2024-09-10T21:08:41.503 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T21:08:41.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:08:41.612 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage} email: hyelloji@redhat.com first_in_suite: false job_id: '7899672' kernel: client: branch: testing flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds - MDS abort because newly corrupt dentry sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: syntax: v1 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtCTET+JXWSFrWMIdPYsX3SSOn20F/v6ffpa+dyd/nD3cVWTI9yapCu61mFqtfz2rVLTHxXDesY+vXlrnb1MNY= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZx1F8fm3r4iIkL74/lM7FVJJ1oK/MvM16ORjqNQQYhtBTar1fcHzKZHp0XBOOC12ccakVORGryDW89jI+DoOc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T21:08:41.613 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T21:08:41.613 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T21:08:41.614 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T21:08:41.615 INFO:teuthology.task.internal:Checking packages... 2024-09-10T21:08:41.636 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T21:08:41.636 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T21:08:41.636 INFO:teuthology.packaging:ref: None 2024-09-10T21:08:41.636 INFO:teuthology.packaging:tag: None 2024-09-10T21:08:41.636 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:08:41.637 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:08:41.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:08:41.783 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:08:41.785 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T21:08:41.792 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T21:08:41.792 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T21:08:41.807 INFO:teuthology.task.internal:Saving configuration 2024-09-10T21:08:41.824 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T21:08:41.831 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T21:08:41.862 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:03:06.728845', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIETmee94oeR9D1gnCZdeq3VpNqj/KCCzbGDsU3CeWnlM'} 2024-09-10T21:08:41.893 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:03:06.730546', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICECW2X2F9Pws06Ze/+vU+IGa2vy9KQsF3O3Y9zxRfY2'} 2024-09-10T21:08:41.893 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T21:08:41.899 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T21:08:41.899 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T21:08:41.899 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T21:08:41.980 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbd8a3c3be0>, signals=[15]) 2024-09-10T21:08:41.980 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T21:08:41.986 INFO:teuthology.task.internal:Opening connections... 2024-09-10T21:08:41.987 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:08:41.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:08:42.077 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:08:42.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:08:42.167 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T21:08:42.174 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-09-10T21:08:42.179 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-09-10T21:08:42.179 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Ubuntu" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="22.04" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_CODENAME=jammy 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:ID=ubuntu 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE=debian 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T21:08:42.228 INFO:teuthology.orchestra.run.smithi097.stdout:UBUNTU_CODENAME=jammy 2024-09-10T21:08:42.229 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-09-10T21:08:42.269 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-09-10T21:08:42.275 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-09-10T21:08:42.275 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-09-10T21:08:42.322 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T21:08:42.322 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="Ubuntu" 2024-09-10T21:08:42.323 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="22.04" 2024-09-10T21:08:42.323 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T21:08:42.323 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_CODENAME=jammy 2024-09-10T21:08:42.323 INFO:teuthology.orchestra.run.smithi194.stdout:ID=ubuntu 2024-09-10T21:08:42.323 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE=debian 2024-09-10T21:08:42.323 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T21:08:42.323 INFO:teuthology.orchestra.run.smithi194.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T21:08:42.324 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T21:08:42.324 INFO:teuthology.orchestra.run.smithi194.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T21:08:42.324 INFO:teuthology.orchestra.run.smithi194.stdout:UBUNTU_CODENAME=jammy 2024-09-10T21:08:42.324 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-09-10T21:08:42.352 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T21:08:42.361 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T21:08:42.367 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T21:08:42.367 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:08:42.370 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:08:42.375 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T21:08:42.381 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T21:08:42.381 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:08:42.417 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:08:42.429 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T21:08:42.500 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T21:08:42.514 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T21:08:42.514 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-10T21:08:42.515 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-10T21:08:42.515 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:08:42.516 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:08:42.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:08:42.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-10T21:08:42.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:08:42.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-10T21:08:42.702 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:42.702 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:08:42.703 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-10T21:08:42.708 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-56-generic 2024-09-10T21:08:42.709 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:42.709 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:08:42.709 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-10T21:08:42.709 DEBUG:teuthology.orchestra.run.smithi097:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T21:08:42.716 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:42.716 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:08:42.716 DEBUG:teuthology.orchestra.run.smithi194:> uname -r 2024-09-10T21:08:42.722 INFO:teuthology.orchestra.run.smithi194.stdout:5.15.0-56-generic 2024-09-10T21:08:42.722 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:42.722 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:08:42.722 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-10T21:08:42.722 DEBUG:teuthology.orchestra.run.smithi194:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T21:08:42.762 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T21:08:42.779 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T21:08:42.897 INFO:teuthology.orchestra.run.smithi097.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: git config --global init.defaultBranch 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 2024-09-10T21:08:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:hint: git branch -m 2024-09-10T21:08:42.899 INFO:teuthology.orchestra.run.smithi097.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T21:08:42.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: git config --global init.defaultBranch 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: 2024-09-10T21:08:42.929 INFO:teuthology.orchestra.run.smithi194.stderr:hint: git branch -m 2024-09-10T21:08:42.930 INFO:teuthology.orchestra.run.smithi194.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T21:08:42.932 DEBUG:teuthology.orchestra.run.smithi194:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T21:08:42.967 DEBUG:teuthology.orchestra.run.smithi097:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T21:08:42.998 DEBUG:teuthology.orchestra.run.smithi194:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T21:08:43.026 DEBUG:teuthology.orchestra.run.smithi097:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T21:08:43.056 DEBUG:teuthology.orchestra.run.smithi194:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T21:09:13.457 INFO:teuthology.orchestra.run.smithi194.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T21:09:13.458 INFO:teuthology.orchestra.run.smithi194.stderr: * [new branch] main -> origin/main 2024-09-10T21:09:13.494 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T21:09:13.494 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T21:09:13.494 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T21:09:13.495 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T21:09:13.495 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T21:09:13.495 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T21:09:13.495 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T21:09:13.495 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T21:09:13.496 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T21:09:13.496 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T21:09:13.497 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T21:09:13.497 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T21:09:13.497 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T21:09:13.497 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T21:09:13.497 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T21:09:13.498 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T21:09:13.499 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T21:09:13.499 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T21:09:13.499 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T21:09:13.499 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T21:09:13.499 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T21:09:13.500 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T21:09:13.500 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T21:09:13.500 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T21:09:13.501 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T21:09:13.501 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T21:09:13.501 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T21:09:13.501 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T21:09:13.501 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T21:09:13.502 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T21:09:13.502 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T21:09:13.502 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T21:09:13.502 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T21:09:13.502 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T21:09:13.503 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T21:09:13.503 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T21:09:13.503 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T21:09:13.503 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T21:09:13.503 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T21:09:13.503 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T21:09:13.504 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T21:09:13.504 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T21:09:13.504 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T21:09:13.504 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T21:09:13.504 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T21:09:13.504 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T21:09:13.505 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T21:09:13.505 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T21:09:13.505 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T21:09:13.505 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T21:09:13.505 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T21:09:13.505 INFO:teuthology.orchestra.run.smithi194.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T21:09:15.717 INFO:teuthology.orchestra.run.smithi097.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T21:09:15.717 INFO:teuthology.orchestra.run.smithi097.stderr: * [new branch] main -> origin/main 2024-09-10T21:09:15.754 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T21:09:15.754 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T21:09:15.754 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T21:09:15.754 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T21:09:15.755 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T21:09:15.755 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T21:09:15.755 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T21:09:15.755 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T21:09:15.755 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T21:09:15.755 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T21:09:15.755 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T21:09:15.759 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T21:09:15.760 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T21:09:15.761 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T21:09:15.761 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T21:09:15.761 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T21:09:15.763 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T21:09:15.764 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T21:09:15.764 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T21:09:15.764 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T21:09:15.764 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T21:09:15.764 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T21:09:18.023 INFO:teuthology.orchestra.run.smithi194.stderr:Updating files: 42% (1483/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 69% (2424/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2891/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T21:09:18.028 INFO:teuthology.orchestra.run.smithi194.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T21:09:18.050 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T21:09:18.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:09:18.199 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:18.199 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T21:09:18.202 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi194.front.sepia.ceph.com to complete... 2024-09-10T21:09:19.194 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10 21:09:19 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T21:09:19.194 INFO:teuthology.orchestra.run.smithi194.stderr:FINISHED --2024-09-10 21:09:19-- 2024-09-10T21:09:19.194 INFO:teuthology.orchestra.run.smithi194.stderr:Total wall clock time: 1.0s 2024-09-10T21:09:19.194 INFO:teuthology.orchestra.run.smithi194.stderr:Downloaded: 1 files, 43M in 0.9s (50.8 MB/s) 2024-09-10T21:09:19.198 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T21:09:19.199 DEBUG:teuthology.orchestra.run.smithi194:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T21:09:19.460 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T21:09:20.287 INFO:teuthology.orchestra.run.smithi097.stderr:Updating files: 35% (1238/3501) Updating files: 36% (1261/3501) Updating files: 37% (1296/3501) Updating files: 38% (1331/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2063/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 69% (2423/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2891/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T21:09:20.292 INFO:teuthology.orchestra.run.smithi097.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T21:09:20.314 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T21:09:20.314 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:09:20.469 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:20.469 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T21:09:20.472 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2024-09-10T21:09:21.354 INFO:teuthology.orchestra.run.smithi194.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T21:09:21.358 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T21:09:21.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10 21:09:21 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T21:09:21.442 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2024-09-10 21:09:21-- 2024-09-10T21:09:21.442 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 0.9s 2024-09-10T21:09:21.442 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 43M in 0.8s (51.9 MB/s) 2024-09-10T21:09:21.446 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T21:09:21.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T21:09:21.497 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:21.682 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T21:09:23.609 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T21:09:23.613 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T21:09:23.723 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:25.543 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:25.672 INFO:teuthology.orchestra.run.smithi194.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:27.492 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:27.611 INFO:teuthology.orchestra.run.smithi097.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:48.237 INFO:teuthology.orchestra.run.smithi194.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:09:48.250 INFO:teuthology.orchestra.run.smithi194.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:09:48.275 INFO:teuthology.orchestra.run.smithi194.stderr:Generating grub configuration file ... 2024-09-10T21:09:48.584 INFO:teuthology.orchestra.run.smithi194.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:48.599 INFO:teuthology.orchestra.run.smithi194.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:48.868 INFO:teuthology.orchestra.run.smithi194.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:09:48.885 INFO:teuthology.orchestra.run.smithi194.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:09:48.944 INFO:teuthology.orchestra.run.smithi194.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:09:48.949 INFO:teuthology.orchestra.run.smithi194.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:09:49.115 INFO:teuthology.orchestra.run.smithi194.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:09:49.115 INFO:teuthology.orchestra.run.smithi194.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:09:49.115 INFO:teuthology.orchestra.run.smithi194.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:09:49.120 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:09:49.135 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:09:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-09-10T21:09:49.154 INFO:teuthology.orchestra.run.smithi194.stderr:done 2024-09-10T21:09:49.341 DEBUG:teuthology.orchestra.run.smithi194:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T21:09:49.391 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T21:09:49.391 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T21:09:49.391 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.406 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.406 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T21:09:49.428 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:09:49.429 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:09:49.430 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T21:09:49.431 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:09:49.432 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:09:49.432 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T21:09:49.443 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:09:49.444 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:09:49.445 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:09:49.446 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T21:09:49.447 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T21:09:49.448 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:09:49.449 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:09:49.450 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:09:49.451 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:09:49.452 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T21:09:49.453 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:09:49.454 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.465 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.508 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:09:49.508 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T21:09:49.508 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:09:49.508 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:09:49.508 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T21:09:49.508 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:09:49.509 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T21:09:49.510 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T21:09:49.511 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:09:49.512 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T21:09:49.513 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:09:49.514 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T21:09:49.515 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T21:09:49.516 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T21:09:49.517 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T21:09:49.518 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:09:49.519 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:09:49.520 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T21:09:49.521 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:09:49.522 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:09:49.523 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:09:49.524 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T21:09:49.525 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:09:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T21:09:49.527 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:09:49.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:09:49.529 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:09:49.530 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:09:49.531 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:09:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:09:49.533 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:09:49.534 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:09:49.535 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:09:49.536 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:09:49.537 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:09:49.538 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:09:49.539 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:09:49.540 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T21:09:49.541 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T21:09:49.542 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T21:09:49.543 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:09:49.544 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:09:49.545 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T21:09:49.546 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T21:09:49.547 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T21:09:49.548 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T21:09:49.549 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:09:49.550 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T21:09:49.551 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T21:09:49.552 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:09:49.553 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:09:49.554 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:09:49.555 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T21:09:49.556 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T21:09:49.557 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:09:49.558 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:09:49.559 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T21:09:49.560 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:09:49.561 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:09:49.561 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T21:09:49.561 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:09:49.561 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:09:49.561 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T21:09:49.561 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:09:49.597 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T21:09:49.598 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T21:09:49.599 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T21:09:49.600 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:09:49.601 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T21:09:49.602 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:09:49.603 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:09:49.604 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:09:49.605 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:09:49.606 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T21:09:49.607 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:09:49.608 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:09:49.609 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:09:49.610 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:09:49.611 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:09:49.612 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T21:09:49.613 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:09:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T21:09:49.615 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:09:49.616 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T21:09:49.617 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T21:09:49.618 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T21:09:49.619 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T21:09:49.620 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T21:09:49.621 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:09:49.622 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T21:09:49.623 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:09:49.624 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T21:09:49.625 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T21:09:49.626 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:09:49.627 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:09:49.628 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:09:49.629 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:09:49.630 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T21:09:49.631 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:09:49.632 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T21:09:49.633 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T21:09:49.634 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:09:49.634 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T21:09:49.634 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T21:09:49.634 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T21:09:49.671 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:09:49.672 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T21:09:49.673 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T21:09:49.674 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:09:49.675 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T21:09:49.676 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T21:09:49.677 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T21:09:49.678 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T21:09:49.679 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T21:09:49.680 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T21:09:49.681 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T21:09:49.682 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:09:49.683 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T21:09:49.684 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:09:49.685 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:09:49.686 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:09:49.687 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T21:09:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:09:49.689 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:09:49.690 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:09:49.691 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T21:09:49.692 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:09:49.693 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:09:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:09:49.695 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T21:09:49.696 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:09:49.697 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T21:09:49.698 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T21:09:49.699 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T21:09:49.700 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:09:49.701 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:09:49.702 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T21:09:49.703 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:09:49.704 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:09:49.705 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:09:49.706 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:09:49.707 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:09:49.708 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T21:09:49.709 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:09:49.741 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:09:49.751 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T21:09:49.751 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:09:49.751 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T21:09:49.751 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T21:09:49.751 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T21:09:49.752 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T21:09:49.753 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T21:09:49.754 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T21:09:49.755 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:09:49.756 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T21:09:49.757 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:09:49.758 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:09:49.759 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:09:49.760 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:09:49.761 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:09:49.762 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:09:49.763 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:09:49.764 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:09:49.765 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:09:49.766 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:09:49.767 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:09:49.768 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T21:09:49.769 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:09:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:09:49.771 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:09:49.772 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:09:49.773 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:09:49.774 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:09:49.775 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:09:49.776 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:09:49.777 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:09:49.778 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:09:49.779 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:09:49.780 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:09:49.781 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:09:49.782 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:09:49.783 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:09:49.784 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:09:49.785 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T21:09:49.786 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T21:09:49.787 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:09:49.788 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:09:49.789 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:09:49.789 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T21:09:49.789 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T21:09:49.789 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T21:09:49.789 INFO:teuthology.orchestra.run.smithi194.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:49.789 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T21:09:49.789 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T21:09:49.794 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.794 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.794 DEBUG:teuthology.orchestra.run.smithi194:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T21:09:49.795 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:09:49.801 INFO:teuthology.orchestra.run.smithi194.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:09:49.801 INFO:teuthology.orchestra.run.smithi194.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:09:49.801 INFO:teuthology.orchestra.run.smithi194.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:09:49.802 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T21:09:49.802 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.802 DEBUG:teuthology.orchestra.run.smithi194:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T21:09:49.814 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:09:49.819 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:09:49.847 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi194.front.sepia.ceph.com to complete... 2024-09-10T21:09:49.971 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:09:49.971 INFO:teuthology.orchestra.run.smithi097.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:09:49.972 INFO:teuthology.orchestra.run.smithi097.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:09:50.007 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-09-10T21:09:50.201 DEBUG:teuthology.orchestra.run.smithi097:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T21:09:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T21:09:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T21:09:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T21:09:50.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:09:50.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:09:50.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:09:50.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:09:50.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:09:50.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:09:50.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:09:50.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:09:50.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:09:50.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:09:50.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:09:50.252 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T21:09:50.252 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T21:09:50.252 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T21:09:50.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:09:50.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T21:09:50.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T21:09:50.253 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T21:09:50.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:09:50.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:09:50.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:09:50.254 INFO:teuthology.orchestra.run.smithi194.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:09:50.255 INFO:teuthology.orchestra.run.smithi194.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:09:50.268 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T21:09:50.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T21:09:50.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:09:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:09:50.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T21:09:50.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:09:50.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:09:50.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:09:50.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:09:50.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:09:50.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:09:50.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T21:09:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:09:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:09:50.280 INFO:teuthology.orchestra.run.smithi194.stderr:Generating grub configuration file ... 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T21:09:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:09:50.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T21:09:50.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T21:09:50.329 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T21:09:50.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:09:50.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T21:09:50.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T21:09:50.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T21:09:50.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T21:09:50.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T21:09:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:09:50.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T21:09:50.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:09:50.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:09:50.340 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:09:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:09:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:09:50.343 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:09:50.344 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:09:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:09:50.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:09:50.347 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:09:50.348 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:09:50.349 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:09:50.350 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:09:50.351 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:09:50.352 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T21:09:50.353 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:09:50.354 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T21:09:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:09:50.356 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:09:50.357 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:09:50.358 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T21:09:50.359 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T21:09:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T21:09:50.361 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:09:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T21:09:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T21:09:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:09:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T21:09:50.366 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:09:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:09:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:09:50.369 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:09:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:09:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:09:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T21:09:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T21:09:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T21:09:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:09:50.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T21:09:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T21:09:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T21:09:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:09:50.423 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:09:50.424 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:09:50.425 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:09:50.426 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:09:50.427 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T21:09:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T21:09:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:09:50.430 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T21:09:50.431 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T21:09:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T21:09:50.433 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:09:50.434 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:09:50.435 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:09:50.436 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:09:50.437 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:09:50.438 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T21:09:50.439 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:09:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:09:50.441 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:09:50.442 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:09:50.443 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:09:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T21:09:50.445 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:09:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T21:09:50.447 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:09:50.448 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T21:09:50.449 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T21:09:50.450 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T21:09:50.451 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T21:09:50.452 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:09:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:09:50.454 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:09:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:09:50.456 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T21:09:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:09:50.458 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:09:50.459 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:09:50.460 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T21:09:50.461 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:09:50.462 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:09:50.463 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:09:50.463 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:09:50.463 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T21:09:50.463 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:09:50.476 INFO:teuthology.orchestra.run.smithi194.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.483 INFO:teuthology.orchestra.run.smithi194.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T21:09:50.509 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:09:50.510 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T21:09:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:09:50.512 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:09:50.513 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T21:09:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:09:50.515 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:09:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T21:09:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T21:09:50.518 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T21:09:50.519 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T21:09:50.520 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T21:09:50.521 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T21:09:50.522 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:09:50.523 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:09:50.524 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T21:09:50.525 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:09:50.526 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:09:50.527 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:09:50.528 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:09:50.529 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:09:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T21:09:50.531 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T21:09:50.532 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T21:09:50.533 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:09:50.534 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:09:50.535 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:09:50.536 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T21:09:50.537 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:09:50.538 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T21:09:50.539 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T21:09:50.540 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:09:50.541 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T21:09:50.542 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T21:09:50.543 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:09:50.544 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:09:50.545 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:09:50.546 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:09:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:09:50.548 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:09:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T21:09:50.550 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:09:50.550 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:09:50.593 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:09:50.593 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:09:50.593 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T21:09:50.594 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:09:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T21:09:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:09:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T21:09:50.598 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:09:50.599 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:09:50.600 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:09:50.601 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:09:50.602 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:09:50.603 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T21:09:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:09:50.605 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:09:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:09:50.607 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:09:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:09:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:09:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:09:50.611 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:09:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:09:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:09:50.614 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:09:50.615 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:09:50.616 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:09:50.617 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:09:50.618 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:09:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:09:50.620 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:09:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:09:50.622 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:09:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:09:50.624 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:09:50.625 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T21:09:50.626 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:09:50.627 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:09:50.628 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T21:09:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T21:09:50.631 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:09:50.632 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:09:50.633 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:09:50.633 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T21:09:50.633 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T21:09:50.633 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T21:09:50.633 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:50.633 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T21:09:50.633 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T21:09:50.638 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.638 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.638 DEBUG:teuthology.orchestra.run.smithi097:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T21:09:50.645 INFO:teuthology.orchestra.run.smithi097.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:09:50.645 INFO:teuthology.orchestra.run.smithi097.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:09:50.645 INFO:teuthology.orchestra.run.smithi097.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:09:50.646 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T21:09:50.646 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:50.646 DEBUG:teuthology.orchestra.run.smithi097:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T21:09:50.688 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi097.front.sepia.ceph.com to complete... 2024-09-10T21:09:50.703 INFO:teuthology.orchestra.run.smithi194.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:09:50.708 INFO:teuthology.orchestra.run.smithi194.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:09:50.744 INFO:teuthology.orchestra.run.smithi194.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:09:50.750 INFO:teuthology.orchestra.run.smithi194.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:09:50.874 INFO:teuthology.orchestra.run.smithi194.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:09:50.874 INFO:teuthology.orchestra.run.smithi194.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:09:50.874 INFO:teuthology.orchestra.run.smithi194.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:09:50.896 INFO:teuthology.orchestra.run.smithi194.stderr:done 2024-09-10T21:09:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:09:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:09:51.084 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-09-10T21:09:51.297 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:09:51.509 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:09:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:09:51.555 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:09:51.668 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:09:51.668 INFO:teuthology.orchestra.run.smithi097.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:09:51.668 INFO:teuthology.orchestra.run.smithi097.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:09:51.686 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-09-10T21:10:22.041 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T21:10:22.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-10T21:10:22.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:10:22.813 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T21:10:22.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:10:22.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:10:40.445 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-09-10T21:10:41.213 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-10T21:10:49.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-10T21:10:49.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:10:50.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:10:50.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:10:52.541 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-09-10T21:10:53.277 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-10T21:11:04.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-10T21:11:04.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:05.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:11:05.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:22.941 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-09-10T21:11:23.709 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-10T21:11:37.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-10T21:11:37.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:38.399 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-09-10T21:11:38.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:11:38.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:38.721 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-10T21:11:39.854 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-10T21:11:39.854 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:11:39.855 DEBUG:teuthology.orchestra.run.smithi194:> uname -r 2024-09-10T21:11:39.904 INFO:teuthology.orchestra.run.smithi194.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:11:39.904 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:11:39.904 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:11:39.904 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T21:11:40.905 DEBUG:teuthology.parallel:result is None 2024-09-10T21:11:56.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:11:56.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:57.004 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-10T21:11:57.632 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:11:57.632 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:11:57.633 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-10T21:11:57.682 INFO:teuthology.orchestra.run.smithi097.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:11:57.683 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:11:57.683 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:11:57.683 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T21:11:58.686 DEBUG:teuthology.parallel:result is None 2024-09-10T21:11:58.686 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T21:11:58.694 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T21:11:58.694 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:11:58.697 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:11:58.706 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T21:11:58.712 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T21:11:58.719 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T21:11:58.719 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:11:58.746 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:11:58.771 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T21:11:58.778 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T21:11:58.778 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:11:58.810 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:11:58.894 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:11:58.917 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:11:58.917 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:11:58.948 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:11:58.950 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T21:11:58.958 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T21:11:58.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:11:58.961 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:11:58.982 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T21:11:58.992 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T21:11:58.992 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:11:59.026 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:11:59.035 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:11:59.084 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:11:59.135 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:11:59.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:11:59.208 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:11:59.217 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:11:59.268 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:11:59.268 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:11:59.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-09-10T21:11:59.344 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-09-10T21:11:59.486 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T21:11:59.493 INFO:teuthology.task.internal:Starting timer... 2024-09-10T21:11:59.494 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T21:11:59.512 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T21:11:59.520 DEBUG:teuthology.task.selinux:Excluding smithi097: OS 'ubuntu' does not support SELinux 2024-09-10T21:11:59.520 DEBUG:teuthology.task.selinux:Excluding smithi194: OS 'ubuntu' does not support SELinux 2024-09-10T21:11:59.521 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T21:11:59.521 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-10T21:11:59.521 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T21:11:59.521 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T21:11:59.529 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T21:11:59.541 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T21:11:59.542 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T21:20:06.851 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2024-09-10T21:20:06.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:20:06.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:20:06.964 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-10T21:20:07.079 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-10T21:20:07.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-10T21:20:07.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:20:07.172 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-09-10T21:20:07.311 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-10T21:20:07.311 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T21:20:07.322 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T21:20:07.322 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:20:07.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:20:07.325 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:20:07.325 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:20:07.360 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:20:07.361 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Command line: ntpd -gq 2024-09-10T21:20:07.361 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: ---------------------------------------------------- 2024-09-10T21:20:07.361 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:20:07.361 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:20:07.361 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: corporation. Support and training for ntp-4 are 2024-09-10T21:20:07.361 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: available at https://www.nwtime.org/support 2024-09-10T21:20:07.361 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: ---------------------------------------------------- 2024-09-10T21:20:07.362 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: proto: precision = 0.043 usec (-24) 2024-09-10T21:20:07.362 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: basedate set to 2022-02-04 2024-09-10T21:20:07.362 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:20:07.362 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:07.362 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:07.363 INFO:teuthology.orchestra.run.smithi097.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:07.363 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:20:07.364 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:20:07.364 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:20:07.364 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Listen normally on 3 enp3s0f1 172.21.15.97:123 2024-09-10T21:20:07.364 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Listen normally on 4 lo [::1]:123 2024-09-10T21:20:07.364 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d805%5]:123 2024-09-10T21:20:07.364 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:07 ntpd[18418]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Command line: ntpd -gq 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: ---------------------------------------------------- 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: corporation. Support and training for ntp-4 are 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: available at https://www.nwtime.org/support 2024-09-10T21:20:07.392 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: ---------------------------------------------------- 2024-09-10T21:20:07.393 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: proto: precision = 0.042 usec (-24) 2024-09-10T21:20:07.393 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: basedate set to 2022-02-04 2024-09-10T21:20:07.393 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:20:07.394 INFO:teuthology.orchestra.run.smithi194.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:07.394 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:07.394 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:07.394 INFO:teuthology.orchestra.run.smithi194.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:07.395 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:20:07.395 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:20:07.395 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:20:07.395 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Listen normally on 3 enp3s0f1 172.21.15.194:123 2024-09-10T21:20:07.395 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Listen normally on 4 lo [::1]:123 2024-09-10T21:20:07.396 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ea5%5]:123 2024-09-10T21:20:07.396 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:07 ntpd[18482]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:08 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:08.361 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:08 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:08 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:08 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:08 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:08.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:08 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:08.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:08 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:10.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:10 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:10.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:10.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:10.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:10.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:10.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:10.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:10 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:12.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:12.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:12.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:12.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:12 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:12.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:12.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:12 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:14.364 INFO:teuthology.orchestra.run.smithi097.stdout:10 Sep 21:20:14 ntpd[18418]: ntpd: time slew -0.002761 s 2024-09-10T21:20:14.364 INFO:teuthology.orchestra.run.smithi097.stdout:ntpd: time slew -0.002761s 2024-09-10T21:20:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:20:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:10 Sep 21:20:14 ntpd[18418]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:20:14.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.393 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:14.394 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:14.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:14.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:14.395 INFO:teuthology.orchestra.run.smithi194.stdout:10 Sep 21:20:14 ntpd[18482]: ntpd: time slew -0.000126 s 2024-09-10T21:20:14.395 INFO:teuthology.orchestra.run.smithi194.stdout:ntpd: time slew -0.000126s 2024-09-10T21:20:14.395 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:20:14.396 INFO:teuthology.orchestra.run.smithi194.stderr:10 Sep 21:20:14 ntpd[18482]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:20:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:20:14.459 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-09-10T21:20:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.459 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.498 INFO:teuthology.orchestra.run.smithi194.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:20:14.499 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================== 2024-09-10T21:20:14.499 INFO:teuthology.orchestra.run.smithi194.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.499 INFO:teuthology.orchestra.run.smithi194.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.499 INFO:teuthology.orchestra.run.smithi194.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.499 INFO:teuthology.orchestra.run.smithi194.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:14.500 INFO:teuthology.run_tasks:Running task install... 2024-09-10T21:20:14.508 DEBUG:teuthology.task.install:project ceph 2024-09-10T21:20:14.508 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T21:20:14.508 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T21:20:14.508 INFO:teuthology.task.install:Using flavor: default 2024-09-10T21:20:14.514 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T21:20:14.514 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T21:20:14.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-key list | grep Ceph 2024-09-10T21:20:14.515 DEBUG:teuthology.orchestra.run.smithi194:> sudo apt-key list | grep Ceph 2024-09-10T21:20:14.616 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:20:14.644 INFO:teuthology.orchestra.run.smithi194.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:20:14.668 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:20:14.668 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:20:14.669 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:20:14.669 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:20:14.669 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:20:14.771 INFO:teuthology.orchestra.run.smithi194.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:20:14.771 INFO:teuthology.orchestra.run.smithi194.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:20:14.772 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:20:14.773 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:20:14.773 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:20:14.826 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:20:14.827 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:20:14.961 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:20:14.961 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:20:14.979 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:20:14.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:20:14.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-09-10T21:20:15.089 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:20:15.089 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:20:15.108 DEBUG:teuthology.orchestra.run.smithi194:> sudo apt-get update 2024-09-10T21:20:15.181 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:20:15.188 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:20:15.205 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:20:15.232 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:20:15.253 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:20:15.277 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:20:15.296 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:20:15.311 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:20:15.332 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:20:15.341 INFO:teuthology.orchestra.run.smithi194.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:20:15.362 INFO:teuthology.orchestra.run.smithi194.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:20:15.384 INFO:teuthology.orchestra.run.smithi194.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:20:15.406 INFO:teuthology.orchestra.run.smithi194.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:20:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:20:15.427 INFO:teuthology.orchestra.run.smithi194.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:20:15.448 INFO:teuthology.orchestra.run.smithi194.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:20:15.491 INFO:teuthology.orchestra.run.smithi194.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:20:15.572 INFO:teuthology.orchestra.run.smithi194.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:20:15.779 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 29.3 kB in 1s (52.9 kB/s) 2024-09-10T21:20:16.022 INFO:teuthology.orchestra.run.smithi194.stdout:Fetched 29.3 kB in 1s (42.7 kB/s) 2024-09-10T21:20:16.819 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-10T21:20:16.834 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-10T21:20:17.054 INFO:teuthology.orchestra.run.smithi194.stdout:Reading package lists... 2024-09-10T21:20:17.068 DEBUG:teuthology.orchestra.run.smithi194:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:20:17.093 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-10T21:20:17.094 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-10T21:20:17.145 INFO:teuthology.orchestra.run.smithi194.stdout:Reading package lists... 2024-09-10T21:20:17.280 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:20:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:20:17.280 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:20:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:20:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:20:17.281 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:20:17.281 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:20:17.282 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-09-10T21:20:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:20:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:20:17.282 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:20:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:20:17.283 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:20:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:20:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:20:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:20:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:20:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:20:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:20:17.284 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-09-10T21:20:17.285 INFO:teuthology.orchestra.run.smithi097.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2024-09-10T21:20:17.286 INFO:teuthology.orchestra.run.smithi097.stdout: btrfs-tools 2024-09-10T21:20:17.327 INFO:teuthology.orchestra.run.smithi194.stdout:Building dependency tree... 2024-09-10T21:20:17.327 INFO:teuthology.orchestra.run.smithi194.stdout:Reading state information... 2024-09-10T21:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-09-10T21:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: socat unzip xmlstarlet zip 2024-09-10T21:20:17.512 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 251 MB of archives. 2024-09-10T21:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:20:17.513 INFO:teuthology.orchestra.run.smithi194.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:20:17.513 INFO:teuthology.orchestra.run.smithi194.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:20:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:20:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:20:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:20:17.514 INFO:teuthology.orchestra.run.smithi194.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:20:17.514 INFO:teuthology.orchestra.run.smithi194.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:20:17.515 INFO:teuthology.orchestra.run.smithi194.stdout:The following additional packages will be installed: 2024-09-10T21:20:17.515 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:20:17.516 INFO:teuthology.orchestra.run.smithi194.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:20:17.517 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:20:17.517 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:20:17.517 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:20:17.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout:Suggested packages: 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:20:17.521 INFO:teuthology.orchestra.run.smithi194.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:20:17.522 INFO:teuthology.orchestra.run.smithi194.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:20:17.522 INFO:teuthology.orchestra.run.smithi194.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:20:17.522 INFO:teuthology.orchestra.run.smithi194.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:20:17.522 INFO:teuthology.orchestra.run.smithi194.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:20:17.522 INFO:teuthology.orchestra.run.smithi194.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:20:17.522 INFO:teuthology.orchestra.run.smithi194.stdout:Recommended packages: 2024-09-10T21:20:17.522 INFO:teuthology.orchestra.run.smithi194.stdout: btrfs-tools 2024-09-10T21:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:20:17.581 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:20:17.593 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:20:17.602 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:20:17.605 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:20:17.616 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:20:17.655 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:20:17.657 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:20:17.664 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:20:17.667 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:20:17.668 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:20:17.670 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:20:17.671 INFO:teuthology.orchestra.run.smithi194.stdout:The following NEW packages will be installed: 2024-09-10T21:20:17.671 INFO:teuthology.orchestra.run.smithi194.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:20:17.671 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:20:17.671 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:20:17.673 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:20:17.674 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:20:17.675 INFO:teuthology.orchestra.run.smithi194.stdout: socat unzip xmlstarlet zip 2024-09-10T21:20:17.677 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:20:17.679 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:20:17.680 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:20:17.682 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:20:17.691 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:20:17.693 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:20:17.695 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:20:17.697 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:20:17.697 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:20:17.709 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:20:17.710 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:20:17.710 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:20:17.710 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:20:17.711 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:20:17.727 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:20:17.727 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:20:17.727 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:20:17.729 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:20:17.730 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:20:17.742 INFO:teuthology.orchestra.run.smithi194.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:20:17.742 INFO:teuthology.orchestra.run.smithi194.stdout:Need to get 251 MB of archives. 2024-09-10T21:20:17.742 INFO:teuthology.orchestra.run.smithi194.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:20:17.742 INFO:teuthology.orchestra.run.smithi194.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:20:17.744 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:20:17.744 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:20:17.745 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:20:17.746 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:20:17.746 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:20:17.760 INFO:teuthology.orchestra.run.smithi194.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:20:17.761 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:20:17.762 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:20:17.762 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:20:17.765 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:20:17.767 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:20:17.767 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:20:17.770 INFO:teuthology.orchestra.run.smithi194.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:20:17.778 INFO:teuthology.orchestra.run.smithi097.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:20:17.778 INFO:teuthology.orchestra.run.smithi097.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:20:17.780 INFO:teuthology.orchestra.run.smithi097.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:20:17.781 INFO:teuthology.orchestra.run.smithi097.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:20:17.782 INFO:teuthology.orchestra.run.smithi097.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:20:17.795 INFO:teuthology.orchestra.run.smithi097.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:20:17.796 INFO:teuthology.orchestra.run.smithi097.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:20:17.798 INFO:teuthology.orchestra.run.smithi097.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:20:17.798 INFO:teuthology.orchestra.run.smithi097.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:20:17.799 INFO:teuthology.orchestra.run.smithi097.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:20:17.812 INFO:teuthology.orchestra.run.smithi194.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:20:17.813 INFO:teuthology.orchestra.run.smithi097.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:20:17.814 INFO:teuthology.orchestra.run.smithi097.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:20:17.823 INFO:teuthology.orchestra.run.smithi097.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:20:17.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:20:17.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:20:17.832 INFO:teuthology.orchestra.run.smithi097.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:20:17.832 INFO:teuthology.orchestra.run.smithi194.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:20:17.833 INFO:teuthology.orchestra.run.smithi097.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:20:17.835 INFO:teuthology.orchestra.run.smithi194.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:20:17.839 INFO:teuthology.orchestra.run.smithi097.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:20:17.846 INFO:teuthology.orchestra.run.smithi194.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:20:17.851 INFO:teuthology.orchestra.run.smithi194.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:20:17.863 INFO:teuthology.orchestra.run.smithi097.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:20:17.864 INFO:teuthology.orchestra.run.smithi097.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:20:17.866 INFO:teuthology.orchestra.run.smithi097.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:20:17.867 INFO:teuthology.orchestra.run.smithi097.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:20:17.868 INFO:teuthology.orchestra.run.smithi097.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:20:17.885 INFO:teuthology.orchestra.run.smithi194.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:20:17.887 INFO:teuthology.orchestra.run.smithi097.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:20:17.888 INFO:teuthology.orchestra.run.smithi194.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:20:17.889 INFO:teuthology.orchestra.run.smithi097.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:20:17.890 INFO:teuthology.orchestra.run.smithi097.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:20:17.895 INFO:teuthology.orchestra.run.smithi194.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:20:17.898 INFO:teuthology.orchestra.run.smithi194.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:20:17.898 INFO:teuthology.orchestra.run.smithi194.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:20:17.899 INFO:teuthology.orchestra.run.smithi194.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:20:17.905 INFO:teuthology.orchestra.run.smithi194.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:20:17.907 INFO:teuthology.orchestra.run.smithi194.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:20:17.907 INFO:teuthology.orchestra.run.smithi097.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:20:17.907 INFO:teuthology.orchestra.run.smithi097.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:20:17.908 INFO:teuthology.orchestra.run.smithi194.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:20:17.909 INFO:teuthology.orchestra.run.smithi097.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:20:17.909 INFO:teuthology.orchestra.run.smithi097.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:20:17.910 INFO:teuthology.orchestra.run.smithi194.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:20:17.910 INFO:teuthology.orchestra.run.smithi097.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:20:17.910 INFO:teuthology.orchestra.run.smithi097.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:20:17.918 INFO:teuthology.orchestra.run.smithi097.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:20:17.918 INFO:teuthology.orchestra.run.smithi097.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:20:17.922 INFO:teuthology.orchestra.run.smithi097.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:20:17.923 INFO:teuthology.orchestra.run.smithi194.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:20:17.924 INFO:teuthology.orchestra.run.smithi194.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:20:17.929 INFO:teuthology.orchestra.run.smithi194.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:20:17.929 INFO:teuthology.orchestra.run.smithi194.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:20:17.929 INFO:teuthology.orchestra.run.smithi194.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:20:17.935 INFO:teuthology.orchestra.run.smithi097.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:20:17.937 INFO:teuthology.orchestra.run.smithi097.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:20:17.937 INFO:teuthology.orchestra.run.smithi097.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:20:17.940 INFO:teuthology.orchestra.run.smithi194.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:20:17.941 INFO:teuthology.orchestra.run.smithi194.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:20:17.941 INFO:teuthology.orchestra.run.smithi097.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:20:17.941 INFO:teuthology.orchestra.run.smithi194.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:20:17.942 INFO:teuthology.orchestra.run.smithi194.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:20:17.942 INFO:teuthology.orchestra.run.smithi194.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:20:17.944 INFO:teuthology.orchestra.run.smithi097.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:20:17.944 INFO:teuthology.orchestra.run.smithi097.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:20:17.945 INFO:teuthology.orchestra.run.smithi097.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:20:17.952 INFO:teuthology.orchestra.run.smithi097.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:20:17.957 INFO:teuthology.orchestra.run.smithi097.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:20:17.958 INFO:teuthology.orchestra.run.smithi194.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:20:17.958 INFO:teuthology.orchestra.run.smithi194.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:20:17.958 INFO:teuthology.orchestra.run.smithi194.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:20:17.959 INFO:teuthology.orchestra.run.smithi097.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:20:17.960 INFO:teuthology.orchestra.run.smithi194.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:20:17.960 INFO:teuthology.orchestra.run.smithi194.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:20:17.969 INFO:teuthology.orchestra.run.smithi097.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:20:17.970 INFO:teuthology.orchestra.run.smithi097.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:20:17.970 INFO:teuthology.orchestra.run.smithi097.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:20:17.971 INFO:teuthology.orchestra.run.smithi097.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:20:17.973 INFO:teuthology.orchestra.run.smithi097.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:20:17.974 INFO:teuthology.orchestra.run.smithi097.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:20:17.974 INFO:teuthology.orchestra.run.smithi194.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:20:17.974 INFO:teuthology.orchestra.run.smithi194.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:20:17.976 INFO:teuthology.orchestra.run.smithi194.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:20:17.976 INFO:teuthology.orchestra.run.smithi194.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:20:17.976 INFO:teuthology.orchestra.run.smithi194.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:20:17.977 INFO:teuthology.orchestra.run.smithi097.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:20:17.978 INFO:teuthology.orchestra.run.smithi097.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:20:17.982 INFO:teuthology.orchestra.run.smithi097.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:20:17.984 INFO:teuthology.orchestra.run.smithi097.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:20:17.986 INFO:teuthology.orchestra.run.smithi097.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:20:17.987 INFO:teuthology.orchestra.run.smithi097.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:20:17.989 INFO:teuthology.orchestra.run.smithi097.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:20:17.991 INFO:teuthology.orchestra.run.smithi194.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:20:17.992 INFO:teuthology.orchestra.run.smithi194.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:20:17.992 INFO:teuthology.orchestra.run.smithi194.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:20:17.997 INFO:teuthology.orchestra.run.smithi194.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:20:17.997 INFO:teuthology.orchestra.run.smithi194.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:20:18.003 INFO:teuthology.orchestra.run.smithi097.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:20:18.008 INFO:teuthology.orchestra.run.smithi194.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:20:18.009 INFO:teuthology.orchestra.run.smithi194.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:20:18.010 INFO:teuthology.orchestra.run.smithi194.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:20:18.012 INFO:teuthology.orchestra.run.smithi194.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:20:18.012 INFO:teuthology.orchestra.run.smithi194.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:20:18.025 INFO:teuthology.orchestra.run.smithi194.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:20:18.026 INFO:teuthology.orchestra.run.smithi194.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:20:18.028 INFO:teuthology.orchestra.run.smithi194.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:20:18.028 INFO:teuthology.orchestra.run.smithi194.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:20:18.029 INFO:teuthology.orchestra.run.smithi194.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:20:18.042 INFO:teuthology.orchestra.run.smithi194.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:20:18.086 INFO:teuthology.orchestra.run.smithi194.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:20:18.087 INFO:teuthology.orchestra.run.smithi194.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:20:18.088 INFO:teuthology.orchestra.run.smithi194.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:20:18.088 INFO:teuthology.orchestra.run.smithi194.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:20:18.089 INFO:teuthology.orchestra.run.smithi194.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:20:18.108 INFO:teuthology.orchestra.run.smithi194.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:20:18.111 INFO:teuthology.orchestra.run.smithi194.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:20:18.111 INFO:teuthology.orchestra.run.smithi194.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:20:18.127 INFO:teuthology.orchestra.run.smithi194.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:20:18.128 INFO:teuthology.orchestra.run.smithi194.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:20:18.128 INFO:teuthology.orchestra.run.smithi194.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:20:18.133 INFO:teuthology.orchestra.run.smithi194.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:20:18.133 INFO:teuthology.orchestra.run.smithi194.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:20:18.133 INFO:teuthology.orchestra.run.smithi194.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:20:18.139 INFO:teuthology.orchestra.run.smithi194.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:20:18.140 INFO:teuthology.orchestra.run.smithi194.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:20:18.143 INFO:teuthology.orchestra.run.smithi194.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:20:18.157 INFO:teuthology.orchestra.run.smithi194.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:20:18.158 INFO:teuthology.orchestra.run.smithi194.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:20:18.158 INFO:teuthology.orchestra.run.smithi194.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:20:18.162 INFO:teuthology.orchestra.run.smithi194.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:20:18.164 INFO:teuthology.orchestra.run.smithi194.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:20:18.165 INFO:teuthology.orchestra.run.smithi194.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:20:18.166 INFO:teuthology.orchestra.run.smithi194.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:20:18.174 INFO:teuthology.orchestra.run.smithi194.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:20:18.178 INFO:teuthology.orchestra.run.smithi194.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:20:18.178 INFO:teuthology.orchestra.run.smithi194.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:20:18.191 INFO:teuthology.orchestra.run.smithi194.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:20:18.191 INFO:teuthology.orchestra.run.smithi194.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:20:18.192 INFO:teuthology.orchestra.run.smithi194.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:20:18.192 INFO:teuthology.orchestra.run.smithi194.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:20:18.194 INFO:teuthology.orchestra.run.smithi194.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:20:18.195 INFO:teuthology.orchestra.run.smithi194.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:20:18.201 INFO:teuthology.orchestra.run.smithi194.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:20:18.208 INFO:teuthology.orchestra.run.smithi194.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:20:18.208 INFO:teuthology.orchestra.run.smithi194.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:20:18.210 INFO:teuthology.orchestra.run.smithi194.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:20:18.225 INFO:teuthology.orchestra.run.smithi194.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:20:18.473 INFO:teuthology.orchestra.run.smithi097.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:20:18.555 INFO:teuthology.orchestra.run.smithi097.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:20:18.557 INFO:teuthology.orchestra.run.smithi097.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:20:18.559 INFO:teuthology.orchestra.run.smithi097.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:20:18.588 INFO:teuthology.orchestra.run.smithi097.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:20:18.697 INFO:teuthology.orchestra.run.smithi097.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:20:19.095 INFO:teuthology.orchestra.run.smithi097.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:20:19.096 INFO:teuthology.orchestra.run.smithi097.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:20:19.104 INFO:teuthology.orchestra.run.smithi097.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:20:19.146 INFO:teuthology.orchestra.run.smithi097.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:20:19.160 INFO:teuthology.orchestra.run.smithi097.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:20:19.162 INFO:teuthology.orchestra.run.smithi097.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:20:19.459 INFO:teuthology.orchestra.run.smithi194.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:20:20.234 INFO:teuthology.orchestra.run.smithi097.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:20:20.367 INFO:teuthology.orchestra.run.smithi097.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:20:20.367 INFO:teuthology.orchestra.run.smithi097.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:20:20.656 INFO:teuthology.orchestra.run.smithi194.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:20:20.898 INFO:teuthology.orchestra.run.smithi194.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:20:21.005 INFO:teuthology.orchestra.run.smithi194.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:20:21.039 INFO:teuthology.orchestra.run.smithi194.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:20:21.093 INFO:teuthology.orchestra.run.smithi194.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:20:21.099 INFO:teuthology.orchestra.run.smithi194.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:20:21.235 INFO:teuthology.orchestra.run.smithi194.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:20:21.342 INFO:teuthology.orchestra.run.smithi097.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:20:21.343 INFO:teuthology.orchestra.run.smithi097.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:20:21.345 INFO:teuthology.orchestra.run.smithi097.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:20:21.345 INFO:teuthology.orchestra.run.smithi097.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:20:21.345 INFO:teuthology.orchestra.run.smithi097.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:20:21.346 INFO:teuthology.orchestra.run.smithi097.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:20:21.611 INFO:teuthology.orchestra.run.smithi097.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:20:22.263 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 251 MB in 4s (60.6 MB/s) 2024-09-10T21:20:22.422 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:20:22.481 INFO:teuthology.orchestra.run.smithi194.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:20:22.485 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:20:22.489 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:20:22.498 INFO:teuthology.orchestra.run.smithi194.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:20:22.529 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:20:22.540 INFO:teuthology.orchestra.run.smithi194.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:20:22.815 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:20:22.829 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:20:22.873 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:20:23.201 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:20:23.215 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:20:23.259 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:20:23.621 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:20:23.635 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:20:23.737 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:20:24.049 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:20:24.064 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:20:24.099 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:20:24.444 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:20:24.458 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:20:24.502 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:20:24.863 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:20:24.878 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:20:24.922 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:20:25.155 INFO:teuthology.orchestra.run.smithi194.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:20:25.358 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:20:25.373 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:20:25.417 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:20:25.561 INFO:teuthology.orchestra.run.smithi194.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:20:25.580 INFO:teuthology.orchestra.run.smithi194.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:20:25.581 INFO:teuthology.orchestra.run.smithi194.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:20:25.705 INFO:teuthology.orchestra.run.smithi194.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:20:25.786 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:20:25.801 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:20:25.844 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:20:26.144 INFO:teuthology.orchestra.run.smithi194.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:20:26.156 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:20:26.170 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:20:26.206 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:20:26.533 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librados2. 2024-09-10T21:20:26.548 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:26.592 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:27.061 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:20:27.076 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:20:27.119 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:20:27.406 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:20:27.420 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:27.455 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:27.589 INFO:teuthology.orchestra.run.smithi194.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:20:27.600 INFO:teuthology.orchestra.run.smithi194.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:20:27.643 INFO:teuthology.orchestra.run.smithi194.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:20:27.788 INFO:teuthology.orchestra.run.smithi194.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:20:27.825 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:20:27.831 INFO:teuthology.orchestra.run.smithi194.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:20:27.833 INFO:teuthology.orchestra.run.smithi194.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:20:27.839 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:27.883 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:28.152 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:20:28.167 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:28.210 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:28.555 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:20:28.570 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:20:28.630 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:28.874 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:20:28.889 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:28.924 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:29.201 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:20:29.217 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:20:29.260 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:29.946 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:20:29.961 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:20:30.004 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:20:30.265 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:20:30.280 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:30.324 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:30.685 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:20:30.699 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:20:30.735 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:20:30.896 INFO:teuthology.orchestra.run.smithi194.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:20:30.978 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:20:30.993 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:20:31.045 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:20:31.254 INFO:teuthology.orchestra.run.smithi194.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:20:31.254 INFO:teuthology.orchestra.run.smithi194.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:20:31.373 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:20:31.389 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:20:31.431 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:20:31.751 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:20:31.766 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:20:31.834 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:20:32.129 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:20:32.143 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:20:32.179 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:20:32.423 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package zip. 2024-09-10T21:20:32.438 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:20:32.506 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:20:32.826 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package unzip. 2024-09-10T21:20:32.840 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:20:32.884 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:20:33.277 INFO:teuthology.orchestra.run.smithi194.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:20:33.278 INFO:teuthology.orchestra.run.smithi194.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:20:33.280 INFO:teuthology.orchestra.run.smithi194.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:20:33.281 INFO:teuthology.orchestra.run.smithi194.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:20:33.284 INFO:teuthology.orchestra.run.smithi194.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:20:33.284 INFO:teuthology.orchestra.run.smithi194.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:20:33.287 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:20:33.302 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:20:33.346 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:20:33.718 INFO:teuthology.orchestra.run.smithi194.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:20:33.736 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:20:33.750 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:33.794 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:34.322 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:20:34.335 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:34.372 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:34.373 INFO:teuthology.orchestra.run.smithi194.stdout:Fetched 251 MB in 16s (15.7 MB/s) 2024-09-10T21:20:34.501 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:20:34.567 INFO:teuthology.orchestra.run.smithi194.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:20:34.571 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:20:34.609 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:20:34.742 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:20:34.757 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:20:34.792 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:20:34.945 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:20:34.958 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:20:35.003 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:20:35.044 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:20:35.058 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:35.102 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:35.331 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:20:35.344 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:20:35.556 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:20:35.570 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:35.574 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:20:35.614 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:35.875 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:20:35.889 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:20:35.975 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:20:36.371 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:20:36.384 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:20:36.437 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:20:36.874 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:20:36.877 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:20:36.887 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:20:36.891 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:36.940 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:20:36.944 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:37.473 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:20:37.485 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:20:37.486 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:20:37.501 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:20:37.514 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:20:37.551 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:20:37.801 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:20:37.813 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:20:37.851 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:20:38.040 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:20:38.055 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:20:38.107 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:20:38.187 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:20:38.202 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:20:38.245 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:20:38.518 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:20:38.533 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:20:38.548 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:20:38.562 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:20:38.585 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:20:38.632 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:20:38.909 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:20:38.923 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:20:38.962 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:20:38.967 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:20:38.977 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:20:39.029 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:20:39.298 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package librados2. 2024-09-10T21:20:39.312 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:20:39.313 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:39.326 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:20:39.349 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:39.370 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:20:39.673 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:20:39.688 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:20:39.731 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:20:39.834 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:20:39.849 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:20:39.901 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:20:40.209 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:20:40.224 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:20:40.237 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:20:40.253 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:40.259 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:20:40.304 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:40.545 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:20:40.560 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:20:40.620 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:20:40.932 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:20:40.947 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:20:40.957 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:20:40.972 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:40.982 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:20:41.023 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:20:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:20:41.342 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:20:41.357 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:41.367 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:20:41.409 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:41.620 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:20:41.635 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:20:41.670 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:20:41.762 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:20:41.776 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:20:41.820 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:41.998 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:20:42.013 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:20:42.057 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:20:42.114 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:20:42.129 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:42.181 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:42.393 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:20:42.408 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:20:42.451 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:20:42.550 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:20:42.565 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:20:42.616 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:42.771 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:20:42.786 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:20:42.829 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:20:43.003 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:20:43.017 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:20:43.069 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:20:43.090 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:20:43.105 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:20:43.140 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:20:43.405 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:20:43.419 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:43.451 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:20:43.466 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:20:43.471 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:43.504 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:20:43.812 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:20:43.827 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:20:43.870 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:20:43.933 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:20:43.943 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:20:43.991 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:20:44.199 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:20:44.213 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:20:44.257 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:20:44.335 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:20:44.345 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:20:44.393 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:20:44.543 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:20:44.557 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:20:44.593 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:20:44.738 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:20:44.748 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:20:44.796 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:20:44.912 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:20:44.924 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:20:44.962 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:20:45.166 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:20:45.176 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:20:45.224 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:20:45.265 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:20:45.280 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:20:45.357 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:20:45.610 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:20:45.625 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:20:45.668 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:20:45.681 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:20:45.718 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:20:45.785 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:20:46.106 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:20:46.113 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package zip. 2024-09-10T21:20:46.119 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:20:46.123 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:20:46.164 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:20:46.171 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:20:46.450 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:20:46.463 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:20:46.500 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:20:46.541 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package unzip. 2024-09-10T21:20:46.552 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:20:46.591 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:20:46.820 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:20:46.833 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:20:46.878 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:20:47.061 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:20:47.072 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:20:47.119 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:20:47.191 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:20:47.205 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:20:47.249 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:20:47.536 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:20:47.549 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:20:47.552 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:20:47.562 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:47.585 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:20:47.602 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:47.847 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:20:47.860 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:20:47.905 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:48.129 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:20:48.139 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:48.179 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:48.310 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:20:48.323 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:48.360 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:48.532 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:20:48.541 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:20:48.582 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:20:48.704 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:20:48.717 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:20:48.762 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:20:48.868 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:20:48.880 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:48.918 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:49.065 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:20:49.078 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:20:49.123 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:20:49.371 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:20:49.382 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:49.420 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:49.426 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:20:49.440 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:49.476 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:49.862 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:20:49.876 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:49.920 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:51.328 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:20:51.338 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:51.383 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:51.501 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:20:51.513 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:51.552 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:52.097 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:20:52.108 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:20:52.136 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph. 2024-09-10T21:20:52.148 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:20:52.149 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:52.177 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:52.480 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:20:52.483 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:20:52.493 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:52.494 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:20:52.533 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:20:52.538 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:52.844 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:20:52.856 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:20:52.893 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:20:52.899 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:20:52.913 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:52.957 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:53.222 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:20:53.233 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:20:53.271 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:20:53.369 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:20:53.383 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:20:53.427 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:53.491 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:20:53.499 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:20:53.566 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:20:53.680 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:20:53.693 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:20:53.729 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:20:53.868 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:20:53.880 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:20:53.918 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:20:54.057 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:20:54.071 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:20:54.116 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:20:54.229 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:20:54.243 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:20:54.296 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:20:54.574 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:20:54.584 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:20:54.623 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:20:54.910 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:20:54.920 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:20:54.959 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:20:55.154 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:20:55.164 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:20:55.203 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:20:55.304 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:20:55.315 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:20:55.380 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:20:55.523 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:20:55.533 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:20:55.573 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:20:55.649 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:20:55.660 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:20:55.691 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:20:55.934 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:20:55.945 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:20:55.984 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:55.993 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:20:56.004 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:20:56.043 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:20:56.329 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:20:56.340 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:20:56.346 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:20:56.356 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:20:56.379 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:20:56.396 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:20:56.690 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:20:56.702 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:20:56.740 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:20:56.749 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:20:56.759 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:20:56.799 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:20:57.076 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:20:57.077 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:20:57.087 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:20:57.088 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:20:57.126 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:20:57.143 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:20:57.396 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:20:57.406 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:20:57.445 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:20:57.790 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:20:57.801 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:20:57.843 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:20:58.151 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:20:58.162 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:20:58.201 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:20:58.512 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:20:58.522 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:20:58.562 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:20:58.882 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:20:58.892 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:20:58.965 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:20:59.267 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:20:59.278 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:20:59.318 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:20:59.456 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:20:59.468 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:20:59.505 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:20:59.620 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:20:59.631 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:20:59.662 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:20:59.923 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:20:59.933 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:20:59.973 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:20:59.984 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:20:59.995 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:21:00.144 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:21:00.336 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:21:00.347 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:21:00.386 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:21:00.506 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:21:00.520 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:21:00.556 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:21:00.755 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:21:00.768 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:21:00.813 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:21:01.085 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:21:01.098 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:21:01.108 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:21:01.119 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:21:01.142 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:21:01.157 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:21:01.677 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:21:01.689 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:21:01.727 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:21:01.826 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:21:01.838 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:01.876 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:02.071 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:21:02.083 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:21:02.122 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:21:02.416 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:21:02.428 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:02.466 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:02.829 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:21:02.842 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:21:02.870 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:21:02.878 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:21:02.881 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:02.920 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:03.181 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:21:03.194 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:21:03.223 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:21:03.224 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:21:03.234 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:21:03.265 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:21:03.501 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:21:03.512 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:21:03.543 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:21:03.550 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:21:03.556 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:21:03.593 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:21:03.928 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:21:03.929 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:21:03.942 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:21:03.943 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:03.987 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:03.987 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:21:04.308 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:21:04.320 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:21:04.358 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:21:04.406 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:21:04.419 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:04.456 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:04.703 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:21:04.715 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:21:04.752 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:21:04.985 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:21:04.996 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:05.026 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:05.030 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:21:05.043 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:21:05.097 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:21:05.409 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:21:05.422 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:21:05.467 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:21:05.779 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph. 2024-09-10T21:21:05.792 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:05.829 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:06.164 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:21:06.176 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:06.190 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:21:06.202 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:06.213 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:06.240 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:06.567 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:21:06.576 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:21:06.578 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:21:06.589 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:06.616 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:21:06.676 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:06.961 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:21:06.973 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:21:07.029 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:21:07.042 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:07.079 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:07.288 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package jq. 2024-09-10T21:21:07.301 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:07.338 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:21:07.423 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:21:07.436 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:21:07.473 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:21:07.574 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package socat. 2024-09-10T21:21:07.589 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:21:07.624 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:21:07.776 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:21:07.788 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:21:07.844 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:07.936 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:21:07.949 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:21:07.985 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:21:08.323 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:21:08.336 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:08.373 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:08.888 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:21:08.900 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:21:08.939 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:09.299 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:21:09.311 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:21:09.349 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:21:09.736 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:21:09.747 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:09.794 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:10.181 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:21:10.193 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:21:10.264 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:21:10.444 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:21:10.458 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:10.502 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:10.616 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:21:10.629 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:21:10.675 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:21:10.881 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:21:10.896 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:11.119 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:21:11.131 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:11.132 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:11.177 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:11.459 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:21:11.473 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:21:11.534 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:21:11.887 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:21:11.900 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:11.944 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:12.205 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:21:12.219 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:12.263 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:12.516 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:21:12.530 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:12.574 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:12.902 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:21:12.916 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:21:12.960 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:21:13.281 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:21:13.295 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:21:13.339 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:21:13.682 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:21:13.695 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:21:13.727 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:21:13.731 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:21:13.741 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:21:13.785 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:21:14.231 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:21:14.245 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:21:14.252 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:21:14.263 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:21:14.281 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:21:14.395 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:21:14.543 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:21:14.557 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:21:14.600 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:21:14.793 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:21:14.804 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:21:14.842 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:21:15.045 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:21:15.059 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:21:15.103 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:21:15.120 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:21:15.132 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:21:15.162 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:21:15.389 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:21:15.403 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:21:15.447 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:21:15.801 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:21:15.803 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:21:15.814 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:21:15.815 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:15.853 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:15.858 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:21:16.112 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:21:16.126 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:21:16.170 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:21:16.531 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:21:16.545 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:21:16.580 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:21:16.781 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:21:16.794 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:21:16.839 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:21:16.918 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:21:16.932 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:21:16.967 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:21:17.201 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:21:17.214 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:21:17.259 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:21:17.413 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:21:17.426 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:21:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:21:17.587 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:21:17.599 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:21:17.637 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:21:17.807 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:21:17.821 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:21:17.865 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:21:17.956 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:21:17.969 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:21:18.006 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:21:18.168 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:21:18.182 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:21:18.251 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:21:18.301 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:21:18.314 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:21:18.389 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:21:18.588 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:21:18.602 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:21:18.646 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:21:18.646 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:21:18.659 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:21:18.704 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:21:18.923 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:21:18.938 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:21:18.982 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:21:19.032 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:21:19.045 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:21:19.090 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:21:19.410 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:21:19.424 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:21:19.468 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:21:19.760 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:21:19.773 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:19.810 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:20.156 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:21:20.169 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:20.231 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:20.514 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:21:20.526 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:21:20.528 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:20.538 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:21:20.563 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:20.576 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:21:20.855 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:21:20.904 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:21:20.916 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:20.945 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:21:21.126 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:21:21.189 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package jq. 2024-09-10T21:21:21.202 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:21.239 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:21:21.486 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:21:21.501 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package socat. 2024-09-10T21:21:21.513 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:21:21.542 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:21:21.725 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:21:21.820 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:21:21.833 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:21:21.861 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:21:22.132 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:21:22.432 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:21:22.445 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:22.491 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:22.732 INFO:teuthology.orchestra.run.smithi097.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:21:22.785 INFO:teuthology.orchestra.run.smithi097.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:21:22.829 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:23.390 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user cephadm....done 2024-09-10T21:21:23.552 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:21:23.845 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:21:24.104 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:21:24.230 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:21:24.470 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:21:24.483 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:24.515 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:21:24.519 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:24.766 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:21:24.865 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:21:24.879 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:24.892 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:21:24.923 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:25.226 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:21:25.227 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:21:25.240 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:21:25.284 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:21:25.570 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:21:25.584 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:25.596 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:21:25.628 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:25.847 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:21:25.862 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:25.889 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:21:25.897 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:26.165 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:21:26.183 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:21:26.196 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:26.241 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:26.441 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:21:26.553 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:21:26.566 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:21:26.610 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:21:26.693 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:21:26.990 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:21:27.003 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:21:27.085 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:21:27.360 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:21:27.370 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:21:27.376 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:21:27.418 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:21:27.629 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:27.806 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:21:27.819 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:21:27.856 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:21:27.930 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:21:28.056 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:21:28.126 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:21:28.138 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:21:28.175 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:21:28.182 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:21:28.308 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:21:28.603 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:21:28.612 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:21:28.624 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:21:28.661 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:21:28.704 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:21:28.964 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:21:28.971 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:21:28.976 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:21:29.014 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:21:29.359 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:21:29.366 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:21:29.371 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:21:29.402 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:21:29.409 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:21:29.466 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:21:29.592 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:21:29.678 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:21:29.690 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:21:29.720 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:21:29.952 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:30.078 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:21:30.081 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:21:30.093 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:21:30.130 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:21:30.354 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:21:30.451 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:21:30.463 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:21:30.492 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:21:30.655 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:21:30.904 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:21:30.916 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:21:30.973 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:21:30.987 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:21:31.124 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:21:31.290 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:21:31.302 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:21:31.340 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:21:31.659 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:21:31.671 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:21:31.709 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:21:31.818 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:21:32.062 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:21:32.074 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:21:32.102 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:21:32.137 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:21:32.228 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:21:32.355 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:21:32.407 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:21:32.419 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:21:32.456 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:21:32.697 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:21:33.115 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:21:33.169 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:21:33.181 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:33.210 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:33.357 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:21:33.776 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:21:33.905 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:21:33.918 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:33.954 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:34.077 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:21:34.272 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:21:34.428 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:21:34.543 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:21:34.692 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:21:34.788 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:21:34.954 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:21:35.056 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:21:35.207 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:35.357 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:21:35.499 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:21:35.842 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:21:35.987 INFO:teuthology.orchestra.run.smithi194.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:21:36.035 INFO:teuthology.orchestra.run.smithi194.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:21:36.179 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:36.748 INFO:teuthology.orchestra.run.smithi194.stdout:Adding system user cephadm....done 2024-09-10T21:21:36.902 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:21:37.073 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:21:37.186 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:21:37.325 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:37.446 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:21:37.459 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:21:37.555 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:21:37.735 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:21:37.840 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:21:37.886 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:21:38.012 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:21:38.116 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:21:38.275 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:21:38.322 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:21:38.581 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:21:38.585 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:21:38.816 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:38.933 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:21:38.953 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:21:39.205 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:21:39.243 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:21:39.464 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:21:39.502 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:21:39.732 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:21:39.804 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:21:39.992 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:21:40.071 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:21:40.276 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:21:40.339 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:21:40.552 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:21:40.607 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:21:40.716 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:21:40.828 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:21:41.088 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:21:41.159 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:21:41.221 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:21:41.348 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:21:41.461 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:21:41.482 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:21:41.750 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:21:41.770 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:21:41.917 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:21:42.072 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:21:42.185 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:21:42.197 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:21:42.306 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:42.379 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:21:42.588 INFO:teuthology.orchestra.run.smithi194.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:21:42.608 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:21:42.632 INFO:teuthology.orchestra.run.smithi194.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:21:42.688 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:21:42.734 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:21:42.806 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:21:43.169 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:21:43.173 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:43.300 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:21:43.445 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:21:43.571 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:21:43.576 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:21:43.848 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:43.877 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:21:43.957 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:21:44.186 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:21:44.258 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:21:44.329 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:21:44.567 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:21:44.946 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:21:44.997 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:21:45.071 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:21:45.265 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:21:45.373 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:21:45.433 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:21:45.559 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:21:45.624 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:21:45.883 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:21:45.885 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:21:46.269 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:21:46.333 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:21:46.570 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:21:46.637 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:21:46.923 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:21:47.005 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:21:47.057 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:47.191 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:47.298 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:21:47.317 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:21:47.616 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:21:47.662 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:47.788 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:47.947 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:21:48.142 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:21:48.249 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:48.375 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:48.410 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:21:48.522 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:48.535 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:21:48.737 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:21:48.845 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:21:49.180 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:21:49.361 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:49.495 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:49.621 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:49.747 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:21:50.418 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:21:50.678 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:50.821 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:21:51.064 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:21:51.206 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:21:51.216 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:51.344 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:51.374 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:21:51.470 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:51.596 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:51.675 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:21:51.722 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:51.848 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:51.943 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:21:52.202 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:52.328 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:21:52.409 INFO:teuthology.orchestra.run.smithi097.stdout:Adding group ceph....done 2024-09-10T21:21:52.621 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:21:52.864 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:21:52.877 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user ceph....done 2024-09-10T21:21:53.047 INFO:teuthology.orchestra.run.smithi097.stdout:Setting system user ceph properties....done 2024-09-10T21:21:53.068 INFO:teuthology.orchestra.run.smithi097.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:21:53.131 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:21:53.212 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:21:53.424 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:21:53.601 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:21:53.692 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:21:53.968 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:21:54.093 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:21:54.266 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:54.403 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:21:54.476 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:21:54.476 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:21:54.529 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:21:54.830 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:21:55.075 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:55.123 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:21:55.201 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:55.391 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:21:55.516 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:21:55.642 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:55.731 INFO:teuthology.orchestra.run.smithi194.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:21:55.774 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:21:55.774 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:21:55.969 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:21:56.070 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:21:56.404 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:56.446 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:21:56.739 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:21:56.787 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:21:56.874 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:21:57.158 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:57.284 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:21:57.618 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:21:57.903 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:57.944 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:21:58.182 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:21:58.182 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:21:58.330 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:21:58.431 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:21:58.710 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:21:58.946 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:58.978 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:21:59.158 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:21:59.158 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:21:59.246 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:21:59.622 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:21:59.789 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:59.916 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:22:00.123 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:00.124 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:00.268 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:22:00.385 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:00.487 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:00.604 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:22:00.757 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:00.974 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:01.017 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:01.099 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:01.225 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:01.225 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:01.251 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:22:01.544 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:01.670 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:01.796 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:01.835 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:02.003 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:02.032 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:22:02.032 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:22:02.129 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:02.272 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:02.651 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:22:02.655 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:02.750 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:22:02.782 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:02.902 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:22:02.907 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:03.067 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:03.193 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:22:04.087 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:22:04.463 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:04.589 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:04.715 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:04.816 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:22:04.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:22:04.866 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:04.899 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-10T21:22:04.992 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:05.081 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-10T21:22:05.082 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-10T21:22:05.112 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:05.265 INFO:teuthology.orchestra.run.smithi097.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:22:05.265 INFO:teuthology.orchestra.run.smithi097.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:22:05.265 INFO:teuthology.orchestra.run.smithi097.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:22:05.265 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:22:05.265 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:22:05.265 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:22:05.266 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:22:05.266 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:22:05.266 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:22:05.266 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:22:05.267 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-09-10T21:22:05.267 INFO:teuthology.orchestra.run.smithi097.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:22:05.268 INFO:teuthology.orchestra.run.smithi097.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:22:05.268 INFO:teuthology.orchestra.run.smithi097.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:22:05.268 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:22:05.268 INFO:teuthology.orchestra.run.smithi097.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:05.269 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-09-10T21:22:05.269 INFO:teuthology.orchestra.run.smithi097.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:22:05.269 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:22:05.269 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:22:05.326 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-09-10T21:22:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:22:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:22:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:22:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:22:05.327 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:22:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:22:05.328 INFO:teuthology.orchestra.run.smithi097.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:05.329 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages will be upgraded: 2024-09-10T21:22:05.329 INFO:teuthology.orchestra.run.smithi097.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:22:05.515 INFO:teuthology.orchestra.run.smithi097.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:22:05.548 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:22:05.548 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:22:05.548 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:22:05.685 INFO:teuthology.orchestra.run.smithi194.stdout:Adding group ceph....done 2024-09-10T21:22:06.022 INFO:teuthology.orchestra.run.smithi194.stdout:Adding system user ceph....done 2024-09-10T21:22:06.140 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:22:06.153 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:22:06.200 INFO:teuthology.orchestra.run.smithi194.stdout:Setting system user ceph properties....done 2024-09-10T21:22:06.220 INFO:teuthology.orchestra.run.smithi194.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:22:06.277 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:22:06.277 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:22:06.278 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:22:06.307 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:22:06.308 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:22:06.312 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:22:06.314 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:22:06.315 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:22:06.317 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:22:06.362 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:22:06.381 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:22:06.381 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:22:06.381 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:22:06.462 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:22:06.740 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:22:07.141 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:22:07.143 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:22:07.143 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:22:07.146 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:22:07.146 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:22:07.151 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:22:07.153 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:22:07.160 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:22:07.160 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:22:07.236 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:22:07.242 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:22:07.292 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:22:07.329 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:22:07.371 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:22:07.371 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:22:07.372 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:22:07.372 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:22:07.392 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:22:07.394 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:22:07.411 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:07.619 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:22:07.619 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:22:07.874 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:22:07.875 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:22:07.875 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:22:07.875 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:22:07.875 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:22:07.875 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:22:07.891 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:22:07.892 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:22:08.279 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:08.283 INFO:teuthology.orchestra.run.smithi097.stdout:Preconfiguring packages ... 2024-09-10T21:22:08.427 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 55.8 MB in 3s (21.8 MB/s) 2024-09-10T21:22:08.430 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:08.717 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:08.720 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:08.859 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:08.958 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:22:08.958 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:22:09.468 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:09.540 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:09.934 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:22:10.540 INFO:teuthology.orchestra.run.smithi097.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:22:10.551 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:10.782 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:22:10.782 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:22:10.815 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package flex. 2024-09-10T21:22:10.886 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:10.889 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:10.930 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:22:11.427 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:11.458 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:22:11.620 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:22:11.641 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:22:11.641 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:22:11.984 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:22:12.253 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:12.531 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:22:12.534 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:12.570 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:12.570 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:12.635 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:13.073 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:13.205 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:13.216 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:13.331 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:13.474 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:13.549 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:13.692 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:13.692 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:13.900 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package bison. 2024-09-10T21:22:13.977 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:22:13.981 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:22:14.028 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:22:14.350 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:14.445 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:22:14.461 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:22:14.476 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:14.512 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:22:14.602 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:14.788 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package dump. 2024-09-10T21:22:14.881 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:22:14.923 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:22:15.149 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:22:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package indent. 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:15.243 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:22:15.257 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:22:15.393 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:22:15.588 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:22:15.604 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:22:15.646 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:22:15.975 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:22:15.991 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:22:16.033 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:22:16.286 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:22:16.302 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:22:16.344 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:22:16.511 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:22:16.673 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:22:16.686 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:16.760 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:17.017 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:22:17.032 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:17.075 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:17.197 INFO:teuthology.orchestra.run.smithi194.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:22:17.200 DEBUG:teuthology.orchestra.run.smithi194:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:22:17.277 INFO:teuthology.orchestra.run.smithi194.stdout:Reading package lists... 2024-09-10T21:22:17.311 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:22:17.325 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:22:17.361 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:22:17.470 INFO:teuthology.orchestra.run.smithi194.stdout:Building dependency tree... 2024-09-10T21:22:17.471 INFO:teuthology.orchestra.run.smithi194.stdout:Reading state information... 2024-09-10T21:22:17.666 INFO:teuthology.orchestra.run.smithi194.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:22:17.666 INFO:teuthology.orchestra.run.smithi194.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:22:17.666 INFO:teuthology.orchestra.run.smithi194.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:22:17.666 INFO:teuthology.orchestra.run.smithi194.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:22:17.666 INFO:teuthology.orchestra.run.smithi194.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:22:17.666 INFO:teuthology.orchestra.run.smithi194.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:22:17.667 INFO:teuthology.orchestra.run.smithi194.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:22:17.667 INFO:teuthology.orchestra.run.smithi194.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:22:17.667 INFO:teuthology.orchestra.run.smithi194.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:22:17.667 INFO:teuthology.orchestra.run.smithi194.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:22:17.668 INFO:teuthology.orchestra.run.smithi194.stdout:The following additional packages will be installed: 2024-09-10T21:22:17.668 INFO:teuthology.orchestra.run.smithi194.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:22:17.668 INFO:teuthology.orchestra.run.smithi194.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:22:17.669 INFO:teuthology.orchestra.run.smithi194.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:22:17.669 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:22:17.669 INFO:teuthology.orchestra.run.smithi194.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:17.671 INFO:teuthology.orchestra.run.smithi194.stdout:Suggested packages: 2024-09-10T21:22:17.671 INFO:teuthology.orchestra.run.smithi194.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:22:17.671 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:22:17.671 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:22:17.698 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:22:17.711 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:22:17.728 INFO:teuthology.orchestra.run.smithi194.stdout:The following NEW packages will be installed: 2024-09-10T21:22:17.728 INFO:teuthology.orchestra.run.smithi194.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:22:17.728 INFO:teuthology.orchestra.run.smithi194.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:22:17.729 INFO:teuthology.orchestra.run.smithi194.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:22:17.729 INFO:teuthology.orchestra.run.smithi194.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:22:17.729 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:22:17.729 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:22:17.729 INFO:teuthology.orchestra.run.smithi194.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:17.730 INFO:teuthology.orchestra.run.smithi194.stdout:The following packages will be upgraded: 2024-09-10T21:22:17.731 INFO:teuthology.orchestra.run.smithi194.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:22:17.756 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:22:17.916 INFO:teuthology.orchestra.run.smithi194.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:22:17.916 INFO:teuthology.orchestra.run.smithi194.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:22:17.917 INFO:teuthology.orchestra.run.smithi194.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:22:17.917 INFO:teuthology.orchestra.run.smithi194.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:22:18.540 INFO:teuthology.orchestra.run.smithi194.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:22:18.556 INFO:teuthology.orchestra.run.smithi194.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:22:18.681 INFO:teuthology.orchestra.run.smithi194.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:22:18.681 INFO:teuthology.orchestra.run.smithi194.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:22:18.682 INFO:teuthology.orchestra.run.smithi194.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:22:18.711 INFO:teuthology.orchestra.run.smithi194.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:22:18.712 INFO:teuthology.orchestra.run.smithi194.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:22:18.715 INFO:teuthology.orchestra.run.smithi194.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:22:18.718 INFO:teuthology.orchestra.run.smithi194.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:22:18.718 INFO:teuthology.orchestra.run.smithi194.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:22:18.721 INFO:teuthology.orchestra.run.smithi194.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:22:18.785 INFO:teuthology.orchestra.run.smithi194.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:22:18.785 INFO:teuthology.orchestra.run.smithi194.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:22:18.786 INFO:teuthology.orchestra.run.smithi194.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:22:18.866 INFO:teuthology.orchestra.run.smithi194.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:22:19.109 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:22:19.122 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:22:19.167 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:19.403 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:22:19.416 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:22:19.455 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:19.808 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:22:19.821 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:22:19.866 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:22:20.228 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:22:20.242 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:22:20.278 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:22:20.533 INFO:teuthology.orchestra.run.smithi194.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:22:20.534 INFO:teuthology.orchestra.run.smithi194.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:22:20.535 INFO:teuthology.orchestra.run.smithi194.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:22:20.538 INFO:teuthology.orchestra.run.smithi194.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:22:20.539 INFO:teuthology.orchestra.run.smithi194.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:22:20.543 INFO:teuthology.orchestra.run.smithi194.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:22:20.545 INFO:teuthology.orchestra.run.smithi194.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:22:20.553 INFO:teuthology.orchestra.run.smithi194.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:22:20.553 INFO:teuthology.orchestra.run.smithi194.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:22:20.598 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:22:20.611 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:22:20.628 INFO:teuthology.orchestra.run.smithi194.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:22:20.628 INFO:teuthology.orchestra.run.smithi194.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:22:20.655 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:22:20.699 INFO:teuthology.orchestra.run.smithi194.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:22:20.721 INFO:teuthology.orchestra.run.smithi194.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:22:20.725 INFO:teuthology.orchestra.run.smithi194.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:22:20.725 INFO:teuthology.orchestra.run.smithi194.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:22:20.726 INFO:teuthology.orchestra.run.smithi194.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:22:20.727 INFO:teuthology.orchestra.run.smithi194.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:22:20.792 INFO:teuthology.orchestra.run.smithi194.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:22:20.793 INFO:teuthology.orchestra.run.smithi194.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:22:21.017 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:22:21.030 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:21.075 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:21.280 INFO:teuthology.orchestra.run.smithi194.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:22:21.281 INFO:teuthology.orchestra.run.smithi194.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:22:21.281 INFO:teuthology.orchestra.run.smithi194.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:22:21.281 INFO:teuthology.orchestra.run.smithi194.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:22:21.281 INFO:teuthology.orchestra.run.smithi194.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:22:21.281 INFO:teuthology.orchestra.run.smithi194.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:22:21.294 INFO:teuthology.orchestra.run.smithi194.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:22:21.294 INFO:teuthology.orchestra.run.smithi194.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:22:21.412 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:22:21.425 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:21.461 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:22:21.679 INFO:teuthology.orchestra.run.smithi194.stdout:Preconfiguring packages ... 2024-09-10T21:22:21.765 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:22:21.778 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:21.823 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:22:21.836 INFO:teuthology.orchestra.run.smithi194.stdout:Fetched 55.8 MB in 4s (15.7 MB/s) 2024-09-10T21:22:22.050 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:22:22.063 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:22:22.108 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:22:22.130 INFO:teuthology.orchestra.run.smithi194.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:22.134 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:22.301 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:22.612 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:22:22.625 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:22:22.654 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:22:22.933 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:23.015 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:22:23.029 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:22:23.073 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:22:23.689 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:22:23.702 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:22:23.755 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:22:24.151 INFO:teuthology.orchestra.run.smithi194.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:22:24.371 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ppp. 2024-09-10T21:22:24.385 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:22:24.422 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package flex. 2024-09-10T21:22:24.495 INFO:teuthology.orchestra.run.smithi194.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:24.499 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:24.537 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:22:24.738 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:22:25.066 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:22:25.068 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:22:25.080 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:22:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:22:25.244 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:22:25.388 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:22:25.399 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:22:25.437 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:22:25.692 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:22:25.749 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:22:25.762 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:22:25.799 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:22:26.069 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:22:26.082 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:26.111 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:26.273 INFO:teuthology.orchestra.run.smithi194.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:22:26.277 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:26.384 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:26.716 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:22:26.725 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:22:26.784 INFO:teuthology.orchestra.run.smithi097.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:22:26.805 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:26.814 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:22:26.941 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:27.248 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:27.307 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:22:27.319 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:27.359 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:27.624 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package bison. 2024-09-10T21:22:27.697 INFO:teuthology.orchestra.run.smithi194.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:22:27.701 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:22:27.751 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:22:28.152 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:22:28.167 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:22:28.210 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:22:28.555 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package dump. 2024-09-10T21:22:28.571 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:22:28.613 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:22:28.835 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:22:28.848 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:22:28.884 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package indent. 2024-09-10T21:22:28.885 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:22:28.900 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:28.942 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:22:29.129 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:22:29.142 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:22:29.179 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:22:29.228 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:22:29.244 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:22:29.286 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:22:29.406 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:22:29.421 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:22:29.464 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:22:29.607 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:22:29.623 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:22:29.665 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:22:29.826 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:22:29.841 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:22:29.884 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:22:29.968 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:22:29.984 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:22:30.026 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:22:30.170 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package tcl. 2024-09-10T21:22:30.183 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:22:30.232 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:22:30.431 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:22:30.447 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:30.489 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:30.614 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:22:30.628 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:22:30.673 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:22:30.708 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:22:30.724 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:30.766 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:31.044 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:22:31.060 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:22:31.061 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:22:31.074 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:22:31.102 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:22:31.118 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:22:31.389 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:22:31.405 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:22:31.422 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:22:31.435 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:22:31.438 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:22:31.489 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:22:31.798 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:22:32.003 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:22:32.104 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:22:32.230 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:32.381 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:22:32.516 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:22:32.641 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:32.733 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:22:32.749 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:22:32.800 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:33.069 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:22:33.084 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:22:33.127 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:33.447 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:22:33.462 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:22:33.504 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:22:33.603 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:22:33.722 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:22:33.816 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:22:33.833 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:22:33.865 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:22:34.100 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:22:34.100 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:22:34.212 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:22:34.227 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:22:34.269 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:22:34.613 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:22:34.629 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:34.671 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:34.835 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:34.935 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:22:34.991 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:22:35.007 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:35.040 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:22:35.086 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:22:35.204 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:22:35.330 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:22:35.456 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:35.470 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:22:35.486 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:35.528 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:22:35.582 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:22:35.733 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:22:35.772 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:22:35.787 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:22:35.830 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:22:36.342 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:22:36.358 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:22:36.392 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:22:36.481 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:22:36.615 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:22:36.710 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:22:36.729 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:22:36.745 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:22:36.775 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:22:36.787 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:22:36.901 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:22:37.052 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:22:37.179 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:22:37.279 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:22:37.369 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:22:37.377 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:22:37.393 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:22:37.435 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:22:37.439 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:22:37.565 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:22:37.775 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:22:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:38.028 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:22:38.184 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package ppp. 2024-09-10T21:22:38.187 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:22:38.200 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:22:38.296 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:22:38.564 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:22:38.905 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:22:38.920 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:22:38.963 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:22:39.053 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:22:39.054 INFO:teuthology.orchestra.run.smithi097.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:22:39.054 INFO:teuthology.orchestra.run.smithi097.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:22:39.054 INFO:teuthology.orchestra.run.smithi097.stdout:- enp3s0f1 2024-09-10T21:22:39.054 INFO:teuthology.orchestra.run.smithi097.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:22:39.054 INFO:teuthology.orchestra.run.smithi097.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:22:39.054 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:22:39.186 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:22:39.232 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:22:39.247 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:22:39.291 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:22:39.558 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:22:39.569 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:22:39.584 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:22:39.619 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:22:39.923 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:22:39.938 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:39.972 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:22:39.981 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:40.611 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:22:40.626 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:22:40.687 INFO:teuthology.orchestra.run.smithi194.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:22:40.717 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:22:40.733 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:40.877 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:22:41.133 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:22:41.167 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:22:41.183 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:41.237 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:42.328 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:22:42.344 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:22:42.387 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:22:42.579 INFO:teuthology.orchestra.run.smithi097.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:22:42.639 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:22:42.654 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:22:42.689 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:22:42.847 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:22:42.848 INFO:teuthology.orchestra.run.smithi097.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:22:42.882 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:22:42.898 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:22:42.932 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:22:43.094 INFO:teuthology.orchestra.run.smithi097.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:22:43.095 INFO:teuthology.orchestra.run.smithi097.stdout:Removing obsolete dictionary files: 2024-09-10T21:22:43.277 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:22:43.292 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:22:43.335 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:22:43.613 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package tcl. 2024-09-10T21:22:43.629 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:22:43.675 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:22:43.809 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:22:43.990 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:22:44.007 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:22:44.040 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:22:44.273 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:44.371 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:22:44.388 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:22:44.446 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:22:44.656 INFO:teuthology.orchestra.run.smithi097.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:22:44.677 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout:This user must also own the server process. 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout:The default text search configuration will be set to "english". 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout:Data page checksums are disabled. 2024-09-10T21:22:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:22:44.721 INFO:teuthology.orchestra.run.smithi097.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:22:44.721 INFO:teuthology.orchestra.run.smithi097.stdout:creating subdirectories ... ok 2024-09-10T21:22:44.721 INFO:teuthology.orchestra.run.smithi097.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:22:44.745 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default max_connections ... 100 2024-09-10T21:22:44.778 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:22:44.792 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:22:44.792 INFO:teuthology.orchestra.run.smithi097.stdout:creating configuration files ... ok 2024-09-10T21:22:44.824 INFO:teuthology.orchestra.run.smithi194.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:22:44.841 INFO:teuthology.orchestra.run.smithi194.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:22:44.883 INFO:teuthology.orchestra.run.smithi194.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:22:44.953 INFO:teuthology.orchestra.run.smithi097.stdout:running bootstrap script ... ok 2024-09-10T21:22:45.169 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:22:45.380 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:22:45.515 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:22:45.541 INFO:teuthology.orchestra.run.smithi097.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:22:45.674 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:45.800 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:22:45.909 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:22:46.010 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:46.072 INFO:teuthology.orchestra.run.smithi097.stdout:syncing data to disk ... ok 2024-09-10T21:22:46.587 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:22:46.948 INFO:teuthology.orchestra.run.smithi194.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:22:47.048 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:22:47.409 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:22:47.409 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:22:48.085 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:48.185 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:22:48.353 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:22:48.479 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:22:48.630 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:22:48.756 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:48.882 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:22:49.008 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:22:49.623 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:22:49.688 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:22:49.758 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:22:49.847 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:22:49.884 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:22:49.942 INFO:teuthology.orchestra.run.smithi194.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:22:50.007 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:22:50.035 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:22:50.133 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:22:50.178 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:22:50.242 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:22:50.343 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:22:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:22:50.440 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:22:50.486 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:22:50.910 INFO:teuthology.orchestra.run.smithi194.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:22:51.013 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:22:51.139 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:22:51.163 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:22:51.271 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:22:51.441 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:51.558 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:22:51.559 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:22:51.710 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:22:51.718 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:22:51.828 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:22:52.404 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:22:52.404 INFO:teuthology.orchestra.run.smithi194.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:22:52.404 INFO:teuthology.orchestra.run.smithi194.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:22:52.404 INFO:teuthology.orchestra.run.smithi194.stdout:- enp3s0f1 2024-09-10T21:22:52.404 INFO:teuthology.orchestra.run.smithi194.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:22:52.404 INFO:teuthology.orchestra.run.smithi194.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:22:52.404 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:22:52.537 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:22:52.694 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:22:52.794 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:22:52.934 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:22:53.331 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:22:53.342 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:22:53.349 DEBUG:teuthology.parallel:result is None 2024-09-10T21:22:54.055 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:22:54.190 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:22:54.299 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:22:54.400 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:22:55.363 INFO:teuthology.orchestra.run.smithi194.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:22:55.606 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:22:55.607 INFO:teuthology.orchestra.run.smithi194.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:22:55.996 INFO:teuthology.orchestra.run.smithi194.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:22:55.996 INFO:teuthology.orchestra.run.smithi194.stdout:Removing obsolete dictionary files: 2024-09-10T21:22:56.704 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:22:57.168 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:22:57.507 INFO:teuthology.orchestra.run.smithi194.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:22:57.537 INFO:teuthology.orchestra.run.smithi194.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:22:57.599 INFO:teuthology.orchestra.run.smithi194.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:22:57.599 INFO:teuthology.orchestra.run.smithi194.stdout:This user must also own the server process. 2024-09-10T21:22:57.599 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:22:57.599 INFO:teuthology.orchestra.run.smithi194.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:22:57.600 INFO:teuthology.orchestra.run.smithi194.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:22:57.600 INFO:teuthology.orchestra.run.smithi194.stdout:The default text search configuration will be set to "english". 2024-09-10T21:22:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:22:57.600 INFO:teuthology.orchestra.run.smithi194.stdout:Data page checksums are disabled. 2024-09-10T21:22:57.600 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:22:57.600 INFO:teuthology.orchestra.run.smithi194.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:22:57.601 INFO:teuthology.orchestra.run.smithi194.stdout:creating subdirectories ... ok 2024-09-10T21:22:57.601 INFO:teuthology.orchestra.run.smithi194.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:22:57.625 INFO:teuthology.orchestra.run.smithi194.stdout:selecting default max_connections ... 100 2024-09-10T21:22:57.658 INFO:teuthology.orchestra.run.smithi194.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:22:57.672 INFO:teuthology.orchestra.run.smithi194.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:22:57.673 INFO:teuthology.orchestra.run.smithi194.stdout:creating configuration files ... ok 2024-09-10T21:22:57.834 INFO:teuthology.orchestra.run.smithi194.stdout:running bootstrap script ... ok 2024-09-10T21:22:58.443 INFO:teuthology.orchestra.run.smithi194.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:22:59.161 INFO:teuthology.orchestra.run.smithi194.stdout:syncing data to disk ... ok 2024-09-10T21:22:59.653 INFO:teuthology.orchestra.run.smithi194.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:23:02.559 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:02.676 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:23:02.801 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:23:02.919 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:03.042 INFO:teuthology.orchestra.run.smithi194.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:03.297 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:23:03.297 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:23:04.081 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:23:04.173 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:23:04.533 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:23:04.693 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:23:06.154 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:23:06.238 INFO:teuthology.orchestra.run.smithi194.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:23:06.789 INFO:teuthology.orchestra.run.smithi194.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:23:06.796 DEBUG:teuthology.parallel:result is None 2024-09-10T21:23:06.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:23:06.974 DEBUG:teuthology.orchestra.run.smithi097:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:23:06.998 INFO:teuthology.orchestra.run.smithi097.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:06.999 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:06.999 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:23:07.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:23:07.147 DEBUG:teuthology.orchestra.run.smithi194:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:23:07.169 INFO:teuthology.orchestra.run.smithi194.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:07.169 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:07.169 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:23:07.171 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T21:23:07.171 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:07.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:23:07.193 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:07.194 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:23:07.231 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T21:23:07.308 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:07.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:23:07.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:23:07.389 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:07.389 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:23:07.408 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:23:07.469 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T21:23:07.469 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:07.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:23:07.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:23:07.553 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:07.553 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:23:07.572 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:23:07.633 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T21:23:07.634 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:07.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:23:07.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:23:07.717 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:07.717 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:23:07.736 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:23:07.797 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T21:23:07.874 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T21:23:07.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-09-10T21:23:07.877 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /var/log/ceph 2024-09-10T21:23:07.895 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T21:23:07.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:23:07.940 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:23:07.958 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T21:23:07.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:23:08.004 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:23:08.028 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T21:23:08.028 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T21:23:08.028 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T21:23:08.028 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:08.028 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:23:08.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:23:08.166 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:08.166 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:23:08.174 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:23:08.263 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T21:23:08.264 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T21:23:08.264 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899672', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtCTET+JXWSFrWMIdPYsX3SSOn20F/v6ffpa+dyd/nD3cVWTI9yapCu61mFqtfz2rVLTHxXDesY+vXlrnb1MNY=', 'smithi194.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZx1F8fm3r4iIkL74/lM7FVJJ1oK/MvM16ORjqNQQYhtBTar1fcHzKZHp0XBOOC12ccakVORGryDW89jI+DoOc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T21:23:08.265 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:23:08.268 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:23:08.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:23:08.320 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:23:08.366 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:08.366 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:23:08.389 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:23:08.390 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-09-10T21:23:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:23:08.442 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:08.442 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 889 Links: 1 2024-09-10T21:23:08.442 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:08.442 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-10 21:16:22.400955408 +0000 2024-09-10T21:23:08.443 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-10 21:16:22.140967610 +0000 2024-09-10T21:23:08.443 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-10 21:16:22.140967610 +0000 2024-09-10T21:23:08.443 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-10 21:16:22.140967610 +0000 2024-09-10T21:23:08.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:23:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-10T21:23:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-10T21:23:08.504 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000395297 s, 1.3 MB/s 2024-09-10T21:23:08.506 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:23:08.564 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-09-10T21:23:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:23:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:08.615 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 905 Links: 1 2024-09-10T21:23:08.615 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:08.615 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-10 21:16:22.952929502 +0000 2024-09-10T21:23:08.615 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-10 21:16:22.704941140 +0000 2024-09-10T21:23:08.615 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-10 21:16:22.704941140 +0000 2024-09-10T21:23:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-10 21:16:22.704941140 +0000 2024-09-10T21:23:08.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:23:08.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-10T21:23:08.676 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-10T21:23:08.676 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000378024 s, 1.4 MB/s 2024-09-10T21:23:08.678 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:23:08.732 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-09-10T21:23:08.782 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:23:08.782 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:08.782 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 922 Links: 1 2024-09-10T21:23:08.782 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:08.782 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-10 21:16:23.500903783 +0000 2024-09-10T21:23:08.782 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-10 21:16:23.248915610 +0000 2024-09-10T21:23:08.782 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-10 21:16:23.248915610 +0000 2024-09-10T21:23:08.783 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-10 21:16:23.248915610 +0000 2024-09-10T21:23:08.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:23:08.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-10T21:23:08.848 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-10T21:23:08.848 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000382113 s, 1.3 MB/s 2024-09-10T21:23:08.850 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:23:08.907 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 942 Links: 1 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-10 21:16:23.808889327 +0000 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-10 21:16:23.804889515 +0000 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-10 21:16:23.804889515 +0000 2024-09-10T21:23:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-10 21:16:23.804889515 +0000 2024-09-10T21:23:08.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:23:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-10T21:23:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-10T21:23:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000407856 s, 1.3 MB/s 2024-09-10T21:23:09.026 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:23:09.084 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:09.084 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:09.084 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:23:09.093 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:23:09.093 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 899 Links: 1 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-10 21:16:23.698753251 +0000 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-10 21:16:23.414755787 +0000 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-10 21:16:23.414755787 +0000 2024-09-10T21:23:09.148 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-10 21:16:23.414755787 +0000 2024-09-10T21:23:09.149 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:23:09.209 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-10T21:23:09.210 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-10T21:23:09.210 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000416693 s, 1.2 MB/s 2024-09-10T21:23:09.212 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:23:09.265 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 914 Links: 1 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-10 21:16:24.266748177 +0000 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-10 21:16:24.022750356 +0000 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-10 21:16:24.022750356 +0000 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-10 21:16:24.022750356 +0000 2024-09-10T21:23:09.317 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:23:09.378 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-10T21:23:09.378 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-10T21:23:09.378 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000384449 s, 1.3 MB/s 2024-09-10T21:23:09.380 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:23:09.437 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 930 Links: 1 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-10 21:16:24.582745355 +0000 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-10 21:16:24.578745391 +0000 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-10 21:16:24.578745391 +0000 2024-09-10T21:23:09.492 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-10 21:16:24.578745391 +0000 2024-09-10T21:23:09.493 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:23:09.554 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-10T21:23:09.554 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-10T21:23:09.554 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000387048 s, 1.3 MB/s 2024-09-10T21:23:09.556 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:23:09.613 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 948 Links: 1 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-10 21:16:25.406737995 +0000 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-10 21:16:25.138740388 +0000 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-10 21:16:25.138740388 +0000 2024-09-10T21:23:09.668 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-10 21:16:25.138740388 +0000 2024-09-10T21:23:09.668 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:23:09.729 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-10T21:23:09.729 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-10T21:23:09.729 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000397935 s, 1.3 MB/s 2024-09-10T21:23:09.731 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:23:09.787 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:09.788 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi194.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:23:09.788 INFO:tasks.ceph:Generating config... 2024-09-10T21:23:09.789 INFO:tasks.ceph:[client] pid file = 2024-09-10T21:23:09.789 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T21:23:09.789 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T21:23:09.790 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T21:23:09.790 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T21:23:09.790 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T21:23:09.790 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T21:23:09.790 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T21:23:09.790 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T21:23:09.790 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T21:23:09.790 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T21:23:09.790 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T21:23:09.790 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T21:23:09.790 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T21:23:09.790 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T21:23:09.790 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T21:23:09.791 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T21:23:09.791 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T21:23:09.791 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T21:23:09.791 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T21:23:09.791 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T21:23:09.791 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T21:23:09.791 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T21:23:09.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T21:23:09.836 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T21:23:09.846 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T21:23:09.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:23:09.956 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97'), ('mon.b', '172.21.15.194'), ('mon.c', '[v2:172.21.15.194:3301,v1:172.21.15.194:6790]')] 2024-09-10T21:23:09.957 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97,172.21.15.194,[v2:172.21.15.194:3301,v1:172.21.15.194:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-10T21:23:09.958 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:09.958 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:23:10.009 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --add b 172.21.15.194 --addv c '[v2:172.21.15.194:3301,v1:172.21.15.194:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:10.100 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:10.100 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid d1b4b680-9a7d-4173-b750-f3c8b85e2f7b 2024-09-10T21:23:10.100 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:fsid d1b4b680-9a7d-4173-b750-f3c8b85e2f7b 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2024-09-10T21:23:10.098049+0000 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-09-10T21:23:10.098049+0000 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 17 (quincy) 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.b 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:2: [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] mon.c 2024-09-10T21:23:10.101 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T21:23:10.103 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:23:10.154 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d1b4b680-9a7d-4173-b750-f3c8b85e2f7b... 2024-09-10T21:23:10.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:23:10.200 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-09-10T21:23:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.240 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.241 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.194,[v2:172.21.15.194:3301,v1:172.21.15.194:6790]" 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on skipped message = False 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = d1b4b680-9a7d-4173-b750-f3c8b85e2f7b 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.242 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-09-10T21:23:10.243 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-09-10T21:23:10.244 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 20 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 20 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 10 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:23:10.245 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 30 2024-09-10T21:23:10.246 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.247 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: mon down mkfs grace = 300 2024-09-10T21:23:10.248 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-09-10T21:23:10.249 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout:[global] 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: chdir = "" 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: auth supported = cephx 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: filestore xattr use omap = true 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: mon clock drift allowed = 1.000 2024-09-10T21:23:10.250 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: osd crush chooseleaf type = 0 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: auth debug = true 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: ms die on old message = true 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: ms die on bug = true 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: mon pg warn max object skew = 0 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: osd pool default size = 2 2024-09-10T21:23:10.251 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd allow primary affinity = true 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd allow pg remap = true 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on no sortbitwise = false 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon warn on too few osds = false 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: mon_allow_pool_size_one = true 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: osd default data pool replay window = 5 2024-09-10T21:23:10.252 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: mon allow pool delete = true 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: mon cluster log file level = debug 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: debug asserts on shutdown = true 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: mon health detail to clog = false 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: mon host = "172.21.15.97,172.21.15.194,[v2:172.21.15.194:3301,v1:172.21.15.194:6790]" 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: ms die on skipped message = False 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: fsid = d1b4b680-9a7d-4173-b750-f3c8b85e2f7b 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout:[osd] 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: osd journal size = 100 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.253 INFO:teuthology.orchestra.run.smithi194.stdout: osd scrub load threshold = 5.0 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd scrub max interval = 600 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd recover clone overlap = true 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd recovery max chunk = 1048576 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug shutdown = true 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug op order = true 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug verify stray on activate = true 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug trim objects = true 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd open classes on start = true 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug pg log writeout = true 2024-09-10T21:23:10.254 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: osd map max advance = 10 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: journal zero on create = true 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: filestore apply finisher threads = 3 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: bdev debug aio = true 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: osd debug misdirected ops = true 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore block size = 96636764160 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: bluestore fsck on mount = True 2024-09-10T21:23:10.255 INFO:teuthology.orchestra.run.smithi194.stdout: debug bluefs = 20 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: debug bluestore = 20 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: debug osd = 20 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: debug rocksdb = 10 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd full ratio = 0.9 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: osd objectstore = bluestore 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout:[mgr] 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: debug mgr = 20 2024-09-10T21:23:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: debug mon = 20 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: debug auth = 20 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: mgr/telemetry/nag = false 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: client mount timeout = 30 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: debug client = 20 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout:[mon] 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: debug ms = 1 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: debug mon = 20 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: debug paxos = 20 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: debug auth = 20 2024-09-10T21:23:10.257 INFO:teuthology.orchestra.run.smithi194.stdout: mon data avail warn = 5 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: mon mgr mkfs grace = 240 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd reporter subtree level = osd 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: mon osd prime pg temp = true 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:23:10.258 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: # 1m isn't quite enough 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: mon down mkfs grace = 300 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout:[client] 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: rgw cache enabled = true 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: rgw enable ops log = true 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: rgw enable usage log = true 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout: pid file = "" 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.a] 2024-09-10T21:23:10.259 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.b] 2024-09-10T21:23:10.260 INFO:teuthology.orchestra.run.smithi194.stdout:[mon.c] 2024-09-10T21:23:10.260 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.a] 2024-09-10T21:23:10.260 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.c] 2024-09-10T21:23:10.260 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.b] 2024-09-10T21:23:10.260 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.d] 2024-09-10T21:23:10.261 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T21:23:10.261 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T21:23:10.362 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T21:23:10.362 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:10.362 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T21:23:10.371 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:10.371 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T21:23:10.421 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:23:10.421 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:10.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:23:10.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:23:10.497 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:10.497 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:10.549 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:23:10.549 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:10.549 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:23:10.549 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:23:10.584 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:10.584 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:10.635 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T21:23:10.635 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T21:23:10.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:23:10.690 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:23:10.695 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:23:10.754 INFO:teuthology.orchestra.run.smithi194.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:23:10.774 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T21:23:10.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:23:10.895 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:23:10.899 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T21:23:10.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:23:11.019 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:23:11.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T21:23:11.042 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:23:11.102 INFO:teuthology.orchestra.run.smithi194.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:23:11.107 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T21:23:11.128 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:23:11.226 INFO:teuthology.orchestra.run.smithi194.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:23:11.230 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T21:23:11.250 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T21:23:11.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T21:23:11.292 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T21:23:11.309 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T21:23:11.394 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T21:23:11.410 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T21:23:11.453 INFO:teuthology.orchestra.run.smithi194.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T21:23:11.470 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T21:23:11.552 INFO:teuthology.orchestra.run.smithi194.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T21:23:11.569 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T21:23:11.569 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi194.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:23:11.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:11.589 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:11.590 INFO:tasks.ceph:role: osd.0 2024-09-10T21:23:11.590 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:23:11.590 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:11.660 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:11.661 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:12.068 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-10T21:23:12.071 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-10T21:23:12.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:12.214 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:12.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:12.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:12.297 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:12.298 INFO:tasks.ceph:role: osd.1 2024-09-10T21:23:12.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:23:12.298 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:12.369 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:12.727 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-10T21:23:12.729 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-10T21:23:12.730 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:12.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:12.870 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:12.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:12.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:12.933 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:12.933 INFO:tasks.ceph:role: osd.2 2024-09-10T21:23:12.934 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:23:12.934 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:23:13.005 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:13.005 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:13.006 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:13.006 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:13.006 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:13.006 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:13.006 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:13.401 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-10T21:23:13.404 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-10T21:23:13.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:13.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:13.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:13.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T21:23:13.593 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:13.593 INFO:tasks.ceph:role: osd.3 2024-09-10T21:23:13.593 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:23:13.593 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:13.665 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:13.666 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:14.053 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-10T21:23:14.056 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-10T21:23:14.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T21:23:14.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T21:23:14.215 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:14.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:14.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:14.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:14.345+0000 7f92995d67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T21:23:14.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:14.349+0000 7f92995d67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T21:23:14.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:14.349+0000 7f92995d67c0 -1 bdev(0x55712141f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:14.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:14.349+0000 7f92995d67c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T21:23:17.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:17.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:17.585+0000 7fcbb99b27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T21:23:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:17.585+0000 7fcbb99b27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T21:23:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:17.585+0000 7fcbb99b27c0 -1 bdev(0x561edffef800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:17.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:17.585+0000 7fcbb99b27c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T21:23:21.047 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:21.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:21.153+0000 7fca7a6127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T21:23:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:21.153+0000 7fca7a6127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T21:23:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:21.153+0000 7fca7a6127c0 -1 bdev(0x55beca55b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:21.153+0000 7fca7a6127c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T21:23:24.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:24.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:24.521+0000 7f176923d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T21:23:24.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:24.521+0000 7f176923d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T21:23:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:24.521+0000 7f176923d7c0 -1 bdev(0x56345debb800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:24.521+0000 7f176923d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T21:23:27.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T21:23:27.815 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T21:23:27.838 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:27.838 INFO:tasks.ceph:role: osd.4 2024-09-10T21:23:27.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:23:27.838 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:23:27.907 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:27.907 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:27.907 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:27.908 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:27.908 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:27.908 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:27.908 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:27.908 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:27.908 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:27.908 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:28.308 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-09-10T21:23:28.311 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-09-10T21:23:28.311 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T21:23:28.412 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T21:23:28.429 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:28.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:28.430 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T21:23:28.490 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:28.491 INFO:tasks.ceph:role: osd.5 2024-09-10T21:23:28.491 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:23:28.491 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:28.558 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:28.559 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:28.967 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-09-10T21:23:28.970 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-09-10T21:23:28.971 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T21:23:29.113 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T21:23:29.130 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:29.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:29.131 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T21:23:29.195 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:29.195 INFO:tasks.ceph:role: osd.6 2024-09-10T21:23:29.196 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:23:29.196 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:29.271 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:29.688 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-09-10T21:23:29.691 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-09-10T21:23:29.691 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T21:23:29.820 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T21:23:29.837 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:29.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:29.838 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T21:23:29.899 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:29.899 INFO:tasks.ceph:role: osd.7 2024-09-10T21:23:29.899 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:23:29.899 DEBUG:teuthology.orchestra.run.smithi194:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:23:29.974 INFO:teuthology.orchestra.run.smithi194.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:29.974 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:29.975 INFO:teuthology.orchestra.run.smithi194.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:30.343 INFO:teuthology.orchestra.run.smithi194.stdout:Discarding blocks...Done. 2024-09-10T21:23:30.346 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi194.front.sepia.ceph.com -o noatime 2024-09-10T21:23:30.346 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T21:23:30.430 DEBUG:teuthology.orchestra.run.smithi194:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T21:23:30.491 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:30.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:30.493 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:30.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:30.610+0000 7f828f4de7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T21:23:30.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:30.614+0000 7f828f4de7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T21:23:30.616 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:30.614+0000 7f828f4de7c0 -1 bdev(0x55d36c8d9800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:30.616 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:30.614+0000 7f828f4de7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T21:23:33.878 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T21:23:33.899 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:33.993 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:33.990+0000 7f43764dc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T21:23:33.994 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:33.990+0000 7f43764dc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T21:23:33.994 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:33.990+0000 7f43764dc7c0 -1 bdev(0x556a46d59800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:33.994 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:33.990+0000 7f43764dc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T21:23:37.231 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T21:23:37.253 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:37.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:37.346+0000 7f3e2743d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T21:23:37.349 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:37.346+0000 7f3e2743d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T21:23:37.349 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:37.346+0000 7f3e2743d7c0 -1 bdev(0x555e14135800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:37.349 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:37.346+0000 7f3e2743d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T21:23:38.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:23:38.321 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:23:40.481 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T21:23:40.501 DEBUG:teuthology.orchestra.run.smithi194:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:40.593 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:40.589+0000 7fbe074d47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T21:23:40.594 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:40.589+0000 7fbe074d47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T21:23:40.594 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:40.589+0000 7fbe074d47c0 -1 bdev(0x56259e583800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:40.594 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:40.589+0000 7fbe074d47c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T21:23:43.796 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T21:23:43.815 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T21:23:43.816 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:43.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T21:23:43.837 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:43.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T21:23:43.902 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:43.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T21:23:43.965 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:43.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T21:23:44.033 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:44.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T21:23:44.097 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:44.097 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T21:23:44.161 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:44.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T21:23:44.225 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.225 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T21:23:44.246 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.246 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T21:23:44.311 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.312 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T21:23:44.379 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.379 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T21:23:44.443 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.443 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T21:23:44.503 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.503 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T21:23:44.567 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.567 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T21:23:44.631 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:44.631 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T21:23:44.640 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:44.640 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T21:23:44.689 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.689 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T21:23:44.698 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:44.698 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T21:23:44.747 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T21:23:44.747 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:23:44.751 DEBUG:teuthology.orchestra.run.smithi194:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw== 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA== 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw== 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q== 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-09-10T21:23:44.795 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q== 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg== 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ== 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA== 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g== 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA== 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ== 2024-09-10T21:23:44.796 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw== 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ== 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q== 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag== 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ== 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2024-09-10T21:23:44.797 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA== 2024-09-10T21:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout:[mgr.y] 2024-09-10T21:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw== 2024-09-10T21:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.a] 2024-09-10T21:23:44.809 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.c] 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.0] 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.1] 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.2] 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.3] 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[mgr.x] 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.b] 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g== 2024-09-10T21:23:44.810 INFO:teuthology.orchestra.run.smithi194.stdout:[mds.d] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA== 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.4] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ== 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.5] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw== 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.6] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ== 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout:[osd.7] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q== 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout:[client.0] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout:[client.2] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag== 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout:[client.1] 2024-09-10T21:23:44.811 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ== 2024-09-10T21:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout:[client.3] 2024-09-10T21:23:44.812 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA== 2024-09-10T21:23:44.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:23:44.848 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:23:44.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:44.915 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:44.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:44.976 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:45.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.036 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.092 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.147 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.204 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.257 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:23:45.260 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:23:45.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:45.315 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:45.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:45.375 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:23:45.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.435 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.491 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.544 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.548 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.603 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:23:45.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.659 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.719 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.779 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.839 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:23:45.896 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T21:23:45.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T21:23:45.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:23:46.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T21:23:46.049 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T21:23:46.068 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:23:46.178 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T21:23:46.198 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T21:23:46.257 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:23:46.363 DEBUG:teuthology.orchestra.run.smithi194:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T21:23:46.382 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:46.385 DEBUG:teuthology.orchestra.run.smithi194:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:46.431 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T21:23:46.431 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:23:46.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:23:46.435 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:23:46.436 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T21:23:46.436 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T21:23:46.474 INFO:tasks.ceph.mon.b:Started 2024-09-10T21:23:46.474 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:23:46.474 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:23:46.477 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:23:46.477 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T21:23:46.477 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T21:23:46.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T21:23:46.480 INFO:tasks.ceph.mgr.y:Started 2024-09-10T21:23:46.480 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T21:23:46.481 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T21:23:46.483 INFO:tasks.ceph.mgr.x:Started 2024-09-10T21:23:46.484 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T21:23:46.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T21:23:46.487 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T21:23:46.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T21:23:46.532 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T21:23:46.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T21:23:46.580 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T21:23:46.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T21:23:46.629 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T21:23:46.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T21:23:46.676 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T21:23:46.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T21:23:46.725 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T21:23:46.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T21:23:46.772 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T21:23:46.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T21:23:46.819 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T21:23:46.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T21:23:46.868 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T21:23:46.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T21:23:46.916 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T21:23:46.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T21:23:46.969 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T21:23:46.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T21:23:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.976+0000 7fc987400640 1 -- 172.21.15.97:0/4051163444 <== mon.2 v2:172.21.15.194:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v4)=0 v4) ==== 109+0+0 (secure 0 0 0) 0x7fc97c010c00 con 0x7fc99015c200 2024-09-10T21:23:46.985 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T21:23:46.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T21:23:46.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 -- 172.21.15.97:0/1572687845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbef80052f0 msgr2=0x7fbf140b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:46.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 --2- 172.21.15.97:0/1572687845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbef80052f0 0x7fbf140b34e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf04010a10 tx=0x7fbf04010e10 comp rx=0 tx=0).stop 2024-09-10T21:23:46.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 -- 172.21.15.97:0/1572687845 shutdown_connections 2024-09-10T21:23:46.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 --2- 172.21.15.97:0/1572687845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf14150520 0x7fbf140c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:46.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 --2- 172.21.15.97:0/1572687845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf14151030 0x7fbf140b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:46.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 --2- 172.21.15.97:0/1572687845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbef80052f0 0x7fbf140b34e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:46.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 -- 172.21.15.97:0/1572687845 >> 172.21.15.97:0/1572687845 conn(0x7fbf140d1a60 msgr2=0x7fbf140d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:46.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 -- 172.21.15.97:0/1572687845 shutdown_connections 2024-09-10T21:23:46.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.980+0000 7fbf19a00640 1 -- 172.21.15.97:0/1572687845 wait complete. 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.992+0000 7fc996200640 1 -- 172.21.15.97:0/4051163444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc99015c200 msgr2=0x7fc99015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.992+0000 7fc996200640 1 --2- 172.21.15.97:0/4051163444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc99015c200 0x7fc99015c680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc97c003ea0 tx=0x7fc97c003f90 comp rx=0 tx=0).stop 2024-09-10T21:23:47.004 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T21:23:47.004 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.992+0000 7fc996200640 1 -- 172.21.15.97:0/4051163444 shutdown_connections 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.992+0000 7fc996200640 1 --2- 172.21.15.97:0/4051163444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99015cbc0 0x7fc99014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.992+0000 7fc996200640 1 --2- 172.21.15.97:0/4051163444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc99015c200 0x7fc99015c680 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.992+0000 7fc996200640 1 --2- 172.21.15.97:0/4051163444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc974004e70 0x7fc99014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:46.992+0000 7fc996200640 1 -- 172.21.15.97:0/4051163444 >> 172.21.15.97:0/4051163444 conn(0x7fc9900d3e70 msgr2=0x7fc99015e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.000+0000 7fc996200640 1 -- 172.21.15.97:0/4051163444 shutdown_connections 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.000+0000 7fc996200640 1 -- 172.21.15.97:0/4051163444 wait complete. 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.000+0000 7f685d000640 1 -- 172.21.15.97:0/3911028829 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f682c005110 con 0x7f683c003a10 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.004+0000 7f684f400640 1 -- 172.21.15.97:0/3911028829 <== mon.1 v2:172.21.15.194:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f6848003c00 con 0x7f683c003a10 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.004+0000 7feadcc00640 1 -- 172.21.15.97:0/1783469531 <== mon.1 v2:172.21.15.194:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fead0010760 con 0x7feae0070d50 2024-09-10T21:23:47.016 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T21:23:47.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.008+0000 7f684f400640 1 -- 172.21.15.97:0/3911028829 <== mon.1 v2:172.21.15.194:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f6848003e30 con 0x7f683c003a10 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 -- 172.21.15.97:0/3747448755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 msgr2=0x7f0648073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 --2- 172.21.15.97:0/3747448755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 0x7f0648073120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06480b2650 tx=0x7f064001cd10 comp rx=0 tx=0).stop 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 -- 172.21.15.97:0/3747448755 shutdown_connections 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 --2- 172.21.15.97:0/3747448755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 0x7f0648073120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 --2- 172.21.15.97:0/3747448755 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 0x7f064815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 --2- 172.21.15.97:0/3747448755 >> v2:172.21.15.97:3300/0 conn(0x7f0648161790 0x7f06480d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 -- 172.21.15.97:0/3747448755 >> 172.21.15.97:0/3747448755 conn(0x7f06480cf890 msgr2=0x7f06480cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f685d000640 1 -- 172.21.15.97:0/3911028829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f683c003a10 msgr2=0x7f68581482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.037 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T21:23:47.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T21:23:47.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f685d000640 1 --2- 172.21.15.97:0/3911028829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f683c003a10 0x7f68581482e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f683c00bb00 tx=0x7f683c005840 comp rx=0 tx=0).stop 2024-09-10T21:23:47.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f685d000640 1 -- 172.21.15.97:0/3911028829 shutdown_connections 2024-09-10T21:23:47.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f685d000640 1 --2- 172.21.15.97:0/3911028829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68581e5290 0x7f685815a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f685d000640 1 --2- 172.21.15.97:0/3911028829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68581f7d30 0x7f6858159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f685d000640 1 --2- 172.21.15.97:0/3911028829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f683c003a10 0x7f68581482e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f685d000640 1 -- 172.21.15.97:0/3911028829 >> 172.21.15.97:0/3911028829 conn(0x7f685806cb40 msgr2=0x7f6858155200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f685d000640 1 -- 172.21.15.97:0/3911028829 shutdown_connections 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f685d000640 1 -- 172.21.15.97:0/3911028829 wait complete. 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 -- 172.21.15.97:0/3747448755 shutdown_connections 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.028+0000 7f064e400640 1 -- 172.21.15.97:0/3747448755 wait complete. 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.032+0000 7f064e400640 1 Processor -- start 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f064e400640 1 -- start start 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f064e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 0x7f064816bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f064e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0648161790 0x7f06480ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f064e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 0x7f06481fb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f064d000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 0x7f064816bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f064d000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 0x7f064816bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:47204/0 (socket says 172.21.15.97:47204) 2024-09-10T21:23:47.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f0647e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0648161790 0x7f06480ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.036+0000 7f0647e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0648161790 0x7f06480ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45482/0 (socket says 172.21.15.97:45482) 2024-09-10T21:23:47.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.040+0000 7f064d000640 1 -- 172.21.15.97:0/3574132568 learned_addr learned my addr 172.21.15.97:0/3574132568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.040+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0648157280 con 0x7f0648161790 2024-09-10T21:23:47.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.040+0000 7f064da00640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 0x7f06481fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.040+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0648157100 con 0x7f0648161b70 2024-09-10T21:23:47.049 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T21:23:47.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T21:23:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.040+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0648157400 con 0x7f06480b43e0 2024-09-10T21:23:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f0647e00640 1 -- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 msgr2=0x7f064816bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f0647e00640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 0x7f064816bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f0647e00640 1 -- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 msgr2=0x7f06481fb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f0647e00640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 0x7f06481fb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f0647e00640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06481fc310 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f0647e00640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0648161790 0x7f06480ce050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f063400c940 tx=0x7f063400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f064da00640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 0x7f06481fb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f064d000640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 0x7f064816bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.044+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634004220 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.048+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f0634004490 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.048+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634007bb0 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.048+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06481fc5f0 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.048+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064816b440 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.048+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f06340053b0 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.048+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06481578a0 con 0x7f0648161790 2024-09-10T21:23:47.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.048+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0634007d50 con 0x7f0648161790 2024-09-10T21:23:47.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.052+0000 7f255e000640 1 -- 172.21.15.97:0/89093390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558073980 msgr2=0x7f2558146ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.052+0000 7f255e000640 1 --2- 172.21.15.97:0/89093390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558073980 0x7f2558146ba0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2558162bb0 tx=0x7f254401c840 comp rx=0 tx=0).stop 2024-09-10T21:23:47.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.052+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f0634014030 con 0x7f0648161790 2024-09-10T21:23:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.056+0000 7f255e000640 1 -- 172.21.15.97:0/89093390 shutdown_connections 2024-09-10T21:23:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.056+0000 7f255e000640 1 --2- 172.21.15.97:0/89093390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2558161ea0 0x7f25581470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.067 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T21:23:47.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T21:23:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.056+0000 7f255e000640 1 --2- 172.21.15.97:0/89093390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558073980 0x7f2558146ba0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.056+0000 7f255e000640 1 --2- 172.21.15.97:0/89093390 >> v2:172.21.15.194:3300/0 conn(0x7f2558161ac0 0x7f25580b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.056+0000 7f255e000640 1 -- 172.21.15.97:0/89093390 >> 172.21.15.97:0/89093390 conn(0x7f255806cb40 msgr2=0x7f255806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.072+0000 7f255e000640 1 -- 172.21.15.97:0/89093390 shutdown_connections 2024-09-10T21:23:47.081 DEBUG:tasks.ceph:set 18 configs 2024-09-10T21:23:47.081 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.081 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.081 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.082 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.076+0000 7f255e000640 1 -- 172.21.15.97:0/89093390 wait complete. 2024-09-10T21:23:47.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.076+0000 7f255e000640 1 Processor -- start 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f255e000640 1 -- start start 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f255e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2558073980 0x7f255813f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f255e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558161ac0 0x7f25581424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f255e000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2558161ea0 0x7f25581429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2557400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2558073980 0x7f255813f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2557400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2558073980 0x7f255813f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:53624/0 (socket says 172.21.15.97:53624) 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2556a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558161ac0 0x7f25581424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2556a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558161ac0 0x7f25581424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45486/0 (socket says 172.21.15.97:45486) 2024-09-10T21:23:47.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f255815f080 con 0x7f2558161ac0 2024-09-10T21:23:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2557400640 1 -- 172.21.15.97:0/263637588 learned_addr learned my addr 172.21.15.97:0/263637588 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f255815ef00 con 0x7f2558073980 2024-09-10T21:23:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f255815f430 con 0x7f2558161ea0 2024-09-10T21:23:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2557e00640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2558161ea0 0x7f25581429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2556a00640 1 -- 172.21.15.97:0/263637588 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2558161ea0 msgr2=0x7f25581429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2556a00640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2558161ea0 0x7f25581429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2556a00640 1 -- 172.21.15.97:0/263637588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2558073980 msgr2=0x7f255813f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2556a00640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2558073980 0x7f255813f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.084+0000 7f2556a00640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255813f820 con 0x7f2558161ac0 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f2556a00640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558161ac0 0x7f25581424b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2558074a70 tx=0x7f2544004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440ba020 con 0x7f2558161ac0 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2558146ba0 con 0x7f2558161ac0 2024-09-10T21:23:47.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2558155f50 con 0x7f2558161ac0 2024-09-10T21:23:47.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f2544004430 con 0x7f2558161ac0 2024-09-10T21:23:47.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440b2040 con 0x7f2558161ac0 2024-09-10T21:23:47.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2544005370 con 0x7f2558161ac0 2024-09-10T21:23:47.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.088+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2544004d90 con 0x7f2558161ac0 2024-09-10T21:23:47.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.092+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25300052a0 con 0x7f2558161ac0 2024-09-10T21:23:47.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.104+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f25440d7950 con 0x7f2558161ac0 2024-09-10T21:23:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.136+0000 7fd76f200640 1 -- 172.21.15.97:0/458356242 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 msgr2=0x7fd77006f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.136+0000 7fd76f200640 1 --2- 172.21.15.97:0/458356242 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 0x7fd77006f3a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd768009c80 tx=0x7fd76801cd70 comp rx=0 tx=0).stop 2024-09-10T21:23:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.140+0000 7fd76f200640 1 -- 172.21.15.97:0/458356242 shutdown_connections 2024-09-10T21:23:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.140+0000 7fd76f200640 1 --2- 172.21.15.97:0/458356242 >> v2:172.21.15.97:3300/0 conn(0x7fd77006f8e0 0x7fd7700cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.140+0000 7fd76f200640 1 --2- 172.21.15.97:0/458356242 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 0x7fd77006f3a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.140+0000 7fd76f200640 1 -- 172.21.15.97:0/458356242 >> 172.21.15.97:0/458356242 conn(0x7fd7700b16a0 msgr2=0x7fd7700b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.160 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.152+0000 7fd76f200640 1 -- 172.21.15.97:0/458356242 shutdown_connections 2024-09-10T21:23:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.152+0000 7fd76f200640 1 -- 172.21.15.97:0/458356242 wait complete. 2024-09-10T21:23:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.156+0000 7fd76f200640 1 Processor -- start 2024-09-10T21:23:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76f200640 1 -- start start 2024-09-10T21:23:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77006f8e0 0x7fd7701fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 0x7fd77014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd77015c000 0x7fd770148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd770168bc0 con 0x7fd77006f8e0 2024-09-10T21:23:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd770168a40 con 0x7fd77015c000 2024-09-10T21:23:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd770168d40 con 0x7fd77015b9e0 2024-09-10T21:23:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd77015c000 0x7fd770148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd77015c000 0x7fd770148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:53632/0 (socket says 172.21.15.97:53632) 2024-09-10T21:23:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 -- 172.21.15.97:0/1250769197 learned_addr learned my addr 172.21.15.97:0/1250769197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76d400640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 0x7fd77014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76de00640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77006f8e0 0x7fd7701fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 -- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 msgr2=0x7fd77014f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 0x7fd77014f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 -- 172.21.15.97:0/1250769197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77006f8e0 msgr2=0x7fd7701fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77006f8e0 0x7fd7701fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 -- 172.21.15.97:0/1250769197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7702004e0 con 0x7fd77015c000 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.168+0000 7fd76e800640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd77015c000 0x7fd770148c30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00b410 tx=0x7fd75c00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.172+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c015020 con 0x7fd77015c000 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.172+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd75c011900 con 0x7fd77015c000 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.172+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd770152080 con 0x7fd77015c000 2024-09-10T21:23:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.172+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77014da10 con 0x7fd77015c000 2024-09-10T21:23:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.172+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c011d30 con 0x7fd77015c000 2024-09-10T21:23:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.172+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd75c01a790 con 0x7fd77015c000 2024-09-10T21:23:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.172+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd75c019c10 con 0x7fd77015c000 2024-09-10T21:23:47.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.184+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7480052a0 con 0x7fd77015c000 2024-09-10T21:23:47.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.192+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd75c01f020 con 0x7fd77015c000 2024-09-10T21:23:47.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.196+0000 7feae7400640 1 -- 172.21.15.97:0/1783469531 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7feab0005320 con 0x7feae0070d50 2024-09-10T21:23:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.212+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25440a57b0 con 0x7f2558161ac0 2024-09-10T21:23:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.220+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06340048c0 con 0x7f0648161790 2024-09-10T21:23:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.220+0000 7feadcc00640 1 -- 172.21.15.97:0/1783469531 <== mon.1 v2:172.21.15.194:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fead0028020 con 0x7feae0070d50 2024-09-10T21:23:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.220+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd75c019e60 con 0x7fd77015c000 2024-09-10T21:23:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.224+0000 7feadcc00640 1 -- 172.21.15.97:0/1783469531 <== mon.1 v2:172.21.15.194:3300/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7fead0010d00 con 0x7feae0070d50 2024-09-10T21:23:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.224+0000 7f6eda400640 1 -- 172.21.15.97:0/1866941594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed415b4f0 msgr2=0x7f6ed415e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.224+0000 7f6eda400640 1 --2- 172.21.15.97:0/1866941594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed415b4f0 0x7f6ed415e9c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4075aa0 tx=0x7f6ebc01c570 comp rx=0 tx=0).stop 2024-09-10T21:23:47.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 -- 172.21.15.97:0/1866941594 shutdown_connections 2024-09-10T21:23:47.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 --2- 172.21.15.97:0/1866941594 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ed40addf0 0x7f6ed4161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 --2- 172.21.15.97:0/1866941594 >> v2:172.21.15.97:3300/0 conn(0x7f6ed40d03b0 0x7f6ed40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 --2- 172.21.15.97:0/1866941594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed415b4f0 0x7f6ed415e9c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 -- 172.21.15.97:0/1866941594 >> 172.21.15.97:0/1866941594 conn(0x7f6ed40cf890 msgr2=0x7f6ed40cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 -- 172.21.15.97:0/1866941594 shutdown_connections 2024-09-10T21:23:47.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 -- 172.21.15.97:0/1866941594 wait complete. 2024-09-10T21:23:47.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.228+0000 7f6eda400640 1 Processor -- start 2024-09-10T21:23:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6eda400640 1 -- start start 2024-09-10T21:23:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6eda400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed40addf0 0x7f6ed41521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6eda400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ed40d03b0 0x7f6ed41602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6eda400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed415b4f0 0x7f6ed4157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6eda400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4072a20 con 0x7f6ed415b4f0 2024-09-10T21:23:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6eda400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed40728a0 con 0x7f6ed40addf0 2024-09-10T21:23:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6eda400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4072ba0 con 0x7f6ed40d03b0 2024-09-10T21:23:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed40addf0 0x7f6ed41521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ed40d03b0 0x7f6ed41602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed415b4f0 0x7f6ed4157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed40addf0 0x7f6ed41521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:53636/0 (socket says 172.21.15.97:53636) 2024-09-10T21:23:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 -- 172.21.15.97:0/3403357930 learned_addr learned my addr 172.21.15.97:0/3403357930 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 -- 172.21.15.97:0/3403357930 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ed40d03b0 msgr2=0x7f6ed41602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 --2- 172.21.15.97:0/3403357930 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ed40d03b0 0x7f6ed41602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 -- 172.21.15.97:0/3403357930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed415b4f0 msgr2=0x7f6ed4157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 --2- 172.21.15.97:0/3403357930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed415b4f0 0x7f6ed4157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 -- 172.21.15.97:0/3403357930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed4153df0 con 0x7f6ed40addf0 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed9000640 1 --2- 172.21.15.97:0/3403357930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed40addf0 0x7f6ed41521a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc01ac20 tx=0x7f6ebc0a9570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7feae7400640 1 -- 172.21.15.97:0/1783469531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feae0070d50 msgr2=0x7feae016dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7feae7400640 1 --2- 172.21.15.97:0/1783469531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feae0070d50 0x7feae016dd80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fead0001100 tx=0x7fead00015c0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed0c00640 1 -- 172.21.15.97:0/3403357930 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0ad070 con 0x7f6ed40addf0 2024-09-10T21:23:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed0c00640 1 -- 172.21.15.97:0/3403357930 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ebc0a9d20 con 0x7f6ed40addf0 2024-09-10T21:23:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7feae7400640 1 -- 172.21.15.97:0/1783469531 shutdown_connections 2024-09-10T21:23:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7feae7400640 1 --2- 172.21.15.97:0/1783469531 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feae0161a30 0x7feae00cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7feae7400640 1 --2- 172.21.15.97:0/1783469531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feae0070d50 0x7feae016dd80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7feae7400640 1 --2- 172.21.15.97:0/1783469531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae006fd60 0x7feae00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7feae7400640 1 -- 172.21.15.97:0/1783469531 >> 172.21.15.97:0/1783469531 conn(0x7feae00cfb10 msgr2=0x7feae0072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.236+0000 7f6ed0c00640 1 -- 172.21.15.97:0/3403357930 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0ad070 con 0x7f6ed40addf0 2024-09-10T21:23:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.240+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed40cc690 con 0x7f6ed40addf0 2024-09-10T21:23:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.240+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed4154370 con 0x7f6ed40addf0 2024-09-10T21:23:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.240+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed40ad8b0 con 0x7f6ed40addf0 2024-09-10T21:23:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.240+0000 7feae7400640 1 -- 172.21.15.97:0/1783469531 shutdown_connections 2024-09-10T21:23:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.240+0000 7feae7400640 1 -- 172.21.15.97:0/1783469531 wait complete. 2024-09-10T21:23:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.244+0000 7f6ed0c00640 1 -- 172.21.15.97:0/3403357930 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6ebc0a8050 con 0x7f6ed40addf0 2024-09-10T21:23:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.244+0000 7f6ed0c00640 1 -- 172.21.15.97:0/3403357930 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ebc0a9a30 con 0x7f6ed40addf0 2024-09-10T21:23:47.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.244+0000 7f6ed0c00640 1 -- 172.21.15.97:0/3403357930 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6ebc0164b0 con 0x7f6ed40addf0 2024-09-10T21:23:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.352+0000 7f1cc7200640 1 -- 172.21.15.97:0/1552840756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc015a870 msgr2=0x7f1cc015b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.352+0000 7f1cc7200640 1 --2- 172.21.15.97:0/1552840756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc015a870 0x7f1cc015b8c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009c80 tx=0x7f1cbc01cd10 comp rx=0 tx=0).stop 2024-09-10T21:23:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.352+0000 7f1cc7200640 1 -- 172.21.15.97:0/1552840756 shutdown_connections 2024-09-10T21:23:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.352+0000 7f1cc7200640 1 --2- 172.21.15.97:0/1552840756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc015a870 0x7f1cc015b8c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.352+0000 7f1cc7200640 1 -- 172.21.15.97:0/1552840756 >> 172.21.15.97:0/1552840756 conn(0x7f1cc00cf890 msgr2=0x7f1cc00cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.352+0000 7f1cc7200640 1 -- 172.21.15.97:0/1552840756 shutdown_connections 2024-09-10T21:23:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.352+0000 7f1cc7200640 1 -- 172.21.15.97:0/1552840756 wait complete. 2024-09-10T21:23:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.356+0000 7f1cc7200640 1 Processor -- start 2024-09-10T21:23:47.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.356+0000 7f1cc7200640 1 -- start start 2024-09-10T21:23:47.365 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc006f820 0x7f1cc01489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cc00735b0 0x7f1cc014f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc7200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc015a870 0x7f1cc014b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0072b30 con 0x7f1cc006f820 2024-09-10T21:23:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00729b0 con 0x7f1cc00735b0 2024-09-10T21:23:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc7200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc0072cb0 con 0x7f1cc015a870 2024-09-10T21:23:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cc00735b0 0x7f1cc014f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cc00735b0 0x7f1cc014f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:46272/0 (socket says 172.21.15.97:46272) 2024-09-10T21:23:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 -- 172.21.15.97:0/378997505 learned_addr learned my addr 172.21.15.97:0/378997505 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc6800640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc015a870 0x7f1cc014b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 -- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc015a870 msgr2=0x7f1cc014b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc015a870 0x7f1cc014b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 -- 172.21.15.97:0/378997505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc006f820 msgr2=0x7f1cc01489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc006f820 0x7f1cc01489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 -- 172.21.15.97:0/378997505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc014d6f0 con 0x7f1cc00735b0 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cc5400640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cc00735b0 0x7f1cc014f7a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0001b40 tx=0x7f1cb0014420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cba000640 1 -- 172.21.15.97:0/378997505 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0001cf0 con 0x7f1cc00735b0 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.360+0000 7f1cba000640 1 -- 172.21.15.97:0/378997505 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb0014d40 con 0x7f1cc00735b0 2024-09-10T21:23:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.364+0000 7f1cba000640 1 -- 172.21.15.97:0/378997505 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0001cf0 con 0x7f1cc00735b0 2024-09-10T21:23:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.364+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc014d9d0 con 0x7f1cc00735b0 2024-09-10T21:23:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.364+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc014df90 con 0x7f1cc00735b0 2024-09-10T21:23:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc00731e0 con 0x7f1cc00735b0 2024-09-10T21:23:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7fc4c9400640 1 -- 172.21.15.97:0/4004874724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c406f820 msgr2=0x7fc4c406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7fc4c9400640 1 --2- 172.21.15.97:0/4004874724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c406f820 0x7fc4c406fcd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009840 tx=0x7fc4b401c640 comp rx=0 tx=0).stop 2024-09-10T21:23:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7fc4c9400640 1 -- 172.21.15.97:0/4004874724 shutdown_connections 2024-09-10T21:23:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7fc4c9400640 1 --2- 172.21.15.97:0/4004874724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c406f820 0x7fc4c406fcd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7fc4c9400640 1 --2- 172.21.15.97:0/4004874724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c415b8c0 0x7fc4c406e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7fc4c9400640 1 -- 172.21.15.97:0/4004874724 >> 172.21.15.97:0/4004874724 conn(0x7fc4c40cf890 msgr2=0x7fc4c40cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7f1cba000640 1 -- 172.21.15.97:0/378997505 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1cb0010850 con 0x7f1cc00735b0 2024-09-10T21:23:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7f1cba000640 1 -- 172.21.15.97:0/378997505 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1cb0018070 con 0x7f1cc00735b0 2024-09-10T21:23:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.368+0000 7f1cba000640 1 -- 172.21.15.97:0/378997505 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1cb0003b00 con 0x7f1cc00735b0 2024-09-10T21:23:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.372+0000 7fc4c9400640 1 -- 172.21.15.97:0/4004874724 shutdown_connections 2024-09-10T21:23:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.372+0000 7fc4c9400640 1 -- 172.21.15.97:0/4004874724 wait complete. 2024-09-10T21:23:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.372+0000 7fc4c9400640 1 Processor -- start 2024-09-10T21:23:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.372+0000 7fc4c9400640 1 -- start start 2024-09-10T21:23:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7f2e6ac00640 1 -- 172.21.15.97:0/2067997191 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c0337b0 msgr2=0x7f2e5c0e74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7f2e6ac00640 1 --2- 172.21.15.97:0/2067997191 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c0337b0 0x7f2e5c0e74c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2e6406ba20 tx=0x7f2e64098980 comp rx=0 tx=0).stop 2024-09-10T21:23:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c406f820 0x7fc4c4162c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c415a870 0x7fc4c4204dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c406f820 0x7fc4c4162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c406f820 0x7fc4c4162c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:58810/0 (socket says 172.21.15.97:58810) 2024-09-10T21:23:47.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7f0e9f200640 1 -- 172.21.15.97:0/701755554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e900361c0 msgr2=0x7f0e90032520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.376+0000 7f0e9f200640 1 --2- 172.21.15.97:0/701755554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e900361c0 0x7f0e90032520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e9806ba20 tx=0x7f0e9809a7c0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c415b8c0 0x7fc4c414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f0e9f200640 1 -- 172.21.15.97:0/701755554 shutdown_connections 2024-09-10T21:23:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f0e9f200640 1 --2- 172.21.15.97:0/701755554 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900365a0 0x7f0e900e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f0e9f200640 1 --2- 172.21.15.97:0/701755554 >> v2:172.21.15.97:3300/0 conn(0x7f0e90031ff0 0x7f0e9000a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f0e9f200640 1 --2- 172.21.15.97:0/701755554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e900361c0 0x7f0e90032520 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f0e9f200640 1 -- 172.21.15.97:0/701755554 >> 172.21.15.97:0/701755554 conn(0x7f0e900511c0 msgr2=0x7f0e900515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c9400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c40724a0 con 0x7fc4c415b8c0 2024-09-10T21:23:47.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f2e6ac00640 1 -- 172.21.15.97:0/2067997191 shutdown_connections 2024-09-10T21:23:47.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f2e6ac00640 1 --2- 172.21.15.97:0/2067997191 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c0337b0 0x7f2e5c0e74c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7f2e6ac00640 1 -- 172.21.15.97:0/2067997191 >> 172.21.15.97:0/2067997191 conn(0x7f2e5c01d950 msgr2=0x7f2e5c054740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c9400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c40b4b90 con 0x7fc4c415a870 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c9400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c4072620 con 0x7fc4c406f820 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3e00640 1 -- 172.21.15.97:0/191297017 learned_addr learned my addr 172.21.15.97:0/191297017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3400640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c415a870 0x7fc4c4204dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c8a00640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c415b8c0 0x7fc4c414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3e00640 1 -- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c415a870 msgr2=0x7fc4c4204dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3e00640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c415a870 0x7fc4c4204dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3e00640 1 -- 172.21.15.97:0/191297017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c415b8c0 msgr2=0x7fc4c414e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3e00640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c415b8c0 0x7fc4c414e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3e00640 1 -- 172.21.15.97:0/191297017 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4158170 con 0x7fc4c406f820 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c3e00640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c406f820 0x7fc4c4162c10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00ca20 tx=0x7fc4bc00cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4bbe00640 1 -- 172.21.15.97:0/191297017 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc017070 con 0x7fc4c406f820 2024-09-10T21:23:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4bbe00640 1 -- 172.21.15.97:0/191297017 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4bc00a7c0 con 0x7fc4c406f820 2024-09-10T21:23:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c4152c90 con 0x7fc4c406f820 2024-09-10T21:23:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4bbe00640 1 -- 172.21.15.97:0/191297017 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00abf0 con 0x7fc4c406f820 2024-09-10T21:23:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c4153250 con 0x7fc4c406f820 2024-09-10T21:23:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4bbe00640 1 -- 172.21.15.97:0/191297017 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fc4bc005070 con 0x7fc4c406f820 2024-09-10T21:23:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.380+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c0052a0 con 0x7fc4c406f820 2024-09-10T21:23:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.384+0000 7fc4bbe00640 1 -- 172.21.15.97:0/191297017 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4bc01cb90 con 0x7fc4c406f820 2024-09-10T21:23:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.384+0000 7f0e9f200640 1 -- 172.21.15.97:0/701755554 shutdown_connections 2024-09-10T21:23:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.384+0000 7f0e9f200640 1 -- 172.21.15.97:0/701755554 wait complete. 2024-09-10T21:23:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.384+0000 7fda96400640 1 -- 172.21.15.97:0/176887520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900adfc0 msgr2=0x7fda9015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.384+0000 7fda96400640 1 --2- 172.21.15.97:0/176887520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900adfc0 0x7fda9015e9c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fda800097f0 tx=0x7fda8001c490 comp rx=0 tx=0).stop 2024-09-10T21:23:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.388+0000 7f2e6ac00640 1 -- 172.21.15.97:0/2067997191 shutdown_connections 2024-09-10T21:23:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.388+0000 7f2e6ac00640 1 -- 172.21.15.97:0/2067997191 wait complete. 2024-09-10T21:23:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.388+0000 7fda96400640 1 -- 172.21.15.97:0/176887520 shutdown_connections 2024-09-10T21:23:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.388+0000 7fda96400640 1 --2- 172.21.15.97:0/176887520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900adfc0 0x7fda9015e9c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.388+0000 7fda96400640 1 -- 172.21.15.97:0/176887520 >> 172.21.15.97:0/176887520 conn(0x7fda900cf890 msgr2=0x7fda900cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.388+0000 7f0e9f200640 1 Processor -- start 2024-09-10T21:23:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.388+0000 7fc4bbe00640 1 -- 172.21.15.97:0/191297017 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fc4bc021070 con 0x7fc4c406f820 2024-09-10T21:23:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.392+0000 7fda96400640 1 -- 172.21.15.97:0/176887520 shutdown_connections 2024-09-10T21:23:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.392+0000 7fda96400640 1 -- 172.21.15.97:0/176887520 wait complete. 2024-09-10T21:23:47.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.392+0000 7f2e6ac00640 1 Processor -- start 2024-09-10T21:23:47.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7fda96400640 1 Processor -- start 2024-09-10T21:23:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9f200640 1 -- start start 2024-09-10T21:23:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e90031ff0 0x7f0e9004a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900361c0 0x7f0e9004abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e900365a0 0x7f0e9016e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e97400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900361c0 0x7f0e9004abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e900365a0 0x7f0e9016e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e900365a0 0x7f0e9016e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43958/0 (socket says 172.21.15.97:43958) 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e97400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900361c0 0x7f0e9004abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:58822/0 (socket says 172.21.15.97:58822) 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9dc00640 1 -- 172.21.15.97:0/4184784440 learned_addr learned my addr 172.21.15.97:0/4184784440 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e900379a0 con 0x7f0e900365a0 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90037820 con 0x7f0e90031ff0 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0e90037b20 con 0x7f0e900361c0 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e97400640 1 -- 172.21.15.97:0/4184784440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e90031ff0 msgr2=0x7f0e9004a690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e97400640 1 --2- 172.21.15.97:0/4184784440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e90031ff0 0x7f0e9004a690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e97400640 1 -- 172.21.15.97:0/4184784440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e900365a0 msgr2=0x7f0e9016e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e97400640 1 --2- 172.21.15.97:0/4184784440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e900365a0 0x7f0e9016e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.396+0000 7f0e97400640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e900225c0 con 0x7f0e900361c0 2024-09-10T21:23:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e97400640 1 --2- 172.21.15.97:0/4184784440 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900361c0 0x7f0e9004abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400b420 tx=0x7f0e8400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.392+0000 7f2e6ac00640 1 -- start start 2024-09-10T21:23:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda96400640 1 -- start start 2024-09-10T21:23:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda96400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda900adfc0 0x7fda90150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e6ac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e5c007050 0x7f2e5c0e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e6ac00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e5c0337b0 0x7f2e5c0e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e6ac00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c039ba0 0x7f2e5c1897a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e6ac00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c0f5280 con 0x7f2e5c007050 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e6ac00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c007640 con 0x7f2e5c0337b0 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e6ac00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c0f5400 con 0x7f2e5c039ba0 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e5c0337b0 0x7f2e5c0e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e5c0337b0 0x7f2e5c0e4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:46306/0 (socket says 172.21.15.97:46306) 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 -- 172.21.15.97:0/532415850 learned_addr learned my addr 172.21.15.97:0/532415850 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e63400640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e5c007050 0x7f2e5c0e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e63e00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c039ba0 0x7f2e5c1897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 -- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c039ba0 msgr2=0x7f2e5c1897a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c039ba0 0x7f2e5c1897a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 -- 172.21.15.97:0/532415850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e5c007050 msgr2=0x7f2e5c0e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e5c007050 0x7f2e5c0e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 -- 172.21.15.97:0/532415850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c04d9c0 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e62a00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e5c0337b0 0x7f2e5c0e4d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b3e0 tx=0x7f2e5000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e57e00640 1 -- 172.21.15.97:0/532415850 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50015020 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e57e00640 1 -- 172.21.15.97:0/532415850 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e50011940 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f2e57e00640 1 -- 172.21.15.97:0/532415850 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50011d90 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c18cfd0 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c188f30 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7f2e57e00640 1 -- 172.21.15.97:0/532415850 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2e50022060 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7f2e57e00640 1 -- 172.21.15.97:0/532415850 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e50019ac0 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c0e74c0 con 0x7f2e5c0337b0 2024-09-10T21:23:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.408+0000 7f2e57e00640 1 -- 172.21.15.97:0/532415850 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2e50019d10 con 0x7f2e5c0337b0 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f06480af080 con 0x7f0648161790 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda96400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900d03b0 0x7fda9014b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda96400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda9015b4f0 0x7fda900b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda96400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda9016be90 con 0x7fda9015b4f0 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda96400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fda9016bd10 con 0x7fda900adfc0 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda96400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fda9016c010 con 0x7fda900d03b0 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900d03b0 0x7fda9014b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900d03b0 0x7fda9014b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:58832/0 (socket says 172.21.15.97:58832) 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 -- 172.21.15.97:0/2068898364 learned_addr learned my addr 172.21.15.97:0/2068898364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda95000640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda900adfc0 0x7fda90150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 -- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda900adfc0 msgr2=0x7fda90150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda900adfc0 0x7fda90150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 -- 172.21.15.97:0/2068898364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda9015b4f0 msgr2=0x7fda900b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda9015b4f0 0x7fda900b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 -- 172.21.15.97:0/2068898364 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda90150910 con 0x7fda900d03b0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8fe00640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900d03b0 0x7fda9014b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fda8001cd30 tx=0x7fda800aa6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8cc00640 1 -- 172.21.15.97:0/2068898364 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda800aacd0 con 0x7fda900d03b0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7fda8cc00640 1 -- 172.21.15.97:0/2068898364 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda800b9030 con 0x7fda900d03b0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7fda8cc00640 1 -- 172.21.15.97:0/2068898364 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80005290 con 0x7fda900d03b0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9014ab60 con 0x7fda900d03b0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda90150590 con 0x7fda900d03b0 2024-09-10T21:23:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9015e9c0 con 0x7fda900d03b0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e8be00640 1 -- 172.21.15.97:0/4184784440 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84011830 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e8be00640 1 -- 172.21.15.97:0/4184784440 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e84016070 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e8be00640 1 -- 172.21.15.97:0/4184784440 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84011d60 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e90022860 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e90022e20 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e8be00640 1 -- 172.21.15.97:0/4184784440 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f0e8401b460 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.400+0000 7f0e8be00640 1 -- 172.21.15.97:0/4184784440 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e8401a8f0 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.404+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e90037fa0 con 0x7f0e900361c0 2024-09-10T21:23:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.408+0000 7f0e8be00640 1 -- 172.21.15.97:0/4184784440 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f0e8401a3c0 con 0x7f0e900361c0 2024-09-10T21:23:47.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.416+0000 7fda8cc00640 1 -- 172.21.15.97:0/2068898364 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fda800046d0 con 0x7fda900d03b0 2024-09-10T21:23:47.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.416+0000 7fda8cc00640 1 -- 172.21.15.97:0/2068898364 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda80003920 con 0x7fda900d03b0 2024-09-10T21:23:47.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.416+0000 7f1786000640 1 -- 172.21.15.97:0/410864648 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17780b2070 con 0x7f17880cf260 2024-09-10T21:23:47.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.416+0000 7f178cc00640 1 -- 172.21.15.97:0/410864648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17880cf260 msgr2=0x7f17880b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.416+0000 7f178cc00640 1 --2- 172.21.15.97:0/410864648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17880cf260 0x7f17880b2a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1778009900 tx=0x7f177801c850 comp rx=0 tx=0).stop 2024-09-10T21:23:47.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.416+0000 7fda8cc00640 1 -- 172.21.15.97:0/2068898364 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fda800c3030 con 0x7fda900d03b0 2024-09-10T21:23:47.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.420+0000 7f0644c00640 1 -- 172.21.15.97:0/3574132568 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f0634004b40 con 0x7f0648161790 2024-09-10T21:23:47.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.424+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f2530005590 con 0x7f2558161ac0 2024-09-10T21:23:47.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f178cc00640 1 -- 172.21.15.97:0/410864648 shutdown_connections 2024-09-10T21:23:47.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f178cc00640 1 --2- 172.21.15.97:0/410864648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17880cf260 0x7f17880b2a30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f178cc00640 1 -- 172.21.15.97:0/410864648 >> 172.21.15.97:0/410864648 conn(0x7f17880b16a0 msgr2=0x7f17880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0648161790 msgr2=0x7f06480ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f064e400640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0648161790 0x7f06480ce050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f063400c940 tx=0x7f063400ce00 comp rx=0 tx=0).stop 2024-09-10T21:23:47.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.432+0000 7f178cc00640 1 -- 172.21.15.97:0/410864648 shutdown_connections 2024-09-10T21:23:47.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.432+0000 7f178cc00640 1 -- 172.21.15.97:0/410864648 wait complete. 2024-09-10T21:23:47.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 shutdown_connections 2024-09-10T21:23:47.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f064e400640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0648161b70 0x7f06481fb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f064e400640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0648161790 0x7f06480ce050 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f064e400640 1 --2- 172.21.15.97:0/3574132568 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06480b43e0 0x7f064816bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.428+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 >> 172.21.15.97:0/3574132568 conn(0x7f06480cf890 msgr2=0x7f06481562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.432+0000 7f178cc00640 1 Processor -- start 2024-09-10T21:23:47.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.436+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 shutdown_connections 2024-09-10T21:23:47.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.436+0000 7f064e400640 1 -- 172.21.15.97:0/3574132568 wait complete. 2024-09-10T21:23:47.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.436+0000 7f254f400640 1 -- 172.21.15.97:0/263637588 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f25440c5020 con 0x7f2558161ac0 2024-09-10T21:23:47.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 -- start start 2024-09-10T21:23:47.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17880cee80 0x7f178806fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17880cf260 0x7f178815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17881501e0 0x7f17881522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1788154530 con 0x7f17880cf260 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f17881543b0 con 0x7f17881501e0 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f17881546b0 con 0x7f17880cee80 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17880cee80 0x7f178806fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17880cee80 0x7f178806fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:58840/0 (socket says 172.21.15.97:58840) 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 -- 172.21.15.97:0/4241244975 learned_addr learned my addr 172.21.15.97:0/4241244975 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1786a00640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17880cf260 0x7f178815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787e00640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17881501e0 0x7f17881522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 -- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17881501e0 msgr2=0x7f17881522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17881501e0 0x7f17881522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 -- 172.21.15.97:0/4241244975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17880cf260 msgr2=0x7f178815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17880cf260 0x7f178815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 -- 172.21.15.97:0/4241244975 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178816c3b0 con 0x7f17880cee80 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f1787400640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17880cee80 0x7f178806fa00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f178000c7c0 tx=0x7f178000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f177f400640 1 -- 172.21.15.97:0/4241244975 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780016020 con 0x7f17880cee80 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f177f400640 1 -- 172.21.15.97:0/4241244975 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f178000ab40 con 0x7f17880cee80 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.440+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1788070300 con 0x7f17880cee80 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.444+0000 7f177f400640 1 -- 172.21.15.97:0/4241244975 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780012420 con 0x7f17880cee80 2024-09-10T21:23:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.444+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f178806ff40 con 0x7f17880cee80 2024-09-10T21:23:47.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.444+0000 7f177f400640 1 -- 172.21.15.97:0/4241244975 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1780012950 con 0x7f17880cee80 2024-09-10T21:23:47.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.444+0000 7f177f400640 1 -- 172.21.15.97:0/4241244975 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f178001a3b0 con 0x7f17880cee80 2024-09-10T21:23:47.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.444+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178816c180 con 0x7f17880cee80 2024-09-10T21:23:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.444+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f6ed40aeac0 con 0x7f6ed40addf0 2024-09-10T21:23:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558161ac0 msgr2=0x7f25581424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558161ac0 0x7f25581424b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2558074a70 tx=0x7f2544004040 comp rx=0 tx=0).stop 2024-09-10T21:23:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 shutdown_connections 2024-09-10T21:23:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2558161ea0 0x7f25581429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2558161ac0 0x7f25581424b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 --2- 172.21.15.97:0/263637588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2558073980 0x7f255813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 >> 172.21.15.97:0/263637588 conn(0x7f255806cb40 msgr2=0x7f255816c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 shutdown_connections 2024-09-10T21:23:47.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.452+0000 7f255e000640 1 -- 172.21.15.97:0/263637588 wait complete. 2024-09-10T21:23:47.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.456+0000 7f6ed0c00640 1 -- 172.21.15.97:0/3403357930 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v9)=0 v9) ==== 115+0+0 (secure 0 0 0) 0x7f6ebc0164b0 con 0x7f6ed40addf0 2024-09-10T21:23:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.456+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed40addf0 msgr2=0x7f6ed41521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.456+0000 7f6eda400640 1 --2- 172.21.15.97:0/3403357930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed40addf0 0x7f6ed41521a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc01ac20 tx=0x7f6ebc0a9570 comp rx=0 tx=0).stop 2024-09-10T21:23:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 shutdown_connections 2024-09-10T21:23:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f6eda400640 1 --2- 172.21.15.97:0/3403357930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed415b4f0 0x7f6ed4157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f6eda400640 1 --2- 172.21.15.97:0/3403357930 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ed40d03b0 0x7f6ed41602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f6eda400640 1 --2- 172.21.15.97:0/3403357930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed40addf0 0x7f6ed41521a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 >> 172.21.15.97:0/3403357930 conn(0x7f6ed40cf890 msgr2=0x7f6ed4158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 shutdown_connections 2024-09-10T21:23:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f6eda400640 1 -- 172.21.15.97:0/3403357930 wait complete. 2024-09-10T21:23:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.460+0000 7f177f400640 1 -- 172.21.15.97:0/4241244975 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1780020070 con 0x7f17880cee80 2024-09-10T21:23:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.480+0000 7f6290200640 1 -- 172.21.15.97:0/4172995112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6280039ba0 msgr2=0x7f6280055060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.480+0000 7f6290200640 1 --2- 172.21.15.97:0/4172995112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6280039ba0 0x7f6280055060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f628000d490 tx=0x7f627401c870 comp rx=0 tx=0).stop 2024-09-10T21:23:47.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.484+0000 7f6290200640 1 -- 172.21.15.97:0/4172995112 shutdown_connections 2024-09-10T21:23:47.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.484+0000 7f6290200640 1 --2- 172.21.15.97:0/4172995112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6280039ba0 0x7f6280055060 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.484+0000 7f6290200640 1 --2- 172.21.15.97:0/4172995112 >> v2:172.21.15.194:3300/0 conn(0x7f6280007050 0x7f62800e74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.484+0000 7f6290200640 1 -- 172.21.15.97:0/4172995112 >> 172.21.15.97:0/4172995112 conn(0x7f628001d950 msgr2=0x7f6280054740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.488+0000 7f6290200640 1 -- 172.21.15.97:0/4172995112 shutdown_connections 2024-09-10T21:23:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.492+0000 7f6290200640 1 -- 172.21.15.97:0/4172995112 wait complete. 2024-09-10T21:23:47.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.492+0000 7f6290200640 1 Processor -- start 2024-09-10T21:23:47.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.496+0000 7f6290200640 1 -- start start 2024-09-10T21:23:47.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6280007050 0x7f62800dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62800337b0 0x7f62800d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6280039ba0 0x7f6280031240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6280006780 con 0x7f62800337b0 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6280006600 con 0x7f6280007050 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6280006900 con 0x7f6280039ba0 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628ce00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6280007050 0x7f62800dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628ce00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6280007050 0x7f62800dacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:46344/0 (socket says 172.21.15.97:46344) 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628ce00640 1 -- 172.21.15.97:0/211822382 learned_addr learned my addr 172.21.15.97:0/211822382 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6287e00640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62800337b0 0x7f62800d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628d800640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6280039ba0 0x7f6280031240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628d800640 1 -- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6280007050 msgr2=0x7f62800dacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628d800640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6280007050 0x7f62800dacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628d800640 1 -- 172.21.15.97:0/211822382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62800337b0 msgr2=0x7f62800d91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628d800640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62800337b0 0x7f62800d91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628d800640 1 -- 172.21.15.97:0/211822382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62800e3cb0 con 0x7f6280039ba0 2024-09-10T21:23:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628ce00640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6280007050 0x7f62800dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f628d800640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6280039ba0 0x7f6280031240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62740ac000 tx=0x7f62740a55f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6285600640 1 -- 172.21.15.97:0/211822382 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740b2040 con 0x7f6280039ba0 2024-09-10T21:23:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6285600640 1 -- 172.21.15.97:0/211822382 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6274004300 con 0x7f6280039ba0 2024-09-10T21:23:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6285600640 1 -- 172.21.15.97:0/211822382 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62740044f0 con 0x7f6280039ba0 2024-09-10T21:23:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62800e3f90 con 0x7f6280039ba0 2024-09-10T21:23:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.500+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62800e34d0 con 0x7f6280039ba0 2024-09-10T21:23:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.504+0000 7f6285600640 1 -- 172.21.15.97:0/211822382 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6274003bf0 con 0x7f6280039ba0 2024-09-10T21:23:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.504+0000 7f6285600640 1 -- 172.21.15.97:0/211822382 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f62740b85e0 con 0x7f6280039ba0 2024-09-10T21:23:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.508+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62800008e0 con 0x7f6280039ba0 2024-09-10T21:23:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.508+0000 7f6285600640 1 -- 172.21.15.97:0/211822382 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f62740c6020 con 0x7f6280039ba0 2024-09-10T21:23:47.522 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.526 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.520+0000 7fd7bb000640 1 -- 172.21.15.97:0/340940978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd798004810 msgr2=0x7fd798004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.520+0000 7fd7bb000640 1 --2- 172.21.15.97:0/340940978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd798004810 0x7fd798004c90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00c660 tx=0x7fd7ac01d380 comp rx=0 tx=0).stop 2024-09-10T21:23:47.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.520+0000 7fd7b9000640 1 -- 172.21.15.97:0/340940978 reap_dead start 2024-09-10T21:23:47.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.520+0000 7fd7bb000640 1 -- 172.21.15.97:0/340940978 shutdown_connections 2024-09-10T21:23:47.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.520+0000 7fd7bb000640 1 -- 172.21.15.97:0/340940978 >> 172.21.15.97:0/340940978 conn(0x7fd7b40cfb10 msgr2=0x7fd7b40cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.524+0000 7fd7bb000640 1 -- 172.21.15.97:0/340940978 shutdown_connections 2024-09-10T21:23:47.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.524+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fd748005590 con 0x7fd77015c000 2024-09-10T21:23:47.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.524+0000 7fd7bb000640 1 -- 172.21.15.97:0/340940978 wait complete. 2024-09-10T21:23:47.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 Processor -- start 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 -- start start 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd798004810 0x7fd7b42083b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b414dc50 0x7fd7b4154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7b414a040 0x7fd7b414a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b40727b0 con 0x7fd7b414dc50 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4072630 con 0x7fd798004810 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7bb000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b4072930 con 0x7fd7b414a040 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b414dc50 0x7fd7b4154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b414dc50 0x7fd7b4154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44010/0 (socket says 172.21.15.97:44010) 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7b3400640 1 -- 172.21.15.97:0/2045918905 learned_addr learned my addr 172.21.15.97:0/2045918905 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.528+0000 7fd7b9000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7b414a040 0x7fd7b414a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7b3e00640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd798004810 0x7fd7b42083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7b9000640 1 -- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd798004810 msgr2=0x7fd7b42083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7b9000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd798004810 0x7fd7b42083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7b9000640 1 -- 172.21.15.97:0/2045918905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b414dc50 msgr2=0x7fd7b4154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7b9000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b414dc50 0x7fd7b4154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7b9000640 1 -- 172.21.15.97:0/2045918905 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b4155290 con 0x7fd7b414a040 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7b9000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7b414a040 0x7fd7b414a4d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7b406de90 tx=0x7fd7a000aa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7abe00640 1 -- 172.21.15.97:0/2045918905 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0005020 con 0x7fd7b414a040 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7abe00640 1 -- 172.21.15.97:0/2045918905 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7a00051e0 con 0x7fd7b414a040 2024-09-10T21:23:47.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7abe00640 1 -- 172.21.15.97:0/2045918905 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0005630 con 0x7fd7b414a040 2024-09-10T21:23:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b40b1570 con 0x7fd7b414a040 2024-09-10T21:23:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd766000640 1 -- 172.21.15.97:0/1250769197 <== mon.1 v2:172.21.15.194:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v10)=0 v10) ==== 106+0+0 (secure 0 0 0) 0x7fd7702004e0 con 0x7fd77015c000 2024-09-10T21:23:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b414aa10 con 0x7fd7b414a040 2024-09-10T21:23:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7abe00640 1 -- 172.21.15.97:0/2045918905 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd7a0004b40 con 0x7fd7b414a040 2024-09-10T21:23:47.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.532+0000 7fd7abe00640 1 -- 172.21.15.97:0/2045918905 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd7a0014830 con 0x7fd7b414a040 2024-09-10T21:23:47.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.536+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd77015c000 msgr2=0x7fd770148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.536+0000 7fd76f200640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd77015c000 0x7fd770148c30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00b410 tx=0x7fd75c00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 shutdown_connections 2024-09-10T21:23:47.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd76f200640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd77015c000 0x7fd770148c30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd76f200640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd77015b9e0 0x7fd77014f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd76f200640 1 --2- 172.21.15.97:0/1250769197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd77006f8e0 0x7fd7701fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 >> 172.21.15.97:0/1250769197 conn(0x7fd7700b16a0 msgr2=0x7fd7700b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b40cc300 con 0x7fd7b414a040 2024-09-10T21:23:47.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 shutdown_connections 2024-09-10T21:23:47.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.540+0000 7fd76f200640 1 -- 172.21.15.97:0/1250769197 wait complete. 2024-09-10T21:23:47.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fc49c005860 con 0x7fc4c406f820 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 -- 172.21.15.97:0/2710577051 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac15a870 msgr2=0x7f3bac15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 --2- 172.21.15.97:0/2710577051 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac15a870 0x7f3bac15b8c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800fb30 tx=0x7f3b98020920 comp rx=0 tx=0).stop 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 -- 172.21.15.97:0/2710577051 shutdown_connections 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 --2- 172.21.15.97:0/2710577051 >> v2:172.21.15.97:3300/0 conn(0x7f3bac06f820 0x7f3bac06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 --2- 172.21.15.97:0/2710577051 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac15a870 0x7f3bac15b8c0 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800fb30 tx=0x7f3b98020920 comp rx=0 tx=0).stop 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 -- 172.21.15.97:0/2710577051 >> 172.21.15.97:0/2710577051 conn(0x7f3bac0cf890 msgr2=0x7f3bac0cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 -- 172.21.15.97:0/2710577051 shutdown_connections 2024-09-10T21:23:47.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.548+0000 7f3bb1600640 1 -- 172.21.15.97:0/2710577051 wait complete. 2024-09-10T21:23:47.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.544+0000 7fd7abe00640 1 -- 172.21.15.97:0/2045918905 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd7a0014a30 con 0x7fd7b414a040 2024-09-10T21:23:47.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.552+0000 7f3bb1600640 1 Processor -- start 2024-09-10T21:23:47.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.556+0000 7f3bb1600640 1 -- start start 2024-09-10T21:23:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.560+0000 7f3bb1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bac06f820 0x7f3bac147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.560+0000 7f3bb1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bac0735b0 0x7f3bac154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.560+0000 7f3bb1600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac1563c0 0x7f3bac14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.560+0000 7f3bb1600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac072620 con 0x7f3bac0735b0 2024-09-10T21:23:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.560+0000 7f3bb1600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0724a0 con 0x7f3bac06f820 2024-09-10T21:23:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.560+0000 7f3bb1600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0727a0 con 0x7f3bac1563c0 2024-09-10T21:23:47.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7fc4bbe00640 1 -- 172.21.15.97:0/191297017 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7fc4bc00ad90 con 0x7fc4c406f820 2024-09-10T21:23:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3babe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bac06f820 0x7f3bac147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3babe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bac06f820 0x7f3bac147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:46384/0 (socket says 172.21.15.97:46384) 2024-09-10T21:23:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3babe00640 1 -- 172.21.15.97:0/4054565405 learned_addr learned my addr 172.21.15.97:0/4054565405 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb0c00640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac1563c0 0x7f3bac14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f1cc01533c0 con 0x7f1cc00735b0 2024-09-10T21:23:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bab400640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bac0735b0 0x7f3bac154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb0c00640 1 -- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bac06f820 msgr2=0x7f3bac147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb0c00640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bac06f820 0x7f3bac147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb0c00640 1 -- 172.21.15.97:0/4054565405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bac0735b0 msgr2=0x7f3bac154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb0c00640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bac0735b0 0x7f3bac154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb0c00640 1 -- 172.21.15.97:0/4054565405 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac149aa0 con 0x7f3bac1563c0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bab400640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bac0735b0 0x7f3bac154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb0c00640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac1563c0 0x7f3bac14b8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4007b50 tx=0x7f3ba4007f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3ba3e00640 1 -- 172.21.15.97:0/4054565405 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4019070 con 0x7f3bac1563c0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3ba3e00640 1 -- 172.21.15.97:0/4054565405 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba4004b30 con 0x7f3bac1563c0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3ba3e00640 1 -- 172.21.15.97:0/4054565405 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400fb00 con 0x7f3bac1563c0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb1600640 1 -- 172.21.15.97:0/4054565405 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac147fd0 con 0x7f3bac1563c0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3bb1600640 1 -- 172.21.15.97:0/4054565405 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac148ee0 con 0x7f3bac1563c0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.564+0000 7f3ba3e00640 1 -- 172.21.15.97:0/4054565405 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3ba4015470 con 0x7f3bac1563c0 2024-09-10T21:23:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7f3ba3e00640 1 -- 172.21.15.97:0/4054565405 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ba401ed10 con 0x7f3bac1563c0 2024-09-10T21:23:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c406f820 msgr2=0x7fc4c4162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c406f820 0x7fc4c4162c10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00ca20 tx=0x7fc4bc00cee0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7f3ba1600640 1 -- 172.21.15.97:0/4054565405 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac15b8c0 con 0x7f3bac1563c0 2024-09-10T21:23:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 shutdown_connections 2024-09-10T21:23:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c415b8c0 0x7fc4c414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4c415a870 0x7fc4c4204dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 --2- 172.21.15.97:0/191297017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4c406f820 0x7fc4c4162c10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 >> 172.21.15.97:0/191297017 conn(0x7fc4c40cf890 msgr2=0x7fc4c415cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 shutdown_connections 2024-09-10T21:23:47.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7fc4c9400640 1 -- 172.21.15.97:0/191297017 wait complete. 2024-09-10T21:23:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7f1cba000640 1 -- 172.21.15.97:0/378997505 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v12)=0 v12) ==== 120+0+0 (secure 0 0 0) 0x7f1cb0003dd0 con 0x7f1cc00735b0 2024-09-10T21:23:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.568+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f2e5c0e6700 con 0x7f2e5c0337b0 2024-09-10T21:23:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.572+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cc00735b0 msgr2=0x7f1cc014f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.572+0000 7f3ba3e00640 1 -- 172.21.15.97:0/4054565405 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f3bac149aa0 con 0x7f3bac1563c0 2024-09-10T21:23:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.572+0000 7f1cc7200640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cc00735b0 0x7f1cc014f7a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0001b40 tx=0x7f1cb0014420 comp rx=0 tx=0).stop 2024-09-10T21:23:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.580+0000 7f2e57e00640 1 -- 172.21.15.97:0/532415850 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f2e500193b0 con 0x7f2e5c0337b0 2024-09-10T21:23:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e5c0337b0 msgr2=0x7f2e5c0e4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e5c0337b0 0x7f2e5c0e4d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b3e0 tx=0x7f2e5000b8a0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 shutdown_connections 2024-09-10T21:23:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e5c039ba0 0x7f2e5c1897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e5c0337b0 0x7f2e5c0e4d60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 --2- 172.21.15.97:0/532415850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e5c007050 0x7f2e5c0e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 >> 172.21.15.97:0/532415850 conn(0x7f2e5c01d950 msgr2=0x7f2e5c04fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 shutdown_connections 2024-09-10T21:23:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.588+0000 7f2e6ac00640 1 -- 172.21.15.97:0/532415850 wait complete. 2024-09-10T21:23:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.592+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 shutdown_connections 2024-09-10T21:23:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.592+0000 7f1cc7200640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc015a870 0x7f1cc014b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.592+0000 7f1cc7200640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cc00735b0 0x7f1cc014f7a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.592+0000 7f1cc7200640 1 --2- 172.21.15.97:0/378997505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc006f820 0x7f1cc01489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.592+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 >> 172.21.15.97:0/378997505 conn(0x7f1cc00cf890 msgr2=0x7f1cc0073cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.596+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 shutdown_connections 2024-09-10T21:23:47.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.596+0000 7f1cc7200640 1 -- 172.21.15.97:0/378997505 wait complete. 2024-09-10T21:23:47.606 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.606 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.608+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fda900b3730 con 0x7fda900d03b0 2024-09-10T21:23:47.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.612+0000 7fda8cc00640 1 -- 172.21.15.97:0/2068898364 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fda800a6040 con 0x7fda900d03b0 2024-09-10T21:23:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.612+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900d03b0 msgr2=0x7fda9014b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.612+0000 7fda96400640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900d03b0 0x7fda9014b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fda8001cd30 tx=0x7fda800aa6f0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.612+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f0e90034cb0 con 0x7f0e900361c0 2024-09-10T21:23:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.616+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 shutdown_connections 2024-09-10T21:23:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.616+0000 7fda96400640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda9015b4f0 0x7fda900b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.616+0000 7fda96400640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda900d03b0 0x7fda9014b640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.616+0000 7fda96400640 1 --2- 172.21.15.97:0/2068898364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda900adfc0 0x7fda90150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.616+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 >> 172.21.15.97:0/2068898364 conn(0x7fda900cf890 msgr2=0x7fda90070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.616+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 shutdown_connections 2024-09-10T21:23:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.616+0000 7fda96400640 1 -- 172.21.15.97:0/2068898364 wait complete. 2024-09-10T21:23:47.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.624+0000 7f0e8be00640 1 -- 172.21.15.97:0/4184784440 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v15)=0 v15) ==== 119+0+0 (secure 0 0 0) 0x7f0e8401a3c0 con 0x7f0e900361c0 2024-09-10T21:23:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.624+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900361c0 msgr2=0x7f0e9004abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.624+0000 7f0e9f200640 1 --2- 172.21.15.97:0/4184784440 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900361c0 0x7f0e9004abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400b420 tx=0x7f0e8400b8e0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.628+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 shutdown_connections 2024-09-10T21:23:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.628+0000 7f0e9f200640 1 --2- 172.21.15.97:0/4184784440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e900365a0 0x7f0e9016e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.628+0000 7f0e9f200640 1 --2- 172.21.15.97:0/4184784440 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e900361c0 0x7f0e9004abd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.628+0000 7f0e9f200640 1 --2- 172.21.15.97:0/4184784440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e90031ff0 0x7f0e9004a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.628+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 >> 172.21.15.97:0/4184784440 conn(0x7f0e900511c0 msgr2=0x7f0e90005c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.628+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 shutdown_connections 2024-09-10T21:23:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.628+0000 7f0e9f200640 1 -- 172.21.15.97:0/4184784440 wait complete. 2024-09-10T21:23:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.636+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f1788158150 con 0x7f17880cee80 2024-09-10T21:23:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.648+0000 7f177f400640 1 -- 172.21.15.97:0/4241244975 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f178001b410 con 0x7f17880cee80 2024-09-10T21:23:47.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17880cee80 msgr2=0x7f178806fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17880cee80 0x7f178806fa00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f178000c7c0 tx=0x7f178000cc80 comp rx=0 tx=0).stop 2024-09-10T21:23:47.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 shutdown_connections 2024-09-10T21:23:47.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17881501e0 0x7f17881522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17880cf260 0x7f178815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 --2- 172.21.15.97:0/4241244975 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17880cee80 0x7f178806fa00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 >> 172.21.15.97:0/4241244975 conn(0x7f17880b16a0 msgr2=0x7f178815f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 shutdown_connections 2024-09-10T21:23:47.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.652+0000 7f178cc00640 1 -- 172.21.15.97:0/4241244975 wait complete. 2024-09-10T21:23:47.662 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.660+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f62800e4440 con 0x7f6280039ba0 2024-09-10T21:23:47.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.664+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fd7b414e130 con 0x7fd7b414a040 2024-09-10T21:23:47.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.664+0000 7f6285600640 1 -- 172.21.15.97:0/211822382 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f62740164f0 con 0x7f6280039ba0 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6280039ba0 msgr2=0x7f6280031240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6280039ba0 0x7f6280031240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62740ac000 tx=0x7f62740a55f0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 shutdown_connections 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6280039ba0 0x7f6280031240 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62800337b0 0x7f62800d91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 --2- 172.21.15.97:0/211822382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6280007050 0x7f62800dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 >> 172.21.15.97:0/211822382 conn(0x7f628001d950 msgr2=0x7f6280039490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 shutdown_connections 2024-09-10T21:23:47.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7f6290200640 1 -- 172.21.15.97:0/211822382 wait complete. 2024-09-10T21:23:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7abe00640 1 -- 172.21.15.97:0/2045918905 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7fd7a0014c10 con 0x7fd7b414a040 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7b414a040 msgr2=0x7fd7b414a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7bb000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7b414a040 0x7fd7b414a4d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7b406de90 tx=0x7fd7a000aa70 comp rx=0 tx=0).stop 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 shutdown_connections 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7bb000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7b414a040 0x7fd7b414a4d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7bb000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b414dc50 0x7fd7b4154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7bb000640 1 --2- 172.21.15.97:0/2045918905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd798004810 0x7fd7b42083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.668+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 >> 172.21.15.97:0/2045918905 conn(0x7fd7b40cfb10 msgr2=0x7fd7b40cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.672+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 shutdown_connections 2024-09-10T21:23:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.672+0000 7fd7bb000640 1 -- 172.21.15.97:0/2045918905 wait complete. 2024-09-10T21:23:47.690 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.690 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.690 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.696+0000 7f3ba1600640 1 -- 172.21.15.97:0/4054565405 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f3bac153e90 con 0x7f3bac1563c0 2024-09-10T21:23:47.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba3e00640 1 -- 172.21.15.97:0/4054565405 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7f3ba401e450 con 0x7f3bac1563c0 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 -- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac1563c0 msgr2=0x7f3bac14b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac1563c0 0x7f3bac14b8c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4007b50 tx=0x7f3ba4007f50 comp rx=0 tx=0).stop 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 -- 172.21.15.97:0/4054565405 shutdown_connections 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bac1563c0 0x7f3bac14b8c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bac0735b0 0x7f3bac154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 --2- 172.21.15.97:0/4054565405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bac06f820 0x7f3bac147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 -- 172.21.15.97:0/4054565405 >> 172.21.15.97:0/4054565405 conn(0x7f3bac0cf890 msgr2=0x7f3bac15ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 -- 172.21.15.97:0/4054565405 shutdown_connections 2024-09-10T21:23:47.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.700+0000 7f3ba1600640 1 -- 172.21.15.97:0/4054565405 wait complete. 2024-09-10T21:23:47.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.729 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi097') 2024-09-10T21:23:47.729 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T21:23:47.730 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T21:23:47.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.904+0000 7f199da00640 1 -- 172.21.15.97:0/3612876509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980b0d50 msgr2=0x7f19980ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.904+0000 7f199da00640 1 --2- 172.21.15.97:0/3612876509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980b0d50 0x7f19980ccde0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1990009bc0 tx=0x7f199001cac0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.904+0000 7f199da00640 1 -- 172.21.15.97:0/3612876509 shutdown_connections 2024-09-10T21:23:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.904+0000 7f199da00640 1 --2- 172.21.15.97:0/3612876509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980b0d50 0x7f19980ccde0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.904+0000 7f199da00640 1 -- 172.21.15.97:0/3612876509 >> 172.21.15.97:0/3612876509 conn(0x7f19980d1c30 msgr2=0x7f19980d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.904+0000 7f199da00640 1 -- 172.21.15.97:0/3612876509 shutdown_connections 2024-09-10T21:23:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.904+0000 7f199da00640 1 -- 172.21.15.97:0/3612876509 wait complete. 2024-09-10T21:23:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 Processor -- start 2024-09-10T21:23:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 -- start start 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980afdd0 0x7f19980c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19980b0d50 0x7f19981671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19980ce640 0x7f19981fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1998169fa0 con 0x7f19980ce640 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1998169e20 con 0x7f19980afdd0 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f199da00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f199816a120 con 0x7f19980b0d50 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19980ce640 0x7f19981fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19980ce640 0x7f19981fbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44036/0 (socket says 172.21.15.97:44036) 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 -- 172.21.15.97:0/166985419 learned_addr learned my addr 172.21.15.97:0/166985419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1996a00640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980afdd0 0x7f19980c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1996000640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19980b0d50 0x7f19981671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 -- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19980b0d50 msgr2=0x7f19981671f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19980b0d50 0x7f19981671f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 -- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980afdd0 msgr2=0x7f19980c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980afdd0 0x7f19980c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 -- 172.21.15.97:0/166985419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19981fbfe0 con 0x7f19980ce640 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f1997400640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19980ce640 0x7f19981fbaa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f19900b0040 tx=0x7f19900039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.908+0000 7f198ea00640 1 -- 172.21.15.97:0/166985419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19900bb070 con 0x7f19980ce640 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.912+0000 7f198ea00640 1 -- 172.21.15.97:0/166985419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1990004a50 con 0x7f19980ce640 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.912+0000 7f198ea00640 1 -- 172.21.15.97:0/166985419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19900b2040 con 0x7f19980ce640 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.912+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199815a220 con 0x7f19980ce640 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.912+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199815a7e0 con 0x7f19980ce640 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.912+0000 7f198ea00640 1 -- 172.21.15.97:0/166985419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f19900b5070 con 0x7f19980ce640 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.912+0000 7f198ea00640 1 -- 172.21.15.97:0/166985419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1990005030 con 0x7f19980ce640 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.912+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19980cea20 con 0x7f19980ce640 2024-09-10T21:23:47.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.916+0000 7f198ea00640 1 -- 172.21.15.97:0/166985419 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f19900c4020 con 0x7f19980ce640 2024-09-10T21:23:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.976+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1998154ac0 con 0x7f19980ce640 2024-09-10T21:23:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f198ea00640 1 -- 172.21.15.97:0/166985419 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f19981fbfe0 con 0x7f19980ce640 2024-09-10T21:23:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19980ce640 msgr2=0x7f19981fbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19980ce640 0x7f19981fbaa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f19900b0040 tx=0x7f19900039f0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 shutdown_connections 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19980ce640 0x7f19981fbaa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19980b0d50 0x7f19981671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 --2- 172.21.15.97:0/166985419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19980afdd0 0x7f19980c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 >> 172.21.15.97:0/166985419 conn(0x7f19980d1c30 msgr2=0x7f19980cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 shutdown_connections 2024-09-10T21:23:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:47.980+0000 7f199da00640 1 -- 172.21.15.97:0/166985419 wait complete. 2024-09-10T21:23:48.007 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T21:23:48.008 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T21:23:48.008 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:48.008 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T21:23:48.028 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:48.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T21:23:48.089 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:48.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T21:23:48.152 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:23:48.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T21:23:48.216 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:48.216 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T21:23:48.236 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:48.237 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T21:23:48.299 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:48.299 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T21:23:48.358 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:23:48.358 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T21:23:48.418 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 8c747c78-dec2-4402-8839-4c4b5f98748b 0 2024-09-10T21:23:48.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- 172.21.15.194:0/89065359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 msgr2=0x7f2d600aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 --2- 172.21.15.194:0/89065359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600aedc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d600b36f0 tx=0x7f2d4c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- 172.21.15.194:0/89065359 shutdown_connections 2024-09-10T21:23:48.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 --2- 172.21.15.194:0/89065359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600aedc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 --2- 172.21.15.194:0/89065359 >> v2:172.21.15.97:3300/0 conn(0x7f2d600ce640 0x7f2d600cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- 172.21.15.194:0/89065359 >> 172.21.15.194:0/89065359 conn(0x7f2d600d1c30 msgr2=0x7f2d600d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:48.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- 172.21.15.194:0/89065359 shutdown_connections 2024-09-10T21:23:48.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- 172.21.15.194:0/89065359 wait complete. 2024-09-10T21:23:48.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 Processor -- start 2024-09-10T21:23:48.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- start start 2024-09-10T21:23:48.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d600af300 0x7f2d6006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:48.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d600ce640 0x7f2d6006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d60169270 con 0x7f2d600ce640 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601690f0 con 0x7f2d600b0d50 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d66a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2d601693f0 con 0x7f2d600af300 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d5f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d5fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d600af300 0x7f2d6006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.597+0000 7f2d5f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50466/0 (socket says 172.21.15.194:50466) 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d5f400640 1 -- 172.21.15.194:0/1085734870 learned_addr learned my addr 172.21.15.194:0/1085734870 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d64a00640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d600ce640 0x7f2d6006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d5f400640 1 -- 172.21.15.194:0/1085734870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d600af300 msgr2=0x7f2d6006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d5f400640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d600af300 0x7f2d6006f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d5f400640 1 -- 172.21.15.194:0/1085734870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d600ce640 msgr2=0x7f2d6006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d5f400640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d600ce640 0x7f2d6006dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d5f400640 1 -- 172.21.15.194:0/1085734870 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6006f8e0 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d64a00640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d600ce640 0x7f2d6006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d5f400640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600c7ca0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009850 tx=0x7f2d4c0a5780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d57e00640 1 -- 172.21.15.194:0/1085734870 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0bc070 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d57e00640 1 -- 172.21.15.194:0/1085734870 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d4c005030 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d57e00640 1 -- 172.21.15.194:0/1085734870 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0b2040 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d601660c0 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6006fbc0 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d57e00640 1 -- 172.21.15.194:0/1085734870 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2d4c004a60 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d57e00640 1 -- 172.21.15.194:0/1085734870 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f2d4c003d90 con 0x7f2d600b0d50 2024-09-10T21:23:48.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.601+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d600aedc0 con 0x7f2d600b0d50 2024-09-10T21:23:48.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.605+0000 7f2d57e00640 1 -- 172.21.15.194:0/1085734870 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2d4c0164b0 con 0x7f2d600b0d50 2024-09-10T21:23:48.678 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.673+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8c747c78-dec2-4402-8839-4c4b5f98748b", "id": 0} v 0) -- 0x7f2d6015f270 con 0x7f2d600b0d50 2024-09-10T21:23:48.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d57e00640 1 -- 172.21.15.194:0/1085734870 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8c747c78-dec2-4402-8839-4c4b5f98748b", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f2d4c0c5020 con 0x7f2d600b0d50 2024-09-10T21:23:48.682 INFO:teuthology.orchestra.run.smithi194.stdout:0 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 msgr2=0x7f2d600c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600c7ca0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009850 tx=0x7f2d4c0a5780 comp rx=0 tx=0).stop 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 shutdown_connections 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d600ce640 0x7f2d6006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d600b0d50 0x7f2d600c7ca0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 --2- 172.21.15.194:0/1085734870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d600af300 0x7f2d6006f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 >> 172.21.15.194:0/1085734870 conn(0x7f2d600d1c30 msgr2=0x7f2d600b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 shutdown_connections 2024-09-10T21:23:48.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.677+0000 7f2d66a00640 1 -- 172.21.15.194:0/1085734870 wait complete. 2024-09-10T21:23:48.701 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new e0d62fef-5f34-4f25-9a73-100f3fbefb45 1 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- 172.21.15.194:0/473823338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0b5210 msgr2=0x7fbaec0cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 --2- 172.21.15.194:0/473823338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0b5210 0x7fbaec0cd1b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009840 tx=0x7fbad401c7d0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- 172.21.15.194:0/473823338 shutdown_connections 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 --2- 172.21.15.194:0/473823338 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaec0d01b0 0x7fbaec0d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 --2- 172.21.15.194:0/473823338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0b5210 0x7fbaec0cd1b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- 172.21.15.194:0/473823338 >> 172.21.15.194:0/473823338 conn(0x7fbaec074200 msgr2=0x7fbaec074610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- 172.21.15.194:0/473823338 shutdown_connections 2024-09-10T21:23:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- 172.21.15.194:0/473823338 wait complete. 2024-09-10T21:23:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 Processor -- start 2024-09-10T21:23:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- start start 2024-09-10T21:23:48.849 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaec0b5210 0x7fbaec1f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:48.849 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0cc140 0x7fbaec15d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:48.849 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec0d01b0 0x7fbaec15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:48.849 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec069330 con 0x7fbaec0d01b0 2024-09-10T21:23:48.849 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0691b0 con 0x7fbaec0cc140 2024-09-10T21:23:48.849 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaf1c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec0694b0 con 0x7fbaec0b5210 2024-09-10T21:23:48.849 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbae3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0cc140 0x7fbaec15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbaeaa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaec0b5210 0x7fbaec1f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbae3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0cc140 0x7fbaec15d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50486/0 (socket says 172.21.15.194:50486) 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.841+0000 7fbae3e00640 1 -- 172.21.15.194:0/1030792431 learned_addr learned my addr 172.21.15.194:0/1030792431 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbaeb400640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec0d01b0 0x7fbaec15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae3e00640 1 -- 172.21.15.194:0/1030792431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaec0b5210 msgr2=0x7fbaec1f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae3e00640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaec0b5210 0x7fbaec1f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae3e00640 1 -- 172.21.15.194:0/1030792431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec0d01b0 msgr2=0x7fbaec15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae3e00640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec0d01b0 0x7fbaec15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae3e00640 1 -- 172.21.15.194:0/1030792431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec1f5a40 con 0x7fbaec0cc140 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbaeb400640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec0d01b0 0x7fbaec15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae3e00640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0cc140 0x7fbaec15d750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009790 tx=0x7fbad40a5740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae2a00640 1 -- 172.21.15.194:0/1030792431 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad40ba020 con 0x7fbaec0cc140 2024-09-10T21:23:48.850 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae2a00640 1 -- 172.21.15.194:0/1030792431 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbad4004430 con 0x7fbaec0cc140 2024-09-10T21:23:48.851 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec15cc70 con 0x7fbaec0cc140 2024-09-10T21:23:48.851 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae2a00640 1 -- 172.21.15.194:0/1030792431 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad40b2040 con 0x7fbaec0cc140 2024-09-10T21:23:48.851 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec15d230 con 0x7fbaec0cc140 2024-09-10T21:23:48.851 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae2a00640 1 -- 172.21.15.194:0/1030792431 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fbad40b5070 con 0x7fbaec0cc140 2024-09-10T21:23:48.851 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbae2a00640 1 -- 172.21.15.194:0/1030792431 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbad40039b0 con 0x7fbaec0cc140 2024-09-10T21:23:48.851 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.845+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec0d05e0 con 0x7fbaec0cc140 2024-09-10T21:23:48.852 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.849+0000 7fbae2a00640 1 -- 172.21.15.194:0/1030792431 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fbad40c4020 con 0x7fbaec0cc140 2024-09-10T21:23:48.921 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.917+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e0d62fef-5f34-4f25-9a73-100f3fbefb45", "id": 1} v 0) -- 0x7fbaec06fa90 con 0x7fbaec0cc140 2024-09-10T21:23:48.925 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbae2a00640 1 -- 172.21.15.194:0/1030792431 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0d62fef-5f34-4f25-9a73-100f3fbefb45", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbad40163f0 con 0x7fbaec0cc140 2024-09-10T21:23:48.926 INFO:teuthology.orchestra.run.smithi194.stdout:1 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0cc140 msgr2=0x7fbaec15d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0cc140 0x7fbaec15d750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009790 tx=0x7fbad40a5740 comp rx=0 tx=0).stop 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 shutdown_connections 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec0d01b0 0x7fbaec15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaec0cc140 0x7fbaec15d750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 --2- 172.21.15.194:0/1030792431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaec0b5210 0x7fbaec1f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 >> 172.21.15.194:0/1030792431 conn(0x7fbaec074200 msgr2=0x7fbaec0cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 shutdown_connections 2024-09-10T21:23:48.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:48.921+0000 7fbaf1c00640 1 -- 172.21.15.194:0/1030792431 wait complete. 2024-09-10T21:23:48.945 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 8a1e08dc-aee0-4208-9e1b-e4541d75f5f6 2 2024-09-10T21:23:49.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 -- 172.21.15.194:0/3872148091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 msgr2=0x7fb1440cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 --2- 172.21.15.194:0/3872148091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 0x7fb1440cea20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb134009840 tx=0x7fb13401c820 comp rx=0 tx=0).stop 2024-09-10T21:23:49.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 -- 172.21.15.194:0/3872148091 shutdown_connections 2024-09-10T21:23:49.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 --2- 172.21.15.194:0/3872148091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 0x7fb1440cea20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 -- 172.21.15.194:0/3872148091 >> 172.21.15.194:0/3872148091 conn(0x7fb1440d1c30 msgr2=0x7fb1440d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 -- 172.21.15.194:0/3872148091 shutdown_connections 2024-09-10T21:23:49.086 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 -- 172.21.15.194:0/3872148091 wait complete. 2024-09-10T21:23:49.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 Processor -- start 2024-09-10T21:23:49.087 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.081+0000 7fb14a400640 1 -- start start 2024-09-10T21:23:49.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1440b0d50 0x7fb144163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1440cd6e0 0x7fb14406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 0x7fb144158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb14416a8a0 con 0x7fb1440cd6e0 2024-09-10T21:23:49.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb14416a720 con 0x7fb1440ce640 2024-09-10T21:23:49.089 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb14416aa20 con 0x7fb1440b0d50 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1440b0d50 0x7fb144163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1440b0d50 0x7fb144163c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:53032/0 (socket says 172.21.15.194:53032) 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 -- 172.21.15.194:0/3431035887 learned_addr learned my addr 172.21.15.194:0/3431035887 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb142a00640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1440cd6e0 0x7fb14406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143e00640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 0x7fb144158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 -- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 msgr2=0x7fb144158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 0x7fb144158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 -- 172.21.15.194:0/3431035887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1440cd6e0 msgr2=0x7fb14406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1440cd6e0 0x7fb14406dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 -- 172.21.15.194:0/3431035887 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb144159220 con 0x7fb1440b0d50 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb142a00640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1440cd6e0 0x7fb14406dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:49.091 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb143400640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1440b0d50 0x7fb144163c30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb134009790 tx=0x7fb134004220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb13b400640 1 -- 172.21.15.194:0/3431035887 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1340b9980 con 0x7fb1440b0d50 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb13b400640 1 -- 172.21.15.194:0/3431035887 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1340b9c10 con 0x7fb1440b0d50 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb144164220 con 0x7fb1440b0d50 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb13b400640 1 -- 172.21.15.194:0/3431035887 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134005030 con 0x7fb1440b0d50 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1441647e0 con 0x7fb1440b0d50 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb13b400640 1 -- 172.21.15.194:0/3431035887 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fb1340c0020 con 0x7fb1440b0d50 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb13b400640 1 -- 172.21.15.194:0/3431035887 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb1340a5860 con 0x7fb1440b0d50 2024-09-10T21:23:49.092 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.085+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14415d440 con 0x7fb1440b0d50 2024-09-10T21:23:49.093 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.089+0000 7fb13b400640 1 -- 172.21.15.194:0/3431035887 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fb14415d440 con 0x7fb1440b0d50 2024-09-10T21:23:49.163 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.157+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8a1e08dc-aee0-4208-9e1b-e4541d75f5f6", "id": 2} v 0) -- 0x7fb1440cea20 con 0x7fb1440b0d50 2024-09-10T21:23:49.168 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb13b400640 1 -- 172.21.15.194:0/3431035887 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8a1e08dc-aee0-4208-9e1b-e4541d75f5f6", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb1340c7070 con 0x7fb1440b0d50 2024-09-10T21:23:49.168 INFO:teuthology.orchestra.run.smithi194.stdout:2 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1440b0d50 msgr2=0x7fb144163c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1440b0d50 0x7fb144163c30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb134009790 tx=0x7fb134004220 comp rx=0 tx=0).stop 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 shutdown_connections 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1440ce640 0x7fb144158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1440cd6e0 0x7fb14406dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 --2- 172.21.15.194:0/3431035887 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1440b0d50 0x7fb144163c30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 >> 172.21.15.194:0/3431035887 conn(0x7fb1440d1c30 msgr2=0x7fb1440d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 shutdown_connections 2024-09-10T21:23:49.170 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.165+0000 7fb14a400640 1 -- 172.21.15.194:0/3431035887 wait complete. 2024-09-10T21:23:49.189 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new dc9ae9e1-4f76-400b-a131-78d535e430f4 3 2024-09-10T21:23:49.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- 172.21.15.194:0/2415537152 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 msgr2=0x7f1e500ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 --2- 172.21.15.194:0/2415537152 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e500ccaa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009870 tx=0x7f1e3c01c860 comp rx=0 tx=0).stop 2024-09-10T21:23:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- 172.21.15.194:0/2415537152 shutdown_connections 2024-09-10T21:23:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 --2- 172.21.15.194:0/2415537152 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e500ccaa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- 172.21.15.194:0/2415537152 >> 172.21.15.194:0/2415537152 conn(0x7f1e50074200 msgr2=0x7f1e50074610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- 172.21.15.194:0/2415537152 shutdown_connections 2024-09-10T21:23:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- 172.21.15.194:0/2415537152 wait complete. 2024-09-10T21:23:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 Processor -- start 2024-09-10T21:23:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- start start 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e500693c0 0x7f1e500c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e50157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e5015bd30 0x7f1e501586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5016a2e0 con 0x7f1e500693c0 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5016a160 con 0x7f1e5015bd30 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e57c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5016a460 con 0x7f1e500d08c0 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e50157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e50157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:53042/0 (socket says 172.21.15.194:53042) 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 -- 172.21.15.194:0/3455404731 learned_addr learned my addr 172.21.15.194:0/3455404731 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e55c00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e5015bd30 0x7f1e501586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e55200640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e500693c0 0x7f1e500c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 -- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e5015bd30 msgr2=0x7f1e501586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e5015bd30 0x7f1e501586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 -- 172.21.15.194:0/3455404731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e500693c0 msgr2=0x7f1e500c2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e500693c0 0x7f1e500c2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.329+0000 7f1e4fe00640 1 -- 172.21.15.194:0/3455404731 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5015b660 con 0x7f1e500d08c0 2024-09-10T21:23:49.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e4fe00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e50157700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0982e0 tx=0x7f1e3c0a5f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e4cc00640 1 -- 172.21.15.194:0/3455404731 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0b2040 con 0x7f1e500d08c0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e4cc00640 1 -- 172.21.15.194:0/3455404731 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e3c0bb070 con 0x7f1e500d08c0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e4cc00640 1 -- 172.21.15.194:0/3455404731 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c003600 con 0x7f1e500d08c0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5015b380 con 0x7f1e500d08c0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e50070370 con 0x7f1e500d08c0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e4cc00640 1 -- 172.21.15.194:0/3455404731 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1e3c0b5070 con 0x7f1e500d08c0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e4cc00640 1 -- 172.21.15.194:0/3455404731 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1e3c0c08f0 con 0x7f1e500d08c0 2024-09-10T21:23:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e500ccaa0 con 0x7f1e500d08c0 2024-09-10T21:23:49.339 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.333+0000 7f1e4cc00640 1 -- 172.21.15.194:0/3455404731 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1e3c016450 con 0x7f1e500d08c0 2024-09-10T21:23:49.408 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.405+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dc9ae9e1-4f76-400b-a131-78d535e430f4", "id": 3} v 0) -- 0x7f1e5006e880 con 0x7f1e500d08c0 2024-09-10T21:23:49.415 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.409+0000 7f1e4cc00640 1 -- 172.21.15.194:0/3455404731 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dc9ae9e1-4f76-400b-a131-78d535e430f4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1e3c0a5780 con 0x7f1e500d08c0 2024-09-10T21:23:49.415 INFO:teuthology.orchestra.run.smithi194.stdout:3 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 msgr2=0x7f1e50157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e50157700 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0982e0 tx=0x7f1e3c0a5f70 comp rx=0 tx=0).stop 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 shutdown_connections 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e5015bd30 0x7f1e501586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e500d08c0 0x7f1e50157700 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 --2- 172.21.15.194:0/3455404731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e500693c0 0x7f1e500c2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 >> 172.21.15.194:0/3455404731 conn(0x7f1e50074200 msgr2=0x7f1e500ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.416 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 shutdown_connections 2024-09-10T21:23:49.417 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.413+0000 7f1e57c00640 1 -- 172.21.15.194:0/3455404731 wait complete. 2024-09-10T21:23:49.433 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 4a9b1edd-edd8-4957-8cb6-1446b54e78f7 4 2024-09-10T21:23:49.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.605+0000 7f45b7200640 1 -- 172.21.15.194:0/1123992398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8154250 msgr2=0x7f45b815aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.610 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.605+0000 7f45b7200640 1 --2- 172.21.15.194:0/1123992398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8154250 0x7f45b815aa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009800 tx=0x7f45b001ddc0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.610 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.605+0000 7f45b7200640 1 -- 172.21.15.194:0/1123992398 shutdown_connections 2024-09-10T21:23:49.610 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.605+0000 7f45b7200640 1 --2- 172.21.15.194:0/1123992398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8154250 0x7f45b815aa00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.610 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.605+0000 7f45b7200640 1 -- 172.21.15.194:0/1123992398 >> 172.21.15.194:0/1123992398 conn(0x7f45b80cf890 msgr2=0x7f45b80cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.610 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.605+0000 7f45b7200640 1 -- 172.21.15.194:0/1123992398 shutdown_connections 2024-09-10T21:23:49.610 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.605+0000 7f45b7200640 1 -- 172.21.15.194:0/1123992398 wait complete. 2024-09-10T21:23:49.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 Processor -- start 2024-09-10T21:23:49.612 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 -- start start 2024-09-10T21:23:49.613 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b806eba0 0x7f45b81461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.613 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45b81583c0 0x7f45b82040d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8156eb0 0x7f45b8155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8169f80 con 0x7f45b8156eb0 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8169e00 con 0x7f45b806eba0 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f45b816a100 con 0x7f45b81583c0 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45b81583c0 0x7f45b82040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b806eba0 0x7f45b81461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b806eba0 0x7f45b81461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50516/0 (socket says 172.21.15.194:50516) 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45b81583c0 0x7f45b82040d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:53056/0 (socket says 172.21.15.194:53056) 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5e00640 1 -- 172.21.15.194:0/504895150 learned_addr learned my addr 172.21.15.194:0/504895150 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b6800640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8156eb0 0x7f45b8155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 -- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b806eba0 msgr2=0x7f45b81461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b806eba0 0x7f45b81461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 -- 172.21.15.194:0/504895150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8156eb0 msgr2=0x7f45b8155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.615 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8156eb0 0x7f45b8155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.616 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 -- 172.21.15.194:0/504895150 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b81466e0 con 0x7f45b81583c0 2024-09-10T21:23:49.616 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5e00640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b806eba0 0x7f45b81461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:49.616 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b6800640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8156eb0 0x7f45b8155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:49.616 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b5400640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45b81583c0 0x7f45b82040d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f45b8075e70 tx=0x7f45b0004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45ae000640 1 -- 172.21.15.194:0/504895150 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0004630 con 0x7f45b81583c0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45ae000640 1 -- 172.21.15.194:0/504895150 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45b00ae430 con 0x7f45b81583c0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45ae000640 1 -- 172.21.15.194:0/504895150 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b00ae680 con 0x7f45b81583c0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b81532b0 con 0x7f45b81583c0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b81f7750 con 0x7f45b81583c0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45ae000640 1 -- 172.21.15.194:0/504895150 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f45b00aebb0 con 0x7f45b81583c0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.609+0000 7f45ae000640 1 -- 172.21.15.194:0/504895150 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f45b00c1bf0 con 0x7f45b81583c0 2024-09-10T21:23:49.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.613+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b815aa00 con 0x7f45b81583c0 2024-09-10T21:23:49.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.613+0000 7f45ae000640 1 -- 172.21.15.194:0/504895150 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f45b00179e0 con 0x7f45b81583c0 2024-09-10T21:23:49.687 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.681+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4a9b1edd-edd8-4957-8cb6-1446b54e78f7", "id": 4} v 0) -- 0x7f45b80d0130 con 0x7f45b81583c0 2024-09-10T21:23:49.694 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45ae000640 1 -- 172.21.15.194:0/504895150 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a9b1edd-edd8-4957-8cb6-1446b54e78f7", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f45b0003c60 con 0x7f45b81583c0 2024-09-10T21:23:49.694 INFO:teuthology.orchestra.run.smithi194.stdout:4 2024-09-10T21:23:49.695 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45b81583c0 msgr2=0x7f45b82040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.695 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45b81583c0 0x7f45b82040d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f45b8075e70 tx=0x7f45b0004380 comp rx=0 tx=0).stop 2024-09-10T21:23:49.695 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 shutdown_connections 2024-09-10T21:23:49.695 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8156eb0 0x7f45b8155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.695 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45b81583c0 0x7f45b82040d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.696 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 --2- 172.21.15.194:0/504895150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45b806eba0 0x7f45b81461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.696 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 >> 172.21.15.194:0/504895150 conn(0x7f45b80cf890 msgr2=0x7f45b8072980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.696 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 shutdown_connections 2024-09-10T21:23:49.696 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.689+0000 7f45b7200640 1 -- 172.21.15.194:0/504895150 wait complete. 2024-09-10T21:23:49.713 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new f888558a-4098-46da-baf8-900de83e509c 5 2024-09-10T21:23:49.856 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.849+0000 7f5dfc200640 1 -- 172.21.15.194:0/3931223041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 msgr2=0x7f5df40cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.856 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.849+0000 7f5dfc200640 1 --2- 172.21.15.194:0/3931223041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df40cff20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009840 tx=0x7f5de001c820 comp rx=0 tx=0).stop 2024-09-10T21:23:49.856 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.849+0000 7f5dfc200640 1 -- 172.21.15.194:0/3931223041 shutdown_connections 2024-09-10T21:23:49.856 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.849+0000 7f5dfc200640 1 --2- 172.21.15.194:0/3931223041 >> v2:172.21.15.97:3300/0 conn(0x7f5df40d0eb0 0x7f5df40cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.857 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.849+0000 7f5dfc200640 1 --2- 172.21.15.194:0/3931223041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df40cff20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.857 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.849+0000 7f5dfc200640 1 -- 172.21.15.194:0/3931223041 >> 172.21.15.194:0/3931223041 conn(0x7f5df40d1c70 msgr2=0x7f5df40d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.857 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 -- 172.21.15.194:0/3931223041 shutdown_connections 2024-09-10T21:23:49.857 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 -- 172.21.15.194:0/3931223041 wait complete. 2024-09-10T21:23:49.860 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 Processor -- start 2024-09-10T21:23:49.860 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 -- start start 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df40cdd40 0x7f5df40b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df40d0eb0 0x7f5df40b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5df416adf0 con 0x7f5df40cdd40 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5df416ac70 con 0x7f5df40d0eb0 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5df416af70 con 0x7f5df40ced60 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfa200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df40d0eb0 0x7f5df40b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df414ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:53078/0 (socket says 172.21.15.194:53078) 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 -- 172.21.15.194:0/437409728 learned_addr learned my addr 172.21.15.194:0/437409728 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df9800640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df40cdd40 0x7f5df40b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 -- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df40d0eb0 msgr2=0x7f5df40b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df40d0eb0 0x7f5df40b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 -- 172.21.15.194:0/437409728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df40cdd40 msgr2=0x7f5df40b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df40cdd40 0x7f5df40b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 -- 172.21.15.194:0/437409728 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df406a010 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfa200640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df40d0eb0 0x7f5df40b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df8e00640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df414ac70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009790 tx=0x7f5de0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df1600640 1 -- 172.21.15.194:0/437409728 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00b2040 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df1600640 1 -- 172.21.15.194:0/437409728 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de0004320 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5df1600640 1 -- 172.21.15.194:0/437409728 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00044f0 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.853+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df40b4260 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.857+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df40b1c20 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.857+0000 7f5df1600640 1 -- 172.21.15.194:0/437409728 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5de00b5030 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.857+0000 7f5df1600640 1 -- 172.21.15.194:0/437409728 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5de0004db0 con 0x7f5df40ced60 2024-09-10T21:23:49.864 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.857+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df40cc2c0 con 0x7f5df40ced60 2024-09-10T21:23:49.865 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.857+0000 7f5df1600640 1 -- 172.21.15.194:0/437409728 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5de0016410 con 0x7f5df40ced60 2024-09-10T21:23:49.902 INFO:tasks.ceph.mgr.x.smithi194.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T21:23:49.902 INFO:tasks.ceph.mgr.x.smithi194.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T21:23:49.902 INFO:tasks.ceph.mgr.x.smithi194.stderr: from numpy import show_config as show_numpy_config 2024-09-10T21:23:49.933 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.929+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f888558a-4098-46da-baf8-900de83e509c", "id": 5} v 0) -- 0x7f5df40ce590 con 0x7f5df40ced60 2024-09-10T21:23:49.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.933+0000 7f5df1600640 1 -- 172.21.15.194:0/437409728 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f888558a-4098-46da-baf8-900de83e509c", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5de00a57d0 con 0x7f5df40ced60 2024-09-10T21:23:49.940 INFO:teuthology.orchestra.run.smithi194.stdout:5 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 msgr2=0x7f5df414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df414ac70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009790 tx=0x7f5de0005dd0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 shutdown_connections 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df40d0eb0 0x7f5df40b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df40ced60 0x7f5df414ac70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 --2- 172.21.15.194:0/437409728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df40cdd40 0x7f5df40b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 >> 172.21.15.194:0/437409728 conn(0x7f5df40d1c70 msgr2=0x7f5df40af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 shutdown_connections 2024-09-10T21:23:49.941 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:49.937+0000 7f5dfc200640 1 -- 172.21.15.194:0/437409728 wait complete. 2024-09-10T21:23:49.959 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 7fa0195a-7dd5-4116-9704-58395df8e70e 6 2024-09-10T21:23:50.102 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 -- 172.21.15.194:0/4249457491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1658161600 msgr2=0x7f16580d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.102 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 --2- 172.21.15.194:0/4249457491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1658161600 0x7f16580d0a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1658073690 tx=0x7f164801c830 comp rx=0 tx=0).stop 2024-09-10T21:23:50.102 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 -- 172.21.15.194:0/4249457491 shutdown_connections 2024-09-10T21:23:50.102 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 --2- 172.21.15.194:0/4249457491 >> v2:172.21.15.97:3300/0 conn(0x7f165806dc50 0x7f16580cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.102 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 --2- 172.21.15.194:0/4249457491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1658161600 0x7f16580d0a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.102 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 -- 172.21.15.194:0/4249457491 >> 172.21.15.194:0/4249457491 conn(0x7f16580d1c30 msgr2=0x7f16580d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:50.103 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 -- 172.21.15.194:0/4249457491 shutdown_connections 2024-09-10T21:23:50.103 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 -- 172.21.15.194:0/4249457491 wait complete. 2024-09-10T21:23:50.103 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 Processor -- start 2024-09-10T21:23:50.103 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.097+0000 7f165e800640 1 -- start start 2024-09-10T21:23:50.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16580b1590 0x7f165815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1658161600 0x7f1658152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1658154ad0 0x7f1658157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f165816b0f0 con 0x7f1658154ad0 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f165816af70 con 0x7f16580b1590 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f165816b270 con 0x7f1658161600 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1658161600 0x7f1658152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1657400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16580b1590 0x7f165815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1658161600 0x7f1658152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:53086/0 (socket says 172.21.15.194:53086) 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 -- 172.21.15.194:0/2951365267 learned_addr learned my addr 172.21.15.194:0/2951365267 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:50.105 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1657400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16580b1590 0x7f165815b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50540/0 (socket says 172.21.15.194:50540) 2024-09-10T21:23:50.106 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1657e00640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1658154ad0 0x7f1658157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 -- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16580b1590 msgr2=0x7f165815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16580b1590 0x7f165815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 -- 172.21.15.194:0/2951365267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1658154ad0 msgr2=0x7f1658157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1658154ad0 0x7f1658157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 -- 172.21.15.194:0/2951365267 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16580b7130 con 0x7f1658161600 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1657400640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16580b1590 0x7f165815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1657e00640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1658154ad0 0x7f1658157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f1656a00640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1658161600 0x7f1658152c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f164400d950 tx=0x7f164400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f164f400640 1 -- 172.21.15.194:0/2951365267 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644017070 con 0x7f1658161600 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f164f400640 1 -- 172.21.15.194:0/2951365267 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1644009d20 con 0x7f1658161600 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f164f400640 1 -- 172.21.15.194:0/2951365267 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16440106c0 con 0x7f1658161600 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16580b6890 con 0x7f1658161600 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f16580b6e50 con 0x7f1658161600 2024-09-10T21:23:50.107 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f164f400640 1 -- 172.21.15.194:0/2951365267 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1644010860 con 0x7f1658161600 2024-09-10T21:23:50.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f164f400640 1 -- 172.21.15.194:0/2951365267 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f1644010b80 con 0x7f1658161600 2024-09-10T21:23:50.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.101+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16580cfe30 con 0x7f1658161600 2024-09-10T21:23:50.108 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.105+0000 7f164f400640 1 -- 172.21.15.194:0/2951365267 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f164401b400 con 0x7f1658161600 2024-09-10T21:23:50.178 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.173+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7fa0195a-7dd5-4116-9704-58395df8e70e", "id": 6} v 0) -- 0x7f16580b2bf0 con 0x7f1658161600 2024-09-10T21:23:50.185 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f164f400640 1 -- 172.21.15.194:0/2951365267 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7fa0195a-7dd5-4116-9704-58395df8e70e", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f164401b5e0 con 0x7f1658161600 2024-09-10T21:23:50.186 INFO:teuthology.orchestra.run.smithi194.stdout:6 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1658161600 msgr2=0x7f1658152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1658161600 0x7f1658152c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f164400d950 tx=0x7f164400de10 comp rx=0 tx=0).stop 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 shutdown_connections 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1658154ad0 0x7f1658157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1658161600 0x7f1658152c40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 --2- 172.21.15.194:0/2951365267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16580b1590 0x7f165815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 >> 172.21.15.194:0/2951365267 conn(0x7f16580d1c30 msgr2=0x7f16580717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:50.187 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 shutdown_connections 2024-09-10T21:23:50.188 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.181+0000 7f165e800640 1 -- 172.21.15.194:0/2951365267 wait complete. 2024-09-10T21:23:50.205 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --cluster ceph osd new 360c2eb5-82b4-4c2b-a316-be76b4817f41 7 2024-09-10T21:23:50.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.341+0000 7fd186200640 1 -- 172.21.15.194:0/623552747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800afdd0 msgr2=0x7fd1800cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.341+0000 7fd186200640 1 --2- 172.21.15.194:0/623552747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800afdd0 0x7fd1800cc6d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009840 tx=0x7fd16c01c820 comp rx=0 tx=0).stop 2024-09-10T21:23:50.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.341+0000 7fd186200640 1 -- 172.21.15.194:0/623552747 shutdown_connections 2024-09-10T21:23:50.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.341+0000 7fd186200640 1 --2- 172.21.15.194:0/623552747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800afdd0 0x7fd1800cc6d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.341+0000 7fd186200640 1 -- 172.21.15.194:0/623552747 >> 172.21.15.194:0/623552747 conn(0x7fd1800d1c30 msgr2=0x7fd1800d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:50.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.341+0000 7fd186200640 1 -- 172.21.15.194:0/623552747 shutdown_connections 2024-09-10T21:23:50.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.341+0000 7fd186200640 1 -- 172.21.15.194:0/623552747 wait complete. 2024-09-10T21:23:50.349 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 Processor -- start 2024-09-10T21:23:50.349 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 -- start start 2024-09-10T21:23:50.351 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1800afdd0 0x7fd18006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.351 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1800b0d50 0x7fd18006e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.351 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800ce640 0x7fd18006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.351 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd180169ef0 con 0x7fd1800ce640 2024-09-10T21:23:50.351 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd180169d70 con 0x7fd1800b0d50 2024-09-10T21:23:50.351 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd18016a070 con 0x7fd1800afdd0 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1800afdd0 0x7fd18006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1800b0d50 0x7fd18006e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1800b0d50 0x7fd18006e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50544/0 (socket says 172.21.15.194:50544) 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 -- 172.21.15.194:0/1030070281 learned_addr learned my addr 172.21.15.194:0/1030070281 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1800afdd0 0x7fd18006dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.194:53088/0 (socket says 172.21.15.194:53088) 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17fe00640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800ce640 0x7fd18006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 -- 172.21.15.194:0/1030070281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1800afdd0 msgr2=0x7fd18006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1800afdd0 0x7fd18006dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 -- 172.21.15.194:0/1030070281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800ce640 msgr2=0x7fd18006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800ce640 0x7fd18006e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 -- 172.21.15.194:0/1030070281 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1800b5f50 con 0x7fd1800b0d50 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd17ea00640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1800b0d50 0x7fd18006e190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd16c01a450 tx=0x7fd16c0a5780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:50.352 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd177400640 1 -- 172.21.15.194:0/1030070281 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0bc070 con 0x7fd1800b0d50 2024-09-10T21:23:50.353 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd177400640 1 -- 172.21.15.194:0/1030070281 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd16c003600 con 0x7fd1800b0d50 2024-09-10T21:23:50.353 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd177400640 1 -- 172.21.15.194:0/1030070281 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0b2040 con 0x7fd1800b0d50 2024-09-10T21:23:50.353 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd180152770 con 0x7fd1800b0d50 2024-09-10T21:23:50.353 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd180166f90 con 0x7fd1800b0d50 2024-09-10T21:23:50.353 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd177400640 1 -- 172.21.15.194:0/1030070281 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd16c0b5070 con 0x7fd1800b0d50 2024-09-10T21:23:50.353 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.345+0000 7fd177400640 1 -- 172.21.15.194:0/1030070281 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fd16c0c1af0 con 0x7fd1800b0d50 2024-09-10T21:23:50.353 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.349+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1800cea20 con 0x7fd1800b0d50 2024-09-10T21:23:50.354 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.349+0000 7fd177400640 1 -- 172.21.15.194:0/1030070281 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd16c0d9950 con 0x7fd1800b0d50 2024-09-10T21:23:50.366 INFO:tasks.ceph.mgr.y.smithi097.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T21:23:50.366 INFO:tasks.ceph.mgr.y.smithi097.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T21:23:50.366 INFO:tasks.ceph.mgr.y.smithi097.stderr: from numpy import show_config as show_numpy_config 2024-09-10T21:23:50.423 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.417+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd new", "uuid": "360c2eb5-82b4-4c2b-a316-be76b4817f41", "id": 7} v 0) -- 0x7fd1800cef40 con 0x7fd1800b0d50 2024-09-10T21:23:50.429 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd177400640 1 -- 172.21.15.194:0/1030070281 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "360c2eb5-82b4-4c2b-a316-be76b4817f41", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fd16c0c7030 con 0x7fd1800b0d50 2024-09-10T21:23:50.430 INFO:teuthology.orchestra.run.smithi194.stdout:7 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1800b0d50 msgr2=0x7fd18006e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1800b0d50 0x7fd18006e190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd16c01a450 tx=0x7fd16c0a5780 comp rx=0 tx=0).stop 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 shutdown_connections 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1800ce640 0x7fd18006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1800b0d50 0x7fd18006e190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 --2- 172.21.15.194:0/1030070281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1800afdd0 0x7fd18006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 >> 172.21.15.194:0/1030070281 conn(0x7fd1800d1c30 msgr2=0x7fd180071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 shutdown_connections 2024-09-10T21:23:50.431 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:23:50.425+0000 7fd186200640 1 -- 172.21.15.194:0/1030070281 wait complete. 2024-09-10T21:23:50.449 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T21:23:50.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T21:23:50.452 INFO:tasks.ceph.osd.0:Started 2024-09-10T21:23:50.453 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T21:23:50.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T21:23:50.456 INFO:tasks.ceph.osd.1:Started 2024-09-10T21:23:50.456 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T21:23:50.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T21:23:50.459 INFO:tasks.ceph.osd.2:Started 2024-09-10T21:23:50.459 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T21:23:50.459 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T21:23:50.463 INFO:tasks.ceph.osd.3:Started 2024-09-10T21:23:50.463 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T21:23:50.463 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T21:23:50.466 INFO:tasks.ceph.osd.4:Started 2024-09-10T21:23:50.466 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T21:23:50.466 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T21:23:50.469 INFO:tasks.ceph.osd.5:Started 2024-09-10T21:23:50.469 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T21:23:50.469 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T21:23:50.472 INFO:tasks.ceph.osd.6:Started 2024-09-10T21:23:50.473 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T21:23:50.473 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T21:23:50.477 INFO:tasks.ceph.osd.7:Started 2024-09-10T21:23:50.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:23:50.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- 172.21.15.97:0/1428303238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c800051a0 msgr2=0x7f2c80005620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 --2- 172.21.15.97:0/1428303238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c800051a0 0x7f2c80005620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c94006f90 tx=0x7f2c9401f680 comp rx=0 tx=0).stop 2024-09-10T21:23:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2c9be00640 1 -- 172.21.15.97:0/1428303238 reap_dead start 2024-09-10T21:23:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- 172.21.15.97:0/1428303238 shutdown_connections 2024-09-10T21:23:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- 172.21.15.97:0/1428303238 >> 172.21.15.97:0/1428303238 conn(0x7f2c9c0cfb10 msgr2=0x7f2c9c0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- 172.21.15.97:0/1428303238 shutdown_connections 2024-09-10T21:23:50.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- 172.21.15.97:0/1428303238 wait complete. 2024-09-10T21:23:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 Processor -- start 2024-09-10T21:23:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- start start 2024-09-10T21:23:50.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c800051a0 0x7f2c9c0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0af780 0x7f2c9c0afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c9c0aa730 0x7f2c9c0aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c0b4460 con 0x7f2c9c0af780 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c0b42e0 con 0x7f2c9c0aa730 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.652+0000 7f2ca2200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9c073be0 con 0x7f2c800051a0 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0af780 0x7f2c9c0afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0af780 0x7f2c9c0afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44070/0 (socket says 172.21.15.97:44070) 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 -- 172.21.15.97:0/4227200587 learned_addr learned my addr 172.21.15.97:0/4227200587 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9be00640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c9c0aa730 0x7f2c9c0aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9b400640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c800051a0 0x7f2c9c0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 -- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c800051a0 msgr2=0x7f2c9c0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c800051a0 0x7f2c9c0cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 -- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c9c0aa730 msgr2=0x7f2c9c0aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c9c0aa730 0x7f2c9c0aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 -- 172.21.15.97:0/4227200587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c0b0120 con 0x7f2c9c0af780 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c9aa00640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0af780 0x7f2c9c0afbe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c840118d0 tx=0x7f2c84011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c93400640 1 -- 172.21.15.97:0/4227200587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8401e070 con 0x7f2c9c0af780 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c93400640 1 -- 172.21.15.97:0/4227200587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c8400fbb0 con 0x7f2c9c0af780 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c93400640 1 -- 172.21.15.97:0/4227200587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84019410 con 0x7f2c9c0af780 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c0ab500 con 0x7f2c9c0af780 2024-09-10T21:23:50.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c0abac0 con 0x7f2c9c0af780 2024-09-10T21:23:50.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c93400640 1 -- 172.21.15.97:0/4227200587 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2c840195b0 con 0x7f2c9c0af780 2024-09-10T21:23:50.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2c93400640 1 -- 172.21.15.97:0/4227200587 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2c84018a80 con 0x7f2c9c0af780 2024-09-10T21:23:50.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.656+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9c070df0 con 0x7f2c9c0af780 2024-09-10T21:23:50.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.660+0000 7f2c93400640 1 -- 172.21.15.97:0/4227200587 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2c84018c80 con 0x7f2c9c0af780 2024-09-10T21:23:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c9c0d0100 con 0x7f2c9c0af780 2024-09-10T21:23:50.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2c93400640 1 -- 172.21.15.97:0/4227200587 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2c84023440 con 0x7f2c9c0af780 2024-09-10T21:23:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:23:50.727 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:23:50.422719+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:23:50.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0af780 msgr2=0x7f2c9c0afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:50.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0af780 0x7f2c9c0afbe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c840118d0 tx=0x7f2c84011d90 comp rx=0 tx=0).stop 2024-09-10T21:23:50.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 shutdown_connections 2024-09-10T21:23:50.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c9c0aa730 0x7f2c9c0aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0af780 0x7f2c9c0afbe0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 --2- 172.21.15.97:0/4227200587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c800051a0 0x7f2c9c0cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:50.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 >> 172.21.15.97:0/4227200587 conn(0x7f2c9c0cfb10 msgr2=0x7f2c9c167080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:50.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 shutdown_connections 2024-09-10T21:23:50.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:23:50.720+0000 7f2ca2200640 1 -- 172.21.15.97:0/4227200587 wait complete. 2024-09-10T21:23:50.746 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T21:23:50.746 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T21:23:51.552 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:23:51.685 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 -- 172.21.15.97:0/2929525812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d0350 msgr2=0x7f84a40709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:51.685 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 --2- 172.21.15.97:0/2929525812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d0350 0x7f84a40709c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8490009900 tx=0x7f849001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.685 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 -- 172.21.15.97:0/2929525812 shutdown_connections 2024-09-10T21:23:51.685 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 --2- 172.21.15.97:0/2929525812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84a40d07e0 0x7f84a4070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.686 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 --2- 172.21.15.97:0/2929525812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d0350 0x7f84a40709c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.686 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 -- 172.21.15.97:0/2929525812 >> 172.21.15.97:0/2929525812 conn(0x7f84a40b3820 msgr2=0x7f84a40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:51.686 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 -- 172.21.15.97:0/2929525812 shutdown_connections 2024-09-10T21:23:51.686 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.676+0000 7f84abe00640 1 -- 172.21.15.97:0/2929525812 wait complete. 2024-09-10T21:23:51.686 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 Processor -- start 2024-09-10T21:23:51.686 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 -- start start 2024-09-10T21:23:51.689 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a40d0350 0x7f84a40b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:51.689 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d07e0 0x7f84a40b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:51.689 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84a415f1a0 0x7f84a406e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:51.689 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40ce070 con 0x7f84a40d0350 2024-09-10T21:23:51.689 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40b4550 con 0x7f84a40d07e0 2024-09-10T21:23:51.689 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f84a40ce1f0 con 0x7f84a415f1a0 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a40d0350 0x7f84a40b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a40d0350 0x7f84a40b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44080/0 (socket says 172.21.15.97:44080) 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 -- 172.21.15.97:0/3695489731 learned_addr learned my addr 172.21.15.97:0/3695489731 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a8a00640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d07e0 0x7f84a40b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9e00640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84a415f1a0 0x7f84a406e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 -- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84a415f1a0 msgr2=0x7f84a406e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84a415f1a0 0x7f84a406e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 -- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d07e0 msgr2=0x7f84a40b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d07e0 0x7f84a40b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 -- 172.21.15.97:0/3695489731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a41f7fb0 con 0x7f84a40d0350 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a8a00640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d07e0 0x7f84a40b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a9400640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a40d0350 0x7f84a40b5b70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f849800c3f0 tx=0x7f849800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a1600640 1 -- 172.21.15.97:0/3695489731 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498017070 con 0x7f84a40d0350 2024-09-10T21:23:51.690 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a1600640 1 -- 172.21.15.97:0/3695489731 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84980053c0 con 0x7f84a40d0350 2024-09-10T21:23:51.691 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a41f8290 con 0x7f84a40d0350 2024-09-10T21:23:51.691 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84a1600640 1 -- 172.21.15.97:0/3695489731 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498003860 con 0x7f84a40d0350 2024-09-10T21:23:51.691 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.680+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41f8850 con 0x7f84a40d0350 2024-09-10T21:23:51.691 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.684+0000 7f84a1600640 1 -- 172.21.15.97:0/3695489731 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f8498003a00 con 0x7f84a40d0350 2024-09-10T21:23:51.691 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.684+0000 7f84a1600640 1 -- 172.21.15.97:0/3695489731 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8498012af0 con 0x7f84a40d0350 2024-09-10T21:23:51.691 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.684+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a40b7c50 con 0x7f84a40d0350 2024-09-10T21:23:51.692 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.684+0000 7f84a1600640 1 -- 172.21.15.97:0/3695489731 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f8498021020 con 0x7f84a40d0350 2024-09-10T21:23:51.751 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84a4070eb0 con 0x7f84a40d0350 2024-09-10T21:23:51.751 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84a1600640 1 -- 172.21.15.97:0/3695489731 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f8498012420 con 0x7f84a40d0350 2024-09-10T21:23:51.752 INFO:teuthology.misc.health.smithi097.stdout: 2024-09-10T21:23:51.752 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":10,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:23:50.422719+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a40d0350 msgr2=0x7f84a40b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a40d0350 0x7f84a40b5b70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f849800c3f0 tx=0x7f849800c8b0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 shutdown_connections 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84a415f1a0 0x7f84a406e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84a40d07e0 0x7f84a40b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 --2- 172.21.15.97:0/3695489731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a40d0350 0x7f84a40b5b70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 >> 172.21.15.97:0/3695489731 conn(0x7f84a40b3820 msgr2=0x7f84a40b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 shutdown_connections 2024-09-10T21:23:51.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:51.744+0000 7f84abe00640 1 -- 172.21.15.97:0/3695489731 wait complete. 2024-09-10T21:23:51.770 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T21:23:51.941 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-09-10T21:23:51.937+0000 7fd8829077c0 -1 Falling back to public interface 2024-09-10T21:23:52.186 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-10T21:23:52.180+0000 7f6f36dd37c0 -1 Falling back to public interface 2024-09-10T21:23:52.201 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-09-10T21:23:52.197+0000 7fd09850b7c0 -1 Falling back to public interface 2024-09-10T21:23:52.222 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-10T21:23:52.216+0000 7f68f9eb27c0 -1 Falling back to public interface 2024-09-10T21:23:52.223 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-10T21:23:52.216+0000 7fb2379197c0 -1 Falling back to public interface 2024-09-10T21:23:52.229 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-09-10T21:23:52.225+0000 7ff25a6377c0 -1 Falling back to public interface 2024-09-10T21:23:52.233 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-09-10T21:23:52.229+0000 7f48a323d7c0 -1 Falling back to public interface 2024-09-10T21:23:52.463 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-10T21:23:52.456+0000 7fc23d1517c0 -1 Falling back to public interface 2024-09-10T21:23:53.072 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-09-10T21:23:53.065+0000 7fabcf600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T21:23:53.074 INFO:tasks.ceph.mgr.x.smithi194.stderr:2024-09-10T21:23:53.069+0000 7fabcf600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T21:23:56.204 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-09-10T21:23:56.196+0000 7fd8829077c0 -1 osd.4 0 log_to_monitors true 2024-09-10T21:23:56.286 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-10T21:23:56.280+0000 7f6f36dd37c0 -1 osd.3 0 log_to_monitors true 2024-09-10T21:23:56.354 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-09-10T21:23:56.348+0000 7fd09850b7c0 -1 osd.6 0 log_to_monitors true 2024-09-10T21:23:56.463 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-10T21:23:56.456+0000 7fb2379197c0 -1 osd.1 0 log_to_monitors true 2024-09-10T21:23:56.508 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-09-10T21:23:56.504+0000 7ff25a6377c0 -1 osd.5 0 log_to_monitors true 2024-09-10T21:23:56.526 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-09-10T21:23:56.520+0000 7f48a323d7c0 -1 osd.7 0 log_to_monitors true 2024-09-10T21:23:56.623 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-10T21:23:56.616+0000 7fc23d1517c0 -1 osd.0 0 log_to_monitors true 2024-09-10T21:23:56.654 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-10T21:23:56.648+0000 7f68f9eb27c0 -1 osd.2 0 log_to_monitors true 2024-09-10T21:23:58.161 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-09-10T21:23:58.156+0000 7fd87d800640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T21:23:58.193 INFO:tasks.ceph.osd.4.smithi194.stderr:2024-09-10T21:23:58.188+0000 7fd873800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:23:58.577 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 -- 172.21.15.97:0/3257801612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e40d2150 msgr2=0x7fc7e41622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/3257801612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e40d2150 0x7fc7e41622b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc7e408e200 tx=0x7fc7d001c830 comp rx=0 tx=0).stop 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 -- 172.21.15.97:0/3257801612 shutdown_connections 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/3257801612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7e41627f0 0x7fc7e40d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/3257801612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e40d2150 0x7fc7e41622b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/3257801612 >> v2:172.21.15.194:3300/0 conn(0x7fc7e4161ed0 0x7fc7e40d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 -- 172.21.15.97:0/3257801612 >> 172.21.15.97:0/3257801612 conn(0x7fc7e409d160 msgr2=0x7fc7e40d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 -- 172.21.15.97:0/3257801612 shutdown_connections 2024-09-10T21:23:58.744 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.736+0000 7fc7e8a00640 1 -- 172.21.15.97:0/3257801612 wait complete. 2024-09-10T21:23:58.746 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 Processor -- start 2024-09-10T21:23:58.746 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 -- start start 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7e40d2150 0x7fc7e40cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e4161ed0 0x7fc7e414eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7e41627f0 0x7fc7e41fc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40cefb0 con 0x7fc7e4161ed0 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40cee30 con 0x7fc7e41627f0 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e40cf130 con 0x7fc7e40d2150 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e4161ed0 0x7fc7e414eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e4161ed0 0x7fc7e414eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43364/0 (socket says 172.21.15.97:43364) 2024-09-10T21:23:58.748 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 -- 172.21.15.97:0/796931367 learned_addr learned my addr 172.21.15.97:0/796931367 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:23:58.749 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e2800640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7e41627f0 0x7fc7e41fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:58.749 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1e00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7e40d2150 0x7fc7e40cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:58.749 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 -- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7e40d2150 msgr2=0x7fc7e40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:58.749 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7e40d2150 0x7fc7e40cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.749 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 -- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7e41627f0 msgr2=0x7fc7e41fc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:58.749 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7e41627f0 0x7fc7e41fc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.749 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 -- 172.21.15.97:0/796931367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e40b9b10 con 0x7fc7e4161ed0 2024-09-10T21:23:58.750 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e1400640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e4161ed0 0x7fc7e414eb40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc7d001aee0 tx=0x7fc7d00a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:58.750 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7da000640 1 -- 172.21.15.97:0/796931367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00b2040 con 0x7fc7e4161ed0 2024-09-10T21:23:58.750 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7da000640 1 -- 172.21.15.97:0/796931367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7d00a6e50 con 0x7fc7e4161ed0 2024-09-10T21:23:58.750 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7da000640 1 -- 172.21.15.97:0/796931367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0004290 con 0x7fc7e4161ed0 2024-09-10T21:23:58.750 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e41fc980 con 0x7fc7e4161ed0 2024-09-10T21:23:58.750 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.740+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e40b9830 con 0x7fc7e4161ed0 2024-09-10T21:23:58.751 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.744+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e40cf5e0 con 0x7fc7e4161ed0 2024-09-10T21:23:58.753 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.748+0000 7fc7da000640 1 -- 172.21.15.97:0/796931367 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 96338+0+0 (secure 0 0 0) 0x7fc7d0005070 con 0x7fc7e4161ed0 2024-09-10T21:23:58.755 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.748+0000 7fc7da000640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7c4072580 0x7fc7c4074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:58.755 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.748+0000 7fc7da000640 1 -- 172.21.15.97:0/796931367 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7fc7d012bca0 con 0x7fc7e4161ed0 2024-09-10T21:23:58.755 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.748+0000 7fc7e1e00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7c4072580 0x7fc7c4074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:58.756 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.748+0000 7fc7e1e00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7c4072580 0x7fc7c4074a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40b2b20 tx=0x7fc7d40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:58.764 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.756+0000 7fc7da000640 1 -- 172.21.15.97:0/796931367 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc7d00164b0 con 0x7fc7e4161ed0 2024-09-10T21:23:58.964 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.956+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7e408dc40 con 0x7fc7e4161ed0 2024-09-10T21:23:58.965 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.956+0000 7fc7da000640 1 -- 172.21.15.97:0/796931367 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7567 (secure 0 0 0) 0x7fc7d00f79f0 con 0x7fc7e4161ed0 2024-09-10T21:23:58.965 INFO:teuthology.misc.health.smithi097.stdout: 2024-09-10T21:23:58.965 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":13,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:23:58.265072+0000","last_up_change":"2024-09-10T21:23:58.265072+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:23:58.968 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7c4072580 msgr2=0x7fc7c4074a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:58.968 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7c4072580 0x7fc7c4074a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40b2b20 tx=0x7fc7d40091e0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.968 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e4161ed0 msgr2=0x7fc7e414eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e4161ed0 0x7fc7e414eb40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc7d001aee0 tx=0x7fc7d00a56d0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 shutdown_connections 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7c4072580 0x7fc7c4074a40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7e41627f0 0x7fc7e41fc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7e4161ed0 0x7fc7e414eb40 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc7d001aee0 tx=0x7fc7d00a56d0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 --2- 172.21.15.97:0/796931367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7e40d2150 0x7fc7e40cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 >> 172.21.15.97:0/796931367 conn(0x7fc7e409d160 msgr2=0x7fc7e41594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 shutdown_connections 2024-09-10T21:23:58.969 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:23:58.960+0000 7fc7e8a00640 1 -- 172.21.15.97:0/796931367 wait complete. 2024-09-10T21:23:58.998 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-10T21:24:00.073 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-09-10T21:24:00.068+0000 7f489e200640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T21:24:00.104 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-09-10T21:24:00.096+0000 7ff255e00640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T21:24:00.105 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-09-10T21:24:00.100+0000 7fd093400640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T21:24:00.109 INFO:tasks.ceph.osd.7.smithi194.stderr:2024-09-10T21:24:00.104+0000 7f4894200640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:00.129 INFO:tasks.ceph.osd.5.smithi194.stderr:2024-09-10T21:24:00.124+0000 7ff24b400640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:00.149 INFO:tasks.ceph.osd.6.smithi194.stderr:2024-09-10T21:24:00.144+0000 7fd089400640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:00.623 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-10T21:24:00.616+0000 7f6f31c00640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T21:24:00.655 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-10T21:24:00.648+0000 7fb232800640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T21:24:00.661 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-10T21:24:00.652+0000 7f6f27c00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:00.692 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-10T21:24:00.684+0000 7fb228800640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:00.719 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-10T21:24:00.712+0000 7f68f4e00640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T21:24:00.746 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-10T21:24:00.740+0000 7f68eae00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:00.831 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-10T21:24:00.824+0000 7fc238000640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T21:24:00.858 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-10T21:24:00.852+0000 7fc22e000640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:05.804 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- 172.21.15.97:0/3031730241 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 msgr2=0x7f366c0723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 --2- 172.21.15.97:0/3031730241 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 0x7f366c0723e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3654009900 tx=0x7f365401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- 172.21.15.97:0/3031730241 shutdown_connections 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 --2- 172.21.15.97:0/3031730241 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 0x7f366c0723e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 --2- 172.21.15.97:0/3031730241 >> v2:172.21.15.194:3300/0 conn(0x7f366c15f170 0x7f366c0d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- 172.21.15.97:0/3031730241 >> 172.21.15.97:0/3031730241 conn(0x7f366c0d1ca0 msgr2=0x7f366c0d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- 172.21.15.97:0/3031730241 shutdown_connections 2024-09-10T21:24:05.946 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- 172.21.15.97:0/3031730241 wait complete. 2024-09-10T21:24:05.947 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 Processor -- start 2024-09-10T21:24:05.947 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- start start 2024-09-10T21:24:05.949 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0cfba0 0x7f366c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:05.949 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f366c15f170 0x7f366c1f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:05.949 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 0x7f366c0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:05.949 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0d03a0 con 0x7f366c0cfba0 2024-09-10T21:24:05.949 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0d0220 con 0x7f366c15f170 2024-09-10T21:24:05.949 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3672e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f366c0d0520 con 0x7f366c15f550 2024-09-10T21:24:05.949 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0cfba0 0x7f366c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0cfba0 0x7f366c0c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43372/0 (socket says 172.21.15.97:43372) 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 -- 172.21.15.97:0/2171031964 learned_addr learned my addr 172.21.15.97:0/2171031964 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3663400640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f366c15f170 0x7f366c1f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f3670e00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 0x7f366c0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 -- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 msgr2=0x7f366c0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 0x7f366c0c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 -- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f366c15f170 msgr2=0x7f366c1f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f366c15f170 0x7f366c1f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.940+0000 7f366be00640 1 -- 172.21.15.97:0/2171031964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c0c75e0 con 0x7f366c0cfba0 2024-09-10T21:24:05.950 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f366be00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0cfba0 0x7f366c0c57d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f365c009d90 tx=0x7f365c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:05.951 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3668c00640 1 -- 172.21.15.97:0/2171031964 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c003b80 con 0x7f366c0cfba0 2024-09-10T21:24:05.951 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3668c00640 1 -- 172.21.15.97:0/2171031964 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f365c003d20 con 0x7f366c0cfba0 2024-09-10T21:24:05.951 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c0c78c0 con 0x7f366c0cfba0 2024-09-10T21:24:05.951 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3668c00640 1 -- 172.21.15.97:0/2171031964 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c004280 con 0x7f366c0cfba0 2024-09-10T21:24:05.952 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c0c7e80 con 0x7f366c0cfba0 2024-09-10T21:24:05.952 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c0d27c0 con 0x7f366c0cfba0 2024-09-10T21:24:05.952 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3668c00640 1 -- 172.21.15.97:0/2171031964 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f365c004420 con 0x7f366c0cfba0 2024-09-10T21:24:05.953 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.944+0000 7f3668c00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36480725e0 0x7f3648074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:05.953 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.948+0000 7f3668c00640 1 -- 172.21.15.97:0/2171031964 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f365c093ba0 con 0x7f366c0cfba0 2024-09-10T21:24:05.953 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.948+0000 7f3663400640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36480725e0 0x7f3648074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:05.954 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.948+0000 7f3663400640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36480725e0 0x7f3648074aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f36540b2000 tx=0x7f365401d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:05.958 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:05.952+0000 7f3668c00640 1 -- 172.21.15.97:0/2171031964 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f365c05eee0 con 0x7f366c0cfba0 2024-09-10T21:24:06.124 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.116+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f366c15e770 con 0x7f366c0cfba0 2024-09-10T21:24:06.125 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.116+0000 7f3668c00640 1 -- 172.21.15.97:0/2171031964 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12920 (secure 0 0 0) 0x7f365c063910 con 0x7f366c0cfba0 2024-09-10T21:24:06.125 INFO:teuthology.misc.health.smithi097.stdout: 2024-09-10T21:24:06.125 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":18,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:04.112352+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:06.127 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36480725e0 msgr2=0x7f3648074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36480725e0 0x7f3648074aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f36540b2000 tx=0x7f365401d910 comp rx=0 tx=0).stop 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0cfba0 msgr2=0x7f366c0c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0cfba0 0x7f366c0c57d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f365c009d90 tx=0x7f365c00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 shutdown_connections 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36480725e0 0x7f3648074aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f366c15f550 0x7f366c0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f366c15f170 0x7f366c1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 --2- 172.21.15.97:0/2171031964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0cfba0 0x7f366c0c57d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 >> 172.21.15.97:0/2171031964 conn(0x7f366c0d1ca0 msgr2=0x7f366c0d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 shutdown_connections 2024-09-10T21:24:06.128 INFO:teuthology.misc.health.smithi097.stderr:2024-09-10T21:24:06.120+0000 7f3672e00640 1 -- 172.21.15.97:0/2171031964 wait complete. 2024-09-10T21:24:06.144 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T21:24:06.144 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T21:24:06.144 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:24:06.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:24:06.148 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:24:06.148 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:24:06.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:24:06.151 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:24:06.151 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:24:06.151 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:24:06.155 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:24:06.155 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:24:06.155 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:24:06.158 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:24:06.158 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T21:24:06.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:06.244 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:24:06.246 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:24:06.256 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:24:06.262 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:24:06.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 -- 172.21.15.97:0/2779305611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 msgr2=0x7f6c500b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:06.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2779305611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c500b2010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009900 tx=0x7f6c4001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 -- 172.21.15.97:0/2779305611 shutdown_connections 2024-09-10T21:24:06.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2779305611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c500d0710 0x7f6c500ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2779305611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c500b2010 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 -- 172.21.15.97:0/2779305611 >> 172.21.15.97:0/2779305611 conn(0x7f6c500d1ca0 msgr2=0x7f6c500d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 -- 172.21.15.97:0/2779305611 shutdown_connections 2024-09-10T21:24:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.304+0000 7f6c55a00640 1 -- 172.21.15.97:0/2779305611 wait complete. 2024-09-10T21:24:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 Processor -- start 2024-09-10T21:24:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 -- start start 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c500cde30 0x7f6c501f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c5014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c500d0710 0x7f6c5006eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500cd3b0 con 0x7f6c500ce5e0 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500cd230 con 0x7f6c500cde30 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6c50169060 con 0x7f6c500d0710 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c5014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c5014dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43408/0 (socket says 172.21.15.97:43408) 2024-09-10T21:24:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 -- 172.21.15.97:0/2982600169 learned_addr learned my addr 172.21.15.97:0/2982600169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:06.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4f400640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c500d0710 0x7f6c5006eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:06.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4ea00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c500cde30 0x7f6c501f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 -- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c500d0710 msgr2=0x7f6c5006eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c500d0710 0x7f6c5006eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 -- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c500cde30 msgr2=0x7f6c501f8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c500cde30 0x7f6c501f8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 -- 172.21.15.97:0/2982600169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500cb410 con 0x7f6c500ce5e0 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c4e000640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c5014dfc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e7a0 tx=0x7f6c3c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c46a00640 1 -- 172.21.15.97:0/2982600169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00ccc0 con 0x7f6c500ce5e0 2024-09-10T21:24:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c46a00640 1 -- 172.21.15.97:0/2982600169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c3c00ce60 con 0x7f6c500ce5e0 2024-09-10T21:24:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c501faed0 con 0x7f6c500ce5e0 2024-09-10T21:24:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c46a00640 1 -- 172.21.15.97:0/2982600169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c005260 con 0x7f6c500ce5e0 2024-09-10T21:24:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.308+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5014e500 con 0x7f6c500ce5e0 2024-09-10T21:24:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.312+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c500b2010 con 0x7f6c500ce5e0 2024-09-10T21:24:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.312+0000 7f6c46a00640 1 -- 172.21.15.97:0/2982600169 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6c3c005400 con 0x7f6c500ce5e0 2024-09-10T21:24:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.312+0000 7f6c46a00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c300725e0 0x7f6c30074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.312+0000 7f6c46a00640 1 -- 172.21.15.97:0/2982600169 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f6c3c093740 con 0x7f6c500ce5e0 2024-09-10T21:24:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.312+0000 7f6c4ea00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c300725e0 0x7f6c30074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.312+0000 7f6c4ea00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c300725e0 0x7f6c30074aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c400af000 tx=0x7f6c4001b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.316+0000 7f6c46a00640 1 -- 172.21.15.97:0/2982600169 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c3c05eb10 con 0x7f6c500ce5e0 2024-09-10T21:24:06.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.516+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c500d0060 con 0x7f6c500ce5e0 2024-09-10T21:24:06.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.516+0000 7f6c46a00640 1 -- 172.21.15.97:0/2982600169 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f6c3c063540 con 0x7f6c500ce5e0 2024-09-10T21:24:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:06.525 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"btime":"2024-09-10T21:24:06:258691+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:24:06.530 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-09-10T21:24:06.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c300725e0 msgr2=0x7f6c30074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c300725e0 0x7f6c30074aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c400af000 tx=0x7f6c4001b120 comp rx=0 tx=0).stop 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 msgr2=0x7f6c5014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c5014dfc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e7a0 tx=0x7f6c3c00ec60 comp rx=0 tx=0).stop 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 shutdown_connections 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c300725e0 0x7f6c30074aa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c500d0710 0x7f6c5006eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c500ce5e0 0x7f6c5014dfc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 --2- 172.21.15.97:0/2982600169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c500cde30 0x7f6c501f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 >> 172.21.15.97:0/2982600169 conn(0x7f6c500d1ca0 msgr2=0x7f6c500ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 shutdown_connections 2024-09-10T21:24:06.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:06.520+0000 7f6c55a00640 1 -- 172.21.15.97:0/2982600169 wait complete. 2024-09-10T21:24:08.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:08.387 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:08.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.692+0000 7feb27400640 1 -- 172.21.15.97:0/492864138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 msgr2=0x7feb040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.692+0000 7feb27400640 1 --2- 172.21.15.97:0/492864138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 0x7feb040051d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb18009ee0 tx=0x7feb1801d5f0 comp rx=0 tx=0).stop 2024-09-10T21:24:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.692+0000 7feb25400640 1 -- 172.21.15.97:0/492864138 reap_dead start 2024-09-10T21:24:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.692+0000 7feb27400640 1 -- 172.21.15.97:0/492864138 shutdown_connections 2024-09-10T21:24:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.692+0000 7feb27400640 1 -- 172.21.15.97:0/492864138 >> 172.21.15.97:0/492864138 conn(0x7feb200d1ca0 msgr2=0x7feb200d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.692+0000 7feb27400640 1 -- 172.21.15.97:0/492864138 shutdown_connections 2024-09-10T21:24:08.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.692+0000 7feb27400640 1 -- 172.21.15.97:0/492864138 wait complete. 2024-09-10T21:24:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 Processor -- start 2024-09-10T21:24:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 -- start start 2024-09-10T21:24:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 0x7feb20154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb20156150 0x7feb200b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb200cace0 0x7feb200cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb200cf250 con 0x7feb04004d50 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb200cf0d0 con 0x7feb200cace0 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7feb200cf3d0 con 0x7feb20156150 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 0x7feb20154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 0x7feb20154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33284/0 (socket says 172.21.15.97:33284) 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 -- 172.21.15.97:0/2225355703 learned_addr learned my addr 172.21.15.97:0/2225355703 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb1fe00640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb20156150 0x7feb200b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb25400640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb200cace0 0x7feb200cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 -- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb20156150 msgr2=0x7feb200b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb20156150 0x7feb200b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 -- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb200cace0 msgr2=0x7feb200cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb200cace0 0x7feb200cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 -- 172.21.15.97:0/2225355703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb20156650 con 0x7feb04004d50 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb24a00640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 0x7feb20154fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb10003ef0 tx=0x7feb10001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb1cc00640 1 -- 172.21.15.97:0/2225355703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10011640 con 0x7feb04004d50 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb1cc00640 1 -- 172.21.15.97:0/2225355703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb10016020 con 0x7feb04004d50 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb1cc00640 1 -- 172.21.15.97:0/2225355703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10011a80 con 0x7feb04004d50 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.696+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb20156930 con 0x7feb04004d50 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.700+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201554e0 con 0x7feb04004d50 2024-09-10T21:24:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.700+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb200cfb80 con 0x7feb04004d50 2024-09-10T21:24:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.700+0000 7feb1cc00640 1 -- 172.21.15.97:0/2225355703 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feb10011c20 con 0x7feb04004d50 2024-09-10T21:24:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.700+0000 7feb1cc00640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb000725e0 0x7feb00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.700+0000 7feb1cc00640 1 -- 172.21.15.97:0/2225355703 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7feb10093c50 con 0x7feb04004d50 2024-09-10T21:24:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.700+0000 7feb1fe00640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb000725e0 0x7feb00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.700+0000 7feb1fe00640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb000725e0 0x7feb00074aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feb0c001db0 tx=0x7feb0c001d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.704+0000 7feb1cc00640 1 -- 172.21.15.97:0/2225355703 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feb10020020 con 0x7feb04004d50 2024-09-10T21:24:08.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.904+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb200d0060 con 0x7feb04004d50 2024-09-10T21:24:08.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.904+0000 7feb1cc00640 1 -- 172.21.15.97:0/2225355703 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3912 (secure 0 0 0) 0x7feb1005f670 con 0x7feb04004d50 2024-09-10T21:24:08.962 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:08.963 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"btime":"2024-09-10T21:24:07:260784+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T21:24:08.964 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-09-10T21:24:08.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb000725e0 msgr2=0x7feb00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:08.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb000725e0 0x7feb00074aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feb0c001db0 tx=0x7feb0c001d00 comp rx=0 tx=0).stop 2024-09-10T21:24:08.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 msgr2=0x7feb20154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:08.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 0x7feb20154fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb10003ef0 tx=0x7feb10001640 comp rx=0 tx=0).stop 2024-09-10T21:24:08.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 shutdown_connections 2024-09-10T21:24:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb000725e0 0x7feb00074aa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb200cace0 0x7feb200cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb20156150 0x7feb200b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 --2- 172.21.15.97:0/2225355703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb04004d50 0x7feb20154fa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 >> 172.21.15.97:0/2225355703 conn(0x7feb200d1ca0 msgr2=0x7feb20164130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:08.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 shutdown_connections 2024-09-10T21:24:08.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:08.908+0000 7feb27400640 1 -- 172.21.15.97:0/2225355703 wait complete. 2024-09-10T21:24:08.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:24:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 -- 172.21.15.97:0/4035613287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c06ec40 msgr2=0x7f515c06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 --2- 172.21.15.97:0/4035613287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c06ec40 0x7f515c06f090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5144009900 tx=0x7f514401f090 comp rx=0 tx=0).stop 2024-09-10T21:24:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 -- 172.21.15.97:0/4035613287 shutdown_connections 2024-09-10T21:24:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 --2- 172.21.15.97:0/4035613287 >> v2:172.21.15.194:3300/0 conn(0x7f515c0ceee0 0x7f515c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 --2- 172.21.15.97:0/4035613287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c06ec40 0x7f515c06f090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 -- 172.21.15.97:0/4035613287 >> 172.21.15.97:0/4035613287 conn(0x7f515c0b1780 msgr2=0x7f515c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 -- 172.21.15.97:0/4035613287 shutdown_connections 2024-09-10T21:24:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 -- 172.21.15.97:0/4035613287 wait complete. 2024-09-10T21:24:09.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 Processor -- start 2024-09-10T21:24:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 -- start start 2024-09-10T21:24:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.100+0000 7f5162e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f515c06ec40 0x7f515c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f515c0ceee0 0x7f515c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c1511f0 0x7f515c154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f515c06e740 con 0x7f515c1511f0 2024-09-10T21:24:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f515c06e5c0 con 0x7f515c0ceee0 2024-09-10T21:24:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f515c06e8c0 con 0x7f515c06ec40 2024-09-10T21:24:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c1511f0 0x7f515c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c1511f0 0x7f515c154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33302/0 (socket says 172.21.15.97:33302) 2024-09-10T21:24:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 -- 172.21.15.97:0/2540272624 learned_addr learned my addr 172.21.15.97:0/2540272624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f515be00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f515c06ec40 0x7f515c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5153400640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f515c0ceee0 0x7f515c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 -- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f515c06ec40 msgr2=0x7f515c149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f515c06ec40 0x7f515c149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 -- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f515c0ceee0 msgr2=0x7f515c158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f515c0ceee0 0x7f515c158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 -- 172.21.15.97:0/2540272624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c149c50 con 0x7f515c1511f0 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5160e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c1511f0 0x7f515c154d60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f515400b970 tx=0x7f515400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5158c00640 1 -- 172.21.15.97:0/2540272624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154019070 con 0x7f515c1511f0 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5158c00640 1 -- 172.21.15.97:0/2540272624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f515400cc50 con 0x7f515c1511f0 2024-09-10T21:24:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5158c00640 1 -- 172.21.15.97:0/2540272624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540146a0 con 0x7f515c1511f0 2024-09-10T21:24:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c14a020 con 0x7f515c1511f0 2024-09-10T21:24:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c159190 con 0x7f515c1511f0 2024-09-10T21:24:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c158930 con 0x7f515c1511f0 2024-09-10T21:24:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.104+0000 7f5158c00640 1 -- 172.21.15.97:0/2540272624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f515401d3b0 con 0x7f515c1511f0 2024-09-10T21:24:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.108+0000 7f5158c00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51380725e0 0x7f5138074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.108+0000 7f5158c00640 1 -- 172.21.15.97:0/2540272624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f5154092e40 con 0x7f515c1511f0 2024-09-10T21:24:09.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.108+0000 7f515be00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51380725e0 0x7f5138074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.108+0000 7f515be00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51380725e0 0x7f5138074aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098e0 tx=0x7f514c009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:09.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.112+0000 7f5158c00640 1 -- 172.21.15.97:0/2540272624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f515c158930 con 0x7f515c1511f0 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f515c150730 con 0x7f515c1511f0 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5158c00640 1 -- 172.21.15.97:0/2540272624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f515405e210 con 0x7f515c1511f0 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:24:09.283 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:24:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51380725e0 msgr2=0x7f5138074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51380725e0 0x7f5138074aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098e0 tx=0x7f514c009060 comp rx=0 tx=0).stop 2024-09-10T21:24:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c1511f0 msgr2=0x7f515c154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c1511f0 0x7f515c154d60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f515400b970 tx=0x7f515400be30 comp rx=0 tx=0).stop 2024-09-10T21:24:09.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 shutdown_connections 2024-09-10T21:24:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51380725e0 0x7f5138074aa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f515c1511f0 0x7f515c154d60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f515c0ceee0 0x7f515c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 --2- 172.21.15.97:0/2540272624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f515c06ec40 0x7f515c149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 >> 172.21.15.97:0/2540272624 conn(0x7f515c0b1780 msgr2=0x7f515c15cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 shutdown_connections 2024-09-10T21:24:09.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.276+0000 7f5162e00640 1 -- 172.21.15.97:0/2540272624 wait complete. 2024-09-10T21:24:09.304 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:24:09.304 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:24:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.444+0000 7fe4be200640 1 -- 172.21.15.97:0/367196330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 msgr2=0x7fe49c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.444+0000 7fe4be200640 1 --2- 172.21.15.97:0/367196330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 0x7fe49c004c90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4b815e650 tx=0x7fe4a401f700 comp rx=0 tx=0).stop 2024-09-10T21:24:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.444+0000 7fe4b7e00640 1 -- 172.21.15.97:0/367196330 reap_dead start 2024-09-10T21:24:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.444+0000 7fe4be200640 1 -- 172.21.15.97:0/367196330 shutdown_connections 2024-09-10T21:24:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.444+0000 7fe4be200640 1 -- 172.21.15.97:0/367196330 >> 172.21.15.97:0/367196330 conn(0x7fe4b80b1780 msgr2=0x7fe4b80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.444+0000 7fe4be200640 1 -- 172.21.15.97:0/367196330 shutdown_connections 2024-09-10T21:24:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.444+0000 7fe4be200640 1 -- 172.21.15.97:0/367196330 wait complete. 2024-09-10T21:24:09.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 Processor -- start 2024-09-10T21:24:09.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 -- start start 2024-09-10T21:24:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 0x7fe4b80cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4b815a680 0x7fe4b80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4b80addf0 0x7fe4b8138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80b49b0 con 0x7fe49c004810 2024-09-10T21:24:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80b4830 con 0x7fe4b815a680 2024-09-10T21:24:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b80b2a80 con 0x7fe4b80addf0 2024-09-10T21:24:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 0x7fe4b80cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 0x7fe4b80cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33312/0 (socket says 172.21.15.97:33312) 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 -- 172.21.15.97:0/3709290741 learned_addr learned my addr 172.21.15.97:0/3709290741 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b6a00640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4b815a680 0x7fe4b80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7e00640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4b80addf0 0x7fe4b8138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 -- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4b80addf0 msgr2=0x7fe4b8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4b80addf0 0x7fe4b8138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 -- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4b815a680 msgr2=0x7fe4b80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4b815a680 0x7fe4b80ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 -- 172.21.15.97:0/3709290741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81393c0 con 0x7fe49c004810 2024-09-10T21:24:09.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4b7400640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 0x7fe4b80cced0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8011c80 tx=0x7fe4a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:09.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4af400640 1 -- 172.21.15.97:0/3709290741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8003d90 con 0x7fe49c004810 2024-09-10T21:24:09.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4af400640 1 -- 172.21.15.97:0/3709290741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4a8004050 con 0x7fe49c004810 2024-09-10T21:24:09.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4af400640 1 -- 172.21.15.97:0/3709290741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8004480 con 0x7fe49c004810 2024-09-10T21:24:09.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b80afe60 con 0x7fe49c004810 2024-09-10T21:24:09.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b80b0420 con 0x7fe49c004810 2024-09-10T21:24:09.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.448+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b80ae200 con 0x7fe49c004810 2024-09-10T21:24:09.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.452+0000 7fe4af400640 1 -- 172.21.15.97:0/3709290741 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe4a8018070 con 0x7fe49c004810 2024-09-10T21:24:09.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.452+0000 7fe4af400640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4980725e0 0x7fe498074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.452+0000 7fe4af400640 1 -- 172.21.15.97:0/3709290741 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fe4a80941b0 con 0x7fe49c004810 2024-09-10T21:24:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.452+0000 7fe4b6a00640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4980725e0 0x7fe498074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.452+0000 7fe4b6a00640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4980725e0 0x7fe498074aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40b0000 tx=0x7fe4a401c540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.456+0000 7fe4af400640 1 -- 172.21.15.97:0/3709290741 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe4b80ae200 con 0x7fe49c004810 2024-09-10T21:24:09.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.624+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe4b80701c0 con 0x7fe49c004810 2024-09-10T21:24:09.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.672+0000 7fe4af400640 1 -- 172.21.15.97:0/3709290741 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fe4a805fb40 con 0x7fe49c004810 2024-09-10T21:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-09-10T21:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4980725e0 msgr2=0x7fe498074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4980725e0 0x7fe498074aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40b0000 tx=0x7fe4a401c540 comp rx=0 tx=0).stop 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 msgr2=0x7fe4b80cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 0x7fe4b80cced0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8011c80 tx=0x7fe4a800f790 comp rx=0 tx=0).stop 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 shutdown_connections 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4980725e0 0x7fe498074aa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4b80addf0 0x7fe4b8138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4b815a680 0x7fe4b80ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 --2- 172.21.15.97:0/3709290741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49c004810 0x7fe4b80cced0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 >> 172.21.15.97:0/3709290741 conn(0x7fe4b80b1780 msgr2=0x7fe4b81414e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 shutdown_connections 2024-09-10T21:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.676+0000 7fe4be200640 1 -- 172.21.15.97:0/3709290741 wait complete. 2024-09-10T21:24:09.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 -- 172.21.15.97:0/1036241215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440af6a0 msgr2=0x7f85440ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 --2- 172.21.15.97:0/1036241215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440af6a0 0x7f85440ccbb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8530009900 tx=0x7f853001c850 comp rx=0 tx=0).stop 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 -- 172.21.15.97:0/1036241215 shutdown_connections 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 --2- 172.21.15.97:0/1036241215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440af6a0 0x7f85440ccbb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 --2- 172.21.15.97:0/1036241215 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85440b5dc0 0x7f85440691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 -- 172.21.15.97:0/1036241215 >> 172.21.15.97:0/1036241215 conn(0x7f8544074160 msgr2=0x7f8544074570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 -- 172.21.15.97:0/1036241215 shutdown_connections 2024-09-10T21:24:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.848+0000 7f854c000640 1 -- 172.21.15.97:0/1036241215 wait complete. 2024-09-10T21:24:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 Processor -- start 2024-09-10T21:24:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 -- start start 2024-09-10T21:24:09.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85440af6a0 0x7f85440c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85440afac0 0x7f854406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440b5dc0 0x7f854406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8544167e50 con 0x7f85440b5dc0 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8544167cd0 con 0x7f85440af6a0 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8544167fd0 con 0x7f85440afac0 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440b5dc0 0x7f854406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440b5dc0 0x7f854406fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33332/0 (socket says 172.21.15.97:33332) 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 -- 172.21.15.97:0/1284658579 learned_addr learned my addr 172.21.15.97:0/1284658579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f8549600640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85440af6a0 0x7f85440c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f8548c00640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85440afac0 0x7f854406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 -- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85440afac0 msgr2=0x7f854406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85440afac0 0x7f854406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 -- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85440af6a0 msgr2=0x7f85440c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85440af6a0 0x7f85440c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 -- 172.21.15.97:0/1284658579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85441ff900 con 0x7f85440b5dc0 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f8548c00640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85440afac0 0x7f854406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854a000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440b5dc0 0x7f854406fcd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c7c0 tx=0x7f853c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f8541600640 1 -- 172.21.15.97:0/1284658579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c003b80 con 0x7f85440b5dc0 2024-09-10T21:24:09.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f8541600640 1 -- 172.21.15.97:0/1284658579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f853c003d20 con 0x7f85440b5dc0 2024-09-10T21:24:09.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85441ffbe0 con 0x7f85440b5dc0 2024-09-10T21:24:09.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f8541600640 1 -- 172.21.15.97:0/1284658579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c00aab0 con 0x7f85440b5dc0 2024-09-10T21:24:09.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.852+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85442001a0 con 0x7f85440b5dc0 2024-09-10T21:24:09.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.856+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85440691b0 con 0x7f85440b5dc0 2024-09-10T21:24:09.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.856+0000 7f8541600640 1 -- 172.21.15.97:0/1284658579 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f853c00ac50 con 0x7f85440b5dc0 2024-09-10T21:24:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.856+0000 7f8541600640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85240725e0 0x7f8524074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.856+0000 7f8541600640 1 -- 172.21.15.97:0/1284658579 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f853c093750 con 0x7f85440b5dc0 2024-09-10T21:24:09.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.860+0000 7f8549600640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85240725e0 0x7f8524074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:09.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.860+0000 7f8549600640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85240725e0 0x7f8524074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85440ca720 tx=0x7f85380091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:09.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:09.860+0000 7f8541600640 1 -- 172.21.15.97:0/1284658579 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f853c05eb30 con 0x7f85440b5dc0 2024-09-10T21:24:10.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.032+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f854406cd80 con 0x7f85440b5dc0 2024-09-10T21:24:10.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.676+0000 7f8541600640 1 -- 172.21.15.97:0/1284658579 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7f853c063560 con 0x7f85440b5dc0 2024-09-10T21:24:10.684 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-09-10T21:24:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85240725e0 msgr2=0x7f8524074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85240725e0 0x7f8524074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85440ca720 tx=0x7f85380091e0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440b5dc0 msgr2=0x7f854406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440b5dc0 0x7f854406fcd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c7c0 tx=0x7f853c00cc80 comp rx=0 tx=0).stop 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 shutdown_connections 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85240725e0 0x7f8524074aa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85440b5dc0 0x7f854406fcd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85440afac0 0x7f854406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 --2- 172.21.15.97:0/1284658579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85440af6a0 0x7f85440c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 >> 172.21.15.97:0/1284658579 conn(0x7f8544074160 msgr2=0x7f854406ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 shutdown_connections 2024-09-10T21:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.680+0000 7f854c000640 1 -- 172.21.15.97:0/1284658579 wait complete. 2024-09-10T21:24:10.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:24:10.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- 172.21.15.97:0/3244611408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540cdcb0 msgr2=0x7f62540cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:10.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 --2- 172.21.15.97:0/3244611408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540cdcb0 0x7f62540cff50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6240009900 tx=0x7f624001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- 172.21.15.97:0/3244611408 shutdown_connections 2024-09-10T21:24:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 --2- 172.21.15.97:0/3244611408 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62540d0490 0x7f62540ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 --2- 172.21.15.97:0/3244611408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540cdcb0 0x7f62540cff50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- 172.21.15.97:0/3244611408 >> 172.21.15.97:0/3244611408 conn(0x7f62540d1a60 msgr2=0x7f62540d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- 172.21.15.97:0/3244611408 shutdown_connections 2024-09-10T21:24:10.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- 172.21.15.97:0/3244611408 wait complete. 2024-09-10T21:24:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 Processor -- start 2024-09-10T21:24:10.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- start start 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62540cdcb0 0x7f625415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540d0490 0x7f6254158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f625415efc0 0x7f6254164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6254168930 con 0x7f62540d0490 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62541687b0 con 0x7f625415efc0 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6253e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6254168ab0 con 0x7f62540cdcb0 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6252000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540d0490 0x7f6254158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.864+0000 7f6252000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540d0490 0x7f6254158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33352/0 (socket says 172.21.15.97:33352) 2024-09-10T21:24:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252000640 1 -- 172.21.15.97:0/1634333625 learned_addr learned my addr 172.21.15.97:0/1634333625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252a00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62540cdcb0 0x7f625415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6253400640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f625415efc0 0x7f6254164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252000640 1 -- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62540cdcb0 msgr2=0x7f625415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252000640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62540cdcb0 0x7f625415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252000640 1 -- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f625415efc0 msgr2=0x7f6254164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252000640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f625415efc0 0x7f6254164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252000640 1 -- 172.21.15.97:0/1634333625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254159840 con 0x7f62540d0490 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6252000640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540d0490 0x7f6254158720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6240009850 tx=0x7f6240004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f624aa00640 1 -- 172.21.15.97:0/1634333625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62400b2040 con 0x7f62540d0490 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f624aa00640 1 -- 172.21.15.97:0/1634333625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6240005070 con 0x7f62540d0490 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6254159de0 con 0x7f62540d0490 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f624aa00640 1 -- 172.21.15.97:0/1634333625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240005480 con 0x7f62540d0490 2024-09-10T21:24:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625415a3a0 con 0x7f62540d0490 2024-09-10T21:24:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62540ae930 con 0x7f62540d0490 2024-09-10T21:24:10.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.868+0000 7f624aa00640 1 -- 172.21.15.97:0/1634333625 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f62400046d0 con 0x7f62540d0490 2024-09-10T21:24:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.872+0000 7f624aa00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62340725e0 0x7f6234074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.872+0000 7f624aa00640 1 -- 172.21.15.97:0/1634333625 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f624012c520 con 0x7f62540d0490 2024-09-10T21:24:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.872+0000 7f6252a00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62340725e0 0x7f6234074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.872+0000 7f6252a00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62340725e0 0x7f6234074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6244001e40 tx=0x7f6244009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:10.876+0000 7f624aa00640 1 -- 172.21.15.97:0/1634333625 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f62400bc060 con 0x7f62540d0490 2024-09-10T21:24:11.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.076+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f625415ba30 con 0x7f62540d0490 2024-09-10T21:24:11.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.708+0000 7f624aa00640 1 -- 172.21.15.97:0/1634333625 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f62400164d0 con 0x7f62540d0490 2024-09-10T21:24:11.714 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T21:24:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62340725e0 msgr2=0x7f6234074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62340725e0 0x7f6234074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6244001e40 tx=0x7f6244009040 comp rx=0 tx=0).stop 2024-09-10T21:24:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540d0490 msgr2=0x7f6254158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540d0490 0x7f6254158720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6240009850 tx=0x7f6240004040 comp rx=0 tx=0).stop 2024-09-10T21:24:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 shutdown_connections 2024-09-10T21:24:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62340725e0 0x7f6234074aa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f625415efc0 0x7f6254164fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62540d0490 0x7f6254158720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 --2- 172.21.15.97:0/1634333625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62540cdcb0 0x7f625415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 >> 172.21.15.97:0/1634333625 conn(0x7f62540d1a60 msgr2=0x7f625415bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:11.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 shutdown_connections 2024-09-10T21:24:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.712+0000 7f6253e00640 1 -- 172.21.15.97:0/1634333625 wait complete. 2024-09-10T21:24:11.776 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T21:24:11.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T21:24:11.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- 172.21.15.97:0/207552379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80d0710 msgr2=0x7f61e80ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 --2- 172.21.15.97:0/207552379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80d0710 0x7f61e80ce9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61e80d1c70 tx=0x7f61d001c800 comp rx=0 tx=0).stop 2024-09-10T21:24:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- 172.21.15.97:0/207552379 shutdown_connections 2024-09-10T21:24:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 --2- 172.21.15.97:0/207552379 >> v2:172.21.15.194:3300/0 conn(0x7f61e80cde30 0x7f61e80cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 --2- 172.21.15.97:0/207552379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80d0710 0x7f61e80ce9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- 172.21.15.97:0/207552379 >> 172.21.15.97:0/207552379 conn(0x7f61e80d1ca0 msgr2=0x7f61e80d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- 172.21.15.97:0/207552379 shutdown_connections 2024-09-10T21:24:11.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- 172.21.15.97:0/207552379 wait complete. 2024-09-10T21:24:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 Processor -- start 2024-09-10T21:24:11.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- start start 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61e80cde30 0x7f61e806df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80ce5e0 0x7f61e80b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61e80d0710 0x7f61e809ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80cd350 con 0x7f61e80ce5e0 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80cd1d0 con 0x7f61e80cde30 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61ed600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f61e8169060 con 0x7f61e80d0710 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80ce5e0 0x7f61e80b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80ce5e0 0x7f61e80b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33366/0 (socket says 172.21.15.97:33366) 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.916+0000 7f61dfe00640 1 -- 172.21.15.97:0/1348976510 learned_addr learned my addr 172.21.15.97:0/1348976510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61e6a00640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61e80cde30 0x7f61e806df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61e7400640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61e80d0710 0x7f61e809ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dfe00640 1 -- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61e80d0710 msgr2=0x7f61e809ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dfe00640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61e80d0710 0x7f61e809ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dfe00640 1 -- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61e80cde30 msgr2=0x7f61e806df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dfe00640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61e80cde30 0x7f61e806df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dfe00640 1 -- 172.21.15.97:0/1348976510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e80b5fd0 con 0x7f61e80ce5e0 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dfe00640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80ce5e0 0x7f61e80b7490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61d0000c10 tx=0x7f61d00a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dea00640 1 -- 172.21.15.97:0/1348976510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00bb070 con 0x7f61e80ce5e0 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dea00640 1 -- 172.21.15.97:0/1348976510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61d0004470 con 0x7f61e80ce5e0 2024-09-10T21:24:11.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dea00640 1 -- 172.21.15.97:0/1348976510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00b2040 con 0x7f61e80ce5e0 2024-09-10T21:24:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e80b62b0 con 0x7f61e80ce5e0 2024-09-10T21:24:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e80b6870 con 0x7f61e80ce5e0 2024-09-10T21:24:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e80b2010 con 0x7f61e80ce5e0 2024-09-10T21:24:11.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dea00640 1 -- 172.21.15.97:0/1348976510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f61d00b5030 con 0x7f61e80ce5e0 2024-09-10T21:24:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dea00640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61c40725e0 0x7f61c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.920+0000 7f61dea00640 1 -- 172.21.15.97:0/1348976510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f61d012c990 con 0x7f61e80ce5e0 2024-09-10T21:24:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.924+0000 7f61e6a00640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61c40725e0 0x7f61c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.924+0000 7f61e6a00640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61c40725e0 0x7f61c4074aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61d8007fd0 tx=0x7f61d8007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:11.928+0000 7f61dea00640 1 -- 172.21.15.97:0/1348976510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f61d00f2de0 con 0x7f61e80ce5e0 2024-09-10T21:24:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.100+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f61e80d0060 con 0x7f61e80ce5e0 2024-09-10T21:24:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.708+0000 7f61dea00640 1 -- 172.21.15.97:0/1348976510 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f61d00163f0 con 0x7f61e80ce5e0 2024-09-10T21:24:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61c40725e0 msgr2=0x7f61c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61c40725e0 0x7f61c4074aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61d8007fd0 tx=0x7f61d8007ea0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80ce5e0 msgr2=0x7f61e80b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80ce5e0 0x7f61e80b7490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61d0000c10 tx=0x7f61d00a5660 comp rx=0 tx=0).stop 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 shutdown_connections 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61c40725e0 0x7f61c4074aa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61e80d0710 0x7f61e809ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61e80ce5e0 0x7f61e80b7490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 --2- 172.21.15.97:0/1348976510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61e80cde30 0x7f61e806df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 >> 172.21.15.97:0/1348976510 conn(0x7f61e80d1ca0 msgr2=0x7f61e80b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 shutdown_connections 2024-09-10T21:24:12.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.712+0000 7f61ed600640 1 -- 172.21.15.97:0/1348976510 wait complete. 2024-09-10T21:24:12.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:24:12.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 -- 172.21.15.97:0/2504695582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 msgr2=0x7f71ec13f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:12.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 --2- 172.21.15.97:0/2504695582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 0x7f71ec13f6f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009900 tx=0x7f71dc01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 -- 172.21.15.97:0/2504695582 shutdown_connections 2024-09-10T21:24:12.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 --2- 172.21.15.97:0/2504695582 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ec0ca090 0x7f71ec0ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 --2- 172.21.15.97:0/2504695582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 0x7f71ec13f6f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 -- 172.21.15.97:0/2504695582 >> 172.21.15.97:0/2504695582 conn(0x7f71ec0cfa40 msgr2=0x7f71ec0cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:12.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 -- 172.21.15.97:0/2504695582 shutdown_connections 2024-09-10T21:24:12.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.884+0000 7f71f2e00640 1 -- 172.21.15.97:0/2504695582 wait complete. 2024-09-10T21:24:12.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 Processor -- start 2024-09-10T21:24:12.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 -- start start 2024-09-10T21:24:12.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec0ca090 0x7f71ec13a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:12.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 0x7f71ec13a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ec13fc30 0x7f71ec1f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec0cf200 con 0x7f71ec0ca090 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec0cf080 con 0x7f71ec13cdb0 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec0cf380 con 0x7f71ec13fc30 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec0ca090 0x7f71ec13a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec0ca090 0x7f71ec13a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33378/0 (socket says 172.21.15.97:33378) 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 -- 172.21.15.97:0/1113400407 learned_addr learned my addr 172.21.15.97:0/1113400407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71e3400640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 0x7f71ec13a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f0e00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ec13fc30 0x7f71ec1f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 -- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ec13fc30 msgr2=0x7f71ec1f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ec13fc30 0x7f71ec1f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 -- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 msgr2=0x7f71ec13a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:12.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 0x7f71ec13a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:12.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 -- 172.21.15.97:0/1113400407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec070f20 con 0x7f71ec0ca090 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71ebe00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec0ca090 0x7f71ec13a650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009850 tx=0x7f71dc0a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71e8c00640 1 -- 172.21.15.97:0/1113400407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0b2040 con 0x7f71ec0ca090 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71e8c00640 1 -- 172.21.15.97:0/1113400407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71dc004380 con 0x7f71ec0ca090 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec0b4540 con 0x7f71ec0ca090 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71e8c00640 1 -- 172.21.15.97:0/1113400407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc005290 con 0x7f71ec0ca090 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.888+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec0b4b00 con 0x7f71ec0ca090 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.892+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec0ca4d0 con 0x7f71ec0ca090 2024-09-10T21:24:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.892+0000 7f71e8c00640 1 -- 172.21.15.97:0/1113400407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71dc005430 con 0x7f71ec0ca090 2024-09-10T21:24:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.892+0000 7f71e8c00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c80725e0 0x7f71c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.892+0000 7f71e8c00640 1 -- 172.21.15.97:0/1113400407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f71dc12cd90 con 0x7f71ec0ca090 2024-09-10T21:24:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.892+0000 7f71e3400640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c80725e0 0x7f71c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:12.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.892+0000 7f71e3400640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c80725e0 0x7f71c8074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0b16e0 tx=0x7f71d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:12.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:12.896+0000 7f71e8c00640 1 -- 172.21.15.97:0/1113400407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71dc0164d0 con 0x7f71ec0ca090 2024-09-10T21:24:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:13.060+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f71ec0d08a0 con 0x7f71ec0ca090 2024-09-10T21:24:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.740+0000 7f71e8c00640 1 -- 172.21.15.97:0/1113400407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f71dc0f83d0 con 0x7f71ec0ca090 2024-09-10T21:24:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' created 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c80725e0 msgr2=0x7f71c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c80725e0 0x7f71c8074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0b16e0 tx=0x7f71d4009040 comp rx=0 tx=0).stop 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec0ca090 msgr2=0x7f71ec13a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec0ca090 0x7f71ec13a650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009850 tx=0x7f71dc0a55a0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 shutdown_connections 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c80725e0 0x7f71c8074aa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71ec13fc30 0x7f71ec1f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71ec13cdb0 0x7f71ec13a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 --2- 172.21.15.97:0/1113400407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec0ca090 0x7f71ec13a650 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 >> 172.21.15.97:0/1113400407 conn(0x7f71ec0cfa40 msgr2=0x7f71ec0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:14.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 shutdown_connections 2024-09-10T21:24:14.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.744+0000 7f71f2e00640 1 -- 172.21.15.97:0/1113400407 wait complete. 2024-09-10T21:24:14.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T21:24:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 -- 172.21.15.97:0/2049861821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 msgr2=0x7f6f740d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/2049861821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f740d27f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009900 tx=0x7f6f6801c8b0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 -- 172.21.15.97:0/2049861821 shutdown_connections 2024-09-10T21:24:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/2049861821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f740ccc60 0x7f6f740b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/2049861821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f740d27f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 -- 172.21.15.97:0/2049861821 >> 172.21.15.97:0/2049861821 conn(0x7f6f740d1cd0 msgr2=0x7f6f740d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 -- 172.21.15.97:0/2049861821 shutdown_connections 2024-09-10T21:24:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.920+0000 7f6f7bc00640 1 -- 172.21.15.97:0/2049861821 wait complete. 2024-09-10T21:24:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 Processor -- start 2024-09-10T21:24:14.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 -- start start 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f740ccc60 0x7f6f7414b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f740cdd40 0x7f6f74154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f741548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74168880 con 0x7f6f7415eed0 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74168700 con 0x7f6f740cdd40 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6f74168a00 con 0x7f6f740ccc60 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f741548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f741548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33404/0 (socket says 172.21.15.97:33404) 2024-09-10T21:24:14.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 -- 172.21.15.97:0/859732174 learned_addr learned my addr 172.21.15.97:0/859732174 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f73e00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f740cdd40 0x7f6f74154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79200640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f740ccc60 0x7f6f7414b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:14.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 -- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f740ccc60 msgr2=0x7f6f7414b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f740ccc60 0x7f6f7414b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 -- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f740cdd40 msgr2=0x7f6f74154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f740cdd40 0x7f6f74154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 -- 172.21.15.97:0/859732174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f740b1de0 con 0x7f6f7415eed0 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f79c00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f741548d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00e9f0 tx=0x7f6f6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f70c00640 1 -- 172.21.15.97:0/859732174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00cd80 con 0x7f6f7415eed0 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f70c00640 1 -- 172.21.15.97:0/859732174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f6c004c20 con 0x7f6f7415eed0 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f740b6a70 con 0x7f6f7415eed0 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f70c00640 1 -- 172.21.15.97:0/859732174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c005020 con 0x7f6f7415eed0 2024-09-10T21:24:14.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f740b6d10 con 0x7f6f7415eed0 2024-09-10T21:24:14.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.924+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7414bb60 con 0x7f6f7415eed0 2024-09-10T21:24:14.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.928+0000 7f6f70c00640 1 -- 172.21.15.97:0/859732174 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f6c00f040 con 0x7f6f7415eed0 2024-09-10T21:24:14.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.928+0000 7f6f70c00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f54072610 0x7f6f54074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:14.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.928+0000 7f6f70c00640 1 -- 172.21.15.97:0/859732174 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f6f6c093c70 con 0x7f6f7415eed0 2024-09-10T21:24:14.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.928+0000 7f6f79200640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f54072610 0x7f6f54074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:14.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.928+0000 7f6f79200640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f54072610 0x7f6f54074ad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f680af000 tx=0x7f6f6801b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:14.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:14.932+0000 7f6f70c00640 1 -- 172.21.15.97:0/859732174 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f740b1de0 con 0x7f6f7415eed0 2024-09-10T21:24:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.100+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f6f740b0470 con 0x7f6f7415eed0 2024-09-10T21:24:15.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.744+0000 7f6f70c00640 1 -- 172.21.15.97:0/859732174 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7f6f6c016030 con 0x7f6f7415eed0 2024-09-10T21:24:15.751 INFO:teuthology.orchestra.run.smithi097.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T21:24:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.744+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f54072610 msgr2=0x7f6f54074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.744+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f54072610 0x7f6f54074ad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f680af000 tx=0x7f6f6801b0f0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.744+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 msgr2=0x7f6f741548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.744+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f741548d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00e9f0 tx=0x7f6f6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 shutdown_connections 2024-09-10T21:24:15.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f54072610 0x7f6f54074ad0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f7415eed0 0x7f6f741548d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f740cdd40 0x7f6f74154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 --2- 172.21.15.97:0/859732174 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f740ccc60 0x7f6f7414b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 >> 172.21.15.97:0/859732174 conn(0x7f6f740d1cd0 msgr2=0x7f6f740cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:15.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 shutdown_connections 2024-09-10T21:24:15.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.748+0000 7f6f7bc00640 1 -- 172.21.15.97:0/859732174 wait complete. 2024-09-10T21:24:15.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T21:24:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.936+0000 7f8173600640 1 -- 172.21.15.97:0/271503152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 msgr2=0x7f8150004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.936+0000 7f8173600640 1 --2- 172.21.15.97:0/271503152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 0x7f8150004c90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8154001860 tx=0x7f815401c8b0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.936+0000 7f8171600640 1 -- 172.21.15.97:0/271503152 reap_dead start 2024-09-10T21:24:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.936+0000 7f8173600640 1 -- 172.21.15.97:0/271503152 shutdown_connections 2024-09-10T21:24:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.936+0000 7f8173600640 1 -- 172.21.15.97:0/271503152 >> 172.21.15.97:0/271503152 conn(0x7f816c0d1c60 msgr2=0x7f816c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.936+0000 7f8173600640 1 -- 172.21.15.97:0/271503152 shutdown_connections 2024-09-10T21:24:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.936+0000 7f8173600640 1 -- 172.21.15.97:0/271503152 wait complete. 2024-09-10T21:24:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 Processor -- start 2024-09-10T21:24:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 -- start start 2024-09-10T21:24:15.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 0x7f816c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:15.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f816c15bdb0 0x7f816c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f816c15d0a0 0x7f816c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f816c167a20 con 0x7f8150004810 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0b0eb0 con 0x7f816c15bdb0 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f816c167ba0 con 0x7f816c15d0a0 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 0x7f816c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 0x7f816c0b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33414/0 (socket says 172.21.15.97:33414) 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 -- 172.21.15.97:0/374095512 learned_addr learned my addr 172.21.15.97:0/374095512 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8163e00640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f816c15bdb0 0x7f816c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 -- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f816c15d0a0 msgr2=0x7f816c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8171600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f816c15d0a0 0x7f816c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f816c15d0a0 0x7f816c15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 -- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f816c15bdb0 msgr2=0x7f816c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f816c15bdb0 0x7f816c1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 -- 172.21.15.97:0/374095512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c0b5220 con 0x7f8150004810 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8163e00640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f816c15bdb0 0x7f816c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8171600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f816c15d0a0 0x7f816c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8170c00640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 0x7f816c0b4ab0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f815c006630 tx=0x7f815c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8169600640 1 -- 172.21.15.97:0/374095512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c015070 con 0x7f8150004810 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8169600640 1 -- 172.21.15.97:0/374095512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f815c011920 con 0x7f8150004810 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8169600640 1 -- 172.21.15.97:0/374095512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c011d70 con 0x7f8150004810 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c0b5500 con 0x7f8150004810 2024-09-10T21:24:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.940+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c0b5ac0 con 0x7f8150004810 2024-09-10T21:24:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.944+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c0afc40 con 0x7f8150004810 2024-09-10T21:24:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.944+0000 7f8169600640 1 -- 172.21.15.97:0/374095512 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f815c0103c0 con 0x7f8150004810 2024-09-10T21:24:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.944+0000 7f8169600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8148072610 0x7f8148074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.944+0000 7f8169600640 1 -- 172.21.15.97:0/374095512 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f815c094630 con 0x7f8150004810 2024-09-10T21:24:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.944+0000 7f8163e00640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8148072610 0x7f8148074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.948+0000 7f8163e00640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8148072610 0x7f8148074ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8154001830 tx=0x7f81540a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:15.948+0000 7f8169600640 1 -- 172.21.15.97:0/374095512 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f815c05f9e0 con 0x7f8150004810 2024-09-10T21:24:16.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.148+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f816c0b13a0 con 0x7f8150004810 2024-09-10T21:24:16.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.764+0000 7f8169600640 1 -- 172.21.15.97:0/374095512 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f815c05f2e0 con 0x7f8150004810 2024-09-10T21:24:16.772 INFO:teuthology.orchestra.run.smithi097.stderr:added data pool 4 to fsmap 2024-09-10T21:24:16.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8148072610 msgr2=0x7f8148074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:16.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8148072610 0x7f8148074ad0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8154001830 tx=0x7f81540a7040 comp rx=0 tx=0).stop 2024-09-10T21:24:16.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 msgr2=0x7f816c0b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:16.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 0x7f816c0b4ab0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f815c006630 tx=0x7f815c0039d0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 shutdown_connections 2024-09-10T21:24:16.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8148072610 0x7f8148074ad0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f816c15d0a0 0x7f816c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f816c15bdb0 0x7f816c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 --2- 172.21.15.97:0/374095512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8150004810 0x7f816c0b4ab0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 >> 172.21.15.97:0/374095512 conn(0x7f816c0d1c60 msgr2=0x7f816c0c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:16.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 shutdown_connections 2024-09-10T21:24:16.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.768+0000 7f8173600640 1 -- 172.21.15.97:0/374095512 wait complete. 2024-09-10T21:24:16.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- 172.21.15.97:0/3998013663 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 msgr2=0x7f10c4003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 --2- 172.21.15.97:0/3998013663 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 0x7f10c4003c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009b60 tx=0x7f10d001d2b0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e5000640 1 -- 172.21.15.97:0/3998013663 reap_dead start 2024-09-10T21:24:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- 172.21.15.97:0/3998013663 shutdown_connections 2024-09-10T21:24:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- 172.21.15.97:0/3998013663 >> 172.21.15.97:0/3998013663 conn(0x7f10e00d1ca0 msgr2=0x7f10e00d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- 172.21.15.97:0/3998013663 shutdown_connections 2024-09-10T21:24:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- 172.21.15.97:0/3998013663 wait complete. 2024-09-10T21:24:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 Processor -- start 2024-09-10T21:24:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- start start 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 0x7f10e00b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e015bb50 0x7f10e00b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e00b05e0 0x7f10e0070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10e00b2190 con 0x7f10e00b05e0 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10e00b2010 con 0x7f10e015bb50 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e7000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f10e00b2310 con 0x7f10e00ce5e0 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e00b05e0 0x7f10e0070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e00b05e0 0x7f10e0070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33422/0 (socket says 172.21.15.97:33422) 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.932+0000 7f10e5000640 1 -- 172.21.15.97:0/1893167558 learned_addr learned my addr 172.21.15.97:0/1893167558 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10dfe00640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 0x7f10e00b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10d7400640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e015bb50 0x7f10e00b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e5000640 1 -- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 msgr2=0x7f10e00b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e5000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 0x7f10e00b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e5000640 1 -- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e015bb50 msgr2=0x7f10e00b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e5000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e015bb50 0x7f10e00b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e5000640 1 -- 172.21.15.97:0/1893167558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e00caaf0 con 0x7f10e00b05e0 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10dfe00640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 0x7f10e00b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10d7400640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e015bb50 0x7f10e00b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e5000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e00b05e0 0x7f10e0070ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10e0073fa0 tx=0x7f10d8006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10dcc00640 1 -- 172.21.15.97:0/1893167558 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8015070 con 0x7f10e00b05e0 2024-09-10T21:24:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10dcc00640 1 -- 172.21.15.97:0/1893167558 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d800fc90 con 0x7f10e00b05e0 2024-09-10T21:24:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e00cad60 con 0x7f10e00b05e0 2024-09-10T21:24:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10dcc00640 1 -- 172.21.15.97:0/1893167558 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8001ba0 con 0x7f10e00b05e0 2024-09-10T21:24:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e00cb320 con 0x7f10e00b05e0 2024-09-10T21:24:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e00cf3c0 con 0x7f10e00b05e0 2024-09-10T21:24:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.936+0000 7f10dcc00640 1 -- 172.21.15.97:0/1893167558 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f10d8011690 con 0x7f10e00b05e0 2024-09-10T21:24:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.940+0000 7f10dcc00640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10bc0725e0 0x7f10bc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.940+0000 7f10dcc00640 1 -- 172.21.15.97:0/1893167558 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f10d8094230 con 0x7f10e00b05e0 2024-09-10T21:24:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.940+0000 7f10dfe00640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10bc0725e0 0x7f10bc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.940+0000 7f10dfe00640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10bc0725e0 0x7f10bc074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ae000 tx=0x7f10d001a050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:16.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:16.944+0000 7f10dcc00640 1 -- 172.21.15.97:0/1893167558 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10d800fe60 con 0x7f10e00b05e0 2024-09-10T21:24:17.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.140+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10e00d0060 con 0x7f10e00b05e0 2024-09-10T21:24:17.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.140+0000 7f10dcc00640 1 -- 172.21.15.97:0/1893167558 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5225 (secure 0 0 0) 0x7f10d805f660 con 0x7f10e00b05e0 2024-09-10T21:24:17.148 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:17.148 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"btime":"2024-09-10T21:24:16:762234+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:16.155336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:17.149 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-09-10T21:24:17.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10bc0725e0 msgr2=0x7f10bc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10bc0725e0 0x7f10bc074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ae000 tx=0x7f10d001a050 comp rx=0 tx=0).stop 2024-09-10T21:24:17.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e00b05e0 msgr2=0x7f10e0070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e00b05e0 0x7f10e0070ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10e0073fa0 tx=0x7f10d8006720 comp rx=0 tx=0).stop 2024-09-10T21:24:17.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 shutdown_connections 2024-09-10T21:24:17.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10bc0725e0 0x7f10bc074aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e00b05e0 0x7f10e0070ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e015bb50 0x7f10e00b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 --2- 172.21.15.97:0/1893167558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e00ce5e0 0x7f10e00b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 >> 172.21.15.97:0/1893167558 conn(0x7f10e00d1ca0 msgr2=0x7f10e006e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:17.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 shutdown_connections 2024-09-10T21:24:17.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.144+0000 7f10e7000640 1 -- 172.21.15.97:0/1893167558 wait complete. 2024-09-10T21:24:17.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:17.273 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:24:17.264+0000 7f9c3b600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:24:17.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 -- 172.21.15.97:0/1510106448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc072d40 msgr2=0x7f3bfc0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1510106448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc072d40 0x7f3bfc0b2a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009bc0 tx=0x7f3bf401cb70 comp rx=0 tx=0).stop 2024-09-10T21:24:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 -- 172.21.15.97:0/1510106448 shutdown_connections 2024-09-10T21:24:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1510106448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc072d40 0x7f3bfc0b2a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1510106448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bfc15b110 0x7f3bfc0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 -- 172.21.15.97:0/1510106448 >> 172.21.15.97:0/1510106448 conn(0x7f3bfc0b1780 msgr2=0x7f3bfc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 -- 172.21.15.97:0/1510106448 shutdown_connections 2024-09-10T21:24:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 -- 172.21.15.97:0/1510106448 wait complete. 2024-09-10T21:24:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 Processor -- start 2024-09-10T21:24:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.304+0000 7f3c03e00640 1 -- start start 2024-09-10T21:24:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc070720 0x7f3bfc1fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bfc072d40 0x7f3bfc14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bfc15b110 0x7f3bfc146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b3ac0 con 0x7f3bfc070720 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b3940 con 0x7f3bfc072d40 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3bfc166b50 con 0x7f3bfc15b110 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc070720 0x7f3bfc1fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc070720 0x7f3bfc1fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50676/0 (socket says 172.21.15.97:50676) 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 -- 172.21.15.97:0/1934023634 learned_addr learned my addr 172.21.15.97:0/1934023634 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01e00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bfc15b110 0x7f3bfc146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c00a00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bfc072d40 0x7f3bfc14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 -- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bfc15b110 msgr2=0x7f3bfc146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bfc15b110 0x7f3bfc146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 -- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bfc072d40 msgr2=0x7f3bfc14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bfc072d40 0x7f3bfc14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 -- 172.21.15.97:0/1934023634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc14ac50 con 0x7f3bfc070720 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c00a00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bfc072d40 0x7f3bfc14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c01400640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc070720 0x7f3bfc1fb920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000ea90 tx=0x7f3bf000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3bf9600640 1 -- 172.21.15.97:0/1934023634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf001b070 con 0x7f3bfc070720 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3bf9600640 1 -- 172.21.15.97:0/1934023634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bf000cc50 con 0x7f3bfc070720 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3bf9600640 1 -- 172.21.15.97:0/1934023634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0016670 con 0x7f3bfc070720 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bfc14af30 con 0x7f3bfc070720 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bfc0af360 con 0x7f3bfc070720 2024-09-10T21:24:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bfc0b2a80 con 0x7f3bfc070720 2024-09-10T21:24:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.308+0000 7f3bf9600640 1 -- 172.21.15.97:0/1934023634 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3bf000b040 con 0x7f3bfc070720 2024-09-10T21:24:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.312+0000 7f3bf9600640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3bdc0725e0 0x7f3bdc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.312+0000 7f3bf9600640 1 -- 172.21.15.97:0/1934023634 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3bf0093690 con 0x7f3bfc070720 2024-09-10T21:24:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.312+0000 7f3c00a00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3bdc0725e0 0x7f3bdc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.312+0000 7f3c00a00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3bdc0725e0 0x7f3bdc074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc1fae70 tx=0x7f3be8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:17.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.316+0000 7f3bf9600640 1 -- 172.21.15.97:0/1934023634 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3bf005e3d0 con 0x7f3bfc070720 2024-09-10T21:24:17.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.480+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bfc0b02d0 con 0x7f3bfc070720 2024-09-10T21:24:17.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.480+0000 7f3bf9600640 1 -- 172.21.15.97:0/1934023634 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f3bf0062e00 con 0x7f3bfc070720 2024-09-10T21:24:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:17.489 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3bdc0725e0 msgr2=0x7f3bdc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3bdc0725e0 0x7f3bdc074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc1fae70 tx=0x7f3be8009040 comp rx=0 tx=0).stop 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc070720 msgr2=0x7f3bfc1fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc070720 0x7f3bfc1fb920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000ea90 tx=0x7f3bf000ef50 comp rx=0 tx=0).stop 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 shutdown_connections 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3bdc0725e0 0x7f3bdc074aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bfc15b110 0x7f3bfc146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bfc072d40 0x7f3bfc14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 --2- 172.21.15.97:0/1934023634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bfc070720 0x7f3bfc1fb920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 >> 172.21.15.97:0/1934023634 conn(0x7f3bfc0b1780 msgr2=0x7f3bfc0b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:17.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 shutdown_connections 2024-09-10T21:24:17.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.484+0000 7f3c03e00640 1 -- 172.21.15.97:0/1934023634 wait complete. 2024-09-10T21:24:17.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 -- 172.21.15.97:0/1311778253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7780d0490 msgr2=0x7fc7780ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 --2- 172.21.15.97:0/1311778253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7780d0490 0x7fc7780ae930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc774009c80 tx=0x7fc77401cda0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 -- 172.21.15.97:0/1311778253 shutdown_connections 2024-09-10T21:24:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 --2- 172.21.15.97:0/1311778253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7780d0490 0x7fc7780ae930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 --2- 172.21.15.97:0/1311778253 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc77815efc0 0x7fc7780cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 -- 172.21.15.97:0/1311778253 >> 172.21.15.97:0/1311778253 conn(0x7fc7780d1a60 msgr2=0x7fc7780d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 -- 172.21.15.97:0/1311778253 shutdown_connections 2024-09-10T21:24:17.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 -- 172.21.15.97:0/1311778253 wait complete. 2024-09-10T21:24:17.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 Processor -- start 2024-09-10T21:24:17.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.644+0000 7fc77f200640 1 -- start start 2024-09-10T21:24:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7780cdcb0 0x7fc778153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7780d0490 0x7fc7780c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77815efc0 0x7fc7780b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc778168b40 con 0x7fc77815efc0 2024-09-10T21:24:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781689c0 con 0x7fc7780d0490 2024-09-10T21:24:17.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc778168cc0 con 0x7fc7780cdcb0 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77815efc0 0x7fc7780b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77815efc0 0x7fc7780b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50684/0 (socket says 172.21.15.97:50684) 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 -- 172.21.15.97:0/370275538 learned_addr learned my addr 172.21.15.97:0/370275538 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77de00640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7780cdcb0 0x7fc778153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77d400640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7780d0490 0x7fc7780c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 -- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7780cdcb0 msgr2=0x7fc778153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7780cdcb0 0x7fc778153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 -- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7780d0490 msgr2=0x7fc7780c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7780d0490 0x7fc7780c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 -- 172.21.15.97:0/370275538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7781f8d00 con 0x7fc77815efc0 2024-09-10T21:24:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77e800640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77815efc0 0x7fc7780b58e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc774009bd0 tx=0x7fc7740a5a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc772000640 1 -- 172.21.15.97:0/370275538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740bb070 con 0x7fc77815efc0 2024-09-10T21:24:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc772000640 1 -- 172.21.15.97:0/370275538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc774004c50 con 0x7fc77815efc0 2024-09-10T21:24:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc772000640 1 -- 172.21.15.97:0/370275538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740b2040 con 0x7fc77815efc0 2024-09-10T21:24:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7781f8fe0 con 0x7fc77815efc0 2024-09-10T21:24:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781f9ac0 con 0x7fc77815efc0 2024-09-10T21:24:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7780ae930 con 0x7fc77815efc0 2024-09-10T21:24:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.648+0000 7fc772000640 1 -- 172.21.15.97:0/370275538 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc7740b5030 con 0x7fc77815efc0 2024-09-10T21:24:17.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.652+0000 7fc772000640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc75c0725e0 0x7fc75c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:17.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.652+0000 7fc772000640 1 -- 172.21.15.97:0/370275538 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc77412cff0 con 0x7fc77815efc0 2024-09-10T21:24:17.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.652+0000 7fc77de00640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc75c0725e0 0x7fc75c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.652+0000 7fc77de00640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc75c0725e0 0x7fc75c074aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00ab40 tx=0x7fc76c00a250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.656+0000 7fc772000640 1 -- 172.21.15.97:0/370275538 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc774016990 con 0x7fc77815efc0 2024-09-10T21:24:17.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.824+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7780cbe60 con 0x7fc77815efc0 2024-09-10T21:24:17.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.824+0000 7fc772000640 1 -- 172.21.15.97:0/370275538 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7fc7740f7d70 con 0x7fc77815efc0 2024-09-10T21:24:17.831 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:17.831 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc75c0725e0 msgr2=0x7fc75c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc75c0725e0 0x7fc75c074aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00ab40 tx=0x7fc76c00a250 comp rx=0 tx=0).stop 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77815efc0 msgr2=0x7fc7780b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77815efc0 0x7fc7780b58e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc774009bd0 tx=0x7fc7740a5a90 comp rx=0 tx=0).stop 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 shutdown_connections 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc75c0725e0 0x7fc75c074aa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc77815efc0 0x7fc7780b58e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7780d0490 0x7fc7780c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 --2- 172.21.15.97:0/370275538 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7780cdcb0 0x7fc778153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:17.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 >> 172.21.15.97:0/370275538 conn(0x7fc7780d1a60 msgr2=0x7fc7780b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:17.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 shutdown_connections 2024-09-10T21:24:17.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:17.828+0000 7fc77f200640 1 -- 172.21.15.97:0/370275538 wait complete. 2024-09-10T21:24:17.869 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- 172.21.15.97:0/3039201056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7724072630 msgr2=0x7f7724072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 --2- 172.21.15.97:0/3039201056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7724072630 0x7f7724072ae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f771c009c80 tx=0x7f771c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- 172.21.15.97:0/3039201056 shutdown_connections 2024-09-10T21:24:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 --2- 172.21.15.97:0/3039201056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7724072630 0x7f7724072ae0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 --2- 172.21.15.97:0/3039201056 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7724070720 0x7f772406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- 172.21.15.97:0/3039201056 >> 172.21.15.97:0/3039201056 conn(0x7f77240b1780 msgr2=0x7f77240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- 172.21.15.97:0/3039201056 shutdown_connections 2024-09-10T21:24:18.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- 172.21.15.97:0/3039201056 wait complete. 2024-09-10T21:24:18.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 Processor -- start 2024-09-10T21:24:18.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- start start 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7724070720 0x7f7724150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7724072630 0x7f77241f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f772415b110 0x7f772414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7724167960 con 0x7f772415b110 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77241677e0 con 0x7f7724072630 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7729400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7724167ae0 con 0x7f7724070720 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7723200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f772415b110 0x7f772414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7722800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7724070720 0x7f7724150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7723200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f772415b110 0x7f772414b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50700/0 (socket says 172.21.15.97:50700) 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f771be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7724072630 0x7f77241f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.004+0000 7f7723200640 1 -- 172.21.15.97:0/3914232792 learned_addr learned my addr 172.21.15.97:0/3914232792 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7723200640 1 -- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7724070720 msgr2=0x7f7724150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7723200640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7724070720 0x7f7724150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7723200640 1 -- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7724072630 msgr2=0x7f77241f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7723200640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7724072630 0x7f77241f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7723200640 1 -- 172.21.15.97:0/3914232792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77241f7740 con 0x7f772415b110 2024-09-10T21:24:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7723200640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f772415b110 0x7f772414b7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f771c009bd0 tx=0x7f771c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f771aa00640 1 -- 172.21.15.97:0/3914232792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c0bb070 con 0x7f772415b110 2024-09-10T21:24:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f771aa00640 1 -- 172.21.15.97:0/3914232792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f771c004d60 con 0x7f772415b110 2024-09-10T21:24:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f771aa00640 1 -- 172.21.15.97:0/3914232792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c0b2040 con 0x7f772415b110 2024-09-10T21:24:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77241f7a10 con 0x7f772415b110 2024-09-10T21:24:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77241f7fd0 con 0x7f772415b110 2024-09-10T21:24:18.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772406e3b0 con 0x7f772415b110 2024-09-10T21:24:18.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f771aa00640 1 -- 172.21.15.97:0/3914232792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f771c0b5030 con 0x7f772415b110 2024-09-10T21:24:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f771aa00640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77000725e0 0x7f7700074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.008+0000 7f771aa00640 1 -- 172.21.15.97:0/3914232792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f771c12d090 con 0x7f772415b110 2024-09-10T21:24:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.012+0000 7f7722800640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77000725e0 0x7f7700074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.012+0000 7f7722800640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77000725e0 0x7f7700074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7714007fd0 tx=0x7f7714009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:18.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.012+0000 7f771aa00640 1 -- 172.21.15.97:0/3914232792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f771c016990 con 0x7f772415b110 2024-09-10T21:24:18.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.208+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77241523d0 con 0x7f772415b110 2024-09-10T21:24:18.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f771aa00640 1 -- 172.21.15.97:0/3914232792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5225 (secure 0 0 0) 0x7f771c0f7d50 con 0x7f772415b110 2024-09-10T21:24:18.218 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:18.218 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"btime":"2024-09-10T21:24:16:762234+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:16.155336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:18.218 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77000725e0 msgr2=0x7f7700074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77000725e0 0x7f7700074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7714007fd0 tx=0x7f7714009040 comp rx=0 tx=0).stop 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f772415b110 msgr2=0x7f772414b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f772415b110 0x7f772414b7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f771c009bd0 tx=0x7f771c009360 comp rx=0 tx=0).stop 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 shutdown_connections 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77000725e0 0x7f7700074aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f772415b110 0x7f772414b7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7724072630 0x7f77241f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 --2- 172.21.15.97:0/3914232792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7724070720 0x7f7724150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 >> 172.21.15.97:0/3914232792 conn(0x7f77240b1780 msgr2=0x7f7724152bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 shutdown_connections 2024-09-10T21:24:18.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.212+0000 7f7729400640 1 -- 172.21.15.97:0/3914232792 wait complete. 2024-09-10T21:24:18.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.372+0000 7f2f06200640 1 -- 172.21.15.97:0/1583943832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f000afdd0 msgr2=0x7f2f000cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.372+0000 7f2f06200640 1 --2- 172.21.15.97:0/1583943832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f000afdd0 0x7f2f000cc6d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80098a0 tx=0x7f2ee801c760 comp rx=0 tx=0).stop 2024-09-10T21:24:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.372+0000 7f2f06200640 1 -- 172.21.15.97:0/1583943832 shutdown_connections 2024-09-10T21:24:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.372+0000 7f2f06200640 1 --2- 172.21.15.97:0/1583943832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f000afdd0 0x7f2f000cc6d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.372+0000 7f2f06200640 1 -- 172.21.15.97:0/1583943832 >> 172.21.15.97:0/1583943832 conn(0x7f2f000d1c30 msgr2=0x7f2f000d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.372+0000 7f2f06200640 1 -- 172.21.15.97:0/1583943832 shutdown_connections 2024-09-10T21:24:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.372+0000 7f2f06200640 1 -- 172.21.15.97:0/1583943832 wait complete. 2024-09-10T21:24:18.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 Processor -- start 2024-09-10T21:24:18.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 -- start start 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f000afdd0 0x7f2f001f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f000b0d50 0x7f2f00159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f000ce640 0x7f2f0006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0016a020 con 0x7f2f000b0d50 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00169ea0 con 0x7f2f000afdd0 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0016a1a0 con 0x7f2f000ce640 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f000b0d50 0x7f2f00159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:18.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f000b0d50 0x7f2f00159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50708/0 (socket says 172.21.15.97:50708) 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 -- 172.21.15.97:0/4211521149 learned_addr learned my addr 172.21.15.97:0/4211521149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 -- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f000ce640 msgr2=0x7f2f0006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f000ce640 0x7f2f0006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 -- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f000afdd0 msgr2=0x7f2f001f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f000afdd0 0x7f2f001f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 -- 172.21.15.97:0/4211521149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f00159100 con 0x7f2f000b0d50 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef6a00640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f000b0d50 0x7f2f00159660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ee801a390 tx=0x7f2ee8004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef7e00640 1 -- 172.21.15.97:0/4211521149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80b2040 con 0x7f2f000b0d50 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef7e00640 1 -- 172.21.15.97:0/4211521149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ee80a5a30 con 0x7f2f000b0d50 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0006df00 con 0x7f2f000b0d50 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2ef7e00640 1 -- 172.21.15.97:0/4211521149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80a5e20 con 0x7f2f000b0d50 2024-09-10T21:24:18.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0006e4c0 con 0x7f2f000b0d50 2024-09-10T21:24:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.376+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f000cea20 con 0x7f2f000b0d50 2024-09-10T21:24:18.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.380+0000 7f2ef7e00640 1 -- 172.21.15.97:0/4211521149 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2ee80b5070 con 0x7f2f000b0d50 2024-09-10T21:24:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.380+0000 7f2ef7e00640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2edc0725e0 0x7f2edc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.380+0000 7f2ef7e00640 1 -- 172.21.15.97:0/4211521149 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2ee812cfc0 con 0x7f2f000b0d50 2024-09-10T21:24:18.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.380+0000 7f2eff400640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2edc0725e0 0x7f2edc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:18.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.380+0000 7f2eff400640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2edc0725e0 0x7f2edc074aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0001560 tx=0x7f2ef00079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:18.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.384+0000 7f2ef7e00640 1 -- 172.21.15.97:0/4211521149 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ee80c4020 con 0x7f2f000b0d50 2024-09-10T21:24:18.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.552+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f0006ab40 con 0x7f2f000b0d50 2024-09-10T21:24:18.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.552+0000 7f2ef7e00640 1 -- 172.21.15.97:0/4211521149 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f2ee8016390 con 0x7f2f000b0d50 2024-09-10T21:24:18.560 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:18.560 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:18.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2edc0725e0 msgr2=0x7f2edc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2edc0725e0 0x7f2edc074aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0001560 tx=0x7f2ef00079e0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f000b0d50 msgr2=0x7f2f00159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f000b0d50 0x7f2f00159660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ee801a390 tx=0x7f2ee8004620 comp rx=0 tx=0).stop 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 shutdown_connections 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2edc0725e0 0x7f2edc074aa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2f000ce640 0x7f2f0006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f000b0d50 0x7f2f00159660 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 --2- 172.21.15.97:0/4211521149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f000afdd0 0x7f2f001f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 >> 172.21.15.97:0/4211521149 conn(0x7f2f000d1c30 msgr2=0x7f2f0015d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 shutdown_connections 2024-09-10T21:24:18.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:18.556+0000 7f2f06200640 1 -- 172.21.15.97:0/4211521149 wait complete. 2024-09-10T21:24:18.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:24:11.081203+0000', 'modified': '2024-09-10T21:24:16.155336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4381': {'gid': 4381, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6835/4001499169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 4001499169}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 4001499169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4381, 'qdb_cluster': [4381]} 2024-09-10T21:24:18.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:24:18.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:24:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 -- 172.21.15.97:0/250465178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb0143f90 msgr2=0x7f8cb00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/250465178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb0143f90 0x7f8cb00b2a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009900 tx=0x7f8ca001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 -- 172.21.15.97:0/250465178 shutdown_connections 2024-09-10T21:24:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/250465178 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8cb006fd50 0x7f8cb00701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/250465178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb0143f90 0x7f8cb00b2a80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 -- 172.21.15.97:0/250465178 >> 172.21.15.97:0/250465178 conn(0x7f8cb00b1780 msgr2=0x7f8cb00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 -- 172.21.15.97:0/250465178 shutdown_connections 2024-09-10T21:24:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.120+0000 7f8cb6c00640 1 -- 172.21.15.97:0/250465178 wait complete. 2024-09-10T21:24:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 Processor -- start 2024-09-10T21:24:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 -- start start 2024-09-10T21:24:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb006fd50 0x7f8cb0140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8cb0143f90 0x7f8cb013dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cb0144690 0x7f8cb01590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0167ca0 con 0x7f8cb006fd50 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0167b20 con 0x7f8cb0144690 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb0167e20 con 0x7f8cb0143f90 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb006fd50 0x7f8cb0140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb006fd50 0x7f8cb0140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50718/0 (socket says 172.21.15.97:50718) 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 -- 172.21.15.97:0/3791857039 learned_addr learned my addr 172.21.15.97:0/3791857039 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb4c00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cb0144690 0x7f8cb01590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8cb0143f90 0x7f8cb013dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 -- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8cb0143f90 msgr2=0x7f8cb013dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8cb0143f90 0x7f8cb013dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 -- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cb0144690 msgr2=0x7f8cb01590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cb0144690 0x7f8cb01590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 -- 172.21.15.97:0/3791857039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb013f4a0 con 0x7f8cb006fd50 2024-09-10T21:24:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cafe00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb006fd50 0x7f8cb0140540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009850 tx=0x7f8ca0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cacc00640 1 -- 172.21.15.97:0/3791857039 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00b2040 con 0x7f8cb006fd50 2024-09-10T21:24:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cacc00640 1 -- 172.21.15.97:0/3791857039 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ca0005030 con 0x7f8cb006fd50 2024-09-10T21:24:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cacc00640 1 -- 172.21.15.97:0/3791857039 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0005460 con 0x7f8cb006fd50 2024-09-10T21:24:19.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb013f780 con 0x7f8cb006fd50 2024-09-10T21:24:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.124+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb013fd40 con 0x7f8cb006fd50 2024-09-10T21:24:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.128+0000 7f8cacc00640 1 -- 172.21.15.97:0/3791857039 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ca00c0060 con 0x7f8cb006fd50 2024-09-10T21:24:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.128+0000 7f8cacc00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c8c0725e0 0x7f8c8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.128+0000 7f8cacc00640 1 -- 172.21.15.97:0/3791857039 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8ca012ce90 con 0x7f8cb006fd50 2024-09-10T21:24:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.128+0000 7f8cb9f62000 1 Processor -- start 2024-09-10T21:24:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.128+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c8c0725e0 0x7f8c8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.128+0000 7f8cb9f62000 1 -- start start 2024-09-10T21:24:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c8c0725e0 0x7f8c8c074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cb01564f0 tx=0x7f8c98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f59f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55933f4c03f0 con 0x55933f5ec2f0 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55933f3fdb20 con 0x55933f5eb9d0 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55933f3ecf50 con 0x55933f59fe10 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f59f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f59f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50734/0 (socket says 172.21.15.97:50734) 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 -- 172.21.15.97:0/4153417611 learned_addr learned my addr 172.21.15.97:0/4153417611 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb4c00640 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cafe00640 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5ebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 -- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 msgr2=0x55933f5a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 -- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 msgr2=0x55933f5ebdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5ebdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 -- 172.21.15.97:0/4153417611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933f3b4590 con 0x55933f5ec2f0 2024-09-10T21:24:19.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f59f8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800b450 tx=0x7f8c98010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4702c2c6f07971f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca4c00640 1 -- 172.21.15.97:0/4153417611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98024070 con 0x55933f5ec2f0 2024-09-10T21:24:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca4c00640 1 -- 172.21.15.97:0/4153417611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c98014aa0 con 0x55933f5ec2f0 2024-09-10T21:24:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8ca4c00640 1 -- 172.21.15.97:0/4153417611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9801f400 con 0x55933f5ec2f0 2024-09-10T21:24:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- 172.21.15.97:0/4153417611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 msgr2=0x55933f59f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f59f8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800b450 tx=0x7f8c98010000 comp rx=0 tx=0).stop 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- 172.21.15.97:0/4153417611 shutdown_connections 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f59f8d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/4153417611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5ebdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- 172.21.15.97:0/4153417611 >> 172.21.15.97:0/4153417611 conn(0x55933f489130 msgr2=0x55933f5cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- 172.21.15.97:0/4153417611 shutdown_connections 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- 172.21.15.97:0/4153417611 wait complete. 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 -- start start 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.132+0000 7f8cb9f62000 1 Processor -- start 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5a1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f5a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55933f3875d0 con 0x55933f5ec2f0 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55933f3f0d60 con 0x55933f5eb9d0 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55933f393500 con 0x55933f59fe10 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 10 client.0 _ll_register_callbacks cb 0x55933f397810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f5a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f5a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50740/0 (socket says 172.21.15.97:50740) 2024-09-10T21:24:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 -- 172.21.15.97:0/172861760 learned_addr learned my addr 172.21.15.97:0/172861760 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 -- 172.21.15.97:0/172861760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 msgr2=0x55933f5a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 -- 172.21.15.97:0/172861760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 msgr2=0x55933f5a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 -- 172.21.15.97:0/172861760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933f38b170 con 0x55933f5ec2f0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb4c00640 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f5a1cc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800be50 tx=0x7f8ca8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea6744bc9a352b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 1 -- 172.21.15.97:0/172861760 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8007d50 con 0x55933f5ec2f0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55933f3850f0 con 0x55933f5ec2f0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 1 -- 172.21.15.97:0/172861760 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8ca8009c80 con 0x55933f5ec2f0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 1 -- 172.21.15.97:0/172861760 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f8ca8022050 con 0x55933f5ec2f0 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 10 client.4458 fetch_fsmap learned FSMap version 7 2024-09-10T21:24:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55933f471ef0 con 0x55933f5ec2f0 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 1 -- 172.21.15.97:0/172861760 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55933f3850f0 con 0x55933f5ec2f0 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 1 -- 172.21.15.97:0/172861760 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f8ca8025020 con 0x55933f5ec2f0 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 10 client.4458 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 10 client.4458 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 20 client.4458 populate_metadata read hostname 'smithi097' 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f5fac70 0x55933f5fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 4 client.4458 mds_command: new command op to 4381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 --> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933f3ffa70 con 0x55933f5fac70 2024-09-10T21:24:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cafe00640 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f5fac70 0x55933f5fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cafe00640 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f5fac70 0x55933f5fd060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41fdd889f964d08 server_cookie=b88bfa73f9a28c05 in_seq=0 out_seq=0 2024-09-10T21:24:19.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 10 client.4458 ms_handle_connect on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:19.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 1 -- 172.21.15.97:0/172861760 <== mds.0 v2:172.21.15.97:6834/4001499169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8ca011ce50 con 0x55933f5fac70 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 10 client.4458 handle_command_reply: tid=0 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 client.4458 shutdown 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 msgr2=0x55933f5a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f5a1cc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800be50 tx=0x7f8ca8007680 comp rx=0 tx=0).stop 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 shutdown_connections 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f5fac70 0x55933f5fd060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5ec2f0 0x55933f5a1cc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5eb9d0 0x55933f5a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8c9d600640 0 client.4458 ms_handle_reset on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/172861760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f5a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 >> 172.21.15.97:0/172861760 conn(0x55933f489130 msgr2=0x55933f59efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 shutdown_connections 2024-09-10T21:24:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.136+0000 7f8cb9f62000 1 -- 172.21.15.97:0/172861760 wait complete. 2024-09-10T21:24:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.140+0000 7f8cb9f62000 20 client.4458 trim_cache size 0 max 16384 2024-09-10T21:24:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 Processor -- start 2024-09-10T21:24:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 -- start start 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f59fe10 0x55933f65d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f65db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f5ec2f0 0x55933f65e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0202530 con 0x55933f5eb9d0 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0158a90 con 0x55933f59fe10 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb0201f10 con 0x55933f5ec2f0 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f65db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f65db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50742/0 (socket says 172.21.15.97:50742) 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 -- 172.21.15.97:0/176037607 learned_addr learned my addr 172.21.15.97:0/176037607 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cb4c00640 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f5ec2f0 0x55933f65e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8cafe00640 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f59fe10 0x55933f65d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 -- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f5ec2f0 msgr2=0x55933f65e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f5ec2f0 0x55933f65e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 -- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f59fe10 msgr2=0x55933f65d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f59fe10 0x55933f65d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 -- 172.21.15.97:0/176037607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933f448d90 con 0x55933f5eb9d0 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.156+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f65db10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800b420 tx=0x7f8c98004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3283a5eef7f7fb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8c9fe00640 1 -- 172.21.15.97:0/176037607 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9802e070 con 0x55933f5eb9d0 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8c9fe00640 1 -- 172.21.15.97:0/176037607 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c98025040 con 0x55933f5eb9d0 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8c9fe00640 1 -- 172.21.15.97:0/176037607 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98024070 con 0x55933f5eb9d0 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- 172.21.15.97:0/176037607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 msgr2=0x55933f65db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f65db10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800b420 tx=0x7f8c98004650 comp rx=0 tx=0).stop 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- 172.21.15.97:0/176037607 shutdown_connections 2024-09-10T21:24:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f5ec2f0 0x55933f65e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f65db10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/176037607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f59fe10 0x55933f65d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- 172.21.15.97:0/176037607 >> 172.21.15.97:0/176037607 conn(0x55933f489130 msgr2=0x55933f59f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- 172.21.15.97:0/176037607 shutdown_connections 2024-09-10T21:24:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- 172.21.15.97:0/176037607 wait complete. 2024-09-10T21:24:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- start start 2024-09-10T21:24:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 Processor -- start 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f601c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f61d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5ec2f0 0x55933f61dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55933f3f0070 con 0x55933f5eb9d0 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55933f3b8ec0 con 0x55933f5ec2f0 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55933f3f5390 con 0x55933f59fe10 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 10 client.0 _ll_register_callbacks cb 0x7f8ca8009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f61d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f61d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50752/0 (socket says 172.21.15.97:50752) 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 -- 172.21.15.97:0/2624323170 learned_addr learned my addr 172.21.15.97:0/2624323170 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cafe00640 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f601c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb4c00640 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5ec2f0 0x55933f61dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 -- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 msgr2=0x55933f601c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f601c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 -- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5ec2f0 msgr2=0x55933f61dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5ec2f0 0x55933f61dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 -- 172.21.15.97:0/2624323170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933f3b3a90 con 0x55933f5eb9d0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8ca7e00640 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f61d580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800b1b0 tx=0x7f8c98015eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d7844b0700ee62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8c9d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8c9d600640 1 -- 172.21.15.97:0/2624323170 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98024070 con 0x55933f5eb9d0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8c9d600640 1 -- 172.21.15.97:0/2624323170 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8c98004370 con 0x55933f5eb9d0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c98025040 con 0x55933f5eb9d0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8c9d600640 1 -- 172.21.15.97:0/2624323170 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f8c9803c050 con 0x55933f5eb9d0 2024-09-10T21:24:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 10 client.4464 fetch_fsmap learned FSMap version 7 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.160+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55933f392050 con 0x55933f5eb9d0 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8c9d600640 1 -- 172.21.15.97:0/2624323170 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c98025040 con 0x55933f5eb9d0 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8c9d600640 1 -- 172.21.15.97:0/2624323170 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f8c9803f020 con 0x55933f5eb9d0 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 10 client.4464 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 10 client.4464 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 20 client.4464 populate_metadata read hostname 'smithi097' 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f61e000 0x55933f661610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 4 client.4464 mds_command: new command op to 4381 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 --> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] -- command(tid 0: {"prefix": "status"}) -- 0x55933f3fb520 con 0x55933f61e000 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cafe00640 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f61e000 0x55933f661610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8c9d600640 10 client.4464 ms_handle_connect on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cafe00640 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f61e000 0x55933f661610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f552ea0bd7bf993f server_cookie=d229bce19bbb65bf in_seq=0 out_seq=0 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8c9d600640 1 -- 172.21.15.97:0/2624323170 <== mds.0 v2:172.21.15.97:6834/4001499169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55933f3fb520 con 0x55933f61e000 2024-09-10T21:24:19.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8c9d600640 10 client.4464 handle_command_reply: tid=0 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 client.4464 shutdown 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 msgr2=0x55933f61d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f61d580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800b1b0 tx=0x7f8c98015eb0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 shutdown_connections 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55933f61e000 0x55933f661610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933f5ec2f0 0x55933f61dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8c9d600640 0 client.4464 ms_handle_reset on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55933f5eb9d0 0x55933f61d580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 --2- 172.21.15.97:0/2624323170 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55933f59fe10 0x55933f601c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 >> 172.21.15.97:0/2624323170 conn(0x55933f489130 msgr2=0x55933f5fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 shutdown_connections 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 1 -- 172.21.15.97:0/2624323170 wait complete. 2024-09-10T21:24:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb9f62000 20 client.4464 trim_cache size 0 max 16384 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4381, 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 7.4549685979999998, 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 7, 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 26, 2024-09-10T21:24:19.178 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 26, 2024-09-10T21:24:19.179 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 12.919001165999999 2024-09-10T21:24:19.179 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:24:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c8c0725e0 msgr2=0x7f8c8c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c8c0725e0 0x7f8c8c074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cb01564f0 tx=0x7f8c98008040 comp rx=0 tx=0).stop 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb006fd50 msgr2=0x7f8cb0140540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb006fd50 0x7f8cb0140540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009850 tx=0x7f8ca0004040 comp rx=0 tx=0).stop 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 shutdown_connections 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c8c0725e0 0x7f8c8c074aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cb0144690 0x7f8cb01590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8cb0143f90 0x7f8cb013dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 --2- 172.21.15.97:0/3791857039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cb006fd50 0x7f8cb0140540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 >> 172.21.15.97:0/3791857039 conn(0x7f8cb00b1780 msgr2=0x7f8cb00cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 shutdown_connections 2024-09-10T21:24:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.164+0000 7f8cb6c00640 1 -- 172.21.15.97:0/3791857039 wait complete. 2024-09-10T21:24:19.190 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:24:19.190 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:19.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:24:19.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.324+0000 7f758b000640 1 -- 172.21.15.97:0/3227076142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 msgr2=0x7f756c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.324+0000 7f758b000640 1 --2- 172.21.15.97:0/3227076142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 0x7f756c003e80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7578006f90 tx=0x7f757801cf20 comp rx=0 tx=0).stop 2024-09-10T21:24:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.324+0000 7f758a600640 1 -- 172.21.15.97:0/3227076142 reap_dead start 2024-09-10T21:24:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.324+0000 7f758b000640 1 -- 172.21.15.97:0/3227076142 shutdown_connections 2024-09-10T21:24:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.324+0000 7f758b000640 1 -- 172.21.15.97:0/3227076142 >> 172.21.15.97:0/3227076142 conn(0x7f75840b2ee0 msgr2=0x7f75840d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.324+0000 7f758b000640 1 -- 172.21.15.97:0/3227076142 shutdown_connections 2024-09-10T21:24:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.324+0000 7f758b000640 1 -- 172.21.15.97:0/3227076142 wait complete. 2024-09-10T21:24:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 Processor -- start 2024-09-10T21:24:19.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 -- start start 2024-09-10T21:24:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 0x7f75840aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75841f7770 0x7f75841fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75841f7bf0 0x7f75840b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75840af980 con 0x7f756c003a60 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f75840af800 con 0x7f75841f7770 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f75840afb00 con 0x7f75841f7bf0 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 0x7f75840aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 0x7f75840aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50768/0 (socket says 172.21.15.97:50768) 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 -- 172.21.15.97:0/3207823891 learned_addr learned my addr 172.21.15.97:0/3207823891 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589200640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75841f7770 0x7f75841fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758a600640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75841f7bf0 0x7f75840b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 -- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75841f7bf0 msgr2=0x7f75840b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75841f7bf0 0x7f75840b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 -- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75841f7770 msgr2=0x7f75841fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75841f7770 0x7f75841fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 -- 172.21.15.97:0/3207823891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7584155630 con 0x7f756c003a60 2024-09-10T21:24:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7589c00640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 0x7f75840aec20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f757801b620 tx=0x7f7578001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7581600640 1 -- 172.21.15.97:0/3207823891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578006340 con 0x7f756c003a60 2024-09-10T21:24:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7581600640 1 -- 172.21.15.97:0/3207823891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75780064e0 con 0x7f756c003a60 2024-09-10T21:24:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f7581600640 1 -- 172.21.15.97:0/3207823891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578001620 con 0x7f756c003a60 2024-09-10T21:24:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7584155910 con 0x7f756c003a60 2024-09-10T21:24:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75841fb750 con 0x7f756c003a60 2024-09-10T21:24:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.328+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7584155e10 con 0x7f756c003a60 2024-09-10T21:24:19.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.332+0000 7f7581600640 1 -- 172.21.15.97:0/3207823891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7578003600 con 0x7f756c003a60 2024-09-10T21:24:19.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.332+0000 7f7581600640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7564072610 0x7f7564074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:19.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.332+0000 7f7581600640 1 -- 172.21.15.97:0/3207823891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f757812ce10 con 0x7f756c003a60 2024-09-10T21:24:19.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.332+0000 7f7589200640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7564072610 0x7f7564074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:19.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.332+0000 7f7589200640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7564072610 0x7f7564074ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f758415dae0 tx=0x7f75700091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:19.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.336+0000 7f7581600640 1 -- 172.21.15.97:0/3207823891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7584155e10 con 0x7f756c003a60 2024-09-10T21:24:19.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.536+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f75841fb570 con 0x7f756c003a60 2024-09-10T21:24:19.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.536+0000 7f7581600640 1 -- 172.21.15.97:0/3207823891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f75841fb570 con 0x7f756c003a60 2024-09-10T21:24:19.544 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T21:24:19.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7564072610 msgr2=0x7f7564074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7564072610 0x7f7564074ad0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f758415dae0 tx=0x7f75700091e0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 msgr2=0x7f75840aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 0x7f75840aec20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f757801b620 tx=0x7f7578001d60 comp rx=0 tx=0).stop 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 shutdown_connections 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7564072610 0x7f7564074ad0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75841f7bf0 0x7f75840b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75841f7770 0x7f75841fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 --2- 172.21.15.97:0/3207823891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f756c003a60 0x7f75840aec20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 >> 172.21.15.97:0/3207823891 conn(0x7f75840b2ee0 msgr2=0x7f75840b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 shutdown_connections 2024-09-10T21:24:19.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19.540+0000 7f758b000640 1 -- 172.21.15.97:0/3207823891 wait complete. 2024-09-10T21:24:19.564 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:24:19.564 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:24:19.564 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:24:19.564 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T21:24:19.564 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:19.565 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:24:19.565 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:24:19.565 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:24:19.565 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T21:24:19.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:19.565 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:19.575 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42434sec preferred_lft 42434sec 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:24:19.576 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:19.578 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:24:19.578 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:19.578 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:19.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:24:19.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-09-10T21:24:19.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-09-10T21:24:19.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:24:19.578 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:19.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:19.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:19.736 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:24:19.755 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:24:19.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:19.758 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:24:19.831 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:24:19.831 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:24:19.831 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:19.832 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:24:19.832 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:19.833 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:19.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:24:19.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:24:19.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:24:19.833 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:19.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:19.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:19.969 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:24:19.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:19.978 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-09-10T21:24:20.029 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:20.029 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:20.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:20.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T21:24:20.030 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:20.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:20.156 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T21:24:20.156 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:20.156 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:20.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T21:24:20.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:24:20.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T21:24:20.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T21:24:20.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T21:24:20.157 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:20.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:20.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:20.361 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:20.361 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:20.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:24:20.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:24:20.361 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:20.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:20.445 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T21:24:20.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:20.446 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:20.501 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T21:24:20.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:20.501 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:20.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-09-10T21:24:20.612 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-09-10T21:24:20.662 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:24:20.663 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:20.664 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:20.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:20.665 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-09-10T21:24:20.721 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T21:24:20.721 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T21:24:20.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-09-10T21:24:20.788 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-09-10T21:24:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-09-10T21:24:20.847 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:20.848 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:20.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:20.849 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-09-10T21:24:20.864 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:2024-09-10T21:24:20.856+0000 7f3294fab580 -1 init, newargv = 0x5645dd388620 newargc=15 2024-09-10T21:24:20.864 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[35714]: starting ceph client 2024-09-10T21:24:20.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[35714]: starting fuse 2024-09-10T21:24:21.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-09-10T21:24:21.997 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/lock 2024-09-10T21:24:22.051 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/bpf 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/fs/nfsd 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-09-10T21:24:22.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/run/rpc_pipefs 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:24:22.053 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:24:22.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-09-10T21:24:22.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:22.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:22.054 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:22.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:22.056 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-09-10T21:24:22.109 INFO:teuthology.orchestra.run.smithi097.stdout:44 2024-09-10T21:24:22.109 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T21:24:22.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-09-10T21:24:22.109 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-09-10T21:24:22.110 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.admin")) 2024-09-10T21:24:22.111 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:24:22.241 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.admin.35714.asok 2024-09-10T21:24:22.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:22.253 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.35714.asok 2024-09-10T21:24:22.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:22.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.35714.asok status 2024-09-10T21:24:22.425 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "admin", 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "35714", 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4476, 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4476 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 912139195 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 912139195 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4476 192.168.0.1:0/912139195", 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/912139195", 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 7, 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 26, 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:24:22.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:22.441 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-09-10T21:24:22.442 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:22.451 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-09-10T21:24:22.451 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:22.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:22.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:22.512 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T21:24:22.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:22.631 INFO:teuthology.orchestra.run.smithi097.stdout:# file: . 2024-09-10T21:24:22.631 INFO:teuthology.orchestra.run.smithi097.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T21:24:22.631 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:22.638 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:22.638 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:22.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 -- 172.21.15.97:0/3549055972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 msgr2=0x7f7f200afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 --2- 172.21.15.97:0/3549055972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200afc40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f200736b0 tx=0x7f7f1801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 -- 172.21.15.97:0/3549055972 shutdown_connections 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 --2- 172.21.15.97:0/3549055972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200afc40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 --2- 172.21.15.97:0/3549055972 >> v2:172.21.15.194:3300/0 conn(0x7f7f2015f350 0x7f7f200b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 -- 172.21.15.97:0/3549055972 >> 172.21.15.97:0/3549055972 conn(0x7f7f200d1c60 msgr2=0x7f7f200d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 -- 172.21.15.97:0/3549055972 shutdown_connections 2024-09-10T21:24:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 -- 172.21.15.97:0/3549055972 wait complete. 2024-09-10T21:24:22.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 Processor -- start 2024-09-10T21:24:22.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.868+0000 7f7f25800640 1 -- start start 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f200d05e0 0x7f7f200b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f2015f350 0x7f7f200b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20168ba0 con 0x7f7f200d0150 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20168a20 con 0x7f7f2015f350 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7f20168d20 con 0x7f7f200d05e0 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200b70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50798/0 (socket says 172.21.15.97:50798) 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 -- 172.21.15.97:0/1883744721 learned_addr learned my addr 172.21.15.97:0/1883744721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1f400640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f2015f350 0x7f7f200b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1e000640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f200d05e0 0x7f7f200b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 -- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f200d05e0 msgr2=0x7f7f200b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f200d05e0 0x7f7f200b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 -- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f2015f350 msgr2=0x7f7f200b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f2015f350 0x7f7f200b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 -- 172.21.15.97:0/1883744721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f201f8d70 con 0x7f7f200d0150 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1e000640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f200d05e0 0x7f7f200b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f1ea00640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200b70f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c9f0 tx=0x7f7f1000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f16a00640 1 -- 172.21.15.97:0/1883744721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10007bb0 con 0x7f7f200d0150 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f16a00640 1 -- 172.21.15.97:0/1883744721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f10007e40 con 0x7f7f200d0150 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f201f9050 con 0x7f7f200d0150 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f16a00640 1 -- 172.21.15.97:0/1883744721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000f690 con 0x7f7f200d0150 2024-09-10T21:24:22.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f201f9610 con 0x7f7f200d0150 2024-09-10T21:24:22.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f200afc40 con 0x7f7f200d0150 2024-09-10T21:24:22.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.872+0000 7f7f16a00640 1 -- 172.21.15.97:0/1883744721 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7f10020030 con 0x7f7f200d0150 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.876+0000 7f7f16a00640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7f000725e0 0x7f7f00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.876+0000 7f7f16a00640 1 -- 172.21.15.97:0/1883744721 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7f10093a00 con 0x7f7f200d0150 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.876+0000 7f7f1e000640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7f000725e0 0x7f7f00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.876+0000 7f7f1e000640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7f000725e0 0x7f7f00074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f200b2510 tx=0x7f7f0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:22.880+0000 7f7f16a00640 1 -- 172.21.15.97:0/1883744721 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7f10014030 con 0x7f7f200d0150 2024-09-10T21:24:23.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7f200b13a0 con 0x7f7f200d0150 2024-09-10T21:24:23.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f16a00640 1 -- 172.21.15.97:0/1883744721 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f7f1005ed90 con 0x7f7f200d0150 2024-09-10T21:24:23.063 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:24:23.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7f000725e0 msgr2=0x7f7f00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7f000725e0 0x7f7f00074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f200b2510 tx=0x7f7f0c009040 comp rx=0 tx=0).stop 2024-09-10T21:24:23.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 msgr2=0x7f7f200b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200b70f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c9f0 tx=0x7f7f1000ceb0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 shutdown_connections 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7f000725e0 0x7f7f00074aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f2015f350 0x7f7f200b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f200d05e0 0x7f7f200b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 --2- 172.21.15.97:0/1883744721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f200d0150 0x7f7f200b70f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.056+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 >> 172.21.15.97:0/1883744721 conn(0x7f7f200d1c60 msgr2=0x7f7f200cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.060+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 shutdown_connections 2024-09-10T21:24:23.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.060+0000 7f7f25800640 1 -- 172.21.15.97:0/1883744721 wait complete. 2024-09-10T21:24:23.085 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:23.085 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:23.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:24:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 -- 172.21.15.97:0/993941172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f049c0039f0 msgr2=0x7f049c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 --2- 172.21.15.97:0/993941172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f049c0039f0 0x7f049c003dd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f04a8001ea0 tx=0x7f04a801cc40 comp rx=0 tx=0).stop 2024-09-10T21:24:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bd800640 1 -- 172.21.15.97:0/993941172 reap_dead start 2024-09-10T21:24:23.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 -- 172.21.15.97:0/993941172 shutdown_connections 2024-09-10T21:24:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 -- 172.21.15.97:0/993941172 >> 172.21.15.97:0/993941172 conn(0x7f04b80b1780 msgr2=0x7f04b80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 -- 172.21.15.97:0/993941172 shutdown_connections 2024-09-10T21:24:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 -- 172.21.15.97:0/993941172 wait complete. 2024-09-10T21:24:23.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 Processor -- start 2024-09-10T21:24:23.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.272+0000 7f04bf800640 1 -- start start 2024-09-10T21:24:23.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f049c0039f0 0x7f04b81fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04b81649f0 0x7f04b815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b8140e90 0x7f04b815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8070680 con 0x7f04b8140e90 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8070500 con 0x7f04b81649f0 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f04b8167070 con 0x7f049c0039f0 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b8140e90 0x7f04b815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b8140e90 0x7f04b815e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50818/0 (socket says 172.21.15.97:50818) 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 -- 172.21.15.97:0/3627314102 learned_addr learned my addr 172.21.15.97:0/3627314102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bce00640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f049c0039f0 0x7f04b81fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04b7e00640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04b81649f0 0x7f04b815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 -- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f049c0039f0 msgr2=0x7f04b81fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f049c0039f0 0x7f04b81fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 -- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04b81649f0 msgr2=0x7f04b815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04b81649f0 0x7f04b815a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 -- 172.21.15.97:0/3627314102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b80ad950 con 0x7f04b8140e90 2024-09-10T21:24:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bd800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b8140e90 0x7f04b815e810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04b815e650 tx=0x7f04b00039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04b4c00640 1 -- 172.21.15.97:0/3627314102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0011630 con 0x7f04b8140e90 2024-09-10T21:24:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04b4c00640 1 -- 172.21.15.97:0/3627314102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b0018070 con 0x7f04b8140e90 2024-09-10T21:24:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b8164e50 con 0x7f04b8140e90 2024-09-10T21:24:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04b4c00640 1 -- 172.21.15.97:0/3627314102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0011ab0 con 0x7f04b8140e90 2024-09-10T21:24:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b81fbba0 con 0x7f04b8140e90 2024-09-10T21:24:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.276+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b80ccee0 con 0x7f04b8140e90 2024-09-10T21:24:23.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.280+0000 7f04b4c00640 1 -- 172.21.15.97:0/3627314102 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04b0011c50 con 0x7f04b8140e90 2024-09-10T21:24:23.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.280+0000 7f04b4c00640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0498072610 0x7f0498074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.280+0000 7f04b4c00640 1 -- 172.21.15.97:0/3627314102 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f04b0093bd0 con 0x7f04b8140e90 2024-09-10T21:24:23.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.280+0000 7f04bce00640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0498072610 0x7f0498074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.280+0000 7f04bce00640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0498072610 0x7f0498074ad0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04a80ac000 tx=0x7f04a80a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.284+0000 7f04b4c00640 1 -- 172.21.15.97:0/3627314102 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04b005eed0 con 0x7f04b8140e90 2024-09-10T21:24:23.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.460+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04b80adb20 con 0x7f04b8140e90 2024-09-10T21:24:23.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04b4c00640 1 -- 172.21.15.97:0/3627314102 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f04b005e7d0 con 0x7f04b8140e90 2024-09-10T21:24:23.470 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0498072610 msgr2=0x7f0498074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0498072610 0x7f0498074ad0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04a80ac000 tx=0x7f04a80a7040 comp rx=0 tx=0).stop 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b8140e90 msgr2=0x7f04b815e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b8140e90 0x7f04b815e810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04b815e650 tx=0x7f04b00039d0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 shutdown_connections 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0498072610 0x7f0498074ad0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04b8140e90 0x7f04b815e810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04b81649f0 0x7f04b815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 --2- 172.21.15.97:0/3627314102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f049c0039f0 0x7f04b81fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 >> 172.21.15.97:0/3627314102 conn(0x7f04b80b1780 msgr2=0x7f04b80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 shutdown_connections 2024-09-10T21:24:23.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:23.464+0000 7f04bf800640 1 -- 172.21.15.97:0/3627314102 wait complete. 2024-09-10T21:24:23.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-09-10T21:24:23.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:23.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:24:23.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[35714]: fuse finished with error 0 and tester_r 0 2024-09-10T21:24:23.517 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:24:29.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:24:29.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:29.619 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T21:24:29.638 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:29.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3598294340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 msgr2=0x7f2a980cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:29.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3598294340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 0x7f2a980cf800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009840 tx=0x7f2a8801f010 comp rx=0 tx=0).stop 2024-09-10T21:24:29.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3598294340 shutdown_connections 2024-09-10T21:24:29.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3598294340 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a980cec10 0x7f2a980cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:29.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3598294340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 0x7f2a980cf800 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:29.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3598294340 >> 172.21.15.97:0/3598294340 conn(0x7f2a980b37c0 msgr2=0x7f2a980b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:29.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3598294340 shutdown_connections 2024-09-10T21:24:29.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.824+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3598294340 wait complete. 2024-09-10T21:24:29.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 Processor -- start 2024-09-10T21:24:29.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 -- start start 2024-09-10T21:24:29.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 0x7f2a981e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:29.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a980cec10 0x7f2a9815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:29.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a980d0280 0x7f2a981e7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:29.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a980cde60 con 0x7f2a980d0280 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2a980b44a0 con 0x7f2a980ce830 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a9dc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2a980ce1a0 con 0x7f2a980cec10 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a980d0280 0x7f2a981e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a980d0280 0x7f2a981e7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40968/0 (socket says 172.21.15.97:40968) 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 -- 172.21.15.97:0/3421871378 learned_addr learned my addr 172.21.15.97:0/3421871378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a8fe00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a980cec10 0x7f2a9815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a96a00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 0x7f2a981e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 -- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a980cec10 msgr2=0x7f2a9815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a980cec10 0x7f2a9815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 -- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 msgr2=0x7f2a981e81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 0x7f2a981e81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:29.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 -- 172.21.15.97:0/3421871378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a981e7d60 con 0x7f2a980d0280 2024-09-10T21:24:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.828+0000 7f2a97400640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a980d0280 0x7f2a981e7820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c6b0 tx=0x7f2a9000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a8ea00640 1 -- 172.21.15.97:0/3421871378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90003b80 con 0x7f2a980d0280 2024-09-10T21:24:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a8ea00640 1 -- 172.21.15.97:0/3421871378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a90003d40 con 0x7f2a980d0280 2024-09-10T21:24:29.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a8ea00640 1 -- 172.21.15.97:0/3421871378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90005330 con 0x7f2a980d0280 2024-09-10T21:24:29.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a981e8800 con 0x7f2a980d0280 2024-09-10T21:24:29.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981e8ec0 con 0x7f2a980d0280 2024-09-10T21:24:29.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a980b2780 con 0x7f2a980d0280 2024-09-10T21:24:29.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a8ea00640 1 -- 172.21.15.97:0/3421871378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a900054d0 con 0x7f2a980d0280 2024-09-10T21:24:29.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a8ea00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a740725e0 0x7f2a74074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:29.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a8ea00640 1 -- 172.21.15.97:0/3421871378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2a900932d0 con 0x7f2a980d0280 2024-09-10T21:24:29.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.832+0000 7f2a96a00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a740725e0 0x7f2a74074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:29.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.836+0000 7f2a96a00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a740725e0 0x7f2a74074aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a880b2000 tx=0x7f2a8801d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:29.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:29.848+0000 7f2a8ea00640 1 -- 172.21.15.97:0/3421871378 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a9005df80 con 0x7f2a980d0280 2024-09-10T21:24:30.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.016+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a98071690 con 0x7f2a980d0280 2024-09-10T21:24:30.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.016+0000 7f2a8ea00640 1 -- 172.21.15.97:0/3421871378 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f2a900629b0 con 0x7f2a980d0280 2024-09-10T21:24:30.024 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:30.024 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a740725e0 msgr2=0x7f2a74074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a740725e0 0x7f2a74074aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a880b2000 tx=0x7f2a8801d850 comp rx=0 tx=0).stop 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a980d0280 msgr2=0x7f2a981e7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a980d0280 0x7f2a981e7820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c6b0 tx=0x7f2a9000cb70 comp rx=0 tx=0).stop 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 shutdown_connections 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a740725e0 0x7f2a74074aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a980d0280 0x7f2a981e7820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a980cec10 0x7f2a9815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 --2- 172.21.15.97:0/3421871378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a980ce830 0x7f2a981e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 >> 172.21.15.97:0/3421871378 conn(0x7f2a980b37c0 msgr2=0x7f2a98071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 shutdown_connections 2024-09-10T21:24:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.020+0000 7f2a9dc00640 1 -- 172.21.15.97:0/3421871378 wait complete. 2024-09-10T21:24:30.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 -- 172.21.15.97:0/2063558385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c015b110 msgr2=0x7f43c0072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 --2- 172.21.15.97:0/2063558385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c015b110 0x7f43c0072b70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43c0072040 tx=0x7f43b801c890 comp rx=0 tx=0).stop 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 -- 172.21.15.97:0/2063558385 shutdown_connections 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 --2- 172.21.15.97:0/2063558385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c015b110 0x7f43c0072b70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 --2- 172.21.15.97:0/2063558385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43c00cf490 0x7f43c0072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 --2- 172.21.15.97:0/2063558385 >> v2:172.21.15.194:3300/0 conn(0x7f43c00cf0b0 0x7f43c006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 -- 172.21.15.97:0/2063558385 >> 172.21.15.97:0/2063558385 conn(0x7f43c00b1780 msgr2=0x7f43c00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 -- 172.21.15.97:0/2063558385 shutdown_connections 2024-09-10T21:24:30.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.180+0000 7f43c6e00640 1 -- 172.21.15.97:0/2063558385 wait complete. 2024-09-10T21:24:30.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 Processor -- start 2024-09-10T21:24:30.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 -- start start 2024-09-10T21:24:30.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c00cf0b0 0x7f43c0138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c00cf490 0x7f43c01f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43c015b110 0x7f43c01fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43c00b2c00 con 0x7f43c00cf0b0 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f43c00b2a80 con 0x7f43c00cf490 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f43c00b2d80 con 0x7f43c015b110 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c00cf0b0 0x7f43c0138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c00cf0b0 0x7f43c0138f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40976/0 (socket says 172.21.15.97:40976) 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 -- 172.21.15.97:0/1787899302 learned_addr learned my addr 172.21.15.97:0/1787899302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43b7400640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c00cf490 0x7f43c01f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c4e00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43c015b110 0x7f43c01fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 -- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43c015b110 msgr2=0x7f43c01fb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43c015b110 0x7f43c01fb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 -- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c00cf490 msgr2=0x7f43c01f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c00cf490 0x7f43c01f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 -- 172.21.15.97:0/1787899302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c014ad50 con 0x7f43c00cf0b0 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bfe00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c00cf0b0 0x7f43c0138f10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43b000ead0 tx=0x7f43b000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bcc00640 1 -- 172.21.15.97:0/1787899302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000ce40 con 0x7f43c00cf0b0 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bcc00640 1 -- 172.21.15.97:0/1787899302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43b0011560 con 0x7f43c00cf0b0 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43bcc00640 1 -- 172.21.15.97:0/1787899302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0011990 con 0x7f43c00cf0b0 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c01504e0 con 0x7f43c00cf0b0 2024-09-10T21:24:30.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c014b2f0 con 0x7f43c00cf0b0 2024-09-10T21:24:30.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.184+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c006e3b0 con 0x7f43c00cf0b0 2024-09-10T21:24:30.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.188+0000 7f43bcc00640 1 -- 172.21.15.97:0/1787899302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f43b0022030 con 0x7f43c00cf0b0 2024-09-10T21:24:30.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.188+0000 7f43bcc00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f439c0725e0 0x7f439c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.188+0000 7f43bcc00640 1 -- 172.21.15.97:0/1787899302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f43b0093e40 con 0x7f43c00cf0b0 2024-09-10T21:24:30.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.188+0000 7f43b7400640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f439c0725e0 0x7f439c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.188+0000 7f43b7400640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f439c0725e0 0x7f439c074aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f43c014d450 tx=0x7f43a800b490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:30.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.192+0000 7f43bcc00640 1 -- 172.21.15.97:0/1787899302 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f43b0016030 con 0x7f43c00cf0b0 2024-09-10T21:24:30.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43c01592b0 con 0x7f43c00cf0b0 2024-09-10T21:24:30.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43bcc00640 1 -- 172.21.15.97:0/1787899302 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f43b005eaf0 con 0x7f43c00cf0b0 2024-09-10T21:24:30.367 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:30.367 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f439c0725e0 msgr2=0x7f439c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f439c0725e0 0x7f439c074aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f43c014d450 tx=0x7f43a800b490 comp rx=0 tx=0).stop 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c00cf0b0 msgr2=0x7f43c0138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c00cf0b0 0x7f43c0138f10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43b000ead0 tx=0x7f43b000ef90 comp rx=0 tx=0).stop 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 shutdown_connections 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f439c0725e0 0x7f439c074aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43c015b110 0x7f43c01fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43c00cf490 0x7f43c01f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 --2- 172.21.15.97:0/1787899302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43c00cf0b0 0x7f43c0138f10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 >> 172.21.15.97:0/1787899302 conn(0x7f43c00b1780 msgr2=0x7f43c01594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:30.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 shutdown_connections 2024-09-10T21:24:30.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.360+0000 7f43c6e00640 1 -- 172.21.15.97:0/1787899302 wait complete. 2024-09-10T21:24:30.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:24:30.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- 172.21.15.97:0/2872049309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 msgr2=0x7fe8c415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 --2- 172.21.15.97:0/2872049309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c415f6d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4141580 tx=0x7fe8b001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- 172.21.15.97:0/2872049309 shutdown_connections 2024-09-10T21:24:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 --2- 172.21.15.97:0/2872049309 >> v2:172.21.15.194:3300/0 conn(0x7fe8c415fc10 0x7fe8c413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 --2- 172.21.15.97:0/2872049309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c415f6d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- 172.21.15.97:0/2872049309 >> 172.21.15.97:0/2872049309 conn(0x7fe8c40b1780 msgr2=0x7fe8c40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- 172.21.15.97:0/2872049309 shutdown_connections 2024-09-10T21:24:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- 172.21.15.97:0/2872049309 wait complete. 2024-09-10T21:24:30.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 Processor -- start 2024-09-10T21:24:30.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- start start 2024-09-10T21:24:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8c40ce5f0 0x7fe8c415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c4154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8c415fc10 0x7fe8c4135e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4168a70 con 0x7fe8c4141d20 2024-09-10T21:24:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4140ca0 con 0x7fe8c415fc10 2024-09-10T21:24:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8cc200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c4168bf0 con 0x7fe8c40ce5f0 2024-09-10T21:24:30.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c4154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40990/0 (socket says 172.21.15.97:40990) 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 -- 172.21.15.97:0/3145089134 learned_addr learned my addr 172.21.15.97:0/3145089134 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8ca200640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8c415fc10 0x7fe8c4135e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c9800640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8c40ce5f0 0x7fe8c415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 -- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8c40ce5f0 msgr2=0x7fe8c415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8c40ce5f0 0x7fe8c415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 -- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8c415fc10 msgr2=0x7fe8c4135e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8c415fc10 0x7fe8c4135e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 -- 172.21.15.97:0/3145089134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c406ee60 con 0x7fe8c4141d20 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c9800640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8c40ce5f0 0x7fe8c415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c8e00640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c4154780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00194b0 tx=0x7fe8b00a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c1600640 1 -- 172.21.15.97:0/3145089134 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00b2040 con 0x7fe8c4141d20 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.528+0000 7fe8c1600640 1 -- 172.21.15.97:0/3145089134 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8b0004400 con 0x7fe8c4141d20 2024-09-10T21:24:30.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8c1600640 1 -- 172.21.15.97:0/3145089134 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0005290 con 0x7fe8c4141d20 2024-09-10T21:24:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c406f140 con 0x7fe8c4141d20 2024-09-10T21:24:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c406f700 con 0x7fe8c4141d20 2024-09-10T21:24:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c415cc40 con 0x7fe8c4141d20 2024-09-10T21:24:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8c1600640 1 -- 172.21.15.97:0/3145089134 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe8b0005430 con 0x7fe8c4141d20 2024-09-10T21:24:30.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8c1600640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8a40725e0 0x7fe8a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8c1600640 1 -- 172.21.15.97:0/3145089134 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe8b012d550 con 0x7fe8c4141d20 2024-09-10T21:24:30.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.532+0000 7fe8c9800640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8a40725e0 0x7fe8a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.536+0000 7fe8c9800640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8a40725e0 0x7fe8a4074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4051100 tx=0x7fe8b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:30.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.536+0000 7fe8c1600640 1 -- 172.21.15.97:0/3145089134 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8b00c4060 con 0x7fe8c4141d20 2024-09-10T21:24:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.736+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe8c415f6d0 con 0x7fe8c4141d20 2024-09-10T21:24:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.800+0000 7fe8c1600640 1 -- 172.21.15.97:0/3145089134 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fe8b0016490 con 0x7fe8c4141d20 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8a40725e0 msgr2=0x7fe8a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8a40725e0 0x7fe8a4074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4051100 tx=0x7fe8b8009040 comp rx=0 tx=0).stop 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 msgr2=0x7fe8c4154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c4154780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00194b0 tx=0x7fe8b00a5740 comp rx=0 tx=0).stop 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 shutdown_connections 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8a40725e0 0x7fe8a4074aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8c415fc10 0x7fe8c4135e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8c4141d20 0x7fe8c4154780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 --2- 172.21.15.97:0/3145089134 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8c40ce5f0 0x7fe8c415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 >> 172.21.15.97:0/3145089134 conn(0x7fe8c40b1780 msgr2=0x7fe8c415d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 shutdown_connections 2024-09-10T21:24:30.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.804+0000 7fe8cc200640 1 -- 172.21.15.97:0/3145089134 wait complete. 2024-09-10T21:24:30.830 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-10T21:24:30.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:24:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.968+0000 7f23d4200640 1 -- 172.21.15.97:0/2725474589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 msgr2=0x7f23b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.968+0000 7f23d4200640 1 --2- 172.21.15.97:0/2725474589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23b4004c90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f23b8005fd0 tx=0x7f23b801cd80 comp rx=0 tx=0).stop 2024-09-10T21:24:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.968+0000 7f23d4200640 1 -- 172.21.15.97:0/2725474589 shutdown_connections 2024-09-10T21:24:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.968+0000 7f23d4200640 1 --2- 172.21.15.97:0/2725474589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23b4004c90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.968+0000 7f23d4200640 1 -- 172.21.15.97:0/2725474589 >> 172.21.15.97:0/2725474589 conn(0x7f23cc0b1780 msgr2=0x7f23cc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.968+0000 7f23d4200640 1 -- 172.21.15.97:0/2725474589 shutdown_connections 2024-09-10T21:24:30.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.968+0000 7f23d4200640 1 -- 172.21.15.97:0/2725474589 wait complete. 2024-09-10T21:24:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 Processor -- start 2024-09-10T21:24:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 -- start start 2024-09-10T21:24:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23cc1f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23cc06ef10 0x7f23cc15d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23cc06f370 0x7f23cc15d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc0b2e10 con 0x7f23b4004810 2024-09-10T21:24:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc0b2c90 con 0x7f23cc06f370 2024-09-10T21:24:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f23cc0701a0 con 0x7f23cc06ef10 2024-09-10T21:24:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23cc1f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23cc1f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41008/0 (socket says 172.21.15.97:41008) 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 -- 172.21.15.97:0/810130159 learned_addr learned my addr 172.21.15.97:0/810130159 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d0e00640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23cc06ef10 0x7f23cc15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 -- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23cc06ef10 msgr2=0x7f23cc15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23cc06ef10 0x7f23cc15d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 -- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23cc06f370 msgr2=0x7f23cc15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23cc06f370 0x7f23cc15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 -- 172.21.15.97:0/810130159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23cc13d6d0 con 0x7f23b4004810 2024-09-10T21:24:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d1800640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23cc1f17f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f23c0001790 tx=0x7f23c000e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23c9600640 1 -- 172.21.15.97:0/810130159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0012850 con 0x7f23b4004810 2024-09-10T21:24:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23c9600640 1 -- 172.21.15.97:0/810130159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23c0012b00 con 0x7f23b4004810 2024-09-10T21:24:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23c9600640 1 -- 172.21.15.97:0/810130159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c001a400 con 0x7f23b4004810 2024-09-10T21:24:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23cc13d950 con 0x7f23b4004810 2024-09-10T21:24:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23cc13df10 con 0x7f23b4004810 2024-09-10T21:24:30.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.972+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23cc070670 con 0x7f23b4004810 2024-09-10T21:24:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.976+0000 7f23c9600640 1 -- 172.21.15.97:0/810130159 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f23c00113d0 con 0x7f23b4004810 2024-09-10T21:24:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.976+0000 7f23c9600640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23ac072610 0x7f23ac074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:30.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.976+0000 7f23c9600640 1 -- 172.21.15.97:0/810130159 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f23c0096a30 con 0x7f23b4004810 2024-09-10T21:24:30.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.976+0000 7f23d0e00640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23ac072610 0x7f23ac074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:30.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.976+0000 7f23d0e00640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23ac072610 0x7f23ac074ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23b80b0000 tx=0x7f23b80a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:30.980+0000 7f23c9600640 1 -- 172.21.15.97:0/810130159 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f23cc13d6d0 con 0x7f23b4004810 2024-09-10T21:24:31.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.184+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f23cc0b0150 con 0x7f23b4004810 2024-09-10T21:24:31.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23c9600640 1 -- 172.21.15.97:0/810130159 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f23c0016030 con 0x7f23b4004810 2024-09-10T21:24:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23ac072610 msgr2=0x7f23ac074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23ac072610 0x7f23ac074ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23b80b0000 tx=0x7f23b80a7040 comp rx=0 tx=0).stop 2024-09-10T21:24:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 msgr2=0x7f23cc1f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23cc1f17f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f23c0001790 tx=0x7f23c000e6b0 comp rx=0 tx=0).stop 2024-09-10T21:24:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 shutdown_connections 2024-09-10T21:24:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23ac072610 0x7f23ac074ad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23cc06f370 0x7f23cc15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23cc06ef10 0x7f23cc15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 --2- 172.21.15.97:0/810130159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23b4004810 0x7f23cc1f17f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.804+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 >> 172.21.15.97:0/810130159 conn(0x7f23cc0b1780 msgr2=0x7f23cc0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.808+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 shutdown_connections 2024-09-10T21:24:31.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.808+0000 7f23d4200640 1 -- 172.21.15.97:0/810130159 wait complete. 2024-09-10T21:24:31.836 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:24:31.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:31.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- 172.21.15.97:0/904497100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d58004810 msgr2=0x7f7d58004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:31.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 --2- 172.21.15.97:0/904497100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d58004810 0x7f7d58004c90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ba0 tx=0x7f7d5c01e410 comp rx=0 tx=0).stop 2024-09-10T21:24:31.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d72800640 1 -- 172.21.15.97:0/904497100 reap_dead start 2024-09-10T21:24:31.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- 172.21.15.97:0/904497100 shutdown_connections 2024-09-10T21:24:31.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- 172.21.15.97:0/904497100 >> 172.21.15.97:0/904497100 conn(0x7f7d740b1780 msgr2=0x7f7d740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- 172.21.15.97:0/904497100 shutdown_connections 2024-09-10T21:24:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- 172.21.15.97:0/904497100 wait complete. 2024-09-10T21:24:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 Processor -- start 2024-09-10T21:24:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- start start 2024-09-10T21:24:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d58004810 0x7f7d741f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d741f68b0 0x7f7d741e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d7414ff70 0x7f7d741503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7406f060 con 0x7f7d741f68b0 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7415b210 con 0x7f7d7414ff70 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d78e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7406f1e0 con 0x7f7d58004810 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d69400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d741f68b0 0x7f7d741e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d69400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d741f68b0 0x7f7d741e6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41032/0 (socket says 172.21.15.97:41032) 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d69400640 1 -- 172.21.15.97:0/1867291341 learned_addr learned my addr 172.21.15.97:0/1867291341 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d72800640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d7414ff70 0x7f7d741503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:31.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.976+0000 7f7d71e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d58004810 0x7f7d741f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d69400640 1 -- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d58004810 msgr2=0x7f7d741f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d69400640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d58004810 0x7f7d741f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d69400640 1 -- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d7414ff70 msgr2=0x7f7d741503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d69400640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d7414ff70 0x7f7d741503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d69400640 1 -- 172.21.15.97:0/1867291341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d741f2fb0 con 0x7f7d741f68b0 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d69400640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d741f68b0 0x7f7d741e6bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b0040 tx=0x7f7d5c0ade80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d6aa00640 1 -- 172.21.15.97:0/1867291341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0bd020 con 0x7f7d741f68b0 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d6aa00640 1 -- 172.21.15.97:0/1867291341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d5c0b3040 con 0x7f7d741f68b0 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d6aa00640 1 -- 172.21.15.97:0/1867291341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0b2050 con 0x7f7d741f68b0 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d741f3290 con 0x7f7d741f68b0 2024-09-10T21:24:31.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d741f3850 con 0x7f7d741f68b0 2024-09-10T21:24:31.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d740cf450 con 0x7f7d741f68b0 2024-09-10T21:24:31.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.980+0000 7f7d6aa00640 1 -- 172.21.15.97:0/1867291341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7d5c00f040 con 0x7f7d741f68b0 2024-09-10T21:24:31.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.984+0000 7f7d6aa00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7d50072610 0x7f7d50074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:31.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.984+0000 7f7d6aa00640 1 -- 172.21.15.97:0/1867291341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7d5c12e500 con 0x7f7d741f68b0 2024-09-10T21:24:31.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.984+0000 7f7d71e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7d50072610 0x7f7d50074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:31.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.984+0000 7f7d71e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7d50072610 0x7f7d50074ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d64004330 tx=0x7f7d64002200 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:31.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:31.988+0000 7f7d6aa00640 1 -- 172.21.15.97:0/1867291341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7d5c018160 con 0x7f7d741f68b0 2024-09-10T21:24:32.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.184+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d740b12a0 con 0x7f7d741f68b0 2024-09-10T21:24:32.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.184+0000 7f7d6aa00640 1 -- 172.21.15.97:0/1867291341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7f7d5c0f91c0 con 0x7f7d741f68b0 2024-09-10T21:24:32.192 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:32.192 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:32.192 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7d50072610 msgr2=0x7f7d50074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7d50072610 0x7f7d50074ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d64004330 tx=0x7f7d64002200 comp rx=0 tx=0).stop 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d741f68b0 msgr2=0x7f7d741e6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d741f68b0 0x7f7d741e6bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b0040 tx=0x7f7d5c0ade80 comp rx=0 tx=0).stop 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 shutdown_connections 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7d50072610 0x7f7d50074ad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d7414ff70 0x7f7d741503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d741f68b0 0x7f7d741e6bb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 --2- 172.21.15.97:0/1867291341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7d58004810 0x7f7d741f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 >> 172.21.15.97:0/1867291341 conn(0x7f7d740b1780 msgr2=0x7f7d740d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 shutdown_connections 2024-09-10T21:24:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.188+0000 7f7d78e00640 1 -- 172.21.15.97:0/1867291341 wait complete. 2024-09-10T21:24:32.214 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.348+0000 7fef78a00640 1 -- 172.21.15.97:0/2802232964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef58004d50 msgr2=0x7fef580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.348+0000 7fef78a00640 1 --2- 172.21.15.97:0/2802232964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef58004d50 0x7fef580051d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c670 tx=0x7fef6c01d300 comp rx=0 tx=0).stop 2024-09-10T21:24:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.348+0000 7fef72800640 1 -- 172.21.15.97:0/2802232964 reap_dead start 2024-09-10T21:24:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.348+0000 7fef78a00640 1 -- 172.21.15.97:0/2802232964 shutdown_connections 2024-09-10T21:24:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.348+0000 7fef78a00640 1 -- 172.21.15.97:0/2802232964 >> 172.21.15.97:0/2802232964 conn(0x7fef740cd680 msgr2=0x7fef740cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.348+0000 7fef78a00640 1 -- 172.21.15.97:0/2802232964 shutdown_connections 2024-09-10T21:24:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.348+0000 7fef78a00640 1 -- 172.21.15.97:0/2802232964 wait complete. 2024-09-10T21:24:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 Processor -- start 2024-09-10T21:24:32.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 -- start start 2024-09-10T21:24:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef58004d50 0x7fef7414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef741f42e0 0x7fef741f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef741f7e10 0x7fef740b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef74074880 con 0x7fef741f7e10 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fef74074700 con 0x7fef741f42e0 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fef74074a00 con 0x7fef58004d50 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef741f7e10 0x7fef740b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef741f7e10 0x7fef740b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41058/0 (socket says 172.21.15.97:41058) 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 -- 172.21.15.97:0/2924758878 learned_addr learned my addr 172.21.15.97:0/2924758878 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef71e00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef58004d50 0x7fef7414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef71400640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef741f42e0 0x7fef741f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 -- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef58004d50 msgr2=0x7fef7414c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef58004d50 0x7fef7414c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 -- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef741f42e0 msgr2=0x7fef741f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef741f42e0 0x7fef741f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 -- 172.21.15.97:0/2924758878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef740b5b90 con 0x7fef741f7e10 2024-09-10T21:24:32.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef72800640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef741f7e10 0x7fef740b4fb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef6c01dba0 tx=0x7fef6c006610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef6a000640 1 -- 172.21.15.97:0/2924758878 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0be070 con 0x7fef741f7e10 2024-09-10T21:24:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef6a000640 1 -- 172.21.15.97:0/2924758878 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef6c006900 con 0x7fef741f7e10 2024-09-10T21:24:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef6a000640 1 -- 172.21.15.97:0/2924758878 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0b2040 con 0x7fef741f7e10 2024-09-10T21:24:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef740b0db0 con 0x7fef741f7e10 2024-09-10T21:24:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.352+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef740b1370 con 0x7fef741f7e10 2024-09-10T21:24:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.356+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef740d07a0 con 0x7fef741f7e10 2024-09-10T21:24:32.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.356+0000 7fef6a000640 1 -- 172.21.15.97:0/2924758878 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fef6c003be0 con 0x7fef741f7e10 2024-09-10T21:24:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.356+0000 7fef6a000640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef54072610 0x7fef54074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.356+0000 7fef6a000640 1 -- 172.21.15.97:0/2924758878 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fef6c12d7a0 con 0x7fef741f7e10 2024-09-10T21:24:32.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.356+0000 7fef71e00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef54072610 0x7fef54074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.360+0000 7fef71e00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef54072610 0x7fef54074ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef64009c60 tx=0x7fef640091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.360+0000 7fef6a000640 1 -- 172.21.15.97:0/2924758878 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fef6c0170b0 con 0x7fef741f7e10 2024-09-10T21:24:32.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.528+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef740b1540 con 0x7fef741f7e10 2024-09-10T21:24:32.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.528+0000 7fef6a000640 1 -- 172.21.15.97:0/2924758878 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7fef6c0f8be0 con 0x7fef741f7e10 2024-09-10T21:24:32.535 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:32.535 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:32.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef54072610 msgr2=0x7fef54074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef54072610 0x7fef54074ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef64009c60 tx=0x7fef640091e0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef741f7e10 msgr2=0x7fef740b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef741f7e10 0x7fef740b4fb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef6c01dba0 tx=0x7fef6c006610 comp rx=0 tx=0).stop 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 shutdown_connections 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef54072610 0x7fef54074ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef741f7e10 0x7fef740b4fb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef741f42e0 0x7fef741f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 --2- 172.21.15.97:0/2924758878 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef58004d50 0x7fef7414c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 >> 172.21.15.97:0/2924758878 conn(0x7fef740cd680 msgr2=0x7fef7406f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 shutdown_connections 2024-09-10T21:24:32.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.532+0000 7fef78a00640 1 -- 172.21.15.97:0/2924758878 wait complete. 2024-09-10T21:24:32.557 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:24:32.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- 172.21.15.97:0/2972259319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 msgr2=0x7fe43c06ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 --2- 172.21.15.97:0/2972259319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c06ec40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe428009900 tx=0x7fe42801f090 comp rx=0 tx=0).stop 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- 172.21.15.97:0/2972259319 shutdown_connections 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 --2- 172.21.15.97:0/2972259319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe43c06f180 0x7fe43c0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 --2- 172.21.15.97:0/2972259319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c06ec40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 --2- 172.21.15.97:0/2972259319 >> v2:172.21.15.194:3300/0 conn(0x7fe43c1511f0 0x7fe43c0b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- 172.21.15.97:0/2972259319 >> 172.21.15.97:0/2972259319 conn(0x7fe43c0b1780 msgr2=0x7fe43c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- 172.21.15.97:0/2972259319 shutdown_connections 2024-09-10T21:24:32.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- 172.21.15.97:0/2972259319 wait complete. 2024-09-10T21:24:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 Processor -- start 2024-09-10T21:24:32.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- start start 2024-09-10T21:24:32.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe43c06f180 0x7fe43c14bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe43c1511f0 0x7fe43c14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c06e530 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c06e3b0 con 0x7fe43c06f180 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe441000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c06e6b0 con 0x7fe43c1511f0 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe439400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe439e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe43c06f180 0x7fe43c14bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe439400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c14f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41082/0 (socket says 172.21.15.97:41082) 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe439400640 1 -- 172.21.15.97:0/2390941897 learned_addr learned my addr 172.21.15.97:0/2390941897 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.692+0000 7fe43a800640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe43c1511f0 0x7fe43c14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe439400640 1 -- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe43c1511f0 msgr2=0x7fe43c14db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe439400640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe43c1511f0 0x7fe43c14db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe439400640 1 -- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe43c06f180 msgr2=0x7fe43c14bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe439400640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe43c06f180 0x7fe43c14bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe439400640 1 -- 172.21.15.97:0/2390941897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c14eb60 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe439400640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c14f920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe428009850 tx=0x7fe4280a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe432000640 1 -- 172.21.15.97:0/2390941897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280b5040 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe432000640 1 -- 172.21.15.97:0/2390941897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4280053e0 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c14ee40 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe432000640 1 -- 172.21.15.97:0/2390941897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280046d0 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c14f400 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c0ceee0 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.696+0000 7fe432000640 1 -- 172.21.15.97:0/2390941897 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe428004890 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.700+0000 7fe432000640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe41c0725e0 0x7fe41c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.700+0000 7fe432000640 1 -- 172.21.15.97:0/2390941897 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe428130740 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.700+0000 7fe439e00640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe41c0725e0 0x7fe41c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:32.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.700+0000 7fe439e00640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe41c0725e0 0x7fe41c074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00ab40 tx=0x7fe42c00a250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:32.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.704+0000 7fe432000640 1 -- 172.21.15.97:0/2390941897 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe428018bf0 con 0x7fe43c0b3cf0 2024-09-10T21:24:32.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.868+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe43c0cfe10 con 0x7fe41c0725e0 2024-09-10T21:24:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.876+0000 7fe432000640 1 -- 172.21.15.97:0/2390941897 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332543 (secure 0 0 0) 0x7fe43c0cfe10 con 0x7fe41c0725e0 2024-09-10T21:24:32.884 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe41c0725e0 msgr2=0x7fe41c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe41c0725e0 0x7fe41c074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00ab40 tx=0x7fe42c00a250 comp rx=0 tx=0).stop 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 msgr2=0x7fe43c14f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c14f920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe428009850 tx=0x7fe4280a8c60 comp rx=0 tx=0).stop 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 shutdown_connections 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe41c0725e0 0x7fe41c074aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe43c1511f0 0x7fe43c14db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe43c0b3cf0 0x7fe43c14f920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 --2- 172.21.15.97:0/2390941897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe43c06f180 0x7fe43c14bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 >> 172.21.15.97:0/2390941897 conn(0x7fe43c0b1780 msgr2=0x7fe43c0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 shutdown_connections 2024-09-10T21:24:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:32.880+0000 7fe441000640 1 -- 172.21.15.97:0/2390941897 wait complete. 2024-09-10T21:24:32.906 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-10T21:24:32.073235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003366"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264781+0000","last_change":"2024-09-10T21:24:10.717118+0000","last_active":"2024-09-10T21:24:17.264781+0000","last_peered":"2024-09-10T21:24:17.264781+0000","last_clean":"2024-09-10T21:24:17.264781+0000","last_became_active":"2024-09-10T21:24:10.716871+0000","last_became_peered":"2024-09-10T21:24:10.716871+0000","last_unstale":"2024-09-10T21:24:17.264781+0000","last_undegraded":"2024-09-10T21:24:17.264781+0000","last_fullsized":"2024-09-10T21:24:17.264781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:26.347318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264579+0000","last_change":"2024-09-10T21:24:11.758545+0000","last_active":"2024-09-10T21:24:17.264579+0000","last_peered":"2024-09-10T21:24:17.264579+0000","last_clean":"2024-09-10T21:24:17.264579+0000","last_became_active":"2024-09-10T21:24:11.758227+0000","last_became_peered":"2024-09-10T21:24:11.758227+0000","last_unstale":"2024-09-10T21:24:17.264579+0000","last_undegraded":"2024-09-10T21:24:17.264579+0000","last_fullsized":"2024-09-10T21:24:17.264579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768689+0000","last_change":"2024-09-10T21:24:15.784509+0000","last_active":"2024-09-10T21:24:16.768689+0000","last_peered":"2024-09-10T21:24:16.768689+0000","last_clean":"2024-09-10T21:24:16.768689+0000","last_became_active":"2024-09-10T21:24:15.777793+0000","last_became_peered":"2024-09-10T21:24:15.777793+0000","last_unstale":"2024-09-10T21:24:16.768689+0000","last_undegraded":"2024-09-10T21:24:16.768689+0000","last_fullsized":"2024-09-10T21:24:16.768689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:55.568506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264784+0000","last_change":"2024-09-10T21:24:10.717927+0000","last_active":"2024-09-10T21:24:17.264784+0000","last_peered":"2024-09-10T21:24:17.264784+0000","last_clean":"2024-09-10T21:24:17.264784+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:17.264784+0000","last_undegraded":"2024-09-10T21:24:17.264784+0000","last_fullsized":"2024-09-10T21:24:17.264784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265060+0000","last_change":"2024-09-10T21:24:11.750430+0000","last_active":"2024-09-10T21:24:17.265060+0000","last_peered":"2024-09-10T21:24:17.265060+0000","last_clean":"2024-09-10T21:24:17.265060+0000","last_became_active":"2024-09-10T21:24:11.749604+0000","last_became_peered":"2024-09-10T21:24:11.749604+0000","last_unstale":"2024-09-10T21:24:17.265060+0000","last_undegraded":"2024-09-10T21:24:17.265060+0000","last_fullsized":"2024-09-10T21:24:17.265060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262600+0000","last_change":"2024-09-10T21:24:15.775067+0000","last_active":"2024-09-10T21:24:17.262600+0000","last_peered":"2024-09-10T21:24:17.262600+0000","last_clean":"2024-09-10T21:24:17.262600+0000","last_became_active":"2024-09-10T21:24:15.773909+0000","last_became_peered":"2024-09-10T21:24:15.773909+0000","last_unstale":"2024-09-10T21:24:17.262600+0000","last_undegraded":"2024-09-10T21:24:17.262600+0000","last_fullsized":"2024-09-10T21:24:17.262600+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:09.297494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265037+0000","last_change":"2024-09-10T21:24:10.711527+0000","last_active":"2024-09-10T21:24:17.265037+0000","last_peered":"2024-09-10T21:24:17.265037+0000","last_clean":"2024-09-10T21:24:17.265037+0000","last_became_active":"2024-09-10T21:24:10.711276+0000","last_became_peered":"2024-09-10T21:24:10.711276+0000","last_unstale":"2024-09-10T21:24:17.265037+0000","last_undegraded":"2024-09-10T21:24:17.265037+0000","last_fullsized":"2024-09-10T21:24:17.265037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263749+0000","last_change":"2024-09-10T21:24:11.756415+0000","last_active":"2024-09-10T21:24:17.263749+0000","last_peered":"2024-09-10T21:24:17.263749+0000","last_clean":"2024-09-10T21:24:17.263749+0000","last_became_active":"2024-09-10T21:24:11.754620+0000","last_became_peered":"2024-09-10T21:24:11.754620+0000","last_unstale":"2024-09-10T21:24:17.263749+0000","last_undegraded":"2024-09-10T21:24:17.263749+0000","last_fullsized":"2024-09-10T21:24:17.263749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:48.122962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262734+0000","last_change":"2024-09-10T21:24:15.782482+0000","last_active":"2024-09-10T21:24:17.262734+0000","last_peered":"2024-09-10T21:24:17.262734+0000","last_clean":"2024-09-10T21:24:17.262734+0000","last_became_active":"2024-09-10T21:24:15.782107+0000","last_became_peered":"2024-09-10T21:24:15.782107+0000","last_unstale":"2024-09-10T21:24:17.262734+0000","last_undegraded":"2024-09-10T21:24:17.262734+0000","last_fullsized":"2024-09-10T21:24:17.262734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.906721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264925+0000","last_change":"2024-09-10T21:24:10.716898+0000","last_active":"2024-09-10T21:24:17.264925+0000","last_peered":"2024-09-10T21:24:17.264925+0000","last_clean":"2024-09-10T21:24:17.264925+0000","last_became_active":"2024-09-10T21:24:10.716644+0000","last_became_peered":"2024-09-10T21:24:10.716644+0000","last_unstale":"2024-09-10T21:24:17.264925+0000","last_undegraded":"2024-09-10T21:24:17.264925+0000","last_fullsized":"2024-09-10T21:24:17.264925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:41.682216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766338+0000","last_change":"2024-09-10T21:24:11.747195+0000","last_active":"2024-09-10T21:24:16.766338+0000","last_peered":"2024-09-10T21:24:16.766338+0000","last_clean":"2024-09-10T21:24:16.766338+0000","last_became_active":"2024-09-10T21:24:11.747038+0000","last_became_peered":"2024-09-10T21:24:11.747038+0000","last_unstale":"2024-09-10T21:24:16.766338+0000","last_undegraded":"2024-09-10T21:24:16.766338+0000","last_fullsized":"2024-09-10T21:24:16.766338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262860+0000","last_change":"2024-09-10T21:24:15.782490+0000","last_active":"2024-09-10T21:24:17.262860+0000","last_peered":"2024-09-10T21:24:17.262860+0000","last_clean":"2024-09-10T21:24:17.262860+0000","last_became_active":"2024-09-10T21:24:15.782079+0000","last_became_peered":"2024-09-10T21:24:15.782079+0000","last_unstale":"2024-09-10T21:24:17.262860+0000","last_undegraded":"2024-09-10T21:24:17.262860+0000","last_fullsized":"2024-09-10T21:24:17.262860+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:42.047447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765875+0000","last_change":"2024-09-10T21:24:11.734798+0000","last_active":"2024-09-10T21:24:16.765875+0000","last_peered":"2024-09-10T21:24:16.765875+0000","last_clean":"2024-09-10T21:24:16.765875+0000","last_became_active":"2024-09-10T21:24:11.734495+0000","last_became_peered":"2024-09-10T21:24:11.734495+0000","last_unstale":"2024-09-10T21:24:16.765875+0000","last_undegraded":"2024-09-10T21:24:16.765875+0000","last_fullsized":"2024-09-10T21:24:16.765875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:08.842495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766019+0000","last_change":"2024-09-10T21:24:10.718178+0000","last_active":"2024-09-10T21:24:16.766019+0000","last_peered":"2024-09-10T21:24:16.766019+0000","last_clean":"2024-09-10T21:24:16.766019+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:16.766019+0000","last_undegraded":"2024-09-10T21:24:16.766019+0000","last_fullsized":"2024-09-10T21:24:16.766019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:01.094807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766145+0000","last_change":"2024-09-10T21:24:15.786334+0000","last_active":"2024-09-10T21:24:16.766145+0000","last_peered":"2024-09-10T21:24:16.766145+0000","last_clean":"2024-09-10T21:24:16.766145+0000","last_became_active":"2024-09-10T21:24:15.785888+0000","last_became_peered":"2024-09-10T21:24:15.785888+0000","last_unstale":"2024-09-10T21:24:16.766145+0000","last_undegraded":"2024-09-10T21:24:16.766145+0000","last_fullsized":"2024-09-10T21:24:16.766145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:27.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769104+0000","last_change":"2024-09-10T21:24:11.743760+0000","last_active":"2024-09-10T21:24:16.769104+0000","last_peered":"2024-09-10T21:24:16.769104+0000","last_clean":"2024-09-10T21:24:16.769104+0000","last_became_active":"2024-09-10T21:24:11.743459+0000","last_became_peered":"2024-09-10T21:24:11.743459+0000","last_unstale":"2024-09-10T21:24:16.769104+0000","last_undegraded":"2024-09-10T21:24:16.769104+0000","last_fullsized":"2024-09-10T21:24:16.769104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:53.958469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770545+0000","last_change":"2024-09-10T21:24:10.719093+0000","last_active":"2024-09-10T21:24:16.770545+0000","last_peered":"2024-09-10T21:24:16.770545+0000","last_clean":"2024-09-10T21:24:16.770545+0000","last_became_active":"2024-09-10T21:24:10.715489+0000","last_became_peered":"2024-09-10T21:24:10.715489+0000","last_unstale":"2024-09-10T21:24:16.770545+0000","last_undegraded":"2024-09-10T21:24:16.770545+0000","last_fullsized":"2024-09-10T21:24:16.770545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264532+0000","last_change":"2024-09-10T21:24:15.777643+0000","last_active":"2024-09-10T21:24:17.264532+0000","last_peered":"2024-09-10T21:24:17.264532+0000","last_clean":"2024-09-10T21:24:17.264532+0000","last_became_active":"2024-09-10T21:24:15.777271+0000","last_became_peered":"2024-09-10T21:24:15.777271+0000","last_unstale":"2024-09-10T21:24:17.264532+0000","last_undegraded":"2024-09-10T21:24:17.264532+0000","last_fullsized":"2024-09-10T21:24:17.264532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:26.509163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263743+0000","last_change":"2024-09-10T21:24:11.724275+0000","last_active":"2024-09-10T21:24:17.263743+0000","last_peered":"2024-09-10T21:24:17.263743+0000","last_clean":"2024-09-10T21:24:17.263743+0000","last_became_active":"2024-09-10T21:24:11.723982+0000","last_became_peered":"2024-09-10T21:24:11.723982+0000","last_unstale":"2024-09-10T21:24:17.263743+0000","last_undegraded":"2024-09-10T21:24:17.263743+0000","last_fullsized":"2024-09-10T21:24:17.263743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:28.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264861+0000","last_change":"2024-09-10T21:24:10.712213+0000","last_active":"2024-09-10T21:24:17.264861+0000","last_peered":"2024-09-10T21:24:17.264861+0000","last_clean":"2024-09-10T21:24:17.264861+0000","last_became_active":"2024-09-10T21:24:10.711951+0000","last_became_peered":"2024-09-10T21:24:10.711951+0000","last_unstale":"2024-09-10T21:24:17.264861+0000","last_undegraded":"2024-09-10T21:24:17.264861+0000","last_fullsized":"2024-09-10T21:24:17.264861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:54.891884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770275+0000","last_change":"2024-09-10T21:24:15.780288+0000","last_active":"2024-09-10T21:24:16.770275+0000","last_peered":"2024-09-10T21:24:16.770275+0000","last_clean":"2024-09-10T21:24:16.770275+0000","last_became_active":"2024-09-10T21:24:15.778928+0000","last_became_peered":"2024-09-10T21:24:15.778928+0000","last_unstale":"2024-09-10T21:24:16.770275+0000","last_undegraded":"2024-09-10T21:24:16.770275+0000","last_fullsized":"2024-09-10T21:24:16.770275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:45.367503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765552+0000","last_change":"2024-09-10T21:24:11.730800+0000","last_active":"2024-09-10T21:24:16.765552+0000","last_peered":"2024-09-10T21:24:16.765552+0000","last_clean":"2024-09-10T21:24:16.765552+0000","last_became_active":"2024-09-10T21:24:11.730474+0000","last_became_peered":"2024-09-10T21:24:11.730474+0000","last_unstale":"2024-09-10T21:24:16.765552+0000","last_undegraded":"2024-09-10T21:24:16.765552+0000","last_fullsized":"2024-09-10T21:24:16.765552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263542+0000","last_change":"2024-09-10T21:24:10.712139+0000","last_active":"2024-09-10T21:24:17.263542+0000","last_peered":"2024-09-10T21:24:17.263542+0000","last_clean":"2024-09-10T21:24:17.263542+0000","last_became_active":"2024-09-10T21:24:10.710501+0000","last_became_peered":"2024-09-10T21:24:10.710501+0000","last_unstale":"2024-09-10T21:24:17.263542+0000","last_undegraded":"2024-09-10T21:24:17.263542+0000","last_fullsized":"2024-09-10T21:24:17.263542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263764+0000","last_change":"2024-09-10T21:24:15.770139+0000","last_active":"2024-09-10T21:24:17.263764+0000","last_peered":"2024-09-10T21:24:17.263764+0000","last_clean":"2024-09-10T21:24:17.263764+0000","last_became_active":"2024-09-10T21:24:15.769554+0000","last_became_peered":"2024-09-10T21:24:15.769554+0000","last_unstale":"2024-09-10T21:24:17.263764+0000","last_undegraded":"2024-09-10T21:24:17.263764+0000","last_fullsized":"2024-09-10T21:24:17.263764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:14.866707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266054+0000","last_change":"2024-09-10T21:24:11.732319+0000","last_active":"2024-09-10T21:24:17.266054+0000","last_peered":"2024-09-10T21:24:17.266054+0000","last_clean":"2024-09-10T21:24:17.266054+0000","last_became_active":"2024-09-10T21:24:11.732038+0000","last_became_peered":"2024-09-10T21:24:11.732038+0000","last_unstale":"2024-09-10T21:24:17.266054+0000","last_undegraded":"2024-09-10T21:24:17.266054+0000","last_fullsized":"2024-09-10T21:24:17.266054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:34.760961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766655+0000","last_change":"2024-09-10T21:24:10.720607+0000","last_active":"2024-09-10T21:24:16.766655+0000","last_peered":"2024-09-10T21:24:16.766655+0000","last_clean":"2024-09-10T21:24:16.766655+0000","last_became_active":"2024-09-10T21:24:10.720144+0000","last_became_peered":"2024-09-10T21:24:10.720144+0000","last_unstale":"2024-09-10T21:24:16.766655+0000","last_undegraded":"2024-09-10T21:24:16.766655+0000","last_fullsized":"2024-09-10T21:24:16.766655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:10.162599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263165+0000","last_change":"2024-09-10T21:24:15.777793+0000","last_active":"2024-09-10T21:24:17.263165+0000","last_peered":"2024-09-10T21:24:17.263165+0000","last_clean":"2024-09-10T21:24:17.263165+0000","last_became_active":"2024-09-10T21:24:15.777358+0000","last_became_peered":"2024-09-10T21:24:15.777358+0000","last_unstale":"2024-09-10T21:24:17.263165+0000","last_undegraded":"2024-09-10T21:24:17.263165+0000","last_fullsized":"2024-09-10T21:24:17.263165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:48.503713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264581+0000","last_change":"2024-09-10T21:24:11.735858+0000","last_active":"2024-09-10T21:24:17.264581+0000","last_peered":"2024-09-10T21:24:17.264581+0000","last_clean":"2024-09-10T21:24:17.264581+0000","last_became_active":"2024-09-10T21:24:11.735599+0000","last_became_peered":"2024-09-10T21:24:11.735599+0000","last_unstale":"2024-09-10T21:24:17.264581+0000","last_undegraded":"2024-09-10T21:24:17.264581+0000","last_fullsized":"2024-09-10T21:24:17.264581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:53.530511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266208+0000","last_change":"2024-09-10T21:24:10.712710+0000","last_active":"2024-09-10T21:24:17.266208+0000","last_peered":"2024-09-10T21:24:17.266208+0000","last_clean":"2024-09-10T21:24:17.266208+0000","last_became_active":"2024-09-10T21:24:10.712400+0000","last_became_peered":"2024-09-10T21:24:10.712400+0000","last_unstale":"2024-09-10T21:24:17.266208+0000","last_undegraded":"2024-09-10T21:24:17.266208+0000","last_fullsized":"2024-09-10T21:24:17.266208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:57.508268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263912+0000","last_change":"2024-09-10T21:24:15.776045+0000","last_active":"2024-09-10T21:24:17.263912+0000","last_peered":"2024-09-10T21:24:17.263912+0000","last_clean":"2024-09-10T21:24:17.263912+0000","last_became_active":"2024-09-10T21:24:15.775199+0000","last_became_peered":"2024-09-10T21:24:15.775199+0000","last_unstale":"2024-09-10T21:24:17.263912+0000","last_undegraded":"2024-09-10T21:24:17.263912+0000","last_fullsized":"2024-09-10T21:24:17.263912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.516796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770569+0000","last_change":"2024-09-10T21:24:11.737947+0000","last_active":"2024-09-10T21:24:16.770569+0000","last_peered":"2024-09-10T21:24:16.770569+0000","last_clean":"2024-09-10T21:24:16.770569+0000","last_became_active":"2024-09-10T21:24:11.735572+0000","last_became_peered":"2024-09-10T21:24:11.735572+0000","last_unstale":"2024-09-10T21:24:16.770569+0000","last_undegraded":"2024-09-10T21:24:16.770569+0000","last_fullsized":"2024-09-10T21:24:16.770569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:41.353969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766677+0000","last_change":"2024-09-10T21:24:10.712429+0000","last_active":"2024-09-10T21:24:16.766677+0000","last_peered":"2024-09-10T21:24:16.766677+0000","last_clean":"2024-09-10T21:24:16.766677+0000","last_became_active":"2024-09-10T21:24:10.712150+0000","last_became_peered":"2024-09-10T21:24:10.712150+0000","last_unstale":"2024-09-10T21:24:16.766677+0000","last_undegraded":"2024-09-10T21:24:16.766677+0000","last_fullsized":"2024-09-10T21:24:16.766677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:37.565989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769802+0000","last_change":"2024-09-10T21:24:15.778516+0000","last_active":"2024-09-10T21:24:16.769802+0000","last_peered":"2024-09-10T21:24:16.769802+0000","last_clean":"2024-09-10T21:24:16.769802+0000","last_became_active":"2024-09-10T21:24:15.778061+0000","last_became_peered":"2024-09-10T21:24:15.778061+0000","last_unstale":"2024-09-10T21:24:16.769802+0000","last_undegraded":"2024-09-10T21:24:16.769802+0000","last_fullsized":"2024-09-10T21:24:16.769802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:38.186163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263379+0000","last_change":"2024-09-10T21:24:11.747693+0000","last_active":"2024-09-10T21:24:17.263379+0000","last_peered":"2024-09-10T21:24:17.263379+0000","last_clean":"2024-09-10T21:24:17.263379+0000","last_became_active":"2024-09-10T21:24:11.747405+0000","last_became_peered":"2024-09-10T21:24:11.747405+0000","last_unstale":"2024-09-10T21:24:17.263379+0000","last_undegraded":"2024-09-10T21:24:17.263379+0000","last_fullsized":"2024-09-10T21:24:17.263379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:42.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264255+0000","last_change":"2024-09-10T21:24:10.711258+0000","last_active":"2024-09-10T21:24:17.264255+0000","last_peered":"2024-09-10T21:24:17.264255+0000","last_clean":"2024-09-10T21:24:17.264255+0000","last_became_active":"2024-09-10T21:24:10.710892+0000","last_became_peered":"2024-09-10T21:24:10.710892+0000","last_unstale":"2024-09-10T21:24:17.264255+0000","last_undegraded":"2024-09-10T21:24:17.264255+0000","last_fullsized":"2024-09-10T21:24:17.264255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769471+0000","last_change":"2024-09-10T21:24:15.786836+0000","last_active":"2024-09-10T21:24:16.769471+0000","last_peered":"2024-09-10T21:24:16.769471+0000","last_clean":"2024-09-10T21:24:16.769471+0000","last_became_active":"2024-09-10T21:24:15.786459+0000","last_became_peered":"2024-09-10T21:24:15.786459+0000","last_unstale":"2024-09-10T21:24:16.769471+0000","last_undegraded":"2024-09-10T21:24:16.769471+0000","last_fullsized":"2024-09-10T21:24:16.769471+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:49.037577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264516+0000","last_change":"2024-09-10T21:24:11.756779+0000","last_active":"2024-09-10T21:24:17.264516+0000","last_peered":"2024-09-10T21:24:17.264516+0000","last_clean":"2024-09-10T21:24:17.264516+0000","last_became_active":"2024-09-10T21:24:11.756493+0000","last_became_peered":"2024-09-10T21:24:11.756493+0000","last_unstale":"2024-09-10T21:24:17.264516+0000","last_undegraded":"2024-09-10T21:24:17.264516+0000","last_fullsized":"2024-09-10T21:24:17.264516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766053+0000","last_change":"2024-09-10T21:24:10.707060+0000","last_active":"2024-09-10T21:24:16.766053+0000","last_peered":"2024-09-10T21:24:16.766053+0000","last_clean":"2024-09-10T21:24:16.766053+0000","last_became_active":"2024-09-10T21:24:10.706798+0000","last_became_peered":"2024-09-10T21:24:10.706798+0000","last_unstale":"2024-09-10T21:24:16.766053+0000","last_undegraded":"2024-09-10T21:24:16.766053+0000","last_fullsized":"2024-09-10T21:24:16.766053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:47.120434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770236+0000","last_change":"2024-09-10T21:24:15.781040+0000","last_active":"2024-09-10T21:24:16.770236+0000","last_peered":"2024-09-10T21:24:16.770236+0000","last_clean":"2024-09-10T21:24:16.770236+0000","last_became_active":"2024-09-10T21:24:15.780677+0000","last_became_peered":"2024-09-10T21:24:15.780677+0000","last_unstale":"2024-09-10T21:24:16.770236+0000","last_undegraded":"2024-09-10T21:24:16.770236+0000","last_fullsized":"2024-09-10T21:24:16.770236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.899193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263779+0000","last_change":"2024-09-10T21:24:11.753935+0000","last_active":"2024-09-10T21:24:17.263779+0000","last_peered":"2024-09-10T21:24:17.263779+0000","last_clean":"2024-09-10T21:24:17.263779+0000","last_became_active":"2024-09-10T21:24:11.753134+0000","last_became_peered":"2024-09-10T21:24:11.753134+0000","last_unstale":"2024-09-10T21:24:17.263779+0000","last_undegraded":"2024-09-10T21:24:17.263779+0000","last_fullsized":"2024-09-10T21:24:17.263779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:41.306076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769295+0000","last_change":"2024-09-10T21:24:10.718908+0000","last_active":"2024-09-10T21:24:16.769295+0000","last_peered":"2024-09-10T21:24:16.769295+0000","last_clean":"2024-09-10T21:24:16.769295+0000","last_became_active":"2024-09-10T21:24:10.718142+0000","last_became_peered":"2024-09-10T21:24:10.718142+0000","last_unstale":"2024-09-10T21:24:16.769295+0000","last_undegraded":"2024-09-10T21:24:16.769295+0000","last_fullsized":"2024-09-10T21:24:16.769295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:56.946441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769872+0000","last_change":"2024-09-10T21:24:15.778921+0000","last_active":"2024-09-10T21:24:16.769872+0000","last_peered":"2024-09-10T21:24:16.769872+0000","last_clean":"2024-09-10T21:24:16.769872+0000","last_became_active":"2024-09-10T21:24:15.778360+0000","last_became_peered":"2024-09-10T21:24:15.778360+0000","last_unstale":"2024-09-10T21:24:16.769872+0000","last_undegraded":"2024-09-10T21:24:16.769872+0000","last_fullsized":"2024-09-10T21:24:16.769872+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:10.163639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768895+0000","last_change":"2024-09-10T21:24:11.758622+0000","last_active":"2024-09-10T21:24:16.768895+0000","last_peered":"2024-09-10T21:24:16.768895+0000","last_clean":"2024-09-10T21:24:16.768895+0000","last_became_active":"2024-09-10T21:24:11.758363+0000","last_became_peered":"2024-09-10T21:24:11.758363+0000","last_unstale":"2024-09-10T21:24:16.768895+0000","last_undegraded":"2024-09-10T21:24:16.768895+0000","last_fullsized":"2024-09-10T21:24:16.768895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765739+0000","last_change":"2024-09-10T21:24:10.720746+0000","last_active":"2024-09-10T21:24:16.765739+0000","last_peered":"2024-09-10T21:24:16.765739+0000","last_clean":"2024-09-10T21:24:16.765739+0000","last_became_active":"2024-09-10T21:24:10.720360+0000","last_became_peered":"2024-09-10T21:24:10.720360+0000","last_unstale":"2024-09-10T21:24:16.765739+0000","last_undegraded":"2024-09-10T21:24:16.765739+0000","last_fullsized":"2024-09-10T21:24:16.765739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:25.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264322+0000","last_change":"2024-09-10T21:24:15.787994+0000","last_active":"2024-09-10T21:24:17.264322+0000","last_peered":"2024-09-10T21:24:17.264322+0000","last_clean":"2024-09-10T21:24:17.264322+0000","last_became_active":"2024-09-10T21:24:15.787532+0000","last_became_peered":"2024-09-10T21:24:15.787532+0000","last_unstale":"2024-09-10T21:24:17.264322+0000","last_undegraded":"2024-09-10T21:24:17.264322+0000","last_fullsized":"2024-09-10T21:24:17.264322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:12.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265574+0000","last_change":"2024-09-10T21:24:11.747694+0000","last_active":"2024-09-10T21:24:17.265574+0000","last_peered":"2024-09-10T21:24:17.265574+0000","last_clean":"2024-09-10T21:24:17.265574+0000","last_became_active":"2024-09-10T21:24:11.746819+0000","last_became_peered":"2024-09-10T21:24:11.746819+0000","last_unstale":"2024-09-10T21:24:17.265574+0000","last_undegraded":"2024-09-10T21:24:17.265574+0000","last_fullsized":"2024-09-10T21:24:17.265574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:14.637932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766139+0000","last_change":"2024-09-10T21:24:10.708595+0000","last_active":"2024-09-10T21:24:16.766139+0000","last_peered":"2024-09-10T21:24:16.766139+0000","last_clean":"2024-09-10T21:24:16.766139+0000","last_became_active":"2024-09-10T21:24:10.708315+0000","last_became_peered":"2024-09-10T21:24:10.708315+0000","last_unstale":"2024-09-10T21:24:16.766139+0000","last_undegraded":"2024-09-10T21:24:16.766139+0000","last_fullsized":"2024-09-10T21:24:16.766139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:39.966992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262962+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:24:17.262962+0000","last_peered":"2024-09-10T21:24:17.262962+0000","last_clean":"2024-09-10T21:24:17.262962+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:24:17.262962+0000","last_undegraded":"2024-09-10T21:24:17.262962+0000","last_fullsized":"2024-09-10T21:24:17.262962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766659+0000","last_change":"2024-09-10T21:24:15.771747+0000","last_active":"2024-09-10T21:24:16.766659+0000","last_peered":"2024-09-10T21:24:16.766659+0000","last_clean":"2024-09-10T21:24:16.766659+0000","last_became_active":"2024-09-10T21:24:15.771197+0000","last_became_peered":"2024-09-10T21:24:15.771197+0000","last_unstale":"2024-09-10T21:24:16.766659+0000","last_undegraded":"2024-09-10T21:24:16.766659+0000","last_fullsized":"2024-09-10T21:24:16.766659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:05.770255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766468+0000","last_change":"2024-09-10T21:24:11.740573+0000","last_active":"2024-09-10T21:24:16.766468+0000","last_peered":"2024-09-10T21:24:16.766468+0000","last_clean":"2024-09-10T21:24:16.766468+0000","last_became_active":"2024-09-10T21:24:11.738994+0000","last_became_peered":"2024-09-10T21:24:11.738994+0000","last_unstale":"2024-09-10T21:24:16.766468+0000","last_undegraded":"2024-09-10T21:24:16.766468+0000","last_fullsized":"2024-09-10T21:24:16.766468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:55.184362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770636+0000","last_change":"2024-09-10T21:24:10.721767+0000","last_active":"2024-09-10T21:24:16.770636+0000","last_peered":"2024-09-10T21:24:16.770636+0000","last_clean":"2024-09-10T21:24:16.770636+0000","last_became_active":"2024-09-10T21:24:10.721444+0000","last_became_peered":"2024-09-10T21:24:10.721444+0000","last_unstale":"2024-09-10T21:24:16.770636+0000","last_undegraded":"2024-09-10T21:24:16.770636+0000","last_fullsized":"2024-09-10T21:24:16.770636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:38.215394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263060+0000","last_change":"2024-09-10T21:24:15.770594+0000","last_active":"2024-09-10T21:24:17.263060+0000","last_peered":"2024-09-10T21:24:17.263060+0000","last_clean":"2024-09-10T21:24:17.263060+0000","last_became_active":"2024-09-10T21:24:15.769704+0000","last_became_peered":"2024-09-10T21:24:15.769704+0000","last_unstale":"2024-09-10T21:24:17.263060+0000","last_undegraded":"2024-09-10T21:24:17.263060+0000","last_fullsized":"2024-09-10T21:24:17.263060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:56.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264863+0000","last_change":"2024-09-10T21:24:11.743773+0000","last_active":"2024-09-10T21:24:17.264863+0000","last_peered":"2024-09-10T21:24:17.264863+0000","last_clean":"2024-09-10T21:24:17.264863+0000","last_became_active":"2024-09-10T21:24:11.743497+0000","last_became_peered":"2024-09-10T21:24:11.743497+0000","last_unstale":"2024-09-10T21:24:17.264863+0000","last_undegraded":"2024-09-10T21:24:17.264863+0000","last_fullsized":"2024-09-10T21:24:17.264863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265338+0000","last_change":"2024-09-10T21:24:10.716493+0000","last_active":"2024-09-10T21:24:17.265338+0000","last_peered":"2024-09-10T21:24:17.265338+0000","last_clean":"2024-09-10T21:24:17.265338+0000","last_became_active":"2024-09-10T21:24:10.716126+0000","last_became_peered":"2024-09-10T21:24:10.716126+0000","last_unstale":"2024-09-10T21:24:17.265338+0000","last_undegraded":"2024-09-10T21:24:17.265338+0000","last_fullsized":"2024-09-10T21:24:17.265338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264298+0000","last_change":"2024-09-10T21:24:15.791234+0000","last_active":"2024-09-10T21:24:17.264298+0000","last_peered":"2024-09-10T21:24:17.264298+0000","last_clean":"2024-09-10T21:24:17.264298+0000","last_became_active":"2024-09-10T21:24:15.790862+0000","last_became_peered":"2024-09-10T21:24:15.790862+0000","last_unstale":"2024-09-10T21:24:17.264298+0000","last_undegraded":"2024-09-10T21:24:17.264298+0000","last_fullsized":"2024-09-10T21:24:17.264298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:41.522844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265147+0000","last_change":"2024-09-10T21:24:11.742947+0000","last_active":"2024-09-10T21:24:17.265147+0000","last_peered":"2024-09-10T21:24:17.265147+0000","last_clean":"2024-09-10T21:24:17.265147+0000","last_became_active":"2024-09-10T21:24:11.741967+0000","last_became_peered":"2024-09-10T21:24:11.741967+0000","last_unstale":"2024-09-10T21:24:17.265147+0000","last_undegraded":"2024-09-10T21:24:17.265147+0000","last_fullsized":"2024-09-10T21:24:17.265147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266168+0000","last_change":"2024-09-10T21:24:10.715721+0000","last_active":"2024-09-10T21:24:17.266168+0000","last_peered":"2024-09-10T21:24:17.266168+0000","last_clean":"2024-09-10T21:24:17.266168+0000","last_became_active":"2024-09-10T21:24:10.715385+0000","last_became_peered":"2024-09-10T21:24:10.715385+0000","last_unstale":"2024-09-10T21:24:17.266168+0000","last_undegraded":"2024-09-10T21:24:17.266168+0000","last_fullsized":"2024-09-10T21:24:17.266168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:30.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765435+0000","last_change":"2024-09-10T21:24:15.770313+0000","last_active":"2024-09-10T21:24:16.765435+0000","last_peered":"2024-09-10T21:24:16.765435+0000","last_clean":"2024-09-10T21:24:16.765435+0000","last_became_active":"2024-09-10T21:24:15.769934+0000","last_became_peered":"2024-09-10T21:24:15.769934+0000","last_unstale":"2024-09-10T21:24:16.765435+0000","last_undegraded":"2024-09-10T21:24:16.765435+0000","last_fullsized":"2024-09-10T21:24:16.765435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:58.034220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264104+0000","last_change":"2024-09-10T21:24:11.752428+0000","last_active":"2024-09-10T21:24:17.264104+0000","last_peered":"2024-09-10T21:24:17.264104+0000","last_clean":"2024-09-10T21:24:17.264104+0000","last_became_active":"2024-09-10T21:24:11.748165+0000","last_became_peered":"2024-09-10T21:24:11.748165+0000","last_unstale":"2024-09-10T21:24:17.264104+0000","last_undegraded":"2024-09-10T21:24:17.264104+0000","last_fullsized":"2024-09-10T21:24:17.264104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265190+0000","last_change":"2024-09-10T21:24:10.717653+0000","last_active":"2024-09-10T21:24:17.265190+0000","last_peered":"2024-09-10T21:24:17.265190+0000","last_clean":"2024-09-10T21:24:17.265190+0000","last_became_active":"2024-09-10T21:24:10.715137+0000","last_became_peered":"2024-09-10T21:24:10.715137+0000","last_unstale":"2024-09-10T21:24:17.265190+0000","last_undegraded":"2024-09-10T21:24:17.265190+0000","last_fullsized":"2024-09-10T21:24:17.265190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:01.438989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262702+0000","last_change":"2024-09-10T21:24:15.782725+0000","last_active":"2024-09-10T21:24:17.262702+0000","last_peered":"2024-09-10T21:24:17.262702+0000","last_clean":"2024-09-10T21:24:17.262702+0000","last_became_active":"2024-09-10T21:24:15.782199+0000","last_became_peered":"2024-09-10T21:24:15.782199+0000","last_unstale":"2024-09-10T21:24:17.262702+0000","last_undegraded":"2024-09-10T21:24:17.262702+0000","last_fullsized":"2024-09-10T21:24:17.262702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:15.226555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765554+0000","last_change":"2024-09-10T21:24:11.757400+0000","last_active":"2024-09-10T21:24:16.765554+0000","last_peered":"2024-09-10T21:24:16.765554+0000","last_clean":"2024-09-10T21:24:16.765554+0000","last_became_active":"2024-09-10T21:24:11.757112+0000","last_became_peered":"2024-09-10T21:24:11.757112+0000","last_unstale":"2024-09-10T21:24:16.765554+0000","last_undegraded":"2024-09-10T21:24:16.765554+0000","last_fullsized":"2024-09-10T21:24:16.765554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263864+0000","last_change":"2024-09-10T21:24:10.701950+0000","last_active":"2024-09-10T21:24:17.263864+0000","last_peered":"2024-09-10T21:24:17.263864+0000","last_clean":"2024-09-10T21:24:17.263864+0000","last_became_active":"2024-09-10T21:24:10.701652+0000","last_became_peered":"2024-09-10T21:24:10.701652+0000","last_unstale":"2024-09-10T21:24:17.263864+0000","last_undegraded":"2024-09-10T21:24:17.263864+0000","last_fullsized":"2024-09-10T21:24:17.263864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.595699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266111+0000","last_change":"2024-09-10T21:24:15.784288+0000","last_active":"2024-09-10T21:24:17.266111+0000","last_peered":"2024-09-10T21:24:17.266111+0000","last_clean":"2024-09-10T21:24:17.266111+0000","last_became_active":"2024-09-10T21:24:15.783890+0000","last_became_peered":"2024-09-10T21:24:15.783890+0000","last_unstale":"2024-09-10T21:24:17.266111+0000","last_undegraded":"2024-09-10T21:24:17.266111+0000","last_fullsized":"2024-09-10T21:24:17.266111+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:59.245790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262943+0000","last_change":"2024-09-10T21:24:11.736914+0000","last_active":"2024-09-10T21:24:17.262943+0000","last_peered":"2024-09-10T21:24:17.262943+0000","last_clean":"2024-09-10T21:24:17.262943+0000","last_became_active":"2024-09-10T21:24:11.734099+0000","last_became_peered":"2024-09-10T21:24:11.734099+0000","last_unstale":"2024-09-10T21:24:17.262943+0000","last_undegraded":"2024-09-10T21:24:17.262943+0000","last_fullsized":"2024-09-10T21:24:17.262943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264000+0000","last_change":"2024-09-10T21:24:10.715530+0000","last_active":"2024-09-10T21:24:17.264000+0000","last_peered":"2024-09-10T21:24:17.264000+0000","last_clean":"2024-09-10T21:24:17.264000+0000","last_became_active":"2024-09-10T21:24:10.715235+0000","last_became_peered":"2024-09-10T21:24:10.715235+0000","last_unstale":"2024-09-10T21:24:17.264000+0000","last_undegraded":"2024-09-10T21:24:17.264000+0000","last_fullsized":"2024-09-10T21:24:17.264000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:12.286087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264900+0000","last_change":"2024-09-10T21:24:15.784572+0000","last_active":"2024-09-10T21:24:17.264900+0000","last_peered":"2024-09-10T21:24:17.264900+0000","last_clean":"2024-09-10T21:24:17.264900+0000","last_became_active":"2024-09-10T21:24:15.784136+0000","last_became_peered":"2024-09-10T21:24:15.784136+0000","last_unstale":"2024-09-10T21:24:17.264900+0000","last_undegraded":"2024-09-10T21:24:17.264900+0000","last_fullsized":"2024-09-10T21:24:17.264900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:12.320754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765877+0000","last_change":"2024-09-10T21:24:11.755458+0000","last_active":"2024-09-10T21:24:16.765877+0000","last_peered":"2024-09-10T21:24:16.765877+0000","last_clean":"2024-09-10T21:24:16.765877+0000","last_became_active":"2024-09-10T21:24:11.755150+0000","last_became_peered":"2024-09-10T21:24:11.755150+0000","last_unstale":"2024-09-10T21:24:16.765877+0000","last_undegraded":"2024-09-10T21:24:16.765877+0000","last_fullsized":"2024-09-10T21:24:16.765877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:34.505560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765994+0000","last_change":"2024-09-10T21:24:10.717709+0000","last_active":"2024-09-10T21:24:16.765994+0000","last_peered":"2024-09-10T21:24:16.765994+0000","last_clean":"2024-09-10T21:24:16.765994+0000","last_became_active":"2024-09-10T21:24:10.717328+0000","last_became_peered":"2024-09-10T21:24:10.717328+0000","last_unstale":"2024-09-10T21:24:16.765994+0000","last_undegraded":"2024-09-10T21:24:16.765994+0000","last_fullsized":"2024-09-10T21:24:16.765994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:37.792691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770443+0000","last_change":"2024-09-10T21:24:15.788432+0000","last_active":"2024-09-10T21:24:16.770443+0000","last_peered":"2024-09-10T21:24:16.770443+0000","last_clean":"2024-09-10T21:24:16.770443+0000","last_became_active":"2024-09-10T21:24:15.788046+0000","last_became_peered":"2024-09-10T21:24:15.788046+0000","last_unstale":"2024-09-10T21:24:16.770443+0000","last_undegraded":"2024-09-10T21:24:16.770443+0000","last_fullsized":"2024-09-10T21:24:16.770443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:04.277488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769322+0000","last_change":"2024-09-10T21:24:11.737027+0000","last_active":"2024-09-10T21:24:16.769322+0000","last_peered":"2024-09-10T21:24:16.769322+0000","last_clean":"2024-09-10T21:24:16.769322+0000","last_became_active":"2024-09-10T21:24:11.736731+0000","last_became_peered":"2024-09-10T21:24:11.736731+0000","last_unstale":"2024-09-10T21:24:16.769322+0000","last_undegraded":"2024-09-10T21:24:16.769322+0000","last_fullsized":"2024-09-10T21:24:16.769322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769804+0000","last_change":"2024-09-10T21:24:10.712131+0000","last_active":"2024-09-10T21:24:16.769804+0000","last_peered":"2024-09-10T21:24:16.769804+0000","last_clean":"2024-09-10T21:24:16.769804+0000","last_became_active":"2024-09-10T21:24:10.711804+0000","last_became_peered":"2024-09-10T21:24:10.711804+0000","last_unstale":"2024-09-10T21:24:16.769804+0000","last_undegraded":"2024-09-10T21:24:16.769804+0000","last_fullsized":"2024-09-10T21:24:16.769804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766399+0000","last_change":"2024-09-10T21:24:15.786086+0000","last_active":"2024-09-10T21:24:16.766399+0000","last_peered":"2024-09-10T21:24:16.766399+0000","last_clean":"2024-09-10T21:24:16.766399+0000","last_became_active":"2024-09-10T21:24:15.785449+0000","last_became_peered":"2024-09-10T21:24:15.785449+0000","last_unstale":"2024-09-10T21:24:16.766399+0000","last_undegraded":"2024-09-10T21:24:16.766399+0000","last_fullsized":"2024-09-10T21:24:16.766399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:51.733023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.740413+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.740134+0000","last_became_peered":"2024-09-10T21:24:11.740134+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:22.624004+0000","last_change":"2024-09-10T21:24:10.707022+0000","last_active":"2024-09-10T21:24:22.624004+0000","last_peered":"2024-09-10T21:24:22.624004+0000","last_clean":"2024-09-10T21:24:22.624004+0000","last_became_active":"2024-09-10T21:24:10.706584+0000","last_became_peered":"2024-09-10T21:24:10.706584+0000","last_unstale":"2024-09-10T21:24:22.624004+0000","last_undegraded":"2024-09-10T21:24:22.624004+0000","last_fullsized":"2024-09-10T21:24:22.624004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:08.525705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265647+0000","last_change":"2024-09-10T21:24:15.789501+0000","last_active":"2024-09-10T21:24:17.265647+0000","last_peered":"2024-09-10T21:24:17.265647+0000","last_clean":"2024-09-10T21:24:17.265647+0000","last_became_active":"2024-09-10T21:24:15.789076+0000","last_became_peered":"2024-09-10T21:24:15.789076+0000","last_unstale":"2024-09-10T21:24:17.265647+0000","last_undegraded":"2024-09-10T21:24:17.265647+0000","last_fullsized":"2024-09-10T21:24:17.265647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:58.299617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766239+0000","last_change":"2024-09-10T21:24:11.757417+0000","last_active":"2024-09-10T21:24:16.766239+0000","last_peered":"2024-09-10T21:24:16.766239+0000","last_clean":"2024-09-10T21:24:16.766239+0000","last_became_active":"2024-09-10T21:24:11.757091+0000","last_became_peered":"2024-09-10T21:24:11.757091+0000","last_unstale":"2024-09-10T21:24:16.766239+0000","last_undegraded":"2024-09-10T21:24:16.766239+0000","last_fullsized":"2024-09-10T21:24:16.766239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:13.080850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770386+0000","last_change":"2024-09-10T21:24:10.707124+0000","last_active":"2024-09-10T21:24:16.770386+0000","last_peered":"2024-09-10T21:24:16.770386+0000","last_clean":"2024-09-10T21:24:16.770386+0000","last_became_active":"2024-09-10T21:24:10.706605+0000","last_became_peered":"2024-09-10T21:24:10.706605+0000","last_unstale":"2024-09-10T21:24:16.770386+0000","last_undegraded":"2024-09-10T21:24:16.770386+0000","last_fullsized":"2024-09-10T21:24:16.770386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262946+0000","last_change":"2024-09-10T21:24:15.776743+0000","last_active":"2024-09-10T21:24:17.262946+0000","last_peered":"2024-09-10T21:24:17.262946+0000","last_clean":"2024-09-10T21:24:17.262946+0000","last_became_active":"2024-09-10T21:24:15.774992+0000","last_became_peered":"2024-09-10T21:24:15.774992+0000","last_unstale":"2024-09-10T21:24:17.262946+0000","last_undegraded":"2024-09-10T21:24:17.262946+0000","last_fullsized":"2024-09-10T21:24:17.262946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:11.593663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265153+0000","last_change":"2024-09-10T21:24:11.750107+0000","last_active":"2024-09-10T21:24:17.265153+0000","last_peered":"2024-09-10T21:24:17.265153+0000","last_clean":"2024-09-10T21:24:17.265153+0000","last_became_active":"2024-09-10T21:24:11.749206+0000","last_became_peered":"2024-09-10T21:24:11.749206+0000","last_unstale":"2024-09-10T21:24:17.265153+0000","last_undegraded":"2024-09-10T21:24:17.265153+0000","last_fullsized":"2024-09-10T21:24:17.265153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265042+0000","last_change":"2024-09-10T21:24:10.711029+0000","last_active":"2024-09-10T21:24:17.265042+0000","last_peered":"2024-09-10T21:24:17.265042+0000","last_clean":"2024-09-10T21:24:17.265042+0000","last_became_active":"2024-09-10T21:24:10.710789+0000","last_became_peered":"2024-09-10T21:24:10.710789+0000","last_unstale":"2024-09-10T21:24:17.265042+0000","last_undegraded":"2024-09-10T21:24:17.265042+0000","last_fullsized":"2024-09-10T21:24:17.265042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:36.260277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768675+0000","last_change":"2024-09-10T21:24:15.779017+0000","last_active":"2024-09-10T21:24:16.768675+0000","last_peered":"2024-09-10T21:24:16.768675+0000","last_clean":"2024-09-10T21:24:16.768675+0000","last_became_active":"2024-09-10T21:24:15.774452+0000","last_became_peered":"2024-09-10T21:24:15.774452+0000","last_unstale":"2024-09-10T21:24:16.768675+0000","last_undegraded":"2024-09-10T21:24:16.768675+0000","last_fullsized":"2024-09-10T21:24:16.768675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:56.032301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264021+0000","last_change":"2024-09-10T21:24:11.748901+0000","last_active":"2024-09-10T21:24:17.264021+0000","last_peered":"2024-09-10T21:24:17.264021+0000","last_clean":"2024-09-10T21:24:17.264021+0000","last_became_active":"2024-09-10T21:24:11.748059+0000","last_became_peered":"2024-09-10T21:24:11.748059+0000","last_unstale":"2024-09-10T21:24:17.264021+0000","last_undegraded":"2024-09-10T21:24:17.264021+0000","last_fullsized":"2024-09-10T21:24:17.264021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:10.697344+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:10.697030+0000","last_became_peered":"2024-09-10T21:24:10.697030+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264496+0000","last_change":"2024-09-10T21:24:15.775122+0000","last_active":"2024-09-10T21:24:17.264496+0000","last_peered":"2024-09-10T21:24:17.264496+0000","last_clean":"2024-09-10T21:24:17.264496+0000","last_became_active":"2024-09-10T21:24:15.774561+0000","last_became_peered":"2024-09-10T21:24:15.774561+0000","last_unstale":"2024-09-10T21:24:17.264496+0000","last_undegraded":"2024-09-10T21:24:17.264496+0000","last_fullsized":"2024-09-10T21:24:17.264496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:19.897395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266159+0000","last_change":"2024-09-10T21:24:15.790852+0000","last_active":"2024-09-10T21:24:17.266159+0000","last_peered":"2024-09-10T21:24:17.266159+0000","last_clean":"2024-09-10T21:24:17.266159+0000","last_became_active":"2024-09-10T21:24:15.786896+0000","last_became_peered":"2024-09-10T21:24:15.786896+0000","last_unstale":"2024-09-10T21:24:17.266159+0000","last_undegraded":"2024-09-10T21:24:17.266159+0000","last_fullsized":"2024-09-10T21:24:17.266159+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:48.900914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264746+0000","last_change":"2024-09-10T21:24:11.732564+0000","last_active":"2024-09-10T21:24:17.264746+0000","last_peered":"2024-09-10T21:24:17.264746+0000","last_clean":"2024-09-10T21:24:17.264746+0000","last_became_active":"2024-09-10T21:24:11.732106+0000","last_became_peered":"2024-09-10T21:24:11.732106+0000","last_unstale":"2024-09-10T21:24:17.264746+0000","last_undegraded":"2024-09-10T21:24:17.264746+0000","last_fullsized":"2024-09-10T21:24:17.264746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.574642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765650+0000","last_change":"2024-09-10T21:24:10.714693+0000","last_active":"2024-09-10T21:24:16.765650+0000","last_peered":"2024-09-10T21:24:16.765650+0000","last_clean":"2024-09-10T21:24:16.765650+0000","last_became_active":"2024-09-10T21:24:10.714223+0000","last_became_peered":"2024-09-10T21:24:10.714223+0000","last_unstale":"2024-09-10T21:24:16.765650+0000","last_undegraded":"2024-09-10T21:24:16.765650+0000","last_fullsized":"2024-09-10T21:24:16.765650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:06.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766392+0000","last_change":"2024-09-10T21:24:15.784105+0000","last_active":"2024-09-10T21:24:16.766392+0000","last_peered":"2024-09-10T21:24:16.766392+0000","last_clean":"2024-09-10T21:24:16.766392+0000","last_became_active":"2024-09-10T21:24:15.783500+0000","last_became_peered":"2024-09-10T21:24:15.783500+0000","last_unstale":"2024-09-10T21:24:16.766392+0000","last_undegraded":"2024-09-10T21:24:16.766392+0000","last_fullsized":"2024-09-10T21:24:16.766392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.513517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263956+0000","last_change":"2024-09-10T21:24:11.738285+0000","last_active":"2024-09-10T21:24:17.263956+0000","last_peered":"2024-09-10T21:24:17.263956+0000","last_clean":"2024-09-10T21:24:17.263956+0000","last_became_active":"2024-09-10T21:24:11.738001+0000","last_became_peered":"2024-09-10T21:24:11.738001+0000","last_unstale":"2024-09-10T21:24:17.263956+0000","last_undegraded":"2024-09-10T21:24:17.263956+0000","last_fullsized":"2024-09-10T21:24:17.263956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769007+0000","last_change":"2024-09-10T21:24:10.710823+0000","last_active":"2024-09-10T21:24:16.769007+0000","last_peered":"2024-09-10T21:24:16.769007+0000","last_clean":"2024-09-10T21:24:16.769007+0000","last_became_active":"2024-09-10T21:24:10.710231+0000","last_became_peered":"2024-09-10T21:24:10.710231+0000","last_unstale":"2024-09-10T21:24:16.769007+0000","last_undegraded":"2024-09-10T21:24:16.769007+0000","last_fullsized":"2024-09-10T21:24:16.769007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:39.865360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765929+0000","last_change":"2024-09-10T21:24:10.711002+0000","last_active":"2024-09-10T21:24:16.765929+0000","last_peered":"2024-09-10T21:24:16.765929+0000","last_clean":"2024-09-10T21:24:16.765929+0000","last_became_active":"2024-09-10T21:24:10.710723+0000","last_became_peered":"2024-09-10T21:24:10.710723+0000","last_unstale":"2024-09-10T21:24:16.765929+0000","last_undegraded":"2024-09-10T21:24:16.765929+0000","last_fullsized":"2024-09-10T21:24:16.765929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:52.955941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:11.744353+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:11.743970+0000","last_became_peered":"2024-09-10T21:24:11.743970+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:35.066178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265974+0000","last_change":"2024-09-10T21:24:15.791043+0000","last_active":"2024-09-10T21:24:17.265974+0000","last_peered":"2024-09-10T21:24:17.265974+0000","last_clean":"2024-09-10T21:24:17.265974+0000","last_became_active":"2024-09-10T21:24:15.790322+0000","last_became_peered":"2024-09-10T21:24:15.790322+0000","last_unstale":"2024-09-10T21:24:17.265974+0000","last_undegraded":"2024-09-10T21:24:17.265974+0000","last_fullsized":"2024-09-10T21:24:17.265974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:18.317248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264568+0000","last_change":"2024-09-10T21:24:10.717526+0000","last_active":"2024-09-10T21:24:17.264568+0000","last_peered":"2024-09-10T21:24:17.264568+0000","last_clean":"2024-09-10T21:24:17.264568+0000","last_became_active":"2024-09-10T21:24:10.715368+0000","last_became_peered":"2024-09-10T21:24:10.715368+0000","last_unstale":"2024-09-10T21:24:17.264568+0000","last_undegraded":"2024-09-10T21:24:17.264568+0000","last_fullsized":"2024-09-10T21:24:17.264568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263197+0000","last_change":"2024-09-10T21:24:11.733617+0000","last_active":"2024-09-10T21:24:17.263197+0000","last_peered":"2024-09-10T21:24:17.263197+0000","last_clean":"2024-09-10T21:24:17.263197+0000","last_became_active":"2024-09-10T21:24:11.733343+0000","last_became_peered":"2024-09-10T21:24:11.733343+0000","last_unstale":"2024-09-10T21:24:17.263197+0000","last_undegraded":"2024-09-10T21:24:17.263197+0000","last_fullsized":"2024-09-10T21:24:17.263197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770202+0000","last_change":"2024-09-10T21:24:15.787912+0000","last_active":"2024-09-10T21:24:16.770202+0000","last_peered":"2024-09-10T21:24:16.770202+0000","last_clean":"2024-09-10T21:24:16.770202+0000","last_became_active":"2024-09-10T21:24:15.787441+0000","last_became_peered":"2024-09-10T21:24:15.787441+0000","last_unstale":"2024-09-10T21:24:16.770202+0000","last_undegraded":"2024-09-10T21:24:16.770202+0000","last_fullsized":"2024-09-10T21:24:16.770202+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:22.856812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262658+0000","last_change":"2024-09-10T21:24:11.747086+0000","last_active":"2024-09-10T21:24:17.262658+0000","last_peered":"2024-09-10T21:24:17.262658+0000","last_clean":"2024-09-10T21:24:17.262658+0000","last_became_active":"2024-09-10T21:24:11.746780+0000","last_became_peered":"2024-09-10T21:24:11.746780+0000","last_unstale":"2024-09-10T21:24:17.262658+0000","last_undegraded":"2024-09-10T21:24:17.262658+0000","last_fullsized":"2024-09-10T21:24:17.262658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:13.288615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769236+0000","last_change":"2024-09-10T21:24:11.732475+0000","last_active":"2024-09-10T21:24:16.769236+0000","last_peered":"2024-09-10T21:24:16.769236+0000","last_clean":"2024-09-10T21:24:16.769236+0000","last_became_active":"2024-09-10T21:24:11.731007+0000","last_became_peered":"2024-09-10T21:24:11.731007+0000","last_unstale":"2024-09-10T21:24:16.769236+0000","last_undegraded":"2024-09-10T21:24:16.769236+0000","last_fullsized":"2024-09-10T21:24:16.769236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765707+0000","last_change":"2024-09-10T21:24:11.744942+0000","last_active":"2024-09-10T21:24:16.765707+0000","last_peered":"2024-09-10T21:24:16.765707+0000","last_clean":"2024-09-10T21:24:16.765707+0000","last_became_active":"2024-09-10T21:24:11.744626+0000","last_became_peered":"2024-09-10T21:24:11.744626+0000","last_unstale":"2024-09-10T21:24:16.765707+0000","last_undegraded":"2024-09-10T21:24:16.765707+0000","last_fullsized":"2024-09-10T21:24:16.765707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:37.461417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264525+0000","last_change":"2024-09-10T21:24:11.745907+0000","last_active":"2024-09-10T21:24:17.264525+0000","last_peered":"2024-09-10T21:24:17.264525+0000","last_clean":"2024-09-10T21:24:17.264525+0000","last_became_active":"2024-09-10T21:24:11.745583+0000","last_became_peered":"2024-09-10T21:24:11.745583+0000","last_unstale":"2024-09-10T21:24:17.264525+0000","last_undegraded":"2024-09-10T21:24:17.264525+0000","last_fullsized":"2024-09-10T21:24:17.264525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:50.845294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770469+0000","last_change":"2024-09-10T21:24:11.756001+0000","last_active":"2024-09-10T21:24:16.770469+0000","last_peered":"2024-09-10T21:24:16.770469+0000","last_clean":"2024-09-10T21:24:16.770469+0000","last_became_active":"2024-09-10T21:24:11.755719+0000","last_became_peered":"2024-09-10T21:24:11.755719+0000","last_unstale":"2024-09-10T21:24:16.770469+0000","last_undegraded":"2024-09-10T21:24:16.770469+0000","last_fullsized":"2024-09-10T21:24:16.770469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:04.237158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769805+0000","last_change":"2024-09-10T21:24:11.740025+0000","last_active":"2024-09-10T21:24:16.769805+0000","last_peered":"2024-09-10T21:24:16.769805+0000","last_clean":"2024-09-10T21:24:16.769805+0000","last_became_active":"2024-09-10T21:24:11.739742+0000","last_became_peered":"2024-09-10T21:24:11.739742+0000","last_unstale":"2024-09-10T21:24:16.769805+0000","last_undegraded":"2024-09-10T21:24:16.769805+0000","last_fullsized":"2024-09-10T21:24:16.769805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:29.650259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263932+0000","last_change":"2024-09-10T21:24:11.749805+0000","last_active":"2024-09-10T21:24:17.263932+0000","last_peered":"2024-09-10T21:24:17.263932+0000","last_clean":"2024-09-10T21:24:17.263932+0000","last_became_active":"2024-09-10T21:24:11.748452+0000","last_became_peered":"2024-09-10T21:24:11.748452+0000","last_unstale":"2024-09-10T21:24:17.263932+0000","last_undegraded":"2024-09-10T21:24:17.263932+0000","last_fullsized":"2024-09-10T21:24:17.263932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:36.508158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264740+0000","last_change":"2024-09-10T21:24:11.750687+0000","last_active":"2024-09-10T21:24:17.264740+0000","last_peered":"2024-09-10T21:24:17.264740+0000","last_clean":"2024-09-10T21:24:17.264740+0000","last_became_active":"2024-09-10T21:24:11.749899+0000","last_became_peered":"2024-09-10T21:24:11.749899+0000","last_unstale":"2024-09-10T21:24:17.264740+0000","last_undegraded":"2024-09-10T21:24:17.264740+0000","last_fullsized":"2024-09-10T21:24:17.264740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765619+0000","last_change":"2024-09-10T21:24:11.734778+0000","last_active":"2024-09-10T21:24:16.765619+0000","last_peered":"2024-09-10T21:24:16.765619+0000","last_clean":"2024-09-10T21:24:16.765619+0000","last_became_active":"2024-09-10T21:24:11.734510+0000","last_became_peered":"2024-09-10T21:24:11.734510+0000","last_unstale":"2024-09-10T21:24:16.765619+0000","last_undegraded":"2024-09-10T21:24:16.765619+0000","last_fullsized":"2024-09-10T21:24:16.765619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:45.729068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263928+0000","last_change":"2024-09-10T21:24:11.758730+0000","last_active":"2024-09-10T21:24:17.263928+0000","last_peered":"2024-09-10T21:24:17.263928+0000","last_clean":"2024-09-10T21:24:17.263928+0000","last_became_active":"2024-09-10T21:24:11.758453+0000","last_became_peered":"2024-09-10T21:24:11.758453+0000","last_unstale":"2024-09-10T21:24:17.263928+0000","last_undegraded":"2024-09-10T21:24:17.263928+0000","last_fullsized":"2024-09-10T21:24:17.263928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:20.045933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263026+0000","last_change":"2024-09-10T21:24:11.727975+0000","last_active":"2024-09-10T21:24:17.263026+0000","last_peered":"2024-09-10T21:24:17.263026+0000","last_clean":"2024-09-10T21:24:17.263026+0000","last_became_active":"2024-09-10T21:24:11.727715+0000","last_became_peered":"2024-09-10T21:24:11.727715+0000","last_unstale":"2024-09-10T21:24:17.263026+0000","last_undegraded":"2024-09-10T21:24:17.263026+0000","last_fullsized":"2024-09-10T21:24:17.263026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264447+0000","last_change":"2024-09-10T21:24:11.752499+0000","last_active":"2024-09-10T21:24:17.264447+0000","last_peered":"2024-09-10T21:24:17.264447+0000","last_clean":"2024-09-10T21:24:17.264447+0000","last_became_active":"2024-09-10T21:24:11.748931+0000","last_became_peered":"2024-09-10T21:24:11.748931+0000","last_unstale":"2024-09-10T21:24:17.264447+0000","last_undegraded":"2024-09-10T21:24:17.264447+0000","last_fullsized":"2024-09-10T21:24:17.264447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:48.334318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264810+0000","last_change":"2024-09-10T21:24:11.742340+0000","last_active":"2024-09-10T21:24:17.264810+0000","last_peered":"2024-09-10T21:24:17.264810+0000","last_clean":"2024-09-10T21:24:17.264810+0000","last_became_active":"2024-09-10T21:24:11.741702+0000","last_became_peered":"2024-09-10T21:24:11.741702+0000","last_unstale":"2024-09-10T21:24:17.264810+0000","last_undegraded":"2024-09-10T21:24:17.264810+0000","last_fullsized":"2024-09-10T21:24:17.264810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766584+0000","last_change":"2024-09-10T21:24:11.744751+0000","last_active":"2024-09-10T21:24:16.766584+0000","last_peered":"2024-09-10T21:24:16.766584+0000","last_clean":"2024-09-10T21:24:16.766584+0000","last_became_active":"2024-09-10T21:24:11.744431+0000","last_became_peered":"2024-09-10T21:24:11.744431+0000","last_unstale":"2024-09-10T21:24:16.766584+0000","last_undegraded":"2024-09-10T21:24:16.766584+0000","last_fullsized":"2024-09-10T21:24:16.766584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:11.693990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770590+0000","last_change":"2024-09-10T21:24:11.748605+0000","last_active":"2024-09-10T21:24:16.770590+0000","last_peered":"2024-09-10T21:24:16.770590+0000","last_clean":"2024-09-10T21:24:16.770590+0000","last_became_active":"2024-09-10T21:24:11.748279+0000","last_became_peered":"2024-09-10T21:24:11.748279+0000","last_unstale":"2024-09-10T21:24:16.770590+0000","last_undegraded":"2024-09-10T21:24:16.770590+0000","last_fullsized":"2024-09-10T21:24:16.770590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766580+0000","last_change":"2024-09-10T21:24:11.750748+0000","last_active":"2024-09-10T21:24:16.766580+0000","last_peered":"2024-09-10T21:24:16.766580+0000","last_clean":"2024-09-10T21:24:16.766580+0000","last_became_active":"2024-09-10T21:24:11.750317+0000","last_became_peered":"2024-09-10T21:24:11.750317+0000","last_unstale":"2024-09-10T21:24:16.766580+0000","last_undegraded":"2024-09-10T21:24:16.766580+0000","last_fullsized":"2024-09-10T21:24:16.766580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:08.259327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765529+0000","last_change":"2024-09-10T21:24:11.757620+0000","last_active":"2024-09-10T21:24:16.765529+0000","last_peered":"2024-09-10T21:24:16.765529+0000","last_clean":"2024-09-10T21:24:16.765529+0000","last_became_active":"2024-09-10T21:24:11.757088+0000","last_became_peered":"2024-09-10T21:24:11.757088+0000","last_unstale":"2024-09-10T21:24:16.765529+0000","last_undegraded":"2024-09-10T21:24:16.765529+0000","last_fullsized":"2024-09-10T21:24:16.765529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263812+0000","last_change":"2024-09-10T21:24:11.745932+0000","last_active":"2024-09-10T21:24:17.263812+0000","last_peered":"2024-09-10T21:24:17.263812+0000","last_clean":"2024-09-10T21:24:17.263812+0000","last_became_active":"2024-09-10T21:24:11.745682+0000","last_became_peered":"2024-09-10T21:24:11.745682+0000","last_unstale":"2024-09-10T21:24:17.263812+0000","last_undegraded":"2024-09-10T21:24:17.263812+0000","last_fullsized":"2024-09-10T21:24:17.263812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769215+0000","last_change":"2024-09-10T21:24:11.740847+0000","last_active":"2024-09-10T21:24:16.769215+0000","last_peered":"2024-09-10T21:24:16.769215+0000","last_clean":"2024-09-10T21:24:16.769215+0000","last_became_active":"2024-09-10T21:24:11.740548+0000","last_became_peered":"2024-09-10T21:24:11.740548+0000","last_unstale":"2024-09-10T21:24:16.769215+0000","last_undegraded":"2024-09-10T21:24:16.769215+0000","last_fullsized":"2024-09-10T21:24:16.769215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770304+0000","last_change":"2024-09-10T21:24:11.742902+0000","last_active":"2024-09-10T21:24:16.770304+0000","last_peered":"2024-09-10T21:24:16.770304+0000","last_clean":"2024-09-10T21:24:16.770304+0000","last_became_active":"2024-09-10T21:24:11.742449+0000","last_became_peered":"2024-09-10T21:24:11.742449+0000","last_unstale":"2024-09-10T21:24:16.770304+0000","last_undegraded":"2024-09-10T21:24:16.770304+0000","last_fullsized":"2024-09-10T21:24:16.770304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:25.302008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264763+0000","last_change":"2024-09-10T21:24:11.742660+0000","last_active":"2024-09-10T21:24:17.264763+0000","last_peered":"2024-09-10T21:24:17.264763+0000","last_clean":"2024-09-10T21:24:17.264763+0000","last_became_active":"2024-09-10T21:24:11.742069+0000","last_became_peered":"2024-09-10T21:24:11.742069+0000","last_unstale":"2024-09-10T21:24:17.264763+0000","last_undegraded":"2024-09-10T21:24:17.264763+0000","last_fullsized":"2024-09-10T21:24:17.264763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:35.526539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264897+0000","last_change":"2024-09-10T21:24:11.743498+0000","last_active":"2024-09-10T21:24:17.264897+0000","last_peered":"2024-09-10T21:24:17.264897+0000","last_clean":"2024-09-10T21:24:17.264897+0000","last_became_active":"2024-09-10T21:24:11.743017+0000","last_became_peered":"2024-09-10T21:24:11.743017+0000","last_unstale":"2024-09-10T21:24:17.264897+0000","last_undegraded":"2024-09-10T21:24:17.264897+0000","last_fullsized":"2024-09-10T21:24:17.264897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.754599+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.754341+0000","last_became_peered":"2024-09-10T21:24:11.754341+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264501+0000","last_change":"2024-09-10T21:24:11.726485+0000","last_active":"2024-09-10T21:24:17.264501+0000","last_peered":"2024-09-10T21:24:17.264501+0000","last_clean":"2024-09-10T21:24:17.264501+0000","last_became_active":"2024-09-10T21:24:11.726157+0000","last_became_peered":"2024-09-10T21:24:11.726157+0000","last_unstale":"2024-09-10T21:24:17.264501+0000","last_undegraded":"2024-09-10T21:24:17.264501+0000","last_fullsized":"2024-09-10T21:24:17.264501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263664+0000","last_change":"2024-09-10T21:24:11.754793+0000","last_active":"2024-09-10T21:24:17.263664+0000","last_peered":"2024-09-10T21:24:17.263664+0000","last_clean":"2024-09-10T21:24:17.263664+0000","last_became_active":"2024-09-10T21:24:11.754372+0000","last_became_peered":"2024-09-10T21:24:11.754372+0000","last_unstale":"2024-09-10T21:24:17.263664+0000","last_undegraded":"2024-09-10T21:24:17.263664+0000","last_fullsized":"2024-09-10T21:24:17.263664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:25.692729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263914+0000","last_change":"2024-09-10T21:24:11.741830+0000","last_active":"2024-09-10T21:24:17.263914+0000","last_peered":"2024-09-10T21:24:17.263914+0000","last_clean":"2024-09-10T21:24:17.263914+0000","last_became_active":"2024-09-10T21:24:11.741543+0000","last_became_peered":"2024-09-10T21:24:11.741543+0000","last_unstale":"2024-09-10T21:24:17.263914+0000","last_undegraded":"2024-09-10T21:24:17.263914+0000","last_fullsized":"2024-09-10T21:24:17.263914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263114+0000","last_change":"2024-09-10T21:24:11.744958+0000","last_active":"2024-09-10T21:24:17.263114+0000","last_peered":"2024-09-10T21:24:17.263114+0000","last_clean":"2024-09-10T21:24:17.263114+0000","last_became_active":"2024-09-10T21:24:11.744611+0000","last_became_peered":"2024-09-10T21:24:11.744611+0000","last_unstale":"2024-09-10T21:24:17.263114+0000","last_undegraded":"2024-09-10T21:24:17.263114+0000","last_fullsized":"2024-09-10T21:24:17.263114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264460+0000","last_change":"2024-09-10T21:24:11.742259+0000","last_active":"2024-09-10T21:24:17.264460+0000","last_peered":"2024-09-10T21:24:17.264460+0000","last_clean":"2024-09-10T21:24:17.264460+0000","last_became_active":"2024-09-10T21:24:11.742010+0000","last_became_peered":"2024-09-10T21:24:11.742010+0000","last_unstale":"2024-09-10T21:24:17.264460+0000","last_undegraded":"2024-09-10T21:24:17.264460+0000","last_fullsized":"2024-09-10T21:24:17.264460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263425+0000","last_change":"2024-09-10T21:24:11.747125+0000","last_active":"2024-09-10T21:24:17.263425+0000","last_peered":"2024-09-10T21:24:17.263425+0000","last_clean":"2024-09-10T21:24:17.263425+0000","last_became_active":"2024-09-10T21:24:11.746844+0000","last_became_peered":"2024-09-10T21:24:11.746844+0000","last_unstale":"2024-09-10T21:24:17.263425+0000","last_undegraded":"2024-09-10T21:24:17.263425+0000","last_fullsized":"2024-09-10T21:24:17.263425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:28.995110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265138+0000","last_change":"2024-09-10T21:24:11.737282+0000","last_active":"2024-09-10T21:24:17.265138+0000","last_peered":"2024-09-10T21:24:17.265138+0000","last_clean":"2024-09-10T21:24:17.265138+0000","last_became_active":"2024-09-10T21:24:11.736985+0000","last_became_peered":"2024-09-10T21:24:11.736985+0000","last_unstale":"2024-09-10T21:24:17.265138+0000","last_undegraded":"2024-09-10T21:24:17.265138+0000","last_fullsized":"2024-09-10T21:24:17.265138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264431+0000","last_change":"2024-09-10T21:24:11.738254+0000","last_active":"2024-09-10T21:24:17.264431+0000","last_peered":"2024-09-10T21:24:17.264431+0000","last_clean":"2024-09-10T21:24:17.264431+0000","last_became_active":"2024-09-10T21:24:11.737963+0000","last_became_peered":"2024-09-10T21:24:11.737963+0000","last_unstale":"2024-09-10T21:24:17.264431+0000","last_undegraded":"2024-09-10T21:24:17.264431+0000","last_fullsized":"2024-09-10T21:24:17.264431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265077+0000","last_change":"2024-09-10T21:24:11.743284+0000","last_active":"2024-09-10T21:24:17.265077+0000","last_peered":"2024-09-10T21:24:17.265077+0000","last_clean":"2024-09-10T21:24:17.265077+0000","last_became_active":"2024-09-10T21:24:11.742655+0000","last_became_peered":"2024-09-10T21:24:11.742655+0000","last_unstale":"2024-09-10T21:24:17.265077+0000","last_undegraded":"2024-09-10T21:24:17.265077+0000","last_fullsized":"2024-09-10T21:24:17.265077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:24:32.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:24:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- 172.21.15.97:0/759891734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 msgr2=0x7fc1d00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/759891734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d00b2a80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009900 tx=0x7fc1b801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- 172.21.15.97:0/759891734 shutdown_connections 2024-09-10T21:24:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/759891734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d00b2a80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- 172.21.15.97:0/759891734 >> 172.21.15.97:0/759891734 conn(0x7fc1d00b1780 msgr2=0x7fc1d00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- 172.21.15.97:0/759891734 shutdown_connections 2024-09-10T21:24:33.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- 172.21.15.97:0/759891734 wait complete. 2024-09-10T21:24:33.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 Processor -- start 2024-09-10T21:24:33.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- start start 2024-09-10T21:24:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1d006fd50 0x7fc1d0138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1d0143f90 0x7fc1d01416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d0141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0169460 con 0x7fc1d0144690 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01692e0 con 0x7fc1d0143f90 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1d4e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d01695e0 con 0x7fc1d006fd50 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d0141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d0141c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41100/0 (socket says 172.21.15.97:41100) 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 -- 172.21.15.97:0/2080440403 learned_addr learned my addr 172.21.15.97:0/2080440403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1c5400640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1d0143f90 0x7fc1d01416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1cde00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1d006fd50 0x7fc1d0138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 -- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1d006fd50 msgr2=0x7fc1d0138f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1d006fd50 0x7fc1d0138f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 -- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1d0143f90 msgr2=0x7fc1d01416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1d0143f90 0x7fc1d01416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1ce800640 1 -- 172.21.15.97:0/2080440403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d00af2c0 con 0x7fc1d0144690 2024-09-10T21:24:33.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.084+0000 7fc1cde00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1d006fd50 0x7fc1d0138f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1ce800640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d0141c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800c3c0 tx=0x7fc1c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1c6a00640 1 -- 172.21.15.97:0/2080440403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8011790 con 0x7fc1d0144690 2024-09-10T21:24:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1c6a00640 1 -- 172.21.15.97:0/2080440403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1c8017070 con 0x7fc1d0144690 2024-09-10T21:24:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1c6a00640 1 -- 172.21.15.97:0/2080440403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8011cc0 con 0x7fc1d0144690 2024-09-10T21:24:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d0136930 con 0x7fc1d0144690 2024-09-10T21:24:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d0136ef0 con 0x7fc1d0144690 2024-09-10T21:24:33.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d00b2a80 con 0x7fc1d0144690 2024-09-10T21:24:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1c6a00640 1 -- 172.21.15.97:0/2080440403 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc1c801c460 con 0x7fc1d0144690 2024-09-10T21:24:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1c6a00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1ac072610 0x7fc1ac074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1c6a00640 1 -- 172.21.15.97:0/2080440403 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc1c8093270 con 0x7fc1d0144690 2024-09-10T21:24:33.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.088+0000 7fc1cde00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1ac072610 0x7fc1ac074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.092+0000 7fc1cde00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1ac072610 0x7fc1ac074ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0001e50 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:33.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.092+0000 7fc1c6a00640 1 -- 172.21.15.97:0/2080440403 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc1c805e600 con 0x7fc1d0144690 2024-09-10T21:24:33.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.256+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1d015ac20 con 0x7fc1ac072610 2024-09-10T21:24:33.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.260+0000 7fc1c6a00640 1 -- 172.21.15.97:0/2080440403 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332543 (secure 0 0 0) 0x7fc1d015ac20 con 0x7fc1ac072610 2024-09-10T21:24:33.272 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:33.274 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:24:33.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1ac072610 msgr2=0x7fc1ac074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1ac072610 0x7fc1ac074ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0001e50 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2024-09-10T21:24:33.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 msgr2=0x7fc1d0141c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d0141c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800c3c0 tx=0x7fc1c800c880 comp rx=0 tx=0).stop 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 shutdown_connections 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1ac072610 0x7fc1ac074ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1d0144690 0x7fc1d0141c20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1d0143f90 0x7fc1d01416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 --2- 172.21.15.97:0/2080440403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1d006fd50 0x7fc1d0138f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.264+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 >> 172.21.15.97:0/2080440403 conn(0x7fc1d00b1780 msgr2=0x7fc1d00ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.268+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 shutdown_connections 2024-09-10T21:24:33.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.268+0000 7fc1d4e00640 1 -- 172.21.15.97:0/2080440403 wait complete. 2024-09-10T21:24:33.292 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-10T21:24:32.073235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003366"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264781+0000","last_change":"2024-09-10T21:24:10.717118+0000","last_active":"2024-09-10T21:24:17.264781+0000","last_peered":"2024-09-10T21:24:17.264781+0000","last_clean":"2024-09-10T21:24:17.264781+0000","last_became_active":"2024-09-10T21:24:10.716871+0000","last_became_peered":"2024-09-10T21:24:10.716871+0000","last_unstale":"2024-09-10T21:24:17.264781+0000","last_undegraded":"2024-09-10T21:24:17.264781+0000","last_fullsized":"2024-09-10T21:24:17.264781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:26.347318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264579+0000","last_change":"2024-09-10T21:24:11.758545+0000","last_active":"2024-09-10T21:24:17.264579+0000","last_peered":"2024-09-10T21:24:17.264579+0000","last_clean":"2024-09-10T21:24:17.264579+0000","last_became_active":"2024-09-10T21:24:11.758227+0000","last_became_peered":"2024-09-10T21:24:11.758227+0000","last_unstale":"2024-09-10T21:24:17.264579+0000","last_undegraded":"2024-09-10T21:24:17.264579+0000","last_fullsized":"2024-09-10T21:24:17.264579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768689+0000","last_change":"2024-09-10T21:24:15.784509+0000","last_active":"2024-09-10T21:24:16.768689+0000","last_peered":"2024-09-10T21:24:16.768689+0000","last_clean":"2024-09-10T21:24:16.768689+0000","last_became_active":"2024-09-10T21:24:15.777793+0000","last_became_peered":"2024-09-10T21:24:15.777793+0000","last_unstale":"2024-09-10T21:24:16.768689+0000","last_undegraded":"2024-09-10T21:24:16.768689+0000","last_fullsized":"2024-09-10T21:24:16.768689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:55.568506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264784+0000","last_change":"2024-09-10T21:24:10.717927+0000","last_active":"2024-09-10T21:24:17.264784+0000","last_peered":"2024-09-10T21:24:17.264784+0000","last_clean":"2024-09-10T21:24:17.264784+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:17.264784+0000","last_undegraded":"2024-09-10T21:24:17.264784+0000","last_fullsized":"2024-09-10T21:24:17.264784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265060+0000","last_change":"2024-09-10T21:24:11.750430+0000","last_active":"2024-09-10T21:24:17.265060+0000","last_peered":"2024-09-10T21:24:17.265060+0000","last_clean":"2024-09-10T21:24:17.265060+0000","last_became_active":"2024-09-10T21:24:11.749604+0000","last_became_peered":"2024-09-10T21:24:11.749604+0000","last_unstale":"2024-09-10T21:24:17.265060+0000","last_undegraded":"2024-09-10T21:24:17.265060+0000","last_fullsized":"2024-09-10T21:24:17.265060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262600+0000","last_change":"2024-09-10T21:24:15.775067+0000","last_active":"2024-09-10T21:24:17.262600+0000","last_peered":"2024-09-10T21:24:17.262600+0000","last_clean":"2024-09-10T21:24:17.262600+0000","last_became_active":"2024-09-10T21:24:15.773909+0000","last_became_peered":"2024-09-10T21:24:15.773909+0000","last_unstale":"2024-09-10T21:24:17.262600+0000","last_undegraded":"2024-09-10T21:24:17.262600+0000","last_fullsized":"2024-09-10T21:24:17.262600+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:09.297494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265037+0000","last_change":"2024-09-10T21:24:10.711527+0000","last_active":"2024-09-10T21:24:17.265037+0000","last_peered":"2024-09-10T21:24:17.265037+0000","last_clean":"2024-09-10T21:24:17.265037+0000","last_became_active":"2024-09-10T21:24:10.711276+0000","last_became_peered":"2024-09-10T21:24:10.711276+0000","last_unstale":"2024-09-10T21:24:17.265037+0000","last_undegraded":"2024-09-10T21:24:17.265037+0000","last_fullsized":"2024-09-10T21:24:17.265037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263749+0000","last_change":"2024-09-10T21:24:11.756415+0000","last_active":"2024-09-10T21:24:17.263749+0000","last_peered":"2024-09-10T21:24:17.263749+0000","last_clean":"2024-09-10T21:24:17.263749+0000","last_became_active":"2024-09-10T21:24:11.754620+0000","last_became_peered":"2024-09-10T21:24:11.754620+0000","last_unstale":"2024-09-10T21:24:17.263749+0000","last_undegraded":"2024-09-10T21:24:17.263749+0000","last_fullsized":"2024-09-10T21:24:17.263749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:48.122962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262734+0000","last_change":"2024-09-10T21:24:15.782482+0000","last_active":"2024-09-10T21:24:17.262734+0000","last_peered":"2024-09-10T21:24:17.262734+0000","last_clean":"2024-09-10T21:24:17.262734+0000","last_became_active":"2024-09-10T21:24:15.782107+0000","last_became_peered":"2024-09-10T21:24:15.782107+0000","last_unstale":"2024-09-10T21:24:17.262734+0000","last_undegraded":"2024-09-10T21:24:17.262734+0000","last_fullsized":"2024-09-10T21:24:17.262734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.906721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264925+0000","last_change":"2024-09-10T21:24:10.716898+0000","last_active":"2024-09-10T21:24:17.264925+0000","last_peered":"2024-09-10T21:24:17.264925+0000","last_clean":"2024-09-10T21:24:17.264925+0000","last_became_active":"2024-09-10T21:24:10.716644+0000","last_became_peered":"2024-09-10T21:24:10.716644+0000","last_unstale":"2024-09-10T21:24:17.264925+0000","last_undegraded":"2024-09-10T21:24:17.264925+0000","last_fullsized":"2024-09-10T21:24:17.264925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:41.682216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766338+0000","last_change":"2024-09-10T21:24:11.747195+0000","last_active":"2024-09-10T21:24:16.766338+0000","last_peered":"2024-09-10T21:24:16.766338+0000","last_clean":"2024-09-10T21:24:16.766338+0000","last_became_active":"2024-09-10T21:24:11.747038+0000","last_became_peered":"2024-09-10T21:24:11.747038+0000","last_unstale":"2024-09-10T21:24:16.766338+0000","last_undegraded":"2024-09-10T21:24:16.766338+0000","last_fullsized":"2024-09-10T21:24:16.766338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262860+0000","last_change":"2024-09-10T21:24:15.782490+0000","last_active":"2024-09-10T21:24:17.262860+0000","last_peered":"2024-09-10T21:24:17.262860+0000","last_clean":"2024-09-10T21:24:17.262860+0000","last_became_active":"2024-09-10T21:24:15.782079+0000","last_became_peered":"2024-09-10T21:24:15.782079+0000","last_unstale":"2024-09-10T21:24:17.262860+0000","last_undegraded":"2024-09-10T21:24:17.262860+0000","last_fullsized":"2024-09-10T21:24:17.262860+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:42.047447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765875+0000","last_change":"2024-09-10T21:24:11.734798+0000","last_active":"2024-09-10T21:24:16.765875+0000","last_peered":"2024-09-10T21:24:16.765875+0000","last_clean":"2024-09-10T21:24:16.765875+0000","last_became_active":"2024-09-10T21:24:11.734495+0000","last_became_peered":"2024-09-10T21:24:11.734495+0000","last_unstale":"2024-09-10T21:24:16.765875+0000","last_undegraded":"2024-09-10T21:24:16.765875+0000","last_fullsized":"2024-09-10T21:24:16.765875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:08.842495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766019+0000","last_change":"2024-09-10T21:24:10.718178+0000","last_active":"2024-09-10T21:24:16.766019+0000","last_peered":"2024-09-10T21:24:16.766019+0000","last_clean":"2024-09-10T21:24:16.766019+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:16.766019+0000","last_undegraded":"2024-09-10T21:24:16.766019+0000","last_fullsized":"2024-09-10T21:24:16.766019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:01.094807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766145+0000","last_change":"2024-09-10T21:24:15.786334+0000","last_active":"2024-09-10T21:24:16.766145+0000","last_peered":"2024-09-10T21:24:16.766145+0000","last_clean":"2024-09-10T21:24:16.766145+0000","last_became_active":"2024-09-10T21:24:15.785888+0000","last_became_peered":"2024-09-10T21:24:15.785888+0000","last_unstale":"2024-09-10T21:24:16.766145+0000","last_undegraded":"2024-09-10T21:24:16.766145+0000","last_fullsized":"2024-09-10T21:24:16.766145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:27.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769104+0000","last_change":"2024-09-10T21:24:11.743760+0000","last_active":"2024-09-10T21:24:16.769104+0000","last_peered":"2024-09-10T21:24:16.769104+0000","last_clean":"2024-09-10T21:24:16.769104+0000","last_became_active":"2024-09-10T21:24:11.743459+0000","last_became_peered":"2024-09-10T21:24:11.743459+0000","last_unstale":"2024-09-10T21:24:16.769104+0000","last_undegraded":"2024-09-10T21:24:16.769104+0000","last_fullsized":"2024-09-10T21:24:16.769104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:53.958469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770545+0000","last_change":"2024-09-10T21:24:10.719093+0000","last_active":"2024-09-10T21:24:16.770545+0000","last_peered":"2024-09-10T21:24:16.770545+0000","last_clean":"2024-09-10T21:24:16.770545+0000","last_became_active":"2024-09-10T21:24:10.715489+0000","last_became_peered":"2024-09-10T21:24:10.715489+0000","last_unstale":"2024-09-10T21:24:16.770545+0000","last_undegraded":"2024-09-10T21:24:16.770545+0000","last_fullsized":"2024-09-10T21:24:16.770545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264532+0000","last_change":"2024-09-10T21:24:15.777643+0000","last_active":"2024-09-10T21:24:17.264532+0000","last_peered":"2024-09-10T21:24:17.264532+0000","last_clean":"2024-09-10T21:24:17.264532+0000","last_became_active":"2024-09-10T21:24:15.777271+0000","last_became_peered":"2024-09-10T21:24:15.777271+0000","last_unstale":"2024-09-10T21:24:17.264532+0000","last_undegraded":"2024-09-10T21:24:17.264532+0000","last_fullsized":"2024-09-10T21:24:17.264532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:26.509163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263743+0000","last_change":"2024-09-10T21:24:11.724275+0000","last_active":"2024-09-10T21:24:17.263743+0000","last_peered":"2024-09-10T21:24:17.263743+0000","last_clean":"2024-09-10T21:24:17.263743+0000","last_became_active":"2024-09-10T21:24:11.723982+0000","last_became_peered":"2024-09-10T21:24:11.723982+0000","last_unstale":"2024-09-10T21:24:17.263743+0000","last_undegraded":"2024-09-10T21:24:17.263743+0000","last_fullsized":"2024-09-10T21:24:17.263743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:28.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264861+0000","last_change":"2024-09-10T21:24:10.712213+0000","last_active":"2024-09-10T21:24:17.264861+0000","last_peered":"2024-09-10T21:24:17.264861+0000","last_clean":"2024-09-10T21:24:17.264861+0000","last_became_active":"2024-09-10T21:24:10.711951+0000","last_became_peered":"2024-09-10T21:24:10.711951+0000","last_unstale":"2024-09-10T21:24:17.264861+0000","last_undegraded":"2024-09-10T21:24:17.264861+0000","last_fullsized":"2024-09-10T21:24:17.264861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:54.891884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770275+0000","last_change":"2024-09-10T21:24:15.780288+0000","last_active":"2024-09-10T21:24:16.770275+0000","last_peered":"2024-09-10T21:24:16.770275+0000","last_clean":"2024-09-10T21:24:16.770275+0000","last_became_active":"2024-09-10T21:24:15.778928+0000","last_became_peered":"2024-09-10T21:24:15.778928+0000","last_unstale":"2024-09-10T21:24:16.770275+0000","last_undegraded":"2024-09-10T21:24:16.770275+0000","last_fullsized":"2024-09-10T21:24:16.770275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:45.367503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765552+0000","last_change":"2024-09-10T21:24:11.730800+0000","last_active":"2024-09-10T21:24:16.765552+0000","last_peered":"2024-09-10T21:24:16.765552+0000","last_clean":"2024-09-10T21:24:16.765552+0000","last_became_active":"2024-09-10T21:24:11.730474+0000","last_became_peered":"2024-09-10T21:24:11.730474+0000","last_unstale":"2024-09-10T21:24:16.765552+0000","last_undegraded":"2024-09-10T21:24:16.765552+0000","last_fullsized":"2024-09-10T21:24:16.765552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263542+0000","last_change":"2024-09-10T21:24:10.712139+0000","last_active":"2024-09-10T21:24:17.263542+0000","last_peered":"2024-09-10T21:24:17.263542+0000","last_clean":"2024-09-10T21:24:17.263542+0000","last_became_active":"2024-09-10T21:24:10.710501+0000","last_became_peered":"2024-09-10T21:24:10.710501+0000","last_unstale":"2024-09-10T21:24:17.263542+0000","last_undegraded":"2024-09-10T21:24:17.263542+0000","last_fullsized":"2024-09-10T21:24:17.263542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263764+0000","last_change":"2024-09-10T21:24:15.770139+0000","last_active":"2024-09-10T21:24:17.263764+0000","last_peered":"2024-09-10T21:24:17.263764+0000","last_clean":"2024-09-10T21:24:17.263764+0000","last_became_active":"2024-09-10T21:24:15.769554+0000","last_became_peered":"2024-09-10T21:24:15.769554+0000","last_unstale":"2024-09-10T21:24:17.263764+0000","last_undegraded":"2024-09-10T21:24:17.263764+0000","last_fullsized":"2024-09-10T21:24:17.263764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:14.866707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266054+0000","last_change":"2024-09-10T21:24:11.732319+0000","last_active":"2024-09-10T21:24:17.266054+0000","last_peered":"2024-09-10T21:24:17.266054+0000","last_clean":"2024-09-10T21:24:17.266054+0000","last_became_active":"2024-09-10T21:24:11.732038+0000","last_became_peered":"2024-09-10T21:24:11.732038+0000","last_unstale":"2024-09-10T21:24:17.266054+0000","last_undegraded":"2024-09-10T21:24:17.266054+0000","last_fullsized":"2024-09-10T21:24:17.266054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:34.760961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766655+0000","last_change":"2024-09-10T21:24:10.720607+0000","last_active":"2024-09-10T21:24:16.766655+0000","last_peered":"2024-09-10T21:24:16.766655+0000","last_clean":"2024-09-10T21:24:16.766655+0000","last_became_active":"2024-09-10T21:24:10.720144+0000","last_became_peered":"2024-09-10T21:24:10.720144+0000","last_unstale":"2024-09-10T21:24:16.766655+0000","last_undegraded":"2024-09-10T21:24:16.766655+0000","last_fullsized":"2024-09-10T21:24:16.766655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:10.162599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263165+0000","last_change":"2024-09-10T21:24:15.777793+0000","last_active":"2024-09-10T21:24:17.263165+0000","last_peered":"2024-09-10T21:24:17.263165+0000","last_clean":"2024-09-10T21:24:17.263165+0000","last_became_active":"2024-09-10T21:24:15.777358+0000","last_became_peered":"2024-09-10T21:24:15.777358+0000","last_unstale":"2024-09-10T21:24:17.263165+0000","last_undegraded":"2024-09-10T21:24:17.263165+0000","last_fullsized":"2024-09-10T21:24:17.263165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:48.503713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264581+0000","last_change":"2024-09-10T21:24:11.735858+0000","last_active":"2024-09-10T21:24:17.264581+0000","last_peered":"2024-09-10T21:24:17.264581+0000","last_clean":"2024-09-10T21:24:17.264581+0000","last_became_active":"2024-09-10T21:24:11.735599+0000","last_became_peered":"2024-09-10T21:24:11.735599+0000","last_unstale":"2024-09-10T21:24:17.264581+0000","last_undegraded":"2024-09-10T21:24:17.264581+0000","last_fullsized":"2024-09-10T21:24:17.264581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:53.530511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266208+0000","last_change":"2024-09-10T21:24:10.712710+0000","last_active":"2024-09-10T21:24:17.266208+0000","last_peered":"2024-09-10T21:24:17.266208+0000","last_clean":"2024-09-10T21:24:17.266208+0000","last_became_active":"2024-09-10T21:24:10.712400+0000","last_became_peered":"2024-09-10T21:24:10.712400+0000","last_unstale":"2024-09-10T21:24:17.266208+0000","last_undegraded":"2024-09-10T21:24:17.266208+0000","last_fullsized":"2024-09-10T21:24:17.266208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:57.508268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263912+0000","last_change":"2024-09-10T21:24:15.776045+0000","last_active":"2024-09-10T21:24:17.263912+0000","last_peered":"2024-09-10T21:24:17.263912+0000","last_clean":"2024-09-10T21:24:17.263912+0000","last_became_active":"2024-09-10T21:24:15.775199+0000","last_became_peered":"2024-09-10T21:24:15.775199+0000","last_unstale":"2024-09-10T21:24:17.263912+0000","last_undegraded":"2024-09-10T21:24:17.263912+0000","last_fullsized":"2024-09-10T21:24:17.263912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.516796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770569+0000","last_change":"2024-09-10T21:24:11.737947+0000","last_active":"2024-09-10T21:24:16.770569+0000","last_peered":"2024-09-10T21:24:16.770569+0000","last_clean":"2024-09-10T21:24:16.770569+0000","last_became_active":"2024-09-10T21:24:11.735572+0000","last_became_peered":"2024-09-10T21:24:11.735572+0000","last_unstale":"2024-09-10T21:24:16.770569+0000","last_undegraded":"2024-09-10T21:24:16.770569+0000","last_fullsized":"2024-09-10T21:24:16.770569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:41.353969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766677+0000","last_change":"2024-09-10T21:24:10.712429+0000","last_active":"2024-09-10T21:24:16.766677+0000","last_peered":"2024-09-10T21:24:16.766677+0000","last_clean":"2024-09-10T21:24:16.766677+0000","last_became_active":"2024-09-10T21:24:10.712150+0000","last_became_peered":"2024-09-10T21:24:10.712150+0000","last_unstale":"2024-09-10T21:24:16.766677+0000","last_undegraded":"2024-09-10T21:24:16.766677+0000","last_fullsized":"2024-09-10T21:24:16.766677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:37.565989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769802+0000","last_change":"2024-09-10T21:24:15.778516+0000","last_active":"2024-09-10T21:24:16.769802+0000","last_peered":"2024-09-10T21:24:16.769802+0000","last_clean":"2024-09-10T21:24:16.769802+0000","last_became_active":"2024-09-10T21:24:15.778061+0000","last_became_peered":"2024-09-10T21:24:15.778061+0000","last_unstale":"2024-09-10T21:24:16.769802+0000","last_undegraded":"2024-09-10T21:24:16.769802+0000","last_fullsized":"2024-09-10T21:24:16.769802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:38.186163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263379+0000","last_change":"2024-09-10T21:24:11.747693+0000","last_active":"2024-09-10T21:24:17.263379+0000","last_peered":"2024-09-10T21:24:17.263379+0000","last_clean":"2024-09-10T21:24:17.263379+0000","last_became_active":"2024-09-10T21:24:11.747405+0000","last_became_peered":"2024-09-10T21:24:11.747405+0000","last_unstale":"2024-09-10T21:24:17.263379+0000","last_undegraded":"2024-09-10T21:24:17.263379+0000","last_fullsized":"2024-09-10T21:24:17.263379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:42.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264255+0000","last_change":"2024-09-10T21:24:10.711258+0000","last_active":"2024-09-10T21:24:17.264255+0000","last_peered":"2024-09-10T21:24:17.264255+0000","last_clean":"2024-09-10T21:24:17.264255+0000","last_became_active":"2024-09-10T21:24:10.710892+0000","last_became_peered":"2024-09-10T21:24:10.710892+0000","last_unstale":"2024-09-10T21:24:17.264255+0000","last_undegraded":"2024-09-10T21:24:17.264255+0000","last_fullsized":"2024-09-10T21:24:17.264255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769471+0000","last_change":"2024-09-10T21:24:15.786836+0000","last_active":"2024-09-10T21:24:16.769471+0000","last_peered":"2024-09-10T21:24:16.769471+0000","last_clean":"2024-09-10T21:24:16.769471+0000","last_became_active":"2024-09-10T21:24:15.786459+0000","last_became_peered":"2024-09-10T21:24:15.786459+0000","last_unstale":"2024-09-10T21:24:16.769471+0000","last_undegraded":"2024-09-10T21:24:16.769471+0000","last_fullsized":"2024-09-10T21:24:16.769471+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:49.037577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264516+0000","last_change":"2024-09-10T21:24:11.756779+0000","last_active":"2024-09-10T21:24:17.264516+0000","last_peered":"2024-09-10T21:24:17.264516+0000","last_clean":"2024-09-10T21:24:17.264516+0000","last_became_active":"2024-09-10T21:24:11.756493+0000","last_became_peered":"2024-09-10T21:24:11.756493+0000","last_unstale":"2024-09-10T21:24:17.264516+0000","last_undegraded":"2024-09-10T21:24:17.264516+0000","last_fullsized":"2024-09-10T21:24:17.264516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766053+0000","last_change":"2024-09-10T21:24:10.707060+0000","last_active":"2024-09-10T21:24:16.766053+0000","last_peered":"2024-09-10T21:24:16.766053+0000","last_clean":"2024-09-10T21:24:16.766053+0000","last_became_active":"2024-09-10T21:24:10.706798+0000","last_became_peered":"2024-09-10T21:24:10.706798+0000","last_unstale":"2024-09-10T21:24:16.766053+0000","last_undegraded":"2024-09-10T21:24:16.766053+0000","last_fullsized":"2024-09-10T21:24:16.766053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:47.120434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770236+0000","last_change":"2024-09-10T21:24:15.781040+0000","last_active":"2024-09-10T21:24:16.770236+0000","last_peered":"2024-09-10T21:24:16.770236+0000","last_clean":"2024-09-10T21:24:16.770236+0000","last_became_active":"2024-09-10T21:24:15.780677+0000","last_became_peered":"2024-09-10T21:24:15.780677+0000","last_unstale":"2024-09-10T21:24:16.770236+0000","last_undegraded":"2024-09-10T21:24:16.770236+0000","last_fullsized":"2024-09-10T21:24:16.770236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.899193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263779+0000","last_change":"2024-09-10T21:24:11.753935+0000","last_active":"2024-09-10T21:24:17.263779+0000","last_peered":"2024-09-10T21:24:17.263779+0000","last_clean":"2024-09-10T21:24:17.263779+0000","last_became_active":"2024-09-10T21:24:11.753134+0000","last_became_peered":"2024-09-10T21:24:11.753134+0000","last_unstale":"2024-09-10T21:24:17.263779+0000","last_undegraded":"2024-09-10T21:24:17.263779+0000","last_fullsized":"2024-09-10T21:24:17.263779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:41.306076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769295+0000","last_change":"2024-09-10T21:24:10.718908+0000","last_active":"2024-09-10T21:24:16.769295+0000","last_peered":"2024-09-10T21:24:16.769295+0000","last_clean":"2024-09-10T21:24:16.769295+0000","last_became_active":"2024-09-10T21:24:10.718142+0000","last_became_peered":"2024-09-10T21:24:10.718142+0000","last_unstale":"2024-09-10T21:24:16.769295+0000","last_undegraded":"2024-09-10T21:24:16.769295+0000","last_fullsized":"2024-09-10T21:24:16.769295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:56.946441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769872+0000","last_change":"2024-09-10T21:24:15.778921+0000","last_active":"2024-09-10T21:24:16.769872+0000","last_peered":"2024-09-10T21:24:16.769872+0000","last_clean":"2024-09-10T21:24:16.769872+0000","last_became_active":"2024-09-10T21:24:15.778360+0000","last_became_peered":"2024-09-10T21:24:15.778360+0000","last_unstale":"2024-09-10T21:24:16.769872+0000","last_undegraded":"2024-09-10T21:24:16.769872+0000","last_fullsized":"2024-09-10T21:24:16.769872+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:10.163639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768895+0000","last_change":"2024-09-10T21:24:11.758622+0000","last_active":"2024-09-10T21:24:16.768895+0000","last_peered":"2024-09-10T21:24:16.768895+0000","last_clean":"2024-09-10T21:24:16.768895+0000","last_became_active":"2024-09-10T21:24:11.758363+0000","last_became_peered":"2024-09-10T21:24:11.758363+0000","last_unstale":"2024-09-10T21:24:16.768895+0000","last_undegraded":"2024-09-10T21:24:16.768895+0000","last_fullsized":"2024-09-10T21:24:16.768895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765739+0000","last_change":"2024-09-10T21:24:10.720746+0000","last_active":"2024-09-10T21:24:16.765739+0000","last_peered":"2024-09-10T21:24:16.765739+0000","last_clean":"2024-09-10T21:24:16.765739+0000","last_became_active":"2024-09-10T21:24:10.720360+0000","last_became_peered":"2024-09-10T21:24:10.720360+0000","last_unstale":"2024-09-10T21:24:16.765739+0000","last_undegraded":"2024-09-10T21:24:16.765739+0000","last_fullsized":"2024-09-10T21:24:16.765739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:25.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264322+0000","last_change":"2024-09-10T21:24:15.787994+0000","last_active":"2024-09-10T21:24:17.264322+0000","last_peered":"2024-09-10T21:24:17.264322+0000","last_clean":"2024-09-10T21:24:17.264322+0000","last_became_active":"2024-09-10T21:24:15.787532+0000","last_became_peered":"2024-09-10T21:24:15.787532+0000","last_unstale":"2024-09-10T21:24:17.264322+0000","last_undegraded":"2024-09-10T21:24:17.264322+0000","last_fullsized":"2024-09-10T21:24:17.264322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:12.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265574+0000","last_change":"2024-09-10T21:24:11.747694+0000","last_active":"2024-09-10T21:24:17.265574+0000","last_peered":"2024-09-10T21:24:17.265574+0000","last_clean":"2024-09-10T21:24:17.265574+0000","last_became_active":"2024-09-10T21:24:11.746819+0000","last_became_peered":"2024-09-10T21:24:11.746819+0000","last_unstale":"2024-09-10T21:24:17.265574+0000","last_undegraded":"2024-09-10T21:24:17.265574+0000","last_fullsized":"2024-09-10T21:24:17.265574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:14.637932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766139+0000","last_change":"2024-09-10T21:24:10.708595+0000","last_active":"2024-09-10T21:24:16.766139+0000","last_peered":"2024-09-10T21:24:16.766139+0000","last_clean":"2024-09-10T21:24:16.766139+0000","last_became_active":"2024-09-10T21:24:10.708315+0000","last_became_peered":"2024-09-10T21:24:10.708315+0000","last_unstale":"2024-09-10T21:24:16.766139+0000","last_undegraded":"2024-09-10T21:24:16.766139+0000","last_fullsized":"2024-09-10T21:24:16.766139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:39.966992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262962+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:24:17.262962+0000","last_peered":"2024-09-10T21:24:17.262962+0000","last_clean":"2024-09-10T21:24:17.262962+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:24:17.262962+0000","last_undegraded":"2024-09-10T21:24:17.262962+0000","last_fullsized":"2024-09-10T21:24:17.262962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766659+0000","last_change":"2024-09-10T21:24:15.771747+0000","last_active":"2024-09-10T21:24:16.766659+0000","last_peered":"2024-09-10T21:24:16.766659+0000","last_clean":"2024-09-10T21:24:16.766659+0000","last_became_active":"2024-09-10T21:24:15.771197+0000","last_became_peered":"2024-09-10T21:24:15.771197+0000","last_unstale":"2024-09-10T21:24:16.766659+0000","last_undegraded":"2024-09-10T21:24:16.766659+0000","last_fullsized":"2024-09-10T21:24:16.766659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:05.770255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766468+0000","last_change":"2024-09-10T21:24:11.740573+0000","last_active":"2024-09-10T21:24:16.766468+0000","last_peered":"2024-09-10T21:24:16.766468+0000","last_clean":"2024-09-10T21:24:16.766468+0000","last_became_active":"2024-09-10T21:24:11.738994+0000","last_became_peered":"2024-09-10T21:24:11.738994+0000","last_unstale":"2024-09-10T21:24:16.766468+0000","last_undegraded":"2024-09-10T21:24:16.766468+0000","last_fullsized":"2024-09-10T21:24:16.766468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:55.184362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770636+0000","last_change":"2024-09-10T21:24:10.721767+0000","last_active":"2024-09-10T21:24:16.770636+0000","last_peered":"2024-09-10T21:24:16.770636+0000","last_clean":"2024-09-10T21:24:16.770636+0000","last_became_active":"2024-09-10T21:24:10.721444+0000","last_became_peered":"2024-09-10T21:24:10.721444+0000","last_unstale":"2024-09-10T21:24:16.770636+0000","last_undegraded":"2024-09-10T21:24:16.770636+0000","last_fullsized":"2024-09-10T21:24:16.770636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:38.215394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263060+0000","last_change":"2024-09-10T21:24:15.770594+0000","last_active":"2024-09-10T21:24:17.263060+0000","last_peered":"2024-09-10T21:24:17.263060+0000","last_clean":"2024-09-10T21:24:17.263060+0000","last_became_active":"2024-09-10T21:24:15.769704+0000","last_became_peered":"2024-09-10T21:24:15.769704+0000","last_unstale":"2024-09-10T21:24:17.263060+0000","last_undegraded":"2024-09-10T21:24:17.263060+0000","last_fullsized":"2024-09-10T21:24:17.263060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:56.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264863+0000","last_change":"2024-09-10T21:24:11.743773+0000","last_active":"2024-09-10T21:24:17.264863+0000","last_peered":"2024-09-10T21:24:17.264863+0000","last_clean":"2024-09-10T21:24:17.264863+0000","last_became_active":"2024-09-10T21:24:11.743497+0000","last_became_peered":"2024-09-10T21:24:11.743497+0000","last_unstale":"2024-09-10T21:24:17.264863+0000","last_undegraded":"2024-09-10T21:24:17.264863+0000","last_fullsized":"2024-09-10T21:24:17.264863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265338+0000","last_change":"2024-09-10T21:24:10.716493+0000","last_active":"2024-09-10T21:24:17.265338+0000","last_peered":"2024-09-10T21:24:17.265338+0000","last_clean":"2024-09-10T21:24:17.265338+0000","last_became_active":"2024-09-10T21:24:10.716126+0000","last_became_peered":"2024-09-10T21:24:10.716126+0000","last_unstale":"2024-09-10T21:24:17.265338+0000","last_undegraded":"2024-09-10T21:24:17.265338+0000","last_fullsized":"2024-09-10T21:24:17.265338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264298+0000","last_change":"2024-09-10T21:24:15.791234+0000","last_active":"2024-09-10T21:24:17.264298+0000","last_peered":"2024-09-10T21:24:17.264298+0000","last_clean":"2024-09-10T21:24:17.264298+0000","last_became_active":"2024-09-10T21:24:15.790862+0000","last_became_peered":"2024-09-10T21:24:15.790862+0000","last_unstale":"2024-09-10T21:24:17.264298+0000","last_undegraded":"2024-09-10T21:24:17.264298+0000","last_fullsized":"2024-09-10T21:24:17.264298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:41.522844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265147+0000","last_change":"2024-09-10T21:24:11.742947+0000","last_active":"2024-09-10T21:24:17.265147+0000","last_peered":"2024-09-10T21:24:17.265147+0000","last_clean":"2024-09-10T21:24:17.265147+0000","last_became_active":"2024-09-10T21:24:11.741967+0000","last_became_peered":"2024-09-10T21:24:11.741967+0000","last_unstale":"2024-09-10T21:24:17.265147+0000","last_undegraded":"2024-09-10T21:24:17.265147+0000","last_fullsized":"2024-09-10T21:24:17.265147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266168+0000","last_change":"2024-09-10T21:24:10.715721+0000","last_active":"2024-09-10T21:24:17.266168+0000","last_peered":"2024-09-10T21:24:17.266168+0000","last_clean":"2024-09-10T21:24:17.266168+0000","last_became_active":"2024-09-10T21:24:10.715385+0000","last_became_peered":"2024-09-10T21:24:10.715385+0000","last_unstale":"2024-09-10T21:24:17.266168+0000","last_undegraded":"2024-09-10T21:24:17.266168+0000","last_fullsized":"2024-09-10T21:24:17.266168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:30.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765435+0000","last_change":"2024-09-10T21:24:15.770313+0000","last_active":"2024-09-10T21:24:16.765435+0000","last_peered":"2024-09-10T21:24:16.765435+0000","last_clean":"2024-09-10T21:24:16.765435+0000","last_became_active":"2024-09-10T21:24:15.769934+0000","last_became_peered":"2024-09-10T21:24:15.769934+0000","last_unstale":"2024-09-10T21:24:16.765435+0000","last_undegraded":"2024-09-10T21:24:16.765435+0000","last_fullsized":"2024-09-10T21:24:16.765435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:58.034220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264104+0000","last_change":"2024-09-10T21:24:11.752428+0000","last_active":"2024-09-10T21:24:17.264104+0000","last_peered":"2024-09-10T21:24:17.264104+0000","last_clean":"2024-09-10T21:24:17.264104+0000","last_became_active":"2024-09-10T21:24:11.748165+0000","last_became_peered":"2024-09-10T21:24:11.748165+0000","last_unstale":"2024-09-10T21:24:17.264104+0000","last_undegraded":"2024-09-10T21:24:17.264104+0000","last_fullsized":"2024-09-10T21:24:17.264104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265190+0000","last_change":"2024-09-10T21:24:10.717653+0000","last_active":"2024-09-10T21:24:17.265190+0000","last_peered":"2024-09-10T21:24:17.265190+0000","last_clean":"2024-09-10T21:24:17.265190+0000","last_became_active":"2024-09-10T21:24:10.715137+0000","last_became_peered":"2024-09-10T21:24:10.715137+0000","last_unstale":"2024-09-10T21:24:17.265190+0000","last_undegraded":"2024-09-10T21:24:17.265190+0000","last_fullsized":"2024-09-10T21:24:17.265190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:01.438989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262702+0000","last_change":"2024-09-10T21:24:15.782725+0000","last_active":"2024-09-10T21:24:17.262702+0000","last_peered":"2024-09-10T21:24:17.262702+0000","last_clean":"2024-09-10T21:24:17.262702+0000","last_became_active":"2024-09-10T21:24:15.782199+0000","last_became_peered":"2024-09-10T21:24:15.782199+0000","last_unstale":"2024-09-10T21:24:17.262702+0000","last_undegraded":"2024-09-10T21:24:17.262702+0000","last_fullsized":"2024-09-10T21:24:17.262702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:15.226555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765554+0000","last_change":"2024-09-10T21:24:11.757400+0000","last_active":"2024-09-10T21:24:16.765554+0000","last_peered":"2024-09-10T21:24:16.765554+0000","last_clean":"2024-09-10T21:24:16.765554+0000","last_became_active":"2024-09-10T21:24:11.757112+0000","last_became_peered":"2024-09-10T21:24:11.757112+0000","last_unstale":"2024-09-10T21:24:16.765554+0000","last_undegraded":"2024-09-10T21:24:16.765554+0000","last_fullsized":"2024-09-10T21:24:16.765554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263864+0000","last_change":"2024-09-10T21:24:10.701950+0000","last_active":"2024-09-10T21:24:17.263864+0000","last_peered":"2024-09-10T21:24:17.263864+0000","last_clean":"2024-09-10T21:24:17.263864+0000","last_became_active":"2024-09-10T21:24:10.701652+0000","last_became_peered":"2024-09-10T21:24:10.701652+0000","last_unstale":"2024-09-10T21:24:17.263864+0000","last_undegraded":"2024-09-10T21:24:17.263864+0000","last_fullsized":"2024-09-10T21:24:17.263864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.595699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266111+0000","last_change":"2024-09-10T21:24:15.784288+0000","last_active":"2024-09-10T21:24:17.266111+0000","last_peered":"2024-09-10T21:24:17.266111+0000","last_clean":"2024-09-10T21:24:17.266111+0000","last_became_active":"2024-09-10T21:24:15.783890+0000","last_became_peered":"2024-09-10T21:24:15.783890+0000","last_unstale":"2024-09-10T21:24:17.266111+0000","last_undegraded":"2024-09-10T21:24:17.266111+0000","last_fullsized":"2024-09-10T21:24:17.266111+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:59.245790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262943+0000","last_change":"2024-09-10T21:24:11.736914+0000","last_active":"2024-09-10T21:24:17.262943+0000","last_peered":"2024-09-10T21:24:17.262943+0000","last_clean":"2024-09-10T21:24:17.262943+0000","last_became_active":"2024-09-10T21:24:11.734099+0000","last_became_peered":"2024-09-10T21:24:11.734099+0000","last_unstale":"2024-09-10T21:24:17.262943+0000","last_undegraded":"2024-09-10T21:24:17.262943+0000","last_fullsized":"2024-09-10T21:24:17.262943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264000+0000","last_change":"2024-09-10T21:24:10.715530+0000","last_active":"2024-09-10T21:24:17.264000+0000","last_peered":"2024-09-10T21:24:17.264000+0000","last_clean":"2024-09-10T21:24:17.264000+0000","last_became_active":"2024-09-10T21:24:10.715235+0000","last_became_peered":"2024-09-10T21:24:10.715235+0000","last_unstale":"2024-09-10T21:24:17.264000+0000","last_undegraded":"2024-09-10T21:24:17.264000+0000","last_fullsized":"2024-09-10T21:24:17.264000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:12.286087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264900+0000","last_change":"2024-09-10T21:24:15.784572+0000","last_active":"2024-09-10T21:24:17.264900+0000","last_peered":"2024-09-10T21:24:17.264900+0000","last_clean":"2024-09-10T21:24:17.264900+0000","last_became_active":"2024-09-10T21:24:15.784136+0000","last_became_peered":"2024-09-10T21:24:15.784136+0000","last_unstale":"2024-09-10T21:24:17.264900+0000","last_undegraded":"2024-09-10T21:24:17.264900+0000","last_fullsized":"2024-09-10T21:24:17.264900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:12.320754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765877+0000","last_change":"2024-09-10T21:24:11.755458+0000","last_active":"2024-09-10T21:24:16.765877+0000","last_peered":"2024-09-10T21:24:16.765877+0000","last_clean":"2024-09-10T21:24:16.765877+0000","last_became_active":"2024-09-10T21:24:11.755150+0000","last_became_peered":"2024-09-10T21:24:11.755150+0000","last_unstale":"2024-09-10T21:24:16.765877+0000","last_undegraded":"2024-09-10T21:24:16.765877+0000","last_fullsized":"2024-09-10T21:24:16.765877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:34.505560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765994+0000","last_change":"2024-09-10T21:24:10.717709+0000","last_active":"2024-09-10T21:24:16.765994+0000","last_peered":"2024-09-10T21:24:16.765994+0000","last_clean":"2024-09-10T21:24:16.765994+0000","last_became_active":"2024-09-10T21:24:10.717328+0000","last_became_peered":"2024-09-10T21:24:10.717328+0000","last_unstale":"2024-09-10T21:24:16.765994+0000","last_undegraded":"2024-09-10T21:24:16.765994+0000","last_fullsized":"2024-09-10T21:24:16.765994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:37.792691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770443+0000","last_change":"2024-09-10T21:24:15.788432+0000","last_active":"2024-09-10T21:24:16.770443+0000","last_peered":"2024-09-10T21:24:16.770443+0000","last_clean":"2024-09-10T21:24:16.770443+0000","last_became_active":"2024-09-10T21:24:15.788046+0000","last_became_peered":"2024-09-10T21:24:15.788046+0000","last_unstale":"2024-09-10T21:24:16.770443+0000","last_undegraded":"2024-09-10T21:24:16.770443+0000","last_fullsized":"2024-09-10T21:24:16.770443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:04.277488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769322+0000","last_change":"2024-09-10T21:24:11.737027+0000","last_active":"2024-09-10T21:24:16.769322+0000","last_peered":"2024-09-10T21:24:16.769322+0000","last_clean":"2024-09-10T21:24:16.769322+0000","last_became_active":"2024-09-10T21:24:11.736731+0000","last_became_peered":"2024-09-10T21:24:11.736731+0000","last_unstale":"2024-09-10T21:24:16.769322+0000","last_undegraded":"2024-09-10T21:24:16.769322+0000","last_fullsized":"2024-09-10T21:24:16.769322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769804+0000","last_change":"2024-09-10T21:24:10.712131+0000","last_active":"2024-09-10T21:24:16.769804+0000","last_peered":"2024-09-10T21:24:16.769804+0000","last_clean":"2024-09-10T21:24:16.769804+0000","last_became_active":"2024-09-10T21:24:10.711804+0000","last_became_peered":"2024-09-10T21:24:10.711804+0000","last_unstale":"2024-09-10T21:24:16.769804+0000","last_undegraded":"2024-09-10T21:24:16.769804+0000","last_fullsized":"2024-09-10T21:24:16.769804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766399+0000","last_change":"2024-09-10T21:24:15.786086+0000","last_active":"2024-09-10T21:24:16.766399+0000","last_peered":"2024-09-10T21:24:16.766399+0000","last_clean":"2024-09-10T21:24:16.766399+0000","last_became_active":"2024-09-10T21:24:15.785449+0000","last_became_peered":"2024-09-10T21:24:15.785449+0000","last_unstale":"2024-09-10T21:24:16.766399+0000","last_undegraded":"2024-09-10T21:24:16.766399+0000","last_fullsized":"2024-09-10T21:24:16.766399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:51.733023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.740413+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.740134+0000","last_became_peered":"2024-09-10T21:24:11.740134+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:22.624004+0000","last_change":"2024-09-10T21:24:10.707022+0000","last_active":"2024-09-10T21:24:22.624004+0000","last_peered":"2024-09-10T21:24:22.624004+0000","last_clean":"2024-09-10T21:24:22.624004+0000","last_became_active":"2024-09-10T21:24:10.706584+0000","last_became_peered":"2024-09-10T21:24:10.706584+0000","last_unstale":"2024-09-10T21:24:22.624004+0000","last_undegraded":"2024-09-10T21:24:22.624004+0000","last_fullsized":"2024-09-10T21:24:22.624004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:08.525705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265647+0000","last_change":"2024-09-10T21:24:15.789501+0000","last_active":"2024-09-10T21:24:17.265647+0000","last_peered":"2024-09-10T21:24:17.265647+0000","last_clean":"2024-09-10T21:24:17.265647+0000","last_became_active":"2024-09-10T21:24:15.789076+0000","last_became_peered":"2024-09-10T21:24:15.789076+0000","last_unstale":"2024-09-10T21:24:17.265647+0000","last_undegraded":"2024-09-10T21:24:17.265647+0000","last_fullsized":"2024-09-10T21:24:17.265647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:58.299617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766239+0000","last_change":"2024-09-10T21:24:11.757417+0000","last_active":"2024-09-10T21:24:16.766239+0000","last_peered":"2024-09-10T21:24:16.766239+0000","last_clean":"2024-09-10T21:24:16.766239+0000","last_became_active":"2024-09-10T21:24:11.757091+0000","last_became_peered":"2024-09-10T21:24:11.757091+0000","last_unstale":"2024-09-10T21:24:16.766239+0000","last_undegraded":"2024-09-10T21:24:16.766239+0000","last_fullsized":"2024-09-10T21:24:16.766239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:13.080850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770386+0000","last_change":"2024-09-10T21:24:10.707124+0000","last_active":"2024-09-10T21:24:16.770386+0000","last_peered":"2024-09-10T21:24:16.770386+0000","last_clean":"2024-09-10T21:24:16.770386+0000","last_became_active":"2024-09-10T21:24:10.706605+0000","last_became_peered":"2024-09-10T21:24:10.706605+0000","last_unstale":"2024-09-10T21:24:16.770386+0000","last_undegraded":"2024-09-10T21:24:16.770386+0000","last_fullsized":"2024-09-10T21:24:16.770386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262946+0000","last_change":"2024-09-10T21:24:15.776743+0000","last_active":"2024-09-10T21:24:17.262946+0000","last_peered":"2024-09-10T21:24:17.262946+0000","last_clean":"2024-09-10T21:24:17.262946+0000","last_became_active":"2024-09-10T21:24:15.774992+0000","last_became_peered":"2024-09-10T21:24:15.774992+0000","last_unstale":"2024-09-10T21:24:17.262946+0000","last_undegraded":"2024-09-10T21:24:17.262946+0000","last_fullsized":"2024-09-10T21:24:17.262946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:11.593663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265153+0000","last_change":"2024-09-10T21:24:11.750107+0000","last_active":"2024-09-10T21:24:17.265153+0000","last_peered":"2024-09-10T21:24:17.265153+0000","last_clean":"2024-09-10T21:24:17.265153+0000","last_became_active":"2024-09-10T21:24:11.749206+0000","last_became_peered":"2024-09-10T21:24:11.749206+0000","last_unstale":"2024-09-10T21:24:17.265153+0000","last_undegraded":"2024-09-10T21:24:17.265153+0000","last_fullsized":"2024-09-10T21:24:17.265153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265042+0000","last_change":"2024-09-10T21:24:10.711029+0000","last_active":"2024-09-10T21:24:17.265042+0000","last_peered":"2024-09-10T21:24:17.265042+0000","last_clean":"2024-09-10T21:24:17.265042+0000","last_became_active":"2024-09-10T21:24:10.710789+0000","last_became_peered":"2024-09-10T21:24:10.710789+0000","last_unstale":"2024-09-10T21:24:17.265042+0000","last_undegraded":"2024-09-10T21:24:17.265042+0000","last_fullsized":"2024-09-10T21:24:17.265042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:36.260277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768675+0000","last_change":"2024-09-10T21:24:15.779017+0000","last_active":"2024-09-10T21:24:16.768675+0000","last_peered":"2024-09-10T21:24:16.768675+0000","last_clean":"2024-09-10T21:24:16.768675+0000","last_became_active":"2024-09-10T21:24:15.774452+0000","last_became_peered":"2024-09-10T21:24:15.774452+0000","last_unstale":"2024-09-10T21:24:16.768675+0000","last_undegraded":"2024-09-10T21:24:16.768675+0000","last_fullsized":"2024-09-10T21:24:16.768675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:56.032301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264021+0000","last_change":"2024-09-10T21:24:11.748901+0000","last_active":"2024-09-10T21:24:17.264021+0000","last_peered":"2024-09-10T21:24:17.264021+0000","last_clean":"2024-09-10T21:24:17.264021+0000","last_became_active":"2024-09-10T21:24:11.748059+0000","last_became_peered":"2024-09-10T21:24:11.748059+0000","last_unstale":"2024-09-10T21:24:17.264021+0000","last_undegraded":"2024-09-10T21:24:17.264021+0000","last_fullsized":"2024-09-10T21:24:17.264021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:10.697344+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:10.697030+0000","last_became_peered":"2024-09-10T21:24:10.697030+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264496+0000","last_change":"2024-09-10T21:24:15.775122+0000","last_active":"2024-09-10T21:24:17.264496+0000","last_peered":"2024-09-10T21:24:17.264496+0000","last_clean":"2024-09-10T21:24:17.264496+0000","last_became_active":"2024-09-10T21:24:15.774561+0000","last_became_peered":"2024-09-10T21:24:15.774561+0000","last_unstale":"2024-09-10T21:24:17.264496+0000","last_undegraded":"2024-09-10T21:24:17.264496+0000","last_fullsized":"2024-09-10T21:24:17.264496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:19.897395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266159+0000","last_change":"2024-09-10T21:24:15.790852+0000","last_active":"2024-09-10T21:24:17.266159+0000","last_peered":"2024-09-10T21:24:17.266159+0000","last_clean":"2024-09-10T21:24:17.266159+0000","last_became_active":"2024-09-10T21:24:15.786896+0000","last_became_peered":"2024-09-10T21:24:15.786896+0000","last_unstale":"2024-09-10T21:24:17.266159+0000","last_undegraded":"2024-09-10T21:24:17.266159+0000","last_fullsized":"2024-09-10T21:24:17.266159+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:48.900914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264746+0000","last_change":"2024-09-10T21:24:11.732564+0000","last_active":"2024-09-10T21:24:17.264746+0000","last_peered":"2024-09-10T21:24:17.264746+0000","last_clean":"2024-09-10T21:24:17.264746+0000","last_became_active":"2024-09-10T21:24:11.732106+0000","last_became_peered":"2024-09-10T21:24:11.732106+0000","last_unstale":"2024-09-10T21:24:17.264746+0000","last_undegraded":"2024-09-10T21:24:17.264746+0000","last_fullsized":"2024-09-10T21:24:17.264746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.574642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765650+0000","last_change":"2024-09-10T21:24:10.714693+0000","last_active":"2024-09-10T21:24:16.765650+0000","last_peered":"2024-09-10T21:24:16.765650+0000","last_clean":"2024-09-10T21:24:16.765650+0000","last_became_active":"2024-09-10T21:24:10.714223+0000","last_became_peered":"2024-09-10T21:24:10.714223+0000","last_unstale":"2024-09-10T21:24:16.765650+0000","last_undegraded":"2024-09-10T21:24:16.765650+0000","last_fullsized":"2024-09-10T21:24:16.765650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:06.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766392+0000","last_change":"2024-09-10T21:24:15.784105+0000","last_active":"2024-09-10T21:24:16.766392+0000","last_peered":"2024-09-10T21:24:16.766392+0000","last_clean":"2024-09-10T21:24:16.766392+0000","last_became_active":"2024-09-10T21:24:15.783500+0000","last_became_peered":"2024-09-10T21:24:15.783500+0000","last_unstale":"2024-09-10T21:24:16.766392+0000","last_undegraded":"2024-09-10T21:24:16.766392+0000","last_fullsized":"2024-09-10T21:24:16.766392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.513517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263956+0000","last_change":"2024-09-10T21:24:11.738285+0000","last_active":"2024-09-10T21:24:17.263956+0000","last_peered":"2024-09-10T21:24:17.263956+0000","last_clean":"2024-09-10T21:24:17.263956+0000","last_became_active":"2024-09-10T21:24:11.738001+0000","last_became_peered":"2024-09-10T21:24:11.738001+0000","last_unstale":"2024-09-10T21:24:17.263956+0000","last_undegraded":"2024-09-10T21:24:17.263956+0000","last_fullsized":"2024-09-10T21:24:17.263956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769007+0000","last_change":"2024-09-10T21:24:10.710823+0000","last_active":"2024-09-10T21:24:16.769007+0000","last_peered":"2024-09-10T21:24:16.769007+0000","last_clean":"2024-09-10T21:24:16.769007+0000","last_became_active":"2024-09-10T21:24:10.710231+0000","last_became_peered":"2024-09-10T21:24:10.710231+0000","last_unstale":"2024-09-10T21:24:16.769007+0000","last_undegraded":"2024-09-10T21:24:16.769007+0000","last_fullsized":"2024-09-10T21:24:16.769007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:39.865360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765929+0000","last_change":"2024-09-10T21:24:10.711002+0000","last_active":"2024-09-10T21:24:16.765929+0000","last_peered":"2024-09-10T21:24:16.765929+0000","last_clean":"2024-09-10T21:24:16.765929+0000","last_became_active":"2024-09-10T21:24:10.710723+0000","last_became_peered":"2024-09-10T21:24:10.710723+0000","last_unstale":"2024-09-10T21:24:16.765929+0000","last_undegraded":"2024-09-10T21:24:16.765929+0000","last_fullsized":"2024-09-10T21:24:16.765929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:52.955941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:11.744353+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:11.743970+0000","last_became_peered":"2024-09-10T21:24:11.743970+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:35.066178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265974+0000","last_change":"2024-09-10T21:24:15.791043+0000","last_active":"2024-09-10T21:24:17.265974+0000","last_peered":"2024-09-10T21:24:17.265974+0000","last_clean":"2024-09-10T21:24:17.265974+0000","last_became_active":"2024-09-10T21:24:15.790322+0000","last_became_peered":"2024-09-10T21:24:15.790322+0000","last_unstale":"2024-09-10T21:24:17.265974+0000","last_undegraded":"2024-09-10T21:24:17.265974+0000","last_fullsized":"2024-09-10T21:24:17.265974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:18.317248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264568+0000","last_change":"2024-09-10T21:24:10.717526+0000","last_active":"2024-09-10T21:24:17.264568+0000","last_peered":"2024-09-10T21:24:17.264568+0000","last_clean":"2024-09-10T21:24:17.264568+0000","last_became_active":"2024-09-10T21:24:10.715368+0000","last_became_peered":"2024-09-10T21:24:10.715368+0000","last_unstale":"2024-09-10T21:24:17.264568+0000","last_undegraded":"2024-09-10T21:24:17.264568+0000","last_fullsized":"2024-09-10T21:24:17.264568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263197+0000","last_change":"2024-09-10T21:24:11.733617+0000","last_active":"2024-09-10T21:24:17.263197+0000","last_peered":"2024-09-10T21:24:17.263197+0000","last_clean":"2024-09-10T21:24:17.263197+0000","last_became_active":"2024-09-10T21:24:11.733343+0000","last_became_peered":"2024-09-10T21:24:11.733343+0000","last_unstale":"2024-09-10T21:24:17.263197+0000","last_undegraded":"2024-09-10T21:24:17.263197+0000","last_fullsized":"2024-09-10T21:24:17.263197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770202+0000","last_change":"2024-09-10T21:24:15.787912+0000","last_active":"2024-09-10T21:24:16.770202+0000","last_peered":"2024-09-10T21:24:16.770202+0000","last_clean":"2024-09-10T21:24:16.770202+0000","last_became_active":"2024-09-10T21:24:15.787441+0000","last_became_peered":"2024-09-10T21:24:15.787441+0000","last_unstale":"2024-09-10T21:24:16.770202+0000","last_undegraded":"2024-09-10T21:24:16.770202+0000","last_fullsized":"2024-09-10T21:24:16.770202+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:22.856812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262658+0000","last_change":"2024-09-10T21:24:11.747086+0000","last_active":"2024-09-10T21:24:17.262658+0000","last_peered":"2024-09-10T21:24:17.262658+0000","last_clean":"2024-09-10T21:24:17.262658+0000","last_became_active":"2024-09-10T21:24:11.746780+0000","last_became_peered":"2024-09-10T21:24:11.746780+0000","last_unstale":"2024-09-10T21:24:17.262658+0000","last_undegraded":"2024-09-10T21:24:17.262658+0000","last_fullsized":"2024-09-10T21:24:17.262658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:13.288615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769236+0000","last_change":"2024-09-10T21:24:11.732475+0000","last_active":"2024-09-10T21:24:16.769236+0000","last_peered":"2024-09-10T21:24:16.769236+0000","last_clean":"2024-09-10T21:24:16.769236+0000","last_became_active":"2024-09-10T21:24:11.731007+0000","last_became_peered":"2024-09-10T21:24:11.731007+0000","last_unstale":"2024-09-10T21:24:16.769236+0000","last_undegraded":"2024-09-10T21:24:16.769236+0000","last_fullsized":"2024-09-10T21:24:16.769236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765707+0000","last_change":"2024-09-10T21:24:11.744942+0000","last_active":"2024-09-10T21:24:16.765707+0000","last_peered":"2024-09-10T21:24:16.765707+0000","last_clean":"2024-09-10T21:24:16.765707+0000","last_became_active":"2024-09-10T21:24:11.744626+0000","last_became_peered":"2024-09-10T21:24:11.744626+0000","last_unstale":"2024-09-10T21:24:16.765707+0000","last_undegraded":"2024-09-10T21:24:16.765707+0000","last_fullsized":"2024-09-10T21:24:16.765707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:37.461417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264525+0000","last_change":"2024-09-10T21:24:11.745907+0000","last_active":"2024-09-10T21:24:17.264525+0000","last_peered":"2024-09-10T21:24:17.264525+0000","last_clean":"2024-09-10T21:24:17.264525+0000","last_became_active":"2024-09-10T21:24:11.745583+0000","last_became_peered":"2024-09-10T21:24:11.745583+0000","last_unstale":"2024-09-10T21:24:17.264525+0000","last_undegraded":"2024-09-10T21:24:17.264525+0000","last_fullsized":"2024-09-10T21:24:17.264525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:50.845294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770469+0000","last_change":"2024-09-10T21:24:11.756001+0000","last_active":"2024-09-10T21:24:16.770469+0000","last_peered":"2024-09-10T21:24:16.770469+0000","last_clean":"2024-09-10T21:24:16.770469+0000","last_became_active":"2024-09-10T21:24:11.755719+0000","last_became_peered":"2024-09-10T21:24:11.755719+0000","last_unstale":"2024-09-10T21:24:16.770469+0000","last_undegraded":"2024-09-10T21:24:16.770469+0000","last_fullsized":"2024-09-10T21:24:16.770469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:04.237158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769805+0000","last_change":"2024-09-10T21:24:11.740025+0000","last_active":"2024-09-10T21:24:16.769805+0000","last_peered":"2024-09-10T21:24:16.769805+0000","last_clean":"2024-09-10T21:24:16.769805+0000","last_became_active":"2024-09-10T21:24:11.739742+0000","last_became_peered":"2024-09-10T21:24:11.739742+0000","last_unstale":"2024-09-10T21:24:16.769805+0000","last_undegraded":"2024-09-10T21:24:16.769805+0000","last_fullsized":"2024-09-10T21:24:16.769805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:29.650259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263932+0000","last_change":"2024-09-10T21:24:11.749805+0000","last_active":"2024-09-10T21:24:17.263932+0000","last_peered":"2024-09-10T21:24:17.263932+0000","last_clean":"2024-09-10T21:24:17.263932+0000","last_became_active":"2024-09-10T21:24:11.748452+0000","last_became_peered":"2024-09-10T21:24:11.748452+0000","last_unstale":"2024-09-10T21:24:17.263932+0000","last_undegraded":"2024-09-10T21:24:17.263932+0000","last_fullsized":"2024-09-10T21:24:17.263932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:36.508158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264740+0000","last_change":"2024-09-10T21:24:11.750687+0000","last_active":"2024-09-10T21:24:17.264740+0000","last_peered":"2024-09-10T21:24:17.264740+0000","last_clean":"2024-09-10T21:24:17.264740+0000","last_became_active":"2024-09-10T21:24:11.749899+0000","last_became_peered":"2024-09-10T21:24:11.749899+0000","last_unstale":"2024-09-10T21:24:17.264740+0000","last_undegraded":"2024-09-10T21:24:17.264740+0000","last_fullsized":"2024-09-10T21:24:17.264740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765619+0000","last_change":"2024-09-10T21:24:11.734778+0000","last_active":"2024-09-10T21:24:16.765619+0000","last_peered":"2024-09-10T21:24:16.765619+0000","last_clean":"2024-09-10T21:24:16.765619+0000","last_became_active":"2024-09-10T21:24:11.734510+0000","last_became_peered":"2024-09-10T21:24:11.734510+0000","last_unstale":"2024-09-10T21:24:16.765619+0000","last_undegraded":"2024-09-10T21:24:16.765619+0000","last_fullsized":"2024-09-10T21:24:16.765619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:45.729068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263928+0000","last_change":"2024-09-10T21:24:11.758730+0000","last_active":"2024-09-10T21:24:17.263928+0000","last_peered":"2024-09-10T21:24:17.263928+0000","last_clean":"2024-09-10T21:24:17.263928+0000","last_became_active":"2024-09-10T21:24:11.758453+0000","last_became_peered":"2024-09-10T21:24:11.758453+0000","last_unstale":"2024-09-10T21:24:17.263928+0000","last_undegraded":"2024-09-10T21:24:17.263928+0000","last_fullsized":"2024-09-10T21:24:17.263928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:20.045933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263026+0000","last_change":"2024-09-10T21:24:11.727975+0000","last_active":"2024-09-10T21:24:17.263026+0000","last_peered":"2024-09-10T21:24:17.263026+0000","last_clean":"2024-09-10T21:24:17.263026+0000","last_became_active":"2024-09-10T21:24:11.727715+0000","last_became_peered":"2024-09-10T21:24:11.727715+0000","last_unstale":"2024-09-10T21:24:17.263026+0000","last_undegraded":"2024-09-10T21:24:17.263026+0000","last_fullsized":"2024-09-10T21:24:17.263026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264447+0000","last_change":"2024-09-10T21:24:11.752499+0000","last_active":"2024-09-10T21:24:17.264447+0000","last_peered":"2024-09-10T21:24:17.264447+0000","last_clean":"2024-09-10T21:24:17.264447+0000","last_became_active":"2024-09-10T21:24:11.748931+0000","last_became_peered":"2024-09-10T21:24:11.748931+0000","last_unstale":"2024-09-10T21:24:17.264447+0000","last_undegraded":"2024-09-10T21:24:17.264447+0000","last_fullsized":"2024-09-10T21:24:17.264447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:48.334318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264810+0000","last_change":"2024-09-10T21:24:11.742340+0000","last_active":"2024-09-10T21:24:17.264810+0000","last_peered":"2024-09-10T21:24:17.264810+0000","last_clean":"2024-09-10T21:24:17.264810+0000","last_became_active":"2024-09-10T21:24:11.741702+0000","last_became_peered":"2024-09-10T21:24:11.741702+0000","last_unstale":"2024-09-10T21:24:17.264810+0000","last_undegraded":"2024-09-10T21:24:17.264810+0000","last_fullsized":"2024-09-10T21:24:17.264810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766584+0000","last_change":"2024-09-10T21:24:11.744751+0000","last_active":"2024-09-10T21:24:16.766584+0000","last_peered":"2024-09-10T21:24:16.766584+0000","last_clean":"2024-09-10T21:24:16.766584+0000","last_became_active":"2024-09-10T21:24:11.744431+0000","last_became_peered":"2024-09-10T21:24:11.744431+0000","last_unstale":"2024-09-10T21:24:16.766584+0000","last_undegraded":"2024-09-10T21:24:16.766584+0000","last_fullsized":"2024-09-10T21:24:16.766584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:11.693990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770590+0000","last_change":"2024-09-10T21:24:11.748605+0000","last_active":"2024-09-10T21:24:16.770590+0000","last_peered":"2024-09-10T21:24:16.770590+0000","last_clean":"2024-09-10T21:24:16.770590+0000","last_became_active":"2024-09-10T21:24:11.748279+0000","last_became_peered":"2024-09-10T21:24:11.748279+0000","last_unstale":"2024-09-10T21:24:16.770590+0000","last_undegraded":"2024-09-10T21:24:16.770590+0000","last_fullsized":"2024-09-10T21:24:16.770590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766580+0000","last_change":"2024-09-10T21:24:11.750748+0000","last_active":"2024-09-10T21:24:16.766580+0000","last_peered":"2024-09-10T21:24:16.766580+0000","last_clean":"2024-09-10T21:24:16.766580+0000","last_became_active":"2024-09-10T21:24:11.750317+0000","last_became_peered":"2024-09-10T21:24:11.750317+0000","last_unstale":"2024-09-10T21:24:16.766580+0000","last_undegraded":"2024-09-10T21:24:16.766580+0000","last_fullsized":"2024-09-10T21:24:16.766580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:08.259327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765529+0000","last_change":"2024-09-10T21:24:11.757620+0000","last_active":"2024-09-10T21:24:16.765529+0000","last_peered":"2024-09-10T21:24:16.765529+0000","last_clean":"2024-09-10T21:24:16.765529+0000","last_became_active":"2024-09-10T21:24:11.757088+0000","last_became_peered":"2024-09-10T21:24:11.757088+0000","last_unstale":"2024-09-10T21:24:16.765529+0000","last_undegraded":"2024-09-10T21:24:16.765529+0000","last_fullsized":"2024-09-10T21:24:16.765529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263812+0000","last_change":"2024-09-10T21:24:11.745932+0000","last_active":"2024-09-10T21:24:17.263812+0000","last_peered":"2024-09-10T21:24:17.263812+0000","last_clean":"2024-09-10T21:24:17.263812+0000","last_became_active":"2024-09-10T21:24:11.745682+0000","last_became_peered":"2024-09-10T21:24:11.745682+0000","last_unstale":"2024-09-10T21:24:17.263812+0000","last_undegraded":"2024-09-10T21:24:17.263812+0000","last_fullsized":"2024-09-10T21:24:17.263812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769215+0000","last_change":"2024-09-10T21:24:11.740847+0000","last_active":"2024-09-10T21:24:16.769215+0000","last_peered":"2024-09-10T21:24:16.769215+0000","last_clean":"2024-09-10T21:24:16.769215+0000","last_became_active":"2024-09-10T21:24:11.740548+0000","last_became_peered":"2024-09-10T21:24:11.740548+0000","last_unstale":"2024-09-10T21:24:16.769215+0000","last_undegraded":"2024-09-10T21:24:16.769215+0000","last_fullsized":"2024-09-10T21:24:16.769215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770304+0000","last_change":"2024-09-10T21:24:11.742902+0000","last_active":"2024-09-10T21:24:16.770304+0000","last_peered":"2024-09-10T21:24:16.770304+0000","last_clean":"2024-09-10T21:24:16.770304+0000","last_became_active":"2024-09-10T21:24:11.742449+0000","last_became_peered":"2024-09-10T21:24:11.742449+0000","last_unstale":"2024-09-10T21:24:16.770304+0000","last_undegraded":"2024-09-10T21:24:16.770304+0000","last_fullsized":"2024-09-10T21:24:16.770304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:25.302008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264763+0000","last_change":"2024-09-10T21:24:11.742660+0000","last_active":"2024-09-10T21:24:17.264763+0000","last_peered":"2024-09-10T21:24:17.264763+0000","last_clean":"2024-09-10T21:24:17.264763+0000","last_became_active":"2024-09-10T21:24:11.742069+0000","last_became_peered":"2024-09-10T21:24:11.742069+0000","last_unstale":"2024-09-10T21:24:17.264763+0000","last_undegraded":"2024-09-10T21:24:17.264763+0000","last_fullsized":"2024-09-10T21:24:17.264763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:35.526539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264897+0000","last_change":"2024-09-10T21:24:11.743498+0000","last_active":"2024-09-10T21:24:17.264897+0000","last_peered":"2024-09-10T21:24:17.264897+0000","last_clean":"2024-09-10T21:24:17.264897+0000","last_became_active":"2024-09-10T21:24:11.743017+0000","last_became_peered":"2024-09-10T21:24:11.743017+0000","last_unstale":"2024-09-10T21:24:17.264897+0000","last_undegraded":"2024-09-10T21:24:17.264897+0000","last_fullsized":"2024-09-10T21:24:17.264897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.754599+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.754341+0000","last_became_peered":"2024-09-10T21:24:11.754341+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264501+0000","last_change":"2024-09-10T21:24:11.726485+0000","last_active":"2024-09-10T21:24:17.264501+0000","last_peered":"2024-09-10T21:24:17.264501+0000","last_clean":"2024-09-10T21:24:17.264501+0000","last_became_active":"2024-09-10T21:24:11.726157+0000","last_became_peered":"2024-09-10T21:24:11.726157+0000","last_unstale":"2024-09-10T21:24:17.264501+0000","last_undegraded":"2024-09-10T21:24:17.264501+0000","last_fullsized":"2024-09-10T21:24:17.264501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263664+0000","last_change":"2024-09-10T21:24:11.754793+0000","last_active":"2024-09-10T21:24:17.263664+0000","last_peered":"2024-09-10T21:24:17.263664+0000","last_clean":"2024-09-10T21:24:17.263664+0000","last_became_active":"2024-09-10T21:24:11.754372+0000","last_became_peered":"2024-09-10T21:24:11.754372+0000","last_unstale":"2024-09-10T21:24:17.263664+0000","last_undegraded":"2024-09-10T21:24:17.263664+0000","last_fullsized":"2024-09-10T21:24:17.263664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:25.692729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263914+0000","last_change":"2024-09-10T21:24:11.741830+0000","last_active":"2024-09-10T21:24:17.263914+0000","last_peered":"2024-09-10T21:24:17.263914+0000","last_clean":"2024-09-10T21:24:17.263914+0000","last_became_active":"2024-09-10T21:24:11.741543+0000","last_became_peered":"2024-09-10T21:24:11.741543+0000","last_unstale":"2024-09-10T21:24:17.263914+0000","last_undegraded":"2024-09-10T21:24:17.263914+0000","last_fullsized":"2024-09-10T21:24:17.263914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263114+0000","last_change":"2024-09-10T21:24:11.744958+0000","last_active":"2024-09-10T21:24:17.263114+0000","last_peered":"2024-09-10T21:24:17.263114+0000","last_clean":"2024-09-10T21:24:17.263114+0000","last_became_active":"2024-09-10T21:24:11.744611+0000","last_became_peered":"2024-09-10T21:24:11.744611+0000","last_unstale":"2024-09-10T21:24:17.263114+0000","last_undegraded":"2024-09-10T21:24:17.263114+0000","last_fullsized":"2024-09-10T21:24:17.263114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264460+0000","last_change":"2024-09-10T21:24:11.742259+0000","last_active":"2024-09-10T21:24:17.264460+0000","last_peered":"2024-09-10T21:24:17.264460+0000","last_clean":"2024-09-10T21:24:17.264460+0000","last_became_active":"2024-09-10T21:24:11.742010+0000","last_became_peered":"2024-09-10T21:24:11.742010+0000","last_unstale":"2024-09-10T21:24:17.264460+0000","last_undegraded":"2024-09-10T21:24:17.264460+0000","last_fullsized":"2024-09-10T21:24:17.264460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263425+0000","last_change":"2024-09-10T21:24:11.747125+0000","last_active":"2024-09-10T21:24:17.263425+0000","last_peered":"2024-09-10T21:24:17.263425+0000","last_clean":"2024-09-10T21:24:17.263425+0000","last_became_active":"2024-09-10T21:24:11.746844+0000","last_became_peered":"2024-09-10T21:24:11.746844+0000","last_unstale":"2024-09-10T21:24:17.263425+0000","last_undegraded":"2024-09-10T21:24:17.263425+0000","last_fullsized":"2024-09-10T21:24:17.263425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:28.995110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265138+0000","last_change":"2024-09-10T21:24:11.737282+0000","last_active":"2024-09-10T21:24:17.265138+0000","last_peered":"2024-09-10T21:24:17.265138+0000","last_clean":"2024-09-10T21:24:17.265138+0000","last_became_active":"2024-09-10T21:24:11.736985+0000","last_became_peered":"2024-09-10T21:24:11.736985+0000","last_unstale":"2024-09-10T21:24:17.265138+0000","last_undegraded":"2024-09-10T21:24:17.265138+0000","last_fullsized":"2024-09-10T21:24:17.265138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264431+0000","last_change":"2024-09-10T21:24:11.738254+0000","last_active":"2024-09-10T21:24:17.264431+0000","last_peered":"2024-09-10T21:24:17.264431+0000","last_clean":"2024-09-10T21:24:17.264431+0000","last_became_active":"2024-09-10T21:24:11.737963+0000","last_became_peered":"2024-09-10T21:24:11.737963+0000","last_unstale":"2024-09-10T21:24:17.264431+0000","last_undegraded":"2024-09-10T21:24:17.264431+0000","last_fullsized":"2024-09-10T21:24:17.264431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265077+0000","last_change":"2024-09-10T21:24:11.743284+0000","last_active":"2024-09-10T21:24:17.265077+0000","last_peered":"2024-09-10T21:24:17.265077+0000","last_clean":"2024-09-10T21:24:17.265077+0000","last_became_active":"2024-09-10T21:24:11.742655+0000","last_became_peered":"2024-09-10T21:24:11.742655+0000","last_unstale":"2024-09-10T21:24:17.265077+0000","last_undegraded":"2024-09-10T21:24:17.265077+0000","last_fullsized":"2024-09-10T21:24:17.265077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:24:33.297 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:24:33.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T21:24:33.297 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T21:24:33.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 -- 172.21.15.97:0/757494150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3850003a40 msgr2=0x7f3850003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 --2- 172.21.15.97:0/757494150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3850003a40 0x7f3850003e60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f385c00c2e0 tx=0x7f385c01f6e0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3871200640 1 -- 172.21.15.97:0/757494150 reap_dead start 2024-09-10T21:24:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 -- 172.21.15.97:0/757494150 shutdown_connections 2024-09-10T21:24:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 -- 172.21.15.97:0/757494150 >> 172.21.15.97:0/757494150 conn(0x7f386c0b1780 msgr2=0x7f386c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 -- 172.21.15.97:0/757494150 shutdown_connections 2024-09-10T21:24:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 -- 172.21.15.97:0/757494150 wait complete. 2024-09-10T21:24:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 Processor -- start 2024-09-10T21:24:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.468+0000 7f3873200640 1 -- start start 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3850003a40 0x7f386c13ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c1f30b0 0x7f386c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f386c15c5c0 0x7f386c141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f386c06f060 con 0x7f386c1f30b0 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f386c15b210 con 0x7f3850003a40 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f386c06f1e0 con 0x7f386c15c5c0 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c1f30b0 0x7f386c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c1f30b0 0x7f386c15c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41114/0 (socket says 172.21.15.97:41114) 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 -- 172.21.15.97:0/1632929792 learned_addr learned my addr 172.21.15.97:0/1632929792 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386be00640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3850003a40 0x7f386c13ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3871200640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f386c15c5c0 0x7f386c141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 -- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f386c15c5c0 msgr2=0x7f386c141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f386c15c5c0 0x7f386c141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 -- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3850003a40 msgr2=0x7f386c13ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3850003a40 0x7f386c13ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 -- 172.21.15.97:0/1632929792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c142430 con 0x7f386c1f30b0 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f386b400640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c1f30b0 0x7f386c15c080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3858011a80 tx=0x7f3858011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3863e00640 1 -- 172.21.15.97:0/1632929792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385800fe10 con 0x7f386c1f30b0 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3863e00640 1 -- 172.21.15.97:0/1632929792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3858016070 con 0x7f386c1f30b0 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3863e00640 1 -- 172.21.15.97:0/1632929792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858012780 con 0x7f386c1f30b0 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1f25d0 con 0x7f386c1f30b0 2024-09-10T21:24:33.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c141c40 con 0x7f386c1f30b0 2024-09-10T21:24:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.472+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386c1f23e0 con 0x7f386c1f30b0 2024-09-10T21:24:33.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.476+0000 7f3863e00640 1 -- 172.21.15.97:0/1632929792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3858012920 con 0x7f386c1f30b0 2024-09-10T21:24:33.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.476+0000 7f3863e00640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f384c0725e0 0x7f384c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.476+0000 7f3863e00640 1 -- 172.21.15.97:0/1632929792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f38580940f0 con 0x7f386c1f30b0 2024-09-10T21:24:33.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.476+0000 7f386be00640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f384c0725e0 0x7f384c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.476+0000 7f386be00640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f384c0725e0 0x7f384c074aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f385c00c2e0 tx=0x7f385c01c520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:33.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.480+0000 7f3863e00640 1 -- 172.21.15.97:0/1632929792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f385805eda0 con 0x7f386c1f30b0 2024-09-10T21:24:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f386c0cf450 con 0x7f386c1f30b0 2024-09-10T21:24:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3863e00640 1 -- 172.21.15.97:0/1632929792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7f38580637d0 con 0x7f386c1f30b0 2024-09-10T21:24:33.655 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f384c0725e0 msgr2=0x7f384c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f384c0725e0 0x7f384c074aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f385c00c2e0 tx=0x7f385c01c520 comp rx=0 tx=0).stop 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c1f30b0 msgr2=0x7f386c15c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c1f30b0 0x7f386c15c080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3858011a80 tx=0x7f3858011f40 comp rx=0 tx=0).stop 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 shutdown_connections 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f384c0725e0 0x7f384c074aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f386c15c5c0 0x7f386c141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386c1f30b0 0x7f386c15c080 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 --2- 172.21.15.97:0/1632929792 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3850003a40 0x7f386c13ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 >> 172.21.15.97:0/1632929792 conn(0x7f386c0b1780 msgr2=0x7f386c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:33.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 shutdown_connections 2024-09-10T21:24:33.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.648+0000 7f3873200640 1 -- 172.21.15.97:0/1632929792 wait complete. 2024-09-10T21:24:33.675 INFO:teuthology.orchestra.run.smithi097.stdout:[{"version":27,"timestamp":"2024-09-10T21:24:00.829733+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"3032.483965"}]}] 2024-09-10T21:24:33.675 INFO:tasks.ceph_manager:config epoch is 27 2024-09-10T21:24:33.675 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T21:24:33.675 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T21:24:33.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T21:24:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- 172.21.15.97:0/4240694206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc070460 msgr2=0x7f03cc0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 --2- 172.21.15.97:0/4240694206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc070460 0x7f03cc0b2a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009bf0 tx=0x7f03c401cbe0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- 172.21.15.97:0/4240694206 shutdown_connections 2024-09-10T21:24:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 --2- 172.21.15.97:0/4240694206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc070460 0x7f03cc0b2a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- 172.21.15.97:0/4240694206 >> 172.21.15.97:0/4240694206 conn(0x7f03cc0b1780 msgr2=0x7f03cc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- 172.21.15.97:0/4240694206 shutdown_connections 2024-09-10T21:24:33.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- 172.21.15.97:0/4240694206 wait complete. 2024-09-10T21:24:33.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 Processor -- start 2024-09-10T21:24:33.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- start start 2024-09-10T21:24:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03cc070460 0x7f03cc142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc143f90 0x7f03cc13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03cc144690 0x7f03cc06f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc168ee0 con 0x7f03cc070460 2024-09-10T21:24:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc168d60 con 0x7f03cc144690 2024-09-10T21:24:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d3800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f03cc169060 con 0x7f03cc143f90 2024-09-10T21:24:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03cc070460 0x7f03cc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03cc070460 0x7f03cc142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41120/0 (socket says 172.21.15.97:41120) 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.812+0000 7f03d0e00640 1 -- 172.21.15.97:0/3839650822 learned_addr learned my addr 172.21.15.97:0/3839650822 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d1800640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03cc144690 0x7f03cc06f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03c3e00640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc143f90 0x7f03cc13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d0e00640 1 -- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc143f90 msgr2=0x7f03cc13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d0e00640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc143f90 0x7f03cc13bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d0e00640 1 -- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03cc144690 msgr2=0x7f03cc06f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d0e00640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03cc144690 0x7f03cc06f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d0e00640 1 -- 172.21.15.97:0/3839650822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc13c2d0 con 0x7f03cc070460 2024-09-10T21:24:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d0e00640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03cc070460 0x7f03cc142440 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00c550 tx=0x7f03bc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03c9600640 1 -- 172.21.15.97:0/3839650822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc017070 con 0x7f03cc070460 2024-09-10T21:24:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03c9600640 1 -- 172.21.15.97:0/3839650822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03bc003600 con 0x7f03cc070460 2024-09-10T21:24:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc141680 con 0x7f03cc070460 2024-09-10T21:24:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03c9600640 1 -- 172.21.15.97:0/3839650822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc003a90 con 0x7f03cc070460 2024-09-10T21:24:33.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc141c40 con 0x7f03cc070460 2024-09-10T21:24:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03cc0b2a80 con 0x7f03cc070460 2024-09-10T21:24:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.816+0000 7f03c9600640 1 -- 172.21.15.97:0/3839650822 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f03bc003c30 con 0x7f03cc070460 2024-09-10T21:24:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.820+0000 7f03c9600640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f03a80725e0 0x7f03a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.820+0000 7f03c9600640 1 -- 172.21.15.97:0/3839650822 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f03bc092b50 con 0x7f03cc070460 2024-09-10T21:24:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.820+0000 7f03c3e00640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f03a80725e0 0x7f03a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.820+0000 7f03c3e00640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f03a80725e0 0x7f03a8074aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03cc13a7d0 tx=0x7f03b4001380 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:33.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:33.820+0000 7f03c9600640 1 -- 172.21.15.97:0/3839650822 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f03bc05d890 con 0x7f03cc070460 2024-09-10T21:24:34.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.028+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f03cc1408a0 con 0x7f03cc070460 2024-09-10T21:24:34.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.032+0000 7f03c9600640 1 -- 172.21.15.97:0/3839650822 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+184989 (secure 0 0 0) 0x7f03bc0622c0 con 0x7f03cc070460 2024-09-10T21:24:34.041 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:34.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f03a80725e0 msgr2=0x7f03a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f03a80725e0 0x7f03a8074aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03cc13a7d0 tx=0x7f03b4001380 comp rx=0 tx=0).stop 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03cc070460 msgr2=0x7f03cc142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03cc070460 0x7f03cc142440 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00c550 tx=0x7f03bc00ca10 comp rx=0 tx=0).stop 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 shutdown_connections 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f03a80725e0 0x7f03a8074aa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03cc144690 0x7f03cc06f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f03cc143f90 0x7f03cc13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 --2- 172.21.15.97:0/3839650822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03cc070460 0x7f03cc142440 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 >> 172.21.15.97:0/3839650822 conn(0x7f03cc0b1780 msgr2=0x7f03cc0ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 shutdown_connections 2024-09-10T21:24:34.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.036+0000 7f03d3800640 1 -- 172.21.15.97:0/3839650822 wait complete. 2024-09-10T21:24:34.060 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":751391564},{"type":"v1","addr":"172.21.15.194:6809","nonce":751391564}]},"active_addr":"172.21.15.194:6809/751391564","active_change":"2024-09-10T21:23:52.052121+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.194:0","nonce":2921094394}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.194:0","nonce":1131427628}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.194:0","nonce":2313013588}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.194:0","nonce":2813304431}]}]} 2024-09-10T21:24:34.065 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T21:24:34.065 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T21:24:34.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 -- 172.21.15.97:0/2596312475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406d180 msgr2=0x7fd2a4070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 --2- 172.21.15.97:0/2596312475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406d180 0x7fd2a4070130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009900 tx=0x7fd28c01c8b0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 -- 172.21.15.97:0/2596312475 shutdown_connections 2024-09-10T21:24:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 --2- 172.21.15.97:0/2596312475 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2a4161820 0x7fd2a40b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 --2- 172.21.15.97:0/2596312475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406d180 0x7fd2a4070130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 -- 172.21.15.97:0/2596312475 >> 172.21.15.97:0/2596312475 conn(0x7fd2a40b2790 msgr2=0x7fd2a40b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 -- 172.21.15.97:0/2596312475 shutdown_connections 2024-09-10T21:24:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 -- 172.21.15.97:0/2596312475 wait complete. 2024-09-10T21:24:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 Processor -- start 2024-09-10T21:24:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.200+0000 7fd2a9600640 1 -- start start 2024-09-10T21:24:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406cda0 0x7fd2a41f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2a406d180 0x7fd2a41fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2a4161820 0x7fd2a415c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a416b0d0 con 0x7fd2a406cda0 2024-09-10T21:24:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a416af50 con 0x7fd2a4161820 2024-09-10T21:24:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a416b250 con 0x7fd2a406d180 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406cda0 0x7fd2a41f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406cda0 0x7fd2a41f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41140/0 (socket says 172.21.15.97:41140) 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 -- 172.21.15.97:0/1316159749 learned_addr learned my addr 172.21.15.97:0/1316159749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd29be00640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2a406d180 0x7fd2a41fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a3400640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2a4161820 0x7fd2a415c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 -- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2a406d180 msgr2=0x7fd2a41fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2a406d180 0x7fd2a41fbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 -- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2a4161820 msgr2=0x7fd2a415c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2a4161820 0x7fd2a415c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 -- 172.21.15.97:0/1316159749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a41f8290 con 0x7fd2a406cda0 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a2a00640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406cda0 0x7fd2a41f7c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c500 tx=0x7fd29400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd29aa00640 1 -- 172.21.15.97:0/1316159749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294017070 con 0x7fd2a406cda0 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd29aa00640 1 -- 172.21.15.97:0/1316159749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2940053c0 con 0x7fd2a406cda0 2024-09-10T21:24:34.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd29aa00640 1 -- 172.21.15.97:0/1316159749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294012890 con 0x7fd2a406cda0 2024-09-10T21:24:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a41f8850 con 0x7fd2a406cda0 2024-09-10T21:24:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a41fc0c0 con 0x7fd2a406cda0 2024-09-10T21:24:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a4070130 con 0x7fd2a406cda0 2024-09-10T21:24:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.204+0000 7fd29aa00640 1 -- 172.21.15.97:0/1316159749 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd294003600 con 0x7fd2a406cda0 2024-09-10T21:24:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.208+0000 7fd29aa00640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2800725e0 0x7fd280074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.208+0000 7fd29aa00640 1 -- 172.21.15.97:0/1316159749 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd294092c10 con 0x7fd2a406cda0 2024-09-10T21:24:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.208+0000 7fd29be00640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2800725e0 0x7fd280074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.208+0000 7fd29be00640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2800725e0 0x7fd280074aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0af000 tx=0x7fd28c01b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.212+0000 7fd29aa00640 1 -- 172.21.15.97:0/1316159749 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd2a415e230 con 0x7fd2a406cda0 2024-09-10T21:24:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2a415dbb0 con 0x7fd2a406cda0 2024-09-10T21:24:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd29aa00640 1 -- 172.21.15.97:0/1316159749 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7fd29405df20 con 0x7fd2a406cda0 2024-09-10T21:24:34.383 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:34.383 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2800725e0 msgr2=0x7fd280074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2800725e0 0x7fd280074aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0af000 tx=0x7fd28c01b0f0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406cda0 msgr2=0x7fd2a41f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406cda0 0x7fd2a41f7c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c500 tx=0x7fd29400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 shutdown_connections 2024-09-10T21:24:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2800725e0 0x7fd280074aa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2a4161820 0x7fd2a415c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2a406d180 0x7fd2a41fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 --2- 172.21.15.97:0/1316159749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2a406cda0 0x7fd2a41f7c40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.376+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 >> 172.21.15.97:0/1316159749 conn(0x7fd2a40b2790 msgr2=0x7fd2a40b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.380+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 shutdown_connections 2024-09-10T21:24:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.380+0000 7fd2a9600640 1 -- 172.21.15.97:0/1316159749 wait complete. 2024-09-10T21:24:34.404 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T21:24:34.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:34.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 -- 172.21.15.97:0/2870746535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 msgr2=0x7f316c0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 --2- 172.21.15.97:0/2870746535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0d2590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f316c06d860 tx=0x7f316001c830 comp rx=0 tx=0).stop 2024-09-10T21:24:34.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 -- 172.21.15.97:0/2870746535 shutdown_connections 2024-09-10T21:24:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 --2- 172.21.15.97:0/2870746535 >> v2:172.21.15.194:3300/0 conn(0x7f316c0cffb0 0x7f316c15ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 --2- 172.21.15.97:0/2870746535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f316c0722c0 0x7f316c15f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 --2- 172.21.15.97:0/2870746535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0d2590 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 -- 172.21.15.97:0/2870746535 >> 172.21.15.97:0/2870746535 conn(0x7f316c0d1a70 msgr2=0x7f316c0d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 -- 172.21.15.97:0/2870746535 shutdown_connections 2024-09-10T21:24:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 -- 172.21.15.97:0/2870746535 wait complete. 2024-09-10T21:24:34.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 Processor -- start 2024-09-10T21:24:34.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.540+0000 7f3172000640 1 -- start start 2024-09-10T21:24:34.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f316c0722c0 0x7f316c1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f316c0cffb0 0x7f316c0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f316c0afea0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f316c0afd20 con 0x7f316c0cffb0 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f316c0b0020 con 0x7f316c0722c0 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41162/0 (socket says 172.21.15.97:41162) 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 -- 172.21.15.97:0/2935147120 learned_addr learned my addr 172.21.15.97:0/2935147120 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3171600640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f316c0cffb0 0x7f316c0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3170c00640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f316c0722c0 0x7f316c1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 -- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f316c0722c0 msgr2=0x7f316c1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f316c0722c0 0x7f316c1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 -- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f316c0cffb0 msgr2=0x7f316c0c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f316c0cffb0 0x7f316c0c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 -- 172.21.15.97:0/2935147120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c0cd620 con 0x7f316c0cfbd0 2024-09-10T21:24:34.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3171600640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f316c0cffb0 0x7f316c0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:34.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f316be00640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0cb390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f315800b550 tx=0x7f315800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3168c00640 1 -- 172.21.15.97:0/2935147120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158014070 con 0x7f316c0cfbd0 2024-09-10T21:24:34.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3168c00640 1 -- 172.21.15.97:0/2935147120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3158010980 con 0x7f316c0cfbd0 2024-09-10T21:24:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c0cb8d0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3168c00640 1 -- 172.21.15.97:0/2935147120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158010dd0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1586c0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.544+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c0b74c0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.548+0000 7f3168c00640 1 -- 172.21.15.97:0/2935147120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f315800f3c0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.548+0000 7f3168c00640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f314c0725e0 0x7f314c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.548+0000 7f3168c00640 1 -- 172.21.15.97:0/2935147120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3158093710 con 0x7f316c0cfbd0 2024-09-10T21:24:34.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.548+0000 7f3170c00640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f314c0725e0 0x7f314c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.548+0000 7f3170c00640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f314c0725e0 0x7f314c074aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31600af000 tx=0x7f316001b070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.552+0000 7f3168c00640 1 -- 172.21.15.97:0/2935147120 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f315805eaa0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f316c0d2510 con 0x7f316c0cfbd0 2024-09-10T21:24:34.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3168c00640 1 -- 172.21.15.97:0/2935147120 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f315805e3a0 con 0x7f316c0cfbd0 2024-09-10T21:24:34.726 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:34.727 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:34.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f314c0725e0 msgr2=0x7f314c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f314c0725e0 0x7f314c074aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31600af000 tx=0x7f316001b070 comp rx=0 tx=0).stop 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 msgr2=0x7f316c0cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0cb390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f315800b550 tx=0x7f315800ba10 comp rx=0 tx=0).stop 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 shutdown_connections 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f314c0725e0 0x7f314c074aa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f316c0cffb0 0x7f316c0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c0cfbd0 0x7f316c0cb390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 --2- 172.21.15.97:0/2935147120 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f316c0722c0 0x7f316c1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 >> 172.21.15.97:0/2935147120 conn(0x7f316c0d1a70 msgr2=0x7f316c0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 shutdown_connections 2024-09-10T21:24:34.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.720+0000 7f3172000640 1 -- 172.21.15.97:0/2935147120 wait complete. 2024-09-10T21:24:34.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T21:24:34.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T21:24:34.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T21:24:34.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T21:24:34.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T21:24:34.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T21:24:34.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T21:24:34.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T21:24:35.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 -- 172.21.15.97:0/4078823732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 msgr2=0x7f41b00530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 --2- 172.21.15.97:0/4078823732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 0x7f41b00530b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009840 tx=0x7f41a401c7d0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 -- 172.21.15.97:0/4078823732 shutdown_connections 2024-09-10T21:24:35.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 --2- 172.21.15.97:0/4078823732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 0x7f41b00530b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 --2- 172.21.15.97:0/4078823732 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41b0034130 0x7f41b0052b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 -- 172.21.15.97:0/4078823732 >> 172.21.15.97:0/4078823732 conn(0x7f41b001e060 msgr2=0x7f41b0054e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 -- 172.21.15.97:0/4078823732 shutdown_connections 2024-09-10T21:24:35.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:34.996+0000 7f41bb200640 1 -- 172.21.15.97:0/4078823732 wait complete. 2024-09-10T21:24:35.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.000+0000 7f41bb200640 1 Processor -- start 2024-09-10T21:24:35.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.000+0000 7f41bb200640 1 -- start start 2024-09-10T21:24:35.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.000+0000 7f41bb200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41b0034130 0x7f41b0184ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.000+0000 7f41bb200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41b0051280 0x7f41b0180340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.000+0000 7f41bb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 0x7f41b0006200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41b0034130 0x7f41b0184ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41b0034130 0x7f41b0184ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:48816/0 (socket says 172.21.15.97:48816) 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41b0051280 0x7f41b0180340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41b0051280 0x7f41b0180340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44170/0 (socket says 172.21.15.97:44170) 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9e00640 1 -- 172.21.15.97:0/4139901569 learned_addr learned my addr 172.21.15.97:0/4139901569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41bb200640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41b00ecb20 con 0x7f41b0055a90 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41bb200640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41b00ec9a0 con 0x7f41b0051280 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41bb200640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f41b00ecca0 con 0x7f41b0034130 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41ba800640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 0x7f41b0006200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 -- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41b0034130 msgr2=0x7f41b0184ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41b0034130 0x7f41b0184ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 -- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 msgr2=0x7f41b0006200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 0x7f41b0006200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b0184ff0 con 0x7f41b0051280 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41ba800640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b0055a90 0x7f41b0006200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:35.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41b9400640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41b0051280 0x7f41b0180340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ac000 tx=0x7f41a40b1bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 -- 172.21.15.97:0/2596284661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740b32c0 msgr2=0x7f9f740aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 --2- 172.21.15.97:0/2596284661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740b32c0 0x7f9f740aef70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9f74073590 tx=0x7f9f5c01c850 comp rx=0 tx=0).stop 2024-09-10T21:24:35.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4004220 con 0x7f41b0051280 2024-09-10T21:24:35.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41a40043c0 con 0x7f41b0051280 2024-09-10T21:24:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 -- 172.21.15.97:0/2596284661 shutdown_connections 2024-09-10T21:24:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 --2- 172.21.15.97:0/2596284661 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f740b1c40 0x7f9f740af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 --2- 172.21.15.97:0/2596284661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740b32c0 0x7f9f740aef70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 --2- 172.21.15.97:0/2596284661 >> v2:172.21.15.194:3300/0 conn(0x7f9f740b1860 0x7f9f740aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 -- 172.21.15.97:0/2596284661 >> 172.21.15.97:0/2596284661 conn(0x7f9f740d1ca0 msgr2=0x7f9f740d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 -- 172.21.15.97:0/2596284661 shutdown_connections 2024-09-10T21:24:35.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f9f7b000640 1 -- 172.21.15.97:0/2596284661 wait complete. 2024-09-10T21:24:35.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4005290 con 0x7f41b0051280 2024-09-10T21:24:35.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41bb200640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b00dac00 con 0x7f41b0051280 2024-09-10T21:24:35.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.004+0000 7f41bb200640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b00db1c0 con 0x7f41b0051280 2024-09-10T21:24:35.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f60d3200640 1 -- 172.21.15.97:0/612395998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b80051d0 msgr2=0x7f60b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f60d3200640 1 --2- 172.21.15.97:0/612395998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b80051d0 0x7f60b800b9d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009e90 tx=0x7f60cc01d470 comp rx=0 tx=0).stop 2024-09-10T21:24:35.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f60d2800640 1 -- 172.21.15.97:0/612395998 reap_dead start 2024-09-10T21:24:35.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f60d3200640 1 -- 172.21.15.97:0/612395998 shutdown_connections 2024-09-10T21:24:35.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f60d3200640 1 -- 172.21.15.97:0/612395998 >> 172.21.15.97:0/612395998 conn(0x7f60d40b16a0 msgr2=0x7f60d40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41b004f4a0 con 0x7f41b0051280 2024-09-10T21:24:35.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f60d3200640 1 -- 172.21.15.97:0/612395998 shutdown_connections 2024-09-10T21:24:35.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f60d3200640 1 -- 172.21.15.97:0/612395998 wait complete. 2024-09-10T21:24:35.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f9f7b000640 1 Processor -- start 2024-09-10T21:24:35.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f9f7b000640 1 -- start start 2024-09-10T21:24:35.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.008+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f41a40c0070 con 0x7f41b0051280 2024-09-10T21:24:35.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 Processor -- start 2024-09-10T21:24:35.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f5f23e00640 1 -- 172.21.15.97:0/4289534716 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b0d50 msgr2=0x7f5f1c0aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f7b000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f740b1860 0x7f9f7413b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f7b000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f740b1c40 0x7f9f741f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f7b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740b32c0 0x7f9f740b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f7b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74160bf0 con 0x7f9f740b32c0 2024-09-10T21:24:35.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f7b000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74160a70 con 0x7f9f740b1c40 2024-09-10T21:24:35.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f7b000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9f740afce0 con 0x7f9f740b1860 2024-09-10T21:24:35.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f5f23e00640 1 --2- 172.21.15.97:0/4289534716 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b0d50 0x7f5f1c0aedc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0b36f0 tx=0x7f5f0801c800 comp rx=0 tx=0).stop 2024-09-10T21:24:35.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f6be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f740b1c40 0x7f9f741f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f6be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f740b1c40 0x7f9f741f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44212/0 (socket says 172.21.15.97:44212) 2024-09-10T21:24:35.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f6be00640 1 -- 172.21.15.97:0/2393203854 learned_addr learned my addr 172.21.15.97:0/2393203854 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41ae000640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4198072630 0x7f4198074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f41a412de40 con 0x7f41b0051280 2024-09-10T21:24:35.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f79000640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740b32c0 0x7f9f740b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f73e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f740b1860 0x7f9f7413b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41ae000640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f419807bdb0 0x7f419807e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f419807e8c0 con 0x7f419807bdb0 2024-09-10T21:24:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f41a412e1f0 con 0x7f41b0051280 2024-09-10T21:24:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41b9e00640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4198072630 0x7f4198074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41ba800640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f419807bdb0 0x7f419807e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 -- start start 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60d415e3a0 0x7f60d40c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60d415e780 0x7f60d41fc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60d414c2e0 0x7f60d414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4072830 con 0x7f60d415e3a0 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60d40726b0 con 0x7f60d415e780 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f60d40729b0 con 0x7f60d414c2e0 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60d415e780 0x7f60d41fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60d415e780 0x7f60d41fc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44214/0 (socket says 172.21.15.97:44214) 2024-09-10T21:24:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 -- 172.21.15.97:0/3439651015 learned_addr learned my addr 172.21.15.97:0/3439651015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f73e00640 1 -- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f740b1c40 msgr2=0x7f9f741f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f73e00640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f740b1c40 0x7f9f741f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f73e00640 1 -- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740b32c0 msgr2=0x7f9f740b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f73e00640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740b32c0 0x7f9f740b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f73e00640 1 -- 172.21.15.97:0/2393203854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7406e2b0 con 0x7f9f740b1860 2024-09-10T21:24:35.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f73e00640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f740b1860 0x7f9f7413b690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b3c0 tx=0x7f9f6400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9053400640 1 -- 172.21.15.97:0/1832568183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c154f10 msgr2=0x7f904c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9053400640 1 --2- 172.21.15.97:0/1832568183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c154f10 0x7f904c0b3ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9038009880 tx=0x7f903801c710 comp rx=0 tx=0).stop 2024-09-10T21:24:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9053400640 1 -- 172.21.15.97:0/1832568183 shutdown_connections 2024-09-10T21:24:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9053400640 1 --2- 172.21.15.97:0/1832568183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c154f10 0x7f904c0b3ca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9053400640 1 -- 172.21.15.97:0/1832568183 >> 172.21.15.97:0/1832568183 conn(0x7f904c0b29a0 msgr2=0x7f904c0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1e00640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60d415e3a0 0x7f60d40c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 -- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60d414c2e0 msgr2=0x7f60d414c790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:24:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60d414c2e0 0x7f60d414c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 -- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60d415e3a0 msgr2=0x7f60d40c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60d415e3a0 0x7f60d40c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41b9e00640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4198072630 0x7f4198074af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41b406b7c0 tx=0x7f41b408e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 -- 172.21.15.97:0/3439651015 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d414ccd0 con 0x7f60d415e780 2024-09-10T21:24:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9053400640 1 -- 172.21.15.97:0/1832568183 shutdown_connections 2024-09-10T21:24:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9053400640 1 -- 172.21.15.97:0/1832568183 wait complete. 2024-09-10T21:24:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f5f23e00640 1 -- 172.21.15.97:0/4289534716 shutdown_connections 2024-09-10T21:24:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f5f23e00640 1 --2- 172.21.15.97:0/4289534716 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b0d50 0x7f5f1c0aedc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f5f23e00640 1 --2- 172.21.15.97:0/4289534716 >> v2:172.21.15.97:3300/0 conn(0x7f5f1c0ce640 0x7f5f1c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f5f23e00640 1 -- 172.21.15.97:0/4289534716 >> 172.21.15.97:0/4289534716 conn(0x7f5f1c0d1c30 msgr2=0x7f5f1c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60d1400640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60d415e780 0x7f60d41fc230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60c000e8f0 tx=0x7f60c000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f41ba800640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f419807bdb0 0x7f419807e1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64011830 con 0x7f9f740b1860 2024-09-10T21:24:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f64016070 con 0x7f9f740b1860 2024-09-10T21:24:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64011d80 con 0x7f9f740b1860 2024-09-10T21:24:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7413bbd0 con 0x7f9f740b1860 2024-09-10T21:24:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7413bf80 con 0x7f9f740b1860 2024-09-10T21:24:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.012+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c000cc60 con 0x7f60d415e780 2024-09-10T21:24:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60c000ce00 con 0x7f60d415e780 2024-09-10T21:24:35.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0005280 con 0x7f60d415e780 2024-09-10T21:24:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f5f23e00640 1 -- 172.21.15.97:0/4289534716 shutdown_connections 2024-09-10T21:24:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f5f23e00640 1 -- 172.21.15.97:0/4289534716 wait complete. 2024-09-10T21:24:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d414f360 con 0x7f60d415e780 2024-09-10T21:24:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f419807e8c0 con 0x7f419807bdb0 2024-09-10T21:24:35.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d414f920 con 0x7f60d415e780 2024-09-10T21:24:35.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 Processor -- start 2024-09-10T21:24:35.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f740ce330 con 0x7f9f740b1860 2024-09-10T21:24:35.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60d40cf7c0 con 0x7f60d415e780 2024-09-10T21:24:35.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 -- start start 2024-09-10T21:24:35.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7fd466400640 1 -- 172.21.15.97:0/1736880929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 msgr2=0x7fd46015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7fd466400640 1 --2- 172.21.15.97:0/1736880929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 0x7fd46015b090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd460070cd0 tx=0x7fd45001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f5f23e00640 1 Processor -- start 2024-09-10T21:24:35.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c073e50 0x7f904c1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f904c0b41e0 0x7f904c145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f904c154f10 0x7f904c1464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0aeb70 con 0x7f904c0b41e0 2024-09-10T21:24:35.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0ae9f0 con 0x7f904c073e50 2024-09-10T21:24:35.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f9053400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0aecf0 con 0x7f904c154f10 2024-09-10T21:24:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f21db800640 1 -- 172.21.15.97:0/3338286196 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21d413e8c0 msgr2=0x7f21d40705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f21db800640 1 --2- 172.21.15.97:0/3338286196 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21d413e8c0 0x7f21d40705c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009bc0 tx=0x7f21cc01cba0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f21db800640 1 -- 172.21.15.97:0/3338286196 shutdown_connections 2024-09-10T21:24:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f21db800640 1 --2- 172.21.15.97:0/3338286196 >> v2:172.21.15.194:3300/0 conn(0x7f21d415c7c0 0x7f21d40cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7f21db800640 1 --2- 172.21.15.97:0/3338286196 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21d413e8c0 0x7f21d40705c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f21db800640 1 -- 172.21.15.97:0/3338286196 >> 172.21.15.97:0/3338286196 conn(0x7f21d40cfb10 msgr2=0x7f21d40cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f23e00640 1 -- start start 2024-09-10T21:24:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9050a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c073e50 0x7f904c1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9050a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c073e50 0x7f904c1439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44230/0 (socket says 172.21.15.97:44230) 2024-09-10T21:24:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9050a00640 1 -- 172.21.15.97:0/1726637835 learned_addr learned my addr 172.21.15.97:0/1726637835 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9051400640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f904c154f10 0x7f904c1464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f21db800640 1 -- 172.21.15.97:0/3338286196 shutdown_connections 2024-09-10T21:24:35.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f21db800640 1 -- 172.21.15.97:0/3338286196 wait complete. 2024-09-10T21:24:35.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9f6401b460 con 0x7f9f740b1860 2024-09-10T21:24:35.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9051400640 1 -- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c073e50 msgr2=0x7f904c1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9051400640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f904c073e50 0x7f904c1439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9051400640 1 -- 172.21.15.97:0/1726637835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f904c0b41e0 msgr2=0x7f904c145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9051400640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f904c0b41e0 0x7f904c145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9051400640 1 -- 172.21.15.97:0/1726637835 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c145e50 con 0x7f904c154f10 2024-09-10T21:24:35.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7fd466400640 1 -- 172.21.15.97:0/1736880929 shutdown_connections 2024-09-10T21:24:35.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7fd466400640 1 --2- 172.21.15.97:0/1736880929 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd46015b5d0 0x7fd46006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7fd466400640 1 --2- 172.21.15.97:0/1736880929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 0x7fd46015b090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.016+0000 7fd466400640 1 --2- 172.21.15.97:0/1736880929 >> v2:172.21.15.97:3300/0 conn(0x7fd460073740 0x7fd4600b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7fd466400640 1 -- 172.21.15.97:0/1736880929 >> 172.21.15.97:0/1736880929 conn(0x7fd4600cfb10 msgr2=0x7fd4600cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0af300 0x7f5f1c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f1c0b0d50 0x7f5f1c071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0ce640 0x7f5f1c15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f23e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c1670e0 con 0x7f5f1c0ce640 2024-09-10T21:24:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f23e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c0b00b0 con 0x7f5f1c0af300 2024-09-10T21:24:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f23e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1c167260 con 0x7f5f1c0b0d50 2024-09-10T21:24:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7fd466400640 1 -- 172.21.15.97:0/1736880929 shutdown_connections 2024-09-10T21:24:35.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f70c00640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f50072630 0x7f9f50074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9f64093200 con 0x7f9f740b1860 2024-09-10T21:24:35.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f70c00640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f9f5007bdb0 0x7f9f5007e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f64013e70 con 0x7f9f5007bdb0 2024-09-10T21:24:35.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f6409a050 con 0x7f9f740b1860 2024-09-10T21:24:35.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f79000640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f9f5007bdb0 0x7f9f5007e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f21db800640 1 Processor -- start 2024-09-10T21:24:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9051400640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f904c154f10 0x7f904c1464a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f904400c110 tx=0x7f904400c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f60c0005420 con 0x7f60d415e780 2024-09-10T21:24:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f21e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0ce640 0x7f5f1c15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f21e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0ce640 0x7f5f1c15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41254/0 (socket says 172.21.15.97:41254) 2024-09-10T21:24:35.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f21e00640 1 -- 172.21.15.97:0/3995872423 learned_addr learned my addr 172.21.15.97:0/3995872423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f5f21400640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0af300 0x7f5f1c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f6be00640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f50072630 0x7f9f50074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f21db800640 1 -- start start 2024-09-10T21:24:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7f9f79000640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f9f5007bdb0 0x7f9f5007e1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f21e00640 1 -- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f1c0b0d50 msgr2=0x7f5f1c071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f21e00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f1c0b0d50 0x7f5f1c071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f21e00640 1 -- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0af300 msgr2=0x7f5f1c0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f21e00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0af300 0x7f5f1c0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f21e00640 1 -- 172.21.15.97:0/3995872423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c0b1980 con 0x7f5f1c0ce640 2024-09-10T21:24:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f21e00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0ce640 0x7f5f1c15ad30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400caa0 tx=0x7f5f1400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.020+0000 7fd466400640 1 -- 172.21.15.97:0/1736880929 wait complete. 2024-09-10T21:24:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60ca000640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60b4072630 0x7f60b4074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f60c0094520 con 0x7f60d415e780 2024-09-10T21:24:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7fd466400640 1 Processor -- start 2024-09-10T21:24:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044018030 con 0x7f904c154f10 2024-09-10T21:24:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9044003980 con 0x7f904c154f10 2024-09-10T21:24:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044003dd0 con 0x7f904c154f10 2024-09-10T21:24:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21db800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21d4070b00 0x7f21d4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21db800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d413e8c0 0x7f21d4154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21db800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21d415c7c0 0x7f21d4154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60d1e00640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60b4072630 0x7f60b4074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60ca000640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f60b407bda0 0x7f60b407e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60c0001d40 con 0x7f60b407bda0 2024-09-10T21:24:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f60c00948d0 con 0x7f60d415e780 2024-09-10T21:24:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21db800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41668c0 con 0x7f21d413e8c0 2024-09-10T21:24:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21db800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4166740 con 0x7f21d415c7c0 2024-09-10T21:24:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21db800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f21d4166a40 con 0x7f21d4070b00 2024-09-10T21:24:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14016020 con 0x7f5f1c0ce640 2024-09-10T21:24:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d413e8c0 0x7f21d4154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f9f6be00640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f50072630 0x7f9f50074af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009230 tx=0x7f9f5c01b090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f1400a930 con 0x7f5f1c0ce640 2024-09-10T21:24:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400ad80 con 0x7f5f1c0ce640 2024-09-10T21:24:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60d2800640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f60b407bda0 0x7f60b407e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c0b1c60 con 0x7f5f1c0ce640 2024-09-10T21:24:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c0b2220 con 0x7f5f1c0ce640 2024-09-10T21:24:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c15cbd0 con 0x7f904c154f10 2024-09-10T21:24:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c15d190 con 0x7f904c154f10 2024-09-10T21:24:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d413e8c0 0x7f21d4154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41268/0 (socket says 172.21.15.97:41268) 2024-09-10T21:24:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d3e00640 1 -- 172.21.15.97:0/325899156 learned_addr learned my addr 172.21.15.97:0/325899156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d8e00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21d4070b00 0x7f21d4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d9800640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21d415c7c0 0x7f21d4154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60d1e00640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60b4072630 0x7f60b4074af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60c4001ec0 tx=0x7f60c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d9800640 1 -- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21d4070b00 msgr2=0x7f21d4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d9800640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21d4070b00 0x7f21d4151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d9800640 1 -- 172.21.15.97:0/325899156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d413e8c0 msgr2=0x7f21d4154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d9800640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d413e8c0 0x7f21d4154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d9800640 1 -- 172.21.15.97:0/325899156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d41531e0 con 0x7f21d415c7c0 2024-09-10T21:24:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d3e00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21d413e8c0 0x7f21d4154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f60d2800640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f60b407bda0 0x7f60b407e1e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7fd466400640 1 -- start start 2024-09-10T21:24:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f21d9800640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21d415c7c0 0x7f21d4154fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21c400e640 tx=0x7f21c400eb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7fa115400640 1 -- 172.21.15.97:0/664749792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1100d1730 msgr2=0x7fa11006eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7fa115400640 1 --2- 172.21.15.97:0/664749792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1100d1730 0x7fa11006eaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa108009bc0 tx=0x7fa10801cb10 comp rx=0 tx=0).stop 2024-09-10T21:24:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f1c0b2600 con 0x7f5f1c0ce640 2024-09-10T21:24:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fa115400640 1 -- 172.21.15.97:0/664749792 shutdown_connections 2024-09-10T21:24:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fa115400640 1 --2- 172.21.15.97:0/664749792 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1100d1730 0x7fa11006eaf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fa115400640 1 --2- 172.21.15.97:0/664749792 >> v2:172.21.15.194:3300/0 conn(0x7fa11009d1b0 0x7fa1100d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fa115400640 1 -- 172.21.15.97:0/664749792 >> 172.21.15.97:0/664749792 conn(0x7fa1100b58b0 msgr2=0x7fa1100b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60c0001d40 con 0x7f60b407bda0 2024-09-10T21:24:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4004220 con 0x7f21d415c7c0 2024-09-10T21:24:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21c40043e0 con 0x7f21d415c7c0 2024-09-10T21:24:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c400ca70 con 0x7f21d415c7c0 2024-09-10T21:24:35.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.024+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f904c1460d0 con 0x7f904c154f10 2024-09-10T21:24:35.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f64013e70 con 0x7f9f5007bdb0 2024-09-10T21:24:35.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fd466400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd460073740 0x7fd4600ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fd466400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 0x7fd460204b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fd466400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd46015b5d0 0x7fd460164810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fd466400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd46015ea00 con 0x7fd460073740 2024-09-10T21:24:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fd466400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd46015e880 con 0x7fd460073b20 2024-09-10T21:24:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fd466400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd46015eb80 con 0x7fd46015b5d0 2024-09-10T21:24:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f1401b400 con 0x7f5f1c0ce640 2024-09-10T21:24:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f904403ab30 con 0x7f904c154f10 2024-09-10T21:24:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f9048c00640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f902c072630 0x7f902c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9044095bd0 con 0x7f904c154f10 2024-09-10T21:24:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d41534c0 con 0x7f21d415c7c0 2024-09-10T21:24:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d413d340 con 0x7f21d415c7c0 2024-09-10T21:24:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21d40705c0 con 0x7f21d415c7c0 2024-09-10T21:24:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fa115400640 1 -- 172.21.15.97:0/664749792 shutdown_connections 2024-09-10T21:24:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fa115400640 1 -- 172.21.15.97:0/664749792 wait complete. 2024-09-10T21:24:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.028+0000 7fa115400640 1 Processor -- start 2024-09-10T21:24:35.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 0x7fd460204b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd46015b5d0 0x7fd460164810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd46015b5d0 0x7fd460164810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:48906/0 (socket says 172.21.15.97:48906) 2024-09-10T21:24:35.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 -- 172.21.15.97:0/3164340473 learned_addr learned my addr 172.21.15.97:0/3164340473 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa115400640 1 -- start start 2024-09-10T21:24:35.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f9050a00640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f902c072630 0x7f902c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f9048c00640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f902c07bda0 0x7f902c07e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f902c07e8b0 con 0x7f902c07bda0 2024-09-10T21:24:35.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9044095f80 con 0x7f904c154f10 2024-09-10T21:24:35.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45f400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd460073740 0x7fd4600ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 -- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 msgr2=0x7fd460204b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f21c400cc10 con 0x7f21d415c7c0 2024-09-10T21:24:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f904be00640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f902c07bda0 0x7f902c07e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 0x7fd460204b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 -- 172.21.15.97:0/3164340473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd460073740 msgr2=0x7fd4600ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd460073740 0x7fd4600ab9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 -- 172.21.15.97:0/3164340473 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd460205070 con 0x7fd46015b5d0 2024-09-10T21:24:35.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45ea00640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd460073b20 0x7fd460204b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f19600640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc072630 0x7f5efc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5f140939d0 con 0x7f5f1c0ce640 2024-09-10T21:24:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f19600640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f5efc07bdb0 0x7f5efc07e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5efc07e8c0 con 0x7f5efc07bdb0 2024-09-10T21:24:35.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f14098220 con 0x7f5f1c0ce640 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa115400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11009d1b0 0x7fa11015ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa115400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1100d1730 0x7fa1100b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa115400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1100d3240 0x7fa1100b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa115400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa11015f5a0 con 0x7fa11009d1b0 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa115400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa11015f420 con 0x7fa1100d1730 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa115400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa11015f720 con 0x7fa1100d3240 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f20a00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f5efc07bdb0 0x7f5efc07e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f21400640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc072630 0x7f5efc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1100d1730 0x7fa1100b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1100d1730 0x7fa1100b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44272/0 (socket says 172.21.15.97:44272) 2024-09-10T21:24:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 -- 172.21.15.97:0/2552541238 learned_addr learned my addr 172.21.15.97:0/2552541238 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10fe00640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11009d1b0 0x7fa11015ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd45fe00640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd46015b5d0 0x7fd460164810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd45800da10 tx=0x7fd45800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d0c00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21b40726b0 0x7f21b4074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458017070 con 0x7fd46015b5d0 2024-09-10T21:24:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd458004b30 con 0x7fd46015b5d0 2024-09-10T21:24:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f21c40941d0 con 0x7f21d415c7c0 2024-09-10T21:24:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d8e00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21b40726b0 0x7f21b4074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d0c00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f21b407be30 0x7f21b407e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21c4016ec0 con 0x7f21b407be30 2024-09-10T21:24:35.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21c40945c0 con 0x7f21d415c7c0 2024-09-10T21:24:35.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 -- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1100d3240 msgr2=0x7fa1100b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1100d3240 0x7fa1100b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 -- 172.21.15.97:0/2552541238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11009d1b0 msgr2=0x7fa11015ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa11009d1b0 0x7fa11015ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7fa10f400640 1 -- 172.21.15.97:0/2552541238 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11015ed20 con 0x7fa1100d1730 2024-09-10T21:24:35.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f5f20a00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f5efc07bdb0 0x7f5efc07e1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.032+0000 7f21d3e00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f21b407be30 0x7f21b407e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7f21d8e00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21b40726b0 0x7f21b4074b70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0af000 tx=0x7f21cc01b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7f5f21400640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc072630 0x7f5efc074af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009b40 tx=0x7f5f100091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fa10f400640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1100d1730 0x7fa1100b2350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e9f0 tx=0x7fa10000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45800be60 con 0x7fd46015b5d0 2024-09-10T21:24:35.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4600ac010 con 0x7fd46015b5d0 2024-09-10T21:24:35.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4600ae9d0 con 0x7fd46015b5d0 2024-09-10T21:24:35.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7f21d3e00640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f21b407be30 0x7f21b407e270 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000cd80 con 0x7fa1100d1730 2024-09-10T21:24:35.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa100010430 con 0x7fa1100d1730 2024-09-10T21:24:35.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100010860 con 0x7fa1100d1730 2024-09-10T21:24:35.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5efc07e8c0 con 0x7f5efc07bdb0 2024-09-10T21:24:35.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7f9050a00640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f902c072630 0x7f902c074af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9038009230 tx=0x7f903801af50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1100cb810 con 0x7fa1100d1730 2024-09-10T21:24:35.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7f904be00640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f902c07bda0 0x7f902c07e1e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1100b83e0 con 0x7fa1100d1730 2024-09-10T21:24:35.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd46006e3b0 con 0x7fd46015b5d0 2024-09-10T21:24:35.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21c4016ec0 con 0x7f21b407be30 2024-09-10T21:24:35.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.036+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa11015fc40 con 0x7fa1100d1730 2024-09-10T21:24:35.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.040+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== osd.5 v2:172.21.15.194:6818/3659294892 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f902c07e8b0 con 0x7f902c07bda0 2024-09-10T21:24:35.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.040+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd458010460 con 0x7fd46015b5d0 2024-09-10T21:24:35.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.040+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa10003daa0 con 0x7fa1100d1730 2024-09-10T21:24:35.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.040+0000 7fd457400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4400725e0 0x7fd440074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.040+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd458094210 con 0x7fd46015b5d0 2024-09-10T21:24:35.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.040+0000 7fd457400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7fd44007bcd0 0x7fd44007e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.040+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd458016ec0 con 0x7fd44007bcd0 2024-09-10T21:24:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4580945c0 con 0x7fd46015b5d0 2024-09-10T21:24:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fd45f400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4400725e0 0x7fd440074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fd45f400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4400725e0 0x7fd440074aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd44c005fd0 tx=0x7fd44c005e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fd45ea00640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7fd44007bcd0 0x7fd44007e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa107e00640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa0ec0726b0 0x7fa0ec074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa1000942d0 con 0x7fa1100d1730 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa107e00640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7fa0ec07be30 0x7fa0ec07e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0ec07e940 con 0x7fa0ec07be30 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1000946c0 con 0x7fa1100d1730 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa114a00640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7fa0ec07be30 0x7fa0ec07e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa10fe00640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa0ec0726b0 0x7fa0ec074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa114a00640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7fa0ec07be30 0x7fa0ec07e270 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa10fe00640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa0ec0726b0 0x7fa0ec074b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1101ff260 tx=0x7fa0f8009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0ec07e940 con 0x7fa0ec07be30 2024-09-10T21:24:35.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.044+0000 7fd45ea00640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7fd44007bcd0 0x7fd44007e110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41b0052310 con 0x7f419807bdb0 2024-09-10T21:24:35.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41ae000640 1 -- 172.21.15.97:0/4139901569 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f41b0052310 con 0x7f419807bdb0 2024-09-10T21:24:35.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== osd.4 v2:172.21.15.194:6800/4073109273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd458016ec0 con 0x7fd44007bcd0 2024-09-10T21:24:35.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f419807bdb0 msgr2=0x7f419807e1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f419807bdb0 0x7f419807e1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4198072630 msgr2=0x7f4198074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4198072630 0x7f4198074af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41b406b7c0 tx=0x7f41b408e040 comp rx=0 tx=0).stop 2024-09-10T21:24:35.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41b0051280 msgr2=0x7f41b0180340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 --2- 172.21.15.97:0/4139901569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41b0051280 0x7f41b0180340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ac000 tx=0x7f41a40b1bb0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41ba800640 1 -- 172.21.15.97:0/4139901569 reap_dead start 2024-09-10T21:24:35.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 shutdown_connections 2024-09-10T21:24:35.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 >> 172.21.15.97:0/4139901569 conn(0x7f41b001e060 msgr2=0x7f41b0031ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 shutdown_connections 2024-09-10T21:24:35.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.048+0000 7f41a3400640 1 -- 172.21.15.97:0/4139901569 wait complete. 2024-09-10T21:24:35.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.052+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60d406f5f0 con 0x7f60b407bda0 2024-09-10T21:24:35.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.052+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21d416b940 con 0x7f21b407be30 2024-09-10T21:24:35.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f21d0c00640 1 -- 172.21.15.97:0/325899156 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21d416b940 con 0x7f21b407be30 2024-09-10T21:24:35.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60ca000640 1 -- 172.21.15.97:0/3439651015 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f60d406f5f0 con 0x7f60b407bda0 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f60b407bda0 msgr2=0x7f60b407e1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f60b407bda0 0x7f60b407e1e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60b4072630 msgr2=0x7f60b4074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60b4072630 0x7f60b4074af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60c4001ec0 tx=0x7f60c4009040 comp rx=0 tx=0).stop 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60d415e780 msgr2=0x7f60d41fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 --2- 172.21.15.97:0/3439651015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60d415e780 0x7f60d41fc230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60c000e8f0 tx=0x7f60c000edb0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d2800640 1 -- 172.21.15.97:0/3439651015 reap_dead start 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 shutdown_connections 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 >> 172.21.15.97:0/3439651015 conn(0x7f60d40b16a0 msgr2=0x7f60d41586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f21b407be30 msgr2=0x7f21b407e270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.056+0000 7f21db800640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f21b407be30 0x7f21b407e270 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f740b0270 con 0x7f9f5007bdb0 2024-09-10T21:24:35.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21b40726b0 msgr2=0x7f21b4074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21b40726b0 0x7f21b4074b70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0af000 tx=0x7f21cc01b3e0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21d415c7c0 msgr2=0x7f21d4154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 --2- 172.21.15.97:0/325899156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21d415c7c0 0x7f21d4154fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21c400e640 tx=0x7f21c400eb00 comp rx=0 tx=0).stop 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21d9800640 1 -- 172.21.15.97:0/325899156 reap_dead start 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 shutdown_connections 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 >> 172.21.15.97:0/325899156 conn(0x7f21d40cfb10 msgr2=0x7f21d4165910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 shutdown_connections 2024-09-10T21:24:35.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f60d3200640 1 -- 172.21.15.97:0/3439651015 wait complete. 2024-09-10T21:24:35.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 shutdown_connections 2024-09-10T21:24:35.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.060+0000 7f21db800640 1 -- 172.21.15.97:0/325899156 wait complete. 2024-09-10T21:24:35.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f70c00640 1 -- 172.21.15.97:0/2393203854 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9f740b0270 con 0x7f9f5007bdb0 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f9f5007bdb0 msgr2=0x7f9f5007e1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f9f5007bdb0 0x7f9f5007e1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f50072630 msgr2=0x7f9f50074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f50072630 0x7f9f50074af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009230 tx=0x7f9f5c01b090 comp rx=0 tx=0).stop 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f740b1860 msgr2=0x7f9f7413b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 --2- 172.21.15.97:0/2393203854 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f740b1860 0x7f9f7413b690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400b3c0 tx=0x7f9f6400b880 comp rx=0 tx=0).stop 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f79000640 1 -- 172.21.15.97:0/2393203854 reap_dead start 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 shutdown_connections 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 >> 172.21.15.97:0/2393203854 conn(0x7f9f740d1ca0 msgr2=0x7f9f7406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 shutdown_connections 2024-09-10T21:24:35.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.064+0000 7f9f7b000640 1 -- 172.21.15.97:0/2393203854 wait complete. 2024-09-10T21:24:35.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5f1c0aedc0 con 0x7f5efc07bdb0 2024-09-10T21:24:35.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f904c0b3ca0 con 0x7f902c07bda0 2024-09-10T21:24:35.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f5f19600640 1 -- 172.21.15.97:0/3995872423 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5f1c0aedc0 con 0x7f5efc07bdb0 2024-09-10T21:24:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f5efc07bdb0 msgr2=0x7f5efc07e1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f5f23e00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f5efc07bdb0 0x7f5efc07e1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f9048c00640 1 -- 172.21.15.97:0/1726637835 <== osd.5 v2:172.21.15.194:6818/3659294892 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f904c0b3ca0 con 0x7f902c07bda0 2024-09-10T21:24:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc072630 msgr2=0x7f5efc074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.068+0000 7f5f23e00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc072630 0x7f5efc074af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009b40 tx=0x7f5f100091e0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0ce640 msgr2=0x7f5f1c15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f5f23e00640 1 --2- 172.21.15.97:0/3995872423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c0ce640 0x7f5f1c15ad30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400caa0 tx=0x7f5f1400cb90 comp rx=0 tx=0).stop 2024-09-10T21:24:35.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f5f21e00640 1 -- 172.21.15.97:0/3995872423 reap_dead start 2024-09-10T21:24:35.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f902c07bda0 msgr2=0x7f902c07e1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f902c07bda0 0x7f902c07e1e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 shutdown_connections 2024-09-10T21:24:35.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f902c072630 msgr2=0x7f902c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 >> 172.21.15.97:0/3995872423 conn(0x7f5f1c0d1c30 msgr2=0x7f5f1c15ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f902c072630 0x7f902c074af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9038009230 tx=0x7f903801af50 comp rx=0 tx=0).stop 2024-09-10T21:24:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f904c154f10 msgr2=0x7f904c1464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 --2- 172.21.15.97:0/1726637835 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f904c154f10 0x7f904c1464a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f904400c110 tx=0x7f904400c5d0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 shutdown_connections 2024-09-10T21:24:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f5f23e00640 1 -- 172.21.15.97:0/3995872423 wait complete. 2024-09-10T21:24:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9051400640 1 -- 172.21.15.97:0/1726637835 reap_dead start 2024-09-10T21:24:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 shutdown_connections 2024-09-10T21:24:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 >> 172.21.15.97:0/1726637835 conn(0x7f904c0b29a0 msgr2=0x7f904c06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 shutdown_connections 2024-09-10T21:24:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.072+0000 7f9053400640 1 -- 172.21.15.97:0/1726637835 wait complete. 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.076+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1100b8e30 con 0x7fa0ec07be30 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.076+0000 7fa107e00640 1 -- 172.21.15.97:0/2552541238 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa1100b8e30 con 0x7fa0ec07be30 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.076+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7fa0ec07be30 msgr2=0x7fa0ec07e270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.076+0000 7fa115400640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7fa0ec07be30 0x7fa0ec07e270 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.076+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa0ec0726b0 msgr2=0x7fa0ec074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.076+0000 7fa115400640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa0ec0726b0 0x7fa0ec074b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1101ff260 tx=0x7fa0f8009060 comp rx=0 tx=0).stop 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.076+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1100d1730 msgr2=0x7fa1100b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fa115400640 1 --2- 172.21.15.97:0/2552541238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1100d1730 0x7fa1100b2350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e9f0 tx=0x7fa10000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fa114a00640 1 -- 172.21.15.97:0/2552541238 reap_dead start 2024-09-10T21:24:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 shutdown_connections 2024-09-10T21:24:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 >> 172.21.15.97:0/2552541238 conn(0x7fa1100b58b0 msgr2=0x7fa110162570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 shutdown_connections 2024-09-10T21:24:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fa115400640 1 -- 172.21.15.97:0/2552541238 wait complete. 2024-09-10T21:24:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd4600cc300 con 0x7fd44007bcd0 2024-09-10T21:24:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd457400640 1 -- 172.21.15.97:0/3164340473 <== osd.4 v2:172.21.15.194:6800/4073109273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd4600cc300 con 0x7fd44007bcd0 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7fd44007bcd0 msgr2=0x7fd44007e110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd466400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7fd44007bcd0 0x7fd44007e110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4400725e0 msgr2=0x7fd440074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd466400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4400725e0 0x7fd440074aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd44c005fd0 tx=0x7fd44c005e60 comp rx=0 tx=0).stop 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd46015b5d0 msgr2=0x7fd460164810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd466400640 1 --2- 172.21.15.97:0/3164340473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd46015b5d0 0x7fd460164810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd45800da10 tx=0x7fd45800ded0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.080+0000 7fd45fe00640 1 -- 172.21.15.97:0/3164340473 reap_dead start 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.084+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 shutdown_connections 2024-09-10T21:24:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.084+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 >> 172.21.15.97:0/3164340473 conn(0x7fd4600cfb10 msgr2=0x7fd46015db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.088+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 shutdown_connections 2024-09-10T21:24:35.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.088+0000 7fd466400640 1 -- 172.21.15.97:0/3164340473 wait complete. 2024-09-10T21:24:35.100 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:35.100 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T21:24:35.109 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-09-10T21:24:35.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T21:24:35.112 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:35.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T21:24:35.115 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:35.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T21:24:35.118 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:35.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T21:24:35.124 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:35.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T21:24:35.127 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-09-10T21:24:35.128 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T21:24:35.129 INFO:teuthology.orchestra.run.smithi097.stdout:55834574858 2024-09-10T21:24:35.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T21:24:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.356+0000 7fcffb400640 1 -- 172.21.15.97:0/3115878632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc00b6f0 msgr2=0x7fcfdc00db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.356+0000 7fcffb400640 1 --2- 172.21.15.97:0/3115878632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc00b6f0 0x7fcfdc00db60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcff4051d40 tx=0x7fcff4097e40 comp rx=0 tx=0).stop 2024-09-10T21:24:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.356+0000 7fcff9400640 1 -- 172.21.15.97:0/3115878632 reap_dead start 2024-09-10T21:24:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.360+0000 7fcffb400640 1 -- 172.21.15.97:0/3115878632 shutdown_connections 2024-09-10T21:24:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.360+0000 7fcffb400640 1 -- 172.21.15.97:0/3115878632 >> 172.21.15.97:0/3115878632 conn(0x7fcfec054570 msgr2=0x7fcfec054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.360+0000 7fcffb400640 1 -- 172.21.15.97:0/3115878632 shutdown_connections 2024-09-10T21:24:35.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.360+0000 7fcffb400640 1 -- 172.21.15.97:0/3115878632 wait complete. 2024-09-10T21:24:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.360+0000 7f5a12800640 1 -- 172.21.15.97:0/82637695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c15b110 msgr2=0x7f5a0c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.360+0000 7f5a12800640 1 --2- 172.21.15.97:0/82637695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c15b110 0x7f5a0c06e3b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c072040 tx=0x7f59f801eff0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7f5a12800640 1 -- 172.21.15.97:0/82637695 shutdown_connections 2024-09-10T21:24:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7f5a12800640 1 --2- 172.21.15.97:0/82637695 >> v2:172.21.15.97:3300/0 conn(0x7f5a0c072630 0x7f5a0c072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7f5a12800640 1 --2- 172.21.15.97:0/82637695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c15b110 0x7f5a0c06e3b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7f5a12800640 1 -- 172.21.15.97:0/82637695 >> 172.21.15.97:0/82637695 conn(0x7f5a0c0b1780 msgr2=0x7f5a0c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 Processor -- start 2024-09-10T21:24:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7f5a12800640 1 -- 172.21.15.97:0/82637695 shutdown_connections 2024-09-10T21:24:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7f5a12800640 1 -- 172.21.15.97:0/82637695 wait complete. 2024-09-10T21:24:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 -- start start 2024-09-10T21:24:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc00b6f0 0x7fcfec1791f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec051230 0x7fcfec021860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcfec0dc220 0x7fcfec04dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0eae00 con 0x7fcfdc00b6f0 2024-09-10T21:24:35.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec0eac80 con 0x7fcfec051230 2024-09-10T21:24:35.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.364+0000 7fcffb400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcfec0eaf80 con 0x7fcfec0dc220 2024-09-10T21:24:35.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f61ef200640 1 -- 172.21.15.97:0/347389902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f0154250 msgr2=0x7f61f015aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f61ef200640 1 --2- 172.21.15.97:0/347389902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f0154250 0x7f61f015aa00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f61e000baf0 tx=0x7f61e001d090 comp rx=0 tx=0).stop 2024-09-10T21:24:35.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f61ef200640 1 -- 172.21.15.97:0/347389902 shutdown_connections 2024-09-10T21:24:35.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f61ef200640 1 --2- 172.21.15.97:0/347389902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f0154250 0x7f61f015aa00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 Processor -- start 2024-09-10T21:24:35.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f61ef200640 1 -- 172.21.15.97:0/347389902 >> 172.21.15.97:0/347389902 conn(0x7f61f00cf890 msgr2=0x7f61f00cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 -- start start 2024-09-10T21:24:35.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f61ef200640 1 -- 172.21.15.97:0/347389902 shutdown_connections 2024-09-10T21:24:35.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f61ef200640 1 -- 172.21.15.97:0/347389902 wait complete. 2024-09-10T21:24:35.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c070720 0x7f5a0c14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a0c072630 0x7f5a0c14de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c15b110 0x7f5a0c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0b2e10 con 0x7f5a0c15b110 2024-09-10T21:24:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0b2c90 con 0x7f5a0c070720 2024-09-10T21:24:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a12800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c0b2f90 con 0x7f5a0c072630 2024-09-10T21:24:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c070720 0x7f5a0c14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c070720 0x7f5a0c14d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44362/0 (socket says 172.21.15.97:44362) 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 -- 172.21.15.97:0/2015396638 learned_addr learned my addr 172.21.15.97:0/2015396638 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0aa00640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a0c072630 0x7f5a0c14de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec051230 0x7fcfec021860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff9400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcfec0dc220 0x7fcfec04dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec051230 0x7fcfec021860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44346/0 (socket says 172.21.15.97:44346) 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 -- 172.21.15.97:0/2976763241 learned_addr learned my addr 172.21.15.97:0/2976763241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 -- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcfec0dc220 msgr2=0x7fcfec04dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcfec0dc220 0x7fcfec04dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f3e03e00640 1 -- 172.21.15.97:0/1596866671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e040b29b0 msgr2=0x7f3e040b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f3e03e00640 1 --2- 172.21.15.97:0/1596866671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e040b29b0 0x7f3e040b1f50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009840 tx=0x7f3df001c760 comp rx=0 tx=0).stop 2024-09-10T21:24:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 -- 172.21.15.97:0/2976763241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc00b6f0 msgr2=0x7fcfec1791f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff8a00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc00b6f0 0x7fcfec1791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc00b6f0 0x7fcfec1791f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff3e00640 1 -- 172.21.15.97:0/2976763241 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec04ff10 con 0x7fcfec051230 2024-09-10T21:24:35.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7fcff8a00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc00b6f0 0x7fcfec1791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 -- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a0c072630 msgr2=0x7f5a0c14de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a0c072630 0x7f5a0c14de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 -- 172.21.15.97:0/2015396638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c15b110 msgr2=0x7f5a0c150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c15b110 0x7f5a0c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.368+0000 7f5a0b400640 1 -- 172.21.15.97:0/2015396638 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c14b020 con 0x7f5a0c070720 2024-09-10T21:24:35.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 -- 172.21.15.97:0/3432760928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 msgr2=0x7f5f040b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 --2- 172.21.15.97:0/3432760928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 0x7f5f040b6e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f040b83c0 tx=0x7f5efc01ca60 comp rx=0 tx=0).stop 2024-09-10T21:24:35.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f3e03e00640 1 -- 172.21.15.97:0/1596866671 shutdown_connections 2024-09-10T21:24:35.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f3e03e00640 1 --2- 172.21.15.97:0/1596866671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e040b29b0 0x7f3e040b1f50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f3e03e00640 1 --2- 172.21.15.97:0/1596866671 >> v2:172.21.15.97:3300/0 conn(0x7f3e04161290 0x7f3e040b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f3e03e00640 1 -- 172.21.15.97:0/1596866671 >> 172.21.15.97:0/1596866671 conn(0x7f3e040d1a60 msgr2=0x7f3e040d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f3e03e00640 1 -- 172.21.15.97:0/1596866671 shutdown_connections 2024-09-10T21:24:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 -- 172.21.15.97:0/3432760928 shutdown_connections 2024-09-10T21:24:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 --2- 172.21.15.97:0/3432760928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 0x7f5f040b6e50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 --2- 172.21.15.97:0/3432760928 >> v2:172.21.15.97:3300/0 conn(0x7f5f040b8630 0x7f5f040d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 --2- 172.21.15.97:0/3432760928 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f0406db10 0x7f5f04166650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 -- 172.21.15.97:0/3432760928 >> 172.21.15.97:0/3432760928 conn(0x7f5f040b9290 msgr2=0x7f5f040b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f3e03e00640 1 -- 172.21.15.97:0/1596866671 wait complete. 2024-09-10T21:24:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5a0b400640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c070720 0x7f5a0c14d8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c10 tx=0x7f59f8005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 -- 172.21.15.97:0/3432760928 shutdown_connections 2024-09-10T21:24:35.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7fcff3e00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec051230 0x7fcfec021860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e680 tx=0x7fcfe000eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 Processor -- start 2024-09-10T21:24:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7fcff0c00640 1 -- 172.21.15.97:0/2976763241 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000cab0 con 0x7fcfec051230 2024-09-10T21:24:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5a03400640 1 -- 172.21.15.97:0/2015396638 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80be070 con 0x7f5a0c070720 2024-09-10T21:24:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5a03400640 1 -- 172.21.15.97:0/2015396638 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f80053e0 con 0x7f5a0c070720 2024-09-10T21:24:35.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5a03400640 1 -- 172.21.15.97:0/2015396638 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80b5040 con 0x7f5a0c070720 2024-09-10T21:24:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 -- start start 2024-09-10T21:24:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7fcffb400640 1 -- 172.21.15.97:0/2976763241 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec050190 con 0x7fcfec051230 2024-09-10T21:24:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7fcff0c00640 1 -- 172.21.15.97:0/2976763241 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe000cc50 con 0x7fcfec051230 2024-09-10T21:24:35.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7fcffb400640 1 -- 172.21.15.97:0/2976763241 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec050750 con 0x7fcfec051230 2024-09-10T21:24:35.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7fcff0c00640 1 -- 172.21.15.97:0/2976763241 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000cab0 con 0x7fcfec051230 2024-09-10T21:24:35.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c14b300 con 0x7f5a0c070720 2024-09-10T21:24:35.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f162f200640 1 -- 172.21.15.97:0/2000856862 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 msgr2=0x7f16300b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f162f200640 1 --2- 172.21.15.97:0/2000856862 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16300b0e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f161c009900 tx=0x7f161c01c820 comp rx=0 tx=0).stop 2024-09-10T21:24:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61f006eba0 0x7f61f014b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f01583c0 0x7f61f00cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61f01499b0 0x7f61f01530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0167d70 con 0x7f61f01499b0 2024-09-10T21:24:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0167bf0 con 0x7f61f01583c0 2024-09-10T21:24:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ef200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f61f0167ef0 con 0x7f61f006eba0 2024-09-10T21:24:35.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c14b8c0 con 0x7f5a0c070720 2024-09-10T21:24:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 -- 172.21.15.97:0/3432760928 wait complete. 2024-09-10T21:24:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 Processor -- start 2024-09-10T21:24:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f3e03e00640 1 Processor -- start 2024-09-10T21:24:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ede00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61f006eba0 0x7f61f014b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f162f200640 1 -- 172.21.15.97:0/2000856862 shutdown_connections 2024-09-10T21:24:35.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ede00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61f006eba0 0x7f61f014b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:48996/0 (socket says 172.21.15.97:48996) 2024-09-10T21:24:35.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f5f0a000640 1 -- start start 2024-09-10T21:24:35.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f162f200640 1 --2- 172.21.15.97:0/2000856862 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16300b0e70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f162f200640 1 --2- 172.21.15.97:0/2000856862 >> v2:172.21.15.194:3300/0 conn(0x7f1630159780 0x7f1630159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f162f200640 1 -- 172.21.15.97:0/2000856862 >> 172.21.15.97:0/2000856862 conn(0x7f16300afa50 msgr2=0x7f1630074980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f162f200640 1 -- 172.21.15.97:0/2000856862 shutdown_connections 2024-09-10T21:24:35.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ede00640 1 -- 172.21.15.97:0/11041999 learned_addr learned my addr 172.21.15.97:0/11041999 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.372+0000 7f61ed400640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f01583c0 0x7f61f00cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f162f200640 1 -- 172.21.15.97:0/2000856862 wait complete. 2024-09-10T21:24:35.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfec01c7e0 con 0x7fcfec051230 2024-09-10T21:24:35.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61ed400640 1 -- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61f006eba0 msgr2=0x7f61f014b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61ed400640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61f006eba0 0x7f61f014b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61ed400640 1 -- 172.21.15.97:0/11041999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61f01499b0 msgr2=0x7f61f01530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61ed400640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61f01499b0 0x7f61f01530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61ed400640 1 -- 172.21.15.97:0/11041999 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f014b8e0 con 0x7f61f01583c0 2024-09-10T21:24:35.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03e00640 1 -- start start 2024-09-10T21:24:35.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c14bcf0 con 0x7f5a0c070720 2024-09-10T21:24:35.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f0a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f0406db10 0x7f5f041630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7fcff0c00640 1 -- 172.21.15.97:0/2976763241 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcfe0019480 con 0x7fcfec051230 2024-09-10T21:24:35.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61ed400640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f01583c0 0x7f61f00cd040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ac000 tx=0x7f61e00ac400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e040b2490 0x7f3e040c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f0a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f040b8630 0x7f5f04203d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f0a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 0x7f5f04161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f0a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f040b5bd0 con 0x7f5f0406db10 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f0a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f040b5a50 con 0x7f5f040d1270 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f0a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5f040b5d50 con 0x7f5f040b8630 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f0406db10 0x7f5f041630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f0406db10 0x7f5f041630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41378/0 (socket says 172.21.15.97:41378) 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f03400640 1 -- 172.21.15.97:0/1092533630 learned_addr learned my addr 172.21.15.97:0/1092533630 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f02a00640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f040b8630 0x7f5f04203d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e040b29b0 0x7f3e04156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e04161290 0x7f3e041570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0416acd0 con 0x7f3e040b2490 2024-09-10T21:24:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0416ab50 con 0x7f3e040b29b0 2024-09-10T21:24:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0416ae50 con 0x7f3e04161290 2024-09-10T21:24:35.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f162f200640 1 Processor -- start 2024-09-10T21:24:35.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61e6000640 1 -- 172.21.15.97:0/11041999 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00b1040 con 0x7f61f01583c0 2024-09-10T21:24:35.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f03e00640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 0x7f5f04161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f162f200640 1 -- start start 2024-09-10T21:24:35.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e04161290 0x7f3e041570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e04161290 0x7f3e041570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:49004/0 (socket says 172.21.15.97:49004) 2024-09-10T21:24:35.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f3e03400640 1 -- 172.21.15.97:0/2526834836 learned_addr learned my addr 172.21.15.97:0/2526834836 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7fcff0c00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcfd4072810 0x7fcfd4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f02a00640 1 -- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 msgr2=0x7f5f04161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f02a00640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 0x7f5f04161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f02a00640 1 -- 172.21.15.97:0/1092533630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f0406db10 msgr2=0x7f5f041630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f02a00640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f0406db10 0x7f5f041630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5f02a00640 1 -- 172.21.15.97:0/1092533630 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f04165680 con 0x7f5f040b8630 2024-09-10T21:24:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7fcff0c00640 1 -- 172.21.15.97:0/2976763241 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcfe0093c20 con 0x7fcfec051230 2024-09-10T21:24:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61e6000640 1 -- 172.21.15.97:0/11041999 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e00bc070 con 0x7f61f01583c0 2024-09-10T21:24:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f61e6000640 1 -- 172.21.15.97:0/11041999 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00050c0 con 0x7f61f01583c0 2024-09-10T21:24:35.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5f02a00640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f040b8630 0x7f5f04203d30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c940 tx=0x7f5ef000ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fcff8a00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcfd4072810 0x7fcfd4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5efb400640 1 -- 172.21.15.97:0/1092533630 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0007bb0 con 0x7f5f040b8630 2024-09-10T21:24:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e02000640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e040b29b0 0x7f3e04156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e02a00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e040b2490 0x7f3e040c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03400640 1 -- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e040b29b0 msgr2=0x7f3e04156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03400640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e040b29b0 0x7f3e04156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03400640 1 -- 172.21.15.97:0/2526834836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e040b2490 msgr2=0x7f3e040c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.376+0000 7f5a03400640 1 -- 172.21.15.97:0/2015396638 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f59f80b8030 con 0x7f5a0c070720 2024-09-10T21:24:35.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5a03400640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59ec072630 0x7f59ec074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f041637a0 con 0x7f5f040b8630 2024-09-10T21:24:35.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f04163d60 con 0x7f5f040b8630 2024-09-10T21:24:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5a03400640 1 -- 172.21.15.97:0/2015396638 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f59f80f74c0 con 0x7f5a0c070720 2024-09-10T21:24:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16301fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16300cd1f0 0x7f1630141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1630159780 0x7f1630157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5efb400640 1 -- 172.21.15.97:0/1092533630 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ef0007e40 con 0x7f5f040b8630 2024-09-10T21:24:35.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5efb400640 1 -- 172.21.15.97:0/1092533630 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0007bb0 con 0x7f5f040b8630 2024-09-10T21:24:35.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5a0aa00640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59ec072630 0x7f59ec074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 -- 172.21.15.97:0/518733863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 msgr2=0x7fc528072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 --2- 172.21.15.97:0/518733863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 0x7fc528072630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0074e0 tx=0x7fc51c01d560 comp rx=0 tx=0).stop 2024-09-10T21:24:35.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1630168d50 con 0x7f1630159780 2024-09-10T21:24:35.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16301fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1630168bd0 con 0x7f16300cd1f0 2024-09-10T21:24:35.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03400640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e040b2490 0x7f3e040c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1630168ed0 con 0x7f163006f6d0 2024-09-10T21:24:35.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16301fb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:49012/0 (socket says 172.21.15.97:49012) 2024-09-10T21:24:35.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 -- 172.21.15.97:0/642481209 learned_addr learned my addr 172.21.15.97:0/642481209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f00cd870 con 0x7f61f01583c0 2024-09-10T21:24:35.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03400640 1 -- 172.21.15.97:0/2526834836 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e040c6620 con 0x7f3e04161290 2024-09-10T21:24:35.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162d400640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16300cd1f0 0x7f1630141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e02a00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e040b2490 0x7f3e040c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:35.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fcff8a00640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcfd4072810 0x7fcfd4074cd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8000c10 tx=0x7fcfe80067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f00cde30 con 0x7f61f01583c0 2024-09-10T21:24:35.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03400640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e04161290 0x7f3e041570f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00cbf0 tx=0x7f3dfc00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 -- 172.21.15.97:0/518733863 shutdown_connections 2024-09-10T21:24:35.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 -- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16300cd1f0 msgr2=0x7f1630141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16300cd1f0 0x7f1630141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 -- 172.21.15.97:0/642481209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1630159780 msgr2=0x7f1630157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1630159780 0x7f1630157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 --2- 172.21.15.97:0/518733863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 0x7fc528072630 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 --2- 172.21.15.97:0/518733863 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 0x7fc52806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 -- 172.21.15.97:0/642481209 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1630141ad0 con 0x7f163006f6d0 2024-09-10T21:24:35.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f978fe00640 1 -- 172.21.15.97:0/3736495504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c0a6b60 con 0x7f97900710b0 2024-09-10T21:24:35.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 -- 172.21.15.97:0/518733863 >> 172.21.15.97:0/518733863 conn(0x7fc5280b1780 msgr2=0x7fc5280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc0052a0 con 0x7f5f040b8630 2024-09-10T21:24:35.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 -- 172.21.15.97:0/518733863 shutdown_connections 2024-09-10T21:24:35.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7fc530400640 1 -- 172.21.15.97:0/518733863 wait complete. 2024-09-10T21:24:35.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f162de00640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16301fb940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f162800ebc0 tx=0x7f162800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3dfaa00640 1 -- 172.21.15.97:0/2526834836 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc016020 con 0x7f3e04161290 2024-09-10T21:24:35.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e040c6900 con 0x7f3e04161290 2024-09-10T21:24:35.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e040c6ec0 con 0x7f3e04161290 2024-09-10T21:24:35.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f5a0aa00640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59ec072630 0x7f59ec074af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c143450 tx=0x7f59fc001420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f015aa00 con 0x7f61f01583c0 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f9796800640 1 -- 172.21.15.97:0/3736495504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900710b0 msgr2=0x7f97900b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f9796800640 1 --2- 172.21.15.97:0/3736495504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900710b0 0x7f97900b39b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f977c009840 tx=0x7f977c01c790 comp rx=0 tx=0).stop 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f9796800640 1 -- 172.21.15.97:0/3736495504 shutdown_connections 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f9796800640 1 --2- 172.21.15.97:0/3736495504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900710b0 0x7f97900b39b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f9796800640 1 --2- 172.21.15.97:0/3736495504 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900afdc0 0x7f979015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f9796800640 1 -- 172.21.15.97:0/3736495504 >> 172.21.15.97:0/3736495504 conn(0x7f97900d09c0 msgr2=0x7f97900d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.380+0000 7f9796800640 1 -- 172.21.15.97:0/3736495504 shutdown_connections 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 -- 172.21.15.97:0/3736495504 wait complete. 2024-09-10T21:24:35.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e040b1f50 con 0x7f3e04161290 2024-09-10T21:24:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f5efb400640 1 -- 172.21.15.97:0/1092533630 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5ef000f450 con 0x7f5f040b8630 2024-09-10T21:24:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 Processor -- start 2024-09-10T21:24:35.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 -- start start 2024-09-10T21:24:35.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 Processor -- start 2024-09-10T21:24:35.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 -- start start 2024-09-10T21:24:35.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fcff0c00640 1 -- 172.21.15.97:0/2976763241 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcfec04ff10 con 0x7fcfec051230 2024-09-10T21:24:35.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f5efb400640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ee40727e0 0x7f5ee4074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f5efb400640 1 -- 172.21.15.97:0/1092533630 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5ef0093880 con 0x7f5f040b8630 2024-09-10T21:24:35.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 0x7fc52813e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f5f03400640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ee40727e0 0x7f5ee4074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 0x7fc5280cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc52815b110 0x7fc52814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc528166fa0 con 0x7fc52815b110 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc528166e20 con 0x7fc5280cf490 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc530400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc528167120 con 0x7fc5280cf0b0 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc52da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 0x7fc52813e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc52e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc52815b110 0x7fc52814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc52da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 0x7fc52813e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:49020/0 (socket says 172.21.15.97:49020) 2024-09-10T21:24:35.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc52da00640 1 -- 172.21.15.97:0/4048310181 learned_addr learned my addr 172.21.15.97:0/4048310181 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900710b0 0x7f97901f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900afdc0 0x7f97900ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7fc52d000640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 0x7fc5280cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97900d15c0 0x7f9790154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f979006ced0 con 0x7f97900afdc0 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f979006cd50 con 0x7f97900d15c0 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.384+0000 7f9796800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f979006d050 con 0x7f97900710b0 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900710b0 0x7f97901f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900710b0 0x7f97901f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:49036/0 (socket says 172.21.15.97:49036) 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 -- 172.21.15.97:0/1158997414 learned_addr learned my addr 172.21.15.97:0/1158997414 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795e00640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97900d15c0 0x7f9790154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f5f03400640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ee40727e0 0x7f5ee4074ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009ca0 tx=0x7f5ef4009260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9794a00640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900afdc0 0x7f97900ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52e400640 1 -- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 msgr2=0x7fc52813e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52e400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 0x7fc52813e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52e400640 1 -- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 msgr2=0x7fc5280cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52e400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 0x7fc5280cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52e400640 1 -- 172.21.15.97:0/4048310181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281519a0 con 0x7fc52815b110 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52d000640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 0x7fc5280cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52da00640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 0x7fc52813e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc52e400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc52815b110 0x7fc52814e210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc52000be20 tx=0x7fc52000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 -- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97900d15c0 msgr2=0x7f9790154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97900d15c0 0x7f9790154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f3dfaa00640 1 -- 172.21.15.97:0/2526834836 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dfc004a50 con 0x7f3e04161290 2024-09-10T21:24:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 -- 172.21.15.97:0/1158997414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900afdc0 msgr2=0x7f97900ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f1626000640 1 -- 172.21.15.97:0/642481209 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628010630 con 0x7f163006f6d0 2024-09-10T21:24:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1630158470 con 0x7f163006f6d0 2024-09-10T21:24:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1630158a30 con 0x7f163006f6d0 2024-09-10T21:24:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f5efb400640 1 -- 172.21.15.97:0/1092533630 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5ef0014030 con 0x7f5f040b8630 2024-09-10T21:24:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc525600640 1 -- 172.21.15.97:0/4048310181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520019070 con 0x7fc52815b110 2024-09-10T21:24:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900afdc0 0x7f97900ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 -- 172.21.15.97:0/1158997414 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97901f82f0 con 0x7f97900710b0 2024-09-10T21:24:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9794a00640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900afdc0 0x7f97900ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc525600640 1 -- 172.21.15.97:0/4048310181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc520014440 con 0x7fc52815b110 2024-09-10T21:24:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc528164900 con 0x7fc52815b110 2024-09-10T21:24:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f9795400640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900710b0 0x7f97901f7db0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f978800efc0 tx=0x7f978800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc525600640 1 -- 172.21.15.97:0/4048310181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5200148d0 con 0x7fc52815b110 2024-09-10T21:24:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52814d250 con 0x7fc52815b110 2024-09-10T21:24:35.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f1626000640 1 -- 172.21.15.97:0/642481209 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16280108c0 con 0x7f163006f6d0 2024-09-10T21:24:35.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f1626000640 1 -- 172.21.15.97:0/642481209 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628010cf0 con 0x7f163006f6d0 2024-09-10T21:24:35.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f5a03400640 1 -- 172.21.15.97:0/2015396638 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f59f8018b50 con 0x7f5a0c070720 2024-09-10T21:24:35.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5281f3bb0 con 0x7fc52815b110 2024-09-10T21:24:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16080052a0 con 0x7f163006f6d0 2024-09-10T21:24:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.388+0000 7f3dfaa00640 1 -- 172.21.15.97:0/2526834836 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00a9f0 con 0x7f3e04161290 2024-09-10T21:24:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f3dfaa00640 1 -- 172.21.15.97:0/2526834836 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3dfc00abd0 con 0x7f3e04161290 2024-09-10T21:24:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f3dfaa00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de4072610 0x7f3de4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f3dfaa00640 1 -- 172.21.15.97:0/2526834836 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3dfc093f30 con 0x7f3e04161290 2024-09-10T21:24:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f61e6000640 1 -- 172.21.15.97:0/11041999 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f61e0005260 con 0x7f61f01583c0 2024-09-10T21:24:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f3dfaa00640 1 -- 172.21.15.97:0/2526834836 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3dfc020020 con 0x7f3e04161290 2024-09-10T21:24:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f978d600640 1 -- 172.21.15.97:0/1158997414 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978801b070 con 0x7f97900710b0 2024-09-10T21:24:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f978d600640 1 -- 172.21.15.97:0/1158997414 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f978800b630 con 0x7f97900710b0 2024-09-10T21:24:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f978d600640 1 -- 172.21.15.97:0/1158997414 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978800ba20 con 0x7f97900710b0 2024-09-10T21:24:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9790165500 con 0x7f97900710b0 2024-09-10T21:24:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7fc525600640 1 -- 172.21.15.97:0/4048310181 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc52001e440 con 0x7fc52815b110 2024-09-10T21:24:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.392+0000 7f61e6000640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61d0072630 0x7f61d0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f61e6000640 1 -- 172.21.15.97:0/11041999 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f61e012ea30 con 0x7f61f01583c0 2024-09-10T21:24:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97900ce000 con 0x7f97900710b0 2024-09-10T21:24:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f3e02a00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de4072610 0x7f3de4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7fc525600640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc50c0725e0 0x7fc50c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7fc525600640 1 -- 172.21.15.97:0/4048310181 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc520093d20 con 0x7fc52815b110 2024-09-10T21:24:35.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f61ede00640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61d0072630 0x7f61d0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7fc52da00640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc50c0725e0 0x7fc50c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f61e6000640 1 -- 172.21.15.97:0/11041999 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f61e0016cb0 con 0x7f61f01583c0 2024-09-10T21:24:35.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f61ede00640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61d0072630 0x7f61d0074af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0019d0 tx=0x7f61dc0014e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97900b39b0 con 0x7f97900710b0 2024-09-10T21:24:35.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f3e02a00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de4072610 0x7f3de4074ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3df400a870 tx=0x7f3df4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.396+0000 7f1626000640 1 -- 172.21.15.97:0/642481209 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1628029080 con 0x7f163006f6d0 2024-09-10T21:24:35.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.400+0000 7f1626000640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f16100725e0 0x7f1610074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.400+0000 7f1626000640 1 -- 172.21.15.97:0/642481209 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1628094770 con 0x7f163006f6d0 2024-09-10T21:24:35.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.400+0000 7f162d400640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f16100725e0 0x7f1610074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.400+0000 7f978d600640 1 -- 172.21.15.97:0/1158997414 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9788028080 con 0x7f97900710b0 2024-09-10T21:24:35.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.400+0000 7f1626000640 1 -- 172.21.15.97:0/642481209 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1628094b90 con 0x7f163006f6d0 2024-09-10T21:24:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.400+0000 7fc52da00640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc50c0725e0 0x7fc50c074aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc518009e20 tx=0x7fc518009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.400+0000 7fc525600640 1 -- 172.21.15.97:0/4048310181 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc520023020 con 0x7fc52815b110 2024-09-10T21:24:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.404+0000 7f978d600640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f97700726b0 0x7f9770074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.404+0000 7f978d600640 1 -- 172.21.15.97:0/1158997414 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f97880944c0 con 0x7f97900710b0 2024-09-10T21:24:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.404+0000 7f978d600640 1 -- 172.21.15.97:0/1158997414 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f97901f82f0 con 0x7f97900710b0 2024-09-10T21:24:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.404+0000 7f9794a00640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f97700726b0 0x7f9770074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:35.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.404+0000 7f9794a00640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f97700726b0 0x7f9770074b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f979015ed80 tx=0x7f9784009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.404+0000 7f162d400640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f16100725e0 0x7f1610074aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1630168520 tx=0x7f1620009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:35.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.748+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5a0c14f5b0 con 0x7f5a0c070720 2024-09-10T21:24:35.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.752+0000 7f5a03400640 1 -- 172.21.15.97:0/2015396638 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f59f80fb250 con 0x7f5a0c070720 2024-09-10T21:24:35.757 INFO:teuthology.orchestra.run.smithi097.stdout:64424509447 2024-09-10T21:24:35.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5edc005860 con 0x7f5f040b8630 2024-09-10T21:24:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcfec0536e0 con 0x7fcfec051230 2024-09-10T21:24:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59ec072630 msgr2=0x7f59ec074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59ec072630 0x7f59ec074af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c143450 tx=0x7f59fc001420 comp rx=0 tx=0).stop 2024-09-10T21:24:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c070720 msgr2=0x7f5a0c14d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c070720 0x7f5a0c14d8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c10 tx=0x7f59f8005dd0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3e040b3d80 con 0x7f3e04161290 2024-09-10T21:24:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 shutdown_connections 2024-09-10T21:24:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59ec072630 0x7f59ec074af0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c15b110 0x7f5a0c150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a0c072630 0x7f5a0c14de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 --2- 172.21.15.97:0/2015396638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a0c070720 0x7f5a0c14d8c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 >> 172.21.15.97:0/2015396638 conn(0x7f5a0c0b1780 msgr2=0x7f5a0c0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7fcff0c00640 1 -- 172.21.15.97:0/2976763241 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcfe005efc0 con 0x7fcfec051230 2024-09-10T21:24:35.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 shutdown_connections 2024-09-10T21:24:35.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.756+0000 7f5a12800640 1 -- 172.21.15.97:0/2015396638 wait complete. 2024-09-10T21:24:35.765 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-09-10T21:24:35.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7f3dfaa00640 1 -- 172.21.15.97:0/2526834836 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3dfc05f410 con 0x7f3e04161290 2024-09-10T21:24:35.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7f5efb400640 1 -- 172.21.15.97:0/1092533630 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ef005e640 con 0x7f5f040b8630 2024-09-10T21:24:35.766 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-09-10T21:24:35.766 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-09-10T21:24:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f61f015a650 con 0x7f61f01583c0 2024-09-10T21:24:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcfd4072810 msgr2=0x7fcfd4074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7fcfe6000640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcfd4072810 0x7fcfd4074cd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8000c10 tx=0x7fcfe80067b0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec051230 msgr2=0x7fcfec021860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7fcfe6000640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec051230 0x7fcfec021860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000e680 tx=0x7fcfe000eb40 comp rx=0 tx=0).stop 2024-09-10T21:24:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.760+0000 7f61e6000640 1 -- 172.21.15.97:0/11041999 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61e00fa060 con 0x7f61f01583c0 2024-09-10T21:24:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 shutdown_connections 2024-09-10T21:24:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcfd4072810 0x7fcfd4074cd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcfec0dc220 0x7fcfec04dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfec051230 0x7fcfec021860 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 --2- 172.21.15.97:0/2976763241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc00b6f0 0x7fcfec1791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 >> 172.21.15.97:0/2976763241 conn(0x7fcfec054570 msgr2=0x7fcfec0df6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ee40727e0 msgr2=0x7f5ee4074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ee40727e0 0x7f5ee4074ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009ca0 tx=0x7f5ef4009260 comp rx=0 tx=0).stop 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f040b8630 msgr2=0x7f5f04203d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f040b8630 0x7f5f04203d30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c940 tx=0x7f5ef000ce00 comp rx=0 tx=0).stop 2024-09-10T21:24:35.771 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-09-10T21:24:35.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 shutdown_connections 2024-09-10T21:24:35.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ee40727e0 0x7f5ee4074ca0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f040d1270 0x7f5f04161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f040b8630 0x7f5f04203d30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 --2- 172.21.15.97:0/1092533630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f0406db10 0x7f5f041630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 >> 172.21.15.97:0/1092533630 conn(0x7f5f040b9290 msgr2=0x7f5f0409bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de4072610 msgr2=0x7f3de4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de4072610 0x7f3de4074ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3df400a870 tx=0x7f3df4008040 comp rx=0 tx=0).stop 2024-09-10T21:24:35.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1608005860 con 0x7f163006f6d0 2024-09-10T21:24:35.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e04161290 msgr2=0x7f3e041570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e04161290 0x7f3e041570f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00cbf0 tx=0x7f3dfc00cce0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 shutdown_connections 2024-09-10T21:24:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 shutdown_connections 2024-09-10T21:24:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de4072610 0x7f3de4074ad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e04161290 0x7f3e041570f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e040b29b0 0x7f3e04156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 --2- 172.21.15.97:0/2526834836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e040b2490 0x7f3e040c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 >> 172.21.15.97:0/2526834836 conn(0x7f3e040d1a60 msgr2=0x7f3e040b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 shutdown_connections 2024-09-10T21:24:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f1626000640 1 -- 172.21.15.97:0/642481209 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f162805f530 con 0x7f163006f6d0 2024-09-10T21:24:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f5f0a000640 1 -- 172.21.15.97:0/1092533630 wait complete. 2024-09-10T21:24:35.775 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-09-10T21:24:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7fcfe6000640 1 -- 172.21.15.97:0/2976763241 wait complete. 2024-09-10T21:24:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 shutdown_connections 2024-09-10T21:24:35.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f3e03e00640 1 -- 172.21.15.97:0/2526834836 wait complete. 2024-09-10T21:24:35.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61d0072630 msgr2=0x7f61d0074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f61ef200640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61d0072630 0x7f61d0074af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0019d0 tx=0x7f61dc0014e0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f01583c0 msgr2=0x7f61f00cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.764+0000 7f61ef200640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f01583c0 0x7f61f00cd040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ac000 tx=0x7f61e00ac400 comp rx=0 tx=0).stop 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 shutdown_connections 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61d0072630 0x7f61d0074af0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61f01499b0 0x7f61f01530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61f01583c0 0x7f61f00cd040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 --2- 172.21.15.97:0/11041999 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61f006eba0 0x7f61f014b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 >> 172.21.15.97:0/11041999 conn(0x7f61f00cf890 msgr2=0x7f61f0073540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 shutdown_connections 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f61ef200640 1 -- 172.21.15.97:0/11041999 wait complete. 2024-09-10T21:24:35.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f979015b670 con 0x7f97900710b0 2024-09-10T21:24:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f978d600640 1 -- 172.21.15.97:0/1158997414 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f978805f280 con 0x7f97900710b0 2024-09-10T21:24:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f16100725e0 msgr2=0x7f1610074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f16100725e0 0x7f1610074aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1630168520 tx=0x7f1620009040 comp rx=0 tx=0).stop 2024-09-10T21:24:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 msgr2=0x7f16301fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16301fb940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f162800ebc0 tx=0x7f162800c6a0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.778 INFO:teuthology.orchestra.run.smithi097.stdout:55834574857 2024-09-10T21:24:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 shutdown_connections 2024-09-10T21:24:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f16100725e0 0x7f1610074aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1630159780 0x7f1630157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16300cd1f0 0x7f1630141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 --2- 172.21.15.97:0/642481209 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f163006f6d0 0x7f16301fb940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 >> 172.21.15.97:0/642481209 conn(0x7f16300afa50 msgr2=0x7f16300ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc528152040 con 0x7fc52815b110 2024-09-10T21:24:35.780 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-09-10T21:24:35.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.768+0000 7fc525600640 1 -- 172.21.15.97:0/4048310181 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc52005ea50 con 0x7fc52815b110 2024-09-10T21:24:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f97700726b0 msgr2=0x7f9770074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7f9796800640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f97700726b0 0x7f9770074b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f979015ed80 tx=0x7f9784009040 comp rx=0 tx=0).stop 2024-09-10T21:24:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900710b0 msgr2=0x7f97901f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7f9796800640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900710b0 0x7f97901f7db0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f978800efc0 tx=0x7f978800c800 comp rx=0 tx=0).stop 2024-09-10T21:24:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc50c0725e0 msgr2=0x7fc50c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc50c0725e0 0x7fc50c074aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc518009e20 tx=0x7fc518009190 comp rx=0 tx=0).stop 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc52815b110 msgr2=0x7fc52814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc52815b110 0x7fc52814e210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc52000be20 tx=0x7fc52000bf10 comp rx=0 tx=0).stop 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 shutdown_connections 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc50c0725e0 0x7fc50c074aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc52815b110 0x7fc52814e210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5280cf490 0x7fc5280cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 --2- 172.21.15.97:0/4048310181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5280cf0b0 0x7fc52813e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.772+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 >> 172.21.15.97:0/4048310181 conn(0x7fc5280b1780 msgr2=0x7fc528151290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 shutdown_connections 2024-09-10T21:24:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 shutdown_connections 2024-09-10T21:24:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f97700726b0 0x7f9770074b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97900d15c0 0x7f9790154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97900afdc0 0x7f97900ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 --2- 172.21.15.97:0/1158997414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97900710b0 0x7f97901f7db0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 >> 172.21.15.97:0/1158997414 conn(0x7f97900d09c0 msgr2=0x7f9790068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 shutdown_connections 2024-09-10T21:24:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f162f200640 1 -- 172.21.15.97:0/642481209 wait complete. 2024-09-10T21:24:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7fc530400640 1 -- 172.21.15.97:0/4048310181 wait complete. 2024-09-10T21:24:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 shutdown_connections 2024-09-10T21:24:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:35.776+0000 7f9796800640 1 -- 172.21.15.97:0/1158997414 wait complete. 2024-09-10T21:24:35.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-09-10T21:24:35.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.5 2024-09-10T21:24:35.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.1 2024-09-10T21:24:35.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.0 2024-09-10T21:24:35.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.3 2024-09-10T21:24:35.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-10T21:24:35.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-09-10T21:24:35.827 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.2 2024-09-10T21:24:36.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T21:24:36.818 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T21:24:36.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T21:24:36.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T21:24:36.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T21:24:36.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T21:24:36.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T21:24:36.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T21:24:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 -- 172.21.15.97:0/4255016476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b080e2ee0 msgr2=0x7f8b08034340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/4255016476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b080e2ee0 0x7f8b08034340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8afc005fd0 tx=0x7f8afc01c7f0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 -- 172.21.15.97:0/4255016476 shutdown_connections 2024-09-10T21:24:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/4255016476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b08035f10 0x7f8b080363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/4255016476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b080e2ee0 0x7f8b08034340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 -- 172.21.15.97:0/4255016476 >> 172.21.15.97:0/4255016476 conn(0x7f8b0801d780 msgr2=0x7f8b08054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 -- 172.21.15.97:0/4255016476 shutdown_connections 2024-09-10T21:24:37.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.024+0000 7f8b0fe00640 1 -- 172.21.15.97:0/4255016476 wait complete. 2024-09-10T21:24:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 Processor -- start 2024-09-10T21:24:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 -- start start 2024-09-10T21:24:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b08005a10 0x7f8b080d6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b08035f10 0x7f8b08031d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b080e2ee0 0x7f8b08032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08033a80 con 0x7f8b08005a10 2024-09-10T21:24:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08033900 con 0x7f8b08035f10 2024-09-10T21:24:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.028+0000 7f8b0fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8b080ecdf0 con 0x7f8b080e2ee0 2024-09-10T21:24:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b08035f10 0x7f8b08031d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b080e2ee0 0x7f8b08032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b08035f10 0x7f8b08031d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44412/0 (socket says 172.21.15.97:44412) 2024-09-10T21:24:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 -- 172.21.15.97:0/2155896656 learned_addr learned my addr 172.21.15.97:0/2155896656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 -- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b080e2ee0 msgr2=0x7f8b08032270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b080e2ee0 0x7f8b08032270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 -- 172.21.15.97:0/2155896656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b08005a10 msgr2=0x7f8b080d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b08005a10 0x7f8b080d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0e000640 1 -- 172.21.15.97:0/2155896656 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b080d6ca0 con 0x7f8b08035f10 2024-09-10T21:24:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.032+0000 7f8b0f400640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b080e2ee0 0x7f8b08032270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:37.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.036+0000 7f8b0e000640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b08035f10 0x7f8b08031d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000d950 tx=0x7f8b0000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.036+0000 7f8b06a00640 1 -- 172.21.15.97:0/2155896656 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00017070 con 0x7f8b08035f10 2024-09-10T21:24:37.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.036+0000 7f8b06a00640 1 -- 172.21.15.97:0/2155896656 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b00009c30 con 0x7f8b08035f10 2024-09-10T21:24:37.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.036+0000 7f8b06a00640 1 -- 172.21.15.97:0/2155896656 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000106c0 con 0x7f8b08035f10 2024-09-10T21:24:37.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.036+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b08052c60 con 0x7f8b08035f10 2024-09-10T21:24:37.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.036+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b08053220 con 0x7f8b08035f10 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b080363a0 con 0x7f8b08035f10 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 -- 172.21.15.97:0/4176457321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815e840 msgr2=0x7fc86806fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 --2- 172.21.15.97:0/4176457321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815e840 0x7fc86806fa60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc858009900 tx=0x7fc85801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 -- 172.21.15.97:0/4176457321 shutdown_connections 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 --2- 172.21.15.97:0/4176457321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815e840 0x7fc86806fa60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 --2- 172.21.15.97:0/4176457321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815eea0 0x7fc86815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 -- 172.21.15.97:0/4176457321 >> 172.21.15.97:0/4176457321 conn(0x7fc8680d09a0 msgr2=0x7fc8680d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 -- 172.21.15.97:0/4176457321 shutdown_connections 2024-09-10T21:24:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7f8b06a00640 1 -- 172.21.15.97:0/2155896656 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8b00010860 con 0x7f8b08035f10 2024-09-10T21:24:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.040+0000 7fc86d800640 1 -- 172.21.15.97:0/4176457321 wait complete. 2024-09-10T21:24:37.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7f8b06a00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8af00725e0 0x7f8af0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7f8b06a00640 1 -- 172.21.15.97:0/2155896656 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8b00093840 con 0x7f8b08035f10 2024-09-10T21:24:37.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7f8b0ea00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8af00725e0 0x7f8af0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 Processor -- start 2024-09-10T21:24:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 -- start start 2024-09-10T21:24:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7f8b0ea00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8af00725e0 0x7f8af0074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ae000 tx=0x7f8afc019590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680d0520 0x7fc868146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815e840 0x7fc86814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815eea0 0x7fc86814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc8681676a0 con 0x7fc86815eea0 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc868070d50 con 0x7fc8680d0520 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc86d800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc868167820 con 0x7fc86815e840 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc867e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680d0520 0x7fc868146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fc867400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815e840 0x7fc86814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815e840 0x7fc86814b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:49080/0 (socket says 172.21.15.97:49080) 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 -- 172.21.15.97:0/3389427466 learned_addr learned my addr 172.21.15.97:0/3389427466 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc86ce00640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815eea0 0x7fc86814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 -- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680d0520 msgr2=0x7fc868146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680d0520 0x7fc868146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 -- 172.21.15.97:0/3389427466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815eea0 msgr2=0x7fc86814be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815eea0 0x7fc86814be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 -- 172.21.15.97:0/3389427466 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86814f5a0 con 0x7fc86815e840 2024-09-10T21:24:37.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc867400640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815e840 0x7fc86814b8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc86000cce0 tx=0x7fc86000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc86ce00640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815eea0 0x7fc86814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:37.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc85fe00640 1 -- 172.21.15.97:0/3389427466 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000f450 con 0x7fc86815e840 2024-09-10T21:24:37.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc85fe00640 1 -- 172.21.15.97:0/3389427466 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc86000f6e0 con 0x7fc86815e840 2024-09-10T21:24:37.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc85fe00640 1 -- 172.21.15.97:0/3389427466 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000faf0 con 0x7fc86815e840 2024-09-10T21:24:37.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7f8b06a00640 1 -- 172.21.15.97:0/2155896656 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b00009e20 con 0x7f8b08035f10 2024-09-10T21:24:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fe90d800640 1 -- 172.21.15.97:0/279500278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90815dd70 msgr2=0x7fe9080d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fe90d800640 1 --2- 172.21.15.97:0/279500278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90815dd70 0x7fe9080d06a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009870 tx=0x7fe8f801c820 comp rx=0 tx=0).stop 2024-09-10T21:24:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fe90d800640 1 -- 172.21.15.97:0/279500278 shutdown_connections 2024-09-10T21:24:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fe90d800640 1 --2- 172.21.15.97:0/279500278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90815dd70 0x7fe9080d06a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fe90d800640 1 -- 172.21.15.97:0/279500278 >> 172.21.15.97:0/279500278 conn(0x7fe9080d3c90 msgr2=0x7fe9080d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.044+0000 7fe90d800640 1 -- 172.21.15.97:0/279500278 shutdown_connections 2024-09-10T21:24:37.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fe90d800640 1 -- 172.21.15.97:0/279500278 wait complete. 2024-09-10T21:24:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fe90d800640 1 Processor -- start 2024-09-10T21:24:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fe90d800640 1 -- start start 2024-09-10T21:24:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86814f880 con 0x7fc86815e840 2024-09-10T21:24:37.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.048+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86814fe40 con 0x7fc86815e840 2024-09-10T21:24:37.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe90d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9080b2c70 0x7fe9080cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe90d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9080cf740 0x7fe908203f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe90d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe908208890 0x7fe908207bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9080b2c70 0x7fe9080cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9080b2c70 0x7fe9080cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44432/0 (socket says 172.21.15.97:44432) 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe907400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe908208890 0x7fe908207bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe907400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe908208890 0x7fe908207bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41450/0 (socket says 172.21.15.97:41450) 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 -- 172.21.15.97:0/930345881 learned_addr learned my addr 172.21.15.97:0/930345881 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe90816c5f0 con 0x7fe908208890 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe90816c470 con 0x7fe9080b2c70 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe90816c770 con 0x7fe9080cf740 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 -- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9080cf740 msgr2=0x7fe908203f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9080cf740 0x7fe908203f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 -- 172.21.15.97:0/930345881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe908208890 msgr2=0x7fe908207bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe908208890 0x7fe908207bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9080c8720 con 0x7fe9080b2c70 2024-09-10T21:24:37.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe906a00640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9080b2c70 0x7fe9080cd7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8000c10 tx=0x7fe8f80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.052+0000 7fe8fea00640 1 -- 172.21.15.97:0/930345881 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8005230 con 0x7fe9080b2c70 2024-09-10T21:24:37.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fe8fea00640 1 -- 172.21.15.97:0/930345881 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8f80bf070 con 0x7fe9080b2c70 2024-09-10T21:24:37.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fe8fea00640 1 -- 172.21.15.97:0/930345881 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80044c0 con 0x7fe9080b2c70 2024-09-10T21:24:37.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86806fa60 con 0x7fc86815e840 2024-09-10T21:24:37.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fc85fe00640 1 -- 172.21.15.97:0/3389427466 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc86000fc90 con 0x7fc86815e840 2024-09-10T21:24:37.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9080c8a00 con 0x7fe9080b2c70 2024-09-10T21:24:37.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9080c8fc0 con 0x7fe9080b2c70 2024-09-10T21:24:37.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9080008e0 con 0x7fe9080b2c70 2024-09-10T21:24:37.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fc85fe00640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8480725e0 0x7fc848074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fc85fe00640 1 -- 172.21.15.97:0/3389427466 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc860093fe0 con 0x7fc86815e840 2024-09-10T21:24:37.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.056+0000 7fc867e00640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8480725e0 0x7fc848074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.060+0000 7fc867e00640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8480725e0 0x7fc848074aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc854007fd0 tx=0x7fc854009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.060+0000 7fe8fea00640 1 -- 172.21.15.97:0/930345881 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe8f80046d0 con 0x7fe9080b2c70 2024-09-10T21:24:37.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.064+0000 7fc85fe00640 1 -- 172.21.15.97:0/3389427466 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc860014030 con 0x7fc86815e840 2024-09-10T21:24:37.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.064+0000 7fe8fea00640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8e8072600 0x7fe8e8074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.064+0000 7fe8fea00640 1 -- 172.21.15.97:0/930345881 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe8f812ef50 con 0x7fe9080b2c70 2024-09-10T21:24:37.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.064+0000 7fe906000640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8e8072600 0x7fe8e8074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.064+0000 7fe8fea00640 1 -- 172.21.15.97:0/930345881 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8f80c8020 con 0x7fe9080b2c70 2024-09-10T21:24:37.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.064+0000 7fe906000640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8e8072600 0x7fe8e8074ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007fd0 tx=0x7fe8f4007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.100+0000 7f7060600640 1 -- 172.21.15.97:0/167259639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7058161d90 msgr2=0x7f7058162170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.100+0000 7f7060600640 1 --2- 172.21.15.97:0/167259639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7058161d90 0x7f7058162170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7044009900 tx=0x7f704401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.100+0000 7f7060600640 1 -- 172.21.15.97:0/167259639 shutdown_connections 2024-09-10T21:24:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.100+0000 7f7060600640 1 --2- 172.21.15.97:0/167259639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f70580b3040 0x7f70580b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.100+0000 7f7060600640 1 --2- 172.21.15.97:0/167259639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7058161d90 0x7f7058162170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.100+0000 7f7060600640 1 -- 172.21.15.97:0/167259639 >> 172.21.15.97:0/167259639 conn(0x7f70580d1ca0 msgr2=0x7f70580d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.104+0000 7f7060600640 1 -- 172.21.15.97:0/167259639 shutdown_connections 2024-09-10T21:24:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.104+0000 7f7060600640 1 -- 172.21.15.97:0/167259639 wait complete. 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.104+0000 7f7060600640 1 Processor -- start 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f7060600640 1 -- start start 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f7060600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70580b3040 0x7f70580b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f7060600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70580cfd30 0x7f7058147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f7060600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7058161d90 0x7f705815df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f7060600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f705806ec00 con 0x7f70580cfd30 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f7060600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f705806ea80 con 0x7f70580b3040 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f7060600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f705806ed80 con 0x7f7058161d90 2024-09-10T21:24:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70580cfd30 0x7f7058147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70580b3040 0x7f70580b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705e600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7058161d90 0x7f705815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70580cfd30 0x7f7058147be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41460/0 (socket says 172.21.15.97:41460) 2024-09-10T21:24:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705d200640 1 -- 172.21.15.97:0/1896631823 learned_addr learned my addr 172.21.15.97:0/1896631823 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705dc00640 1 -- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7058161d90 msgr2=0x7f705815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705dc00640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7058161d90 0x7f705815df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705dc00640 1 -- 172.21.15.97:0/1896631823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70580cfd30 msgr2=0x7f7058147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705dc00640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70580cfd30 0x7f7058147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705dc00640 1 -- 172.21.15.97:0/1896631823 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70580b5220 con 0x7f70580b3040 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705d200640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70580cfd30 0x7f7058147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.108+0000 7f705dc00640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70580b3040 0x7f70580b4ad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70440ac000 tx=0x7f7044005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.112+0000 7f7055600640 1 -- 172.21.15.97:0/1896631823 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440bb070 con 0x7f70580b3040 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.112+0000 7f7055600640 1 -- 172.21.15.97:0/1896631823 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70440043f0 con 0x7f70580b3040 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.112+0000 7f7055600640 1 -- 172.21.15.97:0/1896631823 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440b2040 con 0x7f70580b3040 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.112+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70580c8560 con 0x7f70580b3040 2024-09-10T21:24:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.112+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70580b7890 con 0x7f70580b3040 2024-09-10T21:24:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.116+0000 7f7055600640 1 -- 172.21.15.97:0/1896631823 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f70440b5070 con 0x7f70580b3040 2024-09-10T21:24:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.116+0000 7f7055600640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f70380726b0 0x7f7038074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.116+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70580b24a0 con 0x7f70580b3040 2024-09-10T21:24:37.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.116+0000 7f7055600640 1 -- 172.21.15.97:0/1896631823 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f704412cb90 con 0x7f70580b3040 2024-09-10T21:24:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.120+0000 7f705d200640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f70380726b0 0x7f7038074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.120+0000 7f705d200640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f70380726b0 0x7f7038074b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f705816ec90 tx=0x7f704c00b6f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f7055600640 1 -- 172.21.15.97:0/1896631823 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f70440f1de0 con 0x7f70580b3040 2024-09-10T21:24:37.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 -- 172.21.15.97:0/1341613949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2624143f90 msgr2=0x7f26240b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 --2- 172.21.15.97:0/1341613949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2624143f90 0x7f26240b2a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2610009900 tx=0x7f261001c900 comp rx=0 tx=0).stop 2024-09-10T21:24:37.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 -- 172.21.15.97:0/1341613949 shutdown_connections 2024-09-10T21:24:37.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 --2- 172.21.15.97:0/1341613949 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f262406fd50 0x7f26240701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 --2- 172.21.15.97:0/1341613949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2624143f90 0x7f26240b2a80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 -- 172.21.15.97:0/1341613949 >> 172.21.15.97:0/1341613949 conn(0x7f26240b1780 msgr2=0x7f26240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 -- 172.21.15.97:0/1341613949 shutdown_connections 2024-09-10T21:24:37.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.124+0000 7f2628c00640 1 -- 172.21.15.97:0/1341613949 wait complete. 2024-09-10T21:24:37.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.128+0000 7f2628c00640 1 Processor -- start 2024-09-10T21:24:37.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.128+0000 7f2628c00640 1 -- start start 2024-09-10T21:24:37.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.128+0000 7f2628c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262406fd50 0x7f2624159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.128+0000 7f2628c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2624143f90 0x7f262415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.128+0000 7f2628c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2624144690 0x7f26241335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262406fd50 0x7f2624159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262406fd50 0x7f2624159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44474/0 (socket says 172.21.15.97:44474) 2024-09-10T21:24:37.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621e00640 1 -- 172.21.15.97:0/1469491354 learned_addr learned my addr 172.21.15.97:0/1469491354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2622800640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2624144690 0x7f26241335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f262406f050 con 0x7f2624143f90 2024-09-10T21:24:37.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f262406eed0 con 0x7f262406fd50 2024-09-10T21:24:37.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f262406f1d0 con 0x7f2624144690 2024-09-10T21:24:37.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621400640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2624143f90 0x7f262415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621400640 1 -- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2624144690 msgr2=0x7f26241335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621400640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2624144690 0x7f26241335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621400640 1 -- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262406fd50 msgr2=0x7f2624159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621400640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262406fd50 0x7f2624159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.132+0000 7f2621400640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2624133b10 con 0x7f2624143f90 2024-09-10T21:24:37.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f2622800640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2624144690 0x7f26241335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:37.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f2621e00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262406fd50 0x7f2624159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:37.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f2621400640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2624143f90 0x7f262415a890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f261400c390 tx=0x7f261400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f261a000640 1 -- 172.21.15.97:0/1469491354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614017070 con 0x7f2624143f90 2024-09-10T21:24:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2624132530 con 0x7f2624143f90 2024-09-10T21:24:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2624132af0 con 0x7f2624143f90 2024-09-10T21:24:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f261a000640 1 -- 172.21.15.97:0/1469491354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26140053c0 con 0x7f2624143f90 2024-09-10T21:24:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.136+0000 7f261a000640 1 -- 172.21.15.97:0/1469491354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614017070 con 0x7f2624143f90 2024-09-10T21:24:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.140+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0052a0 con 0x7f2624143f90 2024-09-10T21:24:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.140+0000 7f261a000640 1 -- 172.21.15.97:0/1469491354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2614003600 con 0x7f2624143f90 2024-09-10T21:24:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.144+0000 7f261a000640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26040729e0 0x7f2604074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.144+0000 7f261a000640 1 -- 172.21.15.97:0/1469491354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f26140050d0 con 0x7f2624143f90 2024-09-10T21:24:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.144+0000 7f2621e00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26040729e0 0x7f2604074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.144+0000 7f2621e00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26040729e0 0x7f2604074ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2610000c10 tx=0x7f261001b140 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.152+0000 7f261a000640 1 -- 172.21.15.97:0/1469491354 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f261405d0e0 con 0x7f2624143f90 2024-09-10T21:24:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.156+0000 7fc9a9200640 1 -- 172.21.15.97:0/3623195171 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a4070d50 msgr2=0x7fc9a4162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.156+0000 7fc9a9200640 1 --2- 172.21.15.97:0/3623195171 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a4070d50 0x7fc9a4162350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009c80 tx=0x7fc99c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.156+0000 7fc9a9200640 1 -- 172.21.15.97:0/3623195171 shutdown_connections 2024-09-10T21:24:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.156+0000 7fc9a9200640 1 --2- 172.21.15.97:0/3623195171 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a4070d50 0x7fc9a4162350 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009c80 tx=0x7fc99c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.156+0000 7fc9a9200640 1 --2- 172.21.15.97:0/3623195171 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9a4161a30 0x7fc9a4161e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.156+0000 7fc9a9200640 1 -- 172.21.15.97:0/3623195171 >> 172.21.15.97:0/3623195171 conn(0x7fc9a40cfb10 msgr2=0x7fc9a40cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.156+0000 7fc9a9200640 1 -- 172.21.15.97:0/3623195171 shutdown_connections 2024-09-10T21:24:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.160+0000 7fc9a9200640 1 -- 172.21.15.97:0/3623195171 wait complete. 2024-09-10T21:24:37.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.160+0000 7fc9a9200640 1 Processor -- start 2024-09-10T21:24:37.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.160+0000 7fc9a9200640 1 -- start start 2024-09-10T21:24:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a406fd60 0x7fc9a415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9a4161a30 0x7fc9a40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9a4154d10 0x7fc9a41551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4159be0 con 0x7fc9a4161a30 2024-09-10T21:24:37.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4159a60 con 0x7fc9a406fd60 2024-09-10T21:24:37.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a4159d60 con 0x7fc9a4154d10 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a406fd60 0x7fc9a415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a2800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9a4154d10 0x7fc9a41551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a2800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9a4154d10 0x7fc9a41551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:49136/0 (socket says 172.21.15.97:49136) 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9a4161a30 0x7fc9a40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a2800640 1 -- 172.21.15.97:0/482891078 learned_addr learned my addr 172.21.15.97:0/482891078 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a406fd60 0x7fc9a415c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44500/0 (socket says 172.21.15.97:44500) 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 -- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9a4154d10 msgr2=0x7fc9a41551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9a4154d10 0x7fc9a41551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 -- 172.21.15.97:0/482891078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9a4161a30 msgr2=0x7fc9a40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9a4161a30 0x7fc9a40b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 -- 172.21.15.97:0/482891078 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a415ee30 con 0x7fc9a406fd60 2024-09-10T21:24:37.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1400640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9a4161a30 0x7fc9a40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:37.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a1e00640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a406fd60 0x7fc9a415c5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc99000e9f0 tx=0x7fc99000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc99a000640 1 -- 172.21.15.97:0/482891078 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99000ccc0 con 0x7fc9a406fd60 2024-09-10T21:24:37.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a415e2b0 con 0x7fc9a406fd60 2024-09-10T21:24:37.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a415e870 con 0x7fc9a406fd60 2024-09-10T21:24:37.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc99a000640 1 -- 172.21.15.97:0/482891078 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc990004020 con 0x7fc9a406fd60 2024-09-10T21:24:37.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.164+0000 7fc99a000640 1 -- 172.21.15.97:0/482891078 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99000ce60 con 0x7fc9a406fd60 2024-09-10T21:24:37.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.168+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c0052a0 con 0x7fc9a406fd60 2024-09-10T21:24:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 -- 172.21.15.97:0/814665730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5014073740 msgr2=0x7f50140b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 --2- 172.21.15.97:0/814665730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5014073740 0x7f50140b49f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5000009900 tx=0x7f500001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7fc99a000640 1 -- 172.21.15.97:0/482891078 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc990003600 con 0x7fc9a406fd60 2024-09-10T21:24:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 -- 172.21.15.97:0/814665730 shutdown_connections 2024-09-10T21:24:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 --2- 172.21.15.97:0/814665730 >> v2:172.21.15.194:3300/0 conn(0x7f501415b5d0 0x7f501406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 --2- 172.21.15.97:0/814665730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5014073b20 0x7f501415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 --2- 172.21.15.97:0/814665730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5014073740 0x7f50140b49f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 -- 172.21.15.97:0/814665730 >> 172.21.15.97:0/814665730 conn(0x7f50140cfb10 msgr2=0x7f50140cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 -- 172.21.15.97:0/814665730 shutdown_connections 2024-09-10T21:24:37.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 -- 172.21.15.97:0/814665730 wait complete. 2024-09-10T21:24:37.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 Processor -- start 2024-09-10T21:24:37.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7fc99a000640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9840725e0 0x7fc984074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7fc99a000640 1 -- 172.21.15.97:0/482891078 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc990093d80 con 0x7fc9a406fd60 2024-09-10T21:24:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.172+0000 7f501aa00640 1 -- start start 2024-09-10T21:24:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f501aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5014073740 0x7f50141482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f501aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5014073b20 0x7f5014146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f501aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f501415b5d0 0x7f5014152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f501aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f501415e950 con 0x7f501415b5d0 2024-09-10T21:24:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f501aa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f501415e7d0 con 0x7f5014073740 2024-09-10T21:24:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f501aa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f501415ead0 con 0x7f5014073b20 2024-09-10T21:24:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f501415b5d0 0x7f5014152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f501415b5d0 0x7f5014152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41484/0 (socket says 172.21.15.97:41484) 2024-09-10T21:24:37.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 -- 172.21.15.97:0/4051638730 learned_addr learned my addr 172.21.15.97:0/4051638730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5013400640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5014073b20 0x7f5014146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5013e00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5014073740 0x7f50141482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 -- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5014073b20 msgr2=0x7f5014146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5014073b20 0x7f5014146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 -- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5014073740 msgr2=0x7f50141482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5014073740 0x7f50141482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 -- 172.21.15.97:0/4051638730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5014147490 con 0x7f501415b5d0 2024-09-10T21:24:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5013e00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5014073740 0x7f50141482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f5018a00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f501415b5d0 0x7f5014152a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f500c00da10 tx=0x7f500c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7fc9a1400640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9840725e0 0x7fc984074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7fc99a000640 1 -- 172.21.15.97:0/482891078 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc99005f0a0 con 0x7fc9a406fd60 2024-09-10T21:24:37.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.176+0000 7f500be00640 1 -- 172.21.15.97:0/4051638730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c017070 con 0x7f501415b5d0 2024-09-10T21:24:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.180+0000 7f500be00640 1 -- 172.21.15.97:0/4051638730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f500c004b30 con 0x7f501415b5d0 2024-09-10T21:24:37.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.180+0000 7f500be00640 1 -- 172.21.15.97:0/4051638730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c00be60 con 0x7f501415b5d0 2024-09-10T21:24:37.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.180+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5014147730 con 0x7f501415b5d0 2024-09-10T21:24:37.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.180+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5014147cf0 con 0x7f501415b5d0 2024-09-10T21:24:37.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.180+0000 7fc9a1400640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9840725e0 0x7fc984074aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0af000 tx=0x7fc99c019b40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.180+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501415f030 con 0x7f501415b5d0 2024-09-10T21:24:37.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.184+0000 7f500be00640 1 -- 172.21.15.97:0/4051638730 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f500c003980 con 0x7f501415b5d0 2024-09-10T21:24:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.188+0000 7f500be00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ff4076ac0 0x7f4ff4078f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.188+0000 7f500be00640 1 -- 172.21.15.97:0/4051638730 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f500c0940c0 con 0x7f501415b5d0 2024-09-10T21:24:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.188+0000 7f5013e00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ff4076ac0 0x7f4ff4078f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.188+0000 7f5013e00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ff4076ac0 0x7f4ff4078f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50000af000 tx=0x7f500001b0e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.192+0000 7f500be00640 1 -- 172.21.15.97:0/4051638730 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f500c020070 con 0x7f501415b5d0 2024-09-10T21:24:37.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.192+0000 7f7207e00640 1 -- 172.21.15.97:0/724405764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000ad440 con 0x7f720815ac30 2024-09-10T21:24:37.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.196+0000 7f7210000640 1 -- 172.21.15.97:0/724405764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 msgr2=0x7f720815e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.196+0000 7f7210000640 1 --2- 172.21.15.97:0/724405764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 0x7f720815e5e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7200009bc0 tx=0x7f720001c9c0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.200+0000 7f7210000640 1 -- 172.21.15.97:0/724405764 shutdown_connections 2024-09-10T21:24:37.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.200+0000 7f7210000640 1 --2- 172.21.15.97:0/724405764 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f720815eb20 0x7f720815ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.200+0000 7f7210000640 1 --2- 172.21.15.97:0/724405764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 0x7f720815e5e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.200+0000 7f7210000640 1 -- 172.21.15.97:0/724405764 >> 172.21.15.97:0/724405764 conn(0x7f72080d0be0 msgr2=0x7f72080d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.200+0000 7f7210000640 1 -- 172.21.15.97:0/724405764 shutdown_connections 2024-09-10T21:24:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.200+0000 7f7210000640 1 -- 172.21.15.97:0/724405764 wait complete. 2024-09-10T21:24:37.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.200+0000 7f7210000640 1 Processor -- start 2024-09-10T21:24:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f7210000640 1 -- start start 2024-09-10T21:24:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f7210000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f720806cb40 0x7f7208205590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f7210000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 0x7f7208157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f720d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f720806cb40 0x7f7208205590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f720d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f720806cb40 0x7f7208205590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44534/0 (socket says 172.21.15.97:44534) 2024-09-10T21:24:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f720cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 0x7f7208157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f720cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 0x7f7208157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41490/0 (socket says 172.21.15.97:41490) 2024-09-10T21:24:37.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f720d600640 1 -- 172.21.15.97:0/324497626 learned_addr learned my addr 172.21.15.97:0/324497626 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f7210000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f720815eb20 0x7f7208157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.208+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f720806d700 con 0x7f720815ac30 2024-09-10T21:24:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f720e000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f720815eb20 0x7f7208157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72080b4d00 con 0x7f720806cb40 2024-09-10T21:24:37.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f720806d880 con 0x7f720815eb20 2024-09-10T21:24:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f720e000640 1 -- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f720806cb40 msgr2=0x7f7208205590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f720e000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f720806cb40 0x7f7208205590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f720e000640 1 -- 172.21.15.97:0/324497626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 msgr2=0x7f7208157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f720e000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 0x7f7208157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f720e000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7208205ad0 con 0x7f720815eb20 2024-09-10T21:24:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f720e000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f720815eb20 0x7f7208157640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00faa0 tx=0x7f71fc00ff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7205600640 1 -- 172.21.15.97:0/324497626 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc01a070 con 0x7f720815eb20 2024-09-10T21:24:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7205600640 1 -- 172.21.15.97:0/324497626 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71fc008480 con 0x7f720815eb20 2024-09-10T21:24:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7205600640 1 -- 172.21.15.97:0/324497626 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc006960 con 0x7f720815eb20 2024-09-10T21:24:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72081561a0 con 0x7f720815eb20 2024-09-10T21:24:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7208204d90 con 0x7f720815eb20 2024-09-10T21:24:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.212+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72081566a0 con 0x7f720815eb20 2024-09-10T21:24:37.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.216+0000 7f7205600640 1 -- 172.21.15.97:0/324497626 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71fc006b00 con 0x7f720815eb20 2024-09-10T21:24:37.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.216+0000 7f7205600640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71e80725e0 0x7f71e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.220+0000 7f720d600640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71e80725e0 0x7f71e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.220+0000 7f7205600640 1 -- 172.21.15.97:0/324497626 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f71fc095f70 con 0x7f720815eb20 2024-09-10T21:24:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.220+0000 7f720d600640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71e80725e0 0x7f71e8074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f720815da00 tx=0x7f720000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.220+0000 7f7205600640 1 -- 172.21.15.97:0/324497626 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f72081566a0 con 0x7f720815eb20 2024-09-10T21:24:37.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.404+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8b0804b300 con 0x7f8b08035f10 2024-09-10T21:24:37.414 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.404+0000 7f8b06a00640 1 -- 172.21.15.97:0/2155896656 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b0005e580 con 0x7f8b08035f10 2024-09-10T21:24:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8af00725e0 msgr2=0x7f8af0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8af00725e0 0x7f8af0074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ae000 tx=0x7f8afc019590 comp rx=0 tx=0).stop 2024-09-10T21:24:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b08035f10 msgr2=0x7f8b08031d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b08035f10 0x7f8b08031d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000d950 tx=0x7f8b0000de10 comp rx=0 tx=0).stop 2024-09-10T21:24:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 shutdown_connections 2024-09-10T21:24:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8af00725e0 0x7f8af0074aa0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ae000 tx=0x7f8afc019590 comp rx=0 tx=0).stop 2024-09-10T21:24:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8b080e2ee0 0x7f8b08032270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b08035f10 0x7f8b08031d30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 --2- 172.21.15.97:0/2155896656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b08005a10 0x7f8b080d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.408+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 >> 172.21.15.97:0/2155896656 conn(0x7f8b0801d780 msgr2=0x7f8b080dd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.416+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 shutdown_connections 2024-09-10T21:24:37.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.416+0000 7f8b0fe00640 1 -- 172.21.15.97:0/2155896656 wait complete. 2024-09-10T21:24:37.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.416+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc868074770 con 0x7fc86815e840 2024-09-10T21:24:37.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.416+0000 7fc85fe00640 1 -- 172.21.15.97:0/3389427466 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc86005f3d0 con 0x7fc86815e840 2024-09-10T21:24:37.425 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-09-10T21:24:37.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.420+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe9080c83d0 con 0x7fe9080b2c70 2024-09-10T21:24:37.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.420+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8480725e0 msgr2=0x7fc848074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.420+0000 7fe8fea00640 1 -- 172.21.15.97:0/930345881 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe8f8016410 con 0x7fe9080b2c70 2024-09-10T21:24:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.420+0000 7fc86d800640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8480725e0 0x7fc848074aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc854007fd0 tx=0x7fc854009040 comp rx=0 tx=0).stop 2024-09-10T21:24:37.429 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.420+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815e840 msgr2=0x7fc86814b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815e840 0x7fc86814b8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc86000cce0 tx=0x7fc86000a620 comp rx=0 tx=0).stop 2024-09-10T21:24:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 shutdown_connections 2024-09-10T21:24:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8480725e0 0x7fc848074aa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815eea0 0x7fc86814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc86815e840 0x7fc86814b8c0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc86000cce0 tx=0x7fc86000a620 comp rx=0 tx=0).stop 2024-09-10T21:24:37.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 --2- 172.21.15.97:0/3389427466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680d0520 0x7fc868146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 >> 172.21.15.97:0/3389427466 conn(0x7fc8680d09a0 msgr2=0x7fc868073b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 shutdown_connections 2024-09-10T21:24:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fc86d800640 1 -- 172.21.15.97:0/3389427466 wait complete. 2024-09-10T21:24:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8e8072600 msgr2=0x7fe8e8074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fe90d800640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8e8072600 0x7fe8e8074ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007fd0 tx=0x7fe8f4007980 comp rx=0 tx=0).stop 2024-09-10T21:24:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9080b2c70 msgr2=0x7fe9080cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.424+0000 7fe90d800640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9080b2c70 0x7fe9080cd7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8000c10 tx=0x7fe8f80040c0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 shutdown_connections 2024-09-10T21:24:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8e8072600 0x7fe8e8074ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe908208890 0x7fe908207bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9080cf740 0x7fe908203f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 --2- 172.21.15.97:0/930345881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9080b2c70 0x7fe9080cd7d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 >> 172.21.15.97:0/930345881 conn(0x7fe9080d3c90 msgr2=0x7fe9080b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 shutdown_connections 2024-09-10T21:24:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.428+0000 7fe90d800640 1 -- 172.21.15.97:0/930345881 wait complete. 2024-09-10T21:24:37.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.5 2024-09-10T21:24:37.459 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-09-10T21:24:37.470 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.1 2024-09-10T21:24:37.471 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.468+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f70580b0160 con 0x7f70580b3040 2024-09-10T21:24:37.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.468+0000 7f7055600640 1 -- 172.21.15.97:0/1896631823 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f70440164d0 con 0x7f70580b3040 2024-09-10T21:24:37.476 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f70380726b0 msgr2=0x7f7038074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f70380726b0 0x7f7038074b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f705816ec90 tx=0x7f704c00b6f0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70580b3040 msgr2=0x7f70580b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70580b3040 0x7f70580b4ad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70440ac000 tx=0x7f7044005790 comp rx=0 tx=0).stop 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 shutdown_connections 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f70380726b0 0x7f7038074b70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7058161d90 0x7f705815df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70580cfd30 0x7f7058147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 --2- 172.21.15.97:0/1896631823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70580b3040 0x7f70580b4ad0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 >> 172.21.15.97:0/1896631823 conn(0x7f70580d1ca0 msgr2=0x7f7058071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 shutdown_connections 2024-09-10T21:24:37.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.472+0000 7f7060600640 1 -- 172.21.15.97:0/1896631823 wait complete. 2024-09-10T21:24:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.496+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f25fc005590 con 0x7f2624143f90 2024-09-10T21:24:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f261a000640 1 -- 172.21.15.97:0/1469491354 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2614061b10 con 0x7f2624143f90 2024-09-10T21:24:37.506 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26040729e0 msgr2=0x7f2604074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26040729e0 0x7f2604074ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2610000c10 tx=0x7f261001b140 comp rx=0 tx=0).stop 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2624143f90 msgr2=0x7f262415a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2624143f90 0x7f262415a890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f261400c390 tx=0x7f261400c850 comp rx=0 tx=0).stop 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 shutdown_connections 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26040729e0 0x7f2604074ea0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2624144690 0x7f26241335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2624143f90 0x7f262415a890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 --2- 172.21.15.97:0/1469491354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262406fd50 0x7f2624159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 >> 172.21.15.97:0/1469491354 conn(0x7f26240b1780 msgr2=0x7f26240b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 shutdown_connections 2024-09-10T21:24:37.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.500+0000 7f2628c00640 1 -- 172.21.15.97:0/1469491354 wait complete. 2024-09-10T21:24:37.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.0 2024-09-10T21:24:37.511 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.512+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f720815e0e0 con 0x7f720815eb20 2024-09-10T21:24:37.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.512+0000 7f7205600640 1 -- 172.21.15.97:0/324497626 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71fc006e20 con 0x7f720815eb20 2024-09-10T21:24:37.520 INFO:teuthology.orchestra.run.smithi097.stdout:55834574858 2024-09-10T21:24:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71e80725e0 msgr2=0x7f71e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71e80725e0 0x7f71e8074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f720815da00 tx=0x7f720000b040 comp rx=0 tx=0).stop 2024-09-10T21:24:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f720815eb20 msgr2=0x7f7208157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f720815eb20 0x7f7208157640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00faa0 tx=0x7f71fc00ff60 comp rx=0 tx=0).stop 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 shutdown_connections 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71e80725e0 0x7f71e8074aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f720815eb20 0x7f7208157640 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720815ac30 0x7f7208157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 --2- 172.21.15.97:0/324497626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f720806cb40 0x7f7208205590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 >> 172.21.15.97:0/324497626 conn(0x7f72080d0be0 msgr2=0x7f72080b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 shutdown_connections 2024-09-10T21:24:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.516+0000 7f7210000640 1 -- 172.21.15.97:0/324497626 wait complete. 2024-09-10T21:24:37.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.3 2024-09-10T21:24:37.529 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.536+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc97c005860 con 0x7fc9a406fd60 2024-09-10T21:24:37.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-09-10T21:24:37.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.536+0000 7fc99a000640 1 -- 172.21.15.97:0/482891078 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc99005e9a0 con 0x7fc9a406fd60 2024-09-10T21:24:37.544 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-09-10T21:24:37.544 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9840725e0 msgr2=0x7fc984074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9840725e0 0x7fc984074aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0af000 tx=0x7fc99c019b40 comp rx=0 tx=0).stop 2024-09-10T21:24:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a406fd60 msgr2=0x7fc9a415c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a406fd60 0x7fc9a415c5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc99000e9f0 tx=0x7fc99000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 shutdown_connections 2024-09-10T21:24:37.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9840725e0 0x7fc984074aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9a4154d10 0x7fc9a41551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9a4161a30 0x7fc9a40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 --2- 172.21.15.97:0/482891078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a406fd60 0x7fc9a415c5d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 >> 172.21.15.97:0/482891078 conn(0x7fc9a40cfb10 msgr2=0x7fc9a40ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 shutdown_connections 2024-09-10T21:24:37.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.540+0000 7fc9a9200640 1 -- 172.21.15.97:0/482891078 wait complete. 2024-09-10T21:24:37.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.548+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f50140ce7b0 con 0x7f501415b5d0 2024-09-10T21:24:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.548+0000 7f500be00640 1 -- 172.21.15.97:0/4051638730 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f500c05f4d0 con 0x7f501415b5d0 2024-09-10T21:24:37.556 INFO:teuthology.orchestra.run.smithi097.stdout:64424509450 2024-09-10T21:24:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ff4076ac0 msgr2=0x7f4ff4078f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ff4076ac0 0x7f4ff4078f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50000af000 tx=0x7f500001b0e0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f501415b5d0 msgr2=0x7f5014152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f501415b5d0 0x7f5014152a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f500c00da10 tx=0x7f500c00ded0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 shutdown_connections 2024-09-10T21:24:37.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ff4076ac0 0x7f4ff4078f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f501415b5d0 0x7f5014152a70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5014073b20 0x7f5014146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 --2- 172.21.15.97:0/4051638730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5014073740 0x7f50141482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 >> 172.21.15.97:0/4051638730 conn(0x7f50140cfb10 msgr2=0x7f501406f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 shutdown_connections 2024-09-10T21:24:37.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.552+0000 7f501aa00640 1 -- 172.21.15.97:0/4051638730 wait complete. 2024-09-10T21:24:37.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.2 2024-09-10T21:24:37.569 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-09-10T21:24:37.579 DEBUG:teuthology.parallel:result is None 2024-09-10T21:24:37.580 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:24:37.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:24:37.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- 172.21.15.97:0/367039779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e80cf5f0 msgr2=0x7f84e806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 --2- 172.21.15.97:0/367039779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e80cf5f0 0x7f84e806f090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84e000aa40 tx=0x7f84e0020400 comp rx=0 tx=0).stop 2024-09-10T21:24:37.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- 172.21.15.97:0/367039779 shutdown_connections 2024-09-10T21:24:37.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 --2- 172.21.15.97:0/367039779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e80cf5f0 0x7f84e806f090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- 172.21.15.97:0/367039779 >> 172.21.15.97:0/367039779 conn(0x7f84e80b1780 msgr2=0x7f84e80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- 172.21.15.97:0/367039779 shutdown_connections 2024-09-10T21:24:37.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- 172.21.15.97:0/367039779 wait complete. 2024-09-10T21:24:37.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 Processor -- start 2024-09-10T21:24:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- start start 2024-09-10T21:24:37.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e806ec40 0x7f84e81f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e80cf5f0 0x7f84e80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e81511f0 0x7f84e80b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84e806e850 con 0x7f84e806ec40 2024-09-10T21:24:37.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84e806e6d0 con 0x7f84e81511f0 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84ee200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f84e806e9d0 con 0x7f84e80cf5f0 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e806ec40 0x7f84e81f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e806ec40 0x7f84e81f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59140/0 (socket says 172.21.15.97:59140) 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 -- 172.21.15.97:0/4157650444 learned_addr learned my addr 172.21.15.97:0/4157650444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e6a00640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e80cf5f0 0x7f84e80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7e00640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e81511f0 0x7f84e80b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 -- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e80cf5f0 msgr2=0x7f84e80aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e80cf5f0 0x7f84e80aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 -- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e81511f0 msgr2=0x7f84e80b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e81511f0 0x7f84e80b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 -- 172.21.15.97:0/4157650444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e80b2280 con 0x7f84e806ec40 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.760+0000 7f84e7400640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e806ec40 0x7f84e81f8c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c790 tx=0x7f84d800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84df400640 1 -- 172.21.15.97:0/4157650444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8003b80 con 0x7f84e806ec40 2024-09-10T21:24:37.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84df400640 1 -- 172.21.15.97:0/4157650444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84d8003d20 con 0x7f84e806ec40 2024-09-10T21:24:37.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84df400640 1 -- 172.21.15.97:0/4157650444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800aa90 con 0x7f84e806ec40 2024-09-10T21:24:37.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e81f5000 con 0x7f84e806ec40 2024-09-10T21:24:37.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81f55c0 con 0x7f84e806ec40 2024-09-10T21:24:37.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e806f090 con 0x7f84e806ec40 2024-09-10T21:24:37.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84df400640 1 -- 172.21.15.97:0/4157650444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f84d800ac30 con 0x7f84e806ec40 2024-09-10T21:24:37.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84df400640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c80725e0 0x7f84c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:37.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84df400640 1 -- 172.21.15.97:0/4157650444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f84d8093400 con 0x7f84e806ec40 2024-09-10T21:24:37.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.764+0000 7f84e6a00640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c80725e0 0x7f84c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:37.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.768+0000 7f84e6a00640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c80725e0 0x7f84c8074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84e815a3a0 tx=0x7f84d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:37.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.768+0000 7f84df400640 1 -- 172.21.15.97:0/4157650444 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f84d805e790 con 0x7f84e806ec40 2024-09-10T21:24:37.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.932+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84e80b02e0 con 0x7f84c80725e0 2024-09-10T21:24:37.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.940+0000 7f84df400640 1 -- 172.21.15.97:0/4157650444 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f84e80b02e0 con 0x7f84c80725e0 2024-09-10T21:24:37.949 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:24:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c80725e0 msgr2=0x7f84c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c80725e0 0x7f84c8074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84e815a3a0 tx=0x7f84d4009040 comp rx=0 tx=0).stop 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e806ec40 msgr2=0x7f84e81f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e806ec40 0x7f84e81f8c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c790 tx=0x7f84d800cc50 comp rx=0 tx=0).stop 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 shutdown_connections 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c80725e0 0x7f84c8074aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e81511f0 0x7f84e80b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e80cf5f0 0x7f84e80aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 --2- 172.21.15.97:0/4157650444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e806ec40 0x7f84e81f8c80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 >> 172.21.15.97:0/4157650444 conn(0x7f84e80b1780 msgr2=0x7f84e815cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 shutdown_connections 2024-09-10T21:24:37.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:37.944+0000 7f84ee200640 1 -- 172.21.15.97:0/4157650444 wait complete. 2024-09-10T21:24:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-10T21:24:36.074564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003427"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264781+0000","last_change":"2024-09-10T21:24:10.717118+0000","last_active":"2024-09-10T21:24:17.264781+0000","last_peered":"2024-09-10T21:24:17.264781+0000","last_clean":"2024-09-10T21:24:17.264781+0000","last_became_active":"2024-09-10T21:24:10.716871+0000","last_became_peered":"2024-09-10T21:24:10.716871+0000","last_unstale":"2024-09-10T21:24:17.264781+0000","last_undegraded":"2024-09-10T21:24:17.264781+0000","last_fullsized":"2024-09-10T21:24:17.264781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:26.347318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264579+0000","last_change":"2024-09-10T21:24:11.758545+0000","last_active":"2024-09-10T21:24:17.264579+0000","last_peered":"2024-09-10T21:24:17.264579+0000","last_clean":"2024-09-10T21:24:17.264579+0000","last_became_active":"2024-09-10T21:24:11.758227+0000","last_became_peered":"2024-09-10T21:24:11.758227+0000","last_unstale":"2024-09-10T21:24:17.264579+0000","last_undegraded":"2024-09-10T21:24:17.264579+0000","last_fullsized":"2024-09-10T21:24:17.264579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768689+0000","last_change":"2024-09-10T21:24:15.784509+0000","last_active":"2024-09-10T21:24:16.768689+0000","last_peered":"2024-09-10T21:24:16.768689+0000","last_clean":"2024-09-10T21:24:16.768689+0000","last_became_active":"2024-09-10T21:24:15.777793+0000","last_became_peered":"2024-09-10T21:24:15.777793+0000","last_unstale":"2024-09-10T21:24:16.768689+0000","last_undegraded":"2024-09-10T21:24:16.768689+0000","last_fullsized":"2024-09-10T21:24:16.768689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:55.568506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264784+0000","last_change":"2024-09-10T21:24:10.717927+0000","last_active":"2024-09-10T21:24:17.264784+0000","last_peered":"2024-09-10T21:24:17.264784+0000","last_clean":"2024-09-10T21:24:17.264784+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:17.264784+0000","last_undegraded":"2024-09-10T21:24:17.264784+0000","last_fullsized":"2024-09-10T21:24:17.264784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265060+0000","last_change":"2024-09-10T21:24:11.750430+0000","last_active":"2024-09-10T21:24:17.265060+0000","last_peered":"2024-09-10T21:24:17.265060+0000","last_clean":"2024-09-10T21:24:17.265060+0000","last_became_active":"2024-09-10T21:24:11.749604+0000","last_became_peered":"2024-09-10T21:24:11.749604+0000","last_unstale":"2024-09-10T21:24:17.265060+0000","last_undegraded":"2024-09-10T21:24:17.265060+0000","last_fullsized":"2024-09-10T21:24:17.265060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262600+0000","last_change":"2024-09-10T21:24:15.775067+0000","last_active":"2024-09-10T21:24:17.262600+0000","last_peered":"2024-09-10T21:24:17.262600+0000","last_clean":"2024-09-10T21:24:17.262600+0000","last_became_active":"2024-09-10T21:24:15.773909+0000","last_became_peered":"2024-09-10T21:24:15.773909+0000","last_unstale":"2024-09-10T21:24:17.262600+0000","last_undegraded":"2024-09-10T21:24:17.262600+0000","last_fullsized":"2024-09-10T21:24:17.262600+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:09.297494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265037+0000","last_change":"2024-09-10T21:24:10.711527+0000","last_active":"2024-09-10T21:24:17.265037+0000","last_peered":"2024-09-10T21:24:17.265037+0000","last_clean":"2024-09-10T21:24:17.265037+0000","last_became_active":"2024-09-10T21:24:10.711276+0000","last_became_peered":"2024-09-10T21:24:10.711276+0000","last_unstale":"2024-09-10T21:24:17.265037+0000","last_undegraded":"2024-09-10T21:24:17.265037+0000","last_fullsized":"2024-09-10T21:24:17.265037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263749+0000","last_change":"2024-09-10T21:24:11.756415+0000","last_active":"2024-09-10T21:24:17.263749+0000","last_peered":"2024-09-10T21:24:17.263749+0000","last_clean":"2024-09-10T21:24:17.263749+0000","last_became_active":"2024-09-10T21:24:11.754620+0000","last_became_peered":"2024-09-10T21:24:11.754620+0000","last_unstale":"2024-09-10T21:24:17.263749+0000","last_undegraded":"2024-09-10T21:24:17.263749+0000","last_fullsized":"2024-09-10T21:24:17.263749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:48.122962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262734+0000","last_change":"2024-09-10T21:24:15.782482+0000","last_active":"2024-09-10T21:24:17.262734+0000","last_peered":"2024-09-10T21:24:17.262734+0000","last_clean":"2024-09-10T21:24:17.262734+0000","last_became_active":"2024-09-10T21:24:15.782107+0000","last_became_peered":"2024-09-10T21:24:15.782107+0000","last_unstale":"2024-09-10T21:24:17.262734+0000","last_undegraded":"2024-09-10T21:24:17.262734+0000","last_fullsized":"2024-09-10T21:24:17.262734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.906721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264925+0000","last_change":"2024-09-10T21:24:10.716898+0000","last_active":"2024-09-10T21:24:17.264925+0000","last_peered":"2024-09-10T21:24:17.264925+0000","last_clean":"2024-09-10T21:24:17.264925+0000","last_became_active":"2024-09-10T21:24:10.716644+0000","last_became_peered":"2024-09-10T21:24:10.716644+0000","last_unstale":"2024-09-10T21:24:17.264925+0000","last_undegraded":"2024-09-10T21:24:17.264925+0000","last_fullsized":"2024-09-10T21:24:17.264925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:41.682216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766338+0000","last_change":"2024-09-10T21:24:11.747195+0000","last_active":"2024-09-10T21:24:16.766338+0000","last_peered":"2024-09-10T21:24:16.766338+0000","last_clean":"2024-09-10T21:24:16.766338+0000","last_became_active":"2024-09-10T21:24:11.747038+0000","last_became_peered":"2024-09-10T21:24:11.747038+0000","last_unstale":"2024-09-10T21:24:16.766338+0000","last_undegraded":"2024-09-10T21:24:16.766338+0000","last_fullsized":"2024-09-10T21:24:16.766338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262860+0000","last_change":"2024-09-10T21:24:15.782490+0000","last_active":"2024-09-10T21:24:17.262860+0000","last_peered":"2024-09-10T21:24:17.262860+0000","last_clean":"2024-09-10T21:24:17.262860+0000","last_became_active":"2024-09-10T21:24:15.782079+0000","last_became_peered":"2024-09-10T21:24:15.782079+0000","last_unstale":"2024-09-10T21:24:17.262860+0000","last_undegraded":"2024-09-10T21:24:17.262860+0000","last_fullsized":"2024-09-10T21:24:17.262860+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:42.047447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765875+0000","last_change":"2024-09-10T21:24:11.734798+0000","last_active":"2024-09-10T21:24:16.765875+0000","last_peered":"2024-09-10T21:24:16.765875+0000","last_clean":"2024-09-10T21:24:16.765875+0000","last_became_active":"2024-09-10T21:24:11.734495+0000","last_became_peered":"2024-09-10T21:24:11.734495+0000","last_unstale":"2024-09-10T21:24:16.765875+0000","last_undegraded":"2024-09-10T21:24:16.765875+0000","last_fullsized":"2024-09-10T21:24:16.765875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:08.842495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766019+0000","last_change":"2024-09-10T21:24:10.718178+0000","last_active":"2024-09-10T21:24:16.766019+0000","last_peered":"2024-09-10T21:24:16.766019+0000","last_clean":"2024-09-10T21:24:16.766019+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:16.766019+0000","last_undegraded":"2024-09-10T21:24:16.766019+0000","last_fullsized":"2024-09-10T21:24:16.766019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:01.094807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766145+0000","last_change":"2024-09-10T21:24:15.786334+0000","last_active":"2024-09-10T21:24:16.766145+0000","last_peered":"2024-09-10T21:24:16.766145+0000","last_clean":"2024-09-10T21:24:16.766145+0000","last_became_active":"2024-09-10T21:24:15.785888+0000","last_became_peered":"2024-09-10T21:24:15.785888+0000","last_unstale":"2024-09-10T21:24:16.766145+0000","last_undegraded":"2024-09-10T21:24:16.766145+0000","last_fullsized":"2024-09-10T21:24:16.766145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:27.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769104+0000","last_change":"2024-09-10T21:24:11.743760+0000","last_active":"2024-09-10T21:24:16.769104+0000","last_peered":"2024-09-10T21:24:16.769104+0000","last_clean":"2024-09-10T21:24:16.769104+0000","last_became_active":"2024-09-10T21:24:11.743459+0000","last_became_peered":"2024-09-10T21:24:11.743459+0000","last_unstale":"2024-09-10T21:24:16.769104+0000","last_undegraded":"2024-09-10T21:24:16.769104+0000","last_fullsized":"2024-09-10T21:24:16.769104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:53.958469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770545+0000","last_change":"2024-09-10T21:24:10.719093+0000","last_active":"2024-09-10T21:24:16.770545+0000","last_peered":"2024-09-10T21:24:16.770545+0000","last_clean":"2024-09-10T21:24:16.770545+0000","last_became_active":"2024-09-10T21:24:10.715489+0000","last_became_peered":"2024-09-10T21:24:10.715489+0000","last_unstale":"2024-09-10T21:24:16.770545+0000","last_undegraded":"2024-09-10T21:24:16.770545+0000","last_fullsized":"2024-09-10T21:24:16.770545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264532+0000","last_change":"2024-09-10T21:24:15.777643+0000","last_active":"2024-09-10T21:24:17.264532+0000","last_peered":"2024-09-10T21:24:17.264532+0000","last_clean":"2024-09-10T21:24:17.264532+0000","last_became_active":"2024-09-10T21:24:15.777271+0000","last_became_peered":"2024-09-10T21:24:15.777271+0000","last_unstale":"2024-09-10T21:24:17.264532+0000","last_undegraded":"2024-09-10T21:24:17.264532+0000","last_fullsized":"2024-09-10T21:24:17.264532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:26.509163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263743+0000","last_change":"2024-09-10T21:24:11.724275+0000","last_active":"2024-09-10T21:24:17.263743+0000","last_peered":"2024-09-10T21:24:17.263743+0000","last_clean":"2024-09-10T21:24:17.263743+0000","last_became_active":"2024-09-10T21:24:11.723982+0000","last_became_peered":"2024-09-10T21:24:11.723982+0000","last_unstale":"2024-09-10T21:24:17.263743+0000","last_undegraded":"2024-09-10T21:24:17.263743+0000","last_fullsized":"2024-09-10T21:24:17.263743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:28.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264861+0000","last_change":"2024-09-10T21:24:10.712213+0000","last_active":"2024-09-10T21:24:17.264861+0000","last_peered":"2024-09-10T21:24:17.264861+0000","last_clean":"2024-09-10T21:24:17.264861+0000","last_became_active":"2024-09-10T21:24:10.711951+0000","last_became_peered":"2024-09-10T21:24:10.711951+0000","last_unstale":"2024-09-10T21:24:17.264861+0000","last_undegraded":"2024-09-10T21:24:17.264861+0000","last_fullsized":"2024-09-10T21:24:17.264861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:54.891884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770275+0000","last_change":"2024-09-10T21:24:15.780288+0000","last_active":"2024-09-10T21:24:16.770275+0000","last_peered":"2024-09-10T21:24:16.770275+0000","last_clean":"2024-09-10T21:24:16.770275+0000","last_became_active":"2024-09-10T21:24:15.778928+0000","last_became_peered":"2024-09-10T21:24:15.778928+0000","last_unstale":"2024-09-10T21:24:16.770275+0000","last_undegraded":"2024-09-10T21:24:16.770275+0000","last_fullsized":"2024-09-10T21:24:16.770275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:45.367503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765552+0000","last_change":"2024-09-10T21:24:11.730800+0000","last_active":"2024-09-10T21:24:16.765552+0000","last_peered":"2024-09-10T21:24:16.765552+0000","last_clean":"2024-09-10T21:24:16.765552+0000","last_became_active":"2024-09-10T21:24:11.730474+0000","last_became_peered":"2024-09-10T21:24:11.730474+0000","last_unstale":"2024-09-10T21:24:16.765552+0000","last_undegraded":"2024-09-10T21:24:16.765552+0000","last_fullsized":"2024-09-10T21:24:16.765552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263542+0000","last_change":"2024-09-10T21:24:10.712139+0000","last_active":"2024-09-10T21:24:17.263542+0000","last_peered":"2024-09-10T21:24:17.263542+0000","last_clean":"2024-09-10T21:24:17.263542+0000","last_became_active":"2024-09-10T21:24:10.710501+0000","last_became_peered":"2024-09-10T21:24:10.710501+0000","last_unstale":"2024-09-10T21:24:17.263542+0000","last_undegraded":"2024-09-10T21:24:17.263542+0000","last_fullsized":"2024-09-10T21:24:17.263542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263764+0000","last_change":"2024-09-10T21:24:15.770139+0000","last_active":"2024-09-10T21:24:17.263764+0000","last_peered":"2024-09-10T21:24:17.263764+0000","last_clean":"2024-09-10T21:24:17.263764+0000","last_became_active":"2024-09-10T21:24:15.769554+0000","last_became_peered":"2024-09-10T21:24:15.769554+0000","last_unstale":"2024-09-10T21:24:17.263764+0000","last_undegraded":"2024-09-10T21:24:17.263764+0000","last_fullsized":"2024-09-10T21:24:17.263764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:14.866707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266054+0000","last_change":"2024-09-10T21:24:11.732319+0000","last_active":"2024-09-10T21:24:17.266054+0000","last_peered":"2024-09-10T21:24:17.266054+0000","last_clean":"2024-09-10T21:24:17.266054+0000","last_became_active":"2024-09-10T21:24:11.732038+0000","last_became_peered":"2024-09-10T21:24:11.732038+0000","last_unstale":"2024-09-10T21:24:17.266054+0000","last_undegraded":"2024-09-10T21:24:17.266054+0000","last_fullsized":"2024-09-10T21:24:17.266054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:34.760961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766655+0000","last_change":"2024-09-10T21:24:10.720607+0000","last_active":"2024-09-10T21:24:16.766655+0000","last_peered":"2024-09-10T21:24:16.766655+0000","last_clean":"2024-09-10T21:24:16.766655+0000","last_became_active":"2024-09-10T21:24:10.720144+0000","last_became_peered":"2024-09-10T21:24:10.720144+0000","last_unstale":"2024-09-10T21:24:16.766655+0000","last_undegraded":"2024-09-10T21:24:16.766655+0000","last_fullsized":"2024-09-10T21:24:16.766655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:10.162599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263165+0000","last_change":"2024-09-10T21:24:15.777793+0000","last_active":"2024-09-10T21:24:17.263165+0000","last_peered":"2024-09-10T21:24:17.263165+0000","last_clean":"2024-09-10T21:24:17.263165+0000","last_became_active":"2024-09-10T21:24:15.777358+0000","last_became_peered":"2024-09-10T21:24:15.777358+0000","last_unstale":"2024-09-10T21:24:17.263165+0000","last_undegraded":"2024-09-10T21:24:17.263165+0000","last_fullsized":"2024-09-10T21:24:17.263165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:48.503713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264581+0000","last_change":"2024-09-10T21:24:11.735858+0000","last_active":"2024-09-10T21:24:17.264581+0000","last_peered":"2024-09-10T21:24:17.264581+0000","last_clean":"2024-09-10T21:24:17.264581+0000","last_became_active":"2024-09-10T21:24:11.735599+0000","last_became_peered":"2024-09-10T21:24:11.735599+0000","last_unstale":"2024-09-10T21:24:17.264581+0000","last_undegraded":"2024-09-10T21:24:17.264581+0000","last_fullsized":"2024-09-10T21:24:17.264581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:53.530511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266208+0000","last_change":"2024-09-10T21:24:10.712710+0000","last_active":"2024-09-10T21:24:17.266208+0000","last_peered":"2024-09-10T21:24:17.266208+0000","last_clean":"2024-09-10T21:24:17.266208+0000","last_became_active":"2024-09-10T21:24:10.712400+0000","last_became_peered":"2024-09-10T21:24:10.712400+0000","last_unstale":"2024-09-10T21:24:17.266208+0000","last_undegraded":"2024-09-10T21:24:17.266208+0000","last_fullsized":"2024-09-10T21:24:17.266208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:57.508268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263912+0000","last_change":"2024-09-10T21:24:15.776045+0000","last_active":"2024-09-10T21:24:17.263912+0000","last_peered":"2024-09-10T21:24:17.263912+0000","last_clean":"2024-09-10T21:24:17.263912+0000","last_became_active":"2024-09-10T21:24:15.775199+0000","last_became_peered":"2024-09-10T21:24:15.775199+0000","last_unstale":"2024-09-10T21:24:17.263912+0000","last_undegraded":"2024-09-10T21:24:17.263912+0000","last_fullsized":"2024-09-10T21:24:17.263912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.516796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770569+0000","last_change":"2024-09-10T21:24:11.737947+0000","last_active":"2024-09-10T21:24:16.770569+0000","last_peered":"2024-09-10T21:24:16.770569+0000","last_clean":"2024-09-10T21:24:16.770569+0000","last_became_active":"2024-09-10T21:24:11.735572+0000","last_became_peered":"2024-09-10T21:24:11.735572+0000","last_unstale":"2024-09-10T21:24:16.770569+0000","last_undegraded":"2024-09-10T21:24:16.770569+0000","last_fullsized":"2024-09-10T21:24:16.770569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:41.353969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766677+0000","last_change":"2024-09-10T21:24:10.712429+0000","last_active":"2024-09-10T21:24:16.766677+0000","last_peered":"2024-09-10T21:24:16.766677+0000","last_clean":"2024-09-10T21:24:16.766677+0000","last_became_active":"2024-09-10T21:24:10.712150+0000","last_became_peered":"2024-09-10T21:24:10.712150+0000","last_unstale":"2024-09-10T21:24:16.766677+0000","last_undegraded":"2024-09-10T21:24:16.766677+0000","last_fullsized":"2024-09-10T21:24:16.766677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:37.565989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769802+0000","last_change":"2024-09-10T21:24:15.778516+0000","last_active":"2024-09-10T21:24:16.769802+0000","last_peered":"2024-09-10T21:24:16.769802+0000","last_clean":"2024-09-10T21:24:16.769802+0000","last_became_active":"2024-09-10T21:24:15.778061+0000","last_became_peered":"2024-09-10T21:24:15.778061+0000","last_unstale":"2024-09-10T21:24:16.769802+0000","last_undegraded":"2024-09-10T21:24:16.769802+0000","last_fullsized":"2024-09-10T21:24:16.769802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:38.186163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263379+0000","last_change":"2024-09-10T21:24:11.747693+0000","last_active":"2024-09-10T21:24:17.263379+0000","last_peered":"2024-09-10T21:24:17.263379+0000","last_clean":"2024-09-10T21:24:17.263379+0000","last_became_active":"2024-09-10T21:24:11.747405+0000","last_became_peered":"2024-09-10T21:24:11.747405+0000","last_unstale":"2024-09-10T21:24:17.263379+0000","last_undegraded":"2024-09-10T21:24:17.263379+0000","last_fullsized":"2024-09-10T21:24:17.263379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:42.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264255+0000","last_change":"2024-09-10T21:24:10.711258+0000","last_active":"2024-09-10T21:24:17.264255+0000","last_peered":"2024-09-10T21:24:17.264255+0000","last_clean":"2024-09-10T21:24:17.264255+0000","last_became_active":"2024-09-10T21:24:10.710892+0000","last_became_peered":"2024-09-10T21:24:10.710892+0000","last_unstale":"2024-09-10T21:24:17.264255+0000","last_undegraded":"2024-09-10T21:24:17.264255+0000","last_fullsized":"2024-09-10T21:24:17.264255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769471+0000","last_change":"2024-09-10T21:24:15.786836+0000","last_active":"2024-09-10T21:24:16.769471+0000","last_peered":"2024-09-10T21:24:16.769471+0000","last_clean":"2024-09-10T21:24:16.769471+0000","last_became_active":"2024-09-10T21:24:15.786459+0000","last_became_peered":"2024-09-10T21:24:15.786459+0000","last_unstale":"2024-09-10T21:24:16.769471+0000","last_undegraded":"2024-09-10T21:24:16.769471+0000","last_fullsized":"2024-09-10T21:24:16.769471+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:49.037577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264516+0000","last_change":"2024-09-10T21:24:11.756779+0000","last_active":"2024-09-10T21:24:17.264516+0000","last_peered":"2024-09-10T21:24:17.264516+0000","last_clean":"2024-09-10T21:24:17.264516+0000","last_became_active":"2024-09-10T21:24:11.756493+0000","last_became_peered":"2024-09-10T21:24:11.756493+0000","last_unstale":"2024-09-10T21:24:17.264516+0000","last_undegraded":"2024-09-10T21:24:17.264516+0000","last_fullsized":"2024-09-10T21:24:17.264516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766053+0000","last_change":"2024-09-10T21:24:10.707060+0000","last_active":"2024-09-10T21:24:16.766053+0000","last_peered":"2024-09-10T21:24:16.766053+0000","last_clean":"2024-09-10T21:24:16.766053+0000","last_became_active":"2024-09-10T21:24:10.706798+0000","last_became_peered":"2024-09-10T21:24:10.706798+0000","last_unstale":"2024-09-10T21:24:16.766053+0000","last_undegraded":"2024-09-10T21:24:16.766053+0000","last_fullsized":"2024-09-10T21:24:16.766053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:47.120434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770236+0000","last_change":"2024-09-10T21:24:15.781040+0000","last_active":"2024-09-10T21:24:16.770236+0000","last_peered":"2024-09-10T21:24:16.770236+0000","last_clean":"2024-09-10T21:24:16.770236+0000","last_became_active":"2024-09-10T21:24:15.780677+0000","last_became_peered":"2024-09-10T21:24:15.780677+0000","last_unstale":"2024-09-10T21:24:16.770236+0000","last_undegraded":"2024-09-10T21:24:16.770236+0000","last_fullsized":"2024-09-10T21:24:16.770236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.899193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263779+0000","last_change":"2024-09-10T21:24:11.753935+0000","last_active":"2024-09-10T21:24:17.263779+0000","last_peered":"2024-09-10T21:24:17.263779+0000","last_clean":"2024-09-10T21:24:17.263779+0000","last_became_active":"2024-09-10T21:24:11.753134+0000","last_became_peered":"2024-09-10T21:24:11.753134+0000","last_unstale":"2024-09-10T21:24:17.263779+0000","last_undegraded":"2024-09-10T21:24:17.263779+0000","last_fullsized":"2024-09-10T21:24:17.263779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:41.306076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769295+0000","last_change":"2024-09-10T21:24:10.718908+0000","last_active":"2024-09-10T21:24:16.769295+0000","last_peered":"2024-09-10T21:24:16.769295+0000","last_clean":"2024-09-10T21:24:16.769295+0000","last_became_active":"2024-09-10T21:24:10.718142+0000","last_became_peered":"2024-09-10T21:24:10.718142+0000","last_unstale":"2024-09-10T21:24:16.769295+0000","last_undegraded":"2024-09-10T21:24:16.769295+0000","last_fullsized":"2024-09-10T21:24:16.769295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:56.946441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769872+0000","last_change":"2024-09-10T21:24:15.778921+0000","last_active":"2024-09-10T21:24:16.769872+0000","last_peered":"2024-09-10T21:24:16.769872+0000","last_clean":"2024-09-10T21:24:16.769872+0000","last_became_active":"2024-09-10T21:24:15.778360+0000","last_became_peered":"2024-09-10T21:24:15.778360+0000","last_unstale":"2024-09-10T21:24:16.769872+0000","last_undegraded":"2024-09-10T21:24:16.769872+0000","last_fullsized":"2024-09-10T21:24:16.769872+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:10.163639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768895+0000","last_change":"2024-09-10T21:24:11.758622+0000","last_active":"2024-09-10T21:24:16.768895+0000","last_peered":"2024-09-10T21:24:16.768895+0000","last_clean":"2024-09-10T21:24:16.768895+0000","last_became_active":"2024-09-10T21:24:11.758363+0000","last_became_peered":"2024-09-10T21:24:11.758363+0000","last_unstale":"2024-09-10T21:24:16.768895+0000","last_undegraded":"2024-09-10T21:24:16.768895+0000","last_fullsized":"2024-09-10T21:24:16.768895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765739+0000","last_change":"2024-09-10T21:24:10.720746+0000","last_active":"2024-09-10T21:24:16.765739+0000","last_peered":"2024-09-10T21:24:16.765739+0000","last_clean":"2024-09-10T21:24:16.765739+0000","last_became_active":"2024-09-10T21:24:10.720360+0000","last_became_peered":"2024-09-10T21:24:10.720360+0000","last_unstale":"2024-09-10T21:24:16.765739+0000","last_undegraded":"2024-09-10T21:24:16.765739+0000","last_fullsized":"2024-09-10T21:24:16.765739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:25.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264322+0000","last_change":"2024-09-10T21:24:15.787994+0000","last_active":"2024-09-10T21:24:17.264322+0000","last_peered":"2024-09-10T21:24:17.264322+0000","last_clean":"2024-09-10T21:24:17.264322+0000","last_became_active":"2024-09-10T21:24:15.787532+0000","last_became_peered":"2024-09-10T21:24:15.787532+0000","last_unstale":"2024-09-10T21:24:17.264322+0000","last_undegraded":"2024-09-10T21:24:17.264322+0000","last_fullsized":"2024-09-10T21:24:17.264322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:12.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265574+0000","last_change":"2024-09-10T21:24:11.747694+0000","last_active":"2024-09-10T21:24:17.265574+0000","last_peered":"2024-09-10T21:24:17.265574+0000","last_clean":"2024-09-10T21:24:17.265574+0000","last_became_active":"2024-09-10T21:24:11.746819+0000","last_became_peered":"2024-09-10T21:24:11.746819+0000","last_unstale":"2024-09-10T21:24:17.265574+0000","last_undegraded":"2024-09-10T21:24:17.265574+0000","last_fullsized":"2024-09-10T21:24:17.265574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:14.637932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766139+0000","last_change":"2024-09-10T21:24:10.708595+0000","last_active":"2024-09-10T21:24:16.766139+0000","last_peered":"2024-09-10T21:24:16.766139+0000","last_clean":"2024-09-10T21:24:16.766139+0000","last_became_active":"2024-09-10T21:24:10.708315+0000","last_became_peered":"2024-09-10T21:24:10.708315+0000","last_unstale":"2024-09-10T21:24:16.766139+0000","last_undegraded":"2024-09-10T21:24:16.766139+0000","last_fullsized":"2024-09-10T21:24:16.766139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:39.966992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262962+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:24:17.262962+0000","last_peered":"2024-09-10T21:24:17.262962+0000","last_clean":"2024-09-10T21:24:17.262962+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:24:17.262962+0000","last_undegraded":"2024-09-10T21:24:17.262962+0000","last_fullsized":"2024-09-10T21:24:17.262962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766659+0000","last_change":"2024-09-10T21:24:15.771747+0000","last_active":"2024-09-10T21:24:16.766659+0000","last_peered":"2024-09-10T21:24:16.766659+0000","last_clean":"2024-09-10T21:24:16.766659+0000","last_became_active":"2024-09-10T21:24:15.771197+0000","last_became_peered":"2024-09-10T21:24:15.771197+0000","last_unstale":"2024-09-10T21:24:16.766659+0000","last_undegraded":"2024-09-10T21:24:16.766659+0000","last_fullsized":"2024-09-10T21:24:16.766659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:05.770255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766468+0000","last_change":"2024-09-10T21:24:11.740573+0000","last_active":"2024-09-10T21:24:16.766468+0000","last_peered":"2024-09-10T21:24:16.766468+0000","last_clean":"2024-09-10T21:24:16.766468+0000","last_became_active":"2024-09-10T21:24:11.738994+0000","last_became_peered":"2024-09-10T21:24:11.738994+0000","last_unstale":"2024-09-10T21:24:16.766468+0000","last_undegraded":"2024-09-10T21:24:16.766468+0000","last_fullsized":"2024-09-10T21:24:16.766468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:55.184362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770636+0000","last_change":"2024-09-10T21:24:10.721767+0000","last_active":"2024-09-10T21:24:16.770636+0000","last_peered":"2024-09-10T21:24:16.770636+0000","last_clean":"2024-09-10T21:24:16.770636+0000","last_became_active":"2024-09-10T21:24:10.721444+0000","last_became_peered":"2024-09-10T21:24:10.721444+0000","last_unstale":"2024-09-10T21:24:16.770636+0000","last_undegraded":"2024-09-10T21:24:16.770636+0000","last_fullsized":"2024-09-10T21:24:16.770636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:38.215394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263060+0000","last_change":"2024-09-10T21:24:15.770594+0000","last_active":"2024-09-10T21:24:17.263060+0000","last_peered":"2024-09-10T21:24:17.263060+0000","last_clean":"2024-09-10T21:24:17.263060+0000","last_became_active":"2024-09-10T21:24:15.769704+0000","last_became_peered":"2024-09-10T21:24:15.769704+0000","last_unstale":"2024-09-10T21:24:17.263060+0000","last_undegraded":"2024-09-10T21:24:17.263060+0000","last_fullsized":"2024-09-10T21:24:17.263060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:56.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264863+0000","last_change":"2024-09-10T21:24:11.743773+0000","last_active":"2024-09-10T21:24:17.264863+0000","last_peered":"2024-09-10T21:24:17.264863+0000","last_clean":"2024-09-10T21:24:17.264863+0000","last_became_active":"2024-09-10T21:24:11.743497+0000","last_became_peered":"2024-09-10T21:24:11.743497+0000","last_unstale":"2024-09-10T21:24:17.264863+0000","last_undegraded":"2024-09-10T21:24:17.264863+0000","last_fullsized":"2024-09-10T21:24:17.264863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265338+0000","last_change":"2024-09-10T21:24:10.716493+0000","last_active":"2024-09-10T21:24:17.265338+0000","last_peered":"2024-09-10T21:24:17.265338+0000","last_clean":"2024-09-10T21:24:17.265338+0000","last_became_active":"2024-09-10T21:24:10.716126+0000","last_became_peered":"2024-09-10T21:24:10.716126+0000","last_unstale":"2024-09-10T21:24:17.265338+0000","last_undegraded":"2024-09-10T21:24:17.265338+0000","last_fullsized":"2024-09-10T21:24:17.265338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264298+0000","last_change":"2024-09-10T21:24:15.791234+0000","last_active":"2024-09-10T21:24:17.264298+0000","last_peered":"2024-09-10T21:24:17.264298+0000","last_clean":"2024-09-10T21:24:17.264298+0000","last_became_active":"2024-09-10T21:24:15.790862+0000","last_became_peered":"2024-09-10T21:24:15.790862+0000","last_unstale":"2024-09-10T21:24:17.264298+0000","last_undegraded":"2024-09-10T21:24:17.264298+0000","last_fullsized":"2024-09-10T21:24:17.264298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:41.522844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265147+0000","last_change":"2024-09-10T21:24:11.742947+0000","last_active":"2024-09-10T21:24:17.265147+0000","last_peered":"2024-09-10T21:24:17.265147+0000","last_clean":"2024-09-10T21:24:17.265147+0000","last_became_active":"2024-09-10T21:24:11.741967+0000","last_became_peered":"2024-09-10T21:24:11.741967+0000","last_unstale":"2024-09-10T21:24:17.265147+0000","last_undegraded":"2024-09-10T21:24:17.265147+0000","last_fullsized":"2024-09-10T21:24:17.265147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266168+0000","last_change":"2024-09-10T21:24:10.715721+0000","last_active":"2024-09-10T21:24:17.266168+0000","last_peered":"2024-09-10T21:24:17.266168+0000","last_clean":"2024-09-10T21:24:17.266168+0000","last_became_active":"2024-09-10T21:24:10.715385+0000","last_became_peered":"2024-09-10T21:24:10.715385+0000","last_unstale":"2024-09-10T21:24:17.266168+0000","last_undegraded":"2024-09-10T21:24:17.266168+0000","last_fullsized":"2024-09-10T21:24:17.266168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:30.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765435+0000","last_change":"2024-09-10T21:24:15.770313+0000","last_active":"2024-09-10T21:24:16.765435+0000","last_peered":"2024-09-10T21:24:16.765435+0000","last_clean":"2024-09-10T21:24:16.765435+0000","last_became_active":"2024-09-10T21:24:15.769934+0000","last_became_peered":"2024-09-10T21:24:15.769934+0000","last_unstale":"2024-09-10T21:24:16.765435+0000","last_undegraded":"2024-09-10T21:24:16.765435+0000","last_fullsized":"2024-09-10T21:24:16.765435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:58.034220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264104+0000","last_change":"2024-09-10T21:24:11.752428+0000","last_active":"2024-09-10T21:24:17.264104+0000","last_peered":"2024-09-10T21:24:17.264104+0000","last_clean":"2024-09-10T21:24:17.264104+0000","last_became_active":"2024-09-10T21:24:11.748165+0000","last_became_peered":"2024-09-10T21:24:11.748165+0000","last_unstale":"2024-09-10T21:24:17.264104+0000","last_undegraded":"2024-09-10T21:24:17.264104+0000","last_fullsized":"2024-09-10T21:24:17.264104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265190+0000","last_change":"2024-09-10T21:24:10.717653+0000","last_active":"2024-09-10T21:24:17.265190+0000","last_peered":"2024-09-10T21:24:17.265190+0000","last_clean":"2024-09-10T21:24:17.265190+0000","last_became_active":"2024-09-10T21:24:10.715137+0000","last_became_peered":"2024-09-10T21:24:10.715137+0000","last_unstale":"2024-09-10T21:24:17.265190+0000","last_undegraded":"2024-09-10T21:24:17.265190+0000","last_fullsized":"2024-09-10T21:24:17.265190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:01.438989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262702+0000","last_change":"2024-09-10T21:24:15.782725+0000","last_active":"2024-09-10T21:24:17.262702+0000","last_peered":"2024-09-10T21:24:17.262702+0000","last_clean":"2024-09-10T21:24:17.262702+0000","last_became_active":"2024-09-10T21:24:15.782199+0000","last_became_peered":"2024-09-10T21:24:15.782199+0000","last_unstale":"2024-09-10T21:24:17.262702+0000","last_undegraded":"2024-09-10T21:24:17.262702+0000","last_fullsized":"2024-09-10T21:24:17.262702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:15.226555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765554+0000","last_change":"2024-09-10T21:24:11.757400+0000","last_active":"2024-09-10T21:24:16.765554+0000","last_peered":"2024-09-10T21:24:16.765554+0000","last_clean":"2024-09-10T21:24:16.765554+0000","last_became_active":"2024-09-10T21:24:11.757112+0000","last_became_peered":"2024-09-10T21:24:11.757112+0000","last_unstale":"2024-09-10T21:24:16.765554+0000","last_undegraded":"2024-09-10T21:24:16.765554+0000","last_fullsized":"2024-09-10T21:24:16.765554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263864+0000","last_change":"2024-09-10T21:24:10.701950+0000","last_active":"2024-09-10T21:24:17.263864+0000","last_peered":"2024-09-10T21:24:17.263864+0000","last_clean":"2024-09-10T21:24:17.263864+0000","last_became_active":"2024-09-10T21:24:10.701652+0000","last_became_peered":"2024-09-10T21:24:10.701652+0000","last_unstale":"2024-09-10T21:24:17.263864+0000","last_undegraded":"2024-09-10T21:24:17.263864+0000","last_fullsized":"2024-09-10T21:24:17.263864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.595699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266111+0000","last_change":"2024-09-10T21:24:15.784288+0000","last_active":"2024-09-10T21:24:17.266111+0000","last_peered":"2024-09-10T21:24:17.266111+0000","last_clean":"2024-09-10T21:24:17.266111+0000","last_became_active":"2024-09-10T21:24:15.783890+0000","last_became_peered":"2024-09-10T21:24:15.783890+0000","last_unstale":"2024-09-10T21:24:17.266111+0000","last_undegraded":"2024-09-10T21:24:17.266111+0000","last_fullsized":"2024-09-10T21:24:17.266111+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:59.245790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262943+0000","last_change":"2024-09-10T21:24:11.736914+0000","last_active":"2024-09-10T21:24:17.262943+0000","last_peered":"2024-09-10T21:24:17.262943+0000","last_clean":"2024-09-10T21:24:17.262943+0000","last_became_active":"2024-09-10T21:24:11.734099+0000","last_became_peered":"2024-09-10T21:24:11.734099+0000","last_unstale":"2024-09-10T21:24:17.262943+0000","last_undegraded":"2024-09-10T21:24:17.262943+0000","last_fullsized":"2024-09-10T21:24:17.262943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264000+0000","last_change":"2024-09-10T21:24:10.715530+0000","last_active":"2024-09-10T21:24:17.264000+0000","last_peered":"2024-09-10T21:24:17.264000+0000","last_clean":"2024-09-10T21:24:17.264000+0000","last_became_active":"2024-09-10T21:24:10.715235+0000","last_became_peered":"2024-09-10T21:24:10.715235+0000","last_unstale":"2024-09-10T21:24:17.264000+0000","last_undegraded":"2024-09-10T21:24:17.264000+0000","last_fullsized":"2024-09-10T21:24:17.264000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:12.286087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264900+0000","last_change":"2024-09-10T21:24:15.784572+0000","last_active":"2024-09-10T21:24:17.264900+0000","last_peered":"2024-09-10T21:24:17.264900+0000","last_clean":"2024-09-10T21:24:17.264900+0000","last_became_active":"2024-09-10T21:24:15.784136+0000","last_became_peered":"2024-09-10T21:24:15.784136+0000","last_unstale":"2024-09-10T21:24:17.264900+0000","last_undegraded":"2024-09-10T21:24:17.264900+0000","last_fullsized":"2024-09-10T21:24:17.264900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:12.320754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765877+0000","last_change":"2024-09-10T21:24:11.755458+0000","last_active":"2024-09-10T21:24:16.765877+0000","last_peered":"2024-09-10T21:24:16.765877+0000","last_clean":"2024-09-10T21:24:16.765877+0000","last_became_active":"2024-09-10T21:24:11.755150+0000","last_became_peered":"2024-09-10T21:24:11.755150+0000","last_unstale":"2024-09-10T21:24:16.765877+0000","last_undegraded":"2024-09-10T21:24:16.765877+0000","last_fullsized":"2024-09-10T21:24:16.765877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:34.505560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765994+0000","last_change":"2024-09-10T21:24:10.717709+0000","last_active":"2024-09-10T21:24:16.765994+0000","last_peered":"2024-09-10T21:24:16.765994+0000","last_clean":"2024-09-10T21:24:16.765994+0000","last_became_active":"2024-09-10T21:24:10.717328+0000","last_became_peered":"2024-09-10T21:24:10.717328+0000","last_unstale":"2024-09-10T21:24:16.765994+0000","last_undegraded":"2024-09-10T21:24:16.765994+0000","last_fullsized":"2024-09-10T21:24:16.765994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:37.792691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770443+0000","last_change":"2024-09-10T21:24:15.788432+0000","last_active":"2024-09-10T21:24:16.770443+0000","last_peered":"2024-09-10T21:24:16.770443+0000","last_clean":"2024-09-10T21:24:16.770443+0000","last_became_active":"2024-09-10T21:24:15.788046+0000","last_became_peered":"2024-09-10T21:24:15.788046+0000","last_unstale":"2024-09-10T21:24:16.770443+0000","last_undegraded":"2024-09-10T21:24:16.770443+0000","last_fullsized":"2024-09-10T21:24:16.770443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:04.277488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769322+0000","last_change":"2024-09-10T21:24:11.737027+0000","last_active":"2024-09-10T21:24:16.769322+0000","last_peered":"2024-09-10T21:24:16.769322+0000","last_clean":"2024-09-10T21:24:16.769322+0000","last_became_active":"2024-09-10T21:24:11.736731+0000","last_became_peered":"2024-09-10T21:24:11.736731+0000","last_unstale":"2024-09-10T21:24:16.769322+0000","last_undegraded":"2024-09-10T21:24:16.769322+0000","last_fullsized":"2024-09-10T21:24:16.769322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769804+0000","last_change":"2024-09-10T21:24:10.712131+0000","last_active":"2024-09-10T21:24:16.769804+0000","last_peered":"2024-09-10T21:24:16.769804+0000","last_clean":"2024-09-10T21:24:16.769804+0000","last_became_active":"2024-09-10T21:24:10.711804+0000","last_became_peered":"2024-09-10T21:24:10.711804+0000","last_unstale":"2024-09-10T21:24:16.769804+0000","last_undegraded":"2024-09-10T21:24:16.769804+0000","last_fullsized":"2024-09-10T21:24:16.769804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766399+0000","last_change":"2024-09-10T21:24:15.786086+0000","last_active":"2024-09-10T21:24:16.766399+0000","last_peered":"2024-09-10T21:24:16.766399+0000","last_clean":"2024-09-10T21:24:16.766399+0000","last_became_active":"2024-09-10T21:24:15.785449+0000","last_became_peered":"2024-09-10T21:24:15.785449+0000","last_unstale":"2024-09-10T21:24:16.766399+0000","last_undegraded":"2024-09-10T21:24:16.766399+0000","last_fullsized":"2024-09-10T21:24:16.766399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:51.733023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.740413+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.740134+0000","last_became_peered":"2024-09-10T21:24:11.740134+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:22.624004+0000","last_change":"2024-09-10T21:24:10.707022+0000","last_active":"2024-09-10T21:24:22.624004+0000","last_peered":"2024-09-10T21:24:22.624004+0000","last_clean":"2024-09-10T21:24:22.624004+0000","last_became_active":"2024-09-10T21:24:10.706584+0000","last_became_peered":"2024-09-10T21:24:10.706584+0000","last_unstale":"2024-09-10T21:24:22.624004+0000","last_undegraded":"2024-09-10T21:24:22.624004+0000","last_fullsized":"2024-09-10T21:24:22.624004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:08.525705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265647+0000","last_change":"2024-09-10T21:24:15.789501+0000","last_active":"2024-09-10T21:24:17.265647+0000","last_peered":"2024-09-10T21:24:17.265647+0000","last_clean":"2024-09-10T21:24:17.265647+0000","last_became_active":"2024-09-10T21:24:15.789076+0000","last_became_peered":"2024-09-10T21:24:15.789076+0000","last_unstale":"2024-09-10T21:24:17.265647+0000","last_undegraded":"2024-09-10T21:24:17.265647+0000","last_fullsized":"2024-09-10T21:24:17.265647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:58.299617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766239+0000","last_change":"2024-09-10T21:24:11.757417+0000","last_active":"2024-09-10T21:24:16.766239+0000","last_peered":"2024-09-10T21:24:16.766239+0000","last_clean":"2024-09-10T21:24:16.766239+0000","last_became_active":"2024-09-10T21:24:11.757091+0000","last_became_peered":"2024-09-10T21:24:11.757091+0000","last_unstale":"2024-09-10T21:24:16.766239+0000","last_undegraded":"2024-09-10T21:24:16.766239+0000","last_fullsized":"2024-09-10T21:24:16.766239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:13.080850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770386+0000","last_change":"2024-09-10T21:24:10.707124+0000","last_active":"2024-09-10T21:24:16.770386+0000","last_peered":"2024-09-10T21:24:16.770386+0000","last_clean":"2024-09-10T21:24:16.770386+0000","last_became_active":"2024-09-10T21:24:10.706605+0000","last_became_peered":"2024-09-10T21:24:10.706605+0000","last_unstale":"2024-09-10T21:24:16.770386+0000","last_undegraded":"2024-09-10T21:24:16.770386+0000","last_fullsized":"2024-09-10T21:24:16.770386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262946+0000","last_change":"2024-09-10T21:24:15.776743+0000","last_active":"2024-09-10T21:24:17.262946+0000","last_peered":"2024-09-10T21:24:17.262946+0000","last_clean":"2024-09-10T21:24:17.262946+0000","last_became_active":"2024-09-10T21:24:15.774992+0000","last_became_peered":"2024-09-10T21:24:15.774992+0000","last_unstale":"2024-09-10T21:24:17.262946+0000","last_undegraded":"2024-09-10T21:24:17.262946+0000","last_fullsized":"2024-09-10T21:24:17.262946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:11.593663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265153+0000","last_change":"2024-09-10T21:24:11.750107+0000","last_active":"2024-09-10T21:24:17.265153+0000","last_peered":"2024-09-10T21:24:17.265153+0000","last_clean":"2024-09-10T21:24:17.265153+0000","last_became_active":"2024-09-10T21:24:11.749206+0000","last_became_peered":"2024-09-10T21:24:11.749206+0000","last_unstale":"2024-09-10T21:24:17.265153+0000","last_undegraded":"2024-09-10T21:24:17.265153+0000","last_fullsized":"2024-09-10T21:24:17.265153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265042+0000","last_change":"2024-09-10T21:24:10.711029+0000","last_active":"2024-09-10T21:24:17.265042+0000","last_peered":"2024-09-10T21:24:17.265042+0000","last_clean":"2024-09-10T21:24:17.265042+0000","last_became_active":"2024-09-10T21:24:10.710789+0000","last_became_peered":"2024-09-10T21:24:10.710789+0000","last_unstale":"2024-09-10T21:24:17.265042+0000","last_undegraded":"2024-09-10T21:24:17.265042+0000","last_fullsized":"2024-09-10T21:24:17.265042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:36.260277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768675+0000","last_change":"2024-09-10T21:24:15.779017+0000","last_active":"2024-09-10T21:24:16.768675+0000","last_peered":"2024-09-10T21:24:16.768675+0000","last_clean":"2024-09-10T21:24:16.768675+0000","last_became_active":"2024-09-10T21:24:15.774452+0000","last_became_peered":"2024-09-10T21:24:15.774452+0000","last_unstale":"2024-09-10T21:24:16.768675+0000","last_undegraded":"2024-09-10T21:24:16.768675+0000","last_fullsized":"2024-09-10T21:24:16.768675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:56.032301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264021+0000","last_change":"2024-09-10T21:24:11.748901+0000","last_active":"2024-09-10T21:24:17.264021+0000","last_peered":"2024-09-10T21:24:17.264021+0000","last_clean":"2024-09-10T21:24:17.264021+0000","last_became_active":"2024-09-10T21:24:11.748059+0000","last_became_peered":"2024-09-10T21:24:11.748059+0000","last_unstale":"2024-09-10T21:24:17.264021+0000","last_undegraded":"2024-09-10T21:24:17.264021+0000","last_fullsized":"2024-09-10T21:24:17.264021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:10.697344+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:10.697030+0000","last_became_peered":"2024-09-10T21:24:10.697030+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264496+0000","last_change":"2024-09-10T21:24:15.775122+0000","last_active":"2024-09-10T21:24:17.264496+0000","last_peered":"2024-09-10T21:24:17.264496+0000","last_clean":"2024-09-10T21:24:17.264496+0000","last_became_active":"2024-09-10T21:24:15.774561+0000","last_became_peered":"2024-09-10T21:24:15.774561+0000","last_unstale":"2024-09-10T21:24:17.264496+0000","last_undegraded":"2024-09-10T21:24:17.264496+0000","last_fullsized":"2024-09-10T21:24:17.264496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:19.897395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266159+0000","last_change":"2024-09-10T21:24:15.790852+0000","last_active":"2024-09-10T21:24:17.266159+0000","last_peered":"2024-09-10T21:24:17.266159+0000","last_clean":"2024-09-10T21:24:17.266159+0000","last_became_active":"2024-09-10T21:24:15.786896+0000","last_became_peered":"2024-09-10T21:24:15.786896+0000","last_unstale":"2024-09-10T21:24:17.266159+0000","last_undegraded":"2024-09-10T21:24:17.266159+0000","last_fullsized":"2024-09-10T21:24:17.266159+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:48.900914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264746+0000","last_change":"2024-09-10T21:24:11.732564+0000","last_active":"2024-09-10T21:24:17.264746+0000","last_peered":"2024-09-10T21:24:17.264746+0000","last_clean":"2024-09-10T21:24:17.264746+0000","last_became_active":"2024-09-10T21:24:11.732106+0000","last_became_peered":"2024-09-10T21:24:11.732106+0000","last_unstale":"2024-09-10T21:24:17.264746+0000","last_undegraded":"2024-09-10T21:24:17.264746+0000","last_fullsized":"2024-09-10T21:24:17.264746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.574642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765650+0000","last_change":"2024-09-10T21:24:10.714693+0000","last_active":"2024-09-10T21:24:16.765650+0000","last_peered":"2024-09-10T21:24:16.765650+0000","last_clean":"2024-09-10T21:24:16.765650+0000","last_became_active":"2024-09-10T21:24:10.714223+0000","last_became_peered":"2024-09-10T21:24:10.714223+0000","last_unstale":"2024-09-10T21:24:16.765650+0000","last_undegraded":"2024-09-10T21:24:16.765650+0000","last_fullsized":"2024-09-10T21:24:16.765650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:06.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766392+0000","last_change":"2024-09-10T21:24:15.784105+0000","last_active":"2024-09-10T21:24:16.766392+0000","last_peered":"2024-09-10T21:24:16.766392+0000","last_clean":"2024-09-10T21:24:16.766392+0000","last_became_active":"2024-09-10T21:24:15.783500+0000","last_became_peered":"2024-09-10T21:24:15.783500+0000","last_unstale":"2024-09-10T21:24:16.766392+0000","last_undegraded":"2024-09-10T21:24:16.766392+0000","last_fullsized":"2024-09-10T21:24:16.766392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.513517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263956+0000","last_change":"2024-09-10T21:24:11.738285+0000","last_active":"2024-09-10T21:24:17.263956+0000","last_peered":"2024-09-10T21:24:17.263956+0000","last_clean":"2024-09-10T21:24:17.263956+0000","last_became_active":"2024-09-10T21:24:11.738001+0000","last_became_peered":"2024-09-10T21:24:11.738001+0000","last_unstale":"2024-09-10T21:24:17.263956+0000","last_undegraded":"2024-09-10T21:24:17.263956+0000","last_fullsized":"2024-09-10T21:24:17.263956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769007+0000","last_change":"2024-09-10T21:24:10.710823+0000","last_active":"2024-09-10T21:24:16.769007+0000","last_peered":"2024-09-10T21:24:16.769007+0000","last_clean":"2024-09-10T21:24:16.769007+0000","last_became_active":"2024-09-10T21:24:10.710231+0000","last_became_peered":"2024-09-10T21:24:10.710231+0000","last_unstale":"2024-09-10T21:24:16.769007+0000","last_undegraded":"2024-09-10T21:24:16.769007+0000","last_fullsized":"2024-09-10T21:24:16.769007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:39.865360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765929+0000","last_change":"2024-09-10T21:24:10.711002+0000","last_active":"2024-09-10T21:24:16.765929+0000","last_peered":"2024-09-10T21:24:16.765929+0000","last_clean":"2024-09-10T21:24:16.765929+0000","last_became_active":"2024-09-10T21:24:10.710723+0000","last_became_peered":"2024-09-10T21:24:10.710723+0000","last_unstale":"2024-09-10T21:24:16.765929+0000","last_undegraded":"2024-09-10T21:24:16.765929+0000","last_fullsized":"2024-09-10T21:24:16.765929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:52.955941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:11.744353+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:11.743970+0000","last_became_peered":"2024-09-10T21:24:11.743970+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:35.066178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265974+0000","last_change":"2024-09-10T21:24:15.791043+0000","last_active":"2024-09-10T21:24:17.265974+0000","last_peered":"2024-09-10T21:24:17.265974+0000","last_clean":"2024-09-10T21:24:17.265974+0000","last_became_active":"2024-09-10T21:24:15.790322+0000","last_became_peered":"2024-09-10T21:24:15.790322+0000","last_unstale":"2024-09-10T21:24:17.265974+0000","last_undegraded":"2024-09-10T21:24:17.265974+0000","last_fullsized":"2024-09-10T21:24:17.265974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:18.317248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264568+0000","last_change":"2024-09-10T21:24:10.717526+0000","last_active":"2024-09-10T21:24:17.264568+0000","last_peered":"2024-09-10T21:24:17.264568+0000","last_clean":"2024-09-10T21:24:17.264568+0000","last_became_active":"2024-09-10T21:24:10.715368+0000","last_became_peered":"2024-09-10T21:24:10.715368+0000","last_unstale":"2024-09-10T21:24:17.264568+0000","last_undegraded":"2024-09-10T21:24:17.264568+0000","last_fullsized":"2024-09-10T21:24:17.264568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263197+0000","last_change":"2024-09-10T21:24:11.733617+0000","last_active":"2024-09-10T21:24:17.263197+0000","last_peered":"2024-09-10T21:24:17.263197+0000","last_clean":"2024-09-10T21:24:17.263197+0000","last_became_active":"2024-09-10T21:24:11.733343+0000","last_became_peered":"2024-09-10T21:24:11.733343+0000","last_unstale":"2024-09-10T21:24:17.263197+0000","last_undegraded":"2024-09-10T21:24:17.263197+0000","last_fullsized":"2024-09-10T21:24:17.263197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770202+0000","last_change":"2024-09-10T21:24:15.787912+0000","last_active":"2024-09-10T21:24:16.770202+0000","last_peered":"2024-09-10T21:24:16.770202+0000","last_clean":"2024-09-10T21:24:16.770202+0000","last_became_active":"2024-09-10T21:24:15.787441+0000","last_became_peered":"2024-09-10T21:24:15.787441+0000","last_unstale":"2024-09-10T21:24:16.770202+0000","last_undegraded":"2024-09-10T21:24:16.770202+0000","last_fullsized":"2024-09-10T21:24:16.770202+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:22.856812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262658+0000","last_change":"2024-09-10T21:24:11.747086+0000","last_active":"2024-09-10T21:24:17.262658+0000","last_peered":"2024-09-10T21:24:17.262658+0000","last_clean":"2024-09-10T21:24:17.262658+0000","last_became_active":"2024-09-10T21:24:11.746780+0000","last_became_peered":"2024-09-10T21:24:11.746780+0000","last_unstale":"2024-09-10T21:24:17.262658+0000","last_undegraded":"2024-09-10T21:24:17.262658+0000","last_fullsized":"2024-09-10T21:24:17.262658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:13.288615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769236+0000","last_change":"2024-09-10T21:24:11.732475+0000","last_active":"2024-09-10T21:24:16.769236+0000","last_peered":"2024-09-10T21:24:16.769236+0000","last_clean":"2024-09-10T21:24:16.769236+0000","last_became_active":"2024-09-10T21:24:11.731007+0000","last_became_peered":"2024-09-10T21:24:11.731007+0000","last_unstale":"2024-09-10T21:24:16.769236+0000","last_undegraded":"2024-09-10T21:24:16.769236+0000","last_fullsized":"2024-09-10T21:24:16.769236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765707+0000","last_change":"2024-09-10T21:24:11.744942+0000","last_active":"2024-09-10T21:24:16.765707+0000","last_peered":"2024-09-10T21:24:16.765707+0000","last_clean":"2024-09-10T21:24:16.765707+0000","last_became_active":"2024-09-10T21:24:11.744626+0000","last_became_peered":"2024-09-10T21:24:11.744626+0000","last_unstale":"2024-09-10T21:24:16.765707+0000","last_undegraded":"2024-09-10T21:24:16.765707+0000","last_fullsized":"2024-09-10T21:24:16.765707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:37.461417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264525+0000","last_change":"2024-09-10T21:24:11.745907+0000","last_active":"2024-09-10T21:24:17.264525+0000","last_peered":"2024-09-10T21:24:17.264525+0000","last_clean":"2024-09-10T21:24:17.264525+0000","last_became_active":"2024-09-10T21:24:11.745583+0000","last_became_peered":"2024-09-10T21:24:11.745583+0000","last_unstale":"2024-09-10T21:24:17.264525+0000","last_undegraded":"2024-09-10T21:24:17.264525+0000","last_fullsized":"2024-09-10T21:24:17.264525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:50.845294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770469+0000","last_change":"2024-09-10T21:24:11.756001+0000","last_active":"2024-09-10T21:24:16.770469+0000","last_peered":"2024-09-10T21:24:16.770469+0000","last_clean":"2024-09-10T21:24:16.770469+0000","last_became_active":"2024-09-10T21:24:11.755719+0000","last_became_peered":"2024-09-10T21:24:11.755719+0000","last_unstale":"2024-09-10T21:24:16.770469+0000","last_undegraded":"2024-09-10T21:24:16.770469+0000","last_fullsized":"2024-09-10T21:24:16.770469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:04.237158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769805+0000","last_change":"2024-09-10T21:24:11.740025+0000","last_active":"2024-09-10T21:24:16.769805+0000","last_peered":"2024-09-10T21:24:16.769805+0000","last_clean":"2024-09-10T21:24:16.769805+0000","last_became_active":"2024-09-10T21:24:11.739742+0000","last_became_peered":"2024-09-10T21:24:11.739742+0000","last_unstale":"2024-09-10T21:24:16.769805+0000","last_undegraded":"2024-09-10T21:24:16.769805+0000","last_fullsized":"2024-09-10T21:24:16.769805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:29.650259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263932+0000","last_change":"2024-09-10T21:24:11.749805+0000","last_active":"2024-09-10T21:24:17.263932+0000","last_peered":"2024-09-10T21:24:17.263932+0000","last_clean":"2024-09-10T21:24:17.263932+0000","last_became_active":"2024-09-10T21:24:11.748452+0000","last_became_peered":"2024-09-10T21:24:11.748452+0000","last_unstale":"2024-09-10T21:24:17.263932+0000","last_undegraded":"2024-09-10T21:24:17.263932+0000","last_fullsized":"2024-09-10T21:24:17.263932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:36.508158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264740+0000","last_change":"2024-09-10T21:24:11.750687+0000","last_active":"2024-09-10T21:24:17.264740+0000","last_peered":"2024-09-10T21:24:17.264740+0000","last_clean":"2024-09-10T21:24:17.264740+0000","last_became_active":"2024-09-10T21:24:11.749899+0000","last_became_peered":"2024-09-10T21:24:11.749899+0000","last_unstale":"2024-09-10T21:24:17.264740+0000","last_undegraded":"2024-09-10T21:24:17.264740+0000","last_fullsized":"2024-09-10T21:24:17.264740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765619+0000","last_change":"2024-09-10T21:24:11.734778+0000","last_active":"2024-09-10T21:24:16.765619+0000","last_peered":"2024-09-10T21:24:16.765619+0000","last_clean":"2024-09-10T21:24:16.765619+0000","last_became_active":"2024-09-10T21:24:11.734510+0000","last_became_peered":"2024-09-10T21:24:11.734510+0000","last_unstale":"2024-09-10T21:24:16.765619+0000","last_undegraded":"2024-09-10T21:24:16.765619+0000","last_fullsized":"2024-09-10T21:24:16.765619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:45.729068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263928+0000","last_change":"2024-09-10T21:24:11.758730+0000","last_active":"2024-09-10T21:24:17.263928+0000","last_peered":"2024-09-10T21:24:17.263928+0000","last_clean":"2024-09-10T21:24:17.263928+0000","last_became_active":"2024-09-10T21:24:11.758453+0000","last_became_peered":"2024-09-10T21:24:11.758453+0000","last_unstale":"2024-09-10T21:24:17.263928+0000","last_undegraded":"2024-09-10T21:24:17.263928+0000","last_fullsized":"2024-09-10T21:24:17.263928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:20.045933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263026+0000","last_change":"2024-09-10T21:24:11.727975+0000","last_active":"2024-09-10T21:24:17.263026+0000","last_peered":"2024-09-10T21:24:17.263026+0000","last_clean":"2024-09-10T21:24:17.263026+0000","last_became_active":"2024-09-10T21:24:11.727715+0000","last_became_peered":"2024-09-10T21:24:11.727715+0000","last_unstale":"2024-09-10T21:24:17.263026+0000","last_undegraded":"2024-09-10T21:24:17.263026+0000","last_fullsized":"2024-09-10T21:24:17.263026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264447+0000","last_change":"2024-09-10T21:24:11.752499+0000","last_active":"2024-09-10T21:24:17.264447+0000","last_peered":"2024-09-10T21:24:17.264447+0000","last_clean":"2024-09-10T21:24:17.264447+0000","last_became_active":"2024-09-10T21:24:11.748931+0000","last_became_peered":"2024-09-10T21:24:11.748931+0000","last_unstale":"2024-09-10T21:24:17.264447+0000","last_undegraded":"2024-09-10T21:24:17.264447+0000","last_fullsized":"2024-09-10T21:24:17.264447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:48.334318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264810+0000","last_change":"2024-09-10T21:24:11.742340+0000","last_active":"2024-09-10T21:24:17.264810+0000","last_peered":"2024-09-10T21:24:17.264810+0000","last_clean":"2024-09-10T21:24:17.264810+0000","last_became_active":"2024-09-10T21:24:11.741702+0000","last_became_peered":"2024-09-10T21:24:11.741702+0000","last_unstale":"2024-09-10T21:24:17.264810+0000","last_undegraded":"2024-09-10T21:24:17.264810+0000","last_fullsized":"2024-09-10T21:24:17.264810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766584+0000","last_change":"2024-09-10T21:24:11.744751+0000","last_active":"2024-09-10T21:24:16.766584+0000","last_peered":"2024-09-10T21:24:16.766584+0000","last_clean":"2024-09-10T21:24:16.766584+0000","last_became_active":"2024-09-10T21:24:11.744431+0000","last_became_peered":"2024-09-10T21:24:11.744431+0000","last_unstale":"2024-09-10T21:24:16.766584+0000","last_undegraded":"2024-09-10T21:24:16.766584+0000","last_fullsized":"2024-09-10T21:24:16.766584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:11.693990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770590+0000","last_change":"2024-09-10T21:24:11.748605+0000","last_active":"2024-09-10T21:24:16.770590+0000","last_peered":"2024-09-10T21:24:16.770590+0000","last_clean":"2024-09-10T21:24:16.770590+0000","last_became_active":"2024-09-10T21:24:11.748279+0000","last_became_peered":"2024-09-10T21:24:11.748279+0000","last_unstale":"2024-09-10T21:24:16.770590+0000","last_undegraded":"2024-09-10T21:24:16.770590+0000","last_fullsized":"2024-09-10T21:24:16.770590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766580+0000","last_change":"2024-09-10T21:24:11.750748+0000","last_active":"2024-09-10T21:24:16.766580+0000","last_peered":"2024-09-10T21:24:16.766580+0000","last_clean":"2024-09-10T21:24:16.766580+0000","last_became_active":"2024-09-10T21:24:11.750317+0000","last_became_peered":"2024-09-10T21:24:11.750317+0000","last_unstale":"2024-09-10T21:24:16.766580+0000","last_undegraded":"2024-09-10T21:24:16.766580+0000","last_fullsized":"2024-09-10T21:24:16.766580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:08.259327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765529+0000","last_change":"2024-09-10T21:24:11.757620+0000","last_active":"2024-09-10T21:24:16.765529+0000","last_peered":"2024-09-10T21:24:16.765529+0000","last_clean":"2024-09-10T21:24:16.765529+0000","last_became_active":"2024-09-10T21:24:11.757088+0000","last_became_peered":"2024-09-10T21:24:11.757088+0000","last_unstale":"2024-09-10T21:24:16.765529+0000","last_undegraded":"2024-09-10T21:24:16.765529+0000","last_fullsized":"2024-09-10T21:24:16.765529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263812+0000","last_change":"2024-09-10T21:24:11.745932+0000","last_active":"2024-09-10T21:24:17.263812+0000","last_peered":"2024-09-10T21:24:17.263812+0000","last_clean":"2024-09-10T21:24:17.263812+0000","last_became_active":"2024-09-10T21:24:11.745682+0000","last_became_peered":"2024-09-10T21:24:11.745682+0000","last_unstale":"2024-09-10T21:24:17.263812+0000","last_undegraded":"2024-09-10T21:24:17.263812+0000","last_fullsized":"2024-09-10T21:24:17.263812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769215+0000","last_change":"2024-09-10T21:24:11.740847+0000","last_active":"2024-09-10T21:24:16.769215+0000","last_peered":"2024-09-10T21:24:16.769215+0000","last_clean":"2024-09-10T21:24:16.769215+0000","last_became_active":"2024-09-10T21:24:11.740548+0000","last_became_peered":"2024-09-10T21:24:11.740548+0000","last_unstale":"2024-09-10T21:24:16.769215+0000","last_undegraded":"2024-09-10T21:24:16.769215+0000","last_fullsized":"2024-09-10T21:24:16.769215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770304+0000","last_change":"2024-09-10T21:24:11.742902+0000","last_active":"2024-09-10T21:24:16.770304+0000","last_peered":"2024-09-10T21:24:16.770304+0000","last_clean":"2024-09-10T21:24:16.770304+0000","last_became_active":"2024-09-10T21:24:11.742449+0000","last_became_peered":"2024-09-10T21:24:11.742449+0000","last_unstale":"2024-09-10T21:24:16.770304+0000","last_undegraded":"2024-09-10T21:24:16.770304+0000","last_fullsized":"2024-09-10T21:24:16.770304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:25.302008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264763+0000","last_change":"2024-09-10T21:24:11.742660+0000","last_active":"2024-09-10T21:24:17.264763+0000","last_peered":"2024-09-10T21:24:17.264763+0000","last_clean":"2024-09-10T21:24:17.264763+0000","last_became_active":"2024-09-10T21:24:11.742069+0000","last_became_peered":"2024-09-10T21:24:11.742069+0000","last_unstale":"2024-09-10T21:24:17.264763+0000","last_undegraded":"2024-09-10T21:24:17.264763+0000","last_fullsized":"2024-09-10T21:24:17.264763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:35.526539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264897+0000","last_change":"2024-09-10T21:24:11.743498+0000","last_active":"2024-09-10T21:24:17.264897+0000","last_peered":"2024-09-10T21:24:17.264897+0000","last_clean":"2024-09-10T21:24:17.264897+0000","last_became_active":"2024-09-10T21:24:11.743017+0000","last_became_peered":"2024-09-10T21:24:11.743017+0000","last_unstale":"2024-09-10T21:24:17.264897+0000","last_undegraded":"2024-09-10T21:24:17.264897+0000","last_fullsized":"2024-09-10T21:24:17.264897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.754599+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.754341+0000","last_became_peered":"2024-09-10T21:24:11.754341+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264501+0000","last_change":"2024-09-10T21:24:11.726485+0000","last_active":"2024-09-10T21:24:17.264501+0000","last_peered":"2024-09-10T21:24:17.264501+0000","last_clean":"2024-09-10T21:24:17.264501+0000","last_became_active":"2024-09-10T21:24:11.726157+0000","last_became_peered":"2024-09-10T21:24:11.726157+0000","last_unstale":"2024-09-10T21:24:17.264501+0000","last_undegraded":"2024-09-10T21:24:17.264501+0000","last_fullsized":"2024-09-10T21:24:17.264501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263664+0000","last_change":"2024-09-10T21:24:11.754793+0000","last_active":"2024-09-10T21:24:17.263664+0000","last_peered":"2024-09-10T21:24:17.263664+0000","last_clean":"2024-09-10T21:24:17.263664+0000","last_became_active":"2024-09-10T21:24:11.754372+0000","last_became_peered":"2024-09-10T21:24:11.754372+0000","last_unstale":"2024-09-10T21:24:17.263664+0000","last_undegraded":"2024-09-10T21:24:17.263664+0000","last_fullsized":"2024-09-10T21:24:17.263664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:25.692729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263914+0000","last_change":"2024-09-10T21:24:11.741830+0000","last_active":"2024-09-10T21:24:17.263914+0000","last_peered":"2024-09-10T21:24:17.263914+0000","last_clean":"2024-09-10T21:24:17.263914+0000","last_became_active":"2024-09-10T21:24:11.741543+0000","last_became_peered":"2024-09-10T21:24:11.741543+0000","last_unstale":"2024-09-10T21:24:17.263914+0000","last_undegraded":"2024-09-10T21:24:17.263914+0000","last_fullsized":"2024-09-10T21:24:17.263914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263114+0000","last_change":"2024-09-10T21:24:11.744958+0000","last_active":"2024-09-10T21:24:17.263114+0000","last_peered":"2024-09-10T21:24:17.263114+0000","last_clean":"2024-09-10T21:24:17.263114+0000","last_became_active":"2024-09-10T21:24:11.744611+0000","last_became_peered":"2024-09-10T21:24:11.744611+0000","last_unstale":"2024-09-10T21:24:17.263114+0000","last_undegraded":"2024-09-10T21:24:17.263114+0000","last_fullsized":"2024-09-10T21:24:17.263114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264460+0000","last_change":"2024-09-10T21:24:11.742259+0000","last_active":"2024-09-10T21:24:17.264460+0000","last_peered":"2024-09-10T21:24:17.264460+0000","last_clean":"2024-09-10T21:24:17.264460+0000","last_became_active":"2024-09-10T21:24:11.742010+0000","last_became_peered":"2024-09-10T21:24:11.742010+0000","last_unstale":"2024-09-10T21:24:17.264460+0000","last_undegraded":"2024-09-10T21:24:17.264460+0000","last_fullsized":"2024-09-10T21:24:17.264460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263425+0000","last_change":"2024-09-10T21:24:11.747125+0000","last_active":"2024-09-10T21:24:17.263425+0000","last_peered":"2024-09-10T21:24:17.263425+0000","last_clean":"2024-09-10T21:24:17.263425+0000","last_became_active":"2024-09-10T21:24:11.746844+0000","last_became_peered":"2024-09-10T21:24:11.746844+0000","last_unstale":"2024-09-10T21:24:17.263425+0000","last_undegraded":"2024-09-10T21:24:17.263425+0000","last_fullsized":"2024-09-10T21:24:17.263425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:28.995110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265138+0000","last_change":"2024-09-10T21:24:11.737282+0000","last_active":"2024-09-10T21:24:17.265138+0000","last_peered":"2024-09-10T21:24:17.265138+0000","last_clean":"2024-09-10T21:24:17.265138+0000","last_became_active":"2024-09-10T21:24:11.736985+0000","last_became_peered":"2024-09-10T21:24:11.736985+0000","last_unstale":"2024-09-10T21:24:17.265138+0000","last_undegraded":"2024-09-10T21:24:17.265138+0000","last_fullsized":"2024-09-10T21:24:17.265138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264431+0000","last_change":"2024-09-10T21:24:11.738254+0000","last_active":"2024-09-10T21:24:17.264431+0000","last_peered":"2024-09-10T21:24:17.264431+0000","last_clean":"2024-09-10T21:24:17.264431+0000","last_became_active":"2024-09-10T21:24:11.737963+0000","last_became_peered":"2024-09-10T21:24:11.737963+0000","last_unstale":"2024-09-10T21:24:17.264431+0000","last_undegraded":"2024-09-10T21:24:17.264431+0000","last_fullsized":"2024-09-10T21:24:17.264431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265077+0000","last_change":"2024-09-10T21:24:11.743284+0000","last_active":"2024-09-10T21:24:17.265077+0000","last_peered":"2024-09-10T21:24:17.265077+0000","last_clean":"2024-09-10T21:24:17.265077+0000","last_became_active":"2024-09-10T21:24:11.742655+0000","last_became_peered":"2024-09-10T21:24:11.742655+0000","last_unstale":"2024-09-10T21:24:17.265077+0000","last_undegraded":"2024-09-10T21:24:17.265077+0000","last_fullsized":"2024-09-10T21:24:17.265077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:24:37.975 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:24:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- 172.21.15.97:0/3502967748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 msgr2=0x7fe91006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 --2- 172.21.15.97:0/3502967748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe91006f090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe90000aa40 tx=0x7fe900020400 comp rx=0 tx=0).stop 2024-09-10T21:24:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- 172.21.15.97:0/3502967748 shutdown_connections 2024-09-10T21:24:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 --2- 172.21.15.97:0/3502967748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9100ceee0 0x7fe9100cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 --2- 172.21.15.97:0/3502967748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe91006f090 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- 172.21.15.97:0/3502967748 >> 172.21.15.97:0/3502967748 conn(0x7fe9100b1780 msgr2=0x7fe9100b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- 172.21.15.97:0/3502967748 shutdown_connections 2024-09-10T21:24:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- 172.21.15.97:0/3502967748 wait complete. 2024-09-10T21:24:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 Processor -- start 2024-09-10T21:24:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- start start 2024-09-10T21:24:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe9101e3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9100ceee0 0x7fe9101e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9101511f0 0x7fe9101e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe91006e740 con 0x7fe91006ec40 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe91006e5c0 con 0x7fe9101511f0 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe918800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe91006e8c0 con 0x7fe9100ceee0 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe9101e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe9101e3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59164/0 (socket says 172.21.15.97:59164) 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 -- 172.21.15.97:0/2544783909 learned_addr learned my addr 172.21.15.97:0/2544783909 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe916800640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9101511f0 0x7fe9101e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915400640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9100ceee0 0x7fe9101e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 -- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9100ceee0 msgr2=0x7fe9101e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9100ceee0 0x7fe9101e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 -- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9101511f0 msgr2=0x7fe9101e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9101511f0 0x7fe9101e3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.148+0000 7fe915e00640 1 -- 172.21.15.97:0/2544783909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9101e4a20 con 0x7fe91006ec40 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe915e00640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe9101e3640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe90400b440 tx=0x7fe90400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe90a000640 1 -- 172.21.15.97:0/2544783909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904015020 con 0x7fe91006ec40 2024-09-10T21:24:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe90a000640 1 -- 172.21.15.97:0/2544783909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe904011980 con 0x7fe91006ec40 2024-09-10T21:24:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe90a000640 1 -- 172.21.15.97:0/2544783909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904011dd0 con 0x7fe91006ec40 2024-09-10T21:24:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9101e4740 con 0x7fe91006ec40 2024-09-10T21:24:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9101e2e40 con 0x7fe91006ec40 2024-09-10T21:24:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91006f090 con 0x7fe91006ec40 2024-09-10T21:24:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe90a000640 1 -- 172.21.15.97:0/2544783909 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe90401a400 con 0x7fe91006ec40 2024-09-10T21:24:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe90a000640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8f4072610 0x7fe8f4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.152+0000 7fe90a000640 1 -- 172.21.15.97:0/2544783909 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe904093680 con 0x7fe91006ec40 2024-09-10T21:24:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.156+0000 7fe915400640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8f4072610 0x7fe8f4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.156+0000 7fe915400640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8f4072610 0x7fe8f4074ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b2000 tx=0x7fe90001ec40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:38.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.156+0000 7fe90a000640 1 -- 172.21.15.97:0/2544783909 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe90401f030 con 0x7fe91006ec40 2024-09-10T21:24:38.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.320+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9101e3190 con 0x7fe8f4072610 2024-09-10T21:24:38.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.328+0000 7fe90a000640 1 -- 172.21.15.97:0/2544783909 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7fe9101e3190 con 0x7fe8f4072610 2024-09-10T21:24:38.337 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:24:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8f4072610 msgr2=0x7fe8f4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8f4072610 0x7fe8f4074ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b2000 tx=0x7fe90001ec40 comp rx=0 tx=0).stop 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 msgr2=0x7fe9101e3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe9101e3640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe90400b440 tx=0x7fe90400b900 comp rx=0 tx=0).stop 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 shutdown_connections 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8f4072610 0x7fe8f4074ad0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9101511f0 0x7fe9101e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9100ceee0 0x7fe9101e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 --2- 172.21.15.97:0/2544783909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91006ec40 0x7fe9101e3640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 >> 172.21.15.97:0/2544783909 conn(0x7fe9100b1780 msgr2=0x7fe910150550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 shutdown_connections 2024-09-10T21:24:38.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.332+0000 7fe918800640 1 -- 172.21.15.97:0/2544783909 wait complete. 2024-09-10T21:24:38.358 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-10T21:24:38.075199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219584,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755136,"statfs":{"total":773094113280,"available":772869259264,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003553"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264781+0000","last_change":"2024-09-10T21:24:10.717118+0000","last_active":"2024-09-10T21:24:17.264781+0000","last_peered":"2024-09-10T21:24:17.264781+0000","last_clean":"2024-09-10T21:24:17.264781+0000","last_became_active":"2024-09-10T21:24:10.716871+0000","last_became_peered":"2024-09-10T21:24:10.716871+0000","last_unstale":"2024-09-10T21:24:17.264781+0000","last_undegraded":"2024-09-10T21:24:17.264781+0000","last_fullsized":"2024-09-10T21:24:17.264781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:26.347318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264579+0000","last_change":"2024-09-10T21:24:11.758545+0000","last_active":"2024-09-10T21:24:17.264579+0000","last_peered":"2024-09-10T21:24:17.264579+0000","last_clean":"2024-09-10T21:24:17.264579+0000","last_became_active":"2024-09-10T21:24:11.758227+0000","last_became_peered":"2024-09-10T21:24:11.758227+0000","last_unstale":"2024-09-10T21:24:17.264579+0000","last_undegraded":"2024-09-10T21:24:17.264579+0000","last_fullsized":"2024-09-10T21:24:17.264579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768689+0000","last_change":"2024-09-10T21:24:15.784509+0000","last_active":"2024-09-10T21:24:16.768689+0000","last_peered":"2024-09-10T21:24:16.768689+0000","last_clean":"2024-09-10T21:24:16.768689+0000","last_became_active":"2024-09-10T21:24:15.777793+0000","last_became_peered":"2024-09-10T21:24:15.777793+0000","last_unstale":"2024-09-10T21:24:16.768689+0000","last_undegraded":"2024-09-10T21:24:16.768689+0000","last_fullsized":"2024-09-10T21:24:16.768689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:55.568506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264784+0000","last_change":"2024-09-10T21:24:10.717927+0000","last_active":"2024-09-10T21:24:17.264784+0000","last_peered":"2024-09-10T21:24:17.264784+0000","last_clean":"2024-09-10T21:24:17.264784+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:17.264784+0000","last_undegraded":"2024-09-10T21:24:17.264784+0000","last_fullsized":"2024-09-10T21:24:17.264784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265060+0000","last_change":"2024-09-10T21:24:11.750430+0000","last_active":"2024-09-10T21:24:17.265060+0000","last_peered":"2024-09-10T21:24:17.265060+0000","last_clean":"2024-09-10T21:24:17.265060+0000","last_became_active":"2024-09-10T21:24:11.749604+0000","last_became_peered":"2024-09-10T21:24:11.749604+0000","last_unstale":"2024-09-10T21:24:17.265060+0000","last_undegraded":"2024-09-10T21:24:17.265060+0000","last_fullsized":"2024-09-10T21:24:17.265060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262600+0000","last_change":"2024-09-10T21:24:15.775067+0000","last_active":"2024-09-10T21:24:17.262600+0000","last_peered":"2024-09-10T21:24:17.262600+0000","last_clean":"2024-09-10T21:24:17.262600+0000","last_became_active":"2024-09-10T21:24:15.773909+0000","last_became_peered":"2024-09-10T21:24:15.773909+0000","last_unstale":"2024-09-10T21:24:17.262600+0000","last_undegraded":"2024-09-10T21:24:17.262600+0000","last_fullsized":"2024-09-10T21:24:17.262600+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:09.297494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265037+0000","last_change":"2024-09-10T21:24:10.711527+0000","last_active":"2024-09-10T21:24:17.265037+0000","last_peered":"2024-09-10T21:24:17.265037+0000","last_clean":"2024-09-10T21:24:17.265037+0000","last_became_active":"2024-09-10T21:24:10.711276+0000","last_became_peered":"2024-09-10T21:24:10.711276+0000","last_unstale":"2024-09-10T21:24:17.265037+0000","last_undegraded":"2024-09-10T21:24:17.265037+0000","last_fullsized":"2024-09-10T21:24:17.265037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263749+0000","last_change":"2024-09-10T21:24:11.756415+0000","last_active":"2024-09-10T21:24:17.263749+0000","last_peered":"2024-09-10T21:24:17.263749+0000","last_clean":"2024-09-10T21:24:17.263749+0000","last_became_active":"2024-09-10T21:24:11.754620+0000","last_became_peered":"2024-09-10T21:24:11.754620+0000","last_unstale":"2024-09-10T21:24:17.263749+0000","last_undegraded":"2024-09-10T21:24:17.263749+0000","last_fullsized":"2024-09-10T21:24:17.263749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:48.122962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262734+0000","last_change":"2024-09-10T21:24:15.782482+0000","last_active":"2024-09-10T21:24:17.262734+0000","last_peered":"2024-09-10T21:24:17.262734+0000","last_clean":"2024-09-10T21:24:17.262734+0000","last_became_active":"2024-09-10T21:24:15.782107+0000","last_became_peered":"2024-09-10T21:24:15.782107+0000","last_unstale":"2024-09-10T21:24:17.262734+0000","last_undegraded":"2024-09-10T21:24:17.262734+0000","last_fullsized":"2024-09-10T21:24:17.262734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.906721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264925+0000","last_change":"2024-09-10T21:24:10.716898+0000","last_active":"2024-09-10T21:24:17.264925+0000","last_peered":"2024-09-10T21:24:17.264925+0000","last_clean":"2024-09-10T21:24:17.264925+0000","last_became_active":"2024-09-10T21:24:10.716644+0000","last_became_peered":"2024-09-10T21:24:10.716644+0000","last_unstale":"2024-09-10T21:24:17.264925+0000","last_undegraded":"2024-09-10T21:24:17.264925+0000","last_fullsized":"2024-09-10T21:24:17.264925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:41.682216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766338+0000","last_change":"2024-09-10T21:24:11.747195+0000","last_active":"2024-09-10T21:24:16.766338+0000","last_peered":"2024-09-10T21:24:16.766338+0000","last_clean":"2024-09-10T21:24:16.766338+0000","last_became_active":"2024-09-10T21:24:11.747038+0000","last_became_peered":"2024-09-10T21:24:11.747038+0000","last_unstale":"2024-09-10T21:24:16.766338+0000","last_undegraded":"2024-09-10T21:24:16.766338+0000","last_fullsized":"2024-09-10T21:24:16.766338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262860+0000","last_change":"2024-09-10T21:24:15.782490+0000","last_active":"2024-09-10T21:24:17.262860+0000","last_peered":"2024-09-10T21:24:17.262860+0000","last_clean":"2024-09-10T21:24:17.262860+0000","last_became_active":"2024-09-10T21:24:15.782079+0000","last_became_peered":"2024-09-10T21:24:15.782079+0000","last_unstale":"2024-09-10T21:24:17.262860+0000","last_undegraded":"2024-09-10T21:24:17.262860+0000","last_fullsized":"2024-09-10T21:24:17.262860+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:42.047447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765875+0000","last_change":"2024-09-10T21:24:11.734798+0000","last_active":"2024-09-10T21:24:16.765875+0000","last_peered":"2024-09-10T21:24:16.765875+0000","last_clean":"2024-09-10T21:24:16.765875+0000","last_became_active":"2024-09-10T21:24:11.734495+0000","last_became_peered":"2024-09-10T21:24:11.734495+0000","last_unstale":"2024-09-10T21:24:16.765875+0000","last_undegraded":"2024-09-10T21:24:16.765875+0000","last_fullsized":"2024-09-10T21:24:16.765875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:08.842495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766019+0000","last_change":"2024-09-10T21:24:10.718178+0000","last_active":"2024-09-10T21:24:16.766019+0000","last_peered":"2024-09-10T21:24:16.766019+0000","last_clean":"2024-09-10T21:24:16.766019+0000","last_became_active":"2024-09-10T21:24:10.717662+0000","last_became_peered":"2024-09-10T21:24:10.717662+0000","last_unstale":"2024-09-10T21:24:16.766019+0000","last_undegraded":"2024-09-10T21:24:16.766019+0000","last_fullsized":"2024-09-10T21:24:16.766019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:01.094807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766145+0000","last_change":"2024-09-10T21:24:15.786334+0000","last_active":"2024-09-10T21:24:16.766145+0000","last_peered":"2024-09-10T21:24:16.766145+0000","last_clean":"2024-09-10T21:24:16.766145+0000","last_became_active":"2024-09-10T21:24:15.785888+0000","last_became_peered":"2024-09-10T21:24:15.785888+0000","last_unstale":"2024-09-10T21:24:16.766145+0000","last_undegraded":"2024-09-10T21:24:16.766145+0000","last_fullsized":"2024-09-10T21:24:16.766145+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:27.906530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769104+0000","last_change":"2024-09-10T21:24:11.743760+0000","last_active":"2024-09-10T21:24:16.769104+0000","last_peered":"2024-09-10T21:24:16.769104+0000","last_clean":"2024-09-10T21:24:16.769104+0000","last_became_active":"2024-09-10T21:24:11.743459+0000","last_became_peered":"2024-09-10T21:24:11.743459+0000","last_unstale":"2024-09-10T21:24:16.769104+0000","last_undegraded":"2024-09-10T21:24:16.769104+0000","last_fullsized":"2024-09-10T21:24:16.769104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:53.958469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770545+0000","last_change":"2024-09-10T21:24:10.719093+0000","last_active":"2024-09-10T21:24:16.770545+0000","last_peered":"2024-09-10T21:24:16.770545+0000","last_clean":"2024-09-10T21:24:16.770545+0000","last_became_active":"2024-09-10T21:24:10.715489+0000","last_became_peered":"2024-09-10T21:24:10.715489+0000","last_unstale":"2024-09-10T21:24:16.770545+0000","last_undegraded":"2024-09-10T21:24:16.770545+0000","last_fullsized":"2024-09-10T21:24:16.770545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264532+0000","last_change":"2024-09-10T21:24:15.777643+0000","last_active":"2024-09-10T21:24:17.264532+0000","last_peered":"2024-09-10T21:24:17.264532+0000","last_clean":"2024-09-10T21:24:17.264532+0000","last_became_active":"2024-09-10T21:24:15.777271+0000","last_became_peered":"2024-09-10T21:24:15.777271+0000","last_unstale":"2024-09-10T21:24:17.264532+0000","last_undegraded":"2024-09-10T21:24:17.264532+0000","last_fullsized":"2024-09-10T21:24:17.264532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:26.509163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263743+0000","last_change":"2024-09-10T21:24:11.724275+0000","last_active":"2024-09-10T21:24:17.263743+0000","last_peered":"2024-09-10T21:24:17.263743+0000","last_clean":"2024-09-10T21:24:17.263743+0000","last_became_active":"2024-09-10T21:24:11.723982+0000","last_became_peered":"2024-09-10T21:24:11.723982+0000","last_unstale":"2024-09-10T21:24:17.263743+0000","last_undegraded":"2024-09-10T21:24:17.263743+0000","last_fullsized":"2024-09-10T21:24:17.263743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:28.218402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264861+0000","last_change":"2024-09-10T21:24:10.712213+0000","last_active":"2024-09-10T21:24:17.264861+0000","last_peered":"2024-09-10T21:24:17.264861+0000","last_clean":"2024-09-10T21:24:17.264861+0000","last_became_active":"2024-09-10T21:24:10.711951+0000","last_became_peered":"2024-09-10T21:24:10.711951+0000","last_unstale":"2024-09-10T21:24:17.264861+0000","last_undegraded":"2024-09-10T21:24:17.264861+0000","last_fullsized":"2024-09-10T21:24:17.264861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:54.891884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770275+0000","last_change":"2024-09-10T21:24:15.780288+0000","last_active":"2024-09-10T21:24:16.770275+0000","last_peered":"2024-09-10T21:24:16.770275+0000","last_clean":"2024-09-10T21:24:16.770275+0000","last_became_active":"2024-09-10T21:24:15.778928+0000","last_became_peered":"2024-09-10T21:24:15.778928+0000","last_unstale":"2024-09-10T21:24:16.770275+0000","last_undegraded":"2024-09-10T21:24:16.770275+0000","last_fullsized":"2024-09-10T21:24:16.770275+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:45.367503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765552+0000","last_change":"2024-09-10T21:24:11.730800+0000","last_active":"2024-09-10T21:24:16.765552+0000","last_peered":"2024-09-10T21:24:16.765552+0000","last_clean":"2024-09-10T21:24:16.765552+0000","last_became_active":"2024-09-10T21:24:11.730474+0000","last_became_peered":"2024-09-10T21:24:11.730474+0000","last_unstale":"2024-09-10T21:24:16.765552+0000","last_undegraded":"2024-09-10T21:24:16.765552+0000","last_fullsized":"2024-09-10T21:24:16.765552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263542+0000","last_change":"2024-09-10T21:24:10.712139+0000","last_active":"2024-09-10T21:24:17.263542+0000","last_peered":"2024-09-10T21:24:17.263542+0000","last_clean":"2024-09-10T21:24:17.263542+0000","last_became_active":"2024-09-10T21:24:10.710501+0000","last_became_peered":"2024-09-10T21:24:10.710501+0000","last_unstale":"2024-09-10T21:24:17.263542+0000","last_undegraded":"2024-09-10T21:24:17.263542+0000","last_fullsized":"2024-09-10T21:24:17.263542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:15.323315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263764+0000","last_change":"2024-09-10T21:24:15.770139+0000","last_active":"2024-09-10T21:24:17.263764+0000","last_peered":"2024-09-10T21:24:17.263764+0000","last_clean":"2024-09-10T21:24:17.263764+0000","last_became_active":"2024-09-10T21:24:15.769554+0000","last_became_peered":"2024-09-10T21:24:15.769554+0000","last_unstale":"2024-09-10T21:24:17.263764+0000","last_undegraded":"2024-09-10T21:24:17.263764+0000","last_fullsized":"2024-09-10T21:24:17.263764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:14.866707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266054+0000","last_change":"2024-09-10T21:24:11.732319+0000","last_active":"2024-09-10T21:24:17.266054+0000","last_peered":"2024-09-10T21:24:17.266054+0000","last_clean":"2024-09-10T21:24:17.266054+0000","last_became_active":"2024-09-10T21:24:11.732038+0000","last_became_peered":"2024-09-10T21:24:11.732038+0000","last_unstale":"2024-09-10T21:24:17.266054+0000","last_undegraded":"2024-09-10T21:24:17.266054+0000","last_fullsized":"2024-09-10T21:24:17.266054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:34.760961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766655+0000","last_change":"2024-09-10T21:24:10.720607+0000","last_active":"2024-09-10T21:24:16.766655+0000","last_peered":"2024-09-10T21:24:16.766655+0000","last_clean":"2024-09-10T21:24:16.766655+0000","last_became_active":"2024-09-10T21:24:10.720144+0000","last_became_peered":"2024-09-10T21:24:10.720144+0000","last_unstale":"2024-09-10T21:24:16.766655+0000","last_undegraded":"2024-09-10T21:24:16.766655+0000","last_fullsized":"2024-09-10T21:24:16.766655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:10.162599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263165+0000","last_change":"2024-09-10T21:24:15.777793+0000","last_active":"2024-09-10T21:24:17.263165+0000","last_peered":"2024-09-10T21:24:17.263165+0000","last_clean":"2024-09-10T21:24:17.263165+0000","last_became_active":"2024-09-10T21:24:15.777358+0000","last_became_peered":"2024-09-10T21:24:15.777358+0000","last_unstale":"2024-09-10T21:24:17.263165+0000","last_undegraded":"2024-09-10T21:24:17.263165+0000","last_fullsized":"2024-09-10T21:24:17.263165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:48.503713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264581+0000","last_change":"2024-09-10T21:24:11.735858+0000","last_active":"2024-09-10T21:24:17.264581+0000","last_peered":"2024-09-10T21:24:17.264581+0000","last_clean":"2024-09-10T21:24:17.264581+0000","last_became_active":"2024-09-10T21:24:11.735599+0000","last_became_peered":"2024-09-10T21:24:11.735599+0000","last_unstale":"2024-09-10T21:24:17.264581+0000","last_undegraded":"2024-09-10T21:24:17.264581+0000","last_fullsized":"2024-09-10T21:24:17.264581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:53.530511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266208+0000","last_change":"2024-09-10T21:24:10.712710+0000","last_active":"2024-09-10T21:24:17.266208+0000","last_peered":"2024-09-10T21:24:17.266208+0000","last_clean":"2024-09-10T21:24:17.266208+0000","last_became_active":"2024-09-10T21:24:10.712400+0000","last_became_peered":"2024-09-10T21:24:10.712400+0000","last_unstale":"2024-09-10T21:24:17.266208+0000","last_undegraded":"2024-09-10T21:24:17.266208+0000","last_fullsized":"2024-09-10T21:24:17.266208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:57.508268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263912+0000","last_change":"2024-09-10T21:24:15.776045+0000","last_active":"2024-09-10T21:24:17.263912+0000","last_peered":"2024-09-10T21:24:17.263912+0000","last_clean":"2024-09-10T21:24:17.263912+0000","last_became_active":"2024-09-10T21:24:15.775199+0000","last_became_peered":"2024-09-10T21:24:15.775199+0000","last_unstale":"2024-09-10T21:24:17.263912+0000","last_undegraded":"2024-09-10T21:24:17.263912+0000","last_fullsized":"2024-09-10T21:24:17.263912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.516796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770569+0000","last_change":"2024-09-10T21:24:11.737947+0000","last_active":"2024-09-10T21:24:16.770569+0000","last_peered":"2024-09-10T21:24:16.770569+0000","last_clean":"2024-09-10T21:24:16.770569+0000","last_became_active":"2024-09-10T21:24:11.735572+0000","last_became_peered":"2024-09-10T21:24:11.735572+0000","last_unstale":"2024-09-10T21:24:16.770569+0000","last_undegraded":"2024-09-10T21:24:16.770569+0000","last_fullsized":"2024-09-10T21:24:16.770569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:41.353969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766677+0000","last_change":"2024-09-10T21:24:10.712429+0000","last_active":"2024-09-10T21:24:16.766677+0000","last_peered":"2024-09-10T21:24:16.766677+0000","last_clean":"2024-09-10T21:24:16.766677+0000","last_became_active":"2024-09-10T21:24:10.712150+0000","last_became_peered":"2024-09-10T21:24:10.712150+0000","last_unstale":"2024-09-10T21:24:16.766677+0000","last_undegraded":"2024-09-10T21:24:16.766677+0000","last_fullsized":"2024-09-10T21:24:16.766677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:37.565989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769802+0000","last_change":"2024-09-10T21:24:15.778516+0000","last_active":"2024-09-10T21:24:16.769802+0000","last_peered":"2024-09-10T21:24:16.769802+0000","last_clean":"2024-09-10T21:24:16.769802+0000","last_became_active":"2024-09-10T21:24:15.778061+0000","last_became_peered":"2024-09-10T21:24:15.778061+0000","last_unstale":"2024-09-10T21:24:16.769802+0000","last_undegraded":"2024-09-10T21:24:16.769802+0000","last_fullsized":"2024-09-10T21:24:16.769802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:38.186163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263379+0000","last_change":"2024-09-10T21:24:11.747693+0000","last_active":"2024-09-10T21:24:17.263379+0000","last_peered":"2024-09-10T21:24:17.263379+0000","last_clean":"2024-09-10T21:24:17.263379+0000","last_became_active":"2024-09-10T21:24:11.747405+0000","last_became_peered":"2024-09-10T21:24:11.747405+0000","last_unstale":"2024-09-10T21:24:17.263379+0000","last_undegraded":"2024-09-10T21:24:17.263379+0000","last_fullsized":"2024-09-10T21:24:17.263379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:42.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264255+0000","last_change":"2024-09-10T21:24:10.711258+0000","last_active":"2024-09-10T21:24:17.264255+0000","last_peered":"2024-09-10T21:24:17.264255+0000","last_clean":"2024-09-10T21:24:17.264255+0000","last_became_active":"2024-09-10T21:24:10.710892+0000","last_became_peered":"2024-09-10T21:24:10.710892+0000","last_unstale":"2024-09-10T21:24:17.264255+0000","last_undegraded":"2024-09-10T21:24:17.264255+0000","last_fullsized":"2024-09-10T21:24:17.264255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:03.592055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769471+0000","last_change":"2024-09-10T21:24:15.786836+0000","last_active":"2024-09-10T21:24:16.769471+0000","last_peered":"2024-09-10T21:24:16.769471+0000","last_clean":"2024-09-10T21:24:16.769471+0000","last_became_active":"2024-09-10T21:24:15.786459+0000","last_became_peered":"2024-09-10T21:24:15.786459+0000","last_unstale":"2024-09-10T21:24:16.769471+0000","last_undegraded":"2024-09-10T21:24:16.769471+0000","last_fullsized":"2024-09-10T21:24:16.769471+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:49.037577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264516+0000","last_change":"2024-09-10T21:24:11.756779+0000","last_active":"2024-09-10T21:24:17.264516+0000","last_peered":"2024-09-10T21:24:17.264516+0000","last_clean":"2024-09-10T21:24:17.264516+0000","last_became_active":"2024-09-10T21:24:11.756493+0000","last_became_peered":"2024-09-10T21:24:11.756493+0000","last_unstale":"2024-09-10T21:24:17.264516+0000","last_undegraded":"2024-09-10T21:24:17.264516+0000","last_fullsized":"2024-09-10T21:24:17.264516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766053+0000","last_change":"2024-09-10T21:24:10.707060+0000","last_active":"2024-09-10T21:24:16.766053+0000","last_peered":"2024-09-10T21:24:16.766053+0000","last_clean":"2024-09-10T21:24:16.766053+0000","last_became_active":"2024-09-10T21:24:10.706798+0000","last_became_peered":"2024-09-10T21:24:10.706798+0000","last_unstale":"2024-09-10T21:24:16.766053+0000","last_undegraded":"2024-09-10T21:24:16.766053+0000","last_fullsized":"2024-09-10T21:24:16.766053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:47.120434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770236+0000","last_change":"2024-09-10T21:24:15.781040+0000","last_active":"2024-09-10T21:24:16.770236+0000","last_peered":"2024-09-10T21:24:16.770236+0000","last_clean":"2024-09-10T21:24:16.770236+0000","last_became_active":"2024-09-10T21:24:15.780677+0000","last_became_peered":"2024-09-10T21:24:15.780677+0000","last_unstale":"2024-09-10T21:24:16.770236+0000","last_undegraded":"2024-09-10T21:24:16.770236+0000","last_fullsized":"2024-09-10T21:24:16.770236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.899193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263779+0000","last_change":"2024-09-10T21:24:11.753935+0000","last_active":"2024-09-10T21:24:17.263779+0000","last_peered":"2024-09-10T21:24:17.263779+0000","last_clean":"2024-09-10T21:24:17.263779+0000","last_became_active":"2024-09-10T21:24:11.753134+0000","last_became_peered":"2024-09-10T21:24:11.753134+0000","last_unstale":"2024-09-10T21:24:17.263779+0000","last_undegraded":"2024-09-10T21:24:17.263779+0000","last_fullsized":"2024-09-10T21:24:17.263779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:41.306076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769295+0000","last_change":"2024-09-10T21:24:10.718908+0000","last_active":"2024-09-10T21:24:16.769295+0000","last_peered":"2024-09-10T21:24:16.769295+0000","last_clean":"2024-09-10T21:24:16.769295+0000","last_became_active":"2024-09-10T21:24:10.718142+0000","last_became_peered":"2024-09-10T21:24:10.718142+0000","last_unstale":"2024-09-10T21:24:16.769295+0000","last_undegraded":"2024-09-10T21:24:16.769295+0000","last_fullsized":"2024-09-10T21:24:16.769295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:56.946441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769872+0000","last_change":"2024-09-10T21:24:15.778921+0000","last_active":"2024-09-10T21:24:16.769872+0000","last_peered":"2024-09-10T21:24:16.769872+0000","last_clean":"2024-09-10T21:24:16.769872+0000","last_became_active":"2024-09-10T21:24:15.778360+0000","last_became_peered":"2024-09-10T21:24:15.778360+0000","last_unstale":"2024-09-10T21:24:16.769872+0000","last_undegraded":"2024-09-10T21:24:16.769872+0000","last_fullsized":"2024-09-10T21:24:16.769872+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:10.163639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768895+0000","last_change":"2024-09-10T21:24:11.758622+0000","last_active":"2024-09-10T21:24:16.768895+0000","last_peered":"2024-09-10T21:24:16.768895+0000","last_clean":"2024-09-10T21:24:16.768895+0000","last_became_active":"2024-09-10T21:24:11.758363+0000","last_became_peered":"2024-09-10T21:24:11.758363+0000","last_unstale":"2024-09-10T21:24:16.768895+0000","last_undegraded":"2024-09-10T21:24:16.768895+0000","last_fullsized":"2024-09-10T21:24:16.768895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765739+0000","last_change":"2024-09-10T21:24:10.720746+0000","last_active":"2024-09-10T21:24:16.765739+0000","last_peered":"2024-09-10T21:24:16.765739+0000","last_clean":"2024-09-10T21:24:16.765739+0000","last_became_active":"2024-09-10T21:24:10.720360+0000","last_became_peered":"2024-09-10T21:24:10.720360+0000","last_unstale":"2024-09-10T21:24:16.765739+0000","last_undegraded":"2024-09-10T21:24:16.765739+0000","last_fullsized":"2024-09-10T21:24:16.765739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:25.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264322+0000","last_change":"2024-09-10T21:24:15.787994+0000","last_active":"2024-09-10T21:24:17.264322+0000","last_peered":"2024-09-10T21:24:17.264322+0000","last_clean":"2024-09-10T21:24:17.264322+0000","last_became_active":"2024-09-10T21:24:15.787532+0000","last_became_peered":"2024-09-10T21:24:15.787532+0000","last_unstale":"2024-09-10T21:24:17.264322+0000","last_undegraded":"2024-09-10T21:24:17.264322+0000","last_fullsized":"2024-09-10T21:24:17.264322+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:12.903923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265574+0000","last_change":"2024-09-10T21:24:11.747694+0000","last_active":"2024-09-10T21:24:17.265574+0000","last_peered":"2024-09-10T21:24:17.265574+0000","last_clean":"2024-09-10T21:24:17.265574+0000","last_became_active":"2024-09-10T21:24:11.746819+0000","last_became_peered":"2024-09-10T21:24:11.746819+0000","last_unstale":"2024-09-10T21:24:17.265574+0000","last_undegraded":"2024-09-10T21:24:17.265574+0000","last_fullsized":"2024-09-10T21:24:17.265574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:14.637932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766139+0000","last_change":"2024-09-10T21:24:10.708595+0000","last_active":"2024-09-10T21:24:16.766139+0000","last_peered":"2024-09-10T21:24:16.766139+0000","last_clean":"2024-09-10T21:24:16.766139+0000","last_became_active":"2024-09-10T21:24:10.708315+0000","last_became_peered":"2024-09-10T21:24:10.708315+0000","last_unstale":"2024-09-10T21:24:16.766139+0000","last_undegraded":"2024-09-10T21:24:16.766139+0000","last_fullsized":"2024-09-10T21:24:16.766139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:39.966992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262962+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:24:17.262962+0000","last_peered":"2024-09-10T21:24:17.262962+0000","last_clean":"2024-09-10T21:24:17.262962+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:24:17.262962+0000","last_undegraded":"2024-09-10T21:24:17.262962+0000","last_fullsized":"2024-09-10T21:24:17.262962+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766659+0000","last_change":"2024-09-10T21:24:15.771747+0000","last_active":"2024-09-10T21:24:16.766659+0000","last_peered":"2024-09-10T21:24:16.766659+0000","last_clean":"2024-09-10T21:24:16.766659+0000","last_became_active":"2024-09-10T21:24:15.771197+0000","last_became_peered":"2024-09-10T21:24:15.771197+0000","last_unstale":"2024-09-10T21:24:16.766659+0000","last_undegraded":"2024-09-10T21:24:16.766659+0000","last_fullsized":"2024-09-10T21:24:16.766659+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:05.770255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766468+0000","last_change":"2024-09-10T21:24:11.740573+0000","last_active":"2024-09-10T21:24:16.766468+0000","last_peered":"2024-09-10T21:24:16.766468+0000","last_clean":"2024-09-10T21:24:16.766468+0000","last_became_active":"2024-09-10T21:24:11.738994+0000","last_became_peered":"2024-09-10T21:24:11.738994+0000","last_unstale":"2024-09-10T21:24:16.766468+0000","last_undegraded":"2024-09-10T21:24:16.766468+0000","last_fullsized":"2024-09-10T21:24:16.766468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:55.184362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770636+0000","last_change":"2024-09-10T21:24:10.721767+0000","last_active":"2024-09-10T21:24:16.770636+0000","last_peered":"2024-09-10T21:24:16.770636+0000","last_clean":"2024-09-10T21:24:16.770636+0000","last_became_active":"2024-09-10T21:24:10.721444+0000","last_became_peered":"2024-09-10T21:24:10.721444+0000","last_unstale":"2024-09-10T21:24:16.770636+0000","last_undegraded":"2024-09-10T21:24:16.770636+0000","last_fullsized":"2024-09-10T21:24:16.770636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:38.215394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263060+0000","last_change":"2024-09-10T21:24:15.770594+0000","last_active":"2024-09-10T21:24:17.263060+0000","last_peered":"2024-09-10T21:24:17.263060+0000","last_clean":"2024-09-10T21:24:17.263060+0000","last_became_active":"2024-09-10T21:24:15.769704+0000","last_became_peered":"2024-09-10T21:24:15.769704+0000","last_unstale":"2024-09-10T21:24:17.263060+0000","last_undegraded":"2024-09-10T21:24:17.263060+0000","last_fullsized":"2024-09-10T21:24:17.263060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:56.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264863+0000","last_change":"2024-09-10T21:24:11.743773+0000","last_active":"2024-09-10T21:24:17.264863+0000","last_peered":"2024-09-10T21:24:17.264863+0000","last_clean":"2024-09-10T21:24:17.264863+0000","last_became_active":"2024-09-10T21:24:11.743497+0000","last_became_peered":"2024-09-10T21:24:11.743497+0000","last_unstale":"2024-09-10T21:24:17.264863+0000","last_undegraded":"2024-09-10T21:24:17.264863+0000","last_fullsized":"2024-09-10T21:24:17.264863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:48.837766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265338+0000","last_change":"2024-09-10T21:24:10.716493+0000","last_active":"2024-09-10T21:24:17.265338+0000","last_peered":"2024-09-10T21:24:17.265338+0000","last_clean":"2024-09-10T21:24:17.265338+0000","last_became_active":"2024-09-10T21:24:10.716126+0000","last_became_peered":"2024-09-10T21:24:10.716126+0000","last_unstale":"2024-09-10T21:24:17.265338+0000","last_undegraded":"2024-09-10T21:24:17.265338+0000","last_fullsized":"2024-09-10T21:24:17.265338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264298+0000","last_change":"2024-09-10T21:24:15.791234+0000","last_active":"2024-09-10T21:24:17.264298+0000","last_peered":"2024-09-10T21:24:17.264298+0000","last_clean":"2024-09-10T21:24:17.264298+0000","last_became_active":"2024-09-10T21:24:15.790862+0000","last_became_peered":"2024-09-10T21:24:15.790862+0000","last_unstale":"2024-09-10T21:24:17.264298+0000","last_undegraded":"2024-09-10T21:24:17.264298+0000","last_fullsized":"2024-09-10T21:24:17.264298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:41.522844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265147+0000","last_change":"2024-09-10T21:24:11.742947+0000","last_active":"2024-09-10T21:24:17.265147+0000","last_peered":"2024-09-10T21:24:17.265147+0000","last_clean":"2024-09-10T21:24:17.265147+0000","last_became_active":"2024-09-10T21:24:11.741967+0000","last_became_peered":"2024-09-10T21:24:11.741967+0000","last_unstale":"2024-09-10T21:24:17.265147+0000","last_undegraded":"2024-09-10T21:24:17.265147+0000","last_fullsized":"2024-09-10T21:24:17.265147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:56.130827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266168+0000","last_change":"2024-09-10T21:24:10.715721+0000","last_active":"2024-09-10T21:24:17.266168+0000","last_peered":"2024-09-10T21:24:17.266168+0000","last_clean":"2024-09-10T21:24:17.266168+0000","last_became_active":"2024-09-10T21:24:10.715385+0000","last_became_peered":"2024-09-10T21:24:10.715385+0000","last_unstale":"2024-09-10T21:24:17.266168+0000","last_undegraded":"2024-09-10T21:24:17.266168+0000","last_fullsized":"2024-09-10T21:24:17.266168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:30.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765435+0000","last_change":"2024-09-10T21:24:15.770313+0000","last_active":"2024-09-10T21:24:16.765435+0000","last_peered":"2024-09-10T21:24:16.765435+0000","last_clean":"2024-09-10T21:24:16.765435+0000","last_became_active":"2024-09-10T21:24:15.769934+0000","last_became_peered":"2024-09-10T21:24:15.769934+0000","last_unstale":"2024-09-10T21:24:16.765435+0000","last_undegraded":"2024-09-10T21:24:16.765435+0000","last_fullsized":"2024-09-10T21:24:16.765435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:58.034220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264104+0000","last_change":"2024-09-10T21:24:11.752428+0000","last_active":"2024-09-10T21:24:17.264104+0000","last_peered":"2024-09-10T21:24:17.264104+0000","last_clean":"2024-09-10T21:24:17.264104+0000","last_became_active":"2024-09-10T21:24:11.748165+0000","last_became_peered":"2024-09-10T21:24:11.748165+0000","last_unstale":"2024-09-10T21:24:17.264104+0000","last_undegraded":"2024-09-10T21:24:17.264104+0000","last_fullsized":"2024-09-10T21:24:17.264104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265190+0000","last_change":"2024-09-10T21:24:10.717653+0000","last_active":"2024-09-10T21:24:17.265190+0000","last_peered":"2024-09-10T21:24:17.265190+0000","last_clean":"2024-09-10T21:24:17.265190+0000","last_became_active":"2024-09-10T21:24:10.715137+0000","last_became_peered":"2024-09-10T21:24:10.715137+0000","last_unstale":"2024-09-10T21:24:17.265190+0000","last_undegraded":"2024-09-10T21:24:17.265190+0000","last_fullsized":"2024-09-10T21:24:17.265190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:01.438989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262702+0000","last_change":"2024-09-10T21:24:15.782725+0000","last_active":"2024-09-10T21:24:17.262702+0000","last_peered":"2024-09-10T21:24:17.262702+0000","last_clean":"2024-09-10T21:24:17.262702+0000","last_became_active":"2024-09-10T21:24:15.782199+0000","last_became_peered":"2024-09-10T21:24:15.782199+0000","last_unstale":"2024-09-10T21:24:17.262702+0000","last_undegraded":"2024-09-10T21:24:17.262702+0000","last_fullsized":"2024-09-10T21:24:17.262702+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:15.226555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765554+0000","last_change":"2024-09-10T21:24:11.757400+0000","last_active":"2024-09-10T21:24:16.765554+0000","last_peered":"2024-09-10T21:24:16.765554+0000","last_clean":"2024-09-10T21:24:16.765554+0000","last_became_active":"2024-09-10T21:24:11.757112+0000","last_became_peered":"2024-09-10T21:24:11.757112+0000","last_unstale":"2024-09-10T21:24:16.765554+0000","last_undegraded":"2024-09-10T21:24:16.765554+0000","last_fullsized":"2024-09-10T21:24:16.765554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263864+0000","last_change":"2024-09-10T21:24:10.701950+0000","last_active":"2024-09-10T21:24:17.263864+0000","last_peered":"2024-09-10T21:24:17.263864+0000","last_clean":"2024-09-10T21:24:17.263864+0000","last_became_active":"2024-09-10T21:24:10.701652+0000","last_became_peered":"2024-09-10T21:24:10.701652+0000","last_unstale":"2024-09-10T21:24:17.263864+0000","last_undegraded":"2024-09-10T21:24:17.263864+0000","last_fullsized":"2024-09-10T21:24:17.263864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.595699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266111+0000","last_change":"2024-09-10T21:24:15.784288+0000","last_active":"2024-09-10T21:24:17.266111+0000","last_peered":"2024-09-10T21:24:17.266111+0000","last_clean":"2024-09-10T21:24:17.266111+0000","last_became_active":"2024-09-10T21:24:15.783890+0000","last_became_peered":"2024-09-10T21:24:15.783890+0000","last_unstale":"2024-09-10T21:24:17.266111+0000","last_undegraded":"2024-09-10T21:24:17.266111+0000","last_fullsized":"2024-09-10T21:24:17.266111+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:59.245790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262943+0000","last_change":"2024-09-10T21:24:11.736914+0000","last_active":"2024-09-10T21:24:17.262943+0000","last_peered":"2024-09-10T21:24:17.262943+0000","last_clean":"2024-09-10T21:24:17.262943+0000","last_became_active":"2024-09-10T21:24:11.734099+0000","last_became_peered":"2024-09-10T21:24:11.734099+0000","last_unstale":"2024-09-10T21:24:17.262943+0000","last_undegraded":"2024-09-10T21:24:17.262943+0000","last_fullsized":"2024-09-10T21:24:17.262943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264000+0000","last_change":"2024-09-10T21:24:10.715530+0000","last_active":"2024-09-10T21:24:17.264000+0000","last_peered":"2024-09-10T21:24:17.264000+0000","last_clean":"2024-09-10T21:24:17.264000+0000","last_became_active":"2024-09-10T21:24:10.715235+0000","last_became_peered":"2024-09-10T21:24:10.715235+0000","last_unstale":"2024-09-10T21:24:17.264000+0000","last_undegraded":"2024-09-10T21:24:17.264000+0000","last_fullsized":"2024-09-10T21:24:17.264000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:12.286087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264900+0000","last_change":"2024-09-10T21:24:15.784572+0000","last_active":"2024-09-10T21:24:17.264900+0000","last_peered":"2024-09-10T21:24:17.264900+0000","last_clean":"2024-09-10T21:24:17.264900+0000","last_became_active":"2024-09-10T21:24:15.784136+0000","last_became_peered":"2024-09-10T21:24:15.784136+0000","last_unstale":"2024-09-10T21:24:17.264900+0000","last_undegraded":"2024-09-10T21:24:17.264900+0000","last_fullsized":"2024-09-10T21:24:17.264900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:12.320754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765877+0000","last_change":"2024-09-10T21:24:11.755458+0000","last_active":"2024-09-10T21:24:16.765877+0000","last_peered":"2024-09-10T21:24:16.765877+0000","last_clean":"2024-09-10T21:24:16.765877+0000","last_became_active":"2024-09-10T21:24:11.755150+0000","last_became_peered":"2024-09-10T21:24:11.755150+0000","last_unstale":"2024-09-10T21:24:16.765877+0000","last_undegraded":"2024-09-10T21:24:16.765877+0000","last_fullsized":"2024-09-10T21:24:16.765877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:34.505560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765994+0000","last_change":"2024-09-10T21:24:10.717709+0000","last_active":"2024-09-10T21:24:16.765994+0000","last_peered":"2024-09-10T21:24:16.765994+0000","last_clean":"2024-09-10T21:24:16.765994+0000","last_became_active":"2024-09-10T21:24:10.717328+0000","last_became_peered":"2024-09-10T21:24:10.717328+0000","last_unstale":"2024-09-10T21:24:16.765994+0000","last_undegraded":"2024-09-10T21:24:16.765994+0000","last_fullsized":"2024-09-10T21:24:16.765994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:37.792691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770443+0000","last_change":"2024-09-10T21:24:15.788432+0000","last_active":"2024-09-10T21:24:16.770443+0000","last_peered":"2024-09-10T21:24:16.770443+0000","last_clean":"2024-09-10T21:24:16.770443+0000","last_became_active":"2024-09-10T21:24:15.788046+0000","last_became_peered":"2024-09-10T21:24:15.788046+0000","last_unstale":"2024-09-10T21:24:16.770443+0000","last_undegraded":"2024-09-10T21:24:16.770443+0000","last_fullsized":"2024-09-10T21:24:16.770443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:04.277488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769322+0000","last_change":"2024-09-10T21:24:11.737027+0000","last_active":"2024-09-10T21:24:16.769322+0000","last_peered":"2024-09-10T21:24:16.769322+0000","last_clean":"2024-09-10T21:24:16.769322+0000","last_became_active":"2024-09-10T21:24:11.736731+0000","last_became_peered":"2024-09-10T21:24:11.736731+0000","last_unstale":"2024-09-10T21:24:16.769322+0000","last_undegraded":"2024-09-10T21:24:16.769322+0000","last_fullsized":"2024-09-10T21:24:16.769322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:17.069524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769804+0000","last_change":"2024-09-10T21:24:10.712131+0000","last_active":"2024-09-10T21:24:16.769804+0000","last_peered":"2024-09-10T21:24:16.769804+0000","last_clean":"2024-09-10T21:24:16.769804+0000","last_became_active":"2024-09-10T21:24:10.711804+0000","last_became_peered":"2024-09-10T21:24:10.711804+0000","last_unstale":"2024-09-10T21:24:16.769804+0000","last_undegraded":"2024-09-10T21:24:16.769804+0000","last_fullsized":"2024-09-10T21:24:16.769804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766399+0000","last_change":"2024-09-10T21:24:15.786086+0000","last_active":"2024-09-10T21:24:16.766399+0000","last_peered":"2024-09-10T21:24:16.766399+0000","last_clean":"2024-09-10T21:24:16.766399+0000","last_became_active":"2024-09-10T21:24:15.785449+0000","last_became_peered":"2024-09-10T21:24:15.785449+0000","last_unstale":"2024-09-10T21:24:16.766399+0000","last_undegraded":"2024-09-10T21:24:16.766399+0000","last_fullsized":"2024-09-10T21:24:16.766399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:51.733023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.740413+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.740134+0000","last_became_peered":"2024-09-10T21:24:11.740134+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:22.624004+0000","last_change":"2024-09-10T21:24:10.707022+0000","last_active":"2024-09-10T21:24:22.624004+0000","last_peered":"2024-09-10T21:24:22.624004+0000","last_clean":"2024-09-10T21:24:22.624004+0000","last_became_active":"2024-09-10T21:24:10.706584+0000","last_became_peered":"2024-09-10T21:24:10.706584+0000","last_unstale":"2024-09-10T21:24:22.624004+0000","last_undegraded":"2024-09-10T21:24:22.624004+0000","last_fullsized":"2024-09-10T21:24:22.624004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:08.525705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265647+0000","last_change":"2024-09-10T21:24:15.789501+0000","last_active":"2024-09-10T21:24:17.265647+0000","last_peered":"2024-09-10T21:24:17.265647+0000","last_clean":"2024-09-10T21:24:17.265647+0000","last_became_active":"2024-09-10T21:24:15.789076+0000","last_became_peered":"2024-09-10T21:24:15.789076+0000","last_unstale":"2024-09-10T21:24:17.265647+0000","last_undegraded":"2024-09-10T21:24:17.265647+0000","last_fullsized":"2024-09-10T21:24:17.265647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:58.299617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766239+0000","last_change":"2024-09-10T21:24:11.757417+0000","last_active":"2024-09-10T21:24:16.766239+0000","last_peered":"2024-09-10T21:24:16.766239+0000","last_clean":"2024-09-10T21:24:16.766239+0000","last_became_active":"2024-09-10T21:24:11.757091+0000","last_became_peered":"2024-09-10T21:24:11.757091+0000","last_unstale":"2024-09-10T21:24:16.766239+0000","last_undegraded":"2024-09-10T21:24:16.766239+0000","last_fullsized":"2024-09-10T21:24:16.766239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:13.080850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770386+0000","last_change":"2024-09-10T21:24:10.707124+0000","last_active":"2024-09-10T21:24:16.770386+0000","last_peered":"2024-09-10T21:24:16.770386+0000","last_clean":"2024-09-10T21:24:16.770386+0000","last_became_active":"2024-09-10T21:24:10.706605+0000","last_became_peered":"2024-09-10T21:24:10.706605+0000","last_unstale":"2024-09-10T21:24:16.770386+0000","last_undegraded":"2024-09-10T21:24:16.770386+0000","last_fullsized":"2024-09-10T21:24:16.770386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:21.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262946+0000","last_change":"2024-09-10T21:24:15.776743+0000","last_active":"2024-09-10T21:24:17.262946+0000","last_peered":"2024-09-10T21:24:17.262946+0000","last_clean":"2024-09-10T21:24:17.262946+0000","last_became_active":"2024-09-10T21:24:15.774992+0000","last_became_peered":"2024-09-10T21:24:15.774992+0000","last_unstale":"2024-09-10T21:24:17.262946+0000","last_undegraded":"2024-09-10T21:24:17.262946+0000","last_fullsized":"2024-09-10T21:24:17.262946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:11.593663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265153+0000","last_change":"2024-09-10T21:24:11.750107+0000","last_active":"2024-09-10T21:24:17.265153+0000","last_peered":"2024-09-10T21:24:17.265153+0000","last_clean":"2024-09-10T21:24:17.265153+0000","last_became_active":"2024-09-10T21:24:11.749206+0000","last_became_peered":"2024-09-10T21:24:11.749206+0000","last_unstale":"2024-09-10T21:24:17.265153+0000","last_undegraded":"2024-09-10T21:24:17.265153+0000","last_fullsized":"2024-09-10T21:24:17.265153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265042+0000","last_change":"2024-09-10T21:24:10.711029+0000","last_active":"2024-09-10T21:24:17.265042+0000","last_peered":"2024-09-10T21:24:17.265042+0000","last_clean":"2024-09-10T21:24:17.265042+0000","last_became_active":"2024-09-10T21:24:10.710789+0000","last_became_peered":"2024-09-10T21:24:10.710789+0000","last_unstale":"2024-09-10T21:24:17.265042+0000","last_undegraded":"2024-09-10T21:24:17.265042+0000","last_fullsized":"2024-09-10T21:24:17.265042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:36.260277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.768675+0000","last_change":"2024-09-10T21:24:15.779017+0000","last_active":"2024-09-10T21:24:16.768675+0000","last_peered":"2024-09-10T21:24:16.768675+0000","last_clean":"2024-09-10T21:24:16.768675+0000","last_became_active":"2024-09-10T21:24:15.774452+0000","last_became_peered":"2024-09-10T21:24:15.774452+0000","last_unstale":"2024-09-10T21:24:16.768675+0000","last_undegraded":"2024-09-10T21:24:16.768675+0000","last_fullsized":"2024-09-10T21:24:16.768675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:56.032301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264021+0000","last_change":"2024-09-10T21:24:11.748901+0000","last_active":"2024-09-10T21:24:17.264021+0000","last_peered":"2024-09-10T21:24:17.264021+0000","last_clean":"2024-09-10T21:24:17.264021+0000","last_became_active":"2024-09-10T21:24:11.748059+0000","last_became_peered":"2024-09-10T21:24:11.748059+0000","last_unstale":"2024-09-10T21:24:17.264021+0000","last_undegraded":"2024-09-10T21:24:17.264021+0000","last_fullsized":"2024-09-10T21:24:17.264021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:49.019122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:10.697344+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:10.697030+0000","last_became_peered":"2024-09-10T21:24:10.697030+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:47.355925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264496+0000","last_change":"2024-09-10T21:24:15.775122+0000","last_active":"2024-09-10T21:24:17.264496+0000","last_peered":"2024-09-10T21:24:17.264496+0000","last_clean":"2024-09-10T21:24:17.264496+0000","last_became_active":"2024-09-10T21:24:15.774561+0000","last_became_peered":"2024-09-10T21:24:15.774561+0000","last_unstale":"2024-09-10T21:24:17.264496+0000","last_undegraded":"2024-09-10T21:24:17.264496+0000","last_fullsized":"2024-09-10T21:24:17.264496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:19.897395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.266159+0000","last_change":"2024-09-10T21:24:15.790852+0000","last_active":"2024-09-10T21:24:17.266159+0000","last_peered":"2024-09-10T21:24:17.266159+0000","last_clean":"2024-09-10T21:24:17.266159+0000","last_became_active":"2024-09-10T21:24:15.786896+0000","last_became_peered":"2024-09-10T21:24:15.786896+0000","last_unstale":"2024-09-10T21:24:17.266159+0000","last_undegraded":"2024-09-10T21:24:17.266159+0000","last_fullsized":"2024-09-10T21:24:17.266159+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:48.900914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264746+0000","last_change":"2024-09-10T21:24:11.732564+0000","last_active":"2024-09-10T21:24:17.264746+0000","last_peered":"2024-09-10T21:24:17.264746+0000","last_clean":"2024-09-10T21:24:17.264746+0000","last_became_active":"2024-09-10T21:24:11.732106+0000","last_became_peered":"2024-09-10T21:24:11.732106+0000","last_unstale":"2024-09-10T21:24:17.264746+0000","last_undegraded":"2024-09-10T21:24:17.264746+0000","last_fullsized":"2024-09-10T21:24:17.264746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:28.574642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765650+0000","last_change":"2024-09-10T21:24:10.714693+0000","last_active":"2024-09-10T21:24:16.765650+0000","last_peered":"2024-09-10T21:24:16.765650+0000","last_clean":"2024-09-10T21:24:16.765650+0000","last_became_active":"2024-09-10T21:24:10.714223+0000","last_became_peered":"2024-09-10T21:24:10.714223+0000","last_unstale":"2024-09-10T21:24:16.765650+0000","last_undegraded":"2024-09-10T21:24:16.765650+0000","last_fullsized":"2024-09-10T21:24:16.765650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:06.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766392+0000","last_change":"2024-09-10T21:24:15.784105+0000","last_active":"2024-09-10T21:24:16.766392+0000","last_peered":"2024-09-10T21:24:16.766392+0000","last_clean":"2024-09-10T21:24:16.766392+0000","last_became_active":"2024-09-10T21:24:15.783500+0000","last_became_peered":"2024-09-10T21:24:15.783500+0000","last_unstale":"2024-09-10T21:24:16.766392+0000","last_undegraded":"2024-09-10T21:24:16.766392+0000","last_fullsized":"2024-09-10T21:24:16.766392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.513517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263956+0000","last_change":"2024-09-10T21:24:11.738285+0000","last_active":"2024-09-10T21:24:17.263956+0000","last_peered":"2024-09-10T21:24:17.263956+0000","last_clean":"2024-09-10T21:24:17.263956+0000","last_became_active":"2024-09-10T21:24:11.738001+0000","last_became_peered":"2024-09-10T21:24:11.738001+0000","last_unstale":"2024-09-10T21:24:17.263956+0000","last_undegraded":"2024-09-10T21:24:17.263956+0000","last_fullsized":"2024-09-10T21:24:17.263956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769007+0000","last_change":"2024-09-10T21:24:10.710823+0000","last_active":"2024-09-10T21:24:16.769007+0000","last_peered":"2024-09-10T21:24:16.769007+0000","last_clean":"2024-09-10T21:24:16.769007+0000","last_became_active":"2024-09-10T21:24:10.710231+0000","last_became_peered":"2024-09-10T21:24:10.710231+0000","last_unstale":"2024-09-10T21:24:16.769007+0000","last_undegraded":"2024-09-10T21:24:16.769007+0000","last_fullsized":"2024-09-10T21:24:16.769007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:39.865360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765929+0000","last_change":"2024-09-10T21:24:10.711002+0000","last_active":"2024-09-10T21:24:16.765929+0000","last_peered":"2024-09-10T21:24:16.765929+0000","last_clean":"2024-09-10T21:24:16.765929+0000","last_became_active":"2024-09-10T21:24:10.710723+0000","last_became_peered":"2024-09-10T21:24:10.710723+0000","last_unstale":"2024-09-10T21:24:16.765929+0000","last_undegraded":"2024-09-10T21:24:16.765929+0000","last_fullsized":"2024-09-10T21:24:16.765929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:52.955941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264580+0000","last_change":"2024-09-10T21:24:11.744353+0000","last_active":"2024-09-10T21:24:17.264580+0000","last_peered":"2024-09-10T21:24:17.264580+0000","last_clean":"2024-09-10T21:24:17.264580+0000","last_became_active":"2024-09-10T21:24:11.743970+0000","last_became_peered":"2024-09-10T21:24:11.743970+0000","last_unstale":"2024-09-10T21:24:17.264580+0000","last_undegraded":"2024-09-10T21:24:17.264580+0000","last_fullsized":"2024-09-10T21:24:17.264580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:35.066178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265974+0000","last_change":"2024-09-10T21:24:15.791043+0000","last_active":"2024-09-10T21:24:17.265974+0000","last_peered":"2024-09-10T21:24:17.265974+0000","last_clean":"2024-09-10T21:24:17.265974+0000","last_became_active":"2024-09-10T21:24:15.790322+0000","last_became_peered":"2024-09-10T21:24:15.790322+0000","last_unstale":"2024-09-10T21:24:17.265974+0000","last_undegraded":"2024-09-10T21:24:17.265974+0000","last_fullsized":"2024-09-10T21:24:17.265974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:18.317248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264568+0000","last_change":"2024-09-10T21:24:10.717526+0000","last_active":"2024-09-10T21:24:17.264568+0000","last_peered":"2024-09-10T21:24:17.264568+0000","last_clean":"2024-09-10T21:24:17.264568+0000","last_became_active":"2024-09-10T21:24:10.715368+0000","last_became_peered":"2024-09-10T21:24:10.715368+0000","last_unstale":"2024-09-10T21:24:17.264568+0000","last_undegraded":"2024-09-10T21:24:17.264568+0000","last_fullsized":"2024-09-10T21:24:17.264568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:09.676519+0000","last_clean_scrub_stamp":"2024-09-10T21:24:09.676519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:47.117956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263197+0000","last_change":"2024-09-10T21:24:11.733617+0000","last_active":"2024-09-10T21:24:17.263197+0000","last_peered":"2024-09-10T21:24:17.263197+0000","last_clean":"2024-09-10T21:24:17.263197+0000","last_became_active":"2024-09-10T21:24:11.733343+0000","last_became_peered":"2024-09-10T21:24:11.733343+0000","last_unstale":"2024-09-10T21:24:17.263197+0000","last_undegraded":"2024-09-10T21:24:17.263197+0000","last_fullsized":"2024-09-10T21:24:17.263197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770202+0000","last_change":"2024-09-10T21:24:15.787912+0000","last_active":"2024-09-10T21:24:16.770202+0000","last_peered":"2024-09-10T21:24:16.770202+0000","last_clean":"2024-09-10T21:24:16.770202+0000","last_became_active":"2024-09-10T21:24:15.787441+0000","last_became_peered":"2024-09-10T21:24:15.787441+0000","last_unstale":"2024-09-10T21:24:16.770202+0000","last_undegraded":"2024-09-10T21:24:16.770202+0000","last_fullsized":"2024-09-10T21:24:16.770202+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:14.740475+0000","last_clean_scrub_stamp":"2024-09-10T21:24:14.740475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:22.856812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.262658+0000","last_change":"2024-09-10T21:24:11.747086+0000","last_active":"2024-09-10T21:24:17.262658+0000","last_peered":"2024-09-10T21:24:17.262658+0000","last_clean":"2024-09-10T21:24:17.262658+0000","last_became_active":"2024-09-10T21:24:11.746780+0000","last_became_peered":"2024-09-10T21:24:11.746780+0000","last_unstale":"2024-09-10T21:24:17.262658+0000","last_undegraded":"2024-09-10T21:24:17.262658+0000","last_fullsized":"2024-09-10T21:24:17.262658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:13.288615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769236+0000","last_change":"2024-09-10T21:24:11.732475+0000","last_active":"2024-09-10T21:24:16.769236+0000","last_peered":"2024-09-10T21:24:16.769236+0000","last_clean":"2024-09-10T21:24:16.769236+0000","last_became_active":"2024-09-10T21:24:11.731007+0000","last_became_peered":"2024-09-10T21:24:11.731007+0000","last_unstale":"2024-09-10T21:24:16.769236+0000","last_undegraded":"2024-09-10T21:24:16.769236+0000","last_fullsized":"2024-09-10T21:24:16.769236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765707+0000","last_change":"2024-09-10T21:24:11.744942+0000","last_active":"2024-09-10T21:24:16.765707+0000","last_peered":"2024-09-10T21:24:16.765707+0000","last_clean":"2024-09-10T21:24:16.765707+0000","last_became_active":"2024-09-10T21:24:11.744626+0000","last_became_peered":"2024-09-10T21:24:11.744626+0000","last_unstale":"2024-09-10T21:24:16.765707+0000","last_undegraded":"2024-09-10T21:24:16.765707+0000","last_fullsized":"2024-09-10T21:24:16.765707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:37.461417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264525+0000","last_change":"2024-09-10T21:24:11.745907+0000","last_active":"2024-09-10T21:24:17.264525+0000","last_peered":"2024-09-10T21:24:17.264525+0000","last_clean":"2024-09-10T21:24:17.264525+0000","last_became_active":"2024-09-10T21:24:11.745583+0000","last_became_peered":"2024-09-10T21:24:11.745583+0000","last_unstale":"2024-09-10T21:24:17.264525+0000","last_undegraded":"2024-09-10T21:24:17.264525+0000","last_fullsized":"2024-09-10T21:24:17.264525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:50.845294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770469+0000","last_change":"2024-09-10T21:24:11.756001+0000","last_active":"2024-09-10T21:24:16.770469+0000","last_peered":"2024-09-10T21:24:16.770469+0000","last_clean":"2024-09-10T21:24:16.770469+0000","last_became_active":"2024-09-10T21:24:11.755719+0000","last_became_peered":"2024-09-10T21:24:11.755719+0000","last_unstale":"2024-09-10T21:24:16.770469+0000","last_undegraded":"2024-09-10T21:24:16.770469+0000","last_fullsized":"2024-09-10T21:24:16.770469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:04.237158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769805+0000","last_change":"2024-09-10T21:24:11.740025+0000","last_active":"2024-09-10T21:24:16.769805+0000","last_peered":"2024-09-10T21:24:16.769805+0000","last_clean":"2024-09-10T21:24:16.769805+0000","last_became_active":"2024-09-10T21:24:11.739742+0000","last_became_peered":"2024-09-10T21:24:11.739742+0000","last_unstale":"2024-09-10T21:24:16.769805+0000","last_undegraded":"2024-09-10T21:24:16.769805+0000","last_fullsized":"2024-09-10T21:24:16.769805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:29.650259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263932+0000","last_change":"2024-09-10T21:24:11.749805+0000","last_active":"2024-09-10T21:24:17.263932+0000","last_peered":"2024-09-10T21:24:17.263932+0000","last_clean":"2024-09-10T21:24:17.263932+0000","last_became_active":"2024-09-10T21:24:11.748452+0000","last_became_peered":"2024-09-10T21:24:11.748452+0000","last_unstale":"2024-09-10T21:24:17.263932+0000","last_undegraded":"2024-09-10T21:24:17.263932+0000","last_fullsized":"2024-09-10T21:24:17.263932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:36.508158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264740+0000","last_change":"2024-09-10T21:24:11.750687+0000","last_active":"2024-09-10T21:24:17.264740+0000","last_peered":"2024-09-10T21:24:17.264740+0000","last_clean":"2024-09-10T21:24:17.264740+0000","last_became_active":"2024-09-10T21:24:11.749899+0000","last_became_peered":"2024-09-10T21:24:11.749899+0000","last_unstale":"2024-09-10T21:24:17.264740+0000","last_undegraded":"2024-09-10T21:24:17.264740+0000","last_fullsized":"2024-09-10T21:24:17.264740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765619+0000","last_change":"2024-09-10T21:24:11.734778+0000","last_active":"2024-09-10T21:24:16.765619+0000","last_peered":"2024-09-10T21:24:16.765619+0000","last_clean":"2024-09-10T21:24:16.765619+0000","last_became_active":"2024-09-10T21:24:11.734510+0000","last_became_peered":"2024-09-10T21:24:11.734510+0000","last_unstale":"2024-09-10T21:24:16.765619+0000","last_undegraded":"2024-09-10T21:24:16.765619+0000","last_fullsized":"2024-09-10T21:24:16.765619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:45.729068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263928+0000","last_change":"2024-09-10T21:24:11.758730+0000","last_active":"2024-09-10T21:24:17.263928+0000","last_peered":"2024-09-10T21:24:17.263928+0000","last_clean":"2024-09-10T21:24:17.263928+0000","last_became_active":"2024-09-10T21:24:11.758453+0000","last_became_peered":"2024-09-10T21:24:11.758453+0000","last_unstale":"2024-09-10T21:24:17.263928+0000","last_undegraded":"2024-09-10T21:24:17.263928+0000","last_fullsized":"2024-09-10T21:24:17.263928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:20.045933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263026+0000","last_change":"2024-09-10T21:24:11.727975+0000","last_active":"2024-09-10T21:24:17.263026+0000","last_peered":"2024-09-10T21:24:17.263026+0000","last_clean":"2024-09-10T21:24:17.263026+0000","last_became_active":"2024-09-10T21:24:11.727715+0000","last_became_peered":"2024-09-10T21:24:11.727715+0000","last_unstale":"2024-09-10T21:24:17.263026+0000","last_undegraded":"2024-09-10T21:24:17.263026+0000","last_fullsized":"2024-09-10T21:24:17.263026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:40.969520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264447+0000","last_change":"2024-09-10T21:24:11.752499+0000","last_active":"2024-09-10T21:24:17.264447+0000","last_peered":"2024-09-10T21:24:17.264447+0000","last_clean":"2024-09-10T21:24:17.264447+0000","last_became_active":"2024-09-10T21:24:11.748931+0000","last_became_peered":"2024-09-10T21:24:11.748931+0000","last_unstale":"2024-09-10T21:24:17.264447+0000","last_undegraded":"2024-09-10T21:24:17.264447+0000","last_fullsized":"2024-09-10T21:24:17.264447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:48.334318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264810+0000","last_change":"2024-09-10T21:24:11.742340+0000","last_active":"2024-09-10T21:24:17.264810+0000","last_peered":"2024-09-10T21:24:17.264810+0000","last_clean":"2024-09-10T21:24:17.264810+0000","last_became_active":"2024-09-10T21:24:11.741702+0000","last_became_peered":"2024-09-10T21:24:11.741702+0000","last_unstale":"2024-09-10T21:24:17.264810+0000","last_undegraded":"2024-09-10T21:24:17.264810+0000","last_fullsized":"2024-09-10T21:24:17.264810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766584+0000","last_change":"2024-09-10T21:24:11.744751+0000","last_active":"2024-09-10T21:24:16.766584+0000","last_peered":"2024-09-10T21:24:16.766584+0000","last_clean":"2024-09-10T21:24:16.766584+0000","last_became_active":"2024-09-10T21:24:11.744431+0000","last_became_peered":"2024-09-10T21:24:11.744431+0000","last_unstale":"2024-09-10T21:24:16.766584+0000","last_undegraded":"2024-09-10T21:24:16.766584+0000","last_fullsized":"2024-09-10T21:24:16.766584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:11.693990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770590+0000","last_change":"2024-09-10T21:24:11.748605+0000","last_active":"2024-09-10T21:24:16.770590+0000","last_peered":"2024-09-10T21:24:16.770590+0000","last_clean":"2024-09-10T21:24:16.770590+0000","last_became_active":"2024-09-10T21:24:11.748279+0000","last_became_peered":"2024-09-10T21:24:11.748279+0000","last_unstale":"2024-09-10T21:24:16.770590+0000","last_undegraded":"2024-09-10T21:24:16.770590+0000","last_fullsized":"2024-09-10T21:24:16.770590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.766580+0000","last_change":"2024-09-10T21:24:11.750748+0000","last_active":"2024-09-10T21:24:16.766580+0000","last_peered":"2024-09-10T21:24:16.766580+0000","last_clean":"2024-09-10T21:24:16.766580+0000","last_became_active":"2024-09-10T21:24:11.750317+0000","last_became_peered":"2024-09-10T21:24:11.750317+0000","last_unstale":"2024-09-10T21:24:16.766580+0000","last_undegraded":"2024-09-10T21:24:16.766580+0000","last_fullsized":"2024-09-10T21:24:16.766580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:08.259327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.765529+0000","last_change":"2024-09-10T21:24:11.757620+0000","last_active":"2024-09-10T21:24:16.765529+0000","last_peered":"2024-09-10T21:24:16.765529+0000","last_clean":"2024-09-10T21:24:16.765529+0000","last_became_active":"2024-09-10T21:24:11.757088+0000","last_became_peered":"2024-09-10T21:24:11.757088+0000","last_unstale":"2024-09-10T21:24:16.765529+0000","last_undegraded":"2024-09-10T21:24:16.765529+0000","last_fullsized":"2024-09-10T21:24:16.765529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:00.216061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263812+0000","last_change":"2024-09-10T21:24:11.745932+0000","last_active":"2024-09-10T21:24:17.263812+0000","last_peered":"2024-09-10T21:24:17.263812+0000","last_clean":"2024-09-10T21:24:17.263812+0000","last_became_active":"2024-09-10T21:24:11.745682+0000","last_became_peered":"2024-09-10T21:24:11.745682+0000","last_unstale":"2024-09-10T21:24:17.263812+0000","last_undegraded":"2024-09-10T21:24:17.263812+0000","last_fullsized":"2024-09-10T21:24:17.263812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.769215+0000","last_change":"2024-09-10T21:24:11.740847+0000","last_active":"2024-09-10T21:24:16.769215+0000","last_peered":"2024-09-10T21:24:16.769215+0000","last_clean":"2024-09-10T21:24:16.769215+0000","last_became_active":"2024-09-10T21:24:11.740548+0000","last_became_peered":"2024-09-10T21:24:11.740548+0000","last_unstale":"2024-09-10T21:24:16.769215+0000","last_undegraded":"2024-09-10T21:24:16.769215+0000","last_fullsized":"2024-09-10T21:24:16.769215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.076056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:16.770304+0000","last_change":"2024-09-10T21:24:11.742902+0000","last_active":"2024-09-10T21:24:16.770304+0000","last_peered":"2024-09-10T21:24:16.770304+0000","last_clean":"2024-09-10T21:24:16.770304+0000","last_became_active":"2024-09-10T21:24:11.742449+0000","last_became_peered":"2024-09-10T21:24:11.742449+0000","last_unstale":"2024-09-10T21:24:16.770304+0000","last_undegraded":"2024-09-10T21:24:16.770304+0000","last_fullsized":"2024-09-10T21:24:16.770304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:25.302008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264763+0000","last_change":"2024-09-10T21:24:11.742660+0000","last_active":"2024-09-10T21:24:17.264763+0000","last_peered":"2024-09-10T21:24:17.264763+0000","last_clean":"2024-09-10T21:24:17.264763+0000","last_became_active":"2024-09-10T21:24:11.742069+0000","last_became_peered":"2024-09-10T21:24:11.742069+0000","last_unstale":"2024-09-10T21:24:17.264763+0000","last_undegraded":"2024-09-10T21:24:17.264763+0000","last_fullsized":"2024-09-10T21:24:17.264763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:35.526539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264897+0000","last_change":"2024-09-10T21:24:11.743498+0000","last_active":"2024-09-10T21:24:17.264897+0000","last_peered":"2024-09-10T21:24:17.264897+0000","last_clean":"2024-09-10T21:24:17.264897+0000","last_became_active":"2024-09-10T21:24:11.743017+0000","last_became_peered":"2024-09-10T21:24:11.743017+0000","last_unstale":"2024-09-10T21:24:17.264897+0000","last_undegraded":"2024-09-10T21:24:17.264897+0000","last_fullsized":"2024-09-10T21:24:17.264897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:46.223570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264934+0000","last_change":"2024-09-10T21:24:11.754599+0000","last_active":"2024-09-10T21:24:17.264934+0000","last_peered":"2024-09-10T21:24:17.264934+0000","last_clean":"2024-09-10T21:24:17.264934+0000","last_became_active":"2024-09-10T21:24:11.754341+0000","last_became_peered":"2024-09-10T21:24:11.754341+0000","last_unstale":"2024-09-10T21:24:17.264934+0000","last_undegraded":"2024-09-10T21:24:17.264934+0000","last_fullsized":"2024-09-10T21:24:17.264934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:38.694729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264501+0000","last_change":"2024-09-10T21:24:11.726485+0000","last_active":"2024-09-10T21:24:17.264501+0000","last_peered":"2024-09-10T21:24:17.264501+0000","last_clean":"2024-09-10T21:24:17.264501+0000","last_became_active":"2024-09-10T21:24:11.726157+0000","last_became_peered":"2024-09-10T21:24:11.726157+0000","last_unstale":"2024-09-10T21:24:17.264501+0000","last_undegraded":"2024-09-10T21:24:17.264501+0000","last_fullsized":"2024-09-10T21:24:17.264501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:38.795219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263664+0000","last_change":"2024-09-10T21:24:11.754793+0000","last_active":"2024-09-10T21:24:17.263664+0000","last_peered":"2024-09-10T21:24:17.263664+0000","last_clean":"2024-09-10T21:24:17.263664+0000","last_became_active":"2024-09-10T21:24:11.754372+0000","last_became_peered":"2024-09-10T21:24:11.754372+0000","last_unstale":"2024-09-10T21:24:17.263664+0000","last_undegraded":"2024-09-10T21:24:17.263664+0000","last_fullsized":"2024-09-10T21:24:17.263664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:25.692729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263914+0000","last_change":"2024-09-10T21:24:11.741830+0000","last_active":"2024-09-10T21:24:17.263914+0000","last_peered":"2024-09-10T21:24:17.263914+0000","last_clean":"2024-09-10T21:24:17.263914+0000","last_became_active":"2024-09-10T21:24:11.741543+0000","last_became_peered":"2024-09-10T21:24:11.741543+0000","last_unstale":"2024-09-10T21:24:17.263914+0000","last_undegraded":"2024-09-10T21:24:17.263914+0000","last_fullsized":"2024-09-10T21:24:17.263914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:48.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263114+0000","last_change":"2024-09-10T21:24:11.744958+0000","last_active":"2024-09-10T21:24:17.263114+0000","last_peered":"2024-09-10T21:24:17.263114+0000","last_clean":"2024-09-10T21:24:17.263114+0000","last_became_active":"2024-09-10T21:24:11.744611+0000","last_became_peered":"2024-09-10T21:24:11.744611+0000","last_unstale":"2024-09-10T21:24:17.263114+0000","last_undegraded":"2024-09-10T21:24:17.263114+0000","last_fullsized":"2024-09-10T21:24:17.263114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:02.441517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264460+0000","last_change":"2024-09-10T21:24:11.742259+0000","last_active":"2024-09-10T21:24:17.264460+0000","last_peered":"2024-09-10T21:24:17.264460+0000","last_clean":"2024-09-10T21:24:17.264460+0000","last_became_active":"2024-09-10T21:24:11.742010+0000","last_became_peered":"2024-09-10T21:24:11.742010+0000","last_unstale":"2024-09-10T21:24:17.264460+0000","last_undegraded":"2024-09-10T21:24:17.264460+0000","last_fullsized":"2024-09-10T21:24:17.264460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:44.976150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.263425+0000","last_change":"2024-09-10T21:24:11.747125+0000","last_active":"2024-09-10T21:24:17.263425+0000","last_peered":"2024-09-10T21:24:17.263425+0000","last_clean":"2024-09-10T21:24:17.263425+0000","last_became_active":"2024-09-10T21:24:11.746844+0000","last_became_peered":"2024-09-10T21:24:11.746844+0000","last_unstale":"2024-09-10T21:24:17.263425+0000","last_undegraded":"2024-09-10T21:24:17.263425+0000","last_fullsized":"2024-09-10T21:24:17.263425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:28.995110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265138+0000","last_change":"2024-09-10T21:24:11.737282+0000","last_active":"2024-09-10T21:24:17.265138+0000","last_peered":"2024-09-10T21:24:17.265138+0000","last_clean":"2024-09-10T21:24:17.265138+0000","last_became_active":"2024-09-10T21:24:11.736985+0000","last_became_peered":"2024-09-10T21:24:11.736985+0000","last_unstale":"2024-09-10T21:24:17.265138+0000","last_undegraded":"2024-09-10T21:24:17.265138+0000","last_fullsized":"2024-09-10T21:24:17.265138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:06.102212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.264431+0000","last_change":"2024-09-10T21:24:11.738254+0000","last_active":"2024-09-10T21:24:17.264431+0000","last_peered":"2024-09-10T21:24:17.264431+0000","last_clean":"2024-09-10T21:24:17.264431+0000","last_became_active":"2024-09-10T21:24:11.737963+0000","last_became_peered":"2024-09-10T21:24:11.737963+0000","last_unstale":"2024-09-10T21:24:17.264431+0000","last_undegraded":"2024-09-10T21:24:17.264431+0000","last_fullsized":"2024-09-10T21:24:17.264431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:34.124736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-10T21:24:17.265077+0000","last_change":"2024-09-10T21:24:11.743284+0000","last_active":"2024-09-10T21:24:17.265077+0000","last_peered":"2024-09-10T21:24:17.265077+0000","last_clean":"2024-09-10T21:24:17.265077+0000","last_became_active":"2024-09-10T21:24:11.742655+0000","last_became_peered":"2024-09-10T21:24:11.742655+0000","last_unstale":"2024-09-10T21:24:17.265077+0000","last_undegraded":"2024-09-10T21:24:17.265077+0000","last_fullsized":"2024-09-10T21:24:17.265077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:10.679048+0000","last_clean_scrub_stamp":"2024-09-10T21:24:10.679048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:55.714322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:24:38.363 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:24:38.363 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T21:24:38.363 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T21:24:38.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:24:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- 172.21.15.97:0/83896953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ecc003c70 msgr2=0x7f6ecc004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 --2- 172.21.15.97:0/83896953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ecc003c70 0x7f6ecc004070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800c490 tx=0x7f6ed801d180 comp rx=0 tx=0).stop 2024-09-10T21:24:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eed200640 1 -- 172.21.15.97:0/83896953 reap_dead start 2024-09-10T21:24:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- 172.21.15.97:0/83896953 shutdown_connections 2024-09-10T21:24:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- 172.21.15.97:0/83896953 >> 172.21.15.97:0/83896953 conn(0x7f6ee80d1a60 msgr2=0x7f6ee80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- 172.21.15.97:0/83896953 shutdown_connections 2024-09-10T21:24:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- 172.21.15.97:0/83896953 wait complete. 2024-09-10T21:24:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 Processor -- start 2024-09-10T21:24:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- start start 2024-09-10T21:24:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ecc003c70 0x7f6ee80b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee80b2f10 0x7f6ee8161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee80716c0 0x7f6ee8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee8167b40 con 0x7f6ee80b2f10 2024-09-10T21:24:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee81679c0 con 0x7f6ee80716c0 2024-09-10T21:24:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6eedc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee8167cc0 con 0x7f6ecc003c70 2024-09-10T21:24:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6ee7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee80b2f10 0x7f6ee8161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6ee7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee80b2f10 0x7f6ee8161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59170/0 (socket says 172.21.15.97:59170) 2024-09-10T21:24:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.540+0000 7f6ee7400640 1 -- 172.21.15.97:0/912079155 learned_addr learned my addr 172.21.15.97:0/912079155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6ee7e00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ecc003c70 0x7f6ee80b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6eed200640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee80716c0 0x7f6ee8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6ee7400640 1 -- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ecc003c70 msgr2=0x7f6ee80b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6ee7400640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ecc003c70 0x7f6ee80b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6ee7400640 1 -- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee80716c0 msgr2=0x7f6ee8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6ee7400640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee80716c0 0x7f6ee8071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6ee7400640 1 -- 172.21.15.97:0/912079155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81f8780 con 0x7f6ee80b2f10 2024-09-10T21:24:38.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6ee7400640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee80b2f10 0x7f6ee8161200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4011ae0 tx=0x7f6ed4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:38.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6edfe00640 1 -- 172.21.15.97:0/912079155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed401e070 con 0x7f6ee80b2f10 2024-09-10T21:24:38.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6edfe00640 1 -- 172.21.15.97:0/912079155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ed400fc10 con 0x7f6ee80b2f10 2024-09-10T21:24:38.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6edfe00640 1 -- 172.21.15.97:0/912079155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4019410 con 0x7f6ee80b2f10 2024-09-10T21:24:38.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee81f7c00 con 0x7f6ee80b2f10 2024-09-10T21:24:38.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee81f81c0 con 0x7f6ee80b2f10 2024-09-10T21:24:38.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee80ae930 con 0x7f6ee80b2f10 2024-09-10T21:24:38.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6edfe00640 1 -- 172.21.15.97:0/912079155 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6ed40195b0 con 0x7f6ee80b2f10 2024-09-10T21:24:38.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.544+0000 7f6edfe00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ec80725e0 0x7f6ec8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.548+0000 7f6edfe00640 1 -- 172.21.15.97:0/912079155 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6ed4094100 con 0x7f6ee80b2f10 2024-09-10T21:24:38.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.548+0000 7f6ee7e00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ec80725e0 0x7f6ec8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.548+0000 7f6ee7e00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ec80725e0 0x7f6ec8074aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80ac000 tx=0x7f6ed8019fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:38.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.548+0000 7f6edfe00640 1 -- 172.21.15.97:0/912079155 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6ed405edc0 con 0x7f6ee80b2f10 2024-09-10T21:24:38.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:38.573 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6ee8161740 con 0x7f6ee80b2f10 2024-09-10T21:24:38.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6edfe00640 1 -- 172.21.15.97:0/912079155 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6ed40637f0 con 0x7f6ee80b2f10 2024-09-10T21:24:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:38.758 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:24:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ec80725e0 msgr2=0x7f6ec8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ec80725e0 0x7f6ec8074aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80ac000 tx=0x7f6ed8019fc0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee80b2f10 msgr2=0x7f6ee8161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee80b2f10 0x7f6ee8161200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4011ae0 tx=0x7f6ed4009ec0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 shutdown_connections 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ec80725e0 0x7f6ec8074aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee80716c0 0x7f6ee8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee80b2f10 0x7f6ee8161200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 --2- 172.21.15.97:0/912079155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ecc003c70 0x7f6ee80b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 >> 172.21.15.97:0/912079155 conn(0x7f6ee80d1a60 msgr2=0x7f6ee815c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 shutdown_connections 2024-09-10T21:24:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.752+0000 7f6eedc00640 1 -- 172.21.15.97:0/912079155 wait complete. 2024-09-10T21:24:38.780 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T21:24:38.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 -- 172.21.15.97:0/1827067651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 msgr2=0x7faaec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 --2- 172.21.15.97:0/1827067651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7faaec004c90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ee0 tx=0x7faaf001d5f0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 -- 172.21.15.97:0/1827067651 shutdown_connections 2024-09-10T21:24:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 --2- 172.21.15.97:0/1827067651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7faaec004c90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 -- 172.21.15.97:0/1827067651 >> 172.21.15.97:0/1827067651 conn(0x7fab080b37c0 msgr2=0x7fab080b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 -- 172.21.15.97:0/1827067651 shutdown_connections 2024-09-10T21:24:38.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 -- 172.21.15.97:0/1827067651 wait complete. 2024-09-10T21:24:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.916+0000 7fab0d400640 1 Processor -- start 2024-09-10T21:24:38.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 -- start start 2024-09-10T21:24:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7fab080cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab0809f760 0x7fab080cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080c7250 0x7fab080c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab080ce1a0 con 0x7faaec004810 2024-09-10T21:24:38.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab080cde60 con 0x7fab080c7250 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fab080ce320 con 0x7fab0809f760 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7fab080cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7fab080cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59190/0 (socket says 172.21.15.97:59190) 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 -- 172.21.15.97:0/2484488225 learned_addr learned my addr 172.21.15.97:0/2484488225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7faafde00640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab0809f760 0x7fab080cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab07200640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080c7250 0x7fab080c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 -- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab0809f760 msgr2=0x7fab080cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab0809f760 0x7fab080cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 -- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080c7250 msgr2=0x7fab080c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080c7250 0x7fab080c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 -- 172.21.15.97:0/2484488225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0814ee20 con 0x7faaec004810 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab07200640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080c7250 0x7fab080c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab06800640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7fab080cb3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faaf8006680 tx=0x7faaf80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7faaff400640 1 -- 172.21.15.97:0/2484488225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8015070 con 0x7faaec004810 2024-09-10T21:24:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7faaff400640 1 -- 172.21.15.97:0/2484488225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf8011900 con 0x7faaec004810 2024-09-10T21:24:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7faaff400640 1 -- 172.21.15.97:0/2484488225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011d30 con 0x7faaec004810 2024-09-10T21:24:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab080c8010 con 0x7faaec004810 2024-09-10T21:24:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0814f0e0 con 0x7faaec004810 2024-09-10T21:24:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.920+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab080cfb10 con 0x7faaec004810 2024-09-10T21:24:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.924+0000 7faaff400640 1 -- 172.21.15.97:0/2484488225 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faaf8010420 con 0x7faaec004810 2024-09-10T21:24:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.924+0000 7faaff400640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faae4072610 0x7faae4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.924+0000 7faaff400640 1 -- 172.21.15.97:0/2484488225 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7faaf80944b0 con 0x7faaec004810 2024-09-10T21:24:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.924+0000 7faafde00640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faae4072610 0x7faae4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.924+0000 7faafde00640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faae4072610 0x7faae4074ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faaf00af000 tx=0x7faaf001a390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:38.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:38.928+0000 7faaff400640 1 -- 172.21.15.97:0/2484488225 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faaf805f1f0 con 0x7faaec004810 2024-09-10T21:24:39.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.124+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab080ccd30 con 0x7faaec004810 2024-09-10T21:24:39.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.124+0000 7faaff400640 1 -- 172.21.15.97:0/2484488225 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7faaf8063c20 con 0x7faaec004810 2024-09-10T21:24:39.131 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:39.131 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:39.132 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faae4072610 msgr2=0x7faae4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faae4072610 0x7faae4074ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faaf00af000 tx=0x7faaf001a390 comp rx=0 tx=0).stop 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 msgr2=0x7fab080cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7fab080cb3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faaf8006680 tx=0x7faaf80039d0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 shutdown_connections 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faae4072610 0x7faae4074ad0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080c7250 0x7fab080c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab0809f760 0x7fab080cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 --2- 172.21.15.97:0/2484488225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faaec004810 0x7fab080cb3c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 >> 172.21.15.97:0/2484488225 conn(0x7fab080b37c0 msgr2=0x7fab080cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 shutdown_connections 2024-09-10T21:24:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.128+0000 7fab0d400640 1 -- 172.21.15.97:0/2484488225 wait complete. 2024-09-10T21:24:39.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 -- 172.21.15.97:0/375530385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 msgr2=0x7fa5e40700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 --2- 172.21.15.97:0/375530385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e40700b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b0740 tx=0x7fa5dc01c6e0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 -- 172.21.15.97:0/375530385 shutdown_connections 2024-09-10T21:24:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 --2- 172.21.15.97:0/375530385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e40700b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 --2- 172.21.15.97:0/375530385 >> v2:172.21.15.194:3300/0 conn(0x7fa5e415ed50 0x7fa5e40b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 -- 172.21.15.97:0/375530385 >> 172.21.15.97:0/375530385 conn(0x7fa5e40d0b70 msgr2=0x7fa5e40d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 -- 172.21.15.97:0/375530385 shutdown_connections 2024-09-10T21:24:39.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 -- 172.21.15.97:0/375530385 wait complete. 2024-09-10T21:24:39.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 Processor -- start 2024-09-10T21:24:39.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.288+0000 7fa5ec400640 1 -- start start 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e414e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa5e40d06d0 0x7fa5e41f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5e415ed50 0x7fa5e415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e4167b60 con 0x7fa5e40b4a50 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e41679e0 con 0x7fa5e415ed50 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e4167ce0 con 0x7fa5e40d06d0 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e414e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e414e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59210/0 (socket says 172.21.15.97:59210) 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 -- 172.21.15.97:0/2117146535 learned_addr learned my addr 172.21.15.97:0/2117146535 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ea400640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5e415ed50 0x7fa5e415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9000640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa5e40d06d0 0x7fa5e41f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 -- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa5e40d06d0 msgr2=0x7fa5e41f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa5e40d06d0 0x7fa5e41f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 -- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5e415ed50 msgr2=0x7fa5e415b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5e415ed50 0x7fa5e415b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 -- 172.21.15.97:0/2117146535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e4154910 con 0x7fa5e40b4a50 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e9a00640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e414e210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c7c0 tx=0x7fa5d800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e1600640 1 -- 172.21.15.97:0/2117146535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8016020 con 0x7fa5e40b4a50 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e1600640 1 -- 172.21.15.97:0/2117146535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5d800ab40 con 0x7fa5e40b4a50 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5e1600640 1 -- 172.21.15.97:0/2117146535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8012420 con 0x7fa5e40b4a50 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e4154bf0 con 0x7fa5e40b4a50 2024-09-10T21:24:39.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e41f7b50 con 0x7fa5e40b4a50 2024-09-10T21:24:39.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.292+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e41f7d20 con 0x7fa5e40b4a50 2024-09-10T21:24:39.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.296+0000 7fa5e1600640 1 -- 172.21.15.97:0/2117146535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa5d801b400 con 0x7fa5e40b4a50 2024-09-10T21:24:39.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.296+0000 7fa5e1600640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa5c40725e0 0x7fa5c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.296+0000 7fa5e1600640 1 -- 172.21.15.97:0/2117146535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa5d8093880 con 0x7fa5e40b4a50 2024-09-10T21:24:39.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.296+0000 7fa5e9000640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa5c40725e0 0x7fa5c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.296+0000 7fa5e9000640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa5c40725e0 0x7fa5c4074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5e41557c0 tx=0x7fa5d000b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:39.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.300+0000 7fa5e1600640 1 -- 172.21.15.97:0/2117146535 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa5d8020030 con 0x7fa5e40b4a50 2024-09-10T21:24:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.496+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e4070090 con 0x7fa5e40b4a50 2024-09-10T21:24:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.496+0000 7fa5e1600640 1 -- 172.21.15.97:0/2117146535 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7fa5d805e530 con 0x7fa5e40b4a50 2024-09-10T21:24:39.505 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:39.505 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:39.505 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa5c40725e0 msgr2=0x7fa5c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa5c40725e0 0x7fa5c4074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5e41557c0 tx=0x7fa5d000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 msgr2=0x7fa5e414e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e414e210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800c7c0 tx=0x7fa5d800cc80 comp rx=0 tx=0).stop 2024-09-10T21:24:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 shutdown_connections 2024-09-10T21:24:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa5c40725e0 0x7fa5c4074aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5e415ed50 0x7fa5e415b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa5e40d06d0 0x7fa5e41f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 --2- 172.21.15.97:0/2117146535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5e40b4a50 0x7fa5e414e210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 >> 172.21.15.97:0/2117146535 conn(0x7fa5e40d0b70 msgr2=0x7fa5e415e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 shutdown_connections 2024-09-10T21:24:39.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.500+0000 7fa5ec400640 1 -- 172.21.15.97:0/2117146535 wait complete. 2024-09-10T21:24:39.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:39.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- 172.21.15.97:0/3213599086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 msgr2=0x7f13580ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 --2- 172.21.15.97:0/3213599086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580ce9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1340009900 tx=0x7f134001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- 172.21.15.97:0/3213599086 shutdown_connections 2024-09-10T21:24:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 --2- 172.21.15.97:0/3213599086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580ce9c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- 172.21.15.97:0/3213599086 >> 172.21.15.97:0/3213599086 conn(0x7f13580d1ca0 msgr2=0x7f13580d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- 172.21.15.97:0/3213599086 shutdown_connections 2024-09-10T21:24:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- 172.21.15.97:0/3213599086 wait complete. 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 Processor -- start 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- start start 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13580cde30 0x7f13581f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f13580ce5e0 0x7f135814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1358169d10 con 0x7f13580d0710 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1358169b90 con 0x7f13580cde30 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1358169e90 con 0x7f13580ce5e0 2024-09-10T21:24:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59234/0 (socket says 172.21.15.97:59234) 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 -- 172.21.15.97:0/963392227 learned_addr learned my addr 172.21.15.97:0/963392227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f1357e00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13580cde30 0x7f13581f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f134f400640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f13580ce5e0 0x7f135814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 -- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f13580ce5e0 msgr2=0x7f135814fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f13580ce5e0 0x7f135814fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 -- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13580cde30 msgr2=0x7f13581f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13580cde30 0x7f13581f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.660+0000 7f135ce00640 1 -- 172.21.15.97:0/963392227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1358155d00 con 0x7f13580d0710 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f135ce00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580b5400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f135000c3f0 tx=0x7f135000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f1354c00640 1 -- 172.21.15.97:0/963392227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350011790 con 0x7f13580d0710 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f1354c00640 1 -- 172.21.15.97:0/963392227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1350017070 con 0x7f13580d0710 2024-09-10T21:24:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f1354c00640 1 -- 172.21.15.97:0/963392227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350011cc0 con 0x7f13580d0710 2024-09-10T21:24:39.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1358155a80 con 0x7f13580d0710 2024-09-10T21:24:39.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13581f6150 con 0x7f13580d0710 2024-09-10T21:24:39.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13580b2010 con 0x7f13580d0710 2024-09-10T21:24:39.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f1354c00640 1 -- 172.21.15.97:0/963392227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f135001c460 con 0x7f13580d0710 2024-09-10T21:24:39.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f1354c00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:39.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f1354c00640 1 -- 172.21.15.97:0/963392227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1350093380 con 0x7f13580d0710 2024-09-10T21:24:39.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.664+0000 7f1357e00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:39.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.668+0000 7f1357e00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1358155390 tx=0x7f1348009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:39.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.668+0000 7f1354c00640 1 -- 172.21.15.97:0/963392227 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1358155a80 con 0x7f13580d0710 2024-09-10T21:24:39.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.840+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13580c8b30 con 0x7f13580d0710 2024-09-10T21:24:39.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.840+0000 7f1354c00640 1 -- 172.21.15.97:0/963392227 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f135005e030 con 0x7f13580d0710 2024-09-10T21:24:39.848 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:39.848 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 msgr2=0x7f1334074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1358155390 tx=0x7f1348009040 comp rx=0 tx=0).stop 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 msgr2=0x7f13580b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580b5400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f135000c3f0 tx=0x7f135000c8b0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 shutdown_connections 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13580d0710 0x7f13580b5400 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f13580ce5e0 0x7f135814fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 --2- 172.21.15.97:0/963392227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13580cde30 0x7f13581f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 >> 172.21.15.97:0/963392227 conn(0x7f13580d1ca0 msgr2=0x7f13580b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:39.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 shutdown_connections 2024-09-10T21:24:39.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:39.844+0000 7f135ee00640 1 -- 172.21.15.97:0/963392227 wait complete. 2024-09-10T21:24:39.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.004+0000 7fbc87c00640 1 -- 172.21.15.97:0/3403829597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc68003a40 msgr2=0x7fbc68003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.004+0000 7fbc87c00640 1 --2- 172.21.15.97:0/3403829597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc68003a40 0x7fbc68003e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400c2e0 tx=0x7fbc7401cf20 comp rx=0 tx=0).stop 2024-09-10T21:24:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.004+0000 7fbc85c00640 1 -- 172.21.15.97:0/3403829597 reap_dead start 2024-09-10T21:24:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.004+0000 7fbc87c00640 1 -- 172.21.15.97:0/3403829597 shutdown_connections 2024-09-10T21:24:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.004+0000 7fbc87c00640 1 -- 172.21.15.97:0/3403829597 >> 172.21.15.97:0/3403829597 conn(0x7fbc800b2930 msgr2=0x7fbc800b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.004+0000 7fbc87c00640 1 -- 172.21.15.97:0/3403829597 shutdown_connections 2024-09-10T21:24:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.004+0000 7fbc87c00640 1 -- 172.21.15.97:0/3403829597 wait complete. 2024-09-10T21:24:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 Processor -- start 2024-09-10T21:24:40.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 -- start start 2024-09-10T21:24:40.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc68003a40 0x7fbc80145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc80143330 0x7fbc800d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc800d0f20 0x7fbc8015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80166f70 con 0x7fbc800d0f20 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80166df0 con 0x7fbc68003a40 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbc801670f0 con 0x7fbc80143330 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc800d0f20 0x7fbc8015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc800d0f20 0x7fbc8015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59246/0 (socket says 172.21.15.97:59246) 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 -- 172.21.15.97:0/8794532 learned_addr learned my addr 172.21.15.97:0/8794532 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 -- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc80143330 msgr2=0x7fbc800d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc80143330 0x7fbc800d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 -- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc68003a40 msgr2=0x7fbc80145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc68003a40 0x7fbc80145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 -- 172.21.15.97:0/8794532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc80160830 con 0x7fbc800d0f20 2024-09-10T21:24:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc85c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc800d0f20 0x7fbc8015e680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc80072df0 tx=0x7fbc78001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc7cc00640 1 -- 172.21.15.97:0/8794532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78011870 con 0x7fbc800d0f20 2024-09-10T21:24:40.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc7cc00640 1 -- 172.21.15.97:0/8794532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc78017070 con 0x7fbc800d0f20 2024-09-10T21:24:40.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc80160b10 con 0x7fbc800d0f20 2024-09-10T21:24:40.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc7cc00640 1 -- 172.21.15.97:0/8794532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78011d80 con 0x7fbc800d0f20 2024-09-10T21:24:40.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.008+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc801610d0 con 0x7fbc800d0f20 2024-09-10T21:24:40.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.012+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8006fed0 con 0x7fbc800d0f20 2024-09-10T21:24:40.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.012+0000 7fbc7cc00640 1 -- 172.21.15.97:0/8794532 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbc7801c460 con 0x7fbc800d0f20 2024-09-10T21:24:40.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.012+0000 7fbc7cc00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc60072610 0x7fbc60074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.012+0000 7fbc7cc00640 1 -- 172.21.15.97:0/8794532 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbc78093f10 con 0x7fbc800d0f20 2024-09-10T21:24:40.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.012+0000 7fbc85200640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc60072610 0x7fbc60074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.012+0000 7fbc85200640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc60072610 0x7fbc60074ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc740ab000 tx=0x7fbc68005840 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.016+0000 7fbc7cc00640 1 -- 172.21.15.97:0/8794532 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbc7805f210 con 0x7fbc800d0f20 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc8015e2f0 con 0x7fbc800d0f20 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc7cc00640 1 -- 172.21.15.97:0/8794532 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7fbc7805eb10 con 0x7fbc800d0f20 2024-09-10T21:24:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:40.222 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:40.222 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc60072610 msgr2=0x7fbc60074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc60072610 0x7fbc60074ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc740ab000 tx=0x7fbc68005840 comp rx=0 tx=0).stop 2024-09-10T21:24:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc800d0f20 msgr2=0x7fbc8015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc800d0f20 0x7fbc8015e680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc80072df0 tx=0x7fbc78001a80 comp rx=0 tx=0).stop 2024-09-10T21:24:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 shutdown_connections 2024-09-10T21:24:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc60072610 0x7fbc60074ad0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc800d0f20 0x7fbc8015e680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc80143330 0x7fbc800d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 --2- 172.21.15.97:0/8794532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc68003a40 0x7fbc80145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 >> 172.21.15.97:0/8794532 conn(0x7fbc800b2930 msgr2=0x7fbc800709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 shutdown_connections 2024-09-10T21:24:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.216+0000 7fbc87c00640 1 -- 172.21.15.97:0/8794532 wait complete. 2024-09-10T21:24:40.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:24:11.081203+0000', 'modified': '2024-09-10T21:24:31.188433+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4381': {'gid': 4381, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6835/4001499169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 4001499169}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 4001499169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4381, 'qdb_cluster': [4381]} 2024-09-10T21:24:40.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:24:40.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:24:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 -- 172.21.15.97:0/393603964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c1614f0 msgr2=0x7f100c0d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 --2- 172.21.15.97:0/393603964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c1614f0 0x7f100c0d11f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009bc0 tx=0x7f0ff401ca80 comp rx=0 tx=0).stop 2024-09-10T21:24:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 -- 172.21.15.97:0/393603964 shutdown_connections 2024-09-10T21:24:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 --2- 172.21.15.97:0/393603964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f100c0d29d0 0x7f100c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 --2- 172.21.15.97:0/393603964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c1614f0 0x7f100c0d11f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 -- 172.21.15.97:0/393603964 >> 172.21.15.97:0/393603964 conn(0x7f100c0d3ce0 msgr2=0x7f100c0d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 -- 172.21.15.97:0/393603964 shutdown_connections 2024-09-10T21:24:40.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.784+0000 7f1010a00640 1 -- 172.21.15.97:0/393603964 wait complete. 2024-09-10T21:24:40.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 Processor -- start 2024-09-10T21:24:40.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 -- start start 2024-09-10T21:24:40.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0d29d0 0x7f100c153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f100c161110 0x7f100c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f100c1614f0 0x7f100c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f100c16b060 con 0x7f100c0d29d0 2024-09-10T21:24:40.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f100c16aee0 con 0x7f100c1614f0 2024-09-10T21:24:40.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f100c16b1e0 con 0x7f100c161110 2024-09-10T21:24:40.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0d29d0 0x7f100c153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0d29d0 0x7f100c153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59272/0 (socket says 172.21.15.97:59272) 2024-09-10T21:24:40.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 -- 172.21.15.97:0/1650655354 learned_addr learned my addr 172.21.15.97:0/1650655354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:40.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1002a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f100c161110 0x7f100c1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100be00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f100c1614f0 0x7f100c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 -- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f100c161110 msgr2=0x7f100c1524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f100c161110 0x7f100c1524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 -- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f100c1614f0 msgr2=0x7f100c152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f100c1614f0 0x7f100c152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 -- 172.21.15.97:0/1650655354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c152a00 con 0x7f100c0d29d0 2024-09-10T21:24:40.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f100b400640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0d29d0 0x7f100c153f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c450 tx=0x7f0ffc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1003e00640 1 -- 172.21.15.97:0/1650655354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011790 con 0x7f100c0d29d0 2024-09-10T21:24:40.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1003e00640 1 -- 172.21.15.97:0/1650655354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ffc017070 con 0x7f100c0d29d0 2024-09-10T21:24:40.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c153510 con 0x7f100c0d29d0 2024-09-10T21:24:40.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1003e00640 1 -- 172.21.15.97:0/1650655354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011cc0 con 0x7f100c0d29d0 2024-09-10T21:24:40.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.788+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c153ad0 con 0x7f100c0d29d0 2024-09-10T21:24:40.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f1003e00640 1 -- 172.21.15.97:0/1650655354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0ffc01c460 con 0x7f100c0d29d0 2024-09-10T21:24:40.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f1003e00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0fe8072610 0x7f0fe8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f1003e00640 1 -- 172.21.15.97:0/1650655354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0ffc093240 con 0x7f100c0d29d0 2024-09-10T21:24:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 Processor -- start 2024-09-10T21:24:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 -- start start 2024-09-10T21:24:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f1002a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0fe8072610 0x7f0fe8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f1002a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0fe8072610 0x7f0fe8074ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40af000 tx=0x7f0ff401b2c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b979d0 0x55b479b97db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b982f0 0x55b479b4b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b4c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b479a6c3f0 con 0x55b479b4be10 2024-09-10T21:24:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b4799a9b20 con 0x55b479b982f0 2024-09-10T21:24:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b479998f50 con 0x55b479b979d0 2024-09-10T21:24:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f100be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b4c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f100be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b4c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59278/0 (socket says 172.21.15.97:59278) 2024-09-10T21:24:40.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f100be00640 1 -- 172.21.15.97:0/3178168862 learned_addr learned my addr 172.21.15.97:0/3178168862 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f100b400640 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b979d0 0x55b479b97db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.792+0000 7f1002a00640 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b982f0 0x55b479b4b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 -- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b979d0 msgr2=0x55b479b97db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b979d0 0x55b479b97db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 -- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b982f0 msgr2=0x55b479b4b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b982f0 0x55b479b4b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 -- 172.21.15.97:0/3178168862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b479960590 con 0x55b479b4be10 2024-09-10T21:24:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b4c1f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f100400b6a0 tx=0x7f100400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2ecd6e22aac348 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f0ffbe00640 1 -- 172.21.15.97:0/3178168862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004003b80 con 0x55b479b4be10 2024-09-10T21:24:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f0ffbe00640 1 -- 172.21.15.97:0/3178168862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1004015070 con 0x55b479b4be10 2024-09-10T21:24:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f0ffbe00640 1 -- 172.21.15.97:0/3178168862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004005280 con 0x55b479b4be10 2024-09-10T21:24:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- 172.21.15.97:0/3178168862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 msgr2=0x55b479b4c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b4c1f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f100400b6a0 tx=0x7f100400bb60 comp rx=0 tx=0).stop 2024-09-10T21:24:40.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- 172.21.15.97:0/3178168862 shutdown_connections 2024-09-10T21:24:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b4c1f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b982f0 0x55b479b4b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 --2- 172.21.15.97:0/3178168862 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b979d0 0x55b479b97db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- 172.21.15.97:0/3178168862 >> 172.21.15.97:0/3178168862 conn(0x55b479a35130 msgr2=0x55b479b7ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- 172.21.15.97:0/3178168862 shutdown_connections 2024-09-10T21:24:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- 172.21.15.97:0/3178168862 wait complete. 2024-09-10T21:24:40.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- start start 2024-09-10T21:24:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 Processor -- start 2024-09-10T21:24:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b979d0 0x55b479b78e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b4799335d0 con 0x55b479b4be10 2024-09-10T21:24:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b47999cd60 con 0x55b479b979d0 2024-09-10T21:24:40.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b47993f500 con 0x55b479b982f0 2024-09-10T21:24:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f10154fc000 10 client.0 _ll_register_callbacks cb 0x55b479943810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b786a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59290/0 (socket says 172.21.15.97:59290) 2024-09-10T21:24:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 -- 172.21.15.97:0/3543505979 learned_addr learned my addr 172.21.15.97:0/3543505979 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:40.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f1002a00640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b979d0 0x55b479b78e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 -- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 msgr2=0x55b479b793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 -- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b979d0 msgr2=0x55b479b78e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b979d0 0x55b479b78e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100b400640 1 -- 172.21.15.97:0/3543505979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b479937170 con 0x55b479b4be10 2024-09-10T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f100be00640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.796+0000 7f1002a00640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b979d0 0x55b479b78e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f100b400640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b786a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc091420 tx=0x7f0ffc05d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b9d645565f4091 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:24:40.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 1 -- 172.21.15.97:0/3543505979 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc063d40 con 0x55b479b4be10 2024-09-10T21:24:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 1 -- 172.21.15.97:0/3543505979 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0ffc05d130 con 0x55b479b4be10 2024-09-10T21:24:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4799310f0 con 0x55b479b4be10 2024-09-10T21:24:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 1 -- 172.21.15.97:0/3543505979 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f0ffc05ded0 con 0x55b479b4be10 2024-09-10T21:24:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 10 client.4713 fetch_fsmap learned FSMap version 9 2024-09-10T21:24:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b479a1def0 con 0x55b479b4be10 2024-09-10T21:24:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 1 -- 172.21.15.97:0/3543505979 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b4799310f0 con 0x55b479b4be10 2024-09-10T21:24:40.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 1 -- 172.21.15.97:0/3543505979 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f0ffc059820 con 0x55b479b4be10 2024-09-10T21:24:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 10 client.4713 fetch_fsmap finished waiting for FSMap version 9 2024-09-10T21:24:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 10 client.4713 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:24:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 20 client.4713 populate_metadata read hostname 'smithi097' 2024-09-10T21:24:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479b7a0f0 0x55b479ba9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 4 client.4713 mds_command: new command op to 4381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 --> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4799aba70 con 0x55b479b7a0f0 2024-09-10T21:24:40.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f1002a00640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479b7a0f0 0x55b479ba9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f1002a00640 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479b7a0f0 0x55b479ba9c90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce088ac3ed7dcc7c server_cookie=b4ea212d9aaaa66a in_seq=0 out_seq=0 2024-09-10T21:24:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 10 client.4713 ms_handle_connect on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 1 -- 172.21.15.97:0/3543505979 <== mds.0 v2:172.21.15.97:6834/4001499169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b4799aba70 con 0x55b479b7a0f0 2024-09-10T21:24:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 10 client.4713 handle_command_reply: tid=0 2024-09-10T21:24:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 client.4713 shutdown 2024-09-10T21:24:40.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 msgr2=0x55b479b786a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b786a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc091420 tx=0x7f0ffc05d910 comp rx=0 tx=0).stop 2024-09-10T21:24:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 shutdown_connections 2024-09-10T21:24:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479b7a0f0 0x55b479ba9c90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b979d0 0x55b479b78e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f0ff8c00640 0 client.4713 ms_handle_reset on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:40.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 --2- 172.21.15.97:0/3543505979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b4be10 0x55b479b786a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 >> 172.21.15.97:0/3543505979 conn(0x55b479a35130 msgr2=0x55b479b4afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 shutdown_connections 2024-09-10T21:24:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 1 -- 172.21.15.97:0/3543505979 wait complete. 2024-09-10T21:24:40.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.800+0000 7f10154fc000 20 client.4713 trim_cache size 0 max 16384 2024-09-10T21:24:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 Processor -- start 2024-09-10T21:24:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 -- start start 2024-09-10T21:24:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b4be10 0x55b479b96cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b979d0 0x55b479b97200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b4a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f100c205400 con 0x55b479b979d0 2024-09-10T21:24:40.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f100c204440 con 0x55b479b4be10 2024-09-10T21:24:40.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f100c06dba0 con 0x55b479b982f0 2024-09-10T21:24:40.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b979d0 0x55b479b97200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b979d0 0x55b479b97200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59292/0 (socket says 172.21.15.97:59292) 2024-09-10T21:24:40.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 -- 172.21.15.97:0/1349139028 learned_addr learned my addr 172.21.15.97:0/1349139028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:40.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 -- 172.21.15.97:0/1349139028 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 msgr2=0x55b479b4a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 --2- 172.21.15.97:0/1349139028 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b4a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 -- 172.21.15.97:0/1349139028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b4be10 msgr2=0x55b479b96cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 --2- 172.21.15.97:0/1349139028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b4be10 0x55b479b96cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 -- 172.21.15.97:0/1349139028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4799f4d90 con 0x55b479b979d0 2024-09-10T21:24:40.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f1002a00640 1 --2- 172.21.15.97:0/1349139028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b979d0 0x55b479b97200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009b10 tx=0x7f0ff40ba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ae7dedf8e3e8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f0ffb400640 1 -- 172.21.15.97:0/1349139028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40b2040 con 0x55b479b979d0 2024-09-10T21:24:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f0ffb400640 1 -- 172.21.15.97:0/1349139028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ff4004240 con 0x55b479b979d0 2024-09-10T21:24:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f0ffb400640 1 -- 172.21.15.97:0/1349139028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4004670 con 0x55b479b979d0 2024-09-10T21:24:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 -- 172.21.15.97:0/1349139028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b979d0 msgr2=0x55b479b97200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.820+0000 7f10154fc000 1 --2- 172.21.15.97:0/1349139028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b979d0 0x55b479b97200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009b10 tx=0x7f0ff40ba000 comp rx=0 tx=0).stop 2024-09-10T21:24:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- 172.21.15.97:0/1349139028 shutdown_connections 2024-09-10T21:24:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 --2- 172.21.15.97:0/1349139028 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b982f0 0x55b479b4a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 --2- 172.21.15.97:0/1349139028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b979d0 0x55b479b97200 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 --2- 172.21.15.97:0/1349139028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479b4be10 0x55b479b96cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- 172.21.15.97:0/1349139028 >> 172.21.15.97:0/1349139028 conn(0x55b479a35130 msgr2=0x55b479b965b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- 172.21.15.97:0/1349139028 shutdown_connections 2024-09-10T21:24:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- 172.21.15.97:0/1349139028 wait complete. 2024-09-10T21:24:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- start start 2024-09-10T21:24:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 Processor -- start 2024-09-10T21:24:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b4be10 0x55b479c16620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b982f0 0x55b479c16b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479c170a0 0x55b479babbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b4799335d0 con 0x55b479b982f0 2024-09-10T21:24:40.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b479baece0 con 0x55b479c170a0 2024-09-10T21:24:40.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b479a1da60 con 0x55b479b4be10 2024-09-10T21:24:40.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 10 client.0 _ll_register_callbacks cb 0x7f0ffc05dd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:40.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b982f0 0x55b479c16b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b982f0 0x55b479c16b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59308/0 (socket says 172.21.15.97:59308) 2024-09-10T21:24:40.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 -- 172.21.15.97:0/4234965763 learned_addr learned my addr 172.21.15.97:0/4234965763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:40.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f100b400640 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b4be10 0x55b479c16620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f100be00640 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479c170a0 0x55b479babbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 -- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b4be10 msgr2=0x55b479c16620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b4be10 0x55b479c16620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 -- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479c170a0 msgr2=0x55b479babbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479c170a0 0x55b479babbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 -- 172.21.15.97:0/4234965763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47995fa90 con 0x55b479b982f0 2024-09-10T21:24:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f1002a00640 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b982f0 0x55b479c16b60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009b10 tx=0x7f0ff4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbd91abae598f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f0ff8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:24:40.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f0ff8c00640 1 -- 172.21.15.97:0/4234965763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40da060 con 0x55b479b982f0 2024-09-10T21:24:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff4004240 con 0x55b479b982f0 2024-09-10T21:24:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f0ff8c00640 1 -- 172.21.15.97:0/4234965763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0ff4004610 con 0x55b479b982f0 2024-09-10T21:24:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f0ff8c00640 1 -- 172.21.15.97:0/4234965763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f0ff40e0050 con 0x55b479b982f0 2024-09-10T21:24:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 10 client.4719 fetch_fsmap learned FSMap version 9 2024-09-10T21:24:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b47993e050 con 0x55b479b982f0 2024-09-10T21:24:40.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f0ff8c00640 1 -- 172.21.15.97:0/4234965763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ff4004240 con 0x55b479b982f0 2024-09-10T21:24:40.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f0ff8c00640 1 -- 172.21.15.97:0/4234965763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f0ff40e3020 con 0x55b479b982f0 2024-09-10T21:24:40.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 10 client.4719 fetch_fsmap finished waiting for FSMap version 9 2024-09-10T21:24:40.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 10 client.4719 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:24:40.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 20 client.4719 populate_metadata read hostname 'smithi097' 2024-09-10T21:24:40.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479bac4c0 0x55b479bac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 4 client.4719 mds_command: new command op to 4381 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:24:40.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.824+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 --> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] -- command(tid 0: {"prefix": "status"}) -- 0x55b4799a7520 con 0x55b479bac4c0 2024-09-10T21:24:40.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f100b400640 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479bac4c0 0x55b479bac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f0ff8c00640 10 client.4719 ms_handle_connect on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:40.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f100b400640 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479bac4c0 0x55b479bac8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e79f57a97e8b41 server_cookie=3bd918e2e47e1b4e in_seq=0 out_seq=0 2024-09-10T21:24:40.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f0ff8c00640 1 -- 172.21.15.97:0/4234965763 <== mds.0 v2:172.21.15.97:6834/4001499169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55b4799a7520 con 0x55b479bac4c0 2024-09-10T21:24:40.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f0ff8c00640 10 client.4719 handle_command_reply: tid=0 2024-09-10T21:24:40.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 client.4719 shutdown 2024-09-10T21:24:40.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b982f0 msgr2=0x55b479c16b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b982f0 0x55b479c16b60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009b10 tx=0x7f0ff4004040 comp rx=0 tx=0).stop 2024-09-10T21:24:40.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 shutdown_connections 2024-09-10T21:24:40.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:6834/4001499169,v1:172.21.15.97:6835/4001499169] conn(0x55b479bac4c0 0x55b479bac8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b479c170a0 0x55b479babbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b479b982f0 0x55b479c16b60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f0ff8c00640 0 client.4719 ms_handle_reset on v2:172.21.15.97:6834/4001499169 2024-09-10T21:24:40.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 --2- 172.21.15.97:0/4234965763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b479b4be10 0x55b479c16620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 >> 172.21.15.97:0/4234965763 conn(0x55b479a35130 msgr2=0x55b479b4a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 shutdown_connections 2024-09-10T21:24:40.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 1 -- 172.21.15.97:0/4234965763 wait complete. 2024-09-10T21:24:40.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f10154fc000 20 client.4719 trim_cache size 0 max 16384 2024-09-10T21:24:40.851 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:24:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:24:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:24:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4381, 2024-09-10T21:24:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:24:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:24:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:24:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 29.118509804999999, 2024-09-10T21:24:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 9, 2024-09-10T21:24:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 26, 2024-09-10T21:24:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 26, 2024-09-10T21:24:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 34.582541394000003 2024-09-10T21:24:40.853 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:24:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0fe8072610 msgr2=0x7f0fe8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0fe8072610 0x7f0fe8074ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40af000 tx=0x7f0ff401b2c0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0d29d0 msgr2=0x7f100c153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0d29d0 0x7f100c153f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c450 tx=0x7f0ffc00c850 comp rx=0 tx=0).stop 2024-09-10T21:24:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 shutdown_connections 2024-09-10T21:24:40.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0fe8072610 0x7f0fe8074ad0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f100c1614f0 0x7f100c152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f100c161110 0x7f100c1524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 --2- 172.21.15.97:0/1650655354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0d29d0 0x7f100c153f80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 >> 172.21.15.97:0/1650655354 conn(0x7f100c0d3ce0 msgr2=0x7f100c0b8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 shutdown_connections 2024-09-10T21:24:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:40.828+0000 7f1010a00640 1 -- 172.21.15.97:0/1650655354 wait complete. 2024-09-10T21:24:40.857 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T21:24:40.871 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T21:24:40.871 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T21:24:40.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:40.871 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:24:40.880 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T21:24:40.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:40.881 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:24:40.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:40.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-09-10T21:24:41.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:41.495 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:24:41.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:41.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-09-10T21:24:41.566 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-09-10T21:24:41.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:24:41.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:41.569 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-09-10T21:24:41.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:41.579 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link delete ceph-brx 2024-09-10T21:24:41.645 INFO:teuthology.orchestra.run.smithi194.stderr:Cannot find device "ceph-brx" 2024-09-10T21:24:41.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:24:41.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:41.648 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-09-10T21:24:41.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:41.698 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link delete ceph-brx 2024-09-10T21:24:41.763 INFO:teuthology.orchestra.run.smithi194.stderr:Cannot find device "ceph-brx" 2024-09-10T21:24:41.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:24:41.766 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-10T21:24:41.766 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:24:41.766 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:24:41.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:24:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- 172.21.15.97:0/3282253790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100b1560 msgr2=0x7f7a100b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 --2- 172.21.15.97:0/3282253790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100b1560 0x7f7a100b19b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009900 tx=0x7f7a0801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- 172.21.15.97:0/3282253790 shutdown_connections 2024-09-10T21:24:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 --2- 172.21.15.97:0/3282253790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100b1560 0x7f7a100b19b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 --2- 172.21.15.97:0/3282253790 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a100cec10 0x7f7a100cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- 172.21.15.97:0/3282253790 >> 172.21.15.97:0/3282253790 conn(0x7f7a100b37c0 msgr2=0x7f7a100b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- 172.21.15.97:0/3282253790 shutdown_connections 2024-09-10T21:24:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- 172.21.15.97:0/3282253790 wait complete. 2024-09-10T21:24:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 Processor -- start 2024-09-10T21:24:41.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- start start 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a100b1560 0x7f7a100c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a100ce830 0x7f7a100cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100cec10 0x7f7a101f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a100ce1a0 con 0x7f7a100cec10 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7a100cde60 con 0x7f7a100b1560 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a16600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7a100ce320 con 0x7f7a100ce830 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100cec10 0x7f7a101f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a100b1560 0x7f7a100c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a06a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a100ce830 0x7f7a100cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a100b1560 0x7f7a100c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:44836/0 (socket says 172.21.15.97:44836) 2024-09-10T21:24:41.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0f400640 1 -- 172.21.15.97:0/2688655890 learned_addr learned my addr 172.21.15.97:0/2688655890 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0fe00640 1 -- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a100ce830 msgr2=0x7f7a100cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a100ce830 0x7f7a100cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0fe00640 1 -- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a100b1560 msgr2=0x7f7a100c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a100b1560 0x7f7a100c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.904+0000 7f7a0fe00640 1 -- 172.21.15.97:0/2688655890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a100cb890 con 0x7f7a100cec10 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a0fe00640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100cec10 0x7f7a101f8220 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7a080af000 tx=0x7f7a080a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a07e00640 1 -- 172.21.15.97:0/2688655890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a080b5040 con 0x7f7a100cec10 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a07e00640 1 -- 172.21.15.97:0/2688655890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a08005310 con 0x7f7a100cec10 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a07e00640 1 -- 172.21.15.97:0/2688655890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a080046d0 con 0x7f7a100cec10 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a100cbb30 con 0x7f7a100cec10 2024-09-10T21:24:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1006dc50 con 0x7f7a100cec10 2024-09-10T21:24:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a100cffc0 con 0x7f7a100cec10 2024-09-10T21:24:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a07e00640 1 -- 172.21.15.97:0/2688655890 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7a08004870 con 0x7f7a100cec10 2024-09-10T21:24:41.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a07e00640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f79ec0725e0 0x7f79ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a07e00640 1 -- 172.21.15.97:0/2688655890 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7a08130180 con 0x7f7a100cec10 2024-09-10T21:24:41.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.908+0000 7f7a0f400640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f79ec0725e0 0x7f79ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.912+0000 7f7a0f400640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f79ec0725e0 0x7f79ec074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a00001fa0 tx=0x7f7a00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:41.912+0000 7f7a07e00640 1 -- 172.21.15.97:0/2688655890 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7a080c0030 con 0x7f7a100cec10 2024-09-10T21:24:42.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7a1015e190 con 0x7f7a100cec10 2024-09-10T21:24:42.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a07e00640 1 -- 172.21.15.97:0/2688655890 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f7a08018c30 con 0x7f7a100cec10 2024-09-10T21:24:42.122 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T21:24:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f79ec0725e0 msgr2=0x7f79ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f79ec0725e0 0x7f79ec074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a00001fa0 tx=0x7f7a00009040 comp rx=0 tx=0).stop 2024-09-10T21:24:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100cec10 msgr2=0x7f7a101f8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100cec10 0x7f7a101f8220 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7a080af000 tx=0x7f7a080a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 shutdown_connections 2024-09-10T21:24:42.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f79ec0725e0 0x7f79ec074aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a100cec10 0x7f7a101f8220 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a100ce830 0x7f7a100cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 --2- 172.21.15.97:0/2688655890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a100b1560 0x7f7a100c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 >> 172.21.15.97:0/2688655890 conn(0x7f7a100b37c0 msgr2=0x7f7a100cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:42.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 shutdown_connections 2024-09-10T21:24:42.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42.116+0000 7f7a16600640 1 -- 172.21.15.97:0/2688655890 wait complete. 2024-09-10T21:24:42.144 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:24:42.144 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:24:42.144 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:24:42.144 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:24:42.144 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:24:42.144 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:24:42.145 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:24:42.145 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:24:42.145 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:24:42.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:42.145 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42411sec preferred_lft 42411sec 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:24:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:24:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:42.157 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:24:42.157 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:42.157 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:42.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:24:42.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-09-10T21:24:42.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-09-10T21:24:42.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:24:42.157 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:42.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:42.312 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:24:42.330 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:24:42.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:42.332 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:42.388 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:24:42.389 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:42.389 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:42.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:24:42.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:24:42.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:24:42.389 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:42.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:42.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:42.522 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:24:42.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:42.531 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-09-10T21:24:42.581 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:42.581 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:42.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:24:42.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T21:24:42.582 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:42.706 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T21:24:42.706 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:42.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:42.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:42.847 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:42.847 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:42.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:24:42.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:24:42.847 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:42.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:42.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:42.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:42.971 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:24:42.979 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:24:42.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:42.979 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:24:43.028 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:24:43.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:43.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:24:43.120 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:24:43.121 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:24:43.121 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:24:43.816 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:24:43.816 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:24:43.816 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:24:43.816 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:43.816 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:24:43.828 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:24:43.832 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:24:43.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:43.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:24:43.852 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:43.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:43.953 INFO:teuthology.orchestra.run.smithi097.stdout:client4728 2024-09-10T21:24:43.954 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:43.954 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:43.963 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:44.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:44.059 INFO:teuthology.orchestra.run.smithi097.stdout:client4728 2024-09-10T21:24:44.060 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:44.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:44.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:44.115 INFO:teuthology.orchestra.run.smithi097.stdout:client4728 2024-09-10T21:24:44.116 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1b4b680-9a7d-4173-b750-f3c8b85e2f7b.client4728/status') 2024-09-10T21:24:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:44.174 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4728 (3)192.168.0.1:0/2015294579 2024-09-10T21:24:44.174 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-09-10T21:24:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-09-10T21:24:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-09-10T21:24:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:69 bytes copied, 6.0539e-05 s, 1.1 MB/s 2024-09-10T21:24:44.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:44.182 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-10T21:24:44.183 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:24:44.183 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T21:24:44.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:24:44.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.360+0000 7f9370600640 1 -- 172.21.15.97:0/2342152245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93680b3cf0 msgr2=0x7f936806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.360+0000 7f9370600640 1 --2- 172.21.15.97:0/2342152245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93680b3cf0 0x7f936806ec40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9354009900 tx=0x7f935401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- 172.21.15.97:0/2342152245 shutdown_connections 2024-09-10T21:24:44.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 --2- 172.21.15.97:0/2342152245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93680b3cf0 0x7f936806ec40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 --2- 172.21.15.97:0/2342152245 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f93681511f0 0x7f93680b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- 172.21.15.97:0/2342152245 >> 172.21.15.97:0/2342152245 conn(0x7f93680b1780 msgr2=0x7f93680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- 172.21.15.97:0/2342152245 shutdown_connections 2024-09-10T21:24:44.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- 172.21.15.97:0/2342152245 wait complete. 2024-09-10T21:24:44.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 Processor -- start 2024-09-10T21:24:44.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- start start 2024-09-10T21:24:44.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f936806f180 0x7f936813f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f93680b3cf0 0x7f93681f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93681511f0 0x7f93681e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f936815aac0 con 0x7f93681511f0 2024-09-10T21:24:44.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f936815a940 con 0x7f936806f180 2024-09-10T21:24:44.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f9370600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f936815ac40 con 0x7f93680b3cf0 2024-09-10T21:24:44.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93681511f0 0x7f93681e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93681511f0 0x7f93681e3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59342/0 (socket says 172.21.15.97:59342) 2024-09-10T21:24:44.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 -- 172.21.15.97:0/218668931 learned_addr learned my addr 172.21.15.97:0/218668931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:44.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936d200640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f93680b3cf0 0x7f93681f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936dc00640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f936806f180 0x7f936813f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 -- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f93680b3cf0 msgr2=0x7f93681f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f93680b3cf0 0x7f93681f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 -- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f936806f180 msgr2=0x7f936813f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f936806f180 0x7f936813f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.364+0000 7f936e600640 1 -- 172.21.15.97:0/218668931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9368141f50 con 0x7f93681511f0 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f936e600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93681511f0 0x7f93681e3ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f936000c7a0 tx=0x7f936000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9365600640 1 -- 172.21.15.97:0/218668931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360003b80 con 0x7f93681511f0 2024-09-10T21:24:44.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9365600640 1 -- 172.21.15.97:0/218668931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9360003d40 con 0x7f93681511f0 2024-09-10T21:24:44.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9365600640 1 -- 172.21.15.97:0/218668931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936000aab0 con 0x7f93681511f0 2024-09-10T21:24:44.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9368142230 con 0x7f93681511f0 2024-09-10T21:24:44.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93681427f0 con 0x7f93681511f0 2024-09-10T21:24:44.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93681f2a50 con 0x7f93681511f0 2024-09-10T21:24:44.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9365600640 1 -- 172.21.15.97:0/218668931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f936000ac50 con 0x7f93681511f0 2024-09-10T21:24:44.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9365600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f93480725e0 0x7f9348074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.368+0000 7f9365600640 1 -- 172.21.15.97:0/218668931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f93600932a0 con 0x7f93681511f0 2024-09-10T21:24:44.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.372+0000 7f936dc00640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f93480725e0 0x7f9348074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.372+0000 7f936dc00640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f93480725e0 0x7f9348074aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f936815a3d0 tx=0x7f935c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:44.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.372+0000 7f9365600640 1 -- 172.21.15.97:0/218668931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f936005df50 con 0x7f93681511f0 2024-09-10T21:24:44.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.580+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f936815b090 con 0x7f93681511f0 2024-09-10T21:24:44.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.580+0000 7f9365600640 1 -- 172.21.15.97:0/218668931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f9360062980 con 0x7f93681511f0 2024-09-10T21:24:44.587 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T21:24:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f93480725e0 msgr2=0x7f9348074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f93480725e0 0x7f9348074aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f936815a3d0 tx=0x7f935c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93681511f0 msgr2=0x7f93681e3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93681511f0 0x7f93681e3ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f936000c7a0 tx=0x7f936000cc60 comp rx=0 tx=0).stop 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 shutdown_connections 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f93480725e0 0x7f9348074aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93681511f0 0x7f93681e3ed0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f93680b3cf0 0x7f93681f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 --2- 172.21.15.97:0/218668931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f936806f180 0x7f936813f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 >> 172.21.15.97:0/218668931 conn(0x7f93680b1780 msgr2=0x7f936815cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 shutdown_connections 2024-09-10T21:24:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44.584+0000 7f9370600640 1 -- 172.21.15.97:0/218668931 wait complete. 2024-09-10T21:24:44.608 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:24:44.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:24:44.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:24:44.608 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T21:24:44.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T21:24:44.609 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:24:44.609 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:24:44.609 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:24:44.609 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T21:24:44.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:44.609 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:24:44.618 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:44.618 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42409sec preferred_lft 42409sec 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:44.619 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-09-10T21:24:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:44.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:44.621 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:24:44.685 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:24:44.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:44.688 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:24:44.740 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:24:44.740 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:24:44.740 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:24:44.740 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:24:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:24:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:24:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:24:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:44.741 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:24:44.741 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:44.741 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:44.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:24:44.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:24:44.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:24:44.742 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:44.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:44.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:44.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:44.877 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:24:44.886 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:24:44.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:44.887 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-09-10T21:24:44.936 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T21:24:44.938 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:44.938 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:44.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:24:44.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T21:24:44.938 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:45.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:45.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:45.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:45.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T21:24:45.132 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.133 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:24:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f2:b3:08:7f:a3:08 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:24:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:24:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f0b3:8ff:fe7f:a308/64 scope link 2024-09-10T21:24:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:45.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T21:24:45.197 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.197 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:24:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether f2:b3:08:7f:a3:08 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:24:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:24:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::f0b3:8ff:fe7f:a308/64 scope link 2024-09-10T21:24:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:45.199 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T21:24:45.199 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:45.199 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:45.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T21:24:45.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T21:24:45.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T21:24:45.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T21:24:45.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T21:24:45.200 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:45.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:45.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:45.380 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:45.380 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:24:45.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2024-09-10T21:24:45.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T21:24:45.380 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:24:45.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:45.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:45.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:45.462 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T21:24:45.470 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T21:24:45.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:45.470 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T21:24:45.520 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:24:45.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:45.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T21:24:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T21:24:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T21:24:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:24:45.628 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T21:24:45.628 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T21:24:45.628 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:24:45.628 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:45.628 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:24:45.637 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:24:45.640 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:24:45.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:45.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T21:24:45.660 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:45.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:45.760 INFO:teuthology.orchestra.run.smithi097.stdout:client4734 2024-09-10T21:24:45.761 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:45.761 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:45.769 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:45.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:45.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:45.867 INFO:teuthology.orchestra.run.smithi097.stdout:client4734 2024-09-10T21:24:45.868 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:45.968 INFO:teuthology.orchestra.run.smithi097.stdout:client4734 2024-09-10T21:24:45.968 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1b4b680-9a7d-4173-b750-f3c8b85e2f7b.client4734/status') 2024-09-10T21:24:46.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:46.026 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4734 (3)192.168.0.2:0/3501184934 2024-09-10T21:24:46.026 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-09-10T21:24:46.027 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-09-10T21:24:46.027 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-09-10T21:24:46.027 INFO:teuthology.orchestra.run.smithi097.stderr:69 bytes copied, 4.957e-05 s, 1.4 MB/s 2024-09-10T21:24:46.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:46.034 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-10T21:24:46.034 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:24:46.034 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:24:46.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 -- 172.21.15.97:0/3548184289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480d0710 msgr2=0x7f6c480ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 --2- 172.21.15.97:0/3548184289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480d0710 0x7f6c480ce9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c480d1c70 tx=0x7f6c3401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 -- 172.21.15.97:0/3548184289 shutdown_connections 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 --2- 172.21.15.97:0/3548184289 >> v2:172.21.15.194:3300/0 conn(0x7f6c480cde30 0x7f6c480cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 --2- 172.21.15.97:0/3548184289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480d0710 0x7f6c480ce9c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 -- 172.21.15.97:0/3548184289 >> 172.21.15.97:0/3548184289 conn(0x7f6c480d1ca0 msgr2=0x7f6c480d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 -- 172.21.15.97:0/3548184289 shutdown_connections 2024-09-10T21:24:46.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.212+0000 7f6c4d600640 1 -- 172.21.15.97:0/3548184289 wait complete. 2024-09-10T21:24:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 Processor -- start 2024-09-10T21:24:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 -- start start 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c480cde30 0x7f6c480cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480ce5e0 0x7f6c48158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c480d0710 0x7f6c4806dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c480cd350 con 0x7f6c480ce5e0 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c480cd1d0 con 0x7f6c480d0710 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6c48169060 con 0x7f6c480cde30 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480ce5e0 0x7f6c48158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480ce5e0 0x7f6c48158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59366/0 (socket says 172.21.15.97:59366) 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 -- 172.21.15.97:0/467408306 learned_addr learned my addr 172.21.15.97:0/467408306 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46a00640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c480cde30 0x7f6c480cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c47400640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c480d0710 0x7f6c4806dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 -- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c480cde30 msgr2=0x7f6c480cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c480cde30 0x7f6c480cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 -- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c480d0710 msgr2=0x7f6c4806dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c480d0710 0x7f6c4806dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 -- 172.21.15.97:0/467408306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c481f7960 con 0x7f6c480ce5e0 2024-09-10T21:24:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c46000640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480ce5e0 0x7f6c48158120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b0040 tx=0x7f6c340a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c3ea00640 1 -- 172.21.15.97:0/467408306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340b2040 con 0x7f6c480ce5e0 2024-09-10T21:24:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c3ea00640 1 -- 172.21.15.97:0/467408306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c340044b0 con 0x7f6c480ce5e0 2024-09-10T21:24:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c3ea00640 1 -- 172.21.15.97:0/467408306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34003860 con 0x7f6c480ce5e0 2024-09-10T21:24:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c480c9b10 con 0x7f6c480ce5e0 2024-09-10T21:24:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c480ca0d0 con 0x7f6c480ce5e0 2024-09-10T21:24:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.216+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c481f7b30 con 0x7f6c480ce5e0 2024-09-10T21:24:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.220+0000 7f6c3ea00640 1 -- 172.21.15.97:0/467408306 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6c340c1060 con 0x7f6c480ce5e0 2024-09-10T21:24:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.220+0000 7f6c3ea00640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c280725e0 0x7f6c28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.220+0000 7f6c3ea00640 1 -- 172.21.15.97:0/467408306 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6c3412cb00 con 0x7f6c480ce5e0 2024-09-10T21:24:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.220+0000 7f6c46a00640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c280725e0 0x7f6c28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.220+0000 7f6c46a00640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c280725e0 0x7f6c28074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c380016c0 tx=0x7f6c38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.224+0000 7f6c3ea00640 1 -- 172.21.15.97:0/467408306 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c340164b0 con 0x7f6c480ce5e0 2024-09-10T21:24:46.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.424+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6c480b2040 con 0x7f6c480ce5e0 2024-09-10T21:24:46.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.424+0000 7f6c3ea00640 1 -- 172.21.15.97:0/467408306 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f6c480b2040 con 0x7f6c480ce5e0 2024-09-10T21:24:46.431 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T21:24:46.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.424+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c280725e0 msgr2=0x7f6c28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.424+0000 7f6c4d600640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c280725e0 0x7f6c28074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c380016c0 tx=0x7f6c38009040 comp rx=0 tx=0).stop 2024-09-10T21:24:46.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.424+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480ce5e0 msgr2=0x7f6c48158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.424+0000 7f6c4d600640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480ce5e0 0x7f6c48158120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b0040 tx=0x7f6c340a57d0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 shutdown_connections 2024-09-10T21:24:46.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c280725e0 0x7f6c28074aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c480d0710 0x7f6c4806dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c480ce5e0 0x7f6c48158120 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 --2- 172.21.15.97:0/467408306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c480cde30 0x7f6c480cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 >> 172.21.15.97:0/467408306 conn(0x7f6c480d1ca0 msgr2=0x7f6c480b6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 shutdown_connections 2024-09-10T21:24:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:46.428+0000 7f6c4d600640 1 -- 172.21.15.97:0/467408306 wait complete. 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi194.front.sepia.ceph.com 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:24:46.453 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:24:46.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:46.454 DEBUG:teuthology.orchestra.run.smithi194:> ip addr 2024-09-10T21:24:46.464 INFO:teuthology.orchestra.run.smithi194.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:46.464 INFO:teuthology.orchestra.run.smithi194.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:46.464 INFO:teuthology.orchestra.run.smithi194.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:46.464 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:46.464 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:46.464 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:d9:3a:fe brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: altname enp7s0f0 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:d9:3a:ff brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: altname enp7s0f1 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:88:7e:a4 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:88:7e:a5 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: inet 172.21.15.194/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:24:46.465 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft 42410sec preferred_lft 42410sec 2024-09-10T21:24:46.466 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 fe80::ec4:7aff:fe88:7ea5/64 scope link 2024-09-10T21:24:46.466 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:46.466 INFO:teuthology.orchestra.run.smithi194.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:24:46.466 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 02:42:88:37:df:9e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:46.466 INFO:teuthology.orchestra.run.smithi194.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:24:46.466 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:46.466 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:24:46.466 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:46.466 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:46.467 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:24:46.467 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip addr flush dev ceph-brx 2024-09-10T21:24:46.467 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set ceph-brx up 2024-09-10T21:24:46.467 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:24:46.467 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:46.557 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:46.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:46.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:46.627 DEBUG:teuthology.orchestra.run.smithi194:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:24:46.645 INFO:teuthology.orchestra.run.smithi194.stdout:1 2024-09-10T21:24:46.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:46.647 DEBUG:teuthology.orchestra.run.smithi194:> route 2024-09-10T21:24:46.719 INFO:teuthology.orchestra.run.smithi194.stdout:Kernel IP routing table 2024-09-10T21:24:46.719 INFO:teuthology.orchestra.run.smithi194.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:24:46.719 INFO:teuthology.orchestra.run.smithi194.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:46.720 INFO:teuthology.orchestra.run.smithi194.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:24:46.720 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:46.720 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:46.721 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:24:46.721 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:24:46.721 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:24:46.721 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:46.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:46.852 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:46.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:46.859 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-09-10T21:24:46.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:46.868 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list-id 2024-09-10T21:24:46.917 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:46.917 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:46.917 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:24:46.917 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T21:24:46.917 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:47.006 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:47.042 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T21:24:47.042 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:47.043 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:47.043 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T21:24:47.043 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:24:47.043 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T21:24:47.043 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T21:24:47.043 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T21:24:47.043 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:47.131 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:47.243 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:47.249 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:47.250 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:47.250 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set brx.0 up 2024-09-10T21:24:47.250 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:24:47.250 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:47.296 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:47.326 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:47.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:47.333 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:24:47.384 INFO:teuthology.orchestra.run.smithi194.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:24:47.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:47.385 DEBUG:teuthology.orchestra.run.smithi194:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:24:47.440 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:24:47.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:47.440 DEBUG:teuthology.orchestra.run.smithi194:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T21:24:47.528 INFO:teuthology.orchestra.run.smithi194.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T21:24:47.528 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T21:24:47.528 INFO:teuthology.orchestra.run.smithi194.stdout:invalid new device string format 2024-09-10T21:24:48.232 INFO:teuthology.orchestra.run.smithi194.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T21:24:48.232 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T21:24:48.232 INFO:teuthology.orchestra.run.smithi194.stdout:invalid new device string format 2024-09-10T21:24:48.232 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:48.233 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:48.245 INFO:teuthology.orchestra.run.smithi194.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T21:24:48.245 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T21:24:48.245 INFO:teuthology.orchestra.run.smithi194.stdout:invalid new device string format 2024-09-10T21:24:48.245 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:48.246 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:48.246 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:24:48.246 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:24:48.246 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:24:48.249 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:24:48.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:48.249 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:24:48.269 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:48.364 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:48.376 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:48.382 INFO:teuthology.orchestra.run.smithi194.stdout:client4625 2024-09-10T21:24:48.382 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:48.382 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:48.391 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:48.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:48.482 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:48.488 INFO:teuthology.orchestra.run.smithi194.stdout:client4625 2024-09-10T21:24:48.489 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:48.535 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:48.539 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:48.545 INFO:teuthology.orchestra.run.smithi194.stdout:client4625 2024-09-10T21:24:48.546 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1b4b680-9a7d-4173-b750-f3c8b85e2f7b.client4625/status') 2024-09-10T21:24:48.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:48.651 INFO:teuthology.orchestra.run.smithi194.stdout:instance: client.4625 (3)192.168.0.1:0/3219061798 2024-09-10T21:24:48.651 INFO:teuthology.orchestra.run.smithi194.stdout:blocklisted: false 2024-09-10T21:24:48.651 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2024-09-10T21:24:48.651 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2024-09-10T21:24:48.652 INFO:teuthology.orchestra.run.smithi194.stderr:69 bytes copied, 6.0491e-05 s, 1.1 MB/s 2024-09-10T21:24:48.652 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:48.659 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-10T21:24:48.659 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:24:48.659 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T21:24:48.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:24:48.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 -- 172.21.15.97:0/1861763011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080cd3f0 msgr2=0x7fd80815e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 --2- 172.21.15.97:0/1861763011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080cd3f0 0x7fd80815e730 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009840 tx=0x7fd7f801c7f0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 -- 172.21.15.97:0/1861763011 shutdown_connections 2024-09-10T21:24:48.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 --2- 172.21.15.97:0/1861763011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8080cd3f0 0x7fd80815e730 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 -- 172.21.15.97:0/1861763011 >> 172.21.15.97:0/1861763011 conn(0x7fd80806e2b0 msgr2=0x7fd8080b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 -- 172.21.15.97:0/1861763011 shutdown_connections 2024-09-10T21:24:48.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 -- 172.21.15.97:0/1861763011 wait complete. 2024-09-10T21:24:48.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.804+0000 7fd810800640 1 Processor -- start 2024-09-10T21:24:48.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 -- start start 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8080cd3f0 0x7fd8081494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd8080ce1b0 0x7fd808141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80815b770 0x7fd80814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd80815bd00 con 0x7fd80815b770 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd80815bb80 con 0x7fd8080cd3f0 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd80815be80 con 0x7fd8080ce1b0 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80815b770 0x7fd80814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd8080ce1b0 0x7fd808141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8080cd3f0 0x7fd8081494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd8080ce1b0 0x7fd808141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:43210/0 (socket says 172.21.15.97:43210) 2024-09-10T21:24:48.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80d400640 1 -- 172.21.15.97:0/523595722 learned_addr learned my addr 172.21.15.97:0/523595722 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:48.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80de00640 1 -- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd8080ce1b0 msgr2=0x7fd808141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80de00640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd8080ce1b0 0x7fd808141f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80de00640 1 -- 172.21.15.97:0/523595722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80815b770 msgr2=0x7fd80814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80de00640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80815b770 0x7fd80814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80de00640 1 -- 172.21.15.97:0/523595722 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80814c850 con 0x7fd8080cd3f0 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80d400640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd8080ce1b0 0x7fd808141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80e800640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80815b770 0x7fd80814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd80de00640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8080cd3f0 0x7fd8081494c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c370 tx=0x7fd7fc00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd802000640 1 -- 172.21.15.97:0/523595722 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc017070 con 0x7fd8080cd3f0 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd802000640 1 -- 172.21.15.97:0/523595722 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7fc003600 con 0x7fd8080cd3f0 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80814cb30 con 0x7fd8080cd3f0 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd802000640 1 -- 172.21.15.97:0/523595722 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc003a50 con 0x7fd8080cd3f0 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd808148fa0 con 0x7fd8080cd3f0 2024-09-10T21:24:48.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.808+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80815e730 con 0x7fd8080cd3f0 2024-09-10T21:24:48.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.812+0000 7fd802000640 1 -- 172.21.15.97:0/523595722 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd7fc003bf0 con 0x7fd8080cd3f0 2024-09-10T21:24:48.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.812+0000 7fd802000640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7ec072610 0x7fd7ec074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.812+0000 7fd802000640 1 -- 172.21.15.97:0/523595722 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd7fc092b00 con 0x7fd8080cd3f0 2024-09-10T21:24:48.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.812+0000 7fd80d400640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7ec072610 0x7fd7ec074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.812+0000 7fd80d400640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7ec072610 0x7fd7ec074ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8019400 tx=0x7fd7f801b030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:48.816+0000 7fd802000640 1 -- 172.21.15.97:0/523595722 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd7fc05d7b0 con 0x7fd8080cd3f0 2024-09-10T21:24:49.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.016+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd8080b3460 con 0x7fd8080cd3f0 2024-09-10T21:24:49.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.016+0000 7fd802000640 1 -- 172.21.15.97:0/523595722 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd7fc092e20 con 0x7fd8080cd3f0 2024-09-10T21:24:49.023 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T21:24:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7ec072610 msgr2=0x7fd7ec074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7ec072610 0x7fd7ec074ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8019400 tx=0x7fd7f801b030 comp rx=0 tx=0).stop 2024-09-10T21:24:49.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8080cd3f0 msgr2=0x7fd8081494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8080cd3f0 0x7fd8081494c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c370 tx=0x7fd7fc00c830 comp rx=0 tx=0).stop 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 shutdown_connections 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7ec072610 0x7fd7ec074ad0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80815b770 0x7fd80814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd8080ce1b0 0x7fd808141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 --2- 172.21.15.97:0/523595722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8080cd3f0 0x7fd8081494c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 >> 172.21.15.97:0/523595722 conn(0x7fd80806e2b0 msgr2=0x7fd80806fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 shutdown_connections 2024-09-10T21:24:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:49.020+0000 7fd810800640 1 -- 172.21.15.97:0/523595722 wait complete. 2024-09-10T21:24:49.044 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:24:49.044 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:24:49.044 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi194.front.sepia.ceph.com 2024-09-10T21:24:49.044 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T21:24:49.044 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T21:24:49.044 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:24:49.045 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:24:49.045 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:24:49.045 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T21:24:49.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.045 DEBUG:teuthology.orchestra.run.smithi194:> ip addr 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:d9:3a:fe brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: altname enp7s0f0 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:49.054 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:d9:3a:ff brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: altname enp7s0f1 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:88:7e:a4 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:88:7e:a5 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: inet 172.21.15.194/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft 42408sec preferred_lft 42408sec 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 fe80::ec4:7aff:fe88:7ea5/64 scope link 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 02:42:88:37:df:9e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:24:49.055 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:24:49.056 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.057 DEBUG:teuthology.orchestra.run.smithi194:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:24:49.118 INFO:teuthology.orchestra.run.smithi194.stdout:1 2024-09-10T21:24:49.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.121 DEBUG:teuthology.orchestra.run.smithi194:> route 2024-09-10T21:24:49.175 INFO:teuthology.orchestra.run.smithi194.stdout:Kernel IP routing table 2024-09-10T21:24:49.176 INFO:teuthology.orchestra.run.smithi194.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:24:49.176 INFO:teuthology.orchestra.run.smithi194.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:24:49.176 INFO:teuthology.orchestra.run.smithi194.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:24:49.176 INFO:teuthology.orchestra.run.smithi194.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:24:49.176 INFO:teuthology.orchestra.run.smithi194.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:24:49.176 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:24:49.176 INFO:teuthology.orchestra.run.smithi194.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:49.177 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:49.177 INFO:teuthology.orchestra.run.smithi194.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:24:49.177 INFO:teuthology.orchestra.run.smithi194.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:24:49.177 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:49.177 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:49.177 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:24:49.177 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:24:49.177 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:24:49.177 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:49.266 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:49.306 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:49.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.313 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-09-10T21:24:49.322 INFO:teuthology.orchestra.run.smithi194.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:24:49.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.323 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list-id 2024-09-10T21:24:49.376 INFO:teuthology.orchestra.run.smithi194.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T21:24:49.378 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:49.378 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:49.378 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:24:49.378 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T21:24:49.378 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:49.470 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:49.499 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:49.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.506 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T21:24:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:49.526 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.526 INFO:teuthology.orchestra.run.smithi194.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:24:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 62:84:48:68:8d:9c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:24:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:24:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 fe80::6084:48ff:fe68:8d9c/64 scope link 2024-09-10T21:24:49.527 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.528 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 ::1/128 scope host 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 62:84:48:68:8d:9c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:24:49.592 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.593 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 fe80::6084:48ff:fe68:8d9c/64 scope link 2024-09-10T21:24:49.593 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:24:49.594 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T21:24:49.594 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:49.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:49.764 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:49.771 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:24:49.771 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-09-10T21:24:49.771 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set brx.1 up 2024-09-10T21:24:49.771 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T21:24:49.771 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-09-10T21:24:49.818 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:49.848 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:49.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:49.856 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T21:24:49.865 INFO:teuthology.orchestra.run.smithi194.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T21:24:49.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:24:49.865 DEBUG:teuthology.orchestra.run.smithi194:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T21:24:49.916 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:24:49.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:49.916 DEBUG:teuthology.orchestra.run.smithi194:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T21:24:50.005 INFO:teuthology.orchestra.run.smithi194.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T21:24:50.005 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T21:24:50.005 INFO:teuthology.orchestra.run.smithi194.stdout:invalid new device string format 2024-09-10T21:24:50.022 INFO:teuthology.orchestra.run.smithi194.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T21:24:50.022 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T21:24:50.022 INFO:teuthology.orchestra.run.smithi194.stdout:invalid new device string format 2024-09-10T21:24:50.022 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:50.022 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:50.038 INFO:teuthology.orchestra.run.smithi194.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T21:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T21:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout:invalid new device string format 2024-09-10T21:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:24:50.039 INFO:teuthology.orchestra.run.smithi194.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:24:50.043 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:24:50.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:50.043 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T21:24:50.104 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:50.193 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:50.197 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:50.203 INFO:teuthology.orchestra.run.smithi194.stdout:client4752 2024-09-10T21:24:50.203 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:50.203 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:50.212 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:50.299 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:50.302 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:50.309 INFO:teuthology.orchestra.run.smithi194.stdout:client4752 2024-09-10T21:24:50.309 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:24:50.355 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:24:50.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:50.365 INFO:teuthology.orchestra.run.smithi194.stdout:client4752 2024-09-10T21:24:50.366 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1b4b680-9a7d-4173-b750-f3c8b85e2f7b.client4752/status') 2024-09-10T21:24:50.455 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout:instance: client.4752 (3)172.21.15.194:0/4208352027 2024-09-10T21:24:50.467 INFO:teuthology.orchestra.run.smithi194.stdout:blocklisted: false 2024-09-10T21:24:50.468 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records in 2024-09-10T21:24:50.468 INFO:teuthology.orchestra.run.smithi194.stderr:0+1 records out 2024-09-10T21:24:50.468 INFO:teuthology.orchestra.run.smithi194.stderr:71 bytes copied, 6.031e-05 s, 1.2 MB/s 2024-09-10T21:24:50.469 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-10T21:24:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:24:50.475 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T21:24:50.528 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2024-09-10T21:24:50.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:50.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.672+0000 7f77ae000640 1 -- 172.21.15.97:0/1479280970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a80b0d50 msgr2=0x7f77a80ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:50.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.672+0000 7f77ae000640 1 --2- 172.21.15.97:0/1479280970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a80b0d50 0x7f77a80ccde0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009bc0 tx=0x7f77a001cba0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- 172.21.15.97:0/1479280970 shutdown_connections 2024-09-10T21:24:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 --2- 172.21.15.97:0/1479280970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a80b0d50 0x7f77a80ccde0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- 172.21.15.97:0/1479280970 >> 172.21.15.97:0/1479280970 conn(0x7f77a80d1c30 msgr2=0x7f77a80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- 172.21.15.97:0/1479280970 shutdown_connections 2024-09-10T21:24:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- 172.21.15.97:0/1479280970 wait complete. 2024-09-10T21:24:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 Processor -- start 2024-09-10T21:24:50.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- start start 2024-09-10T21:24:50.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80afdd0 0x7f77a814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a80b0d50 0x7f77a8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77a80ce640 0x7f77a8152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8169ff0 con 0x7f77a80afdd0 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8169e70 con 0x7f77a80b0d50 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77ae000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f77a816a170 con 0x7f77a80ce640 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80afdd0 0x7f77a814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80afdd0 0x7f77a814b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54270/0 (socket says 172.21.15.97:54270) 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 -- 172.21.15.97:0/2428722615 learned_addr learned my addr 172.21.15.97:0/2428722615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 -- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77a80ce640 msgr2=0x7f77a8152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77a80ce640 0x7f77a8152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 -- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a80b0d50 msgr2=0x7f77a8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a80b0d50 0x7f77a8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 -- 172.21.15.97:0/2428722615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a806dcf0 con 0x7f77a80afdd0 2024-09-10T21:24:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f77a7400640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80afdd0 0x7f77a814b030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f779800c400 tx=0x7f779800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f779fe00640 1 -- 172.21.15.97:0/2428722615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798017070 con 0x7f77a80afdd0 2024-09-10T21:24:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.676+0000 7f779fe00640 1 -- 172.21.15.97:0/2428722615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7798005400 con 0x7f77a80afdd0 2024-09-10T21:24:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f779fe00640 1 -- 172.21.15.97:0/2428722615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798003860 con 0x7f77a80afdd0 2024-09-10T21:24:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a806dfd0 con 0x7f77a80afdd0 2024-09-10T21:24:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a806e590 con 0x7f77a80afdd0 2024-09-10T21:24:50.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a80cea20 con 0x7f77a80afdd0 2024-09-10T21:24:50.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f779fe00640 1 -- 172.21.15.97:0/2428722615 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7798003a00 con 0x7f77a80afdd0 2024-09-10T21:24:50.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f779fe00640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77840725e0 0x7f7784074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:50.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f779fe00640 1 -- 172.21.15.97:0/2428722615 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7798092c10 con 0x7f77a80afdd0 2024-09-10T21:24:50.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.680+0000 7f779ea00640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77840725e0 0x7f7784074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:50.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.684+0000 7f779ea00640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77840725e0 0x7f7784074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7790001230 tx=0x7f7790000f40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:50.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.688+0000 7f779fe00640 1 -- 172.21.15.97:0/2428722615 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f779805d8d0 con 0x7f77a80afdd0 2024-09-10T21:24:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.856+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77a814b570 con 0x7f77a80afdd0 2024-09-10T21:24:50.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.856+0000 7f779fe00640 1 -- 172.21.15.97:0/2428722615 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18626 (secure 0 0 0) 0x7f7798062300 con 0x7f77a80afdd0 2024-09-10T21:24:50.864 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:50.864 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:16.155065+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77840725e0 msgr2=0x7f7784074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77840725e0 0x7f7784074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7790001230 tx=0x7f7790000f40 comp rx=0 tx=0).stop 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80afdd0 msgr2=0x7f77a814b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80afdd0 0x7f77a814b030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f779800c400 tx=0x7f779800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 shutdown_connections 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77840725e0 0x7f7784074aa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77a80ce640 0x7f77a8152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a80b0d50 0x7f77a8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 --2- 172.21.15.97:0/2428722615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a80afdd0 0x7f77a814b030 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 >> 172.21.15.97:0/2428722615 conn(0x7f77a80d1c30 msgr2=0x7f77a8158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 shutdown_connections 2024-09-10T21:24:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:50.860+0000 7f77ae000640 1 -- 172.21.15.97:0/2428722615 wait complete. 2024-09-10T21:24:50.886 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:24:02.092708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T21:24:09.636980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T21:24:10.045699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T21:24:13.750840+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T21:24:50.886 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 -- 172.21.15.97:0/3878336461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380afdd0 msgr2=0x7f87380cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 --2- 172.21.15.97:0/3878336461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380afdd0 0x7f87380cc6d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8720009840 tx=0x7f872001c820 comp rx=0 tx=0).stop 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 -- 172.21.15.97:0/3878336461 shutdown_connections 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 --2- 172.21.15.97:0/3878336461 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87380b0d50 0x7f87380ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 --2- 172.21.15.97:0/3878336461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380afdd0 0x7f87380cc6d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 -- 172.21.15.97:0/3878336461 >> 172.21.15.97:0/3878336461 conn(0x7f87380d1c30 msgr2=0x7f87380d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 -- 172.21.15.97:0/3878336461 shutdown_connections 2024-09-10T21:24:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.024+0000 7f873ea00640 1 -- 172.21.15.97:0/3878336461 wait complete. 2024-09-10T21:24:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 Processor -- start 2024-09-10T21:24:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 -- start start 2024-09-10T21:24:51.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87380afdd0 0x7f87381faef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87380b0d50 0x7f8738157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380ce640 0x7f87381f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8738168e30 con 0x7f87380ce640 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8738168cb0 con 0x7f87380b0d50 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8738168fb0 con 0x7f87380afdd0 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380ce640 0x7f87381f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380ce640 0x7f87381f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54298/0 (socket says 172.21.15.97:54298) 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ca00640 1 -- 172.21.15.97:0/2372893770 learned_addr learned my addr 172.21.15.97:0/2372893770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f8737e00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87380afdd0 0x7f87381faef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f872f400640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87380b0d50 0x7f8738157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f872f400640 1 -- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87380afdd0 msgr2=0x7f87381faef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f872f400640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87380afdd0 0x7f87381faef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f872f400640 1 -- 172.21.15.97:0/2372893770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380ce640 msgr2=0x7f87381f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f872f400640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380ce640 0x7f87381f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f872f400640 1 -- 172.21.15.97:0/2372893770 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87381fb430 con 0x7f87380b0d50 2024-09-10T21:24:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f872f400640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87380b0d50 0x7f8738157840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87200ac000 tx=0x7f87200a5760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f8734c00640 1 -- 172.21.15.97:0/2372893770 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87200b2040 con 0x7f87380b0d50 2024-09-10T21:24:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f8734c00640 1 -- 172.21.15.97:0/2372893770 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87200043b0 con 0x7f87380b0d50 2024-09-10T21:24:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87381fb6d0 con 0x7f87380b0d50 2024-09-10T21:24:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f8734c00640 1 -- 172.21.15.97:0/2372893770 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720003860 con 0x7f87380b0d50 2024-09-10T21:24:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381fbc90 con 0x7f87380b0d50 2024-09-10T21:24:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.028+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87380cea20 con 0x7f87380b0d50 2024-09-10T21:24:51.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.032+0000 7f8734c00640 1 -- 172.21.15.97:0/2372893770 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f87200b5070 con 0x7f87380b0d50 2024-09-10T21:24:51.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.032+0000 7f8734c00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87140725e0 0x7f8714074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.032+0000 7f8734c00640 1 -- 172.21.15.97:0/2372893770 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f872012c890 con 0x7f87380b0d50 2024-09-10T21:24:51.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.032+0000 7f8737e00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87140725e0 0x7f8714074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.032+0000 7f8737e00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87140725e0 0x7f8714074aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8728007fd0 tx=0x7f8728009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.036+0000 7f8734c00640 1 -- 172.21.15.97:0/2372893770 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8720016410 con 0x7f87380b0d50 2024-09-10T21:24:51.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87380726e0 con 0x7f87380b0d50 2024-09-10T21:24:51.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f8734c00640 1 -- 172.21.15.97:0/2372893770 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f87201002c0 con 0x7f87380b0d50 2024-09-10T21:24:51.222 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-09-10T21:24:51.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87140725e0 msgr2=0x7f8714074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87140725e0 0x7f8714074aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8728007fd0 tx=0x7f8728009040 comp rx=0 tx=0).stop 2024-09-10T21:24:51.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87380b0d50 msgr2=0x7f8738157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87380b0d50 0x7f8738157840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87200ac000 tx=0x7f87200a5760 comp rx=0 tx=0).stop 2024-09-10T21:24:51.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 shutdown_connections 2024-09-10T21:24:51.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87140725e0 0x7f8714074aa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87380ce640 0x7f87381f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87380b0d50 0x7f8738157840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 --2- 172.21.15.97:0/2372893770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87380afdd0 0x7f87381faef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 >> 172.21.15.97:0/2372893770 conn(0x7f87380d1c30 msgr2=0x7f87380d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 shutdown_connections 2024-09-10T21:24:51.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.216+0000 7f873ea00640 1 -- 172.21.15.97:0/2372893770 wait complete. 2024-09-10T21:24:51.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T21:24:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.384+0000 7f7477000640 1 -- 172.21.15.97:0/240756039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 msgr2=0x7f745400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.384+0000 7f7477000640 1 --2- 172.21.15.97:0/240756039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 0x7f745400b9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f746800e400 tx=0x7f746801d7d0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.384+0000 7f7475000640 1 -- 172.21.15.97:0/240756039 reap_dead start 2024-09-10T21:24:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.384+0000 7f7477000640 1 -- 172.21.15.97:0/240756039 shutdown_connections 2024-09-10T21:24:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.384+0000 7f7477000640 1 -- 172.21.15.97:0/240756039 >> 172.21.15.97:0/240756039 conn(0x7f74700b1780 msgr2=0x7f74700b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.384+0000 7f7477000640 1 -- 172.21.15.97:0/240756039 shutdown_connections 2024-09-10T21:24:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.384+0000 7f7477000640 1 -- 172.21.15.97:0/240756039 wait complete. 2024-09-10T21:24:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 Processor -- start 2024-09-10T21:24:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 -- start start 2024-09-10T21:24:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 0x7f747006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74701f7c50 0x7f74701489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74700cd480 0x7f74700cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74700cf480 con 0x7f74540051d0 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f74700cf300 con 0x7f74700cd480 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f74700cf600 con 0x7f74701f7c50 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 0x7f747006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 0x7f747006e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54314/0 (socket says 172.21.15.97:54314) 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 -- 172.21.15.97:0/1853153336 learned_addr learned my addr 172.21.15.97:0/1853153336 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7467400640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74701f7c50 0x7f74701489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7475000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74700cd480 0x7f74700cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 -- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74701f7c50 msgr2=0x7f74701489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74701f7c50 0x7f74701489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 -- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74700cd480 msgr2=0x7f74700cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74700cd480 0x7f74700cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 -- 172.21.15.97:0/1853153336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747006f1f0 con 0x7f74540051d0 2024-09-10T21:24:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7475000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74700cd480 0x7f74700cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746fe00640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 0x7f747006e910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7460006630 tx=0x7f74600039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746cc00640 1 -- 172.21.15.97:0/1853153336 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460015070 con 0x7f74540051d0 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746cc00640 1 -- 172.21.15.97:0/1853153336 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7460011900 con 0x7f74540051d0 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f746cc00640 1 -- 172.21.15.97:0/1853153336 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460011d50 con 0x7f74540051d0 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747006f4d0 con 0x7f74540051d0 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.388+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7470153040 con 0x7f74540051d0 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.392+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f747015abc0 con 0x7f74540051d0 2024-09-10T21:24:51.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.392+0000 7f746cc00640 1 -- 172.21.15.97:0/1853153336 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7460010420 con 0x7f74540051d0 2024-09-10T21:24:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.392+0000 7f746cc00640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f744c0725e0 0x7f744c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.392+0000 7f746cc00640 1 -- 172.21.15.97:0/1853153336 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7460094270 con 0x7f74540051d0 2024-09-10T21:24:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.392+0000 7f7467400640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f744c0725e0 0x7f744c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.392+0000 7f7467400640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f744c0725e0 0x7f744c074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7470152ec0 tx=0x7f74580091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.396+0000 7f746cc00640 1 -- 172.21.15.97:0/1853153336 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f746005efb0 con 0x7f74540051d0 2024-09-10T21:24:51.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.564+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f747015c2c0 con 0x7f74540051d0 2024-09-10T21:24:51.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.564+0000 7f746cc00640 1 -- 172.21.15.97:0/1853153336 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f74600639e0 con 0x7f74540051d0 2024-09-10T21:24:51.571 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f744c0725e0 msgr2=0x7f744c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f744c0725e0 0x7f744c074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7470152ec0 tx=0x7f74580091e0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 msgr2=0x7f747006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 0x7f747006e910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7460006630 tx=0x7f74600039d0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 shutdown_connections 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f744c0725e0 0x7f744c074aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74700cd480 0x7f74700cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74701f7c50 0x7f74701489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 --2- 172.21.15.97:0/1853153336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74540051d0 0x7f747006e910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 >> 172.21.15.97:0/1853153336 conn(0x7f74700b1780 msgr2=0x7f747015c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 shutdown_connections 2024-09-10T21:24:51.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.568+0000 7f7477000640 1 -- 172.21.15.97:0/1853153336 wait complete. 2024-09-10T21:24:51.593 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T21:24:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 -- 172.21.15.97:0/4056844604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5015f350 msgr2=0x7fbb500b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 --2- 172.21.15.97:0/4056844604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5015f350 0x7fbb500b3070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009840 tx=0x7fbb4401c790 comp rx=0 tx=0).stop 2024-09-10T21:24:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 -- 172.21.15.97:0/4056844604 shutdown_connections 2024-09-10T21:24:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 --2- 172.21.15.97:0/4056844604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb500d05e0 0x7fbb500cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 --2- 172.21.15.97:0/4056844604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5015f350 0x7fbb500b3070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 -- 172.21.15.97:0/4056844604 >> 172.21.15.97:0/4056844604 conn(0x7fbb500d1c60 msgr2=0x7fbb500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 -- 172.21.15.97:0/4056844604 shutdown_connections 2024-09-10T21:24:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 -- 172.21.15.97:0/4056844604 wait complete. 2024-09-10T21:24:51.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.732+0000 7fbb58600640 1 Processor -- start 2024-09-10T21:24:51.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 -- start start 2024-09-10T21:24:51.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb500cff80 0x7fbb5015dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb500d05e0 0x7fbb5015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5015f350 0x7fbb5015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50169180 con 0x7fbb500d05e0 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50169000 con 0x7fbb5015f350 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbb50169300 con 0x7fbb500cff80 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb500d05e0 0x7fbb5015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb500cff80 0x7fbb5015dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb500d05e0 0x7fbb5015a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54332/0 (socket says 172.21.15.97:54332) 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 -- 172.21.15.97:0/23443723 learned_addr learned my addr 172.21.15.97:0/23443723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb56600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5015f350 0x7fbb5015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 -- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb500cff80 msgr2=0x7fbb5015dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb500cff80 0x7fbb5015dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 -- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5015f350 msgr2=0x7fbb5015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5015f350 0x7fbb5015a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 -- 172.21.15.97:0/23443723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb501fc4f0 con 0x7fbb500d05e0 2024-09-10T21:24:51.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb55200640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb500d05e0 0x7fbb5015a040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00ea20 tx=0x7fbb3c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb4d600640 1 -- 172.21.15.97:0/23443723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c00cd80 con 0x7fbb500d05e0 2024-09-10T21:24:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb4d600640 1 -- 172.21.15.97:0/23443723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb3c010430 con 0x7fbb500d05e0 2024-09-10T21:24:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb501f8190 con 0x7fbb500d05e0 2024-09-10T21:24:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb4d600640 1 -- 172.21.15.97:0/23443723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c010860 con 0x7fbb500d05e0 2024-09-10T21:24:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb501f8750 con 0x7fbb500d05e0 2024-09-10T21:24:51.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.736+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb500b13a0 con 0x7fbb500d05e0 2024-09-10T21:24:51.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.740+0000 7fbb4d600640 1 -- 172.21.15.97:0/23443723 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbb3c03daa0 con 0x7fbb500d05e0 2024-09-10T21:24:51.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.740+0000 7fbb4d600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb300725e0 0x7fbb30074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.740+0000 7fbb4d600640 1 -- 172.21.15.97:0/23443723 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbb3c094280 con 0x7fbb500d05e0 2024-09-10T21:24:51.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.740+0000 7fbb55c00640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb300725e0 0x7fbb30074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.740+0000 7fbb55c00640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb300725e0 0x7fbb30074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb440ae000 tx=0x7fbb4401afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.744+0000 7fbb4d600640 1 -- 172.21.15.97:0/23443723 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbb3c05f140 con 0x7fbb500d05e0 2024-09-10T21:24:51.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.908+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb50050fd0 con 0x7fbb500d05e0 2024-09-10T21:24:51.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.908+0000 7fbb4d600640 1 -- 172.21.15.97:0/23443723 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fbb3c063b70 con 0x7fbb500d05e0 2024-09-10T21:24:51.916 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-09-10T21:24:51.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb300725e0 msgr2=0x7fbb30074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb300725e0 0x7fbb30074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb440ae000 tx=0x7fbb4401afd0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb500d05e0 msgr2=0x7fbb5015a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb500d05e0 0x7fbb5015a040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00ea20 tx=0x7fbb3c00eee0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 shutdown_connections 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb300725e0 0x7fbb30074aa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5015f350 0x7fbb5015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb500d05e0 0x7fbb5015a040 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 --2- 172.21.15.97:0/23443723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb500cff80 0x7fbb5015dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 >> 172.21.15.97:0/23443723 conn(0x7fbb500d1c60 msgr2=0x7fbb500b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 shutdown_connections 2024-09-10T21:24:51.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:51.912+0000 7fbb58600640 1 -- 172.21.15.97:0/23443723 wait complete. 2024-09-10T21:24:51.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T21:24:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 -- 172.21.15.97:0/3889559088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 msgr2=0x7f8298004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 --2- 172.21.15.97:0/3889559088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f8298004c90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82a00016c0 tx=0x7f82a001c730 comp rx=0 tx=0).stop 2024-09-10T21:24:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 -- 172.21.15.97:0/3889559088 shutdown_connections 2024-09-10T21:24:52.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 --2- 172.21.15.97:0/3889559088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f8298004c90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 -- 172.21.15.97:0/3889559088 >> 172.21.15.97:0/3889559088 conn(0x7f82b0074160 msgr2=0x7f82b0074570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 -- 172.21.15.97:0/3889559088 shutdown_connections 2024-09-10T21:24:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 -- 172.21.15.97:0/3889559088 wait complete. 2024-09-10T21:24:52.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 Processor -- start 2024-09-10T21:24:52.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 -- start start 2024-09-10T21:24:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.076+0000 7f82b8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f82b00c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82b01fa880 0x7f82b009a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82b009a6f0 0x7f82b0152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82b006cf60 con 0x7f8298004810 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82b006cde0 con 0x7f82b009a6f0 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f82b006d0e0 con 0x7f82b01fa880 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f82b00c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f82b00c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54336/0 (socket says 172.21.15.97:54336) 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 -- 172.21.15.97:0/1707624118 learned_addr learned my addr 172.21.15.97:0/1707624118 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5400640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82b01fa880 0x7f82b009a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b6800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82b009a6f0 0x7f82b0152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 -- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82b01fa880 msgr2=0x7f82b009a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82b01fa880 0x7f82b009a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 -- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82b009a6f0 msgr2=0x7f82b0152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82b009a6f0 0x7f82b0152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 -- 172.21.15.97:0/1707624118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b01516f0 con 0x7f8298004810 2024-09-10T21:24:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b6800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82b009a6f0 0x7f82b0152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b5e00640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f82b00c9f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82a400ea20 tx=0x7f82a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82aa000640 1 -- 172.21.15.97:0/1707624118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4004260 con 0x7f8298004810 2024-09-10T21:24:52.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82aa000640 1 -- 172.21.15.97:0/1707624118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a4004400 con 0x7f8298004810 2024-09-10T21:24:52.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82aa000640 1 -- 172.21.15.97:0/1707624118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a400dba0 con 0x7f8298004810 2024-09-10T21:24:52.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b01519d0 con 0x7f8298004810 2024-09-10T21:24:52.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b0151f90 con 0x7f8298004810 2024-09-10T21:24:52.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b006d640 con 0x7f8298004810 2024-09-10T21:24:52.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.080+0000 7f82aa000640 1 -- 172.21.15.97:0/1707624118 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f82a400b040 con 0x7f8298004810 2024-09-10T21:24:52.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.084+0000 7f82aa000640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8294072610 0x7f8294074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.084+0000 7f82aa000640 1 -- 172.21.15.97:0/1707624118 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f82a4093e30 con 0x7f8298004810 2024-09-10T21:24:52.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.084+0000 7f82b5400640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8294072610 0x7f8294074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.084+0000 7f82b5400640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8294072610 0x7f8294074ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82980057f0 tx=0x7f82a0019520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:52.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.088+0000 7f82aa000640 1 -- 172.21.15.97:0/1707624118 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f82a405f0a0 con 0x7f8298004810 2024-09-10T21:24:52.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.256+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82b00cc780 con 0x7f8298004810 2024-09-10T21:24:52.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.256+0000 7f82aa000640 1 -- 172.21.15.97:0/1707624118 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f82a405e9a0 con 0x7f8298004810 2024-09-10T21:24:52.265 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8294072610 msgr2=0x7f8294074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8294072610 0x7f8294074ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82980057f0 tx=0x7f82a0019520 comp rx=0 tx=0).stop 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 msgr2=0x7f82b00c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f82b00c9f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82a400ea20 tx=0x7f82a400ee20 comp rx=0 tx=0).stop 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 shutdown_connections 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8294072610 0x7f8294074ad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82b009a6f0 0x7f82b0152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82b01fa880 0x7f82b009a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 --2- 172.21.15.97:0/1707624118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298004810 0x7f82b00c9f70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 >> 172.21.15.97:0/1707624118 conn(0x7f82b0074160 msgr2=0x7f82b0099700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:52.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 shutdown_connections 2024-09-10T21:24:52.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.260+0000 7f82b8800640 1 -- 172.21.15.97:0/1707624118 wait complete. 2024-09-10T21:24:52.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-10T21:24:52.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 -- 172.21.15.97:0/1778820542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 msgr2=0x7f95140b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 --2- 172.21.15.97:0/1778820542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140b3070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f95140b1210 tx=0x7f950401c770 comp rx=0 tx=0).stop 2024-09-10T21:24:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 -- 172.21.15.97:0/1778820542 shutdown_connections 2024-09-10T21:24:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 --2- 172.21.15.97:0/1778820542 >> v2:172.21.15.194:3300/0 conn(0x7f95140d05e0 0x7f95140cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 --2- 172.21.15.97:0/1778820542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140b3070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 -- 172.21.15.97:0/1778820542 >> 172.21.15.97:0/1778820542 conn(0x7f95140d1c60 msgr2=0x7f95140d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 -- 172.21.15.97:0/1778820542 shutdown_connections 2024-09-10T21:24:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.420+0000 7f951a200640 1 -- 172.21.15.97:0/1778820542 wait complete. 2024-09-10T21:24:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 Processor -- start 2024-09-10T21:24:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 -- start start 2024-09-10T21:24:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95140cff80 0x7f95140cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f95140d05e0 0x7f95140c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9514169020 con 0x7f951415f350 2024-09-10T21:24:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9514168ea0 con 0x7f95140cff80 2024-09-10T21:24:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f95141691a0 con 0x7f95140d05e0 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54354/0 (socket says 172.21.15.97:54354) 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 -- 172.21.15.97:0/925509197 learned_addr learned my addr 172.21.15.97:0/925509197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f950aa00640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f95140d05e0 0x7f95140c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513400640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95140cff80 0x7f95140cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 -- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f95140d05e0 msgr2=0x7f95140c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f95140d05e0 0x7f95140c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 -- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95140cff80 msgr2=0x7f95140cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95140cff80 0x7f95140cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 -- 172.21.15.97:0/925509197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95140c9d60 con 0x7f951415f350 2024-09-10T21:24:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513400640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95140cff80 0x7f95140cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f9513e00640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140715a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f950c00cbc0 tx=0x7f950c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f950be00640 1 -- 172.21.15.97:0/925509197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c016020 con 0x7f951415f350 2024-09-10T21:24:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f950be00640 1 -- 172.21.15.97:0/925509197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f950c00ab40 con 0x7f951415f350 2024-09-10T21:24:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f950be00640 1 -- 172.21.15.97:0/925509197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c012420 con 0x7f951415f350 2024-09-10T21:24:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95140ca040 con 0x7f951415f350 2024-09-10T21:24:52.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.424+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95140ca320 con 0x7f951415f350 2024-09-10T21:24:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.428+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95140cfa40 con 0x7f951415f350 2024-09-10T21:24:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.428+0000 7f950be00640 1 -- 172.21.15.97:0/925509197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f950c01b400 con 0x7f951415f350 2024-09-10T21:24:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.428+0000 7f950be00640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94f00725e0 0x7f94f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.428+0000 7f950be00640 1 -- 172.21.15.97:0/925509197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6590+0+0 (secure 0 0 0) 0x7f950c093b10 con 0x7f951415f350 2024-09-10T21:24:52.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.428+0000 7f9513400640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94f00725e0 0x7f94f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.428+0000 7f9513400640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94f00725e0 0x7f94f0074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95040af000 tx=0x7f950401afb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:52.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.432+0000 7f950be00640 1 -- 172.21.15.97:0/925509197 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f950c020020 con 0x7f951415f350 2024-09-10T21:24:52.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:52.660+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f95140b13a0 con 0x7f951415f350 2024-09-10T21:24:53.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.200+0000 7f950be00640 1 -- 172.21.15.97:0/925509197 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v57) ==== 137+0+0 (secure 0 0 0) 0x7f950c05ee70 con 0x7f951415f350 2024-09-10T21:24:53.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94f00725e0 msgr2=0x7f94f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94f00725e0 0x7f94f0074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95040af000 tx=0x7f950401afb0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 msgr2=0x7f95140715a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140715a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f950c00cbc0 tx=0x7f950c00ccb0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 shutdown_connections 2024-09-10T21:24:53.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94f00725e0 0x7f94f0074aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f951415f350 0x7f95140715a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f95140d05e0 0x7f95140c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 --2- 172.21.15.97:0/925509197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95140cff80 0x7f95140cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 >> 172.21.15.97:0/925509197 conn(0x7f95140d1c60 msgr2=0x7f95140ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 shutdown_connections 2024-09-10T21:24:53.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.204+0000 7f951a200640 1 -- 172.21.15.97:0/925509197 wait complete. 2024-09-10T21:24:53.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:24:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- 172.21.15.97:0/2744692423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c0d0150 msgr2=0x7f1b3c0afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 --2- 172.21.15.97:0/2744692423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c0d0150 0x7f1b3c0afc40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009bc0 tx=0x7f1b3401cb10 comp rx=0 tx=0).stop 2024-09-10T21:24:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- 172.21.15.97:0/2744692423 shutdown_connections 2024-09-10T21:24:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 --2- 172.21.15.97:0/2744692423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c0d0150 0x7f1b3c0afc40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 --2- 172.21.15.97:0/2744692423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b3c15f350 0x7f1b3c0b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- 172.21.15.97:0/2744692423 >> 172.21.15.97:0/2744692423 conn(0x7f1b3c0d1c60 msgr2=0x7f1b3c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- 172.21.15.97:0/2744692423 shutdown_connections 2024-09-10T21:24:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- 172.21.15.97:0/2744692423 wait complete. 2024-09-10T21:24:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 Processor -- start 2024-09-10T21:24:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- start start 2024-09-10T21:24:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b3c0d0150 0x7f1b3c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b3c0d05e0 0x7f1b3c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c15f350 0x7f1b3c0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c168c30 con 0x7f1b3c15f350 2024-09-10T21:24:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c0b0ee0 con 0x7f1b3c0d05e0 2024-09-10T21:24:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b42a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c168db0 con 0x7f1b3c0d0150 2024-09-10T21:24:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b40a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c15f350 0x7f1b3c0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.376+0000 7f1b40a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c15f350 0x7f1b3c0b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54372/0 (socket says 172.21.15.97:54372) 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b40a00640 1 -- 172.21.15.97:0/730929266 learned_addr learned my addr 172.21.15.97:0/730929266 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b3be00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b3c0d0150 0x7f1b3c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b3b400640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b3c0d05e0 0x7f1b3c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b40a00640 1 -- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b3c0d0150 msgr2=0x7f1b3c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b40a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b3c0d0150 0x7f1b3c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b40a00640 1 -- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b3c0d05e0 msgr2=0x7f1b3c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b40a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b3c0d05e0 0x7f1b3c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b40a00640 1 -- 172.21.15.97:0/730929266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c148ae0 con 0x7f1b3c15f350 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b3b400640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b3c0d05e0 0x7f1b3c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b40a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c15f350 0x7f1b3c0b1b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009b10 tx=0x7f1b340a5a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b33e00640 1 -- 172.21.15.97:0/730929266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340bb070 con 0x7f1b3c15f350 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b33e00640 1 -- 172.21.15.97:0/730929266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b34004e00 con 0x7f1b3c15f350 2024-09-10T21:24:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b33e00640 1 -- 172.21.15.97:0/730929266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340b2040 con 0x7f1b3c15f350 2024-09-10T21:24:53.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c148dc0 con 0x7f1b3c15f350 2024-09-10T21:24:53.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c149380 con 0x7f1b3c15f350 2024-09-10T21:24:53.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0afc40 con 0x7f1b3c15f350 2024-09-10T21:24:53.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b33e00640 1 -- 172.21.15.97:0/730929266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1b340b5030 con 0x7f1b3c15f350 2024-09-10T21:24:53.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.380+0000 7f1b33e00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b1c072610 0x7f1b1c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.384+0000 7f1b33e00640 1 -- 172.21.15.97:0/730929266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f1b3412d410 con 0x7f1b3c15f350 2024-09-10T21:24:53.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.384+0000 7f1b3be00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b1c072610 0x7f1b1c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.384+0000 7f1b3be00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b1c072610 0x7f1b1c074ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006fd0 tx=0x7f1b2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.384+0000 7f1b33e00640 1 -- 172.21.15.97:0/730929266 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1b34016790 con 0x7f1b3c15f350 2024-09-10T21:24:53.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.552+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1b3c0b13a0 con 0x7f1b3c15f350 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b33e00640 1 -- 172.21.15.97:0/730929266 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f1b340f86e0 con 0x7f1b3c15f350 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3032.483965 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3656.533946 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3351.408335 2024-09-10T21:24:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3363.485497 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 8164.284498 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4391.956543 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4391.204852 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4799.171341 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds 20/20 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_ms 2/2 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_frag true 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_verify_scatter true 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:24:53.562 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced client_mount_timeout 600 2024-09-10T21:24:53.563 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_client 20/20 2024-09-10T21:24:53.563 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_ms 1/1 2024-09-10T21:24:53.563 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:24:53.563 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b1c072610 msgr2=0x7f1b1c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b1c072610 0x7f1b1c074ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006fd0 tx=0x7f1b2c008040 comp rx=0 tx=0).stop 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c15f350 msgr2=0x7f1b3c0b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c15f350 0x7f1b3c0b1b00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009b10 tx=0x7f1b340a5a30 comp rx=0 tx=0).stop 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 shutdown_connections 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b1c072610 0x7f1b1c074ad0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c15f350 0x7f1b3c0b1b00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b3c0d05e0 0x7f1b3c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 --2- 172.21.15.97:0/730929266 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b3c0d0150 0x7f1b3c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 >> 172.21.15.97:0/730929266 conn(0x7f1b3c0d1c60 msgr2=0x7f1b3c15c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 shutdown_connections 2024-09-10T21:24:53.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.556+0000 7f1b42a00640 1 -- 172.21.15.97:0/730929266 wait complete. 2024-09-10T21:24:53.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- 172.21.15.97:0/1760623312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140d0490 msgr2=0x7fc8140ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 --2- 172.21.15.97:0/1760623312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140d0490 0x7fc8140ae930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc8140b3410 tx=0x7fc80c01caf0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- 172.21.15.97:0/1760623312 shutdown_connections 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 --2- 172.21.15.97:0/1760623312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140d0490 0x7fc8140ae930 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 --2- 172.21.15.97:0/1760623312 >> v2:172.21.15.194:3300/0 conn(0x7fc8140cdcb0 0x7fc8140cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- 172.21.15.97:0/1760623312 >> 172.21.15.97:0/1760623312 conn(0x7fc8140d1a60 msgr2=0x7fc8140d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- 172.21.15.97:0/1760623312 shutdown_connections 2024-09-10T21:24:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- 172.21.15.97:0/1760623312 wait complete. 2024-09-10T21:24:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 Processor -- start 2024-09-10T21:24:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- start start 2024-09-10T21:24:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140cdcb0 0x7fc814153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8140d0490 0x7fc8140c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81415efc0 0x7fc81406f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc814168930 con 0x7fc8140cdcb0 2024-09-10T21:24:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141687b0 con 0x7fc81415efc0 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc819800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc814168ab0 con 0x7fc8140d0490 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc813e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140cdcb0 0x7fc814153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc813e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140cdcb0 0x7fc814153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54386/0 (socket says 172.21.15.97:54386) 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.720+0000 7fc813e00640 1 -- 172.21.15.97:0/3567035933 learned_addr learned my addr 172.21.15.97:0/3567035933 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc80be00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8140d0490 0x7fc8140c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc818e00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81415efc0 0x7fc81406f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc813e00640 1 -- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8140d0490 msgr2=0x7fc8140c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc813e00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8140d0490 0x7fc8140c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc813e00640 1 -- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81415efc0 msgr2=0x7fc81406f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc813e00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81415efc0 0x7fc81406f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc813e00640 1 -- 172.21.15.97:0/3567035933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc814153c90 con 0x7fc8140cdcb0 2024-09-10T21:24:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc813e00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140cdcb0 0x7fc814153750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c400 tx=0x7fc80400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc810c00640 1 -- 172.21.15.97:0/3567035933 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804017070 con 0x7fc8140cdcb0 2024-09-10T21:24:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc810c00640 1 -- 172.21.15.97:0/3567035933 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc804003600 con 0x7fc8140cdcb0 2024-09-10T21:24:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8140c74d0 con 0x7fc8140cdcb0 2024-09-10T21:24:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc810c00640 1 -- 172.21.15.97:0/3567035933 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804003a50 con 0x7fc8140cdcb0 2024-09-10T21:24:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8140c59e0 con 0x7fc8140cdcb0 2024-09-10T21:24:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8140ae930 con 0x7fc8140cdcb0 2024-09-10T21:24:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc810c00640 1 -- 172.21.15.97:0/3567035933 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc804003bf0 con 0x7fc8140cdcb0 2024-09-10T21:24:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.724+0000 7fc810c00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7f00725e0 0x7fc7f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.728+0000 7fc810c00640 1 -- 172.21.15.97:0/3567035933 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fc804092c10 con 0x7fc8140cdcb0 2024-09-10T21:24:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.728+0000 7fc80be00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7f00725e0 0x7fc7f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.728+0000 7fc80be00640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7f00725e0 0x7fc7f0074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc814157d20 tx=0x7fc7fc0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.728+0000 7fc810c00640 1 -- 172.21.15.97:0/3567035933 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc80405df80 con 0x7fc8140cdcb0 2024-09-10T21:24:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.896+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc8140b3c40 con 0x7fc8140cdcb0 2024-09-10T21:24:53.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.900+0000 7fc810c00640 1 -- 172.21.15.97:0/3567035933 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fc80405d880 con 0x7fc8140cdcb0 2024-09-10T21:24:53.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7f00725e0 msgr2=0x7fc7f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7f00725e0 0x7fc7f0074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc814157d20 tx=0x7fc7fc0091e0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140cdcb0 msgr2=0x7fc814153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140cdcb0 0x7fc814153750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c400 tx=0x7fc80400c8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 shutdown_connections 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7f00725e0 0x7fc7f0074aa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc81415efc0 0x7fc81406f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8140d0490 0x7fc8140c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 --2- 172.21.15.97:0/3567035933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8140cdcb0 0x7fc814153750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 >> 172.21.15.97:0/3567035933 conn(0x7fc8140d1a60 msgr2=0x7fc81415a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 shutdown_connections 2024-09-10T21:24:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:53.904+0000 7fc819800640 1 -- 172.21.15.97:0/3567035933 wait complete. 2024-09-10T21:24:53.929 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:24:53.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:24:54.043 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:24:54.120 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:24:54.120 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:54.120 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:54.129 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:54.129 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:54.184 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:54.185 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:54.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:24:54.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 -- 172.21.15.97:0/4115583367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700ce5e0 msgr2=0x7f09700b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 --2- 172.21.15.97:0/4115583367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700ce5e0 0x7f09700b2010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09700d1c70 tx=0x7f096001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 -- 172.21.15.97:0/4115583367 shutdown_connections 2024-09-10T21:24:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 --2- 172.21.15.97:0/4115583367 >> v2:172.21.15.194:3300/0 conn(0x7f09700cde30 0x7f09700cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 --2- 172.21.15.97:0/4115583367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700ce5e0 0x7f09700b2010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 -- 172.21.15.97:0/4115583367 >> 172.21.15.97:0/4115583367 conn(0x7f09700d1ca0 msgr2=0x7f09700d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 -- 172.21.15.97:0/4115583367 shutdown_connections 2024-09-10T21:24:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.420+0000 7f0974c00640 1 -- 172.21.15.97:0/4115583367 wait complete. 2024-09-10T21:24:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 Processor -- start 2024-09-10T21:24:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 -- start start 2024-09-10T21:24:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700cde30 0x7f0970150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09700ce5e0 0x7f09700b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09700d0710 0x7f09700ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f09700cd350 con 0x7f09700cde30 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f09700cd1d0 con 0x7f09700ce5e0 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0970169060 con 0x7f09700d0710 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700cde30 0x7f0970150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700cde30 0x7f0970150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54408/0 (socket says 172.21.15.97:54408) 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 -- 172.21.15.97:0/1617736852 learned_addr learned my addr 172.21.15.97:0/1617736852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096e800640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09700d0710 0x7f09700ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0967e00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09700ce5e0 0x7f09700b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 -- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09700d0710 msgr2=0x7f09700ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09700d0710 0x7f09700ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 -- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09700ce5e0 msgr2=0x7f09700b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09700ce5e0 0x7f09700b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 -- 172.21.15.97:0/1617736852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09700c99d0 con 0x7f09700cde30 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f096de00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700cde30 0x7f0970150f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09600b0040 tx=0x7f09600a55d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0966000640 1 -- 172.21.15.97:0/1617736852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600b2040 con 0x7f09700cde30 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0966000640 1 -- 172.21.15.97:0/1617736852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0960004470 con 0x7f09700cde30 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09700c9cb0 con 0x7f09700cde30 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0966000640 1 -- 172.21.15.97:0/1617736852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600052d0 con 0x7f09700cde30 2024-09-10T21:24:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09701514c0 con 0x7f09700cde30 2024-09-10T21:24:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.424+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09700b2010 con 0x7f09700cde30 2024-09-10T21:24:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.428+0000 7f0966000640 1 -- 172.21.15.97:0/1617736852 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0960005470 con 0x7f09700cde30 2024-09-10T21:24:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.428+0000 7f0966000640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f094c0725e0 0x7f094c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.428+0000 7f0966000640 1 -- 172.21.15.97:0/1617736852 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f096012cc00 con 0x7f09700cde30 2024-09-10T21:24:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.428+0000 7f0967e00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f094c0725e0 0x7f094c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.428+0000 7f0967e00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f094c0725e0 0x7f094c074aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f09701fad00 tx=0x7f0958008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.432+0000 7f0966000640 1 -- 172.21.15.97:0/1617736852 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09600164b0 con 0x7f09700cde30 2024-09-10T21:24:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.608+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09700c9710 con 0x7f09700cde30 2024-09-10T21:24:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.608+0000 7f0966000640 1 -- 172.21.15.97:0/1617736852 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f09600f7ee0 con 0x7f09700cde30 2024-09-10T21:24:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f094c0725e0 msgr2=0x7f094c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f094c0725e0 0x7f094c074aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f09701fad00 tx=0x7f0958008040 comp rx=0 tx=0).stop 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700cde30 msgr2=0x7f0970150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700cde30 0x7f0970150f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09600b0040 tx=0x7f09600a55d0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 shutdown_connections 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f094c0725e0 0x7f094c074aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09700d0710 0x7f09700ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09700ce5e0 0x7f09700b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 --2- 172.21.15.97:0/1617736852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09700cde30 0x7f0970150f80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 >> 172.21.15.97:0/1617736852 conn(0x7f09700d1ca0 msgr2=0x7f09700ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 shutdown_connections 2024-09-10T21:24:54.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.612+0000 7f0974c00640 1 -- 172.21.15.97:0/1617736852 wait complete. 2024-09-10T21:24:54.637 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:24:54.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:54.637 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:24:54.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:24:54.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:54.676 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:24:54.726 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:54.726 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:54.735 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:54.736 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:54.790 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:54.790 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:54.842 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:24:54.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- 172.21.15.97:0/407394102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0a40051d0 msgr2=0x7fc0a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/407394102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0a40051d0 0x7fc0a400b9d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8001dc0 tx=0x7fc0b801cb30 comp rx=0 tx=0).stop 2024-09-10T21:24:54.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- 172.21.15.97:0/407394102 shutdown_connections 2024-09-10T21:24:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/407394102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0a40051d0 0x7fc0a400b9d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- 172.21.15.97:0/407394102 >> 172.21.15.97:0/407394102 conn(0x7fc0c00b1780 msgr2=0x7fc0c00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- 172.21.15.97:0/407394102 shutdown_connections 2024-09-10T21:24:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- 172.21.15.97:0/407394102 wait complete. 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 Processor -- start 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- start start 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0a40051d0 0x7fc0c0140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c013be70 0x7fc0c00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c013c2f0 0x7fc0c00cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c00b2e10 con 0x7fc0c013be70 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c00b2c90 con 0x7fc0c013c2f0 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.980+0000 7fc0c5a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c0144480 con 0x7fc0a40051d0 2024-09-10T21:24:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c013be70 0x7fc0c00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c013be70 0x7fc0c00cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54422/0 (socket says 172.21.15.97:54422) 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0a40051d0 0x7fc0c0140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b6000640 1 -- 172.21.15.97:0/2412894854 learned_addr learned my addr 172.21.15.97:0/2412894854 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bf400640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c013c2f0 0x7fc0c00cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bea00640 1 -- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c013c2f0 msgr2=0x7fc0c00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bea00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c013c2f0 0x7fc0c00cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bea00640 1 -- 172.21.15.97:0/2412894854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c013be70 msgr2=0x7fc0c00cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bea00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c013be70 0x7fc0c00cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bea00640 1 -- 172.21.15.97:0/2412894854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c0159b30 con 0x7fc0a40051d0 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b6000640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c013be70 0x7fc0c00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0bea00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0a40051d0 0x7fc0c0140230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0011800 tx=0x7fc0b0011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b7400640 1 -- 172.21.15.97:0/2412894854 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0004460 con 0x7fc0a40051d0 2024-09-10T21:24:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b7400640 1 -- 172.21.15.97:0/2412894854 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0b000f9a0 con 0x7fc0a40051d0 2024-09-10T21:24:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c013d640 con 0x7fc0a40051d0 2024-09-10T21:24:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b7400640 1 -- 172.21.15.97:0/2412894854 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000fdb0 con 0x7fc0a40051d0 2024-09-10T21:24:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c01428d0 con 0x7fc0a40051d0 2024-09-10T21:24:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c01430b0 con 0x7fc0a40051d0 2024-09-10T21:24:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.984+0000 7fc0b7400640 1 -- 172.21.15.97:0/2412894854 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc0b0018070 con 0x7fc0a40051d0 2024-09-10T21:24:54.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.988+0000 7fc0b7400640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc09c0725e0 0x7fc09c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.988+0000 7fc0b7400640 1 -- 172.21.15.97:0/2412894854 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fc0b0094240 con 0x7fc0a40051d0 2024-09-10T21:24:54.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.988+0000 7fc0b6000640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc09c0725e0 0x7fc09c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.988+0000 7fc0b6000640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc09c0725e0 0x7fc09c074aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8001230 tx=0x7fc0a80067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:54.992+0000 7fc0b7400640 1 -- 172.21.15.97:0/2412894854 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc0c01430b0 con 0x7fc0a40051d0 2024-09-10T21:24:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.168+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc0c0070470 con 0x7fc0a40051d0 2024-09-10T21:24:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.168+0000 7fc0b7400640 1 -- 172.21.15.97:0/2412894854 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fc0b005ef50 con 0x7fc0a40051d0 2024-09-10T21:24:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc09c0725e0 msgr2=0x7fc09c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc09c0725e0 0x7fc09c074aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8001230 tx=0x7fc0a80067b0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0a40051d0 msgr2=0x7fc0c0140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0a40051d0 0x7fc0c0140230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0011800 tx=0x7fc0b0011cc0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 shutdown_connections 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc09c0725e0 0x7fc09c074aa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0c013c2f0 0x7fc0c00cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0c013be70 0x7fc0c00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 --2- 172.21.15.97:0/2412894854 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0a40051d0 0x7fc0c0140230 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 >> 172.21.15.97:0/2412894854 conn(0x7fc0c00b1780 msgr2=0x7fc0c0072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 shutdown_connections 2024-09-10T21:24:55.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.172+0000 7fc0c5a00640 1 -- 172.21.15.97:0/2412894854 wait complete. 2024-09-10T21:24:55.197 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:24:55.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:55.197 DEBUG:teuthology.orchestra.run.smithi194:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T21:24:55.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:24:55.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:55.234 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:24:55.298 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:55.298 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:55.307 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:55.307 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:55.360 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:24:55.360 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:55.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:24:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- 172.21.15.97:0/771602661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 msgr2=0x7fea68141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 --2- 172.21.15.97:0/771602661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea68141d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea58009900 tx=0x7fea5801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- 172.21.15.97:0/771602661 shutdown_connections 2024-09-10T21:24:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 --2- 172.21.15.97:0/771602661 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea6815fde0 0x7fea6813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 --2- 172.21.15.97:0/771602661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea68141d20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- 172.21.15.97:0/771602661 >> 172.21.15.97:0/771602661 conn(0x7fea680b1780 msgr2=0x7fea680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- 172.21.15.97:0/771602661 shutdown_connections 2024-09-10T21:24:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- 172.21.15.97:0/771602661 wait complete. 2024-09-10T21:24:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 Processor -- start 2024-09-10T21:24:55.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- start start 2024-09-10T21:24:55.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea681351c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea680ce9d0 0x7fea681357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea6815fde0 0x7fea68135d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea68168bf0 con 0x7fea680ce5f0 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea68168a70 con 0x7fea6815fde0 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea6e000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fea68168d70 con 0x7fea680ce9d0 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.596+0000 7fea67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea681351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea681351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54438/0 (socket says 172.21.15.97:54438) 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 -- 172.21.15.97:0/3764917129 learned_addr learned my addr 172.21.15.97:0/3764917129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea66a00640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea680ce9d0 0x7fea681357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67e00640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea6815fde0 0x7fea68135d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 -- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea680ce9d0 msgr2=0x7fea681357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:55.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea680ce9d0 0x7fea681357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 -- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea6815fde0 msgr2=0x7fea68135d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea6815fde0 0x7fea68135d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 -- 172.21.15.97:0/3764917129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6815eb90 con 0x7fea680ce5f0 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea67400640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea681351c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea58009850 tx=0x7fea58004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea5f400640 1 -- 172.21.15.97:0/3764917129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580b2040 con 0x7fea680ce5f0 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea5f400640 1 -- 172.21.15.97:0/3764917129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea58005030 con 0x7fea680ce5f0 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6815ee70 con 0x7fea680ce5f0 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea5f400640 1 -- 172.21.15.97:0/3764917129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58005440 con 0x7fea680ce5f0 2024-09-10T21:24:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea68156e10 con 0x7fea680ce5f0 2024-09-10T21:24:55.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea68141d20 con 0x7fea680ce5f0 2024-09-10T21:24:55.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.600+0000 7fea5f400640 1 -- 172.21.15.97:0/3764917129 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fea580c0060 con 0x7fea680ce5f0 2024-09-10T21:24:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.604+0000 7fea5f400640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea480725e0 0x7fea48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.604+0000 7fea5f400640 1 -- 172.21.15.97:0/3764917129 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fea5812cd10 con 0x7fea680ce5f0 2024-09-10T21:24:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.604+0000 7fea66a00640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea480725e0 0x7fea48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.604+0000 7fea66a00640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea480725e0 0x7fea48074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea6813cbb0 tx=0x7fea54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:55.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.608+0000 7fea5f400640 1 -- 172.21.15.97:0/3764917129 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea58132020 con 0x7fea680ce5f0 2024-09-10T21:24:55.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea68141360 con 0x7fea680ce5f0 2024-09-10T21:24:55.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea5f400640 1 -- 172.21.15.97:0/3764917129 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fea580164d0 con 0x7fea680ce5f0 2024-09-10T21:24:55.794 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:24:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea480725e0 msgr2=0x7fea48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea480725e0 0x7fea48074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea6813cbb0 tx=0x7fea54009040 comp rx=0 tx=0).stop 2024-09-10T21:24:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 msgr2=0x7fea681351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea681351c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea58009850 tx=0x7fea58004040 comp rx=0 tx=0).stop 2024-09-10T21:24:55.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 shutdown_connections 2024-09-10T21:24:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea480725e0 0x7fea48074aa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea6815fde0 0x7fea68135d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea680ce9d0 0x7fea681357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 --2- 172.21.15.97:0/3764917129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea680ce5f0 0x7fea681351c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 >> 172.21.15.97:0/3764917129 conn(0x7fea680b1780 msgr2=0x7fea68072830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 shutdown_connections 2024-09-10T21:24:55.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:55.788+0000 7fea6e000640 1 -- 172.21.15.97:0/3764917129 wait complete. 2024-09-10T21:24:55.815 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T21:24:55.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:55.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T21:24:55.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:24:55.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:55.840 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T21:24:55.890 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:55.890 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:55.900 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:55.900 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:55.949 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:24:55.949 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:24:56.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:24:56.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.136+0000 7fbccea00640 1 -- 172.21.15.97:0/1555779447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 msgr2=0x7fbcac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.136+0000 7fbccea00640 1 --2- 172.21.15.97:0/1555779447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 0x7fbcac004c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40016c0 tx=0x7fbcb401c790 comp rx=0 tx=0).stop 2024-09-10T21:24:56.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.136+0000 7fbccca00640 1 -- 172.21.15.97:0/1555779447 reap_dead start 2024-09-10T21:24:56.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.136+0000 7fbccea00640 1 -- 172.21.15.97:0/1555779447 shutdown_connections 2024-09-10T21:24:56.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.136+0000 7fbccea00640 1 -- 172.21.15.97:0/1555779447 >> 172.21.15.97:0/1555779447 conn(0x7fbcc80d1ca0 msgr2=0x7fbcc80d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:56.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.136+0000 7fbccea00640 1 -- 172.21.15.97:0/1555779447 shutdown_connections 2024-09-10T21:24:56.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.136+0000 7fbccea00640 1 -- 172.21.15.97:0/1555779447 wait complete. 2024-09-10T21:24:56.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 Processor -- start 2024-09-10T21:24:56.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 -- start start 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 0x7fbcc81f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcc81f7920 0x7fbcc81f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcc80b6cc0 0x7fbcc80b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80b2190 con 0x7fbcac004810 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80b2010 con 0x7fbcc80b6cc0 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc80b2310 con 0x7fbcc81f7920 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 0x7fbcc81f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 0x7fbcc81f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54460/0 (socket says 172.21.15.97:54460) 2024-09-10T21:24:56.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 -- 172.21.15.97:0/54993551 learned_addr learned my addr 172.21.15.97:0/54993551 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7400640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcc81f7920 0x7fbcc81f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccca00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcc80b6cc0 0x7fbcc80b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 -- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcc81f7920 msgr2=0x7fbcc81f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcc81f7920 0x7fbcc81f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 -- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcc80b6cc0 msgr2=0x7fbcc80b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcc80b6cc0 0x7fbcc80b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 -- 172.21.15.97:0/54993551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc815a7f0 con 0x7fbcac004810 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccca00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcc80b6cc0 0x7fbcc80b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcc7e00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 0x7fbcc81f5600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800f9d0 tx=0x7fbcb800fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcbfe00640 1 -- 172.21.15.97:0/54993551 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb800bcb0 con 0x7fbcac004810 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcbfe00640 1 -- 172.21.15.97:0/54993551 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcb8011440 con 0x7fbcac004810 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc815aad0 con 0x7fbcac004810 2024-09-10T21:24:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbcbfe00640 1 -- 172.21.15.97:0/54993551 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8011850 con 0x7fbcac004810 2024-09-10T21:24:56.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.140+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc815c4b0 con 0x7fbcac004810 2024-09-10T21:24:56.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.144+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc80cf450 con 0x7fbcac004810 2024-09-10T21:24:56.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.144+0000 7fbcbfe00640 1 -- 172.21.15.97:0/54993551 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbcb8023030 con 0x7fbcac004810 2024-09-10T21:24:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.144+0000 7fbcbfe00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbca8072610 0x7fbca8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.144+0000 7fbcbfe00640 1 -- 172.21.15.97:0/54993551 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fbcb8093fa0 con 0x7fbcac004810 2024-09-10T21:24:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.144+0000 7fbcc7400640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbca8072610 0x7fbca8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.144+0000 7fbcc7400640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbca8072610 0x7fbca8074ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8073250 tx=0x7fbcb40b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.148+0000 7fbcbfe00640 1 -- 172.21.15.97:0/54993551 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbcb805ec20 con 0x7fbcac004810 2024-09-10T21:24:56.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.328+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbcc80d0060 con 0x7fbcac004810 2024-09-10T21:24:56.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.328+0000 7fbcbfe00640 1 -- 172.21.15.97:0/54993551 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fbcb8063650 con 0x7fbcac004810 2024-09-10T21:24:56.336 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:24:56.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbca8072610 msgr2=0x7fbca8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbca8072610 0x7fbca8074ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8073250 tx=0x7fbcb40b0040 comp rx=0 tx=0).stop 2024-09-10T21:24:56.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 msgr2=0x7fbcc81f5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 0x7fbcc81f5600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800f9d0 tx=0x7fbcb800fe90 comp rx=0 tx=0).stop 2024-09-10T21:24:56.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 shutdown_connections 2024-09-10T21:24:56.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbca8072610 0x7fbca8074ad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcc80b6cc0 0x7fbcc80b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcc81f7920 0x7fbcc81f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 --2- 172.21.15.97:0/54993551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcac004810 0x7fbcc81f5600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 >> 172.21.15.97:0/54993551 conn(0x7fbcc80d1ca0 msgr2=0x7fbcc815e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:56.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 shutdown_connections 2024-09-10T21:24:56.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.332+0000 7fbccea00640 1 -- 172.21.15.97:0/54993551 wait complete. 2024-09-10T21:24:56.357 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T21:24:56.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:56.357 DEBUG:teuthology.orchestra.run.smithi194:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T21:24:56.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:24:56.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:24:56.384 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T21:24:56.434 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:24:56.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:56.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- 172.21.15.97:0/245030081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 msgr2=0x7fb7e0141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/245030081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0141d20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009900 tx=0x7fb7d001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- 172.21.15.97:0/245030081 shutdown_connections 2024-09-10T21:24:56.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/245030081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0141d20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- 172.21.15.97:0/245030081 >> 172.21.15.97:0/245030081 conn(0x7fb7e00b1780 msgr2=0x7fb7e00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:56.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- 172.21.15.97:0/245030081 shutdown_connections 2024-09-10T21:24:56.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- 172.21.15.97:0/245030081 wait complete. 2024-09-10T21:24:56.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 Processor -- start 2024-09-10T21:24:56.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- start start 2024-09-10T21:24:56.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0165830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7e00ce9d0 0x7fb7e013ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7e015fde0 0x7fb7e01f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0169dc0 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0169c40 con 0x7fb7e00ce9d0 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7e4e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e0169f40 con 0x7fb7e015fde0 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0165830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0165830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54486/0 (socket says 172.21.15.97:54486) 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 -- 172.21.15.97:0/1903586138 learned_addr learned my addr 172.21.15.97:0/1903586138 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7de800640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7e015fde0 0x7fb7e01f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7d5400640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7e00ce9d0 0x7fb7e013ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 -- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7e015fde0 msgr2=0x7fb7e01f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7e015fde0 0x7fb7e01f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 -- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7e00ce9d0 msgr2=0x7fb7e013ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7e00ce9d0 0x7fb7e013ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 -- 172.21.15.97:0/1903586138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e0166640 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.572+0000 7fb7dde00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0165830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009850 tx=0x7fb7d0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7d6a00640 1 -- 172.21.15.97:0/1903586138 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00b2040 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7d6a00640 1 -- 172.21.15.97:0/1903586138 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7d0005030 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e0166080 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7d6a00640 1 -- 172.21.15.97:0/1903586138 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0005440 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e00b3440 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e0141d20 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7d6a00640 1 -- 172.21.15.97:0/1903586138 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb7d00046d0 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7d6a00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7bc072610 0x7fb7bc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7d6a00640 1 -- 172.21.15.97:0/1903586138 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fb7d012dab0 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.576+0000 7fb7d5400640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7bc072610 0x7fb7bc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.580+0000 7fb7d5400640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7bc072610 0x7fb7bc074ad0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00b3180 tx=0x7fb7c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.580+0000 7fb7d6a00640 1 -- 172.21.15.97:0/1903586138 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb7d00164d0 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7e0141360 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7d6a00640 1 -- 172.21.15.97:0/1903586138 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7fb7d00f8d90 con 0x7fb7e00ce5f0 2024-09-10T21:24:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:56.787 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7bc072610 msgr2=0x7fb7bc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7bc072610 0x7fb7bc074ad0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00b3180 tx=0x7fb7c8009040 comp rx=0 tx=0).stop 2024-09-10T21:24:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 msgr2=0x7fb7e0165830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0165830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009850 tx=0x7fb7d0004040 comp rx=0 tx=0).stop 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 shutdown_connections 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7bc072610 0x7fb7bc074ad0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7e015fde0 0x7fb7e01f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7e00ce9d0 0x7fb7e013ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 --2- 172.21.15.97:0/1903586138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e00ce5f0 0x7fb7e0165830 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 >> 172.21.15.97:0/1903586138 conn(0x7fb7e00b1780 msgr2=0x7fb7e015bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 shutdown_connections 2024-09-10T21:24:56.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.780+0000 7fb7e4e00640 1 -- 172.21.15.97:0/1903586138 wait complete. 2024-09-10T21:24:56.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- 172.21.15.97:0/3342113475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 msgr2=0x7f77c40cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 --2- 172.21.15.97:0/3342113475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40cc6d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77c40c55b0 tx=0x7f77b001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- 172.21.15.97:0/3342113475 shutdown_connections 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 --2- 172.21.15.97:0/3342113475 >> v2:172.21.15.194:3300/0 conn(0x7f77c40b0d50 0x7f77c40ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 --2- 172.21.15.97:0/3342113475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40cc6d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- 172.21.15.97:0/3342113475 >> 172.21.15.97:0/3342113475 conn(0x7f77c40d1c30 msgr2=0x7f77c40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- 172.21.15.97:0/3342113475 shutdown_connections 2024-09-10T21:24:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- 172.21.15.97:0/3342113475 wait complete. 2024-09-10T21:24:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 Processor -- start 2024-09-10T21:24:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- start start 2024-09-10T21:24:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77c40b0d50 0x7f77c4071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c40ce640 0x7f77c41f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4168ee0 con 0x7f77c40afdd0 2024-09-10T21:24:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4168d60 con 0x7f77c40ce640 2024-09-10T21:24:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77cc000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f77c4169060 con 0x7f77c40b0d50 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54496/0 (socket says 172.21.15.97:54496) 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 -- 172.21.15.97:0/96410319 learned_addr learned my addr 172.21.15.97:0/96410319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c8c00640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77c40b0d50 0x7f77c4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77ca000640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c40ce640 0x7f77c41f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 -- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77c40b0d50 msgr2=0x7f77c4071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77c40b0d50 0x7f77c4071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 -- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c40ce640 msgr2=0x7f77c41f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c40ce640 0x7f77c41f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.944+0000 7f77c9600640 1 -- 172.21.15.97:0/96410319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c41f61c0 con 0x7f77c40afdd0 2024-09-10T21:24:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c9600640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40c96a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f77b800c3c0 tx=0x7f77b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c1600640 1 -- 172.21.15.97:0/96410319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8017070 con 0x7f77c40afdd0 2024-09-10T21:24:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c1600640 1 -- 172.21.15.97:0/96410319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b80053c0 con 0x7f77c40afdd0 2024-09-10T21:24:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c41f64a0 con 0x7f77c40afdd0 2024-09-10T21:24:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c1600640 1 -- 172.21.15.97:0/96410319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8003860 con 0x7f77c40afdd0 2024-09-10T21:24:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c40c9180 con 0x7f77c40afdd0 2024-09-10T21:24:56.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c40cea20 con 0x7f77c40afdd0 2024-09-10T21:24:56.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c1600640 1 -- 172.21.15.97:0/96410319 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f77b8003a00 con 0x7f77c40afdd0 2024-09-10T21:24:56.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c1600640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77a40725e0 0x7f77a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c1600640 1 -- 172.21.15.97:0/96410319 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f77b8092b40 con 0x7f77c40afdd0 2024-09-10T21:24:56.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.948+0000 7f77c8c00640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77a40725e0 0x7f77a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.952+0000 7f77c8c00640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77a40725e0 0x7f77a4074aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f77b00af000 tx=0x7f77b001b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:56.952+0000 7f77c1600640 1 -- 172.21.15.97:0/96410319 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77b805d850 con 0x7f77c40afdd0 2024-09-10T21:24:57.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.152+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c40ccac0 con 0x7f77c40afdd0 2024-09-10T21:24:57.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.152+0000 7f77c1600640 1 -- 172.21.15.97:0/96410319 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7f77b8062280 con 0x7f77c40afdd0 2024-09-10T21:24:57.159 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:57.159 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:57.159 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:57.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.152+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77a40725e0 msgr2=0x7f77a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.152+0000 7f77cc000640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77a40725e0 0x7f77a4074aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f77b00af000 tx=0x7f77b001b100 comp rx=0 tx=0).stop 2024-09-10T21:24:57.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.152+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 msgr2=0x7f77c40c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.152+0000 7f77cc000640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40c96a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f77b800c3c0 tx=0x7f77b800c880 comp rx=0 tx=0).stop 2024-09-10T21:24:57.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 shutdown_connections 2024-09-10T21:24:57.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f77a40725e0 0x7f77a4074aa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c40ce640 0x7f77c41f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77c40b0d50 0x7f77c4071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 --2- 172.21.15.97:0/96410319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77c40afdd0 0x7f77c40c96a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 >> 172.21.15.97:0/96410319 conn(0x7f77c40d1c30 msgr2=0x7f77c40b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:57.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 shutdown_connections 2024-09-10T21:24:57.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.156+0000 7f77cc000640 1 -- 172.21.15.97:0/96410319 wait complete. 2024-09-10T21:24:57.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 -- 172.21.15.97:0/14836717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 msgr2=0x7f75641600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 --2- 172.21.15.97:0/14836717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f75641600d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f75640b12d0 tx=0x7f755c01c800 comp rx=0 tx=0).stop 2024-09-10T21:24:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 -- 172.21.15.97:0/14836717 shutdown_connections 2024-09-10T21:24:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 --2- 172.21.15.97:0/14836717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f75641600d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 --2- 172.21.15.97:0/14836717 >> v2:172.21.15.194:3300/0 conn(0x7f75640722c0 0x7f75640d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 -- 172.21.15.97:0/14836717 >> 172.21.15.97:0/14836717 conn(0x7f75640d1a70 msgr2=0x7f75640d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:57.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 -- 172.21.15.97:0/14836717 shutdown_connections 2024-09-10T21:24:57.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 -- 172.21.15.97:0/14836717 wait complete. 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 Processor -- start 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.320+0000 7f756a200640 1 -- start start 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75640722c0 0x7f75640c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75640cfbd0 0x7f756415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f756415bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75640b0a90 con 0x7f75640cffb0 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f75640b0910 con 0x7f75640722c0 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f75640b0c10 con 0x7f75640cfbd0 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f756415bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f756415bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40702/0 (socket says 172.21.15.97:40702) 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 -- 172.21.15.97:0/3855741594 learned_addr learned my addr 172.21.15.97:0/3855741594 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7568e00640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75640722c0 0x7f75640c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7563e00640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75640cfbd0 0x7f756415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 -- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75640cfbd0 msgr2=0x7f756415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75640cfbd0 0x7f756415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 -- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75640722c0 msgr2=0x7f75640c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75640722c0 0x7f75640c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 -- 172.21.15.97:0/3855741594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641600e0 con 0x7f75640cffb0 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7569800640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f756415bbe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f755c009850 tx=0x7f755c0a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7560c00640 1 -- 172.21.15.97:0/3855741594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0bb070 con 0x7f75640cffb0 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7560c00640 1 -- 172.21.15.97:0/3855741594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f755c004490 con 0x7f75640cffb0 2024-09-10T21:24:57.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f7560c00640 1 -- 172.21.15.97:0/3855741594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0b2040 con 0x7f75640cffb0 2024-09-10T21:24:57.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641f8730 con 0x7f75640cffb0 2024-09-10T21:24:57.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641f8cf0 con 0x7f75640cffb0 2024-09-10T21:24:57.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.324+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75640d2590 con 0x7f75640cffb0 2024-09-10T21:24:57.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.328+0000 7f7560c00640 1 -- 172.21.15.97:0/3855741594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f755c0052d0 con 0x7f75640cffb0 2024-09-10T21:24:57.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.328+0000 7f7560c00640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75440725e0 0x7f7544074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.328+0000 7f7560c00640 1 -- 172.21.15.97:0/3855741594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7f755c12d0a0 con 0x7f75640cffb0 2024-09-10T21:24:57.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.328+0000 7f7568e00640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75440725e0 0x7f7544074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.328+0000 7f7568e00640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75440725e0 0x7f7544074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7554005680 tx=0x7f7554003dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:57.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.332+0000 7f7560c00640 1 -- 172.21.15.97:0/3855741594 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f755c0164b0 con 0x7f75640cffb0 2024-09-10T21:24:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.504+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7564159d60 con 0x7f75640cffb0 2024-09-10T21:24:57.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.504+0000 7f7560c00640 1 -- 172.21.15.97:0/3855741594 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18721 (secure 0 0 0) 0x7f755c0f7db0 con 0x7f75640cffb0 2024-09-10T21:24:57.511 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:57.511 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:53.191241+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3f","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.504+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75440725e0 msgr2=0x7f7544074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.504+0000 7f756a200640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75440725e0 0x7f7544074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7554005680 tx=0x7f7554003dd0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.504+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 msgr2=0x7f756415bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.504+0000 7f756a200640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f756415bbe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f755c009850 tx=0x7f755c0a5630 comp rx=0 tx=0).stop 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 shutdown_connections 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75440725e0 0x7f7544074aa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75640cffb0 0x7f756415bbe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75640cfbd0 0x7f756415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 --2- 172.21.15.97:0/3855741594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75640722c0 0x7f75640c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 >> 172.21.15.97:0/3855741594 conn(0x7f75640d1a70 msgr2=0x7f75640cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:57.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 shutdown_connections 2024-09-10T21:24:57.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.508+0000 7f756a200640 1 -- 172.21.15.97:0/3855741594 wait complete. 2024-09-10T21:24:57.532 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:24:57.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 -- 172.21.15.97:0/4185027158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0ce640 msgr2=0x7fab5c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 --2- 172.21.15.97:0/4185027158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0ce640 0x7fab5c0cea20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009bc0 tx=0x7fab4c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 -- 172.21.15.97:0/4185027158 shutdown_connections 2024-09-10T21:24:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 --2- 172.21.15.97:0/4185027158 >> v2:172.21.15.194:3300/0 conn(0x7fab5c0cd6e0 0x7fab5c0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 --2- 172.21.15.97:0/4185027158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0ce640 0x7fab5c0cea20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 -- 172.21.15.97:0/4185027158 >> 172.21.15.97:0/4185027158 conn(0x7fab5c0d1c30 msgr2=0x7fab5c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 -- 172.21.15.97:0/4185027158 shutdown_connections 2024-09-10T21:24:57.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 -- 172.21.15.97:0/4185027158 wait complete. 2024-09-10T21:24:57.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 Processor -- start 2024-09-10T21:24:57.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.668+0000 7fab63000640 1 -- start start 2024-09-10T21:24:57.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0b0d50 0x7fab5c166310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0cd6e0 0x7fab5c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab5c0ce640 0x7fab5c15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c168c70 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c168af0 con 0x7fab5c0ce640 2024-09-10T21:24:57.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c168df0 con 0x7fab5c0b0d50 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0cd6e0 0x7fab5c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0cd6e0 0x7fab5c0cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40718/0 (socket says 172.21.15.97:40718) 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 -- 172.21.15.97:0/103702769 learned_addr learned my addr 172.21.15.97:0/103702769 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab5be00640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0b0d50 0x7fab5c166310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab61000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab5c0ce640 0x7fab5c15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 -- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0b0d50 msgr2=0x7fab5c166310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0b0d50 0x7fab5c166310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 -- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab5c0ce640 msgr2=0x7fab5c15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab5c0ce640 0x7fab5c15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 -- 172.21.15.97:0/103702769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c0cb410 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab61000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab5c0ce640 0x7fab5c15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:57.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab53400640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0cd6e0 0x7fab5c0cab90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c490 tx=0x7fab4400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:57.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab58c00640 1 -- 172.21.15.97:0/103702769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44016020 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c14c090 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab58c00640 1 -- 172.21.15.97:0/103702769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab44012920 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab58c00640 1 -- 172.21.15.97:0/103702769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44012d50 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c0cb990 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.672+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c0cea20 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.676+0000 7fab58c00640 1 -- 172.21.15.97:0/103702769 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fab4401b400 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.676+0000 7fab58c00640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab380725e0 0x7fab38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:57.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.676+0000 7fab58c00640 1 -- 172.21.15.97:0/103702769 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fab44093540 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.676+0000 7fab5be00640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab380725e0 0x7fab38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:57.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.676+0000 7fab5be00640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab380725e0 0x7fab38074aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0af000 tx=0x7fab4c019940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:57.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.680+0000 7fab58c00640 1 -- 172.21.15.97:0/103702769 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fab44020030 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab5c0cbe90 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab58c00640 1 -- 172.21.15.97:0/103702769 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7fab4405e8c0 con 0x7fab5c0cd6e0 2024-09-10T21:24:57.886 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:57.886 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:57.886 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab380725e0 msgr2=0x7fab38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab380725e0 0x7fab38074aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0af000 tx=0x7fab4c019940 comp rx=0 tx=0).stop 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0cd6e0 msgr2=0x7fab5c0cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0cd6e0 0x7fab5c0cab90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c490 tx=0x7fab4400c950 comp rx=0 tx=0).stop 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 shutdown_connections 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab380725e0 0x7fab38074aa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab5c0ce640 0x7fab5c15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab5c0cd6e0 0x7fab5c0cab90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 --2- 172.21.15.97:0/103702769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab5c0b0d50 0x7fab5c166310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 >> 172.21.15.97:0/103702769 conn(0x7fab5c0d1c30 msgr2=0x7fab5c0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 shutdown_connections 2024-09-10T21:24:57.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:57.880+0000 7fab63000640 1 -- 172.21.15.97:0/103702769 wait complete. 2024-09-10T21:24:57.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 -- 172.21.15.97:0/2570044497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640b0d50 msgr2=0x7efc640aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 --2- 172.21.15.97:0/2570044497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640b0d50 0x7efc640aedc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009900 tx=0x7efc4c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 -- 172.21.15.97:0/2570044497 shutdown_connections 2024-09-10T21:24:58.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 --2- 172.21.15.97:0/2570044497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640b0d50 0x7efc640aedc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 --2- 172.21.15.97:0/2570044497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc640ce640 0x7efc640cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 -- 172.21.15.97:0/2570044497 >> 172.21.15.97:0/2570044497 conn(0x7efc640d1c30 msgr2=0x7efc640d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 -- 172.21.15.97:0/2570044497 shutdown_connections 2024-09-10T21:24:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 -- 172.21.15.97:0/2570044497 wait complete. 2024-09-10T21:24:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 Processor -- start 2024-09-10T21:24:58.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.040+0000 7efc6b200640 1 -- start start 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640af300 0x7efc641576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc640b0d50 0x7efc64157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc640ce640 0x7efc640ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc64169270 con 0x7efc640af300 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efc641690f0 con 0x7efc640ce640 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efc641693f0 con 0x7efc640b0d50 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640af300 0x7efc641576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640af300 0x7efc641576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40724/0 (socket says 172.21.15.97:40724) 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 -- 172.21.15.97:0/4031207508 learned_addr learned my addr 172.21.15.97:0/4031207508 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc5be00640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc640b0d50 0x7efc64157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc69200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc640ce640 0x7efc640ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 -- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc640b0d50 msgr2=0x7efc64157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc640b0d50 0x7efc64157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 -- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc640ce640 msgr2=0x7efc640ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc640ce640 0x7efc640ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 -- 172.21.15.97:0/4031207508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc640c5760 con 0x7efc640af300 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc69200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc640ce640 0x7efc640ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc63e00640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640af300 0x7efc641576d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc5400d950 tx=0x7efc5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc60c00640 1 -- 172.21.15.97:0/4031207508 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54017070 con 0x7efc640af300 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc60c00640 1 -- 172.21.15.97:0/4031207508 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc54009d20 con 0x7efc640af300 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc641587d0 con 0x7efc640af300 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc60c00640 1 -- 172.21.15.97:0/4031207508 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc540106c0 con 0x7efc640af300 2024-09-10T21:24:58.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc64158d90 con 0x7efc640af300 2024-09-10T21:24:58.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.044+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc640aedc0 con 0x7efc640af300 2024-09-10T21:24:58.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.048+0000 7efc60c00640 1 -- 172.21.15.97:0/4031207508 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efc54010860 con 0x7efc640af300 2024-09-10T21:24:58.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.048+0000 7efc60c00640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc400725e0 0x7efc40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.048+0000 7efc60c00640 1 -- 172.21.15.97:0/4031207508 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7efc54093a30 con 0x7efc640af300 2024-09-10T21:24:58.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.048+0000 7efc5be00640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc400725e0 0x7efc40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.048+0000 7efc5be00640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc400725e0 0x7efc40074aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0af000 tx=0x7efc4c01b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.052+0000 7efc60c00640 1 -- 172.21.15.97:0/4031207508 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc640c5760 con 0x7efc640af300 2024-09-10T21:24:58.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.252+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc6415e1b0 con 0x7efc640af300 2024-09-10T21:24:58.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.252+0000 7efc60c00640 1 -- 172.21.15.97:0/4031207508 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5225 (secure 0 0 0) 0x7efc5405e6b0 con 0x7efc640af300 2024-09-10T21:24:58.260 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:58.260 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"btime":"2024-09-10T21:24:31:806885+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3537105924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3537105924},{"type":"v1","addr":"172.21.15.194:6837","nonce":3537105924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4294067517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4294067517},{"type":"v1","addr":"172.21.15.97:6833","nonce":4294067517}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4344,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2329499359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2329499359},{"type":"v1","addr":"172.21.15.194:6835","nonce":2329499359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:11.081203+0000","modified":"2024-09-10T21:24:31.188433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/4001499169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4001499169},{"type":"v1","addr":"172.21.15.97:6835","nonce":4001499169}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-10T21:24:58.261 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-09-10T21:24:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc400725e0 msgr2=0x7efc40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc400725e0 0x7efc40074aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0af000 tx=0x7efc4c01b120 comp rx=0 tx=0).stop 2024-09-10T21:24:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640af300 msgr2=0x7efc641576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640af300 0x7efc641576d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc5400d950 tx=0x7efc5400de10 comp rx=0 tx=0).stop 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 shutdown_connections 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc400725e0 0x7efc40074aa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc640ce640 0x7efc640ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc640b0d50 0x7efc64157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 --2- 172.21.15.97:0/4031207508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc640af300 0x7efc641576d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 >> 172.21.15.97:0/4031207508 conn(0x7efc640d1c30 msgr2=0x7efc640d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 shutdown_connections 2024-09-10T21:24:58.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.256+0000 7efc6b200640 1 -- 172.21.15.97:0/4031207508 wait complete. 2024-09-10T21:24:58.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:58.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.416+0000 7fe614800640 1 -- 172.21.15.97:0/712299458 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 msgr2=0x7fe60c0b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.416+0000 7fe614800640 1 --2- 172.21.15.97:0/712299458 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 0x7fe60c0b3070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60001c7d0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.416+0000 7fe614800640 1 -- 172.21.15.97:0/712299458 shutdown_connections 2024-09-10T21:24:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.416+0000 7fe614800640 1 --2- 172.21.15.97:0/712299458 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 0x7fe60c0b3070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.416+0000 7fe614800640 1 -- 172.21.15.97:0/712299458 >> 172.21.15.97:0/712299458 conn(0x7fe60c0d1c60 msgr2=0x7fe60c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.416+0000 7fe614800640 1 -- 172.21.15.97:0/712299458 shutdown_connections 2024-09-10T21:24:58.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.416+0000 7fe614800640 1 -- 172.21.15.97:0/712299458 wait complete. 2024-09-10T21:24:58.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 Processor -- start 2024-09-10T21:24:58.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 -- start start 2024-09-10T21:24:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60c0d0150 0x7fe60c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe60c0d05e0 0x7fe60c0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 0x7fe60c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c169e70 con 0x7fe60c0d0150 2024-09-10T21:24:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c169cf0 con 0x7fe60c0d05e0 2024-09-10T21:24:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe60c169ff0 con 0x7fe60c15f350 2024-09-10T21:24:58.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60c0d0150 0x7fe60c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60c0d0150 0x7fe60c0722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40734/0 (socket says 172.21.15.97:40734) 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 -- 172.21.15.97:0/1944298015 learned_addr learned my addr 172.21.15.97:0/1944298015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611400640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe60c0d05e0 0x7fe60c0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe612800640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 0x7fe60c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 -- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 msgr2=0x7fe60c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 0x7fe60c0c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 -- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe60c0d05e0 msgr2=0x7fe60c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe60c0d05e0 0x7fe60c0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 -- 172.21.15.97:0/1944298015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c06e0a0 con 0x7fe60c0d0150 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611400640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe60c0d05e0 0x7fe60c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe611e00640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60c0d0150 0x7fe60c0722e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe60001ccb0 tx=0x7fe600005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe606000640 1 -- 172.21.15.97:0/1944298015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6000b2040 con 0x7fe60c0d0150 2024-09-10T21:24:58.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe606000640 1 -- 172.21.15.97:0/1944298015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6000bb070 con 0x7fe60c0d0150 2024-09-10T21:24:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe606000640 1 -- 172.21.15.97:0/1944298015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600003600 con 0x7fe60c0d0150 2024-09-10T21:24:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c06e380 con 0x7fe60c0d0150 2024-09-10T21:24:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c06e940 con 0x7fe60c0d0150 2024-09-10T21:24:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.420+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60c0b3070 con 0x7fe60c0d0150 2024-09-10T21:24:58.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.424+0000 7fe606000640 1 -- 172.21.15.97:0/1944298015 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe6000b5070 con 0x7fe60c0d0150 2024-09-10T21:24:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.424+0000 7fe606000640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5f00725e0 0x7fe5f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.424+0000 7fe606000640 1 -- 172.21.15.97:0/1944298015 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fe60012d5f0 con 0x7fe60c0d0150 2024-09-10T21:24:58.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.424+0000 7fe611400640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5f00725e0 0x7fe5f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.424+0000 7fe611400640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5f00725e0 0x7fe5f0074aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007a60 tx=0x7fe5fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.428+0000 7fe606000640 1 -- 172.21.15.97:0/1944298015 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe600016450 con 0x7fe60c0d0150 2024-09-10T21:24:58.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.592+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe60c0b13a0 con 0x7fe60c0d0150 2024-09-10T21:24:58.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe606000640 1 -- 172.21.15.97:0/1944298015 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18721 (secure 0 0 0) 0x7fe6000f8950 con 0x7fe60c0d0150 2024-09-10T21:24:58.602 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:24:58.602 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:24:53.191241+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:09.636980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:10.045699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:13.750840+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3f","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:58.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5f00725e0 msgr2=0x7fe5f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5f00725e0 0x7fe5f0074aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007a60 tx=0x7fe5fc009040 comp rx=0 tx=0).stop 2024-09-10T21:24:58.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60c0d0150 msgr2=0x7fe60c0722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60c0d0150 0x7fe60c0722e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe60001ccb0 tx=0x7fe600005780 comp rx=0 tx=0).stop 2024-09-10T21:24:58.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 shutdown_connections 2024-09-10T21:24:58.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5f00725e0 0x7fe5f0074aa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe60c15f350 0x7fe60c0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe60c0d05e0 0x7fe60c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 --2- 172.21.15.97:0/1944298015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe60c0d0150 0x7fe60c0722e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 >> 172.21.15.97:0/1944298015 conn(0x7fe60c0d1c60 msgr2=0x7fe60c0cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 shutdown_connections 2024-09-10T21:24:58.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.596+0000 7fe614800640 1 -- 172.21.15.97:0/1944298015 wait complete. 2024-09-10T21:24:58.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:24:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 -- 172.21.15.97:0/617341848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47415fde0 msgr2=0x7fc47413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 --2- 172.21.15.97:0/617341848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47415fde0 0x7fc47413f9b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009900 tx=0x7fc46c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 -- 172.21.15.97:0/617341848 shutdown_connections 2024-09-10T21:24:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 --2- 172.21.15.97:0/617341848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47415fde0 0x7fc47413f9b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 --2- 172.21.15.97:0/617341848 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4740ce5f0 0x7fc474141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 -- 172.21.15.97:0/617341848 >> 172.21.15.97:0/617341848 conn(0x7fc4740b1780 msgr2=0x7fc4740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 -- 172.21.15.97:0/617341848 shutdown_connections 2024-09-10T21:24:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 -- 172.21.15.97:0/617341848 wait complete. 2024-09-10T21:24:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.760+0000 7fc478a00640 1 Processor -- start 2024-09-10T21:24:58.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 -- start start 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4740ce5f0 0x7fc4740cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4740ce9d0 0x7fc4741f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47415fde0 0x7fc4740738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc474168a70 con 0x7fc4740ce5f0 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc474168bf0 con 0x7fc47415fde0 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc474168d70 con 0x7fc4740ce9d0 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4740ce5f0 0x7fc4740cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4740ce5f0 0x7fc4740cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40754/0 (socket says 172.21.15.97:40754) 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 -- 172.21.15.97:0/444339949 learned_addr learned my addr 172.21.15.97:0/444339949 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:58.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc472800640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47415fde0 0x7fc4740738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471400640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4740ce9d0 0x7fc4741f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 -- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4740ce9d0 msgr2=0x7fc4741f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4740ce9d0 0x7fc4741f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 -- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47415fde0 msgr2=0x7fc4740738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47415fde0 0x7fc4740738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 -- 172.21.15.97:0/444339949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4741f7fa0 con 0x7fc4740ce5f0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc471e00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4740ce5f0 0x7fc4740cc720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc464009fd0 tx=0x7fc46400c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc46a000640 1 -- 172.21.15.97:0/444339949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464011470 con 0x7fc4740ce5f0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc46a000640 1 -- 172.21.15.97:0/444339949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc464011700 con 0x7fc4740ce5f0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741f8270 con 0x7fc4740ce5f0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc46a000640 1 -- 172.21.15.97:0/444339949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464011b30 con 0x7fc4740ce5f0 2024-09-10T21:24:58.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc474156710 con 0x7fc4740ce5f0 2024-09-10T21:24:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.764+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474141d20 con 0x7fc4740ce5f0 2024-09-10T21:24:58.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.768+0000 7fc46a000640 1 -- 172.21.15.97:0/444339949 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc464022030 con 0x7fc4740ce5f0 2024-09-10T21:24:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.768+0000 7fc46a000640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4540725e0 0x7fc454074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.768+0000 7fc46a000640 1 -- 172.21.15.97:0/444339949 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6590+0+0 (secure 0 0 0) 0x7fc464093f10 con 0x7fc4740ce5f0 2024-09-10T21:24:58.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.768+0000 7fc471400640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4540725e0 0x7fc454074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.768+0000 7fc471400640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4540725e0 0x7fc454074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4740ccc60 tx=0x7fc460009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.772+0000 7fc46a000640 1 -- 172.21.15.97:0/444339949 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc464016030 con 0x7fc4740ce5f0 2024-09-10T21:24:58.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.968+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc474166630 con 0x7fc4740ce5f0 2024-09-10T21:24:58.975 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:24:58.968+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:24:58.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc46a000640 1 -- 172.21.15.97:0/444339949 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fc46405eb90 con 0x7fc4740ce5f0 2024-09-10T21:24:58.977 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4540725e0 msgr2=0x7fc454074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4540725e0 0x7fc454074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4740ccc60 tx=0x7fc460009040 comp rx=0 tx=0).stop 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4740ce5f0 msgr2=0x7fc4740cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4740ce5f0 0x7fc4740cc720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc464009fd0 tx=0x7fc46400c270 comp rx=0 tx=0).stop 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 shutdown_connections 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4540725e0 0x7fc454074aa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47415fde0 0x7fc4740738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4740ce9d0 0x7fc4741f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 --2- 172.21.15.97:0/444339949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4740ce5f0 0x7fc4740cc720 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 >> 172.21.15.97:0/444339949 conn(0x7fc4740b1780 msgr2=0x7fc47415aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 shutdown_connections 2024-09-10T21:24:58.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:58.972+0000 7fc478a00640 1 -- 172.21.15.97:0/444339949 wait complete. 2024-09-10T21:24:58.990 INFO:tasks.ceph.mds.c.smithi097.stderr: -4525> 2024-09-10T21:24:17.264+0000 7f9c3b600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:24:59.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:24:59.039 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:24:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 -- 172.21.15.97:0/435530078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb40d29d0 msgr2=0x7f4eb40cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 --2- 172.21.15.97:0/435530078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb40d29d0 0x7f4eb40cf850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40b68a0 tx=0x7f4eac01cc60 comp rx=0 tx=0).stop 2024-09-10T21:24:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 -- 172.21.15.97:0/435530078 shutdown_connections 2024-09-10T21:24:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 --2- 172.21.15.97:0/435530078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb40d29d0 0x7f4eb40cf850 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 --2- 172.21.15.97:0/435530078 >> v2:172.21.15.194:3300/0 conn(0x7f4eb4161110 0x7f4eb40d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 -- 172.21.15.97:0/435530078 >> 172.21.15.97:0/435530078 conn(0x7f4eb40d3ce0 msgr2=0x7f4eb40d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 -- 172.21.15.97:0/435530078 shutdown_connections 2024-09-10T21:24:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.140+0000 7f4eba800640 1 -- 172.21.15.97:0/435530078 wait complete. 2024-09-10T21:24:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 Processor -- start 2024-09-10T21:24:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 -- start start 2024-09-10T21:24:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4eb40d29d0 0x7f4eb4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb4161110 0x7f4eb40c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb41614f0 0x7f4eb415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb416b010 con 0x7f4eb41614f0 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb416ae90 con 0x7f4eb4161110 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb416b190 con 0x7f4eb40d29d0 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb41614f0 0x7f4eb415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb41614f0 0x7f4eb415f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40782/0 (socket says 172.21.15.97:40782) 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 -- 172.21.15.97:0/2913630078 learned_addr learned my addr 172.21.15.97:0/2913630078 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9400640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4eb40d29d0 0x7f4eb4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb8a00640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb4161110 0x7f4eb40c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 -- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4eb40d29d0 msgr2=0x7f4eb4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4eb40d29d0 0x7f4eb4157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 -- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb4161110 msgr2=0x7f4eb40c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb4161110 0x7f4eb40c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 -- 172.21.15.97:0/2913630078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb4157ff0 con 0x7f4eb41614f0 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb8a00640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb4161110 0x7f4eb40c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb9e00640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb41614f0 0x7f4eb415f630 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4eac01b310 tx=0x7f4eac0a5be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb1600640 1 -- 172.21.15.97:0/2913630078 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0bb070 con 0x7f4eb41614f0 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb1600640 1 -- 172.21.15.97:0/2913630078 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eac004d40 con 0x7f4eb41614f0 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eb1600640 1 -- 172.21.15.97:0/2913630078 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0b2040 con 0x7f4eb41614f0 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb40c8600 con 0x7f4eb41614f0 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.144+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb415bd70 con 0x7f4eb41614f0 2024-09-10T21:24:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.148+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb406aea0 con 0x7f4eb41614f0 2024-09-10T21:24:59.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.148+0000 7f4eb1600640 1 -- 172.21.15.97:0/2913630078 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4eac0b5030 con 0x7f4eb41614f0 2024-09-10T21:24:59.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.148+0000 7f4eb1600640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e940725e0 0x7f4e94074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.148+0000 7f4eb1600640 1 -- 172.21.15.97:0/2913630078 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6676+0+0 (secure 0 0 0) 0x7f4eac12d2a0 con 0x7f4eb41614f0 2024-09-10T21:24:59.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.148+0000 7f4eb9400640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e940725e0 0x7f4e94074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.152+0000 7f4eb9400640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e940725e0 0x7f4e94074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009990 tx=0x7f4ea8009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:59.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.152+0000 7f4eb1600640 1 -- 172.21.15.97:0/2913630078 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4eb40c8600 con 0x7f4eb41614f0 2024-09-10T21:24:59.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.352+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4eb40b9c60 con 0x7f4eb41614f0 2024-09-10T21:24:59.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.972+0000 7f4eb1600640 1 -- 172.21.15.97:0/2913630078 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f4eac016970 con 0x7f4eb41614f0 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e940725e0 msgr2=0x7f4e94074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e940725e0 0x7f4e94074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009990 tx=0x7f4ea8009110 comp rx=0 tx=0).stop 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb41614f0 msgr2=0x7f4eb415f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb41614f0 0x7f4eb415f630 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4eac01b310 tx=0x7f4eac0a5be0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 shutdown_connections 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e940725e0 0x7f4e94074aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eb41614f0 0x7f4eb415f630 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4eb4161110 0x7f4eb40c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 --2- 172.21.15.97:0/2913630078 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4eb40d29d0 0x7f4eb4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 >> 172.21.15.97:0/2913630078 conn(0x7f4eb40d3ce0 msgr2=0x7f4eb40b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 shutdown_connections 2024-09-10T21:24:59.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:24:59.976+0000 7f4eba800640 1 -- 172.21.15.97:0/2913630078 wait complete. 2024-09-10T21:25:00.004 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:25:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- 172.21.15.97:0/3159270591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65006dc50 msgr2=0x7fc6500cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 --2- 172.21.15.97:0/3159270591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65006dc50 0x7fc6500cfe30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc638009900 tx=0x7fc63801c850 comp rx=0 tx=0).stop 2024-09-10T21:25:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- 172.21.15.97:0/3159270591 shutdown_connections 2024-09-10T21:25:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 --2- 172.21.15.97:0/3159270591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65006dc50 0x7fc6500cfe30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 --2- 172.21.15.97:0/3159270591 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc650161600 0x7fc6500d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- 172.21.15.97:0/3159270591 >> 172.21.15.97:0/3159270591 conn(0x7fc6500d1c30 msgr2=0x7fc6500d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- 172.21.15.97:0/3159270591 shutdown_connections 2024-09-10T21:25:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- 172.21.15.97:0/3159270591 wait complete. 2024-09-10T21:25:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 Processor -- start 2024-09-10T21:25:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- start start 2024-09-10T21:25:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6500b1590 0x7fc65014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc650161600 0x7fc6501504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6500b4f20 0x7fc6500b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc65016b200 con 0x7fc6500b4f20 2024-09-10T21:25:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc65016b080 con 0x7fc6500b1590 2024-09-10T21:25:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc656e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc65016b380 con 0x7fc650161600 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.184+0000 7fc654e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6500b4f20 0x7fc6500b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6500b4f20 0x7fc6500b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40794/0 (socket says 172.21.15.97:40794) 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 -- 172.21.15.97:0/1030294384 learned_addr learned my addr 172.21.15.97:0/1030294384 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc64fe00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6500b1590 0x7fc65014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc647e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc650161600 0x7fc6501504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 -- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc650161600 msgr2=0x7fc6501504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc650161600 0x7fc6501504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 -- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6500b1590 msgr2=0x7fc65014f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6500b1590 0x7fc65014f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 -- 172.21.15.97:0/1030294384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65015afb0 con 0x7fc6500b4f20 2024-09-10T21:25:00.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc654e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6500b4f20 0x7fc6500b5390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc64800c730 tx=0x7fc64800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:00.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc64cc00640 1 -- 172.21.15.97:0/1030294384 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648003b80 con 0x7fc6500b4f20 2024-09-10T21:25:00.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc64cc00640 1 -- 172.21.15.97:0/1030294384 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc648003d20 con 0x7fc6500b4f20 2024-09-10T21:25:00.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65015b290 con 0x7fc6500b4f20 2024-09-10T21:25:00.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc64cc00640 1 -- 172.21.15.97:0/1030294384 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64800aa50 con 0x7fc6500b4f20 2024-09-10T21:25:00.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65014f9f0 con 0x7fc6500b4f20 2024-09-10T21:25:00.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6500cfe30 con 0x7fc6500b4f20 2024-09-10T21:25:00.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.188+0000 7fc64cc00640 1 -- 172.21.15.97:0/1030294384 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc64800abf0 con 0x7fc6500b4f20 2024-09-10T21:25:00.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.192+0000 7fc64cc00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc62c072610 0x7fc62c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:00.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.192+0000 7fc64cc00640 1 -- 172.21.15.97:0/1030294384 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6676+0+0 (secure 0 0 0) 0x7fc648093240 con 0x7fc6500b4f20 2024-09-10T21:25:00.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.192+0000 7fc64fe00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc62c072610 0x7fc62c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:00.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.192+0000 7fc64fe00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc62c072610 0x7fc62c074ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc64000bb50 tx=0x7fc64000b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:00.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.196+0000 7fc64cc00640 1 -- 172.21.15.97:0/1030294384 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc64805def0 con 0x7fc6500b4f20 2024-09-10T21:25:00.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:00.368+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc650159970 con 0x7fc6500b4f20 2024-09-10T21:25:01.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.232+0000 7fc64cc00640 1 -- 172.21.15.97:0/1030294384 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7fc648062920 con 0x7fc6500b4f20 2024-09-10T21:25:01.239 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:25:01.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc62c072610 msgr2=0x7fc62c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc62c072610 0x7fc62c074ad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc64000bb50 tx=0x7fc64000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6500b4f20 msgr2=0x7fc6500b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6500b4f20 0x7fc6500b5390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc64800c730 tx=0x7fc64800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 shutdown_connections 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc62c072610 0x7fc62c074ad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6500b4f20 0x7fc6500b5390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc650161600 0x7fc6501504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 --2- 172.21.15.97:0/1030294384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6500b1590 0x7fc65014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 >> 172.21.15.97:0/1030294384 conn(0x7fc6500d1c30 msgr2=0x7fc6500b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 shutdown_connections 2024-09-10T21:25:01.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.236+0000 7fc656e00640 1 -- 172.21.15.97:0/1030294384 wait complete. 2024-09-10T21:25:01.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:25:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- 172.21.15.97:0/4196869503 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77780cdcb0 msgr2=0x7f77780cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 --2- 172.21.15.97:0/4196869503 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77780cdcb0 0x7f77780cfd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7760009900 tx=0x7f776001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- 172.21.15.97:0/4196869503 shutdown_connections 2024-09-10T21:25:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 --2- 172.21.15.97:0/4196869503 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77780cdcb0 0x7f77780cfd80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 --2- 172.21.15.97:0/4196869503 >> v2:172.21.15.194:3300/0 conn(0x7f777815efc0 0x7f77780cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- 172.21.15.97:0/4196869503 >> 172.21.15.97:0/4196869503 conn(0x7f77780d1a60 msgr2=0x7f77780d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- 172.21.15.97:0/4196869503 shutdown_connections 2024-09-10T21:25:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- 172.21.15.97:0/4196869503 wait complete. 2024-09-10T21:25:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 Processor -- start 2024-09-10T21:25:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- start start 2024-09-10T21:25:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77780cdcb0 0x7f7778151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77780d02c0 0x7f77781f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f777815efc0 0x7f77781f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7778168f20 con 0x7f77780d02c0 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7778168da0 con 0x7f77780cdcb0 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f7777200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f77781690a0 con 0x7f777815efc0 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.408+0000 7f776d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77780d02c0 0x7f77781f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77780d02c0 0x7f77781f65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40802/0 (socket says 172.21.15.97:40802) 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 -- 172.21.15.97:0/5159730 learned_addr learned my addr 172.21.15.97:0/5159730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f7775e00640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77780cdcb0 0x7f7778151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f7776800640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f777815efc0 0x7f77781f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 -- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f777815efc0 msgr2=0x7f77781f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f777815efc0 0x7f77781f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 -- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77780cdcb0 msgr2=0x7f7778151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77780cdcb0 0x7f7778151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 -- 172.21.15.97:0/5159730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7778156120 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f7776800640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f777815efc0 0x7f77781f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776d400640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77780d02c0 0x7f77781f65e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f776001a510 tx=0x7f7760005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776ea00640 1 -- 172.21.15.97:0/5159730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77600ac070 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776ea00640 1 -- 172.21.15.97:0/5159730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7760004320 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776ea00640 1 -- 172.21.15.97:0/5159730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77600052a0 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7778156400 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781517f0 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77780ce1e0 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.412+0000 7f776ea00640 1 -- 172.21.15.97:0/5159730 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f77600a6620 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.416+0000 7f776ea00640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7754072610 0x7f7754074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.416+0000 7f776ea00640 1 -- 172.21.15.97:0/5159730 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6238+0+0 (secure 0 0 0) 0x7f776012d6e0 con 0x7f77780d02c0 2024-09-10T21:25:01.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.416+0000 7f7775e00640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7754072610 0x7f7754074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.416+0000 7f7775e00640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7754072610 0x7f7754074ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7768009b40 tx=0x7f77680091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:01.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.420+0000 7f776ea00640 1 -- 172.21.15.97:0/5159730 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77600164d0 con 0x7f77780d02c0 2024-09-10T21:25:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:01.588+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77780b0530 con 0x7f77780d02c0 2024-09-10T21:25:02.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f776ea00640 1 -- 172.21.15.97:0/5159730 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f77600f8550 con 0x7f77780d02c0 2024-09-10T21:25:02.246 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-09-10T21:25:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7754072610 msgr2=0x7f7754074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7754072610 0x7f7754074ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7768009b40 tx=0x7f77680091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77780d02c0 msgr2=0x7f77781f65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77780d02c0 0x7f77781f65e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f776001a510 tx=0x7f7760005e80 comp rx=0 tx=0).stop 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 shutdown_connections 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7754072610 0x7f7754074ad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f777815efc0 0x7f77781f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77780d02c0 0x7f77781f65e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 --2- 172.21.15.97:0/5159730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77780cdcb0 0x7f7778151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 >> 172.21.15.97:0/5159730 conn(0x7f77780d1a60 msgr2=0x7f77780ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 shutdown_connections 2024-09-10T21:25:02.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.240+0000 7f7777200640 1 -- 172.21.15.97:0/5159730 wait complete. 2024-09-10T21:25:02.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 -- 172.21.15.97:0/817223995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624144690 msgr2=0x7f06240b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 --2- 172.21.15.97:0/817223995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624144690 0x7f06240b2a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f060c009840 tx=0x7f060c01c820 comp rx=0 tx=0).stop 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 -- 172.21.15.97:0/817223995 shutdown_connections 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 --2- 172.21.15.97:0/817223995 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f062406fd50 0x7f06240701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 --2- 172.21.15.97:0/817223995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624144690 0x7f06240b2a80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 -- 172.21.15.97:0/817223995 >> 172.21.15.97:0/817223995 conn(0x7f06240b1780 msgr2=0x7f06240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 -- 172.21.15.97:0/817223995 shutdown_connections 2024-09-10T21:25:02.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.412+0000 7f062b400640 1 -- 172.21.15.97:0/817223995 wait complete. 2024-09-10T21:25:02.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 Processor -- start 2024-09-10T21:25:02.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 -- start start 2024-09-10T21:25:02.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062406fd50 0x7f062415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624143f90 0x7f0624159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0624144690 0x7f06240ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0624167c50 con 0x7f062406fd50 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0624167ad0 con 0x7f0624143f90 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0624167dd0 con 0x7f0624144690 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062406fd50 0x7f062415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062406fd50 0x7f062415a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40814/0 (socket says 172.21.15.97:40814) 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 -- 172.21.15.97:0/2677996557 learned_addr learned my addr 172.21.15.97:0/2677996557 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f061be00640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624143f90 0x7f0624159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0629400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0624144690 0x7f06240ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 -- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0624144690 msgr2=0x7f06240ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0624144690 0x7f06240ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 -- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624143f90 msgr2=0x7f0624159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624143f90 0x7f0624159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 -- 172.21.15.97:0/2677996557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0624138de0 con 0x7f062406fd50 2024-09-10T21:25:02.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0629400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0624144690 0x7f06240ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:02.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0628a00640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062406fd50 0x7f062415a640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f061400c500 tx=0x7f061400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:02.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0621600640 1 -- 172.21.15.97:0/2677996557 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614017070 con 0x7f062406fd50 2024-09-10T21:25:02.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0621600640 1 -- 172.21.15.97:0/2677996557 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06140053c0 con 0x7f062406fd50 2024-09-10T21:25:02.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062413b1e0 con 0x7f062406fd50 2024-09-10T21:25:02.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f0621600640 1 -- 172.21.15.97:0/2677996557 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614012890 con 0x7f062406fd50 2024-09-10T21:25:02.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.416+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0624138570 con 0x7f062406fd50 2024-09-10T21:25:02.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.420+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06240701e0 con 0x7f062406fd50 2024-09-10T21:25:02.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.420+0000 7f0621600640 1 -- 172.21.15.97:0/2677996557 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0614003600 con 0x7f062406fd50 2024-09-10T21:25:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.420+0000 7f0621600640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06000725e0 0x7f0600074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.420+0000 7f0621600640 1 -- 172.21.15.97:0/2677996557 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5795+0+0 (secure 0 0 0) 0x7f0614092860 con 0x7f062406fd50 2024-09-10T21:25:02.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.420+0000 7f061be00640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06000725e0 0x7f0600074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.420+0000 7f061be00640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06000725e0 0x7f0600074aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f060c0af000 tx=0x7f060c01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:02.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.424+0000 7f0621600640 1 -- 172.21.15.97:0/2677996557 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f061405d8a0 con 0x7f062406fd50 2024-09-10T21:25:02.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:02.604+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f062413b570 con 0x7f062406fd50 2024-09-10T21:25:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.244+0000 7f0621600640 1 -- 172.21.15.97:0/2677996557 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f06140622d0 con 0x7f062406fd50 2024-09-10T21:25:03.248 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' removed 2024-09-10T21:25:03.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.244+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06000725e0 msgr2=0x7f0600074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:03.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.244+0000 7f062b400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06000725e0 0x7f0600074aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f060c0af000 tx=0x7f060c01b060 comp rx=0 tx=0).stop 2024-09-10T21:25:03.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.244+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062406fd50 msgr2=0x7f062415a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:03.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.244+0000 7f062b400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062406fd50 0x7f062415a640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f061400c500 tx=0x7f061400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 shutdown_connections 2024-09-10T21:25:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06000725e0 0x7f0600074aa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0624144690 0x7f06240ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0624143f90 0x7f0624159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 --2- 172.21.15.97:0/2677996557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062406fd50 0x7f062415a640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 >> 172.21.15.97:0/2677996557 conn(0x7f06240b1780 msgr2=0x7f06240cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 shutdown_connections 2024-09-10T21:25:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.248+0000 7f062b400640 1 -- 172.21.15.97:0/2677996557 wait complete. 2024-09-10T21:25:03.282 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:25:03.282 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:25:03.282 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:25:03.282 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:25:03.282 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:25:03.282 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:25:03.283 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:25:03.283 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:25:03.283 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:25:03.283 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:25:03.283 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:25:03.283 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:25:03.283 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:25:03.284 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:25:03.284 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:25:03.284 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:25:03.295 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:25:03.296 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:25:03.296 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:25:03.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:25:03.298 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:25:03.299 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:25:03.301 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:25:03.301 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:25:03.343 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:25:03.343 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:03.344 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:25:03.345 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:03.351 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:25:03.351 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:03.353 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:25:03.354 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:03.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:25:03.430 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:25:03.431 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:25:03.436 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:25:03.436 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- 172.21.15.97:0/2183571246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8440ceee0 msgr2=0x7fa8440cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 --2- 172.21.15.97:0/2183571246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8440ceee0 0x7fa8440cf370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00a950 tx=0x7fa83c020360 comp rx=0 tx=0).stop 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- 172.21.15.97:0/2183571246 shutdown_connections 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 --2- 172.21.15.97:0/2183571246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8440ceee0 0x7fa8440cf370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 --2- 172.21.15.97:0/2183571246 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa84406ec40 0x7fa84406f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- 172.21.15.97:0/2183571246 >> 172.21.15.97:0/2183571246 conn(0x7fa8440b1780 msgr2=0x7fa8440b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- 172.21.15.97:0/2183571246 shutdown_connections 2024-09-10T21:25:03.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- 172.21.15.97:0/2183571246 wait complete. 2024-09-10T21:25:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 Processor -- start 2024-09-10T21:25:03.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- start start 2024-09-10T21:25:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa84406ec40 0x7fa84414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:03.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa8440ceee0 0x7fa844148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8441511f0 0x7fa8441f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa84406e740 con 0x7fa84406ec40 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa84406e5c0 con 0x7fa8441511f0 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa849c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa84406e8c0 con 0x7fa8440ceee0 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa84406ec40 0x7fa84414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa84406ec40 0x7fa84414d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40852/0 (socket says 172.21.15.97:40852) 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 -- 172.21.15.97:0/1246405746 learned_addr learned my addr 172.21.15.97:0/1246405746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa83a000640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa8440ceee0 0x7fa844148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa843400640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8441511f0 0x7fa8441f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 -- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa8440ceee0 msgr2=0x7fa844148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa8440ceee0 0x7fa844148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 -- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8441511f0 msgr2=0x7fa8441f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8441511f0 0x7fa8441f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.500+0000 7fa842a00640 1 -- 172.21.15.97:0/1246405746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8441f35c0 con 0x7fa84406ec40 2024-09-10T21:25:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa842a00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa84406ec40 0x7fa84414d7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa83400cbe0 tx=0x7fa83400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa83b400640 1 -- 172.21.15.97:0/1246405746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834016020 con 0x7fa84406ec40 2024-09-10T21:25:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa83b400640 1 -- 172.21.15.97:0/1246405746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa83400ab60 con 0x7fa84406ec40 2024-09-10T21:25:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa83b400640 1 -- 172.21.15.97:0/1246405746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834012420 con 0x7fa84406ec40 2024-09-10T21:25:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa844148750 con 0x7fa84406ec40 2024-09-10T21:25:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441f2880 con 0x7fa84406ec40 2024-09-10T21:25:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa844148c70 con 0x7fa84406ec40 2024-09-10T21:25:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa83b400640 1 -- 172.21.15.97:0/1246405746 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa8340125c0 con 0x7fa84406ec40 2024-09-10T21:25:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa83b400640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa820072610 0x7fa820074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa83b400640 1 -- 172.21.15.97:0/1246405746 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7fa8340935f0 con 0x7fa84406ec40 2024-09-10T21:25:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.504+0000 7fa83a000640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa820072610 0x7fa820074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.508+0000 7fa83a000640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa820072610 0x7fa820074ad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa8441594f0 tx=0x7fa82c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.508+0000 7fa83b400640 1 -- 172.21.15.97:0/1246405746 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa834020020 con 0x7fa84406ec40 2024-09-10T21:25:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:03.692+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa84406f060 con 0x7fa84406ec40 2024-09-10T21:25:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa83b400640 1 -- 172.21.15.97:0/1246405746 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7fa8441f35c0 con 0x7fa84406ec40 2024-09-10T21:25:04.270 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-09-10T21:25:04.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa820072610 msgr2=0x7fa820074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa820072610 0x7fa820074ad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa8441594f0 tx=0x7fa82c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa84406ec40 msgr2=0x7fa84414d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa84406ec40 0x7fa84414d7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa83400cbe0 tx=0x7fa83400ccd0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 shutdown_connections 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa820072610 0x7fa820074ad0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8441511f0 0x7fa8441f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa8440ceee0 0x7fa844148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 --2- 172.21.15.97:0/1246405746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa84406ec40 0x7fa84414d7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 >> 172.21.15.97:0/1246405746 conn(0x7fa8440b1780 msgr2=0x7fa84415c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.264+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 shutdown_connections 2024-09-10T21:25:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.268+0000 7fa849c00640 1 -- 172.21.15.97:0/1246405746 wait complete. 2024-09-10T21:25:04.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 -- 172.21.15.97:0/3602973752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600722c0 msgr2=0x7f34600d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 --2- 172.21.15.97:0/3602973752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600722c0 0x7f34600d2590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3448009840 tx=0x7f344801c6d0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 -- 172.21.15.97:0/3602973752 shutdown_connections 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 --2- 172.21.15.97:0/3602973752 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34600cffb0 0x7f34601600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 --2- 172.21.15.97:0/3602973752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600722c0 0x7f34600d2590 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 -- 172.21.15.97:0/3602973752 >> 172.21.15.97:0/3602973752 conn(0x7f34600d1a70 msgr2=0x7f34600d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 -- 172.21.15.97:0/3602973752 shutdown_connections 2024-09-10T21:25:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.432+0000 7f345fe00640 1 -- 172.21.15.97:0/3602973752 wait complete. 2024-09-10T21:25:04.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 Processor -- start 2024-09-10T21:25:04.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 -- start start 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34600722c0 0x7f346015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600cfbd0 0x7f346015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34600cffb0 0x7f34601500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34600b0a60 con 0x7f34600cfbd0 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f34600b08e0 con 0x7f34600722c0 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f34600b0be0 con 0x7f34600cffb0 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600cfbd0 0x7f346015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600cfbd0 0x7f346015fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40872/0 (socket says 172.21.15.97:40872) 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 -- 172.21.15.97:0/3137813991 learned_addr learned my addr 172.21.15.97:0/3137813991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345ea00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34600722c0 0x7f346015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345f400640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34600cffb0 0x7f34601500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 -- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34600cffb0 msgr2=0x7f34601500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34600cffb0 0x7f34601500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 -- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34600722c0 msgr2=0x7f346015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34600722c0 0x7f346015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 -- 172.21.15.97:0/3137813991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3460151f70 con 0x7f34600cfbd0 2024-09-10T21:25:04.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3456000640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600cfbd0 0x7f346015fbe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3448009790 tx=0x7f3448004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3457400640 1 -- 172.21.15.97:0/3137813991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34480bb070 con 0x7f34600cfbd0 2024-09-10T21:25:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3457400640 1 -- 172.21.15.97:0/3137813991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3448005030 con 0x7f34600cfbd0 2024-09-10T21:25:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f3457400640 1 -- 172.21.15.97:0/3137813991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34480b2040 con 0x7f34600cfbd0 2024-09-10T21:25:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346015c6a0 con 0x7f34600cfbd0 2024-09-10T21:25:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346014c120 con 0x7f34600cfbd0 2024-09-10T21:25:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.436+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34600d2590 con 0x7f34600cfbd0 2024-09-10T21:25:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.440+0000 7f3457400640 1 -- 172.21.15.97:0/3137813991 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f34480b5030 con 0x7f34600cfbd0 2024-09-10T21:25:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.440+0000 7f3457400640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f343c0725e0 0x7f343c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.440+0000 7f3457400640 1 -- 172.21.15.97:0/3137813991 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f344812c780 con 0x7f34600cfbd0 2024-09-10T21:25:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.440+0000 7f345ea00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f343c0725e0 0x7f343c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.440+0000 7f345ea00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f343c0725e0 0x7f343c074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3450007fd0 tx=0x7f3450007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.444+0000 7f3457400640 1 -- 172.21.15.97:0/3137813991 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34480f1de0 con 0x7f34600cfbd0 2024-09-10T21:25:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.668+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f34600d2830 con 0x7f34600cfbd0 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.668+0000 7f3457400640 1 -- 172.21.15.97:0/3137813991 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f3448016410 con 0x7f34600cfbd0 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA==", 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.677 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g==", 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.678 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw==", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA==", 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q==", 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q==", 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.681 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg==", 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.682 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ==", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ==", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.683 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw==", 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-09-10T21:25:04.684 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ==", 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q==", 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.685 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q==", 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:25:04.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ==", 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag==", 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.688 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA==", 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.689 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmjCI0FRAATFUoCxw0bU6HSEUnlyrVJQ==", 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mds", 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDiuOBmQ4rKLRAA/NoFK0Ujt9OzL6+sogfwOw==", 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T21:25:04.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDiuOBmFJzKLRAAMNJKnBlJ3cCtXlSDkTgubw==", 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-osd", 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDiuOBmoKnKLRAAiHwB2xGU6M6wV+CRfFljLQ==", 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T21:25:04.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDiuOBm97fKLRAANyAXHHF4ldd1H6J8JUjX3Q==", 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDiuOBmHMfKLRAAF8MBZxtCL6LHd4SHwvxTnw==", 2024-09-10T21:25:04.692 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDiuOBm8tXKLRAAsjpFdbSRI/RxAxZPSvGnAQ==", 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA==", 2024-09-10T21:25:04.693 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw==", 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:04.694 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-10T21:25:04.695 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:25:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f343c0725e0 msgr2=0x7f343c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f343c0725e0 0x7f343c074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3450007fd0 tx=0x7f3450007ea0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600cfbd0 msgr2=0x7f346015fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600cfbd0 0x7f346015fbe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3448009790 tx=0x7f3448004040 comp rx=0 tx=0).stop 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 shutdown_connections 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f343c0725e0 0x7f343c074aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34600cffb0 0x7f34601500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34600cfbd0 0x7f346015fbe0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 --2- 172.21.15.97:0/3137813991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34600722c0 0x7f346015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 >> 172.21.15.97:0/3137813991 conn(0x7f34600d1a70 msgr2=0x7f34600cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 shutdown_connections 2024-09-10T21:25:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.672+0000 7f345fe00640 1 -- 172.21.15.97:0/3137813991 wait complete. 2024-09-10T21:25:04.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T21:25:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.832+0000 7f8d3c200640 1 -- 172.21.15.97:0/886308479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406cda0 msgr2=0x7f8d34152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.832+0000 7f8d3c200640 1 --2- 172.21.15.97:0/886308479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406cda0 0x7f8d34152760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009900 tx=0x7f8d2801c870 comp rx=0 tx=0).stop 2024-09-10T21:25:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.832+0000 7f8d3c200640 1 -- 172.21.15.97:0/886308479 shutdown_connections 2024-09-10T21:25:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.832+0000 7f8d3c200640 1 --2- 172.21.15.97:0/886308479 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d340d01a0 0x7f8d3415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.832+0000 7f8d3c200640 1 --2- 172.21.15.97:0/886308479 >> v2:172.21.15.194:3300/0 conn(0x7f8d3406d180 0x7f8d34152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.832+0000 7f8d3c200640 1 --2- 172.21.15.97:0/886308479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406cda0 0x7f8d34152760 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.832+0000 7f8d3c200640 1 -- 172.21.15.97:0/886308479 >> 172.21.15.97:0/886308479 conn(0x7f8d3406e2b0 msgr2=0x7f8d340b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:04.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 -- 172.21.15.97:0/886308479 shutdown_connections 2024-09-10T21:25:04.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 -- 172.21.15.97:0/886308479 wait complete. 2024-09-10T21:25:04.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 Processor -- start 2024-09-10T21:25:04.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 -- start start 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406cda0 0x7f8d341485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406d180 0x7f8d341f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d340d01a0 0x7f8d34149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34073ad0 con 0x7f8d3406d180 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34073950 con 0x7f8d3406cda0 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3c200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8d34073c50 con 0x7f8d340d01a0 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406d180 0x7f8d341f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406d180 0x7f8d341f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40886/0 (socket says 172.21.15.97:40886) 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 -- 172.21.15.97:0/2182091757 learned_addr learned my addr 172.21.15.97:0/2182091757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d3a200640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d340d01a0 0x7f8d34149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d39800640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406cda0 0x7f8d341485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 -- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d340d01a0 msgr2=0x7f8d34149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d340d01a0 0x7f8d34149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 -- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406cda0 msgr2=0x7f8d341485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406cda0 0x7f8d341485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 -- 172.21.15.97:0/2182091757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d341f87f0 con 0x7f8d3406d180 2024-09-10T21:25:04.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.836+0000 7f8d38e00640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406d180 0x7f8d341f82b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000d950 tx=0x7f8d2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:04.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d31600640 1 -- 172.21.15.97:0/2182091757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20017070 con 0x7f8d3406d180 2024-09-10T21:25:04.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d31600640 1 -- 172.21.15.97:0/2182091757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d20009d20 con 0x7f8d3406d180 2024-09-10T21:25:04.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d31600640 1 -- 172.21.15.97:0/2182091757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d200106c0 con 0x7f8d3406d180 2024-09-10T21:25:04.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d34149fa0 con 0x7f8d3406d180 2024-09-10T21:25:04.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341f7ab0 con 0x7f8d3406d180 2024-09-10T21:25:04.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3415c260 con 0x7f8d3406d180 2024-09-10T21:25:04.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d31600640 1 -- 172.21.15.97:0/2182091757 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8d20010860 con 0x7f8d3406d180 2024-09-10T21:25:04.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d31600640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d14072610 0x7f8d14074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:04.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d31600640 1 -- 172.21.15.97:0/2182091757 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f8d20093380 con 0x7f8d3406d180 2024-09-10T21:25:04.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.840+0000 7f8d39800640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d14072610 0x7f8d14074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.844+0000 7f8d39800640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d14072610 0x7f8d14074ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d280af000 tx=0x7f8d2801b0b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:04.844+0000 7f8d31600640 1 -- 172.21.15.97:0/2182091757 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8d2005e660 con 0x7f8d3406d180 2024-09-10T21:25:05.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.072+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f8d3414f770 con 0x7f8d3406d180 2024-09-10T21:25:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.072+0000 7f8d31600640 1 -- 172.21.15.97:0/2182091757 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f8d20063090 con 0x7f8d3406d180 2024-09-10T21:25:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d14072610 msgr2=0x7f8d14074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d14072610 0x7f8d14074ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d280af000 tx=0x7f8d2801b0b0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406d180 msgr2=0x7f8d341f82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406d180 0x7f8d341f82b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000d950 tx=0x7f8d2000de10 comp rx=0 tx=0).stop 2024-09-10T21:25:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 shutdown_connections 2024-09-10T21:25:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d14072610 0x7f8d14074ad0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d340d01a0 0x7f8d34149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d3406d180 0x7f8d341f82b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 --2- 172.21.15.97:0/2182091757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406cda0 0x7f8d341485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 >> 172.21.15.97:0/2182091757 conn(0x7f8d3406e2b0 msgr2=0x7f8d3415d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 shutdown_connections 2024-09-10T21:25:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.076+0000 7f8d3c200640 1 -- 172.21.15.97:0/2182091757 wait complete. 2024-09-10T21:25:05.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T21:25:05.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- 172.21.15.97:0/1248242007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 msgr2=0x7f45f813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 --2- 172.21.15.97:0/1248242007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f813f9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009c80 tx=0x7f45f001cda0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- 172.21.15.97:0/1248242007 shutdown_connections 2024-09-10T21:25:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 --2- 172.21.15.97:0/1248242007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f813f9b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 --2- 172.21.15.97:0/1248242007 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45f8141d20 0x7f45f815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- 172.21.15.97:0/1248242007 >> 172.21.15.97:0/1248242007 conn(0x7f45f80b1780 msgr2=0x7f45f80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- 172.21.15.97:0/1248242007 shutdown_connections 2024-09-10T21:25:05.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- 172.21.15.97:0/1248242007 wait complete. 2024-09-10T21:25:05.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 Processor -- start 2024-09-10T21:25:05.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- start start 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45f80ce5f0 0x7f45f815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f8141d20 0x7f45f8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f8073640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45f8168bf0 con 0x7f45f815fc10 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45f8168a70 con 0x7f45f8141d20 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45fd000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f45f8168d70 con 0x7f45f80ce5f0 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45f6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f8073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45ed400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f8141d20 0x7f45f8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45f6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f8073640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40904/0 (socket says 172.21.15.97:40904) 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45f6800640 1 -- 172.21.15.97:0/3408856853 learned_addr learned my addr 172.21.15.97:0/3408856853 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.240+0000 7f45f5e00640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45f80ce5f0 0x7f45f815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45f6800640 1 -- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45f80ce5f0 msgr2=0x7f45f815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45f6800640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45f80ce5f0 0x7f45f815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45f6800640 1 -- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f8141d20 msgr2=0x7f45f8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45f6800640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f8141d20 0x7f45f8159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45f6800640 1 -- 172.21.15.97:0/3408856853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f815d200 con 0x7f45f815fc10 2024-09-10T21:25:05.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45f6800640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f8073640 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009bd0 tx=0x7f45f0009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45eea00640 1 -- 172.21.15.97:0/3408856853 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f00bb070 con 0x7f45f815fc10 2024-09-10T21:25:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45eea00640 1 -- 172.21.15.97:0/3408856853 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45f0004d40 con 0x7f45f815fc10 2024-09-10T21:25:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f815d540 con 0x7f45f815fc10 2024-09-10T21:25:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45eea00640 1 -- 172.21.15.97:0/3408856853 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f00b2040 con 0x7f45f815fc10 2024-09-10T21:25:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f815dd80 con 0x7f45f815fc10 2024-09-10T21:25:05.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f815f6d0 con 0x7f45f815fc10 2024-09-10T21:25:05.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45eea00640 1 -- 172.21.15.97:0/3408856853 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f45f00b5030 con 0x7f45f815fc10 2024-09-10T21:25:05.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45eea00640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45d40725e0 0x7f45d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.244+0000 7f45eea00640 1 -- 172.21.15.97:0/3408856853 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f45f012cb50 con 0x7f45f815fc10 2024-09-10T21:25:05.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.248+0000 7f45f5e00640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45d40725e0 0x7f45d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.248+0000 7f45f5e00640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45d40725e0 0x7f45d4074aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f45f8159aa0 tx=0x7f45e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:05.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.248+0000 7f45eea00640 1 -- 172.21.15.97:0/3408856853 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45f815d540 con 0x7f45f815fc10 2024-09-10T21:25:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.480+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f45f80cca70 con 0x7f45f815fc10 2024-09-10T21:25:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45eea00640 1 -- 172.21.15.97:0/3408856853 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f45f0016990 con 0x7f45f815fc10 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45d40725e0 msgr2=0x7f45d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45d40725e0 0x7f45d4074aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f45f8159aa0 tx=0x7f45e8009040 comp rx=0 tx=0).stop 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 msgr2=0x7f45f8073640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f8073640 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009bd0 tx=0x7f45f0009350 comp rx=0 tx=0).stop 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 shutdown_connections 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45d40725e0 0x7f45d4074aa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815fc10 0x7f45f8073640 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45f8141d20 0x7f45f8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 --2- 172.21.15.97:0/3408856853 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45f80ce5f0 0x7f45f815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 >> 172.21.15.97:0/3408856853 conn(0x7f45f80b1780 msgr2=0x7f45f80afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 shutdown_connections 2024-09-10T21:25:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.484+0000 7f45fd000640 1 -- 172.21.15.97:0/3408856853 wait complete. 2024-09-10T21:25:05.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T21:25:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- 172.21.15.97:0/854671282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 msgr2=0x7ff834166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 --2- 172.21.15.97:0/854671282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff834166fe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff834069d60 tx=0x7ff82c01eff0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- 172.21.15.97:0/854671282 shutdown_connections 2024-09-10T21:25:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 --2- 172.21.15.97:0/854671282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff834166fe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- 172.21.15.97:0/854671282 >> 172.21.15.97:0/854671282 conn(0x7ff8340b37c0 msgr2=0x7ff8340b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- 172.21.15.97:0/854671282 shutdown_connections 2024-09-10T21:25:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- 172.21.15.97:0/854671282 wait complete. 2024-09-10T21:25:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 Processor -- start 2024-09-10T21:25:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- start start 2024-09-10T21:25:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff8340cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8340ce830 0x7ff834157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8340cec10 0x7ff834151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8340cddd0 con 0x7ff8340b2570 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8340cdc50 con 0x7ff8340ce830 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff83a600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff8340ce1a0 con 0x7ff8340cec10 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff833400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff8340cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff833400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff8340cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40918/0 (socket says 172.21.15.97:40918) 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff833400640 1 -- 172.21.15.97:0/1874367047 learned_addr learned my addr 172.21.15.97:0/1874367047 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff833e00640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8340cec10 0x7ff834151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.648+0000 7ff82aa00640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8340ce830 0x7ff834157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff833400640 1 -- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8340cec10 msgr2=0x7ff834151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff833400640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8340cec10 0x7ff834151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff833400640 1 -- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8340ce830 msgr2=0x7ff834157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff833400640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8340ce830 0x7ff834157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff833400640 1 -- 172.21.15.97:0/1874367047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8340b6bb0 con 0x7ff8340b2570 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff833e00640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8340cec10 0x7ff834151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff833400640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff8340cb370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff82400c820 tx=0x7ff82400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff82be00640 1 -- 172.21.15.97:0/1874367047 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824017070 con 0x7ff8340b2570 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff82be00640 1 -- 172.21.15.97:0/1874367047 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff824005070 con 0x7ff8340b2570 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8340b6e90 con 0x7ff8340b2570 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff82be00640 1 -- 172.21.15.97:0/1874367047 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8240054c0 con 0x7ff8340b2570 2024-09-10T21:25:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8340cb8b0 con 0x7ff8340b2570 2024-09-10T21:25:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8340b70d0 con 0x7ff8340b2570 2024-09-10T21:25:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.652+0000 7ff82be00640 1 -- 172.21.15.97:0/1874367047 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff824005660 con 0x7ff8340b2570 2024-09-10T21:25:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.656+0000 7ff82be00640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff8100725e0 0x7ff810074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.656+0000 7ff82be00640 1 -- 172.21.15.97:0/1874367047 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ff824092b30 con 0x7ff8340b2570 2024-09-10T21:25:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.656+0000 7ff82aa00640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff8100725e0 0x7ff810074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.656+0000 7ff82aa00640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff8100725e0 0x7ff810074aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8341f5b00 tx=0x7ff81c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:05.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.660+0000 7ff82be00640 1 -- 172.21.15.97:0/1874367047 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8340b70d0 con 0x7ff8340b2570 2024-09-10T21:25:05.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.880+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7ff8340cf800 con 0x7ff8340b2570 2024-09-10T21:25:05.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.884+0000 7ff82be00640 1 -- 172.21.15.97:0/1874367047 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7ff82405e330 con 0x7ff8340b2570 2024-09-10T21:25:05.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff8100725e0 msgr2=0x7ff810074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff8100725e0 0x7ff810074aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8341f5b00 tx=0x7ff81c008040 comp rx=0 tx=0).stop 2024-09-10T21:25:05.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 msgr2=0x7ff8340cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff8340cb370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff82400c820 tx=0x7ff82400cce0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 shutdown_connections 2024-09-10T21:25:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff8100725e0 0x7ff810074aa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8340cec10 0x7ff834151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8340ce830 0x7ff834157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 --2- 172.21.15.97:0/1874367047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8340b2570 0x7ff8340cb370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 >> 172.21.15.97:0/1874367047 conn(0x7ff8340b37c0 msgr2=0x7ff8341581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 shutdown_connections 2024-09-10T21:25:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:05.888+0000 7ff83a600640 1 -- 172.21.15.97:0/1874367047 wait complete. 2024-09-10T21:25:05.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.048+0000 7f135a000640 1 -- 172.21.15.97:0/3236665011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354143f90 msgr2=0x7f13540b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.048+0000 7f135a000640 1 --2- 172.21.15.97:0/3236665011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354143f90 0x7f13540b2a80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1354073080 tx=0x7f134401c6e0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- 172.21.15.97:0/3236665011 shutdown_connections 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 --2- 172.21.15.97:0/3236665011 >> v2:172.21.15.194:3300/0 conn(0x7f135406fd50 0x7f1354070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 --2- 172.21.15.97:0/3236665011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354143f90 0x7f13540b2a80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- 172.21.15.97:0/3236665011 >> 172.21.15.97:0/3236665011 conn(0x7f13540b1780 msgr2=0x7f13540b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- 172.21.15.97:0/3236665011 shutdown_connections 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- 172.21.15.97:0/3236665011 wait complete. 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 Processor -- start 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- start start 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f135406fd50 0x7f135413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1354143f90 0x7f1354140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354144690 0x7f1354138010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1354167ca0 con 0x7f1354144690 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1354167b20 con 0x7f1354143f90 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1354167e20 con 0x7f135406fd50 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354144690 0x7f1354138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354144690 0x7f1354138010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40944/0 (socket says 172.21.15.97:40944) 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 -- 172.21.15.97:0/2592506251 learned_addr learned my addr 172.21.15.97:0/2592506251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1352a00640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1354143f90 0x7f1354140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353400640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f135406fd50 0x7f135413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 -- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f135406fd50 msgr2=0x7f135413bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f135406fd50 0x7f135413bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 -- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1354143f90 msgr2=0x7f1354140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1354143f90 0x7f1354140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 -- 172.21.15.97:0/2592506251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135415c760 con 0x7f1354144690 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f1353e00640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354144690 0x7f1354138010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f134c00ea20 tx=0x7f134c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f134b400640 1 -- 172.21.15.97:0/2592506251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c00cde0 con 0x7f1354144690 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f134b400640 1 -- 172.21.15.97:0/2592506251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f134c004c40 con 0x7f1354144690 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135415ca40 con 0x7f1354144690 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.052+0000 7f134b400640 1 -- 172.21.15.97:0/2592506251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c005280 con 0x7f1354144690 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.056+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135415d000 con 0x7f1354144690 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.056+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13541fafa0 con 0x7f1354144690 2024-09-10T21:25:06.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.056+0000 7f134b400640 1 -- 172.21.15.97:0/2592506251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f134c00f040 con 0x7f1354144690 2024-09-10T21:25:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.056+0000 7f134b400640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.056+0000 7f134b400640 1 -- 172.21.15.97:0/2592506251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f134c093740 con 0x7f1354144690 2024-09-10T21:25:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.056+0000 7f1353400640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.060+0000 7f1353400640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f13440ae000 tx=0x7f134401af20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.060+0000 7f134b400640 1 -- 172.21.15.97:0/2592506251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f134c016030 con 0x7f1354144690 2024-09-10T21:25:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.288+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1354141f60 con 0x7f1354144690 2024-09-10T21:25:06.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f134b400640 1 -- 172.21.15.97:0/2592506251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f134c05e8f0 con 0x7f1354144690 2024-09-10T21:25:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 msgr2=0x7f1334074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f13440ae000 tx=0x7f134401af20 comp rx=0 tx=0).stop 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354144690 msgr2=0x7f1354138010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354144690 0x7f1354138010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f134c00ea20 tx=0x7f134c00eee0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 shutdown_connections 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13340725e0 0x7f1334074aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1354144690 0x7f1354138010 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1354143f90 0x7f1354140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 --2- 172.21.15.97:0/2592506251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f135406fd50 0x7f135413bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 >> 172.21.15.97:0/2592506251 conn(0x7f13540b1780 msgr2=0x7f13540ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 shutdown_connections 2024-09-10T21:25:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.292+0000 7f135a000640 1 -- 172.21.15.97:0/2592506251 wait complete. 2024-09-10T21:25:06.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T21:25:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- 172.21.15.97:0/1303228830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080cdcb0 msgr2=0x7f85080cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 --2- 172.21.15.97:0/1303228830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080cdcb0 0x7f85080cff50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009900 tx=0x7f84f001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- 172.21.15.97:0/1303228830 shutdown_connections 2024-09-10T21:25:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 --2- 172.21.15.97:0/1303228830 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85080d0490 0x7f85080ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 --2- 172.21.15.97:0/1303228830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080cdcb0 0x7f85080cff50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- 172.21.15.97:0/1303228830 >> 172.21.15.97:0/1303228830 conn(0x7f85080d1a60 msgr2=0x7f85080d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- 172.21.15.97:0/1303228830 shutdown_connections 2024-09-10T21:25:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- 172.21.15.97:0/1303228830 wait complete. 2024-09-10T21:25:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 Processor -- start 2024-09-10T21:25:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- start start 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85080cdcb0 0x7f8508150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080d0490 0x7f8508153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f850815efc0 0x7f85080b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8508168930 con 0x7f85080d0490 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f85081687b0 con 0x7f85080cdcb0 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.456+0000 7f850e400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8508168ab0 con 0x7f850815efc0 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080d0490 0x7f8508153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080d0490 0x7f8508153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40962/0 (socket says 172.21.15.97:40962) 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 -- 172.21.15.97:0/971466321 learned_addr learned my addr 172.21.15.97:0/971466321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f850d000640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85080cdcb0 0x7f8508150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f850da00640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f850815efc0 0x7f85080b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 -- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f850815efc0 msgr2=0x7f85080b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f850815efc0 0x7f85080b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 -- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85080cdcb0 msgr2=0x7f8508150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85080cdcb0 0x7f8508150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 -- 172.21.15.97:0/971466321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85081514b0 con 0x7f85080d0490 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f850da00640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f850815efc0 0x7f85080b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f84ffe00640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080d0490 0x7f8508153690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009850 tx=0x7f84f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f8505600640 1 -- 172.21.15.97:0/971466321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f00b2040 con 0x7f85080d0490 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f8505600640 1 -- 172.21.15.97:0/971466321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84f0005030 con 0x7f85080d0490 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f8505600640 1 -- 172.21.15.97:0/971466321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0005440 con 0x7f85080d0490 2024-09-10T21:25:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85081521e0 con 0x7f85080d0490 2024-09-10T21:25:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8508152760 con 0x7f85080d0490 2024-09-10T21:25:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85080ae930 con 0x7f85080d0490 2024-09-10T21:25:06.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.460+0000 7f8505600640 1 -- 172.21.15.97:0/971466321 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f84f00c0060 con 0x7f85080d0490 2024-09-10T21:25:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.464+0000 7f8505600640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84e4072610 0x7f84e4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.464+0000 7f8505600640 1 -- 172.21.15.97:0/971466321 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f84f012c7c0 con 0x7f85080d0490 2024-09-10T21:25:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.464+0000 7f850d000640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84e4072610 0x7f84e4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.464+0000 7f850d000640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84e4072610 0x7f84e4074ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f84f80098e0 tx=0x7f84f8009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.468+0000 7f8505600640 1 -- 172.21.15.97:0/971466321 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f84f00164d0 con 0x7f85080d0490 2024-09-10T21:25:06.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.692+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f85080cfde0 con 0x7f85080d0490 2024-09-10T21:25:06.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f8505600640 1 -- 172.21.15.97:0/971466321 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f84f00f7a00 con 0x7f85080d0490 2024-09-10T21:25:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84e4072610 msgr2=0x7f84e4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84e4072610 0x7f84e4074ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f84f80098e0 tx=0x7f84f8009060 comp rx=0 tx=0).stop 2024-09-10T21:25:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080d0490 msgr2=0x7f8508153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080d0490 0x7f8508153690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009850 tx=0x7f84f0004040 comp rx=0 tx=0).stop 2024-09-10T21:25:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 shutdown_connections 2024-09-10T21:25:06.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84e4072610 0x7f84e4074ad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f850815efc0 0x7f85080b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85080d0490 0x7f8508153690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 --2- 172.21.15.97:0/971466321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85080cdcb0 0x7f8508150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 >> 172.21.15.97:0/971466321 conn(0x7f85080d1a60 msgr2=0x7f8508154160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:06.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 shutdown_connections 2024-09-10T21:25:06.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.696+0000 7f850e400640 1 -- 172.21.15.97:0/971466321 wait complete. 2024-09-10T21:25:06.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T21:25:06.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- 172.21.15.97:0/3734785205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c15eed0 msgr2=0x7fa99c0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/3734785205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c15eed0 0x7fa99c0d27f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009840 tx=0x7fa98c01c7f0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- 172.21.15.97:0/3734785205 shutdown_connections 2024-09-10T21:25:06.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/3734785205 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa99c0ccc60 0x7fa99c0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/3734785205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c15eed0 0x7fa99c0d27f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- 172.21.15.97:0/3734785205 >> 172.21.15.97:0/3734785205 conn(0x7fa99c0d1cd0 msgr2=0x7fa99c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:06.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- 172.21.15.97:0/3734785205 shutdown_connections 2024-09-10T21:25:06.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- 172.21.15.97:0/3734785205 wait complete. 2024-09-10T21:25:06.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 Processor -- start 2024-09-10T21:25:06.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- start start 2024-09-10T21:25:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c0ccc60 0x7fa99c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c0cdd40 0x7fa99c14b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa99c15eed0 0x7fa99c14bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c168880 con 0x7fa99c0cdd40 2024-09-10T21:25:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c168700 con 0x7fa99c0ccc60 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.860+0000 7fa9a0a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c168a00 con 0x7fa99c15eed0 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c0cdd40 0x7fa99c14b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c0cdd40 0x7fa99c14b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40970/0 (socket says 172.21.15.97:40970) 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 -- 172.21.15.97:0/1855492385 learned_addr learned my addr 172.21.15.97:0/1855492385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa999e00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c0ccc60 0x7fa99c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa99a800640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa99c15eed0 0x7fa99c14bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 -- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa99c15eed0 msgr2=0x7fa99c14bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa99c15eed0 0x7fa99c14bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 -- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c0ccc60 msgr2=0x7fa99c1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c0ccc60 0x7fa99c1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 -- 172.21.15.97:0/1855492385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c15d700 con 0x7fa99c0cdd40 2024-09-10T21:25:06.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa991400640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c0cdd40 0x7fa99c14b640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c4d0 tx=0x7fa98400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa992a00640 1 -- 172.21.15.97:0/1855492385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984016020 con 0x7fa99c0cdd40 2024-09-10T21:25:06.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa992a00640 1 -- 172.21.15.97:0/1855492385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa984012920 con 0x7fa99c0cdd40 2024-09-10T21:25:06.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c1599d0 con 0x7fa99c0cdd40 2024-09-10T21:25:06.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa992a00640 1 -- 172.21.15.97:0/1855492385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984012d30 con 0x7fa99c0cdd40 2024-09-10T21:25:06.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c159f90 con 0x7fa99c0cdd40 2024-09-10T21:25:06.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c0cfd60 con 0x7fa99c0cdd40 2024-09-10T21:25:06.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.864+0000 7fa992a00640 1 -- 172.21.15.97:0/1855492385 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa98401b400 con 0x7fa99c0cdd40 2024-09-10T21:25:06.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.868+0000 7fa992a00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa978072610 0x7fa978074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.868+0000 7fa992a00640 1 -- 172.21.15.97:0/1855492385 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fa9840930e0 con 0x7fa99c0cdd40 2024-09-10T21:25:06.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.868+0000 7fa999e00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa978072610 0x7fa978074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.868+0000 7fa999e00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa978072610 0x7fa978074ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0af000 tx=0x7fa98c01b030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:06.872+0000 7fa992a00640 1 -- 172.21.15.97:0/1855492385 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa984020020 con 0x7fa99c0cdd40 2024-09-10T21:25:07.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.092+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fa99c0b0470 con 0x7fa99c0cdd40 2024-09-10T21:25:07.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa992a00640 1 -- 172.21.15.97:0/1855492385 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fa99c15d700 con 0x7fa99c0cdd40 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa978072610 msgr2=0x7fa978074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa978072610 0x7fa978074ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0af000 tx=0x7fa98c01b030 comp rx=0 tx=0).stop 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c0cdd40 msgr2=0x7fa99c14b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c0cdd40 0x7fa99c14b640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c4d0 tx=0x7fa98400c990 comp rx=0 tx=0).stop 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 shutdown_connections 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa978072610 0x7fa978074ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa99c15eed0 0x7fa99c14bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c0cdd40 0x7fa99c14b640 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 --2- 172.21.15.97:0/1855492385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa99c0ccc60 0x7fa99c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 >> 172.21.15.97:0/1855492385 conn(0x7fa99c0d1cd0 msgr2=0x7fa99c0d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 shutdown_connections 2024-09-10T21:25:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.096+0000 7fa9a0a00640 1 -- 172.21.15.97:0/1855492385 wait complete. 2024-09-10T21:25:07.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:25:07.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 -- 172.21.15.97:0/2328381609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 msgr2=0x7f1734004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 --2- 172.21.15.97:0/2328381609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 0x7f1734004810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f17380016c0 tx=0x7f173801c790 comp rx=0 tx=0).stop 2024-09-10T21:25:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f174f400640 1 -- 172.21.15.97:0/2328381609 reap_dead start 2024-09-10T21:25:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 -- 172.21.15.97:0/2328381609 shutdown_connections 2024-09-10T21:25:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 -- 172.21.15.97:0/2328381609 >> 172.21.15.97:0/2328381609 conn(0x7f17500b1780 msgr2=0x7f17500b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 -- 172.21.15.97:0/2328381609 shutdown_connections 2024-09-10T21:25:07.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 -- 172.21.15.97:0/2328381609 wait complete. 2024-09-10T21:25:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 Processor -- start 2024-09-10T21:25:07.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.260+0000 7f1755600640 1 -- start start 2024-09-10T21:25:07.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 0x7f1750159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17501566f0 0x7f1750143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17501333d0 0x7f1750072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17501400c0 con 0x7f1734004060 2024-09-10T21:25:07.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1750142200 con 0x7f17501566f0 2024-09-10T21:25:07.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1750140240 con 0x7f17501333d0 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 0x7f1750159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 0x7f1750159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40988/0 (socket says 172.21.15.97:40988) 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 -- 172.21.15.97:0/2755136409 learned_addr learned my addr 172.21.15.97:0/2755136409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174f400640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17501333d0 0x7f1750072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1746000640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17501566f0 0x7f1750143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 -- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17501333d0 msgr2=0x7f1750072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17501333d0 0x7f1750072630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 -- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17501566f0 msgr2=0x7f1750143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17501566f0 0x7f1750143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 -- 172.21.15.97:0/2755136409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17501f5ff0 con 0x7f1734004060 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f174ea00640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 0x7f1750159640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1740011d70 tx=0x7f174000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1747400640 1 -- 172.21.15.97:0/2755136409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740011e60 con 0x7f1734004060 2024-09-10T21:25:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1747400640 1 -- 172.21.15.97:0/2755136409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1740003d90 con 0x7f1734004060 2024-09-10T21:25:07.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17501f62d0 con 0x7f1734004060 2024-09-10T21:25:07.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1747400640 1 -- 172.21.15.97:0/2755136409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740004290 con 0x7f1734004060 2024-09-10T21:25:07.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17501f6890 con 0x7f1734004060 2024-09-10T21:25:07.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.264+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17501407d0 con 0x7f1734004060 2024-09-10T21:25:07.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.268+0000 7f1747400640 1 -- 172.21.15.97:0/2755136409 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1740018070 con 0x7f1734004060 2024-09-10T21:25:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.268+0000 7f1747400640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f172c0725e0 0x7f172c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.268+0000 7f1747400640 1 -- 172.21.15.97:0/2755136409 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f17400940f0 con 0x7f1734004060 2024-09-10T21:25:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.268+0000 7f1746000640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f172c0725e0 0x7f172c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.268+0000 7f1746000640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f172c0725e0 0x7f172c074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1750070b40 tx=0x7f17380b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:07.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.272+0000 7f1747400640 1 -- 172.21.15.97:0/2755136409 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f174005f3d0 con 0x7f1734004060 2024-09-10T21:25:07.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1750141360 con 0x7f1734004060 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1747400640 1 -- 172.21.15.97:0/2755136409 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f1740063e00 con 0x7f1734004060 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:07.450 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:25:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f172c0725e0 msgr2=0x7f172c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f172c0725e0 0x7f172c074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1750070b40 tx=0x7f17380b1040 comp rx=0 tx=0).stop 2024-09-10T21:25:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 msgr2=0x7f1750159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 0x7f1750159640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1740011d70 tx=0x7f174000f790 comp rx=0 tx=0).stop 2024-09-10T21:25:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 shutdown_connections 2024-09-10T21:25:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f172c0725e0 0x7f172c074aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17501333d0 0x7f1750072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17501566f0 0x7f1750143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 --2- 172.21.15.97:0/2755136409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1734004060 0x7f1750159640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 >> 172.21.15.97:0/2755136409 conn(0x7f17500b1780 msgr2=0x7f1750073200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 shutdown_connections 2024-09-10T21:25:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.444+0000 7f1755600640 1 -- 172.21.15.97:0/2755136409 wait complete. 2024-09-10T21:25:07.471 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:25:07.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:25:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 -- 172.21.15.97:0/2950444081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840ce640 msgr2=0x7f01840cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 --2- 172.21.15.97:0/2950444081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840ce640 0x7f01840cea20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f017401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 -- 172.21.15.97:0/2950444081 shutdown_connections 2024-09-10T21:25:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 --2- 172.21.15.97:0/2950444081 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f01840b0d50 0x7f01840aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 --2- 172.21.15.97:0/2950444081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840ce640 0x7f01840cea20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 -- 172.21.15.97:0/2950444081 >> 172.21.15.97:0/2950444081 conn(0x7f01840d1c30 msgr2=0x7f01840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 -- 172.21.15.97:0/2950444081 shutdown_connections 2024-09-10T21:25:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.608+0000 7f018b600640 1 -- 172.21.15.97:0/2950444081 wait complete. 2024-09-10T21:25:07.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 Processor -- start 2024-09-10T21:25:07.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 -- start start 2024-09-10T21:25:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840af300 0x7f0184158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f01840b0d50 0x7f01840c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01840ce640 0x7f01840c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f01841690f0 con 0x7f01840af300 2024-09-10T21:25:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f01840b00d0 con 0x7f01840ce640 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0184169270 con 0x7f01840b0d50 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840af300 0x7f0184158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840af300 0x7f0184158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41122/0 (socket says 172.21.15.97:41122) 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 -- 172.21.15.97:0/232688187 learned_addr learned my addr 172.21.15.97:0/232688187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f017be00640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f01840b0d50 0x7f01840c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0189600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01840ce640 0x7f01840c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 -- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f01840b0d50 msgr2=0x7f01840c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f01840b0d50 0x7f01840c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 -- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01840ce640 msgr2=0x7f01840c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01840ce640 0x7f01840c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 -- 172.21.15.97:0/232688187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0184163cd0 con 0x7f01840af300 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f017be00640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f01840b0d50 0x7f01840c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0188c00640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840af300 0x7f0184158190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0174009850 tx=0x7f01740a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0181600640 1 -- 172.21.15.97:0/232688187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740b2040 con 0x7f01840af300 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0181600640 1 -- 172.21.15.97:0/232688187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01740044b0 con 0x7f01840af300 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f0181600640 1 -- 172.21.15.97:0/232688187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174005270 con 0x7f01840af300 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841586d0 con 0x7f01840af300 2024-09-10T21:25:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.612+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018406e4a0 con 0x7f01840af300 2024-09-10T21:25:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.616+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01840aedc0 con 0x7f01840af300 2024-09-10T21:25:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.616+0000 7f0181600640 1 -- 172.21.15.97:0/232688187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0174005410 con 0x7f01840af300 2024-09-10T21:25:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.616+0000 7f0181600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01600725e0 0x7f0160074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.616+0000 7f0181600640 1 -- 172.21.15.97:0/232688187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f017412cb00 con 0x7f01840af300 2024-09-10T21:25:07.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.616+0000 7f017be00640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01600725e0 0x7f0160074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:07.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.616+0000 7f017be00640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01600725e0 0x7f0160074aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f016c0058d0 tx=0x7f016c005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:07.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.620+0000 7f0181600640 1 -- 172.21.15.97:0/232688187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f01740164d0 con 0x7f01840af300 2024-09-10T21:25:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:07.788+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f01840c89c0 con 0x7f01840af300 2024-09-10T21:25:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.272+0000 7f0181600640 1 -- 172.21.15.97:0/232688187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f01740f8300 con 0x7f01840af300 2024-09-10T21:25:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-09-10T21:25:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01600725e0 msgr2=0x7f0160074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01600725e0 0x7f0160074aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f016c0058d0 tx=0x7f016c005860 comp rx=0 tx=0).stop 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840af300 msgr2=0x7f0184158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840af300 0x7f0184158190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0174009850 tx=0x7f01740a5810 comp rx=0 tx=0).stop 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 shutdown_connections 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01600725e0 0x7f0160074aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01840ce640 0x7f01840c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f01840b0d50 0x7f01840c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 --2- 172.21.15.97:0/232688187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01840af300 0x7f0184158190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 >> 172.21.15.97:0/232688187 conn(0x7f01840d1c30 msgr2=0x7f01840b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 shutdown_connections 2024-09-10T21:25:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.276+0000 7f018b600640 1 -- 172.21.15.97:0/232688187 wait complete. 2024-09-10T21:25:08.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:25:08.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- 172.21.15.97:0/512258428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50144690 msgr2=0x7efc500b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 --2- 172.21.15.97:0/512258428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50144690 0x7efc500b2a80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc38009840 tx=0x7efc3801c820 comp rx=0 tx=0).stop 2024-09-10T21:25:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- 172.21.15.97:0/512258428 shutdown_connections 2024-09-10T21:25:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 --2- 172.21.15.97:0/512258428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc5006fd50 0x7efc500701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 --2- 172.21.15.97:0/512258428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50144690 0x7efc500b2a80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- 172.21.15.97:0/512258428 >> 172.21.15.97:0/512258428 conn(0x7efc500b1780 msgr2=0x7efc500b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- 172.21.15.97:0/512258428 shutdown_connections 2024-09-10T21:25:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- 172.21.15.97:0/512258428 wait complete. 2024-09-10T21:25:08.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 Processor -- start 2024-09-10T21:25:08.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- start start 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc5006fd50 0x7efc500aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50143f90 0x7efc501f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc50144690 0x7efc501f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc50167c50 con 0x7efc50143f90 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efc50167ad0 con 0x7efc5006fd50 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc55e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efc50167dd0 con 0x7efc50144690 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50143f90 0x7efc501f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50143f90 0x7efc501f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41148/0 (socket says 172.21.15.97:41148) 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 -- 172.21.15.97:0/708792377 learned_addr learned my addr 172.21.15.97:0/708792377 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc4ea00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc5006fd50 0x7efc500aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc4f400640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc50144690 0x7efc501f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 -- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc50144690 msgr2=0x7efc501f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc50144690 0x7efc501f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 -- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc5006fd50 msgr2=0x7efc500aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc5006fd50 0x7efc500aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc46000640 1 -- 172.21.15.97:0/708792377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc50138ba0 con 0x7efc50143f90 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.452+0000 7efc4ea00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc5006fd50 0x7efc500aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc46000640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50143f90 0x7efc501f9060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc3801cd00 tx=0x7efc380a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc47e00640 1 -- 172.21.15.97:0/708792377 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38004040 con 0x7efc50143f90 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc47e00640 1 -- 172.21.15.97:0/708792377 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc380042d0 con 0x7efc50143f90 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc50138e80 con 0x7efc50143f90 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc47e00640 1 -- 172.21.15.97:0/708792377 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38003600 con 0x7efc50143f90 2024-09-10T21:25:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5013c390 con 0x7efc50143f90 2024-09-10T21:25:08.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc500701e0 con 0x7efc50143f90 2024-09-10T21:25:08.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc47e00640 1 -- 172.21.15.97:0/708792377 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efc380037a0 con 0x7efc50143f90 2024-09-10T21:25:08.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc47e00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc2c0725e0 0x7efc2c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.456+0000 7efc47e00640 1 -- 172.21.15.97:0/708792377 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7efc3812e1a0 con 0x7efc50143f90 2024-09-10T21:25:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.460+0000 7efc4ea00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc2c0725e0 0x7efc2c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.460+0000 7efc4ea00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc2c0725e0 0x7efc2c074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efc40001850 tx=0x7efc40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:08.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.460+0000 7efc47e00640 1 -- 172.21.15.97:0/708792377 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc38016410 con 0x7efc50143f90 2024-09-10T21:25:08.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:08.622 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:08.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:08.632+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efc500cc4e0 con 0x7efc50143f90 2024-09-10T21:25:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.280+0000 7efc47e00640 1 -- 172.21.15.97:0/708792377 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7efc380f9960 con 0x7efc50143f90 2024-09-10T21:25:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc2c0725e0 msgr2=0x7efc2c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc2c0725e0 0x7efc2c074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efc40001850 tx=0x7efc40009040 comp rx=0 tx=0).stop 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50143f90 msgr2=0x7efc501f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50143f90 0x7efc501f9060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc3801cd00 tx=0x7efc380a6570 comp rx=0 tx=0).stop 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 shutdown_connections 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc2c0725e0 0x7efc2c074aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc50144690 0x7efc501f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc50143f90 0x7efc501f9060 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 --2- 172.21.15.97:0/708792377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc5006fd50 0x7efc500aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 >> 172.21.15.97:0/708792377 conn(0x7efc500b1780 msgr2=0x7efc500b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:09.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 shutdown_connections 2024-09-10T21:25:09.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.284+0000 7efc55e00640 1 -- 172.21.15.97:0/708792377 wait complete. 2024-09-10T21:25:09.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:25:09.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- 172.21.15.97:0/3313948491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 msgr2=0x7f359c0b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:09.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 --2- 172.21.15.97:0/3313948491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 0x7f359c0b4380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f358c009840 tx=0x7f358c01c5d0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- 172.21.15.97:0/3313948491 shutdown_connections 2024-09-10T21:25:09.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 --2- 172.21.15.97:0/3313948491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 0x7f359c0b4380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- 172.21.15.97:0/3313948491 >> 172.21.15.97:0/3313948491 conn(0x7f359c0b28c0 msgr2=0x7f359c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:09.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- 172.21.15.97:0/3313948491 shutdown_connections 2024-09-10T21:25:09.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- 172.21.15.97:0/3313948491 wait complete. 2024-09-10T21:25:09.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 Processor -- start 2024-09-10T21:25:09.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- start start 2024-09-10T21:25:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c15e750 0x7f359c14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 0x7f359c157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f359c158c60 0x7f359c158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f359c168ef0 con 0x7f359c15e750 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f359c168d70 con 0x7f359c15f150 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f35a2000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f359c169070 con 0x7f359c158c60 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c15e750 0x7f359c14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c15e750 0x7f359c14e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41164/0 (socket says 172.21.15.97:41164) 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 -- 172.21.15.97:0/1885480387 learned_addr learned my addr 172.21.15.97:0/1885480387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359aa00640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 0x7f359c157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359be00640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f359c158c60 0x7f359c158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 -- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f359c158c60 msgr2=0x7f359c158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f359c158c60 0x7f359c158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 -- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 msgr2=0x7f359c157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 0x7f359c157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.472+0000 7f359b400640 1 -- 172.21.15.97:0/1885480387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c15b230 con 0x7f359c15e750 2024-09-10T21:25:09.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f359b400640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c15e750 0x7f359c14e740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f358c009790 tx=0x7f358c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:09.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f3593400640 1 -- 172.21.15.97:0/1885480387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0ad070 con 0x7f359c15e750 2024-09-10T21:25:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f3593400640 1 -- 172.21.15.97:0/1885480387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f358c0aab90 con 0x7f359c15e750 2024-09-10T21:25:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f3593400640 1 -- 172.21.15.97:0/1885480387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0046d0 con 0x7f359c15e750 2024-09-10T21:25:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c1f8260 con 0x7f359c15e750 2024-09-10T21:25:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1f8820 con 0x7f359c15e750 2024-09-10T21:25:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c0b4380 con 0x7f359c15e750 2024-09-10T21:25:09.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f3593400640 1 -- 172.21.15.97:0/1885480387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f358c004870 con 0x7f359c15e750 2024-09-10T21:25:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.476+0000 7f3593400640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f357c0725e0 0x7f357c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:09.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.480+0000 7f3593400640 1 -- 172.21.15.97:0/1885480387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f358c12de80 con 0x7f359c15e750 2024-09-10T21:25:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.480+0000 7f359aa00640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f357c0725e0 0x7f357c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:09.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.480+0000 7f359aa00640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f357c0725e0 0x7f357c074aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3588000fc0 tx=0x7f3588009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:09.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.480+0000 7f3593400640 1 -- 172.21.15.97:0/1885480387 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f358c0b8070 con 0x7f359c15e750 2024-09-10T21:25:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:09.676+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f359c0ce7b0 con 0x7f359c15e750 2024-09-10T21:25:10.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.296+0000 7f3593400640 1 -- 172.21.15.97:0/1885480387 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f358c016310 con 0x7f359c15e750 2024-09-10T21:25:10.305 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 5 and data pool 6 2024-09-10T21:25:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f357c0725e0 msgr2=0x7f357c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f357c0725e0 0x7f357c074aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3588000fc0 tx=0x7f3588009040 comp rx=0 tx=0).stop 2024-09-10T21:25:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c15e750 msgr2=0x7f359c14e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c15e750 0x7f359c14e740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f358c009790 tx=0x7f358c005d80 comp rx=0 tx=0).stop 2024-09-10T21:25:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 shutdown_connections 2024-09-10T21:25:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f357c0725e0 0x7f357c074aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f359c158c60 0x7f359c158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f359c15f150 0x7f359c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 --2- 172.21.15.97:0/1885480387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c15e750 0x7f359c14e740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.304+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 >> 172.21.15.97:0/1885480387 conn(0x7f359c0b28c0 msgr2=0x7f359c0cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.308+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 shutdown_connections 2024-09-10T21:25:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.308+0000 7f35a2000640 1 -- 172.21.15.97:0/1885480387 wait complete. 2024-09-10T21:25:10.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- 172.21.15.97:0/3080744122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 msgr2=0x7fe5d80cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 --2- 172.21.15.97:0/3080744122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d80cea20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009840 tx=0x7fe5cc01c820 comp rx=0 tx=0).stop 2024-09-10T21:25:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- 172.21.15.97:0/3080744122 shutdown_connections 2024-09-10T21:25:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 --2- 172.21.15.97:0/3080744122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d80cea20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- 172.21.15.97:0/3080744122 >> 172.21.15.97:0/3080744122 conn(0x7fe5d80d1c30 msgr2=0x7fe5d80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- 172.21.15.97:0/3080744122 shutdown_connections 2024-09-10T21:25:10.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- 172.21.15.97:0/3080744122 wait complete. 2024-09-10T21:25:10.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 Processor -- start 2024-09-10T21:25:10.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- start start 2024-09-10T21:25:10.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d80b0d50 0x7fe5d814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d80cd6e0 0x7fe5d81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d816a8a0 con 0x7fe5d80ce640 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d816a720 con 0x7fe5d80cd6e0 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5e0800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d816aa20 con 0x7fe5d80b0d50 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d814ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41184/0 (socket says 172.21.15.97:41184) 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.524+0000 7fe5de800640 1 -- 172.21.15.97:0/1224462795 learned_addr learned my addr 172.21.15.97:0/1224462795 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5dd400640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d80cd6e0 0x7fe5d81572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5dde00640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d80b0d50 0x7fe5d814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5de800640 1 -- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d80b0d50 msgr2=0x7fe5d814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5de800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d80b0d50 0x7fe5d814ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5de800640 1 -- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d80cd6e0 msgr2=0x7fe5d81572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5de800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d80cd6e0 0x7fe5d81572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5de800640 1 -- 172.21.15.97:0/1224462795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d814ff40 con 0x7fe5d80ce640 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5dde00640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d80b0d50 0x7fe5d814ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5dd400640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d80cd6e0 0x7fe5d81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5de800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d814ed70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c780 tx=0x7fe5d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5d2000640 1 -- 172.21.15.97:0/1224462795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4004b20 con 0x7fe5d80ce640 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5d2000640 1 -- 172.21.15.97:0/1224462795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5d4004db0 con 0x7fe5d80ce640 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d81501e0 con 0x7fe5d80ce640 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5d2000640 1 -- 172.21.15.97:0/1224462795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400aa90 con 0x7fe5d80ce640 2024-09-10T21:25:10.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d81507a0 con 0x7fe5d80ce640 2024-09-10T21:25:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d80d04b0 con 0x7fe5d80ce640 2024-09-10T21:25:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5d2000640 1 -- 172.21.15.97:0/1224462795 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe5d400ac30 con 0x7fe5d80ce640 2024-09-10T21:25:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.528+0000 7fe5d2000640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5bc0725e0 0x7fe5bc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.532+0000 7fe5d2000640 1 -- 172.21.15.97:0/1224462795 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe5d40935d0 con 0x7fe5d80ce640 2024-09-10T21:25:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.532+0000 7fe5dde00640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5bc0725e0 0x7fe5bc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:10.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.532+0000 7fe5dde00640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5bc0725e0 0x7fe5bc074aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0af000 tx=0x7fe5cc01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:10.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.532+0000 7fe5d2000640 1 -- 172.21.15.97:0/1224462795 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5d4020070 con 0x7fe5d80ce640 2024-09-10T21:25:10.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.700+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5d8075f90 con 0x7fe5d80ce640 2024-09-10T21:25:10.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.700+0000 7fe5d2000640 1 -- 172.21.15.97:0/1224462795 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17039 (secure 0 0 0) 0x7fe5d405eb20 con 0x7fe5d80ce640 2024-09-10T21:25:10.709 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:10.709 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":39,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:25:09.683509+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:25:07.800828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T21:25:08.646331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:10.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5bc0725e0 msgr2=0x7fe5bc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5bc0725e0 0x7fe5bc074aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0af000 tx=0x7fe5cc01b060 comp rx=0 tx=0).stop 2024-09-10T21:25:10.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 msgr2=0x7fe5d814ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d814ed70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c780 tx=0x7fe5d400cc40 comp rx=0 tx=0).stop 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 shutdown_connections 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5bc0725e0 0x7fe5bc074aa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d80ce640 0x7fe5d814ed70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d80cd6e0 0x7fe5d81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 --2- 172.21.15.97:0/1224462795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d80b0d50 0x7fe5d814ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 >> 172.21.15.97:0/1224462795 conn(0x7fe5d80d1c30 msgr2=0x7fe5d8157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 shutdown_connections 2024-09-10T21:25:10.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.704+0000 7fe5e0800640 1 -- 172.21.15.97:0/1224462795 wait complete. 2024-09-10T21:25:10.731 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- 172.21.15.97:0/3200985762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4844004810 msgr2=0x7f4844004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 --2- 172.21.15.97:0/3200985762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4844004810 0x7f4844004c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f484c001860 tx=0x7f484c01eff0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- 172.21.15.97:0/3200985762 shutdown_connections 2024-09-10T21:25:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 --2- 172.21.15.97:0/3200985762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4844004810 0x7f4844004c90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- 172.21.15.97:0/3200985762 >> 172.21.15.97:0/3200985762 conn(0x7f48600d1c30 msgr2=0x7f48600d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- 172.21.15.97:0/3200985762 shutdown_connections 2024-09-10T21:25:10.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- 172.21.15.97:0/3200985762 wait complete. 2024-09-10T21:25:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 Processor -- start 2024-09-10T21:25:10.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- start start 2024-09-10T21:25:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4844004810 0x7f48600c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48600c6170 0x7f4860151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f486006ee60 0x7f48601e8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48600ceba0 con 0x7f48600c6170 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f48600cea20 con 0x7f4844004810 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f4866800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f48600ced20 con 0x7f486006ee60 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48600c6170 0x7f4860151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48600c6170 0x7f4860151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41206/0 (socket says 172.21.15.97:41206) 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 -- 172.21.15.97:0/2760286262 learned_addr learned my addr 172.21.15.97:0/2760286262 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 -- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f486006ee60 msgr2=0x7f48601e8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f486006ee60 0x7f48601e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 -- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4844004810 msgr2=0x7f48600c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4844004810 0x7f48600c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.868+0000 7f485ea00640 1 -- 172.21.15.97:0/2760286262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48600c6670 con 0x7f48600c6170 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f485ea00640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48600c6170 0x7f4860151d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f484c005040 tx=0x7f484c001580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4857400640 1 -- 172.21.15.97:0/2760286262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0a8bc0 con 0x7f48600c6170 2024-09-10T21:25:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4857400640 1 -- 172.21.15.97:0/2760286262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f484c0b7020 con 0x7f48600c6170 2024-09-10T21:25:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48600c6950 con 0x7f48600c6170 2024-09-10T21:25:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4857400640 1 -- 172.21.15.97:0/2760286262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c003990 con 0x7f48600c6170 2024-09-10T21:25:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48600c6f10 con 0x7f48600c6170 2024-09-10T21:25:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48600aef30 con 0x7f48600c6170 2024-09-10T21:25:10.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4857400640 1 -- 172.21.15.97:0/2760286262 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f484c003b30 con 0x7f48600c6170 2024-09-10T21:25:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4857400640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4840072610 0x7f4840074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.872+0000 7f4857400640 1 -- 172.21.15.97:0/2760286262 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f484c130340 con 0x7f48600c6170 2024-09-10T21:25:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.876+0000 7f485f400640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4840072610 0x7f4840074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.876+0000 7f485f400640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4840072610 0x7f4840074ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4850002240 tx=0x7f4850001590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:10.876+0000 7f4857400640 1 -- 172.21.15.97:0/2760286262 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f484c0c1070 con 0x7f48600c6170 2024-09-10T21:25:11.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.048+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f486015dad0 con 0x7f48600c6170 2024-09-10T21:25:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.048+0000 7f4857400640 1 -- 172.21.15.97:0/2760286262 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17039 (secure 0 0 0) 0x7f484c018da0 con 0x7f48600c6170 2024-09-10T21:25:11.056 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:11.056 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":39,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:25:09.683509+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:25:07.800828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T21:25:08.646331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4840072610 msgr2=0x7f4840074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4840072610 0x7f4840074ad0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4850002240 tx=0x7f4850001590 comp rx=0 tx=0).stop 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48600c6170 msgr2=0x7f4860151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48600c6170 0x7f4860151d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f484c005040 tx=0x7f484c001580 comp rx=0 tx=0).stop 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 shutdown_connections 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4840072610 0x7f4840074ad0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f486006ee60 0x7f48601e8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:11.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48600c6170 0x7f4860151d90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 --2- 172.21.15.97:0/2760286262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4844004810 0x7f48600c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 >> 172.21.15.97:0/2760286262 conn(0x7f48600d1c30 msgr2=0x7f486015d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 shutdown_connections 2024-09-10T21:25:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.052+0000 7f4866800640 1 -- 172.21.15.97:0/2760286262 wait complete. 2024-09-10T21:25:11.078 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:25:11.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 -- 172.21.15.97:0/3896098941 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80b2570 msgr2=0x7f08d8166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 --2- 172.21.15.97:0/3896098941 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80b2570 0x7f08d8166fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009870 tx=0x7f08d001f050 comp rx=0 tx=0).stop 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 -- 172.21.15.97:0/3896098941 shutdown_connections 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 --2- 172.21.15.97:0/3896098941 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80b2570 0x7f08d8166fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 -- 172.21.15.97:0/3896098941 >> 172.21.15.97:0/3896098941 conn(0x7f08d80b37c0 msgr2=0x7f08d80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 -- 172.21.15.97:0/3896098941 shutdown_connections 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 -- 172.21.15.97:0/3896098941 wait complete. 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 Processor -- start 2024-09-10T21:25:11.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.216+0000 7f08ddc00640 1 -- start start 2024-09-10T21:25:11.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08d80b2570 0x7f08d814bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:11.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80ce830 0x7f08d80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:11.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cec10 0x7f08d8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08d80b4470 con 0x7f08d80cec10 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08d80cde60 con 0x7f08d80b2570 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f08d80ce1a0 con 0x7f08d80ce830 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cec10 0x7f08d8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cec10 0x7f08d8155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41210/0 (socket says 172.21.15.97:41210) 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 -- 172.21.15.97:0/2805417251 learned_addr learned my addr 172.21.15.97:0/2805417251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d6a00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08d80b2570 0x7f08d814bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d6000640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80ce830 0x7f08d80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 -- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80ce830 msgr2=0x7f08d80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80ce830 0x7f08d80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 -- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08d80b2570 msgr2=0x7f08d814bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08d80b2570 0x7f08d814bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 -- 172.21.15.97:0/2805417251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d80c8680 con 0x7f08d80cec10 2024-09-10T21:25:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d6a00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08d80b2570 0x7f08d814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08d7400640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cec10 0x7f08d8155d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08d00b3040 tx=0x7f08d0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08cea00640 1 -- 172.21.15.97:0/2805417251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d00be070 con 0x7f08d80cec10 2024-09-10T21:25:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08cea00640 1 -- 172.21.15.97:0/2805417251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08d0005320 con 0x7f08d80cec10 2024-09-10T21:25:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08cea00640 1 -- 172.21.15.97:0/2805417251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d00b5040 con 0x7f08d80cec10 2024-09-10T21:25:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d8139660 con 0x7f08d80cec10 2024-09-10T21:25:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d8139c20 con 0x7f08d80cec10 2024-09-10T21:25:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d80cf800 con 0x7f08d80cec10 2024-09-10T21:25:11.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.220+0000 7f08cea00640 1 -- 172.21.15.97:0/2805417251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f08d00b8070 con 0x7f08d80cec10 2024-09-10T21:25:11.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.224+0000 7f08cea00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f08b80725e0 0x7f08b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:11.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.224+0000 7f08cea00640 1 -- 172.21.15.97:0/2805417251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f08d01303d0 con 0x7f08d80cec10 2024-09-10T21:25:11.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.224+0000 7f08d6a00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f08b80725e0 0x7f08b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:11.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.224+0000 7f08d6a00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f08b80725e0 0x7f08b8074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f08c8007fd0 tx=0x7f08c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:11.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.228+0000 7f08cea00640 1 -- 172.21.15.97:0/2805417251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f08d0018bb0 con 0x7f08d80cec10 2024-09-10T21:25:11.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:11.432+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f08d8167300 con 0x7f08d80cec10 2024-09-10T21:25:12.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08cea00640 1 -- 172.21.15.97:0/2805417251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f08d00fb880 con 0x7f08d80cec10 2024-09-10T21:25:12.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f08b80725e0 msgr2=0x7f08b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f08b80725e0 0x7f08b8074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f08c8007fd0 tx=0x7f08c8009040 comp rx=0 tx=0).stop 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cec10 msgr2=0x7f08d8155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cec10 0x7f08d8155d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08d00b3040 tx=0x7f08d0004600 comp rx=0 tx=0).stop 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 shutdown_connections 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f08b80725e0 0x7f08b8074aa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cec10 0x7f08d8155d90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f08d80ce830 0x7f08d80c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 --2- 172.21.15.97:0/2805417251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08d80b2570 0x7f08d814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 >> 172.21.15.97:0/2805417251 conn(0x7f08d80b37c0 msgr2=0x7f08d80aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 shutdown_connections 2024-09-10T21:25:12.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.312+0000 7f08ddc00640 1 -- 172.21.15.97:0/2805417251 wait complete. 2024-09-10T21:25:12.339 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:25:12.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- 172.21.15.97:0/565779047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 msgr2=0x7f76d80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 --2- 172.21.15.97:0/565779047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80cc6d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76d80c55b0 tx=0x7f76c401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- 172.21.15.97:0/565779047 shutdown_connections 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 --2- 172.21.15.97:0/565779047 >> v2:172.21.15.194:3300/0 conn(0x7f76d80b0d50 0x7f76d80ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 --2- 172.21.15.97:0/565779047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80cc6d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- 172.21.15.97:0/565779047 >> 172.21.15.97:0/565779047 conn(0x7f76d80d1c30 msgr2=0x7f76d80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- 172.21.15.97:0/565779047 shutdown_connections 2024-09-10T21:25:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- 172.21.15.97:0/565779047 wait complete. 2024-09-10T21:25:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 Processor -- start 2024-09-10T21:25:12.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- start start 2024-09-10T21:25:12.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76d80b0d50 0x7f76d8071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80ce640 0x7f76d81f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8168ee0 con 0x7f76d80afdd0 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8168d60 con 0x7f76d80ce640 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76e0400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f76d8169060 con 0x7f76d80b0d50 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76dda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76dda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41222/0 (socket says 172.21.15.97:41222) 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.480+0000 7f76dda00640 1 -- 172.21.15.97:0/4144441074 learned_addr learned my addr 172.21.15.97:0/4144441074 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76dd000640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76d80b0d50 0x7f76d8071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76de400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80ce640 0x7f76d81f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76dda00640 1 -- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76d80b0d50 msgr2=0x7f76d8071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76dda00640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76d80b0d50 0x7f76d8071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76dda00640 1 -- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80ce640 msgr2=0x7f76d81f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76dda00640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80ce640 0x7f76d81f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76dda00640 1 -- 172.21.15.97:0/4144441074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d81f61c0 con 0x7f76d80afdd0 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76de400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80ce640 0x7f76d81f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76dda00640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80c96a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c3c0 tx=0x7f76cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76d5600640 1 -- 172.21.15.97:0/4144441074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc017070 con 0x7f76d80afdd0 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76d5600640 1 -- 172.21.15.97:0/4144441074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76cc0053c0 con 0x7f76d80afdd0 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76d5600640 1 -- 172.21.15.97:0/4144441074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc003860 con 0x7f76d80afdd0 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d81f64a0 con 0x7f76d80afdd0 2024-09-10T21:25:12.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d80c9180 con 0x7f76d80afdd0 2024-09-10T21:25:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d80cea20 con 0x7f76d80afdd0 2024-09-10T21:25:12.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76d5600640 1 -- 172.21.15.97:0/4144441074 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76cc003a00 con 0x7f76d80afdd0 2024-09-10T21:25:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.484+0000 7f76d5600640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76b80725e0 0x7f76b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.488+0000 7f76d5600640 1 -- 172.21.15.97:0/4144441074 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f76cc092960 con 0x7f76d80afdd0 2024-09-10T21:25:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.488+0000 7f76dd000640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76b80725e0 0x7f76b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.488+0000 7f76dd000640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76b80725e0 0x7f76b8074aa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76c40af000 tx=0x7f76c401b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:12.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.488+0000 7f76d5600640 1 -- 172.21.15.97:0/4144441074 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f76cc05d850 con 0x7f76d80afdd0 2024-09-10T21:25:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:12.688+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f76d81f6670 con 0x7f76d80afdd0 2024-09-10T21:25:13.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76d5600640 1 -- 172.21.15.97:0/4144441074 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f76cc062280 con 0x7f76d80afdd0 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76b80725e0 msgr2=0x7f76b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76b80725e0 0x7f76b8074aa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76c40af000 tx=0x7f76c401b100 comp rx=0 tx=0).stop 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 msgr2=0x7f76d80c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80c96a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00c3c0 tx=0x7f76cc00c880 comp rx=0 tx=0).stop 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 shutdown_connections 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76b80725e0 0x7f76b8074aa0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80ce640 0x7f76d81f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76d80b0d50 0x7f76d8071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 --2- 172.21.15.97:0/4144441074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76d80afdd0 0x7f76d80c96a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.320+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 >> 172.21.15.97:0/4144441074 conn(0x7f76d80d1c30 msgr2=0x7f76d80b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.324+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 shutdown_connections 2024-09-10T21:25:13.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.324+0000 7f76e0400640 1 -- 172.21.15.97:0/4144441074 wait complete. 2024-09-10T21:25:13.349 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:25:13.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:13.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.484+0000 7f4431e00640 1 -- 172.21.15.97:0/448927247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410003a10 msgr2=0x7f4410003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.484+0000 7f4431e00640 1 --2- 172.21.15.97:0/448927247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410003a10 0x7f4410003e30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f441c009ee0 tx=0x7f441c01d630 comp rx=0 tx=0).stop 2024-09-10T21:25:13.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.484+0000 7f442b400640 1 -- 172.21.15.97:0/448927247 reap_dead start 2024-09-10T21:25:13.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.484+0000 7f4431e00640 1 -- 172.21.15.97:0/448927247 shutdown_connections 2024-09-10T21:25:13.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.484+0000 7f4431e00640 1 -- 172.21.15.97:0/448927247 >> 172.21.15.97:0/448927247 conn(0x7f442c0d1c30 msgr2=0x7f442c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:13.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.484+0000 7f4431e00640 1 -- 172.21.15.97:0/448927247 shutdown_connections 2024-09-10T21:25:13.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.484+0000 7f4431e00640 1 -- 172.21.15.97:0/448927247 wait complete. 2024-09-10T21:25:13.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 Processor -- start 2024-09-10T21:25:13.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 -- start start 2024-09-10T21:25:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4410003a10 0x7f442c1fc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442c15a2b0 0x7f442c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f442c1f7c00 0x7f442c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1670e0 con 0x7f442c15a2b0 2024-09-10T21:25:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0b00d0 con 0x7f442c1f7c00 2024-09-10T21:25:13.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f442c167260 con 0x7f4410003a10 2024-09-10T21:25:13.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442c15a2b0 0x7f442c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442c15a2b0 0x7f442c158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41252/0 (socket says 172.21.15.97:41252) 2024-09-10T21:25:13.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 -- 172.21.15.97:0/2911899660 learned_addr learned my addr 172.21.15.97:0/2911899660 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:13.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f442b400640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f442c1f7c00 0x7f442c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f442aa00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4410003a10 0x7f442c1fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 -- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4410003a10 msgr2=0x7f442c1fc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4410003a10 0x7f442c1fc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 -- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f442c1f7c00 msgr2=0x7f442c0b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f442c1f7c00 0x7f442c0b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 -- 172.21.15.97:0/2911899660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c1f8670 con 0x7f442c15a2b0 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f442aa00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4410003a10 0x7f442c1fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f442b400640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f442c1f7c00 0x7f442c0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4422000640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442c15a2b0 0x7f442c158b70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4414011ab0 tx=0x7f4414011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4423e00640 1 -- 172.21.15.97:0/2911899660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441401e070 con 0x7f442c15a2b0 2024-09-10T21:25:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4423e00640 1 -- 172.21.15.97:0/2911899660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f441400fc70 con 0x7f442c15a2b0 2024-09-10T21:25:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4423e00640 1 -- 172.21.15.97:0/2911899660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414019670 con 0x7f442c15a2b0 2024-09-10T21:25:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c1f8950 con 0x7f442c15a2b0 2024-09-10T21:25:13.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.488+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1f8f10 con 0x7f442c15a2b0 2024-09-10T21:25:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.492+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c0cea20 con 0x7f442c15a2b0 2024-09-10T21:25:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.492+0000 7f4423e00640 1 -- 172.21.15.97:0/2911899660 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4414019810 con 0x7f442c15a2b0 2024-09-10T21:25:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.492+0000 7f4423e00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4408072610 0x7f4408074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.492+0000 7f4423e00640 1 -- 172.21.15.97:0/2911899660 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4414093f50 con 0x7f442c15a2b0 2024-09-10T21:25:13.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.492+0000 7f442aa00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4408072610 0x7f4408074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.496+0000 7f442aa00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4408072610 0x7f4408074ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f441c0af000 tx=0x7f441c01a3d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:13.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.496+0000 7f4423e00640 1 -- 172.21.15.97:0/2911899660 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f441400fe60 con 0x7f442c15a2b0 2024-09-10T21:25:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.692+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f442c1590b0 con 0x7f442c15a2b0 2024-09-10T21:25:13.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4423e00640 1 -- 172.21.15.97:0/2911899660 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f441405edb0 con 0x7f442c15a2b0 2024-09-10T21:25:13.702 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:13.702 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"btime":"2024-09-10T21:25:13:316162+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:12.695121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/2127253870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2127253870},{"type":"v1","addr":"172.21.15.97:6835","nonce":2127253870}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4932,"qdb_cluster":[4932]},"id":2}]} 2024-09-10T21:25:13.703 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-09-10T21:25:13.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4408072610 msgr2=0x7f4408074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4408072610 0x7f4408074ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f441c0af000 tx=0x7f441c01a3d0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442c15a2b0 msgr2=0x7f442c158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442c15a2b0 0x7f442c158b70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4414011ab0 tx=0x7f4414011f70 comp rx=0 tx=0).stop 2024-09-10T21:25:13.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 shutdown_connections 2024-09-10T21:25:13.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4408072610 0x7f4408074ad0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f442c1f7c00 0x7f442c0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442c15a2b0 0x7f442c158b70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 --2- 172.21.15.97:0/2911899660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4410003a10 0x7f442c1fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 >> 172.21.15.97:0/2911899660 conn(0x7f442c0d1c30 msgr2=0x7f442c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:13.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 shutdown_connections 2024-09-10T21:25:13.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.696+0000 7f4431e00640 1 -- 172.21.15.97:0/2911899660 wait complete. 2024-09-10T21:25:13.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- 172.21.15.97:0/585208331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af815fde0 msgr2=0x7f8af8166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 --2- 172.21.15.97:0/585208331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af815fde0 0x7f8af8166850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009bc0 tx=0x7f8af001cba0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- 172.21.15.97:0/585208331 shutdown_connections 2024-09-10T21:25:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 --2- 172.21.15.97:0/585208331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af815fde0 0x7f8af8166850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- 172.21.15.97:0/585208331 >> 172.21.15.97:0/585208331 conn(0x7f8af80b1780 msgr2=0x7f8af80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- 172.21.15.97:0/585208331 shutdown_connections 2024-09-10T21:25:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- 172.21.15.97:0/585208331 wait complete. 2024-09-10T21:25:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 Processor -- start 2024-09-10T21:25:13.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- start start 2024-09-10T21:25:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af80ce5f0 0x7f8af813b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af8141d20 0x7f8af81f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8af815fde0 0x7f8af80b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8af816a130 con 0x7f8af8141d20 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8169fb0 con 0x7f8af80ce5f0 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8b00600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8af816a2b0 con 0x7f8af815fde0 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af8141d20 0x7f8af81f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af8141d20 0x7f8af81f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41264/0 (socket says 172.21.15.97:41264) 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 -- 172.21.15.97:0/2115834050 learned_addr learned my addr 172.21.15.97:0/2115834050 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afe600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8af815fde0 0x7f8af80b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afdc00640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af80ce5f0 0x7f8af813b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 -- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8af815fde0 msgr2=0x7f8af80b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8af815fde0 0x7f8af80b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 -- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af80ce5f0 msgr2=0x7f8af813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af80ce5f0 0x7f8af813b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 -- 172.21.15.97:0/2115834050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af813be60 con 0x7f8af8141d20 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afe600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8af815fde0 0x7f8af80b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afdc00640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af80ce5f0 0x7f8af813b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.860+0000 7f8afd200640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af8141d20 0x7f8af81f8440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c4b0 tx=0x7f8ae400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8af5600640 1 -- 172.21.15.97:0/2115834050 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40052d0 con 0x7f8af8141d20 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8af5600640 1 -- 172.21.15.97:0/2115834050 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ae4005470 con 0x7f8af8141d20 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8af5600640 1 -- 172.21.15.97:0/2115834050 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4012870 con 0x7f8af8141d20 2024-09-10T21:25:13.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af81f8980 con 0x7f8af8141d20 2024-09-10T21:25:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af8164f00 con 0x7f8af8141d20 2024-09-10T21:25:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af80ce9d0 con 0x7f8af8141d20 2024-09-10T21:25:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8af5600640 1 -- 172.21.15.97:0/2115834050 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ae4003600 con 0x7f8af8141d20 2024-09-10T21:25:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8af5600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ad80725e0 0x7f8ad8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8af5600640 1 -- 172.21.15.97:0/2115834050 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f8ae4092ff0 con 0x7f8af8141d20 2024-09-10T21:25:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.864+0000 7f8afdc00640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ad80725e0 0x7f8ad8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.868+0000 7f8afdc00640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ad80725e0 0x7f8ad8074aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000f70 tx=0x7f8aec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:13.868+0000 7f8af5600640 1 -- 172.21.15.97:0/2115834050 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8af813be60 con 0x7f8af8141d20 2024-09-10T21:25:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.036+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8af8141360 con 0x7f8af8141d20 2024-09-10T21:25:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8af5600640 1 -- 172.21.15.97:0/2115834050 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17026 (secure 0 0 0) 0x7f8ae405e4a0 con 0x7f8af8141d20 2024-09-10T21:25:14.046 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:14.046 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":40,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:25:11.301879+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:25:07.800828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T21:25:08.646331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ad80725e0 msgr2=0x7f8ad8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ad80725e0 0x7f8ad8074aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8aec000f70 tx=0x7f8aec009040 comp rx=0 tx=0).stop 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af8141d20 msgr2=0x7f8af81f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af8141d20 0x7f8af81f8440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c4b0 tx=0x7f8ae400c970 comp rx=0 tx=0).stop 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 shutdown_connections 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ad80725e0 0x7f8ad8074aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8af815fde0 0x7f8af80b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8af8141d20 0x7f8af81f8440 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 --2- 172.21.15.97:0/2115834050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8af80ce5f0 0x7f8af813b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 >> 172.21.15.97:0/2115834050 conn(0x7f8af80b1780 msgr2=0x7f8af806eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 shutdown_connections 2024-09-10T21:25:14.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.040+0000 7f8b00600640 1 -- 172.21.15.97:0/2115834050 wait complete. 2024-09-10T21:25:14.066 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:25:14.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 -- 172.21.15.97:0/699324708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 msgr2=0x7f3f280d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/699324708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280d0370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3f28073590 tx=0x7f3f2001ccf0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 -- 172.21.15.97:0/699324708 shutdown_connections 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/699324708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280d0370 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/699324708 >> v2:172.21.15.194:3300/0 conn(0x7f3f2806dc50 0x7f3f280cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/699324708 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f28161600 0x7f3f280d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 -- 172.21.15.97:0/699324708 >> 172.21.15.97:0/699324708 conn(0x7f3f280d1c30 msgr2=0x7f3f280d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 -- 172.21.15.97:0/699324708 shutdown_connections 2024-09-10T21:25:14.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.200+0000 7f3f2ee00640 1 -- 172.21.15.97:0/699324708 wait complete. 2024-09-10T21:25:14.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 Processor -- start 2024-09-10T21:25:14.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 -- start start 2024-09-10T21:25:14.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f28161600 0x7f3f280b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f281564b0 0x7f3f2815fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f280afd10 con 0x7f3f280b1590 2024-09-10T21:25:14.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3f280afb90 con 0x7f3f281564b0 2024-09-10T21:25:14.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3f280afe90 con 0x7f3f28161600 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41274/0 (socket says 172.21.15.97:41274) 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 -- 172.21.15.97:0/3712597692 learned_addr learned my addr 172.21.15.97:0/3712597692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27400640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f28161600 0x7f3f280b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ce00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f281564b0 0x7f3f2815fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 -- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f28161600 msgr2=0x7f3f280b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f28161600 0x7f3f280b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 -- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f281564b0 msgr2=0x7f3f2815fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f281564b0 0x7f3f2815fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 -- 172.21.15.97:0/3712597692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f281f7d40 con 0x7f3f280b1590 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f27e00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280c82b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800d720 tx=0x7f3f1800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:14.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f1fe00640 1 -- 172.21.15.97:0/3712597692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18017070 con 0x7f3f280b1590 2024-09-10T21:25:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f1fe00640 1 -- 172.21.15.97:0/3712597692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f18009d20 con 0x7f3f280b1590 2024-09-10T21:25:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f281f8020 con 0x7f3f280b1590 2024-09-10T21:25:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f1fe00640 1 -- 172.21.15.97:0/3712597692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18005280 con 0x7f3f280b1590 2024-09-10T21:25:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f281f85e0 con 0x7f3f280b1590 2024-09-10T21:25:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.204+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f281f8820 con 0x7f3f280b1590 2024-09-10T21:25:14.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.208+0000 7f3f1fe00640 1 -- 172.21.15.97:0/3712597692 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3f18005420 con 0x7f3f280b1590 2024-09-10T21:25:14.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.208+0000 7f3f1fe00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f08072610 0x7f3f08074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.208+0000 7f3f1fe00640 1 -- 172.21.15.97:0/3712597692 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3f18093530 con 0x7f3f280b1590 2024-09-10T21:25:14.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.208+0000 7f3f27400640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f08072610 0x7f3f08074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.208+0000 7f3f27400640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f08072610 0x7f3f08074ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3f2815bf80 tx=0x7f3f14005e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:14.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.212+0000 7f3f1fe00640 1 -- 172.21.15.97:0/3712597692 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3f1805e9e0 con 0x7f3f280b1590 2024-09-10T21:25:14.272 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:25:14.264+0000 7f46bca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:25:14.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.380+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f280d0a60 con 0x7f3f08072610 2024-09-10T21:25:14.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.384+0000 7f3f1fe00640 1 -- 172.21.15.97:0/3712597692 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f3f280d0a60 con 0x7f3f08072610 2024-09-10T21:25:14.393 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:14.395 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f08072610 msgr2=0x7f3f08074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f08072610 0x7f3f08074ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3f2815bf80 tx=0x7f3f14005e60 comp rx=0 tx=0).stop 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 msgr2=0x7f3f280c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280c82b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800d720 tx=0x7f3f1800dbe0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 shutdown_connections 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f08072610 0x7f3f08074ad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f281564b0 0x7f3f2815fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f28161600 0x7f3f280b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 --2- 172.21.15.97:0/3712597692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f280b1590 0x7f3f280c82b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 >> 172.21.15.97:0/3712597692 conn(0x7f3f280d1c30 msgr2=0x7f3f280b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 shutdown_connections 2024-09-10T21:25:14.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.388+0000 7f3f2ee00640 1 -- 172.21.15.97:0/3712597692 wait complete. 2024-09-10T21:25:14.414 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-10T21:25:14.084398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220952,"kb_used_data":4692,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753768,"statfs":{"total":773094113280,"available":772867858432,"internally_reserved":0,"allocated":4804608,"data_stored":2696968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805430"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314720+0000","last_change":"2024-09-10T21:25:10.313445+0000","last_active":"2024-09-10T21:25:11.314720+0000","last_peered":"2024-09-10T21:25:11.314720+0000","last_clean":"2024-09-10T21:25:11.314720+0000","last_became_active":"2024-09-10T21:25:10.313186+0000","last_became_peered":"2024-09-10T21:25:10.313186+0000","last_unstale":"2024-09-10T21:25:11.314720+0000","last_undegraded":"2024-09-10T21:25:11.314720+0000","last_fullsized":"2024-09-10T21:25:11.314720+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:09.330807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.314930+0000","last_change":"2024-09-10T21:25:10.308813+0000","last_active":"2024-09-10T21:25:10.314930+0000","last_peered":"2024-09-10T21:25:10.314930+0000","last_clean":"2024-09-10T21:25:10.314930+0000","last_became_active":"2024-09-10T21:25:10.308558+0000","last_became_peered":"2024-09-10T21:25:10.308558+0000","last_unstale":"2024-09-10T21:25:10.314930+0000","last_undegraded":"2024-09-10T21:25:10.314930+0000","last_fullsized":"2024-09-10T21:25:10.314930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.051223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313545+0000","last_change":"2024-09-10T21:25:10.322846+0000","last_active":"2024-09-10T21:25:11.313545+0000","last_peered":"2024-09-10T21:25:11.313545+0000","last_clean":"2024-09-10T21:25:11.313545+0000","last_became_active":"2024-09-10T21:25:10.322475+0000","last_became_peered":"2024-09-10T21:25:10.322475+0000","last_unstale":"2024-09-10T21:25:11.313545+0000","last_undegraded":"2024-09-10T21:25:11.313545+0000","last_fullsized":"2024-09-10T21:25:11.313545+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:02.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313174+0000","last_change":"2024-09-10T21:25:10.315205+0000","last_active":"2024-09-10T21:25:11.313174+0000","last_peered":"2024-09-10T21:25:11.313174+0000","last_clean":"2024-09-10T21:25:11.313174+0000","last_became_active":"2024-09-10T21:25:10.314777+0000","last_became_peered":"2024-09-10T21:25:10.314777+0000","last_unstale":"2024-09-10T21:25:11.313174+0000","last_undegraded":"2024-09-10T21:25:11.313174+0000","last_fullsized":"2024-09-10T21:25:11.313174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:57.665201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319200+0000","last_change":"2024-09-10T21:25:10.317144+0000","last_active":"2024-09-10T21:25:10.319200+0000","last_peered":"2024-09-10T21:25:10.319200+0000","last_clean":"2024-09-10T21:25:10.319200+0000","last_became_active":"2024-09-10T21:25:10.312332+0000","last_became_peered":"2024-09-10T21:25:10.312332+0000","last_unstale":"2024-09-10T21:25:10.319200+0000","last_undegraded":"2024-09-10T21:25:10.319200+0000","last_fullsized":"2024-09-10T21:25:10.319200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:31.705951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.315099+0000","last_change":"2024-09-10T21:25:10.311396+0000","last_active":"2024-09-10T21:25:10.315099+0000","last_peered":"2024-09-10T21:25:10.315099+0000","last_clean":"2024-09-10T21:25:10.315099+0000","last_became_active":"2024-09-10T21:25:10.311002+0000","last_became_peered":"2024-09-10T21:25:10.311002+0000","last_unstale":"2024-09-10T21:25:10.315099+0000","last_undegraded":"2024-09-10T21:25:10.315099+0000","last_fullsized":"2024-09-10T21:25:10.315099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:01.124483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319251+0000","last_change":"2024-09-10T21:25:10.313464+0000","last_active":"2024-09-10T21:25:10.319251+0000","last_peered":"2024-09-10T21:25:10.319251+0000","last_clean":"2024-09-10T21:25:10.319251+0000","last_became_active":"2024-09-10T21:25:10.313179+0000","last_became_peered":"2024-09-10T21:25:10.313179+0000","last_unstale":"2024-09-10T21:25:10.319251+0000","last_undegraded":"2024-09-10T21:25:10.319251+0000","last_fullsized":"2024-09-10T21:25:10.319251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:33.104694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312416+0000","last_change":"2024-09-10T21:25:10.323012+0000","last_active":"2024-09-10T21:25:11.312416+0000","last_peered":"2024-09-10T21:25:11.312416+0000","last_clean":"2024-09-10T21:25:11.312416+0000","last_became_active":"2024-09-10T21:25:10.322713+0000","last_became_peered":"2024-09-10T21:25:10.322713+0000","last_unstale":"2024-09-10T21:25:11.312416+0000","last_undegraded":"2024-09-10T21:25:11.312416+0000","last_fullsized":"2024-09-10T21:25:11.312416+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:52.771526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.303958+0000","last_change":"2024-09-10T21:25:10.298663+0000","last_active":"2024-09-10T21:25:10.303958+0000","last_peered":"2024-09-10T21:25:10.303958+0000","last_clean":"2024-09-10T21:25:10.303958+0000","last_became_active":"2024-09-10T21:25:10.298254+0000","last_became_peered":"2024-09-10T21:25:10.298254+0000","last_unstale":"2024-09-10T21:25:10.303958+0000","last_undegraded":"2024-09-10T21:25:10.303958+0000","last_fullsized":"2024-09-10T21:25:10.303958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:21.046869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.331700+0000","last_change":"2024-09-10T21:25:10.321572+0000","last_active":"2024-09-10T21:25:10.331700+0000","last_peered":"2024-09-10T21:25:10.331700+0000","last_clean":"2024-09-10T21:25:10.331700+0000","last_became_active":"2024-09-10T21:25:10.321241+0000","last_became_peered":"2024-09-10T21:25:10.321241+0000","last_unstale":"2024-09-10T21:25:10.331700+0000","last_undegraded":"2024-09-10T21:25:10.331700+0000","last_fullsized":"2024-09-10T21:25:10.331700+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.783496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325131+0000","last_change":"2024-09-10T21:25:10.318363+0000","last_active":"2024-09-10T21:25:10.325131+0000","last_peered":"2024-09-10T21:25:10.325131+0000","last_clean":"2024-09-10T21:25:10.325131+0000","last_became_active":"2024-09-10T21:25:10.318063+0000","last_became_peered":"2024-09-10T21:25:10.318063+0000","last_unstale":"2024-09-10T21:25:10.325131+0000","last_undegraded":"2024-09-10T21:25:10.325131+0000","last_fullsized":"2024-09-10T21:25:10.325131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:29.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311953+0000","last_change":"2024-09-10T21:25:10.304998+0000","last_active":"2024-09-10T21:25:10.311953+0000","last_peered":"2024-09-10T21:25:10.311953+0000","last_clean":"2024-09-10T21:25:10.311953+0000","last_became_active":"2024-09-10T21:25:10.304727+0000","last_became_peered":"2024-09-10T21:25:10.304727+0000","last_unstale":"2024-09-10T21:25:10.311953+0000","last_undegraded":"2024-09-10T21:25:10.311953+0000","last_fullsized":"2024-09-10T21:25:10.311953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.315394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328564+0000","last_change":"2024-09-10T21:25:10.317776+0000","last_active":"2024-09-10T21:25:10.328564+0000","last_peered":"2024-09-10T21:25:10.328564+0000","last_clean":"2024-09-10T21:25:10.328564+0000","last_became_active":"2024-09-10T21:25:10.317377+0000","last_became_peered":"2024-09-10T21:25:10.317377+0000","last_unstale":"2024-09-10T21:25:10.328564+0000","last_undegraded":"2024-09-10T21:25:10.328564+0000","last_fullsized":"2024-09-10T21:25:10.328564+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:36.060550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.317262+0000","last_change":"2024-09-10T21:25:10.310314+0000","last_active":"2024-09-10T21:25:10.317262+0000","last_peered":"2024-09-10T21:25:10.317262+0000","last_clean":"2024-09-10T21:25:10.317262+0000","last_became_active":"2024-09-10T21:25:10.310035+0000","last_became_peered":"2024-09-10T21:25:10.310035+0000","last_unstale":"2024-09-10T21:25:10.317262+0000","last_undegraded":"2024-09-10T21:25:10.317262+0000","last_fullsized":"2024-09-10T21:25:10.317262+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:22.444236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.317934+0000","last_change":"2024-09-10T21:25:10.306305+0000","last_active":"2024-09-10T21:25:10.317934+0000","last_peered":"2024-09-10T21:25:10.317934+0000","last_clean":"2024-09-10T21:25:10.317934+0000","last_became_active":"2024-09-10T21:25:10.305939+0000","last_became_peered":"2024-09-10T21:25:10.305939+0000","last_unstale":"2024-09-10T21:25:10.317934+0000","last_undegraded":"2024-09-10T21:25:10.317934+0000","last_fullsized":"2024-09-10T21:25:10.317934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:10.375583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328602+0000","last_change":"2024-09-10T21:25:10.318444+0000","last_active":"2024-09-10T21:25:10.328602+0000","last_peered":"2024-09-10T21:25:10.328602+0000","last_clean":"2024-09-10T21:25:10.328602+0000","last_became_active":"2024-09-10T21:25:10.318208+0000","last_became_peered":"2024-09-10T21:25:10.318208+0000","last_unstale":"2024-09-10T21:25:10.328602+0000","last_undegraded":"2024-09-10T21:25:10.328602+0000","last_fullsized":"2024-09-10T21:25:10.328602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:06.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335257+0000","last_change":"2024-09-10T21:25:10.330124+0000","last_active":"2024-09-10T21:25:10.335257+0000","last_peered":"2024-09-10T21:25:10.335257+0000","last_clean":"2024-09-10T21:25:10.335257+0000","last_became_active":"2024-09-10T21:25:10.329878+0000","last_became_peered":"2024-09-10T21:25:10.329878+0000","last_unstale":"2024-09-10T21:25:10.335257+0000","last_undegraded":"2024-09-10T21:25:10.335257+0000","last_fullsized":"2024-09-10T21:25:10.335257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:52.357679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335699+0000","last_change":"2024-09-10T21:25:10.326748+0000","last_active":"2024-09-10T21:25:10.335699+0000","last_peered":"2024-09-10T21:25:10.335699+0000","last_clean":"2024-09-10T21:25:10.335699+0000","last_became_active":"2024-09-10T21:25:10.326350+0000","last_became_peered":"2024-09-10T21:25:10.326350+0000","last_unstale":"2024-09-10T21:25:10.335699+0000","last_undegraded":"2024-09-10T21:25:10.335699+0000","last_fullsized":"2024-09-10T21:25:10.335699+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:21.785416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319975+0000","last_change":"2024-09-10T21:25:10.315986+0000","last_active":"2024-09-10T21:25:10.319975+0000","last_peered":"2024-09-10T21:25:10.319975+0000","last_clean":"2024-09-10T21:25:10.319975+0000","last_became_active":"2024-09-10T21:25:10.315762+0000","last_became_peered":"2024-09-10T21:25:10.315762+0000","last_unstale":"2024-09-10T21:25:10.319975+0000","last_undegraded":"2024-09-10T21:25:10.319975+0000","last_fullsized":"2024-09-10T21:25:10.319975+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:32.179380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316483+0000","last_change":"2024-09-10T21:25:10.313229+0000","last_active":"2024-09-10T21:25:10.316483+0000","last_peered":"2024-09-10T21:25:10.316483+0000","last_clean":"2024-09-10T21:25:10.316483+0000","last_became_active":"2024-09-10T21:25:10.312958+0000","last_became_peered":"2024-09-10T21:25:10.312958+0000","last_unstale":"2024-09-10T21:25:10.316483+0000","last_undegraded":"2024-09-10T21:25:10.316483+0000","last_fullsized":"2024-09-10T21:25:10.316483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.863854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325007+0000","last_change":"2024-09-10T21:25:10.316928+0000","last_active":"2024-09-10T21:25:10.325007+0000","last_peered":"2024-09-10T21:25:10.325007+0000","last_clean":"2024-09-10T21:25:10.325007+0000","last_became_active":"2024-09-10T21:25:10.316618+0000","last_became_peered":"2024-09-10T21:25:10.316618+0000","last_unstale":"2024-09-10T21:25:10.325007+0000","last_undegraded":"2024-09-10T21:25:10.325007+0000","last_fullsized":"2024-09-10T21:25:10.325007+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:32.179380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316277+0000","last_change":"2024-09-10T21:25:10.310232+0000","last_active":"2024-09-10T21:25:10.316277+0000","last_peered":"2024-09-10T21:25:10.316277+0000","last_clean":"2024-09-10T21:25:10.316277+0000","last_became_active":"2024-09-10T21:25:10.309968+0000","last_became_peered":"2024-09-10T21:25:10.309968+0000","last_unstale":"2024-09-10T21:25:10.316277+0000","last_undegraded":"2024-09-10T21:25:10.316277+0000","last_fullsized":"2024-09-10T21:25:10.316277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:00.307962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.327601+0000","last_change":"2024-09-10T21:25:10.311739+0000","last_active":"2024-09-10T21:25:10.327601+0000","last_peered":"2024-09-10T21:25:10.327601+0000","last_clean":"2024-09-10T21:25:10.327601+0000","last_became_active":"2024-09-10T21:25:10.311487+0000","last_became_peered":"2024-09-10T21:25:10.311487+0000","last_unstale":"2024-09-10T21:25:10.327601+0000","last_undegraded":"2024-09-10T21:25:10.327601+0000","last_fullsized":"2024-09-10T21:25:10.327601+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:51.233168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312510+0000","last_change":"2024-09-10T21:25:10.317888+0000","last_active":"2024-09-10T21:25:11.312510+0000","last_peered":"2024-09-10T21:25:11.312510+0000","last_clean":"2024-09-10T21:25:11.312510+0000","last_became_active":"2024-09-10T21:25:10.316980+0000","last_became_peered":"2024-09-10T21:25:10.316980+0000","last_unstale":"2024-09-10T21:25:11.312510+0000","last_undegraded":"2024-09-10T21:25:11.312510+0000","last_fullsized":"2024-09-10T21:25:11.312510+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:03.936673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316410+0000","last_change":"2024-09-10T21:25:10.308826+0000","last_active":"2024-09-10T21:25:10.316410+0000","last_peered":"2024-09-10T21:25:10.316410+0000","last_clean":"2024-09-10T21:25:10.316410+0000","last_became_active":"2024-09-10T21:25:10.308552+0000","last_became_peered":"2024-09-10T21:25:10.308552+0000","last_unstale":"2024-09-10T21:25:10.316410+0000","last_undegraded":"2024-09-10T21:25:10.316410+0000","last_fullsized":"2024-09-10T21:25:10.316410+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:52.571926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314396+0000","last_change":"2024-09-10T21:25:10.311824+0000","last_active":"2024-09-10T21:25:11.314396+0000","last_peered":"2024-09-10T21:25:11.314396+0000","last_clean":"2024-09-10T21:25:11.314396+0000","last_became_active":"2024-09-10T21:25:10.311474+0000","last_became_peered":"2024-09-10T21:25:10.311474+0000","last_unstale":"2024-09-10T21:25:11.314396+0000","last_undegraded":"2024-09-10T21:25:11.314396+0000","last_fullsized":"2024-09-10T21:25:11.314396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:57.435532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325320+0000","last_change":"2024-09-10T21:25:10.320611+0000","last_active":"2024-09-10T21:25:10.325320+0000","last_peered":"2024-09-10T21:25:10.325320+0000","last_clean":"2024-09-10T21:25:10.325320+0000","last_became_active":"2024-09-10T21:25:10.320127+0000","last_became_peered":"2024-09-10T21:25:10.320127+0000","last_unstale":"2024-09-10T21:25:10.325320+0000","last_undegraded":"2024-09-10T21:25:10.325320+0000","last_fullsized":"2024-09-10T21:25:10.325320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:47.904295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.331836+0000","last_change":"2024-09-10T21:25:10.321195+0000","last_active":"2024-09-10T21:25:10.331836+0000","last_peered":"2024-09-10T21:25:10.331836+0000","last_clean":"2024-09-10T21:25:10.331836+0000","last_became_active":"2024-09-10T21:25:10.318727+0000","last_became_peered":"2024-09-10T21:25:10.318727+0000","last_unstale":"2024-09-10T21:25:10.331836+0000","last_undegraded":"2024-09-10T21:25:10.331836+0000","last_fullsized":"2024-09-10T21:25:10.331836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:10.293123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.324722+0000","last_change":"2024-09-10T21:25:10.312714+0000","last_active":"2024-09-10T21:25:10.324722+0000","last_peered":"2024-09-10T21:25:10.324722+0000","last_clean":"2024-09-10T21:25:10.324722+0000","last_became_active":"2024-09-10T21:25:10.312456+0000","last_became_peered":"2024-09-10T21:25:10.312456+0000","last_unstale":"2024-09-10T21:25:10.324722+0000","last_undegraded":"2024-09-10T21:25:10.324722+0000","last_fullsized":"2024-09-10T21:25:10.324722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:22.096627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.329642+0000","last_change":"2024-09-10T21:25:10.311184+0000","last_active":"2024-09-10T21:25:10.329642+0000","last_peered":"2024-09-10T21:25:10.329642+0000","last_clean":"2024-09-10T21:25:10.329642+0000","last_became_active":"2024-09-10T21:25:10.310908+0000","last_became_peered":"2024-09-10T21:25:10.310908+0000","last_unstale":"2024-09-10T21:25:10.329642+0000","last_undegraded":"2024-09-10T21:25:10.329642+0000","last_fullsized":"2024-09-10T21:25:10.329642+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:06.378712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319320+0000","last_change":"2024-09-10T21:25:10.313455+0000","last_active":"2024-09-10T21:25:10.319320+0000","last_peered":"2024-09-10T21:25:10.319320+0000","last_clean":"2024-09-10T21:25:10.319320+0000","last_became_active":"2024-09-10T21:25:10.313127+0000","last_became_peered":"2024-09-10T21:25:10.313127+0000","last_unstale":"2024-09-10T21:25:10.319320+0000","last_undegraded":"2024-09-10T21:25:10.319320+0000","last_fullsized":"2024-09-10T21:25:10.319320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:21.046869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312650+0000","last_change":"2024-09-10T21:25:10.314863+0000","last_active":"2024-09-10T21:25:11.312650+0000","last_peered":"2024-09-10T21:25:11.312650+0000","last_clean":"2024-09-10T21:25:11.312650+0000","last_became_active":"2024-09-10T21:25:10.314408+0000","last_became_peered":"2024-09-10T21:25:10.314408+0000","last_unstale":"2024-09-10T21:25:11.312650+0000","last_undegraded":"2024-09-10T21:25:11.312650+0000","last_fullsized":"2024-09-10T21:25:11.312650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:58.901169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.312090+0000","last_change":"2024-09-10T21:25:10.306524+0000","last_active":"2024-09-10T21:25:10.312090+0000","last_peered":"2024-09-10T21:25:10.312090+0000","last_clean":"2024-09-10T21:25:10.312090+0000","last_became_active":"2024-09-10T21:25:10.306215+0000","last_became_peered":"2024-09-10T21:25:10.306215+0000","last_unstale":"2024-09-10T21:25:10.312090+0000","last_undegraded":"2024-09-10T21:25:10.312090+0000","last_fullsized":"2024-09-10T21:25:10.312090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:53.589163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.336665+0000","last_change":"2024-09-10T21:25:09.294230+0000","last_active":"2024-09-10T21:25:10.336665+0000","last_peered":"2024-09-10T21:25:10.336665+0000","last_clean":"2024-09-10T21:25:10.336665+0000","last_became_active":"2024-09-10T21:25:09.293956+0000","last_became_peered":"2024-09-10T21:25:09.293956+0000","last_unstale":"2024-09-10T21:25:10.336665+0000","last_undegraded":"2024-09-10T21:25:10.336665+0000","last_fullsized":"2024-09-10T21:25:10.336665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:20.692813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.322806+0000","last_change":"2024-09-10T21:25:10.319177+0000","last_active":"2024-09-10T21:25:10.322806+0000","last_peered":"2024-09-10T21:25:10.322806+0000","last_clean":"2024-09-10T21:25:10.322806+0000","last_became_active":"2024-09-10T21:25:10.317996+0000","last_became_peered":"2024-09-10T21:25:10.317996+0000","last_unstale":"2024-09-10T21:25:10.322806+0000","last_undegraded":"2024-09-10T21:25:10.322806+0000","last_fullsized":"2024-09-10T21:25:10.322806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:21.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335862+0000","last_change":"2024-09-10T21:25:09.319663+0000","last_active":"2024-09-10T21:25:10.335862+0000","last_peered":"2024-09-10T21:25:10.335862+0000","last_clean":"2024-09-10T21:25:10.335862+0000","last_became_active":"2024-09-10T21:25:09.319435+0000","last_became_peered":"2024-09-10T21:25:09.319435+0000","last_unstale":"2024-09-10T21:25:10.335862+0000","last_undegraded":"2024-09-10T21:25:10.335862+0000","last_fullsized":"2024-09-10T21:25:10.335862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:51.565933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311303+0000","last_change":"2024-09-10T21:25:10.306708+0000","last_active":"2024-09-10T21:25:10.311303+0000","last_peered":"2024-09-10T21:25:10.311303+0000","last_clean":"2024-09-10T21:25:10.311303+0000","last_became_active":"2024-09-10T21:25:10.306429+0000","last_became_peered":"2024-09-10T21:25:10.306429+0000","last_unstale":"2024-09-10T21:25:10.311303+0000","last_undegraded":"2024-09-10T21:25:10.311303+0000","last_fullsized":"2024-09-10T21:25:10.311303+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:25.889472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313672+0000","last_change":"2024-09-10T21:25:09.309569+0000","last_active":"2024-09-10T21:25:11.313672+0000","last_peered":"2024-09-10T21:25:11.313672+0000","last_clean":"2024-09-10T21:25:11.313672+0000","last_became_active":"2024-09-10T21:25:09.309226+0000","last_became_peered":"2024-09-10T21:25:09.309226+0000","last_unstale":"2024-09-10T21:25:11.313672+0000","last_undegraded":"2024-09-10T21:25:11.313672+0000","last_fullsized":"2024-09-10T21:25:11.313672+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:59.647662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.322595+0000","last_change":"2024-09-10T21:25:10.317630+0000","last_active":"2024-09-10T21:25:10.322595+0000","last_peered":"2024-09-10T21:25:10.322595+0000","last_clean":"2024-09-10T21:25:10.322595+0000","last_became_active":"2024-09-10T21:25:10.317241+0000","last_became_peered":"2024-09-10T21:25:10.317241+0000","last_unstale":"2024-09-10T21:25:10.322595+0000","last_undegraded":"2024-09-10T21:25:10.322595+0000","last_fullsized":"2024-09-10T21:25:10.322595+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.863854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.330846+0000","last_change":"2024-09-10T21:25:09.306182+0000","last_active":"2024-09-10T21:25:10.330846+0000","last_peered":"2024-09-10T21:25:10.330846+0000","last_clean":"2024-09-10T21:25:10.330846+0000","last_became_active":"2024-09-10T21:25:09.305934+0000","last_became_peered":"2024-09-10T21:25:09.305934+0000","last_unstale":"2024-09-10T21:25:10.330846+0000","last_undegraded":"2024-09-10T21:25:10.330846+0000","last_fullsized":"2024-09-10T21:25:10.330846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:32.170499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312736+0000","last_change":"2024-09-10T21:25:10.318027+0000","last_active":"2024-09-10T21:25:11.312736+0000","last_peered":"2024-09-10T21:25:11.312736+0000","last_clean":"2024-09-10T21:25:11.312736+0000","last_became_active":"2024-09-10T21:25:10.317330+0000","last_became_peered":"2024-09-10T21:25:10.317330+0000","last_unstale":"2024-09-10T21:25:11.312736+0000","last_undegraded":"2024-09-10T21:25:11.312736+0000","last_fullsized":"2024-09-10T21:25:11.312736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:58.964847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.327782+0000","last_change":"2024-09-10T21:25:09.307932+0000","last_active":"2024-09-10T21:25:10.327782+0000","last_peered":"2024-09-10T21:25:10.327782+0000","last_clean":"2024-09-10T21:25:10.327782+0000","last_became_active":"2024-09-10T21:25:09.307629+0000","last_became_peered":"2024-09-10T21:25:09.307629+0000","last_unstale":"2024-09-10T21:25:10.327782+0000","last_undegraded":"2024-09-10T21:25:10.327782+0000","last_fullsized":"2024-09-10T21:25:10.327782+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:36.143619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319664+0000","last_change":"2024-09-10T21:25:10.312532+0000","last_active":"2024-09-10T21:25:10.319664+0000","last_peered":"2024-09-10T21:25:10.319664+0000","last_clean":"2024-09-10T21:25:10.319664+0000","last_became_active":"2024-09-10T21:25:10.312270+0000","last_became_peered":"2024-09-10T21:25:10.312270+0000","last_unstale":"2024-09-10T21:25:10.319664+0000","last_undegraded":"2024-09-10T21:25:10.319664+0000","last_fullsized":"2024-09-10T21:25:10.319664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:01.124483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335865+0000","last_change":"2024-09-10T21:25:09.304425+0000","last_active":"2024-09-10T21:25:10.335865+0000","last_peered":"2024-09-10T21:25:10.335865+0000","last_clean":"2024-09-10T21:25:10.335865+0000","last_became_active":"2024-09-10T21:25:09.303548+0000","last_became_peered":"2024-09-10T21:25:09.303548+0000","last_unstale":"2024-09-10T21:25:10.335865+0000","last_undegraded":"2024-09-10T21:25:10.335865+0000","last_fullsized":"2024-09-10T21:25:10.335865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:36.287869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.331719+0000","last_change":"2024-09-10T21:25:10.327423+0000","last_active":"2024-09-10T21:25:10.331719+0000","last_peered":"2024-09-10T21:25:10.331719+0000","last_clean":"2024-09-10T21:25:10.331719+0000","last_became_active":"2024-09-10T21:25:10.327148+0000","last_became_peered":"2024-09-10T21:25:10.327148+0000","last_unstale":"2024-09-10T21:25:10.331719+0000","last_undegraded":"2024-09-10T21:25:10.331719+0000","last_fullsized":"2024-09-10T21:25:10.331719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:15.356200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312528+0000","last_change":"2024-09-10T21:25:09.312771+0000","last_active":"2024-09-10T21:25:11.312528+0000","last_peered":"2024-09-10T21:25:11.312528+0000","last_clean":"2024-09-10T21:25:11.312528+0000","last_became_active":"2024-09-10T21:25:09.312584+0000","last_became_peered":"2024-09-10T21:25:09.312584+0000","last_unstale":"2024-09-10T21:25:11.312528+0000","last_undegraded":"2024-09-10T21:25:11.312528+0000","last_fullsized":"2024-09-10T21:25:11.312528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:29.845774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.315143+0000","last_change":"2024-09-10T21:25:10.309742+0000","last_active":"2024-09-10T21:25:10.315143+0000","last_peered":"2024-09-10T21:25:10.315143+0000","last_clean":"2024-09-10T21:25:10.315143+0000","last_became_active":"2024-09-10T21:25:10.309304+0000","last_became_peered":"2024-09-10T21:25:10.309304+0000","last_unstale":"2024-09-10T21:25:10.315143+0000","last_undegraded":"2024-09-10T21:25:10.315143+0000","last_fullsized":"2024-09-10T21:25:10.315143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:06.742437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.300325+0000","last_change":"2024-09-10T21:25:09.307419+0000","last_active":"2024-09-10T21:25:10.300325+0000","last_peered":"2024-09-10T21:25:10.300325+0000","last_clean":"2024-09-10T21:25:10.300325+0000","last_became_active":"2024-09-10T21:25:09.307145+0000","last_became_peered":"2024-09-10T21:25:09.307145+0000","last_unstale":"2024-09-10T21:25:10.300325+0000","last_undegraded":"2024-09-10T21:25:10.300325+0000","last_fullsized":"2024-09-10T21:25:10.300325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:46.430906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313180+0000","last_change":"2024-09-10T21:25:10.319762+0000","last_active":"2024-09-10T21:25:11.313180+0000","last_peered":"2024-09-10T21:25:11.313180+0000","last_clean":"2024-09-10T21:25:11.313180+0000","last_became_active":"2024-09-10T21:25:10.315185+0000","last_became_peered":"2024-09-10T21:25:10.315185+0000","last_unstale":"2024-09-10T21:25:11.313180+0000","last_undegraded":"2024-09-10T21:25:11.313180+0000","last_fullsized":"2024-09-10T21:25:11.313180+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:04.194994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314704+0000","last_change":"2024-09-10T21:25:09.300063+0000","last_active":"2024-09-10T21:25:11.314704+0000","last_peered":"2024-09-10T21:25:11.314704+0000","last_clean":"2024-09-10T21:25:11.314704+0000","last_became_active":"2024-09-10T21:25:09.299765+0000","last_became_peered":"2024-09-10T21:25:09.299765+0000","last_unstale":"2024-09-10T21:25:11.314704+0000","last_undegraded":"2024-09-10T21:25:11.314704+0000","last_fullsized":"2024-09-10T21:25:11.314704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:10.782413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328189+0000","last_change":"2024-09-10T21:25:10.316919+0000","last_active":"2024-09-10T21:25:10.328189+0000","last_peered":"2024-09-10T21:25:10.328189+0000","last_clean":"2024-09-10T21:25:10.328189+0000","last_became_active":"2024-09-10T21:25:10.316645+0000","last_became_peered":"2024-09-10T21:25:10.316645+0000","last_unstale":"2024-09-10T21:25:10.328189+0000","last_undegraded":"2024-09-10T21:25:10.328189+0000","last_fullsized":"2024-09-10T21:25:10.328189+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.051223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293441+0000","last_change":"2024-09-10T21:25:09.316339+0000","last_active":"2024-09-10T21:25:10.293441+0000","last_peered":"2024-09-10T21:25:10.293441+0000","last_clean":"2024-09-10T21:25:10.293441+0000","last_became_active":"2024-09-10T21:25:09.316026+0000","last_became_peered":"2024-09-10T21:25:09.316026+0000","last_unstale":"2024-09-10T21:25:10.293441+0000","last_undegraded":"2024-09-10T21:25:10.293441+0000","last_fullsized":"2024-09-10T21:25:10.293441+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:27.156379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314603+0000","last_change":"2024-09-10T21:25:10.321352+0000","last_active":"2024-09-10T21:25:11.314603+0000","last_peered":"2024-09-10T21:25:11.314603+0000","last_clean":"2024-09-10T21:25:11.314603+0000","last_became_active":"2024-09-10T21:25:10.321028+0000","last_became_peered":"2024-09-10T21:25:10.321028+0000","last_unstale":"2024-09-10T21:25:11.314603+0000","last_undegraded":"2024-09-10T21:25:11.314603+0000","last_fullsized":"2024-09-10T21:25:11.314603+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:42.294158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.339409+0000","last_change":"2024-09-10T21:25:09.314657+0000","last_active":"2024-09-10T21:25:10.339409+0000","last_peered":"2024-09-10T21:25:10.339409+0000","last_clean":"2024-09-10T21:25:10.339409+0000","last_became_active":"2024-09-10T21:25:09.314339+0000","last_became_peered":"2024-09-10T21:25:09.314339+0000","last_unstale":"2024-09-10T21:25:10.339409+0000","last_undegraded":"2024-09-10T21:25:10.339409+0000","last_fullsized":"2024-09-10T21:25:10.339409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:39.109897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311932+0000","last_change":"2024-09-10T21:25:10.305685+0000","last_active":"2024-09-10T21:25:10.311932+0000","last_peered":"2024-09-10T21:25:10.311932+0000","last_clean":"2024-09-10T21:25:10.311932+0000","last_became_active":"2024-09-10T21:25:10.305473+0000","last_became_peered":"2024-09-10T21:25:10.305473+0000","last_unstale":"2024-09-10T21:25:10.311932+0000","last_undegraded":"2024-09-10T21:25:10.311932+0000","last_fullsized":"2024-09-10T21:25:10.311932+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:54.729961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313255+0000","last_change":"2024-09-10T21:25:09.308217+0000","last_active":"2024-09-10T21:25:11.313255+0000","last_peered":"2024-09-10T21:25:11.313255+0000","last_clean":"2024-09-10T21:25:11.313255+0000","last_became_active":"2024-09-10T21:25:09.307894+0000","last_became_peered":"2024-09-10T21:25:09.307894+0000","last_unstale":"2024-09-10T21:25:11.313255+0000","last_undegraded":"2024-09-10T21:25:11.313255+0000","last_fullsized":"2024-09-10T21:25:11.313255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:52.032968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312888+0000","last_change":"2024-09-10T21:25:10.317608+0000","last_active":"2024-09-10T21:25:11.312888+0000","last_peered":"2024-09-10T21:25:11.312888+0000","last_clean":"2024-09-10T21:25:11.312888+0000","last_became_active":"2024-09-10T21:25:10.316630+0000","last_became_peered":"2024-09-10T21:25:10.316630+0000","last_unstale":"2024-09-10T21:25:11.312888+0000","last_undegraded":"2024-09-10T21:25:11.312888+0000","last_fullsized":"2024-09-10T21:25:11.312888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.897181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312729+0000","last_change":"2024-09-10T21:25:09.309657+0000","last_active":"2024-09-10T21:25:11.312729+0000","last_peered":"2024-09-10T21:25:11.312729+0000","last_clean":"2024-09-10T21:25:11.312729+0000","last_became_active":"2024-09-10T21:25:09.309352+0000","last_became_peered":"2024-09-10T21:25:09.309352+0000","last_unstale":"2024-09-10T21:25:11.312729+0000","last_undegraded":"2024-09-10T21:25:11.312729+0000","last_fullsized":"2024-09-10T21:25:11.312729+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:50.728255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.322780+0000","last_change":"2024-09-10T21:25:10.318156+0000","last_active":"2024-09-10T21:25:10.322780+0000","last_peered":"2024-09-10T21:25:10.322780+0000","last_clean":"2024-09-10T21:25:10.322780+0000","last_became_active":"2024-09-10T21:25:10.317819+0000","last_became_peered":"2024-09-10T21:25:10.317819+0000","last_unstale":"2024-09-10T21:25:10.322780+0000","last_undegraded":"2024-09-10T21:25:10.322780+0000","last_fullsized":"2024-09-10T21:25:10.322780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:32.179380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293965+0000","last_change":"2024-09-10T21:25:09.310648+0000","last_active":"2024-09-10T21:25:10.293965+0000","last_peered":"2024-09-10T21:25:10.293965+0000","last_clean":"2024-09-10T21:25:10.293965+0000","last_became_active":"2024-09-10T21:25:09.310206+0000","last_became_peered":"2024-09-10T21:25:09.310206+0000","last_unstale":"2024-09-10T21:25:10.293965+0000","last_undegraded":"2024-09-10T21:25:10.293965+0000","last_fullsized":"2024-09-10T21:25:10.293965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:32.170499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325146+0000","last_change":"2024-09-10T21:25:10.319417+0000","last_active":"2024-09-10T21:25:10.325146+0000","last_peered":"2024-09-10T21:25:10.325146+0000","last_clean":"2024-09-10T21:25:10.325146+0000","last_became_active":"2024-09-10T21:25:10.319148+0000","last_became_peered":"2024-09-10T21:25:10.319148+0000","last_unstale":"2024-09-10T21:25:10.325146+0000","last_undegraded":"2024-09-10T21:25:10.325146+0000","last_fullsized":"2024-09-10T21:25:10.325146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:33.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294230+0000","last_change":"2024-09-10T21:25:09.314970+0000","last_active":"2024-09-10T21:25:10.294230+0000","last_peered":"2024-09-10T21:25:10.294230+0000","last_clean":"2024-09-10T21:25:10.294230+0000","last_became_active":"2024-09-10T21:25:09.314586+0000","last_became_peered":"2024-09-10T21:25:09.314586+0000","last_unstale":"2024-09-10T21:25:10.294230+0000","last_undegraded":"2024-09-10T21:25:10.294230+0000","last_fullsized":"2024-09-10T21:25:10.294230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:20.786558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.312161+0000","last_change":"2024-09-10T21:25:10.305675+0000","last_active":"2024-09-10T21:25:10.312161+0000","last_peered":"2024-09-10T21:25:10.312161+0000","last_clean":"2024-09-10T21:25:10.312161+0000","last_became_active":"2024-09-10T21:25:10.305416+0000","last_became_peered":"2024-09-10T21:25:10.305416+0000","last_unstale":"2024-09-10T21:25:10.312161+0000","last_undegraded":"2024-09-10T21:25:10.312161+0000","last_fullsized":"2024-09-10T21:25:10.312161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:12.301442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":21,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.339011+0000","last_change":"2024-09-10T21:25:09.305368+0000","last_active":"2024-09-10T21:25:10.339011+0000","last_peered":"2024-09-10T21:25:10.339011+0000","last_clean":"2024-09-10T21:25:10.339011+0000","last_became_active":"2024-09-10T21:25:09.305110+0000","last_became_peered":"2024-09-10T21:25:09.305110+0000","last_unstale":"2024-09-10T21:25:10.339011+0000","last_undegraded":"2024-09-10T21:25:10.339011+0000","last_fullsized":"2024-09-10T21:25:10.339011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:38.473532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.315561+0000","last_change":"2024-09-10T21:25:10.311100+0000","last_active":"2024-09-10T21:25:10.315561+0000","last_peered":"2024-09-10T21:25:10.315561+0000","last_clean":"2024-09-10T21:25:10.315561+0000","last_became_active":"2024-09-10T21:25:10.310726+0000","last_became_peered":"2024-09-10T21:25:10.310726+0000","last_unstale":"2024-09-10T21:25:10.315561+0000","last_undegraded":"2024-09-10T21:25:10.315561+0000","last_fullsized":"2024-09-10T21:25:10.315561+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:00.307962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.336095+0000","last_change":"2024-09-10T21:25:09.311810+0000","last_active":"2024-09-10T21:25:10.336095+0000","last_peered":"2024-09-10T21:25:10.336095+0000","last_clean":"2024-09-10T21:25:10.336095+0000","last_became_active":"2024-09-10T21:25:09.311524+0000","last_became_peered":"2024-09-10T21:25:09.311524+0000","last_unstale":"2024-09-10T21:25:10.336095+0000","last_undegraded":"2024-09-10T21:25:10.336095+0000","last_fullsized":"2024-09-10T21:25:10.336095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:53.307462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328597+0000","last_change":"2024-09-10T21:25:10.318304+0000","last_active":"2024-09-10T21:25:10.328597+0000","last_peered":"2024-09-10T21:25:10.328597+0000","last_clean":"2024-09-10T21:25:10.328597+0000","last_became_active":"2024-09-10T21:25:10.318030+0000","last_became_peered":"2024-09-10T21:25:10.318030+0000","last_unstale":"2024-09-10T21:25:10.328597+0000","last_undegraded":"2024-09-10T21:25:10.328597+0000","last_fullsized":"2024-09-10T21:25:10.328597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.441629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.339546+0000","last_change":"2024-09-10T21:25:09.314359+0000","last_active":"2024-09-10T21:25:10.339546+0000","last_peered":"2024-09-10T21:25:10.339546+0000","last_clean":"2024-09-10T21:25:10.339546+0000","last_became_active":"2024-09-10T21:25:09.314043+0000","last_became_peered":"2024-09-10T21:25:09.314043+0000","last_unstale":"2024-09-10T21:25:10.339546+0000","last_undegraded":"2024-09-10T21:25:10.339546+0000","last_fullsized":"2024-09-10T21:25:10.339546+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:35.683426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.321441+0000","last_change":"2024-09-10T21:25:10.313384+0000","last_active":"2024-09-10T21:25:10.321441+0000","last_peered":"2024-09-10T21:25:10.321441+0000","last_clean":"2024-09-10T21:25:10.321441+0000","last_became_active":"2024-09-10T21:25:10.313053+0000","last_became_peered":"2024-09-10T21:25:10.313053+0000","last_unstale":"2024-09-10T21:25:10.321441+0000","last_undegraded":"2024-09-10T21:25:10.321441+0000","last_fullsized":"2024-09-10T21:25:10.321441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:11.679983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335205+0000","last_change":"2024-09-10T21:25:09.305697+0000","last_active":"2024-09-10T21:25:10.335205+0000","last_peered":"2024-09-10T21:25:10.335205+0000","last_clean":"2024-09-10T21:25:10.335205+0000","last_became_active":"2024-09-10T21:25:09.305437+0000","last_became_peered":"2024-09-10T21:25:09.305437+0000","last_unstale":"2024-09-10T21:25:10.335205+0000","last_undegraded":"2024-09-10T21:25:10.335205+0000","last_fullsized":"2024-09-10T21:25:10.335205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:29.845979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.327209+0000","last_change":"2024-09-10T21:25:10.321281+0000","last_active":"2024-09-10T21:25:10.327209+0000","last_peered":"2024-09-10T21:25:10.327209+0000","last_clean":"2024-09-10T21:25:10.327209+0000","last_became_active":"2024-09-10T21:25:10.321119+0000","last_became_peered":"2024-09-10T21:25:10.321119+0000","last_unstale":"2024-09-10T21:25:10.327209+0000","last_undegraded":"2024-09-10T21:25:10.327209+0000","last_fullsized":"2024-09-10T21:25:10.327209+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:21.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.306495+0000","last_change":"2024-09-10T21:25:09.310801+0000","last_active":"2024-09-10T21:25:10.306495+0000","last_peered":"2024-09-10T21:25:10.306495+0000","last_clean":"2024-09-10T21:25:10.306495+0000","last_became_active":"2024-09-10T21:25:09.310550+0000","last_became_peered":"2024-09-10T21:25:09.310550+0000","last_unstale":"2024-09-10T21:25:10.306495+0000","last_undegraded":"2024-09-10T21:25:10.306495+0000","last_fullsized":"2024-09-10T21:25:10.306495+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:00.118490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316219+0000","last_change":"2024-09-10T21:25:10.307738+0000","last_active":"2024-09-10T21:25:10.316219+0000","last_peered":"2024-09-10T21:25:10.316219+0000","last_clean":"2024-09-10T21:25:10.316219+0000","last_became_active":"2024-09-10T21:25:10.307423+0000","last_became_peered":"2024-09-10T21:25:10.307423+0000","last_unstale":"2024-09-10T21:25:10.316219+0000","last_undegraded":"2024-09-10T21:25:10.316219+0000","last_fullsized":"2024-09-10T21:25:10.316219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:23.261976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.330129+0000","last_change":"2024-09-10T21:25:09.302530+0000","last_active":"2024-09-10T21:25:10.330129+0000","last_peered":"2024-09-10T21:25:10.330129+0000","last_clean":"2024-09-10T21:25:10.330129+0000","last_became_active":"2024-09-10T21:25:09.302012+0000","last_became_peered":"2024-09-10T21:25:09.302012+0000","last_unstale":"2024-09-10T21:25:10.330129+0000","last_undegraded":"2024-09-10T21:25:10.330129+0000","last_fullsized":"2024-09-10T21:25:10.330129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:59.301968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312458+0000","last_change":"2024-09-10T21:25:10.325283+0000","last_active":"2024-09-10T21:25:11.312458+0000","last_peered":"2024-09-10T21:25:11.312458+0000","last_clean":"2024-09-10T21:25:11.312458+0000","last_became_active":"2024-09-10T21:25:10.325016+0000","last_became_peered":"2024-09-10T21:25:10.325016+0000","last_unstale":"2024-09-10T21:25:11.312458+0000","last_undegraded":"2024-09-10T21:25:11.312458+0000","last_fullsized":"2024-09-10T21:25:11.312458+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:18.795068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293735+0000","last_change":"2024-09-10T21:25:09.312281+0000","last_active":"2024-09-10T21:25:10.293735+0000","last_peered":"2024-09-10T21:25:10.293735+0000","last_clean":"2024-09-10T21:25:10.293735+0000","last_became_active":"2024-09-10T21:25:09.312048+0000","last_became_peered":"2024-09-10T21:25:09.312048+0000","last_unstale":"2024-09-10T21:25:10.293735+0000","last_undegraded":"2024-09-10T21:25:10.293735+0000","last_fullsized":"2024-09-10T21:25:10.293735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:20.692813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325079+0000","last_change":"2024-09-10T21:25:10.318224+0000","last_active":"2024-09-10T21:25:10.325079+0000","last_peered":"2024-09-10T21:25:10.325079+0000","last_clean":"2024-09-10T21:25:10.325079+0000","last_became_active":"2024-09-10T21:25:10.317857+0000","last_became_peered":"2024-09-10T21:25:10.317857+0000","last_unstale":"2024-09-10T21:25:10.325079+0000","last_undegraded":"2024-09-10T21:25:10.325079+0000","last_fullsized":"2024-09-10T21:25:10.325079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:30.320131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.300577+0000","last_change":"2024-09-10T21:25:09.305739+0000","last_active":"2024-09-10T21:25:10.300577+0000","last_peered":"2024-09-10T21:25:10.300577+0000","last_clean":"2024-09-10T21:25:10.300577+0000","last_became_active":"2024-09-10T21:25:09.305452+0000","last_became_peered":"2024-09-10T21:25:09.305452+0000","last_unstale":"2024-09-10T21:25:10.300577+0000","last_undegraded":"2024-09-10T21:25:10.300577+0000","last_fullsized":"2024-09-10T21:25:10.300577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:57.809201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312821+0000","last_change":"2024-09-10T21:25:10.321948+0000","last_active":"2024-09-10T21:25:11.312821+0000","last_peered":"2024-09-10T21:25:11.312821+0000","last_clean":"2024-09-10T21:25:11.312821+0000","last_became_active":"2024-09-10T21:25:10.321688+0000","last_became_peered":"2024-09-10T21:25:10.321688+0000","last_unstale":"2024-09-10T21:25:11.312821+0000","last_undegraded":"2024-09-10T21:25:11.312821+0000","last_fullsized":"2024-09-10T21:25:11.312821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:29.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293187+0000","last_change":"2024-09-10T21:25:09.309438+0000","last_active":"2024-09-10T21:25:10.293187+0000","last_peered":"2024-09-10T21:25:10.293187+0000","last_clean":"2024-09-10T21:25:10.293187+0000","last_became_active":"2024-09-10T21:25:09.309145+0000","last_became_peered":"2024-09-10T21:25:09.309145+0000","last_unstale":"2024-09-10T21:25:10.293187+0000","last_undegraded":"2024-09-10T21:25:10.293187+0000","last_fullsized":"2024-09-10T21:25:10.293187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:01.976460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316806+0000","last_change":"2024-09-10T21:25:10.313745+0000","last_active":"2024-09-10T21:25:10.316806+0000","last_peered":"2024-09-10T21:25:10.316806+0000","last_clean":"2024-09-10T21:25:10.316806+0000","last_became_active":"2024-09-10T21:25:10.313437+0000","last_became_peered":"2024-09-10T21:25:10.313437+0000","last_unstale":"2024-09-10T21:25:10.316806+0000","last_undegraded":"2024-09-10T21:25:10.316806+0000","last_fullsized":"2024-09-10T21:25:10.316806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:13.617229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.337755+0000","last_change":"2024-09-10T21:25:09.315503+0000","last_active":"2024-09-10T21:25:10.337755+0000","last_peered":"2024-09-10T21:25:10.337755+0000","last_clean":"2024-09-10T21:25:10.337755+0000","last_became_active":"2024-09-10T21:25:09.314377+0000","last_became_peered":"2024-09-10T21:25:09.314377+0000","last_unstale":"2024-09-10T21:25:10.337755+0000","last_undegraded":"2024-09-10T21:25:10.337755+0000","last_fullsized":"2024-09-10T21:25:10.337755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:18.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325049+0000","last_change":"2024-09-10T21:25:10.318217+0000","last_active":"2024-09-10T21:25:10.325049+0000","last_peered":"2024-09-10T21:25:10.325049+0000","last_clean":"2024-09-10T21:25:10.325049+0000","last_became_active":"2024-09-10T21:25:10.317949+0000","last_became_peered":"2024-09-10T21:25:10.317949+0000","last_unstale":"2024-09-10T21:25:10.325049+0000","last_undegraded":"2024-09-10T21:25:10.325049+0000","last_fullsized":"2024-09-10T21:25:10.325049+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:21.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294717+0000","last_change":"2024-09-10T21:25:09.302821+0000","last_active":"2024-09-10T21:25:10.294717+0000","last_peered":"2024-09-10T21:25:10.294717+0000","last_clean":"2024-09-10T21:25:10.294717+0000","last_became_active":"2024-09-10T21:25:09.302539+0000","last_became_peered":"2024-09-10T21:25:09.302539+0000","last_unstale":"2024-09-10T21:25:10.294717+0000","last_undegraded":"2024-09-10T21:25:10.294717+0000","last_fullsized":"2024-09-10T21:25:10.294717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:12.695159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319812+0000","last_change":"2024-09-10T21:25:10.315253+0000","last_active":"2024-09-10T21:25:10.319812+0000","last_peered":"2024-09-10T21:25:10.319812+0000","last_clean":"2024-09-10T21:25:10.319812+0000","last_became_active":"2024-09-10T21:25:10.312954+0000","last_became_peered":"2024-09-10T21:25:10.312954+0000","last_unstale":"2024-09-10T21:25:10.319812+0000","last_undegraded":"2024-09-10T21:25:10.319812+0000","last_fullsized":"2024-09-10T21:25:10.319812+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.863854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294455+0000","last_change":"2024-09-10T21:25:09.303737+0000","last_active":"2024-09-10T21:25:10.294455+0000","last_peered":"2024-09-10T21:25:10.294455+0000","last_clean":"2024-09-10T21:25:10.294455+0000","last_became_active":"2024-09-10T21:25:09.303470+0000","last_became_peered":"2024-09-10T21:25:09.303470+0000","last_unstale":"2024-09-10T21:25:10.294455+0000","last_undegraded":"2024-09-10T21:25:10.294455+0000","last_fullsized":"2024-09-10T21:25:10.294455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:23.982363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319188+0000","last_change":"2024-09-10T21:25:10.316389+0000","last_active":"2024-09-10T21:25:10.319188+0000","last_peered":"2024-09-10T21:25:10.319188+0000","last_clean":"2024-09-10T21:25:10.319188+0000","last_became_active":"2024-09-10T21:25:10.316086+0000","last_became_peered":"2024-09-10T21:25:10.316086+0000","last_unstale":"2024-09-10T21:25:10.319188+0000","last_undegraded":"2024-09-10T21:25:10.319188+0000","last_fullsized":"2024-09-10T21:25:10.319188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:42.986151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312898+0000","last_change":"2024-09-10T21:25:09.309626+0000","last_active":"2024-09-10T21:25:11.312898+0000","last_peered":"2024-09-10T21:25:11.312898+0000","last_clean":"2024-09-10T21:25:11.312898+0000","last_became_active":"2024-09-10T21:25:09.309248+0000","last_became_peered":"2024-09-10T21:25:09.309248+0000","last_unstale":"2024-09-10T21:25:11.312898+0000","last_undegraded":"2024-09-10T21:25:11.312898+0000","last_fullsized":"2024-09-10T21:25:11.312898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:59.924842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.306830+0000","last_change":"2024-09-10T21:25:10.298644+0000","last_active":"2024-09-10T21:25:10.306830+0000","last_peered":"2024-09-10T21:25:10.306830+0000","last_clean":"2024-09-10T21:25:10.306830+0000","last_became_active":"2024-09-10T21:25:10.298318+0000","last_became_peered":"2024-09-10T21:25:10.298318+0000","last_unstale":"2024-09-10T21:25:10.306830+0000","last_undegraded":"2024-09-10T21:25:10.306830+0000","last_fullsized":"2024-09-10T21:25:10.306830+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:22.444236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.303528+0000","last_change":"2024-09-10T21:25:09.309411+0000","last_active":"2024-09-10T21:25:10.303528+0000","last_peered":"2024-09-10T21:25:10.303528+0000","last_clean":"2024-09-10T21:25:10.303528+0000","last_became_active":"2024-09-10T21:25:09.309126+0000","last_became_peered":"2024-09-10T21:25:09.309126+0000","last_unstale":"2024-09-10T21:25:10.303528+0000","last_undegraded":"2024-09-10T21:25:10.303528+0000","last_fullsized":"2024-09-10T21:25:10.303528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:22.045229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311752+0000","last_change":"2024-09-10T21:25:10.307503+0000","last_active":"2024-09-10T21:25:10.311752+0000","last_peered":"2024-09-10T21:25:10.311752+0000","last_clean":"2024-09-10T21:25:10.311752+0000","last_became_active":"2024-09-10T21:25:10.307238+0000","last_became_peered":"2024-09-10T21:25:10.307238+0000","last_unstale":"2024-09-10T21:25:10.311752+0000","last_undegraded":"2024-09-10T21:25:10.311752+0000","last_fullsized":"2024-09-10T21:25:10.311752+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:02.809894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294358+0000","last_change":"2024-09-10T21:25:09.312813+0000","last_active":"2024-09-10T21:25:10.294358+0000","last_peered":"2024-09-10T21:25:10.294358+0000","last_clean":"2024-09-10T21:25:10.294358+0000","last_became_active":"2024-09-10T21:25:09.312511+0000","last_became_peered":"2024-09-10T21:25:09.312511+0000","last_unstale":"2024-09-10T21:25:10.294358+0000","last_undegraded":"2024-09-10T21:25:10.294358+0000","last_fullsized":"2024-09-10T21:25:10.294358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:13.973679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.334988+0000","last_change":"2024-09-10T21:25:10.323177+0000","last_active":"2024-09-10T21:25:10.334988+0000","last_peered":"2024-09-10T21:25:10.334988+0000","last_clean":"2024-09-10T21:25:10.334988+0000","last_became_active":"2024-09-10T21:25:10.320099+0000","last_became_peered":"2024-09-10T21:25:10.320099+0000","last_unstale":"2024-09-10T21:25:10.334988+0000","last_undegraded":"2024-09-10T21:25:10.334988+0000","last_fullsized":"2024-09-10T21:25:10.334988+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:19.572091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312810+0000","last_change":"2024-09-10T21:25:09.314378+0000","last_active":"2024-09-10T21:25:11.312810+0000","last_peered":"2024-09-10T21:25:11.312810+0000","last_clean":"2024-09-10T21:25:11.312810+0000","last_became_active":"2024-09-10T21:25:09.314138+0000","last_became_peered":"2024-09-10T21:25:09.314138+0000","last_unstale":"2024-09-10T21:25:11.312810+0000","last_undegraded":"2024-09-10T21:25:11.312810+0000","last_fullsized":"2024-09-10T21:25:11.312810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:13.188324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.324765+0000","last_change":"2024-09-10T21:25:10.311534+0000","last_active":"2024-09-10T21:25:10.324765+0000","last_peered":"2024-09-10T21:25:10.324765+0000","last_clean":"2024-09-10T21:25:10.324765+0000","last_became_active":"2024-09-10T21:25:10.310769+0000","last_became_peered":"2024-09-10T21:25:10.310769+0000","last_unstale":"2024-09-10T21:25:10.324765+0000","last_undegraded":"2024-09-10T21:25:10.324765+0000","last_fullsized":"2024-09-10T21:25:10.324765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:13.433230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.304039+0000","last_change":"2024-09-10T21:25:09.301797+0000","last_active":"2024-09-10T21:25:10.304039+0000","last_peered":"2024-09-10T21:25:10.304039+0000","last_clean":"2024-09-10T21:25:10.304039+0000","last_became_active":"2024-09-10T21:25:09.301602+0000","last_became_peered":"2024-09-10T21:25:09.301602+0000","last_unstale":"2024-09-10T21:25:10.304039+0000","last_undegraded":"2024-09-10T21:25:10.304039+0000","last_fullsized":"2024-09-10T21:25:10.304039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:05.736443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":143,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.300205+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:25:10.300205+0000","last_peered":"2024-09-10T21:25:10.300205+0000","last_clean":"2024-09-10T21:25:10.300205+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:25:10.300205+0000","last_undegraded":"2024-09-10T21:25:10.300205+0000","last_fullsized":"2024-09-10T21:25:10.300205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1040384,"data_stored":779040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":458752,"data_stored":190284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1040384,"data_stored":778480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":450560,"data_stored":188182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":196518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:14.419 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- 172.21.15.97:0/3290877129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec15e750 msgr2=0x7f94ec0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 --2- 172.21.15.97:0/3290877129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec15e750 0x7f94ec0cd580 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94ec06f5c0 tx=0x7f94e801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- 172.21.15.97:0/3290877129 shutdown_connections 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 --2- 172.21.15.97:0/3290877129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec15e750 0x7f94ec0cd580 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 --2- 172.21.15.97:0/3290877129 >> v2:172.21.15.194:3300/0 conn(0x7f94ec15f150 0x7f94ec0b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- 172.21.15.97:0/3290877129 >> 172.21.15.97:0/3290877129 conn(0x7f94ec0b28c0 msgr2=0x7f94ec0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- 172.21.15.97:0/3290877129 shutdown_connections 2024-09-10T21:25:14.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- 172.21.15.97:0/3290877129 wait complete. 2024-09-10T21:25:14.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 Processor -- start 2024-09-10T21:25:14.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- start start 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94ec15e750 0x7f94ec0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94ec15f150 0x7f94ec0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec1568a0 0x7f94ec156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec168440 con 0x7f94ec1568a0 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec1682c0 con 0x7f94ec15f150 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f4800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f94ec1685c0 con 0x7f94ec15e750 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec1568a0 0x7f94ec156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec1568a0 0x7f94ec156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41298/0 (socket says 172.21.15.97:41298) 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f2800640 1 -- 172.21.15.97:0/2208109466 learned_addr learned my addr 172.21.15.97:0/2208109466 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f1e00640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94ec15e750 0x7f94ec0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.596+0000 7f94f1400640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94ec15f150 0x7f94ec0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f2800640 1 -- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94ec15e750 msgr2=0x7f94ec0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f2800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94ec15e750 0x7f94ec0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f2800640 1 -- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94ec15f150 msgr2=0x7f94ec0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f2800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94ec15f150 0x7f94ec0b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f2800640 1 -- 172.21.15.97:0/2208109466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec1fbe90 con 0x7f94ec1568a0 2024-09-10T21:25:14.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f1400640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94ec15f150 0x7f94ec0b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:14.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f2800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec1568a0 0x7f94ec156d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009bd0 tx=0x7f94e8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:14.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94e6000640 1 -- 172.21.15.97:0/2208109466 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e80bb070 con 0x7f94ec1568a0 2024-09-10T21:25:14.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94e6000640 1 -- 172.21.15.97:0/2208109466 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94e8004e00 con 0x7f94ec1568a0 2024-09-10T21:25:14.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec1f91d0 con 0x7f94ec1568a0 2024-09-10T21:25:14.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94e6000640 1 -- 172.21.15.97:0/2208109466 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e80b2040 con 0x7f94ec1568a0 2024-09-10T21:25:14.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec0ae4e0 con 0x7f94ec1568a0 2024-09-10T21:25:14.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ec0cd580 con 0x7f94ec1568a0 2024-09-10T21:25:14.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.600+0000 7f94e6000640 1 -- 172.21.15.97:0/2208109466 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f94e80b5030 con 0x7f94ec1568a0 2024-09-10T21:25:14.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.604+0000 7f94e6000640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94d00725e0 0x7f94d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.604+0000 7f94e6000640 1 -- 172.21.15.97:0/2208109466 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f94e812d0c0 con 0x7f94ec1568a0 2024-09-10T21:25:14.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.604+0000 7f94f1e00640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94d00725e0 0x7f94d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.604+0000 7f94f1e00640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94d00725e0 0x7f94d0074aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94e0004630 tx=0x7f94e000b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:14.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.608+0000 7f94e6000640 1 -- 172.21.15.97:0/2208109466 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f94e80a5e20 con 0x7f94ec1568a0 2024-09-10T21:25:14.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.768+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94ec1f94b0 con 0x7f94d00725e0 2024-09-10T21:25:14.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.772+0000 7f94e6000640 1 -- 172.21.15.97:0/2208109466 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f94ec1f94b0 con 0x7f94d00725e0 2024-09-10T21:25:14.782 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94d00725e0 msgr2=0x7f94d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94d00725e0 0x7f94d0074aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94e0004630 tx=0x7f94e000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec1568a0 msgr2=0x7f94ec156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec1568a0 0x7f94ec156d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009bd0 tx=0x7f94e8003f60 comp rx=0 tx=0).stop 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 shutdown_connections 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94d00725e0 0x7f94d0074aa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec1568a0 0x7f94ec156d30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94ec15f150 0x7f94ec0b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 --2- 172.21.15.97:0/2208109466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94ec15e750 0x7f94ec0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 >> 172.21.15.97:0/2208109466 conn(0x7f94ec0b28c0 msgr2=0x7f94ec1585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 shutdown_connections 2024-09-10T21:25:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.776+0000 7f94f4800640 1 -- 172.21.15.97:0/2208109466 wait complete. 2024-09-10T21:25:14.803 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-10T21:25:14.084398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220952,"kb_used_data":4692,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753768,"statfs":{"total":773094113280,"available":772867858432,"internally_reserved":0,"allocated":4804608,"data_stored":2696968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805430"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314720+0000","last_change":"2024-09-10T21:25:10.313445+0000","last_active":"2024-09-10T21:25:11.314720+0000","last_peered":"2024-09-10T21:25:11.314720+0000","last_clean":"2024-09-10T21:25:11.314720+0000","last_became_active":"2024-09-10T21:25:10.313186+0000","last_became_peered":"2024-09-10T21:25:10.313186+0000","last_unstale":"2024-09-10T21:25:11.314720+0000","last_undegraded":"2024-09-10T21:25:11.314720+0000","last_fullsized":"2024-09-10T21:25:11.314720+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:09.330807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.314930+0000","last_change":"2024-09-10T21:25:10.308813+0000","last_active":"2024-09-10T21:25:10.314930+0000","last_peered":"2024-09-10T21:25:10.314930+0000","last_clean":"2024-09-10T21:25:10.314930+0000","last_became_active":"2024-09-10T21:25:10.308558+0000","last_became_peered":"2024-09-10T21:25:10.308558+0000","last_unstale":"2024-09-10T21:25:10.314930+0000","last_undegraded":"2024-09-10T21:25:10.314930+0000","last_fullsized":"2024-09-10T21:25:10.314930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.051223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313545+0000","last_change":"2024-09-10T21:25:10.322846+0000","last_active":"2024-09-10T21:25:11.313545+0000","last_peered":"2024-09-10T21:25:11.313545+0000","last_clean":"2024-09-10T21:25:11.313545+0000","last_became_active":"2024-09-10T21:25:10.322475+0000","last_became_peered":"2024-09-10T21:25:10.322475+0000","last_unstale":"2024-09-10T21:25:11.313545+0000","last_undegraded":"2024-09-10T21:25:11.313545+0000","last_fullsized":"2024-09-10T21:25:11.313545+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:02.946600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313174+0000","last_change":"2024-09-10T21:25:10.315205+0000","last_active":"2024-09-10T21:25:11.313174+0000","last_peered":"2024-09-10T21:25:11.313174+0000","last_clean":"2024-09-10T21:25:11.313174+0000","last_became_active":"2024-09-10T21:25:10.314777+0000","last_became_peered":"2024-09-10T21:25:10.314777+0000","last_unstale":"2024-09-10T21:25:11.313174+0000","last_undegraded":"2024-09-10T21:25:11.313174+0000","last_fullsized":"2024-09-10T21:25:11.313174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:57.665201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319200+0000","last_change":"2024-09-10T21:25:10.317144+0000","last_active":"2024-09-10T21:25:10.319200+0000","last_peered":"2024-09-10T21:25:10.319200+0000","last_clean":"2024-09-10T21:25:10.319200+0000","last_became_active":"2024-09-10T21:25:10.312332+0000","last_became_peered":"2024-09-10T21:25:10.312332+0000","last_unstale":"2024-09-10T21:25:10.319200+0000","last_undegraded":"2024-09-10T21:25:10.319200+0000","last_fullsized":"2024-09-10T21:25:10.319200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:31.705951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.315099+0000","last_change":"2024-09-10T21:25:10.311396+0000","last_active":"2024-09-10T21:25:10.315099+0000","last_peered":"2024-09-10T21:25:10.315099+0000","last_clean":"2024-09-10T21:25:10.315099+0000","last_became_active":"2024-09-10T21:25:10.311002+0000","last_became_peered":"2024-09-10T21:25:10.311002+0000","last_unstale":"2024-09-10T21:25:10.315099+0000","last_undegraded":"2024-09-10T21:25:10.315099+0000","last_fullsized":"2024-09-10T21:25:10.315099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:01.124483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319251+0000","last_change":"2024-09-10T21:25:10.313464+0000","last_active":"2024-09-10T21:25:10.319251+0000","last_peered":"2024-09-10T21:25:10.319251+0000","last_clean":"2024-09-10T21:25:10.319251+0000","last_became_active":"2024-09-10T21:25:10.313179+0000","last_became_peered":"2024-09-10T21:25:10.313179+0000","last_unstale":"2024-09-10T21:25:10.319251+0000","last_undegraded":"2024-09-10T21:25:10.319251+0000","last_fullsized":"2024-09-10T21:25:10.319251+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:33.104694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312416+0000","last_change":"2024-09-10T21:25:10.323012+0000","last_active":"2024-09-10T21:25:11.312416+0000","last_peered":"2024-09-10T21:25:11.312416+0000","last_clean":"2024-09-10T21:25:11.312416+0000","last_became_active":"2024-09-10T21:25:10.322713+0000","last_became_peered":"2024-09-10T21:25:10.322713+0000","last_unstale":"2024-09-10T21:25:11.312416+0000","last_undegraded":"2024-09-10T21:25:11.312416+0000","last_fullsized":"2024-09-10T21:25:11.312416+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:52.771526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.303958+0000","last_change":"2024-09-10T21:25:10.298663+0000","last_active":"2024-09-10T21:25:10.303958+0000","last_peered":"2024-09-10T21:25:10.303958+0000","last_clean":"2024-09-10T21:25:10.303958+0000","last_became_active":"2024-09-10T21:25:10.298254+0000","last_became_peered":"2024-09-10T21:25:10.298254+0000","last_unstale":"2024-09-10T21:25:10.303958+0000","last_undegraded":"2024-09-10T21:25:10.303958+0000","last_fullsized":"2024-09-10T21:25:10.303958+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:21.046869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.331700+0000","last_change":"2024-09-10T21:25:10.321572+0000","last_active":"2024-09-10T21:25:10.331700+0000","last_peered":"2024-09-10T21:25:10.331700+0000","last_clean":"2024-09-10T21:25:10.331700+0000","last_became_active":"2024-09-10T21:25:10.321241+0000","last_became_peered":"2024-09-10T21:25:10.321241+0000","last_unstale":"2024-09-10T21:25:10.331700+0000","last_undegraded":"2024-09-10T21:25:10.331700+0000","last_fullsized":"2024-09-10T21:25:10.331700+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.783496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325131+0000","last_change":"2024-09-10T21:25:10.318363+0000","last_active":"2024-09-10T21:25:10.325131+0000","last_peered":"2024-09-10T21:25:10.325131+0000","last_clean":"2024-09-10T21:25:10.325131+0000","last_became_active":"2024-09-10T21:25:10.318063+0000","last_became_peered":"2024-09-10T21:25:10.318063+0000","last_unstale":"2024-09-10T21:25:10.325131+0000","last_undegraded":"2024-09-10T21:25:10.325131+0000","last_fullsized":"2024-09-10T21:25:10.325131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:29.632408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311953+0000","last_change":"2024-09-10T21:25:10.304998+0000","last_active":"2024-09-10T21:25:10.311953+0000","last_peered":"2024-09-10T21:25:10.311953+0000","last_clean":"2024-09-10T21:25:10.311953+0000","last_became_active":"2024-09-10T21:25:10.304727+0000","last_became_peered":"2024-09-10T21:25:10.304727+0000","last_unstale":"2024-09-10T21:25:10.311953+0000","last_undegraded":"2024-09-10T21:25:10.311953+0000","last_fullsized":"2024-09-10T21:25:10.311953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.315394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328564+0000","last_change":"2024-09-10T21:25:10.317776+0000","last_active":"2024-09-10T21:25:10.328564+0000","last_peered":"2024-09-10T21:25:10.328564+0000","last_clean":"2024-09-10T21:25:10.328564+0000","last_became_active":"2024-09-10T21:25:10.317377+0000","last_became_peered":"2024-09-10T21:25:10.317377+0000","last_unstale":"2024-09-10T21:25:10.328564+0000","last_undegraded":"2024-09-10T21:25:10.328564+0000","last_fullsized":"2024-09-10T21:25:10.328564+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:36.060550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.317262+0000","last_change":"2024-09-10T21:25:10.310314+0000","last_active":"2024-09-10T21:25:10.317262+0000","last_peered":"2024-09-10T21:25:10.317262+0000","last_clean":"2024-09-10T21:25:10.317262+0000","last_became_active":"2024-09-10T21:25:10.310035+0000","last_became_peered":"2024-09-10T21:25:10.310035+0000","last_unstale":"2024-09-10T21:25:10.317262+0000","last_undegraded":"2024-09-10T21:25:10.317262+0000","last_fullsized":"2024-09-10T21:25:10.317262+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:22.444236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.317934+0000","last_change":"2024-09-10T21:25:10.306305+0000","last_active":"2024-09-10T21:25:10.317934+0000","last_peered":"2024-09-10T21:25:10.317934+0000","last_clean":"2024-09-10T21:25:10.317934+0000","last_became_active":"2024-09-10T21:25:10.305939+0000","last_became_peered":"2024-09-10T21:25:10.305939+0000","last_unstale":"2024-09-10T21:25:10.317934+0000","last_undegraded":"2024-09-10T21:25:10.317934+0000","last_fullsized":"2024-09-10T21:25:10.317934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:10.375583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328602+0000","last_change":"2024-09-10T21:25:10.318444+0000","last_active":"2024-09-10T21:25:10.328602+0000","last_peered":"2024-09-10T21:25:10.328602+0000","last_clean":"2024-09-10T21:25:10.328602+0000","last_became_active":"2024-09-10T21:25:10.318208+0000","last_became_peered":"2024-09-10T21:25:10.318208+0000","last_unstale":"2024-09-10T21:25:10.328602+0000","last_undegraded":"2024-09-10T21:25:10.328602+0000","last_fullsized":"2024-09-10T21:25:10.328602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:06.858460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335257+0000","last_change":"2024-09-10T21:25:10.330124+0000","last_active":"2024-09-10T21:25:10.335257+0000","last_peered":"2024-09-10T21:25:10.335257+0000","last_clean":"2024-09-10T21:25:10.335257+0000","last_became_active":"2024-09-10T21:25:10.329878+0000","last_became_peered":"2024-09-10T21:25:10.329878+0000","last_unstale":"2024-09-10T21:25:10.335257+0000","last_undegraded":"2024-09-10T21:25:10.335257+0000","last_fullsized":"2024-09-10T21:25:10.335257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:52.357679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335699+0000","last_change":"2024-09-10T21:25:10.326748+0000","last_active":"2024-09-10T21:25:10.335699+0000","last_peered":"2024-09-10T21:25:10.335699+0000","last_clean":"2024-09-10T21:25:10.335699+0000","last_became_active":"2024-09-10T21:25:10.326350+0000","last_became_peered":"2024-09-10T21:25:10.326350+0000","last_unstale":"2024-09-10T21:25:10.335699+0000","last_undegraded":"2024-09-10T21:25:10.335699+0000","last_fullsized":"2024-09-10T21:25:10.335699+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:21.785416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319975+0000","last_change":"2024-09-10T21:25:10.315986+0000","last_active":"2024-09-10T21:25:10.319975+0000","last_peered":"2024-09-10T21:25:10.319975+0000","last_clean":"2024-09-10T21:25:10.319975+0000","last_became_active":"2024-09-10T21:25:10.315762+0000","last_became_peered":"2024-09-10T21:25:10.315762+0000","last_unstale":"2024-09-10T21:25:10.319975+0000","last_undegraded":"2024-09-10T21:25:10.319975+0000","last_fullsized":"2024-09-10T21:25:10.319975+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:32.179380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316483+0000","last_change":"2024-09-10T21:25:10.313229+0000","last_active":"2024-09-10T21:25:10.316483+0000","last_peered":"2024-09-10T21:25:10.316483+0000","last_clean":"2024-09-10T21:25:10.316483+0000","last_became_active":"2024-09-10T21:25:10.312958+0000","last_became_peered":"2024-09-10T21:25:10.312958+0000","last_unstale":"2024-09-10T21:25:10.316483+0000","last_undegraded":"2024-09-10T21:25:10.316483+0000","last_fullsized":"2024-09-10T21:25:10.316483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.863854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325007+0000","last_change":"2024-09-10T21:25:10.316928+0000","last_active":"2024-09-10T21:25:10.325007+0000","last_peered":"2024-09-10T21:25:10.325007+0000","last_clean":"2024-09-10T21:25:10.325007+0000","last_became_active":"2024-09-10T21:25:10.316618+0000","last_became_peered":"2024-09-10T21:25:10.316618+0000","last_unstale":"2024-09-10T21:25:10.325007+0000","last_undegraded":"2024-09-10T21:25:10.325007+0000","last_fullsized":"2024-09-10T21:25:10.325007+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:32.179380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316277+0000","last_change":"2024-09-10T21:25:10.310232+0000","last_active":"2024-09-10T21:25:10.316277+0000","last_peered":"2024-09-10T21:25:10.316277+0000","last_clean":"2024-09-10T21:25:10.316277+0000","last_became_active":"2024-09-10T21:25:10.309968+0000","last_became_peered":"2024-09-10T21:25:10.309968+0000","last_unstale":"2024-09-10T21:25:10.316277+0000","last_undegraded":"2024-09-10T21:25:10.316277+0000","last_fullsized":"2024-09-10T21:25:10.316277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:00.307962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.327601+0000","last_change":"2024-09-10T21:25:10.311739+0000","last_active":"2024-09-10T21:25:10.327601+0000","last_peered":"2024-09-10T21:25:10.327601+0000","last_clean":"2024-09-10T21:25:10.327601+0000","last_became_active":"2024-09-10T21:25:10.311487+0000","last_became_peered":"2024-09-10T21:25:10.311487+0000","last_unstale":"2024-09-10T21:25:10.327601+0000","last_undegraded":"2024-09-10T21:25:10.327601+0000","last_fullsized":"2024-09-10T21:25:10.327601+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:51.233168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312510+0000","last_change":"2024-09-10T21:25:10.317888+0000","last_active":"2024-09-10T21:25:11.312510+0000","last_peered":"2024-09-10T21:25:11.312510+0000","last_clean":"2024-09-10T21:25:11.312510+0000","last_became_active":"2024-09-10T21:25:10.316980+0000","last_became_peered":"2024-09-10T21:25:10.316980+0000","last_unstale":"2024-09-10T21:25:11.312510+0000","last_undegraded":"2024-09-10T21:25:11.312510+0000","last_fullsized":"2024-09-10T21:25:11.312510+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:03.936673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316410+0000","last_change":"2024-09-10T21:25:10.308826+0000","last_active":"2024-09-10T21:25:10.316410+0000","last_peered":"2024-09-10T21:25:10.316410+0000","last_clean":"2024-09-10T21:25:10.316410+0000","last_became_active":"2024-09-10T21:25:10.308552+0000","last_became_peered":"2024-09-10T21:25:10.308552+0000","last_unstale":"2024-09-10T21:25:10.316410+0000","last_undegraded":"2024-09-10T21:25:10.316410+0000","last_fullsized":"2024-09-10T21:25:10.316410+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:52.571926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314396+0000","last_change":"2024-09-10T21:25:10.311824+0000","last_active":"2024-09-10T21:25:11.314396+0000","last_peered":"2024-09-10T21:25:11.314396+0000","last_clean":"2024-09-10T21:25:11.314396+0000","last_became_active":"2024-09-10T21:25:10.311474+0000","last_became_peered":"2024-09-10T21:25:10.311474+0000","last_unstale":"2024-09-10T21:25:11.314396+0000","last_undegraded":"2024-09-10T21:25:11.314396+0000","last_fullsized":"2024-09-10T21:25:11.314396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:57.435532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325320+0000","last_change":"2024-09-10T21:25:10.320611+0000","last_active":"2024-09-10T21:25:10.325320+0000","last_peered":"2024-09-10T21:25:10.325320+0000","last_clean":"2024-09-10T21:25:10.325320+0000","last_became_active":"2024-09-10T21:25:10.320127+0000","last_became_peered":"2024-09-10T21:25:10.320127+0000","last_unstale":"2024-09-10T21:25:10.325320+0000","last_undegraded":"2024-09-10T21:25:10.325320+0000","last_fullsized":"2024-09-10T21:25:10.325320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:47.904295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.331836+0000","last_change":"2024-09-10T21:25:10.321195+0000","last_active":"2024-09-10T21:25:10.331836+0000","last_peered":"2024-09-10T21:25:10.331836+0000","last_clean":"2024-09-10T21:25:10.331836+0000","last_became_active":"2024-09-10T21:25:10.318727+0000","last_became_peered":"2024-09-10T21:25:10.318727+0000","last_unstale":"2024-09-10T21:25:10.331836+0000","last_undegraded":"2024-09-10T21:25:10.331836+0000","last_fullsized":"2024-09-10T21:25:10.331836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:10.293123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.324722+0000","last_change":"2024-09-10T21:25:10.312714+0000","last_active":"2024-09-10T21:25:10.324722+0000","last_peered":"2024-09-10T21:25:10.324722+0000","last_clean":"2024-09-10T21:25:10.324722+0000","last_became_active":"2024-09-10T21:25:10.312456+0000","last_became_peered":"2024-09-10T21:25:10.312456+0000","last_unstale":"2024-09-10T21:25:10.324722+0000","last_undegraded":"2024-09-10T21:25:10.324722+0000","last_fullsized":"2024-09-10T21:25:10.324722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:22.096627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.329642+0000","last_change":"2024-09-10T21:25:10.311184+0000","last_active":"2024-09-10T21:25:10.329642+0000","last_peered":"2024-09-10T21:25:10.329642+0000","last_clean":"2024-09-10T21:25:10.329642+0000","last_became_active":"2024-09-10T21:25:10.310908+0000","last_became_peered":"2024-09-10T21:25:10.310908+0000","last_unstale":"2024-09-10T21:25:10.329642+0000","last_undegraded":"2024-09-10T21:25:10.329642+0000","last_fullsized":"2024-09-10T21:25:10.329642+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:06.378712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319320+0000","last_change":"2024-09-10T21:25:10.313455+0000","last_active":"2024-09-10T21:25:10.319320+0000","last_peered":"2024-09-10T21:25:10.319320+0000","last_clean":"2024-09-10T21:25:10.319320+0000","last_became_active":"2024-09-10T21:25:10.313127+0000","last_became_peered":"2024-09-10T21:25:10.313127+0000","last_unstale":"2024-09-10T21:25:10.319320+0000","last_undegraded":"2024-09-10T21:25:10.319320+0000","last_fullsized":"2024-09-10T21:25:10.319320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:21.046869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312650+0000","last_change":"2024-09-10T21:25:10.314863+0000","last_active":"2024-09-10T21:25:11.312650+0000","last_peered":"2024-09-10T21:25:11.312650+0000","last_clean":"2024-09-10T21:25:11.312650+0000","last_became_active":"2024-09-10T21:25:10.314408+0000","last_became_peered":"2024-09-10T21:25:10.314408+0000","last_unstale":"2024-09-10T21:25:11.312650+0000","last_undegraded":"2024-09-10T21:25:11.312650+0000","last_fullsized":"2024-09-10T21:25:11.312650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:58.901169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.312090+0000","last_change":"2024-09-10T21:25:10.306524+0000","last_active":"2024-09-10T21:25:10.312090+0000","last_peered":"2024-09-10T21:25:10.312090+0000","last_clean":"2024-09-10T21:25:10.312090+0000","last_became_active":"2024-09-10T21:25:10.306215+0000","last_became_peered":"2024-09-10T21:25:10.306215+0000","last_unstale":"2024-09-10T21:25:10.312090+0000","last_undegraded":"2024-09-10T21:25:10.312090+0000","last_fullsized":"2024-09-10T21:25:10.312090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:53.589163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.336665+0000","last_change":"2024-09-10T21:25:09.294230+0000","last_active":"2024-09-10T21:25:10.336665+0000","last_peered":"2024-09-10T21:25:10.336665+0000","last_clean":"2024-09-10T21:25:10.336665+0000","last_became_active":"2024-09-10T21:25:09.293956+0000","last_became_peered":"2024-09-10T21:25:09.293956+0000","last_unstale":"2024-09-10T21:25:10.336665+0000","last_undegraded":"2024-09-10T21:25:10.336665+0000","last_fullsized":"2024-09-10T21:25:10.336665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:20.692813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.322806+0000","last_change":"2024-09-10T21:25:10.319177+0000","last_active":"2024-09-10T21:25:10.322806+0000","last_peered":"2024-09-10T21:25:10.322806+0000","last_clean":"2024-09-10T21:25:10.322806+0000","last_became_active":"2024-09-10T21:25:10.317996+0000","last_became_peered":"2024-09-10T21:25:10.317996+0000","last_unstale":"2024-09-10T21:25:10.322806+0000","last_undegraded":"2024-09-10T21:25:10.322806+0000","last_fullsized":"2024-09-10T21:25:10.322806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:21.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335862+0000","last_change":"2024-09-10T21:25:09.319663+0000","last_active":"2024-09-10T21:25:10.335862+0000","last_peered":"2024-09-10T21:25:10.335862+0000","last_clean":"2024-09-10T21:25:10.335862+0000","last_became_active":"2024-09-10T21:25:09.319435+0000","last_became_peered":"2024-09-10T21:25:09.319435+0000","last_unstale":"2024-09-10T21:25:10.335862+0000","last_undegraded":"2024-09-10T21:25:10.335862+0000","last_fullsized":"2024-09-10T21:25:10.335862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:51.565933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311303+0000","last_change":"2024-09-10T21:25:10.306708+0000","last_active":"2024-09-10T21:25:10.311303+0000","last_peered":"2024-09-10T21:25:10.311303+0000","last_clean":"2024-09-10T21:25:10.311303+0000","last_became_active":"2024-09-10T21:25:10.306429+0000","last_became_peered":"2024-09-10T21:25:10.306429+0000","last_unstale":"2024-09-10T21:25:10.311303+0000","last_undegraded":"2024-09-10T21:25:10.311303+0000","last_fullsized":"2024-09-10T21:25:10.311303+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:25.889472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313672+0000","last_change":"2024-09-10T21:25:09.309569+0000","last_active":"2024-09-10T21:25:11.313672+0000","last_peered":"2024-09-10T21:25:11.313672+0000","last_clean":"2024-09-10T21:25:11.313672+0000","last_became_active":"2024-09-10T21:25:09.309226+0000","last_became_peered":"2024-09-10T21:25:09.309226+0000","last_unstale":"2024-09-10T21:25:11.313672+0000","last_undegraded":"2024-09-10T21:25:11.313672+0000","last_fullsized":"2024-09-10T21:25:11.313672+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:59.647662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.322595+0000","last_change":"2024-09-10T21:25:10.317630+0000","last_active":"2024-09-10T21:25:10.322595+0000","last_peered":"2024-09-10T21:25:10.322595+0000","last_clean":"2024-09-10T21:25:10.322595+0000","last_became_active":"2024-09-10T21:25:10.317241+0000","last_became_peered":"2024-09-10T21:25:10.317241+0000","last_unstale":"2024-09-10T21:25:10.322595+0000","last_undegraded":"2024-09-10T21:25:10.322595+0000","last_fullsized":"2024-09-10T21:25:10.322595+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.863854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.330846+0000","last_change":"2024-09-10T21:25:09.306182+0000","last_active":"2024-09-10T21:25:10.330846+0000","last_peered":"2024-09-10T21:25:10.330846+0000","last_clean":"2024-09-10T21:25:10.330846+0000","last_became_active":"2024-09-10T21:25:09.305934+0000","last_became_peered":"2024-09-10T21:25:09.305934+0000","last_unstale":"2024-09-10T21:25:10.330846+0000","last_undegraded":"2024-09-10T21:25:10.330846+0000","last_fullsized":"2024-09-10T21:25:10.330846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:32.170499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312736+0000","last_change":"2024-09-10T21:25:10.318027+0000","last_active":"2024-09-10T21:25:11.312736+0000","last_peered":"2024-09-10T21:25:11.312736+0000","last_clean":"2024-09-10T21:25:11.312736+0000","last_became_active":"2024-09-10T21:25:10.317330+0000","last_became_peered":"2024-09-10T21:25:10.317330+0000","last_unstale":"2024-09-10T21:25:11.312736+0000","last_undegraded":"2024-09-10T21:25:11.312736+0000","last_fullsized":"2024-09-10T21:25:11.312736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:58.964847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.327782+0000","last_change":"2024-09-10T21:25:09.307932+0000","last_active":"2024-09-10T21:25:10.327782+0000","last_peered":"2024-09-10T21:25:10.327782+0000","last_clean":"2024-09-10T21:25:10.327782+0000","last_became_active":"2024-09-10T21:25:09.307629+0000","last_became_peered":"2024-09-10T21:25:09.307629+0000","last_unstale":"2024-09-10T21:25:10.327782+0000","last_undegraded":"2024-09-10T21:25:10.327782+0000","last_fullsized":"2024-09-10T21:25:10.327782+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:36.143619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319664+0000","last_change":"2024-09-10T21:25:10.312532+0000","last_active":"2024-09-10T21:25:10.319664+0000","last_peered":"2024-09-10T21:25:10.319664+0000","last_clean":"2024-09-10T21:25:10.319664+0000","last_became_active":"2024-09-10T21:25:10.312270+0000","last_became_peered":"2024-09-10T21:25:10.312270+0000","last_unstale":"2024-09-10T21:25:10.319664+0000","last_undegraded":"2024-09-10T21:25:10.319664+0000","last_fullsized":"2024-09-10T21:25:10.319664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:01.124483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335865+0000","last_change":"2024-09-10T21:25:09.304425+0000","last_active":"2024-09-10T21:25:10.335865+0000","last_peered":"2024-09-10T21:25:10.335865+0000","last_clean":"2024-09-10T21:25:10.335865+0000","last_became_active":"2024-09-10T21:25:09.303548+0000","last_became_peered":"2024-09-10T21:25:09.303548+0000","last_unstale":"2024-09-10T21:25:10.335865+0000","last_undegraded":"2024-09-10T21:25:10.335865+0000","last_fullsized":"2024-09-10T21:25:10.335865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:36.287869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.331719+0000","last_change":"2024-09-10T21:25:10.327423+0000","last_active":"2024-09-10T21:25:10.331719+0000","last_peered":"2024-09-10T21:25:10.331719+0000","last_clean":"2024-09-10T21:25:10.331719+0000","last_became_active":"2024-09-10T21:25:10.327148+0000","last_became_peered":"2024-09-10T21:25:10.327148+0000","last_unstale":"2024-09-10T21:25:10.331719+0000","last_undegraded":"2024-09-10T21:25:10.331719+0000","last_fullsized":"2024-09-10T21:25:10.331719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:15.356200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312528+0000","last_change":"2024-09-10T21:25:09.312771+0000","last_active":"2024-09-10T21:25:11.312528+0000","last_peered":"2024-09-10T21:25:11.312528+0000","last_clean":"2024-09-10T21:25:11.312528+0000","last_became_active":"2024-09-10T21:25:09.312584+0000","last_became_peered":"2024-09-10T21:25:09.312584+0000","last_unstale":"2024-09-10T21:25:11.312528+0000","last_undegraded":"2024-09-10T21:25:11.312528+0000","last_fullsized":"2024-09-10T21:25:11.312528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:29.845774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.315143+0000","last_change":"2024-09-10T21:25:10.309742+0000","last_active":"2024-09-10T21:25:10.315143+0000","last_peered":"2024-09-10T21:25:10.315143+0000","last_clean":"2024-09-10T21:25:10.315143+0000","last_became_active":"2024-09-10T21:25:10.309304+0000","last_became_peered":"2024-09-10T21:25:10.309304+0000","last_unstale":"2024-09-10T21:25:10.315143+0000","last_undegraded":"2024-09-10T21:25:10.315143+0000","last_fullsized":"2024-09-10T21:25:10.315143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:06.742437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.300325+0000","last_change":"2024-09-10T21:25:09.307419+0000","last_active":"2024-09-10T21:25:10.300325+0000","last_peered":"2024-09-10T21:25:10.300325+0000","last_clean":"2024-09-10T21:25:10.300325+0000","last_became_active":"2024-09-10T21:25:09.307145+0000","last_became_peered":"2024-09-10T21:25:09.307145+0000","last_unstale":"2024-09-10T21:25:10.300325+0000","last_undegraded":"2024-09-10T21:25:10.300325+0000","last_fullsized":"2024-09-10T21:25:10.300325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:46.430906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313180+0000","last_change":"2024-09-10T21:25:10.319762+0000","last_active":"2024-09-10T21:25:11.313180+0000","last_peered":"2024-09-10T21:25:11.313180+0000","last_clean":"2024-09-10T21:25:11.313180+0000","last_became_active":"2024-09-10T21:25:10.315185+0000","last_became_peered":"2024-09-10T21:25:10.315185+0000","last_unstale":"2024-09-10T21:25:11.313180+0000","last_undegraded":"2024-09-10T21:25:11.313180+0000","last_fullsized":"2024-09-10T21:25:11.313180+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:04.194994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314704+0000","last_change":"2024-09-10T21:25:09.300063+0000","last_active":"2024-09-10T21:25:11.314704+0000","last_peered":"2024-09-10T21:25:11.314704+0000","last_clean":"2024-09-10T21:25:11.314704+0000","last_became_active":"2024-09-10T21:25:09.299765+0000","last_became_peered":"2024-09-10T21:25:09.299765+0000","last_unstale":"2024-09-10T21:25:11.314704+0000","last_undegraded":"2024-09-10T21:25:11.314704+0000","last_fullsized":"2024-09-10T21:25:11.314704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:10.782413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328189+0000","last_change":"2024-09-10T21:25:10.316919+0000","last_active":"2024-09-10T21:25:10.328189+0000","last_peered":"2024-09-10T21:25:10.328189+0000","last_clean":"2024-09-10T21:25:10.328189+0000","last_became_active":"2024-09-10T21:25:10.316645+0000","last_became_peered":"2024-09-10T21:25:10.316645+0000","last_unstale":"2024-09-10T21:25:10.328189+0000","last_undegraded":"2024-09-10T21:25:10.328189+0000","last_fullsized":"2024-09-10T21:25:10.328189+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.051223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293441+0000","last_change":"2024-09-10T21:25:09.316339+0000","last_active":"2024-09-10T21:25:10.293441+0000","last_peered":"2024-09-10T21:25:10.293441+0000","last_clean":"2024-09-10T21:25:10.293441+0000","last_became_active":"2024-09-10T21:25:09.316026+0000","last_became_peered":"2024-09-10T21:25:09.316026+0000","last_unstale":"2024-09-10T21:25:10.293441+0000","last_undegraded":"2024-09-10T21:25:10.293441+0000","last_fullsized":"2024-09-10T21:25:10.293441+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:27.156379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.314603+0000","last_change":"2024-09-10T21:25:10.321352+0000","last_active":"2024-09-10T21:25:11.314603+0000","last_peered":"2024-09-10T21:25:11.314603+0000","last_clean":"2024-09-10T21:25:11.314603+0000","last_became_active":"2024-09-10T21:25:10.321028+0000","last_became_peered":"2024-09-10T21:25:10.321028+0000","last_unstale":"2024-09-10T21:25:11.314603+0000","last_undegraded":"2024-09-10T21:25:11.314603+0000","last_fullsized":"2024-09-10T21:25:11.314603+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:42.294158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.339409+0000","last_change":"2024-09-10T21:25:09.314657+0000","last_active":"2024-09-10T21:25:10.339409+0000","last_peered":"2024-09-10T21:25:10.339409+0000","last_clean":"2024-09-10T21:25:10.339409+0000","last_became_active":"2024-09-10T21:25:09.314339+0000","last_became_peered":"2024-09-10T21:25:09.314339+0000","last_unstale":"2024-09-10T21:25:10.339409+0000","last_undegraded":"2024-09-10T21:25:10.339409+0000","last_fullsized":"2024-09-10T21:25:10.339409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:39.109897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311932+0000","last_change":"2024-09-10T21:25:10.305685+0000","last_active":"2024-09-10T21:25:10.311932+0000","last_peered":"2024-09-10T21:25:10.311932+0000","last_clean":"2024-09-10T21:25:10.311932+0000","last_became_active":"2024-09-10T21:25:10.305473+0000","last_became_peered":"2024-09-10T21:25:10.305473+0000","last_unstale":"2024-09-10T21:25:10.311932+0000","last_undegraded":"2024-09-10T21:25:10.311932+0000","last_fullsized":"2024-09-10T21:25:10.311932+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:54.729961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.313255+0000","last_change":"2024-09-10T21:25:09.308217+0000","last_active":"2024-09-10T21:25:11.313255+0000","last_peered":"2024-09-10T21:25:11.313255+0000","last_clean":"2024-09-10T21:25:11.313255+0000","last_became_active":"2024-09-10T21:25:09.307894+0000","last_became_peered":"2024-09-10T21:25:09.307894+0000","last_unstale":"2024-09-10T21:25:11.313255+0000","last_undegraded":"2024-09-10T21:25:11.313255+0000","last_fullsized":"2024-09-10T21:25:11.313255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:52.032968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312888+0000","last_change":"2024-09-10T21:25:10.317608+0000","last_active":"2024-09-10T21:25:11.312888+0000","last_peered":"2024-09-10T21:25:11.312888+0000","last_clean":"2024-09-10T21:25:11.312888+0000","last_became_active":"2024-09-10T21:25:10.316630+0000","last_became_peered":"2024-09-10T21:25:10.316630+0000","last_unstale":"2024-09-10T21:25:11.312888+0000","last_undegraded":"2024-09-10T21:25:11.312888+0000","last_fullsized":"2024-09-10T21:25:11.312888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.897181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312729+0000","last_change":"2024-09-10T21:25:09.309657+0000","last_active":"2024-09-10T21:25:11.312729+0000","last_peered":"2024-09-10T21:25:11.312729+0000","last_clean":"2024-09-10T21:25:11.312729+0000","last_became_active":"2024-09-10T21:25:09.309352+0000","last_became_peered":"2024-09-10T21:25:09.309352+0000","last_unstale":"2024-09-10T21:25:11.312729+0000","last_undegraded":"2024-09-10T21:25:11.312729+0000","last_fullsized":"2024-09-10T21:25:11.312729+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:50.728255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.322780+0000","last_change":"2024-09-10T21:25:10.318156+0000","last_active":"2024-09-10T21:25:10.322780+0000","last_peered":"2024-09-10T21:25:10.322780+0000","last_clean":"2024-09-10T21:25:10.322780+0000","last_became_active":"2024-09-10T21:25:10.317819+0000","last_became_peered":"2024-09-10T21:25:10.317819+0000","last_unstale":"2024-09-10T21:25:10.322780+0000","last_undegraded":"2024-09-10T21:25:10.322780+0000","last_fullsized":"2024-09-10T21:25:10.322780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:32.179380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293965+0000","last_change":"2024-09-10T21:25:09.310648+0000","last_active":"2024-09-10T21:25:10.293965+0000","last_peered":"2024-09-10T21:25:10.293965+0000","last_clean":"2024-09-10T21:25:10.293965+0000","last_became_active":"2024-09-10T21:25:09.310206+0000","last_became_peered":"2024-09-10T21:25:09.310206+0000","last_unstale":"2024-09-10T21:25:10.293965+0000","last_undegraded":"2024-09-10T21:25:10.293965+0000","last_fullsized":"2024-09-10T21:25:10.293965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:32.170499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325146+0000","last_change":"2024-09-10T21:25:10.319417+0000","last_active":"2024-09-10T21:25:10.325146+0000","last_peered":"2024-09-10T21:25:10.325146+0000","last_clean":"2024-09-10T21:25:10.325146+0000","last_became_active":"2024-09-10T21:25:10.319148+0000","last_became_peered":"2024-09-10T21:25:10.319148+0000","last_unstale":"2024-09-10T21:25:10.325146+0000","last_undegraded":"2024-09-10T21:25:10.325146+0000","last_fullsized":"2024-09-10T21:25:10.325146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:33.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294230+0000","last_change":"2024-09-10T21:25:09.314970+0000","last_active":"2024-09-10T21:25:10.294230+0000","last_peered":"2024-09-10T21:25:10.294230+0000","last_clean":"2024-09-10T21:25:10.294230+0000","last_became_active":"2024-09-10T21:25:09.314586+0000","last_became_peered":"2024-09-10T21:25:09.314586+0000","last_unstale":"2024-09-10T21:25:10.294230+0000","last_undegraded":"2024-09-10T21:25:10.294230+0000","last_fullsized":"2024-09-10T21:25:10.294230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:20.786558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.312161+0000","last_change":"2024-09-10T21:25:10.305675+0000","last_active":"2024-09-10T21:25:10.312161+0000","last_peered":"2024-09-10T21:25:10.312161+0000","last_clean":"2024-09-10T21:25:10.312161+0000","last_became_active":"2024-09-10T21:25:10.305416+0000","last_became_peered":"2024-09-10T21:25:10.305416+0000","last_unstale":"2024-09-10T21:25:10.312161+0000","last_undegraded":"2024-09-10T21:25:10.312161+0000","last_fullsized":"2024-09-10T21:25:10.312161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:12.301442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":21,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.339011+0000","last_change":"2024-09-10T21:25:09.305368+0000","last_active":"2024-09-10T21:25:10.339011+0000","last_peered":"2024-09-10T21:25:10.339011+0000","last_clean":"2024-09-10T21:25:10.339011+0000","last_became_active":"2024-09-10T21:25:09.305110+0000","last_became_peered":"2024-09-10T21:25:09.305110+0000","last_unstale":"2024-09-10T21:25:10.339011+0000","last_undegraded":"2024-09-10T21:25:10.339011+0000","last_fullsized":"2024-09-10T21:25:10.339011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:38.473532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.315561+0000","last_change":"2024-09-10T21:25:10.311100+0000","last_active":"2024-09-10T21:25:10.315561+0000","last_peered":"2024-09-10T21:25:10.315561+0000","last_clean":"2024-09-10T21:25:10.315561+0000","last_became_active":"2024-09-10T21:25:10.310726+0000","last_became_peered":"2024-09-10T21:25:10.310726+0000","last_unstale":"2024-09-10T21:25:10.315561+0000","last_undegraded":"2024-09-10T21:25:10.315561+0000","last_fullsized":"2024-09-10T21:25:10.315561+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:00.307962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.336095+0000","last_change":"2024-09-10T21:25:09.311810+0000","last_active":"2024-09-10T21:25:10.336095+0000","last_peered":"2024-09-10T21:25:10.336095+0000","last_clean":"2024-09-10T21:25:10.336095+0000","last_became_active":"2024-09-10T21:25:09.311524+0000","last_became_peered":"2024-09-10T21:25:09.311524+0000","last_unstale":"2024-09-10T21:25:10.336095+0000","last_undegraded":"2024-09-10T21:25:10.336095+0000","last_fullsized":"2024-09-10T21:25:10.336095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:53.307462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.328597+0000","last_change":"2024-09-10T21:25:10.318304+0000","last_active":"2024-09-10T21:25:10.328597+0000","last_peered":"2024-09-10T21:25:10.328597+0000","last_clean":"2024-09-10T21:25:10.328597+0000","last_became_active":"2024-09-10T21:25:10.318030+0000","last_became_peered":"2024-09-10T21:25:10.318030+0000","last_unstale":"2024-09-10T21:25:10.328597+0000","last_undegraded":"2024-09-10T21:25:10.328597+0000","last_fullsized":"2024-09-10T21:25:10.328597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:07.441629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.339546+0000","last_change":"2024-09-10T21:25:09.314359+0000","last_active":"2024-09-10T21:25:10.339546+0000","last_peered":"2024-09-10T21:25:10.339546+0000","last_clean":"2024-09-10T21:25:10.339546+0000","last_became_active":"2024-09-10T21:25:09.314043+0000","last_became_peered":"2024-09-10T21:25:09.314043+0000","last_unstale":"2024-09-10T21:25:10.339546+0000","last_undegraded":"2024-09-10T21:25:10.339546+0000","last_fullsized":"2024-09-10T21:25:10.339546+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:35.683426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.321441+0000","last_change":"2024-09-10T21:25:10.313384+0000","last_active":"2024-09-10T21:25:10.321441+0000","last_peered":"2024-09-10T21:25:10.321441+0000","last_clean":"2024-09-10T21:25:10.321441+0000","last_became_active":"2024-09-10T21:25:10.313053+0000","last_became_peered":"2024-09-10T21:25:10.313053+0000","last_unstale":"2024-09-10T21:25:10.321441+0000","last_undegraded":"2024-09-10T21:25:10.321441+0000","last_fullsized":"2024-09-10T21:25:10.321441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:11.679983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.335205+0000","last_change":"2024-09-10T21:25:09.305697+0000","last_active":"2024-09-10T21:25:10.335205+0000","last_peered":"2024-09-10T21:25:10.335205+0000","last_clean":"2024-09-10T21:25:10.335205+0000","last_became_active":"2024-09-10T21:25:09.305437+0000","last_became_peered":"2024-09-10T21:25:09.305437+0000","last_unstale":"2024-09-10T21:25:10.335205+0000","last_undegraded":"2024-09-10T21:25:10.335205+0000","last_fullsized":"2024-09-10T21:25:10.335205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:29.845979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.327209+0000","last_change":"2024-09-10T21:25:10.321281+0000","last_active":"2024-09-10T21:25:10.327209+0000","last_peered":"2024-09-10T21:25:10.327209+0000","last_clean":"2024-09-10T21:25:10.327209+0000","last_became_active":"2024-09-10T21:25:10.321119+0000","last_became_peered":"2024-09-10T21:25:10.321119+0000","last_unstale":"2024-09-10T21:25:10.327209+0000","last_undegraded":"2024-09-10T21:25:10.327209+0000","last_fullsized":"2024-09-10T21:25:10.327209+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:21.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.306495+0000","last_change":"2024-09-10T21:25:09.310801+0000","last_active":"2024-09-10T21:25:10.306495+0000","last_peered":"2024-09-10T21:25:10.306495+0000","last_clean":"2024-09-10T21:25:10.306495+0000","last_became_active":"2024-09-10T21:25:09.310550+0000","last_became_peered":"2024-09-10T21:25:09.310550+0000","last_unstale":"2024-09-10T21:25:10.306495+0000","last_undegraded":"2024-09-10T21:25:10.306495+0000","last_fullsized":"2024-09-10T21:25:10.306495+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:00.118490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316219+0000","last_change":"2024-09-10T21:25:10.307738+0000","last_active":"2024-09-10T21:25:10.316219+0000","last_peered":"2024-09-10T21:25:10.316219+0000","last_clean":"2024-09-10T21:25:10.316219+0000","last_became_active":"2024-09-10T21:25:10.307423+0000","last_became_peered":"2024-09-10T21:25:10.307423+0000","last_unstale":"2024-09-10T21:25:10.316219+0000","last_undegraded":"2024-09-10T21:25:10.316219+0000","last_fullsized":"2024-09-10T21:25:10.316219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:23.261976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.330129+0000","last_change":"2024-09-10T21:25:09.302530+0000","last_active":"2024-09-10T21:25:10.330129+0000","last_peered":"2024-09-10T21:25:10.330129+0000","last_clean":"2024-09-10T21:25:10.330129+0000","last_became_active":"2024-09-10T21:25:09.302012+0000","last_became_peered":"2024-09-10T21:25:09.302012+0000","last_unstale":"2024-09-10T21:25:10.330129+0000","last_undegraded":"2024-09-10T21:25:10.330129+0000","last_fullsized":"2024-09-10T21:25:10.330129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:59.301968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312458+0000","last_change":"2024-09-10T21:25:10.325283+0000","last_active":"2024-09-10T21:25:11.312458+0000","last_peered":"2024-09-10T21:25:11.312458+0000","last_clean":"2024-09-10T21:25:11.312458+0000","last_became_active":"2024-09-10T21:25:10.325016+0000","last_became_peered":"2024-09-10T21:25:10.325016+0000","last_unstale":"2024-09-10T21:25:11.312458+0000","last_undegraded":"2024-09-10T21:25:11.312458+0000","last_fullsized":"2024-09-10T21:25:11.312458+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:18.795068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293735+0000","last_change":"2024-09-10T21:25:09.312281+0000","last_active":"2024-09-10T21:25:10.293735+0000","last_peered":"2024-09-10T21:25:10.293735+0000","last_clean":"2024-09-10T21:25:10.293735+0000","last_became_active":"2024-09-10T21:25:09.312048+0000","last_became_peered":"2024-09-10T21:25:09.312048+0000","last_unstale":"2024-09-10T21:25:10.293735+0000","last_undegraded":"2024-09-10T21:25:10.293735+0000","last_fullsized":"2024-09-10T21:25:10.293735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:20.692813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325079+0000","last_change":"2024-09-10T21:25:10.318224+0000","last_active":"2024-09-10T21:25:10.325079+0000","last_peered":"2024-09-10T21:25:10.325079+0000","last_clean":"2024-09-10T21:25:10.325079+0000","last_became_active":"2024-09-10T21:25:10.317857+0000","last_became_peered":"2024-09-10T21:25:10.317857+0000","last_unstale":"2024-09-10T21:25:10.325079+0000","last_undegraded":"2024-09-10T21:25:10.325079+0000","last_fullsized":"2024-09-10T21:25:10.325079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:30.320131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.300577+0000","last_change":"2024-09-10T21:25:09.305739+0000","last_active":"2024-09-10T21:25:10.300577+0000","last_peered":"2024-09-10T21:25:10.300577+0000","last_clean":"2024-09-10T21:25:10.300577+0000","last_became_active":"2024-09-10T21:25:09.305452+0000","last_became_peered":"2024-09-10T21:25:09.305452+0000","last_unstale":"2024-09-10T21:25:10.300577+0000","last_undegraded":"2024-09-10T21:25:10.300577+0000","last_fullsized":"2024-09-10T21:25:10.300577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:57.809201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312821+0000","last_change":"2024-09-10T21:25:10.321948+0000","last_active":"2024-09-10T21:25:11.312821+0000","last_peered":"2024-09-10T21:25:11.312821+0000","last_clean":"2024-09-10T21:25:11.312821+0000","last_became_active":"2024-09-10T21:25:10.321688+0000","last_became_peered":"2024-09-10T21:25:10.321688+0000","last_unstale":"2024-09-10T21:25:11.312821+0000","last_undegraded":"2024-09-10T21:25:11.312821+0000","last_fullsized":"2024-09-10T21:25:11.312821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:29.699913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.293187+0000","last_change":"2024-09-10T21:25:09.309438+0000","last_active":"2024-09-10T21:25:10.293187+0000","last_peered":"2024-09-10T21:25:10.293187+0000","last_clean":"2024-09-10T21:25:10.293187+0000","last_became_active":"2024-09-10T21:25:09.309145+0000","last_became_peered":"2024-09-10T21:25:09.309145+0000","last_unstale":"2024-09-10T21:25:10.293187+0000","last_undegraded":"2024-09-10T21:25:10.293187+0000","last_fullsized":"2024-09-10T21:25:10.293187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:01.976460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.316806+0000","last_change":"2024-09-10T21:25:10.313745+0000","last_active":"2024-09-10T21:25:10.316806+0000","last_peered":"2024-09-10T21:25:10.316806+0000","last_clean":"2024-09-10T21:25:10.316806+0000","last_became_active":"2024-09-10T21:25:10.313437+0000","last_became_peered":"2024-09-10T21:25:10.313437+0000","last_unstale":"2024-09-10T21:25:10.316806+0000","last_undegraded":"2024-09-10T21:25:10.316806+0000","last_fullsized":"2024-09-10T21:25:10.316806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:13.617229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.337755+0000","last_change":"2024-09-10T21:25:09.315503+0000","last_active":"2024-09-10T21:25:10.337755+0000","last_peered":"2024-09-10T21:25:10.337755+0000","last_clean":"2024-09-10T21:25:10.337755+0000","last_became_active":"2024-09-10T21:25:09.314377+0000","last_became_peered":"2024-09-10T21:25:09.314377+0000","last_unstale":"2024-09-10T21:25:10.337755+0000","last_undegraded":"2024-09-10T21:25:10.337755+0000","last_fullsized":"2024-09-10T21:25:10.337755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:18.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.325049+0000","last_change":"2024-09-10T21:25:10.318217+0000","last_active":"2024-09-10T21:25:10.325049+0000","last_peered":"2024-09-10T21:25:10.325049+0000","last_clean":"2024-09-10T21:25:10.325049+0000","last_became_active":"2024-09-10T21:25:10.317949+0000","last_became_peered":"2024-09-10T21:25:10.317949+0000","last_unstale":"2024-09-10T21:25:10.325049+0000","last_undegraded":"2024-09-10T21:25:10.325049+0000","last_fullsized":"2024-09-10T21:25:10.325049+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:21.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294717+0000","last_change":"2024-09-10T21:25:09.302821+0000","last_active":"2024-09-10T21:25:10.294717+0000","last_peered":"2024-09-10T21:25:10.294717+0000","last_clean":"2024-09-10T21:25:10.294717+0000","last_became_active":"2024-09-10T21:25:09.302539+0000","last_became_peered":"2024-09-10T21:25:09.302539+0000","last_unstale":"2024-09-10T21:25:10.294717+0000","last_undegraded":"2024-09-10T21:25:10.294717+0000","last_fullsized":"2024-09-10T21:25:10.294717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:12.695159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319812+0000","last_change":"2024-09-10T21:25:10.315253+0000","last_active":"2024-09-10T21:25:10.319812+0000","last_peered":"2024-09-10T21:25:10.319812+0000","last_clean":"2024-09-10T21:25:10.319812+0000","last_became_active":"2024-09-10T21:25:10.312954+0000","last_became_peered":"2024-09-10T21:25:10.312954+0000","last_unstale":"2024-09-10T21:25:10.319812+0000","last_undegraded":"2024-09-10T21:25:10.319812+0000","last_fullsized":"2024-09-10T21:25:10.319812+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.863854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294455+0000","last_change":"2024-09-10T21:25:09.303737+0000","last_active":"2024-09-10T21:25:10.294455+0000","last_peered":"2024-09-10T21:25:10.294455+0000","last_clean":"2024-09-10T21:25:10.294455+0000","last_became_active":"2024-09-10T21:25:09.303470+0000","last_became_peered":"2024-09-10T21:25:09.303470+0000","last_unstale":"2024-09-10T21:25:10.294455+0000","last_undegraded":"2024-09-10T21:25:10.294455+0000","last_fullsized":"2024-09-10T21:25:10.294455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:23.982363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.319188+0000","last_change":"2024-09-10T21:25:10.316389+0000","last_active":"2024-09-10T21:25:10.319188+0000","last_peered":"2024-09-10T21:25:10.319188+0000","last_clean":"2024-09-10T21:25:10.319188+0000","last_became_active":"2024-09-10T21:25:10.316086+0000","last_became_peered":"2024-09-10T21:25:10.316086+0000","last_unstale":"2024-09-10T21:25:10.319188+0000","last_undegraded":"2024-09-10T21:25:10.319188+0000","last_fullsized":"2024-09-10T21:25:10.319188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:42.986151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312898+0000","last_change":"2024-09-10T21:25:09.309626+0000","last_active":"2024-09-10T21:25:11.312898+0000","last_peered":"2024-09-10T21:25:11.312898+0000","last_clean":"2024-09-10T21:25:11.312898+0000","last_became_active":"2024-09-10T21:25:09.309248+0000","last_became_peered":"2024-09-10T21:25:09.309248+0000","last_unstale":"2024-09-10T21:25:11.312898+0000","last_undegraded":"2024-09-10T21:25:11.312898+0000","last_fullsized":"2024-09-10T21:25:11.312898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:59.924842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.306830+0000","last_change":"2024-09-10T21:25:10.298644+0000","last_active":"2024-09-10T21:25:10.306830+0000","last_peered":"2024-09-10T21:25:10.306830+0000","last_clean":"2024-09-10T21:25:10.306830+0000","last_became_active":"2024-09-10T21:25:10.298318+0000","last_became_peered":"2024-09-10T21:25:10.298318+0000","last_unstale":"2024-09-10T21:25:10.306830+0000","last_undegraded":"2024-09-10T21:25:10.306830+0000","last_fullsized":"2024-09-10T21:25:10.306830+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:22.444236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.303528+0000","last_change":"2024-09-10T21:25:09.309411+0000","last_active":"2024-09-10T21:25:10.303528+0000","last_peered":"2024-09-10T21:25:10.303528+0000","last_clean":"2024-09-10T21:25:10.303528+0000","last_became_active":"2024-09-10T21:25:09.309126+0000","last_became_peered":"2024-09-10T21:25:09.309126+0000","last_unstale":"2024-09-10T21:25:10.303528+0000","last_undegraded":"2024-09-10T21:25:10.303528+0000","last_fullsized":"2024-09-10T21:25:10.303528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:22.045229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.311752+0000","last_change":"2024-09-10T21:25:10.307503+0000","last_active":"2024-09-10T21:25:10.311752+0000","last_peered":"2024-09-10T21:25:10.311752+0000","last_clean":"2024-09-10T21:25:10.311752+0000","last_became_active":"2024-09-10T21:25:10.307238+0000","last_became_peered":"2024-09-10T21:25:10.307238+0000","last_unstale":"2024-09-10T21:25:10.311752+0000","last_undegraded":"2024-09-10T21:25:10.311752+0000","last_fullsized":"2024-09-10T21:25:10.311752+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:02.809894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.294358+0000","last_change":"2024-09-10T21:25:09.312813+0000","last_active":"2024-09-10T21:25:10.294358+0000","last_peered":"2024-09-10T21:25:10.294358+0000","last_clean":"2024-09-10T21:25:10.294358+0000","last_became_active":"2024-09-10T21:25:09.312511+0000","last_became_peered":"2024-09-10T21:25:09.312511+0000","last_unstale":"2024-09-10T21:25:10.294358+0000","last_undegraded":"2024-09-10T21:25:10.294358+0000","last_fullsized":"2024-09-10T21:25:10.294358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:13.973679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.334988+0000","last_change":"2024-09-10T21:25:10.323177+0000","last_active":"2024-09-10T21:25:10.334988+0000","last_peered":"2024-09-10T21:25:10.334988+0000","last_clean":"2024-09-10T21:25:10.334988+0000","last_became_active":"2024-09-10T21:25:10.320099+0000","last_became_peered":"2024-09-10T21:25:10.320099+0000","last_unstale":"2024-09-10T21:25:10.334988+0000","last_undegraded":"2024-09-10T21:25:10.334988+0000","last_fullsized":"2024-09-10T21:25:10.334988+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:19.572091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T21:25:11.312810+0000","last_change":"2024-09-10T21:25:09.314378+0000","last_active":"2024-09-10T21:25:11.312810+0000","last_peered":"2024-09-10T21:25:11.312810+0000","last_clean":"2024-09-10T21:25:11.312810+0000","last_became_active":"2024-09-10T21:25:09.314138+0000","last_became_peered":"2024-09-10T21:25:09.314138+0000","last_unstale":"2024-09-10T21:25:11.312810+0000","last_undegraded":"2024-09-10T21:25:11.312810+0000","last_fullsized":"2024-09-10T21:25:11.312810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:13.188324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.324765+0000","last_change":"2024-09-10T21:25:10.311534+0000","last_active":"2024-09-10T21:25:10.324765+0000","last_peered":"2024-09-10T21:25:10.324765+0000","last_clean":"2024-09-10T21:25:10.324765+0000","last_became_active":"2024-09-10T21:25:10.310769+0000","last_became_peered":"2024-09-10T21:25:10.310769+0000","last_unstale":"2024-09-10T21:25:10.324765+0000","last_undegraded":"2024-09-10T21:25:10.324765+0000","last_fullsized":"2024-09-10T21:25:10.324765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:09.278181+0000","last_clean_scrub_stamp":"2024-09-10T21:25:09.278181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:13.433230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.304039+0000","last_change":"2024-09-10T21:25:09.301797+0000","last_active":"2024-09-10T21:25:10.304039+0000","last_peered":"2024-09-10T21:25:10.304039+0000","last_clean":"2024-09-10T21:25:10.304039+0000","last_became_active":"2024-09-10T21:25:09.301602+0000","last_became_peered":"2024-09-10T21:25:09.301602+0000","last_unstale":"2024-09-10T21:25:10.304039+0000","last_undegraded":"2024-09-10T21:25:10.304039+0000","last_fullsized":"2024-09-10T21:25:10.304039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:25:08.272188+0000","last_clean_scrub_stamp":"2024-09-10T21:25:08.272188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:05.736443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":143,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-10T21:25:10.300205+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:25:10.300205+0000","last_peered":"2024-09-10T21:25:10.300205+0000","last_clean":"2024-09-10T21:25:10.300205+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:25:10.300205+0000","last_undegraded":"2024-09-10T21:25:10.300205+0000","last_fullsized":"2024-09-10T21:25:10.300205+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1040384,"data_stored":779040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":458752,"data_stored":190284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1040384,"data_stored":778480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":450560,"data_stored":188182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":196518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:14.807 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:25:14.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:25:14.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- 172.21.15.97:0/3242390406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 msgr2=0x7eff3c005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 --2- 172.21.15.97:0/3242390406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 0x7eff3c005240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7eff5000bb10 tx=0x7eff5001cde0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57e00640 1 -- 172.21.15.97:0/3242390406 reap_dead start 2024-09-10T21:25:14.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- 172.21.15.97:0/3242390406 shutdown_connections 2024-09-10T21:25:14.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- 172.21.15.97:0/3242390406 >> 172.21.15.97:0/3242390406 conn(0x7eff580d1ca0 msgr2=0x7eff580d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- 172.21.15.97:0/3242390406 shutdown_connections 2024-09-10T21:25:14.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- 172.21.15.97:0/3242390406 wait complete. 2024-09-10T21:25:14.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 Processor -- start 2024-09-10T21:25:14.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- start start 2024-09-10T21:25:15.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 0x7eff580cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff580b7100 0x7eff5809f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff5815c7e0 0x7eff5815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff580cf250 con 0x7eff3c004dc0 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff580cf0d0 con 0x7eff580b7100 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff5e400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7eff580cf3d0 con 0x7eff5815c7e0 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 0x7eff580cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 0x7eff580cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41308/0 (socket says 172.21.15.97:41308) 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 -- 172.21.15.97:0/764294024 learned_addr learned my addr 172.21.15.97:0/764294024 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff4ea00640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff580b7100 0x7eff5809f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57e00640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff5815c7e0 0x7eff5815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 -- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff5815c7e0 msgr2=0x7eff5815cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff5815c7e0 0x7eff5815cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 -- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff580b7100 msgr2=0x7eff5809f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff580b7100 0x7eff5809f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57400640 1 -- 172.21.15.97:0/764294024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff581f4940 con 0x7eff3c004dc0 2024-09-10T21:25:15.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff57e00640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff5815c7e0 0x7eff5815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:15.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.992+0000 7eff4ea00640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff580b7100 0x7eff5809f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:15.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff57400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 0x7eff580cb780 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7eff4800cad0 tx=0x7eff4800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:15.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff4fe00640 1 -- 172.21.15.97:0/764294024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48003b80 con 0x7eff3c004dc0 2024-09-10T21:25:15.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff4fe00640 1 -- 172.21.15.97:0/764294024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff48003d40 con 0x7eff3c004dc0 2024-09-10T21:25:15.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff4fe00640 1 -- 172.21.15.97:0/764294024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4800abd0 con 0x7eff3c004dc0 2024-09-10T21:25:15.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff581f4c20 con 0x7eff3c004dc0 2024-09-10T21:25:15.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff581f51e0 con 0x7eff3c004dc0 2024-09-10T21:25:15.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff580b2010 con 0x7eff3c004dc0 2024-09-10T21:25:15.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff4fe00640 1 -- 172.21.15.97:0/764294024 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff480050d0 con 0x7eff3c004dc0 2024-09-10T21:25:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff4fe00640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff4fe00640 1 -- 172.21.15.97:0/764294024 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7eff48093440 con 0x7eff3c004dc0 2024-09-10T21:25:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:14.996+0000 7eff4ea00640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.000+0000 7eff4ea00640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff40001230 tx=0x7eff4000c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:15.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.000+0000 7eff4fe00640 1 -- 172.21.15.97:0/764294024 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff4805ea20 con 0x7eff3c004dc0 2024-09-10T21:25:15.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.224+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7eff58072430 con 0x7eff3c004dc0 2024-09-10T21:25:15.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.228+0000 7eff4fe00640 1 -- 172.21.15.97:0/764294024 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7eff4805e320 con 0x7eff3c004dc0 2024-09-10T21:25:15.235 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-10T21:25:15.235 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:25:15.235 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-09-10T21:25:15.235 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-09-10T21:25:15.235 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:25:15.235 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 msgr2=0x7eff34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff40001230 tx=0x7eff4000c040 comp rx=0 tx=0).stop 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 msgr2=0x7eff580cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 0x7eff580cb780 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7eff4800cad0 tx=0x7eff4800cf90 comp rx=0 tx=0).stop 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 shutdown_connections 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff5815c7e0 0x7eff5815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff580b7100 0x7eff5809f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 --2- 172.21.15.97:0/764294024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff3c004dc0 0x7eff580cb780 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 >> 172.21.15.97:0/764294024 conn(0x7eff580d1ca0 msgr2=0x7eff580d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 shutdown_connections 2024-09-10T21:25:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.232+0000 7eff5e400640 1 -- 172.21.15.97:0/764294024 wait complete. 2024-09-10T21:25:15.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:15.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 -- 172.21.15.97:0/659562156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 msgr2=0x7f77b40b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 --2- 172.21.15.97:0/659562156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b40b37b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009840 tx=0x7f77a401f010 comp rx=0 tx=0).stop 2024-09-10T21:25:15.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 -- 172.21.15.97:0/659562156 shutdown_connections 2024-09-10T21:25:15.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 --2- 172.21.15.97:0/659562156 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77b40b3cf0 0x7f77b406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 --2- 172.21.15.97:0/659562156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b40b37b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 -- 172.21.15.97:0/659562156 >> 172.21.15.97:0/659562156 conn(0x7f77b40b1780 msgr2=0x7f77b40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:15.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 -- 172.21.15.97:0/659562156 shutdown_connections 2024-09-10T21:25:15.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.388+0000 7f77b8e00640 1 -- 172.21.15.97:0/659562156 wait complete. 2024-09-10T21:25:15.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 Processor -- start 2024-09-10T21:25:15.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 -- start start 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77b406f180 0x7f77b414c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77b40b3cf0 0x7f77b413e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77b406e740 con 0x7f77b41511f0 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77b406e5c0 con 0x7f77b40b3cf0 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f77b406e8c0 con 0x7f77b406f180 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b415bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41336/0 (socket says 172.21.15.97:41336) 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 -- 172.21.15.97:0/1747098761 learned_addr learned my addr 172.21.15.97:0/1747098761 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:15.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77a9400640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77b40b3cf0 0x7f77b413e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b1e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77b406f180 0x7f77b414c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 -- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77b406f180 msgr2=0x7f77b414c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77b406f180 0x7f77b414c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 -- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77b40b3cf0 msgr2=0x7f77b413e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77b40b3cf0 0x7f77b413e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 -- 172.21.15.97:0/1747098761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b414c610 con 0x7f77b41511f0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b2800640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b415bec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f77ac002240 tx=0x7f77ac00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77aaa00640 1 -- 172.21.15.97:0/1747098761 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac017070 con 0x7f77b41511f0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77aaa00640 1 -- 172.21.15.97:0/1747098761 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77ac003c70 con 0x7f77b41511f0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b41f8340 con 0x7f77b41511f0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77aaa00640 1 -- 172.21.15.97:0/1747098761 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0052d0 con 0x7f77b41511f0 2024-09-10T21:25:15.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.392+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b41f8900 con 0x7f77b41511f0 2024-09-10T21:25:15.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.396+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b406ec40 con 0x7f77b41511f0 2024-09-10T21:25:15.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.396+0000 7f77aaa00640 1 -- 172.21.15.97:0/1747098761 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f77ac005680 con 0x7f77b41511f0 2024-09-10T21:25:15.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.396+0000 7f77aaa00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7790072610 0x7f7790074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.396+0000 7f77aaa00640 1 -- 172.21.15.97:0/1747098761 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f77ac092e50 con 0x7f77b41511f0 2024-09-10T21:25:15.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.396+0000 7f77b1e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7790072610 0x7f7790074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.396+0000 7f77b1e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7790072610 0x7f7790074ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77a40b2000 tx=0x7f77a401d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:15.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.400+0000 7f77aaa00640 1 -- 172.21.15.97:0/1747098761 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77ac05dd40 con 0x7f77b41511f0 2024-09-10T21:25:15.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b415ce10 con 0x7f77b41511f0 2024-09-10T21:25:15.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77aaa00640 1 -- 172.21.15.97:0/1747098761 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f77ac062770 con 0x7f77b41511f0 2024-09-10T21:25:15.606 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:15.606 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"btime":"2024-09-10T21:25:13:316162+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:12.695121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/2127253870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2127253870},{"type":"v1","addr":"172.21.15.97:6835","nonce":2127253870}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4932,"qdb_cluster":[4932]},"id":2}]} 2024-09-10T21:25:15.606 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-09-10T21:25:15.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7790072610 msgr2=0x7f7790074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7790072610 0x7f7790074ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77a40b2000 tx=0x7f77a401d850 comp rx=0 tx=0).stop 2024-09-10T21:25:15.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 msgr2=0x7f77b415bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b415bec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f77ac002240 tx=0x7f77ac00cb90 comp rx=0 tx=0).stop 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 shutdown_connections 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7790072610 0x7f7790074ad0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77b41511f0 0x7f77b415bec0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77b40b3cf0 0x7f77b413e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 --2- 172.21.15.97:0/1747098761 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f77b406f180 0x7f77b414c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 >> 172.21.15.97:0/1747098761 conn(0x7f77b40b1780 msgr2=0x7f77b415a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 shutdown_connections 2024-09-10T21:25:15.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:15.600+0000 7f77b8e00640 1 -- 172.21.15.97:0/1747098761 wait complete. 2024-09-10T21:25:16.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:25:09.683475+0000', 'modified': '2024-09-10T21:25:12.695121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4932}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4932': {'gid': 4932, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6835/2127253870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 2127253870}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2127253870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4932, 'qdb_cluster': [4932]} 2024-09-10T21:25:16.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:25:16.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:25:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 -- 172.21.15.97:0/607992332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 msgr2=0x7f86e00cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 --2- 172.21.15.97:0/607992332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e00cf360 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009c80 tx=0x7f86d801cd10 comp rx=0 tx=0).stop 2024-09-10T21:25:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 -- 172.21.15.97:0/607992332 shutdown_connections 2024-09-10T21:25:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 --2- 172.21.15.97:0/607992332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e00cf360 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 --2- 172.21.15.97:0/607992332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f86e015bbe0 0x7f86e015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 -- 172.21.15.97:0/607992332 >> 172.21.15.97:0/607992332 conn(0x7f86e00b2ee0 msgr2=0x7f86e00d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 -- 172.21.15.97:0/607992332 shutdown_connections 2024-09-10T21:25:16.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.168+0000 7f86df200640 1 -- 172.21.15.97:0/607992332 wait complete. 2024-09-10T21:25:16.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 Processor -- start 2024-09-10T21:25:16.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 -- start start 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f86e0155090 0x7f86e00aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86e015bbe0 0x7f86e01f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e01fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86e00afc40 con 0x7f86e015c5d0 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f86e00afac0 con 0x7f86e015bbe0 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f86e00afdc0 con 0x7f86e0155090 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e01fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e01fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41356/0 (socket says 172.21.15.97:41356) 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 -- 172.21.15.97:0/814621472 learned_addr learned my addr 172.21.15.97:0/814621472 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 -- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f86e0155090 msgr2=0x7f86e00aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f86e0155090 0x7f86e00aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 -- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86e015bbe0 msgr2=0x7f86e01f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86e015bbe0 0x7f86e01f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 -- 172.21.15.97:0/814621472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e01f8360 con 0x7f86e015c5d0 2024-09-10T21:25:16.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86de800640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e01fc250 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86d801b3c0 tx=0x7f86d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86d6000640 1 -- 172.21.15.97:0/814621472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d80bb070 con 0x7f86e015c5d0 2024-09-10T21:25:16.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86d6000640 1 -- 172.21.15.97:0/814621472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86d8004a50 con 0x7f86e015c5d0 2024-09-10T21:25:16.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86d6000640 1 -- 172.21.15.97:0/814621472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d80b2040 con 0x7f86e015c5d0 2024-09-10T21:25:16.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e01fc790 con 0x7f86e015c5d0 2024-09-10T21:25:16.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.172+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e01f85d0 con 0x7f86e015c5d0 2024-09-10T21:25:16.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86d6000640 1 -- 172.21.15.97:0/814621472 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f86d80b5030 con 0x7f86e015c5d0 2024-09-10T21:25:16.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 Processor -- start 2024-09-10T21:25:16.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86d6000640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86c00725e0 0x7f86c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 -- start start 2024-09-10T21:25:16.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86d6000640 1 -- 172.21.15.97:0/814621472 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f86d812cfa0 con 0x7f86e015c5d0 2024-09-10T21:25:16.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86c00725e0 0x7f86c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ddbf993f0 con 0x556ddc0c52f0 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ddbed6b20 con 0x556ddc078e10 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556ddbec5f50 con 0x556ddc0c49d0 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86c00725e0 0x7f86c0074aa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86e006e380 tx=0x7f86cc009640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0788d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41358/0 (socket says 172.21.15.97:41358) 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 -- 172.21.15.97:0/330709523 learned_addr learned my addr 172.21.15.97:0/330709523 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dde00640 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86de800640 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 -- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 msgr2=0x556ddc0c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 -- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 msgr2=0x556ddc0791f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0791f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.176+0000 7f86dd400640 1 -- 172.21.15.97:0/330709523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ddbe8d590 con 0x556ddc0c52f0 2024-09-10T21:25:16.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86dd400640 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0788d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00ed40 tx=0x7f86cc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d15d028920f467 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86caa00640 1 -- 172.21.15.97:0/330709523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc020070 con 0x556ddc0c52f0 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86caa00640 1 -- 172.21.15.97:0/330709523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86cc00ed70 con 0x556ddc0c52f0 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86caa00640 1 -- 172.21.15.97:0/330709523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0256c0 con 0x556ddc0c52f0 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- 172.21.15.97:0/330709523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 msgr2=0x556ddc0788d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0788d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00ed40 tx=0x7f86cc011000 comp rx=0 tx=0).stop 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- 172.21.15.97:0/330709523 shutdown_connections 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0788d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/330709523 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- 172.21.15.97:0/330709523 >> 172.21.15.97:0/330709523 conn(0x556ddbf62130 msgr2=0x556ddc0a7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- 172.21.15.97:0/330709523 shutdown_connections 2024-09-10T21:25:16.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- 172.21.15.97:0/330709523 wait complete. 2024-09-10T21:25:16.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- start start 2024-09-10T21:25:16.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 Processor -- start 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ddbe605d0 con 0x556ddc0c52f0 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ddbec9d60 con 0x556ddc078e10 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556ddbe6c500 con 0x556ddc0c49d0 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86e7f3c000 10 client.0 _ll_register_callbacks cb 0x556ddbe70810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41368/0 (socket says 172.21.15.97:41368) 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 -- 172.21.15.97:0/252468084 learned_addr learned my addr 172.21.15.97:0/252468084 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:16.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86dde00640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86dd400640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 -- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 msgr2=0x556ddc0a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 -- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 msgr2=0x556ddc0a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86de800640 1 -- 172.21.15.97:0/252468084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ddbe64170 con 0x556ddc0c52f0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.180+0000 7f86dd400640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86de800640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0a63b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f86d801c780 tx=0x7f86d80f7b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dfec2a1097321d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 1 -- 172.21.15.97:0/252468084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d80f7860 con 0x556ddc0c52f0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 1 -- 172.21.15.97:0/252468084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f86d80f6fb0 con 0x556ddc0c52f0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ddbe5e0f0 con 0x556ddc0c52f0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 1 -- 172.21.15.97:0/252468084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f86d80f8450 con 0x556ddc0c52f0 2024-09-10T21:25:16.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 10 client.5100 fetch_fsmap learned FSMap version 18 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x556ddbf4aef0 con 0x556ddc0c52f0 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 1 -- 172.21.15.97:0/252468084 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556ddbe5e0f0 con 0x556ddc0c52f0 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 1 -- 172.21.15.97:0/252468084 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f86e00cdad0 con 0x556ddc0c52f0 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 10 client.5100 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 10 client.5100 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 20 client.5100 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0a70f0 0x556ddc0d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 4 client.5100 mds_command: new command op to 4932 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 --> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ddbed8a70 con 0x556ddc0a70f0 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86dde00640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0a70f0 0x556ddc0d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86dde00640 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0a70f0 0x556ddc0d6c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc152536f02b5923 server_cookie=84abe0324d6ab815 in_seq=0 out_seq=0 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 10 client.5100 ms_handle_connect on v2:172.21.15.97:6832/2127253870 2024-09-10T21:25:16.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 1 -- 172.21.15.97:0/252468084 <== mds.0 v2:172.21.15.97:6832/2127253870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ddbed8a70 con 0x556ddc0a70f0 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 10 client.5100 handle_command_reply: tid=0 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 client.5100 shutdown 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 msgr2=0x556ddc0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0a63b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f86d801c780 tx=0x7f86d80f7b20 comp rx=0 tx=0).stop 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 shutdown_connections 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0a70f0 0x556ddc0d6c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c52f0 0x556ddc0a63b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c49d0 0x556ddc0a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/252468084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc0a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86bf400640 0 client.5100 ms_handle_reset on v2:172.21.15.97:6832/2127253870 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 >> 172.21.15.97:0/252468084 conn(0x556ddbf62130 msgr2=0x556ddc077fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 shutdown_connections 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 1 -- 172.21.15.97:0/252468084 wait complete. 2024-09-10T21:25:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.184+0000 7f86e7f3c000 20 client.5100 trim_cache size 0 max 16384 2024-09-10T21:25:16.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 Processor -- start 2024-09-10T21:25:16.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 -- start start 2024-09-10T21:25:16.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc078e10 0x556ddc13f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc0c52f0 0x556ddc140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0204900 con 0x556ddc0c49d0 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f86e00703a0 con 0x556ddc0c52f0 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f86e006f6e0 con 0x556ddc078e10 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc13fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41378/0 (socket says 172.21.15.97:41378) 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 -- 172.21.15.97:0/3051898071 learned_addr learned my addr 172.21.15.97:0/3051898071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 -- 172.21.15.97:0/3051898071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc078e10 msgr2=0x556ddc13f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 --2- 172.21.15.97:0/3051898071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc078e10 0x556ddc13f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 -- 172.21.15.97:0/3051898071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc0c52f0 msgr2=0x556ddc140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 --2- 172.21.15.97:0/3051898071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc0c52f0 0x556ddc140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 -- 172.21.15.97:0/3051898071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ddbf21d90 con 0x556ddc0c49d0 2024-09-10T21:25:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86dd400640 1 --2- 172.21.15.97:0/3051898071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc13fd30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f86cc021000 tx=0x7f86cc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4939c40bba33435 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86ca000640 1 -- 172.21.15.97:0/3051898071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc033070 con 0x556ddc0c49d0 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86ca000640 1 -- 172.21.15.97:0/3051898071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86cc00ed70 con 0x556ddc0c49d0 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.204+0000 7f86ca000640 1 -- 172.21.15.97:0/3051898071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc020040 con 0x556ddc0c49d0 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/3051898071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 msgr2=0x556ddc13fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/3051898071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc13fd30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f86cc021000 tx=0x7f86cc0046d0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/3051898071 shutdown_connections 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/3051898071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc0c52f0 0x556ddc140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/3051898071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc13fd30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/3051898071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc078e10 0x556ddc13f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/3051898071 >> 172.21.15.97:0/3051898071 conn(0x556ddbf62130 msgr2=0x556ddc080240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/3051898071 shutdown_connections 2024-09-10T21:25:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/3051898071 wait complete. 2024-09-10T21:25:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- start start 2024-09-10T21:25:16.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 Processor -- start 2024-09-10T21:25:16.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc143620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc143b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c52f0 0x556ddc1440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ddbe91ec0 con 0x556ddc0c49d0 2024-09-10T21:25:16.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f86e006fd30 con 0x556ddc078e10 2024-09-10T21:25:16.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556ddbe605d0 con 0x556ddc0c52f0 2024-09-10T21:25:16.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 10 client.0 _ll_register_callbacks cb 0x7f86d80f43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc143b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41388/0 (socket says 172.21.15.97:41388) 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 -- 172.21.15.97:0/622252004 learned_addr learned my addr 172.21.15.97:0/622252004 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86de800640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c52f0 0x556ddc1440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dde00640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 -- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c52f0 msgr2=0x556ddc1440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c52f0 0x556ddc1440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 -- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 msgr2=0x556ddc143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 -- 172.21.15.97:0/622252004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ddbe8ca90 con 0x556ddc0c49d0 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dde00640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86dd400640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc143b60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x556ddbf203c0 tx=0x7f86cc02ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d3b3362de979fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86bf400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:16.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86bf400640 1 -- 172.21.15.97:0/622252004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc033070 con 0x556ddc0c49d0 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86bf400640 1 -- 172.21.15.97:0/622252004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f86cc038460 con 0x556ddc0c49d0 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86cc00ed70 con 0x556ddc0c49d0 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86bf400640 1 -- 172.21.15.97:0/622252004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f86cc040050 con 0x556ddc0c49d0 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 10 client.5106 fetch_fsmap learned FSMap version 18 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x556ddbe6b050 con 0x556ddc0c49d0 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86bf400640 1 -- 172.21.15.97:0/622252004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86cc00ed70 con 0x556ddc0c49d0 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86bf400640 1 -- 172.21.15.97:0/622252004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f86cc043020 con 0x556ddc0c49d0 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 10 client.5106 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 10 client.5106 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 20 client.5106 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0d8c10 0x556ddc0d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 4 client.5106 mds_command: new command op to 4932 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:25:16.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.208+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 --> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] -- command(tid 0: {"prefix": "status"}) -- 0x556ddbed4520 con 0x556ddc0d8c10 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86dde00640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0d8c10 0x556ddc0d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86dde00640 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0d8c10 0x556ddc0d8ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61783d78b4720f2 server_cookie=ead2c85dbd798688 in_seq=0 out_seq=0 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86bf400640 10 client.5106 ms_handle_connect on v2:172.21.15.97:6832/2127253870 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86bf400640 1 -- 172.21.15.97:0/622252004 <== mds.0 v2:172.21.15.97:6832/2127253870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x556ddbed4520 con 0x556ddc0d8c10 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86bf400640 10 client.5106 handle_command_reply: tid=0 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 client.5106 shutdown 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 msgr2=0x556ddc143b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc143b60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x556ddbf203c0 tx=0x7f86cc02ea60 comp rx=0 tx=0).stop 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 shutdown_connections 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:6832/2127253870,v1:172.21.15.97:6835/2127253870] conn(0x556ddc0d8c10 0x556ddc0d8ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ddc0c52f0 0x556ddc1440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ddc0c49d0 0x556ddc143b60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86bf400640 0 client.5106 ms_handle_reset on v2:172.21.15.97:6832/2127253870 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 --2- 172.21.15.97:0/622252004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ddc078e10 0x556ddc143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 >> 172.21.15.97:0/622252004 conn(0x556ddbf62130 msgr2=0x556ddc080b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 shutdown_connections 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 1 -- 172.21.15.97:0/622252004 wait complete. 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86e7f3c000 20 client.5106 trim_cache size 0 max 16384 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4932, 2024-09-10T21:25:16.222 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.913415208, 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 18, 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 40, 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 40, 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 12.783358256 2024-09-10T21:25:16.223 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86c00725e0 msgr2=0x7f86c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86c00725e0 0x7f86c0074aa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86e006e380 tx=0x7f86cc009640 comp rx=0 tx=0).stop 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 msgr2=0x7f86e01fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e01fc250 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86d801b3c0 tx=0x7f86d8004270 comp rx=0 tx=0).stop 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 shutdown_connections 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86c00725e0 0x7f86c0074aa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e015c5d0 0x7f86e01fc250 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86e015bbe0 0x7f86e01f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 --2- 172.21.15.97:0/814621472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f86e0155090 0x7f86e00aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 >> 172.21.15.97:0/814621472 conn(0x7f86e00b2ee0 msgr2=0x7f86e006d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 shutdown_connections 2024-09-10T21:25:16.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.212+0000 7f86df200640 1 -- 172.21.15.97:0/814621472 wait complete. 2024-09-10T21:25:16.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:25:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 -- 172.21.15.97:0/3929513038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640ce640 msgr2=0x7f71640cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 --2- 172.21.15.97:0/3929513038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640ce640 0x7f71640cea20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f71640b36f0 tx=0x7f715401c800 comp rx=0 tx=0).stop 2024-09-10T21:25:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 -- 172.21.15.97:0/3929513038 shutdown_connections 2024-09-10T21:25:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 --2- 172.21.15.97:0/3929513038 >> v2:172.21.15.194:3300/0 conn(0x7f71640b0d50 0x7f71640aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 --2- 172.21.15.97:0/3929513038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640ce640 0x7f71640cea20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 -- 172.21.15.97:0/3929513038 >> 172.21.15.97:0/3929513038 conn(0x7f71640d1c30 msgr2=0x7f71640d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 -- 172.21.15.97:0/3929513038 shutdown_connections 2024-09-10T21:25:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.372+0000 7f716b000640 1 -- 172.21.15.97:0/3929513038 wait complete. 2024-09-10T21:25:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 Processor -- start 2024-09-10T21:25:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 -- start start 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640af300 0x7f716414e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71640b0d50 0x7f71640b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71640ce640 0x7f716415e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7164169270 con 0x7f71640af300 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71641690f0 con 0x7f71640b0d50 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f71641693f0 con 0x7f71640ce640 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640af300 0x7f716414e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640af300 0x7f716414e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41412/0 (socket says 172.21.15.97:41412) 2024-09-10T21:25:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 -- 172.21.15.97:0/220613723 learned_addr learned my addr 172.21.15.97:0/220613723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f715b400640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71640b0d50 0x7f71640b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7169000640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71640ce640 0x7f716415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 -- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71640ce640 msgr2=0x7f716415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71640ce640 0x7f716415e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 -- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71640b0d50 msgr2=0x7f71640b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71640b0d50 0x7f71640b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 -- 172.21.15.97:0/220613723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71640b3fb0 con 0x7f71640af300 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7163e00640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640af300 0x7f716414e210 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7154009790 tx=0x7f71540a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7160c00640 1 -- 172.21.15.97:0/220613723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540bb070 con 0x7f71640af300 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7160c00640 1 -- 172.21.15.97:0/220613723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7154004470 con 0x7f71640af300 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f7160c00640 1 -- 172.21.15.97:0/220613723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540b2040 con 0x7f71640af300 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71640b3cd0 con 0x7f71640af300 2024-09-10T21:25:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.376+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71640c9080 con 0x7f71640af300 2024-09-10T21:25:16.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.380+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71640aedc0 con 0x7f71640af300 2024-09-10T21:25:16.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.380+0000 7f7160c00640 1 -- 172.21.15.97:0/220613723 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71540b5030 con 0x7f71640af300 2024-09-10T21:25:16.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.380+0000 7f7160c00640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71400725e0 0x7f7140074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.380+0000 7f7160c00640 1 -- 172.21.15.97:0/220613723 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f715412cc80 con 0x7f71640af300 2024-09-10T21:25:16.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.380+0000 7f715b400640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71400725e0 0x7f7140074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.380+0000 7f715b400640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71400725e0 0x7f7140074aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f716414e910 tx=0x7f714c009680 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.384+0000 7f7160c00640 1 -- 172.21.15.97:0/220613723 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71540f2de0 con 0x7f71640af300 2024-09-10T21:25:16.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.584+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71640d08f0 con 0x7f71640af300 2024-09-10T21:25:16.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.584+0000 7f7160c00640 1 -- 172.21.15.97:0/220613723 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f71540163f0 con 0x7f71640af300 2024-09-10T21:25:16.591 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:25:16.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71400725e0 msgr2=0x7f7140074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71400725e0 0x7f7140074aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f716414e910 tx=0x7f714c009680 comp rx=0 tx=0).stop 2024-09-10T21:25:16.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640af300 msgr2=0x7f716414e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640af300 0x7f716414e210 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7154009790 tx=0x7f71540a5650 comp rx=0 tx=0).stop 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 shutdown_connections 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71400725e0 0x7f7140074aa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71640ce640 0x7f716415e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71640b0d50 0x7f71640b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 --2- 172.21.15.97:0/220613723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71640af300 0x7f716414e210 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 >> 172.21.15.97:0/220613723 conn(0x7f71640d1c30 msgr2=0x7f71640d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 shutdown_connections 2024-09-10T21:25:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16.588+0000 7f716b000640 1 -- 172.21.15.97:0/220613723 wait complete. 2024-09-10T21:25:16.611 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:25:16.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:25:16.612 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:25:16.612 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:25:16.612 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:16.612 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:25:16.612 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:25:16.612 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:25:16.612 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:16.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:16.612 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42377sec preferred_lft 42377sec 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:16.625 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:25:16.685 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:25:16.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:16.687 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:25:16.739 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:25:16.739 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:25:16.739 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:16.740 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:25:16.740 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:16.741 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:25:16.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:25:16.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:25:16.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:25:16.741 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:25:16.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:16.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:16.877 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:25:16.886 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:25:16.886 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:25:16.887 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:16.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:16.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:25:16.948 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:16.948 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:16.996 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:16.996 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:16.996 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:17.051 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:25:17.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:17.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:17.144 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:17.144 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:17.144 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:17.159 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:17.159 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:17.159 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:17.159 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:25:17.160 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:25:17.169 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:17.169 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:17.169 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:17.170 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:25:17.170 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:25:17.170 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:25:17.170 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:25:17.170 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:25:17.173 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:25:17.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:17.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:17.194 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:25:17.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:25:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:17.301 INFO:teuthology.orchestra.run.smithi097.stdout:client5056 2024-09-10T21:25:17.302 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:25:17.302 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:17.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:17.311 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-09-10T21:25:17.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:17.368 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-09-10T21:25:17.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:17.423 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-09-10T21:25:17.478 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:25:17.478 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:25:17.478 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:25:17.478 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:25:17.478 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-09-10T21:25:17.478 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:25:17.587 INFO:teuthology.orchestra.run.smithi097.stdout:1099511627776 2024-09-10T21:25:17.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:17.595 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:25:17.595 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:17.645 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:25:17.645 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:17.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:25:17.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 -- 172.21.15.97:0/426405303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae40cca70 msgr2=0x7f3ae40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 --2- 172.21.15.97:0/426405303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae40cca70 0x7f3ae40705f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4073930 tx=0x7f3adc01cb50 comp rx=0 tx=0).stop 2024-09-10T21:25:17.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 -- 172.21.15.97:0/426405303 shutdown_connections 2024-09-10T21:25:17.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 --2- 172.21.15.97:0/426405303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae40cca70 0x7f3ae40705f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 --2- 172.21.15.97:0/426405303 >> v2:172.21.15.194:3300/0 conn(0x7f3ae415f0f0 0x7f3ae40d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 -- 172.21.15.97:0/426405303 >> 172.21.15.97:0/426405303 conn(0x7f3ae40d1b00 msgr2=0x7f3ae40d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:17.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 -- 172.21.15.97:0/426405303 shutdown_connections 2024-09-10T21:25:17.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.879+0000 7f3aea000640 1 -- 172.21.15.97:0/426405303 wait complete. 2024-09-10T21:25:17.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 Processor -- start 2024-09-10T21:25:17.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 -- start start 2024-09-10T21:25:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ae40cca70 0x7f3ae415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ae40cda50 0x7f3ae4148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae415f0f0 0x7f3ae414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae4071550 con 0x7f3ae415f0f0 2024-09-10T21:25:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae40d0930 con 0x7f3ae40cca70 2024-09-10T21:25:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae41688c0 con 0x7f3ae40cda50 2024-09-10T21:25:17.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae415f0f0 0x7f3ae414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae415f0f0 0x7f3ae414b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53776/0 (socket says 172.21.15.97:53776) 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 -- 172.21.15.97:0/2297267448 learned_addr learned my addr 172.21.15.97:0/2297267448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae8c00640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ae40cca70 0x7f3ae415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3adbe00640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ae40cda50 0x7f3ae4148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 -- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ae40cda50 msgr2=0x7f3ae4148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ae40cda50 0x7f3ae4148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 -- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ae40cca70 msgr2=0x7f3ae415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ae40cca70 0x7f3ae415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 -- 172.21.15.97:0/2297267448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae415d450 con 0x7f3ae415f0f0 2024-09-10T21:25:17.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae9600640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae415f0f0 0x7f3ae414b280 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3adc01c5c0 tx=0x7f3adc009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae1600640 1 -- 172.21.15.97:0/2297267448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0b2040 con 0x7f3ae415f0f0 2024-09-10T21:25:17.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae1600640 1 -- 172.21.15.97:0/2297267448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3adc0bb070 con 0x7f3ae415f0f0 2024-09-10T21:25:17.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae415d730 con 0x7f3ae415f0f0 2024-09-10T21:25:17.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3ae1600640 1 -- 172.21.15.97:0/2297267448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc004530 con 0x7f3ae415f0f0 2024-09-10T21:25:17.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae415dcf0 con 0x7f3ae415f0f0 2024-09-10T21:25:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.883+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae40705f0 con 0x7f3ae415f0f0 2024-09-10T21:25:17.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.887+0000 7f3ae1600640 1 -- 172.21.15.97:0/2297267448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3adc0b5070 con 0x7f3ae415f0f0 2024-09-10T21:25:17.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.887+0000 7f3ae1600640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac00725e0 0x7f3ac0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.887+0000 7f3ae1600640 1 -- 172.21.15.97:0/2297267448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3adc12db80 con 0x7f3ae415f0f0 2024-09-10T21:25:17.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.887+0000 7f3ae8c00640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac00725e0 0x7f3ac0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.887+0000 7f3ae8c00640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac00725e0 0x7f3ac0074aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40058d0 tx=0x7f3ad4005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:17.891+0000 7f3ae1600640 1 -- 172.21.15.97:0/2297267448 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3adc016770 con 0x7f3ae415f0f0 2024-09-10T21:25:18.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ae406de60 con 0x7f3ae415f0f0 2024-09-10T21:25:18.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3ae1600640 1 -- 172.21.15.97:0/2297267448 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) ==== 80+0+0 (secure 0 0 0) 0x7f3adc0f9030 con 0x7f3ae415f0f0 2024-09-10T21:25:18.074 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:25:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac00725e0 msgr2=0x7f3ac0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac00725e0 0x7f3ac0074aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40058d0 tx=0x7f3ad4005860 comp rx=0 tx=0).stop 2024-09-10T21:25:18.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae415f0f0 msgr2=0x7f3ae414b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae415f0f0 0x7f3ae414b280 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3adc01c5c0 tx=0x7f3adc009280 comp rx=0 tx=0).stop 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 shutdown_connections 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac00725e0 0x7f3ac0074aa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ae415f0f0 0x7f3ae414b280 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ae40cda50 0x7f3ae4148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 --2- 172.21.15.97:0/2297267448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ae40cca70 0x7f3ae415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 >> 172.21.15.97:0/2297267448 conn(0x7f3ae40d1b00 msgr2=0x7f3ae406f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 shutdown_connections 2024-09-10T21:25:18.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.067+0000 7f3aea000640 1 -- 172.21.15.97:0/2297267448 wait complete. 2024-09-10T21:25:18.096 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:25:18.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:18.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:18.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:25:18.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:18.145 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:25:18.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- 172.21.15.97:0/368390239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80b1560 msgr2=0x7f41a80b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 --2- 172.21.15.97:0/368390239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80b1560 0x7f41a80b19b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009bc0 tx=0x7f41a001f3b0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- 172.21.15.97:0/368390239 shutdown_connections 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 --2- 172.21.15.97:0/368390239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80b1560 0x7f41a80b19b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 --2- 172.21.15.97:0/368390239 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41a80cec10 0x7f41a80cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- 172.21.15.97:0/368390239 >> 172.21.15.97:0/368390239 conn(0x7f41a80b37c0 msgr2=0x7f41a80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- 172.21.15.97:0/368390239 shutdown_connections 2024-09-10T21:25:18.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- 172.21.15.97:0/368390239 wait complete. 2024-09-10T21:25:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 Processor -- start 2024-09-10T21:25:18.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- start start 2024-09-10T21:25:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41a80b1560 0x7f41a80c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80ce830 0x7f41a81f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41a80cec10 0x7f41a8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41a80ce1a0 con 0x7f41a80ce830 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41a80cde60 con 0x7f41a80cec10 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41aee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f41a80ce320 con 0x7f41a80b1560 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80ce830 0x7f41a81f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80ce830 0x7f41a81f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53796/0 (socket says 172.21.15.97:53796) 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41a7400640 1 -- 172.21.15.97:0/3697729875 learned_addr learned my addr 172.21.15.97:0/3697729875 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41a7e00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41a80b1560 0x7f41a80c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.387+0000 7f41ace00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41a80cec10 0x7f41a8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41a7400640 1 -- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41a80b1560 msgr2=0x7f41a80c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41a7400640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41a80b1560 0x7f41a80c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41a7400640 1 -- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41a80cec10 msgr2=0x7f41a8158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41a7400640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41a80cec10 0x7f41a8158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41a7400640 1 -- 172.21.15.97:0/3697729875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a80c6250 con 0x7f41a80ce830 2024-09-10T21:25:18.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41ace00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41a80cec10 0x7f41a8158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:18.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41a7400640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80ce830 0x7f41a81f8340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f419400d950 tx=0x7f419400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f419fe00640 1 -- 172.21.15.97:0/3697729875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194017070 con 0x7f41a80ce830 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f419fe00640 1 -- 172.21.15.97:0/3697729875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4194009d20 con 0x7f41a80ce830 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a815b0e0 con 0x7f41a80ce830 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f419fe00640 1 -- 172.21.15.97:0/3697729875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41940106c0 con 0x7f41a80ce830 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a815b980 con 0x7f41a80ce830 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a815b4d0 con 0x7f41a80ce830 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f419fe00640 1 -- 172.21.15.97:0/3697729875 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4194010860 con 0x7f41a80ce830 2024-09-10T21:25:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f419fe00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41880725e0 0x7f4188074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.391+0000 7f419fe00640 1 -- 172.21.15.97:0/3697729875 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f41940937a0 con 0x7f41a80ce830 2024-09-10T21:25:18.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.395+0000 7f41a7e00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41880725e0 0x7f4188074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.395+0000 7f41a7e00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41880725e0 0x7f4188074aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4198001910 tx=0x7f4198001420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.395+0000 7f419fe00640 1 -- 172.21.15.97:0/3697729875 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f419405e690 con 0x7f41a80ce830 2024-09-10T21:25:18.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.595+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41a80cf800 con 0x7f41a80ce830 2024-09-10T21:25:18.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.595+0000 7f419fe00640 1 -- 172.21.15.97:0/3697729875 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5226 (secure 0 0 0) 0x7f41940630c0 con 0x7f41a80ce830 2024-09-10T21:25:18.604 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:18.604 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"btime":"2024-09-10T21:25:13:316162+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:12.695121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/2127253870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2127253870},{"type":"v1","addr":"172.21.15.97:6835","nonce":2127253870}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4932,"qdb_cluster":[4932]},"id":2}]} 2024-09-10T21:25:18.604 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-09-10T21:25:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41880725e0 msgr2=0x7f4188074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41880725e0 0x7f4188074aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4198001910 tx=0x7f4198001420 comp rx=0 tx=0).stop 2024-09-10T21:25:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80ce830 msgr2=0x7f41a81f8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80ce830 0x7f41a81f8340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f419400d950 tx=0x7f419400de10 comp rx=0 tx=0).stop 2024-09-10T21:25:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 shutdown_connections 2024-09-10T21:25:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41880725e0 0x7f4188074aa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41a80cec10 0x7f41a8158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41a80ce830 0x7f41a81f8340 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 --2- 172.21.15.97:0/3697729875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41a80b1560 0x7f41a80c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 >> 172.21.15.97:0/3697729875 conn(0x7f41a80b37c0 msgr2=0x7f41a80712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 shutdown_connections 2024-09-10T21:25:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.599+0000 7f41aee00640 1 -- 172.21.15.97:0/3697729875 wait complete. 2024-09-10T21:25:18.626 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-10T21:25:18.795 INFO:teuthology.orchestra.run.smithi097.stdout:{"message":"","return_code":0} 2024-09-10T21:25:18.795 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:25:18.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 -- 172.21.15.97:0/3606329800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300ce830 msgr2=0x7f5d300cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 --2- 172.21.15.97:0/3606329800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300ce830 0x7f5d300cf800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009900 tx=0x7f5d2001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 -- 172.21.15.97:0/3606329800 shutdown_connections 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 --2- 172.21.15.97:0/3606329800 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d300cec10 0x7f5d300cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 --2- 172.21.15.97:0/3606329800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300ce830 0x7f5d300cf800 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 -- 172.21.15.97:0/3606329800 >> 172.21.15.97:0/3606329800 conn(0x7f5d300b37c0 msgr2=0x7f5d300b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 -- 172.21.15.97:0/3606329800 shutdown_connections 2024-09-10T21:25:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 -- 172.21.15.97:0/3606329800 wait complete. 2024-09-10T21:25:18.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 Processor -- start 2024-09-10T21:25:18.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.931+0000 7f5d36800640 1 -- start start 2024-09-10T21:25:18.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d300ce830 0x7f5d3014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300cec10 0x7f5d300b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d300d0280 0x7f5d301f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300cde60 con 0x7f5d300cec10 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300b44a0 con 0x7f5d300d0280 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5d300ce1a0 con 0x7f5d300ce830 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300cec10 0x7f5d300b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d300ce830 0x7f5d3014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300cec10 0x7f5d300b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53810/0 (socket says 172.21.15.97:53810) 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 -- 172.21.15.97:0/2440550554 learned_addr learned my addr 172.21.15.97:0/2440550554 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2fe00640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d300d0280 0x7f5d301f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 -- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d300ce830 msgr2=0x7f5d3014b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d300ce830 0x7f5d3014b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 -- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d300d0280 msgr2=0x7f5d301f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d300d0280 0x7f5d301f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 -- 172.21.15.97:0/2440550554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3014b9b0 con 0x7f5d300cec10 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d2ea00640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300cec10 0x7f5d300b5b10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c9a0 tx=0x7f5d1c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d27400640 1 -- 172.21.15.97:0/2440550554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c007c10 con 0x7f5d300cec10 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d27400640 1 -- 172.21.15.97:0/2440550554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d1c005020 con 0x7f5d300cec10 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d27400640 1 -- 172.21.15.97:0/2440550554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c005450 con 0x7f5d300cec10 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d300b6460 con 0x7f5d300cec10 2024-09-10T21:25:18.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d300b6a20 con 0x7f5d300cec10 2024-09-10T21:25:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.935+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d300b71f0 con 0x7f5d300cec10 2024-09-10T21:25:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.939+0000 7f5d27400640 1 -- 172.21.15.97:0/2440550554 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5d1c0047b0 con 0x7f5d300cec10 2024-09-10T21:25:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.939+0000 7f5d27400640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d100725e0 0x7f5d10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.939+0000 7f5d27400640 1 -- 172.21.15.97:0/2440550554 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f5d1c093910 con 0x7f5d300cec10 2024-09-10T21:25:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.939+0000 7f5d2f400640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d100725e0 0x7f5d10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.939+0000 7f5d2f400640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d100725e0 0x7f5d10074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d200b2000 tx=0x7f5d2001d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:18.943+0000 7f5d27400640 1 -- 172.21.15.97:0/2440550554 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5d1c014030 con 0x7f5d300cec10 2024-09-10T21:25:19.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.139+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5d300b2780 con 0x7f5d300cec10 2024-09-10T21:25:19.149 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:25:19.143+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:25:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.143+0000 7f5d27400640 1 -- 172.21.15.97:0/2440550554 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f5d1c05e7f0 con 0x7f5d300cec10 2024-09-10T21:25:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:25:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d100725e0 msgr2=0x7f5d10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d100725e0 0x7f5d10074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d200b2000 tx=0x7f5d2001d910 comp rx=0 tx=0).stop 2024-09-10T21:25:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300cec10 msgr2=0x7f5d300b5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300cec10 0x7f5d300b5b10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c9a0 tx=0x7f5d1c00ce60 comp rx=0 tx=0).stop 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 shutdown_connections 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d100725e0 0x7f5d10074aa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d300d0280 0x7f5d301f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d300cec10 0x7f5d300b5b10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 --2- 172.21.15.97:0/2440550554 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d300ce830 0x7f5d3014b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 >> 172.21.15.97:0/2440550554 conn(0x7f5d300b37c0 msgr2=0x7f5d30071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 shutdown_connections 2024-09-10T21:25:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.147+0000 7f5d36800640 1 -- 172.21.15.97:0/2440550554 wait complete. 2024-09-10T21:25:19.164 INFO:tasks.ceph.mds.c.smithi097.stderr: -5108> 2024-09-10T21:25:14.264+0000 7f46bca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:25:19.175 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:25:19.213 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:25:19.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.215+0000 7f5447b70080 1 Processor -- start 2024-09-10T21:25:19.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.215+0000 7f5447b70080 1 -- start start 2024-09-10T21:25:19.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 0x5617b18e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 0x5617b18e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b196df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5617b1805a00 con 0x5617b198c3c0 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5617b1863f40 con 0x5617b1972060 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5617b1973c90 con 0x5617b1971c80 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b196df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b196df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53848/0 (socket says 172.21.15.97:53848) 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 -- 172.21.15.97:0/1279046005 learned_addr learned my addr 172.21.15.97:0/1279046005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f543fe00640 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 0x5617b18e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5444e00640 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 0x5617b18e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 -- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 msgr2=0x5617b18e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 0x5617b18e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 -- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 msgr2=0x5617b18e36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 0x5617b18e36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 -- 172.21.15.97:0/1279046005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b17dec10 con 0x5617b198c3c0 2024-09-10T21:25:19.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b196df80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5440019b80 tx=0x7f5440018270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe73abfa3ea096f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f543ea00640 1 -- 172.21.15.97:0/1279046005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54400a9b00 con 0x5617b198c3c0 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f543ea00640 1 -- 172.21.15.97:0/1279046005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54400a5040 con 0x5617b198c3c0 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f543ea00640 1 -- 172.21.15.97:0/1279046005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54400bd440 con 0x5617b198c3c0 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- 172.21.15.97:0/1279046005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 msgr2=0x5617b196df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b196df80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5440019b80 tx=0x7f5440018270 comp rx=0 tx=0).stop 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- 172.21.15.97:0/1279046005 shutdown_connections 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b196df80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 0x5617b18e36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- 172.21.15.97:0/1279046005 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 0x5617b18e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- 172.21.15.97:0/1279046005 >> 172.21.15.97:0/1279046005 conn(0x5617b18b6140 msgr2=0x5617b18b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- 172.21.15.97:0/1279046005 shutdown_connections 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- 172.21.15.97:0/1279046005 wait complete. 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 Processor -- start 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- start start 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 0x5617b1970ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 0x5617b18e0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b198d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5617b18e65b0 con 0x5617b198c3c0 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5617b1990240 con 0x5617b1972060 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5447b70080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5617b18e6850 con 0x5617b1971c80 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.219+0000 7f5445800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b198d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b198d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53862/0 (socket says 172.21.15.97:53862) 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 -- 172.21.15.97:0/2526864877 learned_addr learned my addr 172.21.15.97:0/2526864877 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5444e00640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 0x5617b1970ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f543fe00640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 0x5617b18e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 -- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 msgr2=0x5617b1970ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5617b1971c80 0x5617b1970ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 -- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 msgr2=0x5617b18e0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617b1972060 0x5617b18e0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 -- 172.21.15.97:0/2526864877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b18daf70 con 0x5617b198c3c0 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5445800640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b198d9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5440019b80 tx=0x7f5440004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f543d600640 1 -- 172.21.15.97:0/2526864877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54400bd440 con 0x5617b198c3c0 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f543d600640 1 -- 172.21.15.97:0/2526864877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54400c7070 con 0x5617b198c3c0 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f543d600640 1 -- 172.21.15.97:0/2526864877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54400bd840 con 0x5617b198c3c0 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5617b18db210 con 0x5617b198c3c0 2024-09-10T21:25:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b18db750 con 0x5617b198c3c0 2024-09-10T21:25:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f543d600640 1 -- 172.21.15.97:0/2526864877 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f54400bb070 con 0x5617b198c3c0 2024-09-10T21:25:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f543d600640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5424070fc0 0x7f5424073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.223+0000 7f543d600640 1 -- 172.21.15.97:0/2526864877 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f544012e340 con 0x5617b198c3c0 2024-09-10T21:25:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.227+0000 7f5444e00640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5424070fc0 0x7f5424073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.227+0000 7f5447b70080 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5617b18e0bb0 0x5617b18dc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.227+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5617b1a27120 con 0x5617b18e0bb0 2024-09-10T21:25:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.227+0000 7f543fe00640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5617b18e0bb0 0x5617b18dc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.227+0000 7f5444e00640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5424070fc0 0x7f5424073480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5438011570 tx=0x7f54380017e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.227+0000 7f543fe00640 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5617b18e0bb0 0x5617b18dc640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f543fe00640 1 -- 172.21.15.97:0/2526864877 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v41'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f543000b950 con 0x5617b18e0bb0 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5617b18e0bb0 msgr2=0x5617b18dc640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5617b18e0bb0 0x5617b18dc640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5424070fc0 msgr2=0x7f5424073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5424070fc0 0x7f5424073480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5438011570 tx=0x7f54380017e0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 msgr2=0x5617b198d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 --2- 172.21.15.97:0/2526864877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617b198c3c0 0x5617b198d9b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5440019b80 tx=0x7f5440004550 comp rx=0 tx=0).stop 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5445800640 1 -- 172.21.15.97:0/2526864877 reap_dead start 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 shutdown_connections 2024-09-10T21:25:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 >> 172.21.15.97:0/2526864877 conn(0x5617b18b6140 msgr2=0x5617b198b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 shutdown_connections 2024-09-10T21:25:19.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.231+0000 7f5447b70080 1 -- 172.21.15.97:0/2526864877 wait complete. 2024-09-10T21:25:19.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:25:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- 172.21.15.97:0/279939366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f481511f0 msgr2=0x7f4f480b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/279939366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f481511f0 0x7f4f480b37b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009900 tx=0x7f4f3801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- 172.21.15.97:0/279939366 shutdown_connections 2024-09-10T21:25:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/279939366 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f480b3cf0 0x7f4f480ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/279939366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f481511f0 0x7f4f480b37b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- 172.21.15.97:0/279939366 >> 172.21.15.97:0/279939366 conn(0x7f4f480b1780 msgr2=0x7f4f480b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- 172.21.15.97:0/279939366 shutdown_connections 2024-09-10T21:25:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- 172.21.15.97:0/279939366 wait complete. 2024-09-10T21:25:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 Processor -- start 2024-09-10T21:25:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- start start 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f480b3cf0 0x7f4f481f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f480cf420 0x7f4f4814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f481511f0 0x7f4f48150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4806e530 con 0x7f4f480cf420 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4806e3b0 con 0x7f4f480b3cf0 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4fa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4806e6b0 con 0x7f4f481511f0 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f480cf420 0x7f4f4814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f480cf420 0x7f4f4814fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53886/0 (socket says 172.21.15.97:53886) 2024-09-10T21:25:19.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 -- 172.21.15.97:0/661148799 learned_addr learned my addr 172.21.15.97:0/661148799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4d000640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f480b3cf0 0x7f4f481f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f4da00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f481511f0 0x7f4f48150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 -- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f481511f0 msgr2=0x7f4f48150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f481511f0 0x7f4f48150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 -- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f480b3cf0 msgr2=0x7f4f481f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f480b3cf0 0x7f4f481f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 -- 172.21.15.97:0/661148799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f48152fc0 con 0x7f4f480cf420 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.423+0000 7f4f3fe00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f480cf420 0x7f4f4814fd00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d980 tx=0x7f4f3000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f45600640 1 -- 172.21.15.97:0/661148799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30017070 con 0x7f4f480cf420 2024-09-10T21:25:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f45600640 1 -- 172.21.15.97:0/661148799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f30009d60 con 0x7f4f480cf420 2024-09-10T21:25:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f45600640 1 -- 172.21.15.97:0/661148799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30005280 con 0x7f4f480cf420 2024-09-10T21:25:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f481532a0 con 0x7f4f480cf420 2024-09-10T21:25:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f481f7dd0 con 0x7f4f480cf420 2024-09-10T21:25:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f480ceee0 con 0x7f4f480cf420 2024-09-10T21:25:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f45600640 1 -- 172.21.15.97:0/661148799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f30005420 con 0x7f4f480cf420 2024-09-10T21:25:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f45600640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f240725e0 0x7f4f24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.427+0000 7f4f45600640 1 -- 172.21.15.97:0/661148799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f4f300934c0 con 0x7f4f480cf420 2024-09-10T21:25:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.431+0000 7f4f4d000640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f240725e0 0x7f4f24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.431+0000 7f4f4d000640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f240725e0 0x7f4f24074aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f380b2000 tx=0x7f4f3801d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.435+0000 7f4f45600640 1 -- 172.21.15.97:0/661148799 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4f3005e400 con 0x7f4f480cf420 2024-09-10T21:25:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:19.635+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4f480b3930 con 0x7f4f480cf420 2024-09-10T21:25:20.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f45600640 1 -- 172.21.15.97:0/661148799 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v20) ==== 162+0+0 (secure 0 0 0) 0x7f4f30067170 con 0x7f4f480cf420 2024-09-10T21:25:20.153 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:25:20.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f240725e0 msgr2=0x7f4f24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f240725e0 0x7f4f24074aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f380b2000 tx=0x7f4f3801d910 comp rx=0 tx=0).stop 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f480cf420 msgr2=0x7f4f4814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f480cf420 0x7f4f4814fd00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d980 tx=0x7f4f3000de40 comp rx=0 tx=0).stop 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 shutdown_connections 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f240725e0 0x7f4f24074aa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f481511f0 0x7f4f48150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f480cf420 0x7f4f4814fd00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 --2- 172.21.15.97:0/661148799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f480b3cf0 0x7f4f481f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 >> 172.21.15.97:0/661148799 conn(0x7f4f480b1780 msgr2=0x7f4f4815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 shutdown_connections 2024-09-10T21:25:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.147+0000 7f4f4fa00640 1 -- 172.21.15.97:0/661148799 wait complete. 2024-09-10T21:25:20.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:20.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.311+0000 7f47f0200640 1 -- 172.21.15.97:0/679085918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d0003a40 msgr2=0x7f47d0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.311+0000 7f47f0200640 1 --2- 172.21.15.97:0/679085918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d0003a40 0x7f47d0003e60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00c3c0 tx=0x7f47dc01cf60 comp rx=0 tx=0).stop 2024-09-10T21:25:20.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.311+0000 7f47ee200640 1 -- 172.21.15.97:0/679085918 reap_dead start 2024-09-10T21:25:20.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.311+0000 7f47f0200640 1 -- 172.21.15.97:0/679085918 shutdown_connections 2024-09-10T21:25:20.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.311+0000 7f47f0200640 1 -- 172.21.15.97:0/679085918 >> 172.21.15.97:0/679085918 conn(0x7f47e80d1c30 msgr2=0x7f47e80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.311+0000 7f47f0200640 1 -- 172.21.15.97:0/679085918 shutdown_connections 2024-09-10T21:25:20.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.311+0000 7f47f0200640 1 -- 172.21.15.97:0/679085918 wait complete. 2024-09-10T21:25:20.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 Processor -- start 2024-09-10T21:25:20.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 -- start start 2024-09-10T21:25:20.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47d0003a40 0x7f47e80b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e80c9710 0x7f47e8071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47e8157d90 0x7f47e8158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81678e0 con 0x7f47e80c9710 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47e80cda40 con 0x7f47d0003a40 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f47e8167a60 con 0x7f47e8157d90 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e80c9710 0x7f47e8071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e80c9710 0x7f47e8071f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53892/0 (socket says 172.21.15.97:53892) 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 -- 172.21.15.97:0/194289948 learned_addr learned my addr 172.21.15.97:0/194289948 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ee200640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47e8157d90 0x7f47e8158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ed800640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47d0003a40 0x7f47e80b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 -- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47e8157d90 msgr2=0x7f47e8158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47e8157d90 0x7f47e8158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 -- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47d0003a40 msgr2=0x7f47e80b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47d0003a40 0x7f47e80b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 -- 172.21.15.97:0/194289948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e80b5030 con 0x7f47e80c9710 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47ece00640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e80c9710 0x7f47e8071f70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f47d4004ee0 tx=0x7f47d4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47e5600640 1 -- 172.21.15.97:0/194289948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4015020 con 0x7f47e80c9710 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47e5600640 1 -- 172.21.15.97:0/194289948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47d4011960 con 0x7f47e80c9710 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47e5600640 1 -- 172.21.15.97:0/194289948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4011d90 con 0x7f47e80c9710 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e80b5310 con 0x7f47e80c9710 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e80c9b90 con 0x7f47e80c9710 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.315+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e80cc6d0 con 0x7f47e80c9710 2024-09-10T21:25:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.319+0000 7f47e5600640 1 -- 172.21.15.97:0/194289948 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f47d401a400 con 0x7f47e80c9710 2024-09-10T21:25:20.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.319+0000 7f47e5600640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47c8072610 0x7f47c8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.319+0000 7f47e5600640 1 -- 172.21.15.97:0/194289948 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f47d40941c0 con 0x7f47e80c9710 2024-09-10T21:25:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.319+0000 7f47ed800640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47c8072610 0x7f47c8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.319+0000 7f47ed800640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47c8072610 0x7f47c8074ad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ac000 tx=0x7f47dc019da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.323+0000 7f47e5600640 1 -- 172.21.15.97:0/194289948 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47d401f020 con 0x7f47e80c9710 2024-09-10T21:25:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.519+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47e80cc040 con 0x7f47e80c9710 2024-09-10T21:25:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47e5600640 1 -- 172.21.15.97:0/194289948 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5214 (secure 0 0 0) 0x7f47d405f6c0 con 0x7f47e80c9710 2024-09-10T21:25:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:20.529 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":21,"btime":"2024-09-10T21:25:20:154455+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:20.154403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-10T21:25:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 21 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47c8072610 msgr2=0x7f47c8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47c8072610 0x7f47c8074ad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ac000 tx=0x7f47dc019da0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e80c9710 msgr2=0x7f47e8071f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e80c9710 0x7f47e8071f70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f47d4004ee0 tx=0x7f47d4001640 comp rx=0 tx=0).stop 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 shutdown_connections 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47c8072610 0x7f47c8074ad0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47e8157d90 0x7f47e8158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e80c9710 0x7f47e8071f70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 --2- 172.21.15.97:0/194289948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47d0003a40 0x7f47e80b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 >> 172.21.15.97:0/194289948 conn(0x7f47e80d1c30 msgr2=0x7f47e8163c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 shutdown_connections 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:20.523+0000 7f47f0200640 1 -- 172.21.15.97:0/194289948 wait complete. 2024-09-10T21:25:20.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:25:09.683475+0000', 'modified': '2024-09-10T21:25:20.154403+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5148': {'gid': 5148, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/95042702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 95042702}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 95042702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:25:20.954 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5148:up:replay 2024-09-10T21:25:21.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 -- 172.21.15.97:0/1776749029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 msgr2=0x7f3d9015f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 --2- 172.21.15.97:0/1776749029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015f6d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3d90141580 tx=0x7f3d7c01c8a0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 -- 172.21.15.97:0/1776749029 shutdown_connections 2024-09-10T21:25:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 --2- 172.21.15.97:0/1776749029 >> v2:172.21.15.194:3300/0 conn(0x7f3d9015fc10 0x7f3d9013f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 --2- 172.21.15.97:0/1776749029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015f6d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 -- 172.21.15.97:0/1776749029 >> 172.21.15.97:0/1776749029 conn(0x7f3d900b1780 msgr2=0x7f3d900b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 -- 172.21.15.97:0/1776749029 shutdown_connections 2024-09-10T21:25:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.099+0000 7f3d95a00640 1 -- 172.21.15.97:0/1776749029 wait complete. 2024-09-10T21:25:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 Processor -- start 2024-09-10T21:25:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 -- start start 2024-09-10T21:25:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d900ce5f0 0x7f3d901524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d9015fc10 0x7f3d9013ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90168a70 con 0x7f3d90141d20 2024-09-10T21:25:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90168bf0 con 0x7f3d9015fc10 2024-09-10T21:25:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3d90168d70 con 0x7f3d900ce5f0 2024-09-10T21:25:22.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53918/0 (socket says 172.21.15.97:53918) 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 -- 172.21.15.97:0/4163504636 learned_addr learned my addr 172.21.15.97:0/4163504636 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8f400640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d9015fc10 0x7f3d9013ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8ea00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d900ce5f0 0x7f3d901524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 -- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d900ce5f0 msgr2=0x7f3d901524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d900ce5f0 0x7f3d901524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 -- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d9015fc10 msgr2=0x7f3d9013ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d9015fc10 0x7f3d9013ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 -- 172.21.15.97:0/4163504636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9015dca0 con 0x7f3d90141d20 2024-09-10T21:25:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8f400640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d9015fc10 0x7f3d9013ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8ea00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d900ce5f0 0x7f3d901524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d8e000640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015ea60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0b0040 tx=0x7f3d7c0a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d86a00640 1 -- 172.21.15.97:0/4163504636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0b2040 con 0x7f3d90141d20 2024-09-10T21:25:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d86a00640 1 -- 172.21.15.97:0/4163504636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d7c004440 con 0x7f3d90141d20 2024-09-10T21:25:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d86a00640 1 -- 172.21.15.97:0/4163504636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0052d0 con 0x7f3d90141d20 2024-09-10T21:25:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9015df80 con 0x7f3d90141d20 2024-09-10T21:25:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.103+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9013d230 con 0x7f3d90141d20 2024-09-10T21:25:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.107+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9015f6d0 con 0x7f3d90141d20 2024-09-10T21:25:22.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.107+0000 7f3d86a00640 1 -- 172.21.15.97:0/4163504636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3d7c005470 con 0x7f3d90141d20 2024-09-10T21:25:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.107+0000 7f3d86a00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3d700725e0 0x7f3d70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.107+0000 7f3d86a00640 1 -- 172.21.15.97:0/4163504636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f3d7c12cfe0 con 0x7f3d90141d20 2024-09-10T21:25:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.107+0000 7f3d8ea00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3d700725e0 0x7f3d70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.111+0000 7f3d8ea00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3d700725e0 0x7f3d70074aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d80007fd0 tx=0x7f3d80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:22.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.111+0000 7f3d86a00640 1 -- 172.21.15.97:0/4163504636 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d7c016490 con 0x7f3d90141d20 2024-09-10T21:25:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d90141340 con 0x7f3d90141d20 2024-09-10T21:25:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d86a00640 1 -- 172.21.15.97:0/4163504636 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5214 (secure 0 0 0) 0x7f3d7c0f8530 con 0x7f3d90141d20 2024-09-10T21:25:22.318 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:22.318 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":23,"btime":"2024-09-10T21:25:22:159458+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:21.161982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-10T21:25:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 23 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3d700725e0 msgr2=0x7f3d70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3d700725e0 0x7f3d70074aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d80007fd0 tx=0x7f3d80009040 comp rx=0 tx=0).stop 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 msgr2=0x7f3d9015ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015ea60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0b0040 tx=0x7f3d7c0a5720 comp rx=0 tx=0).stop 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 shutdown_connections 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3d700725e0 0x7f3d70074aa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d9015fc10 0x7f3d9013ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d90141d20 0x7f3d9015ea60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 --2- 172.21.15.97:0/4163504636 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d900ce5f0 0x7f3d901524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 >> 172.21.15.97:0/4163504636 conn(0x7f3d900b1780 msgr2=0x7f3d9006ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 shutdown_connections 2024-09-10T21:25:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:22.311+0000 7f3d95a00640 1 -- 172.21.15.97:0/4163504636 wait complete. 2024-09-10T21:25:22.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:25:09.683475+0000', 'modified': '2024-09-10T21:25:21.161982+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5148': {'gid': 5148, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.97:6835/95042702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 95042702}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 95042702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:25:22.742 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5148:up:rejoin 2024-09-10T21:25:23.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.887+0000 7ff0f6e00640 1 -- 172.21.15.97:0/3017633344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d40039f0 msgr2=0x7ff0d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.887+0000 7ff0f6e00640 1 --2- 172.21.15.97:0/3017633344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d40039f0 0x7ff0d4003dd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0001730 tx=0x7ff0e001efb0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.887+0000 7ff0f4e00640 1 -- 172.21.15.97:0/3017633344 reap_dead start 2024-09-10T21:25:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.887+0000 7ff0f6e00640 1 -- 172.21.15.97:0/3017633344 shutdown_connections 2024-09-10T21:25:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.887+0000 7ff0f6e00640 1 -- 172.21.15.97:0/3017633344 >> 172.21.15.97:0/3017633344 conn(0x7ff0f00b37c0 msgr2=0x7ff0f00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.887+0000 7ff0f6e00640 1 -- 172.21.15.97:0/3017633344 shutdown_connections 2024-09-10T21:25:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.887+0000 7ff0f6e00640 1 -- 172.21.15.97:0/3017633344 wait complete. 2024-09-10T21:25:23.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 Processor -- start 2024-09-10T21:25:23.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 -- start start 2024-09-10T21:25:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d40039f0 0x7ff0f0164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0f014fa00 0x7ff0f0164160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0f015a670 0x7ff0f015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f00cddd0 con 0x7ff0f014fa00 2024-09-10T21:25:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f00cdc50 con 0x7ff0d40039f0 2024-09-10T21:25:23.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f00ce1a0 con 0x7ff0f015a670 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0f014fa00 0x7ff0f0164160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0f014fa00 0x7ff0f0164160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53948/0 (socket says 172.21.15.97:53948) 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 -- 172.21.15.97:0/1820930841 learned_addr learned my addr 172.21.15.97:0/1820930841 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 -- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0f015a670 msgr2=0x7ff0f015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0f015a670 0x7ff0f015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 -- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d40039f0 msgr2=0x7ff0f0164870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d40039f0 0x7ff0f0164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 -- 172.21.15.97:0/1820930841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f01636c0 con 0x7ff0f014fa00 2024-09-10T21:25:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0e7400640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0f014fa00 0x7ff0f0164160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80066e0 tx=0x7ff0d80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0ecc00640 1 -- 172.21.15.97:0/1820930841 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8015070 con 0x7ff0f014fa00 2024-09-10T21:25:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0ecc00640 1 -- 172.21.15.97:0/1820930841 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0d8011900 con 0x7ff0f014fa00 2024-09-10T21:25:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f00b05e0 con 0x7ff0f014fa00 2024-09-10T21:25:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0ecc00640 1 -- 172.21.15.97:0/1820930841 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8011d50 con 0x7ff0f014fa00 2024-09-10T21:25:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f00b0200 con 0x7ff0f014fa00 2024-09-10T21:25:23.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.891+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f00cf800 con 0x7ff0f014fa00 2024-09-10T21:25:23.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.895+0000 7ff0ecc00640 1 -- 172.21.15.97:0/1820930841 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff0d8010420 con 0x7ff0f014fa00 2024-09-10T21:25:23.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.895+0000 7ff0ecc00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0cc072610 0x7ff0cc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.895+0000 7ff0ecc00640 1 -- 172.21.15.97:0/1820930841 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff0d8094450 con 0x7ff0f014fa00 2024-09-10T21:25:23.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.895+0000 7ff0efe00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0cc072610 0x7ff0cc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.895+0000 7ff0efe00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0cc072610 0x7ff0cc074ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40057f0 tx=0x7ff0d4005840 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:23.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:23.899+0000 7ff0ecc00640 1 -- 172.21.15.97:0/1820930841 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0d805f8b0 con 0x7ff0f014fa00 2024-09-10T21:25:24.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0f00cc910 con 0x7ff0f014fa00 2024-09-10T21:25:24.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0ecc00640 1 -- 172.21.15.97:0/1820930841 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5221 (secure 0 0 0) 0x7ff0d805f1b0 con 0x7ff0f014fa00 2024-09-10T21:25:24.106 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:24.106 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"btime":"2024-09-10T21:25:23:161523+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:23.161515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5148,"qdb_cluster":[5148]},"id":2}]} 2024-09-10T21:25:24.106 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0cc072610 msgr2=0x7ff0cc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0cc072610 0x7ff0cc074ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40057f0 tx=0x7ff0d4005840 comp rx=0 tx=0).stop 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0f014fa00 msgr2=0x7ff0f0164160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0f014fa00 0x7ff0f0164160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80066e0 tx=0x7ff0d80039d0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 shutdown_connections 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0cc072610 0x7ff0cc074ad0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0f015a670 0x7ff0f015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0f014fa00 0x7ff0f0164160 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 --2- 172.21.15.97:0/1820930841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d40039f0 0x7ff0f0164870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 >> 172.21.15.97:0/1820930841 conn(0x7ff0f00b37c0 msgr2=0x7ff0f0150af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 shutdown_connections 2024-09-10T21:25:24.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.099+0000 7ff0f6e00640 1 -- 172.21.15.97:0/1820930841 wait complete. 2024-09-10T21:25:24.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:25:09.683475+0000', 'modified': '2024-09-10T21:25:23.161515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5148': {'gid': 5148, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6835/95042702', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 95042702}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 95042702}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5148, 'qdb_cluster': [5148]} 2024-09-10T21:25:24.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:25:24.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:25:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 -- 172.21.15.97:0/4125947109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb52c003ab0 msgr2=0x7fb52c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 --2- 172.21.15.97:0/4125947109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb52c003ab0 0x7fb52c003ed0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb5380016c0 tx=0x7fb53801c750 comp rx=0 tx=0).stop 2024-09-10T21:25:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 -- 172.21.15.97:0/4125947109 shutdown_connections 2024-09-10T21:25:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 --2- 172.21.15.97:0/4125947109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb52c003ab0 0x7fb52c003ed0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 -- 172.21.15.97:0/4125947109 >> 172.21.15.97:0/4125947109 conn(0x7fb5480d1cd0 msgr2=0x7fb5480d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 -- 172.21.15.97:0/4125947109 shutdown_connections 2024-09-10T21:25:24.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 -- 172.21.15.97:0/4125947109 wait complete. 2024-09-10T21:25:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 Processor -- start 2024-09-10T21:25:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 -- start start 2024-09-10T21:25:24.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.667+0000 7fb54ee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003ab0 0x7fb5480b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5480b1ec0 0x7fb5481fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5481fc510 0x7fb5481fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5480afa60 con 0x7fb5480b1ec0 2024-09-10T21:25:24.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb5480af8e0 con 0x7fb5481fc510 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb5480b0470 con 0x7fb52c003ab0 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5480b1ec0 0x7fb5481fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5480b1ec0 0x7fb5481fb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53962/0 (socket says 172.21.15.97:53962) 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 -- 172.21.15.97:0/2947108911 learned_addr learned my addr 172.21.15.97:0/2947108911 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547e00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003ab0 0x7fb5480b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ce00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5481fc510 0x7fb5481fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 -- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003ab0 msgr2=0x7fb5480b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003ab0 0x7fb5480b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 -- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5481fc510 msgr2=0x7fb5481fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5481fc510 0x7fb5481fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 -- 172.21.15.97:0/2947108911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb548155bf0 con 0x7fb5480b1ec0 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ce00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5481fc510 0x7fb5481fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:24.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb547400640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5480b1ec0 0x7fb5481fb900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb534011a40 tx=0x7fb534011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb53fe00640 1 -- 172.21.15.97:0/2947108911 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53401e070 con 0x7fb5480b1ec0 2024-09-10T21:25:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb53fe00640 1 -- 172.21.15.97:0/2947108911 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb53400fe70 con 0x7fb5480b1ec0 2024-09-10T21:25:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb548155ed0 con 0x7fb5480b1ec0 2024-09-10T21:25:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb53fe00640 1 -- 172.21.15.97:0/2947108911 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5340042b0 con 0x7fb5480b1ec0 2024-09-10T21:25:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb548156490 con 0x7fb5480b1ec0 2024-09-10T21:25:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.671+0000 7fb53fe00640 1 -- 172.21.15.97:0/2947108911 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb534003600 con 0x7fb5480b1ec0 2024-09-10T21:25:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb53fe00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528072610 0x7fb528074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb53fe00640 1 -- 172.21.15.97:0/2947108911 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fb534093fd0 con 0x7fb5480b1ec0 2024-09-10T21:25:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 Processor -- start 2024-09-10T21:25:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- start start 2024-09-10T21:25:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528072610 0x7fb528074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cab9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528072610 0x7fb528074ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb52c004790 tx=0x7fb538019540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3ca6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3ca6e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a3c98e3f0 con 0x559a3cab99d0 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a3c8cbb20 con 0x559a3caba2f0 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559a3c8baf50 con 0x559a3ca6de10 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cab9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cab9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53978/0 (socket says 172.21.15.97:53978) 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 -- 172.21.15.97:0/2301781206 learned_addr learned my addr 172.21.15.97:0/2301781206 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547400640 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3ca6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb54ce00640 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3ca6e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 -- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 msgr2=0x559a3ca6e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3ca6e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 -- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 msgr2=0x559a3ca6d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3ca6d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 -- 172.21.15.97:0/2301781206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3c882590 con 0x559a3cab99d0 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb54ce00640 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3ca6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb547e00640 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cab9db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb53801ae50 tx=0x7fb5380b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249f9fe444f68cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb53cc00640 1 -- 172.21.15.97:0/2301781206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5380b9030 con 0x559a3cab99d0 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb53cc00640 1 -- 172.21.15.97:0/2301781206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5380c5070 con 0x559a3cab99d0 2024-09-10T21:25:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb53cc00640 1 -- 172.21.15.97:0/2301781206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5380c0690 con 0x559a3cab99d0 2024-09-10T21:25:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- 172.21.15.97:0/2301781206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 msgr2=0x559a3cab9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cab9db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb53801ae50 tx=0x7fb5380b5000 comp rx=0 tx=0).stop 2024-09-10T21:25:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- 172.21.15.97:0/2301781206 shutdown_connections 2024-09-10T21:25:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3ca6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3ca6d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2301781206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cab9db0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- 172.21.15.97:0/2301781206 >> 172.21.15.97:0/2301781206 conn(0x559a3c957130 msgr2=0x559a3ca9ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- 172.21.15.97:0/2301781206 shutdown_connections 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.675+0000 7fb5522e5000 1 -- 172.21.15.97:0/2301781206 wait complete. 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 -- start start 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 Processor -- start 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3ca6de10 0x559a3ca9a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3ca9ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3caba2f0 0x559a3ca9b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a3c8555d0 con 0x559a3caba2f0 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a3c8bed60 con 0x559a3ca6de10 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559a3c861500 con 0x559a3cab99d0 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 10 client.0 _ll_register_callbacks cb 0x559a3c865810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3caba2f0 0x559a3ca9b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3caba2f0 0x559a3ca9b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53994/0 (socket says 172.21.15.97:53994) 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 -- 172.21.15.97:0/2632920024 learned_addr learned my addr 172.21.15.97:0/2632920024 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb547e00640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3ca6de10 0x559a3ca9a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb547400640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3ca9ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 -- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 msgr2=0x559a3ca9ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3ca9ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 -- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3ca6de10 msgr2=0x559a3ca9a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3ca6de10 0x559a3ca9a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 -- 172.21.15.97:0/2632920024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3c859170 con 0x559a3caba2f0 2024-09-10T21:25:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb547e00640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3ca6de10 0x559a3ca9a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb54ce00640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3caba2f0 0x559a3ca9b3b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb540001340 tx=0x7fb540006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4933ffa944dcb389 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb531600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb531600640 1 -- 172.21.15.97:0/2632920024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540012640 con 0x559a3caba2f0 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a3c8530f0 con 0x559a3caba2f0 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb531600640 1 -- 172.21.15.97:0/2632920024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fb540018b10 con 0x559a3caba2f0 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb531600640 1 -- 172.21.15.97:0/2632920024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb540018d10 con 0x559a3caba2f0 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 10 client.5193 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x559a3c93fef0 con 0x559a3caba2f0 2024-09-10T21:25:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb531600640 1 -- 172.21.15.97:0/2632920024 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559a3c8530f0 con 0x559a3caba2f0 2024-09-10T21:25:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb531600640 1 -- 172.21.15.97:0/2632920024 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7fb540025070 con 0x559a3caba2f0 2024-09-10T21:25:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 10 client.5193 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 10 client.5193 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 20 client.5193 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3ca9c0f0 0x559a3cacbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 4 client.5193 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a3c8cda70 con 0x559a3ca9c0f0 2024-09-10T21:25:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.679+0000 7fb547e00640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3ca9c0f0 0x559a3cacbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb547e00640 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3ca9c0f0 0x559a3cacbc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b57a0ddbed8796 server_cookie=ebd32b523a393fbc in_seq=0 out_seq=0 2024-09-10T21:25:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb531600640 10 client.5193 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb531600640 1 -- 172.21.15.97:0/2632920024 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a3c8cda70 con 0x559a3ca9c0f0 2024-09-10T21:25:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb531600640 10 client.5193 handle_command_reply: tid=0 2024-09-10T21:25:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 client.5193 shutdown 2024-09-10T21:25:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3caba2f0 msgr2=0x559a3ca9b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3caba2f0 0x559a3ca9b3b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb540001340 tx=0x7fb540006430 comp rx=0 tx=0).stop 2024-09-10T21:25:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 shutdown_connections 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3ca9c0f0 0x559a3cacbc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3caba2f0 0x559a3ca9b3b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3ca9ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb531600640 0 client.5193 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 --2- 172.21.15.97:0/2632920024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3ca6de10 0x559a3ca9a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 >> 172.21.15.97:0/2632920024 conn(0x559a3c957130 msgr2=0x559a3ca6cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 shutdown_connections 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 1 -- 172.21.15.97:0/2632920024 wait complete. 2024-09-10T21:25:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.683+0000 7fb5522e5000 20 client.5193 trim_cache size 0 max 16384 2024-09-10T21:25:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.699+0000 7fb5522e5000 1 Processor -- start 2024-09-10T21:25:24.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.699+0000 7fb5522e5000 1 -- start start 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3cb2bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cb2c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb2c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5482058f0 con 0x559a3cab99d0 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb5482048f0 con 0x559a3caba2f0 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb548203ce0 con 0x559a3ca6de10 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cb2c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cb2c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54002/0 (socket says 172.21.15.97:54002) 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 -- 172.21.15.97:0/1678169687 learned_addr learned my addr 172.21.15.97:0/1678169687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547e00640 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3cb2bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb54ce00640 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb2c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 -- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 msgr2=0x559a3cb2bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3cb2bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 -- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 msgr2=0x559a3cb2c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb2c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 -- 172.21.15.97:0/1678169687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3c916d90 con 0x559a3cab99d0 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb547400640 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cb2c100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb53405c0e0 tx=0x7fb53405e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9538123cd577f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb533e00640 1 -- 172.21.15.97:0/1678169687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5340682c0 con 0x559a3cab99d0 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb533e00640 1 -- 172.21.15.97:0/1678169687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb53405b0e0 con 0x559a3cab99d0 2024-09-10T21:25:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb533e00640 1 -- 172.21.15.97:0/1678169687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53404c470 con 0x559a3cab99d0 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- 172.21.15.97:0/1678169687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 msgr2=0x559a3cb2c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cb2c100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb53405c0e0 tx=0x7fb53405e980 comp rx=0 tx=0).stop 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- 172.21.15.97:0/1678169687 shutdown_connections 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb2c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3cab99d0 0x559a3cb2c100 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 --2- 172.21.15.97:0/1678169687 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3ca6de10 0x559a3cb2bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- 172.21.15.97:0/1678169687 >> 172.21.15.97:0/1678169687 conn(0x559a3c957130 msgr2=0x559a3cac6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- 172.21.15.97:0/1678169687 shutdown_connections 2024-09-10T21:25:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- 172.21.15.97:0/1678169687 wait complete. 2024-09-10T21:25:24.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 -- start start 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.703+0000 7fb5522e5000 1 Processor -- start 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3ca6de10 0x559a3cb2fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3cb30350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb30890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a3cad0ce0 con 0x559a3ca6de10 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a3c8c3390 con 0x559a3caba2f0 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559a3c8ca8f0 con 0x559a3cab99d0 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 10 client.0 _ll_register_callbacks cb 0x7fb540001500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3ca6de10 0x559a3cb2fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3ca6de10 0x559a3cb2fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54014/0 (socket says 172.21.15.97:54014) 2024-09-10T21:25:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 -- 172.21.15.97:0/998791543 learned_addr learned my addr 172.21.15.97:0/998791543 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547400640 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3cb30350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb54ce00640 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb30890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 -- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 msgr2=0x559a3cb30350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3cb30350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 -- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 msgr2=0x559a3cb30890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb30890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 -- 172.21.15.97:0/998791543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3c881a90 con 0x559a3ca6de10 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547e00640 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3ca6de10 0x559a3cb2fe10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb5380c2040 tx=0x7fb5380a50f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0dcc67bf96ae8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 1 -- 172.21.15.97:0/998791543 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5380b9040 con 0x559a3ca6de10 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 1 -- 172.21.15.97:0/998791543 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fb5380ce9e0 con 0x559a3ca6de10 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb53405b0e0 con 0x559a3ca6de10 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 1 -- 172.21.15.97:0/998791543 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5380d7050 con 0x559a3ca6de10 2024-09-10T21:25:24.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 10 client.5199 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x559a3c860050 con 0x559a3ca6de10 2024-09-10T21:25:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 1 -- 172.21.15.97:0/998791543 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb53405b0e0 con 0x559a3ca6de10 2024-09-10T21:25:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 1 -- 172.21.15.97:0/998791543 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7fb5380da020 con 0x559a3ca6de10 2024-09-10T21:25:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 10 client.5199 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 10 client.5199 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 20 client.5199 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:24.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3cacdbb0 0x559a3cacdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 4 client.5199 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:25:24.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "status"}) -- 0x559a3c8c9520 con 0x559a3cacdbb0 2024-09-10T21:25:24.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547400640 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3cacdbb0 0x559a3cacdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb547400640 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3cacdbb0 0x559a3cacdf90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c32bbb33cc522c server_cookie=c4dbce03e1f8b1ed in_seq=0 out_seq=0 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 10 client.5199 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 1 -- 172.21.15.97:0/998791543 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559a3c8c9520 con 0x559a3cacdbb0 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 10 client.5199 handle_command_reply: tid=0 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 client.5199 shutdown 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3ca6de10 msgr2=0x559a3cb2fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3ca6de10 0x559a3cb2fe10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb5380c2040 tx=0x7fb5380a50f0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 shutdown_connections 2024-09-10T21:25:24.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x559a3cacdbb0 0x559a3cacdf90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a3caba2f0 0x559a3cb30890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a3cab99d0 0x559a3cb30350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb531600640 0 client.5199 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 --2- 172.21.15.97:0/998791543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a3ca6de10 0x559a3cb2fe10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 >> 172.21.15.97:0/998791543 conn(0x559a3c957130 msgr2=0x559a3ca75b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 shutdown_connections 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 1 -- 172.21.15.97:0/998791543 wait complete. 2024-09-10T21:25:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb5522e5000 20 client.5199 trim_cache size 0 max 16384 2024-09-10T21:25:24.724 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:25:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:25:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:25:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5148, 2024-09-10T21:25:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:25:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:25:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:25:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 4.5539753630000002, 2024-09-10T21:25:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 24, 2024-09-10T21:25:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 41, 2024-09-10T21:25:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 41, 2024-09-10T21:25:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 5.4989226980000003 2024-09-10T21:25:24.725 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:25:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528072610 msgr2=0x7fb528074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb54ee00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528072610 0x7fb528074ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb52c004790 tx=0x7fb538019540 comp rx=0 tx=0).stop 2024-09-10T21:25:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5480b1ec0 msgr2=0x7fb5481fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.707+0000 7fb54ee00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5480b1ec0 0x7fb5481fb900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb534011a40 tx=0x7fb534011f00 comp rx=0 tx=0).stop 2024-09-10T21:25:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 shutdown_connections 2024-09-10T21:25:24.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528072610 0x7fb528074ad0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5481fc510 0x7fb5481fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5480b1ec0 0x7fb5481fb900 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 --2- 172.21.15.97:0/2947108911 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003ab0 0x7fb5480b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 >> 172.21.15.97:0/2947108911 conn(0x7fb5480d1cd0 msgr2=0x7fb5480cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 shutdown_connections 2024-09-10T21:25:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.711+0000 7fb54ee00640 1 -- 172.21.15.97:0/2947108911 wait complete. 2024-09-10T21:25:24.736 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:25:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- 172.21.15.97:0/2094956354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99500d27f0 msgr2=0x7f99500cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 --2- 172.21.15.97:0/2094956354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99500d27f0 0x7f99500cc550 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f993c009840 tx=0x7f993c01c820 comp rx=0 tx=0).stop 2024-09-10T21:25:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- 172.21.15.97:0/2094956354 shutdown_connections 2024-09-10T21:25:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 --2- 172.21.15.97:0/2094956354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99500d27f0 0x7f99500cc550 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- 172.21.15.97:0/2094956354 >> 172.21.15.97:0/2094956354 conn(0x7f99500d1cd0 msgr2=0x7f99500d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- 172.21.15.97:0/2094956354 shutdown_connections 2024-09-10T21:25:24.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- 172.21.15.97:0/2094956354 wait complete. 2024-09-10T21:25:24.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 Processor -- start 2024-09-10T21:25:24.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- start start 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99500cca90 0x7f995014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99500d27f0 0x7f9950155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f995015eed0 0x7f99500b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9950169fc0 con 0x7f995015eed0 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9950169e40 con 0x7f99500cca90 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9957000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f995016a140 con 0x7f99500d27f0 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f995015eed0 0x7f99500b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f995015eed0 0x7f99500b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54026/0 (socket says 172.21.15.97:54026) 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 -- 172.21.15.97:0/628458654 learned_addr learned my addr 172.21.15.97:0/628458654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f994f400640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99500d27f0 0x7f9950155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f994fe00640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99500cca90 0x7f995014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 -- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99500d27f0 msgr2=0x7f9950155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99500d27f0 0x7f9950155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 -- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99500cca90 msgr2=0x7f995014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99500cca90 0x7f995014d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.871+0000 7f9955000640 1 -- 172.21.15.97:0/628458654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99500b6d50 con 0x7f995015eed0 2024-09-10T21:25:24.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9955000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f995015eed0 0x7f99500b71e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f994800c4d0 tx=0x7f994800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9947e00640 1 -- 172.21.15.97:0/628458654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99480052d0 con 0x7f995015eed0 2024-09-10T21:25:24.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9947e00640 1 -- 172.21.15.97:0/628458654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9948005470 con 0x7f995015eed0 2024-09-10T21:25:24.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9947e00640 1 -- 172.21.15.97:0/628458654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948012870 con 0x7f995015eed0 2024-09-10T21:25:24.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99500b4d40 con 0x7f995015eed0 2024-09-10T21:25:24.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99500b5300 con 0x7f995015eed0 2024-09-10T21:25:24.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99500cc550 con 0x7f995015eed0 2024-09-10T21:25:24.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9947e00640 1 -- 172.21.15.97:0/628458654 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9948003600 con 0x7f995015eed0 2024-09-10T21:25:24.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9947e00640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99300725e0 0x7f9930074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.875+0000 7f9947e00640 1 -- 172.21.15.97:0/628458654 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f9948092e70 con 0x7f995015eed0 2024-09-10T21:25:24.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.879+0000 7f994fe00640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99300725e0 0x7f9930074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.879+0000 7f994fe00640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99300725e0 0x7f9930074aa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f99400012f0 tx=0x7f9940009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:24.879+0000 7f9947e00640 1 -- 172.21.15.97:0/628458654 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f994805dc80 con 0x7f995015eed0 2024-09-10T21:25:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f99500cfd60 con 0x7f995015eed0 2024-09-10T21:25:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9947e00640 1 -- 172.21.15.97:0/628458654 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f99480626b0 con 0x7f995015eed0 2024-09-10T21:25:25.086 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99300725e0 msgr2=0x7f9930074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99300725e0 0x7f9930074aa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f99400012f0 tx=0x7f9940009040 comp rx=0 tx=0).stop 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f995015eed0 msgr2=0x7f99500b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f995015eed0 0x7f99500b71e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f994800c4d0 tx=0x7f994800c990 comp rx=0 tx=0).stop 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 shutdown_connections 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99300725e0 0x7f9930074aa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f995015eed0 0x7f99500b71e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99500d27f0 0x7f9950155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 --2- 172.21.15.97:0/628458654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99500cca90 0x7f995014d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 >> 172.21.15.97:0/628458654 conn(0x7f99500d1cd0 msgr2=0x7f9950156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 shutdown_connections 2024-09-10T21:25:25.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25.079+0000 7f9957000640 1 -- 172.21.15.97:0/628458654 wait complete. 2024-09-10T21:25:25.106 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:25:25.107 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:25.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.107 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:25:25.116 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:25:25.116 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:25:25.116 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:25.117 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42368sec preferred_lft 42368sec 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:25:25.119 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:25.119 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:25:25.119 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.119 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:25:25.119 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:25:25.119 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:25:25.119 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:25.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.120 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:25:25.187 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:25:25.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.189 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:25:25.244 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:25:25.244 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:25:25.244 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:25:25.244 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:25:25.244 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:25:25.245 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:25:25.245 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:25:25.245 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:25.245 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:25.245 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:25.245 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:25:25.245 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:25.245 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:25:25.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:25:25.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:25:25.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:25:25.246 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:25:25.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:25.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.384 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:25:25.393 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:25:25.393 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:25:25.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:25.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:25:25.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:25.455 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:25.504 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:25.504 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:25.504 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:25.560 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:25:25.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:25.652 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:25.652 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:25.652 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:25.667 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:25.667 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:25.667 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:25.667 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:25:25.668 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:25:25.676 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:25.676 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:25.676 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:25.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:25:25.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:25:25.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:25:25.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:25:25.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:25:25.680 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:25:25.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:25.700 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:25:25.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:25:25.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:25.800 INFO:teuthology.orchestra.run.smithi097.stdout:client5211 2024-09-10T21:25:25.800 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:25:25.800 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:25.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.809 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-09-10T21:25:25.863 INFO:teuthology.orchestra.run.smithi097.stdout:file_undamaged 2024-09-10T21:25:25.864 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:25:25.864 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:25:25.864 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:25:25.864 DEBUG:teuthology.orchestra.run.smithi097:> import json 2024-09-10T21:25:25.864 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(e.errno) 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> print(json.dumps( 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T21:25:25.865 DEBUG:teuthology.orchestra.run.smithi097:> indent=2)) 2024-09-10T21:25:25.866 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:25:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:25 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-09-10T21:25:25.988 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T21:25:25.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:26.162 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:25:26.155+0000 7fc9d0000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:25:26.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.167+0000 7f47fca00640 1 -- 172.21.15.97:0/2719634994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f815efc0 msgr2=0x7f47f80cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.167+0000 7f47fca00640 1 --2- 172.21.15.97:0/2719634994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f815efc0 0x7f47f80cf840 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009900 tx=0x7f47e801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.167+0000 7f47fca00640 1 -- 172.21.15.97:0/2719634994 shutdown_connections 2024-09-10T21:25:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.167+0000 7f47fca00640 1 --2- 172.21.15.97:0/2719634994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f815efc0 0x7f47f80cf840 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.167+0000 7f47fca00640 1 -- 172.21.15.97:0/2719634994 >> 172.21.15.97:0/2719634994 conn(0x7f47f80d1a60 msgr2=0x7f47f80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.167+0000 7f47fca00640 1 -- 172.21.15.97:0/2719634994 shutdown_connections 2024-09-10T21:25:26.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.167+0000 7f47fca00640 1 -- 172.21.15.97:0/2719634994 wait complete. 2024-09-10T21:25:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 Processor -- start 2024-09-10T21:25:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 -- start start 2024-09-10T21:25:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47f80cdcb0 0x7f47f806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f80d0490 0x7f47f81fae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f815efc0 0x7f47f81fb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47f8169b50 con 0x7f47f80d0490 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81699d0 con 0x7f47f815efc0 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f47f8169cd0 con 0x7f47f80cdcb0 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f80d0490 0x7f47f81fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f80d0490 0x7f47f81fae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54052/0 (socket says 172.21.15.97:54052) 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 -- 172.21.15.97:0/774436988 learned_addr learned my addr 172.21.15.97:0/774436988 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f7400640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47f80cdcb0 0x7f47f806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f7e00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f815efc0 0x7f47f81fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 -- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47f80cdcb0 msgr2=0x7f47f806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47f80cdcb0 0x7f47f806e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 -- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f815efc0 msgr2=0x7f47f81fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f815efc0 0x7f47f81fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 -- 172.21.15.97:0/774436988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f80b3240 con 0x7f47f80d0490 2024-09-10T21:25:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47f6a00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f80d0490 0x7f47f81fae00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c3f0 tx=0x7f47e400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47ef400640 1 -- 172.21.15.97:0/774436988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4017070 con 0x7f47f80d0490 2024-09-10T21:25:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47ef400640 1 -- 172.21.15.97:0/774436988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e40053c0 con 0x7f47f80d0490 2024-09-10T21:25:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f806e820 con 0x7f47f80d0490 2024-09-10T21:25:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47ef400640 1 -- 172.21.15.97:0/774436988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4003860 con 0x7f47f80d0490 2024-09-10T21:25:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.171+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f806ede0 con 0x7f47f80d0490 2024-09-10T21:25:26.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.175+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f80ae930 con 0x7f47f80d0490 2024-09-10T21:25:26.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.175+0000 7f47ef400640 1 -- 172.21.15.97:0/774436988 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f47e4003a00 con 0x7f47f80d0490 2024-09-10T21:25:26.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.175+0000 7f47ef400640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47d80725e0 0x7f47d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.175+0000 7f47ef400640 1 -- 172.21.15.97:0/774436988 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f47e4092920 con 0x7f47f80d0490 2024-09-10T21:25:26.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.175+0000 7f47f7400640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47d80725e0 0x7f47d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.175+0000 7f47f7400640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47d80725e0 0x7f47d8074aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47e80af000 tx=0x7f47e801b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.179+0000 7f47ef400640 1 -- 172.21.15.97:0/774436988 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47e405d7c0 con 0x7f47f80d0490 2024-09-10T21:25:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.383+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47f80b2280 con 0x7f47f80d0490 2024-09-10T21:25:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.383+0000 7f47ef400640 1 -- 172.21.15.97:0/774436988 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5221 (secure 0 0 0) 0x7f47e40621f0 con 0x7f47f80d0490 2024-09-10T21:25:26.391 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:26.391 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"btime":"2024-09-10T21:25:23:161523+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:23.161515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5148,"qdb_cluster":[5148]},"id":2}]} 2024-09-10T21:25:26.391 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.383+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47d80725e0 msgr2=0x7f47d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.383+0000 7f47fca00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47d80725e0 0x7f47d8074aa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47e80af000 tx=0x7f47e801b120 comp rx=0 tx=0).stop 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f80d0490 msgr2=0x7f47f81fae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f80d0490 0x7f47f81fae00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c3f0 tx=0x7f47e400c8b0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 shutdown_connections 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47d80725e0 0x7f47d8074aa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f815efc0 0x7f47f81fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f80d0490 0x7f47f81fae00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 --2- 172.21.15.97:0/774436988 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f47f80cdcb0 0x7f47f806e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 >> 172.21.15.97:0/774436988 conn(0x7f47f80d1a60 msgr2=0x7f47f8157070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 shutdown_connections 2024-09-10T21:25:26.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.387+0000 7f47fca00640 1 -- 172.21.15.97:0/774436988 wait complete. 2024-09-10T21:25:26.413 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-10T21:25:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 -- 172.21.15.97:0/2075660289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 msgr2=0x7f208c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 --2- 172.21.15.97:0/2075660289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 0x7f208c004c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2090009b30 tx=0x7f209001d230 comp rx=0 tx=0).stop 2024-09-10T21:25:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a6800640 1 -- 172.21.15.97:0/2075660289 reap_dead start 2024-09-10T21:25:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 -- 172.21.15.97:0/2075660289 shutdown_connections 2024-09-10T21:25:26.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 -- 172.21.15.97:0/2075660289 >> 172.21.15.97:0/2075660289 conn(0x7f20a80d0090 msgr2=0x7f20a80d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 -- 172.21.15.97:0/2075660289 shutdown_connections 2024-09-10T21:25:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 -- 172.21.15.97:0/2075660289 wait complete. 2024-09-10T21:25:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 Processor -- start 2024-09-10T21:25:26.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.551+0000 7f20a7200640 1 -- start start 2024-09-10T21:25:26.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 0x7f20a809d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20a8159aa0 0x7f20a80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20a80cc0d0 0x7f20a80c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20a80d0d90 con 0x7f208c004810 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f20a80b2620 con 0x7f20a8159aa0 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f20a80d0f10 con 0x7f20a80cc0d0 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 0x7f20a809d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 0x7f20a809d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54062/0 (socket says 172.21.15.97:54062) 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 -- 172.21.15.97:0/3541205707 learned_addr learned my addr 172.21.15.97:0/3541205707 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a6800640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20a80cc0d0 0x7f20a80c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f209fe00640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20a8159aa0 0x7f20a80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 -- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20a80cc0d0 msgr2=0x7f20a80c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20a80cc0d0 0x7f20a80c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 -- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20a8159aa0 msgr2=0x7f20a80c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20a8159aa0 0x7f20a80c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 -- 172.21.15.97:0/3541205707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a81f79a0 con 0x7f208c004810 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f209fe00640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20a8159aa0 0x7f20a80c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a6800640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20a80cc0d0 0x7f20a80c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a5e00640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 0x7f20a809d670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f209800ea40 tx=0x7f209800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f209e000640 1 -- 172.21.15.97:0/3541205707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209800dcd0 con 0x7f208c004810 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f209e000640 1 -- 172.21.15.97:0/3541205707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f209800de70 con 0x7f208c004810 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a81f7c40 con 0x7f208c004810 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f209e000640 1 -- 172.21.15.97:0/3541205707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20980042b0 con 0x7f208c004810 2024-09-10T21:25:26.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81f8200 con 0x7f208c004810 2024-09-10T21:25:26.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.555+0000 7f209e000640 1 -- 172.21.15.97:0/3541205707 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f209800b040 con 0x7f208c004810 2024-09-10T21:25:26.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f209e000640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20840725e0 0x7f2084074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f209e000640 1 -- 172.21.15.97:0/3541205707 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f2098093c90 con 0x7f208c004810 2024-09-10T21:25:26.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 Processor -- start 2024-09-10T21:25:26.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- start start 2024-09-10T21:25:26.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f209fe00640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20840725e0 0x7f2084074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f209fe00640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20840725e0 0x7f2084074aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f20900ae000 tx=0x7f2090019fd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3b370 0x55d6c8a3b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3bc90 0x55d6c89e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c89e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6c88d0ce0 con 0x55d6c89e92c0 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d6c8844c70 con 0x55d6c8a3bc90 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d6c88340a0 con 0x55d6c8a3b370 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c89e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a5e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3b370 0x55d6c8a3b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c89e96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54066/0 (socket says 172.21.15.97:54066) 2024-09-10T21:25:26.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a6800640 1 -- 172.21.15.97:0/1937460036 learned_addr learned my addr 172.21.15.97:0/1937460036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:26.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f209fe00640 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3bc90 0x55d6c89e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a5e00640 1 -- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3bc90 msgr2=0x55d6c89e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a5e00640 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3bc90 0x55d6c89e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a5e00640 1 -- 172.21.15.97:0/1937460036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 msgr2=0x55d6c89e96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a5e00640 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c89e96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a5e00640 1 -- 172.21.15.97:0/1937460036 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c880f270 con 0x55d6c8a3b370 2024-09-10T21:25:26.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a6800640 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c89e96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:26.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20a5e00640 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3b370 0x55d6c8a3b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2098016000 tx=0x7f209805e490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fef21bdd964bfcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f2096a00640 1 -- 172.21.15.97:0/1937460036 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098083030 con 0x55d6c8a3b370 2024-09-10T21:25:26.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f2096a00640 1 -- 172.21.15.97:0/1937460036 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f209805f180 con 0x55d6c8a3b370 2024-09-10T21:25:26.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f2096a00640 1 -- 172.21.15.97:0/1937460036 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209805ea80 con 0x55d6c8a3b370 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3b370 msgr2=0x55d6c8a3b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3b370 0x55d6c8a3b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2098016000 tx=0x7f209805e490 comp rx=0 tx=0).stop 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- 172.21.15.97:0/1937460036 shutdown_connections 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c89e96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3bc90 0x55d6c89e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 --2- 172.21.15.97:0/1937460036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3b370 0x55d6c8a3b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- 172.21.15.97:0/1937460036 >> 172.21.15.97:0/1937460036 conn(0x55d6c89e5270 msgr2=0x55d6c89e7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- 172.21.15.97:0/1937460036 shutdown_connections 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.559+0000 7f20b0001000 1 -- 172.21.15.97:0/1937460036 wait complete. 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 -- start start 2024-09-10T21:25:26.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 Processor -- start 2024-09-10T21:25:26.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c89e92c0 0x55d6c8a422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c8a3b370 0x55d6c8a42820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8a42d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6c87ce920 con 0x55d6c8a3b370 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d6c8837eb0 con 0x55d6c89e92c0 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d6c84f9820 con 0x55d6c8a3bc90 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 10 client.0 _ll_register_callbacks cb 0x55d6c88127d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c8a3b370 0x55d6c8a42820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c8a3b370 0x55d6c8a42820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54068/0 (socket says 172.21.15.97:54068) 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 -- 172.21.15.97:0/1568597750 learned_addr learned my addr 172.21.15.97:0/1568597750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20a6800640 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8a42d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20a5e00640 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c89e92c0 0x55d6c8a422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 -- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 msgr2=0x55d6c8a42d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8a42d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 -- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c89e92c0 msgr2=0x55d6c8a422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c89e92c0 0x55d6c8a422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 -- 172.21.15.97:0/1568597750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c87e7530 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f209fe00640 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c8a3b370 0x55d6c8a42820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f209001d060 tx=0x7f20900091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66cbefee12f819e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f208b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f208b400640 1 -- 172.21.15.97:0/1568597750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20900c9030 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6c87cc3e0 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f208b400640 1 -- 172.21.15.97:0/1568597750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f20900b7510 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f208b400640 1 -- 172.21.15.97:0/1568597750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20900b1040 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 10 client.5232 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55d6c87d2460 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f208b400640 1 -- 172.21.15.97:0/1568597750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d6c87cc3e0 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f208b400640 1 -- 172.21.15.97:0/1568597750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f20900047d0 con 0x55d6c8a3b370 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 10 client.5232 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:26.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 10 client.5232 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 20 client.5232 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a4a6b0 0x55d6c8a4caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 4 client.5232 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6c8846bc0 con 0x55d6c8a4a6b0 2024-09-10T21:25:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.563+0000 7f20a5e00640 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a4a6b0 0x55d6c8a4caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20a5e00640 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a4a6b0 0x55d6c8a4caa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a92605b959f09db server_cookie=7130e10f3b4db955 in_seq=0 out_seq=0 2024-09-10T21:25:26.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f208b400640 10 client.5232 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f208b400640 1 -- 172.21.15.97:0/1568597750 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d6c8846bc0 con 0x55d6c8a4a6b0 2024-09-10T21:25:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f208b400640 10 client.5232 handle_command_reply: tid=0 2024-09-10T21:25:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 client.5232 shutdown 2024-09-10T21:25:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c8a3b370 msgr2=0x55d6c8a42820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c8a3b370 0x55d6c8a42820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f209001d060 tx=0x7f20900091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 shutdown_connections 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a4a6b0 0x55d6c8a4caa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8a42d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c8a3b370 0x55d6c8a42820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 --2- 172.21.15.97:0/1568597750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c89e92c0 0x55d6c8a422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f208b400640 0 client.5232 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 >> 172.21.15.97:0/1568597750 conn(0x55d6c89e5270 msgr2=0x55d6c89e7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 shutdown_connections 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 1 -- 172.21.15.97:0/1568597750 wait complete. 2024-09-10T21:25:26.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.567+0000 7f20b0001000 20 client.5232 trim_cache size 0 max 16384 2024-09-10T21:25:26.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 Processor -- start 2024-09-10T21:25:26.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 -- start start 2024-09-10T21:25:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8a432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 0x55d6c8a43830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8a43d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6c89f24f0 con 0x55d6c89e92c0 2024-09-10T21:25:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d6c88f3fa0 con 0x55d6c8a3b370 2024-09-10T21:25:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.579+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d6c87df2e0 con 0x55d6c8a3bc90 2024-09-10T21:25:26.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8a432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8a432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54082/0 (socket says 172.21.15.97:54082) 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/4185605306 learned_addr learned my addr 172.21.15.97:0/4185605306 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/4185605306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 msgr2=0x55d6c8a43d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- 172.21.15.97:0/4185605306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8a43d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/4185605306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 msgr2=0x55d6c8a43830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- 172.21.15.97:0/4185605306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 0x55d6c8a43830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/4185605306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c88b9cf0 con 0x55d6c89e92c0 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- 172.21.15.97:0/4185605306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8a432f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f209805ea10 tx=0x7f209805e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4e8db0bf36e9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f2096000640 1 -- 172.21.15.97:0/4185605306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098083040 con 0x55d6c89e92c0 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f2096000640 1 -- 172.21.15.97:0/4185605306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f209805ea80 con 0x55d6c89e92c0 2024-09-10T21:25:26.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f2096000640 1 -- 172.21.15.97:0/4185605306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209805b0d0 con 0x55d6c89e92c0 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- 172.21.15.97:0/4185605306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 msgr2=0x55d6c8a432f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 --2- 172.21.15.97:0/4185605306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8a432f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f209805ea10 tx=0x7f209805e490 comp rx=0 tx=0).stop 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- 172.21.15.97:0/4185605306 shutdown_connections 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 --2- 172.21.15.97:0/4185605306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8a43d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 --2- 172.21.15.97:0/4185605306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 0x55d6c8a43830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 --2- 172.21.15.97:0/4185605306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8a432f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- 172.21.15.97:0/4185605306 >> 172.21.15.97:0/4185605306 conn(0x55d6c89e5270 msgr2=0x55d6c89e84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- 172.21.15.97:0/4185605306 shutdown_connections 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- 172.21.15.97:0/4185605306 wait complete. 2024-09-10T21:25:26.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- start start 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 Processor -- start 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8ab0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 0x55d6c8ab1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8ab16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6c87f1650 con 0x55d6c89e92c0 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d6c8843a40 con 0x55d6c8a3b370 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d6c88002e0 con 0x55d6c8a3bc90 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20b0001000 10 client.0 _ll_register_callbacks cb 0x7f208c003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8ab0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8ab0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54090/0 (socket says 172.21.15.97:54090) 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/3718766281 learned_addr learned my addr 172.21.15.97:0/3718766281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f209fe00640 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 0x55d6c8ab1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a6800640 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8ab16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 msgr2=0x55d6c8ab16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8ab16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 msgr2=0x55d6c8ab1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 0x55d6c8ab1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.583+0000 7f20a5e00640 1 -- 172.21.15.97:0/3718766281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c87eae30 con 0x55d6c89e92c0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a5e00640 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8ab0c20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f209805bcf0 tx=0x7f209800abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0bb5d76372436d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 1 -- 172.21.15.97:0/3718766281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209808c070 con 0x55d6c89e92c0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 1 -- 172.21.15.97:0/3718766281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f2098063150 con 0x55d6c89e92c0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f209805ea80 con 0x55d6c89e92c0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 1 -- 172.21.15.97:0/3718766281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f209808a050 con 0x55d6c89e92c0 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 10 client.5238 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:26.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55d6c87d92b0 con 0x55d6c89e92c0 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 1 -- 172.21.15.97:0/3718766281 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f209805ea80 con 0x55d6c89e92c0 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 1 -- 172.21.15.97:0/3718766281 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f209805ece0 con 0x55d6c89e92c0 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 10 client.5238 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 10 client.5238 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 20 client.5238 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a46670 0x55d6c8a46a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 4 client.5238 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55d6c8a52dc0 con 0x55d6c8a46670 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f209fe00640 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a46670 0x55d6c8a46a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f209fe00640 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a46670 0x55d6c8a46a50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24eb342e07b14877 server_cookie=47617b2b8437753 in_seq=0 out_seq=0 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 10 client.5238 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 1 -- 172.21.15.97:0/3718766281 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x55d6c8a52dc0 con 0x55d6c8a46670 2024-09-10T21:25:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 10 client.5238 handle_command_reply: tid=0 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 client.5238 shutdown 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 msgr2=0x55d6c8ab0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8ab0c20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f209805bcf0 tx=0x7f209800abb0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 shutdown_connections 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55d6c8a46670 0x55d6c8a46a50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d6c8a3bc90 0x55d6c8ab16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d6c8a3b370 0x55d6c8ab1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 --2- 172.21.15.97:0/3718766281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6c89e92c0 0x55d6c8ab0c20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f208b400640 0 client.5238 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 >> 172.21.15.97:0/3718766281 conn(0x55d6c89e5270 msgr2=0x55d6c89ed9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 shutdown_connections 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 1 -- 172.21.15.97:0/3718766281 wait complete. 2024-09-10T21:25:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20b0001000 20 client.5238 trim_cache size 0 max 16384 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "dentry", 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 2054577795, 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511627776, 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: "frag": "*", 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: "dname": "file_to_be_damaged", 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_id": "head", 2024-09-10T21:25:26.600 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/subdir/file_to_be_damaged" 2024-09-10T21:25:26.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:26.601 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:25:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20840725e0 msgr2=0x7f2084074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20840725e0 0x7f2084074aa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f20900ae000 tx=0x7f2090019fd0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 msgr2=0x7f20a809d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 0x7f20a809d670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f209800ea40 tx=0x7f209800ef00 comp rx=0 tx=0).stop 2024-09-10T21:25:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 shutdown_connections 2024-09-10T21:25:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20840725e0 0x7f2084074aa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20a80cc0d0 0x7f20a80c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20a8159aa0 0x7f20a80c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 --2- 172.21.15.97:0/3541205707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f208c004810 0x7f20a809d670 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 >> 172.21.15.97:0/3541205707 conn(0x7f20a80d0090 msgr2=0x7f20a8157200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 shutdown_connections 2024-09-10T21:25:26.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26.587+0000 7f20a7200640 1 -- 172.21.15.97:0/3541205707 wait complete. 2024-09-10T21:25:26.614 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:25:26.614 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-09-10T21:25:26.614 DEBUG:teuthology.orchestra.run.smithi097:> import errno 2024-09-10T21:25:26.615 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:25:26.615 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-09-10T21:25:26.615 DEBUG:teuthology.orchestra.run.smithi097:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-10T21:25:26.615 DEBUG:teuthology.orchestra.run.smithi097:> f.close() 2024-09-10T21:25:26.615 DEBUG:teuthology.orchestra.run.smithi097:> except IOError as e: 2024-09-10T21:25:26.615 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(errno.EIO) 2024-09-10T21:25:26.615 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:25:26.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T21:25:26.688 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> import json 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(e.errno) 2024-09-10T21:25:26.689 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:25:26.690 DEBUG:teuthology.orchestra.run.smithi097:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T21:25:26.690 DEBUG:teuthology.orchestra.run.smithi097:> print(json.dumps( 2024-09-10T21:25:26.690 DEBUG:teuthology.orchestra.run.smithi097:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T21:25:26.690 DEBUG:teuthology.orchestra.run.smithi097:> indent=2)) 2024-09-10T21:25:26.690 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:25:26.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:26 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T21:25:26.811 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T21:25:26.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:26.812 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-10T21:25:26.866 INFO:teuthology.orchestra.run.smithi097.stdout:2 2024-09-10T21:25:26.866 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:25:26.866 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:26.921 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:25:26.921 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:26.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:25:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- 172.21.15.97:0/1909343733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4680ce5f0 msgr2=0x7ff468141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 --2- 172.21.15.97:0/1909343733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4680ce5f0 0x7ff468141d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff458009bc0 tx=0x7ff45801cba0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- 172.21.15.97:0/1909343733 shutdown_connections 2024-09-10T21:25:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 --2- 172.21.15.97:0/1909343733 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff46815fde0 0x7ff46813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 --2- 172.21.15.97:0/1909343733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4680ce5f0 0x7ff468141d20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- 172.21.15.97:0/1909343733 >> 172.21.15.97:0/1909343733 conn(0x7ff4680b1780 msgr2=0x7ff4680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- 172.21.15.97:0/1909343733 shutdown_connections 2024-09-10T21:25:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- 172.21.15.97:0/1909343733 wait complete. 2024-09-10T21:25:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 Processor -- start 2024-09-10T21:25:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- start start 2024-09-10T21:25:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4680ce5f0 0x7ff46813c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4680ce9d0 0x7ff4681f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff46815fde0 0x7ff4681f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff468168bf0 con 0x7ff46815fde0 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff468168a70 con 0x7ff4680ce5f0 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff46de00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff468168d70 con 0x7ff4680ce9d0 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff467400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff46815fde0 0x7ff4681f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff467400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff46815fde0 0x7ff4681f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54114/0 (socket says 172.21.15.97:54114) 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff467400640 1 -- 172.21.15.97:0/2717200849 learned_addr learned my addr 172.21.15.97:0/2717200849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff45e000640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4680ce9d0 0x7ff4681f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.155+0000 7ff466a00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4680ce5f0 0x7ff46813c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff467400640 1 -- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4680ce9d0 msgr2=0x7ff4681f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff467400640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4680ce9d0 0x7ff4681f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff467400640 1 -- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4680ce5f0 msgr2=0x7ff46813c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff467400640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4680ce5f0 0x7ff46813c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff467400640 1 -- 172.21.15.97:0/2717200849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff468074060 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff467400640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff46815fde0 0x7ff4681f8590 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff46000d660 tx=0x7ff46000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff45fe00640 1 -- 172.21.15.97:0/2717200849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460017070 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff45fe00640 1 -- 172.21.15.97:0/2717200849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff460005310 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff468074340 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff45fe00640 1 -- 172.21.15.97:0/2717200849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460009820 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff468072e90 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff468141d20 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff45fe00640 1 -- 172.21.15.97:0/2717200849 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff4600099c0 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff45fe00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff4440725e0 0x7ff444074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.159+0000 7ff45fe00640 1 -- 172.21.15.97:0/2717200849 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff460093480 con 0x7ff46815fde0 2024-09-10T21:25:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.163+0000 7ff466a00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff4440725e0 0x7ff444074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.163+0000 7ff466a00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff4440725e0 0x7ff444074aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4580af000 tx=0x7ff45801b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.167+0000 7ff45fe00640 1 -- 172.21.15.97:0/2717200849 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff46005e320 con 0x7ff46815fde0 2024-09-10T21:25:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff4680cca70 con 0x7ff46815fde0 2024-09-10T21:25:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff45fe00640 1 -- 172.21.15.97:0/2717200849 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) ==== 80+0+122 (secure 0 0 0) 0x7ff460067090 con 0x7ff46815fde0 2024-09-10T21:25:27.349 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2127253870 2024-09-11T21:25:19.147709+0000 2024-09-10T21:25:27.350 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2127253870 2024-09-11T21:25:19.147709+0000 2024-09-10T21:25:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:listed 2 entries 2024-09-10T21:25:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff4440725e0 msgr2=0x7ff444074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff4440725e0 0x7ff444074aa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4580af000 tx=0x7ff45801b3e0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff46815fde0 msgr2=0x7ff4681f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff46815fde0 0x7ff4681f8590 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff46000d660 tx=0x7ff46000db20 comp rx=0 tx=0).stop 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 shutdown_connections 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff4440725e0 0x7ff444074aa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff46815fde0 0x7ff4681f8590 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4680ce9d0 0x7ff4681f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 --2- 172.21.15.97:0/2717200849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4680ce5f0 0x7ff46813c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 >> 172.21.15.97:0/2717200849 conn(0x7ff4680b1780 msgr2=0x7ff46813d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 shutdown_connections 2024-09-10T21:25:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.343+0000 7ff46de00640 1 -- 172.21.15.97:0/2717200849 wait complete. 2024-09-10T21:25:27.371 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:25:27.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:27.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:25:27.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.396 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:25:27.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2024-09-10T21:25:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- 172.21.15.97:0/2969994608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0cec10 msgr2=0x7f30ec0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 --2- 172.21.15.97:0/2969994608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0cec10 0x7f30ec0cfd40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0735a0 tx=0x7f30d801f0b0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- 172.21.15.97:0/2969994608 shutdown_connections 2024-09-10T21:25:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 --2- 172.21.15.97:0/2969994608 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f30ec0d0280 0x7f30ec15fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 --2- 172.21.15.97:0/2969994608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0cec10 0x7f30ec0cfd40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 --2- 172.21.15.97:0/2969994608 >> v2:172.21.15.194:3300/0 conn(0x7f30ec0ce830 0x7f30ec0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- 172.21.15.97:0/2969994608 >> 172.21.15.97:0/2969994608 conn(0x7f30ec0b37c0 msgr2=0x7f30ec0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- 172.21.15.97:0/2969994608 shutdown_connections 2024-09-10T21:25:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- 172.21.15.97:0/2969994608 wait complete. 2024-09-10T21:25:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 Processor -- start 2024-09-10T21:25:27.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- start start 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f30ec0ce830 0x7f30ec1f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30ec0cec10 0x7f30ec0c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0d0280 0x7f30ec150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0b4450 con 0x7f30ec0d0280 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0cde60 con 0x7f30ec0cec10 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30f1200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec0ce1a0 con 0x7f30ec0ce830 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0d0280 0x7f30ec150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0d0280 0x7f30ec150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53362/0 (socket says 172.21.15.97:53362) 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 -- 172.21.15.97:0/299140588 learned_addr learned my addr 172.21.15.97:0/299140588 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 -- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f30ec0ce830 msgr2=0x7f30ec1f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f30ec0ce830 0x7f30ec1f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 -- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30ec0cec10 msgr2=0x7f30ec0c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30ec0cec10 0x7f30ec0c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 -- 172.21.15.97:0/299140588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec1f6b00 con 0x7f30ec0d0280 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30ea800640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0d0280 0x7f30ec150030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f30e400eab0 tx=0x7f30e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30e2000640 1 -- 172.21.15.97:0/299140588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e400cea0 con 0x7f30ec0d0280 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.635+0000 7f30e2000640 1 -- 172.21.15.97:0/299140588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e4004b30 con 0x7f30ec0d0280 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30e2000640 1 -- 172.21.15.97:0/299140588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4005020 con 0x7f30ec0d0280 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec1f6260 con 0x7f30ec0d0280 2024-09-10T21:25:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1f6820 con 0x7f30ec0d0280 2024-09-10T21:25:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30e2000640 1 -- 172.21.15.97:0/299140588 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f30e40051c0 con 0x7f30ec0d0280 2024-09-10T21:25:27.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30f46fd000 1 Processor -- start 2024-09-10T21:25:27.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30e2000640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f30cc0725e0 0x7f30cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30f46fd000 1 -- start start 2024-09-10T21:25:27.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30e2000640 1 -- 172.21.15.97:0/299140588 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f30e4093fd0 con 0x7f30ec0d0280 2024-09-10T21:25:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.639+0000 7f30e9400640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f30cc0725e0 0x7f30cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 0x555d86db4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86db5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555d86c0ba40 con 0x555d86dfde80 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555d86bfae70 con 0x555d86dfe7a0 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555d86bfec80 con 0x555d86db4c90 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53372/0 (socket says 172.21.15.97:53372) 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 -- 172.21.15.97:0/2044467556 learned_addr learned my addr 172.21.15.97:0/2044467556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9400640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f30cc0725e0 0x7f30cc074aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30d80b2000 tx=0x7f30d801be50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9400640 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 0x555d86db4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30ea800640 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86db5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 -- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 msgr2=0x555d86db5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86db5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 -- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 msgr2=0x555d86db4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 0x555d86db4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 -- 172.21.15.97:0/2044467556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d86c44040 con 0x555d86dfde80 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30e9e00640 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfe260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009c50 tx=0x7f30dc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad61883786e88594 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30d6a00640 1 -- 172.21.15.97:0/2044467556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc005020 con 0x555d86dfde80 2024-09-10T21:25:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30d6a00640 1 -- 172.21.15.97:0/2044467556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30dc0051c0 con 0x555d86dfde80 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30d6a00640 1 -- 172.21.15.97:0/2044467556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0055d0 con 0x555d86dfde80 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- 172.21.15.97:0/2044467556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 msgr2=0x555d86dfe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfe260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009c50 tx=0x7f30dc00e720 comp rx=0 tx=0).stop 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- 172.21.15.97:0/2044467556 shutdown_connections 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86db5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 0x555d86db4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 --2- 172.21.15.97:0/2044467556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfe260 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- 172.21.15.97:0/2044467556 >> 172.21.15.97:0/2044467556 conn(0x555d86dac800 msgr2=0x555d86daec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- 172.21.15.97:0/2044467556 shutdown_connections 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- 172.21.15.97:0/2044467556 wait complete. 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 -- start start 2024-09-10T21:25:27.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.643+0000 7f30f46fd000 1 Processor -- start 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86de85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfde80 0x555d86de8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfe7a0 0x555d86de9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555d86c3eb60 con 0x555d86dfe7a0 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555d86ba4950 con 0x555d86dfde80 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555d86bb0bb0 con 0x555d86db4c90 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 10 client.0 _ll_register_callbacks cb 0x555d86ba13f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfe7a0 0x555d86de9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfe7a0 0x555d86de9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53388/0 (socket says 172.21.15.97:53388) 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 -- 172.21.15.97:0/170585119 learned_addr learned my addr 172.21.15.97:0/170585119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30e9400640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfde80 0x555d86de8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30e9e00640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86de85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 -- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 msgr2=0x555d86de85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86de85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 -- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfde80 msgr2=0x555d86de8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfde80 0x555d86de8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 -- 172.21.15.97:0/170585119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d86ba3b30 con 0x555d86dfe7a0 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30e9400640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfde80 0x555d86de8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30e9e00640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86de85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30ea800640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfe7a0 0x555d86de9050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c760 tx=0x7f30e405f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b23b3a1bd3f4753 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:27.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 1 -- 172.21.15.97:0/170585119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4090060 con 0x555d86dfe7a0 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 1 -- 172.21.15.97:0/170585119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f30e405e020 con 0x555d86dfe7a0 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d86b931e0 con 0x555d86dfe7a0 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 1 -- 172.21.15.97:0/170585119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f30e4078070 con 0x555d86dfe7a0 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 10 client.5259 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x555d86b990a0 con 0x555d86dfe7a0 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 1 -- 172.21.15.97:0/170585119 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555d86b931e0 con 0x555d86dfe7a0 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 1 -- 172.21.15.97:0/170585119 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f30e4082040 con 0x555d86dfe7a0 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 10 client.5259 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 10 client.5259 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 20 client.5259 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e0d120 0x555d86e0f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 4 client.5259 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d86c0d990 con 0x555d86e0d120 2024-09-10T21:25:27.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30e9e00640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e0d120 0x555d86e0f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30e9e00640 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e0d120 0x555d86e0f510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34bee62d6ef88b0e server_cookie=72686fa20351e86d in_seq=0 out_seq=0 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 10 client.5259 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 1 -- 172.21.15.97:0/170585119 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d86c0d990 con 0x555d86e0d120 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 10 client.5259 handle_command_reply: tid=0 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 client.5259 shutdown 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfe7a0 msgr2=0x555d86de9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfe7a0 0x555d86de9050 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c760 tx=0x7f30e405f3e0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 shutdown_connections 2024-09-10T21:25:27.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e0d120 0x555d86e0f510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfe7a0 0x555d86de9050 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfde80 0x555d86de8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30cb400640 0 client.5259 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 --2- 172.21.15.97:0/170585119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86db4c90 0x555d86de85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.647+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 >> 172.21.15.97:0/170585119 conn(0x555d86dac800 msgr2=0x555d86db5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.651+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 shutdown_connections 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.651+0000 7f30f46fd000 1 -- 172.21.15.97:0/170585119 wait complete. 2024-09-10T21:25:27.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.651+0000 7f30f46fd000 20 client.5259 trim_cache size 0 max 16384 2024-09-10T21:25:27.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 Processor -- start 2024-09-10T21:25:27.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 -- start start 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86db4c90 0x555d86dfd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfde80 0x555d86dfd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 0x555d86dfbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec06f5c0 con 0x555d86db4c90 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0b7280 con 0x555d86dfe7a0 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec06ef30 con 0x555d86dfde80 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86db4c90 0x555d86dfd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86db4c90 0x555d86dfd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53396/0 (socket says 172.21.15.97:53396) 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 -- 172.21.15.97:0/3861929371 learned_addr learned my addr 172.21.15.97:0/3861929371 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 -- 172.21.15.97:0/3861929371 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfde80 msgr2=0x555d86dfd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 --2- 172.21.15.97:0/3861929371 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfde80 0x555d86dfd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 -- 172.21.15.97:0/3861929371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 msgr2=0x555d86dfbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 --2- 172.21.15.97:0/3861929371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 0x555d86dfbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 -- 172.21.15.97:0/3861929371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d86c7fdc0 con 0x555d86db4c90 2024-09-10T21:25:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30e9e00640 1 --2- 172.21.15.97:0/3861929371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86db4c90 0x555d86dfd0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f30dc016000 tx=0x7f30dc0091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb02c84b1b738f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.667+0000 7f30d6000640 1 -- 172.21.15.97:0/3861929371 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc031070 con 0x555d86db4c90 2024-09-10T21:25:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30d6000640 1 -- 172.21.15.97:0/3861929371 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30dc004350 con 0x555d86db4c90 2024-09-10T21:25:27.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30d6000640 1 -- 172.21.15.97:0/3861929371 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003860 con 0x555d86db4c90 2024-09-10T21:25:27.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- 172.21.15.97:0/3861929371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86db4c90 msgr2=0x555d86dfd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 --2- 172.21.15.97:0/3861929371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86db4c90 0x555d86dfd0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f30dc016000 tx=0x7f30dc0091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- 172.21.15.97:0/3861929371 shutdown_connections 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 --2- 172.21.15.97:0/3861929371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86dfe7a0 0x555d86dfbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 --2- 172.21.15.97:0/3861929371 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfde80 0x555d86dfd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 --2- 172.21.15.97:0/3861929371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86db4c90 0x555d86dfd0a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- 172.21.15.97:0/3861929371 >> 172.21.15.97:0/3861929371 conn(0x555d86dac800 msgr2=0x555d86daebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- 172.21.15.97:0/3861929371 shutdown_connections 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- 172.21.15.97:0/3861929371 wait complete. 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- start start 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 Processor -- start 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86db4c90 0x555d86dfd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfe7a0 0x555d86e10560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555d86bc6f40 con 0x555d86dfde80 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555d86bc6760 con 0x555d86db4c90 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555d86bfdf90 con 0x555d86dfe7a0 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 10 client.0 _ll_register_callbacks cb 0x7f30e405e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53406/0 (socket says 172.21.15.97:53406) 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 -- 172.21.15.97:0/863505417 learned_addr learned my addr 172.21.15.97:0/863505417 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9e00640 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86db4c90 0x555d86dfd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30ea800640 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfe7a0 0x555d86e10560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 -- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfe7a0 msgr2=0x555d86e10560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfe7a0 0x555d86e10560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 -- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86db4c90 msgr2=0x555d86dfd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86db4c90 0x555d86dfd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 -- 172.21.15.97:0/863505417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d86b91620 con 0x555d86dfde80 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30e9400640 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfd6b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f30d801f950 tx=0x7f30d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c214570682bc11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30cb400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30cb400640 1 -- 172.21.15.97:0/863505417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80b5040 con 0x555d86dfde80 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30cb400640 1 -- 172.21.15.97:0/863505417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f30d80046d0 con 0x555d86dfde80 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30dc004350 con 0x555d86dfde80 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30cb400640 1 -- 172.21.15.97:0/863505417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f30d80d5050 con 0x555d86dfde80 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 10 client.5265 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.671+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x555d86bc3af0 con 0x555d86dfde80 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30cb400640 1 -- 172.21.15.97:0/863505417 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30dc004350 con 0x555d86dfde80 2024-09-10T21:25:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30cb400640 1 -- 172.21.15.97:0/863505417 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f30d80d8020 con 0x555d86dfde80 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 10 client.5265 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 10 client.5265 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 20 client.5265 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e10aa0 0x555d86e10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 4 client.5265 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) -- 0x555d86c09440 con 0x555d86e10aa0 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30e9e00640 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e10aa0 0x555d86e10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30e9e00640 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e10aa0 0x555d86e10eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e1cf68ea6a718a server_cookie=3a9327f8bbfeb85c in_seq=0 out_seq=0 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30cb400640 10 client.5265 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30cb400640 1 -- 172.21.15.97:0/863505417 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d86c09440 con 0x555d86e10aa0 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30cb400640 10 client.5265 handle_command_reply: tid=0 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 client.5265 shutdown 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 msgr2=0x555d86dfd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfd6b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f30d801f950 tx=0x7f30d8009260 comp rx=0 tx=0).stop 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 shutdown_connections 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x555d86e10aa0 0x555d86e10eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555d86dfe7a0 0x555d86e10560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d86dfde80 0x555d86dfd6b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 --2- 172.21.15.97:0/863505417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d86db4c90 0x555d86dfd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30cb400640 0 client.5265 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 >> 172.21.15.97:0/863505417 conn(0x555d86dac800 msgr2=0x555d86daf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 shutdown_connections 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 1 -- 172.21.15.97:0/863505417 wait complete. 2024-09-10T21:25:27.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f46fd000 20 client.5265 trim_cache size 0 max 16384 2024-09-10T21:25:27.689 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:25:27.689 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-09-10T21:25:27.689 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "9b46f0a5-0061-401d-ad97-092c37a37be8", 2024-09-10T21:25:27.689 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-09-10T21:25:27.689 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:25:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f30cc0725e0 msgr2=0x7f30cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f30cc0725e0 0x7f30cc074aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30d80b2000 tx=0x7f30d801be50 comp rx=0 tx=0).stop 2024-09-10T21:25:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0d0280 msgr2=0x7f30ec150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0d0280 0x7f30ec150030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f30e400eab0 tx=0x7f30e400ef70 comp rx=0 tx=0).stop 2024-09-10T21:25:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 shutdown_connections 2024-09-10T21:25:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f30cc0725e0 0x7f30cc074aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30ec0d0280 0x7f30ec150030 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30ec0cec10 0x7f30ec0c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 --2- 172.21.15.97:0/299140588 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f30ec0ce830 0x7f30ec1f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 >> 172.21.15.97:0/299140588 conn(0x7f30ec0b37c0 msgr2=0x7f30ec0aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 shutdown_connections 2024-09-10T21:25:27.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.675+0000 7f30f1200640 1 -- 172.21.15.97:0/299140588 wait complete. 2024-09-10T21:25:27.701 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "9b46f0a5-0061-401d-ad97-092c37a37be8", "mode": "asynchronous" } 2024-09-10T21:25:27.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-10T21:25:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.839+0000 7f1c23a00640 1 -- 172.21.15.97:0/2133272506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c04003a60 msgr2=0x7f1c04003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.839+0000 7f1c23a00640 1 --2- 172.21.15.97:0/2133272506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c04003a60 0x7f1c04003e80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1c100016f0 tx=0x7f1c1001c750 comp rx=0 tx=0).stop 2024-09-10T21:25:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.839+0000 7f1c23a00640 1 -- 172.21.15.97:0/2133272506 shutdown_connections 2024-09-10T21:25:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.839+0000 7f1c23a00640 1 --2- 172.21.15.97:0/2133272506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c04003a60 0x7f1c04003e80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.839+0000 7f1c23a00640 1 -- 172.21.15.97:0/2133272506 >> 172.21.15.97:0/2133272506 conn(0x7f1c1c0d1c60 msgr2=0x7f1c1c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.839+0000 7f1c23a00640 1 -- 172.21.15.97:0/2133272506 shutdown_connections 2024-09-10T21:25:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.839+0000 7f1c23a00640 1 -- 172.21.15.97:0/2133272506 wait complete. 2024-09-10T21:25:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 Processor -- start 2024-09-10T21:25:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 -- start start 2024-09-10T21:25:27.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c04003a60 0x7f1c1c0b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1589a0 0x7f1c1c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c1c158e20 0x7f1c1c156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c06dfe0 con 0x7f1c1c1589a0 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c06de60 con 0x7f1c1c158e20 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c06e160 con 0x7f1c04003a60 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1589a0 0x7f1c1c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1589a0 0x7f1c1c156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53424/0 (socket says 172.21.15.97:53424) 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 -- 172.21.15.97:0/3634195217 learned_addr learned my addr 172.21.15.97:0/3634195217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c21000640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c04003a60 0x7f1c1c0b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c21a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c1c158e20 0x7f1c1c156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 -- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c04003a60 msgr2=0x7f1c1c0b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c04003a60 0x7f1c1c0b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 -- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c1c158e20 msgr2=0x7f1c1c156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c1c158e20 0x7f1c1c156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 -- 172.21.15.97:0/3634195217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c15e170 con 0x7f1c1c1589a0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1589a0 0x7f1c1c156a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c08011ae0 tx=0x7f1c08009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c18c00640 1 -- 172.21.15.97:0/3634195217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0800fc50 con 0x7f1c1c1589a0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c18c00640 1 -- 172.21.15.97:0/3634195217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c0800fdf0 con 0x7f1c1c1589a0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c18c00640 1 -- 172.21.15.97:0/3634195217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08004280 con 0x7f1c1c1589a0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c15e3f0 con 0x7f1c1c1589a0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.843+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1f77e0 con 0x7f1c1c1589a0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c18c00640 1 -- 172.21.15.97:0/3634195217 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1c08003600 con 0x7f1c1c1589a0 2024-09-10T21:25:27.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 Processor -- start 2024-09-10T21:25:27.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c18c00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 -- start start 2024-09-10T21:25:27.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c18c00640 1 -- 172.21.15.97:0/3634195217 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f1c08093d40 con 0x7f1c1c1589a0 2024-09-10T21:25:27.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c21a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e9262680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e9262fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b2e914acb0 con 0x55b2e9262bc0 2024-09-10T21:25:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b2e9060310 con 0x55b2e92ac1a0 2024-09-10T21:25:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b2e90bdb90 con 0x55b2e92acac0 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c21a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c069c50 tx=0x7f1c14006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c21a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e9262fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e9262680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c21a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e9262fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53438/0 (socket says 172.21.15.97:53438) 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c21a00640 1 -- 172.21.15.97:0/3575619123 learned_addr learned my addr 172.21.15.97:0/3575619123 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c21000640 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c1be00640 1 -- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 msgr2=0x55b2e92ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c1be00640 1 -- 172.21.15.97:0/3575619123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 msgr2=0x55b2e9262fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e9262fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c1be00640 1 -- 172.21.15.97:0/3575619123 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e9061c60 con 0x55b2e92acac0 2024-09-10T21:25:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.847+0000 7f1c21a00640 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e9262fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e9262680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c08092080 tx=0x7f1c0805e410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c34b53c5ffb1a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c0d600640 1 -- 172.21.15.97:0/3575619123 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0805fe60 con 0x55b2e92acac0 2024-09-10T21:25:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c0d600640 1 -- 172.21.15.97:0/3575619123 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c08068030 con 0x55b2e92acac0 2024-09-10T21:25:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c0d600640 1 -- 172.21.15.97:0/3575619123 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08063200 con 0x55b2e92acac0 2024-09-10T21:25:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 msgr2=0x55b2e9262680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e9262680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c08092080 tx=0x7f1c0805e410 comp rx=0 tx=0).stop 2024-09-10T21:25:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- 172.21.15.97:0/3575619123 shutdown_connections 2024-09-10T21:25:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e9262fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e9262680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3575619123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- 172.21.15.97:0/3575619123 >> 172.21.15.97:0/3575619123 conn(0x55b2e9268d20 msgr2=0x55b2e9260f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- 172.21.15.97:0/3575619123 shutdown_connections 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- 172.21.15.97:0/3575619123 wait complete. 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- start start 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 Processor -- start 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e9262bc0 0x55b2e92b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92ac1a0 0x55b2e92b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e92b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b2e90b0dd0 con 0x55b2e92ac1a0 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b2e90acfc0 con 0x55b2e9262bc0 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b2e90524b0 con 0x55b2e92acac0 2024-09-10T21:25:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c26e09000 10 client.0 _ll_register_callbacks cb 0x55b2e90307d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92ac1a0 0x55b2e92b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92ac1a0 0x55b2e92b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53452/0 (socket says 172.21.15.97:53452) 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 -- 172.21.15.97:0/3036269748 learned_addr learned my addr 172.21.15.97:0/3036269748 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c21a00640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e92b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c21000640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e9262bc0 0x55b2e92b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 -- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 msgr2=0x55b2e92b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e92b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 -- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e9262bc0 msgr2=0x55b2e92b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e9262bc0 0x55b2e92b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 -- 172.21.15.97:0/3036269748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e904b1f0 con 0x55b2e92ac1a0 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c21a00640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e92b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c1be00640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92ac1a0 0x55b2e92b3a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800f850 tx=0x7f1c08063740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f223b80a45845d0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.851+0000 7f1c02000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 1 -- 172.21.15.97:0/3036269748 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0804c020 con 0x55b2e92ac1a0 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2e9045330 con 0x55b2e92ac1a0 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 1 -- 172.21.15.97:0/3036269748 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f1c0805de40 con 0x55b2e92ac1a0 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 1 -- 172.21.15.97:0/3036269748 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1c0805fe60 con 0x55b2e92ac1a0 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 10 client.5277 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b2e9031fa0 con 0x55b2e92ac1a0 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 1 -- 172.21.15.97:0/3036269748 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b2e9045330 con 0x55b2e92ac1a0 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 1 -- 172.21.15.97:0/3036269748 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f1c0805a530 con 0x55b2e92ac1a0 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 10 client.5277 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 10 client.5277 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 20 client.5277 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e92bb900 0x55b2e92bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 4 client.5277 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2e90bfae0 con 0x55b2e92bb900 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c21000640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e92bb900 0x55b2e92bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c21000640 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e92bb900 0x55b2e92bdcf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9d97efb452b92 server_cookie=4591465f73028410 in_seq=0 out_seq=0 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 10 client.5277 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 1 -- 172.21.15.97:0/3036269748 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b2e90bfae0 con 0x55b2e92bb900 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 10 client.5277 handle_command_reply: tid=0 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 client.5277 shutdown 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92ac1a0 msgr2=0x55b2e92b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92ac1a0 0x55b2e92b3a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800f850 tx=0x7f1c08063740 comp rx=0 tx=0).stop 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 shutdown_connections 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e92bb900 0x55b2e92bdcf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92acac0 0x55b2e92b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92ac1a0 0x55b2e92b3a70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c02000640 0 client.5277 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 --2- 172.21.15.97:0/3036269748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e9262bc0 0x55b2e92b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 >> 172.21.15.97:0/3036269748 conn(0x55b2e9268d20 msgr2=0x55b2e9260f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 shutdown_connections 2024-09-10T21:25:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 1 -- 172.21.15.97:0/3036269748 wait complete. 2024-09-10T21:25:27.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.855+0000 7f1c26e09000 20 client.5277 trim_cache size 0 max 16384 2024-09-10T21:25:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 Processor -- start 2024-09-10T21:25:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- start start 2024-09-10T21:25:27.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e9262bc0 0x55b2e9263fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92acac0 0x55b2e92b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c204c60 con 0x55b2e92acac0 2024-09-10T21:25:27.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c203c40 con 0x55b2e92ac1a0 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c202f80 con 0x55b2e9262bc0 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92acac0 0x55b2e92b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92acac0 0x55b2e92b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53460/0 (socket says 172.21.15.97:53460) 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 -- 172.21.15.97:0/351606998 learned_addr learned my addr 172.21.15.97:0/351606998 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c1be00640 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21000640 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e9262bc0 0x55b2e9263fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 -- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e9262bc0 msgr2=0x55b2e9263fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e9262bc0 0x55b2e9263fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 -- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 msgr2=0x55b2e92644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 -- 172.21.15.97:0/351606998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e910a660 con 0x55b2e92acac0 2024-09-10T21:25:27.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c1be00640 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:27.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c21a00640 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92acac0 0x55b2e92b6310 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1c14003cd0 tx=0x7f1c14014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2097dcfd2239888 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c0cc00640 1 -- 172.21.15.97:0/351606998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14003d00 con 0x55b2e92acac0 2024-09-10T21:25:27.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c0cc00640 1 -- 172.21.15.97:0/351606998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c14016070 con 0x55b2e92acac0 2024-09-10T21:25:27.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c0cc00640 1 -- 172.21.15.97:0/351606998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140206a0 con 0x55b2e92acac0 2024-09-10T21:25:27.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- 172.21.15.97:0/351606998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92acac0 msgr2=0x55b2e92b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92acac0 0x55b2e92b6310 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1c14003cd0 tx=0x7f1c14014000 comp rx=0 tx=0).stop 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- 172.21.15.97:0/351606998 shutdown_connections 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e92acac0 0x55b2e92b6310 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92ac1a0 0x55b2e92644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 --2- 172.21.15.97:0/351606998 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e9262bc0 0x55b2e9263fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- 172.21.15.97:0/351606998 >> 172.21.15.97:0/351606998 conn(0x55b2e9268d20 msgr2=0x55b2e9262180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- 172.21.15.97:0/351606998 shutdown_connections 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.875+0000 7f1c26e09000 1 -- 172.21.15.97:0/351606998 wait complete. 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 -- start start 2024-09-10T21:25:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 Processor -- start 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e92b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92ac1a0 0x55b2e92b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92acac0 0x55b2e92de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b2e926bd40 con 0x55b2e9262bc0 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b2e9269100 con 0x55b2e92acac0 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b2e90a7ec0 con 0x55b2e92ac1a0 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 10 client.0 _ll_register_callbacks cb 0x7f1c14001b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e92b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e92b9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53468/0 (socket says 172.21.15.97:53468) 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 -- 172.21.15.97:0/2444059181 learned_addr learned my addr 172.21.15.97:0/2444059181 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21a00640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92acac0 0x55b2e92de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c1be00640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92ac1a0 0x55b2e92b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 -- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92ac1a0 msgr2=0x55b2e92b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92ac1a0 0x55b2e92b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 -- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92acac0 msgr2=0x55b2e92de210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92acac0 0x55b2e92de210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 -- 172.21.15.97:0/2444059181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e9077090 con 0x55b2e9262bc0 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c1be00640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92ac1a0 0x55b2e92b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21a00640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92acac0 0x55b2e92de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c21000640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e92b9560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000fe0 tx=0x7f1c10006a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236de41179798933 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 1 -- 172.21.15.97:0/2444059181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10006270 con 0x55b2e9262bc0 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 1 -- 172.21.15.97:0/2444059181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f1c10006410 con 0x55b2e9262bc0 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c14016070 con 0x55b2e9262bc0 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 1 -- 172.21.15.97:0/2444059181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1c100cd050 con 0x55b2e9262bc0 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 10 client.5283 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b2e90435f0 con 0x55b2e9262bc0 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 1 -- 172.21.15.97:0/2444059181 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c14016070 con 0x55b2e9262bc0 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 1 -- 172.21.15.97:0/2444059181 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f1c10003600 con 0x55b2e9262bc0 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 10 client.5283 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 10 client.5283 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-10T21:25:27.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 20 client.5283 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e9321c70 0x55b2e9322080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 4 client.5283 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2e90bb590 con 0x55b2e9321c70 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c1be00640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e9321c70 0x55b2e9322080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c1be00640 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e9321c70 0x55b2e9322080 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5805a888572878 server_cookie=a7b1730dc72de39a in_seq=0 out_seq=0 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 10 client.5283 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 1 -- 172.21.15.97:0/2444059181 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+573 (crc 0 0 0) 0x55b2e90bb590 con 0x55b2e9321c70 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c02000640 10 client.5283 handle_command_reply: tid=0 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.879+0000 7f1c26e09000 1 client.5283 shutdown 2024-09-10T21:25:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 msgr2=0x55b2e92b9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e92b9560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000fe0 tx=0x7f1c10006a20 comp rx=0 tx=0).stop 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 shutdown_connections 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55b2e9321c70 0x55b2e9322080 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2e92acac0 0x55b2e92de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b2e92ac1a0 0x55b2e92b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 --2- 172.21.15.97:0/2444059181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b2e9262bc0 0x55b2e92b9560 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c02000640 0 client.5283 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 >> 172.21.15.97:0/2444059181 conn(0x55b2e9268d20 msgr2=0x55b2e9262fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 shutdown_connections 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 1 -- 172.21.15.97:0/2444059181 wait complete. 2024-09-10T21:25:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c26e09000 20 client.5283 trim_cache size 0 max 16384 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: "9b46f0a5-0061-401d-ad97-092c37a37be8": { 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/subdir", 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "9b46f0a5-0061-401d-ad97-092c37a37be8", 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "repair" 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:27.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_stats": { 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "9b46f0a5-0061-401d-ad97-092c37a37be8": { 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "start_time": "2024-09-10T21:25:27.675887+0000", 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/subdir", 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_started": 0, 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_passed": 0, 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_failed": 0, 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_skipped": 0 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:27.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:27.898 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:25:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 msgr2=0x7f1bfc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c069c50 tx=0x7f1c14006040 comp rx=0 tx=0).stop 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1589a0 msgr2=0x7f1c1c156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1589a0 0x7f1c1c156a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c08011ae0 tx=0x7f1c08009f20 comp rx=0 tx=0).stop 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 shutdown_connections 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c1c158e20 0x7f1c1c156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1589a0 0x7f1c1c156a90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 --2- 172.21.15.97:0/3634195217 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c04003a60 0x7f1c1c0b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 >> 172.21.15.97:0/3634195217 conn(0x7f1c1c0d1c60 msgr2=0x7f1c1c0714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 shutdown_connections 2024-09-10T21:25:27.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:27.883+0000 7f1c23a00640 1 -- 172.21.15.97:0/3634195217 wait complete. 2024-09-10T21:25:27.907 INFO:tasks.cephfs.filesystem:scrub status for tag:9b46f0a5-0061-401d-ad97-092c37a37be8 - {'path': '/subdir', 'tag': '9b46f0a5-0061-401d-ad97-092c37a37be8', 'options': 'repair'} 2024-09-10T21:25:38.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:38.693 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:57.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:57.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-10T21:25:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 -- 172.21.15.97:0/3943519114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 msgr2=0x7f19c80cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 --2- 172.21.15.97:0/3943519114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c80cf370 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009bc0 tx=0x7f19c001f3b0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 -- 172.21.15.97:0/3943519114 shutdown_connections 2024-09-10T21:25:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 --2- 172.21.15.97:0/3943519114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c80cf370 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 --2- 172.21.15.97:0/3943519114 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19c806ec40 0x7f19c806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 -- 172.21.15.97:0/3943519114 >> 172.21.15.97:0/3943519114 conn(0x7f19c80b1780 msgr2=0x7f19c80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 -- 172.21.15.97:0/3943519114 shutdown_connections 2024-09-10T21:25:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.059+0000 7f19cde00640 1 -- 172.21.15.97:0/3943519114 wait complete. 2024-09-10T21:25:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 Processor -- start 2024-09-10T21:25:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 -- start start 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19c806ec40 0x7f19c815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c81e7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19c81511f0 0x7f19c81f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19c806e740 con 0x7f19c80ceee0 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19c806e5c0 con 0x7f19c806ec40 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f19c806e8c0 con 0x7f19c81511f0 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c81e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c81e7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52834/0 (socket says 172.21.15.97:52834) 2024-09-10T21:25:58.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 -- 172.21.15.97:0/1345520518 learned_addr learned my addr 172.21.15.97:0/1345520518 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19c7400640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19c81511f0 0x7f19c81f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19c806ec40 0x7f19c815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 -- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19c81511f0 msgr2=0x7f19c81f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19c81511f0 0x7f19c81f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 -- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19c806ec40 msgr2=0x7f19c815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19c806ec40 0x7f19c815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 -- 172.21.15.97:0/1345520518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c815bf60 con 0x7f19c80ceee0 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19c806ec40 0x7f19c815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c81e7030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19b00058d0 tx=0x7f19b000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bf400640 1 -- 172.21.15.97:0/1345520518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0017070 con 0x7f19c80ceee0 2024-09-10T21:25:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bf400640 1 -- 172.21.15.97:0/1345520518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b0009d20 con 0x7f19c80ceee0 2024-09-10T21:25:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c815c240 con 0x7f19c80ceee0 2024-09-10T21:25:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19bf400640 1 -- 172.21.15.97:0/1345520518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b00106c0 con 0x7f19c80ceee0 2024-09-10T21:25:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.063+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c81e7570 con 0x7f19c80ceee0 2024-09-10T21:25:58.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19bf400640 1 -- 172.21.15.97:0/1345520518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f19b0010860 con 0x7f19c80ceee0 2024-09-10T21:25:58.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19bf400640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19a40725e0 0x7f19a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19bf400640 1 -- 172.21.15.97:0/1345520518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f19b0093920 con 0x7f19c80ceee0 2024-09-10T21:25:58.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 Processor -- start 2024-09-10T21:25:58.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 -- start start 2024-09-10T21:25:58.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19a40725e0 0x7f19a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19a40725e0 0x7f19a4074aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19b8007a60 tx=0x7f19b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8c1a0 0x563313e8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313e42680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e42bc0 0x563313e42fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563313d2acb0 con 0x563313e8cac0 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563313c40310 con 0x563313e42bc0 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.067+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563313c9db90 con 0x563313e8c1a0 2024-09-10T21:25:58.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313e42680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19c6a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8c1a0 0x563313e8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313e42680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52848/0 (socket says 172.21.15.97:52848) 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 -- 172.21.15.97:0/1990984590 learned_addr learned my addr 172.21.15.97:0/1990984590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19c7400640 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e42bc0 0x563313e42fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 -- 172.21.15.97:0/1990984590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8c1a0 msgr2=0x563313e8c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8c1a0 0x563313e8c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 -- 172.21.15.97:0/1990984590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e42bc0 msgr2=0x563313e42fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e42bc0 0x563313e42fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 -- 172.21.15.97:0/1990984590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563313c41c60 con 0x563313e8cac0 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313e42680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f19b005b960 tx=0x7f19b005e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984b610d98765e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19b7e00640 1 -- 172.21.15.97:0/1990984590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0021040 con 0x563313e8cac0 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19b7e00640 1 -- 172.21.15.97:0/1990984590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b0063120 con 0x563313e8cac0 2024-09-10T21:25:58.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19b7e00640 1 -- 172.21.15.97:0/1990984590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0010de0 con 0x563313e8cac0 2024-09-10T21:25:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- 172.21.15.97:0/1990984590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 msgr2=0x563313e42680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313e42680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f19b005b960 tx=0x7f19b005e100 comp rx=0 tx=0).stop 2024-09-10T21:25:58.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- 172.21.15.97:0/1990984590 shutdown_connections 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e42bc0 0x563313e42fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313e42680 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 --2- 172.21.15.97:0/1990984590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8c1a0 0x563313e8c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- 172.21.15.97:0/1990984590 >> 172.21.15.97:0/1990984590 conn(0x563313e48d20 msgr2=0x563313e40f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- 172.21.15.97:0/1990984590 shutdown_connections 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- 172.21.15.97:0/1990984590 wait complete. 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- start start 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 Processor -- start 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313e93530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8c1a0 0x563313e93a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8cac0 0x563313e93fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563313c90dd0 con 0x563313e8c1a0 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563313c8cfc0 con 0x563313e8cac0 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563313c324b0 con 0x563313e42bc0 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19d1244000 10 client.0 _ll_register_callbacks cb 0x563313c107d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8c1a0 0x563313e93a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8c1a0 0x563313e93a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52864/0 (socket says 172.21.15.97:52864) 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19bfe00640 1 -- 172.21.15.97:0/217079432 learned_addr learned my addr 172.21.15.97:0/217079432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19c7400640 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8cac0 0x563313e93fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.071+0000 7f19c6a00640 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313e93530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19bfe00640 1 -- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 msgr2=0x563313e93530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19bfe00640 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313e93530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19bfe00640 1 -- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8cac0 msgr2=0x563313e93fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19bfe00640 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8cac0 0x563313e93fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19bfe00640 1 -- 172.21.15.97:0/217079432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563313c2b1f0 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19bfe00640 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8c1a0 0x563313e93a70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f19b005b960 tx=0x7f19b005ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cc0fd01b16c6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 1 -- 172.21.15.97:0/217079432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0021040 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 1 -- 172.21.15.97:0/217079432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f19b005d950 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563313c25330 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 1 -- 172.21.15.97:0/217079432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f19b0091050 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 10 client.5298 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x563313c11fa0 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 1 -- 172.21.15.97:0/217079432 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563313c25330 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 1 -- 172.21.15.97:0/217079432 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f19b0065330 con 0x563313e8c1a0 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 10 client.5298 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:58.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 10 client.5298 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-10T21:25:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 20 client.5298 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313e9b900 0x563313e9dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 4 client.5298 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563313c9fae0 con 0x563313e9b900 2024-09-10T21:25:58.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19c6a00640 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313e9b900 0x563313e9dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19c6a00640 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313e9b900 0x563313e9dcf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214fb9e86d2e1a87 server_cookie=eef28441abcb86ab in_seq=0 out_seq=0 2024-09-10T21:25:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 10 client.5298 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 1 -- 172.21.15.97:0/217079432 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563313c9fae0 con 0x563313e9b900 2024-09-10T21:25:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 10 client.5298 handle_command_reply: tid=0 2024-09-10T21:25:58.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 client.5298 shutdown 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8c1a0 msgr2=0x563313e93a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8c1a0 0x563313e93a70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f19b005b960 tx=0x7f19b005ac10 comp rx=0 tx=0).stop 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 shutdown_connections 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313e9b900 0x563313e9dcf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8cac0 0x563313e93fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8c1a0 0x563313e93a70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 --2- 172.21.15.97:0/217079432 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313e93530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19b4c00640 0 client.5298 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 >> 172.21.15.97:0/217079432 conn(0x563313e48d20 msgr2=0x563313e40f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 shutdown_connections 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 1 -- 172.21.15.97:0/217079432 wait complete. 2024-09-10T21:25:58.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.075+0000 7f19d1244000 20 client.5298 trim_cache size 0 max 16384 2024-09-10T21:25:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 Processor -- start 2024-09-10T21:25:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- start start 2024-09-10T21:25:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e42bc0 0x563313efdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313efe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8cac0 0x563313efe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563313e42a40 con 0x563313e42bc0 2024-09-10T21:25:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8201780 con 0x563313e8c1a0 2024-09-10T21:25:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f19c8200800 con 0x563313e8cac0 2024-09-10T21:25:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e42bc0 0x563313efdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e42bc0 0x563313efdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52870/0 (socket says 172.21.15.97:52870) 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 -- 172.21.15.97:0/1335319630 learned_addr learned my addr 172.21.15.97:0/1335319630 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313efe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c7400640 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8cac0 0x563313efe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 -- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8cac0 msgr2=0x563313efe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8cac0 0x563313efe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 -- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 msgr2=0x563313efe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313efe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 -- 172.21.15.97:0/1335319630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563313cea660 con 0x563313e42bc0 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e42bc0 0x563313efdc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f19b80015c0 tx=0x7f19b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4add24cccdf49d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19b7400640 1 -- 172.21.15.97:0/1335319630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80154f0 con 0x563313e42bc0 2024-09-10T21:25:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19b7400640 1 -- 172.21.15.97:0/1335319630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b8015690 con 0x563313e42bc0 2024-09-10T21:25:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19b7400640 1 -- 172.21.15.97:0/1335319630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80158a0 con 0x563313e42bc0 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- 172.21.15.97:0/1335319630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e42bc0 msgr2=0x563313efdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e42bc0 0x563313efdc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f19b80015c0 tx=0x7f19b8014000 comp rx=0 tx=0).stop 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- 172.21.15.97:0/1335319630 shutdown_connections 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e8cac0 0x563313efe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313efe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 --2- 172.21.15.97:0/1335319630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e42bc0 0x563313efdc30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- 172.21.15.97:0/1335319630 >> 172.21.15.97:0/1335319630 conn(0x563313e48d20 msgr2=0x563313e442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- 172.21.15.97:0/1335319630 shutdown_connections 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.095+0000 7f19d1244000 1 -- 172.21.15.97:0/1335319630 wait complete. 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 -- start start 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 Processor -- start 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313ea2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313ea3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313ebe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19c82001e0 con 0x563313e8cac0 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8152630 con 0x563313e8c1a0 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563313952680 con 0x563313e42bc0 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 10 client.0 _ll_register_callbacks cb 0x7f19b0083070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313ebe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313ebe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52872/0 (socket says 172.21.15.97:52872) 2024-09-10T21:25:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 -- 172.21.15.97:0/1606944660 learned_addr learned my addr 172.21.15.97:0/1606944660 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313ea2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19bfe00640 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313ea3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 -- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 msgr2=0x563313ea2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313ea2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 -- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 msgr2=0x563313ea3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313ea3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 -- 172.21.15.97:0/1606944660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563313c57090 con 0x563313e8cac0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c7400640 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313ebe210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f19c001cfe0 tx=0x7f19c0004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d258b13c306e656 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19b4c00640 1 -- 172.21.15.97:0/1606944660 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c00b4040 con 0x563313e8cac0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19b4c00640 1 -- 172.21.15.97:0/1606944660 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f19c0003b80 con 0x563313e8cac0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b8015690 con 0x563313e8cac0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19b4c00640 1 -- 172.21.15.97:0/1606944660 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f19c00cc050 con 0x563313e8cac0 2024-09-10T21:25:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 10 client.5304 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x563313c235f0 con 0x563313e8cac0 2024-09-10T21:25:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19b4c00640 1 -- 172.21.15.97:0/1606944660 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b8015690 con 0x563313e8cac0 2024-09-10T21:25:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19b4c00640 1 -- 172.21.15.97:0/1606944660 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f19c00cf020 con 0x563313e8cac0 2024-09-10T21:25:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 10 client.5304 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 10 client.5304 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 20 client.5304 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313f01640 0x563313f01a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 4 client.5304 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563313c9b590 con 0x563313f01640 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.099+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313f01640 0x563313f01a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19c6a00640 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313f01640 0x563313f01a50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdf44f816d89fdf server_cookie=1fbf98c8b8916752 in_seq=0 out_seq=0 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19b4c00640 10 client.5304 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19b4c00640 1 -- 172.21.15.97:0/1606944660 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+380 (crc 0 0 0) 0x563313c9b590 con 0x563313f01640 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19b4c00640 10 client.5304 handle_command_reply: tid=0 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 client.5304 shutdown 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 msgr2=0x563313ebe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313ebe210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f19c001cfe0 tx=0x7f19c0004dd0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 shutdown_connections 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x563313f01640 0x563313f01a50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563313e8cac0 0x563313ebe210 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563313e8c1a0 0x563313ea3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19b4c00640 0 client.5304 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 --2- 172.21.15.97:0/1606944660 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563313e42bc0 0x563313ea2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 >> 172.21.15.97:0/1606944660 conn(0x563313e48d20 msgr2=0x563313e9dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 shutdown_connections 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 1 -- 172.21.15.97:0/1606944660 wait complete. 2024-09-10T21:25:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19d1244000 20 client.5304 trim_cache size 0 max 16384 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {}, 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_stats": { 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "9b46f0a5-0061-401d-ad97-092c37a37be8": { 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "start_time": "2024-09-10T21:25:27.675887+0000", 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/subdir", 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_started": 0, 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_passed": 0, 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_failed": 0, 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_skipped": 0 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:58.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:25:58.111 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:25:58.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19a40725e0 msgr2=0x7f19a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19a40725e0 0x7f19a4074aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19b8007a60 tx=0x7f19b8009040 comp rx=0 tx=0).stop 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 msgr2=0x7f19c81e7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c81e7030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f19b00058d0 tx=0x7f19b000dde0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 shutdown_connections 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19a40725e0 0x7f19a4074aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19c81511f0 0x7f19c81f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c80ceee0 0x7f19c81e7030 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 --2- 172.21.15.97:0/1345520518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19c806ec40 0x7f19c815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 >> 172.21.15.97:0/1345520518 conn(0x7f19c80b1780 msgr2=0x7f19c8150310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 shutdown_connections 2024-09-10T21:25:58.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.103+0000 7f19cde00640 1 -- 172.21.15.97:0/1345520518 wait complete. 2024-09-10T21:25:58.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:25:58.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:25:58.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 -- 172.21.15.97:0/881925673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480ce640 msgr2=0x7f5a480cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/881925673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480ce640 0x7f5a480cea20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009840 tx=0x7f5a3801c820 comp rx=0 tx=0).stop 2024-09-10T21:25:58.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 -- 172.21.15.97:0/881925673 shutdown_connections 2024-09-10T21:25:58.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/881925673 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a480b0d50 0x7f5a480aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/881925673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480ce640 0x7f5a480cea20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 -- 172.21.15.97:0/881925673 >> 172.21.15.97:0/881925673 conn(0x7f5a480d1c30 msgr2=0x7f5a480d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 -- 172.21.15.97:0/881925673 shutdown_connections 2024-09-10T21:25:58.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.263+0000 7f5a4ec00640 1 -- 172.21.15.97:0/881925673 wait complete. 2024-09-10T21:25:58.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 Processor -- start 2024-09-10T21:25:58.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 -- start start 2024-09-10T21:25:58.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480af300 0x7f5a481fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a480b0d50 0x7f5a4815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a480ce640 0x7f5a4815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a48169270 con 0x7f5a480af300 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481690f0 con 0x7f5a480ce640 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5a481693f0 con 0x7f5a480b0d50 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480af300 0x7f5a481fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480af300 0x7f5a481fb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52884/0 (socket says 172.21.15.97:52884) 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 -- 172.21.15.97:0/1547824630 learned_addr learned my addr 172.21.15.97:0/1547824630 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47400640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a480b0d50 0x7f5a4815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4cc00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a480ce640 0x7f5a4815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 -- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a480b0d50 msgr2=0x7f5a4815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a480b0d50 0x7f5a4815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 -- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a480ce640 msgr2=0x7f5a4815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a480ce640 0x7f5a4815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 -- 172.21.15.97:0/1547824630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4806dc50 con 0x7f5a480af300 2024-09-10T21:25:58.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a47e00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480af300 0x7f5a481fb0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009790 tx=0x7f5a380a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a3fe00640 1 -- 172.21.15.97:0/1547824630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a380bb070 con 0x7f5a480af300 2024-09-10T21:25:58.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a3fe00640 1 -- 172.21.15.97:0/1547824630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a38005030 con 0x7f5a480af300 2024-09-10T21:25:58.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a480b4b00 con 0x7f5a480af300 2024-09-10T21:25:58.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a3fe00640 1 -- 172.21.15.97:0/1547824630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a380b2040 con 0x7f5a480af300 2024-09-10T21:25:58.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a481fb960 con 0x7f5a480af300 2024-09-10T21:25:58.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.267+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a480aedc0 con 0x7f5a480af300 2024-09-10T21:25:58.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.271+0000 7f5a3fe00640 1 -- 172.21.15.97:0/1547824630 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5a380b5030 con 0x7f5a480af300 2024-09-10T21:25:58.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.271+0000 7f5a3fe00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5a280725e0 0x7f5a28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:58.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.271+0000 7f5a3fe00640 1 -- 172.21.15.97:0/1547824630 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f5a3812cb80 con 0x7f5a480af300 2024-09-10T21:25:58.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.271+0000 7f5a47400640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5a280725e0 0x7f5a28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:58.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.271+0000 7f5a47400640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5a280725e0 0x7f5a28074aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a340058d0 tx=0x7f5a34005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:58.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.275+0000 7f5a3fe00640 1 -- 172.21.15.97:0/1547824630 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a38016410 con 0x7f5a480af300 2024-09-10T21:25:58.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a480ceb50 con 0x7f5a480af300 2024-09-10T21:25:58.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a3fe00640 1 -- 172.21.15.97:0/1547824630 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f5a380f7fa0 con 0x7f5a480af300 2024-09-10T21:25:58.482 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5a280725e0 msgr2=0x7f5a28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5a280725e0 0x7f5a28074aa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a340058d0 tx=0x7f5a34005860 comp rx=0 tx=0).stop 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480af300 msgr2=0x7f5a481fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480af300 0x7f5a481fb0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009790 tx=0x7f5a380a5700 comp rx=0 tx=0).stop 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 shutdown_connections 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5a280725e0 0x7f5a28074aa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a480ce640 0x7f5a4815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5a480b0d50 0x7f5a4815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 --2- 172.21.15.97:0/1547824630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a480af300 0x7f5a481fb0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:58.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 >> 172.21.15.97:0/1547824630 conn(0x7f5a480d1c30 msgr2=0x7f5a480af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:58.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 shutdown_connections 2024-09-10T21:25:58.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58.479+0000 7f5a4ec00640 1 -- 172.21.15.97:0/1547824630 wait complete. 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:25:58.503 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:58.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:58.504 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:25:58.513 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42335sec preferred_lft 42335sec 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:58.514 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:25:58.515 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:58.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:58.516 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:25:58.581 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:25:58.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:58.584 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:25:58.643 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:25:58.643 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:25:58.643 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:25:58.643 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:25:58.643 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:25:58.644 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:25:58.644 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:25:58.644 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:58.644 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:58.644 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:25:58.644 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:25:58.644 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:58.644 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:25:58.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:25:58.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:25:58.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:25:58.645 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:25:58.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:58.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:58.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:58.776 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:25:58.786 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:25:58.786 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:25:58.786 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:58.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:58.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:25:58.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:58.847 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:58.900 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:58.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:58.900 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:58.955 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:25:58.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:58.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:59.043 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:59.044 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:59.044 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:59.059 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:59.059 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:59.059 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:59.059 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:25:59.059 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:25:59.071 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:25:59.074 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:25:59.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:59.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:59.094 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:25:59.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:25:59.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:59.190 INFO:teuthology.orchestra.run.smithi097.stdout:client4994 2024-09-10T21:25:59.191 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:25:59.191 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:59.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:59.200 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-10T21:25:59.249 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:25:59.249 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:25:59.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 Processor -- start 2024-09-10T21:25:59.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 -- start start 2024-09-10T21:25:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c67e940 0x56251c723970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 0x56251c723eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c7092a0 0x56251c7243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56251c59da00 con 0x56251c67e940 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56251c5fbf40 con 0x56251c7092a0 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feee204d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56251c70b080 con 0x56251c708b00 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feede800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c67e940 0x56251c723970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feede800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c67e940 0x56251c723970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52906/0 (socket says 172.21.15.97:52906) 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.335+0000 7feede800640 1 -- 172.21.15.97:0/1076532770 learned_addr learned my addr 172.21.15.97:0/1076532770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feed7e00640 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 0x56251c723eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c7092a0 0x56251c7243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 -- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 msgr2=0x56251c723eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 0x56251c723eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 -- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c7092a0 msgr2=0x56251c7243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c7092a0 0x56251c7243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 -- 172.21.15.97:0/1076532770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251c576c10 con 0x56251c67e940 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c67e940 0x56251c723970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2a0 tx=0x7feed000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc80dcbc0e37649a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedd400640 1 -- 172.21.15.97:0/1076532770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed000a040 con 0x56251c67e940 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedd400640 1 -- 172.21.15.97:0/1076532770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed000f620 con 0x56251c67e940 2024-09-10T21:25:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedd400640 1 -- 172.21.15.97:0/1076532770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed000fa70 con 0x56251c67e940 2024-09-10T21:25:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- 172.21.15.97:0/1076532770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c67e940 msgr2=0x56251c723970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c67e940 0x56251c723970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2a0 tx=0x7feed000b760 comp rx=0 tx=0).stop 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- 172.21.15.97:0/1076532770 shutdown_connections 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c7092a0 0x56251c7243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 0x56251c723eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 --2- 172.21.15.97:0/1076532770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c67e940 0x56251c723970 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- 172.21.15.97:0/1076532770 >> 172.21.15.97:0/1076532770 conn(0x56251c67d520 msgr2=0x56251c725300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- 172.21.15.97:0/1076532770 shutdown_connections 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- 172.21.15.97:0/1076532770 wait complete. 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 Processor -- start 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- start start 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c67e940 0x56251c720510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 0x56251c708050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c7092a0 0x56251c7219e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56251c724d30 con 0x56251c7092a0 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56251c728a10 con 0x56251c67e940 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feee204d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56251c680630 con 0x56251c708b00 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c7092a0 0x56251c7219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c7092a0 0x56251c7219e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52922/0 (socket says 172.21.15.97:52922) 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 -- 172.21.15.97:0/1960391516 learned_addr learned my addr 172.21.15.97:0/1960391516 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feed7e00640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 0x56251c708050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c67e940 0x56251c720510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 -- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 msgr2=0x56251c708050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56251c708b00 0x56251c708050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 -- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c67e940 msgr2=0x56251c720510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c67e940 0x56251c720510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 -- 172.21.15.97:0/1960391516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251c71f5f0 con 0x56251c7092a0 2024-09-10T21:25:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feede800640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56251c67e940 0x56251c720510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.339+0000 7feedf200640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c7092a0 0x56251c7219e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feed80ad040 tx=0x7feed80aaf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feed7400640 1 -- 172.21.15.97:0/1960391516 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80ba030 con 0x56251c7092a0 2024-09-10T21:25:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feed7400640 1 -- 172.21.15.97:0/1960391516 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed80b0040 con 0x56251c7092a0 2024-09-10T21:25:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feed7400640 1 -- 172.21.15.97:0/1960391516 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80af040 con 0x56251c7092a0 2024-09-10T21:25:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56251c71f890 con 0x56251c7092a0 2024-09-10T21:25:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251c71fdd0 con 0x56251c7092a0 2024-09-10T21:25:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feed7400640 1 -- 172.21.15.97:0/1960391516 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feed80b5040 con 0x56251c7092a0 2024-09-10T21:25:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feed7400640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feebc070ff0 0x7feebc0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feed7400640 1 -- 172.21.15.97:0/1960391516 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7feed812cc10 con 0x56251c7092a0 2024-09-10T21:25:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feee204d080 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56251c721f20 0x56251c7bf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feede800640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feebc070ff0 0x7feebc0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) -- 0x56251c7bfcb0 con 0x56251c721f20 2024-09-10T21:25:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.343+0000 7feed7e00640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56251c721f20 0x56251c7bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feed7e00640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56251c721f20 0x56251c7bf770 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feede800640 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feebc070ff0 0x7feebc0734b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feed000c010 tx=0x7feed0000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feed7e00640 1 -- 172.21.15.97:0/1960391516 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v43'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feec800c750 con 0x56251c721f20 2024-09-10T21:25:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56251c721f20 msgr2=0x56251c7bf770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56251c721f20 0x56251c7bf770 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feebc070ff0 msgr2=0x7feebc0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feebc070ff0 0x7feebc0734b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feed000c010 tx=0x7feed0000ee0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c7092a0 msgr2=0x56251c7219e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 --2- 172.21.15.97:0/1960391516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56251c7092a0 0x56251c7219e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feed80ad040 tx=0x7feed80aaf30 comp rx=0 tx=0).stop 2024-09-10T21:25:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feedf200640 1 -- 172.21.15.97:0/1960391516 reap_dead start 2024-09-10T21:25:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 shutdown_connections 2024-09-10T21:25:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 >> 172.21.15.97:0/1960391516 conn(0x56251c67d520 msgr2=0x56251c69bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 shutdown_connections 2024-09-10T21:25:59.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.347+0000 7feee204d080 1 -- 172.21.15.97:0/1960391516 wait complete. 2024-09-10T21:25:59.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 -- 172.21.15.97:0/2870971779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 msgr2=0x7fddf006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2870971779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 0x7fddf006e3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdde000aa10 tx=0x7fdde001dc00 comp rx=0 tx=0).stop 2024-09-10T21:25:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 -- 172.21.15.97:0/2870971779 shutdown_connections 2024-09-10T21:25:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2870971779 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddf00cf490 0x7fddf0072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2870971779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 0x7fddf006e3b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 -- 172.21.15.97:0/2870971779 >> 172.21.15.97:0/2870971779 conn(0x7fddf00b1780 msgr2=0x7fddf00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 -- 172.21.15.97:0/2870971779 shutdown_connections 2024-09-10T21:25:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 -- 172.21.15.97:0/2870971779 wait complete. 2024-09-10T21:25:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 Processor -- start 2024-09-10T21:25:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.535+0000 7fddf4e00640 1 -- start start 2024-09-10T21:25:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 0x7fddf01f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddf00cf490 0x7fddf015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf015b110 0x7fddf01fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0166f30 con 0x7fddf015b110 2024-09-10T21:25:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0166db0 con 0x7fddf00cf0b0 2024-09-10T21:25:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fddf01670b0 con 0x7fddf00cf490 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf015b110 0x7fddf01fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf015b110 0x7fddf01fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52932/0 (socket says 172.21.15.97:52932) 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 -- 172.21.15.97:0/2183730739 learned_addr learned my addr 172.21.15.97:0/2183730739 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fdded400640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddf00cf490 0x7fddf015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddede00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 0x7fddf01f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 -- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddf00cf490 msgr2=0x7fddf015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddf00cf490 0x7fddf015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 -- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 msgr2=0x7fddf01f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 0x7fddf01f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 -- 172.21.15.97:0/2183730739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf01f9340 con 0x7fddf015b110 2024-09-10T21:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddee800640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf015b110 0x7fddf01fb9e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdde800c710 tx=0x7fdde800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fdde6000640 1 -- 172.21.15.97:0/2183730739 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8003b80 con 0x7fddf015b110 2024-09-10T21:25:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fdde6000640 1 -- 172.21.15.97:0/2183730739 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde8003d20 con 0x7fddf015b110 2024-09-10T21:25:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fdde6000640 1 -- 172.21.15.97:0/2183730739 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde800aab0 con 0x7fddf015b110 2024-09-10T21:25:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf01fc350 con 0x7fddf015b110 2024-09-10T21:25:59.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf01f8600 con 0x7fddf015b110 2024-09-10T21:25:59.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.539+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf01f83f0 con 0x7fddf015b110 2024-09-10T21:25:59.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.543+0000 7fdde6000640 1 -- 172.21.15.97:0/2183730739 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdde800ac50 con 0x7fddf015b110 2024-09-10T21:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.543+0000 7fdde6000640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddd00725e0 0x7fddd0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.543+0000 7fdde6000640 1 -- 172.21.15.97:0/2183730739 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7fdde8093180 con 0x7fddf015b110 2024-09-10T21:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.543+0000 7fddede00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddd00725e0 0x7fddd0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.543+0000 7fddede00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddd00725e0 0x7fddd0074aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdde00b0000 tx=0x7fdde001c440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.547+0000 7fdde6000640 1 -- 172.21.15.97:0/2183730739 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fddf01f9340 con 0x7fddf015b110 2024-09-10T21:25:59.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.743+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddf006e3b0 con 0x7fddf015b110 2024-09-10T21:25:59.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fdde6000640 1 -- 172.21.15.97:0/2183730739 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5221 (secure 0 0 0) 0x7fdde800ce20 con 0x7fddf015b110 2024-09-10T21:25:59.749 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:25:59.749 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"btime":"2024-09-10T21:25:23:161523+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:23.161515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5148,"qdb_cluster":[5148]},"id":2}]} 2024-09-10T21:25:59.749 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddd00725e0 msgr2=0x7fddd0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddd00725e0 0x7fddd0074aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdde00b0000 tx=0x7fdde001c440 comp rx=0 tx=0).stop 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf015b110 msgr2=0x7fddf01fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf015b110 0x7fddf01fb9e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdde800c710 tx=0x7fdde800cbd0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 shutdown_connections 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddd00725e0 0x7fddd0074aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf015b110 0x7fddf01fb9e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddf00cf490 0x7fddf015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 --2- 172.21.15.97:0/2183730739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddf00cf0b0 0x7fddf01f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 >> 172.21.15.97:0/2183730739 conn(0x7fddf00b1780 msgr2=0x7fddf015b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 shutdown_connections 2024-09-10T21:25:59.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.747+0000 7fddf4e00640 1 -- 172.21.15.97:0/2183730739 wait complete. 2024-09-10T21:25:59.769 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 2054577795 2024-09-10T21:25:59.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- 172.21.15.97:0/2803539609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c15efc0 msgr2=0x7f2e2c0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 --2- 172.21.15.97:0/2803539609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c15efc0 0x7f2e2c0cf840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c06daf0 tx=0x7f2e1c01c800 comp rx=0 tx=0).stop 2024-09-10T21:25:59.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- 172.21.15.97:0/2803539609 shutdown_connections 2024-09-10T21:25:59.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 --2- 172.21.15.97:0/2803539609 >> v2:172.21.15.194:3300/0 conn(0x7f2e2c0d0490 0x7f2e2c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 --2- 172.21.15.97:0/2803539609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c15efc0 0x7f2e2c0cf840 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- 172.21.15.97:0/2803539609 >> 172.21.15.97:0/2803539609 conn(0x7f2e2c0d1a60 msgr2=0x7f2e2c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- 172.21.15.97:0/2803539609 shutdown_connections 2024-09-10T21:25:59.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- 172.21.15.97:0/2803539609 wait complete. 2024-09-10T21:25:59.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 Processor -- start 2024-09-10T21:25:59.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- start start 2024-09-10T21:25:59.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e2c0cdcb0 0x7f2e2c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d0490 0x7f2e2c157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e2c15efc0 0x7f2e2c0b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c1687b0 con 0x7f2e2c0d0490 2024-09-10T21:25:59.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c0afc50 con 0x7f2e2c0cdcb0 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e30a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2e2c168930 con 0x7f2e2c15efc0 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e22a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d0490 0x7f2e2c157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e22a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d0490 0x7f2e2c157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52954/0 (socket says 172.21.15.97:52954) 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e22a00640 1 -- 172.21.15.97:0/4223043704 learned_addr learned my addr 172.21.15.97:0/4223043704 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e2b400640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e2c0cdcb0 0x7f2e2c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.911+0000 7f2e2be00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e2c15efc0 0x7f2e2c0b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e22a00640 1 -- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e2c15efc0 msgr2=0x7f2e2c0b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e22a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e2c15efc0 0x7f2e2c0b6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e22a00640 1 -- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e2c0cdcb0 msgr2=0x7f2e2c150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e22a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e2c0cdcb0 0x7f2e2c150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e22a00640 1 -- 172.21.15.97:0/4223043704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2c158050 con 0x7f2e2c0d0490 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e22a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d0490 0x7f2e2c157700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400c490 tx=0x7f2e1400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e23e00640 1 -- 172.21.15.97:0/4223043704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14016020 con 0x7f2e2c0d0490 2024-09-10T21:25:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e23e00640 1 -- 172.21.15.97:0/4223043704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e14012940 con 0x7f2e2c0d0490 2024-09-10T21:25:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2c1f8770 con 0x7f2e2c0d0490 2024-09-10T21:25:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e23e00640 1 -- 172.21.15.97:0/4223043704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14012d90 con 0x7f2e2c0d0490 2024-09-10T21:25:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2c1f8d30 con 0x7f2e2c0d0490 2024-09-10T21:25:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.915+0000 7f2e23e00640 1 -- 172.21.15.97:0/4223043704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2e1401b400 con 0x7f2e2c0d0490 2024-09-10T21:25:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e23e00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e080725e0 0x7f2e08074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 Processor -- start 2024-09-10T21:25:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e23e00640 1 -- 172.21.15.97:0/4223043704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f2e140936b0 con 0x7f2e2c0d0490 2024-09-10T21:25:59.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- start start 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e080725e0 0x7f2e08074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd78261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd78266f0 0x55abd77d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd77d96f0 0x55abd77d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55abd76bea80 con 0x55abd7825dd0 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abd7633c90 con 0x55abd77d96f0 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55abd76230c0 con 0x55abd78266f0 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd78261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd78261b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52968/0 (socket says 172.21.15.97:52968) 2024-09-10T21:25:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 -- 172.21.15.97:0/1353169571 learned_addr learned my addr 172.21.15.97:0/1353169571 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2be00640 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd77d96f0 0x55abd77d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e22a00640 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd78266f0 0x55abd77d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e080725e0 0x7f2e08074aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0af000 tx=0x7f2e1c01b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 -- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd78266f0 msgr2=0x55abd77d91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd78266f0 0x55abd77d91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 -- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd77d96f0 msgr2=0x55abd77d9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd77d96f0 0x55abd77d9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 -- 172.21.15.97:0/1353169571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd75c7350 con 0x55abd7825dd0 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e2b400640 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd78261b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c01cce0 tx=0x7f2e1c0b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c437449b002e0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e1be00640 1 -- 172.21.15.97:0/1353169571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0b2040 con 0x55abd7825dd0 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e1be00640 1 -- 172.21.15.97:0/1353169571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e1c0c9070 con 0x55abd7825dd0 2024-09-10T21:25:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e1be00640 1 -- 172.21.15.97:0/1353169571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0046d0 con 0x55abd7825dd0 2024-09-10T21:25:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- 172.21.15.97:0/1353169571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 msgr2=0x55abd78261b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd78261b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c01cce0 tx=0x7f2e1c0b7000 comp rx=0 tx=0).stop 2024-09-10T21:25:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- 172.21.15.97:0/1353169571 shutdown_connections 2024-09-10T21:25:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd77d96f0 0x55abd77d9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd78266f0 0x55abd77d91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 --2- 172.21.15.97:0/1353169571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd78261b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- 172.21.15.97:0/1353169571 >> 172.21.15.97:0/1353169571 conn(0x55abd7806be0 msgr2=0x55abd7809000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- 172.21.15.97:0/1353169571 shutdown_connections 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.919+0000 7f2e354ae000 1 -- 172.21.15.97:0/1353169571 wait complete. 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 -- start start 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 Processor -- start 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd780a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd780aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd780b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55abd76a7600 con 0x55abd7825dd0 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abd76e21e0 con 0x55abd78266f0 2024-09-10T21:25:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55abd75ccb10 con 0x55abd77d96f0 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 10 client.0 _ll_register_callbacks cb 0x55abd77c2c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd780aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd780aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52982/0 (socket says 172.21.15.97:52982) 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e2b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd780a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 -- 172.21.15.97:0/2348855821 learned_addr learned my addr 172.21.15.97:0/2348855821 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e2be00640 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd780b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 -- 172.21.15.97:0/2348855821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 msgr2=0x55abd780a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd780a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 -- 172.21.15.97:0/2348855821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 msgr2=0x55abd780b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd780b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 -- 172.21.15.97:0/2348855821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd75c1260 con 0x55abd7825dd0 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e22a00640 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd780aeb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e140918c0 tx=0x7f2e1405e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fcc219e064d399 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e18c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e18c00640 1 -- 172.21.15.97:0/2348855821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14062f70 con 0x55abd7825dd0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e18c00640 1 -- 172.21.15.97:0/2348855821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f2e14065180 con 0x55abd7825dd0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55abd75bb1e0 con 0x55abd7825dd0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e18c00640 1 -- 172.21.15.97:0/2348855821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2e14067990 con 0x55abd7825dd0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 10 client.5340 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55abd75ecac0 con 0x55abd7825dd0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e18c00640 1 -- 172.21.15.97:0/2348855821 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55abd75bb1e0 con 0x55abd7825dd0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e18c00640 1 -- 172.21.15.97:0/2348855821 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f2e14083180 con 0x55abd7825dd0 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 10 client.5340 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 10 client.5340 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 20 client.5340 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd780be00 0x55abd7836e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 4 client.5340 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd7635be0 con 0x55abd780be00 2024-09-10T21:25:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd780be00 0x55abd7836e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e18c00640 10 client.5340 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.923+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd780be00 0x55abd7836e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a2796a031a1aa0 server_cookie=468c761545083008 in_seq=0 out_seq=0 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e18c00640 1 -- 172.21.15.97:0/2348855821 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55abd7635be0 con 0x55abd780be00 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e18c00640 10 client.5340 handle_command_reply: tid=0 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 client.5340 shutdown 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 msgr2=0x55abd780aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd780aeb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e140918c0 tx=0x7f2e1405e050 comp rx=0 tx=0).stop 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 shutdown_connections 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd780be00 0x55abd7836e70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd780b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd7825dd0 0x55abd780aeb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2348855821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd780a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e18c00640 0 client.5340 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 >> 172.21.15.97:0/2348855821 conn(0x55abd7806be0 msgr2=0x55abd7808fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 shutdown_connections 2024-09-10T21:25:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 1 -- 172.21.15.97:0/2348855821 wait complete. 2024-09-10T21:25:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.927+0000 7f2e354ae000 20 client.5340 trim_cache size 0 max 16384 2024-09-10T21:25:59.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 Processor -- start 2024-09-10T21:25:59.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 -- start start 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd77d96f0 0x55abd7897dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd7825dd0 0x55abd7898300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd7898840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c06f5a0 con 0x55abd77d96f0 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c151910 con 0x55abd78266f0 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2e2c06ef50 con 0x55abd7825dd0 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.939+0000 7f2e2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd77d96f0 0x55abd7897dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd77d96f0 0x55abd7897dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52986/0 (socket says 172.21.15.97:52986) 2024-09-10T21:25:59.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 -- 172.21.15.97:0/2324231139 learned_addr learned my addr 172.21.15.97:0/2324231139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e22a00640 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd7825dd0 0x55abd7898300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd7898840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 -- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd7825dd0 msgr2=0x55abd7898300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd7825dd0 0x55abd7898300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 -- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 msgr2=0x55abd7898840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd7898840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 -- 172.21.15.97:0/2324231139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd76a7a90 con 0x55abd77d96f0 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e22a00640 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd7825dd0 0x55abd7898300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd77d96f0 0x55abd7897dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c019410 tx=0x7f2e1c0a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6ccfd231cddba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e1b400640 1 -- 172.21.15.97:0/2324231139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0b2040 con 0x55abd77d96f0 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e1b400640 1 -- 172.21.15.97:0/2324231139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e1c0c9040 con 0x55abd77d96f0 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e1b400640 1 -- 172.21.15.97:0/2324231139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c004c80 con 0x55abd77d96f0 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- 172.21.15.97:0/2324231139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd77d96f0 msgr2=0x55abd7897dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd77d96f0 0x55abd7897dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c019410 tx=0x7f2e1c0a7040 comp rx=0 tx=0).stop 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- 172.21.15.97:0/2324231139 shutdown_connections 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd78266f0 0x55abd7898840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd7825dd0 0x55abd7898300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2324231139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd77d96f0 0x55abd7897dc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- 172.21.15.97:0/2324231139 >> 172.21.15.97:0/2324231139 conn(0x55abd7806be0 msgr2=0x55abd7837690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- 172.21.15.97:0/2324231139 shutdown_connections 2024-09-10T21:25:59.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- 172.21.15.97:0/2324231139 wait complete. 2024-09-10T21:25:59.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- start start 2024-09-10T21:25:59.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 Processor -- start 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd789bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd7825dd0 0x55abd789c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd78266f0 0x55abd789c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55abd7632a60 con 0x55abd78266f0 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abd75c7b10 con 0x55abd7825dd0 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55abd76261e0 con 0x55abd77d96f0 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e354ae000 10 client.0 _ll_register_callbacks cb 0x7f2e1400c3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd78266f0 0x55abd789c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd78266f0 0x55abd789c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53000/0 (socket says 172.21.15.97:53000) 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 -- 172.21.15.97:0/2327380157 learned_addr learned my addr 172.21.15.97:0/2327380157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:25:59.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd789bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e22a00640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd7825dd0 0x55abd789c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 -- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 msgr2=0x55abd789bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd789bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 -- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd7825dd0 msgr2=0x55abd789c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd7825dd0 0x55abd789c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e2be00640 1 -- 172.21.15.97:0/2327380157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd75c8330 con 0x55abd78266f0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.943+0000 7f2e22a00640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd7825dd0 0x55abd789c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e2be00640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd78266f0 0x55abd789c7c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2e24004350 tx=0x7f2e24004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e857797fec8b114 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 1 -- 172.21.15.97:0/2327380157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24010e00 con 0x55abd78266f0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 1 -- 172.21.15.97:0/2327380157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f2e2401c430 con 0x55abd78266f0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1c0c9040 con 0x55abd78266f0 2024-09-10T21:25:59.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 1 -- 172.21.15.97:0/2327380157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2e24028050 con 0x55abd78266f0 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 10 client.5346 fetch_fsmap learned FSMap version 24 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55abd75ebcb0 con 0x55abd78266f0 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 1 -- 172.21.15.97:0/2327380157 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e1c0c9040 con 0x55abd78266f0 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 1 -- 172.21.15.97:0/2327380157 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f2e2402b020 con 0x55abd78266f0 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 10 client.5346 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 10 client.5346 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 20 client.5346 populate_metadata read hostname 'smithi097' 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd7858970 0x55abd7857b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 4 client.5346 mds_command: new command op to 5148 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2054577795}] 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 --> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2054577795}) -- 0x55abd7631690 con 0x55abd7858970 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd7858970 0x55abd7857b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e2b400640 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd7858970 0x55abd7857b00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6ef06e511de290 server_cookie=ec27221ec9c93caf in_seq=0 out_seq=0 2024-09-10T21:25:59.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 10 client.5346 ms_handle_connect on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 1 -- 172.21.15.97:0/2327380157 <== mds.0 v2:172.21.15.97:6832/95042702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55abd7631690 con 0x55abd7858970 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 10 client.5346 handle_command_reply: tid=0 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 client.5346 shutdown 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd78266f0 msgr2=0x55abd789c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd78266f0 0x55abd789c7c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2e24004350 tx=0x7f2e24004810 comp rx=0 tx=0).stop 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 shutdown_connections 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:6832/95042702,v1:172.21.15.97:6835/95042702] conn(0x55abd7858970 0x55abd7857b00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abd78266f0 0x55abd789c7c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abd7825dd0 0x55abd789c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e18c00640 0 client.5346 ms_handle_reset on v2:172.21.15.97:6832/95042702 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 --2- 172.21.15.97:0/2327380157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abd77d96f0 0x55abd789bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 >> 172.21.15.97:0/2327380157 conn(0x55abd7806be0 msgr2=0x55abd7837da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 shutdown_connections 2024-09-10T21:25:59.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 1 -- 172.21.15.97:0/2327380157 wait complete. 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e354ae000 20 client.5346 trim_cache size 0 max 16384 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e080725e0 msgr2=0x7f2e08074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e080725e0 0x7f2e08074aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0af000 tx=0x7f2e1c01b040 comp rx=0 tx=0).stop 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d0490 msgr2=0x7f2e2c157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d0490 0x7f2e2c157700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400c490 tx=0x7f2e1400c950 comp rx=0 tx=0).stop 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 shutdown_connections 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e080725e0 0x7f2e08074aa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e2c15efc0 0x7f2e2c0b6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d0490 0x7f2e2c157700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 --2- 172.21.15.97:0/4223043704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e2c0cdcb0 0x7f2e2c150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 >> 172.21.15.97:0/4223043704 conn(0x7f2e2c0d1a60 msgr2=0x7f2e2c0b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 shutdown_connections 2024-09-10T21:25:59.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:25:59.947+0000 7f2e30a00640 1 -- 172.21.15.97:0/4223043704 wait complete. 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> import errno 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> f.close() 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> except IOError as e: 2024-09-10T21:25:59.970 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(errno.EIO) 2024-09-10T21:25:59.971 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:26:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:26:00.049 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:26:00.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:00.159 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:00.260 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:26:00.260 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:00.260 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:00.260 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:00.270 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:00.270 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:00.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:26:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.503+0000 7ff156800640 1 -- 172.21.15.97:0/4151175181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 msgr2=0x7ff134004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.503+0000 7ff156800640 1 --2- 172.21.15.97:0/4151175181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 0x7ff134004c90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff138009b60 tx=0x7ff13801d270 comp rx=0 tx=0).stop 2024-09-10T21:26:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.503+0000 7ff14fe00640 1 -- 172.21.15.97:0/4151175181 reap_dead start 2024-09-10T21:26:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.503+0000 7ff156800640 1 -- 172.21.15.97:0/4151175181 shutdown_connections 2024-09-10T21:26:00.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.503+0000 7ff156800640 1 -- 172.21.15.97:0/4151175181 >> 172.21.15.97:0/4151175181 conn(0x7ff1500d1c30 msgr2=0x7ff1500d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:00.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.503+0000 7ff156800640 1 -- 172.21.15.97:0/4151175181 shutdown_connections 2024-09-10T21:26:00.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.503+0000 7ff156800640 1 -- 172.21.15.97:0/4151175181 wait complete. 2024-09-10T21:26:00.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 Processor -- start 2024-09-10T21:26:00.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 -- start start 2024-09-10T21:26:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 0x7ff1501f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1501f33d0 0x7ff1500cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff1500cba70 0x7ff1500c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1500afec0 con 0x7ff134004810 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff15015c1d0 con 0x7ff1501f33d0 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff156800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff1500b0040 con 0x7ff1500cba70 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 0x7ff1501f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 0x7ff1501f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53016/0 (socket says 172.21.15.97:53016) 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 -- 172.21.15.97:0/177570705 learned_addr learned my addr 172.21.15.97:0/177570705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14fe00640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff1500cba70 0x7ff1500c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff146a00640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1501f33d0 0x7ff1500cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 -- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff1500cba70 msgr2=0x7ff1500c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:00.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff1500cba70 0x7ff1500c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 -- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1501f33d0 msgr2=0x7ff1500cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1501f33d0 0x7ff1500cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 -- 172.21.15.97:0/177570705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1500c59b0 con 0x7ff134004810 2024-09-10T21:26:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff146a00640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1501f33d0 0x7ff1500cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:00.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff14f400640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 0x7ff1501f2e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff140010ca0 tx=0x7ff14000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff147e00640 1 -- 172.21.15.97:0/177570705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140017070 con 0x7ff134004810 2024-09-10T21:26:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff147e00640 1 -- 172.21.15.97:0/177570705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff140011750 con 0x7ff134004810 2024-09-10T21:26:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.507+0000 7ff147e00640 1 -- 172.21.15.97:0/177570705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140011ba0 con 0x7ff134004810 2024-09-10T21:26:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.511+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1500c5c90 con 0x7ff134004810 2024-09-10T21:26:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.511+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1500c6250 con 0x7ff134004810 2024-09-10T21:26:00.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.511+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1500cbe80 con 0x7ff134004810 2024-09-10T21:26:00.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.511+0000 7ff147e00640 1 -- 172.21.15.97:0/177570705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff14002c080 con 0x7ff134004810 2024-09-10T21:26:00.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.515+0000 7ff147e00640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff12c0725e0 0x7ff12c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:00.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.515+0000 7ff147e00640 1 -- 172.21.15.97:0/177570705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7ff140094490 con 0x7ff134004810 2024-09-10T21:26:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.515+0000 7ff146a00640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff12c0725e0 0x7ff12c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.515+0000 7ff146a00640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff12c0725e0 0x7ff12c074aa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff1380ae000 tx=0x7ff13801a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.515+0000 7ff147e00640 1 -- 172.21.15.97:0/177570705 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1500cbe80 con 0x7ff134004810 2024-09-10T21:26:00.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff1500cece0 con 0x7ff134004810 2024-09-10T21:26:00.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff147e00640 1 -- 172.21.15.97:0/177570705 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v43) ==== 80+0+122 (secure 0 0 0) 0x7ff14005fac0 con 0x7ff134004810 2024-09-10T21:26:00.698 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2127253870 2024-09-11T21:25:19.147709+0000 2024-09-10T21:26:00.698 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2127253870 2024-09-11T21:25:19.147709+0000 2024-09-10T21:26:00.699 INFO:teuthology.orchestra.run.smithi097.stderr:listed 2 entries 2024-09-10T21:26:00.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff12c0725e0 msgr2=0x7ff12c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:00.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff12c0725e0 0x7ff12c074aa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff1380ae000 tx=0x7ff13801a010 comp rx=0 tx=0).stop 2024-09-10T21:26:00.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 msgr2=0x7ff1501f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 0x7ff1501f2e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff140010ca0 tx=0x7ff14000e720 comp rx=0 tx=0).stop 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 shutdown_connections 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff12c0725e0 0x7ff12c074aa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff1500cba70 0x7ff1500c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1501f33d0 0x7ff1500cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 --2- 172.21.15.97:0/177570705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff134004810 0x7ff1501f2e90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 >> 172.21.15.97:0/177570705 conn(0x7ff1500d1c30 msgr2=0x7ff1500d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 shutdown_connections 2024-09-10T21:26:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:00.695+0000 7ff156800640 1 -- 172.21.15.97:0/177570705 wait complete. 2024-09-10T21:26:00.719 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:26:00.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:00.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:26:00.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:26:00.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:00.747 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:26:00.796 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:00.797 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:00.797 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:00.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:00.806 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:00.807 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:00.807 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:00.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:26:00.852 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:00.852 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:00.852 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:00.861 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:00.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:01.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- 172.21.15.97:0/3428382501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0cec10 msgr2=0x7f6b6c0cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 --2- 172.21.15.97:0/3428382501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0cec10 0x7f6b6c0cff10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0b4860 tx=0x7f6b5401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- 172.21.15.97:0/3428382501 shutdown_connections 2024-09-10T21:26:01.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 --2- 172.21.15.97:0/3428382501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b6c0b1560 0x7f6b6c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 --2- 172.21.15.97:0/3428382501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0cec10 0x7f6b6c0cff10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- 172.21.15.97:0/3428382501 >> 172.21.15.97:0/3428382501 conn(0x7f6b6c0b37c0 msgr2=0x7f6b6c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:01.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- 172.21.15.97:0/3428382501 shutdown_connections 2024-09-10T21:26:01.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- 172.21.15.97:0/3428382501 wait complete. 2024-09-10T21:26:01.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 Processor -- start 2024-09-10T21:26:01.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- start start 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0b1560 0x7f6b6c0b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b6c0ce830 0x7f6b6c0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b6c0cec10 0x7f6b6c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c0ce320 con 0x7f6b6c0b1560 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c0ce1a0 con 0x7f6b6c0ce830 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b71400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6c0d0450 con 0x7f6b6c0cec10 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0b1560 0x7f6b6c0b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0b1560 0x7f6b6c0b73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53034/0 (socket says 172.21.15.97:53034) 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 -- 172.21.15.97:0/1262410261 learned_addr learned my addr 172.21.15.97:0/1262410261 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:01.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b61e00640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b6c0ce830 0x7f6b6c0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6b200640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b6c0cec10 0x7f6b6c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 -- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b6c0cec10 msgr2=0x7f6b6c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b6c0cec10 0x7f6b6c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 -- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b6c0ce830 msgr2=0x7f6b6c0c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b6c0ce830 0x7f6b6c0c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 -- 172.21.15.97:0/1262410261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c1f8390 con 0x7f6b6c0b1560 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b61e00640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b6c0ce830 0x7f6b6c0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6b200640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b6c0cec10 0x7f6b6c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.035+0000 7f6b6a800640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0b1560 0x7f6b6c0b73b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c7a0 tx=0x7f6b5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:01.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b63400640 1 -- 172.21.15.97:0/1262410261 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c017070 con 0x7f6b6c0b1560 2024-09-10T21:26:01.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b63400640 1 -- 172.21.15.97:0/1262410261 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b5c005070 con 0x7f6b6c0b1560 2024-09-10T21:26:01.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c1f8670 con 0x7f6b6c0b1560 2024-09-10T21:26:01.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b63400640 1 -- 172.21.15.97:0/1262410261 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c005500 con 0x7f6b6c0b1560 2024-09-10T21:26:01.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c1f8c30 con 0x7f6b6c0b1560 2024-09-10T21:26:01.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c0b1970 con 0x7f6b6c0b1560 2024-09-10T21:26:01.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b63400640 1 -- 172.21.15.97:0/1262410261 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6b5c00a5e0 con 0x7f6b6c0b1560 2024-09-10T21:26:01.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b63400640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b480725e0 0x7f6b48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b63400640 1 -- 172.21.15.97:0/1262410261 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f6b5c092ed0 con 0x7f6b6c0b1560 2024-09-10T21:26:01.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.039+0000 7f6b61e00640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b480725e0 0x7f6b48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.043+0000 7f6b61e00640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b480725e0 0x7f6b48074aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6b540b2000 tx=0x7f6b5401d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:01.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.043+0000 7f6b63400640 1 -- 172.21.15.97:0/1262410261 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6b5c05e390 con 0x7f6b6c0b1560 2024-09-10T21:26:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.243+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b6c0aeaa0 con 0x7f6b6c0b1560 2024-09-10T21:26:01.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.243+0000 7f6b63400640 1 -- 172.21.15.97:0/1262410261 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5221 (secure 0 0 0) 0x7f6b5c05dc90 con 0x7f6b6c0b1560 2024-09-10T21:26:01.246 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:01.247 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"btime":"2024-09-10T21:25:23:161523+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:23.161515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5148,"qdb_cluster":[5148]},"id":2}]} 2024-09-10T21:26:01.247 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-09-10T21:26:01.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b480725e0 msgr2=0x7f6b48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b480725e0 0x7f6b48074aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6b540b2000 tx=0x7f6b5401d910 comp rx=0 tx=0).stop 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0b1560 msgr2=0x7f6b6c0b73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0b1560 0x7f6b6c0b73b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c7a0 tx=0x7f6b5c00cc60 comp rx=0 tx=0).stop 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 shutdown_connections 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b480725e0 0x7f6b48074aa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b6c0cec10 0x7f6b6c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b6c0ce830 0x7f6b6c0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 --2- 172.21.15.97:0/1262410261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6c0b1560 0x7f6b6c0b73b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 >> 172.21.15.97:0/1262410261 conn(0x7f6b6c0b37c0 msgr2=0x7f6b6c15d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 shutdown_connections 2024-09-10T21:26:01.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.247+0000 7f6b71400640 1 -- 172.21.15.97:0/1262410261 wait complete. 2024-09-10T21:26:01.268 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.407+0000 7f1db3200640 1 -- 172.21.15.97:0/1980176214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d90004d50 msgr2=0x7f1d900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.407+0000 7f1db3200640 1 --2- 172.21.15.97:0/1980176214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d90004d50 0x7f1d900051d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1da400c640 tx=0x7f1da401d2a0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.407+0000 7f1db1200640 1 -- 172.21.15.97:0/1980176214 reap_dead start 2024-09-10T21:26:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.407+0000 7f1db3200640 1 -- 172.21.15.97:0/1980176214 shutdown_connections 2024-09-10T21:26:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.407+0000 7f1db3200640 1 -- 172.21.15.97:0/1980176214 >> 172.21.15.97:0/1980176214 conn(0x7f1dac0b37c0 msgr2=0x7f1dac0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:01.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.407+0000 7f1db3200640 1 -- 172.21.15.97:0/1980176214 shutdown_connections 2024-09-10T21:26:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.407+0000 7f1db3200640 1 -- 172.21.15.97:0/1980176214 wait complete. 2024-09-10T21:26:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 Processor -- start 2024-09-10T21:26:01.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 -- start start 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d90004d50 0x7f1dac1f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1dac0c9b90 0x7f1dac1f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dac0b6ef0 0x7f1dac1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac0cddd0 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac0cdc50 con 0x7f1d90004d50 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac0ce1a0 con 0x7f1dac0c9b90 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dac0b6ef0 0x7f1dac1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dac0b6ef0 0x7f1dac1f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53056/0 (socket says 172.21.15.97:53056) 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 -- 172.21.15.97:0/1441391935 learned_addr learned my addr 172.21.15.97:0/1441391935 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:01.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1dabe00640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d90004d50 0x7f1dac1f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1da3400640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1dac0c9b90 0x7f1dac1f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 -- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1dac0c9b90 msgr2=0x7f1dac1f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1dac0c9b90 0x7f1dac1f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 -- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d90004d50 msgr2=0x7f1dac1f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d90004d50 0x7f1dac1f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 -- 172.21.15.97:0/1441391935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac0ca0e0 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db1200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dac0b6ef0 0x7f1dac1f3e30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1da400c640 tx=0x7f1da4006680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1da8c00640 1 -- 172.21.15.97:0/1441391935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da40b2030 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1da8c00640 1 -- 172.21.15.97:0/1441391935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1da4004e40 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac0ca3c0 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1da8c00640 1 -- 172.21.15.97:0/1441391935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4001fe0 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1f4e10 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.411+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac0b26e0 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.415+0000 7f1da8c00640 1 -- 172.21.15.97:0/1441391935 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1da40066f0 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.415+0000 7f1da8c00640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d88072610 0x7f1d88074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.415+0000 7f1da8c00640 1 -- 172.21.15.97:0/1441391935 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f1da40b6020 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.415+0000 7f1dabe00640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d88072610 0x7f1d88074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.415+0000 7f1dabe00640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d88072610 0x7f1d88074ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009c60 tx=0x7f1d9c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:01.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.419+0000 7f1da8c00640 1 -- 172.21.15.97:0/1441391935 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1da4017050 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.615+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dac071d10 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.615+0000 7f1da8c00640 1 -- 172.21.15.97:0/1441391935 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5221 (secure 0 0 0) 0x7f1da40f8b20 con 0x7f1dac0b6ef0 2024-09-10T21:26:01.619 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:01.619 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"btime":"2024-09-10T21:25:23:161523+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:23.161515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5148,"qdb_cluster":[5148]},"id":2}]} 2024-09-10T21:26:01.620 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-09-10T21:26:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d88072610 msgr2=0x7f1d88074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d88072610 0x7f1d88074ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009c60 tx=0x7f1d9c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dac0b6ef0 msgr2=0x7f1dac1f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dac0b6ef0 0x7f1dac1f3e30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1da400c640 tx=0x7f1da4006680 comp rx=0 tx=0).stop 2024-09-10T21:26:01.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 shutdown_connections 2024-09-10T21:26:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d88072610 0x7f1d88074ad0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dac0b6ef0 0x7f1dac1f3e30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1dac0c9b90 0x7f1dac1f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 --2- 172.21.15.97:0/1441391935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d90004d50 0x7f1dac1f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 >> 172.21.15.97:0/1441391935 conn(0x7f1dac0b37c0 msgr2=0x7f1dac0b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 shutdown_connections 2024-09-10T21:26:01.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.619+0000 7f1db3200640 1 -- 172.21.15.97:0/1441391935 wait complete. 2024-09-10T21:26:01.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:01.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 -- 172.21.15.97:0/3775084608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144690 msgr2=0x7fd15c0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 --2- 172.21.15.97:0/3775084608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144690 0x7fd15c0b2a80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd144009840 tx=0x7fd14401c820 comp rx=0 tx=0).stop 2024-09-10T21:26:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 -- 172.21.15.97:0/3775084608 shutdown_connections 2024-09-10T21:26:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 --2- 172.21.15.97:0/3775084608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144690 0x7fd15c0b2a80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 -- 172.21.15.97:0/3775084608 >> 172.21.15.97:0/3775084608 conn(0x7fd15c0b1780 msgr2=0x7fd15c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:01.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 -- 172.21.15.97:0/3775084608 shutdown_connections 2024-09-10T21:26:01.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 -- 172.21.15.97:0/3775084608 wait complete. 2024-09-10T21:26:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 Processor -- start 2024-09-10T21:26:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.787+0000 7fd161a00640 1 -- start start 2024-09-10T21:26:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd15c06fd50 0x7fd15c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c143f90 0x7fd15c0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd15c144690 0x7fd15c0730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c169410 con 0x7fd15c143f90 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c169290 con 0x7fd15c144690 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd15c169590 con 0x7fd15c06fd50 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c143f90 0x7fd15c0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c143f90 0x7fd15c0aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53066/0 (socket says 172.21.15.97:53066) 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 -- 172.21.15.97:0/2594492952 learned_addr learned my addr 172.21.15.97:0/2594492952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd15b400640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd15c144690 0x7fd15c0730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd15aa00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd15c06fd50 0x7fd15c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 -- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd15c06fd50 msgr2=0x7fd15c15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd15c06fd50 0x7fd15c15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 -- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd15c144690 msgr2=0x7fd15c0730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd15c144690 0x7fd15c0730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 -- 172.21.15.97:0/2594492952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c0af370 con 0x7fd15c143f90 2024-09-10T21:26:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd152000640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c143f90 0x7fd15c0aee30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd144009790 tx=0x7fd1440a6780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd153400640 1 -- 172.21.15.97:0/2594492952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144004140 con 0x7fd15c143f90 2024-09-10T21:26:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd153400640 1 -- 172.21.15.97:0/2594492952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1440043d0 con 0x7fd15c143f90 2024-09-10T21:26:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd153400640 1 -- 172.21.15.97:0/2594492952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144003600 con 0x7fd15c143f90 2024-09-10T21:26:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c1f8050 con 0x7fd15c143f90 2024-09-10T21:26:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c1f8610 con 0x7fd15c143f90 2024-09-10T21:26:01.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c0b2a80 con 0x7fd15c143f90 2024-09-10T21:26:01.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.791+0000 7fd153400640 1 -- 172.21.15.97:0/2594492952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1440037a0 con 0x7fd15c143f90 2024-09-10T21:26:01.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.795+0000 7fd153400640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd138072610 0x7fd138074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:01.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.795+0000 7fd153400640 1 -- 172.21.15.97:0/2594492952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7fd14412daa0 con 0x7fd15c143f90 2024-09-10T21:26:01.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.795+0000 7fd15aa00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd138072610 0x7fd138074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:01.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.795+0000 7fd15aa00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd138072610 0x7fd138074ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0012f0 tx=0x7fd14c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:01.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.799+0000 7fd153400640 1 -- 172.21.15.97:0/2594492952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd144016410 con 0x7fd15c143f90 2024-09-10T21:26:01.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.963+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd15c13db40 con 0x7fd15c143f90 2024-09-10T21:26:01.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.963+0000 7fd153400640 1 -- 172.21.15.97:0/2594492952 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17190 (secure 0 0 0) 0x7fd1440f8f50 con 0x7fd15c143f90 2024-09-10T21:26:01.968 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:01.968 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:25:53.238937+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:25:07.800828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T21:25:08.646331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.9","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/2127253870":"2024-09-11T21:25:19.147709+0000","172.21.15.97:6832/2127253870":"2024-09-11T21:25:19.147709+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd138072610 msgr2=0x7fd138074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd138072610 0x7fd138074ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0012f0 tx=0x7fd14c009040 comp rx=0 tx=0).stop 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c143f90 msgr2=0x7fd15c0aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c143f90 0x7fd15c0aee30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd144009790 tx=0x7fd1440a6780 comp rx=0 tx=0).stop 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 shutdown_connections 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd138072610 0x7fd138074ad0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd15c144690 0x7fd15c0730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c143f90 0x7fd15c0aee30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 --2- 172.21.15.97:0/2594492952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd15c06fd50 0x7fd15c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 >> 172.21.15.97:0/2594492952 conn(0x7fd15c0b1780 msgr2=0x7fd15c0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 shutdown_connections 2024-09-10T21:26:01.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:01.967+0000 7fd161a00640 1 -- 172.21.15.97:0/2594492952 wait complete. 2024-09-10T21:26:01.990 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:26:01.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 -- 172.21.15.97:0/783195768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240cec10 msgr2=0x7fd4240cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 --2- 172.21.15.97:0/783195768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240cec10 0x7fd4240cff10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd410009900 tx=0x7fd41001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 -- 172.21.15.97:0/783195768 shutdown_connections 2024-09-10T21:26:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 --2- 172.21.15.97:0/783195768 >> v2:172.21.15.194:3300/0 conn(0x7fd4240b1560 0x7fd4240b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 --2- 172.21.15.97:0/783195768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240cec10 0x7fd4240cff10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 -- 172.21.15.97:0/783195768 >> 172.21.15.97:0/783195768 conn(0x7fd4240b37c0 msgr2=0x7fd4240b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:02.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 -- 172.21.15.97:0/783195768 shutdown_connections 2024-09-10T21:26:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.127+0000 7fd42a200640 1 -- 172.21.15.97:0/783195768 wait complete. 2024-09-10T21:26:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 Processor -- start 2024-09-10T21:26:02.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 -- start start 2024-09-10T21:26:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240b1560 0x7fd4241fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4240ce830 0x7fd4241fc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4240cec10 0x7fd4241fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240ce1a0 con 0x7fd4240ce830 2024-09-10T21:26:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240cde60 con 0x7fd4240cec10 2024-09-10T21:26:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd4240ce320 con 0x7fd4240b1560 2024-09-10T21:26:02.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4240ce830 0x7fd4241fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4240ce830 0x7fd4241fc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53082/0 (socket says 172.21.15.97:53082) 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 -- 172.21.15.97:0/236686740 learned_addr learned my addr 172.21.15.97:0/236686740 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd423e00640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4240cec10 0x7fd4241fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd423400640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240b1560 0x7fd4241fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 -- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240b1560 msgr2=0x7fd4241fbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240b1560 0x7fd4241fbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 -- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4240cec10 msgr2=0x7fd4241fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4240cec10 0x7fd4241fcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 -- 172.21.15.97:0/236686740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4241e90c0 con 0x7fd4240ce830 2024-09-10T21:26:02.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd422a00640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4240ce830 0x7fd4241fc560 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd410009850 tx=0x7fd410005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd41b400640 1 -- 172.21.15.97:0/236686740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100b5040 con 0x7fd4240ce830 2024-09-10T21:26:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd41b400640 1 -- 172.21.15.97:0/236686740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4100be070 con 0x7fd4240ce830 2024-09-10T21:26:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd41b400640 1 -- 172.21.15.97:0/236686740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410003b30 con 0x7fd4240ce830 2024-09-10T21:26:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4241e93a0 con 0x7fd4240ce830 2024-09-10T21:26:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.131+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4241e9960 con 0x7fd4240ce830 2024-09-10T21:26:02.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.135+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4241e9d40 con 0x7fd4240ce830 2024-09-10T21:26:02.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.135+0000 7fd41b400640 1 -- 172.21.15.97:0/236686740 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd410003cd0 con 0x7fd4240ce830 2024-09-10T21:26:02.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.135+0000 7fd41b400640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4040725e0 0x7fd404074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.135+0000 7fd41b400640 1 -- 172.21.15.97:0/236686740 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7fd410130ea0 con 0x7fd4240ce830 2024-09-10T21:26:02.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.135+0000 7fd423400640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4040725e0 0x7fd404074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.139+0000 7fd423400640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4040725e0 0x7fd404074aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4240697f0 tx=0x7fd414009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:02.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.139+0000 7fd41b400640 1 -- 172.21.15.97:0/236686740 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4241e9d40 con 0x7fd4240ce830 2024-09-10T21:26:02.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4240cf800 con 0x7fd4240ce830 2024-09-10T21:26:02.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd41b400640 1 -- 172.21.15.97:0/236686740 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5221 (secure 0 0 0) 0x7fd410018c30 con 0x7fd4240ce830 2024-09-10T21:26:02.346 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:02.346 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"btime":"2024-09-10T21:25:23:161523+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:23.161515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5148,"qdb_cluster":[5148]},"id":2}]} 2024-09-10T21:26:02.346 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4040725e0 msgr2=0x7fd404074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4040725e0 0x7fd404074aa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4240697f0 tx=0x7fd414009040 comp rx=0 tx=0).stop 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4240ce830 msgr2=0x7fd4241fc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4240ce830 0x7fd4241fc560 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd410009850 tx=0x7fd410005790 comp rx=0 tx=0).stop 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 shutdown_connections 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4040725e0 0x7fd404074aa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4240cec10 0x7fd4241fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4240ce830 0x7fd4241fc560 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 --2- 172.21.15.97:0/236686740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4240b1560 0x7fd4241fbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 >> 172.21.15.97:0/236686740 conn(0x7fd4240b37c0 msgr2=0x7fd424071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 shutdown_connections 2024-09-10T21:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.343+0000 7fd42a200640 1 -- 172.21.15.97:0/236686740 wait complete. 2024-09-10T21:26:02.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- 172.21.15.97:0/3653257764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80d0280 msgr2=0x7f75b815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 --2- 172.21.15.97:0/3653257764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80d0280 0x7f75b815fd20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f75b80738d0 tx=0x7f75b001f580 comp rx=0 tx=0).stop 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- 172.21.15.97:0/3653257764 shutdown_connections 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 --2- 172.21.15.97:0/3653257764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80d0280 0x7f75b815fd20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 --2- 172.21.15.97:0/3653257764 >> v2:172.21.15.194:3300/0 conn(0x7f75b80cec10 0x7f75b80cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 --2- 172.21.15.97:0/3653257764 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75b80ce830 0x7f75b80cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- 172.21.15.97:0/3653257764 >> 172.21.15.97:0/3653257764 conn(0x7f75b80b37c0 msgr2=0x7f75b80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- 172.21.15.97:0/3653257764 shutdown_connections 2024-09-10T21:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- 172.21.15.97:0/3653257764 wait complete. 2024-09-10T21:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 Processor -- start 2024-09-10T21:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- start start 2024-09-10T21:26:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75b80ce830 0x7f75b8149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80cec10 0x7f75b80c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75b80d0280 0x7f75b81f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75b80b4420 con 0x7f75b80cec10 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f75b80cde60 con 0x7f75b80d0280 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75bca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f75b80ce1a0 con 0x7f75b80ce830 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80cec10 0x7f75b80c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80cec10 0x7f75b80c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53096/0 (socket says 172.21.15.97:53096) 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 -- 172.21.15.97:0/3209729625 learned_addr learned my addr 172.21.15.97:0/3209729625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5e00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75b80ce830 0x7f75b8149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b6800640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75b80d0280 0x7f75b81f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 -- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75b80ce830 msgr2=0x7f75b8149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75b80ce830 0x7f75b8149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 -- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75b80d0280 msgr2=0x7f75b81f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75b80d0280 0x7f75b81f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 -- 172.21.15.97:0/3209729625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b81f6730 con 0x7f75b80cec10 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5e00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75b80ce830 0x7f75b8149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.507+0000 7f75b5400640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80cec10 0x7f75b80c6100 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d980 tx=0x7f75a400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75ae000640 1 -- 172.21.15.97:0/3209729625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4017070 con 0x7f75b80cec10 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75ae000640 1 -- 172.21.15.97:0/3209729625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75a4009d20 con 0x7f75b80cec10 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75ae000640 1 -- 172.21.15.97:0/3209729625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a40106c0 con 0x7f75b80cec10 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b81f6a10 con 0x7f75b80cec10 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b80c5be0 con 0x7f75b80cec10 2024-09-10T21:26:02.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b815fd20 con 0x7f75b80cec10 2024-09-10T21:26:02.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75ae000640 1 -- 172.21.15.97:0/3209729625 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f75a4010860 con 0x7f75b80cec10 2024-09-10T21:26:02.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75ae000640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75980725e0 0x7f7598074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.511+0000 7f75ae000640 1 -- 172.21.15.97:0/3209729625 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f75a4093800 con 0x7f75b80cec10 2024-09-10T21:26:02.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.515+0000 7f75b5e00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75980725e0 0x7f7598074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.515+0000 7f75b5e00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75980725e0 0x7f7598074aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75a80058d0 tx=0x7f75a8005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:02.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.515+0000 7f75ae000640 1 -- 172.21.15.97:0/3209729625 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f75a405ecb0 con 0x7f75b80cec10 2024-09-10T21:26:02.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.711+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75b80cc060 con 0x7f75b80cec10 2024-09-10T21:26:02.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.711+0000 7f75ae000640 1 -- 172.21.15.97:0/3209729625 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5221 (secure 0 0 0) 0x7f75a405e5b0 con 0x7f75b80cec10 2024-09-10T21:26:02.716 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:02.716 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":24,"btime":"2024-09-10T21:25:23:161523+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:25:09.683475+0000","modified":"2024-09-10T21:25:23.161515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5148":{"gid":5148,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/95042702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":95042702},{"type":"v1","addr":"172.21.15.97:6835","nonce":95042702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5148,"qdb_cluster":[5148]},"id":2}]} 2024-09-10T21:26:02.717 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 24 2024-09-10T21:26:02.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75980725e0 msgr2=0x7f7598074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75980725e0 0x7f7598074aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75a80058d0 tx=0x7f75a8005860 comp rx=0 tx=0).stop 2024-09-10T21:26:02.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80cec10 msgr2=0x7f75b80c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80cec10 0x7f75b80c6100 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d980 tx=0x7f75a400de40 comp rx=0 tx=0).stop 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 shutdown_connections 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75980725e0 0x7f7598074aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f75b80d0280 0x7f75b81f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75b80cec10 0x7f75b80c6100 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 --2- 172.21.15.97:0/3209729625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f75b80ce830 0x7f75b8149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 >> 172.21.15.97:0/3209729625 conn(0x7f75b80b37c0 msgr2=0x7f75b80b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 shutdown_connections 2024-09-10T21:26:02.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.715+0000 7f75bca00640 1 -- 172.21.15.97:0/3209729625 wait complete. 2024-09-10T21:26:02.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:02.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 -- 172.21.15.97:0/2871408712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80b0d50 msgr2=0x7f1bd80aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 --2- 172.21.15.97:0/2871408712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80b0d50 0x7f1bd80aedc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8074d10 tx=0x7f1bc001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 -- 172.21.15.97:0/2871408712 shutdown_connections 2024-09-10T21:26:02.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 --2- 172.21.15.97:0/2871408712 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bd80af300 0x7f1bd80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 --2- 172.21.15.97:0/2871408712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80b0d50 0x7f1bd80aedc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 --2- 172.21.15.97:0/2871408712 >> v2:172.21.15.194:3300/0 conn(0x7f1bd80ce640 0x7f1bd80cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 -- 172.21.15.97:0/2871408712 >> 172.21.15.97:0/2871408712 conn(0x7f1bd80d1c30 msgr2=0x7f1bd80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:02.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 -- 172.21.15.97:0/2871408712 shutdown_connections 2024-09-10T21:26:02.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.875+0000 7f1bdee00640 1 -- 172.21.15.97:0/2871408712 wait complete. 2024-09-10T21:26:02.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 Processor -- start 2024-09-10T21:26:02.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 -- start start 2024-09-10T21:26:02.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80af300 0x7f1bd80c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd80b0d50 0x7f1bd8157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bd80ce640 0x7f1bd80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd80cd270 con 0x7f1bd80af300 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd80cd0f0 con 0x7f1bd80b0d50 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd80cd3f0 con 0x7f1bd80ce640 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80af300 0x7f1bd80c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80af300 0x7f1bd80c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53116/0 (socket says 172.21.15.97:53116) 2024-09-10T21:26:02.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 -- 172.21.15.97:0/3671849317 learned_addr learned my addr 172.21.15.97:0/3671849317 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:02.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bcf400640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd80b0d50 0x7f1bd8157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdce00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bd80ce640 0x7f1bd80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 -- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bd80ce640 msgr2=0x7f1bd80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bd80ce640 0x7f1bd80b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 -- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd80b0d50 msgr2=0x7f1bd8157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd80b0d50 0x7f1bd8157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 -- 172.21.15.97:0/3671849317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd80b78c0 con 0x7f1bd80af300 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bcf400640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd80b0d50 0x7f1bd8157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdce00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bd80ce640 0x7f1bd80b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd7e00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80af300 0x7f1bd80c63e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800ccb0 tx=0x7f1bc800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd4c00640 1 -- 172.21.15.97:0/3671849317 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800f450 con 0x7f1bd80af300 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd4c00640 1 -- 172.21.15.97:0/3671849317 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bc800f6e0 con 0x7f1bd80af300 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd80c5900 con 0x7f1bd80af300 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bd4c00640 1 -- 172.21.15.97:0/3671849317 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800fb30 con 0x7f1bd80af300 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd8157970 con 0x7f1bd80af300 2024-09-10T21:26:02.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.879+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd80cd950 con 0x7f1bd80af300 2024-09-10T21:26:02.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.883+0000 7f1bd4c00640 1 -- 172.21.15.97:0/3671849317 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1bc8020030 con 0x7f1bd80af300 2024-09-10T21:26:02.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.883+0000 7f1bd4c00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bb4072610 0x7f1bb4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:02.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.883+0000 7f1bd4c00640 1 -- 172.21.15.97:0/3671849317 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f1bc8093ce0 con 0x7f1bd80af300 2024-09-10T21:26:02.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.883+0000 7f1bcf400640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bb4072610 0x7f1bb4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:02.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.883+0000 7f1bcf400640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bb4072610 0x7f1bb4074ad0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00af000 tx=0x7f1bc001b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:02.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:02.887+0000 7f1bd4c00640 1 -- 172.21.15.97:0/3671849317 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1bc8014030 con 0x7f1bd80af300 2024-09-10T21:26:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bd80aedd0 con 0x7f1bd80af300 2024-09-10T21:26:03.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bd4c00640 1 -- 172.21.15.97:0/3671849317 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17190 (secure 0 0 0) 0x7f1bc805f1a0 con 0x7f1bd80af300 2024-09-10T21:26:03.058 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:03.058 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:25:53.238937+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:25:07.800828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T21:25:08.646331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.9","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/2127253870":"2024-09-11T21:25:19.147709+0000","172.21.15.97:6832/2127253870":"2024-09-11T21:25:19.147709+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bb4072610 msgr2=0x7f1bb4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bb4072610 0x7f1bb4074ad0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00af000 tx=0x7f1bc001b100 comp rx=0 tx=0).stop 2024-09-10T21:26:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80af300 msgr2=0x7f1bd80c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80af300 0x7f1bd80c63e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800ccb0 tx=0x7f1bc800a620 comp rx=0 tx=0).stop 2024-09-10T21:26:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 shutdown_connections 2024-09-10T21:26:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bb4072610 0x7f1bb4074ad0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bd80ce640 0x7f1bd80b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bd80b0d50 0x7f1bd8157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 --2- 172.21.15.97:0/3671849317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bd80af300 0x7f1bd80c63e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 >> 172.21.15.97:0/3671849317 conn(0x7f1bd80d1c30 msgr2=0x7f1bd815b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 shutdown_connections 2024-09-10T21:26:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.055+0000 7f1bdee00640 1 -- 172.21.15.97:0/3671849317 wait complete. 2024-09-10T21:26:03.079 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:26:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.215+0000 7f9a1d400640 1 -- 172.21.15.97:0/4005802365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a180cf5f0 msgr2=0x7f9a1806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.215+0000 7f9a1d400640 1 --2- 172.21.15.97:0/4005802365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a180cf5f0 0x7f9a1806f090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009bf0 tx=0x7f9a1001f390 comp rx=0 tx=0).stop 2024-09-10T21:26:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.215+0000 7f9a1d400640 1 -- 172.21.15.97:0/4005802365 shutdown_connections 2024-09-10T21:26:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.215+0000 7f9a1d400640 1 --2- 172.21.15.97:0/4005802365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a180cf5f0 0x7f9a1806f090 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.215+0000 7f9a1d400640 1 -- 172.21.15.97:0/4005802365 >> 172.21.15.97:0/4005802365 conn(0x7f9a180b1780 msgr2=0x7f9a180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.215+0000 7f9a1d400640 1 -- 172.21.15.97:0/4005802365 shutdown_connections 2024-09-10T21:26:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.215+0000 7f9a1d400640 1 -- 172.21.15.97:0/4005802365 wait complete. 2024-09-10T21:26:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 Processor -- start 2024-09-10T21:26:03.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 -- start start 2024-09-10T21:26:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a1806ec40 0x7f9a180ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a180cf5f0 0x7f9a181593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a181511f0 0x7f9a181440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1815abd0 con 0x7f9a181511f0 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1815aa50 con 0x7f9a1806ec40 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1815ad50 con 0x7f9a180cf5f0 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a181511f0 0x7f9a181440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a181511f0 0x7f9a181440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53122/0 (socket says 172.21.15.97:53122) 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 -- 172.21.15.97:0/1311271802 learned_addr learned my addr 172.21.15.97:0/1311271802 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 -- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a180cf5f0 msgr2=0x7f9a181593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a180cf5f0 0x7f9a181593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 -- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a1806ec40 msgr2=0x7f9a180ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a16800640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a1806ec40 0x7f9a180ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a1806ec40 0x7f9a180ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 -- 172.21.15.97:0/1311271802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a180b25f0 con 0x7f9a181511f0 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a16800640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a1806ec40 0x7f9a180ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:03.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a17200640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a181511f0 0x7f9a181440c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a1001bfa0 tx=0x7f9a100af580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a0f400640 1 -- 172.21.15.97:0/1311271802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100b4040 con 0x7f9a181511f0 2024-09-10T21:26:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a0f400640 1 -- 172.21.15.97:0/1311271802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a100bf070 con 0x7f9a181511f0 2024-09-10T21:26:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a0f400640 1 -- 172.21.15.97:0/1311271802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100b09b0 con 0x7f9a181511f0 2024-09-10T21:26:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.219+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a181f8cb0 con 0x7f9a181511f0 2024-09-10T21:26:03.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.223+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1814b9d0 con 0x7f9a181511f0 2024-09-10T21:26:03.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.223+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1806f090 con 0x7f9a181511f0 2024-09-10T21:26:03.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.223+0000 7f9a0f400640 1 -- 172.21.15.97:0/1311271802 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9a100b0b70 con 0x7f9a181511f0 2024-09-10T21:26:03.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.223+0000 7f9a0f400640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.223+0000 7f9a0f400640 1 -- 172.21.15.97:0/1311271802 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f9a10131840 con 0x7f9a181511f0 2024-09-10T21:26:03.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.223+0000 7f9a16800640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:03.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.227+0000 7f9a16800640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a08000f70 tx=0x7f9a08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:03.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.227+0000 7f9a0f400640 1 -- 172.21.15.97:0/1311271802 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a10018ef0 con 0x7f9a181511f0 2024-09-10T21:26:03.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.427+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9a181e84f0 con 0x7f9a181511f0 2024-09-10T21:26:03.431 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:26:03.427+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:26:03.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.431+0000 7f9a0f400640 1 -- 172.21.15.97:0/1311271802 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f9a100fcc60 con 0x7f9a181511f0 2024-09-10T21:26:03.434 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:26:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 msgr2=0x7f99f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a08000f70 tx=0x7f9a08009040 comp rx=0 tx=0).stop 2024-09-10T21:26:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a181511f0 msgr2=0x7f9a181440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a181511f0 0x7f9a181440c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a1001bfa0 tx=0x7f9a100af580 comp rx=0 tx=0).stop 2024-09-10T21:26:03.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 shutdown_connections 2024-09-10T21:26:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a181511f0 0x7f9a181440c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a180cf5f0 0x7f9a181593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 --2- 172.21.15.97:0/1311271802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a1806ec40 0x7f9a180ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 >> 172.21.15.97:0/1311271802 conn(0x7f9a180b1780 msgr2=0x7f9a180ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 shutdown_connections 2024-09-10T21:26:03.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.435+0000 7f9a1d400640 1 -- 172.21.15.97:0/1311271802 wait complete. 2024-09-10T21:26:03.446 INFO:tasks.ceph.mds.c.smithi097.stderr: -2998> 2024-09-10T21:25:26.155+0000 7fc9d0000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:26:03.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:26:03.492 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:26:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 -- 172.21.15.97:0/4221899086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14070460 msgr2=0x7fec140b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 --2- 172.21.15.97:0/4221899086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14070460 0x7fec140b2a80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009870 tx=0x7fec0c01c780 comp rx=0 tx=0).stop 2024-09-10T21:26:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 -- 172.21.15.97:0/4221899086 shutdown_connections 2024-09-10T21:26:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 --2- 172.21.15.97:0/4221899086 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14070460 0x7fec140b2a80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 -- 172.21.15.97:0/4221899086 >> 172.21.15.97:0/4221899086 conn(0x7fec140b1780 msgr2=0x7fec140b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 -- 172.21.15.97:0/4221899086 shutdown_connections 2024-09-10T21:26:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 -- 172.21.15.97:0/4221899086 wait complete. 2024-09-10T21:26:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 Processor -- start 2024-09-10T21:26:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.603+0000 7fec19400640 1 -- start start 2024-09-10T21:26:03.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec14070460 0x7fec1415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14143f90 0x7fec14142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec14144690 0x7fec1413e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fec14168f80 con 0x7fec14144690 2024-09-10T21:26:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fec14168e00 con 0x7fec14070460 2024-09-10T21:26:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fec14169100 con 0x7fec14143f90 2024-09-10T21:26:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec14144690 0x7fec1413e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec12800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec14070460 0x7fec1415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec14144690 0x7fec1413e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53146/0 (socket says 172.21.15.97:53146) 2024-09-10T21:26:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 -- 172.21.15.97:0/1231521283 learned_addr learned my addr 172.21.15.97:0/1231521283 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec09e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14143f90 0x7fec14142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 -- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14143f90 msgr2=0x7fec14142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14143f90 0x7fec14142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 -- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec14070460 msgr2=0x7fec1415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec14070460 0x7fec1415a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 -- 172.21.15.97:0/1231521283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec14136610 con 0x7fec14144690 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec13200640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec14144690 0x7fec1413e250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fec0c01cc60 tx=0x7fec0c005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec0b400640 1 -- 172.21.15.97:0/1231521283 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c004030 con 0x7fec14144690 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec0b400640 1 -- 172.21.15.97:0/1231521283 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec0c0041d0 con 0x7fec14144690 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec141368f0 con 0x7fec14144690 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec0b400640 1 -- 172.21.15.97:0/1231521283 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0a86f0 con 0x7fec14144690 2024-09-10T21:26:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec14136eb0 con 0x7fec14144690 2024-09-10T21:26:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec140b2a80 con 0x7fec14144690 2024-09-10T21:26:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.607+0000 7fec0b400640 1 -- 172.21.15.97:0/1231521283 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fec0c0c3030 con 0x7fec14144690 2024-09-10T21:26:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.611+0000 7fec0b400640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febf00725e0 0x7febf0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.611+0000 7fec0b400640 1 -- 172.21.15.97:0/1231521283 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6331+0+0 (secure 0 0 0) 0x7fec0c12d310 con 0x7fec14144690 2024-09-10T21:26:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.611+0000 7fec12800640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febf00725e0 0x7febf0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:03.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.611+0000 7fec12800640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febf00725e0 0x7febf0074aa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec040015b0 tx=0x7fec04001ae0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:03.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.615+0000 7fec0b400640 1 -- 172.21.15.97:0/1231521283 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fec0c0ac030 con 0x7fec14144690 2024-09-10T21:26:03.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:03.811+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fec14140090 con 0x7fec14144690 2024-09-10T21:26:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.435+0000 7fec0b400640 1 -- 172.21.15.97:0/1231521283 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7fec0c016370 con 0x7fec14144690 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febf00725e0 msgr2=0x7febf0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febf00725e0 0x7febf0074aa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec040015b0 tx=0x7fec04001ae0 comp rx=0 tx=0).stop 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec14144690 msgr2=0x7fec1413e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec14144690 0x7fec1413e250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fec0c01cc60 tx=0x7fec0c005690 comp rx=0 tx=0).stop 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 shutdown_connections 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febf00725e0 0x7febf0074aa0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec14144690 0x7fec1413e250 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fec14143f90 0x7fec14142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:04.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 --2- 172.21.15.97:0/1231521283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec14070460 0x7fec1415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 >> 172.21.15.97:0/1231521283 conn(0x7fec140b1780 msgr2=0x7fec140ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 shutdown_connections 2024-09-10T21:26:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.439+0000 7fec19400640 1 -- 172.21.15.97:0/1231521283 wait complete. 2024-09-10T21:26:04.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:26:04.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 -- 172.21.15.97:0/2268324649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0004030 msgr2=0x7f53a0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:04.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 --2- 172.21.15.97:0/2268324649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0004030 0x7f53a0004810 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f53bc15e650 tx=0x7f53a801f700 comp rx=0 tx=0).stop 2024-09-10T21:26:04.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53bb400640 1 -- 172.21.15.97:0/2268324649 reap_dead start 2024-09-10T21:26:04.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 -- 172.21.15.97:0/2268324649 shutdown_connections 2024-09-10T21:26:04.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 -- 172.21.15.97:0/2268324649 >> 172.21.15.97:0/2268324649 conn(0x7f53bc0b1780 msgr2=0x7f53bc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 -- 172.21.15.97:0/2268324649 shutdown_connections 2024-09-10T21:26:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 -- 172.21.15.97:0/2268324649 wait complete. 2024-09-10T21:26:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 Processor -- start 2024-09-10T21:26:04.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.651+0000 7f53c1600640 1 -- start start 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53a0004030 0x7f53bc158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc15a5c0 0x7f53bc0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53bc0adfb0 0x7f53bc0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc0b49b0 con 0x7f53bc15a5c0 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc0b4830 con 0x7f53bc0adfb0 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f53bc0b2a80 con 0x7f53a0004030 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc15a5c0 0x7f53bc0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc15a5c0 0x7f53bc0ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53176/0 (socket says 172.21.15.97:53176) 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 -- 172.21.15.97:0/3638969958 learned_addr learned my addr 172.21.15.97:0/3638969958 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53baa00640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53a0004030 0x7f53bc158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53bb400640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53bc0adfb0 0x7f53bc0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:04.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 -- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53a0004030 msgr2=0x7f53bc158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53a0004030 0x7f53bc158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 -- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53bc0adfb0 msgr2=0x7f53bc0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53bc0adfb0 0x7f53bc0afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 -- 172.21.15.97:0/3638969958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc0cdb00 con 0x7f53bc15a5c0 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53ba000640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc15a5c0 0x7f53bc0ada70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f53a8026410 tx=0x7f53a8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53b2a00640 1 -- 172.21.15.97:0/3638969958 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80bf070 con 0x7f53bc15a5c0 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53b2a00640 1 -- 172.21.15.97:0/3638969958 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53a8006090 con 0x7f53bc15a5c0 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53b2a00640 1 -- 172.21.15.97:0/3638969958 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80b3070 con 0x7f53bc15a5c0 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc0cdde0 con 0x7f53bc15a5c0 2024-09-10T21:26:04.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc0ce3a0 con 0x7f53bc15a5c0 2024-09-10T21:26:04.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc0701c0 con 0x7f53bc15a5c0 2024-09-10T21:26:04.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.655+0000 7f53b2a00640 1 -- 172.21.15.97:0/3638969958 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f53a80b9070 con 0x7f53bc15a5c0 2024-09-10T21:26:04.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.659+0000 7f53b2a00640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f539c0725e0 0x7f539c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:04.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.659+0000 7f53b2a00640 1 -- 172.21.15.97:0/3638969958 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6331+0+0 (secure 0 0 0) 0x7f53a8130710 con 0x7f53bc15a5c0 2024-09-10T21:26:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.659+0000 7f53baa00640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f539c0725e0 0x7f539c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:04.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.659+0000 7f53baa00640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f539c0725e0 0x7f539c074aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009f30 tx=0x7f53ac009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:04.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.663+0000 7f53b2a00640 1 -- 172.21.15.97:0/3638969958 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f53a80194b0 con 0x7f53bc15a5c0 2024-09-10T21:26:04.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:04.831+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f53bc0ae390 con 0x7f53bc15a5c0 2024-09-10T21:26:05.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.439+0000 7f53b2a00640 1 -- 172.21.15.97:0/3638969958 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) ==== 181+0+0 (secure 0 0 0) 0x7f53a8104020 con 0x7f53bc15a5c0 2024-09-10T21:26:05.441 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:26:05.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.439+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f539c0725e0 msgr2=0x7f539c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:05.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.439+0000 7f53c1600640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f539c0725e0 0x7f539c074aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009f30 tx=0x7f53ac009500 comp rx=0 tx=0).stop 2024-09-10T21:26:05.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.439+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc15a5c0 msgr2=0x7f53bc0ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:05.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.439+0000 7f53c1600640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc15a5c0 0x7f53bc0ada70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f53a8026410 tx=0x7f53a8001d80 comp rx=0 tx=0).stop 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 shutdown_connections 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f539c0725e0 0x7f539c074aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53bc0adfb0 0x7f53bc0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc15a5c0 0x7f53bc0ada70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 --2- 172.21.15.97:0/3638969958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53a0004030 0x7f53bc158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 >> 172.21.15.97:0/3638969958 conn(0x7f53bc0b1780 msgr2=0x7f53bc0c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 shutdown_connections 2024-09-10T21:26:05.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.443+0000 7f53c1600640 1 -- 172.21.15.97:0/3638969958 wait complete. 2024-09-10T21:26:05.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 -- 172.21.15.97:0/3568617290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc0b3280 msgr2=0x7f7acc0d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/3568617290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc0b3280 0x7f7acc0d24c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009900 tx=0x7f7ab401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 -- 172.21.15.97:0/3568617290 shutdown_connections 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/3568617290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc0b3280 0x7f7acc0d24c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/3568617290 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7acc15b9c0 0x7f7acc0d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 -- 172.21.15.97:0/3568617290 >> 172.21.15.97:0/3568617290 conn(0x7f7acc0d3e70 msgr2=0x7f7acc0d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 -- 172.21.15.97:0/3568617290 shutdown_connections 2024-09-10T21:26:05.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.607+0000 7f7ad2a00640 1 -- 172.21.15.97:0/3568617290 wait complete. 2024-09-10T21:26:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 Processor -- start 2024-09-10T21:26:05.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 -- start start 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7acc0b3280 0x7f7acc06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc15b9c0 0x7f7acc1f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7acc15ee70 0x7f7acc152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc16b1c0 con 0x7f7acc15b9c0 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc16b040 con 0x7f7acc15ee70 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc16b340 con 0x7f7acc0b3280 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc15b9c0 0x7f7acc1f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc15b9c0 0x7f7acc1f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53200/0 (socket says 172.21.15.97:53200) 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 -- 172.21.15.97:0/4154093155 learned_addr learned my addr 172.21.15.97:0/4154093155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:05.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7acbe00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7acc0b3280 0x7f7acc06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad0a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7acc15ee70 0x7f7acc152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 -- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7acc0b3280 msgr2=0x7f7acc06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7acc0b3280 0x7f7acc06ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 -- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7acc15ee70 msgr2=0x7f7acc152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7acc15ee70 0x7f7acc152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 -- 172.21.15.97:0/4154093155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc155b40 con 0x7f7acc15b9c0 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad0a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7acc15ee70 0x7f7acc152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7acbe00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7acc0b3280 0x7f7acc06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:05.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac3400640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc15b9c0 0x7f7acc1f79f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009850 tx=0x7f7ab4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac8c00640 1 -- 172.21.15.97:0/4154093155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40b2040 con 0x7f7acc15b9c0 2024-09-10T21:26:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac8c00640 1 -- 172.21.15.97:0/4154093155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ab4005070 con 0x7f7acc15b9c0 2024-09-10T21:26:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ac8c00640 1 -- 172.21.15.97:0/4154093155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40044f0 con 0x7f7acc15b9c0 2024-09-10T21:26:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7acc1f82a0 con 0x7f7acc15b9c0 2024-09-10T21:26:05.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.611+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7acc155e00 con 0x7f7acc15b9c0 2024-09-10T21:26:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.615+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7acc161a30 con 0x7f7acc15b9c0 2024-09-10T21:26:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.615+0000 7f7ac8c00640 1 -- 172.21.15.97:0/4154093155 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ab4005480 con 0x7f7acc15b9c0 2024-09-10T21:26:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.615+0000 7f7ac8c00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7aa80725e0 0x7f7aa8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:05.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.615+0000 7f7ac8c00640 1 -- 172.21.15.97:0/4154093155 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5893+0+0 (secure 0 0 0) 0x7f7ab412ccf0 con 0x7f7acc15b9c0 2024-09-10T21:26:05.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.615+0000 7f7acbe00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7aa80725e0 0x7f7aa8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:05.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.615+0000 7f7acbe00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7aa80725e0 0x7f7aa8074aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7acc15e480 tx=0x7f7abc0095a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:05.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.619+0000 7f7ac8c00640 1 -- 172.21.15.97:0/4154093155 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7acc1f82a0 con 0x7f7acc15b9c0 2024-09-10T21:26:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:05.783+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7acc08db10 con 0x7f7acc15b9c0 2024-09-10T21:26:06.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ac8c00640 1 -- 172.21.15.97:0/4154093155 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) ==== 169+0+0 (secure 0 0 0) 0x7f7ab40164d0 con 0x7f7acc15b9c0 2024-09-10T21:26:06.449 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7aa80725e0 msgr2=0x7f7aa8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7aa80725e0 0x7f7aa8074aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7acc15e480 tx=0x7f7abc0095a0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc15b9c0 msgr2=0x7f7acc1f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc15b9c0 0x7f7acc1f79f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009850 tx=0x7f7ab4004040 comp rx=0 tx=0).stop 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 shutdown_connections 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7aa80725e0 0x7f7aa8074aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7acc15ee70 0x7f7acc152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7acc15b9c0 0x7f7acc1f79f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 --2- 172.21.15.97:0/4154093155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7acc0b3280 0x7f7acc06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 >> 172.21.15.97:0/4154093155 conn(0x7f7acc0d3e70 msgr2=0x7f7acc0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 shutdown_connections 2024-09-10T21:26:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.447+0000 7f7ad2a00640 1 -- 172.21.15.97:0/4154093155 wait complete. 2024-09-10T21:26:06.475 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:26:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- 172.21.15.97:0/324217965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 msgr2=0x7fd1400b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 --2- 172.21.15.97:0/324217965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400b2a80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd128009840 tx=0x7fd12801c820 comp rx=0 tx=0).stop 2024-09-10T21:26:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- 172.21.15.97:0/324217965 shutdown_connections 2024-09-10T21:26:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 --2- 172.21.15.97:0/324217965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400b2a80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- 172.21.15.97:0/324217965 >> 172.21.15.97:0/324217965 conn(0x7fd1400b1780 msgr2=0x7fd1400b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- 172.21.15.97:0/324217965 shutdown_connections 2024-09-10T21:26:06.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- 172.21.15.97:0/324217965 wait complete. 2024-09-10T21:26:06.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 Processor -- start 2024-09-10T21:26:06.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- start start 2024-09-10T21:26:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd14006fd50 0x7fd140159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd140143f90 0x7fd140136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd140169410 con 0x7fd140144690 2024-09-10T21:26:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd140169290 con 0x7fd140143f90 2024-09-10T21:26:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.615+0000 7fd145800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd140169590 con 0x7fd14006fd50 2024-09-10T21:26:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53208/0 (socket says 172.21.15.97:53208) 2024-09-10T21:26:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 -- 172.21.15.97:0/2372529428 learned_addr learned my addr 172.21.15.97:0/2372529428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd136000640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd140143f90 0x7fd140136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13ea00640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd14006fd50 0x7fd140159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 -- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd14006fd50 msgr2=0x7fd140159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd14006fd50 0x7fd140159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 -- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd140143f90 msgr2=0x7fd140136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd140143f90 0x7fd140136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 -- 172.21.15.97:0/2372529428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14015a4d0 con 0x7fd140144690 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd13f400640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400730a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c3a0 tx=0x7fd13800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd137400640 1 -- 172.21.15.97:0/2372529428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138017070 con 0x7fd140144690 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd137400640 1 -- 172.21.15.97:0/2372529428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd138003600 con 0x7fd140144690 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd140138b20 con 0x7fd140144690 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd137400640 1 -- 172.21.15.97:0/2372529428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138003a50 con 0x7fd140144690 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1401390e0 con 0x7fd140144690 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1400b2a80 con 0x7fd140144690 2024-09-10T21:26:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.619+0000 7fd137400640 1 -- 172.21.15.97:0/2372529428 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd138003bf0 con 0x7fd140144690 2024-09-10T21:26:06.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.623+0000 7fd137400640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd11c0725e0 0x7fd11c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.623+0000 7fd137400640 1 -- 172.21.15.97:0/2372529428 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fd138092710 con 0x7fd140144690 2024-09-10T21:26:06.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.623+0000 7fd13ea00640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd11c0725e0 0x7fd11c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.623+0000 7fd13ea00640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd11c0725e0 0x7fd11c074aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd130005680 tx=0x7fd130003dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:06.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.627+0000 7fd137400640 1 -- 172.21.15.97:0/2372529428 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd14015a4d0 con 0x7fd140144690 2024-09-10T21:26:06.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.791+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fd14006f7e0 con 0x7fd140144690 2024-09-10T21:26:06.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd137400640 1 -- 172.21.15.97:0/2372529428 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v29) ==== 71+0+0 (secure 0 0 0) 0x7fd13805d820 con 0x7fd140144690 2024-09-10T21:26:06.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd11c0725e0 msgr2=0x7fd11c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd11c0725e0 0x7fd11c074aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd130005680 tx=0x7fd130003dd0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 msgr2=0x7fd1400730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400730a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c3a0 tx=0x7fd13800c860 comp rx=0 tx=0).stop 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 shutdown_connections 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd11c0725e0 0x7fd11c074aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd140144690 0x7fd1400730a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd140143f90 0x7fd140136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 --2- 172.21.15.97:0/2372529428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd14006fd50 0x7fd140159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 >> 172.21.15.97:0/2372529428 conn(0x7fd1400b1780 msgr2=0x7fd1400cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 shutdown_connections 2024-09-10T21:26:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.799+0000 7fd145800640 1 -- 172.21.15.97:0/2372529428 wait complete. 2024-09-10T21:26:06.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-10T21:26:06.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 -- 172.21.15.97:0/4188409763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe300b1590 msgr2=0x7efe300d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 --2- 172.21.15.97:0/4188409763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe300b1590 0x7efe300d0370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efe3006f650 tx=0x7efe2801d550 comp rx=0 tx=0).stop 2024-09-10T21:26:06.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 -- 172.21.15.97:0/4188409763 shutdown_connections 2024-09-10T21:26:06.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 --2- 172.21.15.97:0/4188409763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe300b1590 0x7efe300d0370 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 --2- 172.21.15.97:0/4188409763 >> v2:172.21.15.194:3300/0 conn(0x7efe3006dc50 0x7efe300cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 -- 172.21.15.97:0/4188409763 >> 172.21.15.97:0/4188409763 conn(0x7efe300d1c30 msgr2=0x7efe300d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 -- 172.21.15.97:0/4188409763 shutdown_connections 2024-09-10T21:26:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.971+0000 7efe36200640 1 -- 172.21.15.97:0/4188409763 wait complete. 2024-09-10T21:26:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 Processor -- start 2024-09-10T21:26:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 -- start start 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe300b1590 0x7efe3014ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe30161600 0x7efe300b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe301fac00 0x7efe3014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe3016ad20 con 0x7efe301fac00 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efe300b07c0 con 0x7efe30161600 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efe3016aea0 con 0x7efe300b1590 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe301fac00 0x7efe3014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe301fac00 0x7efe3014f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53236/0 (socket says 172.21.15.97:53236) 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 -- 172.21.15.97:0/2458394654 learned_addr learned my addr 172.21.15.97:0/2458394654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2f400640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe300b1590 0x7efe3014ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe27e00640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe30161600 0x7efe300b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 -- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe300b1590 msgr2=0x7efe3014ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe300b1590 0x7efe3014ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 -- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe30161600 msgr2=0x7efe300b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe30161600 0x7efe300b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 -- 172.21.15.97:0/2458394654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30150590 con 0x7efe301fac00 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe2fe00640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe301fac00 0x7efe3014f3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe28009b10 tx=0x7efe28009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe27400640 1 -- 172.21.15.97:0/2458394654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280bb070 con 0x7efe301fac00 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe27400640 1 -- 172.21.15.97:0/2458394654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe28004d40 con 0x7efe301fac00 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe30150870 con 0x7efe301fac00 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe27400640 1 -- 172.21.15.97:0/2458394654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280b2040 con 0x7efe301fac00 2024-09-10T21:26:06.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe30150e30 con 0x7efe301fac00 2024-09-10T21:26:06.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.975+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300d0a60 con 0x7efe301fac00 2024-09-10T21:26:06.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.979+0000 7efe27400640 1 -- 172.21.15.97:0/2458394654 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe280b5030 con 0x7efe301fac00 2024-09-10T21:26:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.979+0000 7efe27400640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe0c0725e0 0x7efe0c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.979+0000 7efe27400640 1 -- 172.21.15.97:0/2458394654 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7efe2812cfd0 con 0x7efe301fac00 2024-09-10T21:26:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.979+0000 7efe2f400640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe0c0725e0 0x7efe0c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:06.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.979+0000 7efe2f400640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe0c0725e0 0x7efe0c074aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe200015f0 tx=0x7efe20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:06.983+0000 7efe27400640 1 -- 172.21.15.97:0/2458394654 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe28016770 con 0x7efe301fac00 2024-09-10T21:26:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.207+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7efe3015b520 con 0x7efe301fac00 2024-09-10T21:26:07.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.447+0000 7efe27400640 1 -- 172.21.15.97:0/2458394654 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v117) ==== 134+0+0 (secure 0 0 0) 0x7efe280f8730 con 0x7efe301fac00 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe0c0725e0 msgr2=0x7efe0c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe0c0725e0 0x7efe0c074aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe200015f0 tx=0x7efe20009040 comp rx=0 tx=0).stop 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe301fac00 msgr2=0x7efe3014f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe301fac00 0x7efe3014f3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe28009b10 tx=0x7efe28009240 comp rx=0 tx=0).stop 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 shutdown_connections 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe0c0725e0 0x7efe0c074aa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe301fac00 0x7efe3014f3e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe30161600 0x7efe300b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 --2- 172.21.15.97:0/2458394654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe300b1590 0x7efe3014ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 >> 172.21.15.97:0/2458394654 conn(0x7efe300d1c30 msgr2=0x7efe300b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 shutdown_connections 2024-09-10T21:26:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.451+0000 7efe36200640 1 -- 172.21.15.97:0/2458394654 wait complete. 2024-09-10T21:26:07.473 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2024-09-10T21:26:07.474 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2024-09-10T21:26:07.474 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2024-09-10T21:26:07.474 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 -- 172.21.15.97:0/748032147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 msgr2=0x7f21b0074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 --2- 172.21.15.97:0/748032147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b0074380 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009c80 tx=0x7f21a001cd70 comp rx=0 tx=0).stop 2024-09-10T21:26:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 -- 172.21.15.97:0/748032147 shutdown_connections 2024-09-10T21:26:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 --2- 172.21.15.97:0/748032147 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21b00b0110 0x7f21b00cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 --2- 172.21.15.97:0/748032147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b0074380 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 -- 172.21.15.97:0/748032147 >> 172.21.15.97:0/748032147 conn(0x7f21b00b28c0 msgr2=0x7f21b00b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 -- 172.21.15.97:0/748032147 shutdown_connections 2024-09-10T21:26:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.611+0000 7f21b7400640 1 -- 172.21.15.97:0/748032147 wait complete. 2024-09-10T21:26:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 Processor -- start 2024-09-10T21:26:07.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 -- start start 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21b00b0110 0x7f21b014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21b00cfad0 0x7f21b0154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b006f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0168bc0 con 0x7f21b015bdd0 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0074a00 con 0x7f21b00b0110 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f21b0168d40 con 0x7f21b00cfad0 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b006f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33428/0 (socket says 172.21.15.97:33428) 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 -- 172.21.15.97:0/3486027973 learned_addr learned my addr 172.21.15.97:0/3486027973 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b4a00640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21b00b0110 0x7f21b014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21afe00640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21b00cfad0 0x7f21b0154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 -- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21b00cfad0 msgr2=0x7f21b0154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21b00cfad0 0x7f21b0154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 -- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21b00b0110 msgr2=0x7f21b014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21b00b0110 0x7f21b014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 -- 172.21.15.97:0/3486027973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b0153da0 con 0x7f21b015bdd0 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b5400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b006f620 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c970 tx=0x7f21a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:07.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21acc00640 1 -- 172.21.15.97:0/3486027973 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8007bb0 con 0x7f21b015bdd0 2024-09-10T21:26:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21acc00640 1 -- 172.21.15.97:0/3486027973 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21a800f450 con 0x7f21b015bdd0 2024-09-10T21:26:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b0154080 con 0x7f21b015bdd0 2024-09-10T21:26:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21acc00640 1 -- 172.21.15.97:0/3486027973 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8007e60 con 0x7f21b015bdd0 2024-09-10T21:26:07.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.615+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b0154640 con 0x7f21b015bdd0 2024-09-10T21:26:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.619+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b0154810 con 0x7f21b015bdd0 2024-09-10T21:26:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.619+0000 7f21acc00640 1 -- 172.21.15.97:0/3486027973 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f21a800f8a0 con 0x7f21b015bdd0 2024-09-10T21:26:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.619+0000 7f21acc00640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21900725e0 0x7f2190074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.619+0000 7f21acc00640 1 -- 172.21.15.97:0/3486027973 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f21a8093910 con 0x7f21b015bdd0 2024-09-10T21:26:07.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.619+0000 7f21b4a00640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21900725e0 0x7f2190074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.619+0000 7f21b4a00640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21900725e0 0x7f2190074aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f21a00af000 tx=0x7f21a001b5b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:07.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.623+0000 7f21acc00640 1 -- 172.21.15.97:0/3486027973 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f21a8014030 con 0x7f21b015bdd0 2024-09-10T21:26:07.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.787+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21b00cff20 con 0x7f21b015bdd0 2024-09-10T21:26:07.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21acc00640 1 -- 172.21.15.97:0/3486027973 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+13455 (secure 0 0 0) 0x7f21a805f100 con 0x7f21b015bdd0 2024-09-10T21:26:07.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:07.792 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":46,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:06.441040+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/95042702":"2024-09-11T21:26:03.430572+0000","172.21.15.97:6835/2127253870":"2024-09-11T21:25:19.147709+0000","172.21.15.97:6832/95042702":"2024-09-11T21:26:03.430572+0000","172.21.15.97:6832/2127253870":"2024-09-11T21:25:19.147709+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21900725e0 msgr2=0x7f2190074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21900725e0 0x7f2190074aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f21a00af000 tx=0x7f21a001b5b0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 msgr2=0x7f21b006f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b006f620 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c970 tx=0x7f21a800ce30 comp rx=0 tx=0).stop 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 shutdown_connections 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21900725e0 0x7f2190074aa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21b015bdd0 0x7f21b006f620 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21b00cfad0 0x7f21b0154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 --2- 172.21.15.97:0/3486027973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21b00b0110 0x7f21b014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 >> 172.21.15.97:0/3486027973 conn(0x7f21b00b28c0 msgr2=0x7f21b00b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 shutdown_connections 2024-09-10T21:26:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.791+0000 7f21b7400640 1 -- 172.21.15.97:0/3486027973 wait complete. 2024-09-10T21:26:07.814 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:24:02.092708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:26:07.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 -- 172.21.15.97:0/3932279226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00cd6e0 msgr2=0x7f59d00af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 --2- 172.21.15.97:0/3932279226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00cd6e0 0x7f59d00af4d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009c80 tx=0x7f59c801cda0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 -- 172.21.15.97:0/3932279226 shutdown_connections 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 --2- 172.21.15.97:0/3932279226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00cd6e0 0x7f59d00af4d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 --2- 172.21.15.97:0/3932279226 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59d00ce640 0x7f59d00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 -- 172.21.15.97:0/3932279226 >> 172.21.15.97:0/3932279226 conn(0x7f59d00d1c30 msgr2=0x7f59d00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 -- 172.21.15.97:0/3932279226 shutdown_connections 2024-09-10T21:26:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.955+0000 7f59d5600640 1 -- 172.21.15.97:0/3932279226 wait complete. 2024-09-10T21:26:07.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 Processor -- start 2024-09-10T21:26:07.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 -- start start 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00b0d50 0x7f59d014ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59d00cd6e0 0x7f59d01549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d00ce640 0x7f59d0155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0168cc0 con 0x7f59d00ce640 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0168b40 con 0x7f59d00b0d50 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f59d0168e40 con 0x7f59d00cd6e0 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d00ce640 0x7f59d0155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d00ce640 0x7f59d0155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33444/0 (socket says 172.21.15.97:33444) 2024-09-10T21:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 -- 172.21.15.97:0/501069418 learned_addr learned my addr 172.21.15.97:0/501069418 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cea00640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00b0d50 0x7f59d014ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59ce000640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59d00cd6e0 0x7f59d01549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 -- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59d00cd6e0 msgr2=0x7f59d01549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59d00cd6e0 0x7f59d01549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 -- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00b0d50 msgr2=0x7f59d014ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00b0d50 0x7f59d014ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 -- 172.21.15.97:0/501069418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d006dc50 con 0x7f59d00ce640 2024-09-10T21:26:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59cf400640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d00ce640 0x7f59d0155320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f59c801b450 tx=0x7f59c8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59c6a00640 1 -- 172.21.15.97:0/501069418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80bb070 con 0x7f59d00ce640 2024-09-10T21:26:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59c6a00640 1 -- 172.21.15.97:0/501069418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59c8004d40 con 0x7f59d00ce640 2024-09-10T21:26:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59c6a00640 1 -- 172.21.15.97:0/501069418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b2040 con 0x7f59d00ce640 2024-09-10T21:26:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d006df30 con 0x7f59d00ce640 2024-09-10T21:26:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d006e4f0 con 0x7f59d00ce640 2024-09-10T21:26:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.959+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d00cea20 con 0x7f59d00ce640 2024-09-10T21:26:07.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.963+0000 7f59c6a00640 1 -- 172.21.15.97:0/501069418 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f59c80b5030 con 0x7f59d00ce640 2024-09-10T21:26:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.963+0000 7f59c6a00640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59b0072610 0x7f59b0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.963+0000 7f59c6a00640 1 -- 172.21.15.97:0/501069418 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f59c812cbe0 con 0x7f59d00ce640 2024-09-10T21:26:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.963+0000 7f59cea00640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59b0072610 0x7f59b0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.963+0000 7f59cea00640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59b0072610 0x7f59b0074ad0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009b40 tx=0x7f59c00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:07.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:07.967+0000 7f59c6a00640 1 -- 172.21.15.97:0/501069418 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f59c8016990 con 0x7f59d00ce640 2024-09-10T21:26:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.131+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59d00b34d0 con 0x7f59d00ce640 2024-09-10T21:26:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.131+0000 7f59c6a00640 1 -- 172.21.15.97:0/501069418 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f59c80f7d80 con 0x7f59d00ce640 2024-09-10T21:26:08.135 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-09-10T21:26:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59b0072610 msgr2=0x7f59b0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59b0072610 0x7f59b0074ad0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009b40 tx=0x7f59c00091e0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d00ce640 msgr2=0x7f59d0155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d00ce640 0x7f59d0155320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f59c801b450 tx=0x7f59c8003f60 comp rx=0 tx=0).stop 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 shutdown_connections 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f59b0072610 0x7f59b0074ad0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d00ce640 0x7f59d0155320 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59d00cd6e0 0x7f59d01549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 --2- 172.21.15.97:0/501069418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59d00b0d50 0x7f59d014ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 >> 172.21.15.97:0/501069418 conn(0x7f59d00d1c30 msgr2=0x7f59d00b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 shutdown_connections 2024-09-10T21:26:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.135+0000 7f59d5600640 1 -- 172.21.15.97:0/501069418 wait complete. 2024-09-10T21:26:08.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-10T21:26:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- 172.21.15.97:0/576514922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 msgr2=0x7f20080b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 --2- 172.21.15.97:0/576514922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f20080b37b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800a980 tx=0x7f1ff8020340 comp rx=0 tx=0).stop 2024-09-10T21:26:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- 172.21.15.97:0/576514922 shutdown_connections 2024-09-10T21:26:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 --2- 172.21.15.97:0/576514922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f20080b37b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- 172.21.15.97:0/576514922 >> 172.21.15.97:0/576514922 conn(0x7f20080b1780 msgr2=0x7f20080b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- 172.21.15.97:0/576514922 shutdown_connections 2024-09-10T21:26:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- 172.21.15.97:0/576514922 wait complete. 2024-09-10T21:26:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 Processor -- start 2024-09-10T21:26:08.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- start start 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20080b3cf0 0x7f20081f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20080ceee0 0x7f2008153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f200814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f200815b290 con 0x7f20081511f0 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f200815b110 con 0x7f20080ceee0 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f200da00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f20080cf350 con 0x7f20080b3cf0 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f2007400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f200814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f2007400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f200814df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33464/0 (socket says 172.21.15.97:33464) 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f2007400640 1 -- 172.21.15.97:0/3457508467 learned_addr learned my addr 172.21.15.97:0/3457508467 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f1ffe000640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20080ceee0 0x7f2008153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:08.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.295+0000 7f2006a00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20080b3cf0 0x7f20081f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f2007400640 1 -- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20080b3cf0 msgr2=0x7f20081f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f2007400640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20080b3cf0 0x7f20081f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f2007400640 1 -- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20080ceee0 msgr2=0x7f2008153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f2007400640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20080ceee0 0x7f2008153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f2007400640 1 -- 172.21.15.97:0/3457508467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20081f8c20 con 0x7f20081511f0 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f2007400640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f200814df90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f200000c780 tx=0x7f200000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f1fff400640 1 -- 172.21.15.97:0/3457508467 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004b70 con 0x7f20081511f0 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f1fff400640 1 -- 172.21.15.97:0/3457508467 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2000004e00 con 0x7f20081511f0 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f1fff400640 1 -- 172.21.15.97:0/3457508467 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000aa90 con 0x7f20081511f0 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20081e3fc0 con 0x7f20081511f0 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081f7ee0 con 0x7f20081511f0 2024-09-10T21:26:08.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200815a090 con 0x7f20081511f0 2024-09-10T21:26:08.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f1fff400640 1 -- 172.21.15.97:0/3457508467 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f200000ac30 con 0x7f20081511f0 2024-09-10T21:26:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f1fff400640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fe40725e0 0x7f1fe4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.299+0000 7f1fff400640 1 -- 172.21.15.97:0/3457508467 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f2000093340 con 0x7f20081511f0 2024-09-10T21:26:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.303+0000 7f2006a00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fe40725e0 0x7f1fe4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:08.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.303+0000 7f2006a00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fe40725e0 0x7f1fe4074aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80b2000 tx=0x7f1ff801eb80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:08.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.303+0000 7f1fff400640 1 -- 172.21.15.97:0/3457508467 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2000020070 con 0x7f20081511f0 2024-09-10T21:26:08.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:08.531+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7f2008159db0 con 0x7f20081511f0 2024-09-10T21:26:08.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:08.749 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:09.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.455+0000 7f1fff400640 1 -- 172.21.15.97:0/3457508467 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v119) ==== 144+0+0 (secure 0 0 0) 0x7f200005ebd0 con 0x7f20081511f0 2024-09-10T21:26:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.455+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fe40725e0 msgr2=0x7f1fe4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.455+0000 7f200da00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fe40725e0 0x7f1fe4074aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80b2000 tx=0x7f1ff801eb80 comp rx=0 tx=0).stop 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 msgr2=0x7f200814df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f200814df90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f200000c780 tx=0x7f200000cc40 comp rx=0 tx=0).stop 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 shutdown_connections 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fe40725e0 0x7f1fe4074aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20081511f0 0x7f200814df90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20080ceee0 0x7f2008153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 --2- 172.21.15.97:0/3457508467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20080b3cf0 0x7f20081f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 >> 172.21.15.97:0/3457508467 conn(0x7f20080b1780 msgr2=0x7f200815bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 shutdown_connections 2024-09-10T21:26:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.459+0000 7f200da00640 1 -- 172.21.15.97:0/3457508467 wait complete. 2024-09-10T21:26:09.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:26:09.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- 172.21.15.97:0/3663846161 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f229415f120 msgr2=0x7f22940af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 --2- 172.21.15.97:0/3663846161 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f229415f120 0x7f22940af520 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2284009840 tx=0x7f228401c7f0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- 172.21.15.97:0/3663846161 shutdown_connections 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 --2- 172.21.15.97:0/3663846161 >> v2:172.21.15.194:3300/0 conn(0x7f2294160220 0x7f22940ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 --2- 172.21.15.97:0/3663846161 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f229415f120 0x7f22940af520 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- 172.21.15.97:0/3663846161 >> 172.21.15.97:0/3663846161 conn(0x7f22940b3c70 msgr2=0x7f22940b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- 172.21.15.97:0/3663846161 shutdown_connections 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- 172.21.15.97:0/3663846161 wait complete. 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 Processor -- start 2024-09-10T21:26:09.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- start start 2024-09-10T21:26:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229406cb40 0x7f229414a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229415f120 0x7f2294099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2294160220 0x7f2294157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22940cea50 con 0x7f229406cb40 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22940d0a90 con 0x7f229415f120 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2299800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f229406cf70 con 0x7f2294160220 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2292a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229406cb40 0x7f229414a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2292a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229406cb40 0x7f229414a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33474/0 (socket says 172.21.15.97:33474) 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2292a00640 1 -- 172.21.15.97:0/1907344977 learned_addr learned my addr 172.21.15.97:0/1907344977 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f228a000640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229415f120 0x7f2294099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.627+0000 7f2293400640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2294160220 0x7f2294157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2292a00640 1 -- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2294160220 msgr2=0x7f2294157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2292a00640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2294160220 0x7f2294157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2292a00640 1 -- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229415f120 msgr2=0x7f2294099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2292a00640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229415f120 0x7f2294099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2292a00640 1 -- 172.21.15.97:0/1907344977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2294099ec0 con 0x7f229406cb40 2024-09-10T21:26:09.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2292a00640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229406cb40 0x7f229414a6b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2284009790 tx=0x7f22840a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:09.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f228b400640 1 -- 172.21.15.97:0/1907344977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22840b2040 con 0x7f229406cb40 2024-09-10T21:26:09.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f228b400640 1 -- 172.21.15.97:0/1907344977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2284004430 con 0x7f229406cb40 2024-09-10T21:26:09.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f228b400640 1 -- 172.21.15.97:0/1907344977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284005280 con 0x7f229406cb40 2024-09-10T21:26:09.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229414ad70 con 0x7f229406cb40 2024-09-10T21:26:09.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229414b330 con 0x7f229406cb40 2024-09-10T21:26:09.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22940ce320 con 0x7f229406cb40 2024-09-10T21:26:09.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f228b400640 1 -- 172.21.15.97:0/1907344977 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f22840b5070 con 0x7f229406cb40 2024-09-10T21:26:09.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f228b400640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f22700725e0 0x7f2270074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.631+0000 7f228b400640 1 -- 172.21.15.97:0/1907344977 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f228412c790 con 0x7f229406cb40 2024-09-10T21:26:09.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.635+0000 7f228a000640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f22700725e0 0x7f2270074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.635+0000 7f228a000640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f22700725e0 0x7f2270074aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f227c009a40 tx=0x7f227c010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:09.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.635+0000 7f228b400640 1 -- 172.21.15.97:0/1907344977 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2284016410 con 0x7f229406cb40 2024-09-10T21:26:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.803+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f22940d1080 con 0x7f229406cb40 2024-09-10T21:26:09.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.803+0000 7f228b400640 1 -- 172.21.15.97:0/1907344977 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v29) ==== 59+0+1998 (secure 0 0 0) 0x7f22840f7940 con 0x7f229406cb40 2024-09-10T21:26:09.806 INFO:teuthology.orchestra.run.smithi097.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:26:09.806 INFO:teuthology.orchestra.run.smithi097.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:26:09.806 INFO:teuthology.orchestra.run.smithi097.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:26:09.806 INFO:teuthology.orchestra.run.smithi097.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3032.483965 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3656.533946 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3351.408335 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3363.485497 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 8164.284498 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4391.956543 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4391.204852 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4799.171341 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds 20/20 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_ms 2/2 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_frag true 2024-09-10T21:26:09.807 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_verify_scatter true 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced client_mount_timeout 600 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_client 20/20 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_ms 1/1 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:26:09.808 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:26:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.803+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f22700725e0 msgr2=0x7f2270074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.803+0000 7f2299800640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f22700725e0 0x7f2270074aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f227c009a40 tx=0x7f227c010040 comp rx=0 tx=0).stop 2024-09-10T21:26:09.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.803+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229406cb40 msgr2=0x7f229414a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.803+0000 7f2299800640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229406cb40 0x7f229414a6b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2284009790 tx=0x7f22840a5750 comp rx=0 tx=0).stop 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 shutdown_connections 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f22700725e0 0x7f2270074aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2294160220 0x7f2294157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229415f120 0x7f2294099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 --2- 172.21.15.97:0/1907344977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229406cb40 0x7f229414a6b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 >> 172.21.15.97:0/1907344977 conn(0x7f22940b3c70 msgr2=0x7f22940cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 shutdown_connections 2024-09-10T21:26:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.807+0000 7f2299800640 1 -- 172.21.15.97:0/1907344977 wait complete. 2024-09-10T21:26:09.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:26:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- 172.21.15.97:0/1328728288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60141d20 msgr2=0x7f5e6015f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 --2- 172.21.15.97:0/1328728288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60141d20 0x7f5e6015f6d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009840 tx=0x7f5e4801c820 comp rx=0 tx=0).stop 2024-09-10T21:26:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- 172.21.15.97:0/1328728288 shutdown_connections 2024-09-10T21:26:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 --2- 172.21.15.97:0/1328728288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e60141d20 0x7f5e6015f6d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- 172.21.15.97:0/1328728288 >> 172.21.15.97:0/1328728288 conn(0x7f5e600b1780 msgr2=0x7f5e600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- 172.21.15.97:0/1328728288 shutdown_connections 2024-09-10T21:26:09.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- 172.21.15.97:0/1328728288 wait complete. 2024-09-10T21:26:09.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 Processor -- start 2024-09-10T21:26:09.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- start start 2024-09-10T21:26:09.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e600ce5f0 0x7f5e6013b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e60141d20 0x7f5e6013b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e6015fc10 0x7f5e600b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60169cb0 con 0x7f5e600ce5f0 2024-09-10T21:26:09.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60169b30 con 0x7f5e6015fc10 2024-09-10T21:26:09.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e65200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5e60169e30 con 0x7f5e60141d20 2024-09-10T21:26:09.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e5de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e600ce5f0 0x7f5e6013b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e5de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e600ce5f0 0x7f5e6013b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33488/0 (socket says 172.21.15.97:33488) 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e5de00640 1 -- 172.21.15.97:0/1878313008 learned_addr learned my addr 172.21.15.97:0/1878313008 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e5e800640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e6015fc10 0x7f5e600b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e55400640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e60141d20 0x7f5e6013b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e5de00640 1 -- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e60141d20 msgr2=0x7f5e6013b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e5de00640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e60141d20 0x7f5e6013b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.971+0000 7f5e5de00640 1 -- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e6015fc10 msgr2=0x7f5e600b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e5de00640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e6015fc10 0x7f5e600b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e5de00640 1 -- 172.21.15.97:0/1878313008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e60142f70 con 0x7f5e600ce5f0 2024-09-10T21:26:09.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e5de00640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e600ce5f0 0x7f5e6013b710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c7f0 tx=0x7f5e5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:09.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e56a00640 1 -- 172.21.15.97:0/1878313008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50016020 con 0x7f5e600ce5f0 2024-09-10T21:26:09.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e56a00640 1 -- 172.21.15.97:0/1878313008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e5000ab40 con 0x7f5e600ce5f0 2024-09-10T21:26:09.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e56a00640 1 -- 172.21.15.97:0/1878313008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50012420 con 0x7f5e600ce5f0 2024-09-10T21:26:09.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e60143230 con 0x7f5e600ce5f0 2024-09-10T21:26:09.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e600b3460 con 0x7f5e600ce5f0 2024-09-10T21:26:09.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6015f6d0 con 0x7f5e600ce5f0 2024-09-10T21:26:09.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.975+0000 7f5e56a00640 1 -- 172.21.15.97:0/1878313008 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5e5001b400 con 0x7f5e600ce5f0 2024-09-10T21:26:09.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.979+0000 7f5e56a00640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e3c0725e0 0x7f5e3c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:09.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.979+0000 7f5e56a00640 1 -- 172.21.15.97:0/1878313008 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f5e50093390 con 0x7f5e600ce5f0 2024-09-10T21:26:09.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.979+0000 7f5e55400640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e3c0725e0 0x7f5e3c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:09.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.979+0000 7f5e55400640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e3c0725e0 0x7f5e3c074aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e480af000 tx=0x7f5e4801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:09.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:09.983+0000 7f5e56a00640 1 -- 172.21.15.97:0/1878313008 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5e50020030 con 0x7f5e600ce5f0 2024-09-10T21:26:10.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.147+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5e60141360 con 0x7f5e600ce5f0 2024-09-10T21:26:10.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e56a00640 1 -- 172.21.15.97:0/1878313008 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f5e5005e530 con 0x7f5e600ce5f0 2024-09-10T21:26:10.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e3c0725e0 msgr2=0x7f5e3c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e3c0725e0 0x7f5e3c074aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e480af000 tx=0x7f5e4801b060 comp rx=0 tx=0).stop 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e600ce5f0 msgr2=0x7f5e6013b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e600ce5f0 0x7f5e6013b710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c7f0 tx=0x7f5e5000ccb0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 shutdown_connections 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e3c0725e0 0x7f5e3c074aa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e6015fc10 0x7f5e600b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e60141d20 0x7f5e6013b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 --2- 172.21.15.97:0/1878313008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e600ce5f0 0x7f5e6013b710 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 >> 172.21.15.97:0/1878313008 conn(0x7f5e600b1780 msgr2=0x7f5e600b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 shutdown_connections 2024-09-10T21:26:10.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.151+0000 7f5e65200640 1 -- 172.21.15.97:0/1878313008 wait complete. 2024-09-10T21:26:10.175 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:26:10.175 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:10.283 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:10.367 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:26:10.368 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:10.368 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:10.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:26:10.376 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:10.376 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:10.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:10.386 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:10.386 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:10.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:26:10.428 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:10.428 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:10.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:10.437 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:26:10.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:10.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 -- 172.21.15.97:0/2997741974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 msgr2=0x7fc0d00aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 --2- 172.21.15.97:0/2997741974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00aef70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00748b0 tx=0x7fc0c001cd80 comp rx=0 tx=0).stop 2024-09-10T21:26:10.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 -- 172.21.15.97:0/2997741974 shutdown_connections 2024-09-10T21:26:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 --2- 172.21.15.97:0/2997741974 >> v2:172.21.15.194:3300/0 conn(0x7fc0d00702d0 0x7fc0d00b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 --2- 172.21.15.97:0/2997741974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00aef70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 -- 172.21.15.97:0/2997741974 >> 172.21.15.97:0/2997741974 conn(0x7fc0d006cb40 msgr2=0x7fc0d006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 -- 172.21.15.97:0/2997741974 shutdown_connections 2024-09-10T21:26:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.607+0000 7fc0d5800640 1 -- 172.21.15.97:0/2997741974 wait complete. 2024-09-10T21:26:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 Processor -- start 2024-09-10T21:26:10.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 -- start start 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0d006e220 0x7fc0d0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0d00702d0 0x7fc0d00ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0167b70 con 0x7fc0d015a5e0 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01679f0 con 0x7fc0d00702d0 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d0167cf0 con 0x7fc0d006e220 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33508/0 (socket says 172.21.15.97:33508) 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 -- 172.21.15.97:0/3458810775 learned_addr learned my addr 172.21.15.97:0/3458810775 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0cfe00640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0d006e220 0x7fc0d0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0c7400640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0d00702d0 0x7fc0d00ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 -- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0d006e220 msgr2=0x7fc0d0153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0d006e220 0x7fc0d0153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 -- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0d00702d0 msgr2=0x7fc0d00ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0d00702d0 0x7fc0d00ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 -- 172.21.15.97:0/3458810775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d01fafd0 con 0x7fc0d015a5e0 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0c7400640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0d00702d0 0x7fc0d00ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d4e00640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00cd270 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e9a0 tx=0x7fc0c800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0ccc00640 1 -- 172.21.15.97:0/3458810775 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c800ccc0 con 0x7fc0d015a5e0 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0ccc00640 1 -- 172.21.15.97:0/3458810775 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0c800ce60 con 0x7fc0d015a5e0 2024-09-10T21:26:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0ccc00640 1 -- 172.21.15.97:0/3458810775 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8005280 con 0x7fc0d015a5e0 2024-09-10T21:26:10.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d01fb2b0 con 0x7fc0d015a5e0 2024-09-10T21:26:10.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.611+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d01fb870 con 0x7fc0d015a5e0 2024-09-10T21:26:10.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.615+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d00b3820 con 0x7fc0d015a5e0 2024-09-10T21:26:10.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.615+0000 7fc0ccc00640 1 -- 172.21.15.97:0/3458810775 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc0c8005420 con 0x7fc0d015a5e0 2024-09-10T21:26:10.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.615+0000 7fc0ccc00640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0ac0725e0 0x7fc0ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.615+0000 7fc0ccc00640 1 -- 172.21.15.97:0/3458810775 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fc0c8093bd0 con 0x7fc0d015a5e0 2024-09-10T21:26:10.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.615+0000 7fc0cfe00640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0ac0725e0 0x7fc0ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.619+0000 7fc0cfe00640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0ac0725e0 0x7fc0ac074aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00af000 tx=0x7fc0c001b5c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:10.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.619+0000 7fc0ccc00640 1 -- 172.21.15.97:0/3458810775 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc0c805f330 con 0x7fc0d015a5e0 2024-09-10T21:26:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0d0159720 con 0x7fc0d015a5e0 2024-09-10T21:26:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0ccc00640 1 -- 172.21.15.97:0/3458810775 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3911 (secure 0 0 0) 0x7fc0c805ec30 con 0x7fc0d015a5e0 2024-09-10T21:26:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"btime":"2024-09-10T21:26:04:433585+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/1809102798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1809102798},{"type":"v1","addr":"172.21.15.194:6836","nonce":1809102798}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/2065214545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":2065214545},{"type":"v1","addr":"172.21.15.194:6837","nonce":2065214545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/225192623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":225192623},{"type":"v1","addr":"172.21.15.97:6834","nonce":225192623}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/728381088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":728381088},{"type":"v1","addr":"172.21.15.97:6835","nonce":728381088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-09-10T21:26:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0ac0725e0 msgr2=0x7fc0ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0ac0725e0 0x7fc0ac074aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00af000 tx=0x7fc0c001b5c0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 msgr2=0x7fc0d00cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00cd270 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e9a0 tx=0x7fc0c800ee60 comp rx=0 tx=0).stop 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 shutdown_connections 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0ac0725e0 0x7fc0ac074aa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d015a5e0 0x7fc0d00cd270 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0d00702d0 0x7fc0d00ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 --2- 172.21.15.97:0/3458810775 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0d006e220 0x7fc0d0153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 >> 172.21.15.97:0/3458810775 conn(0x7fc0d006cb40 msgr2=0x7fc0d00b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 shutdown_connections 2024-09-10T21:26:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:10.819+0000 7fc0d5800640 1 -- 172.21.15.97:0/3458810775 wait complete. 2024-09-10T21:26:10.845 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:26:10.845 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:26:10.845 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:26:10.846 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:10.846 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:26:10.846 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:26:10.846 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:26:10.846 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:10.846 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:26:10.846 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:26:10.847 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:26:10.847 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:10.847 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:26:10.847 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:26:10.847 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:26:10.847 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:10.857 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:26:10.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:26:10.858 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:26:10.859 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:26:10.861 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:26:10.861 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:26:10.862 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:26:10.862 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:26:10.902 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:26:10.903 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:10.904 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:26:10.904 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:10.911 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:26:10.911 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:10.913 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:26:10.913 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:10.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:26:10.989 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:26:10.990 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:26:11.003 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:26:11.003 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:26:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- 172.21.15.97:0/3927792016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 msgr2=0x7fd48c072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 --2- 172.21.15.97:0/3927792016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c072b70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd48c069e70 tx=0x7fd48401cd30 comp rx=0 tx=0).stop 2024-09-10T21:26:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- 172.21.15.97:0/3927792016 shutdown_connections 2024-09-10T21:26:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 --2- 172.21.15.97:0/3927792016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c072b70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 --2- 172.21.15.97:0/3927792016 >> v2:172.21.15.194:3300/0 conn(0x7fd48c0cf490 0x7fd48c072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 --2- 172.21.15.97:0/3927792016 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd48c0cf0b0 0x7fd48c06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- 172.21.15.97:0/3927792016 >> 172.21.15.97:0/3927792016 conn(0x7fd48c0b1780 msgr2=0x7fd48c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- 172.21.15.97:0/3927792016 shutdown_connections 2024-09-10T21:26:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- 172.21.15.97:0/3927792016 wait complete. 2024-09-10T21:26:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 Processor -- start 2024-09-10T21:26:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- start start 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd48c0cf0b0 0x7fd48c13ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd48c0cf490 0x7fd48c1fa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c166b50 con 0x7fd48c15b110 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c15e6f0 con 0x7fd48c0cf0b0 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd492200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c166cd0 con 0x7fd48c0cf490 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c15b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33542/0 (socket says 172.21.15.97:33542) 2024-09-10T21:26:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 -- 172.21.15.97:0/1449710273 learned_addr learned my addr 172.21.15.97:0/1449710273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48b400640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd48c0cf0b0 0x7fd48c13ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd482a00640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd48c0cf490 0x7fd48c1fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 -- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd48c0cf490 msgr2=0x7fd48c1fa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd48c0cf490 0x7fd48c1fa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 -- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd48c0cf0b0 msgr2=0x7fd48c13ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd48c0cf0b0 0x7fd48c13ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48be00640 1 -- 172.21.15.97:0/1449710273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1fae90 con 0x7fd48c15b110 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd482a00640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd48c0cf490 0x7fd48c1fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.063+0000 7fd48b400640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd48c0cf0b0 0x7fd48c13ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd48be00640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c15b5e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd484009bd0 tx=0x7fd484005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd483e00640 1 -- 172.21.15.97:0/1449710273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840bb070 con 0x7fd48c15b110 2024-09-10T21:26:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd483e00640 1 -- 172.21.15.97:0/1449710273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd484004b40 con 0x7fd48c15b110 2024-09-10T21:26:11.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd483e00640 1 -- 172.21.15.97:0/1449710273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840b2040 con 0x7fd48c15b110 2024-09-10T21:26:11.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c13b410 con 0x7fd48c15b110 2024-09-10T21:26:11.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c13b9d0 con 0x7fd48c15b110 2024-09-10T21:26:11.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c1502d0 con 0x7fd48c15b110 2024-09-10T21:26:11.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd483e00640 1 -- 172.21.15.97:0/1449710273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4840b5030 con 0x7fd48c15b110 2024-09-10T21:26:11.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd483e00640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd468072610 0x7fd468074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.067+0000 7fd483e00640 1 -- 172.21.15.97:0/1449710273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fd48412d0f0 con 0x7fd48c15b110 2024-09-10T21:26:11.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.071+0000 7fd48b400640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd468072610 0x7fd468074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.071+0000 7fd48b400640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd468072610 0x7fd468074ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009ac0 tx=0x7fd47c005f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.071+0000 7fd483e00640 1 -- 172.21.15.97:0/1449710273 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4840c4020 con 0x7fd48c15b110 2024-09-10T21:26:11.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.247+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd48c06e3b0 con 0x7fd48c15b110 2024-09-10T21:26:11.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.475+0000 7fd483e00640 1 -- 172.21.15.97:0/1449710273 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7fd48c1fae90 con 0x7fd48c15b110 2024-09-10T21:26:11.478 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-09-10T21:26:11.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd468072610 msgr2=0x7fd468074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd468072610 0x7fd468074ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009ac0 tx=0x7fd47c005f90 comp rx=0 tx=0).stop 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 msgr2=0x7fd48c15b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c15b5e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd484009bd0 tx=0x7fd484005b00 comp rx=0 tx=0).stop 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 shutdown_connections 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd468072610 0x7fd468074ad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd48c15b110 0x7fd48c15b5e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd48c0cf490 0x7fd48c1fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 --2- 172.21.15.97:0/1449710273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd48c0cf0b0 0x7fd48c13ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 >> 172.21.15.97:0/1449710273 conn(0x7fd48c0b1780 msgr2=0x7fd48c153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 shutdown_connections 2024-09-10T21:26:11.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.479+0000 7fd492200640 1 -- 172.21.15.97:0/1449710273 wait complete. 2024-09-10T21:26:11.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 -- 172.21.15.97:0/3624140497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec0cf0b0 msgr2=0x7faeec06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 --2- 172.21.15.97:0/3624140497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec0cf0b0 0x7faeec06e3b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faeec0af6c0 tx=0x7faedc01c890 comp rx=0 tx=0).stop 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 -- 172.21.15.97:0/3624140497 shutdown_connections 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 --2- 172.21.15.97:0/3624140497 >> v2:172.21.15.194:3300/0 conn(0x7faeec0cf490 0x7faeec072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 --2- 172.21.15.97:0/3624140497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec0cf0b0 0x7faeec06e3b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 -- 172.21.15.97:0/3624140497 >> 172.21.15.97:0/3624140497 conn(0x7faeec0b1780 msgr2=0x7faeec0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 -- 172.21.15.97:0/3624140497 shutdown_connections 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 -- 172.21.15.97:0/3624140497 wait complete. 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 Processor -- start 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.643+0000 7faef2400640 1 -- start start 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec0cf0b0 0x7faeec13e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec0cf490 0x7faeec154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec15b110 0x7faeec0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faeec166f90 con 0x7faeec15b110 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faeec166e10 con 0x7faeec0cf0b0 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7faeec167110 con 0x7faeec0cf490 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec15b110 0x7faeec0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeeb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec0cf0b0 0x7faeec13e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeeb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec0cf0b0 0x7faeec13e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:43768/0 (socket says 172.21.15.97:43768) 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeeb400640 1 -- 172.21.15.97:0/2968082079 learned_addr learned my addr 172.21.15.97:0/2968082079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeeaa00640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec0cf490 0x7faeec154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeebe00640 1 -- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec0cf490 msgr2=0x7faeec154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeebe00640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec0cf490 0x7faeec154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeebe00640 1 -- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec0cf0b0 msgr2=0x7faeec13e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeebe00640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec0cf0b0 0x7faeec13e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeebe00640 1 -- 172.21.15.97:0/2968082079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec14dda0 con 0x7faeec15b110 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faeebe00640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec15b110 0x7faeec0b0790 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faee400c700 tx=0x7faee400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faee3400640 1 -- 172.21.15.97:0/2968082079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4003b80 con 0x7faeec15b110 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faee3400640 1 -- 172.21.15.97:0/2968082079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faee4003d20 con 0x7faeec15b110 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec1fb450 con 0x7faeec15b110 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faee3400640 1 -- 172.21.15.97:0/2968082079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4005330 con 0x7faeec15b110 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec1fba10 con 0x7faeec15b110 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.647+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec06e3b0 con 0x7faeec15b110 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.651+0000 7faee3400640 1 -- 172.21.15.97:0/2968082079 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faee40054d0 con 0x7faeec15b110 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.651+0000 7faee3400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.651+0000 7faee3400640 1 -- 172.21.15.97:0/2968082079 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7faee4092bc0 con 0x7faeec15b110 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.651+0000 7faeeb400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.651+0000 7faeeb400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faedc01a4c0 tx=0x7faedc01b0d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.655+0000 7faee3400640 1 -- 172.21.15.97:0/2968082079 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faee405de00 con 0x7faeec15b110 2024-09-10T21:26:11.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.879+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7faeec1543d0 con 0x7faeec15b110 2024-09-10T21:26:11.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.879+0000 7faee3400640 1 -- 172.21.15.97:0/2968082079 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7faee4062830 con 0x7faeec15b110 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA==", 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.893 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g==", 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-09-10T21:26:11.894 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw==", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA==", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q==", 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q==", 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg==", 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ==", 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.898 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ==", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw==", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.899 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ==", 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-09-10T21:26:11.900 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q==", 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q==", 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:26:11.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ==", 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.902 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag==", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA==", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:11.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmjCI0FRAATFUoCxw0bU6HSEUnlyrVJQ==", 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.904 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA==", 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw==", 2024-09-10T21:26:11.905 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:11.906 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:26:11.906 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:26:11.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:11.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:11.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-10T21:26:11.906 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:26:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 msgr2=0x7faecc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faedc01a4c0 tx=0x7faedc01b0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec15b110 msgr2=0x7faeec0b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec15b110 0x7faeec0b0790 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faee400c700 tx=0x7faee400cbc0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 shutdown_connections 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec15b110 0x7faeec0b0790 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec0cf490 0x7faeec154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 --2- 172.21.15.97:0/2968082079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec0cf0b0 0x7faeec13e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 >> 172.21.15.97:0/2968082079 conn(0x7faeec0b1780 msgr2=0x7faeec164340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 shutdown_connections 2024-09-10T21:26:11.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:11.883+0000 7faef2400640 1 -- 172.21.15.97:0/2968082079 wait complete. 2024-09-10T21:26:11.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:26:12.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 -- 172.21.15.97:0/1804784600 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4070720 msgr2=0x7f76b4072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 --2- 172.21.15.97:0/1804784600 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4070720 0x7f76b4072630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f76a000a950 tx=0x7f76a001db70 comp rx=0 tx=0).stop 2024-09-10T21:26:12.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 -- 172.21.15.97:0/1804784600 shutdown_connections 2024-09-10T21:26:12.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 --2- 172.21.15.97:0/1804784600 >> v2:172.21.15.194:3300/0 conn(0x7f76b4072b70 0x7f76b40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 --2- 172.21.15.97:0/1804784600 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4070720 0x7f76b4072630 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 -- 172.21.15.97:0/1804784600 >> 172.21.15.97:0/1804784600 conn(0x7f76b40b1780 msgr2=0x7f76b40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:12.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 -- 172.21.15.97:0/1804784600 shutdown_connections 2024-09-10T21:26:12.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.047+0000 7f76bba00640 1 -- 172.21.15.97:0/1804784600 wait complete. 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 Processor -- start 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 -- start start 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4070720 0x7f76b4137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4072b70 0x7f76b4152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b415b110 0x7f76b40b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76b40b3a30 con 0x7f76b415b110 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f76b40b38b0 con 0x7f76b4070720 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f76b4166b50 con 0x7f76b4072b70 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b415b110 0x7f76b40b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b415b110 0x7f76b40b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33566/0 (socket says 172.21.15.97:33566) 2024-09-10T21:26:12.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 -- 172.21.15.97:0/3954707829 learned_addr learned my addr 172.21.15.97:0/3954707829 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9000640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4070720 0x7f76b4137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b3e00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4072b70 0x7f76b4152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 -- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4072b70 msgr2=0x7f76b4152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4072b70 0x7f76b4152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 -- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4070720 msgr2=0x7f76b4137630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4070720 0x7f76b4137630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 -- 172.21.15.97:0/3954707829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b4149b10 con 0x7f76b415b110 2024-09-10T21:26:12.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b9a00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b415b110 0x7f76b40b09a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e9b0 tx=0x7f76ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:12.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b0c00640 1 -- 172.21.15.97:0/3954707829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00cd80 con 0x7f76b415b110 2024-09-10T21:26:12.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b0c00640 1 -- 172.21.15.97:0/3954707829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76ac004020 con 0x7f76b415b110 2024-09-10T21:26:12.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76b0c00640 1 -- 172.21.15.97:0/3954707829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac004450 con 0x7f76b415b110 2024-09-10T21:26:12.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b4144830 con 0x7f76b415b110 2024-09-10T21:26:12.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.051+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b4149270 con 0x7f76b415b110 2024-09-10T21:26:12.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.055+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b4072630 con 0x7f76b415b110 2024-09-10T21:26:12.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.055+0000 7f76b0c00640 1 -- 172.21.15.97:0/3954707829 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76ac003600 con 0x7f76b415b110 2024-09-10T21:26:12.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.055+0000 7f76b0c00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76940725e0 0x7f7694074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.055+0000 7f76b0c00640 1 -- 172.21.15.97:0/3954707829 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f76ac0939d0 con 0x7f76b415b110 2024-09-10T21:26:12.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.055+0000 7f76b9000640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76940725e0 0x7f7694074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.055+0000 7f76b9000640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76940725e0 0x7f7694074aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76a40017b0 tx=0x7f76a4001740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:12.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.059+0000 7f76b0c00640 1 -- 172.21.15.97:0/3954707829 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f76ac05eb90 con 0x7f76b415b110 2024-09-10T21:26:12.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f76b40af7f0 con 0x7f76b415b110 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76b0c00640 1 -- 172.21.15.97:0/3954707829 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+65 (secure 0 0 0) 0x7f76ac0635c0 con 0x7f76b415b110 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:12.229 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:26:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76940725e0 msgr2=0x7f7694074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76940725e0 0x7f7694074aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76a40017b0 tx=0x7f76a4001740 comp rx=0 tx=0).stop 2024-09-10T21:26:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b415b110 msgr2=0x7f76b40b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b415b110 0x7f76b40b09a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e9b0 tx=0x7f76ac00ee70 comp rx=0 tx=0).stop 2024-09-10T21:26:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 shutdown_connections 2024-09-10T21:26:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76940725e0 0x7f7694074aa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76b415b110 0x7f76b40b09a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4072b70 0x7f76b4152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 --2- 172.21.15.97:0/3954707829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4070720 0x7f76b4137630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 >> 172.21.15.97:0/3954707829 conn(0x7f76b40b1780 msgr2=0x7f76b40b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:12.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 shutdown_connections 2024-09-10T21:26:12.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.227+0000 7f76bba00640 1 -- 172.21.15.97:0/3954707829 wait complete. 2024-09-10T21:26:12.249 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:26:12.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- 172.21.15.97:0/1975871981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 msgr2=0x7ff0500aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 --2- 172.21.15.97:0/1975871981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff0500aedc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- 172.21.15.97:0/1975871981 shutdown_connections 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 --2- 172.21.15.97:0/1975871981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff0500aedc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 --2- 172.21.15.97:0/1975871981 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0500ce640 0x7ff0500cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- 172.21.15.97:0/1975871981 >> 172.21.15.97:0/1975871981 conn(0x7ff0500d1c30 msgr2=0x7ff0500d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- 172.21.15.97:0/1975871981 shutdown_connections 2024-09-10T21:26:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- 172.21.15.97:0/1975871981 wait complete. 2024-09-10T21:26:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 Processor -- start 2024-09-10T21:26:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- start start 2024-09-10T21:26:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0500af300 0x7ff05006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff05006f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0500ce640 0x7ff050164380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff050169270 con 0x7ff0500b0d50 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501690f0 con 0x7ff0500af300 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff056000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff0501693f0 con 0x7ff0500ce640 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff05006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff05006f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33582/0 (socket says 172.21.15.97:33582) 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 -- 172.21.15.97:0/568466577 learned_addr learned my addr 172.21.15.97:0/568466577 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff04f400640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0500af300 0x7ff05006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff04fe00640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0500ce640 0x7ff050164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 -- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0500ce640 msgr2=0x7ff050164380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0500ce640 0x7ff050164380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 -- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0500af300 msgr2=0x7ff05006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0500af300 0x7ff05006dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.395+0000 7ff046a00640 1 -- 172.21.15.97:0/568466577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0501fc700 con 0x7ff0500b0d50 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff046a00640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff05006f020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff038009850 tx=0x7ff038004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff047e00640 1 -- 172.21.15.97:0/568466577 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b2040 con 0x7ff0500b0d50 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff047e00640 1 -- 172.21.15.97:0/568466577 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff038005030 con 0x7ff0500b0d50 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0501fbaf0 con 0x7ff0500b0d50 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff047e00640 1 -- 172.21.15.97:0/568466577 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038005460 con 0x7ff0500b0d50 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501fc0b0 con 0x7ff0500b0d50 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0500aedc0 con 0x7ff0500b0d50 2024-09-10T21:26:12.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff047e00640 1 -- 172.21.15.97:0/568466577 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff0380c0060 con 0x7ff0500b0d50 2024-09-10T21:26:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff047e00640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff02c0725e0 0x7ff02c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff047e00640 1 -- 172.21.15.97:0/568466577 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7ff03812c820 con 0x7ff0500b0d50 2024-09-10T21:26:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.399+0000 7ff04f400640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff02c0725e0 0x7ff02c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.403+0000 7ff04f400640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff02c0725e0 0x7ff02c074aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0500cc1f0 tx=0x7ff04000a780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:12.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.403+0000 7ff047e00640 1 -- 172.21.15.97:0/568466577 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0380164d0 con 0x7ff0500b0d50 2024-09-10T21:26:12.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:12.575+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff050164f50 con 0x7ff0500b0d50 2024-09-10T21:26:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.495+0000 7ff047e00640 1 -- 172.21.15.97:0/568466577 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) ==== 138+0+0 (secure 0 0 0) 0x7ff0380f80b0 con 0x7ff0500b0d50 2024-09-10T21:26:13.498 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-09-10T21:26:13.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff02c0725e0 msgr2=0x7ff02c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:13.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff02c0725e0 0x7ff02c074aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0500cc1f0 tx=0x7ff04000a780 comp rx=0 tx=0).stop 2024-09-10T21:26:13.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 msgr2=0x7ff05006f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:13.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff05006f020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff038009850 tx=0x7ff038004040 comp rx=0 tx=0).stop 2024-09-10T21:26:13.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 shutdown_connections 2024-09-10T21:26:13.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff02c0725e0 0x7ff02c074aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0500ce640 0x7ff050164380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0500b0d50 0x7ff05006f020 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 --2- 172.21.15.97:0/568466577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0500af300 0x7ff05006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 >> 172.21.15.97:0/568466577 conn(0x7ff0500d1c30 msgr2=0x7ff050155f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 shutdown_connections 2024-09-10T21:26:13.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.499+0000 7ff056000640 1 -- 172.21.15.97:0/568466577 wait complete. 2024-09-10T21:26:13.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:26:13.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 -- 172.21.15.97:0/2612838721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b1e00 msgr2=0x7f0644161db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 --2- 172.21.15.97:0/2612838721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b1e00 0x7f0644161db0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0630009840 tx=0x7f063001c790 comp rx=0 tx=0).stop 2024-09-10T21:26:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 -- 172.21.15.97:0/2612838721 shutdown_connections 2024-09-10T21:26:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 --2- 172.21.15.97:0/2612838721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06440b42e0 0x7f06441622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 --2- 172.21.15.97:0/2612838721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b1e00 0x7f0644161db0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 -- 172.21.15.97:0/2612838721 >> 172.21.15.97:0/2612838721 conn(0x7f06440b37c0 msgr2=0x7f06440b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 -- 172.21.15.97:0/2612838721 shutdown_connections 2024-09-10T21:26:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 -- 172.21.15.97:0/2612838721 wait complete. 2024-09-10T21:26:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 Processor -- start 2024-09-10T21:26:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.675+0000 7f064b600640 1 -- start start 2024-09-10T21:26:13.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06440b2bb0 0x7f06441f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:13.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b42e0 0x7f06440afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:13.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06440c8710 0x7f06440b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:13.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f064416b080 con 0x7f06440b42e0 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f06440d15e0 con 0x7f06440c8710 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f064416b200 con 0x7f06440b2bb0 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b42e0 0x7f06440afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b42e0 0x7f06440afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33600/0 (socket says 172.21.15.97:33600) 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 -- 172.21.15.97:0/1293734866 learned_addr learned my addr 172.21.15.97:0/1293734866 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0648c00640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06440b2bb0 0x7f06441f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0649600640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06440c8710 0x7f06440b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 -- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06440b2bb0 msgr2=0x7f06441f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06440b2bb0 0x7f06441f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 -- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06440c8710 msgr2=0x7f06440b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06440c8710 0x7f06440b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:13.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 -- 172.21.15.97:0/1293734866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0644142c50 con 0x7f06440b42e0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0643e00640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b42e0 0x7f06440afdf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0630009790 tx=0x7f06300a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0640c00640 1 -- 172.21.15.97:0/1293734866 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06300b2040 con 0x7f06440b42e0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0640c00640 1 -- 172.21.15.97:0/1293734866 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06300042c0 con 0x7f06440b42e0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f0640c00640 1 -- 172.21.15.97:0/1293734866 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630005020 con 0x7f06440b42e0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0644142f30 con 0x7f06440b42e0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06441434f0 con 0x7f06440b42e0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.679+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06440b2f90 con 0x7f06440b42e0 2024-09-10T21:26:13.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.683+0000 7f0640c00640 1 -- 172.21.15.97:0/1293734866 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f06300b5070 con 0x7f06440b42e0 2024-09-10T21:26:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.683+0000 7f0640c00640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06240725e0 0x7f0624074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.683+0000 7f0640c00640 1 -- 172.21.15.97:0/1293734866 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5619+0+0 (secure 0 0 0) 0x7f063012c910 con 0x7f06440b42e0 2024-09-10T21:26:13.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.683+0000 7f0648c00640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06240725e0 0x7f0624074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:13.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.683+0000 7f0648c00640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06240725e0 0x7f0624074aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0638000f70 tx=0x7f0638009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:13.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.687+0000 7f0640c00640 1 -- 172.21.15.97:0/1293734866 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0630016410 con 0x7f06440b42e0 2024-09-10T21:26:13.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:13.855+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f06440b3f70 con 0x7f06440b42e0 2024-09-10T21:26:14.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.503+0000 7f0640c00640 1 -- 172.21.15.97:0/1293734866 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) ==== 170+0+0 (secure 0 0 0) 0x7f06300f8150 con 0x7f06440b42e0 2024-09-10T21:26:14.506 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-09-10T21:26:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06240725e0 msgr2=0x7f0624074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06240725e0 0x7f0624074aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0638000f70 tx=0x7f0638009040 comp rx=0 tx=0).stop 2024-09-10T21:26:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b42e0 msgr2=0x7f06440afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b42e0 0x7f06440afdf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0630009790 tx=0x7f06300a56f0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 shutdown_connections 2024-09-10T21:26:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06240725e0 0x7f0624074aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06440c8710 0x7f06440b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06440b42e0 0x7f06440afdf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 --2- 172.21.15.97:0/1293734866 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06440b2bb0 0x7f06441f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 >> 172.21.15.97:0/1293734866 conn(0x7f06440b37c0 msgr2=0x7f064415bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 shutdown_connections 2024-09-10T21:26:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.507+0000 7f064b600640 1 -- 172.21.15.97:0/1293734866 wait complete. 2024-09-10T21:26:14.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:26:14.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 -- 172.21.15.97:0/222371051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 msgr2=0x7fef080cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 --2- 172.21.15.97:0/222371051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef080cf840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feefc009840 tx=0x7feefc01c820 comp rx=0 tx=0).stop 2024-09-10T21:26:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 -- 172.21.15.97:0/222371051 shutdown_connections 2024-09-10T21:26:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 --2- 172.21.15.97:0/222371051 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef080d0490 0x7fef080ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 --2- 172.21.15.97:0/222371051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef080cf840 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 -- 172.21.15.97:0/222371051 >> 172.21.15.97:0/222371051 conn(0x7fef080d1a60 msgr2=0x7fef080d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 -- 172.21.15.97:0/222371051 shutdown_connections 2024-09-10T21:26:14.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 -- 172.21.15.97:0/222371051 wait complete. 2024-09-10T21:26:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 Processor -- start 2024-09-10T21:26:14.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.691+0000 7fef0ec00640 1 -- start start 2024-09-10T21:26:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef080cdcb0 0x7fef0814ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef080d0490 0x7fef0815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef08150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef08168930 con 0x7fef0815efc0 2024-09-10T21:26:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fef081687b0 con 0x7fef080cdcb0 2024-09-10T21:26:14.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fef08168ab0 con 0x7fef080d0490 2024-09-10T21:26:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef08150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef08150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33624/0 (socket says 172.21.15.97:33624) 2024-09-10T21:26:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 -- 172.21.15.97:0/2419124272 learned_addr learned my addr 172.21.15.97:0/2419124272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0d800640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef080cdcb0 0x7fef0814ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ce00640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef080d0490 0x7fef0815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:14.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 -- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef080d0490 msgr2=0x7fef0815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef080d0490 0x7fef0815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 -- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef080cdcb0 msgr2=0x7fef0814ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef080cdcb0 0x7fef0814ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 -- 172.21.15.97:0/2419124272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0815ac40 con 0x7fef0815efc0 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0d800640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef080cdcb0 0x7fef0814ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0e200640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef08150610 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fef0000ea20 tx=0x7fef0000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef05600640 1 -- 172.21.15.97:0/2419124272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0000cde0 con 0x7fef0815efc0 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef05600640 1 -- 172.21.15.97:0/2419124272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef00004c40 con 0x7fef0815efc0 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0814f590 con 0x7fef0815efc0 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef05600640 1 -- 172.21.15.97:0/2419124272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00005280 con 0x7fef0815efc0 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0814fb50 con 0x7fef0815efc0 2024-09-10T21:26:14.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.695+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef080ae930 con 0x7fef0815efc0 2024-09-10T21:26:14.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.699+0000 7fef05600640 1 -- 172.21.15.97:0/2419124272 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fef0000f040 con 0x7fef0815efc0 2024-09-10T21:26:14.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.699+0000 7fef05600640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feee8072610 0x7feee8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:14.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.699+0000 7fef05600640 1 -- 172.21.15.97:0/2419124272 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6001+0+0 (secure 0 0 0) 0x7fef00093900 con 0x7fef0815efc0 2024-09-10T21:26:14.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.699+0000 7fef0d800640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feee8072610 0x7feee8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:14.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.699+0000 7fef0d800640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feee8072610 0x7feee8074ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feefc0af000 tx=0x7feefc01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.703+0000 7fef05600640 1 -- 172.21.15.97:0/2419124272 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fef0815ac40 con 0x7fef0815efc0 2024-09-10T21:26:14.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:14.899+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fef080b01c0 con 0x7fef0815efc0 2024-09-10T21:26:15.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.503+0000 7fef05600640 1 -- 172.21.15.97:0/2419124272 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v29) ==== 172+0+0 (secure 0 0 0) 0x7fef00016030 con 0x7fef0815efc0 2024-09-10T21:26:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 7 and data pool 8 2024-09-10T21:26:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feee8072610 msgr2=0x7feee8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feee8072610 0x7feee8074ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feefc0af000 tx=0x7feefc01b060 comp rx=0 tx=0).stop 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 msgr2=0x7fef08150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef08150610 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fef0000ea20 tx=0x7fef0000eee0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 shutdown_connections 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feee8072610 0x7feee8074ad0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef0815efc0 0x7fef08150610 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fef080d0490 0x7fef0815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 --2- 172.21.15.97:0/2419124272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef080cdcb0 0x7fef0814ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.507+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 >> 172.21.15.97:0/2419124272 conn(0x7fef080d1a60 msgr2=0x7fef080cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:15.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.511+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 shutdown_connections 2024-09-10T21:26:15.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.511+0000 7fef0ec00640 1 -- 172.21.15.97:0/2419124272 wait complete. 2024-09-10T21:26:15.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:15.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 -- 172.21.15.97:0/1444012949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540b52a0 msgr2=0x7f0b540d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 --2- 172.21.15.97:0/1444012949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540b52a0 0x7f0b540d24b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009c80 tx=0x7f0b4c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 -- 172.21.15.97:0/1444012949 shutdown_connections 2024-09-10T21:26:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 --2- 172.21.15.97:0/1444012949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540b52a0 0x7f0b540d24b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 --2- 172.21.15.97:0/1444012949 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b54161790 0x7f0b540b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 -- 172.21.15.97:0/1444012949 >> 172.21.15.97:0/1444012949 conn(0x7f0b540d3ea0 msgr2=0x7f0b540d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 -- 172.21.15.97:0/1444012949 shutdown_connections 2024-09-10T21:26:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.687+0000 7f0b59800640 1 -- 172.21.15.97:0/1444012949 wait complete. 2024-09-10T21:26:15.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 Processor -- start 2024-09-10T21:26:15.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 -- start start 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b540b52a0 0x7f0b540b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540d0770 0x7f0b5414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b54161790 0x7f0b54157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540c77d0 con 0x7f0b540d0770 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540c7650 con 0x7f0b540b52a0 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0b540d29f0 con 0x7f0b54161790 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540d0770 0x7f0b5414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540d0770 0x7f0b5414e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33644/0 (socket says 172.21.15.97:33644) 2024-09-10T21:26:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 -- 172.21.15.97:0/3827883561 learned_addr learned my addr 172.21.15.97:0/3827883561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:15.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b53400640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b54161790 0x7f0b54157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:15.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52a00640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b540b52a0 0x7f0b540b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 -- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b54161790 msgr2=0x7f0b54157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b54161790 0x7f0b54157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 -- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b540b52a0 msgr2=0x7f0b540b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b540b52a0 0x7f0b540b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 -- 172.21.15.97:0/3827883561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540b8c60 con 0x7f0b540d0770 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b52000640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540d0770 0x7f0b5414e920 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c4b0 tx=0x7f0b4000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b4aa00640 1 -- 172.21.15.97:0/3827883561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b400052d0 con 0x7f0b540d0770 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b4aa00640 1 -- 172.21.15.97:0/3827883561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b40005470 con 0x7f0b540d0770 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b540b8f40 con 0x7f0b540d0770 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b4aa00640 1 -- 172.21.15.97:0/3827883561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40012870 con 0x7f0b540d0770 2024-09-10T21:26:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.691+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b540b9500 con 0x7f0b540d0770 2024-09-10T21:26:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.695+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b540b4d60 con 0x7f0b540d0770 2024-09-10T21:26:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.695+0000 7f0b4aa00640 1 -- 172.21.15.97:0/3827883561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0b40003600 con 0x7f0b540d0770 2024-09-10T21:26:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.695+0000 7f0b4aa00640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b340725e0 0x7f0b34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.695+0000 7f0b4aa00640 1 -- 172.21.15.97:0/3827883561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f0b40092f30 con 0x7f0b540d0770 2024-09-10T21:26:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.695+0000 7f0b52a00640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b340725e0 0x7f0b34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:15.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.695+0000 7f0b52a00640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b340725e0 0x7f0b34074aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400bb50 tx=0x7f0b4400b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:15.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.699+0000 7f0b4aa00640 1 -- 172.21.15.97:0/3827883561 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b4005dd90 con 0x7f0b540d0770 2024-09-10T21:26:15.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b54158cb0 con 0x7f0b540d0770 2024-09-10T21:26:15.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b4aa00640 1 -- 172.21.15.97:0/3827883561 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16979 (secure 0 0 0) 0x7f0b400627c0 con 0x7f0b540d0770 2024-09-10T21:26:15.869 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:15.870 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":53,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:14.903523+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:12.586949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:13.866309+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b340725e0 msgr2=0x7f0b34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b340725e0 0x7f0b34074aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400bb50 tx=0x7f0b4400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540d0770 msgr2=0x7f0b5414e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540d0770 0x7f0b5414e920 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c4b0 tx=0x7f0b4000c970 comp rx=0 tx=0).stop 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 shutdown_connections 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b340725e0 0x7f0b34074aa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b54161790 0x7f0b54157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b540d0770 0x7f0b5414e920 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 --2- 172.21.15.97:0/3827883561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b540b52a0 0x7f0b540b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:15.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 >> 172.21.15.97:0/3827883561 conn(0x7f0b540d3ea0 msgr2=0x7f0b540d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:15.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 shutdown_connections 2024-09-10T21:26:15.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:15.867+0000 7f0b59800640 1 -- 172.21.15.97:0/3827883561 wait complete. 2024-09-10T21:26:15.891 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:16.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- 172.21.15.97:0/3295957358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 msgr2=0x7f1fcc0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 --2- 172.21.15.97:0/3295957358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0cfd40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009900 tx=0x7f1fb401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- 172.21.15.97:0/3295957358 shutdown_connections 2024-09-10T21:26:16.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 --2- 172.21.15.97:0/3295957358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0cfd40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 --2- 172.21.15.97:0/3295957358 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1fcc0ce830 0x7f1fcc0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- 172.21.15.97:0/3295957358 >> 172.21.15.97:0/3295957358 conn(0x7f1fcc0b37c0 msgr2=0x7f1fcc0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:16.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- 172.21.15.97:0/3295957358 shutdown_connections 2024-09-10T21:26:16.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- 172.21.15.97:0/3295957358 wait complete. 2024-09-10T21:26:16.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 Processor -- start 2024-09-10T21:26:16.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- start start 2024-09-10T21:26:16.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1fcc0ce830 0x7f1fcc1f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fcc0d0280 0x7f1fcc0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0b4480 con 0x7f1fcc0cec10 2024-09-10T21:26:16.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0cde60 con 0x7f1fcc0d0280 2024-09-10T21:26:16.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fd1600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc0ce1a0 con 0x7f1fcc0ce830 2024-09-10T21:26:16.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33664/0 (socket says 172.21.15.97:33664) 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 -- 172.21.15.97:0/4078175706 learned_addr learned my addr 172.21.15.97:0/4078175706 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fcb400640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fcc0d0280 0x7f1fcc0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fcaa00640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1fcc0ce830 0x7f1fcc1f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 -- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1fcc0ce830 msgr2=0x7f1fcc1f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1fcc0ce830 0x7f1fcc1f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 -- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fcc0d0280 msgr2=0x7f1fcc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fcc0d0280 0x7f1fcc0c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 -- 172.21.15.97:0/4078175706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc1f47b0 con 0x7f1fcc0cec10 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fcb400640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fcc0d0280 0x7f1fcc0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.031+0000 7f1fc2000640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0c8120 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1fb401cd00 tx=0x7f1fb40a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fc3400640 1 -- 172.21.15.97:0/4078175706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40be070 con 0x7f1fcc0cec10 2024-09-10T21:26:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fc3400640 1 -- 172.21.15.97:0/4078175706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fb4005400 con 0x7f1fcc0cec10 2024-09-10T21:26:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc06e990 con 0x7f1fcc0cec10 2024-09-10T21:26:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fc3400640 1 -- 172.21.15.97:0/4078175706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40b5040 con 0x7f1fcc0cec10 2024-09-10T21:26:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc06ef10 con 0x7f1fcc0cec10 2024-09-10T21:26:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc0b2780 con 0x7f1fcc0cec10 2024-09-10T21:26:16.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fc3400640 1 -- 172.21.15.97:0/4078175706 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1fb40b8030 con 0x7f1fcc0cec10 2024-09-10T21:26:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fc3400640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fa80725d0 0x7f1fa8074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fc3400640 1 -- 172.21.15.97:0/4078175706 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1fb4130580 con 0x7f1fcc0cec10 2024-09-10T21:26:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.035+0000 7f1fcaa00640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fa80725d0 0x7f1fa8074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.039+0000 7f1fcaa00640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fa80725d0 0x7f1fa8074a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00bbb0 tx=0x7f1fbc00b590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:16.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.039+0000 7f1fc3400640 1 -- 172.21.15.97:0/4078175706 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1fcc1f47b0 con 0x7f1fcc0cec10 2024-09-10T21:26:16.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.207+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fcc071f80 con 0x7f1fcc0cec10 2024-09-10T21:26:16.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.207+0000 7f1fc3400640 1 -- 172.21.15.97:0/4078175706 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16979 (secure 0 0 0) 0x7f1fb4018c30 con 0x7f1fcc0cec10 2024-09-10T21:26:16.210 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:16.210 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":53,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:14.903523+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:12.586949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:13.866309+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.207+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fa80725d0 msgr2=0x7f1fa8074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.207+0000 7f1fd1600640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fa80725d0 0x7f1fa8074a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00bbb0 tx=0x7f1fbc00b590 comp rx=0 tx=0).stop 2024-09-10T21:26:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.207+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 msgr2=0x7f1fcc0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.207+0000 7f1fd1600640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0c8120 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1fb401cd00 tx=0x7f1fb40a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 shutdown_connections 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1fa80725d0 0x7f1fa8074a90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fcc0d0280 0x7f1fcc0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fcc0cec10 0x7f1fcc0c8120 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 --2- 172.21.15.97:0/4078175706 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1fcc0ce830 0x7f1fcc1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 >> 172.21.15.97:0/4078175706 conn(0x7f1fcc0b37c0 msgr2=0x7f1fcc0cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 shutdown_connections 2024-09-10T21:26:16.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.211+0000 7f1fd1600640 1 -- 172.21.15.97:0/4078175706 wait complete. 2024-09-10T21:26:16.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:26:16.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- 172.21.15.97:0/1194839501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0d0490 msgr2=0x7fbb5c0ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 --2- 172.21.15.97:0/1194839501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0d0490 0x7fbb5c0ae930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009c80 tx=0x7fbb5401cda0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- 172.21.15.97:0/1194839501 shutdown_connections 2024-09-10T21:26:16.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 --2- 172.21.15.97:0/1194839501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0d0490 0x7fbb5c0ae930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 --2- 172.21.15.97:0/1194839501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb5c15efc0 0x7fbb5c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- 172.21.15.97:0/1194839501 >> 172.21.15.97:0/1194839501 conn(0x7fbb5c0d1a60 msgr2=0x7fbb5c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:16.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- 172.21.15.97:0/1194839501 shutdown_connections 2024-09-10T21:26:16.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- 172.21.15.97:0/1194839501 wait complete. 2024-09-10T21:26:16.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 Processor -- start 2024-09-10T21:26:16.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- start start 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0cdcb0 0x7fbb5c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb5c0d0490 0x7fbb5c15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5c15efc0 0x7fbb5c0b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c168b40 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c1689c0 con 0x7fbb5c15efc0 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c168cc0 con 0x7fbb5c0d0490 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0cdcb0 0x7fbb5c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0cdcb0 0x7fbb5c0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33676/0 (socket says 172.21.15.97:33676) 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 -- 172.21.15.97:0/3319973688 learned_addr learned my addr 172.21.15.97:0/3319973688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb53e00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb5c0d0490 0x7fbb5c15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5b400640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5c15efc0 0x7fbb5c0b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 -- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb5c0d0490 msgr2=0x7fbb5c15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb5c0d0490 0x7fbb5c15ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 -- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5c15efc0 msgr2=0x7fbb5c0b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5c15efc0 0x7fbb5c0b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.375+0000 7fbb5aa00640 1 -- 172.21.15.97:0/3319973688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0ca2e0 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb5aa00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0cdcb0 0x7fbb5c0c9cb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0058d0 tx=0x7fbb4c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb52a00640 1 -- 172.21.15.97:0/3319973688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c017070 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb52a00640 1 -- 172.21.15.97:0/3319973688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb4c009d20 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c0ca5c0 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb52a00640 1 -- 172.21.15.97:0/3319973688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0106c0 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c0cab80 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c0ae930 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb52a00640 1 -- 172.21.15.97:0/3319973688 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbb4c010860 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb52a00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb380725e0 0x7fbb38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:16.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb52a00640 1 -- 172.21.15.97:0/3319973688 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fbb4c0938b0 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.379+0000 7fbb53e00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb380725e0 0x7fbb38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.383+0000 7fbb53e00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb380725e0 0x7fbb38074aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000f70 tx=0x7fbb44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:16.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.383+0000 7fbb52a00640 1 -- 172.21.15.97:0/3319973688 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbb4c05edf0 con 0x7fbb5c0cdcb0 2024-09-10T21:26:16.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:16.583+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbb5c070790 con 0x7fbb5c0cdcb0 2024-09-10T21:26:17.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.531+0000 7fbb52a00640 1 -- 172.21.15.97:0/3319973688 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7fbb4c05e6f0 con 0x7fbb5c0cdcb0 2024-09-10T21:26:17.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb380725e0 msgr2=0x7fbb38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb380725e0 0x7fbb38074aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000f70 tx=0x7fbb44009040 comp rx=0 tx=0).stop 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0cdcb0 msgr2=0x7fbb5c0c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0cdcb0 0x7fbb5c0c9cb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0058d0 tx=0x7fbb4c00dde0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 shutdown_connections 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbb380725e0 0x7fbb38074aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb5c15efc0 0x7fbb5c0b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbb5c0d0490 0x7fbb5c15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 --2- 172.21.15.97:0/3319973688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb5c0cdcb0 0x7fbb5c0c9cb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 >> 172.21.15.97:0/3319973688 conn(0x7fbb5c0d1a60 msgr2=0x7fbb5c070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 shutdown_connections 2024-09-10T21:26:17.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.535+0000 7fbb5be00640 1 -- 172.21.15.97:0/3319973688 wait complete. 2024-09-10T21:26:17.561 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:26:17.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:26:17.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- 172.21.15.97:0/2052285915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d0003a10 msgr2=0x7f25d0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 --2- 172.21.15.97:0/2052285915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d0003a10 0x7f25d0003e30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009b60 tx=0x7f25dc01d270 comp rx=0 tx=0).stop 2024-09-10T21:26:17.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f0c00640 1 -- 172.21.15.97:0/2052285915 reap_dead start 2024-09-10T21:26:17.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- 172.21.15.97:0/2052285915 shutdown_connections 2024-09-10T21:26:17.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- 172.21.15.97:0/2052285915 >> 172.21.15.97:0/2052285915 conn(0x7f25ec0b1780 msgr2=0x7f25ec0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- 172.21.15.97:0/2052285915 shutdown_connections 2024-09-10T21:26:17.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- 172.21.15.97:0/2052285915 wait complete. 2024-09-10T21:26:17.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 Processor -- start 2024-09-10T21:26:17.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- start start 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25d0003a10 0x7f25ec148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ec14b6b0 0x7f25ec1517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ec151d00 0x7f25ec1e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0b3b40 con 0x7f25ec151d00 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0b39c0 con 0x7f25ec14b6b0 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f2c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec06e3b0 con 0x7f25d0003a10 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ec151d00 0x7f25ec1e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ec151d00 0x7f25ec1e6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41570/0 (socket says 172.21.15.97:41570) 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25f0c00640 1 -- 172.21.15.97:0/1678663718 learned_addr learned my addr 172.21.15.97:0/1678663718 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25e3400640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ec14b6b0 0x7f25ec1517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.707+0000 7f25ebe00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25d0003a10 0x7f25ec148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f0c00640 1 -- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25d0003a10 msgr2=0x7f25ec148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f0c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25d0003a10 0x7f25ec148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f0c00640 1 -- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ec14b6b0 msgr2=0x7f25ec1517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f0c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ec14b6b0 0x7f25ec1517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f0c00640 1 -- 172.21.15.97:0/1678663718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec14a610 con 0x7f25ec151d00 2024-09-10T21:26:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f0c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ec151d00 0x7f25ec1e6e60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25ec074f10 tx=0x7f25e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25e8c00640 1 -- 172.21.15.97:0/1678663718 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4017070 con 0x7f25ec151d00 2024-09-10T21:26:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25e8c00640 1 -- 172.21.15.97:0/1678663718 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25e4013a30 con 0x7f25ec151d00 2024-09-10T21:26:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25e8c00640 1 -- 172.21.15.97:0/1678663718 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e400f450 con 0x7f25ec151d00 2024-09-10T21:26:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec14a8f0 con 0x7f25ec151d00 2024-09-10T21:26:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec14aeb0 con 0x7f25ec151d00 2024-09-10T21:26:17.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec0ceee0 con 0x7f25ec151d00 2024-09-10T21:26:17.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.711+0000 7f25e8c00640 1 -- 172.21.15.97:0/1678663718 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f25e4010040 con 0x7f25ec151d00 2024-09-10T21:26:17.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.715+0000 7f25e8c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c8072610 0x7f25c8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.715+0000 7f25e8c00640 1 -- 172.21.15.97:0/1678663718 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f25e4096e00 con 0x7f25ec151d00 2024-09-10T21:26:17.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.715+0000 7f25ebe00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c8072610 0x7f25c8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.715+0000 7f25ebe00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c8072610 0x7f25c8074ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0ae000 tx=0x7f25dc01a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.719+0000 7f25e8c00640 1 -- 172.21.15.97:0/1678663718 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f25e4061cf0 con 0x7f25ec151d00 2024-09-10T21:26:17.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:17.915+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f25ec050ef0 con 0x7f25ec151d00 2024-09-10T21:26:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.539+0000 7f25e8c00640 1 -- 172.21.15.97:0/1678663718 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f25e4066720 con 0x7f25ec151d00 2024-09-10T21:26:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c8072610 msgr2=0x7f25c8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c8072610 0x7f25c8074ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0ae000 tx=0x7f25dc01a010 comp rx=0 tx=0).stop 2024-09-10T21:26:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ec151d00 msgr2=0x7f25ec1e6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ec151d00 0x7f25ec1e6e60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25ec074f10 tx=0x7f25e4011a30 comp rx=0 tx=0).stop 2024-09-10T21:26:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 shutdown_connections 2024-09-10T21:26:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c8072610 0x7f25c8074ad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ec151d00 0x7f25ec1e6e60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ec14b6b0 0x7f25ec1517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 --2- 172.21.15.97:0/1678663718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25d0003a10 0x7f25ec148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 >> 172.21.15.97:0/1678663718 conn(0x7f25ec0b1780 msgr2=0x7f25ec0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 shutdown_connections 2024-09-10T21:26:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.543+0000 7f25f2c00640 1 -- 172.21.15.97:0/1678663718 wait complete. 2024-09-10T21:26:18.567 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:26:18.567 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:18.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 -- 172.21.15.97:0/1562162405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cfbd0 msgr2=0x7fe2000d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1562162405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cfbd0 0x7fe2000d2590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009840 tx=0x7fe1f001c790 comp rx=0 tx=0).stop 2024-09-10T21:26:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 -- 172.21.15.97:0/1562162405 shutdown_connections 2024-09-10T21:26:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1562162405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cfbd0 0x7fe2000d2590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 -- 172.21.15.97:0/1562162405 >> 172.21.15.97:0/1562162405 conn(0x7fe2000d1a70 msgr2=0x7fe2000d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 -- 172.21.15.97:0/1562162405 shutdown_connections 2024-09-10T21:26:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 -- 172.21.15.97:0/1562162405 wait complete. 2024-09-10T21:26:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.707+0000 7fe1ff200640 1 Processor -- start 2024-09-10T21:26:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 -- start start 2024-09-10T21:26:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000722c0 0x7fe200155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000cfbd0 0x7fe2000b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cffb0 0x7fe200147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe200169e10 con 0x7fe2000cffb0 2024-09-10T21:26:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe200169c90 con 0x7fe2000cfbd0 2024-09-10T21:26:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe200169f90 con 0x7fe2000722c0 2024-09-10T21:26:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cffb0 0x7fe200147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cffb0 0x7fe200147820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41588/0 (socket says 172.21.15.97:41588) 2024-09-10T21:26:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 -- 172.21.15.97:0/1803418846 learned_addr learned my addr 172.21.15.97:0/1803418846 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1f5400640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000cfbd0 0x7fe2000b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fde00640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000722c0 0x7fe200155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 -- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000722c0 msgr2=0x7fe200155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000722c0 0x7fe200155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 -- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000cfbd0 msgr2=0x7fe2000b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000cfbd0 0x7fe2000b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 -- 172.21.15.97:0/1803418846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2000cc650 con 0x7fe2000cffb0 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1fe800640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cffb0 0x7fe200147820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800cb10 tx=0x7fe1f800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1f6a00640 1 -- 172.21.15.97:0/1803418846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8016020 con 0x7fe2000cffb0 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1f6a00640 1 -- 172.21.15.97:0/1803418846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1f800ce50 con 0x7fe2000cffb0 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2000b43b0 con 0x7fe2000cffb0 2024-09-10T21:26:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1f6a00640 1 -- 172.21.15.97:0/1803418846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800adc0 con 0x7fe2000cffb0 2024-09-10T21:26:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2000cea60 con 0x7fe2000cffb0 2024-09-10T21:26:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.711+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2000cf070 con 0x7fe2000cffb0 2024-09-10T21:26:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.715+0000 7fe1f6a00640 1 -- 172.21.15.97:0/1803418846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe1f801a3b0 con 0x7fe2000cffb0 2024-09-10T21:26:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.715+0000 7fe1f6a00640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc072610 0x7fe1dc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:18.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.715+0000 7fe1f6a00640 1 -- 172.21.15.97:0/1803418846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe1f80934e0 con 0x7fe2000cffb0 2024-09-10T21:26:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.715+0000 7fe1fde00640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc072610 0x7fe1dc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:18.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.715+0000 7fe1fde00640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc072610 0x7fe1dc074ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00af000 tx=0x7fe1f001afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:18.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.719+0000 7fe1f6a00640 1 -- 172.21.15.97:0/1803418846 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1f8020030 con 0x7fe2000cffb0 2024-09-10T21:26:18.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2000d2590 con 0x7fe2000cffb0 2024-09-10T21:26:18.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1f6a00640 1 -- 172.21.15.97:0/1803418846 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5223 (secure 0 0 0) 0x7fe2000b43b0 con 0x7fe2000cffb0 2024-09-10T21:26:18.922 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:18.922 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"btime":"2024-09-10T21:26:18:534778+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:17.919032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5484":{"gid":5484,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6834/3721865460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3721865460},{"type":"v1","addr":"172.21.15.97:6834","nonce":3721865460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5484,"qdb_cluster":[5484]},"id":3}]} 2024-09-10T21:26:18.922 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 33 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc072610 msgr2=0x7fe1dc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc072610 0x7fe1dc074ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00af000 tx=0x7fe1f001afd0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cffb0 msgr2=0x7fe200147820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cffb0 0x7fe200147820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800cb10 tx=0x7fe1f800cc00 comp rx=0 tx=0).stop 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 shutdown_connections 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc072610 0x7fe1dc074ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000cffb0 0x7fe200147820 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000cfbd0 0x7fe2000b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 --2- 172.21.15.97:0/1803418846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000722c0 0x7fe200155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 >> 172.21.15.97:0/1803418846 conn(0x7fe2000d1a70 msgr2=0x7fe200159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 shutdown_connections 2024-09-10T21:26:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:18.919+0000 7fe1ff200640 1 -- 172.21.15.97:0/1803418846 wait complete. 2024-09-10T21:26:18.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:19.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- 172.21.15.97:0/816701513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2bc003a60 msgr2=0x7fa2bc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 --2- 172.21.15.97:0/816701513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2bc003a60 0x7fa2bc003e80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8006fd0 tx=0x7fa2c801ca70 comp rx=0 tx=0).stop 2024-09-10T21:26:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- 172.21.15.97:0/816701513 shutdown_connections 2024-09-10T21:26:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 --2- 172.21.15.97:0/816701513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2bc003a60 0x7fa2bc003e80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- 172.21.15.97:0/816701513 >> 172.21.15.97:0/816701513 conn(0x7fa2d80b1780 msgr2=0x7fa2d80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- 172.21.15.97:0/816701513 shutdown_connections 2024-09-10T21:26:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- 172.21.15.97:0/816701513 wait complete. 2024-09-10T21:26:19.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 Processor -- start 2024-09-10T21:26:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- start start 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2bc003a60 0x7fa2d8155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d8155f40 0x7fa2d8142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa2d8142940 0x7fa2d8158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d815ce00 con 0x7fa2d8155f40 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d815cc80 con 0x7fa2bc003a60 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.083+0000 7fa2df200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d8157ef0 con 0x7fa2d8142940 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d8155f40 0x7fa2d8142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2bc003a60 0x7fa2d8155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d8155f40 0x7fa2d8142400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41598/0 (socket says 172.21.15.97:41598) 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 -- 172.21.15.97:0/2243405161 learned_addr learned my addr 172.21.15.97:0/2243405161 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2dd200640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa2d8142940 0x7fa2d8158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 -- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa2d8142940 msgr2=0x7fa2d8158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa2d8142940 0x7fa2d8158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 -- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2bc003a60 msgr2=0x7fa2d8155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2bc003a60 0x7fa2d8155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 -- 172.21.15.97:0/2243405161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d81574b0 con 0x7fa2d8155f40 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2cf400640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d8155f40 0x7fa2d8142400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0006630 tx=0x7fa2c00039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2d4c00640 1 -- 172.21.15.97:0/2243405161 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0011700 con 0x7fa2d8155f40 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2d4c00640 1 -- 172.21.15.97:0/2243405161 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2c0016020 con 0x7fa2d8155f40 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2d4c00640 1 -- 172.21.15.97:0/2243405161 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0011b40 con 0x7fa2d8155f40 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d8158140 con 0x7fa2d8155f40 2024-09-10T21:26:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d8158700 con 0x7fa2d8155f40 2024-09-10T21:26:19.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d813fd90 con 0x7fa2d8155f40 2024-09-10T21:26:19.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.087+0000 7fa2d4c00640 1 -- 172.21.15.97:0/2243405161 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa2c0011ce0 con 0x7fa2d8155f40 2024-09-10T21:26:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.091+0000 7fa2d4c00640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa2b40725e0 0x7fa2b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.091+0000 7fa2d4c00640 1 -- 172.21.15.97:0/2243405161 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa2c0093e10 con 0x7fa2d8155f40 2024-09-10T21:26:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.091+0000 7fa2d7e00640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa2b40725e0 0x7fa2b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.091+0000 7fa2d7e00640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa2b40725e0 0x7fa2b4074aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80aa000 tx=0x7fa2c80b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.095+0000 7fa2d4c00640 1 -- 172.21.15.97:0/2243405161 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa2c0097030 con 0x7fa2d8155f40 2024-09-10T21:26:19.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.263+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2d81588d0 con 0x7fa2d8155f40 2024-09-10T21:26:19.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2d4c00640 1 -- 172.21.15.97:0/2243405161 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16966 (secure 0 0 0) 0x7fa2c0020020 con 0x7fa2d8155f40 2024-09-10T21:26:19.268 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:19.268 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:16.518412+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:12.586949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:13.866309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa2b40725e0 msgr2=0x7fa2b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa2b40725e0 0x7fa2b4074aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80aa000 tx=0x7fa2c80b0040 comp rx=0 tx=0).stop 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d8155f40 msgr2=0x7fa2d8142400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d8155f40 0x7fa2d8142400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0006630 tx=0x7fa2c00039d0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 shutdown_connections 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa2b40725e0 0x7fa2b4074aa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa2d8142940 0x7fa2d8158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d8155f40 0x7fa2d8142400 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 --2- 172.21.15.97:0/2243405161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2bc003a60 0x7fa2d8155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 >> 172.21.15.97:0/2243405161 conn(0x7fa2d80b1780 msgr2=0x7fa2d815ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 shutdown_connections 2024-09-10T21:26:19.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.267+0000 7fa2df200640 1 -- 172.21.15.97:0/2243405161 wait complete. 2024-09-10T21:26:19.290 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:26:19.290 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 -- 172.21.15.97:0/3086850049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06d180 msgr2=0x7fe34c070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 --2- 172.21.15.97:0/3086850049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06d180 0x7fe34c070130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe338009840 tx=0x7fe33801c710 comp rx=0 tx=0).stop 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 -- 172.21.15.97:0/3086850049 shutdown_connections 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 --2- 172.21.15.97:0/3086850049 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe34c161820 0x7fe34c0b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 --2- 172.21.15.97:0/3086850049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06d180 0x7fe34c070130 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 -- 172.21.15.97:0/3086850049 >> 172.21.15.97:0/3086850049 conn(0x7fe34c0b2790 msgr2=0x7fe34c0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 -- 172.21.15.97:0/3086850049 shutdown_connections 2024-09-10T21:26:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.427+0000 7fe350a00640 1 -- 172.21.15.97:0/3086850049 wait complete. 2024-09-10T21:26:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 Processor -- start 2024-09-10T21:26:19.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 -- start start 2024-09-10T21:26:19.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06cda0 0x7fe34c13a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe34c06d180 0x7fe34c1601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34c161820 0x7fe34c14c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c16b0f0 con 0x7fe34c161820 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c16af70 con 0x7fe34c06cda0 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe34c16b270 con 0x7fe34c06d180 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34c161820 0x7fe34c14c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34c161820 0x7fe34c14c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41604/0 (socket says 172.21.15.97:41604) 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 -- 172.21.15.97:0/2538279578 learned_addr learned my addr 172.21.15.97:0/2538279578 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe349400640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe34c06d180 0x7fe34c1601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe349e00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06cda0 0x7fe34c13a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 -- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe34c06d180 msgr2=0x7fe34c1601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe34c06d180 0x7fe34c1601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 -- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06cda0 msgr2=0x7fe34c13a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06cda0 0x7fe34c13a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 -- 172.21.15.97:0/2538279578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c151fd0 con 0x7fe34c161820 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe34a800640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34c161820 0x7fe34c14c370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe34400ea80 tx=0x7fe34400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe342000640 1 -- 172.21.15.97:0/2538279578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344004b30 con 0x7fe34c161820 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe342000640 1 -- 172.21.15.97:0/2538279578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe344004cf0 con 0x7fe34c161820 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34c1522b0 con 0x7fe34c161820 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe342000640 1 -- 172.21.15.97:0/2538279578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344005280 con 0x7fe34c161820 2024-09-10T21:26:19.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34c14be50 con 0x7fe34c161820 2024-09-10T21:26:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.431+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34c15dbb0 con 0x7fe34c161820 2024-09-10T21:26:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.435+0000 7fe342000640 1 -- 172.21.15.97:0/2538279578 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe344005420 con 0x7fe34c161820 2024-09-10T21:26:19.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.435+0000 7fe342000640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe32c072610 0x7fe32c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.435+0000 7fe342000640 1 -- 172.21.15.97:0/2538279578 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe344093fd0 con 0x7fe34c161820 2024-09-10T21:26:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.435+0000 7fe349e00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe32c072610 0x7fe32c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.435+0000 7fe349e00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe32c072610 0x7fe32c074ad0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001e20 tx=0x7fe33c001d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.439+0000 7fe342000640 1 -- 172.21.15.97:0/2538279578 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe34405ee30 con 0x7fe34c161820 2024-09-10T21:26:19.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.603+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe34c15d8d0 con 0x7fe32c072610 2024-09-10T21:26:19.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.607+0000 7fe342000640 1 -- 172.21.15.97:0/2538279578 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249758 (secure 0 0 0) 0x7fe34c15d8d0 con 0x7fe32c072610 2024-09-10T21:26:19.613 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:19.614 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:26:19.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe32c072610 msgr2=0x7fe32c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe32c072610 0x7fe32c074ad0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001e20 tx=0x7fe33c001d70 comp rx=0 tx=0).stop 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34c161820 msgr2=0x7fe34c14c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34c161820 0x7fe34c14c370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe34400ea80 tx=0x7fe34400ef40 comp rx=0 tx=0).stop 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 shutdown_connections 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe32c072610 0x7fe32c074ad0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe34c161820 0x7fe34c14c370 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe34c06d180 0x7fe34c1601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 --2- 172.21.15.97:0/2538279578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe34c06cda0 0x7fe34c13a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 >> 172.21.15.97:0/2538279578 conn(0x7fe34c0b2790 msgr2=0x7fe34c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 shutdown_connections 2024-09-10T21:26:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.611+0000 7fe350a00640 1 -- 172.21.15.97:0/2538279578 wait complete. 2024-09-10T21:26:19.633 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-10T21:26:18.100132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":164,"acting":164,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222232,"kb_used_data":5984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752488,"statfs":{"total":773094113280,"available":772866547712,"internally_reserved":0,"allocated":6127616,"data_stored":3352429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.595487"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549183+0000","last_change":"2024-09-10T21:26:15.539541+0000","last_active":"2024-09-10T21:26:15.549183+0000","last_peered":"2024-09-10T21:26:15.549183+0000","last_clean":"2024-09-10T21:26:15.549183+0000","last_became_active":"2024-09-10T21:26:15.539095+0000","last_became_peered":"2024-09-10T21:26:15.539095+0000","last_unstale":"2024-09-10T21:26:15.549183+0000","last_undegraded":"2024-09-10T21:26:15.549183+0000","last_fullsized":"2024-09-10T21:26:15.549183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.543900+0000","last_change":"2024-09-10T21:26:15.535693+0000","last_active":"2024-09-10T21:26:15.543900+0000","last_peered":"2024-09-10T21:26:15.543900+0000","last_clean":"2024-09-10T21:26:15.543900+0000","last_became_active":"2024-09-10T21:26:15.535421+0000","last_became_peered":"2024-09-10T21:26:15.535421+0000","last_unstale":"2024-09-10T21:26:15.543900+0000","last_undegraded":"2024-09-10T21:26:15.543900+0000","last_fullsized":"2024-09-10T21:26:15.543900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546167+0000","last_change":"2024-09-10T21:26:15.537322+0000","last_active":"2024-09-10T21:26:15.546167+0000","last_peered":"2024-09-10T21:26:15.546167+0000","last_clean":"2024-09-10T21:26:15.546167+0000","last_became_active":"2024-09-10T21:26:15.537039+0000","last_became_peered":"2024-09-10T21:26:15.537039+0000","last_unstale":"2024-09-10T21:26:15.546167+0000","last_undegraded":"2024-09-10T21:26:15.546167+0000","last_fullsized":"2024-09-10T21:26:15.546167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:11.598464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551006+0000","last_change":"2024-09-10T21:26:15.547237+0000","last_active":"2024-09-10T21:26:15.551006+0000","last_peered":"2024-09-10T21:26:15.551006+0000","last_clean":"2024-09-10T21:26:15.551006+0000","last_became_active":"2024-09-10T21:26:15.546563+0000","last_became_peered":"2024-09-10T21:26:15.546563+0000","last_unstale":"2024-09-10T21:26:15.551006+0000","last_undegraded":"2024-09-10T21:26:15.551006+0000","last_fullsized":"2024-09-10T21:26:15.551006+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544695+0000","last_change":"2024-09-10T21:26:15.534547+0000","last_active":"2024-09-10T21:26:15.544695+0000","last_peered":"2024-09-10T21:26:15.544695+0000","last_clean":"2024-09-10T21:26:15.544695+0000","last_became_active":"2024-09-10T21:26:15.534280+0000","last_became_peered":"2024-09-10T21:26:15.534280+0000","last_unstale":"2024-09-10T21:26:15.544695+0000","last_undegraded":"2024-09-10T21:26:15.544695+0000","last_fullsized":"2024-09-10T21:26:15.544695+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:48.054252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548817+0000","last_change":"2024-09-10T21:26:15.533183+0000","last_active":"2024-09-10T21:26:15.548817+0000","last_peered":"2024-09-10T21:26:15.548817+0000","last_clean":"2024-09-10T21:26:15.548817+0000","last_became_active":"2024-09-10T21:26:15.532995+0000","last_became_peered":"2024-09-10T21:26:15.532995+0000","last_unstale":"2024-09-10T21:26:15.548817+0000","last_undegraded":"2024-09-10T21:26:15.548817+0000","last_fullsized":"2024-09-10T21:26:15.548817+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:27.005167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546214+0000","last_change":"2024-09-10T21:26:15.541612+0000","last_active":"2024-09-10T21:26:15.546214+0000","last_peered":"2024-09-10T21:26:15.546214+0000","last_clean":"2024-09-10T21:26:15.546214+0000","last_became_active":"2024-09-10T21:26:15.541343+0000","last_became_peered":"2024-09-10T21:26:15.541343+0000","last_unstale":"2024-09-10T21:26:15.546214+0000","last_undegraded":"2024-09-10T21:26:15.546214+0000","last_fullsized":"2024-09-10T21:26:15.546214+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548595+0000","last_change":"2024-09-10T21:26:15.536440+0000","last_active":"2024-09-10T21:26:15.548595+0000","last_peered":"2024-09-10T21:26:15.548595+0000","last_clean":"2024-09-10T21:26:15.548595+0000","last_became_active":"2024-09-10T21:26:15.535896+0000","last_became_peered":"2024-09-10T21:26:15.535896+0000","last_unstale":"2024-09-10T21:26:15.548595+0000","last_undegraded":"2024-09-10T21:26:15.548595+0000","last_fullsized":"2024-09-10T21:26:15.548595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:57.577430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549667+0000","last_change":"2024-09-10T21:26:15.538325+0000","last_active":"2024-09-10T21:26:15.549667+0000","last_peered":"2024-09-10T21:26:15.549667+0000","last_clean":"2024-09-10T21:26:15.549667+0000","last_became_active":"2024-09-10T21:26:15.536197+0000","last_became_peered":"2024-09-10T21:26:15.536197+0000","last_unstale":"2024-09-10T21:26:15.549667+0000","last_undegraded":"2024-09-10T21:26:15.549667+0000","last_fullsized":"2024-09-10T21:26:15.549667+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520191+0000","last_change":"2024-09-10T21:26:15.518220+0000","last_active":"2024-09-10T21:26:15.520191+0000","last_peered":"2024-09-10T21:26:15.520191+0000","last_clean":"2024-09-10T21:26:15.520191+0000","last_became_active":"2024-09-10T21:26:15.516898+0000","last_became_peered":"2024-09-10T21:26:15.516898+0000","last_unstale":"2024-09-10T21:26:15.520191+0000","last_undegraded":"2024-09-10T21:26:15.520191+0000","last_fullsized":"2024-09-10T21:26:15.520191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.482589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547572+0000","last_change":"2024-09-10T21:26:15.545455+0000","last_active":"2024-09-10T21:26:15.547572+0000","last_peered":"2024-09-10T21:26:15.547572+0000","last_clean":"2024-09-10T21:26:15.547572+0000","last_became_active":"2024-09-10T21:26:15.545207+0000","last_became_peered":"2024-09-10T21:26:15.545207+0000","last_unstale":"2024-09-10T21:26:15.547572+0000","last_undegraded":"2024-09-10T21:26:15.547572+0000","last_fullsized":"2024-09-10T21:26:15.547572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549843+0000","last_change":"2024-09-10T21:26:15.545646+0000","last_active":"2024-09-10T21:26:15.549843+0000","last_peered":"2024-09-10T21:26:15.549843+0000","last_clean":"2024-09-10T21:26:15.549843+0000","last_became_active":"2024-09-10T21:26:15.544941+0000","last_became_peered":"2024-09-10T21:26:15.544941+0000","last_unstale":"2024-09-10T21:26:15.549843+0000","last_undegraded":"2024-09-10T21:26:15.549843+0000","last_fullsized":"2024-09-10T21:26:15.549843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542087+0000","last_change":"2024-09-10T21:26:15.536666+0000","last_active":"2024-09-10T21:26:15.542087+0000","last_peered":"2024-09-10T21:26:15.542087+0000","last_clean":"2024-09-10T21:26:15.542087+0000","last_became_active":"2024-09-10T21:26:15.536410+0000","last_became_peered":"2024-09-10T21:26:15.536410+0000","last_unstale":"2024-09-10T21:26:15.542087+0000","last_undegraded":"2024-09-10T21:26:15.542087+0000","last_fullsized":"2024-09-10T21:26:15.542087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:15.595334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.529423+0000","last_change":"2024-09-10T21:26:15.521400+0000","last_active":"2024-09-10T21:26:15.529423+0000","last_peered":"2024-09-10T21:26:15.529423+0000","last_clean":"2024-09-10T21:26:15.529423+0000","last_became_active":"2024-09-10T21:26:15.520076+0000","last_became_peered":"2024-09-10T21:26:15.520076+0000","last_unstale":"2024-09-10T21:26:15.529423+0000","last_undegraded":"2024-09-10T21:26:15.529423+0000","last_fullsized":"2024-09-10T21:26:15.529423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:13.921295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549231+0000","last_change":"2024-09-10T21:26:15.544245+0000","last_active":"2024-09-10T21:26:15.549231+0000","last_peered":"2024-09-10T21:26:15.549231+0000","last_clean":"2024-09-10T21:26:15.549231+0000","last_became_active":"2024-09-10T21:26:15.539109+0000","last_became_peered":"2024-09-10T21:26:15.539109+0000","last_unstale":"2024-09-10T21:26:15.549231+0000","last_undegraded":"2024-09-10T21:26:15.549231+0000","last_fullsized":"2024-09-10T21:26:15.549231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.531781+0000","last_change":"2024-09-10T21:26:15.520766+0000","last_active":"2024-09-10T21:26:15.531781+0000","last_peered":"2024-09-10T21:26:15.531781+0000","last_clean":"2024-09-10T21:26:15.531781+0000","last_became_active":"2024-09-10T21:26:15.520453+0000","last_became_peered":"2024-09-10T21:26:15.520453+0000","last_unstale":"2024-09-10T21:26:15.531781+0000","last_undegraded":"2024-09-10T21:26:15.531781+0000","last_fullsized":"2024-09-10T21:26:15.531781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:58.808915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545207+0000","last_change":"2024-09-10T21:26:15.540350+0000","last_active":"2024-09-10T21:26:15.545207+0000","last_peered":"2024-09-10T21:26:15.545207+0000","last_clean":"2024-09-10T21:26:15.545207+0000","last_became_active":"2024-09-10T21:26:15.535466+0000","last_became_peered":"2024-09-10T21:26:15.535466+0000","last_unstale":"2024-09-10T21:26:15.545207+0000","last_undegraded":"2024-09-10T21:26:15.545207+0000","last_fullsized":"2024-09-10T21:26:15.545207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542035+0000","last_change":"2024-09-10T21:26:15.534826+0000","last_active":"2024-09-10T21:26:15.542035+0000","last_peered":"2024-09-10T21:26:15.542035+0000","last_clean":"2024-09-10T21:26:15.542035+0000","last_became_active":"2024-09-10T21:26:15.534533+0000","last_became_peered":"2024-09-10T21:26:15.534533+0000","last_unstale":"2024-09-10T21:26:15.542035+0000","last_undegraded":"2024-09-10T21:26:15.542035+0000","last_fullsized":"2024-09-10T21:26:15.542035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:28.481727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549208+0000","last_change":"2024-09-10T21:26:15.539933+0000","last_active":"2024-09-10T21:26:15.549208+0000","last_peered":"2024-09-10T21:26:15.549208+0000","last_clean":"2024-09-10T21:26:15.549208+0000","last_became_active":"2024-09-10T21:26:15.539551+0000","last_became_peered":"2024-09-10T21:26:15.539551+0000","last_unstale":"2024-09-10T21:26:15.549208+0000","last_undegraded":"2024-09-10T21:26:15.549208+0000","last_fullsized":"2024-09-10T21:26:15.549208+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546220+0000","last_change":"2024-09-10T21:26:15.541772+0000","last_active":"2024-09-10T21:26:15.546220+0000","last_peered":"2024-09-10T21:26:15.546220+0000","last_clean":"2024-09-10T21:26:15.546220+0000","last_became_active":"2024-09-10T21:26:15.541512+0000","last_became_peered":"2024-09-10T21:26:15.541512+0000","last_unstale":"2024-09-10T21:26:15.546220+0000","last_undegraded":"2024-09-10T21:26:15.546220+0000","last_fullsized":"2024-09-10T21:26:15.546220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550226+0000","last_change":"2024-09-10T21:26:15.545746+0000","last_active":"2024-09-10T21:26:15.550226+0000","last_peered":"2024-09-10T21:26:15.550226+0000","last_clean":"2024-09-10T21:26:15.550226+0000","last_became_active":"2024-09-10T21:26:15.545498+0000","last_became_peered":"2024-09-10T21:26:15.545498+0000","last_unstale":"2024-09-10T21:26:15.550226+0000","last_undegraded":"2024-09-10T21:26:15.550226+0000","last_fullsized":"2024-09-10T21:26:15.550226+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549974+0000","last_change":"2024-09-10T21:26:15.546272+0000","last_active":"2024-09-10T21:26:15.549974+0000","last_peered":"2024-09-10T21:26:15.549974+0000","last_clean":"2024-09-10T21:26:15.549974+0000","last_became_active":"2024-09-10T21:26:15.537090+0000","last_became_peered":"2024-09-10T21:26:15.537090+0000","last_unstale":"2024-09-10T21:26:15.549974+0000","last_undegraded":"2024-09-10T21:26:15.549974+0000","last_fullsized":"2024-09-10T21:26:15.549974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546287+0000","last_change":"2024-09-10T21:26:15.541138+0000","last_active":"2024-09-10T21:26:15.546287+0000","last_peered":"2024-09-10T21:26:15.546287+0000","last_clean":"2024-09-10T21:26:15.546287+0000","last_became_active":"2024-09-10T21:26:15.540540+0000","last_became_peered":"2024-09-10T21:26:15.540540+0000","last_unstale":"2024-09-10T21:26:15.546287+0000","last_undegraded":"2024-09-10T21:26:15.546287+0000","last_fullsized":"2024-09-10T21:26:15.546287+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:27.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550232+0000","last_change":"2024-09-10T21:26:15.544506+0000","last_active":"2024-09-10T21:26:15.550232+0000","last_peered":"2024-09-10T21:26:15.550232+0000","last_clean":"2024-09-10T21:26:15.550232+0000","last_became_active":"2024-09-10T21:26:15.544258+0000","last_became_peered":"2024-09-10T21:26:15.544258+0000","last_unstale":"2024-09-10T21:26:15.550232+0000","last_undegraded":"2024-09-10T21:26:15.550232+0000","last_fullsized":"2024-09-10T21:26:15.550232+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547473+0000","last_change":"2024-09-10T21:26:15.542753+0000","last_active":"2024-09-10T21:26:15.547473+0000","last_peered":"2024-09-10T21:26:15.547473+0000","last_clean":"2024-09-10T21:26:15.547473+0000","last_became_active":"2024-09-10T21:26:15.542492+0000","last_became_peered":"2024-09-10T21:26:15.542492+0000","last_unstale":"2024-09-10T21:26:15.547473+0000","last_undegraded":"2024-09-10T21:26:15.547473+0000","last_fullsized":"2024-09-10T21:26:15.547473+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:02.922870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550251+0000","last_change":"2024-09-10T21:26:14.530389+0000","last_active":"2024-09-10T21:26:15.550251+0000","last_peered":"2024-09-10T21:26:15.550251+0000","last_clean":"2024-09-10T21:26:15.550251+0000","last_became_active":"2024-09-10T21:26:14.530024+0000","last_became_peered":"2024-09-10T21:26:14.530024+0000","last_unstale":"2024-09-10T21:26:15.550251+0000","last_undegraded":"2024-09-10T21:26:15.550251+0000","last_fullsized":"2024-09-10T21:26:15.550251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:44.886118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550140+0000","last_change":"2024-09-10T21:26:15.546106+0000","last_active":"2024-09-10T21:26:15.550140+0000","last_peered":"2024-09-10T21:26:15.550140+0000","last_clean":"2024-09-10T21:26:15.550140+0000","last_became_active":"2024-09-10T21:26:15.545789+0000","last_became_peered":"2024-09-10T21:26:15.545789+0000","last_unstale":"2024-09-10T21:26:15.550140+0000","last_undegraded":"2024-09-10T21:26:15.550140+0000","last_fullsized":"2024-09-10T21:26:15.550140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:20.084655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.537228+0000","last_change":"2024-09-10T21:26:14.533996+0000","last_active":"2024-09-10T21:26:15.537228+0000","last_peered":"2024-09-10T21:26:15.537228+0000","last_clean":"2024-09-10T21:26:15.537228+0000","last_became_active":"2024-09-10T21:26:14.533618+0000","last_became_peered":"2024-09-10T21:26:14.533618+0000","last_unstale":"2024-09-10T21:26:15.537228+0000","last_undegraded":"2024-09-10T21:26:15.537228+0000","last_fullsized":"2024-09-10T21:26:15.537228+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:43.185675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546302+0000","last_change":"2024-09-10T21:26:15.541441+0000","last_active":"2024-09-10T21:26:15.546302+0000","last_peered":"2024-09-10T21:26:15.546302+0000","last_clean":"2024-09-10T21:26:15.546302+0000","last_became_active":"2024-09-10T21:26:15.540879+0000","last_became_peered":"2024-09-10T21:26:15.540879+0000","last_unstale":"2024-09-10T21:26:15.546302+0000","last_undegraded":"2024-09-10T21:26:15.546302+0000","last_fullsized":"2024-09-10T21:26:15.546302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:37.706884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545342+0000","last_change":"2024-09-10T21:26:14.529412+0000","last_active":"2024-09-10T21:26:15.545342+0000","last_peered":"2024-09-10T21:26:15.545342+0000","last_clean":"2024-09-10T21:26:15.545342+0000","last_became_active":"2024-09-10T21:26:14.529112+0000","last_became_peered":"2024-09-10T21:26:14.529112+0000","last_unstale":"2024-09-10T21:26:15.545342+0000","last_undegraded":"2024-09-10T21:26:15.545342+0000","last_fullsized":"2024-09-10T21:26:15.545342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:17.649740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548940+0000","last_change":"2024-09-10T21:26:15.537699+0000","last_active":"2024-09-10T21:26:15.548940+0000","last_peered":"2024-09-10T21:26:15.548940+0000","last_clean":"2024-09-10T21:26:15.548940+0000","last_became_active":"2024-09-10T21:26:15.533943+0000","last_became_peered":"2024-09-10T21:26:15.533943+0000","last_unstale":"2024-09-10T21:26:15.548940+0000","last_undegraded":"2024-09-10T21:26:15.548940+0000","last_fullsized":"2024-09-10T21:26:15.548940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:27.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538486+0000","last_change":"2024-09-10T21:26:14.529444+0000","last_active":"2024-09-10T21:26:15.538486+0000","last_peered":"2024-09-10T21:26:15.538486+0000","last_clean":"2024-09-10T21:26:15.538486+0000","last_became_active":"2024-09-10T21:26:14.529049+0000","last_became_peered":"2024-09-10T21:26:14.529049+0000","last_unstale":"2024-09-10T21:26:15.538486+0000","last_undegraded":"2024-09-10T21:26:15.538486+0000","last_fullsized":"2024-09-10T21:26:15.538486+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:52.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541439+0000","last_change":"2024-09-10T21:26:15.536274+0000","last_active":"2024-09-10T21:26:15.541439+0000","last_peered":"2024-09-10T21:26:15.541439+0000","last_clean":"2024-09-10T21:26:15.541439+0000","last_became_active":"2024-09-10T21:26:15.536085+0000","last_became_peered":"2024-09-10T21:26:15.536085+0000","last_unstale":"2024-09-10T21:26:15.541439+0000","last_undegraded":"2024-09-10T21:26:15.541439+0000","last_fullsized":"2024-09-10T21:26:15.541439+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:10.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544943+0000","last_change":"2024-09-10T21:26:15.540471+0000","last_active":"2024-09-10T21:26:15.544943+0000","last_peered":"2024-09-10T21:26:15.544943+0000","last_clean":"2024-09-10T21:26:15.544943+0000","last_became_active":"2024-09-10T21:26:15.539374+0000","last_became_peered":"2024-09-10T21:26:15.539374+0000","last_unstale":"2024-09-10T21:26:15.544943+0000","last_undegraded":"2024-09-10T21:26:15.544943+0000","last_fullsized":"2024-09-10T21:26:15.544943+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:53.409133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539556+0000","last_change":"2024-09-10T21:26:14.528783+0000","last_active":"2024-09-10T21:26:15.539556+0000","last_peered":"2024-09-10T21:26:15.539556+0000","last_clean":"2024-09-10T21:26:15.539556+0000","last_became_active":"2024-09-10T21:26:14.528509+0000","last_became_peered":"2024-09-10T21:26:14.528509+0000","last_unstale":"2024-09-10T21:26:15.539556+0000","last_undegraded":"2024-09-10T21:26:15.539556+0000","last_fullsized":"2024-09-10T21:26:15.539556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:48.911075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549136+0000","last_change":"2024-09-10T21:26:15.540735+0000","last_active":"2024-09-10T21:26:15.549136+0000","last_peered":"2024-09-10T21:26:15.549136+0000","last_clean":"2024-09-10T21:26:15.549136+0000","last_became_active":"2024-09-10T21:26:15.538227+0000","last_became_peered":"2024-09-10T21:26:15.538227+0000","last_unstale":"2024-09-10T21:26:15.549136+0000","last_undegraded":"2024-09-10T21:26:15.549136+0000","last_fullsized":"2024-09-10T21:26:15.549136+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:27.252740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533460+0000","last_change":"2024-09-10T21:26:14.527343+0000","last_active":"2024-09-10T21:26:15.533460+0000","last_peered":"2024-09-10T21:26:15.533460+0000","last_clean":"2024-09-10T21:26:15.533460+0000","last_became_active":"2024-09-10T21:26:14.527053+0000","last_became_peered":"2024-09-10T21:26:14.527053+0000","last_unstale":"2024-09-10T21:26:15.533460+0000","last_undegraded":"2024-09-10T21:26:15.533460+0000","last_fullsized":"2024-09-10T21:26:15.533460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:49.564562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.528756+0000","last_change":"2024-09-10T21:26:15.519138+0000","last_active":"2024-09-10T21:26:15.528756+0000","last_peered":"2024-09-10T21:26:15.528756+0000","last_clean":"2024-09-10T21:26:15.528756+0000","last_became_active":"2024-09-10T21:26:15.518903+0000","last_became_peered":"2024-09-10T21:26:15.518903+0000","last_unstale":"2024-09-10T21:26:15.528756+0000","last_undegraded":"2024-09-10T21:26:15.528756+0000","last_fullsized":"2024-09-10T21:26:15.528756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:24.898753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533518+0000","last_change":"2024-09-10T21:26:15.528432+0000","last_active":"2024-09-10T21:26:15.533518+0000","last_peered":"2024-09-10T21:26:15.533518+0000","last_clean":"2024-09-10T21:26:15.533518+0000","last_became_active":"2024-09-10T21:26:15.526149+0000","last_became_peered":"2024-09-10T21:26:15.526149+0000","last_unstale":"2024-09-10T21:26:15.533518+0000","last_undegraded":"2024-09-10T21:26:15.533518+0000","last_fullsized":"2024-09-10T21:26:15.533518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511121+0000","last_change":"2024-09-10T21:26:14.522970+0000","last_active":"2024-09-10T21:26:15.511121+0000","last_peered":"2024-09-10T21:26:15.511121+0000","last_clean":"2024-09-10T21:26:15.511121+0000","last_became_active":"2024-09-10T21:26:14.522696+0000","last_became_peered":"2024-09-10T21:26:14.522696+0000","last_unstale":"2024-09-10T21:26:15.511121+0000","last_undegraded":"2024-09-10T21:26:15.511121+0000","last_fullsized":"2024-09-10T21:26:15.511121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:26.009061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542110+0000","last_change":"2024-09-10T21:26:15.537430+0000","last_active":"2024-09-10T21:26:15.542110+0000","last_peered":"2024-09-10T21:26:15.542110+0000","last_clean":"2024-09-10T21:26:15.542110+0000","last_became_active":"2024-09-10T21:26:15.537174+0000","last_became_peered":"2024-09-10T21:26:15.537174+0000","last_unstale":"2024-09-10T21:26:15.542110+0000","last_undegraded":"2024-09-10T21:26:15.542110+0000","last_fullsized":"2024-09-10T21:26:15.542110+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:18.652981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510906+0000","last_change":"2024-09-10T21:26:14.516829+0000","last_active":"2024-09-10T21:26:15.510906+0000","last_peered":"2024-09-10T21:26:15.510906+0000","last_clean":"2024-09-10T21:26:15.510906+0000","last_became_active":"2024-09-10T21:26:14.516564+0000","last_became_peered":"2024-09-10T21:26:14.516564+0000","last_unstale":"2024-09-10T21:26:15.510906+0000","last_undegraded":"2024-09-10T21:26:15.510906+0000","last_fullsized":"2024-09-10T21:26:15.510906+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:32.378883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549908+0000","last_change":"2024-09-10T21:26:15.545818+0000","last_active":"2024-09-10T21:26:15.549908+0000","last_peered":"2024-09-10T21:26:15.549908+0000","last_clean":"2024-09-10T21:26:15.549908+0000","last_became_active":"2024-09-10T21:26:15.545383+0000","last_became_peered":"2024-09-10T21:26:15.545383+0000","last_unstale":"2024-09-10T21:26:15.549908+0000","last_undegraded":"2024-09-10T21:26:15.549908+0000","last_fullsized":"2024-09-10T21:26:15.549908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549760+0000","last_change":"2024-09-10T21:26:14.531817+0000","last_active":"2024-09-10T21:26:15.549760+0000","last_peered":"2024-09-10T21:26:15.549760+0000","last_clean":"2024-09-10T21:26:15.549760+0000","last_became_active":"2024-09-10T21:26:14.531387+0000","last_became_peered":"2024-09-10T21:26:14.531387+0000","last_unstale":"2024-09-10T21:26:15.549760+0000","last_undegraded":"2024-09-10T21:26:15.549760+0000","last_fullsized":"2024-09-10T21:26:15.549760+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:58.645631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547276+0000","last_change":"2024-09-10T21:26:15.543162+0000","last_active":"2024-09-10T21:26:15.547276+0000","last_peered":"2024-09-10T21:26:15.547276+0000","last_clean":"2024-09-10T21:26:15.547276+0000","last_became_active":"2024-09-10T21:26:15.542913+0000","last_became_peered":"2024-09-10T21:26:15.542913+0000","last_unstale":"2024-09-10T21:26:15.547276+0000","last_undegraded":"2024-09-10T21:26:15.547276+0000","last_fullsized":"2024-09-10T21:26:15.547276+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.553669+0000","last_change":"2024-09-10T21:26:14.533065+0000","last_active":"2024-09-10T21:26:15.553669+0000","last_peered":"2024-09-10T21:26:15.553669+0000","last_clean":"2024-09-10T21:26:15.553669+0000","last_became_active":"2024-09-10T21:26:14.532788+0000","last_became_peered":"2024-09-10T21:26:14.532788+0000","last_unstale":"2024-09-10T21:26:15.553669+0000","last_undegraded":"2024-09-10T21:26:15.553669+0000","last_fullsized":"2024-09-10T21:26:15.553669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:26.313137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541575+0000","last_change":"2024-09-10T21:26:15.540226+0000","last_active":"2024-09-10T21:26:15.541575+0000","last_peered":"2024-09-10T21:26:15.541575+0000","last_clean":"2024-09-10T21:26:15.541575+0000","last_became_active":"2024-09-10T21:26:15.539910+0000","last_became_peered":"2024-09-10T21:26:15.539910+0000","last_unstale":"2024-09-10T21:26:15.541575+0000","last_undegraded":"2024-09-10T21:26:15.541575+0000","last_fullsized":"2024-09-10T21:26:15.541575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:20.009997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538731+0000","last_change":"2024-09-10T21:26:14.526625+0000","last_active":"2024-09-10T21:26:15.538731+0000","last_peered":"2024-09-10T21:26:15.538731+0000","last_clean":"2024-09-10T21:26:15.538731+0000","last_became_active":"2024-09-10T21:26:14.526301+0000","last_became_peered":"2024-09-10T21:26:14.526301+0000","last_unstale":"2024-09-10T21:26:15.538731+0000","last_undegraded":"2024-09-10T21:26:15.538731+0000","last_fullsized":"2024-09-10T21:26:15.538731+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:36.925973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550279+0000","last_change":"2024-09-10T21:26:15.547448+0000","last_active":"2024-09-10T21:26:15.550279+0000","last_peered":"2024-09-10T21:26:15.550279+0000","last_clean":"2024-09-10T21:26:15.550279+0000","last_became_active":"2024-09-10T21:26:15.546869+0000","last_became_peered":"2024-09-10T21:26:15.546869+0000","last_unstale":"2024-09-10T21:26:15.550279+0000","last_undegraded":"2024-09-10T21:26:15.550279+0000","last_fullsized":"2024-09-10T21:26:15.550279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511264+0000","last_change":"2024-09-10T21:26:14.526509+0000","last_active":"2024-09-10T21:26:15.511264+0000","last_peered":"2024-09-10T21:26:15.511264+0000","last_clean":"2024-09-10T21:26:15.511264+0000","last_became_active":"2024-09-10T21:26:14.526211+0000","last_became_peered":"2024-09-10T21:26:14.526211+0000","last_unstale":"2024-09-10T21:26:15.511264+0000","last_undegraded":"2024-09-10T21:26:15.511264+0000","last_fullsized":"2024-09-10T21:26:15.511264+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:19.196183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539061+0000","last_change":"2024-09-10T21:26:15.533375+0000","last_active":"2024-09-10T21:26:15.539061+0000","last_peered":"2024-09-10T21:26:15.539061+0000","last_clean":"2024-09-10T21:26:15.539061+0000","last_became_active":"2024-09-10T21:26:15.533154+0000","last_became_peered":"2024-09-10T21:26:15.533154+0000","last_unstale":"2024-09-10T21:26:15.539061+0000","last_undegraded":"2024-09-10T21:26:15.539061+0000","last_fullsized":"2024-09-10T21:26:15.539061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:56.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.525900+0000","last_change":"2024-09-10T21:26:14.523579+0000","last_active":"2024-09-10T21:26:15.525900+0000","last_peered":"2024-09-10T21:26:15.525900+0000","last_clean":"2024-09-10T21:26:15.525900+0000","last_became_active":"2024-09-10T21:26:14.523321+0000","last_became_peered":"2024-09-10T21:26:14.523321+0000","last_unstale":"2024-09-10T21:26:15.525900+0000","last_undegraded":"2024-09-10T21:26:15.525900+0000","last_fullsized":"2024-09-10T21:26:15.525900+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:39.550410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538375+0000","last_change":"2024-09-10T21:26:14.526055+0000","last_active":"2024-09-10T21:26:15.538375+0000","last_peered":"2024-09-10T21:26:15.538375+0000","last_clean":"2024-09-10T21:26:15.538375+0000","last_became_active":"2024-09-10T21:26:14.525734+0000","last_became_peered":"2024-09-10T21:26:14.525734+0000","last_unstale":"2024-09-10T21:26:15.538375+0000","last_undegraded":"2024-09-10T21:26:15.538375+0000","last_fullsized":"2024-09-10T21:26:15.538375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:15.163346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549528+0000","last_change":"2024-09-10T21:26:15.545468+0000","last_active":"2024-09-10T21:26:15.549528+0000","last_peered":"2024-09-10T21:26:15.549528+0000","last_clean":"2024-09-10T21:26:15.549528+0000","last_became_active":"2024-09-10T21:26:15.545137+0000","last_became_peered":"2024-09-10T21:26:15.545137+0000","last_unstale":"2024-09-10T21:26:15.549528+0000","last_undegraded":"2024-09-10T21:26:15.549528+0000","last_fullsized":"2024-09-10T21:26:15.549528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:12.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538792+0000","last_change":"2024-09-10T21:26:14.534718+0000","last_active":"2024-09-10T21:26:15.538792+0000","last_peered":"2024-09-10T21:26:15.538792+0000","last_clean":"2024-09-10T21:26:15.538792+0000","last_became_active":"2024-09-10T21:26:14.534222+0000","last_became_peered":"2024-09-10T21:26:14.534222+0000","last_unstale":"2024-09-10T21:26:15.538792+0000","last_undegraded":"2024-09-10T21:26:15.538792+0000","last_fullsized":"2024-09-10T21:26:15.538792+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:10.810025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550202+0000","last_change":"2024-09-10T21:26:15.546824+0000","last_active":"2024-09-10T21:26:15.550202+0000","last_peered":"2024-09-10T21:26:15.550202+0000","last_clean":"2024-09-10T21:26:15.550202+0000","last_became_active":"2024-09-10T21:26:15.546286+0000","last_became_peered":"2024-09-10T21:26:15.546286+0000","last_unstale":"2024-09-10T21:26:15.550202+0000","last_undegraded":"2024-09-10T21:26:15.550202+0000","last_fullsized":"2024-09-10T21:26:15.550202+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:45.889359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533005+0000","last_change":"2024-09-10T21:26:14.527929+0000","last_active":"2024-09-10T21:26:15.533005+0000","last_peered":"2024-09-10T21:26:15.533005+0000","last_clean":"2024-09-10T21:26:15.533005+0000","last_became_active":"2024-09-10T21:26:14.527583+0000","last_became_peered":"2024-09-10T21:26:14.527583+0000","last_unstale":"2024-09-10T21:26:15.533005+0000","last_undegraded":"2024-09-10T21:26:15.533005+0000","last_fullsized":"2024-09-10T21:26:15.533005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:20.085776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545249+0000","last_change":"2024-09-10T21:26:15.541261+0000","last_active":"2024-09-10T21:26:15.545249+0000","last_peered":"2024-09-10T21:26:15.545249+0000","last_clean":"2024-09-10T21:26:15.545249+0000","last_became_active":"2024-09-10T21:26:15.541018+0000","last_became_peered":"2024-09-10T21:26:15.541018+0000","last_unstale":"2024-09-10T21:26:15.545249+0000","last_undegraded":"2024-09-10T21:26:15.545249+0000","last_fullsized":"2024-09-10T21:26:15.545249+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539384+0000","last_change":"2024-09-10T21:26:14.528002+0000","last_active":"2024-09-10T21:26:15.539384+0000","last_peered":"2024-09-10T21:26:15.539384+0000","last_clean":"2024-09-10T21:26:15.539384+0000","last_became_active":"2024-09-10T21:26:14.527640+0000","last_became_peered":"2024-09-10T21:26:14.527640+0000","last_unstale":"2024-09-10T21:26:15.539384+0000","last_undegraded":"2024-09-10T21:26:15.539384+0000","last_fullsized":"2024-09-10T21:26:15.539384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:40.905930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542762+0000","last_change":"2024-09-10T21:26:15.533913+0000","last_active":"2024-09-10T21:26:15.542762+0000","last_peered":"2024-09-10T21:26:15.542762+0000","last_clean":"2024-09-10T21:26:15.542762+0000","last_became_active":"2024-09-10T21:26:15.533607+0000","last_became_peered":"2024-09-10T21:26:15.533607+0000","last_unstale":"2024-09-10T21:26:15.542762+0000","last_undegraded":"2024-09-10T21:26:15.542762+0000","last_fullsized":"2024-09-10T21:26:15.542762+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:53.763483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524585+0000","last_change":"2024-09-10T21:26:14.530739+0000","last_active":"2024-09-10T21:26:15.524585+0000","last_peered":"2024-09-10T21:26:15.524585+0000","last_clean":"2024-09-10T21:26:15.524585+0000","last_became_active":"2024-09-10T21:26:14.530428+0000","last_became_peered":"2024-09-10T21:26:14.530428+0000","last_unstale":"2024-09-10T21:26:15.524585+0000","last_undegraded":"2024-09-10T21:26:15.524585+0000","last_fullsized":"2024-09-10T21:26:15.524585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:47.051011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547356+0000","last_change":"2024-09-10T21:26:15.543515+0000","last_active":"2024-09-10T21:26:15.547356+0000","last_peered":"2024-09-10T21:26:15.547356+0000","last_clean":"2024-09-10T21:26:15.547356+0000","last_became_active":"2024-09-10T21:26:15.543291+0000","last_became_peered":"2024-09-10T21:26:15.543291+0000","last_unstale":"2024-09-10T21:26:15.547356+0000","last_undegraded":"2024-09-10T21:26:15.547356+0000","last_fullsized":"2024-09-10T21:26:15.547356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551224+0000","last_change":"2024-09-10T21:26:14.532927+0000","last_active":"2024-09-10T21:26:15.551224+0000","last_peered":"2024-09-10T21:26:15.551224+0000","last_clean":"2024-09-10T21:26:15.551224+0000","last_became_active":"2024-09-10T21:26:14.532625+0000","last_became_peered":"2024-09-10T21:26:14.532625+0000","last_unstale":"2024-09-10T21:26:15.551224+0000","last_undegraded":"2024-09-10T21:26:15.551224+0000","last_fullsized":"2024-09-10T21:26:15.551224+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:36.431767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532586+0000","last_change":"2024-09-10T21:26:15.529708+0000","last_active":"2024-09-10T21:26:15.532586+0000","last_peered":"2024-09-10T21:26:15.532586+0000","last_clean":"2024-09-10T21:26:15.532586+0000","last_became_active":"2024-09-10T21:26:15.529431+0000","last_became_peered":"2024-09-10T21:26:15.529431+0000","last_unstale":"2024-09-10T21:26:15.532586+0000","last_undegraded":"2024-09-10T21:26:15.532586+0000","last_fullsized":"2024-09-10T21:26:15.532586+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:08.029645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549692+0000","last_change":"2024-09-10T21:26:15.544629+0000","last_active":"2024-09-10T21:26:15.549692+0000","last_peered":"2024-09-10T21:26:15.549692+0000","last_clean":"2024-09-10T21:26:15.549692+0000","last_became_active":"2024-09-10T21:26:15.540919+0000","last_became_peered":"2024-09-10T21:26:15.540919+0000","last_unstale":"2024-09-10T21:26:15.549692+0000","last_undegraded":"2024-09-10T21:26:15.549692+0000","last_fullsized":"2024-09-10T21:26:15.549692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520315+0000","last_change":"2024-09-10T21:26:14.529271+0000","last_active":"2024-09-10T21:26:15.520315+0000","last_peered":"2024-09-10T21:26:15.520315+0000","last_clean":"2024-09-10T21:26:15.520315+0000","last_became_active":"2024-09-10T21:26:14.528963+0000","last_became_peered":"2024-09-10T21:26:14.528963+0000","last_unstale":"2024-09-10T21:26:15.520315+0000","last_undegraded":"2024-09-10T21:26:15.520315+0000","last_fullsized":"2024-09-10T21:26:15.520315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:14.592093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544636+0000","last_change":"2024-09-10T21:26:15.540533+0000","last_active":"2024-09-10T21:26:15.544636+0000","last_peered":"2024-09-10T21:26:15.544636+0000","last_clean":"2024-09-10T21:26:15.544636+0000","last_became_active":"2024-09-10T21:26:15.540237+0000","last_became_peered":"2024-09-10T21:26:15.540237+0000","last_unstale":"2024-09-10T21:26:15.544636+0000","last_undegraded":"2024-09-10T21:26:15.544636+0000","last_fullsized":"2024-09-10T21:26:15.544636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:49.612063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520099+0000","last_change":"2024-09-10T21:26:14.529847+0000","last_active":"2024-09-10T21:26:15.520099+0000","last_peered":"2024-09-10T21:26:15.520099+0000","last_clean":"2024-09-10T21:26:15.520099+0000","last_became_active":"2024-09-10T21:26:14.529074+0000","last_became_peered":"2024-09-10T21:26:14.529074+0000","last_unstale":"2024-09-10T21:26:15.520099+0000","last_undegraded":"2024-09-10T21:26:15.520099+0000","last_fullsized":"2024-09-10T21:26:15.520099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:27.478486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532977+0000","last_change":"2024-09-10T21:26:15.528858+0000","last_active":"2024-09-10T21:26:15.532977+0000","last_peered":"2024-09-10T21:26:15.532977+0000","last_clean":"2024-09-10T21:26:15.532977+0000","last_became_active":"2024-09-10T21:26:15.527735+0000","last_became_peered":"2024-09-10T21:26:15.527735+0000","last_unstale":"2024-09-10T21:26:15.532977+0000","last_undegraded":"2024-09-10T21:26:15.532977+0000","last_fullsized":"2024-09-10T21:26:15.532977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:31.109223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544806+0000","last_change":"2024-09-10T21:26:14.529877+0000","last_active":"2024-09-10T21:26:15.544806+0000","last_peered":"2024-09-10T21:26:15.544806+0000","last_clean":"2024-09-10T21:26:15.544806+0000","last_became_active":"2024-09-10T21:26:14.529580+0000","last_became_peered":"2024-09-10T21:26:14.529580+0000","last_unstale":"2024-09-10T21:26:15.544806+0000","last_undegraded":"2024-09-10T21:26:15.544806+0000","last_fullsized":"2024-09-10T21:26:15.544806+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:41.737020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549409+0000","last_change":"2024-09-10T21:26:15.541706+0000","last_active":"2024-09-10T21:26:15.549409+0000","last_peered":"2024-09-10T21:26:15.549409+0000","last_clean":"2024-09-10T21:26:15.549409+0000","last_became_active":"2024-09-10T21:26:15.540365+0000","last_became_peered":"2024-09-10T21:26:15.540365+0000","last_unstale":"2024-09-10T21:26:15.549409+0000","last_undegraded":"2024-09-10T21:26:15.549409+0000","last_fullsized":"2024-09-10T21:26:15.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:43.648100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.530063+0000","last_change":"2024-09-10T21:26:14.532072+0000","last_active":"2024-09-10T21:26:15.530063+0000","last_peered":"2024-09-10T21:26:15.530063+0000","last_clean":"2024-09-10T21:26:15.530063+0000","last_became_active":"2024-09-10T21:26:14.530711+0000","last_became_peered":"2024-09-10T21:26:14.530711+0000","last_unstale":"2024-09-10T21:26:15.530063+0000","last_undegraded":"2024-09-10T21:26:15.530063+0000","last_fullsized":"2024-09-10T21:26:15.530063+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:15.362786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549756+0000","last_change":"2024-09-10T21:26:15.542738+0000","last_active":"2024-09-10T21:26:15.549756+0000","last_peered":"2024-09-10T21:26:15.549756+0000","last_clean":"2024-09-10T21:26:15.549756+0000","last_became_active":"2024-09-10T21:26:15.542497+0000","last_became_peered":"2024-09-10T21:26:15.542497+0000","last_unstale":"2024-09-10T21:26:15.549756+0000","last_undegraded":"2024-09-10T21:26:15.549756+0000","last_fullsized":"2024-09-10T21:26:15.549756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:59.431959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520974+0000","last_change":"2024-09-10T21:26:14.530559+0000","last_active":"2024-09-10T21:26:15.520974+0000","last_peered":"2024-09-10T21:26:15.520974+0000","last_clean":"2024-09-10T21:26:15.520974+0000","last_became_active":"2024-09-10T21:26:14.529513+0000","last_became_peered":"2024-09-10T21:26:14.529513+0000","last_unstale":"2024-09-10T21:26:15.520974+0000","last_undegraded":"2024-09-10T21:26:15.520974+0000","last_fullsized":"2024-09-10T21:26:15.520974+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:10.595223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538120+0000","last_change":"2024-09-10T21:26:15.533560+0000","last_active":"2024-09-10T21:26:15.538120+0000","last_peered":"2024-09-10T21:26:15.538120+0000","last_clean":"2024-09-10T21:26:15.538120+0000","last_became_active":"2024-09-10T21:26:15.533313+0000","last_became_peered":"2024-09-10T21:26:15.533313+0000","last_unstale":"2024-09-10T21:26:15.538120+0000","last_undegraded":"2024-09-10T21:26:15.538120+0000","last_fullsized":"2024-09-10T21:26:15.538120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548560+0000","last_change":"2024-09-10T21:26:15.536138+0000","last_active":"2024-09-10T21:26:15.548560+0000","last_peered":"2024-09-10T21:26:15.548560+0000","last_clean":"2024-09-10T21:26:15.548560+0000","last_became_active":"2024-09-10T21:26:15.535521+0000","last_became_peered":"2024-09-10T21:26:15.535521+0000","last_unstale":"2024-09-10T21:26:15.548560+0000","last_undegraded":"2024-09-10T21:26:15.548560+0000","last_fullsized":"2024-09-10T21:26:15.548560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:20.575951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.526019+0000","last_change":"2024-09-10T21:26:14.519898+0000","last_active":"2024-09-10T21:26:15.526019+0000","last_peered":"2024-09-10T21:26:15.526019+0000","last_clean":"2024-09-10T21:26:15.526019+0000","last_became_active":"2024-09-10T21:26:14.519606+0000","last_became_peered":"2024-09-10T21:26:14.519606+0000","last_unstale":"2024-09-10T21:26:15.526019+0000","last_undegraded":"2024-09-10T21:26:15.526019+0000","last_fullsized":"2024-09-10T21:26:15.526019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.608822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544613+0000","last_change":"2024-09-10T21:26:15.532747+0000","last_active":"2024-09-10T21:26:15.544613+0000","last_peered":"2024-09-10T21:26:15.544613+0000","last_clean":"2024-09-10T21:26:15.544613+0000","last_became_active":"2024-09-10T21:26:15.532482+0000","last_became_peered":"2024-09-10T21:26:15.532482+0000","last_unstale":"2024-09-10T21:26:15.544613+0000","last_undegraded":"2024-09-10T21:26:15.544613+0000","last_fullsized":"2024-09-10T21:26:15.544613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:40.553651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.509867+0000","last_change":"2024-09-10T21:26:14.526821+0000","last_active":"2024-09-10T21:26:15.509867+0000","last_peered":"2024-09-10T21:26:15.509867+0000","last_clean":"2024-09-10T21:26:15.509867+0000","last_became_active":"2024-09-10T21:26:14.526561+0000","last_became_peered":"2024-09-10T21:26:14.526561+0000","last_unstale":"2024-09-10T21:26:15.509867+0000","last_undegraded":"2024-09-10T21:26:15.509867+0000","last_fullsized":"2024-09-10T21:26:15.509867+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541912+0000","last_change":"2024-09-10T21:26:15.538996+0000","last_active":"2024-09-10T21:26:15.541912+0000","last_peered":"2024-09-10T21:26:15.541912+0000","last_clean":"2024-09-10T21:26:15.541912+0000","last_became_active":"2024-09-10T21:26:15.538393+0000","last_became_peered":"2024-09-10T21:26:15.538393+0000","last_unstale":"2024-09-10T21:26:15.541912+0000","last_undegraded":"2024-09-10T21:26:15.541912+0000","last_fullsized":"2024-09-10T21:26:15.541912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:33.376220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524045+0000","last_change":"2024-09-10T21:26:14.533819+0000","last_active":"2024-09-10T21:26:15.524045+0000","last_peered":"2024-09-10T21:26:15.524045+0000","last_clean":"2024-09-10T21:26:15.524045+0000","last_became_active":"2024-09-10T21:26:14.533554+0000","last_became_peered":"2024-09-10T21:26:14.533554+0000","last_unstale":"2024-09-10T21:26:15.524045+0000","last_undegraded":"2024-09-10T21:26:15.524045+0000","last_fullsized":"2024-09-10T21:26:15.524045+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:55.449678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510018+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:26:15.510018+0000","last_peered":"2024-09-10T21:26:15.510018+0000","last_clean":"2024-09-10T21:26:15.510018+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:26:15.510018+0000","last_undegraded":"2024-09-10T21:26:15.510018+0000","last_fullsized":"2024-09-10T21:26:15.510018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":275651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1216512,"data_stored":865963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":610304,"data_stored":268428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1204224,"data_stored":858796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27612,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":585728,"data_stored":255552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:26:19.638 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:26:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.815+0000 7ff710600640 1 -- 172.21.15.97:0/2770475345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f00052f0 msgr2=0x7ff6f000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.815+0000 7ff710600640 1 --2- 172.21.15.97:0/2770475345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f00052f0 0x7ff6f000b9d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff7000016f0 tx=0x7ff70001c7b0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.815+0000 7ff710600640 1 -- 172.21.15.97:0/2770475345 shutdown_connections 2024-09-10T21:26:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.815+0000 7ff710600640 1 --2- 172.21.15.97:0/2770475345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f00052f0 0x7ff6f000b9d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.815+0000 7ff710600640 1 -- 172.21.15.97:0/2770475345 >> 172.21.15.97:0/2770475345 conn(0x7ff7080b1780 msgr2=0x7ff7080b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.815+0000 7ff710600640 1 -- 172.21.15.97:0/2770475345 shutdown_connections 2024-09-10T21:26:19.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.815+0000 7ff710600640 1 -- 172.21.15.97:0/2770475345 wait complete. 2024-09-10T21:26:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 Processor -- start 2024-09-10T21:26:19.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 -- start start 2024-09-10T21:26:19.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f00052f0 0x7ff70813c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7081544f0 0x7ff7080afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff708154950 0x7ff7080b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080b2e10 con 0x7ff7081544f0 2024-09-10T21:26:19.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080b2c90 con 0x7ff708154950 2024-09-10T21:26:19.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff708144480 con 0x7ff6f00052f0 2024-09-10T21:26:19.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7081544f0 0x7ff7080afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7081544f0 0x7ff7080afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41610/0 (socket says 172.21.15.97:41610) 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 -- 172.21.15.97:0/2916370856 learned_addr learned my addr 172.21.15.97:0/2916370856 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70dc00640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f00052f0 0x7ff70813c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70e600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff708154950 0x7ff7080b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 -- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f00052f0 msgr2=0x7ff70813c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f00052f0 0x7ff70813c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 -- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff708154950 msgr2=0x7ff7080b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff708154950 0x7ff7080b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 -- 172.21.15.97:0/2916370856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7081f4150 con 0x7ff7081544f0 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70e600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff708154950 0x7ff7080b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:19.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff70d200640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7081544f0 0x7ff7080afe70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4000fc0 tx=0x7ff6f400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff705600640 1 -- 172.21.15.97:0/2916370856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4015070 con 0x7ff7081544f0 2024-09-10T21:26:19.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff705600640 1 -- 172.21.15.97:0/2916370856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6f400ca00 con 0x7ff7081544f0 2024-09-10T21:26:19.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff705600640 1 -- 172.21.15.97:0/2916370856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400ce30 con 0x7ff7081544f0 2024-09-10T21:26:19.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7081f4430 con 0x7ff7081544f0 2024-09-10T21:26:19.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7081f49f0 con 0x7ff7081544f0 2024-09-10T21:26:19.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.819+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7081f4bf0 con 0x7ff7081544f0 2024-09-10T21:26:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.823+0000 7ff705600640 1 -- 172.21.15.97:0/2916370856 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff6f4011460 con 0x7ff7081544f0 2024-09-10T21:26:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.823+0000 7ff705600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e8072610 0x7ff6e8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.823+0000 7ff705600640 1 -- 172.21.15.97:0/2916370856 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7ff6f4093a10 con 0x7ff7081544f0 2024-09-10T21:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.823+0000 7ff70dc00640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e8072610 0x7ff6e8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.823+0000 7ff70dc00640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e8072610 0x7ff6e8074ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009ec0 tx=0x7ff6fc009510 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.827+0000 7ff705600640 1 -- 172.21.15.97:0/2916370856 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7081f4bf0 con 0x7ff7081544f0 2024-09-10T21:26:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:19.995+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff708070470 con 0x7ff6e8072610 2024-09-10T21:26:20.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.003+0000 7ff705600640 1 -- 172.21.15.97:0/2916370856 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249758 (secure 0 0 0) 0x7ff708070470 con 0x7ff6e8072610 2024-09-10T21:26:20.014 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:20.015 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:26:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e8072610 msgr2=0x7ff6e8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e8072610 0x7ff6e8074ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009ec0 tx=0x7ff6fc009510 comp rx=0 tx=0).stop 2024-09-10T21:26:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7081544f0 msgr2=0x7ff7080afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7081544f0 0x7ff7080afe70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4000fc0 tx=0x7ff6f400ea90 comp rx=0 tx=0).stop 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 shutdown_connections 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e8072610 0x7ff6e8074ad0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff708154950 0x7ff7080b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7081544f0 0x7ff7080afe70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 --2- 172.21.15.97:0/2916370856 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f00052f0 0x7ff70813c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 >> 172.21.15.97:0/2916370856 conn(0x7ff7080b1780 msgr2=0x7ff7080ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 shutdown_connections 2024-09-10T21:26:20.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.007+0000 7ff710600640 1 -- 172.21.15.97:0/2916370856 wait complete. 2024-09-10T21:26:20.028 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-10T21:26:18.100132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":164,"acting":164,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222232,"kb_used_data":5984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752488,"statfs":{"total":773094113280,"available":772866547712,"internally_reserved":0,"allocated":6127616,"data_stored":3352429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.595487"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549183+0000","last_change":"2024-09-10T21:26:15.539541+0000","last_active":"2024-09-10T21:26:15.549183+0000","last_peered":"2024-09-10T21:26:15.549183+0000","last_clean":"2024-09-10T21:26:15.549183+0000","last_became_active":"2024-09-10T21:26:15.539095+0000","last_became_peered":"2024-09-10T21:26:15.539095+0000","last_unstale":"2024-09-10T21:26:15.549183+0000","last_undegraded":"2024-09-10T21:26:15.549183+0000","last_fullsized":"2024-09-10T21:26:15.549183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.543900+0000","last_change":"2024-09-10T21:26:15.535693+0000","last_active":"2024-09-10T21:26:15.543900+0000","last_peered":"2024-09-10T21:26:15.543900+0000","last_clean":"2024-09-10T21:26:15.543900+0000","last_became_active":"2024-09-10T21:26:15.535421+0000","last_became_peered":"2024-09-10T21:26:15.535421+0000","last_unstale":"2024-09-10T21:26:15.543900+0000","last_undegraded":"2024-09-10T21:26:15.543900+0000","last_fullsized":"2024-09-10T21:26:15.543900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546167+0000","last_change":"2024-09-10T21:26:15.537322+0000","last_active":"2024-09-10T21:26:15.546167+0000","last_peered":"2024-09-10T21:26:15.546167+0000","last_clean":"2024-09-10T21:26:15.546167+0000","last_became_active":"2024-09-10T21:26:15.537039+0000","last_became_peered":"2024-09-10T21:26:15.537039+0000","last_unstale":"2024-09-10T21:26:15.546167+0000","last_undegraded":"2024-09-10T21:26:15.546167+0000","last_fullsized":"2024-09-10T21:26:15.546167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:11.598464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551006+0000","last_change":"2024-09-10T21:26:15.547237+0000","last_active":"2024-09-10T21:26:15.551006+0000","last_peered":"2024-09-10T21:26:15.551006+0000","last_clean":"2024-09-10T21:26:15.551006+0000","last_became_active":"2024-09-10T21:26:15.546563+0000","last_became_peered":"2024-09-10T21:26:15.546563+0000","last_unstale":"2024-09-10T21:26:15.551006+0000","last_undegraded":"2024-09-10T21:26:15.551006+0000","last_fullsized":"2024-09-10T21:26:15.551006+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544695+0000","last_change":"2024-09-10T21:26:15.534547+0000","last_active":"2024-09-10T21:26:15.544695+0000","last_peered":"2024-09-10T21:26:15.544695+0000","last_clean":"2024-09-10T21:26:15.544695+0000","last_became_active":"2024-09-10T21:26:15.534280+0000","last_became_peered":"2024-09-10T21:26:15.534280+0000","last_unstale":"2024-09-10T21:26:15.544695+0000","last_undegraded":"2024-09-10T21:26:15.544695+0000","last_fullsized":"2024-09-10T21:26:15.544695+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:48.054252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548817+0000","last_change":"2024-09-10T21:26:15.533183+0000","last_active":"2024-09-10T21:26:15.548817+0000","last_peered":"2024-09-10T21:26:15.548817+0000","last_clean":"2024-09-10T21:26:15.548817+0000","last_became_active":"2024-09-10T21:26:15.532995+0000","last_became_peered":"2024-09-10T21:26:15.532995+0000","last_unstale":"2024-09-10T21:26:15.548817+0000","last_undegraded":"2024-09-10T21:26:15.548817+0000","last_fullsized":"2024-09-10T21:26:15.548817+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:27.005167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546214+0000","last_change":"2024-09-10T21:26:15.541612+0000","last_active":"2024-09-10T21:26:15.546214+0000","last_peered":"2024-09-10T21:26:15.546214+0000","last_clean":"2024-09-10T21:26:15.546214+0000","last_became_active":"2024-09-10T21:26:15.541343+0000","last_became_peered":"2024-09-10T21:26:15.541343+0000","last_unstale":"2024-09-10T21:26:15.546214+0000","last_undegraded":"2024-09-10T21:26:15.546214+0000","last_fullsized":"2024-09-10T21:26:15.546214+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548595+0000","last_change":"2024-09-10T21:26:15.536440+0000","last_active":"2024-09-10T21:26:15.548595+0000","last_peered":"2024-09-10T21:26:15.548595+0000","last_clean":"2024-09-10T21:26:15.548595+0000","last_became_active":"2024-09-10T21:26:15.535896+0000","last_became_peered":"2024-09-10T21:26:15.535896+0000","last_unstale":"2024-09-10T21:26:15.548595+0000","last_undegraded":"2024-09-10T21:26:15.548595+0000","last_fullsized":"2024-09-10T21:26:15.548595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:57.577430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549667+0000","last_change":"2024-09-10T21:26:15.538325+0000","last_active":"2024-09-10T21:26:15.549667+0000","last_peered":"2024-09-10T21:26:15.549667+0000","last_clean":"2024-09-10T21:26:15.549667+0000","last_became_active":"2024-09-10T21:26:15.536197+0000","last_became_peered":"2024-09-10T21:26:15.536197+0000","last_unstale":"2024-09-10T21:26:15.549667+0000","last_undegraded":"2024-09-10T21:26:15.549667+0000","last_fullsized":"2024-09-10T21:26:15.549667+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520191+0000","last_change":"2024-09-10T21:26:15.518220+0000","last_active":"2024-09-10T21:26:15.520191+0000","last_peered":"2024-09-10T21:26:15.520191+0000","last_clean":"2024-09-10T21:26:15.520191+0000","last_became_active":"2024-09-10T21:26:15.516898+0000","last_became_peered":"2024-09-10T21:26:15.516898+0000","last_unstale":"2024-09-10T21:26:15.520191+0000","last_undegraded":"2024-09-10T21:26:15.520191+0000","last_fullsized":"2024-09-10T21:26:15.520191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.482589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547572+0000","last_change":"2024-09-10T21:26:15.545455+0000","last_active":"2024-09-10T21:26:15.547572+0000","last_peered":"2024-09-10T21:26:15.547572+0000","last_clean":"2024-09-10T21:26:15.547572+0000","last_became_active":"2024-09-10T21:26:15.545207+0000","last_became_peered":"2024-09-10T21:26:15.545207+0000","last_unstale":"2024-09-10T21:26:15.547572+0000","last_undegraded":"2024-09-10T21:26:15.547572+0000","last_fullsized":"2024-09-10T21:26:15.547572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549843+0000","last_change":"2024-09-10T21:26:15.545646+0000","last_active":"2024-09-10T21:26:15.549843+0000","last_peered":"2024-09-10T21:26:15.549843+0000","last_clean":"2024-09-10T21:26:15.549843+0000","last_became_active":"2024-09-10T21:26:15.544941+0000","last_became_peered":"2024-09-10T21:26:15.544941+0000","last_unstale":"2024-09-10T21:26:15.549843+0000","last_undegraded":"2024-09-10T21:26:15.549843+0000","last_fullsized":"2024-09-10T21:26:15.549843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542087+0000","last_change":"2024-09-10T21:26:15.536666+0000","last_active":"2024-09-10T21:26:15.542087+0000","last_peered":"2024-09-10T21:26:15.542087+0000","last_clean":"2024-09-10T21:26:15.542087+0000","last_became_active":"2024-09-10T21:26:15.536410+0000","last_became_peered":"2024-09-10T21:26:15.536410+0000","last_unstale":"2024-09-10T21:26:15.542087+0000","last_undegraded":"2024-09-10T21:26:15.542087+0000","last_fullsized":"2024-09-10T21:26:15.542087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:15.595334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.529423+0000","last_change":"2024-09-10T21:26:15.521400+0000","last_active":"2024-09-10T21:26:15.529423+0000","last_peered":"2024-09-10T21:26:15.529423+0000","last_clean":"2024-09-10T21:26:15.529423+0000","last_became_active":"2024-09-10T21:26:15.520076+0000","last_became_peered":"2024-09-10T21:26:15.520076+0000","last_unstale":"2024-09-10T21:26:15.529423+0000","last_undegraded":"2024-09-10T21:26:15.529423+0000","last_fullsized":"2024-09-10T21:26:15.529423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:13.921295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549231+0000","last_change":"2024-09-10T21:26:15.544245+0000","last_active":"2024-09-10T21:26:15.549231+0000","last_peered":"2024-09-10T21:26:15.549231+0000","last_clean":"2024-09-10T21:26:15.549231+0000","last_became_active":"2024-09-10T21:26:15.539109+0000","last_became_peered":"2024-09-10T21:26:15.539109+0000","last_unstale":"2024-09-10T21:26:15.549231+0000","last_undegraded":"2024-09-10T21:26:15.549231+0000","last_fullsized":"2024-09-10T21:26:15.549231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.531781+0000","last_change":"2024-09-10T21:26:15.520766+0000","last_active":"2024-09-10T21:26:15.531781+0000","last_peered":"2024-09-10T21:26:15.531781+0000","last_clean":"2024-09-10T21:26:15.531781+0000","last_became_active":"2024-09-10T21:26:15.520453+0000","last_became_peered":"2024-09-10T21:26:15.520453+0000","last_unstale":"2024-09-10T21:26:15.531781+0000","last_undegraded":"2024-09-10T21:26:15.531781+0000","last_fullsized":"2024-09-10T21:26:15.531781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:58.808915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545207+0000","last_change":"2024-09-10T21:26:15.540350+0000","last_active":"2024-09-10T21:26:15.545207+0000","last_peered":"2024-09-10T21:26:15.545207+0000","last_clean":"2024-09-10T21:26:15.545207+0000","last_became_active":"2024-09-10T21:26:15.535466+0000","last_became_peered":"2024-09-10T21:26:15.535466+0000","last_unstale":"2024-09-10T21:26:15.545207+0000","last_undegraded":"2024-09-10T21:26:15.545207+0000","last_fullsized":"2024-09-10T21:26:15.545207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542035+0000","last_change":"2024-09-10T21:26:15.534826+0000","last_active":"2024-09-10T21:26:15.542035+0000","last_peered":"2024-09-10T21:26:15.542035+0000","last_clean":"2024-09-10T21:26:15.542035+0000","last_became_active":"2024-09-10T21:26:15.534533+0000","last_became_peered":"2024-09-10T21:26:15.534533+0000","last_unstale":"2024-09-10T21:26:15.542035+0000","last_undegraded":"2024-09-10T21:26:15.542035+0000","last_fullsized":"2024-09-10T21:26:15.542035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:28.481727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549208+0000","last_change":"2024-09-10T21:26:15.539933+0000","last_active":"2024-09-10T21:26:15.549208+0000","last_peered":"2024-09-10T21:26:15.549208+0000","last_clean":"2024-09-10T21:26:15.549208+0000","last_became_active":"2024-09-10T21:26:15.539551+0000","last_became_peered":"2024-09-10T21:26:15.539551+0000","last_unstale":"2024-09-10T21:26:15.549208+0000","last_undegraded":"2024-09-10T21:26:15.549208+0000","last_fullsized":"2024-09-10T21:26:15.549208+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546220+0000","last_change":"2024-09-10T21:26:15.541772+0000","last_active":"2024-09-10T21:26:15.546220+0000","last_peered":"2024-09-10T21:26:15.546220+0000","last_clean":"2024-09-10T21:26:15.546220+0000","last_became_active":"2024-09-10T21:26:15.541512+0000","last_became_peered":"2024-09-10T21:26:15.541512+0000","last_unstale":"2024-09-10T21:26:15.546220+0000","last_undegraded":"2024-09-10T21:26:15.546220+0000","last_fullsized":"2024-09-10T21:26:15.546220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550226+0000","last_change":"2024-09-10T21:26:15.545746+0000","last_active":"2024-09-10T21:26:15.550226+0000","last_peered":"2024-09-10T21:26:15.550226+0000","last_clean":"2024-09-10T21:26:15.550226+0000","last_became_active":"2024-09-10T21:26:15.545498+0000","last_became_peered":"2024-09-10T21:26:15.545498+0000","last_unstale":"2024-09-10T21:26:15.550226+0000","last_undegraded":"2024-09-10T21:26:15.550226+0000","last_fullsized":"2024-09-10T21:26:15.550226+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549974+0000","last_change":"2024-09-10T21:26:15.546272+0000","last_active":"2024-09-10T21:26:15.549974+0000","last_peered":"2024-09-10T21:26:15.549974+0000","last_clean":"2024-09-10T21:26:15.549974+0000","last_became_active":"2024-09-10T21:26:15.537090+0000","last_became_peered":"2024-09-10T21:26:15.537090+0000","last_unstale":"2024-09-10T21:26:15.549974+0000","last_undegraded":"2024-09-10T21:26:15.549974+0000","last_fullsized":"2024-09-10T21:26:15.549974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546287+0000","last_change":"2024-09-10T21:26:15.541138+0000","last_active":"2024-09-10T21:26:15.546287+0000","last_peered":"2024-09-10T21:26:15.546287+0000","last_clean":"2024-09-10T21:26:15.546287+0000","last_became_active":"2024-09-10T21:26:15.540540+0000","last_became_peered":"2024-09-10T21:26:15.540540+0000","last_unstale":"2024-09-10T21:26:15.546287+0000","last_undegraded":"2024-09-10T21:26:15.546287+0000","last_fullsized":"2024-09-10T21:26:15.546287+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:27.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550232+0000","last_change":"2024-09-10T21:26:15.544506+0000","last_active":"2024-09-10T21:26:15.550232+0000","last_peered":"2024-09-10T21:26:15.550232+0000","last_clean":"2024-09-10T21:26:15.550232+0000","last_became_active":"2024-09-10T21:26:15.544258+0000","last_became_peered":"2024-09-10T21:26:15.544258+0000","last_unstale":"2024-09-10T21:26:15.550232+0000","last_undegraded":"2024-09-10T21:26:15.550232+0000","last_fullsized":"2024-09-10T21:26:15.550232+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547473+0000","last_change":"2024-09-10T21:26:15.542753+0000","last_active":"2024-09-10T21:26:15.547473+0000","last_peered":"2024-09-10T21:26:15.547473+0000","last_clean":"2024-09-10T21:26:15.547473+0000","last_became_active":"2024-09-10T21:26:15.542492+0000","last_became_peered":"2024-09-10T21:26:15.542492+0000","last_unstale":"2024-09-10T21:26:15.547473+0000","last_undegraded":"2024-09-10T21:26:15.547473+0000","last_fullsized":"2024-09-10T21:26:15.547473+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:02.922870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550251+0000","last_change":"2024-09-10T21:26:14.530389+0000","last_active":"2024-09-10T21:26:15.550251+0000","last_peered":"2024-09-10T21:26:15.550251+0000","last_clean":"2024-09-10T21:26:15.550251+0000","last_became_active":"2024-09-10T21:26:14.530024+0000","last_became_peered":"2024-09-10T21:26:14.530024+0000","last_unstale":"2024-09-10T21:26:15.550251+0000","last_undegraded":"2024-09-10T21:26:15.550251+0000","last_fullsized":"2024-09-10T21:26:15.550251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:44.886118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550140+0000","last_change":"2024-09-10T21:26:15.546106+0000","last_active":"2024-09-10T21:26:15.550140+0000","last_peered":"2024-09-10T21:26:15.550140+0000","last_clean":"2024-09-10T21:26:15.550140+0000","last_became_active":"2024-09-10T21:26:15.545789+0000","last_became_peered":"2024-09-10T21:26:15.545789+0000","last_unstale":"2024-09-10T21:26:15.550140+0000","last_undegraded":"2024-09-10T21:26:15.550140+0000","last_fullsized":"2024-09-10T21:26:15.550140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:20.084655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.537228+0000","last_change":"2024-09-10T21:26:14.533996+0000","last_active":"2024-09-10T21:26:15.537228+0000","last_peered":"2024-09-10T21:26:15.537228+0000","last_clean":"2024-09-10T21:26:15.537228+0000","last_became_active":"2024-09-10T21:26:14.533618+0000","last_became_peered":"2024-09-10T21:26:14.533618+0000","last_unstale":"2024-09-10T21:26:15.537228+0000","last_undegraded":"2024-09-10T21:26:15.537228+0000","last_fullsized":"2024-09-10T21:26:15.537228+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:43.185675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546302+0000","last_change":"2024-09-10T21:26:15.541441+0000","last_active":"2024-09-10T21:26:15.546302+0000","last_peered":"2024-09-10T21:26:15.546302+0000","last_clean":"2024-09-10T21:26:15.546302+0000","last_became_active":"2024-09-10T21:26:15.540879+0000","last_became_peered":"2024-09-10T21:26:15.540879+0000","last_unstale":"2024-09-10T21:26:15.546302+0000","last_undegraded":"2024-09-10T21:26:15.546302+0000","last_fullsized":"2024-09-10T21:26:15.546302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:37.706884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545342+0000","last_change":"2024-09-10T21:26:14.529412+0000","last_active":"2024-09-10T21:26:15.545342+0000","last_peered":"2024-09-10T21:26:15.545342+0000","last_clean":"2024-09-10T21:26:15.545342+0000","last_became_active":"2024-09-10T21:26:14.529112+0000","last_became_peered":"2024-09-10T21:26:14.529112+0000","last_unstale":"2024-09-10T21:26:15.545342+0000","last_undegraded":"2024-09-10T21:26:15.545342+0000","last_fullsized":"2024-09-10T21:26:15.545342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:17.649740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548940+0000","last_change":"2024-09-10T21:26:15.537699+0000","last_active":"2024-09-10T21:26:15.548940+0000","last_peered":"2024-09-10T21:26:15.548940+0000","last_clean":"2024-09-10T21:26:15.548940+0000","last_became_active":"2024-09-10T21:26:15.533943+0000","last_became_peered":"2024-09-10T21:26:15.533943+0000","last_unstale":"2024-09-10T21:26:15.548940+0000","last_undegraded":"2024-09-10T21:26:15.548940+0000","last_fullsized":"2024-09-10T21:26:15.548940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:27.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538486+0000","last_change":"2024-09-10T21:26:14.529444+0000","last_active":"2024-09-10T21:26:15.538486+0000","last_peered":"2024-09-10T21:26:15.538486+0000","last_clean":"2024-09-10T21:26:15.538486+0000","last_became_active":"2024-09-10T21:26:14.529049+0000","last_became_peered":"2024-09-10T21:26:14.529049+0000","last_unstale":"2024-09-10T21:26:15.538486+0000","last_undegraded":"2024-09-10T21:26:15.538486+0000","last_fullsized":"2024-09-10T21:26:15.538486+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:52.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541439+0000","last_change":"2024-09-10T21:26:15.536274+0000","last_active":"2024-09-10T21:26:15.541439+0000","last_peered":"2024-09-10T21:26:15.541439+0000","last_clean":"2024-09-10T21:26:15.541439+0000","last_became_active":"2024-09-10T21:26:15.536085+0000","last_became_peered":"2024-09-10T21:26:15.536085+0000","last_unstale":"2024-09-10T21:26:15.541439+0000","last_undegraded":"2024-09-10T21:26:15.541439+0000","last_fullsized":"2024-09-10T21:26:15.541439+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:10.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544943+0000","last_change":"2024-09-10T21:26:15.540471+0000","last_active":"2024-09-10T21:26:15.544943+0000","last_peered":"2024-09-10T21:26:15.544943+0000","last_clean":"2024-09-10T21:26:15.544943+0000","last_became_active":"2024-09-10T21:26:15.539374+0000","last_became_peered":"2024-09-10T21:26:15.539374+0000","last_unstale":"2024-09-10T21:26:15.544943+0000","last_undegraded":"2024-09-10T21:26:15.544943+0000","last_fullsized":"2024-09-10T21:26:15.544943+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:53.409133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539556+0000","last_change":"2024-09-10T21:26:14.528783+0000","last_active":"2024-09-10T21:26:15.539556+0000","last_peered":"2024-09-10T21:26:15.539556+0000","last_clean":"2024-09-10T21:26:15.539556+0000","last_became_active":"2024-09-10T21:26:14.528509+0000","last_became_peered":"2024-09-10T21:26:14.528509+0000","last_unstale":"2024-09-10T21:26:15.539556+0000","last_undegraded":"2024-09-10T21:26:15.539556+0000","last_fullsized":"2024-09-10T21:26:15.539556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:48.911075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549136+0000","last_change":"2024-09-10T21:26:15.540735+0000","last_active":"2024-09-10T21:26:15.549136+0000","last_peered":"2024-09-10T21:26:15.549136+0000","last_clean":"2024-09-10T21:26:15.549136+0000","last_became_active":"2024-09-10T21:26:15.538227+0000","last_became_peered":"2024-09-10T21:26:15.538227+0000","last_unstale":"2024-09-10T21:26:15.549136+0000","last_undegraded":"2024-09-10T21:26:15.549136+0000","last_fullsized":"2024-09-10T21:26:15.549136+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:27.252740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533460+0000","last_change":"2024-09-10T21:26:14.527343+0000","last_active":"2024-09-10T21:26:15.533460+0000","last_peered":"2024-09-10T21:26:15.533460+0000","last_clean":"2024-09-10T21:26:15.533460+0000","last_became_active":"2024-09-10T21:26:14.527053+0000","last_became_peered":"2024-09-10T21:26:14.527053+0000","last_unstale":"2024-09-10T21:26:15.533460+0000","last_undegraded":"2024-09-10T21:26:15.533460+0000","last_fullsized":"2024-09-10T21:26:15.533460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:49.564562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.528756+0000","last_change":"2024-09-10T21:26:15.519138+0000","last_active":"2024-09-10T21:26:15.528756+0000","last_peered":"2024-09-10T21:26:15.528756+0000","last_clean":"2024-09-10T21:26:15.528756+0000","last_became_active":"2024-09-10T21:26:15.518903+0000","last_became_peered":"2024-09-10T21:26:15.518903+0000","last_unstale":"2024-09-10T21:26:15.528756+0000","last_undegraded":"2024-09-10T21:26:15.528756+0000","last_fullsized":"2024-09-10T21:26:15.528756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:24.898753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533518+0000","last_change":"2024-09-10T21:26:15.528432+0000","last_active":"2024-09-10T21:26:15.533518+0000","last_peered":"2024-09-10T21:26:15.533518+0000","last_clean":"2024-09-10T21:26:15.533518+0000","last_became_active":"2024-09-10T21:26:15.526149+0000","last_became_peered":"2024-09-10T21:26:15.526149+0000","last_unstale":"2024-09-10T21:26:15.533518+0000","last_undegraded":"2024-09-10T21:26:15.533518+0000","last_fullsized":"2024-09-10T21:26:15.533518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511121+0000","last_change":"2024-09-10T21:26:14.522970+0000","last_active":"2024-09-10T21:26:15.511121+0000","last_peered":"2024-09-10T21:26:15.511121+0000","last_clean":"2024-09-10T21:26:15.511121+0000","last_became_active":"2024-09-10T21:26:14.522696+0000","last_became_peered":"2024-09-10T21:26:14.522696+0000","last_unstale":"2024-09-10T21:26:15.511121+0000","last_undegraded":"2024-09-10T21:26:15.511121+0000","last_fullsized":"2024-09-10T21:26:15.511121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:26.009061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542110+0000","last_change":"2024-09-10T21:26:15.537430+0000","last_active":"2024-09-10T21:26:15.542110+0000","last_peered":"2024-09-10T21:26:15.542110+0000","last_clean":"2024-09-10T21:26:15.542110+0000","last_became_active":"2024-09-10T21:26:15.537174+0000","last_became_peered":"2024-09-10T21:26:15.537174+0000","last_unstale":"2024-09-10T21:26:15.542110+0000","last_undegraded":"2024-09-10T21:26:15.542110+0000","last_fullsized":"2024-09-10T21:26:15.542110+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:18.652981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510906+0000","last_change":"2024-09-10T21:26:14.516829+0000","last_active":"2024-09-10T21:26:15.510906+0000","last_peered":"2024-09-10T21:26:15.510906+0000","last_clean":"2024-09-10T21:26:15.510906+0000","last_became_active":"2024-09-10T21:26:14.516564+0000","last_became_peered":"2024-09-10T21:26:14.516564+0000","last_unstale":"2024-09-10T21:26:15.510906+0000","last_undegraded":"2024-09-10T21:26:15.510906+0000","last_fullsized":"2024-09-10T21:26:15.510906+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:32.378883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549908+0000","last_change":"2024-09-10T21:26:15.545818+0000","last_active":"2024-09-10T21:26:15.549908+0000","last_peered":"2024-09-10T21:26:15.549908+0000","last_clean":"2024-09-10T21:26:15.549908+0000","last_became_active":"2024-09-10T21:26:15.545383+0000","last_became_peered":"2024-09-10T21:26:15.545383+0000","last_unstale":"2024-09-10T21:26:15.549908+0000","last_undegraded":"2024-09-10T21:26:15.549908+0000","last_fullsized":"2024-09-10T21:26:15.549908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549760+0000","last_change":"2024-09-10T21:26:14.531817+0000","last_active":"2024-09-10T21:26:15.549760+0000","last_peered":"2024-09-10T21:26:15.549760+0000","last_clean":"2024-09-10T21:26:15.549760+0000","last_became_active":"2024-09-10T21:26:14.531387+0000","last_became_peered":"2024-09-10T21:26:14.531387+0000","last_unstale":"2024-09-10T21:26:15.549760+0000","last_undegraded":"2024-09-10T21:26:15.549760+0000","last_fullsized":"2024-09-10T21:26:15.549760+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:58.645631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547276+0000","last_change":"2024-09-10T21:26:15.543162+0000","last_active":"2024-09-10T21:26:15.547276+0000","last_peered":"2024-09-10T21:26:15.547276+0000","last_clean":"2024-09-10T21:26:15.547276+0000","last_became_active":"2024-09-10T21:26:15.542913+0000","last_became_peered":"2024-09-10T21:26:15.542913+0000","last_unstale":"2024-09-10T21:26:15.547276+0000","last_undegraded":"2024-09-10T21:26:15.547276+0000","last_fullsized":"2024-09-10T21:26:15.547276+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.553669+0000","last_change":"2024-09-10T21:26:14.533065+0000","last_active":"2024-09-10T21:26:15.553669+0000","last_peered":"2024-09-10T21:26:15.553669+0000","last_clean":"2024-09-10T21:26:15.553669+0000","last_became_active":"2024-09-10T21:26:14.532788+0000","last_became_peered":"2024-09-10T21:26:14.532788+0000","last_unstale":"2024-09-10T21:26:15.553669+0000","last_undegraded":"2024-09-10T21:26:15.553669+0000","last_fullsized":"2024-09-10T21:26:15.553669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:26.313137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541575+0000","last_change":"2024-09-10T21:26:15.540226+0000","last_active":"2024-09-10T21:26:15.541575+0000","last_peered":"2024-09-10T21:26:15.541575+0000","last_clean":"2024-09-10T21:26:15.541575+0000","last_became_active":"2024-09-10T21:26:15.539910+0000","last_became_peered":"2024-09-10T21:26:15.539910+0000","last_unstale":"2024-09-10T21:26:15.541575+0000","last_undegraded":"2024-09-10T21:26:15.541575+0000","last_fullsized":"2024-09-10T21:26:15.541575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:20.009997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538731+0000","last_change":"2024-09-10T21:26:14.526625+0000","last_active":"2024-09-10T21:26:15.538731+0000","last_peered":"2024-09-10T21:26:15.538731+0000","last_clean":"2024-09-10T21:26:15.538731+0000","last_became_active":"2024-09-10T21:26:14.526301+0000","last_became_peered":"2024-09-10T21:26:14.526301+0000","last_unstale":"2024-09-10T21:26:15.538731+0000","last_undegraded":"2024-09-10T21:26:15.538731+0000","last_fullsized":"2024-09-10T21:26:15.538731+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:36.925973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550279+0000","last_change":"2024-09-10T21:26:15.547448+0000","last_active":"2024-09-10T21:26:15.550279+0000","last_peered":"2024-09-10T21:26:15.550279+0000","last_clean":"2024-09-10T21:26:15.550279+0000","last_became_active":"2024-09-10T21:26:15.546869+0000","last_became_peered":"2024-09-10T21:26:15.546869+0000","last_unstale":"2024-09-10T21:26:15.550279+0000","last_undegraded":"2024-09-10T21:26:15.550279+0000","last_fullsized":"2024-09-10T21:26:15.550279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511264+0000","last_change":"2024-09-10T21:26:14.526509+0000","last_active":"2024-09-10T21:26:15.511264+0000","last_peered":"2024-09-10T21:26:15.511264+0000","last_clean":"2024-09-10T21:26:15.511264+0000","last_became_active":"2024-09-10T21:26:14.526211+0000","last_became_peered":"2024-09-10T21:26:14.526211+0000","last_unstale":"2024-09-10T21:26:15.511264+0000","last_undegraded":"2024-09-10T21:26:15.511264+0000","last_fullsized":"2024-09-10T21:26:15.511264+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:19.196183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539061+0000","last_change":"2024-09-10T21:26:15.533375+0000","last_active":"2024-09-10T21:26:15.539061+0000","last_peered":"2024-09-10T21:26:15.539061+0000","last_clean":"2024-09-10T21:26:15.539061+0000","last_became_active":"2024-09-10T21:26:15.533154+0000","last_became_peered":"2024-09-10T21:26:15.533154+0000","last_unstale":"2024-09-10T21:26:15.539061+0000","last_undegraded":"2024-09-10T21:26:15.539061+0000","last_fullsized":"2024-09-10T21:26:15.539061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:56.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.525900+0000","last_change":"2024-09-10T21:26:14.523579+0000","last_active":"2024-09-10T21:26:15.525900+0000","last_peered":"2024-09-10T21:26:15.525900+0000","last_clean":"2024-09-10T21:26:15.525900+0000","last_became_active":"2024-09-10T21:26:14.523321+0000","last_became_peered":"2024-09-10T21:26:14.523321+0000","last_unstale":"2024-09-10T21:26:15.525900+0000","last_undegraded":"2024-09-10T21:26:15.525900+0000","last_fullsized":"2024-09-10T21:26:15.525900+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:39.550410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538375+0000","last_change":"2024-09-10T21:26:14.526055+0000","last_active":"2024-09-10T21:26:15.538375+0000","last_peered":"2024-09-10T21:26:15.538375+0000","last_clean":"2024-09-10T21:26:15.538375+0000","last_became_active":"2024-09-10T21:26:14.525734+0000","last_became_peered":"2024-09-10T21:26:14.525734+0000","last_unstale":"2024-09-10T21:26:15.538375+0000","last_undegraded":"2024-09-10T21:26:15.538375+0000","last_fullsized":"2024-09-10T21:26:15.538375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:15.163346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549528+0000","last_change":"2024-09-10T21:26:15.545468+0000","last_active":"2024-09-10T21:26:15.549528+0000","last_peered":"2024-09-10T21:26:15.549528+0000","last_clean":"2024-09-10T21:26:15.549528+0000","last_became_active":"2024-09-10T21:26:15.545137+0000","last_became_peered":"2024-09-10T21:26:15.545137+0000","last_unstale":"2024-09-10T21:26:15.549528+0000","last_undegraded":"2024-09-10T21:26:15.549528+0000","last_fullsized":"2024-09-10T21:26:15.549528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:12.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538792+0000","last_change":"2024-09-10T21:26:14.534718+0000","last_active":"2024-09-10T21:26:15.538792+0000","last_peered":"2024-09-10T21:26:15.538792+0000","last_clean":"2024-09-10T21:26:15.538792+0000","last_became_active":"2024-09-10T21:26:14.534222+0000","last_became_peered":"2024-09-10T21:26:14.534222+0000","last_unstale":"2024-09-10T21:26:15.538792+0000","last_undegraded":"2024-09-10T21:26:15.538792+0000","last_fullsized":"2024-09-10T21:26:15.538792+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:10.810025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550202+0000","last_change":"2024-09-10T21:26:15.546824+0000","last_active":"2024-09-10T21:26:15.550202+0000","last_peered":"2024-09-10T21:26:15.550202+0000","last_clean":"2024-09-10T21:26:15.550202+0000","last_became_active":"2024-09-10T21:26:15.546286+0000","last_became_peered":"2024-09-10T21:26:15.546286+0000","last_unstale":"2024-09-10T21:26:15.550202+0000","last_undegraded":"2024-09-10T21:26:15.550202+0000","last_fullsized":"2024-09-10T21:26:15.550202+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:45.889359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533005+0000","last_change":"2024-09-10T21:26:14.527929+0000","last_active":"2024-09-10T21:26:15.533005+0000","last_peered":"2024-09-10T21:26:15.533005+0000","last_clean":"2024-09-10T21:26:15.533005+0000","last_became_active":"2024-09-10T21:26:14.527583+0000","last_became_peered":"2024-09-10T21:26:14.527583+0000","last_unstale":"2024-09-10T21:26:15.533005+0000","last_undegraded":"2024-09-10T21:26:15.533005+0000","last_fullsized":"2024-09-10T21:26:15.533005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:20.085776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545249+0000","last_change":"2024-09-10T21:26:15.541261+0000","last_active":"2024-09-10T21:26:15.545249+0000","last_peered":"2024-09-10T21:26:15.545249+0000","last_clean":"2024-09-10T21:26:15.545249+0000","last_became_active":"2024-09-10T21:26:15.541018+0000","last_became_peered":"2024-09-10T21:26:15.541018+0000","last_unstale":"2024-09-10T21:26:15.545249+0000","last_undegraded":"2024-09-10T21:26:15.545249+0000","last_fullsized":"2024-09-10T21:26:15.545249+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539384+0000","last_change":"2024-09-10T21:26:14.528002+0000","last_active":"2024-09-10T21:26:15.539384+0000","last_peered":"2024-09-10T21:26:15.539384+0000","last_clean":"2024-09-10T21:26:15.539384+0000","last_became_active":"2024-09-10T21:26:14.527640+0000","last_became_peered":"2024-09-10T21:26:14.527640+0000","last_unstale":"2024-09-10T21:26:15.539384+0000","last_undegraded":"2024-09-10T21:26:15.539384+0000","last_fullsized":"2024-09-10T21:26:15.539384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:40.905930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542762+0000","last_change":"2024-09-10T21:26:15.533913+0000","last_active":"2024-09-10T21:26:15.542762+0000","last_peered":"2024-09-10T21:26:15.542762+0000","last_clean":"2024-09-10T21:26:15.542762+0000","last_became_active":"2024-09-10T21:26:15.533607+0000","last_became_peered":"2024-09-10T21:26:15.533607+0000","last_unstale":"2024-09-10T21:26:15.542762+0000","last_undegraded":"2024-09-10T21:26:15.542762+0000","last_fullsized":"2024-09-10T21:26:15.542762+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:53.763483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524585+0000","last_change":"2024-09-10T21:26:14.530739+0000","last_active":"2024-09-10T21:26:15.524585+0000","last_peered":"2024-09-10T21:26:15.524585+0000","last_clean":"2024-09-10T21:26:15.524585+0000","last_became_active":"2024-09-10T21:26:14.530428+0000","last_became_peered":"2024-09-10T21:26:14.530428+0000","last_unstale":"2024-09-10T21:26:15.524585+0000","last_undegraded":"2024-09-10T21:26:15.524585+0000","last_fullsized":"2024-09-10T21:26:15.524585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:47.051011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547356+0000","last_change":"2024-09-10T21:26:15.543515+0000","last_active":"2024-09-10T21:26:15.547356+0000","last_peered":"2024-09-10T21:26:15.547356+0000","last_clean":"2024-09-10T21:26:15.547356+0000","last_became_active":"2024-09-10T21:26:15.543291+0000","last_became_peered":"2024-09-10T21:26:15.543291+0000","last_unstale":"2024-09-10T21:26:15.547356+0000","last_undegraded":"2024-09-10T21:26:15.547356+0000","last_fullsized":"2024-09-10T21:26:15.547356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551224+0000","last_change":"2024-09-10T21:26:14.532927+0000","last_active":"2024-09-10T21:26:15.551224+0000","last_peered":"2024-09-10T21:26:15.551224+0000","last_clean":"2024-09-10T21:26:15.551224+0000","last_became_active":"2024-09-10T21:26:14.532625+0000","last_became_peered":"2024-09-10T21:26:14.532625+0000","last_unstale":"2024-09-10T21:26:15.551224+0000","last_undegraded":"2024-09-10T21:26:15.551224+0000","last_fullsized":"2024-09-10T21:26:15.551224+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:36.431767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532586+0000","last_change":"2024-09-10T21:26:15.529708+0000","last_active":"2024-09-10T21:26:15.532586+0000","last_peered":"2024-09-10T21:26:15.532586+0000","last_clean":"2024-09-10T21:26:15.532586+0000","last_became_active":"2024-09-10T21:26:15.529431+0000","last_became_peered":"2024-09-10T21:26:15.529431+0000","last_unstale":"2024-09-10T21:26:15.532586+0000","last_undegraded":"2024-09-10T21:26:15.532586+0000","last_fullsized":"2024-09-10T21:26:15.532586+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:08.029645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549692+0000","last_change":"2024-09-10T21:26:15.544629+0000","last_active":"2024-09-10T21:26:15.549692+0000","last_peered":"2024-09-10T21:26:15.549692+0000","last_clean":"2024-09-10T21:26:15.549692+0000","last_became_active":"2024-09-10T21:26:15.540919+0000","last_became_peered":"2024-09-10T21:26:15.540919+0000","last_unstale":"2024-09-10T21:26:15.549692+0000","last_undegraded":"2024-09-10T21:26:15.549692+0000","last_fullsized":"2024-09-10T21:26:15.549692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520315+0000","last_change":"2024-09-10T21:26:14.529271+0000","last_active":"2024-09-10T21:26:15.520315+0000","last_peered":"2024-09-10T21:26:15.520315+0000","last_clean":"2024-09-10T21:26:15.520315+0000","last_became_active":"2024-09-10T21:26:14.528963+0000","last_became_peered":"2024-09-10T21:26:14.528963+0000","last_unstale":"2024-09-10T21:26:15.520315+0000","last_undegraded":"2024-09-10T21:26:15.520315+0000","last_fullsized":"2024-09-10T21:26:15.520315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:14.592093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544636+0000","last_change":"2024-09-10T21:26:15.540533+0000","last_active":"2024-09-10T21:26:15.544636+0000","last_peered":"2024-09-10T21:26:15.544636+0000","last_clean":"2024-09-10T21:26:15.544636+0000","last_became_active":"2024-09-10T21:26:15.540237+0000","last_became_peered":"2024-09-10T21:26:15.540237+0000","last_unstale":"2024-09-10T21:26:15.544636+0000","last_undegraded":"2024-09-10T21:26:15.544636+0000","last_fullsized":"2024-09-10T21:26:15.544636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:49.612063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520099+0000","last_change":"2024-09-10T21:26:14.529847+0000","last_active":"2024-09-10T21:26:15.520099+0000","last_peered":"2024-09-10T21:26:15.520099+0000","last_clean":"2024-09-10T21:26:15.520099+0000","last_became_active":"2024-09-10T21:26:14.529074+0000","last_became_peered":"2024-09-10T21:26:14.529074+0000","last_unstale":"2024-09-10T21:26:15.520099+0000","last_undegraded":"2024-09-10T21:26:15.520099+0000","last_fullsized":"2024-09-10T21:26:15.520099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:27.478486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532977+0000","last_change":"2024-09-10T21:26:15.528858+0000","last_active":"2024-09-10T21:26:15.532977+0000","last_peered":"2024-09-10T21:26:15.532977+0000","last_clean":"2024-09-10T21:26:15.532977+0000","last_became_active":"2024-09-10T21:26:15.527735+0000","last_became_peered":"2024-09-10T21:26:15.527735+0000","last_unstale":"2024-09-10T21:26:15.532977+0000","last_undegraded":"2024-09-10T21:26:15.532977+0000","last_fullsized":"2024-09-10T21:26:15.532977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:31.109223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544806+0000","last_change":"2024-09-10T21:26:14.529877+0000","last_active":"2024-09-10T21:26:15.544806+0000","last_peered":"2024-09-10T21:26:15.544806+0000","last_clean":"2024-09-10T21:26:15.544806+0000","last_became_active":"2024-09-10T21:26:14.529580+0000","last_became_peered":"2024-09-10T21:26:14.529580+0000","last_unstale":"2024-09-10T21:26:15.544806+0000","last_undegraded":"2024-09-10T21:26:15.544806+0000","last_fullsized":"2024-09-10T21:26:15.544806+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:41.737020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549409+0000","last_change":"2024-09-10T21:26:15.541706+0000","last_active":"2024-09-10T21:26:15.549409+0000","last_peered":"2024-09-10T21:26:15.549409+0000","last_clean":"2024-09-10T21:26:15.549409+0000","last_became_active":"2024-09-10T21:26:15.540365+0000","last_became_peered":"2024-09-10T21:26:15.540365+0000","last_unstale":"2024-09-10T21:26:15.549409+0000","last_undegraded":"2024-09-10T21:26:15.549409+0000","last_fullsized":"2024-09-10T21:26:15.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:43.648100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.530063+0000","last_change":"2024-09-10T21:26:14.532072+0000","last_active":"2024-09-10T21:26:15.530063+0000","last_peered":"2024-09-10T21:26:15.530063+0000","last_clean":"2024-09-10T21:26:15.530063+0000","last_became_active":"2024-09-10T21:26:14.530711+0000","last_became_peered":"2024-09-10T21:26:14.530711+0000","last_unstale":"2024-09-10T21:26:15.530063+0000","last_undegraded":"2024-09-10T21:26:15.530063+0000","last_fullsized":"2024-09-10T21:26:15.530063+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:15.362786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:14.497932+0000","last_change":"2024-09-10T21:26:14.497932+0000","last_active":"2024-09-10T21:26:14.497932+0000","last_peered":"2024-09-10T21:26:14.497932+0000","last_clean":"2024-09-10T21:26:14.497932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:14.497932+0000","last_undegraded":"2024-09-10T21:26:14.497932+0000","last_fullsized":"2024-09-10T21:26:14.497932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549756+0000","last_change":"2024-09-10T21:26:15.542738+0000","last_active":"2024-09-10T21:26:15.549756+0000","last_peered":"2024-09-10T21:26:15.549756+0000","last_clean":"2024-09-10T21:26:15.549756+0000","last_became_active":"2024-09-10T21:26:15.542497+0000","last_became_peered":"2024-09-10T21:26:15.542497+0000","last_unstale":"2024-09-10T21:26:15.549756+0000","last_undegraded":"2024-09-10T21:26:15.549756+0000","last_fullsized":"2024-09-10T21:26:15.549756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:59.431959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520974+0000","last_change":"2024-09-10T21:26:14.530559+0000","last_active":"2024-09-10T21:26:15.520974+0000","last_peered":"2024-09-10T21:26:15.520974+0000","last_clean":"2024-09-10T21:26:15.520974+0000","last_became_active":"2024-09-10T21:26:14.529513+0000","last_became_peered":"2024-09-10T21:26:14.529513+0000","last_unstale":"2024-09-10T21:26:15.520974+0000","last_undegraded":"2024-09-10T21:26:15.520974+0000","last_fullsized":"2024-09-10T21:26:15.520974+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:10.595223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538120+0000","last_change":"2024-09-10T21:26:15.533560+0000","last_active":"2024-09-10T21:26:15.538120+0000","last_peered":"2024-09-10T21:26:15.538120+0000","last_clean":"2024-09-10T21:26:15.538120+0000","last_became_active":"2024-09-10T21:26:15.533313+0000","last_became_peered":"2024-09-10T21:26:15.533313+0000","last_unstale":"2024-09-10T21:26:15.538120+0000","last_undegraded":"2024-09-10T21:26:15.538120+0000","last_fullsized":"2024-09-10T21:26:15.538120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:26:13.494691+0000","last_change":"2024-09-10T21:26:13.494691+0000","last_active":"2024-09-10T21:26:13.494691+0000","last_peered":"2024-09-10T21:26:13.494691+0000","last_clean":"2024-09-10T21:26:13.494691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:26:13.494691+0000","last_undegraded":"2024-09-10T21:26:13.494691+0000","last_fullsized":"2024-09-10T21:26:13.494691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548560+0000","last_change":"2024-09-10T21:26:15.536138+0000","last_active":"2024-09-10T21:26:15.548560+0000","last_peered":"2024-09-10T21:26:15.548560+0000","last_clean":"2024-09-10T21:26:15.548560+0000","last_became_active":"2024-09-10T21:26:15.535521+0000","last_became_peered":"2024-09-10T21:26:15.535521+0000","last_unstale":"2024-09-10T21:26:15.548560+0000","last_undegraded":"2024-09-10T21:26:15.548560+0000","last_fullsized":"2024-09-10T21:26:15.548560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:20.575951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.526019+0000","last_change":"2024-09-10T21:26:14.519898+0000","last_active":"2024-09-10T21:26:15.526019+0000","last_peered":"2024-09-10T21:26:15.526019+0000","last_clean":"2024-09-10T21:26:15.526019+0000","last_became_active":"2024-09-10T21:26:14.519606+0000","last_became_peered":"2024-09-10T21:26:14.519606+0000","last_unstale":"2024-09-10T21:26:15.526019+0000","last_undegraded":"2024-09-10T21:26:15.526019+0000","last_fullsized":"2024-09-10T21:26:15.526019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.608822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544613+0000","last_change":"2024-09-10T21:26:15.532747+0000","last_active":"2024-09-10T21:26:15.544613+0000","last_peered":"2024-09-10T21:26:15.544613+0000","last_clean":"2024-09-10T21:26:15.544613+0000","last_became_active":"2024-09-10T21:26:15.532482+0000","last_became_peered":"2024-09-10T21:26:15.532482+0000","last_unstale":"2024-09-10T21:26:15.544613+0000","last_undegraded":"2024-09-10T21:26:15.544613+0000","last_fullsized":"2024-09-10T21:26:15.544613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:40.553651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.509867+0000","last_change":"2024-09-10T21:26:14.526821+0000","last_active":"2024-09-10T21:26:15.509867+0000","last_peered":"2024-09-10T21:26:15.509867+0000","last_clean":"2024-09-10T21:26:15.509867+0000","last_became_active":"2024-09-10T21:26:14.526561+0000","last_became_peered":"2024-09-10T21:26:14.526561+0000","last_unstale":"2024-09-10T21:26:15.509867+0000","last_undegraded":"2024-09-10T21:26:15.509867+0000","last_fullsized":"2024-09-10T21:26:15.509867+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541912+0000","last_change":"2024-09-10T21:26:15.538996+0000","last_active":"2024-09-10T21:26:15.541912+0000","last_peered":"2024-09-10T21:26:15.541912+0000","last_clean":"2024-09-10T21:26:15.541912+0000","last_became_active":"2024-09-10T21:26:15.538393+0000","last_became_peered":"2024-09-10T21:26:15.538393+0000","last_unstale":"2024-09-10T21:26:15.541912+0000","last_undegraded":"2024-09-10T21:26:15.541912+0000","last_fullsized":"2024-09-10T21:26:15.541912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:33.376220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524045+0000","last_change":"2024-09-10T21:26:14.533819+0000","last_active":"2024-09-10T21:26:15.524045+0000","last_peered":"2024-09-10T21:26:15.524045+0000","last_clean":"2024-09-10T21:26:15.524045+0000","last_became_active":"2024-09-10T21:26:14.533554+0000","last_became_peered":"2024-09-10T21:26:14.533554+0000","last_unstale":"2024-09-10T21:26:15.524045+0000","last_undegraded":"2024-09-10T21:26:15.524045+0000","last_fullsized":"2024-09-10T21:26:15.524045+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:55.449678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510018+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:26:15.510018+0000","last_peered":"2024-09-10T21:26:15.510018+0000","last_clean":"2024-09-10T21:26:15.510018+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:26:15.510018+0000","last_undegraded":"2024-09-10T21:26:15.510018+0000","last_fullsized":"2024-09-10T21:26:15.510018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":275651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1216512,"data_stored":865963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":610304,"data_stored":268428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1204224,"data_stored":858796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27612,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":585728,"data_stored":255552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:26:20.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:26:20.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- 172.21.15.97:0/3247113422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 msgr2=0x7f64c806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 --2- 172.21.15.97:0/3247113422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c806ec40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64c80cc590 tx=0x7f64b401f0b0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- 172.21.15.97:0/3247113422 shutdown_connections 2024-09-10T21:26:20.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 --2- 172.21.15.97:0/3247113422 >> v2:172.21.15.194:3300/0 conn(0x7f64c806f180 0x7f64c80ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 --2- 172.21.15.97:0/3247113422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c806ec40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 --2- 172.21.15.97:0/3247113422 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64c81511f0 0x7f64c80b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- 172.21.15.97:0/3247113422 >> 172.21.15.97:0/3247113422 conn(0x7f64c80b1780 msgr2=0x7f64c80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- 172.21.15.97:0/3247113422 shutdown_connections 2024-09-10T21:26:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- 172.21.15.97:0/3247113422 wait complete. 2024-09-10T21:26:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 Processor -- start 2024-09-10T21:26:20.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- start start 2024-09-10T21:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64c806f180 0x7f64c813de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c80cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64c81511f0 0x7f64c814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64c806e530 con 0x7f64c80b3cf0 2024-09-10T21:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f64c806e3b0 con 0x7f64c81511f0 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f64c806e6b0 con 0x7f64c806f180 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64c7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c80cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64c7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c80cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41628/0 (socket says 172.21.15.97:41628) 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64c7400640 1 -- 172.21.15.97:0/2887166658 learned_addr learned my addr 172.21.15.97:0/2887166658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64c7e00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64c806f180 0x7f64c813de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.215+0000 7f64cce00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64c81511f0 0x7f64c814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64c7400640 1 -- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64c806f180 msgr2=0x7f64c813de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64c7400640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64c806f180 0x7f64c813de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64c7400640 1 -- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64c81511f0 msgr2=0x7f64c814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64c7400640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64c81511f0 0x7f64c814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64c7400640 1 -- 172.21.15.97:0/2887166658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c80ce9f0 con 0x7f64c80b3cf0 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64cce00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64c81511f0 0x7f64c814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64c7400640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c80cdad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f64b40b3040 tx=0x7f64b4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64bfe00640 1 -- 172.21.15.97:0/2887166658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40b5040 con 0x7f64c80b3cf0 2024-09-10T21:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64bfe00640 1 -- 172.21.15.97:0/2887166658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64b4005330 con 0x7f64c80b3cf0 2024-09-10T21:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c80cc7e0 con 0x7f64c80b3cf0 2024-09-10T21:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64bfe00640 1 -- 172.21.15.97:0/2887166658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40046d0 con 0x7f64c80b3cf0 2024-09-10T21:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c80ccda0 con 0x7f64c80b3cf0 2024-09-10T21:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c80ceee0 con 0x7f64c80b3cf0 2024-09-10T21:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64bfe00640 1 -- 172.21.15.97:0/2887166658 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f64b40b8070 con 0x7f64c80b3cf0 2024-09-10T21:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64bfe00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64a80725d0 0x7f64a8074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64bfe00640 1 -- 172.21.15.97:0/2887166658 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f64b412ffe0 con 0x7f64c80b3cf0 2024-09-10T21:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.219+0000 7f64c7e00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64a80725d0 0x7f64a8074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.223+0000 7f64c7e00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64a80725d0 0x7f64a8074a90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64c81f90c0 tx=0x7f64b8010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:20.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.223+0000 7f64bfe00640 1 -- 172.21.15.97:0/2887166658 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f64b40bf060 con 0x7f64c80b3cf0 2024-09-10T21:26:20.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.387+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64c8159560 con 0x7f64a80725d0 2024-09-10T21:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.391+0000 7f64bfe00640 1 -- 172.21.15.97:0/2887166658 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f64b805a840 con 0x7f64a80725d0 2024-09-10T21:26:20.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:20.398 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64a80725d0 msgr2=0x7f64a8074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64a80725d0 0x7f64a8074a90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64c81f90c0 tx=0x7f64b8010040 comp rx=0 tx=0).stop 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 msgr2=0x7f64c80cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c80cdad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f64b40b3040 tx=0x7f64b4005760 comp rx=0 tx=0).stop 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 shutdown_connections 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64a80725d0 0x7f64a8074a90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64c81511f0 0x7f64c814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64c80b3cf0 0x7f64c80cdad0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 --2- 172.21.15.97:0/2887166658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64c806f180 0x7f64c813de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 >> 172.21.15.97:0/2887166658 conn(0x7f64c80b1780 msgr2=0x7f64c815ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 shutdown_connections 2024-09-10T21:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.395+0000 7f64cee00640 1 -- 172.21.15.97:0/2887166658 wait complete. 2024-09-10T21:26:20.417 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-10T21:26:20.100580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222288,"kb_used_data":6052,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752432,"statfs":{"total":773094113280,"available":772866490368,"internally_reserved":0,"allocated":6197248,"data_stored":3380868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001474"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549183+0000","last_change":"2024-09-10T21:26:15.539541+0000","last_active":"2024-09-10T21:26:15.549183+0000","last_peered":"2024-09-10T21:26:15.549183+0000","last_clean":"2024-09-10T21:26:15.549183+0000","last_became_active":"2024-09-10T21:26:15.539095+0000","last_became_peered":"2024-09-10T21:26:15.539095+0000","last_unstale":"2024-09-10T21:26:15.549183+0000","last_undegraded":"2024-09-10T21:26:15.549183+0000","last_fullsized":"2024-09-10T21:26:15.549183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.543900+0000","last_change":"2024-09-10T21:26:15.535693+0000","last_active":"2024-09-10T21:26:15.543900+0000","last_peered":"2024-09-10T21:26:15.543900+0000","last_clean":"2024-09-10T21:26:15.543900+0000","last_became_active":"2024-09-10T21:26:15.535421+0000","last_became_peered":"2024-09-10T21:26:15.535421+0000","last_unstale":"2024-09-10T21:26:15.543900+0000","last_undegraded":"2024-09-10T21:26:15.543900+0000","last_fullsized":"2024-09-10T21:26:15.543900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039167+0000","last_change":"2024-09-10T21:26:15.537145+0000","last_active":"2024-09-10T21:26:17.039167+0000","last_peered":"2024-09-10T21:26:17.039167+0000","last_clean":"2024-09-10T21:26:17.039167+0000","last_became_active":"2024-09-10T21:26:15.536886+0000","last_became_peered":"2024-09-10T21:26:15.536886+0000","last_unstale":"2024-09-10T21:26:17.039167+0000","last_undegraded":"2024-09-10T21:26:17.039167+0000","last_fullsized":"2024-09-10T21:26:17.039167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:20.043744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546167+0000","last_change":"2024-09-10T21:26:15.537322+0000","last_active":"2024-09-10T21:26:15.546167+0000","last_peered":"2024-09-10T21:26:15.546167+0000","last_clean":"2024-09-10T21:26:15.546167+0000","last_became_active":"2024-09-10T21:26:15.537039+0000","last_became_peered":"2024-09-10T21:26:15.537039+0000","last_unstale":"2024-09-10T21:26:15.546167+0000","last_undegraded":"2024-09-10T21:26:15.546167+0000","last_fullsized":"2024-09-10T21:26:15.546167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:11.598464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551006+0000","last_change":"2024-09-10T21:26:15.547237+0000","last_active":"2024-09-10T21:26:15.551006+0000","last_peered":"2024-09-10T21:26:15.551006+0000","last_clean":"2024-09-10T21:26:15.551006+0000","last_became_active":"2024-09-10T21:26:15.546563+0000","last_became_peered":"2024-09-10T21:26:15.546563+0000","last_unstale":"2024-09-10T21:26:15.551006+0000","last_undegraded":"2024-09-10T21:26:15.551006+0000","last_fullsized":"2024-09-10T21:26:15.551006+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544695+0000","last_change":"2024-09-10T21:26:15.534547+0000","last_active":"2024-09-10T21:26:15.544695+0000","last_peered":"2024-09-10T21:26:15.544695+0000","last_clean":"2024-09-10T21:26:15.544695+0000","last_became_active":"2024-09-10T21:26:15.534280+0000","last_became_peered":"2024-09-10T21:26:15.534280+0000","last_unstale":"2024-09-10T21:26:15.544695+0000","last_undegraded":"2024-09-10T21:26:15.544695+0000","last_fullsized":"2024-09-10T21:26:15.544695+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:48.054252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548817+0000","last_change":"2024-09-10T21:26:15.533183+0000","last_active":"2024-09-10T21:26:15.548817+0000","last_peered":"2024-09-10T21:26:15.548817+0000","last_clean":"2024-09-10T21:26:15.548817+0000","last_became_active":"2024-09-10T21:26:15.532995+0000","last_became_peered":"2024-09-10T21:26:15.532995+0000","last_unstale":"2024-09-10T21:26:15.548817+0000","last_undegraded":"2024-09-10T21:26:15.548817+0000","last_fullsized":"2024-09-10T21:26:15.548817+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:27.005167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546214+0000","last_change":"2024-09-10T21:26:15.541612+0000","last_active":"2024-09-10T21:26:15.546214+0000","last_peered":"2024-09-10T21:26:15.546214+0000","last_clean":"2024-09-10T21:26:15.546214+0000","last_became_active":"2024-09-10T21:26:15.541343+0000","last_became_peered":"2024-09-10T21:26:15.541343+0000","last_unstale":"2024-09-10T21:26:15.546214+0000","last_undegraded":"2024-09-10T21:26:15.546214+0000","last_fullsized":"2024-09-10T21:26:15.546214+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039368+0000","last_change":"2024-09-10T21:26:15.531640+0000","last_active":"2024-09-10T21:26:17.039368+0000","last_peered":"2024-09-10T21:26:17.039368+0000","last_clean":"2024-09-10T21:26:17.039368+0000","last_became_active":"2024-09-10T21:26:15.531347+0000","last_became_peered":"2024-09-10T21:26:15.531347+0000","last_unstale":"2024-09-10T21:26:17.039368+0000","last_undegraded":"2024-09-10T21:26:17.039368+0000","last_fullsized":"2024-09-10T21:26:17.039368+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:34.692481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548595+0000","last_change":"2024-09-10T21:26:15.536440+0000","last_active":"2024-09-10T21:26:15.548595+0000","last_peered":"2024-09-10T21:26:15.548595+0000","last_clean":"2024-09-10T21:26:15.548595+0000","last_became_active":"2024-09-10T21:26:15.535896+0000","last_became_peered":"2024-09-10T21:26:15.535896+0000","last_unstale":"2024-09-10T21:26:15.548595+0000","last_undegraded":"2024-09-10T21:26:15.548595+0000","last_fullsized":"2024-09-10T21:26:15.548595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:57.577430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549667+0000","last_change":"2024-09-10T21:26:15.538325+0000","last_active":"2024-09-10T21:26:15.549667+0000","last_peered":"2024-09-10T21:26:15.549667+0000","last_clean":"2024-09-10T21:26:15.549667+0000","last_became_active":"2024-09-10T21:26:15.536197+0000","last_became_peered":"2024-09-10T21:26:15.536197+0000","last_unstale":"2024-09-10T21:26:15.549667+0000","last_undegraded":"2024-09-10T21:26:15.549667+0000","last_fullsized":"2024-09-10T21:26:15.549667+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520191+0000","last_change":"2024-09-10T21:26:15.518220+0000","last_active":"2024-09-10T21:26:15.520191+0000","last_peered":"2024-09-10T21:26:15.520191+0000","last_clean":"2024-09-10T21:26:15.520191+0000","last_became_active":"2024-09-10T21:26:15.516898+0000","last_became_peered":"2024-09-10T21:26:15.516898+0000","last_unstale":"2024-09-10T21:26:15.520191+0000","last_undegraded":"2024-09-10T21:26:15.520191+0000","last_fullsized":"2024-09-10T21:26:15.520191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.482589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547572+0000","last_change":"2024-09-10T21:26:15.545455+0000","last_active":"2024-09-10T21:26:15.547572+0000","last_peered":"2024-09-10T21:26:15.547572+0000","last_clean":"2024-09-10T21:26:15.547572+0000","last_became_active":"2024-09-10T21:26:15.545207+0000","last_became_peered":"2024-09-10T21:26:15.545207+0000","last_unstale":"2024-09-10T21:26:15.547572+0000","last_undegraded":"2024-09-10T21:26:15.547572+0000","last_fullsized":"2024-09-10T21:26:15.547572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549843+0000","last_change":"2024-09-10T21:26:15.545646+0000","last_active":"2024-09-10T21:26:15.549843+0000","last_peered":"2024-09-10T21:26:15.549843+0000","last_clean":"2024-09-10T21:26:15.549843+0000","last_became_active":"2024-09-10T21:26:15.544941+0000","last_became_peered":"2024-09-10T21:26:15.544941+0000","last_unstale":"2024-09-10T21:26:15.549843+0000","last_undegraded":"2024-09-10T21:26:15.549843+0000","last_fullsized":"2024-09-10T21:26:15.549843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542087+0000","last_change":"2024-09-10T21:26:15.536666+0000","last_active":"2024-09-10T21:26:15.542087+0000","last_peered":"2024-09-10T21:26:15.542087+0000","last_clean":"2024-09-10T21:26:15.542087+0000","last_became_active":"2024-09-10T21:26:15.536410+0000","last_became_peered":"2024-09-10T21:26:15.536410+0000","last_unstale":"2024-09-10T21:26:15.542087+0000","last_undegraded":"2024-09-10T21:26:15.542087+0000","last_fullsized":"2024-09-10T21:26:15.542087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:15.595334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.529423+0000","last_change":"2024-09-10T21:26:15.521400+0000","last_active":"2024-09-10T21:26:15.529423+0000","last_peered":"2024-09-10T21:26:15.529423+0000","last_clean":"2024-09-10T21:26:15.529423+0000","last_became_active":"2024-09-10T21:26:15.520076+0000","last_became_peered":"2024-09-10T21:26:15.520076+0000","last_unstale":"2024-09-10T21:26:15.529423+0000","last_undegraded":"2024-09-10T21:26:15.529423+0000","last_fullsized":"2024-09-10T21:26:15.529423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:13.921295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549231+0000","last_change":"2024-09-10T21:26:15.544245+0000","last_active":"2024-09-10T21:26:15.549231+0000","last_peered":"2024-09-10T21:26:15.549231+0000","last_clean":"2024-09-10T21:26:15.549231+0000","last_became_active":"2024-09-10T21:26:15.539109+0000","last_became_peered":"2024-09-10T21:26:15.539109+0000","last_unstale":"2024-09-10T21:26:15.549231+0000","last_undegraded":"2024-09-10T21:26:15.549231+0000","last_fullsized":"2024-09-10T21:26:15.549231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.531781+0000","last_change":"2024-09-10T21:26:15.520766+0000","last_active":"2024-09-10T21:26:15.531781+0000","last_peered":"2024-09-10T21:26:15.531781+0000","last_clean":"2024-09-10T21:26:15.531781+0000","last_became_active":"2024-09-10T21:26:15.520453+0000","last_became_peered":"2024-09-10T21:26:15.520453+0000","last_unstale":"2024-09-10T21:26:15.531781+0000","last_undegraded":"2024-09-10T21:26:15.531781+0000","last_fullsized":"2024-09-10T21:26:15.531781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:58.808915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039229+0000","last_change":"2024-09-10T21:26:15.534009+0000","last_active":"2024-09-10T21:26:17.039229+0000","last_peered":"2024-09-10T21:26:17.039229+0000","last_clean":"2024-09-10T21:26:17.039229+0000","last_became_active":"2024-09-10T21:26:15.533744+0000","last_became_peered":"2024-09-10T21:26:15.533744+0000","last_unstale":"2024-09-10T21:26:17.039229+0000","last_undegraded":"2024-09-10T21:26:17.039229+0000","last_fullsized":"2024-09-10T21:26:17.039229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:39.504012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545207+0000","last_change":"2024-09-10T21:26:15.540350+0000","last_active":"2024-09-10T21:26:15.545207+0000","last_peered":"2024-09-10T21:26:15.545207+0000","last_clean":"2024-09-10T21:26:15.545207+0000","last_became_active":"2024-09-10T21:26:15.535466+0000","last_became_peered":"2024-09-10T21:26:15.535466+0000","last_unstale":"2024-09-10T21:26:15.545207+0000","last_undegraded":"2024-09-10T21:26:15.545207+0000","last_fullsized":"2024-09-10T21:26:15.545207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542035+0000","last_change":"2024-09-10T21:26:15.534826+0000","last_active":"2024-09-10T21:26:15.542035+0000","last_peered":"2024-09-10T21:26:15.542035+0000","last_clean":"2024-09-10T21:26:15.542035+0000","last_became_active":"2024-09-10T21:26:15.534533+0000","last_became_peered":"2024-09-10T21:26:15.534533+0000","last_unstale":"2024-09-10T21:26:15.542035+0000","last_undegraded":"2024-09-10T21:26:15.542035+0000","last_fullsized":"2024-09-10T21:26:15.542035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:28.481727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038929+0000","last_change":"2024-09-10T21:26:15.534654+0000","last_active":"2024-09-10T21:26:17.038929+0000","last_peered":"2024-09-10T21:26:17.038929+0000","last_clean":"2024-09-10T21:26:17.038929+0000","last_became_active":"2024-09-10T21:26:15.534361+0000","last_became_peered":"2024-09-10T21:26:15.534361+0000","last_unstale":"2024-09-10T21:26:17.038929+0000","last_undegraded":"2024-09-10T21:26:17.038929+0000","last_fullsized":"2024-09-10T21:26:17.038929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.584980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038631+0000","last_change":"2024-09-10T21:26:15.532658+0000","last_active":"2024-09-10T21:26:17.038631+0000","last_peered":"2024-09-10T21:26:17.038631+0000","last_clean":"2024-09-10T21:26:17.038631+0000","last_became_active":"2024-09-10T21:26:15.532354+0000","last_became_peered":"2024-09-10T21:26:15.532354+0000","last_unstale":"2024-09-10T21:26:17.038631+0000","last_undegraded":"2024-09-10T21:26:17.038631+0000","last_fullsized":"2024-09-10T21:26:17.038631+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.984958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038705+0000","last_change":"2024-09-10T21:26:15.534760+0000","last_active":"2024-09-10T21:26:17.038705+0000","last_peered":"2024-09-10T21:26:17.038705+0000","last_clean":"2024-09-10T21:26:17.038705+0000","last_became_active":"2024-09-10T21:26:15.534482+0000","last_became_peered":"2024-09-10T21:26:15.534482+0000","last_unstale":"2024-09-10T21:26:17.038705+0000","last_undegraded":"2024-09-10T21:26:17.038705+0000","last_fullsized":"2024-09-10T21:26:17.038705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:34.222614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549208+0000","last_change":"2024-09-10T21:26:15.539933+0000","last_active":"2024-09-10T21:26:15.549208+0000","last_peered":"2024-09-10T21:26:15.549208+0000","last_clean":"2024-09-10T21:26:15.549208+0000","last_became_active":"2024-09-10T21:26:15.539551+0000","last_became_peered":"2024-09-10T21:26:15.539551+0000","last_unstale":"2024-09-10T21:26:15.549208+0000","last_undegraded":"2024-09-10T21:26:15.549208+0000","last_fullsized":"2024-09-10T21:26:15.549208+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039409+0000","last_change":"2024-09-10T21:26:15.518300+0000","last_active":"2024-09-10T21:26:17.039409+0000","last_peered":"2024-09-10T21:26:17.039409+0000","last_clean":"2024-09-10T21:26:17.039409+0000","last_became_active":"2024-09-10T21:26:15.518016+0000","last_became_peered":"2024-09-10T21:26:15.518016+0000","last_unstale":"2024-09-10T21:26:17.039409+0000","last_undegraded":"2024-09-10T21:26:17.039409+0000","last_fullsized":"2024-09-10T21:26:17.039409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:19.497285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546220+0000","last_change":"2024-09-10T21:26:15.541772+0000","last_active":"2024-09-10T21:26:15.546220+0000","last_peered":"2024-09-10T21:26:15.546220+0000","last_clean":"2024-09-10T21:26:15.546220+0000","last_became_active":"2024-09-10T21:26:15.541512+0000","last_became_peered":"2024-09-10T21:26:15.541512+0000","last_unstale":"2024-09-10T21:26:15.546220+0000","last_undegraded":"2024-09-10T21:26:15.546220+0000","last_fullsized":"2024-09-10T21:26:15.546220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550226+0000","last_change":"2024-09-10T21:26:15.545746+0000","last_active":"2024-09-10T21:26:15.550226+0000","last_peered":"2024-09-10T21:26:15.550226+0000","last_clean":"2024-09-10T21:26:15.550226+0000","last_became_active":"2024-09-10T21:26:15.545498+0000","last_became_peered":"2024-09-10T21:26:15.545498+0000","last_unstale":"2024-09-10T21:26:15.550226+0000","last_undegraded":"2024-09-10T21:26:15.550226+0000","last_fullsized":"2024-09-10T21:26:15.550226+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549974+0000","last_change":"2024-09-10T21:26:15.546272+0000","last_active":"2024-09-10T21:26:15.549974+0000","last_peered":"2024-09-10T21:26:15.549974+0000","last_clean":"2024-09-10T21:26:15.549974+0000","last_became_active":"2024-09-10T21:26:15.537090+0000","last_became_peered":"2024-09-10T21:26:15.537090+0000","last_unstale":"2024-09-10T21:26:15.549974+0000","last_undegraded":"2024-09-10T21:26:15.549974+0000","last_fullsized":"2024-09-10T21:26:15.549974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546287+0000","last_change":"2024-09-10T21:26:15.541138+0000","last_active":"2024-09-10T21:26:15.546287+0000","last_peered":"2024-09-10T21:26:15.546287+0000","last_clean":"2024-09-10T21:26:15.546287+0000","last_became_active":"2024-09-10T21:26:15.540540+0000","last_became_peered":"2024-09-10T21:26:15.540540+0000","last_unstale":"2024-09-10T21:26:15.546287+0000","last_undegraded":"2024-09-10T21:26:15.546287+0000","last_fullsized":"2024-09-10T21:26:15.546287+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:27.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038685+0000","last_change":"2024-09-10T21:26:15.534852+0000","last_active":"2024-09-10T21:26:17.038685+0000","last_peered":"2024-09-10T21:26:17.038685+0000","last_clean":"2024-09-10T21:26:17.038685+0000","last_became_active":"2024-09-10T21:26:15.534449+0000","last_became_peered":"2024-09-10T21:26:15.534449+0000","last_unstale":"2024-09-10T21:26:17.038685+0000","last_undegraded":"2024-09-10T21:26:17.038685+0000","last_fullsized":"2024-09-10T21:26:17.038685+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:14.802445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550232+0000","last_change":"2024-09-10T21:26:15.544506+0000","last_active":"2024-09-10T21:26:15.550232+0000","last_peered":"2024-09-10T21:26:15.550232+0000","last_clean":"2024-09-10T21:26:15.550232+0000","last_became_active":"2024-09-10T21:26:15.544258+0000","last_became_peered":"2024-09-10T21:26:15.544258+0000","last_unstale":"2024-09-10T21:26:15.550232+0000","last_undegraded":"2024-09-10T21:26:15.550232+0000","last_fullsized":"2024-09-10T21:26:15.550232+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547473+0000","last_change":"2024-09-10T21:26:15.542753+0000","last_active":"2024-09-10T21:26:15.547473+0000","last_peered":"2024-09-10T21:26:15.547473+0000","last_clean":"2024-09-10T21:26:15.547473+0000","last_became_active":"2024-09-10T21:26:15.542492+0000","last_became_peered":"2024-09-10T21:26:15.542492+0000","last_unstale":"2024-09-10T21:26:15.547473+0000","last_undegraded":"2024-09-10T21:26:15.547473+0000","last_fullsized":"2024-09-10T21:26:15.547473+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:02.922870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550251+0000","last_change":"2024-09-10T21:26:14.530389+0000","last_active":"2024-09-10T21:26:15.550251+0000","last_peered":"2024-09-10T21:26:15.550251+0000","last_clean":"2024-09-10T21:26:15.550251+0000","last_became_active":"2024-09-10T21:26:14.530024+0000","last_became_peered":"2024-09-10T21:26:14.530024+0000","last_unstale":"2024-09-10T21:26:15.550251+0000","last_undegraded":"2024-09-10T21:26:15.550251+0000","last_fullsized":"2024-09-10T21:26:15.550251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:44.886118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550140+0000","last_change":"2024-09-10T21:26:15.546106+0000","last_active":"2024-09-10T21:26:15.550140+0000","last_peered":"2024-09-10T21:26:15.550140+0000","last_clean":"2024-09-10T21:26:15.550140+0000","last_became_active":"2024-09-10T21:26:15.545789+0000","last_became_peered":"2024-09-10T21:26:15.545789+0000","last_unstale":"2024-09-10T21:26:15.550140+0000","last_undegraded":"2024-09-10T21:26:15.550140+0000","last_fullsized":"2024-09-10T21:26:15.550140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:20.084655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.537228+0000","last_change":"2024-09-10T21:26:14.533996+0000","last_active":"2024-09-10T21:26:15.537228+0000","last_peered":"2024-09-10T21:26:15.537228+0000","last_clean":"2024-09-10T21:26:15.537228+0000","last_became_active":"2024-09-10T21:26:14.533618+0000","last_became_peered":"2024-09-10T21:26:14.533618+0000","last_unstale":"2024-09-10T21:26:15.537228+0000","last_undegraded":"2024-09-10T21:26:15.537228+0000","last_fullsized":"2024-09-10T21:26:15.537228+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:43.185675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546302+0000","last_change":"2024-09-10T21:26:15.541441+0000","last_active":"2024-09-10T21:26:15.546302+0000","last_peered":"2024-09-10T21:26:15.546302+0000","last_clean":"2024-09-10T21:26:15.546302+0000","last_became_active":"2024-09-10T21:26:15.540879+0000","last_became_peered":"2024-09-10T21:26:15.540879+0000","last_unstale":"2024-09-10T21:26:15.546302+0000","last_undegraded":"2024-09-10T21:26:15.546302+0000","last_fullsized":"2024-09-10T21:26:15.546302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:37.706884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545342+0000","last_change":"2024-09-10T21:26:14.529412+0000","last_active":"2024-09-10T21:26:15.545342+0000","last_peered":"2024-09-10T21:26:15.545342+0000","last_clean":"2024-09-10T21:26:15.545342+0000","last_became_active":"2024-09-10T21:26:14.529112+0000","last_became_peered":"2024-09-10T21:26:14.529112+0000","last_unstale":"2024-09-10T21:26:15.545342+0000","last_undegraded":"2024-09-10T21:26:15.545342+0000","last_fullsized":"2024-09-10T21:26:15.545342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:17.649740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548940+0000","last_change":"2024-09-10T21:26:15.537699+0000","last_active":"2024-09-10T21:26:15.548940+0000","last_peered":"2024-09-10T21:26:15.548940+0000","last_clean":"2024-09-10T21:26:15.548940+0000","last_became_active":"2024-09-10T21:26:15.533943+0000","last_became_peered":"2024-09-10T21:26:15.533943+0000","last_unstale":"2024-09-10T21:26:15.548940+0000","last_undegraded":"2024-09-10T21:26:15.548940+0000","last_fullsized":"2024-09-10T21:26:15.548940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:27.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538486+0000","last_change":"2024-09-10T21:26:14.529444+0000","last_active":"2024-09-10T21:26:15.538486+0000","last_peered":"2024-09-10T21:26:15.538486+0000","last_clean":"2024-09-10T21:26:15.538486+0000","last_became_active":"2024-09-10T21:26:14.529049+0000","last_became_peered":"2024-09-10T21:26:14.529049+0000","last_unstale":"2024-09-10T21:26:15.538486+0000","last_undegraded":"2024-09-10T21:26:15.538486+0000","last_fullsized":"2024-09-10T21:26:15.538486+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:52.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541439+0000","last_change":"2024-09-10T21:26:15.536274+0000","last_active":"2024-09-10T21:26:15.541439+0000","last_peered":"2024-09-10T21:26:15.541439+0000","last_clean":"2024-09-10T21:26:15.541439+0000","last_became_active":"2024-09-10T21:26:15.536085+0000","last_became_peered":"2024-09-10T21:26:15.536085+0000","last_unstale":"2024-09-10T21:26:15.541439+0000","last_undegraded":"2024-09-10T21:26:15.541439+0000","last_fullsized":"2024-09-10T21:26:15.541439+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:10.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038897+0000","last_change":"2024-09-10T21:26:14.531407+0000","last_active":"2024-09-10T21:26:17.038897+0000","last_peered":"2024-09-10T21:26:17.038897+0000","last_clean":"2024-09-10T21:26:17.038897+0000","last_became_active":"2024-09-10T21:26:14.531128+0000","last_became_peered":"2024-09-10T21:26:14.531128+0000","last_unstale":"2024-09-10T21:26:17.038897+0000","last_undegraded":"2024-09-10T21:26:17.038897+0000","last_fullsized":"2024-09-10T21:26:17.038897+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:54.077340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544943+0000","last_change":"2024-09-10T21:26:15.540471+0000","last_active":"2024-09-10T21:26:15.544943+0000","last_peered":"2024-09-10T21:26:15.544943+0000","last_clean":"2024-09-10T21:26:15.544943+0000","last_became_active":"2024-09-10T21:26:15.539374+0000","last_became_peered":"2024-09-10T21:26:15.539374+0000","last_unstale":"2024-09-10T21:26:15.544943+0000","last_undegraded":"2024-09-10T21:26:15.544943+0000","last_fullsized":"2024-09-10T21:26:15.544943+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:53.409133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539556+0000","last_change":"2024-09-10T21:26:14.528783+0000","last_active":"2024-09-10T21:26:15.539556+0000","last_peered":"2024-09-10T21:26:15.539556+0000","last_clean":"2024-09-10T21:26:15.539556+0000","last_became_active":"2024-09-10T21:26:14.528509+0000","last_became_peered":"2024-09-10T21:26:14.528509+0000","last_unstale":"2024-09-10T21:26:15.539556+0000","last_undegraded":"2024-09-10T21:26:15.539556+0000","last_fullsized":"2024-09-10T21:26:15.539556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:48.911075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549136+0000","last_change":"2024-09-10T21:26:15.540735+0000","last_active":"2024-09-10T21:26:15.549136+0000","last_peered":"2024-09-10T21:26:15.549136+0000","last_clean":"2024-09-10T21:26:15.549136+0000","last_became_active":"2024-09-10T21:26:15.538227+0000","last_became_peered":"2024-09-10T21:26:15.538227+0000","last_unstale":"2024-09-10T21:26:15.549136+0000","last_undegraded":"2024-09-10T21:26:15.549136+0000","last_fullsized":"2024-09-10T21:26:15.549136+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:27.252740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533460+0000","last_change":"2024-09-10T21:26:14.527343+0000","last_active":"2024-09-10T21:26:15.533460+0000","last_peered":"2024-09-10T21:26:15.533460+0000","last_clean":"2024-09-10T21:26:15.533460+0000","last_became_active":"2024-09-10T21:26:14.527053+0000","last_became_peered":"2024-09-10T21:26:14.527053+0000","last_unstale":"2024-09-10T21:26:15.533460+0000","last_undegraded":"2024-09-10T21:26:15.533460+0000","last_fullsized":"2024-09-10T21:26:15.533460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:49.564562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.528756+0000","last_change":"2024-09-10T21:26:15.519138+0000","last_active":"2024-09-10T21:26:15.528756+0000","last_peered":"2024-09-10T21:26:15.528756+0000","last_clean":"2024-09-10T21:26:15.528756+0000","last_became_active":"2024-09-10T21:26:15.518903+0000","last_became_peered":"2024-09-10T21:26:15.518903+0000","last_unstale":"2024-09-10T21:26:15.528756+0000","last_undegraded":"2024-09-10T21:26:15.528756+0000","last_fullsized":"2024-09-10T21:26:15.528756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:24.898753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038776+0000","last_change":"2024-09-10T21:26:14.522827+0000","last_active":"2024-09-10T21:26:17.038776+0000","last_peered":"2024-09-10T21:26:17.038776+0000","last_clean":"2024-09-10T21:26:17.038776+0000","last_became_active":"2024-09-10T21:26:14.522537+0000","last_became_peered":"2024-09-10T21:26:14.522537+0000","last_unstale":"2024-09-10T21:26:17.038776+0000","last_undegraded":"2024-09-10T21:26:17.038776+0000","last_fullsized":"2024-09-10T21:26:17.038776+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:15.763134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533518+0000","last_change":"2024-09-10T21:26:15.528432+0000","last_active":"2024-09-10T21:26:15.533518+0000","last_peered":"2024-09-10T21:26:15.533518+0000","last_clean":"2024-09-10T21:26:15.533518+0000","last_became_active":"2024-09-10T21:26:15.526149+0000","last_became_peered":"2024-09-10T21:26:15.526149+0000","last_unstale":"2024-09-10T21:26:15.533518+0000","last_undegraded":"2024-09-10T21:26:15.533518+0000","last_fullsized":"2024-09-10T21:26:15.533518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511121+0000","last_change":"2024-09-10T21:26:14.522970+0000","last_active":"2024-09-10T21:26:15.511121+0000","last_peered":"2024-09-10T21:26:15.511121+0000","last_clean":"2024-09-10T21:26:15.511121+0000","last_became_active":"2024-09-10T21:26:14.522696+0000","last_became_peered":"2024-09-10T21:26:14.522696+0000","last_unstale":"2024-09-10T21:26:15.511121+0000","last_undegraded":"2024-09-10T21:26:15.511121+0000","last_fullsized":"2024-09-10T21:26:15.511121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:26.009061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542110+0000","last_change":"2024-09-10T21:26:15.537430+0000","last_active":"2024-09-10T21:26:15.542110+0000","last_peered":"2024-09-10T21:26:15.542110+0000","last_clean":"2024-09-10T21:26:15.542110+0000","last_became_active":"2024-09-10T21:26:15.537174+0000","last_became_peered":"2024-09-10T21:26:15.537174+0000","last_unstale":"2024-09-10T21:26:15.542110+0000","last_undegraded":"2024-09-10T21:26:15.542110+0000","last_fullsized":"2024-09-10T21:26:15.542110+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:18.652981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510906+0000","last_change":"2024-09-10T21:26:14.516829+0000","last_active":"2024-09-10T21:26:15.510906+0000","last_peered":"2024-09-10T21:26:15.510906+0000","last_clean":"2024-09-10T21:26:15.510906+0000","last_became_active":"2024-09-10T21:26:14.516564+0000","last_became_peered":"2024-09-10T21:26:14.516564+0000","last_unstale":"2024-09-10T21:26:15.510906+0000","last_undegraded":"2024-09-10T21:26:15.510906+0000","last_fullsized":"2024-09-10T21:26:15.510906+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:32.378883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549908+0000","last_change":"2024-09-10T21:26:15.545818+0000","last_active":"2024-09-10T21:26:15.549908+0000","last_peered":"2024-09-10T21:26:15.549908+0000","last_clean":"2024-09-10T21:26:15.549908+0000","last_became_active":"2024-09-10T21:26:15.545383+0000","last_became_peered":"2024-09-10T21:26:15.545383+0000","last_unstale":"2024-09-10T21:26:15.549908+0000","last_undegraded":"2024-09-10T21:26:15.549908+0000","last_fullsized":"2024-09-10T21:26:15.549908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549760+0000","last_change":"2024-09-10T21:26:14.531817+0000","last_active":"2024-09-10T21:26:15.549760+0000","last_peered":"2024-09-10T21:26:15.549760+0000","last_clean":"2024-09-10T21:26:15.549760+0000","last_became_active":"2024-09-10T21:26:14.531387+0000","last_became_peered":"2024-09-10T21:26:14.531387+0000","last_unstale":"2024-09-10T21:26:15.549760+0000","last_undegraded":"2024-09-10T21:26:15.549760+0000","last_fullsized":"2024-09-10T21:26:15.549760+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:58.645631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547276+0000","last_change":"2024-09-10T21:26:15.543162+0000","last_active":"2024-09-10T21:26:15.547276+0000","last_peered":"2024-09-10T21:26:15.547276+0000","last_clean":"2024-09-10T21:26:15.547276+0000","last_became_active":"2024-09-10T21:26:15.542913+0000","last_became_peered":"2024-09-10T21:26:15.542913+0000","last_unstale":"2024-09-10T21:26:15.547276+0000","last_undegraded":"2024-09-10T21:26:15.547276+0000","last_fullsized":"2024-09-10T21:26:15.547276+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.553669+0000","last_change":"2024-09-10T21:26:14.533065+0000","last_active":"2024-09-10T21:26:15.553669+0000","last_peered":"2024-09-10T21:26:15.553669+0000","last_clean":"2024-09-10T21:26:15.553669+0000","last_became_active":"2024-09-10T21:26:14.532788+0000","last_became_peered":"2024-09-10T21:26:14.532788+0000","last_unstale":"2024-09-10T21:26:15.553669+0000","last_undegraded":"2024-09-10T21:26:15.553669+0000","last_fullsized":"2024-09-10T21:26:15.553669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:26.313137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541575+0000","last_change":"2024-09-10T21:26:15.540226+0000","last_active":"2024-09-10T21:26:15.541575+0000","last_peered":"2024-09-10T21:26:15.541575+0000","last_clean":"2024-09-10T21:26:15.541575+0000","last_became_active":"2024-09-10T21:26:15.539910+0000","last_became_peered":"2024-09-10T21:26:15.539910+0000","last_unstale":"2024-09-10T21:26:15.541575+0000","last_undegraded":"2024-09-10T21:26:15.541575+0000","last_fullsized":"2024-09-10T21:26:15.541575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:20.009997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538731+0000","last_change":"2024-09-10T21:26:14.526625+0000","last_active":"2024-09-10T21:26:15.538731+0000","last_peered":"2024-09-10T21:26:15.538731+0000","last_clean":"2024-09-10T21:26:15.538731+0000","last_became_active":"2024-09-10T21:26:14.526301+0000","last_became_peered":"2024-09-10T21:26:14.526301+0000","last_unstale":"2024-09-10T21:26:15.538731+0000","last_undegraded":"2024-09-10T21:26:15.538731+0000","last_fullsized":"2024-09-10T21:26:15.538731+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:36.925973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550279+0000","last_change":"2024-09-10T21:26:15.547448+0000","last_active":"2024-09-10T21:26:15.550279+0000","last_peered":"2024-09-10T21:26:15.550279+0000","last_clean":"2024-09-10T21:26:15.550279+0000","last_became_active":"2024-09-10T21:26:15.546869+0000","last_became_peered":"2024-09-10T21:26:15.546869+0000","last_unstale":"2024-09-10T21:26:15.550279+0000","last_undegraded":"2024-09-10T21:26:15.550279+0000","last_fullsized":"2024-09-10T21:26:15.550279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511264+0000","last_change":"2024-09-10T21:26:14.526509+0000","last_active":"2024-09-10T21:26:15.511264+0000","last_peered":"2024-09-10T21:26:15.511264+0000","last_clean":"2024-09-10T21:26:15.511264+0000","last_became_active":"2024-09-10T21:26:14.526211+0000","last_became_peered":"2024-09-10T21:26:14.526211+0000","last_unstale":"2024-09-10T21:26:15.511264+0000","last_undegraded":"2024-09-10T21:26:15.511264+0000","last_fullsized":"2024-09-10T21:26:15.511264+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:19.196183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539061+0000","last_change":"2024-09-10T21:26:15.533375+0000","last_active":"2024-09-10T21:26:15.539061+0000","last_peered":"2024-09-10T21:26:15.539061+0000","last_clean":"2024-09-10T21:26:15.539061+0000","last_became_active":"2024-09-10T21:26:15.533154+0000","last_became_peered":"2024-09-10T21:26:15.533154+0000","last_unstale":"2024-09-10T21:26:15.539061+0000","last_undegraded":"2024-09-10T21:26:15.539061+0000","last_fullsized":"2024-09-10T21:26:15.539061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:56.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.525900+0000","last_change":"2024-09-10T21:26:14.523579+0000","last_active":"2024-09-10T21:26:15.525900+0000","last_peered":"2024-09-10T21:26:15.525900+0000","last_clean":"2024-09-10T21:26:15.525900+0000","last_became_active":"2024-09-10T21:26:14.523321+0000","last_became_peered":"2024-09-10T21:26:14.523321+0000","last_unstale":"2024-09-10T21:26:15.525900+0000","last_undegraded":"2024-09-10T21:26:15.525900+0000","last_fullsized":"2024-09-10T21:26:15.525900+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:39.550410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039427+0000","last_change":"2024-09-10T21:26:15.528299+0000","last_active":"2024-09-10T21:26:17.039427+0000","last_peered":"2024-09-10T21:26:17.039427+0000","last_clean":"2024-09-10T21:26:17.039427+0000","last_became_active":"2024-09-10T21:26:15.528045+0000","last_became_peered":"2024-09-10T21:26:15.528045+0000","last_unstale":"2024-09-10T21:26:17.039427+0000","last_undegraded":"2024-09-10T21:26:17.039427+0000","last_fullsized":"2024-09-10T21:26:17.039427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:43.372388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538375+0000","last_change":"2024-09-10T21:26:14.526055+0000","last_active":"2024-09-10T21:26:15.538375+0000","last_peered":"2024-09-10T21:26:15.538375+0000","last_clean":"2024-09-10T21:26:15.538375+0000","last_became_active":"2024-09-10T21:26:14.525734+0000","last_became_peered":"2024-09-10T21:26:14.525734+0000","last_unstale":"2024-09-10T21:26:15.538375+0000","last_undegraded":"2024-09-10T21:26:15.538375+0000","last_fullsized":"2024-09-10T21:26:15.538375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:15.163346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549528+0000","last_change":"2024-09-10T21:26:15.545468+0000","last_active":"2024-09-10T21:26:15.549528+0000","last_peered":"2024-09-10T21:26:15.549528+0000","last_clean":"2024-09-10T21:26:15.549528+0000","last_became_active":"2024-09-10T21:26:15.545137+0000","last_became_peered":"2024-09-10T21:26:15.545137+0000","last_unstale":"2024-09-10T21:26:15.549528+0000","last_undegraded":"2024-09-10T21:26:15.549528+0000","last_fullsized":"2024-09-10T21:26:15.549528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:12.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538792+0000","last_change":"2024-09-10T21:26:14.534718+0000","last_active":"2024-09-10T21:26:15.538792+0000","last_peered":"2024-09-10T21:26:15.538792+0000","last_clean":"2024-09-10T21:26:15.538792+0000","last_became_active":"2024-09-10T21:26:14.534222+0000","last_became_peered":"2024-09-10T21:26:14.534222+0000","last_unstale":"2024-09-10T21:26:15.538792+0000","last_undegraded":"2024-09-10T21:26:15.538792+0000","last_fullsized":"2024-09-10T21:26:15.538792+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:10.810025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550202+0000","last_change":"2024-09-10T21:26:15.546824+0000","last_active":"2024-09-10T21:26:15.550202+0000","last_peered":"2024-09-10T21:26:15.550202+0000","last_clean":"2024-09-10T21:26:15.550202+0000","last_became_active":"2024-09-10T21:26:15.546286+0000","last_became_peered":"2024-09-10T21:26:15.546286+0000","last_unstale":"2024-09-10T21:26:15.550202+0000","last_undegraded":"2024-09-10T21:26:15.550202+0000","last_fullsized":"2024-09-10T21:26:15.550202+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:45.889359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533005+0000","last_change":"2024-09-10T21:26:14.527929+0000","last_active":"2024-09-10T21:26:15.533005+0000","last_peered":"2024-09-10T21:26:15.533005+0000","last_clean":"2024-09-10T21:26:15.533005+0000","last_became_active":"2024-09-10T21:26:14.527583+0000","last_became_peered":"2024-09-10T21:26:14.527583+0000","last_unstale":"2024-09-10T21:26:15.533005+0000","last_undegraded":"2024-09-10T21:26:15.533005+0000","last_fullsized":"2024-09-10T21:26:15.533005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:20.085776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545249+0000","last_change":"2024-09-10T21:26:15.541261+0000","last_active":"2024-09-10T21:26:15.545249+0000","last_peered":"2024-09-10T21:26:15.545249+0000","last_clean":"2024-09-10T21:26:15.545249+0000","last_became_active":"2024-09-10T21:26:15.541018+0000","last_became_peered":"2024-09-10T21:26:15.541018+0000","last_unstale":"2024-09-10T21:26:15.545249+0000","last_undegraded":"2024-09-10T21:26:15.545249+0000","last_fullsized":"2024-09-10T21:26:15.545249+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539384+0000","last_change":"2024-09-10T21:26:14.528002+0000","last_active":"2024-09-10T21:26:15.539384+0000","last_peered":"2024-09-10T21:26:15.539384+0000","last_clean":"2024-09-10T21:26:15.539384+0000","last_became_active":"2024-09-10T21:26:14.527640+0000","last_became_peered":"2024-09-10T21:26:14.527640+0000","last_unstale":"2024-09-10T21:26:15.539384+0000","last_undegraded":"2024-09-10T21:26:15.539384+0000","last_fullsized":"2024-09-10T21:26:15.539384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:40.905930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542762+0000","last_change":"2024-09-10T21:26:15.533913+0000","last_active":"2024-09-10T21:26:15.542762+0000","last_peered":"2024-09-10T21:26:15.542762+0000","last_clean":"2024-09-10T21:26:15.542762+0000","last_became_active":"2024-09-10T21:26:15.533607+0000","last_became_peered":"2024-09-10T21:26:15.533607+0000","last_unstale":"2024-09-10T21:26:15.542762+0000","last_undegraded":"2024-09-10T21:26:15.542762+0000","last_fullsized":"2024-09-10T21:26:15.542762+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:53.763483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524585+0000","last_change":"2024-09-10T21:26:14.530739+0000","last_active":"2024-09-10T21:26:15.524585+0000","last_peered":"2024-09-10T21:26:15.524585+0000","last_clean":"2024-09-10T21:26:15.524585+0000","last_became_active":"2024-09-10T21:26:14.530428+0000","last_became_peered":"2024-09-10T21:26:14.530428+0000","last_unstale":"2024-09-10T21:26:15.524585+0000","last_undegraded":"2024-09-10T21:26:15.524585+0000","last_fullsized":"2024-09-10T21:26:15.524585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:47.051011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547356+0000","last_change":"2024-09-10T21:26:15.543515+0000","last_active":"2024-09-10T21:26:15.547356+0000","last_peered":"2024-09-10T21:26:15.547356+0000","last_clean":"2024-09-10T21:26:15.547356+0000","last_became_active":"2024-09-10T21:26:15.543291+0000","last_became_peered":"2024-09-10T21:26:15.543291+0000","last_unstale":"2024-09-10T21:26:15.547356+0000","last_undegraded":"2024-09-10T21:26:15.547356+0000","last_fullsized":"2024-09-10T21:26:15.547356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551224+0000","last_change":"2024-09-10T21:26:14.532927+0000","last_active":"2024-09-10T21:26:15.551224+0000","last_peered":"2024-09-10T21:26:15.551224+0000","last_clean":"2024-09-10T21:26:15.551224+0000","last_became_active":"2024-09-10T21:26:14.532625+0000","last_became_peered":"2024-09-10T21:26:14.532625+0000","last_unstale":"2024-09-10T21:26:15.551224+0000","last_undegraded":"2024-09-10T21:26:15.551224+0000","last_fullsized":"2024-09-10T21:26:15.551224+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:36.431767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532586+0000","last_change":"2024-09-10T21:26:15.529708+0000","last_active":"2024-09-10T21:26:15.532586+0000","last_peered":"2024-09-10T21:26:15.532586+0000","last_clean":"2024-09-10T21:26:15.532586+0000","last_became_active":"2024-09-10T21:26:15.529431+0000","last_became_peered":"2024-09-10T21:26:15.529431+0000","last_unstale":"2024-09-10T21:26:15.532586+0000","last_undegraded":"2024-09-10T21:26:15.532586+0000","last_fullsized":"2024-09-10T21:26:15.532586+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:08.029645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038684+0000","last_change":"2024-09-10T21:26:14.526244+0000","last_active":"2024-09-10T21:26:17.038684+0000","last_peered":"2024-09-10T21:26:17.038684+0000","last_clean":"2024-09-10T21:26:17.038684+0000","last_became_active":"2024-09-10T21:26:14.525952+0000","last_became_peered":"2024-09-10T21:26:14.525952+0000","last_unstale":"2024-09-10T21:26:17.038684+0000","last_undegraded":"2024-09-10T21:26:17.038684+0000","last_fullsized":"2024-09-10T21:26:17.038684+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:48.228747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549692+0000","last_change":"2024-09-10T21:26:15.544629+0000","last_active":"2024-09-10T21:26:15.549692+0000","last_peered":"2024-09-10T21:26:15.549692+0000","last_clean":"2024-09-10T21:26:15.549692+0000","last_became_active":"2024-09-10T21:26:15.540919+0000","last_became_peered":"2024-09-10T21:26:15.540919+0000","last_unstale":"2024-09-10T21:26:15.549692+0000","last_undegraded":"2024-09-10T21:26:15.549692+0000","last_fullsized":"2024-09-10T21:26:15.549692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520315+0000","last_change":"2024-09-10T21:26:14.529271+0000","last_active":"2024-09-10T21:26:15.520315+0000","last_peered":"2024-09-10T21:26:15.520315+0000","last_clean":"2024-09-10T21:26:15.520315+0000","last_became_active":"2024-09-10T21:26:14.528963+0000","last_became_peered":"2024-09-10T21:26:14.528963+0000","last_unstale":"2024-09-10T21:26:15.520315+0000","last_undegraded":"2024-09-10T21:26:15.520315+0000","last_fullsized":"2024-09-10T21:26:15.520315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:14.592093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544636+0000","last_change":"2024-09-10T21:26:15.540533+0000","last_active":"2024-09-10T21:26:15.544636+0000","last_peered":"2024-09-10T21:26:15.544636+0000","last_clean":"2024-09-10T21:26:15.544636+0000","last_became_active":"2024-09-10T21:26:15.540237+0000","last_became_peered":"2024-09-10T21:26:15.540237+0000","last_unstale":"2024-09-10T21:26:15.544636+0000","last_undegraded":"2024-09-10T21:26:15.544636+0000","last_fullsized":"2024-09-10T21:26:15.544636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:49.612063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520099+0000","last_change":"2024-09-10T21:26:14.529847+0000","last_active":"2024-09-10T21:26:15.520099+0000","last_peered":"2024-09-10T21:26:15.520099+0000","last_clean":"2024-09-10T21:26:15.520099+0000","last_became_active":"2024-09-10T21:26:14.529074+0000","last_became_peered":"2024-09-10T21:26:14.529074+0000","last_unstale":"2024-09-10T21:26:15.520099+0000","last_undegraded":"2024-09-10T21:26:15.520099+0000","last_fullsized":"2024-09-10T21:26:15.520099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:27.478486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532977+0000","last_change":"2024-09-10T21:26:15.528858+0000","last_active":"2024-09-10T21:26:15.532977+0000","last_peered":"2024-09-10T21:26:15.532977+0000","last_clean":"2024-09-10T21:26:15.532977+0000","last_became_active":"2024-09-10T21:26:15.527735+0000","last_became_peered":"2024-09-10T21:26:15.527735+0000","last_unstale":"2024-09-10T21:26:15.532977+0000","last_undegraded":"2024-09-10T21:26:15.532977+0000","last_fullsized":"2024-09-10T21:26:15.532977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:31.109223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544806+0000","last_change":"2024-09-10T21:26:14.529877+0000","last_active":"2024-09-10T21:26:15.544806+0000","last_peered":"2024-09-10T21:26:15.544806+0000","last_clean":"2024-09-10T21:26:15.544806+0000","last_became_active":"2024-09-10T21:26:14.529580+0000","last_became_peered":"2024-09-10T21:26:14.529580+0000","last_unstale":"2024-09-10T21:26:15.544806+0000","last_undegraded":"2024-09-10T21:26:15.544806+0000","last_fullsized":"2024-09-10T21:26:15.544806+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:41.737020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549409+0000","last_change":"2024-09-10T21:26:15.541706+0000","last_active":"2024-09-10T21:26:15.549409+0000","last_peered":"2024-09-10T21:26:15.549409+0000","last_clean":"2024-09-10T21:26:15.549409+0000","last_became_active":"2024-09-10T21:26:15.540365+0000","last_became_peered":"2024-09-10T21:26:15.540365+0000","last_unstale":"2024-09-10T21:26:15.549409+0000","last_undegraded":"2024-09-10T21:26:15.549409+0000","last_fullsized":"2024-09-10T21:26:15.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:43.648100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.530063+0000","last_change":"2024-09-10T21:26:14.532072+0000","last_active":"2024-09-10T21:26:15.530063+0000","last_peered":"2024-09-10T21:26:15.530063+0000","last_clean":"2024-09-10T21:26:15.530063+0000","last_became_active":"2024-09-10T21:26:14.530711+0000","last_became_peered":"2024-09-10T21:26:14.530711+0000","last_unstale":"2024-09-10T21:26:15.530063+0000","last_undegraded":"2024-09-10T21:26:15.530063+0000","last_fullsized":"2024-09-10T21:26:15.530063+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:15.362786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038829+0000","last_change":"2024-09-10T21:26:15.524631+0000","last_active":"2024-09-10T21:26:17.038829+0000","last_peered":"2024-09-10T21:26:17.038829+0000","last_clean":"2024-09-10T21:26:17.038829+0000","last_became_active":"2024-09-10T21:26:15.524305+0000","last_became_peered":"2024-09-10T21:26:15.524305+0000","last_unstale":"2024-09-10T21:26:17.038829+0000","last_undegraded":"2024-09-10T21:26:17.038829+0000","last_fullsized":"2024-09-10T21:26:17.038829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:36.491077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039016+0000","last_change":"2024-09-10T21:26:14.525570+0000","last_active":"2024-09-10T21:26:17.039016+0000","last_peered":"2024-09-10T21:26:17.039016+0000","last_clean":"2024-09-10T21:26:17.039016+0000","last_became_active":"2024-09-10T21:26:14.525294+0000","last_became_peered":"2024-09-10T21:26:14.525294+0000","last_unstale":"2024-09-10T21:26:17.039016+0000","last_undegraded":"2024-09-10T21:26:17.039016+0000","last_fullsized":"2024-09-10T21:26:17.039016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:26.579850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549756+0000","last_change":"2024-09-10T21:26:15.542738+0000","last_active":"2024-09-10T21:26:15.549756+0000","last_peered":"2024-09-10T21:26:15.549756+0000","last_clean":"2024-09-10T21:26:15.549756+0000","last_became_active":"2024-09-10T21:26:15.542497+0000","last_became_peered":"2024-09-10T21:26:15.542497+0000","last_unstale":"2024-09-10T21:26:15.549756+0000","last_undegraded":"2024-09-10T21:26:15.549756+0000","last_fullsized":"2024-09-10T21:26:15.549756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:59.431959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520974+0000","last_change":"2024-09-10T21:26:14.530559+0000","last_active":"2024-09-10T21:26:15.520974+0000","last_peered":"2024-09-10T21:26:15.520974+0000","last_clean":"2024-09-10T21:26:15.520974+0000","last_became_active":"2024-09-10T21:26:14.529513+0000","last_became_peered":"2024-09-10T21:26:14.529513+0000","last_unstale":"2024-09-10T21:26:15.520974+0000","last_undegraded":"2024-09-10T21:26:15.520974+0000","last_fullsized":"2024-09-10T21:26:15.520974+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:10.595223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538120+0000","last_change":"2024-09-10T21:26:15.533560+0000","last_active":"2024-09-10T21:26:15.538120+0000","last_peered":"2024-09-10T21:26:15.538120+0000","last_clean":"2024-09-10T21:26:15.538120+0000","last_became_active":"2024-09-10T21:26:15.533313+0000","last_became_peered":"2024-09-10T21:26:15.533313+0000","last_unstale":"2024-09-10T21:26:15.538120+0000","last_undegraded":"2024-09-10T21:26:15.538120+0000","last_fullsized":"2024-09-10T21:26:15.538120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038798+0000","last_change":"2024-09-10T21:26:14.525715+0000","last_active":"2024-09-10T21:26:17.038798+0000","last_peered":"2024-09-10T21:26:17.038798+0000","last_clean":"2024-09-10T21:26:17.038798+0000","last_became_active":"2024-09-10T21:26:14.525367+0000","last_became_peered":"2024-09-10T21:26:14.525367+0000","last_unstale":"2024-09-10T21:26:17.038798+0000","last_undegraded":"2024-09-10T21:26:17.038798+0000","last_fullsized":"2024-09-10T21:26:17.038798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:36.377629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548560+0000","last_change":"2024-09-10T21:26:15.536138+0000","last_active":"2024-09-10T21:26:15.548560+0000","last_peered":"2024-09-10T21:26:15.548560+0000","last_clean":"2024-09-10T21:26:15.548560+0000","last_became_active":"2024-09-10T21:26:15.535521+0000","last_became_peered":"2024-09-10T21:26:15.535521+0000","last_unstale":"2024-09-10T21:26:15.548560+0000","last_undegraded":"2024-09-10T21:26:15.548560+0000","last_fullsized":"2024-09-10T21:26:15.548560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:20.575951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.526019+0000","last_change":"2024-09-10T21:26:14.519898+0000","last_active":"2024-09-10T21:26:15.526019+0000","last_peered":"2024-09-10T21:26:15.526019+0000","last_clean":"2024-09-10T21:26:15.526019+0000","last_became_active":"2024-09-10T21:26:14.519606+0000","last_became_peered":"2024-09-10T21:26:14.519606+0000","last_unstale":"2024-09-10T21:26:15.526019+0000","last_undegraded":"2024-09-10T21:26:15.526019+0000","last_fullsized":"2024-09-10T21:26:15.526019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.608822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544613+0000","last_change":"2024-09-10T21:26:15.532747+0000","last_active":"2024-09-10T21:26:15.544613+0000","last_peered":"2024-09-10T21:26:15.544613+0000","last_clean":"2024-09-10T21:26:15.544613+0000","last_became_active":"2024-09-10T21:26:15.532482+0000","last_became_peered":"2024-09-10T21:26:15.532482+0000","last_unstale":"2024-09-10T21:26:15.544613+0000","last_undegraded":"2024-09-10T21:26:15.544613+0000","last_fullsized":"2024-09-10T21:26:15.544613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:40.553651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.509867+0000","last_change":"2024-09-10T21:26:14.526821+0000","last_active":"2024-09-10T21:26:15.509867+0000","last_peered":"2024-09-10T21:26:15.509867+0000","last_clean":"2024-09-10T21:26:15.509867+0000","last_became_active":"2024-09-10T21:26:14.526561+0000","last_became_peered":"2024-09-10T21:26:14.526561+0000","last_unstale":"2024-09-10T21:26:15.509867+0000","last_undegraded":"2024-09-10T21:26:15.509867+0000","last_fullsized":"2024-09-10T21:26:15.509867+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541912+0000","last_change":"2024-09-10T21:26:15.538996+0000","last_active":"2024-09-10T21:26:15.541912+0000","last_peered":"2024-09-10T21:26:15.541912+0000","last_clean":"2024-09-10T21:26:15.541912+0000","last_became_active":"2024-09-10T21:26:15.538393+0000","last_became_peered":"2024-09-10T21:26:15.538393+0000","last_unstale":"2024-09-10T21:26:15.541912+0000","last_undegraded":"2024-09-10T21:26:15.541912+0000","last_fullsized":"2024-09-10T21:26:15.541912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:33.376220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524045+0000","last_change":"2024-09-10T21:26:14.533819+0000","last_active":"2024-09-10T21:26:15.524045+0000","last_peered":"2024-09-10T21:26:15.524045+0000","last_clean":"2024-09-10T21:26:15.524045+0000","last_became_active":"2024-09-10T21:26:14.533554+0000","last_became_peered":"2024-09-10T21:26:14.533554+0000","last_unstale":"2024-09-10T21:26:15.524045+0000","last_undegraded":"2024-09-10T21:26:15.524045+0000","last_fullsized":"2024-09-10T21:26:15.524045+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:55.449678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510018+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:26:15.510018+0000","last_peered":"2024-09-10T21:26:15.510018+0000","last_clean":"2024-09-10T21:26:15.510018+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:26:15.510018+0000","last_undegraded":"2024-09-10T21:26:15.510018+0000","last_fullsized":"2024-09-10T21:26:15.510018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":275651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1216512,"data_stored":865963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":610304,"data_stored":268428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1204224,"data_stored":858796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":283991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:26:20.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:26:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- 172.21.15.97:0/3641840167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 msgr2=0x7fe5d40cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3641840167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d40cff50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009840 tx=0x7fe5bc01c790 comp rx=0 tx=0).stop 2024-09-10T21:26:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- 172.21.15.97:0/3641840167 shutdown_connections 2024-09-10T21:26:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3641840167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d40cff50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- 172.21.15.97:0/3641840167 >> 172.21.15.97:0/3641840167 conn(0x7fe5d40d1a60 msgr2=0x7fe5d40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- 172.21.15.97:0/3641840167 shutdown_connections 2024-09-10T21:26:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- 172.21.15.97:0/3641840167 wait complete. 2024-09-10T21:26:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 Processor -- start 2024-09-10T21:26:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- start start 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d4156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d40d0490 0x7fe5d41f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d415efc0 0x7fe5d40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d4169b50 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41699d0 con 0x7fe5d40d0490 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d4169cd0 con 0x7fe5d415efc0 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d4156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d4156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41654/0 (socket says 172.21.15.97:41654) 2024-09-10T21:26:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d1e00640 1 -- 172.21.15.97:0/3750424130 learned_addr learned my addr 172.21.15.97:0/3750424130 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5c9400640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d40d0490 0x7fe5d41f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.599+0000 7fe5d2800640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d415efc0 0x7fe5d40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d1e00640 1 -- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d415efc0 msgr2=0x7fe5d40b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d1e00640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d415efc0 0x7fe5d40b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d1e00640 1 -- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d40d0490 msgr2=0x7fe5d41f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d1e00640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d40d0490 0x7fe5d41f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d1e00640 1 -- 172.21.15.97:0/3750424130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d40b3270 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d2800640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d415efc0 0x7fe5d40b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d1e00640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d4156100 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c3f0 tx=0x7fe5c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5caa00640 1 -- 172.21.15.97:0/3750424130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4017070 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5caa00640 1 -- 172.21.15.97:0/3750424130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5c4003600 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5caa00640 1 -- 172.21.15.97:0/3750424130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4003a50 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d41670c0 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d41f7920 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d40ae930 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5caa00640 1 -- 172.21.15.97:0/3750424130 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe5c4003bf0 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5caa00640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5b00725e0 0x7fe5b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:20.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.603+0000 7fe5caa00640 1 -- 172.21.15.97:0/3750424130 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe5c40928e0 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.607+0000 7fe5c9400640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5b00725e0 0x7fe5b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:20.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.607+0000 7fe5c9400640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5b00725e0 0x7fe5b0074aa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0af000 tx=0x7fe5bc01afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.607+0000 7fe5caa00640 1 -- 172.21.15.97:0/3750424130 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5c405d7d0 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe5d41577b0 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5caa00640 1 -- 172.21.15.97:0/3750424130 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1216 (secure 0 0 0) 0x7fe5c4062200 con 0x7fe5d40cdcb0 2024-09-10T21:26:20.834 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:20.834 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":154,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1726003441,"num_in_osds":8,"osd_in_since":1726003430,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":227622912,"bytes_avail":772866490368,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":33,"btime":"2024-09-10T21:26:18:534778+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5484}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T21:26:06.097266+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T21:26:20.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5b00725e0 msgr2=0x7fe5b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5b00725e0 0x7fe5b0074aa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0af000 tx=0x7fe5bc01afd0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 msgr2=0x7fe5d4156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d4156100 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c3f0 tx=0x7fe5c400c8b0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 shutdown_connections 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5b00725e0 0x7fe5b0074aa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe5d415efc0 0x7fe5d40b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5d40d0490 0x7fe5d41f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 --2- 172.21.15.97:0/3750424130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5d40cdcb0 0x7fe5d4156100 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 >> 172.21.15.97:0/3750424130 conn(0x7fe5d40d1a60 msgr2=0x7fe5d40b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 shutdown_connections 2024-09-10T21:26:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.831+0000 7fe5d3200640 1 -- 172.21.15.97:0/3750424130 wait complete. 2024-09-10T21:26:20.855 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:26:20.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- 172.21.15.97:0/922523755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 msgr2=0x7f3e10141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 --2- 172.21.15.97:0/922523755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e10141d20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009900 tx=0x7f3e0001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- 172.21.15.97:0/922523755 shutdown_connections 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 --2- 172.21.15.97:0/922523755 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e1015fde0 0x7f3e1013f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 --2- 172.21.15.97:0/922523755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e10141d20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- 172.21.15.97:0/922523755 >> 172.21.15.97:0/922523755 conn(0x7f3e100b1780 msgr2=0x7f3e100b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- 172.21.15.97:0/922523755 shutdown_connections 2024-09-10T21:26:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- 172.21.15.97:0/922523755 wait complete. 2024-09-10T21:26:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 Processor -- start 2024-09-10T21:26:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- start start 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e1013a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e100ce9d0 0x7f3e101423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e1015fde0 0x7f3e1015e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e10168a70 con 0x7f3e100ce5f0 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3e10140d00 con 0x7f3e1015fde0 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e16200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3e10168bf0 con 0x7f3e100ce9d0 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e1013a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e1013a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41670/0 (socket says 172.21.15.97:41670) 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 -- 172.21.15.97:0/2886426332 learned_addr learned my addr 172.21.15.97:0/2886426332 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 -- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e100ce9d0 msgr2=0x7f3e101423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e100ce9d0 0x7f3e101423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 -- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e1015fde0 msgr2=0x7f3e1015e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e1015fde0 0x7f3e1015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.995+0000 7f3e0f400640 1 -- 172.21.15.97:0/2886426332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e10155d10 con 0x7f3e100ce5f0 2024-09-10T21:26:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e0f400640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e1013a860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009850 tx=0x7f3e00004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e07e00640 1 -- 172.21.15.97:0/2886426332 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000bb070 con 0x7f3e100ce5f0 2024-09-10T21:26:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e07e00640 1 -- 172.21.15.97:0/2886426332 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e00003600 con 0x7f3e100ce5f0 2024-09-10T21:26:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e10155ff0 con 0x7f3e100ce5f0 2024-09-10T21:26:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e07e00640 1 -- 172.21.15.97:0/2886426332 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000b2040 con 0x7f3e100ce5f0 2024-09-10T21:26:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e101558a0 con 0x7f3e100ce5f0 2024-09-10T21:26:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e10156230 con 0x7f3e100ce5f0 2024-09-10T21:26:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e07e00640 1 -- 172.21.15.97:0/2886426332 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3e00003a10 con 0x7f3e100ce5f0 2024-09-10T21:26:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e07e00640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dec0725e0 0x7f3dec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:20.999+0000 7f3e07e00640 1 -- 172.21.15.97:0/2886426332 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3e0012d110 con 0x7f3e100ce5f0 2024-09-10T21:26:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.003+0000 7f3e06a00640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dec0725e0 0x7f3dec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.003+0000 7f3e06a00640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dec0725e0 0x7f3dec074aa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000f70 tx=0x7f3df80079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.003+0000 7f3e07e00640 1 -- 172.21.15.97:0/2886426332 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3e10156230 con 0x7f3e100ce5f0 2024-09-10T21:26:21.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.167+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e10141d20 con 0x7f3dec0725e0 2024-09-10T21:26:21.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.171+0000 7f3e07e00640 1 -- 172.21.15.97:0/2886426332 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f3e10141d20 con 0x7f3dec0725e0 2024-09-10T21:26:21.178 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:21.180 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:26:21.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dec0725e0 msgr2=0x7f3dec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dec0725e0 0x7f3dec074aa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000f70 tx=0x7f3df80079c0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 msgr2=0x7f3e1013a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e1013a860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009850 tx=0x7f3e00004040 comp rx=0 tx=0).stop 2024-09-10T21:26:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 shutdown_connections 2024-09-10T21:26:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dec0725e0 0x7f3dec074aa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e1015fde0 0x7f3e1015e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e100ce9d0 0x7f3e101423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 --2- 172.21.15.97:0/2886426332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e100ce5f0 0x7f3e1013a860 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 >> 172.21.15.97:0/2886426332 conn(0x7f3e100b1780 msgr2=0x7f3e1015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 shutdown_connections 2024-09-10T21:26:21.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:21.175+0000 7f3e16200640 1 -- 172.21.15.97:0/2886426332 wait complete. 2024-09-10T21:26:21.200 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-10T21:26:20.100580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222288,"kb_used_data":6052,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752432,"statfs":{"total":773094113280,"available":772866490368,"internally_reserved":0,"allocated":6197248,"data_stored":3380868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001474"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549183+0000","last_change":"2024-09-10T21:26:15.539541+0000","last_active":"2024-09-10T21:26:15.549183+0000","last_peered":"2024-09-10T21:26:15.549183+0000","last_clean":"2024-09-10T21:26:15.549183+0000","last_became_active":"2024-09-10T21:26:15.539095+0000","last_became_peered":"2024-09-10T21:26:15.539095+0000","last_unstale":"2024-09-10T21:26:15.549183+0000","last_undegraded":"2024-09-10T21:26:15.549183+0000","last_fullsized":"2024-09-10T21:26:15.549183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.543900+0000","last_change":"2024-09-10T21:26:15.535693+0000","last_active":"2024-09-10T21:26:15.543900+0000","last_peered":"2024-09-10T21:26:15.543900+0000","last_clean":"2024-09-10T21:26:15.543900+0000","last_became_active":"2024-09-10T21:26:15.535421+0000","last_became_peered":"2024-09-10T21:26:15.535421+0000","last_unstale":"2024-09-10T21:26:15.543900+0000","last_undegraded":"2024-09-10T21:26:15.543900+0000","last_fullsized":"2024-09-10T21:26:15.543900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039167+0000","last_change":"2024-09-10T21:26:15.537145+0000","last_active":"2024-09-10T21:26:17.039167+0000","last_peered":"2024-09-10T21:26:17.039167+0000","last_clean":"2024-09-10T21:26:17.039167+0000","last_became_active":"2024-09-10T21:26:15.536886+0000","last_became_peered":"2024-09-10T21:26:15.536886+0000","last_unstale":"2024-09-10T21:26:17.039167+0000","last_undegraded":"2024-09-10T21:26:17.039167+0000","last_fullsized":"2024-09-10T21:26:17.039167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:20.043744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546167+0000","last_change":"2024-09-10T21:26:15.537322+0000","last_active":"2024-09-10T21:26:15.546167+0000","last_peered":"2024-09-10T21:26:15.546167+0000","last_clean":"2024-09-10T21:26:15.546167+0000","last_became_active":"2024-09-10T21:26:15.537039+0000","last_became_peered":"2024-09-10T21:26:15.537039+0000","last_unstale":"2024-09-10T21:26:15.546167+0000","last_undegraded":"2024-09-10T21:26:15.546167+0000","last_fullsized":"2024-09-10T21:26:15.546167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:11.598464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551006+0000","last_change":"2024-09-10T21:26:15.547237+0000","last_active":"2024-09-10T21:26:15.551006+0000","last_peered":"2024-09-10T21:26:15.551006+0000","last_clean":"2024-09-10T21:26:15.551006+0000","last_became_active":"2024-09-10T21:26:15.546563+0000","last_became_peered":"2024-09-10T21:26:15.546563+0000","last_unstale":"2024-09-10T21:26:15.551006+0000","last_undegraded":"2024-09-10T21:26:15.551006+0000","last_fullsized":"2024-09-10T21:26:15.551006+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544695+0000","last_change":"2024-09-10T21:26:15.534547+0000","last_active":"2024-09-10T21:26:15.544695+0000","last_peered":"2024-09-10T21:26:15.544695+0000","last_clean":"2024-09-10T21:26:15.544695+0000","last_became_active":"2024-09-10T21:26:15.534280+0000","last_became_peered":"2024-09-10T21:26:15.534280+0000","last_unstale":"2024-09-10T21:26:15.544695+0000","last_undegraded":"2024-09-10T21:26:15.544695+0000","last_fullsized":"2024-09-10T21:26:15.544695+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:48.054252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548817+0000","last_change":"2024-09-10T21:26:15.533183+0000","last_active":"2024-09-10T21:26:15.548817+0000","last_peered":"2024-09-10T21:26:15.548817+0000","last_clean":"2024-09-10T21:26:15.548817+0000","last_became_active":"2024-09-10T21:26:15.532995+0000","last_became_peered":"2024-09-10T21:26:15.532995+0000","last_unstale":"2024-09-10T21:26:15.548817+0000","last_undegraded":"2024-09-10T21:26:15.548817+0000","last_fullsized":"2024-09-10T21:26:15.548817+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:27.005167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546214+0000","last_change":"2024-09-10T21:26:15.541612+0000","last_active":"2024-09-10T21:26:15.546214+0000","last_peered":"2024-09-10T21:26:15.546214+0000","last_clean":"2024-09-10T21:26:15.546214+0000","last_became_active":"2024-09-10T21:26:15.541343+0000","last_became_peered":"2024-09-10T21:26:15.541343+0000","last_unstale":"2024-09-10T21:26:15.546214+0000","last_undegraded":"2024-09-10T21:26:15.546214+0000","last_fullsized":"2024-09-10T21:26:15.546214+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039368+0000","last_change":"2024-09-10T21:26:15.531640+0000","last_active":"2024-09-10T21:26:17.039368+0000","last_peered":"2024-09-10T21:26:17.039368+0000","last_clean":"2024-09-10T21:26:17.039368+0000","last_became_active":"2024-09-10T21:26:15.531347+0000","last_became_peered":"2024-09-10T21:26:15.531347+0000","last_unstale":"2024-09-10T21:26:17.039368+0000","last_undegraded":"2024-09-10T21:26:17.039368+0000","last_fullsized":"2024-09-10T21:26:17.039368+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:34.692481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548595+0000","last_change":"2024-09-10T21:26:15.536440+0000","last_active":"2024-09-10T21:26:15.548595+0000","last_peered":"2024-09-10T21:26:15.548595+0000","last_clean":"2024-09-10T21:26:15.548595+0000","last_became_active":"2024-09-10T21:26:15.535896+0000","last_became_peered":"2024-09-10T21:26:15.535896+0000","last_unstale":"2024-09-10T21:26:15.548595+0000","last_undegraded":"2024-09-10T21:26:15.548595+0000","last_fullsized":"2024-09-10T21:26:15.548595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:57.577430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549667+0000","last_change":"2024-09-10T21:26:15.538325+0000","last_active":"2024-09-10T21:26:15.549667+0000","last_peered":"2024-09-10T21:26:15.549667+0000","last_clean":"2024-09-10T21:26:15.549667+0000","last_became_active":"2024-09-10T21:26:15.536197+0000","last_became_peered":"2024-09-10T21:26:15.536197+0000","last_unstale":"2024-09-10T21:26:15.549667+0000","last_undegraded":"2024-09-10T21:26:15.549667+0000","last_fullsized":"2024-09-10T21:26:15.549667+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520191+0000","last_change":"2024-09-10T21:26:15.518220+0000","last_active":"2024-09-10T21:26:15.520191+0000","last_peered":"2024-09-10T21:26:15.520191+0000","last_clean":"2024-09-10T21:26:15.520191+0000","last_became_active":"2024-09-10T21:26:15.516898+0000","last_became_peered":"2024-09-10T21:26:15.516898+0000","last_unstale":"2024-09-10T21:26:15.520191+0000","last_undegraded":"2024-09-10T21:26:15.520191+0000","last_fullsized":"2024-09-10T21:26:15.520191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:52.482589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547572+0000","last_change":"2024-09-10T21:26:15.545455+0000","last_active":"2024-09-10T21:26:15.547572+0000","last_peered":"2024-09-10T21:26:15.547572+0000","last_clean":"2024-09-10T21:26:15.547572+0000","last_became_active":"2024-09-10T21:26:15.545207+0000","last_became_peered":"2024-09-10T21:26:15.545207+0000","last_unstale":"2024-09-10T21:26:15.547572+0000","last_undegraded":"2024-09-10T21:26:15.547572+0000","last_fullsized":"2024-09-10T21:26:15.547572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549843+0000","last_change":"2024-09-10T21:26:15.545646+0000","last_active":"2024-09-10T21:26:15.549843+0000","last_peered":"2024-09-10T21:26:15.549843+0000","last_clean":"2024-09-10T21:26:15.549843+0000","last_became_active":"2024-09-10T21:26:15.544941+0000","last_became_peered":"2024-09-10T21:26:15.544941+0000","last_unstale":"2024-09-10T21:26:15.549843+0000","last_undegraded":"2024-09-10T21:26:15.549843+0000","last_fullsized":"2024-09-10T21:26:15.549843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542087+0000","last_change":"2024-09-10T21:26:15.536666+0000","last_active":"2024-09-10T21:26:15.542087+0000","last_peered":"2024-09-10T21:26:15.542087+0000","last_clean":"2024-09-10T21:26:15.542087+0000","last_became_active":"2024-09-10T21:26:15.536410+0000","last_became_peered":"2024-09-10T21:26:15.536410+0000","last_unstale":"2024-09-10T21:26:15.542087+0000","last_undegraded":"2024-09-10T21:26:15.542087+0000","last_fullsized":"2024-09-10T21:26:15.542087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:15.595334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.529423+0000","last_change":"2024-09-10T21:26:15.521400+0000","last_active":"2024-09-10T21:26:15.529423+0000","last_peered":"2024-09-10T21:26:15.529423+0000","last_clean":"2024-09-10T21:26:15.529423+0000","last_became_active":"2024-09-10T21:26:15.520076+0000","last_became_peered":"2024-09-10T21:26:15.520076+0000","last_unstale":"2024-09-10T21:26:15.529423+0000","last_undegraded":"2024-09-10T21:26:15.529423+0000","last_fullsized":"2024-09-10T21:26:15.529423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:13.921295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549231+0000","last_change":"2024-09-10T21:26:15.544245+0000","last_active":"2024-09-10T21:26:15.549231+0000","last_peered":"2024-09-10T21:26:15.549231+0000","last_clean":"2024-09-10T21:26:15.549231+0000","last_became_active":"2024-09-10T21:26:15.539109+0000","last_became_peered":"2024-09-10T21:26:15.539109+0000","last_unstale":"2024-09-10T21:26:15.549231+0000","last_undegraded":"2024-09-10T21:26:15.549231+0000","last_fullsized":"2024-09-10T21:26:15.549231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.531781+0000","last_change":"2024-09-10T21:26:15.520766+0000","last_active":"2024-09-10T21:26:15.531781+0000","last_peered":"2024-09-10T21:26:15.531781+0000","last_clean":"2024-09-10T21:26:15.531781+0000","last_became_active":"2024-09-10T21:26:15.520453+0000","last_became_peered":"2024-09-10T21:26:15.520453+0000","last_unstale":"2024-09-10T21:26:15.531781+0000","last_undegraded":"2024-09-10T21:26:15.531781+0000","last_fullsized":"2024-09-10T21:26:15.531781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:58.808915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039229+0000","last_change":"2024-09-10T21:26:15.534009+0000","last_active":"2024-09-10T21:26:17.039229+0000","last_peered":"2024-09-10T21:26:17.039229+0000","last_clean":"2024-09-10T21:26:17.039229+0000","last_became_active":"2024-09-10T21:26:15.533744+0000","last_became_peered":"2024-09-10T21:26:15.533744+0000","last_unstale":"2024-09-10T21:26:17.039229+0000","last_undegraded":"2024-09-10T21:26:17.039229+0000","last_fullsized":"2024-09-10T21:26:17.039229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:39.504012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545207+0000","last_change":"2024-09-10T21:26:15.540350+0000","last_active":"2024-09-10T21:26:15.545207+0000","last_peered":"2024-09-10T21:26:15.545207+0000","last_clean":"2024-09-10T21:26:15.545207+0000","last_became_active":"2024-09-10T21:26:15.535466+0000","last_became_peered":"2024-09-10T21:26:15.535466+0000","last_unstale":"2024-09-10T21:26:15.545207+0000","last_undegraded":"2024-09-10T21:26:15.545207+0000","last_fullsized":"2024-09-10T21:26:15.545207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:48.167182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542035+0000","last_change":"2024-09-10T21:26:15.534826+0000","last_active":"2024-09-10T21:26:15.542035+0000","last_peered":"2024-09-10T21:26:15.542035+0000","last_clean":"2024-09-10T21:26:15.542035+0000","last_became_active":"2024-09-10T21:26:15.534533+0000","last_became_peered":"2024-09-10T21:26:15.534533+0000","last_unstale":"2024-09-10T21:26:15.542035+0000","last_undegraded":"2024-09-10T21:26:15.542035+0000","last_fullsized":"2024-09-10T21:26:15.542035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:28.481727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038929+0000","last_change":"2024-09-10T21:26:15.534654+0000","last_active":"2024-09-10T21:26:17.038929+0000","last_peered":"2024-09-10T21:26:17.038929+0000","last_clean":"2024-09-10T21:26:17.038929+0000","last_became_active":"2024-09-10T21:26:15.534361+0000","last_became_peered":"2024-09-10T21:26:15.534361+0000","last_unstale":"2024-09-10T21:26:17.038929+0000","last_undegraded":"2024-09-10T21:26:17.038929+0000","last_fullsized":"2024-09-10T21:26:17.038929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.584980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038631+0000","last_change":"2024-09-10T21:26:15.532658+0000","last_active":"2024-09-10T21:26:17.038631+0000","last_peered":"2024-09-10T21:26:17.038631+0000","last_clean":"2024-09-10T21:26:17.038631+0000","last_became_active":"2024-09-10T21:26:15.532354+0000","last_became_peered":"2024-09-10T21:26:15.532354+0000","last_unstale":"2024-09-10T21:26:17.038631+0000","last_undegraded":"2024-09-10T21:26:17.038631+0000","last_fullsized":"2024-09-10T21:26:17.038631+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.984958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038705+0000","last_change":"2024-09-10T21:26:15.534760+0000","last_active":"2024-09-10T21:26:17.038705+0000","last_peered":"2024-09-10T21:26:17.038705+0000","last_clean":"2024-09-10T21:26:17.038705+0000","last_became_active":"2024-09-10T21:26:15.534482+0000","last_became_peered":"2024-09-10T21:26:15.534482+0000","last_unstale":"2024-09-10T21:26:17.038705+0000","last_undegraded":"2024-09-10T21:26:17.038705+0000","last_fullsized":"2024-09-10T21:26:17.038705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:34.222614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549208+0000","last_change":"2024-09-10T21:26:15.539933+0000","last_active":"2024-09-10T21:26:15.549208+0000","last_peered":"2024-09-10T21:26:15.549208+0000","last_clean":"2024-09-10T21:26:15.549208+0000","last_became_active":"2024-09-10T21:26:15.539551+0000","last_became_peered":"2024-09-10T21:26:15.539551+0000","last_unstale":"2024-09-10T21:26:15.549208+0000","last_undegraded":"2024-09-10T21:26:15.549208+0000","last_fullsized":"2024-09-10T21:26:15.549208+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039409+0000","last_change":"2024-09-10T21:26:15.518300+0000","last_active":"2024-09-10T21:26:17.039409+0000","last_peered":"2024-09-10T21:26:17.039409+0000","last_clean":"2024-09-10T21:26:17.039409+0000","last_became_active":"2024-09-10T21:26:15.518016+0000","last_became_peered":"2024-09-10T21:26:15.518016+0000","last_unstale":"2024-09-10T21:26:17.039409+0000","last_undegraded":"2024-09-10T21:26:17.039409+0000","last_fullsized":"2024-09-10T21:26:17.039409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:19.497285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546220+0000","last_change":"2024-09-10T21:26:15.541772+0000","last_active":"2024-09-10T21:26:15.546220+0000","last_peered":"2024-09-10T21:26:15.546220+0000","last_clean":"2024-09-10T21:26:15.546220+0000","last_became_active":"2024-09-10T21:26:15.541512+0000","last_became_peered":"2024-09-10T21:26:15.541512+0000","last_unstale":"2024-09-10T21:26:15.546220+0000","last_undegraded":"2024-09-10T21:26:15.546220+0000","last_fullsized":"2024-09-10T21:26:15.546220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:41.403113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550226+0000","last_change":"2024-09-10T21:26:15.545746+0000","last_active":"2024-09-10T21:26:15.550226+0000","last_peered":"2024-09-10T21:26:15.550226+0000","last_clean":"2024-09-10T21:26:15.550226+0000","last_became_active":"2024-09-10T21:26:15.545498+0000","last_became_peered":"2024-09-10T21:26:15.545498+0000","last_unstale":"2024-09-10T21:26:15.550226+0000","last_undegraded":"2024-09-10T21:26:15.550226+0000","last_fullsized":"2024-09-10T21:26:15.550226+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549974+0000","last_change":"2024-09-10T21:26:15.546272+0000","last_active":"2024-09-10T21:26:15.549974+0000","last_peered":"2024-09-10T21:26:15.549974+0000","last_clean":"2024-09-10T21:26:15.549974+0000","last_became_active":"2024-09-10T21:26:15.537090+0000","last_became_peered":"2024-09-10T21:26:15.537090+0000","last_unstale":"2024-09-10T21:26:15.549974+0000","last_undegraded":"2024-09-10T21:26:15.549974+0000","last_fullsized":"2024-09-10T21:26:15.549974+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546287+0000","last_change":"2024-09-10T21:26:15.541138+0000","last_active":"2024-09-10T21:26:15.546287+0000","last_peered":"2024-09-10T21:26:15.546287+0000","last_clean":"2024-09-10T21:26:15.546287+0000","last_became_active":"2024-09-10T21:26:15.540540+0000","last_became_peered":"2024-09-10T21:26:15.540540+0000","last_unstale":"2024-09-10T21:26:15.546287+0000","last_undegraded":"2024-09-10T21:26:15.546287+0000","last_fullsized":"2024-09-10T21:26:15.546287+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:27.088710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038685+0000","last_change":"2024-09-10T21:26:15.534852+0000","last_active":"2024-09-10T21:26:17.038685+0000","last_peered":"2024-09-10T21:26:17.038685+0000","last_clean":"2024-09-10T21:26:17.038685+0000","last_became_active":"2024-09-10T21:26:15.534449+0000","last_became_peered":"2024-09-10T21:26:15.534449+0000","last_unstale":"2024-09-10T21:26:17.038685+0000","last_undegraded":"2024-09-10T21:26:17.038685+0000","last_fullsized":"2024-09-10T21:26:17.038685+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:14.802445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550232+0000","last_change":"2024-09-10T21:26:15.544506+0000","last_active":"2024-09-10T21:26:15.550232+0000","last_peered":"2024-09-10T21:26:15.550232+0000","last_clean":"2024-09-10T21:26:15.550232+0000","last_became_active":"2024-09-10T21:26:15.544258+0000","last_became_peered":"2024-09-10T21:26:15.544258+0000","last_unstale":"2024-09-10T21:26:15.550232+0000","last_undegraded":"2024-09-10T21:26:15.550232+0000","last_fullsized":"2024-09-10T21:26:15.550232+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547473+0000","last_change":"2024-09-10T21:26:15.542753+0000","last_active":"2024-09-10T21:26:15.547473+0000","last_peered":"2024-09-10T21:26:15.547473+0000","last_clean":"2024-09-10T21:26:15.547473+0000","last_became_active":"2024-09-10T21:26:15.542492+0000","last_became_peered":"2024-09-10T21:26:15.542492+0000","last_unstale":"2024-09-10T21:26:15.547473+0000","last_undegraded":"2024-09-10T21:26:15.547473+0000","last_fullsized":"2024-09-10T21:26:15.547473+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:02.922870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550251+0000","last_change":"2024-09-10T21:26:14.530389+0000","last_active":"2024-09-10T21:26:15.550251+0000","last_peered":"2024-09-10T21:26:15.550251+0000","last_clean":"2024-09-10T21:26:15.550251+0000","last_became_active":"2024-09-10T21:26:14.530024+0000","last_became_peered":"2024-09-10T21:26:14.530024+0000","last_unstale":"2024-09-10T21:26:15.550251+0000","last_undegraded":"2024-09-10T21:26:15.550251+0000","last_fullsized":"2024-09-10T21:26:15.550251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:44.886118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550140+0000","last_change":"2024-09-10T21:26:15.546106+0000","last_active":"2024-09-10T21:26:15.550140+0000","last_peered":"2024-09-10T21:26:15.550140+0000","last_clean":"2024-09-10T21:26:15.550140+0000","last_became_active":"2024-09-10T21:26:15.545789+0000","last_became_peered":"2024-09-10T21:26:15.545789+0000","last_unstale":"2024-09-10T21:26:15.550140+0000","last_undegraded":"2024-09-10T21:26:15.550140+0000","last_fullsized":"2024-09-10T21:26:15.550140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:20.084655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.537228+0000","last_change":"2024-09-10T21:26:14.533996+0000","last_active":"2024-09-10T21:26:15.537228+0000","last_peered":"2024-09-10T21:26:15.537228+0000","last_clean":"2024-09-10T21:26:15.537228+0000","last_became_active":"2024-09-10T21:26:14.533618+0000","last_became_peered":"2024-09-10T21:26:14.533618+0000","last_unstale":"2024-09-10T21:26:15.537228+0000","last_undegraded":"2024-09-10T21:26:15.537228+0000","last_fullsized":"2024-09-10T21:26:15.537228+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:43.185675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.546302+0000","last_change":"2024-09-10T21:26:15.541441+0000","last_active":"2024-09-10T21:26:15.546302+0000","last_peered":"2024-09-10T21:26:15.546302+0000","last_clean":"2024-09-10T21:26:15.546302+0000","last_became_active":"2024-09-10T21:26:15.540879+0000","last_became_peered":"2024-09-10T21:26:15.540879+0000","last_unstale":"2024-09-10T21:26:15.546302+0000","last_undegraded":"2024-09-10T21:26:15.546302+0000","last_fullsized":"2024-09-10T21:26:15.546302+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:37.706884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545342+0000","last_change":"2024-09-10T21:26:14.529412+0000","last_active":"2024-09-10T21:26:15.545342+0000","last_peered":"2024-09-10T21:26:15.545342+0000","last_clean":"2024-09-10T21:26:15.545342+0000","last_became_active":"2024-09-10T21:26:14.529112+0000","last_became_peered":"2024-09-10T21:26:14.529112+0000","last_unstale":"2024-09-10T21:26:15.545342+0000","last_undegraded":"2024-09-10T21:26:15.545342+0000","last_fullsized":"2024-09-10T21:26:15.545342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:17.649740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548940+0000","last_change":"2024-09-10T21:26:15.537699+0000","last_active":"2024-09-10T21:26:15.548940+0000","last_peered":"2024-09-10T21:26:15.548940+0000","last_clean":"2024-09-10T21:26:15.548940+0000","last_became_active":"2024-09-10T21:26:15.533943+0000","last_became_peered":"2024-09-10T21:26:15.533943+0000","last_unstale":"2024-09-10T21:26:15.548940+0000","last_undegraded":"2024-09-10T21:26:15.548940+0000","last_fullsized":"2024-09-10T21:26:15.548940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:27.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538486+0000","last_change":"2024-09-10T21:26:14.529444+0000","last_active":"2024-09-10T21:26:15.538486+0000","last_peered":"2024-09-10T21:26:15.538486+0000","last_clean":"2024-09-10T21:26:15.538486+0000","last_became_active":"2024-09-10T21:26:14.529049+0000","last_became_peered":"2024-09-10T21:26:14.529049+0000","last_unstale":"2024-09-10T21:26:15.538486+0000","last_undegraded":"2024-09-10T21:26:15.538486+0000","last_fullsized":"2024-09-10T21:26:15.538486+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:52.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541439+0000","last_change":"2024-09-10T21:26:15.536274+0000","last_active":"2024-09-10T21:26:15.541439+0000","last_peered":"2024-09-10T21:26:15.541439+0000","last_clean":"2024-09-10T21:26:15.541439+0000","last_became_active":"2024-09-10T21:26:15.536085+0000","last_became_peered":"2024-09-10T21:26:15.536085+0000","last_unstale":"2024-09-10T21:26:15.541439+0000","last_undegraded":"2024-09-10T21:26:15.541439+0000","last_fullsized":"2024-09-10T21:26:15.541439+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:10.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038897+0000","last_change":"2024-09-10T21:26:14.531407+0000","last_active":"2024-09-10T21:26:17.038897+0000","last_peered":"2024-09-10T21:26:17.038897+0000","last_clean":"2024-09-10T21:26:17.038897+0000","last_became_active":"2024-09-10T21:26:14.531128+0000","last_became_peered":"2024-09-10T21:26:14.531128+0000","last_unstale":"2024-09-10T21:26:17.038897+0000","last_undegraded":"2024-09-10T21:26:17.038897+0000","last_fullsized":"2024-09-10T21:26:17.038897+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:54.077340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544943+0000","last_change":"2024-09-10T21:26:15.540471+0000","last_active":"2024-09-10T21:26:15.544943+0000","last_peered":"2024-09-10T21:26:15.544943+0000","last_clean":"2024-09-10T21:26:15.544943+0000","last_became_active":"2024-09-10T21:26:15.539374+0000","last_became_peered":"2024-09-10T21:26:15.539374+0000","last_unstale":"2024-09-10T21:26:15.544943+0000","last_undegraded":"2024-09-10T21:26:15.544943+0000","last_fullsized":"2024-09-10T21:26:15.544943+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:53.409133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539556+0000","last_change":"2024-09-10T21:26:14.528783+0000","last_active":"2024-09-10T21:26:15.539556+0000","last_peered":"2024-09-10T21:26:15.539556+0000","last_clean":"2024-09-10T21:26:15.539556+0000","last_became_active":"2024-09-10T21:26:14.528509+0000","last_became_peered":"2024-09-10T21:26:14.528509+0000","last_unstale":"2024-09-10T21:26:15.539556+0000","last_undegraded":"2024-09-10T21:26:15.539556+0000","last_fullsized":"2024-09-10T21:26:15.539556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:48.911075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549136+0000","last_change":"2024-09-10T21:26:15.540735+0000","last_active":"2024-09-10T21:26:15.549136+0000","last_peered":"2024-09-10T21:26:15.549136+0000","last_clean":"2024-09-10T21:26:15.549136+0000","last_became_active":"2024-09-10T21:26:15.538227+0000","last_became_peered":"2024-09-10T21:26:15.538227+0000","last_unstale":"2024-09-10T21:26:15.549136+0000","last_undegraded":"2024-09-10T21:26:15.549136+0000","last_fullsized":"2024-09-10T21:26:15.549136+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:27.252740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533460+0000","last_change":"2024-09-10T21:26:14.527343+0000","last_active":"2024-09-10T21:26:15.533460+0000","last_peered":"2024-09-10T21:26:15.533460+0000","last_clean":"2024-09-10T21:26:15.533460+0000","last_became_active":"2024-09-10T21:26:14.527053+0000","last_became_peered":"2024-09-10T21:26:14.527053+0000","last_unstale":"2024-09-10T21:26:15.533460+0000","last_undegraded":"2024-09-10T21:26:15.533460+0000","last_fullsized":"2024-09-10T21:26:15.533460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:49.564562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.528756+0000","last_change":"2024-09-10T21:26:15.519138+0000","last_active":"2024-09-10T21:26:15.528756+0000","last_peered":"2024-09-10T21:26:15.528756+0000","last_clean":"2024-09-10T21:26:15.528756+0000","last_became_active":"2024-09-10T21:26:15.518903+0000","last_became_peered":"2024-09-10T21:26:15.518903+0000","last_unstale":"2024-09-10T21:26:15.528756+0000","last_undegraded":"2024-09-10T21:26:15.528756+0000","last_fullsized":"2024-09-10T21:26:15.528756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:24.898753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038776+0000","last_change":"2024-09-10T21:26:14.522827+0000","last_active":"2024-09-10T21:26:17.038776+0000","last_peered":"2024-09-10T21:26:17.038776+0000","last_clean":"2024-09-10T21:26:17.038776+0000","last_became_active":"2024-09-10T21:26:14.522537+0000","last_became_peered":"2024-09-10T21:26:14.522537+0000","last_unstale":"2024-09-10T21:26:17.038776+0000","last_undegraded":"2024-09-10T21:26:17.038776+0000","last_fullsized":"2024-09-10T21:26:17.038776+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:15.763134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533518+0000","last_change":"2024-09-10T21:26:15.528432+0000","last_active":"2024-09-10T21:26:15.533518+0000","last_peered":"2024-09-10T21:26:15.533518+0000","last_clean":"2024-09-10T21:26:15.533518+0000","last_became_active":"2024-09-10T21:26:15.526149+0000","last_became_peered":"2024-09-10T21:26:15.526149+0000","last_unstale":"2024-09-10T21:26:15.533518+0000","last_undegraded":"2024-09-10T21:26:15.533518+0000","last_fullsized":"2024-09-10T21:26:15.533518+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511121+0000","last_change":"2024-09-10T21:26:14.522970+0000","last_active":"2024-09-10T21:26:15.511121+0000","last_peered":"2024-09-10T21:26:15.511121+0000","last_clean":"2024-09-10T21:26:15.511121+0000","last_became_active":"2024-09-10T21:26:14.522696+0000","last_became_peered":"2024-09-10T21:26:14.522696+0000","last_unstale":"2024-09-10T21:26:15.511121+0000","last_undegraded":"2024-09-10T21:26:15.511121+0000","last_fullsized":"2024-09-10T21:26:15.511121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:26.009061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542110+0000","last_change":"2024-09-10T21:26:15.537430+0000","last_active":"2024-09-10T21:26:15.542110+0000","last_peered":"2024-09-10T21:26:15.542110+0000","last_clean":"2024-09-10T21:26:15.542110+0000","last_became_active":"2024-09-10T21:26:15.537174+0000","last_became_peered":"2024-09-10T21:26:15.537174+0000","last_unstale":"2024-09-10T21:26:15.542110+0000","last_undegraded":"2024-09-10T21:26:15.542110+0000","last_fullsized":"2024-09-10T21:26:15.542110+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:18.652981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510906+0000","last_change":"2024-09-10T21:26:14.516829+0000","last_active":"2024-09-10T21:26:15.510906+0000","last_peered":"2024-09-10T21:26:15.510906+0000","last_clean":"2024-09-10T21:26:15.510906+0000","last_became_active":"2024-09-10T21:26:14.516564+0000","last_became_peered":"2024-09-10T21:26:14.516564+0000","last_unstale":"2024-09-10T21:26:15.510906+0000","last_undegraded":"2024-09-10T21:26:15.510906+0000","last_fullsized":"2024-09-10T21:26:15.510906+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:32.378883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549908+0000","last_change":"2024-09-10T21:26:15.545818+0000","last_active":"2024-09-10T21:26:15.549908+0000","last_peered":"2024-09-10T21:26:15.549908+0000","last_clean":"2024-09-10T21:26:15.549908+0000","last_became_active":"2024-09-10T21:26:15.545383+0000","last_became_peered":"2024-09-10T21:26:15.545383+0000","last_unstale":"2024-09-10T21:26:15.549908+0000","last_undegraded":"2024-09-10T21:26:15.549908+0000","last_fullsized":"2024-09-10T21:26:15.549908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549760+0000","last_change":"2024-09-10T21:26:14.531817+0000","last_active":"2024-09-10T21:26:15.549760+0000","last_peered":"2024-09-10T21:26:15.549760+0000","last_clean":"2024-09-10T21:26:15.549760+0000","last_became_active":"2024-09-10T21:26:14.531387+0000","last_became_peered":"2024-09-10T21:26:14.531387+0000","last_unstale":"2024-09-10T21:26:15.549760+0000","last_undegraded":"2024-09-10T21:26:15.549760+0000","last_fullsized":"2024-09-10T21:26:15.549760+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:58.645631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547276+0000","last_change":"2024-09-10T21:26:15.543162+0000","last_active":"2024-09-10T21:26:15.547276+0000","last_peered":"2024-09-10T21:26:15.547276+0000","last_clean":"2024-09-10T21:26:15.547276+0000","last_became_active":"2024-09-10T21:26:15.542913+0000","last_became_peered":"2024-09-10T21:26:15.542913+0000","last_unstale":"2024-09-10T21:26:15.547276+0000","last_undegraded":"2024-09-10T21:26:15.547276+0000","last_fullsized":"2024-09-10T21:26:15.547276+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.553669+0000","last_change":"2024-09-10T21:26:14.533065+0000","last_active":"2024-09-10T21:26:15.553669+0000","last_peered":"2024-09-10T21:26:15.553669+0000","last_clean":"2024-09-10T21:26:15.553669+0000","last_became_active":"2024-09-10T21:26:14.532788+0000","last_became_peered":"2024-09-10T21:26:14.532788+0000","last_unstale":"2024-09-10T21:26:15.553669+0000","last_undegraded":"2024-09-10T21:26:15.553669+0000","last_fullsized":"2024-09-10T21:26:15.553669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:26.313137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541575+0000","last_change":"2024-09-10T21:26:15.540226+0000","last_active":"2024-09-10T21:26:15.541575+0000","last_peered":"2024-09-10T21:26:15.541575+0000","last_clean":"2024-09-10T21:26:15.541575+0000","last_became_active":"2024-09-10T21:26:15.539910+0000","last_became_peered":"2024-09-10T21:26:15.539910+0000","last_unstale":"2024-09-10T21:26:15.541575+0000","last_undegraded":"2024-09-10T21:26:15.541575+0000","last_fullsized":"2024-09-10T21:26:15.541575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:20.009997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538731+0000","last_change":"2024-09-10T21:26:14.526625+0000","last_active":"2024-09-10T21:26:15.538731+0000","last_peered":"2024-09-10T21:26:15.538731+0000","last_clean":"2024-09-10T21:26:15.538731+0000","last_became_active":"2024-09-10T21:26:14.526301+0000","last_became_peered":"2024-09-10T21:26:14.526301+0000","last_unstale":"2024-09-10T21:26:15.538731+0000","last_undegraded":"2024-09-10T21:26:15.538731+0000","last_fullsized":"2024-09-10T21:26:15.538731+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:36.925973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550279+0000","last_change":"2024-09-10T21:26:15.547448+0000","last_active":"2024-09-10T21:26:15.550279+0000","last_peered":"2024-09-10T21:26:15.550279+0000","last_clean":"2024-09-10T21:26:15.550279+0000","last_became_active":"2024-09-10T21:26:15.546869+0000","last_became_peered":"2024-09-10T21:26:15.546869+0000","last_unstale":"2024-09-10T21:26:15.550279+0000","last_undegraded":"2024-09-10T21:26:15.550279+0000","last_fullsized":"2024-09-10T21:26:15.550279+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.511264+0000","last_change":"2024-09-10T21:26:14.526509+0000","last_active":"2024-09-10T21:26:15.511264+0000","last_peered":"2024-09-10T21:26:15.511264+0000","last_clean":"2024-09-10T21:26:15.511264+0000","last_became_active":"2024-09-10T21:26:14.526211+0000","last_became_peered":"2024-09-10T21:26:14.526211+0000","last_unstale":"2024-09-10T21:26:15.511264+0000","last_undegraded":"2024-09-10T21:26:15.511264+0000","last_fullsized":"2024-09-10T21:26:15.511264+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:19.196183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539061+0000","last_change":"2024-09-10T21:26:15.533375+0000","last_active":"2024-09-10T21:26:15.539061+0000","last_peered":"2024-09-10T21:26:15.539061+0000","last_clean":"2024-09-10T21:26:15.539061+0000","last_became_active":"2024-09-10T21:26:15.533154+0000","last_became_peered":"2024-09-10T21:26:15.533154+0000","last_unstale":"2024-09-10T21:26:15.539061+0000","last_undegraded":"2024-09-10T21:26:15.539061+0000","last_fullsized":"2024-09-10T21:26:15.539061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:56.283303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.525900+0000","last_change":"2024-09-10T21:26:14.523579+0000","last_active":"2024-09-10T21:26:15.525900+0000","last_peered":"2024-09-10T21:26:15.525900+0000","last_clean":"2024-09-10T21:26:15.525900+0000","last_became_active":"2024-09-10T21:26:14.523321+0000","last_became_peered":"2024-09-10T21:26:14.523321+0000","last_unstale":"2024-09-10T21:26:15.525900+0000","last_undegraded":"2024-09-10T21:26:15.525900+0000","last_fullsized":"2024-09-10T21:26:15.525900+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:39.550410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039427+0000","last_change":"2024-09-10T21:26:15.528299+0000","last_active":"2024-09-10T21:26:17.039427+0000","last_peered":"2024-09-10T21:26:17.039427+0000","last_clean":"2024-09-10T21:26:17.039427+0000","last_became_active":"2024-09-10T21:26:15.528045+0000","last_became_peered":"2024-09-10T21:26:15.528045+0000","last_unstale":"2024-09-10T21:26:17.039427+0000","last_undegraded":"2024-09-10T21:26:17.039427+0000","last_fullsized":"2024-09-10T21:26:17.039427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:43.372388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538375+0000","last_change":"2024-09-10T21:26:14.526055+0000","last_active":"2024-09-10T21:26:15.538375+0000","last_peered":"2024-09-10T21:26:15.538375+0000","last_clean":"2024-09-10T21:26:15.538375+0000","last_became_active":"2024-09-10T21:26:14.525734+0000","last_became_peered":"2024-09-10T21:26:14.525734+0000","last_unstale":"2024-09-10T21:26:15.538375+0000","last_undegraded":"2024-09-10T21:26:15.538375+0000","last_fullsized":"2024-09-10T21:26:15.538375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:15.163346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549528+0000","last_change":"2024-09-10T21:26:15.545468+0000","last_active":"2024-09-10T21:26:15.549528+0000","last_peered":"2024-09-10T21:26:15.549528+0000","last_clean":"2024-09-10T21:26:15.549528+0000","last_became_active":"2024-09-10T21:26:15.545137+0000","last_became_peered":"2024-09-10T21:26:15.545137+0000","last_unstale":"2024-09-10T21:26:15.549528+0000","last_undegraded":"2024-09-10T21:26:15.549528+0000","last_fullsized":"2024-09-10T21:26:15.549528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:12.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538792+0000","last_change":"2024-09-10T21:26:14.534718+0000","last_active":"2024-09-10T21:26:15.538792+0000","last_peered":"2024-09-10T21:26:15.538792+0000","last_clean":"2024-09-10T21:26:15.538792+0000","last_became_active":"2024-09-10T21:26:14.534222+0000","last_became_peered":"2024-09-10T21:26:14.534222+0000","last_unstale":"2024-09-10T21:26:15.538792+0000","last_undegraded":"2024-09-10T21:26:15.538792+0000","last_fullsized":"2024-09-10T21:26:15.538792+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:10.810025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.550202+0000","last_change":"2024-09-10T21:26:15.546824+0000","last_active":"2024-09-10T21:26:15.550202+0000","last_peered":"2024-09-10T21:26:15.550202+0000","last_clean":"2024-09-10T21:26:15.550202+0000","last_became_active":"2024-09-10T21:26:15.546286+0000","last_became_peered":"2024-09-10T21:26:15.546286+0000","last_unstale":"2024-09-10T21:26:15.550202+0000","last_undegraded":"2024-09-10T21:26:15.550202+0000","last_fullsized":"2024-09-10T21:26:15.550202+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:45.889359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.533005+0000","last_change":"2024-09-10T21:26:14.527929+0000","last_active":"2024-09-10T21:26:15.533005+0000","last_peered":"2024-09-10T21:26:15.533005+0000","last_clean":"2024-09-10T21:26:15.533005+0000","last_became_active":"2024-09-10T21:26:14.527583+0000","last_became_peered":"2024-09-10T21:26:14.527583+0000","last_unstale":"2024-09-10T21:26:15.533005+0000","last_undegraded":"2024-09-10T21:26:15.533005+0000","last_fullsized":"2024-09-10T21:26:15.533005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:20.085776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.545249+0000","last_change":"2024-09-10T21:26:15.541261+0000","last_active":"2024-09-10T21:26:15.545249+0000","last_peered":"2024-09-10T21:26:15.545249+0000","last_clean":"2024-09-10T21:26:15.545249+0000","last_became_active":"2024-09-10T21:26:15.541018+0000","last_became_peered":"2024-09-10T21:26:15.541018+0000","last_unstale":"2024-09-10T21:26:15.545249+0000","last_undegraded":"2024-09-10T21:26:15.545249+0000","last_fullsized":"2024-09-10T21:26:15.545249+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.539384+0000","last_change":"2024-09-10T21:26:14.528002+0000","last_active":"2024-09-10T21:26:15.539384+0000","last_peered":"2024-09-10T21:26:15.539384+0000","last_clean":"2024-09-10T21:26:15.539384+0000","last_became_active":"2024-09-10T21:26:14.527640+0000","last_became_peered":"2024-09-10T21:26:14.527640+0000","last_unstale":"2024-09-10T21:26:15.539384+0000","last_undegraded":"2024-09-10T21:26:15.539384+0000","last_fullsized":"2024-09-10T21:26:15.539384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:40.905930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.542762+0000","last_change":"2024-09-10T21:26:15.533913+0000","last_active":"2024-09-10T21:26:15.542762+0000","last_peered":"2024-09-10T21:26:15.542762+0000","last_clean":"2024-09-10T21:26:15.542762+0000","last_became_active":"2024-09-10T21:26:15.533607+0000","last_became_peered":"2024-09-10T21:26:15.533607+0000","last_unstale":"2024-09-10T21:26:15.542762+0000","last_undegraded":"2024-09-10T21:26:15.542762+0000","last_fullsized":"2024-09-10T21:26:15.542762+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:53.763483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524585+0000","last_change":"2024-09-10T21:26:14.530739+0000","last_active":"2024-09-10T21:26:15.524585+0000","last_peered":"2024-09-10T21:26:15.524585+0000","last_clean":"2024-09-10T21:26:15.524585+0000","last_became_active":"2024-09-10T21:26:14.530428+0000","last_became_peered":"2024-09-10T21:26:14.530428+0000","last_unstale":"2024-09-10T21:26:15.524585+0000","last_undegraded":"2024-09-10T21:26:15.524585+0000","last_fullsized":"2024-09-10T21:26:15.524585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:47.051011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.547356+0000","last_change":"2024-09-10T21:26:15.543515+0000","last_active":"2024-09-10T21:26:15.547356+0000","last_peered":"2024-09-10T21:26:15.547356+0000","last_clean":"2024-09-10T21:26:15.547356+0000","last_became_active":"2024-09-10T21:26:15.543291+0000","last_became_peered":"2024-09-10T21:26:15.543291+0000","last_unstale":"2024-09-10T21:26:15.547356+0000","last_undegraded":"2024-09-10T21:26:15.547356+0000","last_fullsized":"2024-09-10T21:26:15.547356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.551224+0000","last_change":"2024-09-10T21:26:14.532927+0000","last_active":"2024-09-10T21:26:15.551224+0000","last_peered":"2024-09-10T21:26:15.551224+0000","last_clean":"2024-09-10T21:26:15.551224+0000","last_became_active":"2024-09-10T21:26:14.532625+0000","last_became_peered":"2024-09-10T21:26:14.532625+0000","last_unstale":"2024-09-10T21:26:15.551224+0000","last_undegraded":"2024-09-10T21:26:15.551224+0000","last_fullsized":"2024-09-10T21:26:15.551224+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:36.431767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532586+0000","last_change":"2024-09-10T21:26:15.529708+0000","last_active":"2024-09-10T21:26:15.532586+0000","last_peered":"2024-09-10T21:26:15.532586+0000","last_clean":"2024-09-10T21:26:15.532586+0000","last_became_active":"2024-09-10T21:26:15.529431+0000","last_became_peered":"2024-09-10T21:26:15.529431+0000","last_unstale":"2024-09-10T21:26:15.532586+0000","last_undegraded":"2024-09-10T21:26:15.532586+0000","last_fullsized":"2024-09-10T21:26:15.532586+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:08.029645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038684+0000","last_change":"2024-09-10T21:26:14.526244+0000","last_active":"2024-09-10T21:26:17.038684+0000","last_peered":"2024-09-10T21:26:17.038684+0000","last_clean":"2024-09-10T21:26:17.038684+0000","last_became_active":"2024-09-10T21:26:14.525952+0000","last_became_peered":"2024-09-10T21:26:14.525952+0000","last_unstale":"2024-09-10T21:26:17.038684+0000","last_undegraded":"2024-09-10T21:26:17.038684+0000","last_fullsized":"2024-09-10T21:26:17.038684+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:48.228747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549692+0000","last_change":"2024-09-10T21:26:15.544629+0000","last_active":"2024-09-10T21:26:15.549692+0000","last_peered":"2024-09-10T21:26:15.549692+0000","last_clean":"2024-09-10T21:26:15.549692+0000","last_became_active":"2024-09-10T21:26:15.540919+0000","last_became_peered":"2024-09-10T21:26:15.540919+0000","last_unstale":"2024-09-10T21:26:15.549692+0000","last_undegraded":"2024-09-10T21:26:15.549692+0000","last_fullsized":"2024-09-10T21:26:15.549692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520315+0000","last_change":"2024-09-10T21:26:14.529271+0000","last_active":"2024-09-10T21:26:15.520315+0000","last_peered":"2024-09-10T21:26:15.520315+0000","last_clean":"2024-09-10T21:26:15.520315+0000","last_became_active":"2024-09-10T21:26:14.528963+0000","last_became_peered":"2024-09-10T21:26:14.528963+0000","last_unstale":"2024-09-10T21:26:15.520315+0000","last_undegraded":"2024-09-10T21:26:15.520315+0000","last_fullsized":"2024-09-10T21:26:15.520315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:14.592093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544636+0000","last_change":"2024-09-10T21:26:15.540533+0000","last_active":"2024-09-10T21:26:15.544636+0000","last_peered":"2024-09-10T21:26:15.544636+0000","last_clean":"2024-09-10T21:26:15.544636+0000","last_became_active":"2024-09-10T21:26:15.540237+0000","last_became_peered":"2024-09-10T21:26:15.540237+0000","last_unstale":"2024-09-10T21:26:15.544636+0000","last_undegraded":"2024-09-10T21:26:15.544636+0000","last_fullsized":"2024-09-10T21:26:15.544636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:49.612063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520099+0000","last_change":"2024-09-10T21:26:14.529847+0000","last_active":"2024-09-10T21:26:15.520099+0000","last_peered":"2024-09-10T21:26:15.520099+0000","last_clean":"2024-09-10T21:26:15.520099+0000","last_became_active":"2024-09-10T21:26:14.529074+0000","last_became_peered":"2024-09-10T21:26:14.529074+0000","last_unstale":"2024-09-10T21:26:15.520099+0000","last_undegraded":"2024-09-10T21:26:15.520099+0000","last_fullsized":"2024-09-10T21:26:15.520099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:27.478486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.532977+0000","last_change":"2024-09-10T21:26:15.528858+0000","last_active":"2024-09-10T21:26:15.532977+0000","last_peered":"2024-09-10T21:26:15.532977+0000","last_clean":"2024-09-10T21:26:15.532977+0000","last_became_active":"2024-09-10T21:26:15.527735+0000","last_became_peered":"2024-09-10T21:26:15.527735+0000","last_unstale":"2024-09-10T21:26:15.532977+0000","last_undegraded":"2024-09-10T21:26:15.532977+0000","last_fullsized":"2024-09-10T21:26:15.532977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:31.109223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544806+0000","last_change":"2024-09-10T21:26:14.529877+0000","last_active":"2024-09-10T21:26:15.544806+0000","last_peered":"2024-09-10T21:26:15.544806+0000","last_clean":"2024-09-10T21:26:15.544806+0000","last_became_active":"2024-09-10T21:26:14.529580+0000","last_became_peered":"2024-09-10T21:26:14.529580+0000","last_unstale":"2024-09-10T21:26:15.544806+0000","last_undegraded":"2024-09-10T21:26:15.544806+0000","last_fullsized":"2024-09-10T21:26:15.544806+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:41.737020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549409+0000","last_change":"2024-09-10T21:26:15.541706+0000","last_active":"2024-09-10T21:26:15.549409+0000","last_peered":"2024-09-10T21:26:15.549409+0000","last_clean":"2024-09-10T21:26:15.549409+0000","last_became_active":"2024-09-10T21:26:15.540365+0000","last_became_peered":"2024-09-10T21:26:15.540365+0000","last_unstale":"2024-09-10T21:26:15.549409+0000","last_undegraded":"2024-09-10T21:26:15.549409+0000","last_fullsized":"2024-09-10T21:26:15.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:43.648100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.530063+0000","last_change":"2024-09-10T21:26:14.532072+0000","last_active":"2024-09-10T21:26:15.530063+0000","last_peered":"2024-09-10T21:26:15.530063+0000","last_clean":"2024-09-10T21:26:15.530063+0000","last_became_active":"2024-09-10T21:26:14.530711+0000","last_became_peered":"2024-09-10T21:26:14.530711+0000","last_unstale":"2024-09-10T21:26:15.530063+0000","last_undegraded":"2024-09-10T21:26:15.530063+0000","last_fullsized":"2024-09-10T21:26:15.530063+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:15.362786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038829+0000","last_change":"2024-09-10T21:26:15.524631+0000","last_active":"2024-09-10T21:26:17.038829+0000","last_peered":"2024-09-10T21:26:17.038829+0000","last_clean":"2024-09-10T21:26:17.038829+0000","last_became_active":"2024-09-10T21:26:15.524305+0000","last_became_peered":"2024-09-10T21:26:15.524305+0000","last_unstale":"2024-09-10T21:26:17.038829+0000","last_undegraded":"2024-09-10T21:26:17.038829+0000","last_fullsized":"2024-09-10T21:26:17.038829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:36.491077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039016+0000","last_change":"2024-09-10T21:26:14.525570+0000","last_active":"2024-09-10T21:26:17.039016+0000","last_peered":"2024-09-10T21:26:17.039016+0000","last_clean":"2024-09-10T21:26:17.039016+0000","last_became_active":"2024-09-10T21:26:14.525294+0000","last_became_peered":"2024-09-10T21:26:14.525294+0000","last_unstale":"2024-09-10T21:26:17.039016+0000","last_undegraded":"2024-09-10T21:26:17.039016+0000","last_fullsized":"2024-09-10T21:26:17.039016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:26.579850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.549756+0000","last_change":"2024-09-10T21:26:15.542738+0000","last_active":"2024-09-10T21:26:15.549756+0000","last_peered":"2024-09-10T21:26:15.549756+0000","last_clean":"2024-09-10T21:26:15.549756+0000","last_became_active":"2024-09-10T21:26:15.542497+0000","last_became_peered":"2024-09-10T21:26:15.542497+0000","last_unstale":"2024-09-10T21:26:15.549756+0000","last_undegraded":"2024-09-10T21:26:15.549756+0000","last_fullsized":"2024-09-10T21:26:15.549756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:59.431959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.520974+0000","last_change":"2024-09-10T21:26:14.530559+0000","last_active":"2024-09-10T21:26:15.520974+0000","last_peered":"2024-09-10T21:26:15.520974+0000","last_clean":"2024-09-10T21:26:15.520974+0000","last_became_active":"2024-09-10T21:26:14.529513+0000","last_became_peered":"2024-09-10T21:26:14.529513+0000","last_unstale":"2024-09-10T21:26:15.520974+0000","last_undegraded":"2024-09-10T21:26:15.520974+0000","last_fullsized":"2024-09-10T21:26:15.520974+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:10.595223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.538120+0000","last_change":"2024-09-10T21:26:15.533560+0000","last_active":"2024-09-10T21:26:15.538120+0000","last_peered":"2024-09-10T21:26:15.538120+0000","last_clean":"2024-09-10T21:26:15.538120+0000","last_became_active":"2024-09-10T21:26:15.533313+0000","last_became_peered":"2024-09-10T21:26:15.533313+0000","last_unstale":"2024-09-10T21:26:15.538120+0000","last_undegraded":"2024-09-10T21:26:15.538120+0000","last_fullsized":"2024-09-10T21:26:15.538120+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:37.435008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038798+0000","last_change":"2024-09-10T21:26:14.525715+0000","last_active":"2024-09-10T21:26:17.038798+0000","last_peered":"2024-09-10T21:26:17.038798+0000","last_clean":"2024-09-10T21:26:17.038798+0000","last_became_active":"2024-09-10T21:26:14.525367+0000","last_became_peered":"2024-09-10T21:26:14.525367+0000","last_unstale":"2024-09-10T21:26:17.038798+0000","last_undegraded":"2024-09-10T21:26:17.038798+0000","last_fullsized":"2024-09-10T21:26:17.038798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:36.377629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.548560+0000","last_change":"2024-09-10T21:26:15.536138+0000","last_active":"2024-09-10T21:26:15.548560+0000","last_peered":"2024-09-10T21:26:15.548560+0000","last_clean":"2024-09-10T21:26:15.548560+0000","last_became_active":"2024-09-10T21:26:15.535521+0000","last_became_peered":"2024-09-10T21:26:15.535521+0000","last_unstale":"2024-09-10T21:26:15.548560+0000","last_undegraded":"2024-09-10T21:26:15.548560+0000","last_fullsized":"2024-09-10T21:26:15.548560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:20.575951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.526019+0000","last_change":"2024-09-10T21:26:14.519898+0000","last_active":"2024-09-10T21:26:15.526019+0000","last_peered":"2024-09-10T21:26:15.526019+0000","last_clean":"2024-09-10T21:26:15.526019+0000","last_became_active":"2024-09-10T21:26:14.519606+0000","last_became_peered":"2024-09-10T21:26:14.519606+0000","last_unstale":"2024-09-10T21:26:15.526019+0000","last_undegraded":"2024-09-10T21:26:15.526019+0000","last_fullsized":"2024-09-10T21:26:15.526019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.608822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.544613+0000","last_change":"2024-09-10T21:26:15.532747+0000","last_active":"2024-09-10T21:26:15.544613+0000","last_peered":"2024-09-10T21:26:15.544613+0000","last_clean":"2024-09-10T21:26:15.544613+0000","last_became_active":"2024-09-10T21:26:15.532482+0000","last_became_peered":"2024-09-10T21:26:15.532482+0000","last_unstale":"2024-09-10T21:26:15.544613+0000","last_undegraded":"2024-09-10T21:26:15.544613+0000","last_fullsized":"2024-09-10T21:26:15.544613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:40.553651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.509867+0000","last_change":"2024-09-10T21:26:14.526821+0000","last_active":"2024-09-10T21:26:15.509867+0000","last_peered":"2024-09-10T21:26:15.509867+0000","last_clean":"2024-09-10T21:26:15.509867+0000","last_became_active":"2024-09-10T21:26:14.526561+0000","last_became_peered":"2024-09-10T21:26:14.526561+0000","last_unstale":"2024-09-10T21:26:15.509867+0000","last_undegraded":"2024-09-10T21:26:15.509867+0000","last_fullsized":"2024-09-10T21:26:15.509867+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.541912+0000","last_change":"2024-09-10T21:26:15.538996+0000","last_active":"2024-09-10T21:26:15.541912+0000","last_peered":"2024-09-10T21:26:15.541912+0000","last_clean":"2024-09-10T21:26:15.541912+0000","last_became_active":"2024-09-10T21:26:15.538393+0000","last_became_peered":"2024-09-10T21:26:15.538393+0000","last_unstale":"2024-09-10T21:26:15.541912+0000","last_undegraded":"2024-09-10T21:26:15.541912+0000","last_fullsized":"2024-09-10T21:26:15.541912+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:33.376220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.524045+0000","last_change":"2024-09-10T21:26:14.533819+0000","last_active":"2024-09-10T21:26:15.524045+0000","last_peered":"2024-09-10T21:26:15.524045+0000","last_clean":"2024-09-10T21:26:15.524045+0000","last_became_active":"2024-09-10T21:26:14.533554+0000","last_became_peered":"2024-09-10T21:26:14.533554+0000","last_unstale":"2024-09-10T21:26:15.524045+0000","last_undegraded":"2024-09-10T21:26:15.524045+0000","last_fullsized":"2024-09-10T21:26:15.524045+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:55.449678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":170,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T21:26:15.510018+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:26:15.510018+0000","last_peered":"2024-09-10T21:26:15.510018+0000","last_clean":"2024-09-10T21:26:15.510018+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:26:15.510018+0000","last_undegraded":"2024-09-10T21:26:15.510018+0000","last_fullsized":"2024-09-10T21:26:15.510018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":275651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1216512,"data_stored":865963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":610304,"data_stored":268428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1204224,"data_stored":858796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":283991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:26:21.480 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:26:21.479+0000 7f3dc1400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:26:24.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:26:24.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 -- 172.21.15.97:0/3038573914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100b0d50 msgr2=0x7f9a100ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 --2- 172.21.15.97:0/3038573914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100b0d50 0x7f9a100ccc10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009bf0 tx=0x7f9a0c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 -- 172.21.15.97:0/3038573914 shutdown_connections 2024-09-10T21:26:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 --2- 172.21.15.97:0/3038573914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100b0d50 0x7f9a100ccc10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 --2- 172.21.15.97:0/3038573914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a100afdd0 0x7f9a100cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 -- 172.21.15.97:0/3038573914 >> 172.21.15.97:0/3038573914 conn(0x7f9a100d1c30 msgr2=0x7f9a100d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 -- 172.21.15.97:0/3038573914 shutdown_connections 2024-09-10T21:26:24.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.355+0000 7f9a18800640 1 -- 172.21.15.97:0/3038573914 wait complete. 2024-09-10T21:26:24.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 Processor -- start 2024-09-10T21:26:24.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 -- start start 2024-09-10T21:26:24.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100afdd0 0x7f9a100b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a100b0d50 0x7f9a100b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a100ce640 0x7f9a1015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a10168f40 con 0x7f9a100afdd0 2024-09-10T21:26:24.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a10168dc0 con 0x7f9a100b0d50 2024-09-10T21:26:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9a101690c0 con 0x7f9a100ce640 2024-09-10T21:26:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100afdd0 0x7f9a100b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100afdd0 0x7f9a100b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41686/0 (socket says 172.21.15.97:41686) 2024-09-10T21:26:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 -- 172.21.15.97:0/2698421894 learned_addr learned my addr 172.21.15.97:0/2698421894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:24.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15400640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a100b0d50 0x7f9a100b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a16800640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a100ce640 0x7f9a1015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 -- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a100ce640 msgr2=0x7f9a1015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a100ce640 0x7f9a1015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 -- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a100b0d50 msgr2=0x7f9a100b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a100b0d50 0x7f9a100b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 -- 172.21.15.97:0/2698421894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a100b6830 con 0x7f9a100afdd0 2024-09-10T21:26:24.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a15e00640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100afdd0 0x7f9a100b5c70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c3c0 tx=0x7f9a0400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:24.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a0a000640 1 -- 172.21.15.97:0/2698421894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04011790 con 0x7f9a100afdd0 2024-09-10T21:26:24.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a0a000640 1 -- 172.21.15.97:0/2698421894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a04017070 con 0x7f9a100afdd0 2024-09-10T21:26:24.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a0a000640 1 -- 172.21.15.97:0/2698421894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04011d40 con 0x7f9a100afdd0 2024-09-10T21:26:24.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a100b6b10 con 0x7f9a100afdd0 2024-09-10T21:26:24.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.359+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1006dcf0 con 0x7f9a100afdd0 2024-09-10T21:26:24.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.363+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1006def0 con 0x7f9a100afdd0 2024-09-10T21:26:24.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.363+0000 7f9a0a000640 1 -- 172.21.15.97:0/2698421894 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9a0401c460 con 0x7f9a100afdd0 2024-09-10T21:26:24.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.363+0000 7f9a0a000640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.363+0000 7f9a0a000640 1 -- 172.21.15.97:0/2698421894 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f9a040931b0 con 0x7f9a100afdd0 2024-09-10T21:26:24.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.363+0000 7f9a15400640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.363+0000 7f9a15400640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9a00005fd0 tx=0x7f9a00005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:24.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.367+0000 7f9a0a000640 1 -- 172.21.15.97:0/2698421894 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a1006def0 con 0x7f9a100afdd0 2024-09-10T21:26:24.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.531+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a100cea20 con 0x7f99f40725e0 2024-09-10T21:26:24.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.535+0000 7f9a0a000640 1 -- 172.21.15.97:0/2698421894 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7f9a100cea20 con 0x7f99f40725e0 2024-09-10T21:26:24.544 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:26:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 msgr2=0x7f99f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9a00005fd0 tx=0x7f9a00005860 comp rx=0 tx=0).stop 2024-09-10T21:26:24.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100afdd0 msgr2=0x7f9a100b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100afdd0 0x7f9a100b5c70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c3c0 tx=0x7f9a0400c880 comp rx=0 tx=0).stop 2024-09-10T21:26:24.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 shutdown_connections 2024-09-10T21:26:24.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99f40725e0 0x7f99f4074aa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a100ce640 0x7f9a1015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a100b0d50 0x7f9a100b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 --2- 172.21.15.97:0/2698421894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a100afdd0 0x7f9a100b5c70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 >> 172.21.15.97:0/2698421894 conn(0x7f9a100d1c30 msgr2=0x7f9a10165ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 shutdown_connections 2024-09-10T21:26:24.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.539+0000 7f9a18800640 1 -- 172.21.15.97:0/2698421894 wait complete. 2024-09-10T21:26:24.565 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-10T21:26:24.101805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222444,"kb_used_data":6184,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752276,"statfs":{"total":773094113280,"available":772866330624,"internally_reserved":0,"allocated":6332416,"data_stored":3443852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.593100"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038037+0000","last_change":"2024-09-10T21:26:15.539541+0000","last_active":"2024-09-10T21:26:17.038037+0000","last_peered":"2024-09-10T21:26:17.038037+0000","last_clean":"2024-09-10T21:26:17.038037+0000","last_became_active":"2024-09-10T21:26:15.539095+0000","last_became_peered":"2024-09-10T21:26:15.539095+0000","last_unstale":"2024-09-10T21:26:17.038037+0000","last_undegraded":"2024-09-10T21:26:17.038037+0000","last_fullsized":"2024-09-10T21:26:17.038037+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:06.299404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039484+0000","last_change":"2024-09-10T21:26:15.535693+0000","last_active":"2024-09-10T21:26:17.039484+0000","last_peered":"2024-09-10T21:26:17.039484+0000","last_clean":"2024-09-10T21:26:17.039484+0000","last_became_active":"2024-09-10T21:26:15.535421+0000","last_became_peered":"2024-09-10T21:26:15.535421+0000","last_unstale":"2024-09-10T21:26:17.039484+0000","last_undegraded":"2024-09-10T21:26:17.039484+0000","last_fullsized":"2024-09-10T21:26:17.039484+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:39.968653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039167+0000","last_change":"2024-09-10T21:26:15.537145+0000","last_active":"2024-09-10T21:26:17.039167+0000","last_peered":"2024-09-10T21:26:17.039167+0000","last_clean":"2024-09-10T21:26:17.039167+0000","last_became_active":"2024-09-10T21:26:15.536886+0000","last_became_peered":"2024-09-10T21:26:15.536886+0000","last_unstale":"2024-09-10T21:26:17.039167+0000","last_undegraded":"2024-09-10T21:26:17.039167+0000","last_fullsized":"2024-09-10T21:26:17.039167+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:20.043744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526432+0000","last_change":"2024-09-10T21:26:15.537322+0000","last_active":"2024-09-10T21:26:16.526432+0000","last_peered":"2024-09-10T21:26:16.526432+0000","last_clean":"2024-09-10T21:26:16.526432+0000","last_became_active":"2024-09-10T21:26:15.537039+0000","last_became_peered":"2024-09-10T21:26:15.537039+0000","last_unstale":"2024-09-10T21:26:16.526432+0000","last_undegraded":"2024-09-10T21:26:16.526432+0000","last_fullsized":"2024-09-10T21:26:16.526432+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:57.577430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038455+0000","last_change":"2024-09-10T21:26:15.547237+0000","last_active":"2024-09-10T21:26:17.038455+0000","last_peered":"2024-09-10T21:26:17.038455+0000","last_clean":"2024-09-10T21:26:17.038455+0000","last_became_active":"2024-09-10T21:26:15.546563+0000","last_became_peered":"2024-09-10T21:26:15.546563+0000","last_unstale":"2024-09-10T21:26:17.038455+0000","last_undegraded":"2024-09-10T21:26:17.038455+0000","last_fullsized":"2024-09-10T21:26:17.038455+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:43.648100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532736+0000","last_change":"2024-09-10T21:26:15.534547+0000","last_active":"2024-09-10T21:26:16.532736+0000","last_peered":"2024-09-10T21:26:16.532736+0000","last_clean":"2024-09-10T21:26:16.532736+0000","last_became_active":"2024-09-10T21:26:15.534280+0000","last_became_peered":"2024-09-10T21:26:15.534280+0000","last_unstale":"2024-09-10T21:26:16.532736+0000","last_undegraded":"2024-09-10T21:26:16.532736+0000","last_fullsized":"2024-09-10T21:26:16.532736+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:20.725293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533287+0000","last_change":"2024-09-10T21:26:15.533183+0000","last_active":"2024-09-10T21:26:16.533287+0000","last_peered":"2024-09-10T21:26:16.533287+0000","last_clean":"2024-09-10T21:26:16.533287+0000","last_became_active":"2024-09-10T21:26:15.532995+0000","last_became_peered":"2024-09-10T21:26:15.532995+0000","last_unstale":"2024-09-10T21:26:16.533287+0000","last_undegraded":"2024-09-10T21:26:16.533287+0000","last_fullsized":"2024-09-10T21:26:16.533287+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:06.299404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533088+0000","last_change":"2024-09-10T21:26:15.541612+0000","last_active":"2024-09-10T21:26:16.533088+0000","last_peered":"2024-09-10T21:26:16.533088+0000","last_clean":"2024-09-10T21:26:16.533088+0000","last_became_active":"2024-09-10T21:26:15.541343+0000","last_became_peered":"2024-09-10T21:26:15.541343+0000","last_unstale":"2024-09-10T21:26:16.533088+0000","last_undegraded":"2024-09-10T21:26:16.533088+0000","last_fullsized":"2024-09-10T21:26:16.533088+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:42.591294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039368+0000","last_change":"2024-09-10T21:26:15.531640+0000","last_active":"2024-09-10T21:26:17.039368+0000","last_peered":"2024-09-10T21:26:17.039368+0000","last_clean":"2024-09-10T21:26:17.039368+0000","last_became_active":"2024-09-10T21:26:15.531347+0000","last_became_peered":"2024-09-10T21:26:15.531347+0000","last_unstale":"2024-09-10T21:26:17.039368+0000","last_undegraded":"2024-09-10T21:26:17.039368+0000","last_fullsized":"2024-09-10T21:26:17.039368+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:34.692481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534561+0000","last_change":"2024-09-10T21:26:15.536440+0000","last_active":"2024-09-10T21:26:16.534561+0000","last_peered":"2024-09-10T21:26:16.534561+0000","last_clean":"2024-09-10T21:26:16.534561+0000","last_became_active":"2024-09-10T21:26:15.535896+0000","last_became_peered":"2024-09-10T21:26:15.535896+0000","last_unstale":"2024-09-10T21:26:16.534561+0000","last_undegraded":"2024-09-10T21:26:16.534561+0000","last_fullsized":"2024-09-10T21:26:16.534561+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:43.296525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037788+0000","last_change":"2024-09-10T21:26:15.538325+0000","last_active":"2024-09-10T21:26:17.037788+0000","last_peered":"2024-09-10T21:26:17.037788+0000","last_clean":"2024-09-10T21:26:17.037788+0000","last_became_active":"2024-09-10T21:26:15.536197+0000","last_became_peered":"2024-09-10T21:26:15.536197+0000","last_unstale":"2024-09-10T21:26:17.037788+0000","last_undegraded":"2024-09-10T21:26:17.037788+0000","last_fullsized":"2024-09-10T21:26:17.037788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:33.376220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532791+0000","last_change":"2024-09-10T21:26:15.518220+0000","last_active":"2024-09-10T21:26:16.532791+0000","last_peered":"2024-09-10T21:26:16.532791+0000","last_clean":"2024-09-10T21:26:16.532791+0000","last_became_active":"2024-09-10T21:26:15.516898+0000","last_became_peered":"2024-09-10T21:26:15.516898+0000","last_unstale":"2024-09-10T21:26:16.532791+0000","last_undegraded":"2024-09-10T21:26:16.532791+0000","last_fullsized":"2024-09-10T21:26:16.532791+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:57.314204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532835+0000","last_change":"2024-09-10T21:26:15.545455+0000","last_active":"2024-09-10T21:26:16.532835+0000","last_peered":"2024-09-10T21:26:16.532835+0000","last_clean":"2024-09-10T21:26:16.532835+0000","last_became_active":"2024-09-10T21:26:15.545207+0000","last_became_peered":"2024-09-10T21:26:15.545207+0000","last_unstale":"2024-09-10T21:26:16.532835+0000","last_undegraded":"2024-09-10T21:26:16.532835+0000","last_fullsized":"2024-09-10T21:26:16.532835+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:47.987190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037905+0000","last_change":"2024-09-10T21:26:15.545646+0000","last_active":"2024-09-10T21:26:17.037905+0000","last_peered":"2024-09-10T21:26:17.037905+0000","last_clean":"2024-09-10T21:26:17.037905+0000","last_became_active":"2024-09-10T21:26:15.544941+0000","last_became_peered":"2024-09-10T21:26:15.544941+0000","last_unstale":"2024-09-10T21:26:17.037905+0000","last_undegraded":"2024-09-10T21:26:17.037905+0000","last_fullsized":"2024-09-10T21:26:17.037905+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:37.069165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526174+0000","last_change":"2024-09-10T21:26:15.536666+0000","last_active":"2024-09-10T21:26:16.526174+0000","last_peered":"2024-09-10T21:26:16.526174+0000","last_clean":"2024-09-10T21:26:16.526174+0000","last_became_active":"2024-09-10T21:26:15.536410+0000","last_became_peered":"2024-09-10T21:26:15.536410+0000","last_unstale":"2024-09-10T21:26:16.526174+0000","last_undegraded":"2024-09-10T21:26:16.526174+0000","last_fullsized":"2024-09-10T21:26:16.526174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:27.316378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532717+0000","last_change":"2024-09-10T21:26:15.521400+0000","last_active":"2024-09-10T21:26:16.532717+0000","last_peered":"2024-09-10T21:26:16.532717+0000","last_clean":"2024-09-10T21:26:16.532717+0000","last_became_active":"2024-09-10T21:26:15.520076+0000","last_became_peered":"2024-09-10T21:26:15.520076+0000","last_unstale":"2024-09-10T21:26:16.532717+0000","last_undegraded":"2024-09-10T21:26:16.532717+0000","last_fullsized":"2024-09-10T21:26:16.532717+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:59.137606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038246+0000","last_change":"2024-09-10T21:26:15.544245+0000","last_active":"2024-09-10T21:26:17.038246+0000","last_peered":"2024-09-10T21:26:17.038246+0000","last_clean":"2024-09-10T21:26:17.038246+0000","last_became_active":"2024-09-10T21:26:15.539109+0000","last_became_peered":"2024-09-10T21:26:15.539109+0000","last_unstale":"2024-09-10T21:26:17.038246+0000","last_undegraded":"2024-09-10T21:26:17.038246+0000","last_fullsized":"2024-09-10T21:26:17.038246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:11.521376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526828+0000","last_change":"2024-09-10T21:26:15.520766+0000","last_active":"2024-09-10T21:26:16.526828+0000","last_peered":"2024-09-10T21:26:16.526828+0000","last_clean":"2024-09-10T21:26:16.526828+0000","last_became_active":"2024-09-10T21:26:15.520453+0000","last_became_peered":"2024-09-10T21:26:15.520453+0000","last_unstale":"2024-09-10T21:26:16.526828+0000","last_undegraded":"2024-09-10T21:26:16.526828+0000","last_fullsized":"2024-09-10T21:26:16.526828+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:27.252740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039229+0000","last_change":"2024-09-10T21:26:15.534009+0000","last_active":"2024-09-10T21:26:17.039229+0000","last_peered":"2024-09-10T21:26:17.039229+0000","last_clean":"2024-09-10T21:26:17.039229+0000","last_became_active":"2024-09-10T21:26:15.533744+0000","last_became_peered":"2024-09-10T21:26:15.533744+0000","last_unstale":"2024-09-10T21:26:17.039229+0000","last_undegraded":"2024-09-10T21:26:17.039229+0000","last_fullsized":"2024-09-10T21:26:17.039229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:39.504012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038771+0000","last_change":"2024-09-10T21:26:15.540350+0000","last_active":"2024-09-10T21:26:17.038771+0000","last_peered":"2024-09-10T21:26:17.038771+0000","last_clean":"2024-09-10T21:26:17.038771+0000","last_became_active":"2024-09-10T21:26:15.535466+0000","last_became_peered":"2024-09-10T21:26:15.535466+0000","last_unstale":"2024-09-10T21:26:17.038771+0000","last_undegraded":"2024-09-10T21:26:17.038771+0000","last_fullsized":"2024-09-10T21:26:17.038771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:59.431959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526379+0000","last_change":"2024-09-10T21:26:15.534826+0000","last_active":"2024-09-10T21:26:16.526379+0000","last_peered":"2024-09-10T21:26:16.526379+0000","last_clean":"2024-09-10T21:26:16.526379+0000","last_became_active":"2024-09-10T21:26:15.534533+0000","last_became_peered":"2024-09-10T21:26:15.534533+0000","last_unstale":"2024-09-10T21:26:16.526379+0000","last_undegraded":"2024-09-10T21:26:16.526379+0000","last_fullsized":"2024-09-10T21:26:16.526379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:20.575951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038929+0000","last_change":"2024-09-10T21:26:15.534654+0000","last_active":"2024-09-10T21:26:17.038929+0000","last_peered":"2024-09-10T21:26:17.038929+0000","last_clean":"2024-09-10T21:26:17.038929+0000","last_became_active":"2024-09-10T21:26:15.534361+0000","last_became_peered":"2024-09-10T21:26:15.534361+0000","last_unstale":"2024-09-10T21:26:17.038929+0000","last_undegraded":"2024-09-10T21:26:17.038929+0000","last_fullsized":"2024-09-10T21:26:17.038929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.584980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038631+0000","last_change":"2024-09-10T21:26:15.532658+0000","last_active":"2024-09-10T21:26:17.038631+0000","last_peered":"2024-09-10T21:26:17.038631+0000","last_clean":"2024-09-10T21:26:17.038631+0000","last_became_active":"2024-09-10T21:26:15.532354+0000","last_became_peered":"2024-09-10T21:26:15.532354+0000","last_unstale":"2024-09-10T21:26:17.038631+0000","last_undegraded":"2024-09-10T21:26:17.038631+0000","last_fullsized":"2024-09-10T21:26:17.038631+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:45.984958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038705+0000","last_change":"2024-09-10T21:26:15.534760+0000","last_active":"2024-09-10T21:26:17.038705+0000","last_peered":"2024-09-10T21:26:17.038705+0000","last_clean":"2024-09-10T21:26:17.038705+0000","last_became_active":"2024-09-10T21:26:15.534482+0000","last_became_peered":"2024-09-10T21:26:15.534482+0000","last_unstale":"2024-09-10T21:26:17.038705+0000","last_undegraded":"2024-09-10T21:26:17.038705+0000","last_fullsized":"2024-09-10T21:26:17.038705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:34.222614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038066+0000","last_change":"2024-09-10T21:26:15.539933+0000","last_active":"2024-09-10T21:26:17.038066+0000","last_peered":"2024-09-10T21:26:17.038066+0000","last_clean":"2024-09-10T21:26:17.038066+0000","last_became_active":"2024-09-10T21:26:15.539551+0000","last_became_peered":"2024-09-10T21:26:15.539551+0000","last_unstale":"2024-09-10T21:26:17.038066+0000","last_undegraded":"2024-09-10T21:26:17.038066+0000","last_fullsized":"2024-09-10T21:26:17.038066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:04.879933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039409+0000","last_change":"2024-09-10T21:26:15.518300+0000","last_active":"2024-09-10T21:26:17.039409+0000","last_peered":"2024-09-10T21:26:17.039409+0000","last_clean":"2024-09-10T21:26:17.039409+0000","last_became_active":"2024-09-10T21:26:15.518016+0000","last_became_peered":"2024-09-10T21:26:15.518016+0000","last_unstale":"2024-09-10T21:26:17.039409+0000","last_undegraded":"2024-09-10T21:26:17.039409+0000","last_fullsized":"2024-09-10T21:26:17.039409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:19.497285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533208+0000","last_change":"2024-09-10T21:26:15.541772+0000","last_active":"2024-09-10T21:26:16.533208+0000","last_peered":"2024-09-10T21:26:16.533208+0000","last_clean":"2024-09-10T21:26:16.533208+0000","last_became_active":"2024-09-10T21:26:15.541512+0000","last_became_peered":"2024-09-10T21:26:15.541512+0000","last_unstale":"2024-09-10T21:26:16.533208+0000","last_undegraded":"2024-09-10T21:26:16.533208+0000","last_fullsized":"2024-09-10T21:26:16.533208+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:58.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038967+0000","last_change":"2024-09-10T21:26:15.545746+0000","last_active":"2024-09-10T21:26:17.038967+0000","last_peered":"2024-09-10T21:26:17.038967+0000","last_clean":"2024-09-10T21:26:17.038967+0000","last_became_active":"2024-09-10T21:26:15.545498+0000","last_became_peered":"2024-09-10T21:26:15.545498+0000","last_unstale":"2024-09-10T21:26:17.038967+0000","last_undegraded":"2024-09-10T21:26:17.038967+0000","last_fullsized":"2024-09-10T21:26:17.038967+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:12.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037708+0000","last_change":"2024-09-10T21:26:15.546272+0000","last_active":"2024-09-10T21:26:17.037708+0000","last_peered":"2024-09-10T21:26:17.037708+0000","last_clean":"2024-09-10T21:26:17.037708+0000","last_became_active":"2024-09-10T21:26:15.537090+0000","last_became_peered":"2024-09-10T21:26:15.537090+0000","last_unstale":"2024-09-10T21:26:17.037708+0000","last_undegraded":"2024-09-10T21:26:17.037708+0000","last_fullsized":"2024-09-10T21:26:17.037708+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:10.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038217+0000","last_change":"2024-09-10T21:26:15.541138+0000","last_active":"2024-09-10T21:26:17.038217+0000","last_peered":"2024-09-10T21:26:17.038217+0000","last_clean":"2024-09-10T21:26:17.038217+0000","last_became_active":"2024-09-10T21:26:15.540540+0000","last_became_peered":"2024-09-10T21:26:15.540540+0000","last_unstale":"2024-09-10T21:26:17.038217+0000","last_undegraded":"2024-09-10T21:26:17.038217+0000","last_fullsized":"2024-09-10T21:26:17.038217+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:25.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038685+0000","last_change":"2024-09-10T21:26:15.534852+0000","last_active":"2024-09-10T21:26:17.038685+0000","last_peered":"2024-09-10T21:26:17.038685+0000","last_clean":"2024-09-10T21:26:17.038685+0000","last_became_active":"2024-09-10T21:26:15.534449+0000","last_became_peered":"2024-09-10T21:26:15.534449+0000","last_unstale":"2024-09-10T21:26:17.038685+0000","last_undegraded":"2024-09-10T21:26:17.038685+0000","last_fullsized":"2024-09-10T21:26:17.038685+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:14.802445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037944+0000","last_change":"2024-09-10T21:26:15.544506+0000","last_active":"2024-09-10T21:26:17.037944+0000","last_peered":"2024-09-10T21:26:17.037944+0000","last_clean":"2024-09-10T21:26:17.037944+0000","last_became_active":"2024-09-10T21:26:15.544258+0000","last_became_peered":"2024-09-10T21:26:15.544258+0000","last_unstale":"2024-09-10T21:26:17.037944+0000","last_undegraded":"2024-09-10T21:26:17.037944+0000","last_fullsized":"2024-09-10T21:26:17.037944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:57.577430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037886+0000","last_change":"2024-09-10T21:26:15.542753+0000","last_active":"2024-09-10T21:26:17.037886+0000","last_peered":"2024-09-10T21:26:17.037886+0000","last_clean":"2024-09-10T21:26:17.037886+0000","last_became_active":"2024-09-10T21:26:15.542492+0000","last_became_peered":"2024-09-10T21:26:15.542492+0000","last_unstale":"2024-09-10T21:26:17.037886+0000","last_undegraded":"2024-09-10T21:26:17.037886+0000","last_fullsized":"2024-09-10T21:26:17.037886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:22.467188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039240+0000","last_change":"2024-09-10T21:26:14.530389+0000","last_active":"2024-09-10T21:26:17.039240+0000","last_peered":"2024-09-10T21:26:17.039240+0000","last_clean":"2024-09-10T21:26:17.039240+0000","last_became_active":"2024-09-10T21:26:14.530024+0000","last_became_peered":"2024-09-10T21:26:14.530024+0000","last_unstale":"2024-09-10T21:26:17.039240+0000","last_undegraded":"2024-09-10T21:26:17.039240+0000","last_fullsized":"2024-09-10T21:26:17.039240+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:44.886118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037890+0000","last_change":"2024-09-10T21:26:15.546106+0000","last_active":"2024-09-10T21:26:17.037890+0000","last_peered":"2024-09-10T21:26:17.037890+0000","last_clean":"2024-09-10T21:26:17.037890+0000","last_became_active":"2024-09-10T21:26:15.545789+0000","last_became_peered":"2024-09-10T21:26:15.545789+0000","last_unstale":"2024-09-10T21:26:17.037890+0000","last_undegraded":"2024-09-10T21:26:17.037890+0000","last_fullsized":"2024-09-10T21:26:17.037890+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:00.355736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532838+0000","last_change":"2024-09-10T21:26:14.533996+0000","last_active":"2024-09-10T21:26:16.532838+0000","last_peered":"2024-09-10T21:26:16.532838+0000","last_clean":"2024-09-10T21:26:16.532838+0000","last_became_active":"2024-09-10T21:26:14.533618+0000","last_became_peered":"2024-09-10T21:26:14.533618+0000","last_unstale":"2024-09-10T21:26:16.532838+0000","last_undegraded":"2024-09-10T21:26:16.532838+0000","last_fullsized":"2024-09-10T21:26:16.532838+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:43.185675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038517+0000","last_change":"2024-09-10T21:26:15.541441+0000","last_active":"2024-09-10T21:26:17.038517+0000","last_peered":"2024-09-10T21:26:17.038517+0000","last_clean":"2024-09-10T21:26:17.038517+0000","last_became_active":"2024-09-10T21:26:15.540879+0000","last_became_peered":"2024-09-10T21:26:15.540879+0000","last_unstale":"2024-09-10T21:26:17.038517+0000","last_undegraded":"2024-09-10T21:26:17.038517+0000","last_fullsized":"2024-09-10T21:26:17.038517+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:00.252504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534598+0000","last_change":"2024-09-10T21:26:14.529412+0000","last_active":"2024-09-10T21:26:16.534598+0000","last_peered":"2024-09-10T21:26:16.534598+0000","last_clean":"2024-09-10T21:26:16.534598+0000","last_became_active":"2024-09-10T21:26:14.529112+0000","last_became_peered":"2024-09-10T21:26:14.529112+0000","last_unstale":"2024-09-10T21:26:16.534598+0000","last_undegraded":"2024-09-10T21:26:16.534598+0000","last_fullsized":"2024-09-10T21:26:16.534598+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:17.649740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534211+0000","last_change":"2024-09-10T21:26:15.537699+0000","last_active":"2024-09-10T21:26:16.534211+0000","last_peered":"2024-09-10T21:26:16.534211+0000","last_clean":"2024-09-10T21:26:16.534211+0000","last_became_active":"2024-09-10T21:26:15.533943+0000","last_became_peered":"2024-09-10T21:26:15.533943+0000","last_unstale":"2024-09-10T21:26:16.534211+0000","last_undegraded":"2024-09-10T21:26:16.534211+0000","last_fullsized":"2024-09-10T21:26:16.534211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:28.685486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039118+0000","last_change":"2024-09-10T21:26:14.529444+0000","last_active":"2024-09-10T21:26:17.039118+0000","last_peered":"2024-09-10T21:26:17.039118+0000","last_clean":"2024-09-10T21:26:17.039118+0000","last_became_active":"2024-09-10T21:26:14.529049+0000","last_became_peered":"2024-09-10T21:26:14.529049+0000","last_unstale":"2024-09-10T21:26:17.039118+0000","last_undegraded":"2024-09-10T21:26:17.039118+0000","last_fullsized":"2024-09-10T21:26:17.039118+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:52.760242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038980+0000","last_change":"2024-09-10T21:26:15.536274+0000","last_active":"2024-09-10T21:26:17.038980+0000","last_peered":"2024-09-10T21:26:17.038980+0000","last_clean":"2024-09-10T21:26:17.038980+0000","last_became_active":"2024-09-10T21:26:15.536085+0000","last_became_peered":"2024-09-10T21:26:15.536085+0000","last_unstale":"2024-09-10T21:26:17.038980+0000","last_undegraded":"2024-09-10T21:26:17.038980+0000","last_fullsized":"2024-09-10T21:26:17.038980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:50.896138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038897+0000","last_change":"2024-09-10T21:26:14.531407+0000","last_active":"2024-09-10T21:26:17.038897+0000","last_peered":"2024-09-10T21:26:17.038897+0000","last_clean":"2024-09-10T21:26:17.038897+0000","last_became_active":"2024-09-10T21:26:14.531128+0000","last_became_peered":"2024-09-10T21:26:14.531128+0000","last_unstale":"2024-09-10T21:26:17.038897+0000","last_undegraded":"2024-09-10T21:26:17.038897+0000","last_fullsized":"2024-09-10T21:26:17.038897+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:54.077340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038505+0000","last_change":"2024-09-10T21:26:15.540471+0000","last_active":"2024-09-10T21:26:17.038505+0000","last_peered":"2024-09-10T21:26:17.038505+0000","last_clean":"2024-09-10T21:26:17.038505+0000","last_became_active":"2024-09-10T21:26:15.539374+0000","last_became_peered":"2024-09-10T21:26:15.539374+0000","last_unstale":"2024-09-10T21:26:17.038505+0000","last_undegraded":"2024-09-10T21:26:17.038505+0000","last_fullsized":"2024-09-10T21:26:17.038505+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:45.415466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038375+0000","last_change":"2024-09-10T21:26:14.528783+0000","last_active":"2024-09-10T21:26:17.038375+0000","last_peered":"2024-09-10T21:26:17.038375+0000","last_clean":"2024-09-10T21:26:17.038375+0000","last_became_active":"2024-09-10T21:26:14.528509+0000","last_became_peered":"2024-09-10T21:26:14.528509+0000","last_unstale":"2024-09-10T21:26:17.038375+0000","last_undegraded":"2024-09-10T21:26:17.038375+0000","last_fullsized":"2024-09-10T21:26:17.038375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:48.911075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534561+0000","last_change":"2024-09-10T21:26:15.540735+0000","last_active":"2024-09-10T21:26:16.534561+0000","last_peered":"2024-09-10T21:26:16.534561+0000","last_clean":"2024-09-10T21:26:16.534561+0000","last_became_active":"2024-09-10T21:26:15.538227+0000","last_became_peered":"2024-09-10T21:26:15.538227+0000","last_unstale":"2024-09-10T21:26:16.534561+0000","last_undegraded":"2024-09-10T21:26:16.534561+0000","last_fullsized":"2024-09-10T21:26:16.534561+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:54.337627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038165+0000","last_change":"2024-09-10T21:26:14.527343+0000","last_active":"2024-09-10T21:26:17.038165+0000","last_peered":"2024-09-10T21:26:17.038165+0000","last_clean":"2024-09-10T21:26:17.038165+0000","last_became_active":"2024-09-10T21:26:14.527053+0000","last_became_peered":"2024-09-10T21:26:14.527053+0000","last_unstale":"2024-09-10T21:26:17.038165+0000","last_undegraded":"2024-09-10T21:26:17.038165+0000","last_fullsized":"2024-09-10T21:26:17.038165+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:49.564562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526941+0000","last_change":"2024-09-10T21:26:15.519138+0000","last_active":"2024-09-10T21:26:16.526941+0000","last_peered":"2024-09-10T21:26:16.526941+0000","last_clean":"2024-09-10T21:26:16.526941+0000","last_became_active":"2024-09-10T21:26:15.518903+0000","last_became_peered":"2024-09-10T21:26:15.518903+0000","last_unstale":"2024-09-10T21:26:16.526941+0000","last_undegraded":"2024-09-10T21:26:16.526941+0000","last_fullsized":"2024-09-10T21:26:16.526941+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:43.648100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038776+0000","last_change":"2024-09-10T21:26:14.522827+0000","last_active":"2024-09-10T21:26:17.038776+0000","last_peered":"2024-09-10T21:26:17.038776+0000","last_clean":"2024-09-10T21:26:17.038776+0000","last_became_active":"2024-09-10T21:26:14.522537+0000","last_became_peered":"2024-09-10T21:26:14.522537+0000","last_unstale":"2024-09-10T21:26:17.038776+0000","last_undegraded":"2024-09-10T21:26:17.038776+0000","last_fullsized":"2024-09-10T21:26:17.038776+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:15.763134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038782+0000","last_change":"2024-09-10T21:26:15.528432+0000","last_active":"2024-09-10T21:26:17.038782+0000","last_peered":"2024-09-10T21:26:17.038782+0000","last_clean":"2024-09-10T21:26:17.038782+0000","last_became_active":"2024-09-10T21:26:15.526149+0000","last_became_peered":"2024-09-10T21:26:15.526149+0000","last_unstale":"2024-09-10T21:26:17.038782+0000","last_undegraded":"2024-09-10T21:26:17.038782+0000","last_fullsized":"2024-09-10T21:26:17.038782+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:59.431959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.527007+0000","last_change":"2024-09-10T21:26:14.522970+0000","last_active":"2024-09-10T21:26:16.527007+0000","last_peered":"2024-09-10T21:26:16.527007+0000","last_clean":"2024-09-10T21:26:16.527007+0000","last_became_active":"2024-09-10T21:26:14.522696+0000","last_became_peered":"2024-09-10T21:26:14.522696+0000","last_unstale":"2024-09-10T21:26:16.527007+0000","last_undegraded":"2024-09-10T21:26:16.527007+0000","last_fullsized":"2024-09-10T21:26:16.527007+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:26.009061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526182+0000","last_change":"2024-09-10T21:26:15.537430+0000","last_active":"2024-09-10T21:26:16.526182+0000","last_peered":"2024-09-10T21:26:16.526182+0000","last_clean":"2024-09-10T21:26:16.526182+0000","last_became_active":"2024-09-10T21:26:15.537174+0000","last_became_peered":"2024-09-10T21:26:15.537174+0000","last_unstale":"2024-09-10T21:26:16.526182+0000","last_undegraded":"2024-09-10T21:26:16.526182+0000","last_fullsized":"2024-09-10T21:26:16.526182+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:27.005167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526824+0000","last_change":"2024-09-10T21:26:14.516829+0000","last_active":"2024-09-10T21:26:16.526824+0000","last_peered":"2024-09-10T21:26:16.526824+0000","last_clean":"2024-09-10T21:26:16.526824+0000","last_became_active":"2024-09-10T21:26:14.516564+0000","last_became_peered":"2024-09-10T21:26:14.516564+0000","last_unstale":"2024-09-10T21:26:16.526824+0000","last_undegraded":"2024-09-10T21:26:16.526824+0000","last_fullsized":"2024-09-10T21:26:16.526824+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:32.378883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037876+0000","last_change":"2024-09-10T21:26:15.545818+0000","last_active":"2024-09-10T21:26:17.037876+0000","last_peered":"2024-09-10T21:26:17.037876+0000","last_clean":"2024-09-10T21:26:17.037876+0000","last_became_active":"2024-09-10T21:26:15.545383+0000","last_became_peered":"2024-09-10T21:26:15.545383+0000","last_unstale":"2024-09-10T21:26:17.037876+0000","last_undegraded":"2024-09-10T21:26:17.037876+0000","last_fullsized":"2024-09-10T21:26:17.037876+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:59.431959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038543+0000","last_change":"2024-09-10T21:26:14.531817+0000","last_active":"2024-09-10T21:26:17.038543+0000","last_peered":"2024-09-10T21:26:17.038543+0000","last_clean":"2024-09-10T21:26:17.038543+0000","last_became_active":"2024-09-10T21:26:14.531387+0000","last_became_peered":"2024-09-10T21:26:14.531387+0000","last_unstale":"2024-09-10T21:26:17.038543+0000","last_undegraded":"2024-09-10T21:26:17.038543+0000","last_fullsized":"2024-09-10T21:26:17.038543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:58.645631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038299+0000","last_change":"2024-09-10T21:26:15.543162+0000","last_active":"2024-09-10T21:26:17.038299+0000","last_peered":"2024-09-10T21:26:17.038299+0000","last_clean":"2024-09-10T21:26:17.038299+0000","last_became_active":"2024-09-10T21:26:15.542913+0000","last_became_peered":"2024-09-10T21:26:15.542913+0000","last_unstale":"2024-09-10T21:26:17.038299+0000","last_undegraded":"2024-09-10T21:26:17.038299+0000","last_fullsized":"2024-09-10T21:26:17.038299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:27.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":21,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534219+0000","last_change":"2024-09-10T21:26:14.533065+0000","last_active":"2024-09-10T21:26:16.534219+0000","last_peered":"2024-09-10T21:26:16.534219+0000","last_clean":"2024-09-10T21:26:16.534219+0000","last_became_active":"2024-09-10T21:26:14.532788+0000","last_became_peered":"2024-09-10T21:26:14.532788+0000","last_unstale":"2024-09-10T21:26:16.534219+0000","last_undegraded":"2024-09-10T21:26:16.534219+0000","last_fullsized":"2024-09-10T21:26:16.534219+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:26.313137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038508+0000","last_change":"2024-09-10T21:26:15.540226+0000","last_active":"2024-09-10T21:26:17.038508+0000","last_peered":"2024-09-10T21:26:17.038508+0000","last_clean":"2024-09-10T21:26:17.038508+0000","last_became_active":"2024-09-10T21:26:15.539910+0000","last_became_peered":"2024-09-10T21:26:15.539910+0000","last_unstale":"2024-09-10T21:26:17.038508+0000","last_undegraded":"2024-09-10T21:26:17.038508+0000","last_fullsized":"2024-09-10T21:26:17.038508+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:35.441487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532903+0000","last_change":"2024-09-10T21:26:14.526625+0000","last_active":"2024-09-10T21:26:16.532903+0000","last_peered":"2024-09-10T21:26:16.532903+0000","last_clean":"2024-09-10T21:26:16.532903+0000","last_became_active":"2024-09-10T21:26:14.526301+0000","last_became_peered":"2024-09-10T21:26:14.526301+0000","last_unstale":"2024-09-10T21:26:16.532903+0000","last_undegraded":"2024-09-10T21:26:16.532903+0000","last_fullsized":"2024-09-10T21:26:16.532903+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:36.925973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037919+0000","last_change":"2024-09-10T21:26:15.547448+0000","last_active":"2024-09-10T21:26:17.037919+0000","last_peered":"2024-09-10T21:26:17.037919+0000","last_clean":"2024-09-10T21:26:17.037919+0000","last_became_active":"2024-09-10T21:26:15.546869+0000","last_became_peered":"2024-09-10T21:26:15.546869+0000","last_unstale":"2024-09-10T21:26:17.037919+0000","last_undegraded":"2024-09-10T21:26:17.037919+0000","last_fullsized":"2024-09-10T21:26:17.037919+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:16.043230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526236+0000","last_change":"2024-09-10T21:26:14.526509+0000","last_active":"2024-09-10T21:26:16.526236+0000","last_peered":"2024-09-10T21:26:16.526236+0000","last_clean":"2024-09-10T21:26:16.526236+0000","last_became_active":"2024-09-10T21:26:14.526211+0000","last_became_peered":"2024-09-10T21:26:14.526211+0000","last_unstale":"2024-09-10T21:26:16.526236+0000","last_undegraded":"2024-09-10T21:26:16.526236+0000","last_fullsized":"2024-09-10T21:26:16.526236+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:19.196183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533291+0000","last_change":"2024-09-10T21:26:15.533375+0000","last_active":"2024-09-10T21:26:16.533291+0000","last_peered":"2024-09-10T21:26:16.533291+0000","last_clean":"2024-09-10T21:26:16.533291+0000","last_became_active":"2024-09-10T21:26:15.533154+0000","last_became_peered":"2024-09-10T21:26:15.533154+0000","last_unstale":"2024-09-10T21:26:16.533291+0000","last_undegraded":"2024-09-10T21:26:16.533291+0000","last_fullsized":"2024-09-10T21:26:16.533291+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:45.889359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037987+0000","last_change":"2024-09-10T21:26:14.523579+0000","last_active":"2024-09-10T21:26:17.037987+0000","last_peered":"2024-09-10T21:26:17.037987+0000","last_clean":"2024-09-10T21:26:17.037987+0000","last_became_active":"2024-09-10T21:26:14.523321+0000","last_became_peered":"2024-09-10T21:26:14.523321+0000","last_unstale":"2024-09-10T21:26:17.037987+0000","last_undegraded":"2024-09-10T21:26:17.037987+0000","last_fullsized":"2024-09-10T21:26:17.037987+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:39.550410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039427+0000","last_change":"2024-09-10T21:26:15.528299+0000","last_active":"2024-09-10T21:26:17.039427+0000","last_peered":"2024-09-10T21:26:17.039427+0000","last_clean":"2024-09-10T21:26:17.039427+0000","last_became_active":"2024-09-10T21:26:15.528045+0000","last_became_peered":"2024-09-10T21:26:15.528045+0000","last_unstale":"2024-09-10T21:26:17.039427+0000","last_undegraded":"2024-09-10T21:26:17.039427+0000","last_fullsized":"2024-09-10T21:26:17.039427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:43.372388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532940+0000","last_change":"2024-09-10T21:26:14.526055+0000","last_active":"2024-09-10T21:26:16.532940+0000","last_peered":"2024-09-10T21:26:16.532940+0000","last_clean":"2024-09-10T21:26:16.532940+0000","last_became_active":"2024-09-10T21:26:14.525734+0000","last_became_peered":"2024-09-10T21:26:14.525734+0000","last_unstale":"2024-09-10T21:26:16.532940+0000","last_undegraded":"2024-09-10T21:26:16.532940+0000","last_fullsized":"2024-09-10T21:26:16.532940+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:15.163346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533323+0000","last_change":"2024-09-10T21:26:15.545468+0000","last_active":"2024-09-10T21:26:16.533323+0000","last_peered":"2024-09-10T21:26:16.533323+0000","last_clean":"2024-09-10T21:26:16.533323+0000","last_became_active":"2024-09-10T21:26:15.545137+0000","last_became_peered":"2024-09-10T21:26:15.545137+0000","last_unstale":"2024-09-10T21:26:16.533323+0000","last_undegraded":"2024-09-10T21:26:16.533323+0000","last_fullsized":"2024-09-10T21:26:16.533323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:50.896138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534468+0000","last_change":"2024-09-10T21:26:14.534718+0000","last_active":"2024-09-10T21:26:16.534468+0000","last_peered":"2024-09-10T21:26:16.534468+0000","last_clean":"2024-09-10T21:26:16.534468+0000","last_became_active":"2024-09-10T21:26:14.534222+0000","last_became_peered":"2024-09-10T21:26:14.534222+0000","last_unstale":"2024-09-10T21:26:16.534468+0000","last_undegraded":"2024-09-10T21:26:16.534468+0000","last_fullsized":"2024-09-10T21:26:16.534468+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:10.810025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037505+0000","last_change":"2024-09-10T21:26:15.546824+0000","last_active":"2024-09-10T21:26:17.037505+0000","last_peered":"2024-09-10T21:26:17.037505+0000","last_clean":"2024-09-10T21:26:17.037505+0000","last_became_active":"2024-09-10T21:26:15.546286+0000","last_became_peered":"2024-09-10T21:26:15.546286+0000","last_unstale":"2024-09-10T21:26:17.037505+0000","last_undegraded":"2024-09-10T21:26:17.037505+0000","last_fullsized":"2024-09-10T21:26:17.037505+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:44.045792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533460+0000","last_change":"2024-09-10T21:26:14.527929+0000","last_active":"2024-09-10T21:26:16.533460+0000","last_peered":"2024-09-10T21:26:16.533460+0000","last_clean":"2024-09-10T21:26:16.533460+0000","last_became_active":"2024-09-10T21:26:14.527583+0000","last_became_peered":"2024-09-10T21:26:14.527583+0000","last_unstale":"2024-09-10T21:26:16.533460+0000","last_undegraded":"2024-09-10T21:26:16.533460+0000","last_fullsized":"2024-09-10T21:26:16.533460+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:20.085776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038371+0000","last_change":"2024-09-10T21:26:15.541261+0000","last_active":"2024-09-10T21:26:17.038371+0000","last_peered":"2024-09-10T21:26:17.038371+0000","last_clean":"2024-09-10T21:26:17.038371+0000","last_became_active":"2024-09-10T21:26:15.541018+0000","last_became_peered":"2024-09-10T21:26:15.541018+0000","last_unstale":"2024-09-10T21:26:17.038371+0000","last_undegraded":"2024-09-10T21:26:17.038371+0000","last_fullsized":"2024-09-10T21:26:17.038371+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:27.252740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526745+0000","last_change":"2024-09-10T21:26:14.528002+0000","last_active":"2024-09-10T21:26:16.526745+0000","last_peered":"2024-09-10T21:26:16.526745+0000","last_clean":"2024-09-10T21:26:16.526745+0000","last_became_active":"2024-09-10T21:26:14.527640+0000","last_became_peered":"2024-09-10T21:26:14.527640+0000","last_unstale":"2024-09-10T21:26:16.526745+0000","last_undegraded":"2024-09-10T21:26:16.526745+0000","last_fullsized":"2024-09-10T21:26:16.526745+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:40.905930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.532554+0000","last_change":"2024-09-10T21:26:15.533913+0000","last_active":"2024-09-10T21:26:16.532554+0000","last_peered":"2024-09-10T21:26:16.532554+0000","last_clean":"2024-09-10T21:26:16.532554+0000","last_became_active":"2024-09-10T21:26:15.533607+0000","last_became_peered":"2024-09-10T21:26:15.533607+0000","last_unstale":"2024-09-10T21:26:16.532554+0000","last_undegraded":"2024-09-10T21:26:16.532554+0000","last_fullsized":"2024-09-10T21:26:16.532554+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:16.984327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039176+0000","last_change":"2024-09-10T21:26:14.530739+0000","last_active":"2024-09-10T21:26:17.039176+0000","last_peered":"2024-09-10T21:26:17.039176+0000","last_clean":"2024-09-10T21:26:17.039176+0000","last_became_active":"2024-09-10T21:26:14.530428+0000","last_became_peered":"2024-09-10T21:26:14.530428+0000","last_unstale":"2024-09-10T21:26:17.039176+0000","last_undegraded":"2024-09-10T21:26:17.039176+0000","last_fullsized":"2024-09-10T21:26:17.039176+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:47.051011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039019+0000","last_change":"2024-09-10T21:26:15.543515+0000","last_active":"2024-09-10T21:26:17.039019+0000","last_peered":"2024-09-10T21:26:17.039019+0000","last_clean":"2024-09-10T21:26:17.039019+0000","last_became_active":"2024-09-10T21:26:15.543291+0000","last_became_peered":"2024-09-10T21:26:15.543291+0000","last_unstale":"2024-09-10T21:26:17.039019+0000","last_undegraded":"2024-09-10T21:26:17.039019+0000","last_fullsized":"2024-09-10T21:26:17.039019+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:50.896138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037680+0000","last_change":"2024-09-10T21:26:14.532927+0000","last_active":"2024-09-10T21:26:17.037680+0000","last_peered":"2024-09-10T21:26:17.037680+0000","last_clean":"2024-09-10T21:26:17.037680+0000","last_became_active":"2024-09-10T21:26:14.532625+0000","last_became_peered":"2024-09-10T21:26:14.532625+0000","last_unstale":"2024-09-10T21:26:17.037680+0000","last_undegraded":"2024-09-10T21:26:17.037680+0000","last_fullsized":"2024-09-10T21:26:17.037680+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:36.431767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526563+0000","last_change":"2024-09-10T21:26:15.529708+0000","last_active":"2024-09-10T21:26:16.526563+0000","last_peered":"2024-09-10T21:26:16.526563+0000","last_clean":"2024-09-10T21:26:16.526563+0000","last_became_active":"2024-09-10T21:26:15.529431+0000","last_became_peered":"2024-09-10T21:26:15.529431+0000","last_unstale":"2024-09-10T21:26:16.526563+0000","last_undegraded":"2024-09-10T21:26:16.526563+0000","last_fullsized":"2024-09-10T21:26:16.526563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:27.674749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038684+0000","last_change":"2024-09-10T21:26:14.526244+0000","last_active":"2024-09-10T21:26:17.038684+0000","last_peered":"2024-09-10T21:26:17.038684+0000","last_clean":"2024-09-10T21:26:17.038684+0000","last_became_active":"2024-09-10T21:26:14.525952+0000","last_became_peered":"2024-09-10T21:26:14.525952+0000","last_unstale":"2024-09-10T21:26:17.038684+0000","last_undegraded":"2024-09-10T21:26:17.038684+0000","last_fullsized":"2024-09-10T21:26:17.038684+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:48.228747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.037873+0000","last_change":"2024-09-10T21:26:15.544629+0000","last_active":"2024-09-10T21:26:17.037873+0000","last_peered":"2024-09-10T21:26:17.037873+0000","last_clean":"2024-09-10T21:26:17.037873+0000","last_became_active":"2024-09-10T21:26:15.540919+0000","last_became_peered":"2024-09-10T21:26:15.540919+0000","last_unstale":"2024-09-10T21:26:17.037873+0000","last_undegraded":"2024-09-10T21:26:17.037873+0000","last_fullsized":"2024-09-10T21:26:17.037873+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:37.653560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533679+0000","last_change":"2024-09-10T21:26:14.529271+0000","last_active":"2024-09-10T21:26:16.533679+0000","last_peered":"2024-09-10T21:26:16.533679+0000","last_clean":"2024-09-10T21:26:16.533679+0000","last_became_active":"2024-09-10T21:26:14.528963+0000","last_became_peered":"2024-09-10T21:26:14.528963+0000","last_unstale":"2024-09-10T21:26:16.533679+0000","last_undegraded":"2024-09-10T21:26:16.533679+0000","last_fullsized":"2024-09-10T21:26:16.533679+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:14.592093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534305+0000","last_change":"2024-09-10T21:26:15.540533+0000","last_active":"2024-09-10T21:26:16.534305+0000","last_peered":"2024-09-10T21:26:16.534305+0000","last_clean":"2024-09-10T21:26:16.534305+0000","last_became_active":"2024-09-10T21:26:15.540237+0000","last_became_peered":"2024-09-10T21:26:15.540237+0000","last_unstale":"2024-09-10T21:26:16.534305+0000","last_undegraded":"2024-09-10T21:26:16.534305+0000","last_fullsized":"2024-09-10T21:26:16.534305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:33.376220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533495+0000","last_change":"2024-09-10T21:26:14.529847+0000","last_active":"2024-09-10T21:26:16.533495+0000","last_peered":"2024-09-10T21:26:16.533495+0000","last_clean":"2024-09-10T21:26:16.533495+0000","last_became_active":"2024-09-10T21:26:14.529074+0000","last_became_peered":"2024-09-10T21:26:14.529074+0000","last_unstale":"2024-09-10T21:26:16.533495+0000","last_undegraded":"2024-09-10T21:26:16.533495+0000","last_fullsized":"2024-09-10T21:26:16.533495+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:27.478486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526111+0000","last_change":"2024-09-10T21:26:15.528858+0000","last_active":"2024-09-10T21:26:16.526111+0000","last_peered":"2024-09-10T21:26:16.526111+0000","last_clean":"2024-09-10T21:26:16.526111+0000","last_became_active":"2024-09-10T21:26:15.527735+0000","last_became_peered":"2024-09-10T21:26:15.527735+0000","last_unstale":"2024-09-10T21:26:16.526111+0000","last_undegraded":"2024-09-10T21:26:16.526111+0000","last_fullsized":"2024-09-10T21:26:16.526111+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:56.452919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038162+0000","last_change":"2024-09-10T21:26:14.529877+0000","last_active":"2024-09-10T21:26:17.038162+0000","last_peered":"2024-09-10T21:26:17.038162+0000","last_clean":"2024-09-10T21:26:17.038162+0000","last_became_active":"2024-09-10T21:26:14.529580+0000","last_became_peered":"2024-09-10T21:26:14.529580+0000","last_unstale":"2024-09-10T21:26:17.038162+0000","last_undegraded":"2024-09-10T21:26:17.038162+0000","last_fullsized":"2024-09-10T21:26:17.038162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:41.737020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533847+0000","last_change":"2024-09-10T21:26:15.541706+0000","last_active":"2024-09-10T21:26:16.533847+0000","last_peered":"2024-09-10T21:26:16.533847+0000","last_clean":"2024-09-10T21:26:16.533847+0000","last_became_active":"2024-09-10T21:26:15.540365+0000","last_became_peered":"2024-09-10T21:26:15.540365+0000","last_unstale":"2024-09-10T21:26:16.533847+0000","last_undegraded":"2024-09-10T21:26:16.533847+0000","last_fullsized":"2024-09-10T21:26:16.533847+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:39.968653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038594+0000","last_change":"2024-09-10T21:26:14.532072+0000","last_active":"2024-09-10T21:26:17.038594+0000","last_peered":"2024-09-10T21:26:17.038594+0000","last_clean":"2024-09-10T21:26:17.038594+0000","last_became_active":"2024-09-10T21:26:14.530711+0000","last_became_peered":"2024-09-10T21:26:14.530711+0000","last_unstale":"2024-09-10T21:26:17.038594+0000","last_undegraded":"2024-09-10T21:26:17.038594+0000","last_fullsized":"2024-09-10T21:26:17.038594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:15.362786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038829+0000","last_change":"2024-09-10T21:26:15.524631+0000","last_active":"2024-09-10T21:26:17.038829+0000","last_peered":"2024-09-10T21:26:17.038829+0000","last_clean":"2024-09-10T21:26:17.038829+0000","last_became_active":"2024-09-10T21:26:15.524305+0000","last_became_peered":"2024-09-10T21:26:15.524305+0000","last_unstale":"2024-09-10T21:26:17.038829+0000","last_undegraded":"2024-09-10T21:26:17.038829+0000","last_fullsized":"2024-09-10T21:26:17.038829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:36.491077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039016+0000","last_change":"2024-09-10T21:26:14.525570+0000","last_active":"2024-09-10T21:26:17.039016+0000","last_peered":"2024-09-10T21:26:17.039016+0000","last_clean":"2024-09-10T21:26:17.039016+0000","last_became_active":"2024-09-10T21:26:14.525294+0000","last_became_peered":"2024-09-10T21:26:14.525294+0000","last_unstale":"2024-09-10T21:26:17.039016+0000","last_undegraded":"2024-09-10T21:26:17.039016+0000","last_fullsized":"2024-09-10T21:26:17.039016+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:26.579850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534195+0000","last_change":"2024-09-10T21:26:15.542738+0000","last_active":"2024-09-10T21:26:16.534195+0000","last_peered":"2024-09-10T21:26:16.534195+0000","last_clean":"2024-09-10T21:26:16.534195+0000","last_became_active":"2024-09-10T21:26:15.542497+0000","last_became_peered":"2024-09-10T21:26:15.542497+0000","last_unstale":"2024-09-10T21:26:16.534195+0000","last_undegraded":"2024-09-10T21:26:16.534195+0000","last_fullsized":"2024-09-10T21:26:16.534195+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:07.366327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534080+0000","last_change":"2024-09-10T21:26:14.530559+0000","last_active":"2024-09-10T21:26:16.534080+0000","last_peered":"2024-09-10T21:26:16.534080+0000","last_clean":"2024-09-10T21:26:16.534080+0000","last_became_active":"2024-09-10T21:26:14.529513+0000","last_became_peered":"2024-09-10T21:26:14.529513+0000","last_unstale":"2024-09-10T21:26:16.534080+0000","last_undegraded":"2024-09-10T21:26:16.534080+0000","last_fullsized":"2024-09-10T21:26:16.534080+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:10.595223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.039417+0000","last_change":"2024-09-10T21:26:15.533560+0000","last_active":"2024-09-10T21:26:17.039417+0000","last_peered":"2024-09-10T21:26:17.039417+0000","last_clean":"2024-09-10T21:26:17.039417+0000","last_became_active":"2024-09-10T21:26:15.533313+0000","last_became_peered":"2024-09-10T21:26:15.533313+0000","last_unstale":"2024-09-10T21:26:17.039417+0000","last_undegraded":"2024-09-10T21:26:17.039417+0000","last_fullsized":"2024-09-10T21:26:17.039417+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:42.948410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038798+0000","last_change":"2024-09-10T21:26:14.525715+0000","last_active":"2024-09-10T21:26:17.038798+0000","last_peered":"2024-09-10T21:26:17.038798+0000","last_clean":"2024-09-10T21:26:17.038798+0000","last_became_active":"2024-09-10T21:26:14.525367+0000","last_became_peered":"2024-09-10T21:26:14.525367+0000","last_unstale":"2024-09-10T21:26:17.038798+0000","last_undegraded":"2024-09-10T21:26:17.038798+0000","last_fullsized":"2024-09-10T21:26:17.038798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:36.377629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534823+0000","last_change":"2024-09-10T21:26:15.536138+0000","last_active":"2024-09-10T21:26:16.534823+0000","last_peered":"2024-09-10T21:26:16.534823+0000","last_clean":"2024-09-10T21:26:16.534823+0000","last_became_active":"2024-09-10T21:26:15.535521+0000","last_became_peered":"2024-09-10T21:26:15.535521+0000","last_unstale":"2024-09-10T21:26:16.534823+0000","last_undegraded":"2024-09-10T21:26:16.534823+0000","last_fullsized":"2024-09-10T21:26:16.534823+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:38.733815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038026+0000","last_change":"2024-09-10T21:26:14.519898+0000","last_active":"2024-09-10T21:26:17.038026+0000","last_peered":"2024-09-10T21:26:17.038026+0000","last_clean":"2024-09-10T21:26:17.038026+0000","last_became_active":"2024-09-10T21:26:14.519606+0000","last_became_peered":"2024-09-10T21:26:14.519606+0000","last_unstale":"2024-09-10T21:26:17.038026+0000","last_undegraded":"2024-09-10T21:26:17.038026+0000","last_fullsized":"2024-09-10T21:26:17.038026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.608822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.533980+0000","last_change":"2024-09-10T21:26:15.532747+0000","last_active":"2024-09-10T21:26:16.533980+0000","last_peered":"2024-09-10T21:26:16.533980+0000","last_clean":"2024-09-10T21:26:16.533980+0000","last_became_active":"2024-09-10T21:26:15.532482+0000","last_became_peered":"2024-09-10T21:26:15.532482+0000","last_unstale":"2024-09-10T21:26:16.533980+0000","last_undegraded":"2024-09-10T21:26:16.533980+0000","last_fullsized":"2024-09-10T21:26:16.533980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:10.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526041+0000","last_change":"2024-09-10T21:26:14.526821+0000","last_active":"2024-09-10T21:26:16.526041+0000","last_peered":"2024-09-10T21:26:16.526041+0000","last_clean":"2024-09-10T21:26:16.526041+0000","last_became_active":"2024-09-10T21:26:14.526561+0000","last_became_peered":"2024-09-10T21:26:14.526561+0000","last_unstale":"2024-09-10T21:26:16.526041+0000","last_undegraded":"2024-09-10T21:26:16.526041+0000","last_fullsized":"2024-09-10T21:26:16.526041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:44.332401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:17.038982+0000","last_change":"2024-09-10T21:26:15.538996+0000","last_active":"2024-09-10T21:26:17.038982+0000","last_peered":"2024-09-10T21:26:17.038982+0000","last_clean":"2024-09-10T21:26:17.038982+0000","last_became_active":"2024-09-10T21:26:15.538393+0000","last_became_peered":"2024-09-10T21:26:15.538393+0000","last_unstale":"2024-09-10T21:26:17.038982+0000","last_undegraded":"2024-09-10T21:26:17.038982+0000","last_fullsized":"2024-09-10T21:26:17.038982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:14.497932+0000","last_clean_scrub_stamp":"2024-09-10T21:26:14.497932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:12.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.534965+0000","last_change":"2024-09-10T21:26:14.533819+0000","last_active":"2024-09-10T21:26:16.534965+0000","last_peered":"2024-09-10T21:26:16.534965+0000","last_clean":"2024-09-10T21:26:16.534965+0000","last_became_active":"2024-09-10T21:26:14.533554+0000","last_became_peered":"2024-09-10T21:26:14.533554+0000","last_unstale":"2024-09-10T21:26:16.534965+0000","last_undegraded":"2024-09-10T21:26:16.534965+0000","last_fullsized":"2024-09-10T21:26:16.534965+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:13.494691+0000","last_clean_scrub_stamp":"2024-09-10T21:26:13.494691+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:55.449678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T21:26:16.526168+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:26:16.526168+0000","last_peered":"2024-09-10T21:26:16.526168+0000","last_clean":"2024-09-10T21:26:16.526168+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:26:16.526168+0000","last_undegraded":"2024-09-10T21:26:16.526168+0000","last_fullsized":"2024-09-10T21:26:16.526168+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":643072,"data_stored":282343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":282869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":1245184,"data_stored":874357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":282377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":282321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1232896,"data_stored":872725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":282869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":283991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:26:24.574 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:26:24.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:26:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.759+0000 7ff80ae00640 1 -- 172.21.15.97:0/1556909016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e80052f0 msgr2=0x7ff7e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.759+0000 7ff80ae00640 1 --2- 172.21.15.97:0/1556909016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e80052f0 0x7ff7e800b9d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009e00 tx=0x7ff7fc01d170 comp rx=0 tx=0).stop 2024-09-10T21:26:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.759+0000 7ff80ae00640 1 -- 172.21.15.97:0/1556909016 shutdown_connections 2024-09-10T21:26:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.759+0000 7ff80ae00640 1 --2- 172.21.15.97:0/1556909016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e80052f0 0x7ff7e800b9d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.759+0000 7ff80ae00640 1 -- 172.21.15.97:0/1556909016 >> 172.21.15.97:0/1556909016 conn(0x7ff8040b37c0 msgr2=0x7ff8040b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.759+0000 7ff80ae00640 1 -- 172.21.15.97:0/1556909016 shutdown_connections 2024-09-10T21:26:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.759+0000 7ff80ae00640 1 -- 172.21.15.97:0/1556909016 wait complete. 2024-09-10T21:26:24.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 Processor -- start 2024-09-10T21:26:24.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 -- start start 2024-09-10T21:26:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7e80052f0 0x7ff8040b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff804155e40 0x7ff8040b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8040b5000 0x7ff80415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8040ce320 con 0x7ff8040b5000 2024-09-10T21:26:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8040ce1a0 con 0x7ff804155e40 2024-09-10T21:26:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff8040b42e0 con 0x7ff7e80052f0 2024-09-10T21:26:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8040b5000 0x7ff80415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8040b5000 0x7ff80415ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41704/0 (socket says 172.21.15.97:41704) 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 -- 172.21.15.97:0/326034192 learned_addr learned my addr 172.21.15.97:0/326034192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff803e00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7e80052f0 0x7ff8040b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff7fbe00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff804155e40 0x7ff8040b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 -- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7e80052f0 msgr2=0x7ff8040b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7e80052f0 0x7ff8040b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 -- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff804155e40 msgr2=0x7ff8040b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff804155e40 0x7ff8040b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 -- 172.21.15.97:0/326034192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8040b54f0 con 0x7ff8040b5000 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff7fbe00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff804155e40 0x7ff8040b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:24.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff808e00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8040b5000 0x7ff80415ea50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc01cfa0 tx=0x7ff7fc0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff800c00640 1 -- 172.21.15.97:0/326034192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0bd070 con 0x7ff8040b5000 2024-09-10T21:26:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff800c00640 1 -- 172.21.15.97:0/326034192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7fc0a5e60 con 0x7ff8040b5000 2024-09-10T21:26:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8040b57d0 con 0x7ff8040b5000 2024-09-10T21:26:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff800c00640 1 -- 172.21.15.97:0/326034192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc009a00 con 0x7ff8040b5000 2024-09-10T21:26:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8040b5d90 con 0x7ff8040b5000 2024-09-10T21:26:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.763+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8040cf800 con 0x7ff8040b5000 2024-09-10T21:26:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.767+0000 7ff800c00640 1 -- 172.21.15.97:0/326034192 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7fc00f040 con 0x7ff8040b5000 2024-09-10T21:26:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.767+0000 7ff800c00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7e0072610 0x7ff7e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.767+0000 7ff800c00640 1 -- 172.21.15.97:0/326034192 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7ff7fc12d9b0 con 0x7ff8040b5000 2024-09-10T21:26:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.767+0000 7ff803e00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7e0072610 0x7ff7e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.767+0000 7ff803e00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7e0072610 0x7ff7e0074ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8041e8bb0 tx=0x7ff7f4006c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.771+0000 7ff800c00640 1 -- 172.21.15.97:0/326034192 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7fc016e80 con 0x7ff8040b5000 2024-09-10T21:26:24.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.995+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff8040b0d10 con 0x7ff8040b5000 2024-09-10T21:26:25.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff800c00640 1 -- 172.21.15.97:0/326034192 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7ff7fc009280 con 0x7ff8040b5000 2024-09-10T21:26:25.001 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-10T21:26:25.001 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:26:25.001 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-09-10T21:26:25.001 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-09-10T21:26:25.001 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:26:25.001 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7e0072610 msgr2=0x7ff7e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7e0072610 0x7ff7e0074ad0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8041e8bb0 tx=0x7ff7f4006c50 comp rx=0 tx=0).stop 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8040b5000 msgr2=0x7ff80415ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8040b5000 0x7ff80415ea50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc01cfa0 tx=0x7ff7fc0046a0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 shutdown_connections 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7e0072610 0x7ff7e0074ad0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8040b5000 0x7ff80415ea50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff804155e40 0x7ff8040b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 --2- 172.21.15.97:0/326034192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7e80052f0 0x7ff8040b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 >> 172.21.15.97:0/326034192 conn(0x7ff8040b37c0 msgr2=0x7ff8040af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 shutdown_connections 2024-09-10T21:26:25.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:24.999+0000 7ff80ae00640 1 -- 172.21.15.97:0/326034192 wait complete. 2024-09-10T21:26:25.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:25.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.159+0000 7f6542200640 1 -- 172.21.15.97:0/559816007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0ce830 msgr2=0x7f653c0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.159+0000 7f6542200640 1 --2- 172.21.15.97:0/559816007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0ce830 0x7f653c0cf800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f653c0742f0 tx=0x7f652c01f580 comp rx=0 tx=0).stop 2024-09-10T21:26:25.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.159+0000 7f6542200640 1 -- 172.21.15.97:0/559816007 shutdown_connections 2024-09-10T21:26:25.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.159+0000 7f6542200640 1 --2- 172.21.15.97:0/559816007 >> v2:172.21.15.194:3300/0 conn(0x7f653c0d0280 0x7f653c15fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.159+0000 7f6542200640 1 --2- 172.21.15.97:0/559816007 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f653c0cec10 0x7f653c0cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.159+0000 7f6542200640 1 --2- 172.21.15.97:0/559816007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0ce830 0x7f653c0cf800 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.159+0000 7f6542200640 1 -- 172.21.15.97:0/559816007 >> 172.21.15.97:0/559816007 conn(0x7f653c0b37c0 msgr2=0x7f653c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 -- 172.21.15.97:0/559816007 shutdown_connections 2024-09-10T21:26:25.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 -- 172.21.15.97:0/559816007 wait complete. 2024-09-10T21:26:25.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 Processor -- start 2024-09-10T21:26:25.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 -- start start 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f653c0ce830 0x7f653c1f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f653c0cec10 0x7f653c1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0d0280 0x7f653c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f653c0b4480 con 0x7f653c0d0280 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f653c0cde60 con 0x7f653c0cec10 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6542200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f653c0ce1a0 con 0x7f653c0ce830 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0d0280 0x7f653c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0d0280 0x7f653c15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41716/0 (socket says 172.21.15.97:41716) 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 -- 172.21.15.97:0/1205050497 learned_addr learned my addr 172.21.15.97:0/1205050497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:25.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653b400640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f653c0ce830 0x7f653c1f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f6532a00640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f653c0cec10 0x7f653c1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 -- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f653c0ce830 msgr2=0x7f653c1f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f653c0ce830 0x7f653c1f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 -- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f653c0cec10 msgr2=0x7f653c1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f653c0cec10 0x7f653c1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 -- 172.21.15.97:0/1205050497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c15c610 con 0x7f653c0d0280 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.163+0000 7f653be00640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0d0280 0x7f653c15a370 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f653400e740 tx=0x7f653400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6533e00640 1 -- 172.21.15.97:0/1205050497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400ccc0 con 0x7f653c0d0280 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6533e00640 1 -- 172.21.15.97:0/1205050497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f653400ce80 con 0x7f653c0d0280 2024-09-10T21:26:25.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6533e00640 1 -- 172.21.15.97:0/1205050497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534005280 con 0x7f653c0d0280 2024-09-10T21:26:25.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653c15c890 con 0x7f653c0d0280 2024-09-10T21:26:25.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653c1f36a0 con 0x7f653c0d0280 2024-09-10T21:26:25.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653c15fd20 con 0x7f653c0d0280 2024-09-10T21:26:25.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6533e00640 1 -- 172.21.15.97:0/1205050497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6534005420 con 0x7f653c0d0280 2024-09-10T21:26:25.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6533e00640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6518072610 0x7f6518074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f6533e00640 1 -- 172.21.15.97:0/1205050497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f6534093af0 con 0x7f653c0d0280 2024-09-10T21:26:25.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.167+0000 7f653b400640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6518072610 0x7f6518074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.171+0000 7f653b400640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6518072610 0x7f6518074ad0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f652c0b2000 tx=0x7f652c01ddc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.171+0000 7f6533e00640 1 -- 172.21.15.97:0/1205050497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f653c15c610 con 0x7f653c0d0280 2024-09-10T21:26:25.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.371+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f653c071a40 con 0x7f653c0d0280 2024-09-10T21:26:25.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.371+0000 7f6533e00640 1 -- 172.21.15.97:0/1205050497 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5223 (secure 0 0 0) 0x7f653405e9f0 con 0x7f653c0d0280 2024-09-10T21:26:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:25.375 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"btime":"2024-09-10T21:26:18:534778+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:17.919032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5484":{"gid":5484,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6834/3721865460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3721865460},{"type":"v1","addr":"172.21.15.97:6834","nonce":3721865460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5484,"qdb_cluster":[5484]},"id":3}]} 2024-09-10T21:26:25.375 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 33 2024-09-10T21:26:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6518072610 msgr2=0x7f6518074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6518072610 0x7f6518074ad0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f652c0b2000 tx=0x7f652c01ddc0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0d0280 msgr2=0x7f653c15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0d0280 0x7f653c15a370 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f653400e740 tx=0x7f653400ec00 comp rx=0 tx=0).stop 2024-09-10T21:26:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 shutdown_connections 2024-09-10T21:26:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6518072610 0x7f6518074ad0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f653c0d0280 0x7f653c15a370 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f653c0cec10 0x7f653c1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 --2- 172.21.15.97:0/1205050497 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f653c0ce830 0x7f653c1f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 >> 172.21.15.97:0/1205050497 conn(0x7f653c0b37c0 msgr2=0x7f653c0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 shutdown_connections 2024-09-10T21:26:25.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.375+0000 7f6542200640 1 -- 172.21.15.97:0/1205050497 wait complete. 2024-09-10T21:26:25.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:14.903489+0000', 'modified': '2024-09-10T21:26:17.919032+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5484}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5484': {'gid': 5484, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6834/3721865460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 3721865460}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 3721865460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5484, 'qdb_cluster': [5484]} 2024-09-10T21:26:25.801 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:26:25.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:26:25.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf5600640 1 -- 172.21.15.97:0/1893460102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cd40051d0 msgr2=0x7f4cd400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf5600640 1 --2- 172.21.15.97:0/1893460102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cd40051d0 0x7f4cd400b9d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00cc060 tx=0x7f4ce801cc20 comp rx=0 tx=0).stop 2024-09-10T21:26:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf4c00640 1 -- 172.21.15.97:0/1893460102 reap_dead start 2024-09-10T21:26:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf5600640 1 -- 172.21.15.97:0/1893460102 shutdown_connections 2024-09-10T21:26:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf5600640 1 -- 172.21.15.97:0/1893460102 >> 172.21.15.97:0/1893460102 conn(0x7f4cf00d1a60 msgr2=0x7f4cf00d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf5600640 1 -- 172.21.15.97:0/1893460102 shutdown_connections 2024-09-10T21:26:25.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf5600640 1 -- 172.21.15.97:0/1893460102 wait complete. 2024-09-10T21:26:25.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.943+0000 7f4cf5600640 1 Processor -- start 2024-09-10T21:26:25.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 -- start start 2024-09-10T21:26:25.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cd40051d0 0x7f4cf01553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf0155c50 0x7f4cf0070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4cf0070ea0 0x7f4cf00ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00cfbd0 con 0x7f4cf0155c50 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00cfa50 con 0x7f4cd40051d0 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf00af040 con 0x7f4cf0070ea0 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf0155c50 0x7f4cf0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf0155c50 0x7f4cf0070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41738/0 (socket says 172.21.15.97:41738) 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 -- 172.21.15.97:0/2364531219 learned_addr learned my addr 172.21.15.97:0/2364531219 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cefe00640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cd40051d0 0x7f4cf01553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4cf0070ea0 0x7f4cf00ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 -- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4cf0070ea0 msgr2=0x7f4cf00ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4cf0070ea0 0x7f4cf00ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 -- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cd40051d0 msgr2=0x7f4cf01553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cd40051d0 0x7f4cf01553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 -- 172.21.15.97:0/2364531219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf0155920 con 0x7f4cf0155c50 2024-09-10T21:26:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cefe00640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cd40051d0 0x7f4cf01553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cef400640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf0155c50 0x7f4cf0070960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0126c0 tx=0x7f4cdc012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4ce7e00640 1 -- 172.21.15.97:0/2364531219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc010ae0 con 0x7f4cf0155c50 2024-09-10T21:26:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4ce7e00640 1 -- 172.21.15.97:0/2364531219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cdc017030 con 0x7f4cf0155c50 2024-09-10T21:26:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf00c5650 con 0x7f4cf0155c50 2024-09-10T21:26:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4ce7e00640 1 -- 172.21.15.97:0/2364531219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc013450 con 0x7f4cf0155c50 2024-09-10T21:26:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.947+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf00c5c10 con 0x7f4cf0155c50 2024-09-10T21:26:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4ce7e00640 1 -- 172.21.15.97:0/2364531219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4cdc0135f0 con 0x7f4cf0155c50 2024-09-10T21:26:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4ce7e00640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4cd0072610 0x7f4cd0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 Processor -- start 2024-09-10T21:26:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4ce7e00640 1 -- 172.21.15.97:0/2364531219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4cdc096fb0 con 0x7f4cf0155c50 2024-09-10T21:26:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 -- start start 2024-09-10T21:26:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cefe00640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4cd0072610 0x7f4cd0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734cdc0 0x563c9734d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734d6e0 0x563c97300ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c97301600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563c9721d3f0 con 0x563c97301220 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c9715ab20 con 0x563c9734cdc0 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563c97149f50 con 0x563c9734d6e0 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cefe00640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4cd0072610 0x7f4cd0074ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0073490 tx=0x7f4ce00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c97301600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c97301600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41748/0 (socket says 172.21.15.97:41748) 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 -- 172.21.15.97:0/1038843867 learned_addr learned my addr 172.21.15.97:0/1038843867 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cef400640 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734d6e0 0x563c97300ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cefe00640 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734cdc0 0x563c9734d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 -- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734d6e0 msgr2=0x563c97300ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734d6e0 0x563c97300ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 -- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734cdc0 msgr2=0x563c9734d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734cdc0 0x563c9734d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 -- 172.21.15.97:0/1038843867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c97111590 con 0x563c97301220 2024-09-10T21:26:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cefe00640 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734cdc0 0x563c9734d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c97301600 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4ce801d4c0 tx=0x7f4cd4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f245a7fead1d95b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4ce4c00640 1 -- 172.21.15.97:0/1038843867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80a5ad0 con 0x563c97301220 2024-09-10T21:26:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4ce4c00640 1 -- 172.21.15.97:0/1038843867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ce8003980 con 0x563c97301220 2024-09-10T21:26:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.951+0000 7f4ce4c00640 1 -- 172.21.15.97:0/1038843867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8003d50 con 0x563c97301220 2024-09-10T21:26:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1038843867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 msgr2=0x563c97301600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c97301600 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4ce801d4c0 tx=0x7f4cd4005840 comp rx=0 tx=0).stop 2024-09-10T21:26:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1038843867 shutdown_connections 2024-09-10T21:26:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c97301600 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734d6e0 0x563c97300ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1038843867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734cdc0 0x563c9734d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1038843867 >> 172.21.15.97:0/1038843867 conn(0x563c971e6130 msgr2=0x563c9732be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1038843867 shutdown_connections 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1038843867 wait complete. 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- start start 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 Processor -- start 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c97301220 0x563c97337360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734d6e0 0x563c97337de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563c970e45d0 con 0x563c9734d6e0 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c9714dd60 con 0x563c97301220 2024-09-10T21:26:25.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563c970f0500 con 0x563c9734cdc0 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 10 client.0 _ll_register_callbacks cb 0x563c970f4810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734d6e0 0x563c97337de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734d6e0 0x563c97337de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41750/0 (socket says 172.21.15.97:41750) 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 -- 172.21.15.97:0/170217982 learned_addr learned my addr 172.21.15.97:0/170217982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cefe00640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c97301220 0x563c97337360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cef400640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 -- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 msgr2=0x563c973378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 -- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c97301220 msgr2=0x563c97337360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c97301220 0x563c97337360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 -- 172.21.15.97:0/170217982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c970e8170 con 0x563c9734d6e0 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cef400640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734d6e0 0x563c97337de0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4ce801d4c0 tx=0x7f4ce8003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca2e13d405803b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cd9600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cd9600640 1 -- 172.21.15.97:0/170217982 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8003d50 con 0x563c9734d6e0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cd9600640 1 -- 172.21.15.97:0/170217982 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4ce80bf990 con 0x563c9734d6e0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c970e20f0 con 0x563c9734d6e0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cd9600640 1 -- 172.21.15.97:0/170217982 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4ce80bfb90 con 0x563c9734d6e0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 10 client.5652 fetch_fsmap learned FSMap version 33 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x563c971ceef0 con 0x563c9734d6e0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.955+0000 7f4cd9600640 1 -- 172.21.15.97:0/170217982 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563c970e20f0 con 0x563c9734d6e0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cd9600640 1 -- 172.21.15.97:0/170217982 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7f4ce80b8020 con 0x563c9734d6e0 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 10 client.5652 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 10 client.5652 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 20 client.5652 populate_metadata read hostname 'smithi097' 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c973387f0 0x563c9735e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 4 client.5652 mds_command: new command op to 5484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:25.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 --> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c9715ca70 con 0x563c973387f0 2024-09-10T21:26:25.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cefe00640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c973387f0 0x563c9735e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cefe00640 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c973387f0 0x563c9735e070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7318fce50ea40780 server_cookie=4cd4d0e01c685628 in_seq=0 out_seq=0 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cd9600640 10 client.5652 ms_handle_connect on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cd9600640 1 -- 172.21.15.97:0/170217982 <== mds.0 v2:172.21.15.97:6832/3721865460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c9715ca70 con 0x563c973387f0 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cd9600640 10 client.5652 handle_command_reply: tid=0 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 client.5652 shutdown 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734d6e0 msgr2=0x563c97337de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734d6e0 0x563c97337de0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4ce801d4c0 tx=0x7f4ce8003a90 comp rx=0 tx=0).stop 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 shutdown_connections 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c973387f0 0x563c9735e070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734d6e0 0x563c97337de0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cd9600640 0 client.5652 ms_handle_reset on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/170217982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c97301220 0x563c97337360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 >> 172.21.15.97:0/170217982 conn(0x563c971e6130 msgr2=0x563c972fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 shutdown_connections 2024-09-10T21:26:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/170217982 wait complete. 2024-09-10T21:26:25.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.959+0000 7f4cf9f9e000 20 client.5652 trim_cache size 0 max 16384 2024-09-10T21:26:25.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 Processor -- start 2024-09-10T21:26:25.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- start start 2024-09-10T21:26:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c97301220 0x563c97309a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734cdc0 0x563c97309fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c9735e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563c9714dd60 con 0x563c9734cdc0 2024-09-10T21:26:25.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c970f0500 con 0x563c9734d6e0 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563c971cea60 con 0x563c97301220 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734cdc0 0x563c97309fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734cdc0 0x563c97309fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41752/0 (socket says 172.21.15.97:41752) 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 -- 172.21.15.97:0/1334205228 learned_addr learned my addr 172.21.15.97:0/1334205228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cefe00640 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c97301220 0x563c97309a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c9735e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 -- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c97301220 msgr2=0x563c97309a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c97301220 0x563c97309a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 -- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 msgr2=0x563c9735e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c9735e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 -- 172.21.15.97:0/1334205228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c971a5d90 con 0x563c9734cdc0 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cef400640 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734cdc0 0x563c97309fa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc017000 tx=0x7f4cdc0624f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1018208cffda8e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cdbe00640 1 -- 172.21.15.97:0/1334205228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc094070 con 0x563c9734cdc0 2024-09-10T21:26:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cdbe00640 1 -- 172.21.15.97:0/1334205228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cdc06b2f0 con 0x563c9734cdc0 2024-09-10T21:26:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cdbe00640 1 -- 172.21.15.97:0/1334205228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc07b070 con 0x563c9734cdc0 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1334205228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734cdc0 msgr2=0x563c97309fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734cdc0 0x563c97309fa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc017000 tx=0x7f4cdc0624f0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1334205228 shutdown_connections 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c9735e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c9734cdc0 0x563c97309fa0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/1334205228 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c97301220 0x563c97309a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1334205228 >> 172.21.15.97:0/1334205228 conn(0x563c971e6130 msgr2=0x563c973291b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1334205228 shutdown_connections 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/1334205228 wait complete. 2024-09-10T21:26:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.979+0000 7f4cf9f9e000 1 -- start start 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 Processor -- start 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c973c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c973c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563c97209730 con 0x563c97301220 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c9714d070 con 0x563c9734d6e0 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563c97115ec0 con 0x563c9734cdc0 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 10 client.0 _ll_register_callbacks cb 0x7f4ce0005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c973c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c973c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41766/0 (socket says 172.21.15.97:41766) 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 -- 172.21.15.97:0/857299444 learned_addr learned my addr 172.21.15.97:0/857299444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c973c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cef400640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 -- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 msgr2=0x563c973c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 -- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 msgr2=0x563c973c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c973c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 -- 172.21.15.97:0/857299444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c97110a90 con 0x563c97301220 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf4c00640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c973c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cefe00640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c973c2de0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563c97150a30 tx=0x7f4ce0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64887fc3e8d7a860 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 1 -- 172.21.15.97:0/857299444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0035060 con 0x563c97301220 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 1 -- 172.21.15.97:0/857299444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4ce001cc90 con 0x563c97301220 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cdc06b2f0 con 0x563c97301220 2024-09-10T21:26:25.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 1 -- 172.21.15.97:0/857299444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4ce003b050 con 0x563c97301220 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 10 client.5658 fetch_fsmap learned FSMap version 33 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x563c970ef050 con 0x563c97301220 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 1 -- 172.21.15.97:0/857299444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cdc06b2f0 con 0x563c97301220 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 1 -- 172.21.15.97:0/857299444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7f4ce003e020 con 0x563c97301220 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 10 client.5658 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 10 client.5658 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 20 client.5658 populate_metadata read hostname 'smithi097' 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c97361e80 0x563c97360fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 4 client.5658 mds_command: new command op to 5484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 --> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] -- command(tid 0: {"prefix": "status"}) -- 0x563c97158520 con 0x563c97361e80 2024-09-10T21:26:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cef400640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c97361e80 0x563c97360fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cef400640 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c97361e80 0x563c97360fa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9faea76408273fe4 server_cookie=6858c43b9614defa in_seq=0 out_seq=0 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 10 client.5658 ms_handle_connect on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 1 -- 172.21.15.97:0/857299444 <== mds.0 v2:172.21.15.97:6832/3721865460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x563c97158520 con 0x563c97361e80 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 10 client.5658 handle_command_reply: tid=0 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 client.5658 shutdown 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 msgr2=0x563c973c2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c973c2de0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563c97150a30 tx=0x7f4ce0015040 comp rx=0 tx=0).stop 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 shutdown_connections 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x563c97361e80 0x563c97360fa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c9734d6e0 0x563c973c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563c9734cdc0 0x563c973c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cd9600640 0 client.5658 ms_handle_reset on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:25.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 --2- 172.21.15.97:0/857299444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c97301220 0x563c973c2de0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.983+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 >> 172.21.15.97:0/857299444 conn(0x563c971e6130 msgr2=0x563c9730a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 shutdown_connections 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf9f9e000 1 -- 172.21.15.97:0/857299444 wait complete. 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf9f9e000 20 client.5658 trim_cache size 0 max 16384 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5484, 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:26:25.993 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:26:25.994 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:26:25.994 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 10.479580925, 2024-09-10T21:26:25.994 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 33, 2024-09-10T21:26:25.994 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 54, 2024-09-10T21:26:25.994 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 54, 2024-09-10T21:26:25.994 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 14.994600898 2024-09-10T21:26:25.994 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4cd0072610 msgr2=0x7f4cd0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4cd0072610 0x7f4cd0074ad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0073490 tx=0x7f4ce00091e0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf0155c50 msgr2=0x7f4cf0070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf0155c50 0x7f4cf0070960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0126c0 tx=0x7f4cdc012b80 comp rx=0 tx=0).stop 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 shutdown_connections 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4cd0072610 0x7f4cd0074ad0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4cf0070ea0 0x7f4cf00ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf0155c50 0x7f4cf0070960 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 --2- 172.21.15.97:0/2364531219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cd40051d0 0x7f4cf01553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 >> 172.21.15.97:0/2364531219 conn(0x7f4cf00d1a60 msgr2=0x7f4cf015c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:25.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 shutdown_connections 2024-09-10T21:26:25.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:25.987+0000 7f4cf5600640 1 -- 172.21.15.97:0/2364531219 wait complete. 2024-09-10T21:26:26.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:26:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.147+0000 7f0cdae00640 1 -- 172.21.15.97:0/1519518981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 msgr2=0x7f0cd40cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.147+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1519518981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40cc780 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009900 tx=0x7f0cc401c850 comp rx=0 tx=0).stop 2024-09-10T21:26:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.147+0000 7f0cdae00640 1 -- 172.21.15.97:0/1519518981 shutdown_connections 2024-09-10T21:26:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.147+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1519518981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40cc780 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.147+0000 7f0cdae00640 1 -- 172.21.15.97:0/1519518981 >> 172.21.15.97:0/1519518981 conn(0x7f0cd40d1c70 msgr2=0x7f0cd40d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.147+0000 7f0cdae00640 1 -- 172.21.15.97:0/1519518981 shutdown_connections 2024-09-10T21:26:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.147+0000 7f0cdae00640 1 -- 172.21.15.97:0/1519518981 wait complete. 2024-09-10T21:26:26.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 Processor -- start 2024-09-10T21:26:26.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 -- start start 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd40cd3d0 0x7f0cd40c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd40d2840 0x7f0cd40c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4169dd0 con 0x7f0cd415bf70 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd4169c50 con 0x7f0cd40d2840 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd4169f50 con 0x7f0cd40cd3d0 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41792/0 (socket says 172.21.15.97:41792) 2024-09-10T21:26:26.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 -- 172.21.15.97:0/1998741032 learned_addr learned my addr 172.21.15.97:0/1998741032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0ccb400640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd40d2840 0x7f0cd40c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd3e00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd40cd3d0 0x7f0cd40c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 -- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd40cd3d0 msgr2=0x7f0cd40c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd40cd3d0 0x7f0cd40c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 -- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd40d2840 msgr2=0x7f0cd40c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd40d2840 0x7f0cd40c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 -- 172.21.15.97:0/1998741032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd406ac50 con 0x7f0cd415bf70 2024-09-10T21:26:26.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd8e00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40c7f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00c7d0 tx=0x7f0ccc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:26.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd0c00640 1 -- 172.21.15.97:0/1998741032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc003b80 con 0x7f0cd415bf70 2024-09-10T21:26:26.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd0c00640 1 -- 172.21.15.97:0/1998741032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ccc003d40 con 0x7f0cd415bf70 2024-09-10T21:26:26.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd415a010 con 0x7f0cd415bf70 2024-09-10T21:26:26.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cd0c00640 1 -- 172.21.15.97:0/1998741032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00aab0 con 0x7f0cd415bf70 2024-09-10T21:26:26.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd40c5a60 con 0x7f0cd415bf70 2024-09-10T21:26:26.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.151+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd40b1c60 con 0x7f0cd415bf70 2024-09-10T21:26:26.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.155+0000 7f0cd0c00640 1 -- 172.21.15.97:0/1998741032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0ccc00ac50 con 0x7f0cd415bf70 2024-09-10T21:26:26.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.155+0000 7f0cd0c00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cb00725e0 0x7f0cb0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.155+0000 7f0cd0c00640 1 -- 172.21.15.97:0/1998741032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0ccc093110 con 0x7f0cd415bf70 2024-09-10T21:26:26.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.155+0000 7f0cd3e00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cb00725e0 0x7f0cb0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:26.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.155+0000 7f0cd3e00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cb00725e0 0x7f0cb0074aa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40af000 tx=0x7f0cc401b090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:26.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.159+0000 7f0cd0c00640 1 -- 172.21.15.97:0/1998741032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0ccc05e010 con 0x7f0cd415bf70 2024-09-10T21:26:26.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.359+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0cd406f070 con 0x7f0cd415bf70 2024-09-10T21:26:26.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.359+0000 7f0cd0c00640 1 -- 172.21.15.97:0/1998741032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f0ccc062a40 con 0x7f0cd415bf70 2024-09-10T21:26:26.364 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:26:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cb00725e0 msgr2=0x7f0cb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cb00725e0 0x7f0cb0074aa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40af000 tx=0x7f0cc401b090 comp rx=0 tx=0).stop 2024-09-10T21:26:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 msgr2=0x7f0cd40c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40c7f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00c7d0 tx=0x7f0ccc00cc90 comp rx=0 tx=0).stop 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 shutdown_connections 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cb00725e0 0x7f0cb0074aa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd415bf70 0x7f0cd40c7f40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd40d2840 0x7f0cd40c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 --2- 172.21.15.97:0/1998741032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd40cd3d0 0x7f0cd40c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 >> 172.21.15.97:0/1998741032 conn(0x7f0cd40d1c70 msgr2=0x7f0cd4071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 shutdown_connections 2024-09-10T21:26:26.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26.363+0000 7f0cdae00640 1 -- 172.21.15.97:0/1998741032 wait complete. 2024-09-10T21:26:26.385 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:26:26.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:26:26.385 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:26:26.385 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:26:26.385 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:26:26.386 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:26:26.386 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:26:26.386 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:26:26.386 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:26:26.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:26.386 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:26:26.395 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:26:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:26:26.395 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:26:26.396 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42307sec preferred_lft 42307sec 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:26:26.397 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:26:26.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:26.399 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:26:26.457 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:26:26.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:26.460 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:26:26.515 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:26:26.516 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:26:26.516 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:26:26.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:26:26.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:26:26.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:26:26.516 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:26:26.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:26:26.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:26:26.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:26.650 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:26:26.658 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:26:26.658 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:26:26.660 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:26:26.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:26.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:26:26.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:26:26.722 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:26:26.776 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:26:26.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:26:26.776 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:26:26.828 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:26:26.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:26.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:26:26.920 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:26:26.920 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:26:26.921 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:26:26.935 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:26:26.935 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:26:26.936 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:26:26.936 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:26:26.936 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:26:26.945 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:26:26.945 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:26:26.945 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:26:26.945 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:26:26.946 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:26:26.946 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:26:26.946 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:26:26.946 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:26:26.949 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:26:26.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:26.949 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:26:26.969 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:26:27.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:26:27.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:26:27.073 INFO:teuthology.orchestra.run.smithi097.stdout:client5237 2024-09-10T21:26:27.074 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:27.074 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:27.082 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/b') 2024-09-10T21:26:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:26:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:26:27.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-10T21:26:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.327+0000 7f6440400640 1 -- 172.21.15.97:0/4122874799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380ce640 msgr2=0x7f64380cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.327+0000 7f6440400640 1 --2- 172.21.15.97:0/4122874799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380ce640 0x7f64380cea20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f642c009900 tx=0x7f642c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.327+0000 7f6440400640 1 -- 172.21.15.97:0/4122874799 shutdown_connections 2024-09-10T21:26:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.327+0000 7f6440400640 1 --2- 172.21.15.97:0/4122874799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380ce640 0x7f64380cea20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.327+0000 7f6440400640 1 -- 172.21.15.97:0/4122874799 >> 172.21.15.97:0/4122874799 conn(0x7f64380d1c30 msgr2=0x7f64380d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.327+0000 7f6440400640 1 -- 172.21.15.97:0/4122874799 shutdown_connections 2024-09-10T21:26:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.327+0000 7f6440400640 1 -- 172.21.15.97:0/4122874799 wait complete. 2024-09-10T21:26:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 Processor -- start 2024-09-10T21:26:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 -- start start 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b0d50 0x7f643806e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64380cd6e0 0x7f64380c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64380ce640 0x7f64380c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f643816a8a0 con 0x7f64380b0d50 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f643816a720 con 0x7f64380cd6e0 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f643816aa20 con 0x7f64380ce640 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b0d50 0x7f643806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b0d50 0x7f643806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52024/0 (socket says 172.21.15.97:52024) 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 -- 172.21.15.97:0/1671800354 learned_addr learned my addr 172.21.15.97:0/1671800354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643d000640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64380cd6e0 0x7f64380c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643e400640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64380ce640 0x7f64380c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 -- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64380ce640 msgr2=0x7f64380c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64380ce640 0x7f64380c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 -- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64380cd6e0 msgr2=0x7f64380c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64380cd6e0 0x7f64380c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 -- 172.21.15.97:0/1671800354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6438154a50 con 0x7f64380b0d50 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f643da00640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b0d50 0x7f643806e940 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f642c009850 tx=0x7f642c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6435600640 1 -- 172.21.15.97:0/1671800354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b2040 con 0x7f64380b0d50 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6435600640 1 -- 172.21.15.97:0/1671800354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f642c0036f0 con 0x7f64380b0d50 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643815c350 con 0x7f64380b0d50 2024-09-10T21:26:27.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6435600640 1 -- 172.21.15.97:0/1671800354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c003b00 con 0x7f64380b0d50 2024-09-10T21:26:27.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.331+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643815c610 con 0x7f64380b0d50 2024-09-10T21:26:27.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6435600640 1 -- 172.21.15.97:0/1671800354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f642c003ca0 con 0x7f64380b0d50 2024-09-10T21:26:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6435600640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6418072610 0x7f6418074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6435600640 1 -- 172.21.15.97:0/1671800354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f642c12cba0 con 0x7f64380b0d50 2024-09-10T21:26:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 Processor -- start 2024-09-10T21:26:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 -- start start 2024-09-10T21:26:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f643d000640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6418072610 0x7f6418074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f643d000640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6418072610 0x7f6418074ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f64380b3010 tx=0x7f6424009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f8761a0 0x55667f876580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f82c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f82cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55667f714cb0 con 0x55667f8761a0 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55667f62a310 con 0x55667f82cbc0 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55667f687b90 con 0x55667f876ac0 2024-09-10T21:26:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f643da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f8761a0 0x55667f876580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f643da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f8761a0 0x55667f876580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52038/0 (socket says 172.21.15.97:52038) 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f643da00640 1 -- 172.21.15.97:0/1453707282 learned_addr learned my addr 172.21.15.97:0/1453707282 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f643d000640 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f82c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.335+0000 7f643e400640 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f82cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 -- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 msgr2=0x55667f82c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f82c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 -- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 msgr2=0x55667f82cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f82cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 -- 172.21.15.97:0/1453707282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55667f62bc60 con 0x55667f8761a0 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643e400640 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f82cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f8761a0 0x55667f876580 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f642c0b0040 tx=0x7f642c0f39e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3d2bcf2d1d49db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f642a000640 1 -- 172.21.15.97:0/1453707282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0bd070 con 0x55667f8761a0 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f642a000640 1 -- 172.21.15.97:0/1453707282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f642c0f8110 con 0x55667f8761a0 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f642a000640 1 -- 172.21.15.97:0/1453707282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0f7a10 con 0x55667f8761a0 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- 172.21.15.97:0/1453707282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f8761a0 msgr2=0x55667f876580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f8761a0 0x55667f876580 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f642c0b0040 tx=0x7f642c0f39e0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- 172.21.15.97:0/1453707282 shutdown_connections 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f82cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f82c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 --2- 172.21.15.97:0/1453707282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f8761a0 0x55667f876580 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- 172.21.15.97:0/1453707282 >> 172.21.15.97:0/1453707282 conn(0x55667f832d20 msgr2=0x55667f82af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- 172.21.15.97:0/1453707282 shutdown_connections 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- 172.21.15.97:0/1453707282 wait complete. 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- start start 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 Processor -- start 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f82cbc0 0x55667f87d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 0x55667f87d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f87dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55667f67add0 con 0x55667f82cbc0 2024-09-10T21:26:27.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55667f676fc0 con 0x55667f8761a0 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55667f61c4b0 con 0x55667f876ac0 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f6443735000 10 client.0 _ll_register_callbacks cb 0x55667f5fa7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f82cbc0 0x55667f87d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f82cbc0 0x55667f87d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52050/0 (socket says 172.21.15.97:52050) 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 -- 172.21.15.97:0/3815334610 learned_addr learned my addr 172.21.15.97:0/3815334610 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643e400640 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f87dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643d000640 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 0x55667f87d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 -- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 msgr2=0x55667f87dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f87dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 -- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 msgr2=0x55667f87d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 0x55667f87d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.339+0000 7f643da00640 1 -- 172.21.15.97:0/3815334610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55667f6151f0 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f643da00640 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f82cbc0 0x55667f87d320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f642c009900 tx=0x7f642c0f8110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9419522ec462320b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 1 -- 172.21.15.97:0/3815334610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0bd040 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 1 -- 172.21.15.97:0/3815334610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f642c100c80 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55667f60f330 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 1 -- 172.21.15.97:0/3815334610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f642c100f10 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 10 client.5682 fetch_fsmap learned FSMap version 33 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55667f5fbfa0 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 1 -- 172.21.15.97:0/3815334610 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55667f60f330 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 1 -- 172.21.15.97:0/3815334610 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7f642c0f6c70 con 0x55667f82cbc0 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 10 client.5682 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 10 client.5682 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T21:26:27.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 20 client.5682 populate_metadata read hostname 'smithi097' 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8856f0 0x55667f887ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 4 client.5682 mds_command: new command op to 5484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 --> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55667f689ae0 con 0x55667f8856f0 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f643d000640 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8856f0 0x55667f887ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 10 client.5682 ms_handle_connect on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f643d000640 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8856f0 0x55667f887ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdeb63e2da2bdf36 server_cookie=d2992dafc2abad63 in_seq=0 out_seq=0 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 1 -- 172.21.15.97:0/3815334610 <== mds.0 v2:172.21.15.97:6832/3721865460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55667f689ae0 con 0x55667f8856f0 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 10 client.5682 handle_command_reply: tid=0 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 client.5682 shutdown 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f82cbc0 msgr2=0x55667f87d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f82cbc0 0x55667f87d320 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f642c009900 tx=0x7f642c0f8110 comp rx=0 tx=0).stop 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 shutdown_connections 2024-09-10T21:26:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8856f0 0x55667f887ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f876ac0 0x55667f87dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 0x55667f87d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 --2- 172.21.15.97:0/3815334610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f82cbc0 0x55667f87d320 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f641ea00640 0 client.5682 ms_handle_reset on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 >> 172.21.15.97:0/3815334610 conn(0x55667f832d20 msgr2=0x55667f82af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 shutdown_connections 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 1 -- 172.21.15.97:0/3815334610 wait complete. 2024-09-10T21:26:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.343+0000 7f6443735000 20 client.5682 trim_cache size 0 max 16384 2024-09-10T21:26:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 Processor -- start 2024-09-10T21:26:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 -- start start 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f82cbc0 0x55667f8e7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 0x55667f8e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8e8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6438204bc0 con 0x55667f876ac0 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6438203c90 con 0x55667f8761a0 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6438203020 con 0x55667f82cbc0 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8e8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52058/0 (socket says 172.21.15.97:52058) 2024-09-10T21:26:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 -- 172.21.15.97:0/2011830575 learned_addr learned my addr 172.21.15.97:0/2011830575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 -- 172.21.15.97:0/2011830575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f82cbc0 msgr2=0x55667f8e7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 --2- 172.21.15.97:0/2011830575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f82cbc0 0x55667f8e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 -- 172.21.15.97:0/2011830575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 msgr2=0x55667f8e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 --2- 172.21.15.97:0/2011830575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 0x55667f8e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 -- 172.21.15.97:0/2011830575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55667f6d4660 con 0x55667f876ac0 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.359+0000 7f643e400640 1 --2- 172.21.15.97:0/2011830575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8e8290 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f64300053b0 tx=0x7f6430005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6828f43410cee61e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6429600640 1 -- 172.21.15.97:0/2011830575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643001a070 con 0x55667f876ac0 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6429600640 1 -- 172.21.15.97:0/2011830575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6430006b60 con 0x55667f876ac0 2024-09-10T21:26:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6429600640 1 -- 172.21.15.97:0/2011830575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300128e0 con 0x55667f876ac0 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- 172.21.15.97:0/2011830575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 msgr2=0x55667f8e8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 --2- 172.21.15.97:0/2011830575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8e8290 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f64300053b0 tx=0x7f6430005870 comp rx=0 tx=0).stop 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- 172.21.15.97:0/2011830575 shutdown_connections 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 --2- 172.21.15.97:0/2011830575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8e8290 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 --2- 172.21.15.97:0/2011830575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f8761a0 0x55667f8e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 --2- 172.21.15.97:0/2011830575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f82cbc0 0x55667f8e7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- 172.21.15.97:0/2011830575 >> 172.21.15.97:0/2011830575 conn(0x55667f832d20 msgr2=0x55667f82e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- 172.21.15.97:0/2011830575 shutdown_connections 2024-09-10T21:26:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- 172.21.15.97:0/2011830575 wait complete. 2024-09-10T21:26:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- start start 2024-09-10T21:26:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 Processor -- start 2024-09-10T21:26:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f88c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f8761a0 0x55667f88c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55667f676fc0 con 0x55667f876ac0 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55667f835d40 con 0x55667f82cbc0 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55667f833100 con 0x55667f8761a0 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f6443735000 10 client.0 _ll_register_callbacks cb 0x7f6424001350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f88c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52066/0 (socket says 172.21.15.97:52066) 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 -- 172.21.15.97:0/3201069476 learned_addr learned my addr 172.21.15.97:0/3201069476 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643d000640 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f8761a0 0x55667f88c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 -- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f8761a0 msgr2=0x55667f88c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f8761a0 0x55667f88c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 -- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 msgr2=0x55667f88c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f88c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 -- 172.21.15.97:0/3201069476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55667f641090 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643da00640 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f88c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f643e400640 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8a8000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55667f80cf50 tx=0x7f6430006db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860abcefbdcb80c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f641ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.363+0000 7f641ea00640 1 -- 172.21.15.97:0/3201069476 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643001a070 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f641ea00640 1 -- 172.21.15.97:0/3201069476 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f6430007430 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6430006b60 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f641ea00640 1 -- 172.21.15.97:0/3201069476 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6430007630 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 10 client.5688 fetch_fsmap learned FSMap version 33 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55667f60d5f0 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f641ea00640 1 -- 172.21.15.97:0/3201069476 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6430006b60 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f641ea00640 1 -- 172.21.15.97:0/3201069476 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 33) ==== 3502+0+0 (secure 0 0 0) 0x7f6430033080 con 0x55667f876ac0 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 10 client.5688 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:26:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 10 client.5688 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T21:26:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 20 client.5688 populate_metadata read hostname 'smithi097' 2024-09-10T21:26:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8a89f0 0x55667f8f3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 4 client.5688 mds_command: new command op to 5484 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:26:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 --> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55667f685590 con 0x55667f8a89f0 2024-09-10T21:26:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f643da00640 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8a89f0 0x55667f8f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f641ea00640 10 client.5688 ms_handle_connect on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.367+0000 7f643da00640 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8a89f0 0x55667f8f3470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1763b0c0c644346f server_cookie=5a1170157566c9b3 in_seq=0 out_seq=0 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f641ea00640 1 -- 172.21.15.97:0/3201069476 <== mds.0 v2:172.21.15.97:6832/3721865460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55667f685590 con 0x55667f8a89f0 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f641ea00640 10 client.5688 handle_command_reply: tid=0 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 client.5688 shutdown 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 msgr2=0x55667f8a8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8a8000 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55667f80cf50 tx=0x7f6430006db0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 shutdown_connections 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:6832/3721865460,v1:172.21.15.97:6834/3721865460] conn(0x55667f8a89f0 0x55667f8f3470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55667f876ac0 0x55667f8a8000 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55667f8761a0 0x55667f88c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 --2- 172.21.15.97:0/3201069476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55667f82cbc0 0x55667f88c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f641ea00640 0 client.5688 ms_handle_reset on v2:172.21.15.97:6832/3721865460 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 >> 172.21.15.97:0/3201069476 conn(0x55667f832d20 msgr2=0x55667f887ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 shutdown_connections 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 1 -- 172.21.15.97:0/3201069476 wait complete. 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.379+0000 7f6443735000 20 client.5688 trim_cache size 0 max 16384 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "message": "", 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-09-10T21:26:27.385 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:26:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6418072610 msgr2=0x7f6418074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6418072610 0x7f6418074ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f64380b3010 tx=0x7f6424009040 comp rx=0 tx=0).stop 2024-09-10T21:26:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b0d50 msgr2=0x7f643806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b0d50 0x7f643806e940 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f642c009850 tx=0x7f642c004220 comp rx=0 tx=0).stop 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 shutdown_connections 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6418072610 0x7f6418074ad0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f64380ce640 0x7f64380c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64380cd6e0 0x7f64380c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 --2- 172.21.15.97:0/1671800354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64380b0d50 0x7f643806e940 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 >> 172.21.15.97:0/1671800354 conn(0x7f64380d1c30 msgr2=0x7f64380b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 shutdown_connections 2024-09-10T21:26:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.383+0000 7f6440400640 1 -- 172.21.15.97:0/1671800354 wait complete. 2024-09-10T21:26:27.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-10T21:26:27.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- 172.21.15.97:0/3120132471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44e0004810 msgr2=0x7f44e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 --2- 172.21.15.97:0/3120132471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44e0004810 0x7f44e0004c90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005fd0 tx=0x7f44e401cd60 comp rx=0 tx=0).stop 2024-09-10T21:26:27.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4501600640 1 -- 172.21.15.97:0/3120132471 reap_dead start 2024-09-10T21:26:27.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- 172.21.15.97:0/3120132471 shutdown_connections 2024-09-10T21:26:27.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- 172.21.15.97:0/3120132471 >> 172.21.15.97:0/3120132471 conn(0x7f44fc0b37c0 msgr2=0x7f44fc0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- 172.21.15.97:0/3120132471 shutdown_connections 2024-09-10T21:26:27.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- 172.21.15.97:0/3120132471 wait complete. 2024-09-10T21:26:27.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 Processor -- start 2024-09-10T21:26:27.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- start start 2024-09-10T21:26:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44e0004810 0x7f44fc0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44fc1f8910 0x7f44fc15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44fc0b7510 0x7f44fc0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0b4450 con 0x7f44fc1f8910 2024-09-10T21:26:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc0cde60 con 0x7f44e0004810 2024-09-10T21:26:27.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4503600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f44fc0ce1a0 con 0x7f44fc0b7510 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44fc1f8910 0x7f44fc15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44fc1f8910 0x7f44fc15b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52070/0 (socket says 172.21.15.97:52070) 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 -- 172.21.15.97:0/3714033514 learned_addr learned my addr 172.21.15.97:0/3714033514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4500c00640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44e0004810 0x7f44fc0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f4501600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44fc0b7510 0x7f44fc0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 -- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44fc0b7510 msgr2=0x7f44fc0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44fc0b7510 0x7f44fc0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 -- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44e0004810 msgr2=0x7f44fc0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44e0004810 0x7f44fc0c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 -- 172.21.15.97:0/3714033514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc15bde0 con 0x7f44fc1f8910 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.547+0000 7f44f3e00640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44fc1f8910 0x7f44fc15b8a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f44e401c7d0 tx=0x7f44e4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f44f9600640 1 -- 172.21.15.97:0/3714033514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40bc070 con 0x7f44fc1f8910 2024-09-10T21:26:27.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f44f9600640 1 -- 172.21.15.97:0/3714033514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44e4005330 con 0x7f44fc1f8910 2024-09-10T21:26:27.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f44f9600640 1 -- 172.21.15.97:0/3714033514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40ab070 con 0x7f44fc1f8910 2024-09-10T21:26:27.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc0c9b60 con 0x7f44fc1f8910 2024-09-10T21:26:27.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc1f7e30 con 0x7f44fc1f8910 2024-09-10T21:26:27.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc0b2730 con 0x7f44fc1f8910 2024-09-10T21:26:27.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f44f9600640 1 -- 172.21.15.97:0/3714033514 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f44e40046d0 con 0x7f44fc1f8910 2024-09-10T21:26:27.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f44f9600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44d8072610 0x7f44d8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f44f9600640 1 -- 172.21.15.97:0/3714033514 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f44e412d0c0 con 0x7f44fc1f8910 2024-09-10T21:26:27.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.551+0000 7f4500c00640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44d8072610 0x7f44d8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.555+0000 7f4500c00640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44d8072610 0x7f44d8074ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44fc1f2390 tx=0x7f44ec00d410 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.555+0000 7f44f9600640 1 -- 172.21.15.97:0/3714033514 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44e40f2de0 con 0x7f44fc1f8910 2024-09-10T21:26:27.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.723+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7f44fc0714b0 con 0x7f44fc1f8910 2024-09-10T21:26:27.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.727+0000 7f44f9600640 1 -- 172.21.15.97:0/3714033514 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v31)=0 v31) ==== 135+0+0 (secure 0 0 0) 0x7f44e4016a70 con 0x7f44fc1f8910 2024-09-10T21:26:27.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44d8072610 msgr2=0x7f44d8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44d8072610 0x7f44d8074ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44fc1f2390 tx=0x7f44ec00d410 comp rx=0 tx=0).stop 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44fc1f8910 msgr2=0x7f44fc15b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44fc1f8910 0x7f44fc15b8a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f44e401c7d0 tx=0x7f44e4004620 comp rx=0 tx=0).stop 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 shutdown_connections 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44d8072610 0x7f44d8074ad0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44fc0b7510 0x7f44fc0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44fc1f8910 0x7f44fc15b8a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 --2- 172.21.15.97:0/3714033514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44e0004810 0x7f44fc0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 >> 172.21.15.97:0/3714033514 conn(0x7f44fc0b37c0 msgr2=0x7f44fc06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 shutdown_connections 2024-09-10T21:26:27.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.731+0000 7f4503600640 1 -- 172.21.15.97:0/3714033514 wait complete. 2024-09-10T21:26:27.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-10T21:26:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- 172.21.15.97:0/1078284834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3e8003fe0 msgr2=0x7fe3e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 --2- 172.21.15.97:0/1078284834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3e8003fe0 0x7fe3e8004810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe40415f720 tx=0x7fe3f001d1d0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe403400640 1 -- 172.21.15.97:0/1078284834 reap_dead start 2024-09-10T21:26:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- 172.21.15.97:0/1078284834 shutdown_connections 2024-09-10T21:26:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- 172.21.15.97:0/1078284834 >> 172.21.15.97:0/1078284834 conn(0x7fe4040cd680 msgr2=0x7fe4040cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- 172.21.15.97:0/1078284834 shutdown_connections 2024-09-10T21:26:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- 172.21.15.97:0/1078284834 wait complete. 2024-09-10T21:26:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 Processor -- start 2024-09-10T21:26:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- start start 2024-09-10T21:26:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3e8003fe0 0x7fe40415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe404149340 0x7fe4041599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4041497a0 0x7fe404159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4040b4d90 con 0x7fe4041497a0 2024-09-10T21:26:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4040ce3b0 con 0x7fe3e8003fe0 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.895+0000 7fe409800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe4040b4f10 con 0x7fe404149340 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4041497a0 0x7fe404159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4041497a0 0x7fe404159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52086/0 (socket says 172.21.15.97:52086) 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 -- 172.21.15.97:0/168594845 learned_addr learned my addr 172.21.15.97:0/168594845 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe402000640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe404149340 0x7fe4041599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe402a00640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3e8003fe0 0x7fe40415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 -- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe404149340 msgr2=0x7fe4041599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe404149340 0x7fe4041599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 -- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3e8003fe0 msgr2=0x7fe40415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3e8003fe0 0x7fe40415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 -- 172.21.15.97:0/168594845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40415d570 con 0x7fe4041497a0 2024-09-10T21:26:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe403400640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4041497a0 0x7fe404159ee0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe40415c170 tx=0x7fe3fc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe3faa00640 1 -- 172.21.15.97:0/168594845 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc01e070 con 0x7fe4041497a0 2024-09-10T21:26:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe3faa00640 1 -- 172.21.15.97:0/168594845 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3fc00fcc0 con 0x7fe4041497a0 2024-09-10T21:26:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40415d850 con 0x7fe4041497a0 2024-09-10T21:26:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe3faa00640 1 -- 172.21.15.97:0/168594845 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc019690 con 0x7fe4041497a0 2024-09-10T21:26:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40415de10 con 0x7fe4041497a0 2024-09-10T21:26:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4040b52b0 con 0x7fe4041497a0 2024-09-10T21:26:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.899+0000 7fe3faa00640 1 -- 172.21.15.97:0/168594845 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe3fc0223b0 con 0x7fe4041497a0 2024-09-10T21:26:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.903+0000 7fe3faa00640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe3e40725e0 0x7fe3e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.903+0000 7fe3faa00640 1 -- 172.21.15.97:0/168594845 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe3fc0940a0 con 0x7fe4041497a0 2024-09-10T21:26:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.903+0000 7fe402a00640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe3e40725e0 0x7fe3e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.903+0000 7fe402a00640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe3e40725e0 0x7fe3e4074aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4006fd0 tx=0x7fe3f4001f20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:27.903+0000 7fe3faa00640 1 -- 172.21.15.97:0/168594845 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe3fc05ef00 con 0x7fe4041497a0 2024-09-10T21:26:28.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.067+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7fe4040aeb10 con 0x7fe4041497a0 2024-09-10T21:26:28.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.071+0000 7fe3faa00640 1 -- 172.21.15.97:0/168594845 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7fe3fc063930 con 0x7fe4041497a0 2024-09-10T21:26:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe3e40725e0 msgr2=0x7fe3e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe3e40725e0 0x7fe3e4074aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4006fd0 tx=0x7fe3f4001f20 comp rx=0 tx=0).stop 2024-09-10T21:26:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4041497a0 msgr2=0x7fe404159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4041497a0 0x7fe404159ee0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe40415c170 tx=0x7fe3fc011c40 comp rx=0 tx=0).stop 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 shutdown_connections 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe3e40725e0 0x7fe3e4074aa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4041497a0 0x7fe404159ee0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe404149340 0x7fe4041599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 --2- 172.21.15.97:0/168594845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3e8003fe0 0x7fe40415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 >> 172.21.15.97:0/168594845 conn(0x7fe4040cd680 msgr2=0x7fe4040af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 shutdown_connections 2024-09-10T21:26:28.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:28.075+0000 7fe409800640 1 -- 172.21.15.97:0/168594845 wait complete. 2024-09-10T21:26:33.098 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mv a/b a/c; sync .') 2024-09-10T21:26:33.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:26:33.157 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:26:33.155+0000 7f3dc4600640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5237:44 nref=7 cr=0x561a87d5bc00)) (dversion lock w=1 last_client=5237) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x561a87ca5680] 2024-09-10T21:26:33.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:26:33.178 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:33.178 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:33.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:26:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.379+0000 7fc42f000640 1 -- 172.21.15.97:0/337839919 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc428141d20 msgr2=0x7fc40c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.379+0000 7fc42f000640 1 --2- 172.21.15.97:0/337839919 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc428141d20 0x7fc40c004810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc410009b60 tx=0x7fc41001d2b0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.379+0000 7fc42d000640 1 -- 172.21.15.97:0/337839919 reap_dead start 2024-09-10T21:26:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.379+0000 7fc42f000640 1 -- 172.21.15.97:0/337839919 shutdown_connections 2024-09-10T21:26:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.379+0000 7fc42f000640 1 -- 172.21.15.97:0/337839919 >> 172.21.15.97:0/337839919 conn(0x7fc4280b1780 msgr2=0x7fc4280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.379+0000 7fc42f000640 1 -- 172.21.15.97:0/337839919 shutdown_connections 2024-09-10T21:26:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.379+0000 7fc42f000640 1 -- 172.21.15.97:0/337839919 wait complete. 2024-09-10T21:26:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 Processor -- start 2024-09-10T21:26:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 -- start start 2024-09-10T21:26:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc428141d20 0x7fc42806ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4280b4030 0x7fc4280af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4280ae1b0 0x7fc42815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc428167b20 con 0x7fc4280b4030 2024-09-10T21:26:33.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc428160080 con 0x7fc428141d20 2024-09-10T21:26:33.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc428167ca0 con 0x7fc4280ae1b0 2024-09-10T21:26:33.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4280b4030 0x7fc4280af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4280b4030 0x7fc4280af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52112/0 (socket says 172.21.15.97:52112) 2024-09-10T21:26:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 -- 172.21.15.97:0/3343850385 learned_addr learned my addr 172.21.15.97:0/3343850385 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42d000640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4280ae1b0 0x7fc42815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc427e00640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc428141d20 0x7fc42806ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 -- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4280ae1b0 msgr2=0x7fc42815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4280ae1b0 0x7fc42815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 -- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc428141d20 msgr2=0x7fc42806ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc428141d20 0x7fc42806ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 -- 172.21.15.97:0/3343850385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4280b44f0 con 0x7fc4280b4030 2024-09-10T21:26:33.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc427e00640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc428141d20 0x7fc42806ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:33.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc41f400640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4280b4030 0x7fc4280af000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc41001cd20 tx=0x7fc410005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:33.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc424c00640 1 -- 172.21.15.97:0/3343850385 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100b1040 con 0x7fc4280b4030 2024-09-10T21:26:33.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc424c00640 1 -- 172.21.15.97:0/3343850385 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4100bb070 con 0x7fc4280b4030 2024-09-10T21:26:33.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc424c00640 1 -- 172.21.15.97:0/3343850385 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410004290 con 0x7fc4280b4030 2024-09-10T21:26:33.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4280b47d0 con 0x7fc4280b4030 2024-09-10T21:26:33.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.383+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42806fab0 con 0x7fc4280b4030 2024-09-10T21:26:33.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.387+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42813f9b0 con 0x7fc4280b4030 2024-09-10T21:26:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.387+0000 7fc424c00640 1 -- 172.21.15.97:0/3343850385 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc4100046d0 con 0x7fc4280b4030 2024-09-10T21:26:33.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.387+0000 7fc424c00640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc404072610 0x7fc404074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.387+0000 7fc424c00640 1 -- 172.21.15.97:0/3343850385 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc41012e260 con 0x7fc4280b4030 2024-09-10T21:26:33.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.387+0000 7fc427e00640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc404072610 0x7fc404074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.387+0000 7fc427e00640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc404072610 0x7fc404074ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc418006fd0 tx=0x7fc418006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:33.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.391+0000 7fc424c00640 1 -- 172.21.15.97:0/3343850385 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc410016fc0 con 0x7fc4280b4030 2024-09-10T21:26:33.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.567+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc428155b60 con 0x7fc4280b4030 2024-09-10T21:26:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.567+0000 7fc424c00640 1 -- 172.21.15.97:0/3343850385 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7fc4100f9710 con 0x7fc4280b4030 2024-09-10T21:26:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:26:33.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc404072610 msgr2=0x7fc404074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc404072610 0x7fc404074ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc418006fd0 tx=0x7fc418006b20 comp rx=0 tx=0).stop 2024-09-10T21:26:33.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4280b4030 msgr2=0x7fc4280af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4280b4030 0x7fc4280af000 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc41001cd20 tx=0x7fc410005540 comp rx=0 tx=0).stop 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 shutdown_connections 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc404072610 0x7fc404074ad0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4280ae1b0 0x7fc42815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4280b4030 0x7fc4280af000 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 --2- 172.21.15.97:0/3343850385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc428141d20 0x7fc42806ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 >> 172.21.15.97:0/3343850385 conn(0x7fc4280b1780 msgr2=0x7fc4281415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 shutdown_connections 2024-09-10T21:26:33.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.571+0000 7fc42f000640 1 -- 172.21.15.97:0/3343850385 wait complete. 2024-09-10T21:26:33.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:26:33.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:33.594 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:26:33.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:26:33.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:26:33.623 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:26:33.672 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.855+0000 7f1ecf400640 1 -- 172.21.15.97:0/920822639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eac004d50 msgr2=0x7f1eac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.855+0000 7f1ecf400640 1 --2- 172.21.15.97:0/920822639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eac004d50 0x7f1eac0051d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c610 tx=0x7f1ec001d2a0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.855+0000 7f1ecd400640 1 -- 172.21.15.97:0/920822639 reap_dead start 2024-09-10T21:26:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- 172.21.15.97:0/920822639 shutdown_connections 2024-09-10T21:26:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- 172.21.15.97:0/920822639 >> 172.21.15.97:0/920822639 conn(0x7f1ec80d3e70 msgr2=0x7f1ec80d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- 172.21.15.97:0/920822639 shutdown_connections 2024-09-10T21:26:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- 172.21.15.97:0/920822639 wait complete. 2024-09-10T21:26:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 Processor -- start 2024-09-10T21:26:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- start start 2024-09-10T21:26:33.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1eac004d50 0x7f1ec80c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8157930 0x7f1ec8157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ec80c7de0 0x7f1ec80b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec815f650 con 0x7f1ec8157930 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80d24c0 con 0x7f1ec80c7de0 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecf400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec80d2640 con 0x7f1eac004d50 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8157930 0x7f1ec8157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8157930 0x7f1ec8157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52134/0 (socket says 172.21.15.97:52134) 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 -- 172.21.15.97:0/2823651355 learned_addr learned my addr 172.21.15.97:0/2823651355 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecd400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ec80c7de0 0x7f1ec80b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecca00640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1eac004d50 0x7f1ec80c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 -- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1eac004d50 msgr2=0x7f1ec80c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1eac004d50 0x7f1ec80c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 -- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ec80c7de0 msgr2=0x7f1ec80b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ec80c7de0 0x7f1ec80b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ebfe00640 1 -- 172.21.15.97:0/2823651355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec80c8870 con 0x7f1ec8157930 2024-09-10T21:26:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.859+0000 7f1ecd400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ec80c7de0 0x7f1ec80b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ebfe00640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8157930 0x7f1ec8157070 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0011b10 tx=0x7f1eb0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ec5600640 1 -- 172.21.15.97:0/2823651355 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb001e070 con 0x7f1ec8157930 2024-09-10T21:26:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ec5600640 1 -- 172.21.15.97:0/2823651355 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1eb0004460 con 0x7f1ec8157930 2024-09-10T21:26:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ec5600640 1 -- 172.21.15.97:0/2823651355 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000fc40 con 0x7f1ec8157930 2024-09-10T21:26:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec80d1610 con 0x7f1ec8157930 2024-09-10T21:26:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec80d0ed0 con 0x7f1ec8157930 2024-09-10T21:26:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec80d1f80 con 0x7f1ec8157930 2024-09-10T21:26:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ec5600640 1 -- 172.21.15.97:0/2823651355 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1eb0003600 con 0x7f1ec8157930 2024-09-10T21:26:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ec5600640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ea40725e0 0x7f1ea4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ec5600640 1 -- 172.21.15.97:0/2823651355 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f1eb00940a0 con 0x7f1ec8157930 2024-09-10T21:26:33.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.863+0000 7f1ecca00640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ea40725e0 0x7f1ea4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:33.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.867+0000 7f1ecca00640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ea40725e0 0x7f1ea4074aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009d80 tx=0x7f1eb80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:33.867+0000 7f1ec5600640 1 -- 172.21.15.97:0/2823651355 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1eb005efc0 con 0x7f1ec8157930 2024-09-10T21:26:34.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.067+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ec808db10 con 0x7f1ec8157930 2024-09-10T21:26:34.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.067+0000 7f1ec5600640 1 -- 172.21.15.97:0/2823651355 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5223 (secure 0 0 0) 0x7f1eb00639f0 con 0x7f1ec8157930 2024-09-10T21:26:34.070 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:34.071 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"btime":"2024-09-10T21:26:18:534778+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:17.919032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5484":{"gid":5484,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6834/3721865460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3721865460},{"type":"v1","addr":"172.21.15.97:6834","nonce":3721865460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5484,"qdb_cluster":[5484]},"id":3}]} 2024-09-10T21:26:34.071 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 33 2024-09-10T21:26:34.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ea40725e0 msgr2=0x7f1ea4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ea40725e0 0x7f1ea4074aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009d80 tx=0x7f1eb80091e0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8157930 msgr2=0x7f1ec8157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8157930 0x7f1ec8157070 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0011b10 tx=0x7f1eb0011f10 comp rx=0 tx=0).stop 2024-09-10T21:26:34.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 shutdown_connections 2024-09-10T21:26:34.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ea40725e0 0x7f1ea4074aa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ec80c7de0 0x7f1ec80b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8157930 0x7f1ec8157070 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 --2- 172.21.15.97:0/2823651355 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1eac004d50 0x7f1ec80c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 >> 172.21.15.97:0/2823651355 conn(0x7f1ec80d3e70 msgr2=0x7f1ec80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:34.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 shutdown_connections 2024-09-10T21:26:34.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.071+0000 7f1ecf400640 1 -- 172.21.15.97:0/2823651355 wait complete. 2024-09-10T21:26:34.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:34.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 -- 172.21.15.97:0/2855301523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 msgr2=0x7f38c00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 --2- 172.21.15.97:0/2855301523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c00b2a80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f38c0072040 tx=0x7f38b801cac0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 -- 172.21.15.97:0/2855301523 shutdown_connections 2024-09-10T21:26:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 --2- 172.21.15.97:0/2855301523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c00b2a80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 --2- 172.21.15.97:0/2855301523 >> v2:172.21.15.194:3300/0 conn(0x7f38c00cf0b0 0x7f38c006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 -- 172.21.15.97:0/2855301523 >> 172.21.15.97:0/2855301523 conn(0x7f38c00b1780 msgr2=0x7f38c00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 -- 172.21.15.97:0/2855301523 shutdown_connections 2024-09-10T21:26:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 -- 172.21.15.97:0/2855301523 wait complete. 2024-09-10T21:26:34.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.231+0000 7f38c4e00640 1 Processor -- start 2024-09-10T21:26:34.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 -- start start 2024-09-10T21:26:34.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38c00cf0b0 0x7f38c013d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38c00cf490 0x7f38c01403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c01fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38c00b3b60 con 0x7f38c015b110 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f38c00b39e0 con 0x7f38c00cf490 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f38c0070970 con 0x7f38c00cf0b0 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c01fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c01fa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52152/0 (socket says 172.21.15.97:52152) 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 -- 172.21.15.97:0/4115232033 learned_addr learned my addr 172.21.15.97:0/4115232033 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38bde00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38c00cf0b0 0x7f38c013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38bd400640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38c00cf490 0x7f38c01403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 -- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38c00cf0b0 msgr2=0x7f38c013d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38c00cf0b0 0x7f38c013d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 -- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38c00cf490 msgr2=0x7f38c01403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38c00cf490 0x7f38c01403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 -- 172.21.15.97:0/4115232033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c01fb090 con 0x7f38c015b110 2024-09-10T21:26:34.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38bd400640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38c00cf490 0x7f38c01403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:34.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38be800640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c01fa7b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f38b80095c0 tx=0x7f38b8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:34.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38b6000640 1 -- 172.21.15.97:0/4115232033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b80bb070 con 0x7f38c015b110 2024-09-10T21:26:34.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38b6000640 1 -- 172.21.15.97:0/4115232033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b8004d40 con 0x7f38c015b110 2024-09-10T21:26:34.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c01fb370 con 0x7f38c015b110 2024-09-10T21:26:34.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38b6000640 1 -- 172.21.15.97:0/4115232033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b80b2040 con 0x7f38c015b110 2024-09-10T21:26:34.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c013feb0 con 0x7f38c015b110 2024-09-10T21:26:34.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.235+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c01400b0 con 0x7f38c015b110 2024-09-10T21:26:34.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.239+0000 7f38b6000640 1 -- 172.21.15.97:0/4115232033 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f38b80b5030 con 0x7f38c015b110 2024-09-10T21:26:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.239+0000 7f38b6000640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f38a00725e0 0x7f38a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.239+0000 7f38b6000640 1 -- 172.21.15.97:0/4115232033 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f38b812cf90 con 0x7f38c015b110 2024-09-10T21:26:34.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.239+0000 7f38bde00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f38a00725e0 0x7f38a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.239+0000 7f38bde00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f38a00725e0 0x7f38a0074aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38b000bd00 tx=0x7f38b000b490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.243+0000 7f38b6000640 1 -- 172.21.15.97:0/4115232033 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f38b80166e0 con 0x7f38c015b110 2024-09-10T21:26:34.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.439+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38c0163e70 con 0x7f38c015b110 2024-09-10T21:26:34.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.439+0000 7f38b6000640 1 -- 172.21.15.97:0/4115232033 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5223 (secure 0 0 0) 0x7f38b80f7e80 con 0x7f38c015b110 2024-09-10T21:26:34.443 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:34.443 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"btime":"2024-09-10T21:26:18:534778+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:17.919032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5484":{"gid":5484,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6834/3721865460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3721865460},{"type":"v1","addr":"172.21.15.97:6834","nonce":3721865460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5484,"qdb_cluster":[5484]},"id":3}]} 2024-09-10T21:26:34.443 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 33 2024-09-10T21:26:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f38a00725e0 msgr2=0x7f38a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f38a00725e0 0x7f38a0074aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38b000bd00 tx=0x7f38b000b490 comp rx=0 tx=0).stop 2024-09-10T21:26:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 msgr2=0x7f38c01fa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c01fa7b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f38b80095c0 tx=0x7f38b8005b00 comp rx=0 tx=0).stop 2024-09-10T21:26:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 shutdown_connections 2024-09-10T21:26:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f38a00725e0 0x7f38a0074aa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c015b110 0x7f38c01fa7b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38c00cf490 0x7f38c01403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 --2- 172.21.15.97:0/4115232033 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38c00cf0b0 0x7f38c013d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 >> 172.21.15.97:0/4115232033 conn(0x7f38c00b1780 msgr2=0x7f38c00af940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 shutdown_connections 2024-09-10T21:26:34.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.443+0000 7f38c4e00640 1 -- 172.21.15.97:0/4115232033 wait complete. 2024-09-10T21:26:34.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /a 0 2024-09-10T21:26:34.622 INFO:teuthology.orchestra.run.smithi097.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-10T21:26:33.155212+0000","btime":"2024-09-10T21:26:27.175272+0000","mode":16893,"uid":1000,"gid":1271,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-10T21:26:33.155212+0000","atime":"2024-09-10T21:26:27.175272+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-10T21:26:33.155212+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T21:26:33.155212+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T21:26:33.155212+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"inline_data_version":18446744073709551615,"inline_data_length":0,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-10T21:26:34.623 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-10T21:26:33.155212+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-10T21:26:27.175272+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-10T21:26:27.175272+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-10T21:26:33.155212+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-10T21:26:33.155212+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1271, "inline_data_length": 0, "inline_data_version": 18446744073709551615, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2024-09-10T21:26:33.155212+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-10T21:26:33.155212+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-09-10T21:26:34.623 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-10T21:26:33.155212+0000', 'btime': '2024-09-10T21:26:27.175272+0000', 'mode': 16893, 'uid': 1000, 'gid': 1271, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-10T21:26:33.155212+0000', 'atime': '2024-09-10T21:26:27.175272+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-10T21:26:33.155212+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T21:26:33.155212+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T21:26:33.155212+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'inline_data_version': 18446744073709551615, 'inline_data_length': 0, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-10T21:26:34.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:26:34.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 -- 172.21.15.97:0/1086693184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 msgr2=0x7f40040cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 --2- 172.21.15.97:0/1086693184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 0x7f40040cc6d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009840 tx=0x7f3fec01c820 comp rx=0 tx=0).stop 2024-09-10T21:26:34.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 -- 172.21.15.97:0/1086693184 shutdown_connections 2024-09-10T21:26:34.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 --2- 172.21.15.97:0/1086693184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f40040b0d50 0x7f40040ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 --2- 172.21.15.97:0/1086693184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 0x7f40040cc6d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 -- 172.21.15.97:0/1086693184 >> 172.21.15.97:0/1086693184 conn(0x7f40040d1c30 msgr2=0x7f40040d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:34.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 -- 172.21.15.97:0/1086693184 shutdown_connections 2024-09-10T21:26:34.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.763+0000 7f4009000640 1 -- 172.21.15.97:0/1086693184 wait complete. 2024-09-10T21:26:34.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 Processor -- start 2024-09-10T21:26:34.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 -- start start 2024-09-10T21:26:34.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 0x7f40040c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40040b0d50 0x7f40040ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f40040ce640 0x7f4004148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4004168e90 con 0x7f40040b0d50 2024-09-10T21:26:34.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4004168d10 con 0x7f40040afdd0 2024-09-10T21:26:34.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4004169010 con 0x7f40040ce640 2024-09-10T21:26:34.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40040b0d50 0x7f40040ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40040b0d50 0x7f40040ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52158/0 (socket says 172.21.15.97:52158) 2024-09-10T21:26:34.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 -- 172.21.15.97:0/3533555322 learned_addr learned my addr 172.21.15.97:0/3533555322 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:34.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4001e00640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 0x7f40040c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4002800640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f40040ce640 0x7f4004148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 -- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f40040ce640 msgr2=0x7f4004148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f40040ce640 0x7f4004148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 -- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 msgr2=0x7f40040c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 0x7f40040c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 -- 172.21.15.97:0/3533555322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40040c9600 con 0x7f40040b0d50 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ff9400640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40040b0d50 0x7f40040ca300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009790 tx=0x7f3fec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ffaa00640 1 -- 172.21.15.97:0/3533555322 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec004430 con 0x7f40040b0d50 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ffaa00640 1 -- 172.21.15.97:0/3533555322 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fec005110 con 0x7f40040b0d50 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f3ffaa00640 1 -- 172.21.15.97:0/3533555322 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec005520 con 0x7f40040b0d50 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40040c98e0 con 0x7f40040b0d50 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40041572f0 con 0x7f40040b0d50 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.767+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40040cea20 con 0x7f40040b0d50 2024-09-10T21:26:34.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.771+0000 7f3ffaa00640 1 -- 172.21.15.97:0/3533555322 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3fec0046d0 con 0x7f40040b0d50 2024-09-10T21:26:34.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.771+0000 7f3ffaa00640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3fe00725e0 0x7f3fe0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:34.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.771+0000 7f3ffaa00640 1 -- 172.21.15.97:0/3533555322 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f3fec131490 con 0x7f40040b0d50 2024-09-10T21:26:34.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.771+0000 7f4001e00640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3fe00725e0 0x7f3fe0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:34.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.771+0000 7f4001e00640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3fe00725e0 0x7f3fe0074aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f40040c5fd0 tx=0x7f3ff4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:34.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.775+0000 7f3ffaa00640 1 -- 172.21.15.97:0/3533555322 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3fec0ba070 con 0x7f40040b0d50 2024-09-10T21:26:34.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.975+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f40040d2260 con 0x7f40040b0d50 2024-09-10T21:26:34.977 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:26:34.975+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:26:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.975+0000 7f3ffaa00640 1 -- 172.21.15.97:0/3533555322 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7f3fec016410 con 0x7f40040b0d50 2024-09-10T21:26:34.979 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:26:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3fe00725e0 msgr2=0x7f3fe0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3fe00725e0 0x7f3fe0074aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f40040c5fd0 tx=0x7f3ff4009040 comp rx=0 tx=0).stop 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40040b0d50 msgr2=0x7f40040ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40040b0d50 0x7f40040ca300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009790 tx=0x7f3fec004040 comp rx=0 tx=0).stop 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 shutdown_connections 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3fe00725e0 0x7f3fe0074aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f40040ce640 0x7f4004148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40040b0d50 0x7f40040ca300 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 --2- 172.21.15.97:0/3533555322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40040afdd0 0x7f40040c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 >> 172.21.15.97:0/3533555322 conn(0x7f40040d1c30 msgr2=0x7f40040b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 shutdown_connections 2024-09-10T21:26:34.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:34.979+0000 7f4009000640 1 -- 172.21.15.97:0/3533555322 wait complete. 2024-09-10T21:26:34.990 INFO:tasks.ceph.mds.a.smithi097.stderr: -4266> 2024-09-10T21:26:21.479+0000 7f3dc1400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:26:34.997 INFO:tasks.ceph.mds.a.smithi097.stderr: -339> 2024-09-10T21:26:33.155+0000 7f3dc4600640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5237:44 nref=7 cr=0x561a87d5bc00)) (dversion lock w=1 last_client=5237) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x561a87ca5680] 2024-09-10T21:26:35.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-10T21:26:35.038 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:26:35.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- 172.21.15.97:0/3393119810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd0143f90 msgr2=0x7f0cd00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 --2- 172.21.15.97:0/3393119810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd0143f90 0x7f0cd00b2a80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009c80 tx=0x7f0cc001cda0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- 172.21.15.97:0/3393119810 shutdown_connections 2024-09-10T21:26:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 --2- 172.21.15.97:0/3393119810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd0143f90 0x7f0cd00b2a80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- 172.21.15.97:0/3393119810 >> 172.21.15.97:0/3393119810 conn(0x7f0cd00b1780 msgr2=0x7f0cd00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- 172.21.15.97:0/3393119810 shutdown_connections 2024-09-10T21:26:35.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- 172.21.15.97:0/3393119810 wait complete. 2024-09-10T21:26:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 Processor -- start 2024-09-10T21:26:35.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- start start 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd006fd50 0x7f0cd0159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd0143f90 0x7f0cd015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd0144690 0x7f0cd006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd0168ed0 con 0x7f0cd006fd50 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd0168d50 con 0x7f0cd0144690 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cd6800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd0169050 con 0x7f0cd0143f90 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd006fd50 0x7f0cd0159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd006fd50 0x7f0cd0159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52196/0 (socket says 172.21.15.97:52196) 2024-09-10T21:26:35.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 -- 172.21.15.97:0/556152396 learned_addr learned my addr 172.21.15.97:0/556152396 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cc6a00640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd0143f90 0x7f0cd015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccfe00640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd0144690 0x7f0cd006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 -- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd0143f90 msgr2=0x7f0cd015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd0143f90 0x7f0cd015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 -- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd0144690 msgr2=0x7f0cd006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd0144690 0x7f0cd006ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 -- 172.21.15.97:0/556152396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd015bdf0 con 0x7f0cd006fd50 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cc6a00640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd0143f90 0x7f0cd015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0ccf400640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd006fd50 0x7f0cd0159fd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0000f90 tx=0x7f0cc0009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.147+0000 7f0cc7e00640 1 -- 172.21.15.97:0/556152396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00bb070 con 0x7f0cd006fd50 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cc7e00640 1 -- 172.21.15.97:0/556152396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cc0004d40 con 0x7f0cd006fd50 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cc7e00640 1 -- 172.21.15.97:0/556152396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00b2040 con 0x7f0cd006fd50 2024-09-10T21:26:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd015c0d0 con 0x7f0cd006fd50 2024-09-10T21:26:35.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd0140310 con 0x7f0cd006fd50 2024-09-10T21:26:35.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd00b2a80 con 0x7f0cd006fd50 2024-09-10T21:26:35.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cc7e00640 1 -- 172.21.15.97:0/556152396 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0cc00b5030 con 0x7f0cd006fd50 2024-09-10T21:26:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cc7e00640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cac0725e0 0x7f0cac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.151+0000 7f0cc7e00640 1 -- 172.21.15.97:0/556152396 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f0cc012ce80 con 0x7f0cd006fd50 2024-09-10T21:26:35.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.155+0000 7f0cc6a00640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cac0725e0 0x7f0cac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.155+0000 7f0cc6a00640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cac0725e0 0x7f0cac074aa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8001230 tx=0x7f0cb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.155+0000 7f0cc7e00640 1 -- 172.21.15.97:0/556152396 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0cc0016990 con 0x7f0cd006fd50 2024-09-10T21:26:35.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.323+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f0cd00b1ee0 con 0x7f0cd006fd50 2024-09-10T21:26:35.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.323+0000 7f0cc7e00640 1 -- 172.21.15.97:0/556152396 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7f0cc00f8380 con 0x7f0cd006fd50 2024-09-10T21:26:35.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cac0725e0 msgr2=0x7f0cac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cac0725e0 0x7f0cac074aa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8001230 tx=0x7f0cb8009040 comp rx=0 tx=0).stop 2024-09-10T21:26:35.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd006fd50 msgr2=0x7f0cd0159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd006fd50 0x7f0cd0159fd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0000f90 tx=0x7f0cc0009340 comp rx=0 tx=0).stop 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 shutdown_connections 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0cac0725e0 0x7f0cac074aa0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0cd0144690 0x7f0cd006ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0cd0143f90 0x7f0cd015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 --2- 172.21.15.97:0/556152396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cd006fd50 0x7f0cd0159fd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 >> 172.21.15.97:0/556152396 conn(0x7f0cd00b1780 msgr2=0x7f0cd00b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 shutdown_connections 2024-09-10T21:26:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.327+0000 7f0cd6800640 1 -- 172.21.15.97:0/556152396 wait complete. 2024-09-10T21:26:35.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-10T21:26:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- 172.21.15.97:0/2008015335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f34073bf0 msgr2=0x7f1f340ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 --2- 172.21.15.97:0/2008015335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f34073bf0 0x7f1f340ce5b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f340b36a0 tx=0x7f1f1c01c800 comp rx=0 tx=0).stop 2024-09-10T21:26:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- 172.21.15.97:0/2008015335 shutdown_connections 2024-09-10T21:26:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 --2- 172.21.15.97:0/2008015335 >> v2:172.21.15.194:3300/0 conn(0x7f1f340744e0 0x7f1f340cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 --2- 172.21.15.97:0/2008015335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f34073bf0 0x7f1f340ce5b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- 172.21.15.97:0/2008015335 >> 172.21.15.97:0/2008015335 conn(0x7f1f340cd460 msgr2=0x7f1f340cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- 172.21.15.97:0/2008015335 shutdown_connections 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- 172.21.15.97:0/2008015335 wait complete. 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 Processor -- start 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- start start 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f34073bf0 0x7f1f34149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f340744e0 0x7f1f34147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f3415aa30 0x7f1f340c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f341678d0 con 0x7f1f340744e0 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f34167750 con 0x7f1f34073bf0 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f3a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1f34167a50 con 0x7f1f3415aa30 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f340744e0 0x7f1f34147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f340744e0 0x7f1f34147df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52210/0 (socket says 172.21.15.97:52210) 2024-09-10T21:26:35.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.535+0000 7f1f2be00640 1 -- 172.21.15.97:0/681679240 learned_addr learned my addr 172.21.15.97:0/681679240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f2be00640 1 -- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f3415aa30 msgr2=0x7f1f340c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f39600640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f3415aa30 0x7f1f340c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f38c00640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f34073bf0 0x7f1f34149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f2be00640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f3415aa30 0x7f1f340c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f2be00640 1 -- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f34073bf0 msgr2=0x7f1f34149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f2be00640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f34073bf0 0x7f1f34149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f2be00640 1 -- 172.21.15.97:0/681679240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f340b56f0 con 0x7f1f340744e0 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f38c00640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f34073bf0 0x7f1f34149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f39600640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f3415aa30 0x7f1f340c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f2be00640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f340744e0 0x7f1f34147df0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c000c10 tx=0x7f1f1c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f31600640 1 -- 172.21.15.97:0/681679240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0044f0 con 0x7f1f340744e0 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f31600640 1 -- 172.21.15.97:0/681679240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f1c0be070 con 0x7f1f340744e0 2024-09-10T21:26:35.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f34148760 con 0x7f1f340744e0 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f31600640 1 -- 172.21.15.97:0/681679240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c005340 con 0x7f1f340744e0 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f34148d20 con 0x7f1f340744e0 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f340ce5b0 con 0x7f1f340744e0 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f31600640 1 -- 172.21.15.97:0/681679240 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1f1c0054e0 con 0x7f1f340744e0 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.539+0000 7f1f31600640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f100725e0 0x7f1f10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.543+0000 7f1f31600640 1 -- 172.21.15.97:0/681679240 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f1f1c12e510 con 0x7f1f340744e0 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.543+0000 7f1f38c00640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f100725e0 0x7f1f10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.543+0000 7f1f38c00640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f100725e0 0x7f1f10074aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000f70 tx=0x7f1f24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.543+0000 7f1f31600640 1 -- 172.21.15.97:0/681679240 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1f1c0163f0 con 0x7f1f340744e0 2024-09-10T21:26:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.711+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7f1f340b3b40 con 0x7f1f340744e0 2024-09-10T21:26:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.711+0000 7f1f31600640 1 -- 172.21.15.97:0/681679240 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v33)=0 v33) ==== 135+0+0 (secure 0 0 0) 0x7f1f1c0f9a10 con 0x7f1f340744e0 2024-09-10T21:26:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f100725e0 msgr2=0x7f1f10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f100725e0 0x7f1f10074aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000f70 tx=0x7f1f24009040 comp rx=0 tx=0).stop 2024-09-10T21:26:35.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f340744e0 msgr2=0x7f1f34147df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f340744e0 0x7f1f34147df0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c000c10 tx=0x7f1f1c004040 comp rx=0 tx=0).stop 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 shutdown_connections 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f100725e0 0x7f1f10074aa0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f3415aa30 0x7f1f340c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f340744e0 0x7f1f34147df0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 --2- 172.21.15.97:0/681679240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f34073bf0 0x7f1f34149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 >> 172.21.15.97:0/681679240 conn(0x7f1f340cd460 msgr2=0x7f1f3406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 shutdown_connections 2024-09-10T21:26:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.715+0000 7f1f3a000640 1 -- 172.21.15.97:0/681679240 wait complete. 2024-09-10T21:26:35.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:26:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.875+0000 7fbf86600640 1 -- 172.21.15.97:0/443553691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 msgr2=0x7fbf80166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.875+0000 7fbf86600640 1 --2- 172.21.15.97:0/443553691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf80166850 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009900 tx=0x7fbf7801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.875+0000 7fbf86600640 1 -- 172.21.15.97:0/443553691 shutdown_connections 2024-09-10T21:26:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.875+0000 7fbf86600640 1 --2- 172.21.15.97:0/443553691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf80166850 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.875+0000 7fbf86600640 1 -- 172.21.15.97:0/443553691 >> 172.21.15.97:0/443553691 conn(0x7fbf800b1780 msgr2=0x7fbf800b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 -- 172.21.15.97:0/443553691 shutdown_connections 2024-09-10T21:26:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 -- 172.21.15.97:0/443553691 wait complete. 2024-09-10T21:26:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 Processor -- start 2024-09-10T21:26:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 -- start start 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf800ce5f0 0x7fbf80165590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf80141d20 0x7fbf801387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf8015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8016a130 con 0x7fbf8015fde0 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80169fb0 con 0x7fbf80141d20 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf86600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8016a2b0 con 0x7fbf800ce5f0 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf8015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf8015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52234/0 (socket says 172.21.15.97:52234) 2024-09-10T21:26:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 -- 172.21.15.97:0/3910191057 learned_addr learned my addr 172.21.15.97:0/3910191057 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7f400640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf800ce5f0 0x7fbf80165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf76a00640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf80141d20 0x7fbf801387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 -- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf800ce5f0 msgr2=0x7fbf80165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf800ce5f0 0x7fbf80165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 -- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf80141d20 msgr2=0x7fbf801387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf80141d20 0x7fbf801387d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 -- 172.21.15.97:0/3910191057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf80138d10 con 0x7fbf8015fde0 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7f400640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf800ce5f0 0x7fbf80165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf76a00640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf80141d20 0x7fbf801387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.879+0000 7fbf7fe00640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf8015d540 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf780b0040 tx=0x7fbf780a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf77e00640 1 -- 172.21.15.97:0/3910191057 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780bd070 con 0x7fbf8015fde0 2024-09-10T21:26:35.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf77e00640 1 -- 172.21.15.97:0/3910191057 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf78004430 con 0x7fbf8015fde0 2024-09-10T21:26:35.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf80165b70 con 0x7fbf8015fde0 2024-09-10T21:26:35.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf77e00640 1 -- 172.21.15.97:0/3910191057 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780b2040 con 0x7fbf8015fde0 2024-09-10T21:26:35.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf80166130 con 0x7fbf8015fde0 2024-09-10T21:26:35.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf800ce9d0 con 0x7fbf8015fde0 2024-09-10T21:26:35.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf77e00640 1 -- 172.21.15.97:0/3910191057 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf78003860 con 0x7fbf8015fde0 2024-09-10T21:26:35.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf77e00640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf5c0725e0 0x7fbf5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf77e00640 1 -- 172.21.15.97:0/3910191057 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7fbf7812d270 con 0x7fbf8015fde0 2024-09-10T21:26:35.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.883+0000 7fbf7f400640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf5c0725e0 0x7fbf5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.887+0000 7fbf7f400640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf5c0725e0 0x7fbf5c074aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbf70007fd0 tx=0x7fbf70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:35.887+0000 7fbf77e00640 1 -- 172.21.15.97:0/3910191057 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf780164d0 con 0x7fbf8015fde0 2024-09-10T21:26:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.087+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbf80141360 con 0x7fbf8015fde0 2024-09-10T21:26:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.555+0000 7fbf77e00640 1 -- 172.21.15.97:0/3910191057 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v36) ==== 162+0+0 (secure 0 0 0) 0x7fbf780f86e0 con 0x7fbf8015fde0 2024-09-10T21:26:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf5c0725e0 msgr2=0x7fbf5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf5c0725e0 0x7fbf5c074aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbf70007fd0 tx=0x7fbf70009040 comp rx=0 tx=0).stop 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 msgr2=0x7fbf8015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf8015d540 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf780b0040 tx=0x7fbf780a5770 comp rx=0 tx=0).stop 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 shutdown_connections 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf5c0725e0 0x7fbf5c074aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8015fde0 0x7fbf8015d540 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf80141d20 0x7fbf801387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 --2- 172.21.15.97:0/3910191057 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf800ce5f0 0x7fbf80165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 >> 172.21.15.97:0/3910191057 conn(0x7fbf800b1780 msgr2=0x7fbf8013c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:36.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 shutdown_connections 2024-09-10T21:26:36.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.559+0000 7fbf86600640 1 -- 172.21.15.97:0/3910191057 wait complete. 2024-09-10T21:26:36.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 -- 172.21.15.97:0/2833716169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480afdd0 msgr2=0x7f6b480cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/2833716169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480afdd0 0x7f6b480cc6d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009900 tx=0x7f6b3001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 -- 172.21.15.97:0/2833716169 shutdown_connections 2024-09-10T21:26:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/2833716169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b480b0d50 0x7f6b480ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/2833716169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480afdd0 0x7f6b480cc6d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 -- 172.21.15.97:0/2833716169 >> 172.21.15.97:0/2833716169 conn(0x7f6b480d1c30 msgr2=0x7f6b480d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:36.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 -- 172.21.15.97:0/2833716169 shutdown_connections 2024-09-10T21:26:36.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.727+0000 7f6b4ea00640 1 -- 172.21.15.97:0/2833716169 wait complete. 2024-09-10T21:26:36.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 Processor -- start 2024-09-10T21:26:36.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 -- start start 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b480afdd0 0x7f6b480ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480b0d50 0x7f6b480c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b480ce640 0x7f6b481490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48168e90 con 0x7f6b480b0d50 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48168d10 con 0x7f6b480ce640 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6b48169010 con 0x7f6b480afdd0 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480b0d50 0x7f6b480c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480b0d50 0x7f6b480c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52266/0 (socket says 172.21.15.97:52266) 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 -- 172.21.15.97:0/709672501 learned_addr learned my addr 172.21.15.97:0/709672501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b47e00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b480afdd0 0x7f6b480ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:36.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ca00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b480ce640 0x7f6b481490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 -- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b480afdd0 msgr2=0x7f6b480ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b480afdd0 0x7f6b480ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 -- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b480ce640 msgr2=0x7f6b481490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b480ce640 0x7f6b481490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 -- 172.21.15.97:0/709672501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b480c8920 con 0x7f6b480b0d50 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ca00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b480ce640 0x7f6b481490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b3fe00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480b0d50 0x7f6b480c6c10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009850 tx=0x7f6b300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b44c00640 1 -- 172.21.15.97:0/709672501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b300b2040 con 0x7f6b480b0d50 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b44c00640 1 -- 172.21.15.97:0/709672501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b30005030 con 0x7f6b480b0d50 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b44c00640 1 -- 172.21.15.97:0/709672501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30005460 con 0x7f6b480b0d50 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b480c8f50 con 0x7f6b480b0d50 2024-09-10T21:26:36.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b480c9510 con 0x7f6b480b0d50 2024-09-10T21:26:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.731+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b480c8ce0 con 0x7f6b480b0d50 2024-09-10T21:26:36.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.735+0000 7f6b44c00640 1 -- 172.21.15.97:0/709672501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6b300046d0 con 0x7f6b480b0d50 2024-09-10T21:26:36.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.735+0000 7f6b44c00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b240725e0 0x7f6b24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:36.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.735+0000 7f6b44c00640 1 -- 172.21.15.97:0/709672501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f6b3012cfe0 con 0x7f6b480b0d50 2024-09-10T21:26:36.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.735+0000 7f6b47e00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b240725e0 0x7f6b24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:36.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.735+0000 7f6b47e00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b240725e0 0x7f6b24074aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b38007fd0 tx=0x7f6b38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:36.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.739+0000 7f6b44c00640 1 -- 172.21.15.97:0/709672501 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6b300164d0 con 0x7f6b480b0d50 2024-09-10T21:26:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.939+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b480cea20 con 0x7f6b480b0d50 2024-09-10T21:26:36.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.939+0000 7f6b44c00640 1 -- 172.21.15.97:0/709672501 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5217 (secure 0 0 0) 0x7f6b300f84d0 con 0x7f6b480b0d50 2024-09-10T21:26:36.943 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:36.943 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"btime":"2024-09-10T21:26:36:557641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:36.557613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T21:26:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2024-09-10T21:26:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b240725e0 msgr2=0x7f6b24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b240725e0 0x7f6b24074aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b38007fd0 tx=0x7f6b38009040 comp rx=0 tx=0).stop 2024-09-10T21:26:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480b0d50 msgr2=0x7f6b480c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480b0d50 0x7f6b480c6c10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009850 tx=0x7f6b300040c0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 shutdown_connections 2024-09-10T21:26:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b240725e0 0x7f6b24074aa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b480ce640 0x7f6b481490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b480b0d50 0x7f6b480c6c10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 --2- 172.21.15.97:0/709672501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b480afdd0 0x7f6b480ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 >> 172.21.15.97:0/709672501 conn(0x7f6b480d1c30 msgr2=0x7f6b480b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 shutdown_connections 2024-09-10T21:26:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:36.943+0000 7f6b4ea00640 1 -- 172.21.15.97:0/709672501 wait complete. 2024-09-10T21:26:36.964 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-10T21:26:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.103+0000 7f483e800640 1 -- 172.21.15.97:0/3280701884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c003a10 msgr2=0x7f481c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.103+0000 7f483e800640 1 --2- 172.21.15.97:0/3280701884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481c003a10 0x7f481c003e30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4828009b60 tx=0x7f48380b0220 comp rx=0 tx=0).stop 2024-09-10T21:26:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 -- 172.21.15.97:0/3280701884 reap_dead start 2024-09-10T21:26:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- 172.21.15.97:0/3280701884 shutdown_connections 2024-09-10T21:26:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- 172.21.15.97:0/3280701884 >> 172.21.15.97:0/3280701884 conn(0x7f48380d1c30 msgr2=0x7f48380d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- 172.21.15.97:0/3280701884 shutdown_connections 2024-09-10T21:26:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- 172.21.15.97:0/3280701884 wait complete. 2024-09-10T21:26:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 Processor -- start 2024-09-10T21:26:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- start start 2024-09-10T21:26:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481c003a10 0x7f48381fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838150680 0x7f48381f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838162a00 0x7f483815f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4838166c60 con 0x7f4838162a00 2024-09-10T21:26:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4838166ae0 con 0x7f4838150680 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f483e800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4838166de0 con 0x7f481c003a10 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838162a00 0x7f483815f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838162a00 0x7f483815f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52278/0 (socket says 172.21.15.97:52278) 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 -- 172.21.15.97:0/2138866590 learned_addr learned my addr 172.21.15.97:0/2138866590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4836a00640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838150680 0x7f48381f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837400640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481c003a10 0x7f48381fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 -- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481c003a10 msgr2=0x7f48381fbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481c003a10 0x7f48381fbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 -- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838150680 msgr2=0x7f48381f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838150680 0x7f48381f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 -- 172.21.15.97:0/2138866590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4838165380 con 0x7f4838162a00 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.107+0000 7f4837e00640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838162a00 0x7f483815f820 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4838074d10 tx=0x7f483000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f482f400640 1 -- 172.21.15.97:0/2138866590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830009ab0 con 0x7f4838162a00 2024-09-10T21:26:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f482f400640 1 -- 172.21.15.97:0/2138866590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4830003d90 con 0x7f4838162a00 2024-09-10T21:26:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f482f400640 1 -- 172.21.15.97:0/2138866590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830004290 con 0x7f4838162a00 2024-09-10T21:26:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4838165660 con 0x7f4838162a00 2024-09-10T21:26:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483815e1a0 con 0x7f4838162a00 2024-09-10T21:26:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f482f400640 1 -- 172.21.15.97:0/2138866590 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4830018020 con 0x7f4838162a00 2024-09-10T21:26:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f482f400640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f48180725e0 0x7f4818074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f482f400640 1 -- 172.21.15.97:0/2138866590 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f4830094000 con 0x7f4838162a00 2024-09-10T21:26:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f4841c36000 1 Processor -- start 2024-09-10T21:26:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.111+0000 7f4837400640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f48180725e0 0x7f4818074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- start start 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f48180725e0 0x7f4818074aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48280ae000 tx=0x7f482801a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b085580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b085ac0 0x56098b03b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56098af23cb0 con 0x56098b0851a0 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56098ae39310 con 0x56098b085ac0 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56098ae96b90 con 0x56098b03bbc0 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b085580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b085580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52284/0 (socket says 172.21.15.97:52284) 2024-09-10T21:26:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 -- 172.21.15.97:0/455177181 learned_addr learned my addr 172.21.15.97:0/455177181 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4836a00640 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b085ac0 0x56098b03b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837e00640 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 -- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 msgr2=0x56098b03bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b03bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 -- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b085ac0 msgr2=0x56098b03b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b085ac0 0x56098b03b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 -- 172.21.15.97:0/455177181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098ae3ac60 con 0x56098b0851a0 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4837400640 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b085580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4828009b60 tx=0x7f48280b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2b31635ec3e876 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4823e00640 1 -- 172.21.15.97:0/455177181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280b1040 con 0x56098b0851a0 2024-09-10T21:26:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4823e00640 1 -- 172.21.15.97:0/455177181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4828004880 con 0x56098b0851a0 2024-09-10T21:26:37.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4823e00640 1 -- 172.21.15.97:0/455177181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828004cd0 con 0x56098b0851a0 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- 172.21.15.97:0/455177181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 msgr2=0x56098b085580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b085580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4828009b60 tx=0x7f48280b8000 comp rx=0 tx=0).stop 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- 172.21.15.97:0/455177181 shutdown_connections 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b03bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b085ac0 0x56098b03b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 --2- 172.21.15.97:0/455177181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b085580 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- 172.21.15.97:0/455177181 >> 172.21.15.97:0/455177181 conn(0x56098b041d20 msgr2=0x56098b039f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- 172.21.15.97:0/455177181 shutdown_connections 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- 172.21.15.97:0/455177181 wait complete. 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 -- start start 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.115+0000 7f4841c36000 1 Processor -- start 2024-09-10T21:26:37.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b03bbc0 0x56098b08c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b08ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b085ac0 0x56098b08cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56098ae89dd0 con 0x56098b0851a0 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56098ae85fc0 con 0x56098b03bbc0 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56098ae2b4b0 con 0x56098b085ac0 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 10 client.0 _ll_register_callbacks cb 0x56098ae097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b08ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b08ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52298/0 (socket says 172.21.15.97:52298) 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 -- 172.21.15.97:0/2659692845 learned_addr learned my addr 172.21.15.97:0/2659692845 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4837400640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b03bbc0 0x56098b08c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4837e00640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b085ac0 0x56098b08cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 -- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b085ac0 msgr2=0x56098b08cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b085ac0 0x56098b08cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 -- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b03bbc0 msgr2=0x56098b08c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b03bbc0 0x56098b08c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 -- 172.21.15.97:0/2659692845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56098ae241f0 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4837400640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b03bbc0 0x56098b08c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4836a00640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b08ca70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f48240062f0 tx=0x7f48240039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0578c05c5f84dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4820c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4820c00640 1 -- 172.21.15.97:0/2659692845 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824011630 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56098ae1e330 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4820c00640 1 -- 172.21.15.97:0/2659692845 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f482400cae0 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4820c00640 1 -- 172.21.15.97:0/2659692845 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4824023050 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 10 client.5775 fetch_fsmap learned FSMap version 36 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x56098ae0afa0 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4820c00640 1 -- 172.21.15.97:0/2659692845 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56098ae1e330 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4820c00640 1 -- 172.21.15.97:0/2659692845 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f4824026020 con 0x56098b0851a0 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 10 client.5775 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:26:37.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 10 client.5775 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 20 client.5775 populate_metadata read hostname 'smithi097' 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b094900 0x56098b096cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 4 client.5775 mds_command: new command op to 5730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 --> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56098ae98ae0 con 0x56098b094900 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4837400640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b094900 0x56098b096cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4820c00640 10 client.5775 ms_handle_connect on v2:172.21.15.97:6832/2218970160 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.119+0000 7f4837400640 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b094900 0x56098b096cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e2b163ec4f2ff8 server_cookie=ea9d6b9d5f9676d2 in_seq=0 out_seq=0 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4820c00640 1 -- 172.21.15.97:0/2659692845 <== mds.0 v2:172.21.15.97:6832/2218970160 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56098ae98ae0 con 0x56098b094900 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4820c00640 10 client.5775 handle_command_reply: tid=0 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 client.5775 shutdown 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 msgr2=0x56098b08ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b08ca70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f48240062f0 tx=0x7f48240039d0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 shutdown_connections 2024-09-10T21:26:37.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b085ac0 0x56098b08cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b0851a0 0x56098b08ca70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b094900 0x56098b096cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 --2- 172.21.15.97:0/2659692845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b03bbc0 0x56098b08c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 >> 172.21.15.97:0/2659692845 conn(0x56098b041d20 msgr2=0x56098b039f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4820c00640 0 client.5775 ms_handle_reset on v2:172.21.15.97:6832/2218970160 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 shutdown_connections 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 1 -- 172.21.15.97:0/2659692845 wait complete. 2024-09-10T21:26:37.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.123+0000 7f4841c36000 20 client.5775 trim_cache size 0 max 16384 2024-09-10T21:26:37.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.135+0000 7f4841c36000 1 Processor -- start 2024-09-10T21:26:37.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.135+0000 7f4841c36000 1 -- start start 2024-09-10T21:26:37.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b041d20 0x56098b0f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b03bbc0 0x56098b0f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4838205300 con 0x56098b03bbc0 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f48382044d0 con 0x56098b0851a0 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f48380b5e20 con 0x56098b041d20 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b03bbc0 0x56098b0f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b03bbc0 0x56098b0f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52310/0 (socket says 172.21.15.97:52310) 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 -- 172.21.15.97:0/1138158498 learned_addr learned my addr 172.21.15.97:0/1138158498 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4837e00640 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4837400640 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b041d20 0x56098b0f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 -- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b041d20 msgr2=0x56098b0f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b041d20 0x56098b0f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 -- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 msgr2=0x56098b0f84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0f84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 -- 172.21.15.97:0/1138158498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098aee3660 con 0x56098b03bbc0 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4836a00640 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b03bbc0 0x56098b0f7f70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4824006f30 tx=0x7f4824001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a371a86368080a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4823400640 1 -- 172.21.15.97:0/1138158498 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482402c070 con 0x56098b03bbc0 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4823400640 1 -- 172.21.15.97:0/1138158498 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48240119c0 con 0x56098b03bbc0 2024-09-10T21:26:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4823400640 1 -- 172.21.15.97:0/1138158498 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824015040 con 0x56098b03bbc0 2024-09-10T21:26:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- 172.21.15.97:0/1138158498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b03bbc0 msgr2=0x56098b0f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b03bbc0 0x56098b0f7f70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4824006f30 tx=0x7f4824001a40 comp rx=0 tx=0).stop 2024-09-10T21:26:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- 172.21.15.97:0/1138158498 shutdown_connections 2024-09-10T21:26:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b03bbc0 0x56098b0f7f70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 --2- 172.21.15.97:0/1138158498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b041d20 0x56098b0f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- 172.21.15.97:0/1138158498 >> 172.21.15.97:0/1138158498 conn(0x56098b038270 msgr2=0x56098b03bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- 172.21.15.97:0/1138158498 shutdown_connections 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- 172.21.15.97:0/1138158498 wait complete. 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 -- start start 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.139+0000 7f4841c36000 1 Processor -- start 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b0fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b041d20 0x56098b0fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56098ae89dd0 con 0x56098b041d20 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56098b044d40 con 0x56098b0851a0 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56098b042100 con 0x56098b03bbc0 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 10 client.0 _ll_register_callbacks cb 0x7f4824006e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:37.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4836a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b041d20 0x56098b0fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b0fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b0fb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:46014/0 (socket says 172.21.15.97:46014) 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 -- 172.21.15.97:0/73555316 learned_addr learned my addr 172.21.15.97:0/73555316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837e00640 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 -- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 msgr2=0x56098b0fbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0fbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 -- 172.21.15.97:0/73555316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b041d20 msgr2=0x56098b0fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b041d20 0x56098b0fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 -- 172.21.15.97:0/73555316 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x56098ae50090 con 0x56098b03bbc0 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4837400640 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b0fb380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4828009a80 tx=0x7f4828005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8291278406f2da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3301/0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 1 -- 172.21.15.97:0/73555316 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280b1040 con 0x56098b03bbc0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48240119c0 con 0x56098b03bbc0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 1 -- 172.21.15.97:0/73555316 <== mon.2 v2:172.21.15.194:3301/0 2 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f4828000bd0 con 0x56098b03bbc0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 1 -- 172.21.15.97:0/73555316 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4828004020 con 0x56098b03bbc0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 10 client.5312 fetch_fsmap learned FSMap version 36 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x56098ae1c5f0 con 0x56098b03bbc0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 1 -- 172.21.15.97:0/73555316 <== mon.2 v2:172.21.15.194:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48240119c0 con 0x56098b03bbc0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 1 -- 172.21.15.97:0/73555316 <== mon.2 v2:172.21.15.194:3301/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f48280041c0 con 0x56098b03bbc0 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 10 client.5312 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 10 client.5312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 20 client.5312 populate_metadata read hostname 'smithi097' 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b085ac0 0x56098b0b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 4 client.5312 mds_command: new command op to 5730 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 --> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56098b09d010 con 0x56098b085ac0 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4836a00640 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b085ac0 0x56098b0b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4836a00640 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b085ac0 0x56098b0b7690 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d466e3f90c69bb56 server_cookie=668d7bc48b3d05bd in_seq=0 out_seq=0 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 10 client.5312 ms_handle_connect on v2:172.21.15.97:6832/2218970160 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 1 -- 172.21.15.97:0/73555316 <== mds.0 v2:172.21.15.97:6832/2218970160 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56098b09d010 con 0x56098b085ac0 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 10 client.5312 handle_command_reply: tid=0 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 client.5312 shutdown 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 msgr2=0x56098b0fb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b0fb380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4828009a80 tx=0x7f4828005800 comp rx=0 tx=0).stop 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 shutdown_connections 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.97:6832/2218970160,v1:172.21.15.97:6834/2218970160] conn(0x56098b085ac0 0x56098b0b7690 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56098b0851a0 0x56098b0fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4820c00640 0 client.5312 ms_handle_reset on v2:172.21.15.97:6832/2218970160 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56098b041d20 0x56098b0fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 --2- 172.21.15.97:0/73555316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56098b03bbc0 0x56098b0fb380 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 >> 172.21.15.97:0/73555316 conn(0x56098b038270 msgr2=0x56098b03c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 shutdown_connections 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 1 -- 172.21.15.97:0/73555316 wait complete. 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.143+0000 7f4841c36000 20 client.5312 trim_cache size 0 max 16384 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stdout: "message": "", 2024-09-10T21:26:37.153 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-09-10T21:26:37.154 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:26:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f48180725e0 msgr2=0x7f4818074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f48180725e0 0x7f4818074aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48280ae000 tx=0x7f482801a010 comp rx=0 tx=0).stop 2024-09-10T21:26:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838162a00 msgr2=0x7f483815f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838162a00 0x7f483815f820 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4838074d10 tx=0x7f483000f790 comp rx=0 tx=0).stop 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 shutdown_connections 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f48180725e0 0x7f4818074aa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838162a00 0x7f483815f820 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838150680 0x7f48381f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 --2- 172.21.15.97:0/2138866590 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481c003a10 0x7f48381fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 >> 172.21.15.97:0/2138866590 conn(0x7f48380d1c30 msgr2=0x7f48380c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 shutdown_connections 2024-09-10T21:26:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.147+0000 7f483e800640 1 -- 172.21.15.97:0/2138866590 wait complete. 2024-09-10T21:26:37.167 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- 172.21.15.97:0/1337864144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce5f0 msgr2=0x7f0e38141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/1337864144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce5f0 0x7f0e38141d20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009840 tx=0x7f0e2801c820 comp rx=0 tx=0).stop 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- 172.21.15.97:0/1337864144 shutdown_connections 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/1337864144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce5f0 0x7f0e38141d20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- 172.21.15.97:0/1337864144 >> 172.21.15.97:0/1337864144 conn(0x7f0e380b1780 msgr2=0x7f0e380b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- 172.21.15.97:0/1337864144 shutdown_connections 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- 172.21.15.97:0/1337864144 wait complete. 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 Processor -- start 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- start start 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e380ce5f0 0x7f0e3813b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce9d0 0x7f0e380b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e3815fde0 0x7f0e380b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38169d60 con 0x7f0e380ce9d0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38169be0 con 0x7f0e380ce5f0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e3ce00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0e38169ee0 con 0x7f0e3815fde0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce9d0 0x7f0e380b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce9d0 0x7f0e380b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49778/0 (socket says 172.21.15.97:49778) 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 -- 172.21.15.97:0/2151408872 learned_addr learned my addr 172.21.15.97:0/2151408872 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e35e00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e380ce5f0 0x7f0e3813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e36800640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e3815fde0 0x7f0e380b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 -- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e3815fde0 msgr2=0x7f0e380b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e3815fde0 0x7f0e380b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 -- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e380ce5f0 msgr2=0x7f0e3813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e380ce5f0 0x7f0e3813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.307+0000 7f0e2d400640 1 -- 172.21.15.97:0/2151408872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3813ba90 con 0x7f0e380ce9d0 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e2d400640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce9d0 0x7f0e380b3bd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000c500 tx=0x7f0e2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e2ea00640 1 -- 172.21.15.97:0/2151408872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20017070 con 0x7f0e380ce9d0 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e2ea00640 1 -- 172.21.15.97:0/2151408872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e200053c0 con 0x7f0e380ce9d0 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e2ea00640 1 -- 172.21.15.97:0/2151408872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20012890 con 0x7f0e380ce9d0 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e381297a0 con 0x7f0e380ce9d0 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e38129d60 con 0x7f0e380ce9d0 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e38141d20 con 0x7f0e380ce9d0 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e2ea00640 1 -- 172.21.15.97:0/2151408872 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0e20003600 con 0x7f0e380ce9d0 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e2ea00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e14072610 0x7f0e14074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e2ea00640 1 -- 172.21.15.97:0/2151408872 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f0e200929a0 con 0x7f0e380ce9d0 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.311+0000 7f0e35e00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e14072610 0x7f0e14074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.315+0000 7f0e35e00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e14072610 0x7f0e14074ad0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e280af000 tx=0x7f0e2801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.315+0000 7f0e2ea00640 1 -- 172.21.15.97:0/2151408872 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0e2005d7c0 con 0x7f0e380ce9d0 2024-09-10T21:26:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.511+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e38141360 con 0x7f0e380ce9d0 2024-09-10T21:26:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e2ea00640 1 -- 172.21.15.97:0/2151408872 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5217 (secure 0 0 0) 0x7f0e200621f0 con 0x7f0e380ce9d0 2024-09-10T21:26:37.516 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:37.517 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"btime":"2024-09-10T21:26:36:557641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:36.557613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T21:26:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e14072610 msgr2=0x7f0e14074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e14072610 0x7f0e14074ad0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e280af000 tx=0x7f0e2801b060 comp rx=0 tx=0).stop 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce9d0 msgr2=0x7f0e380b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce9d0 0x7f0e380b3bd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000c500 tx=0x7f0e2000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 shutdown_connections 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e14072610 0x7f0e14074ad0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e3815fde0 0x7f0e380b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e380ce9d0 0x7f0e380b3bd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 --2- 172.21.15.97:0/2151408872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e380ce5f0 0x7f0e3813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 >> 172.21.15.97:0/2151408872 conn(0x7f0e380b1780 msgr2=0x7f0e38073e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 shutdown_connections 2024-09-10T21:26:37.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.515+0000 7f0e3ce00640 1 -- 172.21.15.97:0/2151408872 wait complete. 2024-09-10T21:26:37.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:37.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- 172.21.15.97:0/1900706940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc0ceee0 msgr2=0x7f3bcc0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 --2- 172.21.15.97:0/1900706940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc0ceee0 0x7f3bcc0cf350 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400aa10 tx=0x7f3bc40203e0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- 172.21.15.97:0/1900706940 shutdown_connections 2024-09-10T21:26:37.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 --2- 172.21.15.97:0/1900706940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc0ceee0 0x7f3bcc0cf350 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 --2- 172.21.15.97:0/1900706940 >> v2:172.21.15.194:3300/0 conn(0x7f3bcc06ec40 0x7f3bcc06f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- 172.21.15.97:0/1900706940 >> 172.21.15.97:0/1900706940 conn(0x7f3bcc0b1780 msgr2=0x7f3bcc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- 172.21.15.97:0/1900706940 shutdown_connections 2024-09-10T21:26:37.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- 172.21.15.97:0/1900706940 wait complete. 2024-09-10T21:26:37.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 Processor -- start 2024-09-10T21:26:37.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- start start 2024-09-10T21:26:37.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bcc06ec40 0x7f3bcc1e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bcc0ceee0 0x7f3bcc1f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc1511f0 0x7f3bcc13dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc06e740 con 0x7f3bcc1511f0 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc06e5c0 con 0x7f3bcc06ec40 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3bcc06e8c0 con 0x7f3bcc0ceee0 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc1511f0 0x7f3bcc13dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc1511f0 0x7f3bcc13dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49798/0 (socket says 172.21.15.97:49798) 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 -- 172.21.15.97:0/3237972485 learned_addr learned my addr 172.21.15.97:0/3237972485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bcbe00640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bcc06ec40 0x7f3bcc1e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bc3400640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bcc0ceee0 0x7f3bcc1f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 -- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bcc0ceee0 msgr2=0x7f3bcc1f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bcc0ceee0 0x7f3bcc1f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 -- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bcc06ec40 msgr2=0x7f3bcc1e29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bcc06ec40 0x7f3bcc1e29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.687+0000 7f3bd1200640 1 -- 172.21.15.97:0/3237972485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc141670 con 0x7f3bcc1511f0 2024-09-10T21:26:37.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bd1200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc1511f0 0x7f3bcc13dc30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3bc401e010 tx=0x7f3bc4005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bc8c00640 1 -- 172.21.15.97:0/3237972485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40bf070 con 0x7f3bcc1511f0 2024-09-10T21:26:37.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bc8c00640 1 -- 172.21.15.97:0/3237972485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bc40b0c50 con 0x7f3bcc1511f0 2024-09-10T21:26:37.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc1408b0 con 0x7f3bcc1511f0 2024-09-10T21:26:37.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bc8c00640 1 -- 172.21.15.97:0/3237972485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40b5040 con 0x7f3bcc1511f0 2024-09-10T21:26:37.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc140e70 con 0x7f3bcc1511f0 2024-09-10T21:26:37.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc06f090 con 0x7f3bcc1511f0 2024-09-10T21:26:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bc8c00640 1 -- 172.21.15.97:0/3237972485 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3bc40b8070 con 0x7f3bcc1511f0 2024-09-10T21:26:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bc8c00640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ba80725e0 0x7f3ba8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bc8c00640 1 -- 172.21.15.97:0/3237972485 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f3bc4130710 con 0x7f3bcc1511f0 2024-09-10T21:26:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.691+0000 7f3bcbe00640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ba80725e0 0x7f3ba8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.695+0000 7f3bcbe00640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ba80725e0 0x7f3ba8074aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc001e40 tx=0x7f3bbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.695+0000 7f3bc8c00640 1 -- 172.21.15.97:0/3237972485 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3bc4019f40 con 0x7f3bcc1511f0 2024-09-10T21:26:37.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.895+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bcc0b1560 con 0x7f3bcc1511f0 2024-09-10T21:26:37.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.895+0000 7f3bc8c00640 1 -- 172.21.15.97:0/3237972485 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5220 (secure 0 0 0) 0x7f3bc40b0e20 con 0x7f3bcc1511f0 2024-09-10T21:26:37.899 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:37.899 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":37,"btime":"2024-09-10T21:26:37:559584+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:36.571098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T21:26:37.899 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 37 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ba80725e0 msgr2=0x7f3ba8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ba80725e0 0x7f3ba8074aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc001e40 tx=0x7f3bbc009040 comp rx=0 tx=0).stop 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc1511f0 msgr2=0x7f3bcc13dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc1511f0 0x7f3bcc13dc30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3bc401e010 tx=0x7f3bc4005e90 comp rx=0 tx=0).stop 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 shutdown_connections 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ba80725e0 0x7f3ba8074aa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bcc1511f0 0x7f3bcc13dc30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3bcc0ceee0 0x7f3bcc1f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 --2- 172.21.15.97:0/3237972485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bcc06ec40 0x7f3bcc1e29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 >> 172.21.15.97:0/3237972485 conn(0x7f3bcc0b1780 msgr2=0x7f3bcc15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 shutdown_connections 2024-09-10T21:26:37.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:37.899+0000 7f3bd3200640 1 -- 172.21.15.97:0/3237972485 wait complete. 2024-09-10T21:26:37.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.059+0000 7f3e07200640 1 -- 172.21.15.97:0/1466150347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 msgr2=0x7f3de8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.059+0000 7f3e07200640 1 --2- 172.21.15.97:0/1466150347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3de8004c90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3e0015c040 tx=0x7f3df001c6f0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.059+0000 7f3e07200640 1 -- 172.21.15.97:0/1466150347 shutdown_connections 2024-09-10T21:26:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.059+0000 7f3e07200640 1 --2- 172.21.15.97:0/1466150347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3de8004c90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.059+0000 7f3e07200640 1 -- 172.21.15.97:0/1466150347 >> 172.21.15.97:0/1466150347 conn(0x7f3e000b2ee0 msgr2=0x7f3e000d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.059+0000 7f3e07200640 1 -- 172.21.15.97:0/1466150347 shutdown_connections 2024-09-10T21:26:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.059+0000 7f3e07200640 1 -- 172.21.15.97:0/1466150347 wait complete. 2024-09-10T21:26:38.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 Processor -- start 2024-09-10T21:26:38.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 -- start start 2024-09-10T21:26:38.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3e001506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e0006d7a0 0x7f3e0014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e000ae760 0x7f3e000aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0015c8a0 con 0x7f3de8004810 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0015c720 con 0x7f3e000ae760 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3e000cd2c0 con 0x7f3e0006d7a0 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3e001506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3e001506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49822/0 (socket says 172.21.15.97:49822) 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 -- 172.21.15.97:0/3908600902 learned_addr learned my addr 172.21.15.97:0/3908600902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05400640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e0006d7a0 0x7f3e0014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e06800640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e000ae760 0x7f3e000aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 -- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e0006d7a0 msgr2=0x7f3e0014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e0006d7a0 0x7f3e0014a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 -- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e000ae760 msgr2=0x7f3e000aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e000ae760 0x7f3e000aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 -- 172.21.15.97:0/3908600902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0006e7b0 con 0x7f3de8004810 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05400640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e0006d7a0 0x7f3e0014a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e05e00640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3e001506d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3df4006680 tx=0x7f3df40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3dfa000640 1 -- 172.21.15.97:0/3908600902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4015070 con 0x7f3de8004810 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3dfa000640 1 -- 172.21.15.97:0/3908600902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3df4011900 con 0x7f3de8004810 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3dfa000640 1 -- 172.21.15.97:0/3908600902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4011d50 con 0x7f3de8004810 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0006df10 con 0x7f3de8004810 2024-09-10T21:26:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0006e4d0 con 0x7f3de8004810 2024-09-10T21:26:38.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.063+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e000cd7b0 con 0x7f3de8004810 2024-09-10T21:26:38.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.067+0000 7f3dfa000640 1 -- 172.21.15.97:0/3908600902 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3df4010420 con 0x7f3de8004810 2024-09-10T21:26:38.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.067+0000 7f3dfa000640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de40725e0 0x7f3de4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.067+0000 7f3dfa000640 1 -- 172.21.15.97:0/3908600902 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f3df4094100 con 0x7f3de8004810 2024-09-10T21:26:38.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.067+0000 7f3e05400640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de40725e0 0x7f3de4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.067+0000 7f3e05400640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de40725e0 0x7f3de4074aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3de80057f0 tx=0x7f3df00194e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.071+0000 7f3dfa000640 1 -- 172.21.15.97:0/3908600902 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3df405f5f0 con 0x7f3de8004810 2024-09-10T21:26:38.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.267+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e000ccc60 con 0x7f3de8004810 2024-09-10T21:26:38.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.267+0000 7f3dfa000640 1 -- 172.21.15.97:0/3908600902 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5220 (secure 0 0 0) 0x7f3df405eef0 con 0x7f3de8004810 2024-09-10T21:26:38.272 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:38.272 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":37,"btime":"2024-09-10T21:26:37:559584+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:36.571098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T21:26:38.272 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 37 2024-09-10T21:26:38.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de40725e0 msgr2=0x7f3de4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de40725e0 0x7f3de4074aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3de80057f0 tx=0x7f3df00194e0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 msgr2=0x7f3e001506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3e001506d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3df4006680 tx=0x7f3df40039d0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 shutdown_connections 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3de40725e0 0x7f3de4074aa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e000ae760 0x7f3e000aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e0006d7a0 0x7f3e0014a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 --2- 172.21.15.97:0/3908600902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de8004810 0x7f3e001506d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 >> 172.21.15.97:0/3908600902 conn(0x7f3e000b2ee0 msgr2=0x7f3e000b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 shutdown_connections 2024-09-10T21:26:38.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.271+0000 7f3e07200640 1 -- 172.21.15.97:0/3908600902 wait complete. 2024-09-10T21:26:38.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /a 0 2024-09-10T21:26:38.449 INFO:teuthology.orchestra.run.smithi097.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-10T21:26:33.155212+0000","btime":"2024-09-10T21:26:27.175272+0000","mode":16893,"uid":1000,"gid":1271,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-10T21:26:33.155212+0000","atime":"2024-09-10T21:26:27.175272+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-10T21:26:33.155212+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T21:26:33.155212+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T21:26:33.155212+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"inline_data_version":18446744073709551615,"inline_data_length":0,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirfrag":1,"dirty":1},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-10T21:26:38.450 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-10T21:26:33.155212+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-10T21:26:27.175272+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-10T21:26:27.175272+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-10T21:26:33.155212+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-10T21:26:33.155212+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1271, "inline_data_length": 0, "inline_data_version": 18446744073709551615, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16893, "mtime": "2024-09-10T21:26:33.155212+0000", "nestlock": {}, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-10T21:26:33.155212+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-09-10T21:26:38.450 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-10T21:26:33.155212+0000', 'btime': '2024-09-10T21:26:27.175272+0000', 'mode': 16893, 'uid': 1000, 'gid': 1271, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-10T21:26:33.155212+0000', 'atime': '2024-09-10T21:26:27.175272+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-10T21:26:33.155212+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T21:26:33.155212+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T21:26:33.155212+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'inline_data_version': 18446744073709551615, 'inline_data_length': 0, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirfrag': 1, 'dirty': 1}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-10T21:26:38.450 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:26:38.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:38.542 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:38.623 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:26:38.624 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:38.624 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:38.624 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:38.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:26:38.633 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:38.633 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:38.633 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:38.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:38.642 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:38.642 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:38.642 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:38.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:26:38.684 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:26:38.684 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:38.684 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:38.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:38.692 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:38.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:38.804 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 -- 172.21.15.97:0/3064044236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 msgr2=0x7f72fc13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 --2- 172.21.15.97:0/3064044236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc13f9b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009c80 tx=0x7f72f401cda0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 -- 172.21.15.97:0/3064044236 shutdown_connections 2024-09-10T21:26:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 --2- 172.21.15.97:0/3064044236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc13f9b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 --2- 172.21.15.97:0/3064044236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72fc0ce5f0 0x7f72fc141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 -- 172.21.15.97:0/3064044236 >> 172.21.15.97:0/3064044236 conn(0x7f72fc0b1780 msgr2=0x7f72fc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 -- 172.21.15.97:0/3064044236 shutdown_connections 2024-09-10T21:26:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 -- 172.21.15.97:0/3064044236 wait complete. 2024-09-10T21:26:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 Processor -- start 2024-09-10T21:26:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.867+0000 7f7303a00640 1 -- start start 2024-09-10T21:26:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72fc0ce5f0 0x7f72fc15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72fc0ce9d0 0x7f72fc15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc168bf0 con 0x7f72fc15fde0 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc168a70 con 0x7f72fc0ce9d0 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f72fc168d70 con 0x7f72fc0ce5f0 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc073490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49840/0 (socket says 172.21.15.97:49840) 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 -- 172.21.15.97:0/1398878979 learned_addr learned my addr 172.21.15.97:0/1398878979 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f72f3e00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72fc0ce9d0 0x7f72fc15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301000640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72fc0ce5f0 0x7f72fc15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 -- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72fc0ce5f0 msgr2=0x7f72fc15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72fc0ce5f0 0x7f72fc15e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 -- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72fc0ce9d0 msgr2=0x7f72fc15ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72fc0ce9d0 0x7f72fc15ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 -- 172.21.15.97:0/1398878979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc15d920 con 0x7f72fc15fde0 2024-09-10T21:26:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7301a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc073490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009bd0 tx=0x7f72f4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f72f9600640 1 -- 172.21.15.97:0/1398878979 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40bb070 con 0x7f72fc15fde0 2024-09-10T21:26:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f72f9600640 1 -- 172.21.15.97:0/1398878979 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72f4004b40 con 0x7f72fc15fde0 2024-09-10T21:26:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f72f9600640 1 -- 172.21.15.97:0/1398878979 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40b2040 con 0x7f72fc15fde0 2024-09-10T21:26:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc15dc00 con 0x7f72fc15fde0 2024-09-10T21:26:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc15e1c0 con 0x7f72fc15fde0 2024-09-10T21:26:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc072f70 con 0x7f72fc15fde0 2024-09-10T21:26:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.871+0000 7f72f9600640 1 -- 172.21.15.97:0/1398878979 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f72f40b5030 con 0x7f72fc15fde0 2024-09-10T21:26:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.875+0000 7f72f9600640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72d8072610 0x7f72d8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.875+0000 7f72f9600640 1 -- 172.21.15.97:0/1398878979 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f72f412cd70 con 0x7f72fc15fde0 2024-09-10T21:26:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.875+0000 7f7301000640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72d8072610 0x7f72d8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.875+0000 7f7301000640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72d8072610 0x7f72d8074ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009be0 tx=0x7f72ec0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:38.879+0000 7f72f9600640 1 -- 172.21.15.97:0/1398878979 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f72f4016990 con 0x7f72fc15fde0 2024-09-10T21:26:39.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.079+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72fc072c90 con 0x7f72fc15fde0 2024-09-10T21:26:39.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.079+0000 7f72f9600640 1 -- 172.21.15.97:0/1398878979 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5217 (secure 0 0 0) 0x7f72f40f7b80 con 0x7f72fc15fde0 2024-09-10T21:26:39.083 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:39.083 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":38,"btime":"2024-09-10T21:26:38:561354+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:37.563408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T21:26:39.084 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 38 2024-09-10T21:26:39.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72d8072610 msgr2=0x7f72d8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72d8072610 0x7f72d8074ad0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009be0 tx=0x7f72ec0091e0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 msgr2=0x7f72fc073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc073490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009bd0 tx=0x7f72f4005b00 comp rx=0 tx=0).stop 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 shutdown_connections 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72d8072610 0x7f72d8074ad0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72fc15fde0 0x7f72fc073490 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72fc0ce9d0 0x7f72fc15ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 --2- 172.21.15.97:0/1398878979 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72fc0ce5f0 0x7f72fc15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 >> 172.21.15.97:0/1398878979 conn(0x7f72fc0b1780 msgr2=0x7f72fc15c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 shutdown_connections 2024-09-10T21:26:39.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.083+0000 7f7303a00640 1 -- 172.21.15.97:0/1398878979 wait complete. 2024-09-10T21:26:39.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:39.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- 172.21.15.97:0/611736447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a41511f0 msgr2=0x7f19a40b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 --2- 172.21.15.97:0/611736447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a41511f0 0x7f19a40b37b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1998009930 tx=0x7f199801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- 172.21.15.97:0/611736447 shutdown_connections 2024-09-10T21:26:39.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 --2- 172.21.15.97:0/611736447 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19a40cf5f0 0x7f19a415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 --2- 172.21.15.97:0/611736447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a41511f0 0x7f19a40b37b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- 172.21.15.97:0/611736447 >> 172.21.15.97:0/611736447 conn(0x7f19a40b1780 msgr2=0x7f19a40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:39.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- 172.21.15.97:0/611736447 shutdown_connections 2024-09-10T21:26:39.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- 172.21.15.97:0/611736447 wait complete. 2024-09-10T21:26:39.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 Processor -- start 2024-09-10T21:26:39.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- start start 2024-09-10T21:26:39.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a40b3cf0 0x7f19a41546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19a40cf5f0 0x7f19a414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a41511f0 0x7f19a41f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19a406e530 con 0x7f19a40b3cf0 2024-09-10T21:26:39.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19a406e3b0 con 0x7f19a41511f0 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19ac400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f19a406e6b0 con 0x7f19a40cf5f0 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a40b3cf0 0x7f19a41546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a40b3cf0 0x7f19a41546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49866/0 (socket says 172.21.15.97:49866) 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 -- 172.21.15.97:0/2747574560 learned_addr learned my addr 172.21.15.97:0/2747574560 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9000640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19a40cf5f0 0x7f19a414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19aa400640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a41511f0 0x7f19a41f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 -- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19a40cf5f0 msgr2=0x7f19a414efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19a40cf5f0 0x7f19a414efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 -- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a41511f0 msgr2=0x7f19a41f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a41511f0 0x7f19a41f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 -- 172.21.15.97:0/2747574560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a4152290 con 0x7f19a40b3cf0 2024-09-10T21:26:39.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.287+0000 7f19a9a00640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a40b3cf0 0x7f19a41546b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1998009850 tx=0x7f1998005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19a1600640 1 -- 172.21.15.97:0/2747574560 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980b5040 con 0x7f19a40b3cf0 2024-09-10T21:26:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19a1600640 1 -- 172.21.15.97:0/2747574560 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19980038f0 con 0x7f19a40b3cf0 2024-09-10T21:26:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19a1600640 1 -- 172.21.15.97:0/2747574560 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998003d00 con 0x7f19a40b3cf0 2024-09-10T21:26:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a4152570 con 0x7f19a40b3cf0 2024-09-10T21:26:39.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a414f4e0 con 0x7f19a40b3cf0 2024-09-10T21:26:39.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a415a730 con 0x7f19a40b3cf0 2024-09-10T21:26:39.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19a1600640 1 -- 172.21.15.97:0/2747574560 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f19980b8070 con 0x7f19a40b3cf0 2024-09-10T21:26:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19a1600640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19840725e0 0x7f1984074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.291+0000 7f19a1600640 1 -- 172.21.15.97:0/2747574560 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f19981301f0 con 0x7f19a40b3cf0 2024-09-10T21:26:39.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.295+0000 7f19a9000640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19840725e0 0x7f1984074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.295+0000 7f19a9000640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19840725e0 0x7f1984074aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f19a40aea40 tx=0x7f1990009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:39.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.295+0000 7f19a1600640 1 -- 172.21.15.97:0/2747574560 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1998018c30 con 0x7f19a40b3cf0 2024-09-10T21:26:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.491+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19a40afcb0 con 0x7f19a40b3cf0 2024-09-10T21:26:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19a1600640 1 -- 172.21.15.97:0/2747574560 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5217 (secure 0 0 0) 0x7f19980fb6e0 con 0x7f19a40b3cf0 2024-09-10T21:26:39.497 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:39.497 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":38,"btime":"2024-09-10T21:26:38:561354+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:37.563408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T21:26:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 38 2024-09-10T21:26:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19840725e0 msgr2=0x7f1984074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19840725e0 0x7f1984074aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f19a40aea40 tx=0x7f1990009040 comp rx=0 tx=0).stop 2024-09-10T21:26:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a40b3cf0 msgr2=0x7f19a41546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a40b3cf0 0x7f19a41546b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1998009850 tx=0x7f1998005de0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 shutdown_connections 2024-09-10T21:26:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19840725e0 0x7f1984074aa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19a41511f0 0x7f19a41f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19a40cf5f0 0x7f19a414efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 --2- 172.21.15.97:0/2747574560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19a40b3cf0 0x7f19a41546b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 >> 172.21.15.97:0/2747574560 conn(0x7f19a40b1780 msgr2=0x7f19a40b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 shutdown_connections 2024-09-10T21:26:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.495+0000 7f19ac400640 1 -- 172.21.15.97:0/2747574560 wait complete. 2024-09-10T21:26:39.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:39.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- 172.21.15.97:0/3003146351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780cf420 msgr2=0x7f637815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 --2- 172.21.15.97:0/3003146351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780cf420 0x7f637815a730 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6378074f10 tx=0x7f637001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- 172.21.15.97:0/3003146351 shutdown_connections 2024-09-10T21:26:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 --2- 172.21.15.97:0/3003146351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780cf420 0x7f637815a730 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 --2- 172.21.15.97:0/3003146351 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63780b3cf0 0x7f63780ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 --2- 172.21.15.97:0/3003146351 >> v2:172.21.15.194:3300/0 conn(0x7f63781511f0 0x7f63780b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- 172.21.15.97:0/3003146351 >> 172.21.15.97:0/3003146351 conn(0x7f63780b1780 msgr2=0x7f63780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- 172.21.15.97:0/3003146351 shutdown_connections 2024-09-10T21:26:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- 172.21.15.97:0/3003146351 wait complete. 2024-09-10T21:26:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 Processor -- start 2024-09-10T21:26:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- start start 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780b3cf0 0x7f63781f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63780cf420 0x7f6378154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63781511f0 0x7f6378151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f637815dd20 con 0x7f63780b3cf0 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f637815b150 con 0x7f63781511f0 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.663+0000 7f637dc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f637815dea0 con 0x7f63780cf420 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780b3cf0 0x7f63781f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780b3cf0 0x7f63781f40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49888/0 (socket says 172.21.15.97:49888) 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 -- 172.21.15.97:0/1219166922 learned_addr learned my addr 172.21.15.97:0/1219166922 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376000640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63780cf420 0x7f6378154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6377400640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63781511f0 0x7f6378151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 -- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63780cf420 msgr2=0x7f6378154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63780cf420 0x7f6378154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 -- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63781511f0 msgr2=0x7f6378151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63781511f0 0x7f6378151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 -- 172.21.15.97:0/1219166922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63781f45e0 con 0x7f63780b3cf0 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f6376a00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780b3cf0 0x7f63781f40a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f636800eaa0 tx=0x7f636800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f636ea00640 1 -- 172.21.15.97:0/1219166922 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800ce10 con 0x7f63780b3cf0 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f636ea00640 1 -- 172.21.15.97:0/1219166922 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6368011470 con 0x7f63780b3cf0 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f636ea00640 1 -- 172.21.15.97:0/1219166922 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680118a0 con 0x7f63780b3cf0 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6378154aa0 con 0x7f63780b3cf0 2024-09-10T21:26:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6378153a80 con 0x7f63780b3cf0 2024-09-10T21:26:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6378154190 con 0x7f63780b3cf0 2024-09-10T21:26:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.667+0000 7f636ea00640 1 -- 172.21.15.97:0/1219166922 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f636800f040 con 0x7f63780b3cf0 2024-09-10T21:26:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.671+0000 7f636ea00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63580725e0 0x7f6358074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.671+0000 7f636ea00640 1 -- 172.21.15.97:0/1219166922 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f6368093b20 con 0x7f63780b3cf0 2024-09-10T21:26:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.671+0000 7f6376000640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63580725e0 0x7f6358074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:39.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.671+0000 7f6376000640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63580725e0 0x7f6358074aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6378142530 tx=0x7f636400b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:39.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.671+0000 7f636ea00640 1 -- 172.21.15.97:0/1219166922 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f63781f45e0 con 0x7f63780b3cf0 2024-09-10T21:26:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.839+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f637815a730 con 0x7f63780b3cf0 2024-09-10T21:26:39.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.839+0000 7f636ea00640 1 -- 172.21.15.97:0/1219166922 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+17095 (secure 0 0 0) 0x7f6368016030 con 0x7f63780b3cf0 2024-09-10T21:26:39.843 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:39.843 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:34.976781+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:12.586949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:13.866309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6834/3721865460":"2024-09-11T21:26:34.976757+0000","172.21.15.97:6832/3721865460":"2024-09-11T21:26:34.976757+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63580725e0 msgr2=0x7f6358074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63580725e0 0x7f6358074aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6378142530 tx=0x7f636400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780b3cf0 msgr2=0x7f63781f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780b3cf0 0x7f63781f40a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f636800eaa0 tx=0x7f636800ef60 comp rx=0 tx=0).stop 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 shutdown_connections 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63580725e0 0x7f6358074aa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63781511f0 0x7f6378151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63780cf420 0x7f6378154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 --2- 172.21.15.97:0/1219166922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63780b3cf0 0x7f63781f40a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 >> 172.21.15.97:0/1219166922 conn(0x7f63780b1780 msgr2=0x7f637815c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 shutdown_connections 2024-09-10T21:26:39.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:39.843+0000 7f637dc00640 1 -- 172.21.15.97:0/1219166922 wait complete. 2024-09-10T21:26:39.864 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:26:39.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:40.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 -- 172.21.15.97:0/246464384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 msgr2=0x7f20f81600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 --2- 172.21.15.97:0/246464384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f81600d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f20f80b12d0 tx=0x7f20f001cac0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 -- 172.21.15.97:0/246464384 shutdown_connections 2024-09-10T21:26:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 --2- 172.21.15.97:0/246464384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f81600d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 --2- 172.21.15.97:0/246464384 >> v2:172.21.15.194:3300/0 conn(0x7f20f80722c0 0x7f20f80d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 -- 172.21.15.97:0/246464384 >> 172.21.15.97:0/246464384 conn(0x7f20f80d1a70 msgr2=0x7f20f80d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 -- 172.21.15.97:0/246464384 shutdown_connections 2024-09-10T21:26:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.003+0000 7f20fde00640 1 -- 172.21.15.97:0/246464384 wait complete. 2024-09-10T21:26:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 Processor -- start 2024-09-10T21:26:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 -- start start 2024-09-10T21:26:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80722c0 0x7f20f80cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20f80cfbd0 0x7f20f815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f815c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80b0a90 con 0x7f20f80cffb0 2024-09-10T21:26:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80b0910 con 0x7f20f80722c0 2024-09-10T21:26:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f20f80b0c10 con 0x7f20f80cfbd0 2024-09-10T21:26:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f815c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49900/0 (socket says 172.21.15.97:49900) 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 -- 172.21.15.97:0/3181437967 learned_addr learned my addr 172.21.15.97:0/3181437967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20f7e00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20f80cfbd0 0x7f20f815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fca00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80722c0 0x7f20f80cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 -- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20f80cfbd0 msgr2=0x7f20f815c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20f80cfbd0 0x7f20f815c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 -- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80722c0 msgr2=0x7f20f80cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80722c0 0x7f20f80cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 -- 172.21.15.97:0/3181437967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f815e500 con 0x7f20f80cffb0 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fd400640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f815c990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009b10 tx=0x7f20f0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20f4c00640 1 -- 172.21.15.97:0/3181437967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00bc070 con 0x7f20f80cffb0 2024-09-10T21:26:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20f4c00640 1 -- 172.21.15.97:0/3181437967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20f00a5e50 con 0x7f20f80cffb0 2024-09-10T21:26:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20f4c00640 1 -- 172.21.15.97:0/3181437967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f00b2040 con 0x7f20f80cffb0 2024-09-10T21:26:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f815e7a0 con 0x7f20f80cffb0 2024-09-10T21:26:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.007+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f815ed60 con 0x7f20f80cffb0 2024-09-10T21:26:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.011+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f80d2590 con 0x7f20f80cffb0 2024-09-10T21:26:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.011+0000 7f20f4c00640 1 -- 172.21.15.97:0/3181437967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f20f00b5070 con 0x7f20f80cffb0 2024-09-10T21:26:40.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.011+0000 7f20f4c00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20d80725e0 0x7f20d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.011+0000 7f20f4c00640 1 -- 172.21.15.97:0/3181437967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f20f012ce30 con 0x7f20f80cffb0 2024-09-10T21:26:40.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.011+0000 7f20fca00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20d80725e0 0x7f20d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.011+0000 7f20fca00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20d80725e0 0x7f20d8074aa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f20ec007fd0 tx=0x7f20ec007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:40.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.015+0000 7f20f4c00640 1 -- 172.21.15.97:0/3181437967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f20f0016770 con 0x7f20f80cffb0 2024-09-10T21:26:40.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.215+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20f81652d0 con 0x7f20f80cffb0 2024-09-10T21:26:40.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.215+0000 7f20f4c00640 1 -- 172.21.15.97:0/3181437967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5224 (secure 0 0 0) 0x7f20f00f7c40 con 0x7f20f80cffb0 2024-09-10T21:26:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:40.218 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":39,"btime":"2024-09-10T21:26:39:563007+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:39.563002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-09-10T21:26:40.218 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 39 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.215+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20d80725e0 msgr2=0x7f20d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.215+0000 7f20fde00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20d80725e0 0x7f20d8074aa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f20ec007fd0 tx=0x7f20ec007ea0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.215+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 msgr2=0x7f20f815c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.215+0000 7f20fde00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f815c990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009b10 tx=0x7f20f0009310 comp rx=0 tx=0).stop 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 shutdown_connections 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20d80725e0 0x7f20d8074aa0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cffb0 0x7f20f815c990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20f80cfbd0 0x7f20f815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 --2- 172.21.15.97:0/3181437967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80722c0 0x7f20f80cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 >> 172.21.15.97:0/3181437967 conn(0x7f20f80d1a70 msgr2=0x7f20f80cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 shutdown_connections 2024-09-10T21:26:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.219+0000 7f20fde00640 1 -- 172.21.15.97:0/3181437967 wait complete. 2024-09-10T21:26:40.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- 172.21.15.97:0/285132652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57681511f0 msgr2=0x7f57680b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 --2- 172.21.15.97:0/285132652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57681511f0 0x7f57680b37b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f57680cc590 tx=0x7f575801f580 comp rx=0 tx=0).stop 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- 172.21.15.97:0/285132652 shutdown_connections 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 --2- 172.21.15.97:0/285132652 >> v2:172.21.15.194:3300/0 conn(0x7f576806f180 0x7f57680ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 --2- 172.21.15.97:0/285132652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57680b3cf0 0x7f576806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 --2- 172.21.15.97:0/285132652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57681511f0 0x7f57680b37b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- 172.21.15.97:0/285132652 >> 172.21.15.97:0/285132652 conn(0x7f57680b1780 msgr2=0x7f57680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- 172.21.15.97:0/285132652 shutdown_connections 2024-09-10T21:26:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- 172.21.15.97:0/285132652 wait complete. 2024-09-10T21:26:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 Processor -- start 2024-09-10T21:26:40.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- start start 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f576806f180 0x7f57680cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680b3cf0 0x7f57681543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57681511f0 0x7f57681f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f576806e530 con 0x7f57680b3cf0 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f576806e3b0 con 0x7f57681511f0 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f576e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f576806e6b0 con 0x7f576806f180 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680b3cf0 0x7f57681543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680b3cf0 0x7f57681543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49922/0 (socket says 172.21.15.97:49922) 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 -- 172.21.15.97:0/3349892167 learned_addr learned my addr 172.21.15.97:0/3349892167 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5767400640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f576806f180 0x7f57680cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5767e00640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57681511f0 0x7f57681f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 -- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f576806f180 msgr2=0x7f57680cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f576806f180 0x7f57680cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 -- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57681511f0 msgr2=0x7f57681f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57681511f0 0x7f57681f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.379+0000 7f5766a00640 1 -- 172.21.15.97:0/3349892167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5768152af0 con 0x7f57680b3cf0 2024-09-10T21:26:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f5766a00640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680b3cf0 0x7f57681543b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f575400c970 tx=0x7f575400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f575f400640 1 -- 172.21.15.97:0/3349892167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754007bb0 con 0x7f57680b3cf0 2024-09-10T21:26:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f575f400640 1 -- 172.21.15.97:0/3349892167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5754005020 con 0x7f57680b3cf0 2024-09-10T21:26:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f575f400640 1 -- 172.21.15.97:0/3349892167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754007e60 con 0x7f57680b3cf0 2024-09-10T21:26:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5768152e30 con 0x7f57680b3cf0 2024-09-10T21:26:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5768153670 con 0x7f57680b3cf0 2024-09-10T21:26:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57680ceee0 con 0x7f57680b3cf0 2024-09-10T21:26:40.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f575f400640 1 -- 172.21.15.97:0/3349892167 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5754005470 con 0x7f57680b3cf0 2024-09-10T21:26:40.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f575f400640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57480725e0 0x7f5748074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f575f400640 1 -- 172.21.15.97:0/3349892167 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7f57540938d0 con 0x7f57680b3cf0 2024-09-10T21:26:40.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.383+0000 7f5767400640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57480725e0 0x7f5748074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.387+0000 7f5767400640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57480725e0 0x7f5748074aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57580b2000 tx=0x7f575801ddc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.387+0000 7f575f400640 1 -- 172.21.15.97:0/3349892167 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5768152e30 con 0x7f57680b3cf0 2024-09-10T21:26:40.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.587+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57680b4240 con 0x7f57680b3cf0 2024-09-10T21:26:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.587+0000 7f575f400640 1 -- 172.21.15.97:0/3349892167 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5224 (secure 0 0 0) 0x7f5754014030 con 0x7f57680b3cf0 2024-09-10T21:26:40.591 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:40.591 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":39,"btime":"2024-09-10T21:26:39:563007+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:14.903489+0000","modified":"2024-09-10T21:26:39.563002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6834/2218970160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2218970160},{"type":"v1","addr":"172.21.15.97:6834","nonce":2218970160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-09-10T21:26:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 39 2024-09-10T21:26:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57480725e0 msgr2=0x7f5748074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57480725e0 0x7f5748074aa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57580b2000 tx=0x7f575801ddc0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680b3cf0 msgr2=0x7f57681543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680b3cf0 0x7f57681543b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f575400c970 tx=0x7f575400ce30 comp rx=0 tx=0).stop 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 shutdown_connections 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57480725e0 0x7f5748074aa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57681511f0 0x7f57681f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57680b3cf0 0x7f57681543b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 --2- 172.21.15.97:0/3349892167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f576806f180 0x7f57680cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 >> 172.21.15.97:0/3349892167 conn(0x7f57680b1780 msgr2=0x7f576815cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 shutdown_connections 2024-09-10T21:26:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.591+0000 7f576e200640 1 -- 172.21.15.97:0/3349892167 wait complete. 2024-09-10T21:26:40.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:40.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 -- 172.21.15.97:0/28589707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 msgr2=0x7fdb80166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 --2- 172.21.15.97:0/28589707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80166850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009900 tx=0x7fdb7801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 -- 172.21.15.97:0/28589707 shutdown_connections 2024-09-10T21:26:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 --2- 172.21.15.97:0/28589707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80166850 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 -- 172.21.15.97:0/28589707 >> 172.21.15.97:0/28589707 conn(0x7fdb800b1780 msgr2=0x7fdb800b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 -- 172.21.15.97:0/28589707 shutdown_connections 2024-09-10T21:26:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 -- 172.21.15.97:0/28589707 wait complete. 2024-09-10T21:26:40.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 Processor -- start 2024-09-10T21:26:40.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 -- start start 2024-09-10T21:26:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.751+0000 7fdb85600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb800ce5f0 0x7fdb800b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb80141d20 0x7fdb800b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8016a130 con 0x7fdb8015fde0 2024-09-10T21:26:40.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80169fb0 con 0x7fdb80141d20 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8016a2b0 con 0x7fdb800ce5f0 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80072630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49938/0 (socket says 172.21.15.97:49938) 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 -- 172.21.15.97:0/2605205573 learned_addr learned my addr 172.21.15.97:0/2605205573 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7ea00640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb800ce5f0 0x7fdb800b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb76000640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb80141d20 0x7fdb800b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 -- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb800ce5f0 msgr2=0x7fdb800b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb800ce5f0 0x7fdb800b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 -- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb80141d20 msgr2=0x7fdb800b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb80141d20 0x7fdb800b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 -- 172.21.15.97:0/2605205573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb801f7c50 con 0x7fdb8015fde0 2024-09-10T21:26:40.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb7f400640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80072630 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b0040 tx=0x7fdb780a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:40.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb77400640 1 -- 172.21.15.97:0/2605205573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780bd070 con 0x7fdb8015fde0 2024-09-10T21:26:40.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb77400640 1 -- 172.21.15.97:0/2605205573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb78004430 con 0x7fdb8015fde0 2024-09-10T21:26:40.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb801f7ec0 con 0x7fdb8015fde0 2024-09-10T21:26:40.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb77400640 1 -- 172.21.15.97:0/2605205573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780b2040 con 0x7fdb8015fde0 2024-09-10T21:26:40.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801f8480 con 0x7fdb8015fde0 2024-09-10T21:26:40.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb800ce9d0 con 0x7fdb8015fde0 2024-09-10T21:26:40.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.755+0000 7fdb77400640 1 -- 172.21.15.97:0/2605205573 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb78003860 con 0x7fdb8015fde0 2024-09-10T21:26:40.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.759+0000 7fdb77400640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb5c0725e0 0x7fdb5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:40.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.759+0000 7fdb77400640 1 -- 172.21.15.97:0/2605205573 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7fdb7812d1c0 con 0x7fdb8015fde0 2024-09-10T21:26:40.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.759+0000 7fdb7ea00640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb5c0725e0 0x7fdb5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:40.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.759+0000 7fdb7ea00640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb5c0725e0 0x7fdb5c074aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb700012f0 tx=0x7fdb70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:40.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.763+0000 7fdb77400640 1 -- 172.21.15.97:0/2605205573 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb780164d0 con 0x7fdb8015fde0 2024-09-10T21:26:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.927+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb801f8650 con 0x7fdb8015fde0 2024-09-10T21:26:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.927+0000 7fdb77400640 1 -- 172.21.15.97:0/2605205573 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+17095 (secure 0 0 0) 0x7fdb780f7fe0 con 0x7fdb8015fde0 2024-09-10T21:26:40.930 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:40.931 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:34.976781+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:12.586949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:13.866309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6834/3721865460":"2024-09-11T21:26:34.976757+0000","172.21.15.97:6832/3721865460":"2024-09-11T21:26:34.976757+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb5c0725e0 msgr2=0x7fdb5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb5c0725e0 0x7fdb5c074aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb700012f0 tx=0x7fdb70009040 comp rx=0 tx=0).stop 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 msgr2=0x7fdb80072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80072630 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdb780b0040 tx=0x7fdb780a5770 comp rx=0 tx=0).stop 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 shutdown_connections 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb5c0725e0 0x7fdb5c074aa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb8015fde0 0x7fdb80072630 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb80141d20 0x7fdb800b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 --2- 172.21.15.97:0/2605205573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb800ce5f0 0x7fdb800b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 >> 172.21.15.97:0/2605205573 conn(0x7fdb800b1780 msgr2=0x7fdb8013ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 shutdown_connections 2024-09-10T21:26:40.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:40.931+0000 7fdb85600640 1 -- 172.21.15.97:0/2605205573 wait complete. 2024-09-10T21:26:40.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:26:40.988 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:26:40.983+0000 7efd41400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:26:41.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- 172.21.15.97:0/3842788357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb415a730 msgr2=0x7fbeb415abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3842788357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb415a730 0x7fbeb415abc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009c80 tx=0x7fbeac01f5a0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- 172.21.15.97:0/3842788357 shutdown_connections 2024-09-10T21:26:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3842788357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb415a730 0x7fbeb415abc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3842788357 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbeb40ceee0 0x7fbeb40cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- 172.21.15.97:0/3842788357 >> 172.21.15.97:0/3842788357 conn(0x7fbeb40b1780 msgr2=0x7fbeb40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- 172.21.15.97:0/3842788357 shutdown_connections 2024-09-10T21:26:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- 172.21.15.97:0/3842788357 wait complete. 2024-09-10T21:26:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 Processor -- start 2024-09-10T21:26:41.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- start start 2024-09-10T21:26:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb40ceee0 0x7fbeb4147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbeb41511f0 0x7fbeb41f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbeb415a730 0x7fbeb4148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb406e530 con 0x7fbeb40ceee0 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb406e3b0 con 0x7fbeb41511f0 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb9000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb406e6b0 con 0x7fbeb415a730 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb40ceee0 0x7fbeb4147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb40ceee0 0x7fbeb4147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49958/0 (socket says 172.21.15.97:49958) 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 -- 172.21.15.97:0/3378989912 learned_addr learned my addr 172.21.15.97:0/3378989912 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb2800640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbeb415a730 0x7fbeb4148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbea9400640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbeb41511f0 0x7fbeb41f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 -- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbeb415a730 msgr2=0x7fbeb4148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbeb415a730 0x7fbeb4148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 -- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbeb41511f0 msgr2=0x7fbeb41f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbeb41511f0 0x7fbeb41f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb1e00640 1 -- 172.21.15.97:0/3378989912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb4159210 con 0x7fbeb40ceee0 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.095+0000 7fbeb2800640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbeb415a730 0x7fbeb4148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeb1e00640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb40ceee0 0x7fbeb4147510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbea400c490 tx=0x7fbea4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeaaa00640 1 -- 172.21.15.97:0/3378989912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4017070 con 0x7fbeb40ceee0 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeaaa00640 1 -- 172.21.15.97:0/3378989912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbea4003600 con 0x7fbeb40ceee0 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41594f0 con 0x7fbeb40ceee0 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeaaa00640 1 -- 172.21.15.97:0/3378989912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003a50 con 0x7fbeb40ceee0 2024-09-10T21:26:41.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb4159ab0 con 0x7fbeb40ceee0 2024-09-10T21:26:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb40cf300 con 0x7fbeb40ceee0 2024-09-10T21:26:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeaaa00640 1 -- 172.21.15.97:0/3378989912 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbea4003bf0 con 0x7fbeb40ceee0 2024-09-10T21:26:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeaaa00640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe90072610 0x7fbe90074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbeaaa00640 1 -- 172.21.15.97:0/3378989912 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6187+0+0 (secure 0 0 0) 0x7fbea4092990 con 0x7fbeb40ceee0 2024-09-10T21:26:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.099+0000 7fbea9400640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe90072610 0x7fbe90074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.103+0000 7fbea9400640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe90072610 0x7fbe90074ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0058d0 tx=0x7fbe9c005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.103+0000 7fbeaaa00640 1 -- 172.21.15.97:0/3378989912 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbea405df20 con 0x7fbeb40ceee0 2024-09-10T21:26:41.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.303+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbeb414fb60 con 0x7fbeb40ceee0 2024-09-10T21:26:41.306 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:26:41.303+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:26:41.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.307+0000 7fbeaaa00640 1 -- 172.21.15.97:0/3378989912 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7fbea405d820 con 0x7fbeb40ceee0 2024-09-10T21:26:41.310 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:26:41.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe90072610 msgr2=0x7fbe90074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe90072610 0x7fbe90074ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0058d0 tx=0x7fbe9c005860 comp rx=0 tx=0).stop 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb40ceee0 msgr2=0x7fbeb4147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb40ceee0 0x7fbeb4147510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbea400c490 tx=0x7fbea4001e80 comp rx=0 tx=0).stop 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 shutdown_connections 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe90072610 0x7fbe90074ad0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbeb415a730 0x7fbeb4148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbeb41511f0 0x7fbeb41f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 --2- 172.21.15.97:0/3378989912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb40ceee0 0x7fbeb4147510 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 >> 172.21.15.97:0/3378989912 conn(0x7fbeb40b1780 msgr2=0x7fbeb415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 shutdown_connections 2024-09-10T21:26:41.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.311+0000 7fbeb9000640 1 -- 172.21.15.97:0/3378989912 wait complete. 2024-09-10T21:26:41.317 INFO:tasks.ceph.mds.a.smithi097.stderr: -14> 2024-09-10T21:26:40.983+0000 7efd41400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:26:41.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:26:41.356 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- 172.21.15.97:0/3718861249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015e750 msgr2=0x7f05b00cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 --2- 172.21.15.97:0/3718861249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015e750 0x7f05b00cd580 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f05b00d0a10 tx=0x7f05a801c850 comp rx=0 tx=0).stop 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- 172.21.15.97:0/3718861249 shutdown_connections 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 --2- 172.21.15.97:0/3718861249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015e750 0x7f05b00cd580 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 --2- 172.21.15.97:0/3718861249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05b015f150 0x7f05b00b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- 172.21.15.97:0/3718861249 >> 172.21.15.97:0/3718861249 conn(0x7f05b00b28c0 msgr2=0x7f05b00b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- 172.21.15.97:0/3718861249 shutdown_connections 2024-09-10T21:26:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- 172.21.15.97:0/3718861249 wait complete. 2024-09-10T21:26:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 Processor -- start 2024-09-10T21:26:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- start start 2024-09-10T21:26:41.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05b015e750 0x7f05b00aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015f150 0x7f05b00b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05b0070b90 0x7f05b0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f05b0168440 con 0x7f05b015f150 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f05b01682c0 con 0x7f05b0070b90 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05b6800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f05b01685c0 con 0x7f05b015e750 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05a6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015f150 0x7f05b00b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05a6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015f150 0x7f05b00b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49992/0 (socket says 172.21.15.97:49992) 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05a6a00640 1 -- 172.21.15.97:0/439001745 learned_addr learned my addr 172.21.15.97:0/439001745 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05af400640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05b015e750 0x7f05b00aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.499+0000 7f05afe00640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05b0070b90 0x7f05b0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a6a00640 1 -- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05b015e750 msgr2=0x7f05b00aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a6a00640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05b015e750 0x7f05b00aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a6a00640 1 -- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05b0070b90 msgr2=0x7f05b0071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a6a00640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05b0070b90 0x7f05b0071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a6a00640 1 -- 172.21.15.97:0/439001745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b01f9340 con 0x7f05b015f150 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a6a00640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015f150 0x7f05b00b1d40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f059800c410 tx=0x7f059800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a7e00640 1 -- 172.21.15.97:0/439001745 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598011790 con 0x7f05b015f150 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a7e00640 1 -- 172.21.15.97:0/439001745 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0598017070 con 0x7f05b015f150 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b01f9060 con 0x7f05b015f150 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a7e00640 1 -- 172.21.15.97:0/439001745 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598011d00 con 0x7f05b015f150 2024-09-10T21:26:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b00ae5c0 con 0x7f05b015f150 2024-09-10T21:26:41.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b00cd580 con 0x7f05b015f150 2024-09-10T21:26:41.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a7e00640 1 -- 172.21.15.97:0/439001745 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f059801c460 con 0x7f05b015f150 2024-09-10T21:26:41.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a7e00640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f058c0725e0 0x7f058c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:41.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.503+0000 7f05a7e00640 1 -- 172.21.15.97:0/439001745 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6273+0+0 (secure 0 0 0) 0x7f0598092fe0 con 0x7f05b015f150 2024-09-10T21:26:41.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.507+0000 7f05af400640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f058c0725e0 0x7f058c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:41.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.507+0000 7f05af400640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f058c0725e0 0x7f058c074aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05a0005fd0 tx=0x7f05a000a590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:41.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.507+0000 7f05a7e00640 1 -- 172.21.15.97:0/439001745 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0598021020 con 0x7f05b015f150 2024-09-10T21:26:41.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:41.707+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f05b00ce7b0 con 0x7f05b015f150 2024-09-10T21:26:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05a7e00640 1 -- 172.21.15.97:0/439001745 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7f0598003810 con 0x7f05b015f150 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f058c0725e0 msgr2=0x7f058c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f058c0725e0 0x7f058c074aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05a0005fd0 tx=0x7f05a000a590 comp rx=0 tx=0).stop 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015f150 msgr2=0x7f05b00b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015f150 0x7f05b00b1d40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f059800c410 tx=0x7f059800c8d0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 shutdown_connections 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f058c0725e0 0x7f058c074aa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05b0070b90 0x7f05b0071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05b015f150 0x7f05b00b1d40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 --2- 172.21.15.97:0/439001745 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05b015e750 0x7f05b00aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 >> 172.21.15.97:0/439001745 conn(0x7f05b00b28c0 msgr2=0x7f05b015cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 shutdown_connections 2024-09-10T21:26:42.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.311+0000 7f05b6800640 1 -- 172.21.15.97:0/439001745 wait complete. 2024-09-10T21:26:42.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 -- 172.21.15.97:0/3982934332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a200ce5f0 msgr2=0x7f6a20141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 --2- 172.21.15.97:0/3982934332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a200ce5f0 0x7f6a20141d20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009900 tx=0x7f6a1001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 -- 172.21.15.97:0/3982934332 shutdown_connections 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 --2- 172.21.15.97:0/3982934332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a200ce9d0 0x7f6a2015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 --2- 172.21.15.97:0/3982934332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a200ce5f0 0x7f6a20141d20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 -- 172.21.15.97:0/3982934332 >> 172.21.15.97:0/3982934332 conn(0x7f6a200b1780 msgr2=0x7f6a200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 -- 172.21.15.97:0/3982934332 shutdown_connections 2024-09-10T21:26:42.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.479+0000 7f6a27600640 1 -- 172.21.15.97:0/3982934332 wait complete. 2024-09-10T21:26:42.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 Processor -- start 2024-09-10T21:26:42.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 -- start start 2024-09-10T21:26:42.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a200ce5f0 0x7f6a2013ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:42.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a200ce9d0 0x7f6a2006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a2015fc10 0x7f6a201fb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20169020 con 0x7f6a2015fc10 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20168ea0 con 0x7f6a200ce5f0 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6a201691a0 con 0x7f6a200ce9d0 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a2015fc10 0x7f6a201fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a2015fc10 0x7f6a201fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50006/0 (socket says 172.21.15.97:50006) 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 -- 172.21.15.97:0/1361601133 learned_addr learned my addr 172.21.15.97:0/1361601133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a24c00640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a200ce5f0 0x7f6a2013ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a17e00640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a200ce9d0 0x7f6a2006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 -- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a200ce9d0 msgr2=0x7f6a2006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a200ce9d0 0x7f6a2006f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 -- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a200ce5f0 msgr2=0x7f6a2013ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:42.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a200ce5f0 0x7f6a2013ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 -- 172.21.15.97:0/1361601133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2013b140 con 0x7f6a2015fc10 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a17e00640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a200ce9d0 0x7f6a2006f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a25600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a2015fc10 0x7f6a201fb660 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c760 tx=0x7f6a1800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a1d600640 1 -- 172.21.15.97:0/1361601133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18003db0 con 0x7f6a2015fc10 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a1d600640 1 -- 172.21.15.97:0/1361601133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a1800ce10 con 0x7f6a2015fc10 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2006fc80 con 0x7f6a2015fc10 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a1d600640 1 -- 172.21.15.97:0/1361601133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180052d0 con 0x7f6a2015fc10 2024-09-10T21:26:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a20070240 con 0x7f6a2015fc10 2024-09-10T21:26:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.483+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20141d20 con 0x7f6a2015fc10 2024-09-10T21:26:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.487+0000 7f6a1d600640 1 -- 172.21.15.97:0/1361601133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6a18005470 con 0x7f6a2015fc10 2024-09-10T21:26:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.487+0000 7f6a1d600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69fc0725e0 0x7f69fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.487+0000 7f6a1d600640 1 -- 172.21.15.97:0/1361601133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6273+0+0 (secure 0 0 0) 0x7f6a180931e0 con 0x7f6a2015fc10 2024-09-10T21:26:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.487+0000 7f6a24c00640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69fc0725e0 0x7f69fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.487+0000 7f6a24c00640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69fc0725e0 0x7f69fc074aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6a100af000 tx=0x7f6a1001b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:42.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.491+0000 7f6a1d600640 1 -- 172.21.15.97:0/1361601133 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6a1805e5e0 con 0x7f6a2015fc10 2024-09-10T21:26:42.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:42.659+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a20141360 con 0x7f6a2015fc10 2024-09-10T21:26:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.311+0000 7f6a1d600640 1 -- 172.21.15.97:0/1361601133 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) ==== 181+0+0 (secure 0 0 0) 0x7f6a1805dee0 con 0x7f6a2015fc10 2024-09-10T21:26:43.314 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:26:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69fc0725e0 msgr2=0x7f69fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69fc0725e0 0x7f69fc074aa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6a100af000 tx=0x7f6a1001b120 comp rx=0 tx=0).stop 2024-09-10T21:26:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a2015fc10 msgr2=0x7f6a201fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a2015fc10 0x7f6a201fb660 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c760 tx=0x7f6a1800cc20 comp rx=0 tx=0).stop 2024-09-10T21:26:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 shutdown_connections 2024-09-10T21:26:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69fc0725e0 0x7f69fc074aa0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a2015fc10 0x7f6a201fb660 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a200ce9d0 0x7f6a2006f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 --2- 172.21.15.97:0/1361601133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a200ce5f0 0x7f6a2013ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 >> 172.21.15.97:0/1361601133 conn(0x7f6a200b1780 msgr2=0x7f6a2015f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 shutdown_connections 2024-09-10T21:26:43.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.315+0000 7f6a27600640 1 -- 172.21.15.97:0/1361601133 wait complete. 2024-09-10T21:26:43.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:26:43.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- 172.21.15.97:0/448940025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 msgr2=0x7fe7d813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:43.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/448940025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d813f9b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8074800 tx=0x7fe7d001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- 172.21.15.97:0/448940025 shutdown_connections 2024-09-10T21:26:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/448940025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d813f9b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/448940025 >> v2:172.21.15.194:3300/0 conn(0x7fe7d8141d20 0x7fe7d815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- 172.21.15.97:0/448940025 >> 172.21.15.97:0/448940025 conn(0x7fe7d80b1780 msgr2=0x7fe7d80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- 172.21.15.97:0/448940025 shutdown_connections 2024-09-10T21:26:43.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- 172.21.15.97:0/448940025 wait complete. 2024-09-10T21:26:43.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 Processor -- start 2024-09-10T21:26:43.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- start start 2024-09-10T21:26:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7d80ce5f0 0x7fe7d80b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7d8141d20 0x7fe7d815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8168a70 con 0x7fe7d815fc10 2024-09-10T21:26:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8140cd0 con 0x7fe7d80ce5f0 2024-09-10T21:26:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.487+0000 7fe7ddc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d8168bf0 con 0x7fe7d8141d20 2024-09-10T21:26:43.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:43.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d806f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50024/0 (socket says 172.21.15.97:50024) 2024-09-10T21:26:43.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 -- 172.21.15.97:0/3704754240 learned_addr learned my addr 172.21.15.97:0/3704754240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 -- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7d8141d20 msgr2=0x7fe7d815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7d8141d20 0x7fe7d815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 -- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7d80ce5f0 msgr2=0x7fe7d80b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d6a00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7d80ce5f0 0x7fe7d80b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7d80ce5f0 0x7fe7d80b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 -- 172.21.15.97:0/3704754240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d80b3630 con 0x7fe7d815fc10 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d6a00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7d80ce5f0 0x7fe7d80b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7d7400640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d806f140 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe7d001af50 tx=0x7fe7d00a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7cf400640 1 -- 172.21.15.97:0/3704754240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00b2040 con 0x7fe7d815fc10 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7cf400640 1 -- 172.21.15.97:0/3704754240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7d0004430 con 0x7fe7d815fc10 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7cf400640 1 -- 172.21.15.97:0/3704754240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0003860 con 0x7fe7d815fc10 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d80b38a0 con 0x7fe7d815fc10 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d815d2b0 con 0x7fe7d815fc10 2024-09-10T21:26:43.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d815f6d0 con 0x7fe7d815fc10 2024-09-10T21:26:43.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.491+0000 7fe7cf400640 1 -- 172.21.15.97:0/3704754240 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe7d0003a00 con 0x7fe7d815fc10 2024-09-10T21:26:43.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.495+0000 7fe7cf400640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7b40725e0 0x7fe7b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:43.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.495+0000 7fe7cf400640 1 -- 172.21.15.97:0/3704754240 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5835+0+0 (secure 0 0 0) 0x7fe7d012d4e0 con 0x7fe7d815fc10 2024-09-10T21:26:43.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.495+0000 7fe7d6a00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7b40725e0 0x7fe7b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.495+0000 7fe7d6a00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7b40725e0 0x7fe7b4074aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8007fd0 tx=0x7fe7c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:43.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.499+0000 7fe7cf400640 1 -- 172.21.15.97:0/3704754240 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe7d0016490 con 0x7fe7d815fc10 2024-09-10T21:26:43.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:43.663+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7d813c600 con 0x7fe7d815fc10 2024-09-10T21:26:44.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7cf400640 1 -- 172.21.15.97:0/3704754240 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) ==== 169+0+0 (secure 0 0 0) 0x7fe7d00f8aa0 con 0x7fe7d815fc10 2024-09-10T21:26:44.320 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7b40725e0 msgr2=0x7fe7b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7b40725e0 0x7fe7b4074aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8007fd0 tx=0x7fe7c8009040 comp rx=0 tx=0).stop 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 msgr2=0x7fe7d806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d806f140 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe7d001af50 tx=0x7fe7d00a56c0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 shutdown_connections 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7b40725e0 0x7fe7b4074aa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7d815fc10 0x7fe7d806f140 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7d8141d20 0x7fe7d815ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 --2- 172.21.15.97:0/3704754240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7d80ce5f0 0x7fe7d80b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 >> 172.21.15.97:0/3704754240 conn(0x7fe7d80b1780 msgr2=0x7fe7d80ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:44.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 shutdown_connections 2024-09-10T21:26:44.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.319+0000 7fe7ddc00640 1 -- 172.21.15.97:0/3704754240 wait complete. 2024-09-10T21:26:44.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:26:44.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 -- 172.21.15.97:0/2938899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e4160220 msgr2=0x7f57e40ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 --2- 172.21.15.97:0/2938899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e4160220 0x7f57e40ce320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f57e4069990 tx=0x7f57d001c7b0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 -- 172.21.15.97:0/2938899 shutdown_connections 2024-09-10T21:26:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 --2- 172.21.15.97:0/2938899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e4160220 0x7f57e40ce320 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 --2- 172.21.15.97:0/2938899 >> v2:172.21.15.194:3300/0 conn(0x7f57e415f120 0x7f57e40af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 -- 172.21.15.97:0/2938899 >> 172.21.15.97:0/2938899 conn(0x7f57e40b3c70 msgr2=0x7f57e40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 -- 172.21.15.97:0/2938899 shutdown_connections 2024-09-10T21:26:44.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 -- 172.21.15.97:0/2938899 wait complete. 2024-09-10T21:26:44.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 Processor -- start 2024-09-10T21:26:44.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.495+0000 7f57ec600640 1 -- start start 2024-09-10T21:26:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e406cb40 0x7f57e41f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57e415f120 0x7f57e41494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57e4160220 0x7f57e414ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f57e40cea60 con 0x7f57e406cb40 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f57e40d0a90 con 0x7f57e4160220 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f57e406cf50 con 0x7f57e415f120 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e406cb40 0x7f57e41f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e406cb40 0x7f57e41f8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50036/0 (socket says 172.21.15.97:50036) 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 -- 172.21.15.97:0/1640609073 learned_addr learned my addr 172.21.15.97:0/1640609073 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 -- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57e415f120 msgr2=0x7f57e41494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57e415f120 0x7f57e41494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 -- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57e4160220 msgr2=0x7f57e414ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57e4160220 0x7f57e414ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 -- 172.21.15.97:0/1640609073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e41f9140 con 0x7f57e406cb40 2024-09-10T21:26:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e9c00640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e406cb40 0x7f57e41f8b40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f57d800d950 tx=0x7f57d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e1600640 1 -- 172.21.15.97:0/1640609073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8017070 con 0x7f57e406cb40 2024-09-10T21:26:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e1600640 1 -- 172.21.15.97:0/1640609073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57d8009d20 con 0x7f57e406cb40 2024-09-10T21:26:44.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e1600640 1 -- 172.21.15.97:0/1640609073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d80106c0 con 0x7f57e406cb40 2024-09-10T21:26:44.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e41f9420 con 0x7f57e406cb40 2024-09-10T21:26:44.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e41f99e0 con 0x7f57e406cb40 2024-09-10T21:26:44.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e40ce320 con 0x7f57e406cb40 2024-09-10T21:26:44.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.499+0000 7f57e1600640 1 -- 172.21.15.97:0/1640609073 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f57d8010860 con 0x7f57e406cb40 2024-09-10T21:26:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.503+0000 7f57e1600640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57c40725e0 0x7f57c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.503+0000 7f57e1600640 1 -- 172.21.15.97:0/1640609073 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f57d8093510 con 0x7f57e406cb40 2024-09-10T21:26:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.503+0000 7f57e9200640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57c40725e0 0x7f57c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.503+0000 7f57e9200640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57c40725e0 0x7f57c4074aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f57d00af000 tx=0x7f57d0019550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:44.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.507+0000 7f57e1600640 1 -- 172.21.15.97:0/1640609073 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f57d805ed10 con 0x7f57e406cb40 2024-09-10T21:26:44.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.671+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f57e40d1080 con 0x7f57e406cb40 2024-09-10T21:26:44.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.675+0000 7f57e1600640 1 -- 172.21.15.97:0/1640609073 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v34) ==== 71+0+0 (secure 0 0 0) 0x7f57d805e610 con 0x7f57e406cb40 2024-09-10T21:26:44.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.675+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57c40725e0 msgr2=0x7f57c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.675+0000 7f57ec600640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57c40725e0 0x7f57c4074aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f57d00af000 tx=0x7f57d0019550 comp rx=0 tx=0).stop 2024-09-10T21:26:44.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.675+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e406cb40 msgr2=0x7f57e41f8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.675+0000 7f57ec600640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e406cb40 0x7f57e41f8b40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f57d800d950 tx=0x7f57d800de10 comp rx=0 tx=0).stop 2024-09-10T21:26:44.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 shutdown_connections 2024-09-10T21:26:44.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f57c40725e0 0x7f57c4074aa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57e4160220 0x7f57e414ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57e415f120 0x7f57e41494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 --2- 172.21.15.97:0/1640609073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57e406cb40 0x7f57e41f8b40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 >> 172.21.15.97:0/1640609073 conn(0x7f57e40b3c70 msgr2=0x7f57e40ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:44.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 shutdown_connections 2024-09-10T21:26:44.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.679+0000 7f57ec600640 1 -- 172.21.15.97:0/1640609073 wait complete. 2024-09-10T21:26:44.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-10T21:26:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 -- 172.21.15.97:0/3853228029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00b1560 msgr2=0x7efdd00b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 --2- 172.21.15.97:0/3853228029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00b1560 0x7efdd00b1970 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efdd0069d60 tx=0x7efdc801f540 comp rx=0 tx=0).stop 2024-09-10T21:26:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 -- 172.21.15.97:0/3853228029 shutdown_connections 2024-09-10T21:26:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 --2- 172.21.15.97:0/3853228029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00b1560 0x7efdd00b1970 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 --2- 172.21.15.97:0/3853228029 >> v2:172.21.15.194:3300/0 conn(0x7efdd00ce830 0x7efdd00cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 -- 172.21.15.97:0/3853228029 >> 172.21.15.97:0/3853228029 conn(0x7efdd00b37c0 msgr2=0x7efdd00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 -- 172.21.15.97:0/3853228029 shutdown_connections 2024-09-10T21:26:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 -- 172.21.15.97:0/3853228029 wait complete. 2024-09-10T21:26:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 Processor -- start 2024-09-10T21:26:44.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.839+0000 7efdd7e00640 1 -- start start 2024-09-10T21:26:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdd00b1560 0x7efdd00c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdd00ce830 0x7efdd00b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00cec10 0x7efdd015ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efdd00ce1a0 con 0x7efdd00cec10 2024-09-10T21:26:44.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efdd00cde60 con 0x7efdd00b1560 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efdd00ce320 con 0x7efdd00ce830 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00cec10 0x7efdd015ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00cec10 0x7efdd015ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50060/0 (socket says 172.21.15.97:50060) 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 -- 172.21.15.97:0/2263964416 learned_addr learned my addr 172.21.15.97:0/2263964416 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd4a00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdd00ce830 0x7efdd00b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5400640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdd00b1560 0x7efdd00c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 -- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdd00ce830 msgr2=0x7efdd00b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdd00ce830 0x7efdd00b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 -- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdd00b1560 msgr2=0x7efdd00c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdd00b1560 0x7efdd00c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:44.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 -- 172.21.15.97:0/2263964416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd00c7cd0 con 0x7efdd00cec10 2024-09-10T21:26:44.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5400640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdd00b1560 0x7efdd00c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:44.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd5e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00cec10 0x7efdd015ede0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efdc801dbf0 tx=0x7efdc80af560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:44.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdcd600640 1 -- 172.21.15.97:0/2263964416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc80b4040 con 0x7efdd00cec10 2024-09-10T21:26:44.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdcd600640 1 -- 172.21.15.97:0/2263964416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdc80bf070 con 0x7efdd00cec10 2024-09-10T21:26:44.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd00c5760 con 0x7efdd00cec10 2024-09-10T21:26:44.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdcd600640 1 -- 172.21.15.97:0/2263964416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc80b0930 con 0x7efdd00cec10 2024-09-10T21:26:44.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd00c5fe0 con 0x7efdd00cec10 2024-09-10T21:26:44.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.843+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd00cf800 con 0x7efdd00cec10 2024-09-10T21:26:44.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.847+0000 7efdcd600640 1 -- 172.21.15.97:0/2263964416 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efdc80b0ad0 con 0x7efdd00cec10 2024-09-10T21:26:44.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.847+0000 7efdcd600640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdb00725e0 0x7efdb0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:44.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.847+0000 7efdcd600640 1 -- 172.21.15.97:0/2263964416 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7efdc81311e0 con 0x7efdd00cec10 2024-09-10T21:26:44.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.847+0000 7efdd5400640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdb00725e0 0x7efdb0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:44.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.847+0000 7efdd5400640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdb00725e0 0x7efdb0074aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006fd0 tx=0x7efdc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:44.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:44.851+0000 7efdcd600640 1 -- 172.21.15.97:0/2263964416 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efdc80c9020 con 0x7efdd00cec10 2024-09-10T21:26:45.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.071+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7efdd0072220 con 0x7efdd00cec10 2024-09-10T21:26:45.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.319+0000 7efdcd600640 1 -- 172.21.15.97:0/2263964416 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v151) ==== 141+0+0 (secure 0 0 0) 0x7efdc80190a0 con 0x7efdd00cec10 2024-09-10T21:26:45.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdb00725e0 msgr2=0x7efdb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdb00725e0 0x7efdb0074aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efdc4006fd0 tx=0x7efdc4008040 comp rx=0 tx=0).stop 2024-09-10T21:26:45.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00cec10 msgr2=0x7efdd015ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00cec10 0x7efdd015ede0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efdc801dbf0 tx=0x7efdc80af560 comp rx=0 tx=0).stop 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 shutdown_connections 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdb00725e0 0x7efdb0074aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd00cec10 0x7efdd015ede0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdd00ce830 0x7efdd00b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 --2- 172.21.15.97:0/2263964416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdd00b1560 0x7efdd00c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 >> 172.21.15.97:0/2263964416 conn(0x7efdd00b37c0 msgr2=0x7efdd00c4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 shutdown_connections 2024-09-10T21:26:45.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.323+0000 7efdd7e00640 1 -- 172.21.15.97:0/2263964416 wait complete. 2024-09-10T21:26:45.344 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2024-09-10T21:26:45.344 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-10T21:26:45.345 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2024-09-10T21:26:45.345 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:26:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 -- 172.21.15.97:0/2251773786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68145740 msgr2=0x7fbf680d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 --2- 172.21.15.97:0/2251773786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68145740 0x7fbf680d1690 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf68076530 tx=0x7fbf5801c880 comp rx=0 tx=0).stop 2024-09-10T21:26:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 -- 172.21.15.97:0/2251773786 shutdown_connections 2024-09-10T21:26:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 --2- 172.21.15.97:0/2251773786 >> v2:172.21.15.194:3300/0 conn(0x7fbf6815f7d0 0x7fbf6815fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 --2- 172.21.15.97:0/2251773786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68145740 0x7fbf680d1690 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 -- 172.21.15.97:0/2251773786 >> 172.21.15.97:0/2251773786 conn(0x7fbf680d0b70 msgr2=0x7fbf680d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 -- 172.21.15.97:0/2251773786 shutdown_connections 2024-09-10T21:26:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.483+0000 7fbf6d200640 1 -- 172.21.15.97:0/2251773786 wait complete. 2024-09-10T21:26:45.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 Processor -- start 2024-09-10T21:26:45.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 -- start start 2024-09-10T21:26:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf68145740 0x7fbf681f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6815f7d0 0x7fbf681528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68160180 0x7fbf68140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68168cd0 con 0x7fbf68160180 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf68168b50 con 0x7fbf6815f7d0 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbf68168e50 con 0x7fbf68145740 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68160180 0x7fbf68140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68160180 0x7fbf68140650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50080/0 (socket says 172.21.15.97:50080) 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 -- 172.21.15.97:0/1748145827 learned_addr learned my addr 172.21.15.97:0/1748145827 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf65e00640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf68145740 0x7fbf681f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf5fe00640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6815f7d0 0x7fbf681528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 -- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf68145740 msgr2=0x7fbf681f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf68145740 0x7fbf681f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 -- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6815f7d0 msgr2=0x7fbf681528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6815f7d0 0x7fbf681528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 -- 172.21.15.97:0/1748145827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf681533b0 con 0x7fbf68160180 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf65e00640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf68145740 0x7fbf681f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf66800640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68160180 0x7fbf68140650 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000b990 tx=0x7fbf6000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf5e000640 1 -- 172.21.15.97:0/1748145827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60019070 con 0x7fbf68160180 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf5e000640 1 -- 172.21.15.97:0/1748145827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf6000cc90 con 0x7fbf68160180 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf68153690 con 0x7fbf68160180 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf5e000640 1 -- 172.21.15.97:0/1748145827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600146a0 con 0x7fbf68160180 2024-09-10T21:26:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6806f9f0 con 0x7fbf68160180 2024-09-10T21:26:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.487+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf680d1690 con 0x7fbf68160180 2024-09-10T21:26:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.491+0000 7fbf5e000640 1 -- 172.21.15.97:0/1748145827 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf60014840 con 0x7fbf68160180 2024-09-10T21:26:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.491+0000 7fbf5e000640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf440725e0 0x7fbf44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.491+0000 7fbf5e000640 1 -- 172.21.15.97:0/1748145827 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7fbf60093190 con 0x7fbf68160180 2024-09-10T21:26:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.491+0000 7fbf65e00640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf440725e0 0x7fbf44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.491+0000 7fbf65e00640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf440725e0 0x7fbf44074aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbf580af000 tx=0x7fbf5801b0c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:45.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.495+0000 7fbf5e000640 1 -- 172.21.15.97:0/1748145827 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf6005e990 con 0x7fbf68160180 2024-09-10T21:26:45.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.663+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf680b13e0 con 0x7fbf68160180 2024-09-10T21:26:45.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.663+0000 7fbf5e000640 1 -- 172.21.15.97:0/1748145827 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+13459 (secure 0 0 0) 0x7fbf6005e290 con 0x7fbf68160180 2024-09-10T21:26:45.667 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:45.667 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":58,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:44.314760+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6834/2218970160":"2024-09-11T21:26:41.306469+0000","172.21.15.97:6832/2218970160":"2024-09-11T21:26:41.306469+0000","172.21.15.97:6834/3721865460":"2024-09-11T21:26:34.976757+0000","172.21.15.97:6832/3721865460":"2024-09-11T21:26:34.976757+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:26:45.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf440725e0 msgr2=0x7fbf44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf440725e0 0x7fbf44074aa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbf580af000 tx=0x7fbf5801b0c0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68160180 msgr2=0x7fbf68140650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68160180 0x7fbf68140650 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000b990 tx=0x7fbf6000be50 comp rx=0 tx=0).stop 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 shutdown_connections 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf440725e0 0x7fbf44074aa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf68160180 0x7fbf68140650 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6815f7d0 0x7fbf681528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 --2- 172.21.15.97:0/1748145827 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf68145740 0x7fbf681f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 >> 172.21.15.97:0/1748145827 conn(0x7fbf680d0b70 msgr2=0x7fbf680d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 shutdown_connections 2024-09-10T21:26:45.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.667+0000 7fbf6d200640 1 -- 172.21.15.97:0/1748145827 wait complete. 2024-09-10T21:26:45.688 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:24:02.092708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:26:45.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:26:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.831+0000 7f6f4ba00640 1 -- 172.21.15.97:0/202717537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f2c003a40 msgr2=0x7f6f2c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.831+0000 7f6f4ba00640 1 --2- 172.21.15.97:0/202717537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f2c003a40 0x7f6f2c003e60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c2e0 tx=0x7f6f3801cfa0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.831+0000 7f6f49a00640 1 -- 172.21.15.97:0/202717537 reap_dead start 2024-09-10T21:26:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.831+0000 7f6f4ba00640 1 -- 172.21.15.97:0/202717537 shutdown_connections 2024-09-10T21:26:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.831+0000 7f6f4ba00640 1 -- 172.21.15.97:0/202717537 >> 172.21.15.97:0/202717537 conn(0x7f6f440b1780 msgr2=0x7f6f440b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.831+0000 7f6f4ba00640 1 -- 172.21.15.97:0/202717537 shutdown_connections 2024-09-10T21:26:45.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.831+0000 7f6f4ba00640 1 -- 172.21.15.97:0/202717537 wait complete. 2024-09-10T21:26:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 Processor -- start 2024-09-10T21:26:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 -- start start 2024-09-10T21:26:45.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f2c003a40 0x7f6f4415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415f7b0 0x7f6f4415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f44163e20 0x7f6f44164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44070630 con 0x7f6f4415f7b0 2024-09-10T21:26:45.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6f440704b0 con 0x7f6f2c003a40 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6f44167070 con 0x7f6f44163e20 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415f7b0 0x7f6f4415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415f7b0 0x7f6f4415fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50098/0 (socket says 172.21.15.97:50098) 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f49000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f2c003a40 0x7f6f4415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 -- 172.21.15.97:0/2799063146 learned_addr learned my addr 172.21.15.97:0/2799063146 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f49a00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f44163e20 0x7f6f44164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 -- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f44163e20 msgr2=0x7f6f44164260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f44163e20 0x7f6f44164260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 -- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f2c003a40 msgr2=0x7f6f4415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f2c003a40 0x7f6f4415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 -- 172.21.15.97:0/2799063146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f441f8280 con 0x7f6f4415f7b0 2024-09-10T21:26:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f43e00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415f7b0 0x7f6f4415fc30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000f9d0 tx=0x7f6f3000fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f40c00640 1 -- 172.21.15.97:0/2799063146 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000bcb0 con 0x7f6f4415f7b0 2024-09-10T21:26:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f40c00640 1 -- 172.21.15.97:0/2799063146 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f30011440 con 0x7f6f4415f7b0 2024-09-10T21:26:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f40c00640 1 -- 172.21.15.97:0/2799063146 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30011850 con 0x7f6f4415f7b0 2024-09-10T21:26:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f441f8560 con 0x7f6f4415f7b0 2024-09-10T21:26:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.835+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f441f8b20 con 0x7f6f4415f7b0 2024-09-10T21:26:45.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.839+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f440ccee0 con 0x7f6f4415f7b0 2024-09-10T21:26:45.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.839+0000 7f6f40c00640 1 -- 172.21.15.97:0/2799063146 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f300119f0 con 0x7f6f4415f7b0 2024-09-10T21:26:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.839+0000 7f6f40c00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f240725e0 0x7f6f24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.839+0000 7f6f40c00640 1 -- 172.21.15.97:0/2799063146 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f6f30093c30 con 0x7f6f4415f7b0 2024-09-10T21:26:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.839+0000 7f6f49000640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f240725e0 0x7f6f24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:45.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.839+0000 7f6f49000640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f240725e0 0x7f6f24074aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f380ac000 tx=0x7f6f38019de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:45.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:45.843+0000 7f6f40c00640 1 -- 172.21.15.97:0/2799063146 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f3005ede0 con 0x7f6f4415f7b0 2024-09-10T21:26:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f441615f0 con 0x7f6f4415f7b0 2024-09-10T21:26:46.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f40c00640 1 -- 172.21.15.97:0/2799063146 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f6f30063810 con 0x7f6f4415f7b0 2024-09-10T21:26:46.009 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-09-10T21:26:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f240725e0 msgr2=0x7f6f24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f240725e0 0x7f6f24074aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f380ac000 tx=0x7f6f38019de0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415f7b0 msgr2=0x7f6f4415fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415f7b0 0x7f6f4415fc30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000f9d0 tx=0x7f6f3000fe90 comp rx=0 tx=0).stop 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 shutdown_connections 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6f240725e0 0x7f6f24074aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6f44163e20 0x7f6f44164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415f7b0 0x7f6f4415fc30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 --2- 172.21.15.97:0/2799063146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f2c003a40 0x7f6f4415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 >> 172.21.15.97:0/2799063146 conn(0x7f6f440b1780 msgr2=0x7f6f44160970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 shutdown_connections 2024-09-10T21:26:46.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.007+0000 7f6f4ba00640 1 -- 172.21.15.97:0/2799063146 wait complete. 2024-09-10T21:26:46.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-10T21:26:46.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 -- 172.21.15.97:0/1372917480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe8004810 msgr2=0x7f4fe8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:46.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 --2- 172.21.15.97:0/1372917480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe8004810 0x7f4fe8004c90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0016c0 tx=0x7f4fec01c6e0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 -- 172.21.15.97:0/1372917480 shutdown_connections 2024-09-10T21:26:46.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 --2- 172.21.15.97:0/1372917480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe8004810 0x7f4fe8004c90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 -- 172.21.15.97:0/1372917480 >> 172.21.15.97:0/1372917480 conn(0x7f50000d1c60 msgr2=0x7f50000d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:46.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 -- 172.21.15.97:0/1372917480 shutdown_connections 2024-09-10T21:26:46.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 -- 172.21.15.97:0/1372917480 wait complete. 2024-09-10T21:26:46.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 Processor -- start 2024-09-10T21:26:46.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.171+0000 7f5007e00640 1 -- start start 2024-09-10T21:26:46.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe8004810 0x7f5000159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:46.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500015b580 0x7f5000159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:46.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50000c9310 0x7f50000c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:46.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f50000b1f50 con 0x7f500015b580 2024-09-10T21:26:46.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f50000b1dd0 con 0x7f50000c9310 2024-09-10T21:26:46.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f50000b20d0 con 0x7f4fe8004810 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500015b580 0x7f5000159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500015b580 0x7f5000159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50122/0 (socket says 172.21.15.97:50122) 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 -- 172.21.15.97:0/2291338786 learned_addr learned my addr 172.21.15.97:0/2291338786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5005400640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe8004810 0x7f5000159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5005e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50000c9310 0x7f50000c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 -- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe8004810 msgr2=0x7f5000159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe8004810 0x7f5000159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 -- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50000c9310 msgr2=0x7f50000c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50000c9310 0x7f50000c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 -- 172.21.15.97:0/2291338786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50000cb9c0 con 0x7f500015b580 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5005e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50000c9310 0x7f50000c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5004a00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500015b580 0x7f5000159540 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4fec01ade0 tx=0x7f4fec006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:46.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f4ffd600640 1 -- 172.21.15.97:0/2291338786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec003680 con 0x7f500015b580 2024-09-10T21:26:46.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f4ffd600640 1 -- 172.21.15.97:0/2291338786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fec003910 con 0x7f500015b580 2024-09-10T21:26:46.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500015aaa0 con 0x7f500015b580 2024-09-10T21:26:46.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f4ffd600640 1 -- 172.21.15.97:0/2291338786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec003d00 con 0x7f500015b580 2024-09-10T21:26:46.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500015b060 con 0x7f500015b580 2024-09-10T21:26:46.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50000afc40 con 0x7f500015b580 2024-09-10T21:26:46.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.175+0000 7f4ffd600640 1 -- 172.21.15.97:0/2291338786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4fec0a5520 con 0x7f500015b580 2024-09-10T21:26:46.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.179+0000 7f4ffd600640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe00725e0 0x7f4fe0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:46.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.179+0000 7f4ffd600640 1 -- 172.21.15.97:0/2291338786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f4fec12c890 con 0x7f500015b580 2024-09-10T21:26:46.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.179+0000 7f5005400640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe00725e0 0x7f4fe0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:46.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.179+0000 7f5005400640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe00725e0 0x7f4fe0074aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4001ec0 tx=0x7f4ff4006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:46.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.183+0000 7f4ffd600640 1 -- 172.21.15.97:0/2291338786 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4fec0b3070 con 0x7f500015b580 2024-09-10T21:26:46.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:46.411+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7f50000b13a0 con 0x7f500015b580 2024-09-10T21:26:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f4ffd600640 1 -- 172.21.15.97:0/2291338786 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v153) ==== 145+0+0 (secure 0 0 0) 0x7f4fec016470 con 0x7f500015b580 2024-09-10T21:26:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe00725e0 msgr2=0x7f4fe0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe00725e0 0x7f4fe0074aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4001ec0 tx=0x7f4ff4006040 comp rx=0 tx=0).stop 2024-09-10T21:26:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500015b580 msgr2=0x7f5000159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500015b580 0x7f5000159540 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4fec01ade0 tx=0x7f4fec006950 comp rx=0 tx=0).stop 2024-09-10T21:26:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 shutdown_connections 2024-09-10T21:26:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe00725e0 0x7f4fe0074aa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50000c9310 0x7f50000c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500015b580 0x7f5000159540 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 --2- 172.21.15.97:0/2291338786 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe8004810 0x7f5000159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 >> 172.21.15.97:0/2291338786 conn(0x7f50000d1c60 msgr2=0x7f50000cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 shutdown_connections 2024-09-10T21:26:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.327+0000 7f5007e00640 1 -- 172.21.15.97:0/2291338786 wait complete. 2024-09-10T21:26:47.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:26:47.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 -- 172.21.15.97:0/2776817305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100b1560 msgr2=0x7fc3100b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 --2- 172.21.15.97:0/2776817305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100b1560 0x7fc3100b1990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc308009900 tx=0x7fc30801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 -- 172.21.15.97:0/2776817305 shutdown_connections 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 --2- 172.21.15.97:0/2776817305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100b1560 0x7fc3100b1990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 --2- 172.21.15.97:0/2776817305 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3100ce830 0x7fc3100cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 -- 172.21.15.97:0/2776817305 >> 172.21.15.97:0/2776817305 conn(0x7fc3100b37c0 msgr2=0x7fc3100b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 -- 172.21.15.97:0/2776817305 shutdown_connections 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 -- 172.21.15.97:0/2776817305 wait complete. 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 Processor -- start 2024-09-10T21:26:47.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 -- start start 2024-09-10T21:26:47.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.491+0000 7fc317000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3100b1560 0x7fc3100ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3100ce830 0x7fc31015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100cec10 0x7fc31006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3100ce1a0 con 0x7fc3100cec10 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc3100cde60 con 0x7fc3100ce830 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc3100ce320 con 0x7fc3100b1560 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100cec10 0x7fc31006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3100b1560 0x7fc3100ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100cec10 0x7fc31006f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51308/0 (socket says 172.21.15.97:51308) 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc30f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3100ce830 0x7fc31015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 -- 172.21.15.97:0/2479168704 learned_addr learned my addr 172.21.15.97:0/2479168704 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 -- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3100b1560 msgr2=0x7fc3100ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3100b1560 0x7fc3100ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 -- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3100ce830 msgr2=0x7fc31015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3100ce830 0x7fc31015c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 -- 172.21.15.97:0/2479168704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3100ca890 con 0x7fc3100cec10 2024-09-10T21:26:47.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc315000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100cec10 0x7fc31006f3b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc30801d780 tx=0x7fc308004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:47.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc307e00640 1 -- 172.21.15.97:0/2479168704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080be070 con 0x7fc3100cec10 2024-09-10T21:26:47.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc307e00640 1 -- 172.21.15.97:0/2479168704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3080053e0 con 0x7fc3100cec10 2024-09-10T21:26:47.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc307e00640 1 -- 172.21.15.97:0/2479168704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080b5040 con 0x7fc3100cec10 2024-09-10T21:26:47.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3100c67b0 con 0x7fc3100cec10 2024-09-10T21:26:47.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3100c6d70 con 0x7fc3100cec10 2024-09-10T21:26:47.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3100cf800 con 0x7fc3100cec10 2024-09-10T21:26:47.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.495+0000 7fc307e00640 1 -- 172.21.15.97:0/2479168704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc3080b8030 con 0x7fc3100cec10 2024-09-10T21:26:47.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.499+0000 7fc307e00640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2f00725e0 0x7fc2f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.499+0000 7fc307e00640 1 -- 172.21.15.97:0/2479168704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7fc30812fd10 con 0x7fc3100cec10 2024-09-10T21:26:47.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.499+0000 7fc30fe00640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2f00725e0 0x7fc2f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.499+0000 7fc30fe00640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2f00725e0 0x7fc2f0074aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc300009ef0 tx=0x7fc300009680 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:47.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.503+0000 7fc307e00640 1 -- 172.21.15.97:0/2479168704 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc308135020 con 0x7fc3100cec10 2024-09-10T21:26:47.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.671+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc3100b2d20 con 0x7fc3100cec10 2024-09-10T21:26:47.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.671+0000 7fc307e00640 1 -- 172.21.15.97:0/2479168704 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) ==== 59+0+1998 (secure 0 0 0) 0x7fc308018c30 con 0x7fc3100cec10 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3032.483965 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3656.533946 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3351.408335 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3363.485497 2024-09-10T21:26:47.674 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 8164.284498 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4391.956543 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4391.204852 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4799.171341 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds 20/20 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_ms 2/2 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_frag true 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_verify_scatter true 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:26:47.675 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:26:47.676 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced client_mount_timeout 600 2024-09-10T21:26:47.676 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_client 20/20 2024-09-10T21:26:47.676 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_ms 1/1 2024-09-10T21:26:47.676 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:26:47.676 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.671+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2f00725e0 msgr2=0x7fc2f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.671+0000 7fc317000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2f00725e0 0x7fc2f0074aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc300009ef0 tx=0x7fc300009680 comp rx=0 tx=0).stop 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.671+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100cec10 msgr2=0x7fc31006f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.671+0000 7fc317000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100cec10 0x7fc31006f3b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc30801d780 tx=0x7fc308004600 comp rx=0 tx=0).stop 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 shutdown_connections 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2f00725e0 0x7fc2f0074aa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3100cec10 0x7fc31006f3b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3100ce830 0x7fc31015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 --2- 172.21.15.97:0/2479168704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3100b1560 0x7fc3100ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 >> 172.21.15.97:0/2479168704 conn(0x7fc3100b37c0 msgr2=0x7fc31015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 shutdown_connections 2024-09-10T21:26:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.675+0000 7fc317000640 1 -- 172.21.15.97:0/2479168704 wait complete. 2024-09-10T21:26:47.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- 172.21.15.97:0/1842982937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5880ccc60 msgr2=0x7fb5880b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 --2- 172.21.15.97:0/1842982937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5880ccc60 0x7fb5880b0490 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb580009c80 tx=0x7fb58001cd30 comp rx=0 tx=0).stop 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- 172.21.15.97:0/1842982937 shutdown_connections 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 --2- 172.21.15.97:0/1842982937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5880ccc60 0x7fb5880b0490 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 --2- 172.21.15.97:0/1842982937 >> v2:172.21.15.194:3300/0 conn(0x7fb58815eed0 0x7fb5880d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- 172.21.15.97:0/1842982937 >> 172.21.15.97:0/1842982937 conn(0x7fb5880d1cd0 msgr2=0x7fb5880d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- 172.21.15.97:0/1842982937 shutdown_connections 2024-09-10T21:26:47.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- 172.21.15.97:0/1842982937 wait complete. 2024-09-10T21:26:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 Processor -- start 2024-09-10T21:26:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- start start 2024-09-10T21:26:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5880ccc60 0x7fb5880b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5880cdd40 0x7fb5880cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58815eed0 0x7fb58806ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb588168880 con 0x7fb58815eed0 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb588168700 con 0x7fb5880cdd40 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb58d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb588168a00 con 0x7fb5880ccc60 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58815eed0 0x7fb58806ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58815eed0 0x7fb58806ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51332/0 (socket says 172.21.15.97:51332) 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 -- 172.21.15.97:0/3388200472 learned_addr learned my addr 172.21.15.97:0/3388200472 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb585e00640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5880ccc60 0x7fb5880b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb585400640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5880cdd40 0x7fb5880cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 -- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5880ccc60 msgr2=0x7fb5880b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5880ccc60 0x7fb5880b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 -- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5880cdd40 msgr2=0x7fb5880cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5880cdd40 0x7fb5880cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb586800640 1 -- 172.21.15.97:0/3388200472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880b5960 con 0x7fb58815eed0 2024-09-10T21:26:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb585e00640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5880ccc60 0x7fb5880b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.839+0000 7fb585400640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5880cdd40 0x7fb5880cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb586800640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58815eed0 0x7fb58806ea90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb5800b0040 tx=0x7fb580003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb57e000640 1 -- 172.21.15.97:0/3388200472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800bc070 con 0x7fb58815eed0 2024-09-10T21:26:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb57e000640 1 -- 172.21.15.97:0/3388200472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5800044b0 con 0x7fb58815eed0 2024-09-10T21:26:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb57e000640 1 -- 172.21.15.97:0/3388200472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5800b2040 con 0x7fb58815eed0 2024-09-10T21:26:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5880b5c00 con 0x7fb58815eed0 2024-09-10T21:26:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5880b5ea0 con 0x7fb58815eed0 2024-09-10T21:26:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5880b0490 con 0x7fb58815eed0 2024-09-10T21:26:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb57e000640 1 -- 172.21.15.97:0/3388200472 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb5800b5070 con 0x7fb58815eed0 2024-09-10T21:26:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb57e000640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5680725e0 0x7fb568074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb57e000640 1 -- 172.21.15.97:0/3388200472 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7fb58012cfe0 con 0x7fb58815eed0 2024-09-10T21:26:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.843+0000 7fb585e00640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5680725e0 0x7fb568074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:47.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.847+0000 7fb585e00640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5680725e0 0x7fb568074aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb578005fd0 tx=0x7fb578005d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:47.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:47.847+0000 7fb57e000640 1 -- 172.21.15.97:0/3388200472 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb580016950 con 0x7fb58815eed0 2024-09-10T21:26:48.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.011+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb5880727a0 con 0x7fb58815eed0 2024-09-10T21:26:48.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.015+0000 7fb57e000640 1 -- 172.21.15.97:0/3388200472 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fb5800f8900 con 0x7fb58815eed0 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5680725e0 msgr2=0x7fb568074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5680725e0 0x7fb568074aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb578005fd0 tx=0x7fb578005d00 comp rx=0 tx=0).stop 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58815eed0 msgr2=0x7fb58806ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58815eed0 0x7fb58806ea90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb5800b0040 tx=0x7fb580003f60 comp rx=0 tx=0).stop 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 shutdown_connections 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5680725e0 0x7fb568074aa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb58815eed0 0x7fb58806ea90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5880cdd40 0x7fb5880cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 --2- 172.21.15.97:0/3388200472 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5880ccc60 0x7fb5880b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 >> 172.21.15.97:0/3388200472 conn(0x7fb5880d1cd0 msgr2=0x7fb588156450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 shutdown_connections 2024-09-10T21:26:48.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.019+0000 7fb58d200640 1 -- 172.21.15.97:0/3388200472 wait complete. 2024-09-10T21:26:48.042 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:26:48.042 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:48.130 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:26:48.224 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:26:48.224 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:48.224 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:48.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:26:48.233 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:48.233 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:48.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:48.242 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:26:48.242 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:48.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:26:48.284 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:26:48.284 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:26:48.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:26:48.293 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:26:48.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:26:48.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- 172.21.15.97:0/2302239149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 msgr2=0x7f32580cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 --2- 172.21.15.97:0/2302239149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f32580cfa40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f32580b3000 tx=0x7f324001c570 comp rx=0 tx=0).stop 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- 172.21.15.97:0/2302239149 shutdown_connections 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 --2- 172.21.15.97:0/2302239149 >> v2:172.21.15.194:3300/0 conn(0x7f32580cff80 0x7f32580b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 --2- 172.21.15.97:0/2302239149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f32580cfa40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 --2- 172.21.15.97:0/2302239149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f325815f350 0x7f32580b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- 172.21.15.97:0/2302239149 >> 172.21.15.97:0/2302239149 conn(0x7f32580d1c60 msgr2=0x7f32580d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- 172.21.15.97:0/2302239149 shutdown_connections 2024-09-10T21:26:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- 172.21.15.97:0/2302239149 wait complete. 2024-09-10T21:26:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 Processor -- start 2024-09-10T21:26:48.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- start start 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32580cff80 0x7f325806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f3258159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f325815f350 0x7f32581f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32580b0390 con 0x7f32580d05e0 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32580b0210 con 0x7f32580cff80 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f325de00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f32580b0510 con 0x7f325815f350 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f324e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f3258159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f324e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f3258159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51346/0 (socket says 172.21.15.97:51346) 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f324e000640 1 -- 172.21.15.97:0/3614962618 learned_addr learned my addr 172.21.15.97:0/3614962618 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f3256a00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32580cff80 0x7f325806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:48.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f3257400640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f325815f350 0x7f32581f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f324e000640 1 -- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f325815f350 msgr2=0x7f32581f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f324e000640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f325815f350 0x7f32581f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f324e000640 1 -- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32580cff80 msgr2=0x7f325806ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.467+0000 7f324e000640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32580cff80 0x7f325806ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324e000640 1 -- 172.21.15.97:0/3614962618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32580b4550 con 0x7f32580d05e0 2024-09-10T21:26:48.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324e000640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f3258159890 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f32400ad040 tx=0x7f32400ab4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324fe00640 1 -- 172.21.15.97:0/3614962618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400abad0 con 0x7f32580d05e0 2024-09-10T21:26:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324fe00640 1 -- 172.21.15.97:0/3614962618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32400b9070 con 0x7f32580d05e0 2024-09-10T21:26:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324fe00640 1 -- 172.21.15.97:0/3614962618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240005020 con 0x7f32580d05e0 2024-09-10T21:26:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325806f450 con 0x7f32580d05e0 2024-09-10T21:26:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325806fa10 con 0x7f32580d05e0 2024-09-10T21:26:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32580cfa40 con 0x7f32580d05e0 2024-09-10T21:26:48.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324fe00640 1 -- 172.21.15.97:0/3614962618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f32400051c0 con 0x7f32580d05e0 2024-09-10T21:26:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324fe00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32340725e0 0x7f3234074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.471+0000 7f324fe00640 1 -- 172.21.15.97:0/3614962618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5421+0+0 (secure 0 0 0) 0x7f324012d530 con 0x7f32580d05e0 2024-09-10T21:26:48.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.475+0000 7f3256a00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32340725e0 0x7f3234074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:48.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.475+0000 7f3256a00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32340725e0 0x7f3234074aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f324800bad0 tx=0x7f324800b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.475+0000 7f324fe00640 1 -- 172.21.15.97:0/3614962618 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f32400164b0 con 0x7f32580d05e0 2024-09-10T21:26:48.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.675+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32580b13a0 con 0x7f32580d05e0 2024-09-10T21:26:48.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.675+0000 7f324fe00640 1 -- 172.21.15.97:0/3614962618 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+3911 (secure 0 0 0) 0x7f32400f86d0 con 0x7f32580d05e0 2024-09-10T21:26:48.679 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:48.679 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":41,"btime":"2024-09-10T21:26:42:310041+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/61681303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":61681303},{"type":"v1","addr":"172.21.15.194:6837","nonce":61681303}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3289996627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3289996627},{"type":"v1","addr":"172.21.15.97:6835","nonce":3289996627}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3588396445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3588396445},{"type":"v1","addr":"172.21.15.194:6836","nonce":3588396445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3362092215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3362092215},{"type":"v1","addr":"172.21.15.97:6834","nonce":3362092215}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[]} 2024-09-10T21:26:48.679 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 41 2024-09-10T21:26:48.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32340725e0 msgr2=0x7f3234074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:48.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32340725e0 0x7f3234074aa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f324800bad0 tx=0x7f324800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 msgr2=0x7f3258159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:48.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f3258159890 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f32400ad040 tx=0x7f32400ab4c0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 shutdown_connections 2024-09-10T21:26:48.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32340725e0 0x7f3234074aa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f325815f350 0x7f32581f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32580d05e0 0x7f3258159890 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 --2- 172.21.15.97:0/3614962618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32580cff80 0x7f325806ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:48.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 >> 172.21.15.97:0/3614962618 conn(0x7f32580d1c60 msgr2=0x7f32580cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:48.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 shutdown_connections 2024-09-10T21:26:48.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:48.679+0000 7f325de00640 1 -- 172.21.15.97:0/3614962618 wait complete. 2024-09-10T21:26:48.700 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:26:48.700 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:26:48.700 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:26:48.700 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:48.700 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:26:48.700 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:26:48.701 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:26:48.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:48.701 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:26:48.701 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:26:48.701 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:26:48.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:48.701 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:26:48.701 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:26:48.702 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:26:48.702 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:48.715 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:26:48.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:26:48.717 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:26:48.717 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:26:48.719 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:26:48.719 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:26:48.761 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:26:48.762 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:48.763 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:26:48.763 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:48.765 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:26:48.765 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:48.851 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:26:48.855 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:26:48.855 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:26:48.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-10T21:26:48.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T21:26:48.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-10T21:26:54.716 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:26:54.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:26:54.720 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:26:54.720 DEBUG:teuthology.parallel:result is None 2024-09-10T21:26:54.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:26:54.814 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:26:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.870+0000 7f10ec000640 1 -- 172.21.15.97:0/3134742570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e415fde0 msgr2=0x7f10e413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.870+0000 7f10ec000640 1 --2- 172.21.15.97:0/3134742570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e415fde0 0x7f10e413f9b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009c80 tx=0x7f10dc01cda0 comp rx=0 tx=0).stop 2024-09-10T21:26:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- 172.21.15.97:0/3134742570 shutdown_connections 2024-09-10T21:26:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 --2- 172.21.15.97:0/3134742570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e415fde0 0x7f10e413f9b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 --2- 172.21.15.97:0/3134742570 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e40ce5f0 0x7f10e4141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- 172.21.15.97:0/3134742570 >> 172.21.15.97:0/3134742570 conn(0x7f10e40b1780 msgr2=0x7f10e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- 172.21.15.97:0/3134742570 shutdown_connections 2024-09-10T21:26:54.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- 172.21.15.97:0/3134742570 wait complete. 2024-09-10T21:26:54.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 Processor -- start 2024-09-10T21:26:54.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- start start 2024-09-10T21:26:54.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e40ce5f0 0x7f10e4153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:54.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ce9d0 0x7f10e41faf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:54.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e415fde0 0x7f10e41fb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:54.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4168bf0 con 0x7f10e40ce5f0 2024-09-10T21:26:54.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4168a70 con 0x7f10e40ce9d0 2024-09-10T21:26:54.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ec000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f10e4168d70 con 0x7f10e415fde0 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e40ce5f0 0x7f10e4153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e40ce5f0 0x7f10e4153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51390/0 (socket says 172.21.15.97:51390) 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 -- 172.21.15.97:0/722672344 learned_addr learned my addr 172.21.15.97:0/722672344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e8c00640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ce9d0 0x7f10e41faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10ea000640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e415fde0 0x7f10e41fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 -- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e415fde0 msgr2=0x7f10e41fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e415fde0 0x7f10e41fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 -- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ce9d0 msgr2=0x7f10e41faf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ce9d0 0x7f10e41faf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 -- 172.21.15.97:0/722672344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e41543a0 con 0x7f10e40ce5f0 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e8c00640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ce9d0 0x7f10e41faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:54.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e9600640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e40ce5f0 0x7f10e4153e60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d980 tx=0x7f10d800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:54.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.874+0000 7f10e1600640 1 -- 172.21.15.97:0/722672344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8017070 con 0x7f10e40ce5f0 2024-09-10T21:26:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10e1600640 1 -- 172.21.15.97:0/722672344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d8009d20 con 0x7f10e40ce5f0 2024-09-10T21:26:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10e1600640 1 -- 172.21.15.97:0/722672344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80106c0 con 0x7f10e40ce5f0 2024-09-10T21:26:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e413e350 con 0x7f10e40ce5f0 2024-09-10T21:26:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e415d740 con 0x7f10e40ce5f0 2024-09-10T21:26:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e4141d20 con 0x7f10e40ce5f0 2024-09-10T21:26:54.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10e1600640 1 -- 172.21.15.97:0/722672344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f10d8010860 con 0x7f10e40ce5f0 2024-09-10T21:26:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10e1600640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10c4072610 0x7f10c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10e1600640 1 -- 172.21.15.97:0/722672344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5421+0+0 (secure 0 0 0) 0x7f10d8093510 con 0x7f10e40ce5f0 2024-09-10T21:26:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.878+0000 7f10e8c00640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10c4072610 0x7f10c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:54.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.882+0000 7f10e8c00640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10c4072610 0x7f10c4074ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f10e40b1750 tx=0x7f10d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:54.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:54.882+0000 7f10e1600640 1 -- 172.21.15.97:0/722672344 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10d805ed00 con 0x7f10e40ce5f0 2024-09-10T21:26:55.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:55.058+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f10e4141360 con 0x7f10e40ce5f0 2024-09-10T21:26:55.633 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-10T21:26:55.633 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T21:26:55.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-10T21:26:56.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.354+0000 7f10e1600640 1 -- 172.21.15.97:0/722672344 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7f10d805e600 con 0x7f10e40ce5f0 2024-09-10T21:26:56.359 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-09-10T21:26:56.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10c4072610 msgr2=0x7f10c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10c4072610 0x7f10c4074ad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f10e40b1750 tx=0x7f10d0009040 comp rx=0 tx=0).stop 2024-09-10T21:26:56.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e40ce5f0 msgr2=0x7f10e4153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e40ce5f0 0x7f10e4153e60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d980 tx=0x7f10d800de40 comp rx=0 tx=0).stop 2024-09-10T21:26:56.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 shutdown_connections 2024-09-10T21:26:56.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10c4072610 0x7f10c4074ad0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f10e415fde0 0x7f10e41fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ce9d0 0x7f10e41faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 --2- 172.21.15.97:0/722672344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10e40ce5f0 0x7f10e4153e60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 >> 172.21.15.97:0/722672344 conn(0x7f10e40b1780 msgr2=0x7f10e4155980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 shutdown_connections 2024-09-10T21:26:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.358+0000 7f10ec000640 1 -- 172.21.15.97:0/722672344 wait complete. 2024-09-10T21:26:56.387 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:26:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 -- 172.21.15.97:0/1002519197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c004b70 msgr2=0x7f051c004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 --2- 172.21.15.97:0/1002519197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c004b70 0x7f051c004f90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0528006fa0 tx=0x7f052801f680 comp rx=0 tx=0).stop 2024-09-10T21:26:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f0539e00640 1 -- 172.21.15.97:0/1002519197 reap_dead start 2024-09-10T21:26:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 -- 172.21.15.97:0/1002519197 shutdown_connections 2024-09-10T21:26:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 -- 172.21.15.97:0/1002519197 >> 172.21.15.97:0/1002519197 conn(0x7f05340b1780 msgr2=0x7f05340b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 -- 172.21.15.97:0/1002519197 shutdown_connections 2024-09-10T21:26:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 -- 172.21.15.97:0/1002519197 wait complete. 2024-09-10T21:26:56.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 Processor -- start 2024-09-10T21:26:56.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.530+0000 7f053be00640 1 -- start start 2024-09-10T21:26:56.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f051c004b70 0x7f05341539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05341568e0 0x7f0534152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0534150430 0x7f05341508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f053406f3f0 con 0x7f0534150430 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f053406f270 con 0x7f051c004b70 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f053406e3b0 con 0x7f05341568e0 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0534150430 0x7f05341508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0534150430 0x7f05341508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51398/0 (socket says 172.21.15.97:51398) 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f051c004b70 0x7f05341539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 -- 172.21.15.97:0/3092493571 learned_addr learned my addr 172.21.15.97:0/3092493571 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0538a00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05341568e0 0x7f0534152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 -- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05341568e0 msgr2=0x7f0534152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05341568e0 0x7f0534152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 -- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f051c004b70 msgr2=0x7f05341539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f051c004b70 0x7f05341539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 -- 172.21.15.97:0/3092493571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05341f0710 con 0x7f0534150430 2024-09-10T21:26:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0539e00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0534150430 0x7f05341508e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0534074ee0 tx=0x7f052c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0531600640 1 -- 172.21.15.97:0/3092493571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c003980 con 0x7f0534150430 2024-09-10T21:26:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0531600640 1 -- 172.21.15.97:0/3092493571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f052c015070 con 0x7f0534150430 2024-09-10T21:26:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0531600640 1 -- 172.21.15.97:0/3092493571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c004030 con 0x7f0534150430 2024-09-10T21:26:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05341595b0 con 0x7f0534150430 2024-09-10T21:26:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0534159b70 con 0x7f0534150430 2024-09-10T21:26:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05341f08e0 con 0x7f0534150430 2024-09-10T21:26:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.534+0000 7f0531600640 1 -- 172.21.15.97:0/3092493571 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f052c0041d0 con 0x7f0534150430 2024-09-10T21:26:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.538+0000 7f0531600640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0514072610 0x7f0514074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.538+0000 7f0531600640 1 -- 172.21.15.97:0/3092493571 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f052c093a10 con 0x7f0534150430 2024-09-10T21:26:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.538+0000 7f0539400640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0514072610 0x7f0514074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.538+0000 7f0539400640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0514072610 0x7f0514074ad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05280b0000 tx=0x7f052801c4c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:56.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.542+0000 7f0531600640 1 -- 172.21.15.97:0/3092493571 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f052c05ec50 con 0x7f0534150430 2024-09-10T21:26:56.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f053406e940 con 0x7f0534150430 2024-09-10T21:26:56.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f0531600640 1 -- 172.21.15.97:0/3092493571 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f052c063680 con 0x7f0534150430 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA==", 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:56.770 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g==", 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.771 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw==", 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.772 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA==", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q==", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.773 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q==", 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-09-10T21:26:56.774 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg==", 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ==", 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.775 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ==", 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw==", 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ==", 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.777 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q==", 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q==", 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ==", 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.779 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag==", 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA==", 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.781 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmjCI0FRAATFUoCxw0bU6HSEUnlyrVJQ==", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA==", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:26:56.782 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw==", 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-10T21:26:56.783 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:26:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0514072610 msgr2=0x7f0514074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0514072610 0x7f0514074ad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05280b0000 tx=0x7f052801c4c0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0534150430 msgr2=0x7f05341508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0534150430 0x7f05341508e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0534074ee0 tx=0x7f052c010f40 comp rx=0 tx=0).stop 2024-09-10T21:26:56.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 shutdown_connections 2024-09-10T21:26:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0514072610 0x7f0514074ad0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0534150430 0x7f05341508e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f05341568e0 0x7f0534152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 --2- 172.21.15.97:0/3092493571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f051c004b70 0x7f05341539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.766+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 >> 172.21.15.97:0/3092493571 conn(0x7f05340b1780 msgr2=0x7f05340b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.770+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 shutdown_connections 2024-09-10T21:26:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.770+0000 7f053be00640 1 -- 172.21.15.97:0/3092493571 wait complete. 2024-09-10T21:26:56.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 -- 172.21.15.97:0/2598135622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 msgr2=0x7fc7f00ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 --2- 172.21.15.97:0/2598135622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00ae930 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00b3410 tx=0x7fc7e801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 -- 172.21.15.97:0/2598135622 shutdown_connections 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 --2- 172.21.15.97:0/2598135622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00ae930 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 --2- 172.21.15.97:0/2598135622 >> v2:172.21.15.194:3300/0 conn(0x7fc7f00cdcb0 0x7fc7f00cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 -- 172.21.15.97:0/2598135622 >> 172.21.15.97:0/2598135622 conn(0x7fc7f00d1a60 msgr2=0x7fc7f00d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 -- 172.21.15.97:0/2598135622 shutdown_connections 2024-09-10T21:26:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.930+0000 7fc7f7000640 1 -- 172.21.15.97:0/2598135622 wait complete. 2024-09-10T21:26:56.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 Processor -- start 2024-09-10T21:26:56.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 -- start start 2024-09-10T21:26:56.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7f00cdcb0 0x7fc7f0153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7f015efc0 0x7fc7f0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f0168930 con 0x7fc7f00d0490 2024-09-10T21:26:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01687b0 con 0x7fc7f015efc0 2024-09-10T21:26:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f0168ab0 con 0x7fc7f00cdcb0 2024-09-10T21:26:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51422/0 (socket says 172.21.15.97:51422) 2024-09-10T21:26:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 -- 172.21.15.97:0/1588556010 learned_addr learned my addr 172.21.15.97:0/1588556010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f6600640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7f015efc0 0x7fc7f0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 -- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7f00cdcb0 msgr2=0x7fc7f0153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5c00640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7f00cdcb0 0x7fc7f0153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7f00cdcb0 0x7fc7f0153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 -- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7f015efc0 msgr2=0x7fc7f0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7f015efc0 0x7fc7f0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 -- 172.21.15.97:0/1588556010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f00b6830 con 0x7fc7f00d0490 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f6600640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7f015efc0 0x7fc7f0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f5200640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00b5f30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00d9b0 tx=0x7fc7dc00de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7ed600640 1 -- 172.21.15.97:0/1588556010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc017070 con 0x7fc7f00d0490 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7ed600640 1 -- 172.21.15.97:0/1588556010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7dc009cc0 con 0x7fc7f00d0490 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7ed600640 1 -- 172.21.15.97:0/1588556010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc005280 con 0x7fc7f00d0490 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f01fad40 con 0x7fc7f00d0490 2024-09-10T21:26:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f00b6470 con 0x7fc7f00d0490 2024-09-10T21:26:56.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.934+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f00ae930 con 0x7fc7f00d0490 2024-09-10T21:26:56.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.938+0000 7fc7ed600640 1 -- 172.21.15.97:0/1588556010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc7dc005420 con 0x7fc7f00d0490 2024-09-10T21:26:56.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.938+0000 7fc7ed600640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7d00725e0 0x7fc7d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:56.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.938+0000 7fc7ed600640 1 -- 172.21.15.97:0/1588556010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fc7dc093320 con 0x7fc7f00d0490 2024-09-10T21:26:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.938+0000 7fc7f5c00640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7d00725e0 0x7fc7d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.938+0000 7fc7f5c00640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7d00725e0 0x7fc7d0074aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400a750 tx=0x7fc7e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:56.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:56.942+0000 7fc7ed600640 1 -- 172.21.15.97:0/1588556010 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc7dc05ebb0 con 0x7fc7f00d0490 2024-09-10T21:26:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc7f015ad70 con 0x7fc7f00d0490 2024-09-10T21:26:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7ed600640 1 -- 172.21.15.97:0/1588556010 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7fc7dc0672d0 con 0x7fc7f00d0490 2024-09-10T21:26:57.112 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:26:57.113 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:26:57.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:26:57.113 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-09-10T21:26:57.113 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-09-10T21:26:57.113 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:26:57.113 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:26:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7d00725e0 msgr2=0x7fc7d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7d00725e0 0x7fc7d0074aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400a750 tx=0x7fc7e4008040 comp rx=0 tx=0).stop 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 msgr2=0x7fc7f00b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00b5f30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00d9b0 tx=0x7fc7dc00de70 comp rx=0 tx=0).stop 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 shutdown_connections 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7d00725e0 0x7fc7d0074aa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7f015efc0 0x7fc7f0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f00d0490 0x7fc7f00b5f30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 --2- 172.21.15.97:0/1588556010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7f00cdcb0 0x7fc7f0153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 >> 172.21.15.97:0/1588556010 conn(0x7fc7f00d1a60 msgr2=0x7fc7f006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 shutdown_connections 2024-09-10T21:26:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.110+0000 7fc7f7000640 1 -- 172.21.15.97:0/1588556010 wait complete. 2024-09-10T21:26:57.133 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:26:57.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:26:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.274+0000 7fe77f000640 1 -- 172.21.15.97:0/1293957529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 msgr2=0x7fe75c005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.274+0000 7fe77f000640 1 --2- 172.21.15.97:0/1293957529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 0x7fe75c005920 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe760006f90 tx=0x7fe76001f610 comp rx=0 tx=0).stop 2024-09-10T21:26:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.274+0000 7fe77d000640 1 -- 172.21.15.97:0/1293957529 reap_dead start 2024-09-10T21:26:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.274+0000 7fe77f000640 1 -- 172.21.15.97:0/1293957529 shutdown_connections 2024-09-10T21:26:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.274+0000 7fe77f000640 1 -- 172.21.15.97:0/1293957529 >> 172.21.15.97:0/1293957529 conn(0x7fe7780b1780 msgr2=0x7fe7780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.274+0000 7fe77f000640 1 -- 172.21.15.97:0/1293957529 shutdown_connections 2024-09-10T21:26:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.274+0000 7fe77f000640 1 -- 172.21.15.97:0/1293957529 wait complete. 2024-09-10T21:26:57.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 Processor -- start 2024-09-10T21:26:57.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 -- start start 2024-09-10T21:26:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 0x7fe778144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe778159800 0x7fe778157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe778157d50 0x7fe7781f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe77815ae40 con 0x7fe75c0052e0 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe7780b3aa0 con 0x7fe778157d50 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe77815afc0 con 0x7fe778159800 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 0x7fe778144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 0x7fe778144500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51440/0 (socket says 172.21.15.97:51440) 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 -- 172.21.15.97:0/2352618615 learned_addr learned my addr 172.21.15.97:0/2352618615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77d000640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe778157d50 0x7fe7781f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe76f400640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe778159800 0x7fe778157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 -- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe778159800 msgr2=0x7fe778157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe778159800 0x7fe778157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 -- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe778157d50 msgr2=0x7fe7781f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe778157d50 0x7fe7781f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 -- 172.21.15.97:0/2352618615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe778144a40 con 0x7fe75c0052e0 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe76f400640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe778159800 0x7fe778157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe777e00640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 0x7fe778144500 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe768011ae0 tx=0x7fe768009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe774c00640 1 -- 172.21.15.97:0/2352618615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76801e070 con 0x7fe75c0052e0 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe774c00640 1 -- 172.21.15.97:0/2352618615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe76800fe40 con 0x7fe75c0052e0 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe778156840 con 0x7fe75c0052e0 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe774c00640 1 -- 172.21.15.97:0/2352618615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7680042b0 con 0x7fe75c0052e0 2024-09-10T21:26:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.278+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe778156e00 con 0x7fe75c0052e0 2024-09-10T21:26:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.282+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7780cf470 con 0x7fe75c0052e0 2024-09-10T21:26:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.282+0000 7fe774c00640 1 -- 172.21.15.97:0/2352618615 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe768003600 con 0x7fe75c0052e0 2024-09-10T21:26:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.282+0000 7fe774c00640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7540725e0 0x7fe754074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.282+0000 7fe774c00640 1 -- 172.21.15.97:0/2352618615 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fe768093d40 con 0x7fe75c0052e0 2024-09-10T21:26:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.282+0000 7fe76f400640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7540725e0 0x7fe754074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:57.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.282+0000 7fe76f400640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7540725e0 0x7fe754074aa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7600b0000 tx=0x7fe76001c450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:57.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.286+0000 7fe774c00640 1 -- 172.21.15.97:0/2352618615 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe76805f040 con 0x7fe75c0052e0 2024-09-10T21:26:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:57.458+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe77815a980 con 0x7fe75c0052e0 2024-09-10T21:26:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.370+0000 7fe774c00640 1 -- 172.21.15.97:0/2352618615 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7fe768063a70 con 0x7fe75c0052e0 2024-09-10T21:26:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-09-10T21:26:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7540725e0 msgr2=0x7fe754074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7540725e0 0x7fe754074aa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7600b0000 tx=0x7fe76001c450 comp rx=0 tx=0).stop 2024-09-10T21:26:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 msgr2=0x7fe778144500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 0x7fe778144500 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe768011ae0 tx=0x7fe768009ea0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 shutdown_connections 2024-09-10T21:26:58.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7540725e0 0x7fe754074aa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe778157d50 0x7fe7781f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe778159800 0x7fe778157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 --2- 172.21.15.97:0/2352618615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe75c0052e0 0x7fe778144500 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 >> 172.21.15.97:0/2352618615 conn(0x7fe7780b1780 msgr2=0x7fe7780cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:58.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 shutdown_connections 2024-09-10T21:26:58.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.374+0000 7fe77f000640 1 -- 172.21.15.97:0/2352618615 wait complete. 2024-09-10T21:26:58.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 -- 172.21.15.97:0/4183305753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 msgr2=0x7fdcac0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 --2- 172.21.15.97:0/4183305753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac0cea20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0c8c90 tx=0x7fdc9c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 -- 172.21.15.97:0/4183305753 shutdown_connections 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 --2- 172.21.15.97:0/4183305753 >> v2:172.21.15.194:3300/0 conn(0x7fdcac0cd6e0 0x7fdcac0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 --2- 172.21.15.97:0/4183305753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac0cea20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 -- 172.21.15.97:0/4183305753 >> 172.21.15.97:0/4183305753 conn(0x7fdcac0d1c30 msgr2=0x7fdcac0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 -- 172.21.15.97:0/4183305753 shutdown_connections 2024-09-10T21:26:58.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.554+0000 7fdcb3600640 1 -- 172.21.15.97:0/4183305753 wait complete. 2024-09-10T21:26:58.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 Processor -- start 2024-09-10T21:26:58.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 -- start start 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0b0d50 0x7fdcac164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac0cd6e0 0x7fdcac15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac168cd0 con 0x7fdcac0ce640 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac168b50 con 0x7fdcac0cd6e0 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdcac168e50 con 0x7fdcac0b0d50 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52480/0 (socket says 172.21.15.97:52480) 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 -- 172.21.15.97:0/2151394763 learned_addr learned my addr 172.21.15.97:0/2151394763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb0c00640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0b0d50 0x7fdcac164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdca3e00640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac0cd6e0 0x7fdcac15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 -- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0b0d50 msgr2=0x7fdcac164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0b0d50 0x7fdcac164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 -- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac0cd6e0 msgr2=0x7fdcac15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac0cd6e0 0x7fdcac15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 -- 172.21.15.97:0/2151394763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac1651b0 con 0x7fdcac0ce640 2024-09-10T21:26:58.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb1600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac154c10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdca400e960 tx=0x7fdca400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:58.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdca9600640 1 -- 172.21.15.97:0/2151394763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4004d30 con 0x7fdcac0ce640 2024-09-10T21:26:58.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdca9600640 1 -- 172.21.15.97:0/2151394763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdca400cb20 con 0x7fdcac0ce640 2024-09-10T21:26:58.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdca9600640 1 -- 172.21.15.97:0/2151394763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4005020 con 0x7fdcac0ce640 2024-09-10T21:26:58.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac165490 con 0x7fdcac0ce640 2024-09-10T21:26:58.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.558+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac165a50 con 0x7fdcac0ce640 2024-09-10T21:26:58.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.562+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcac0cea20 con 0x7fdcac0ce640 2024-09-10T21:26:58.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.562+0000 7fdca9600640 1 -- 172.21.15.97:0/2151394763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdca400f040 con 0x7fdcac0ce640 2024-09-10T21:26:58.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.562+0000 7fdca9600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc880725e0 0x7fdc88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.562+0000 7fdca9600640 1 -- 172.21.15.97:0/2151394763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5619+0+0 (secure 0 0 0) 0x7fdca4093ce0 con 0x7fdcac0ce640 2024-09-10T21:26:58.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.562+0000 7fdcb0c00640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc880725e0 0x7fdc88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.562+0000 7fdcb0c00640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc880725e0 0x7fdc88074aa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0af000 tx=0x7fdc9c019660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:58.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.566+0000 7fdca9600640 1 -- 172.21.15.97:0/2151394763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdca4016030 con 0x7fdcac0ce640 2024-09-10T21:26:58.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:58.734+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdcac155920 con 0x7fdcac0ce640 2024-09-10T21:26:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.374+0000 7fdca9600640 1 -- 172.21.15.97:0/2151394763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7fdcac155920 con 0x7fdcac0ce640 2024-09-10T21:26:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-09-10T21:26:59.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc880725e0 msgr2=0x7fdc88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:59.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc880725e0 0x7fdc88074aa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0af000 tx=0x7fdc9c019660 comp rx=0 tx=0).stop 2024-09-10T21:26:59.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 msgr2=0x7fdcac154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:59.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac154c10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdca400e960 tx=0x7fdca400ee20 comp rx=0 tx=0).stop 2024-09-10T21:26:59.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 shutdown_connections 2024-09-10T21:26:59.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc880725e0 0x7fdc88074aa0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0ce640 0x7fdcac154c10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac0cd6e0 0x7fdcac15ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 --2- 172.21.15.97:0/2151394763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0b0d50 0x7fdcac164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 >> 172.21.15.97:0/2151394763 conn(0x7fdcac0d1c30 msgr2=0x7fdcac155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:59.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 shutdown_connections 2024-09-10T21:26:59.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.378+0000 7fdcb3600640 1 -- 172.21.15.97:0/2151394763 wait complete. 2024-09-10T21:26:59.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:26:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 -- 172.21.15.97:0/4231418377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea006cb40 msgr2=0x7fcea006cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 --2- 172.21.15.97:0/4231418377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea006cb40 0x7fcea006cfc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce98009bc0 tx=0x7fce9801cba0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 -- 172.21.15.97:0/4231418377 shutdown_connections 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 --2- 172.21.15.97:0/4231418377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea006cb40 0x7fcea006cfc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 --2- 172.21.15.97:0/4231418377 >> v2:172.21.15.194:3300/0 conn(0x7fcea0073a70 0x7fcea00b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 -- 172.21.15.97:0/4231418377 >> 172.21.15.97:0/4231418377 conn(0x7fcea00b28c0 msgr2=0x7fcea00b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 -- 172.21.15.97:0/4231418377 shutdown_connections 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.562+0000 7fcea4a00640 1 -- 172.21.15.97:0/4231418377 wait complete. 2024-09-10T21:26:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 Processor -- start 2024-09-10T21:26:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 -- start start 2024-09-10T21:26:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea006cb40 0x7fcea01fb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea0073a70 0x7fcea013e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea00d0060 0x7fcea014d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcea016ae40 con 0x7fcea006cb40 2024-09-10T21:26:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcea00cf550 con 0x7fcea0073a70 2024-09-10T21:26:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcea016afc0 con 0x7fcea00d0060 2024-09-10T21:26:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea006cb40 0x7fcea01fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea006cb40 0x7fcea01fb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52486/0 (socket says 172.21.15.97:52486) 2024-09-10T21:26:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 -- 172.21.15.97:0/2207195960 learned_addr learned my addr 172.21.15.97:0/2207195960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9d400640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea0073a70 0x7fcea013e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9e800640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea00d0060 0x7fcea014d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 -- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea00d0060 msgr2=0x7fcea014d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea00d0060 0x7fcea014d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 -- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea0073a70 msgr2=0x7fcea013e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea0073a70 0x7fcea013e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 -- 172.21.15.97:0/2207195960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea00af780 con 0x7fcea006cb40 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce9de00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea006cb40 0x7fcea01fb830 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c500 tx=0x7fce9000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce96000640 1 -- 172.21.15.97:0/2207195960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90016020 con 0x7fcea006cb40 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce96000640 1 -- 172.21.15.97:0/2207195960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce90012920 con 0x7fcea006cb40 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea014c940 con 0x7fcea006cb40 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fce96000640 1 -- 172.21.15.97:0/2207195960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90012d30 con 0x7fcea006cb40 2024-09-10T21:26:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea0161290 con 0x7fcea006cb40 2024-09-10T21:26:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.566+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea0158180 con 0x7fcea006cb40 2024-09-10T21:26:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.570+0000 7fce96000640 1 -- 172.21.15.97:0/2207195960 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fce9001b400 con 0x7fcea006cb40 2024-09-10T21:26:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.570+0000 7fce96000640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce800725e0 0x7fce80074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.570+0000 7fce96000640 1 -- 172.21.15.97:0/2207195960 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6001+0+0 (secure 0 0 0) 0x7fce90093620 con 0x7fcea006cb40 2024-09-10T21:26:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.570+0000 7fce9d400640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce800725e0 0x7fce80074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.570+0000 7fce9d400640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce800725e0 0x7fce80074aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009600 tx=0x7fce8c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.574+0000 7fce96000640 1 -- 172.21.15.97:0/2207195960 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce90020020 con 0x7fcea006cb40 2024-09-10T21:26:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:26:59.774+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcea0157ea0 con 0x7fcea006cb40 2024-09-10T21:27:00.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.402+0000 7fce96000640 1 -- 172.21.15.97:0/2207195960 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v44) ==== 173+0+0 (secure 0 0 0) 0x7fce9005e4f0 con 0x7fcea006cb40 2024-09-10T21:27:00.407 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 9 and data pool 10 2024-09-10T21:27:00.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce800725e0 msgr2=0x7fce80074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce800725e0 0x7fce80074aa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009600 tx=0x7fce8c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea006cb40 msgr2=0x7fcea01fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea006cb40 0x7fcea01fb830 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c500 tx=0x7fce9000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 shutdown_connections 2024-09-10T21:27:00.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce800725e0 0x7fce80074aa0 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009600 tx=0x7fce8c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcea00d0060 0x7fcea014d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea0073a70 0x7fcea013e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 --2- 172.21.15.97:0/2207195960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea006cb40 0x7fcea01fb830 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.410+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 >> 172.21.15.97:0/2207195960 conn(0x7fcea00b28c0 msgr2=0x7fcea00afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.418+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 shutdown_connections 2024-09-10T21:27:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.422+0000 7fcea4a00640 1 -- 172.21.15.97:0/2207195960 wait complete. 2024-09-10T21:27:00.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:27:00.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 -- 172.21.15.97:0/2497232465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0a880afdd0 msgr2=0x7f0a880cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 --2- 172.21.15.97:0/2497232465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0a880afdd0 0x7f0a880cc6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009870 tx=0x7f0a7001c820 comp rx=0 tx=0).stop 2024-09-10T21:27:00.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 -- 172.21.15.97:0/2497232465 shutdown_connections 2024-09-10T21:27:00.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 --2- 172.21.15.97:0/2497232465 >> v2:172.21.15.194:3300/0 conn(0x7f0a880b0d50 0x7f0a880ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 --2- 172.21.15.97:0/2497232465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0a880afdd0 0x7f0a880cc6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 -- 172.21.15.97:0/2497232465 >> 172.21.15.97:0/2497232465 conn(0x7f0a880d1c30 msgr2=0x7f0a880d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:00.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 -- 172.21.15.97:0/2497232465 shutdown_connections 2024-09-10T21:27:00.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.590+0000 7f0a8d200640 1 -- 172.21.15.97:0/2497232465 wait complete. 2024-09-10T21:27:00.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 Processor -- start 2024-09-10T21:27:00.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 -- start start 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a880afdd0 0x7f0a881f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a880b0d50 0x7f0a881571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0a880ce640 0x7f0a880b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88168ee0 con 0x7f0a880b0d50 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88168d60 con 0x7f0a880afdd0 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0a88169060 con 0x7f0a880ce640 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a880b0d50 0x7f0a881571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:00.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a880b0d50 0x7f0a881571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52502/0 (socket says 172.21.15.97:52502) 2024-09-10T21:27:00.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 -- 172.21.15.97:0/4241525439 learned_addr learned my addr 172.21.15.97:0/4241525439 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 -- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0a880ce640 msgr2=0x7f0a880b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0a880ce640 0x7f0a880b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 -- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a880afdd0 msgr2=0x7f0a881f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a880afdd0 0x7f0a881f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 -- 172.21.15.97:0/4241525439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a88157730 con 0x7f0a880b0d50 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7d400640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a880b0d50 0x7f0a881571f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0a7001a450 tx=0x7f0a700a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7ea00640 1 -- 172.21.15.97:0/4241525439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a700bb070 con 0x7f0a880b0d50 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7ea00640 1 -- 172.21.15.97:0/4241525439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a70004470 con 0x7f0a880b0d50 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a7ea00640 1 -- 172.21.15.97:0/4241525439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a700b2040 con 0x7f0a880b0d50 2024-09-10T21:27:00.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a880b7550 con 0x7f0a880b0d50 2024-09-10T21:27:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.594+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a880c7140 con 0x7f0a880b0d50 2024-09-10T21:27:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.598+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a880cea20 con 0x7f0a880b0d50 2024-09-10T21:27:00.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.598+0000 7f0a7ea00640 1 -- 172.21.15.97:0/4241525439 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0a700b5030 con 0x7f0a880b0d50 2024-09-10T21:27:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.598+0000 7f0a7ea00640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0a640725e0 0x7f0a64074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.598+0000 7f0a7ea00640 1 -- 172.21.15.97:0/4241525439 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f0a7012ca30 con 0x7f0a880b0d50 2024-09-10T21:27:00.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.598+0000 7f0a85e00640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0a640725e0 0x7f0a64074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:00.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.598+0000 7f0a85e00640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0a640725e0 0x7f0a64074aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0a780046c0 tx=0x7f0a78007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:00.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.602+0000 7f0a7ea00640 1 -- 172.21.15.97:0/4241525439 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0a700f1de0 con 0x7f0a880b0d50 2024-09-10T21:27:00.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a880d2010 con 0x7f0a880b0d50 2024-09-10T21:27:00.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a7ea00640 1 -- 172.21.15.97:0/4241525439 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17029 (secure 0 0 0) 0x7f0a70016410 con 0x7f0a880b0d50 2024-09-10T21:27:00.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:00.773 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:59.777405+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:57.469287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:58.747272+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:27:00.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0a640725e0 msgr2=0x7f0a64074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0a640725e0 0x7f0a64074aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0a780046c0 tx=0x7f0a78007910 comp rx=0 tx=0).stop 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a880b0d50 msgr2=0x7f0a881571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a880b0d50 0x7f0a881571f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0a7001a450 tx=0x7f0a700a5760 comp rx=0 tx=0).stop 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 shutdown_connections 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0a640725e0 0x7f0a64074aa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0a880ce640 0x7f0a880b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a880b0d50 0x7f0a881571f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 --2- 172.21.15.97:0/4241525439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a880afdd0 0x7f0a881f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 >> 172.21.15.97:0/4241525439 conn(0x7f0a880d1c30 msgr2=0x7f0a8815a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 shutdown_connections 2024-09-10T21:27:00.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.770+0000 7f0a8d200640 1 -- 172.21.15.97:0/4241525439 wait complete. 2024-09-10T21:27:00.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:27:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 -- 172.21.15.97:0/2139810645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b40d0490 msgr2=0x7f69b40ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 --2- 172.21.15.97:0/2139810645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b40d0490 0x7f69b40ae930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f69b4073490 tx=0x7f69ac01cb80 comp rx=0 tx=0).stop 2024-09-10T21:27:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 -- 172.21.15.97:0/2139810645 shutdown_connections 2024-09-10T21:27:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 --2- 172.21.15.97:0/2139810645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b40d0490 0x7f69b40ae930 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 --2- 172.21.15.97:0/2139810645 >> v2:172.21.15.194:3300/0 conn(0x7f69b415efc0 0x7f69b40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 -- 172.21.15.97:0/2139810645 >> 172.21.15.97:0/2139810645 conn(0x7f69b40d1a60 msgr2=0x7f69b40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 -- 172.21.15.97:0/2139810645 shutdown_connections 2024-09-10T21:27:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 -- 172.21.15.97:0/2139810645 wait complete. 2024-09-10T21:27:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 Processor -- start 2024-09-10T21:27:00.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.934+0000 7f69b9e00640 1 -- start start 2024-09-10T21:27:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69b40cdcb0 0x7f69b41513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69b40d0490 0x7f69b41587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b415efc0 0x7f69b415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4168930 con 0x7f69b415efc0 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f69b41687b0 con 0x7f69b40d0490 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f69b4168ab0 con 0x7f69b40cdcb0 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b415efc0 0x7f69b415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b415efc0 0x7f69b415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52520/0 (socket says 172.21.15.97:52520) 2024-09-10T21:27:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 -- 172.21.15.97:0/3258821553 learned_addr learned my addr 172.21.15.97:0/3258821553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b8a00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69b40cdcb0 0x7f69b41513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69abe00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69b40d0490 0x7f69b41587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 -- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69b40cdcb0 msgr2=0x7f69b41513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69b40cdcb0 0x7f69b41513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 -- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69b40d0490 msgr2=0x7f69b41587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69b40d0490 0x7f69b41587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 -- 172.21.15.97:0/3258821553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b4158d00 con 0x7f69b415efc0 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69abe00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69b40d0490 0x7f69b41587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9400640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b415efc0 0x7f69b415d060 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009b10 tx=0x7f69ac0a5a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b1600640 1 -- 172.21.15.97:0/3258821553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0bb070 con 0x7f69b415efc0 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b1600640 1 -- 172.21.15.97:0/3258821553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69ac004e00 con 0x7f69b415efc0 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b1600640 1 -- 172.21.15.97:0/3258821553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0b2040 con 0x7f69b415efc0 2024-09-10T21:27:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b41594b0 con 0x7f69b415efc0 2024-09-10T21:27:00.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b415c580 con 0x7f69b415efc0 2024-09-10T21:27:00.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.938+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b40ae930 con 0x7f69b415efc0 2024-09-10T21:27:00.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.942+0000 7f69b1600640 1 -- 172.21.15.97:0/3258821553 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f69ac0b5030 con 0x7f69b415efc0 2024-09-10T21:27:00.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.942+0000 7f69b1600640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69900725e0 0x7f6990074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:00.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.942+0000 7f69b1600640 1 -- 172.21.15.97:0/3258821553 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f69ac12d100 con 0x7f69b415efc0 2024-09-10T21:27:00.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.942+0000 7f69b8a00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69900725e0 0x7f6990074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:00.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.942+0000 7f69b8a00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69900725e0 0x7f6990074aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69a400ab40 tx=0x7f69a400a250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:00.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:00.946+0000 7f69b1600640 1 -- 172.21.15.97:0/3258821553 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f69ac016770 con 0x7f69b415efc0 2024-09-10T21:27:01.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.110+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69b415a3d0 con 0x7f69b415efc0 2024-09-10T21:27:01.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.110+0000 7f69b1600640 1 -- 172.21.15.97:0/3258821553 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+17029 (secure 0 0 0) 0x7f69ac0f7ff0 con 0x7f69b415efc0 2024-09-10T21:27:01.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:01.115 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":66,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:26:59.777405+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:57.469287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:58.747272+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:27:01.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69900725e0 msgr2=0x7f6990074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:01.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69900725e0 0x7f6990074aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69a400ab40 tx=0x7f69a400a250 comp rx=0 tx=0).stop 2024-09-10T21:27:01.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b415efc0 msgr2=0x7f69b415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:01.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b415efc0 0x7f69b415d060 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009b10 tx=0x7f69ac0a5a30 comp rx=0 tx=0).stop 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 shutdown_connections 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69900725e0 0x7f6990074aa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69b415efc0 0x7f69b415d060 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69b40d0490 0x7f69b41587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 --2- 172.21.15.97:0/3258821553 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f69b40cdcb0 0x7f69b41513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 >> 172.21.15.97:0/3258821553 conn(0x7f69b40d1a60 msgr2=0x7f69b415abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 shutdown_connections 2024-09-10T21:27:01.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.114+0000 7f69b9e00640 1 -- 172.21.15.97:0/3258821553 wait complete. 2024-09-10T21:27:01.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:27:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- 172.21.15.97:0/435040745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e841511f0 msgr2=0x7f0e840b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 --2- 172.21.15.97:0/435040745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e841511f0 0x7f0e840b37b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009930 tx=0x7f0e7401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- 172.21.15.97:0/435040745 shutdown_connections 2024-09-10T21:27:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 --2- 172.21.15.97:0/435040745 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e840ceee0 0x7f0e840cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 --2- 172.21.15.97:0/435040745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e841511f0 0x7f0e840b37b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- 172.21.15.97:0/435040745 >> 172.21.15.97:0/435040745 conn(0x7f0e840b1780 msgr2=0x7f0e840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:01.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- 172.21.15.97:0/435040745 shutdown_connections 2024-09-10T21:27:01.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- 172.21.15.97:0/435040745 wait complete. 2024-09-10T21:27:01.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 Processor -- start 2024-09-10T21:27:01.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- start start 2024-09-10T21:27:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e840b3cf0 0x7f0e840aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e840ceee0 0x7f0e8414f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e841511f0 0x7f0e8414ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:01.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8406e740 con 0x7f0e840ceee0 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8406e5c0 con 0x7f0e841511f0 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e8b000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8406e8c0 con 0x7f0e840b3cf0 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e7b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e840ceee0 0x7f0e8414f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e7b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e840ceee0 0x7f0e8414f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52538/0 (socket says 172.21.15.97:52538) 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e7b400640 1 -- 172.21.15.97:0/3827850197 learned_addr learned my addr 172.21.15.97:0/3827850197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e83e00640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e840b3cf0 0x7f0e840aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.278+0000 7f0e89000640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e841511f0 0x7f0e8414ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e7b400640 1 -- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e840b3cf0 msgr2=0x7f0e840aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e7b400640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e840b3cf0 0x7f0e840aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e7b400640 1 -- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e841511f0 msgr2=0x7f0e8414ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e7b400640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e841511f0 0x7f0e8414ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e7b400640 1 -- 172.21.15.97:0/3827850197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e84155c50 con 0x7f0e840ceee0 2024-09-10T21:27:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e7b400640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e840ceee0 0x7f0e8414f9c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c450 tx=0x7f0e6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e80c00640 1 -- 172.21.15.97:0/3827850197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c012830 con 0x7f0e840ceee0 2024-09-10T21:27:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e80c00640 1 -- 172.21.15.97:0/3827850197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e6c017070 con 0x7f0e840ceee0 2024-09-10T21:27:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e80c00640 1 -- 172.21.15.97:0/3827850197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c012d80 con 0x7f0e840ceee0 2024-09-10T21:27:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e841f3a90 con 0x7f0e840ceee0 2024-09-10T21:27:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e841f4050 con 0x7f0e840ceee0 2024-09-10T21:27:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e840cf370 con 0x7f0e840ceee0 2024-09-10T21:27:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e80c00640 1 -- 172.21.15.97:0/3827850197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0e6c01c460 con 0x7f0e840ceee0 2024-09-10T21:27:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.282+0000 7f0e80c00640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e60072610 0x7f0e60074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:01.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.286+0000 7f0e80c00640 1 -- 172.21.15.97:0/3827850197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f0e6c092f90 con 0x7f0e840ceee0 2024-09-10T21:27:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.286+0000 7f0e83e00640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e60072610 0x7f0e60074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.286+0000 7f0e83e00640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e60072610 0x7f0e60074ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e740b2000 tx=0x7f0e7401d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:01.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.286+0000 7f0e80c00640 1 -- 172.21.15.97:0/3827850197 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0e6c05df20 con 0x7f0e840ceee0 2024-09-10T21:27:01.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:01.490+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0e840b01c0 con 0x7f0e840ceee0 2024-09-10T21:27:02.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.410+0000 7f0e80c00640 1 -- 172.21.15.97:0/3827850197 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) ==== 118+0+0 (secure 0 0 0) 0x7f0e6c062950 con 0x7f0e840ceee0 2024-09-10T21:27:02.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e60072610 msgr2=0x7f0e60074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e60072610 0x7f0e60074ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e740b2000 tx=0x7f0e7401d910 comp rx=0 tx=0).stop 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e840ceee0 msgr2=0x7f0e8414f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e840ceee0 0x7f0e8414f9c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c450 tx=0x7f0e6c00c910 comp rx=0 tx=0).stop 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 shutdown_connections 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e60072610 0x7f0e60074ad0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e841511f0 0x7f0e8414ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e840ceee0 0x7f0e8414f9c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 --2- 172.21.15.97:0/3827850197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e840b3cf0 0x7f0e840aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 >> 172.21.15.97:0/3827850197 conn(0x7f0e840b1780 msgr2=0x7f0e8415cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 shutdown_connections 2024-09-10T21:27:02.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.414+0000 7f0e8b000640 1 -- 172.21.15.97:0/3827850197 wait complete. 2024-09-10T21:27:02.439 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:27:02.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:27:02.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- 172.21.15.97:0/100890842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 msgr2=0x7f8e9c0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/100890842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c0b2a80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c073080 tx=0x7f8e8c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- 172.21.15.97:0/100890842 shutdown_connections 2024-09-10T21:27:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/100890842 >> v2:172.21.15.194:3300/0 conn(0x7f8e9c06fd50 0x7f8e9c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/100890842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c0b2a80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- 172.21.15.97:0/100890842 >> 172.21.15.97:0/100890842 conn(0x7f8e9c0b1780 msgr2=0x7f8e9c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- 172.21.15.97:0/100890842 shutdown_connections 2024-09-10T21:27:02.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- 172.21.15.97:0/100890842 wait complete. 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 Processor -- start 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- start start 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e9c06fd50 0x7f8e9c1fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e9c144690 0x7f8e9c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c167bf0 con 0x7f8e9c143f90 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c167a70 con 0x7f8e9c06fd50 2024-09-10T21:27:02.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8ea1a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9c167d70 con 0x7f8e9c144690 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8e92000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8e92000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c15cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52564/0 (socket says 172.21.15.97:52564) 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8e92000640 1 -- 172.21.15.97:0/1814312900 learned_addr learned my addr 172.21.15.97:0/1814312900 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.582+0000 7f8e9aa00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e9c06fd50 0x7f8e9c1fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e9b400640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e9c144690 0x7f8e9c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e92000640 1 -- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e9c144690 msgr2=0x7f8e9c06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e92000640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e9c144690 0x7f8e9c06ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e92000640 1 -- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e9c06fd50 msgr2=0x7f8e9c1fb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e92000640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e9c06fd50 0x7f8e9c1fb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:02.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e92000640 1 -- 172.21.15.97:0/1814312900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c15e720 con 0x7f8e9c143f90 2024-09-10T21:27:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e92000640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c15cc50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c3c0 tx=0x7f8e8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e93400640 1 -- 172.21.15.97:0/1814312900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84017070 con 0x7f8e9c143f90 2024-09-10T21:27:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e93400640 1 -- 172.21.15.97:0/1814312900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e840053c0 con 0x7f8e9c143f90 2024-09-10T21:27:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e93400640 1 -- 172.21.15.97:0/1814312900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84003860 con 0x7f8e9c143f90 2024-09-10T21:27:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c15ea00 con 0x7f8e9c143f90 2024-09-10T21:27:02.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c15efc0 con 0x7f8e9c143f90 2024-09-10T21:27:02.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c070160 con 0x7f8e9c143f90 2024-09-10T21:27:02.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e93400640 1 -- 172.21.15.97:0/1814312900 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8e84003a00 con 0x7f8e9c143f90 2024-09-10T21:27:02.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.586+0000 7f8e93400640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e780725e0 0x7f8e78074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:02.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.590+0000 7f8e93400640 1 -- 172.21.15.97:0/1814312900 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8e84092a00 con 0x7f8e9c143f90 2024-09-10T21:27:02.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.590+0000 7f8e9aa00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e780725e0 0x7f8e78074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:02.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.590+0000 7f8e9aa00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e780725e0 0x7f8e78074aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0af000 tx=0x7f8e8c01b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:02.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.590+0000 7f8e93400640 1 -- 172.21.15.97:0/1814312900 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e9c0706a0 con 0x7f8e9c143f90 2024-09-10T21:27:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:02.790+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8e9c0cc4e0 con 0x7f8e9c143f90 2024-09-10T21:27:03.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.414+0000 7f8e93400640 1 -- 172.21.15.97:0/1814312900 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f8e8405d860 con 0x7f8e9c143f90 2024-09-10T21:27:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e780725e0 msgr2=0x7f8e78074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e780725e0 0x7f8e78074aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0af000 tx=0x7f8e8c01b100 comp rx=0 tx=0).stop 2024-09-10T21:27:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 msgr2=0x7f8e9c15cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c15cc50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c3c0 tx=0x7f8e8400c880 comp rx=0 tx=0).stop 2024-09-10T21:27:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 shutdown_connections 2024-09-10T21:27:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e780725e0 0x7f8e78074aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e9c144690 0x7f8e9c06ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9c143f90 0x7f8e9c15cc50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 --2- 172.21.15.97:0/1814312900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e9c06fd50 0x7f8e9c1fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 >> 172.21.15.97:0/1814312900 conn(0x7f8e9c0b1780 msgr2=0x7f8e9c15da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 shutdown_connections 2024-09-10T21:27:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.418+0000 7f8ea1a00640 1 -- 172.21.15.97:0/1814312900 wait complete. 2024-09-10T21:27:03.441 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:27:03.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:03.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 -- 172.21.15.97:0/129075332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 msgr2=0x7f151c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 --2- 172.21.15.97:0/129075332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15c100 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f151c069e70 tx=0x7f151401cd80 comp rx=0 tx=0).stop 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 -- 172.21.15.97:0/129075332 shutdown_connections 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 --2- 172.21.15.97:0/129075332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15c100 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 --2- 172.21.15.97:0/129075332 >> v2:172.21.15.194:3300/0 conn(0x7f151c0ce9d0 0x7f151c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 --2- 172.21.15.97:0/129075332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f151c0ce5f0 0x7f151c141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 -- 172.21.15.97:0/129075332 >> 172.21.15.97:0/129075332 conn(0x7f151c0b1780 msgr2=0x7f151c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 -- 172.21.15.97:0/129075332 shutdown_connections 2024-09-10T21:27:03.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.578+0000 7f1523800640 1 -- 172.21.15.97:0/129075332 wait complete. 2024-09-10T21:27:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 Processor -- start 2024-09-10T21:27:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 -- start start 2024-09-10T21:27:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f151c0ce5f0 0x7f151c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f151c0ce9d0 0x7f151c0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f151c140290 con 0x7f151c15fc10 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f151c140110 con 0x7f151c0ce9d0 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f151c140410 con 0x7f151c0ce5f0 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52572/0 (socket says 172.21.15.97:52572) 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 -- 172.21.15.97:0/1050572730 learned_addr learned my addr 172.21.15.97:0/1050572730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f151be00640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f151c0ce9d0 0x7f151c0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1520e00640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f151c0ce5f0 0x7f151c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 -- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f151c0ce5f0 msgr2=0x7f151c1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f151c0ce5f0 0x7f151c1545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 -- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f151c0ce9d0 msgr2=0x7f151c0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f151c0ce9d0 0x7f151c0b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 -- 172.21.15.97:0/1050572730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c15f990 con 0x7f151c15fc10 2024-09-10T21:27:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f151be00640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f151c0ce9d0 0x7f151c0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1521800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15e260 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1514009bd0 tx=0x7f1514005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1518c00640 1 -- 172.21.15.97:0/1050572730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140bb070 con 0x7f151c15fc10 2024-09-10T21:27:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1518c00640 1 -- 172.21.15.97:0/1050572730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1514004d40 con 0x7f151c15fc10 2024-09-10T21:27:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1518c00640 1 -- 172.21.15.97:0/1050572730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140b2040 con 0x7f151c15fc10 2024-09-10T21:27:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c0b2b80 con 0x7f151c15fc10 2024-09-10T21:27:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.582+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c0b3140 con 0x7f151c15fc10 2024-09-10T21:27:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.586+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151c15f6d0 con 0x7f151c15fc10 2024-09-10T21:27:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.586+0000 7f1518c00640 1 -- 172.21.15.97:0/1050572730 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f15140b5030 con 0x7f151c15fc10 2024-09-10T21:27:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.586+0000 7f1518c00640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f14fc072610 0x7f14fc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.586+0000 7f1518c00640 1 -- 172.21.15.97:0/1050572730 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f151412ced0 con 0x7f151c15fc10 2024-09-10T21:27:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.586+0000 7f1520e00640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f14fc072610 0x7f14fc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.586+0000 7f1520e00640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f14fc072610 0x7f14fc074ad0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f150c005fd0 tx=0x7f150c005e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.590+0000 7f1518c00640 1 -- 172.21.15.97:0/1050572730 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1514016970 con 0x7f151c15fc10 2024-09-10T21:27:03.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.790+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f151c141360 con 0x7f151c15fc10 2024-09-10T21:27:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.790+0000 7f1518c00640 1 -- 172.21.15.97:0/1050572730 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5230 (secure 0 0 0) 0x7f15140f8380 con 0x7f151c15fc10 2024-09-10T21:27:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:03.794 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":48,"btime":"2024-09-10T21:27:03:415307+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:02.795427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/2609945029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2609945029},{"type":"v1","addr":"172.21.15.97:6835","nonce":2609945029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-10T21:27:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 48 2024-09-10T21:27:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f14fc072610 msgr2=0x7f14fc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f14fc072610 0x7f14fc074ad0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f150c005fd0 tx=0x7f150c005e60 comp rx=0 tx=0).stop 2024-09-10T21:27:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 msgr2=0x7f151c15e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15e260 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1514009bd0 tx=0x7f1514005b00 comp rx=0 tx=0).stop 2024-09-10T21:27:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 shutdown_connections 2024-09-10T21:27:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f14fc072610 0x7f14fc074ad0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c15fc10 0x7f151c15e260 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f151c0ce9d0 0x7f151c0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 --2- 172.21.15.97:0/1050572730 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f151c0ce5f0 0x7f151c1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 >> 172.21.15.97:0/1050572730 conn(0x7f151c0b1780 msgr2=0x7f151c06f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:03.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 shutdown_connections 2024-09-10T21:27:03.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.794+0000 7f1523800640 1 -- 172.21.15.97:0/1050572730 wait complete. 2024-09-10T21:27:03.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:27:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.950+0000 7fc80b800640 1 -- 172.21.15.97:0/3707231904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8040cf5f0 msgr2=0x7fc80406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.950+0000 7fc80b800640 1 --2- 172.21.15.97:0/3707231904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8040cf5f0 0x7fc80406f090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00ad00 tx=0x7fc7fc0206b0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- 172.21.15.97:0/3707231904 shutdown_connections 2024-09-10T21:27:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 --2- 172.21.15.97:0/3707231904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8040cf5f0 0x7fc80406f090 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- 172.21.15.97:0/3707231904 >> 172.21.15.97:0/3707231904 conn(0x7fc8040b1780 msgr2=0x7fc8040b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- 172.21.15.97:0/3707231904 shutdown_connections 2024-09-10T21:27:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- 172.21.15.97:0/3707231904 wait complete. 2024-09-10T21:27:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 Processor -- start 2024-09-10T21:27:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- start start 2024-09-10T21:27:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc80406ec40 0x7fc80414ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8040cf5f0 0x7fc8040b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8041511f0 0x7fc8040b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc80406e7f0 con 0x7fc80406ec40 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc80406e670 con 0x7fc8041511f0 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc80b800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc80406e970 con 0x7fc8040cf5f0 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc80406ec40 0x7fc80414ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc80406ec40 0x7fc80414ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52598/0 (socket says 172.21.15.97:52598) 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 -- 172.21.15.97:0/2638793364 learned_addr learned my addr 172.21.15.97:0/2638793364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc809800640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8041511f0 0x7fc8040b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc7fbe00640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8040cf5f0 0x7fc8040b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 -- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8040cf5f0 msgr2=0x7fc8040b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8040cf5f0 0x7fc8040b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 -- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8041511f0 msgr2=0x7fc8040b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8041511f0 0x7fc8040b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 -- 172.21.15.97:0/2638793364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80414ef70 con 0x7fc80406ec40 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.954+0000 7fc808e00640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc80406ec40 0x7fc80414ea30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c500 tx=0x7fc7f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc801600640 1 -- 172.21.15.97:0/2638793364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4017070 con 0x7fc80406ec40 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc801600640 1 -- 172.21.15.97:0/2638793364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f4003600 con 0x7fc80406ec40 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc801600640 1 -- 172.21.15.97:0/2638793364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4003a90 con 0x7fc80406ec40 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8041f8ad0 con 0x7fc80406ec40 2024-09-10T21:27:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc804151b40 con 0x7fc80406ec40 2024-09-10T21:27:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80406f070 con 0x7fc80406ec40 2024-09-10T21:27:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc801600640 1 -- 172.21.15.97:0/2638793364 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc7f4003c30 con 0x7fc80406ec40 2024-09-10T21:27:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc801600640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7e00725e0 0x7fc7e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.958+0000 7fc801600640 1 -- 172.21.15.97:0/2638793364 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fc7f4092ba0 con 0x7fc80406ec40 2024-09-10T21:27:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.962+0000 7fc7fbe00640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7e00725e0 0x7fc7e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:03.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.962+0000 7fc7fbe00640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7e00725e0 0x7fc7e0074aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc8041e9600 tx=0x7fc7ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:03.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:03.962+0000 7fc801600640 1 -- 172.21.15.97:0/2638793364 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc7f405e1a0 con 0x7fc80406ec40 2024-09-10T21:27:04.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.130+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc804076120 con 0x7fc80406ec40 2024-09-10T21:27:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc801600640 1 -- 172.21.15.97:0/2638793364 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17016 (secure 0 0 0) 0x7fc7f405daa0 con 0x7fc80406ec40 2024-09-10T21:27:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":67,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:27:01.387862+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:57.469287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:58.747272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:27:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7e00725e0 msgr2=0x7fc7e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7e00725e0 0x7fc7e0074aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc8041e9600 tx=0x7fc7ec008040 comp rx=0 tx=0).stop 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc80406ec40 msgr2=0x7fc80414ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc80406ec40 0x7fc80414ea30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c500 tx=0x7fc7f400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 shutdown_connections 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7e00725e0 0x7fc7e0074aa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8041511f0 0x7fc8040b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8040cf5f0 0x7fc8040b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 --2- 172.21.15.97:0/2638793364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc80406ec40 0x7fc80414ea30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 >> 172.21.15.97:0/2638793364 conn(0x7fc8040b1780 msgr2=0x7fc8040cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 shutdown_connections 2024-09-10T21:27:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.134+0000 7fc80b800640 1 -- 172.21.15.97:0/2638793364 wait complete. 2024-09-10T21:27:04.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:27:04.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:27:04.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- 172.21.15.97:0/3446229900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c15fc10 msgr2=0x7fcc6c13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 --2- 172.21.15.97:0/3446229900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c15fc10 0x7fcc6c13f9b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c074800 tx=0x7fcc6401cd60 comp rx=0 tx=0).stop 2024-09-10T21:27:04.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- 172.21.15.97:0/3446229900 shutdown_connections 2024-09-10T21:27:04.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 --2- 172.21.15.97:0/3446229900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c15fc10 0x7fcc6c13f9b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 --2- 172.21.15.97:0/3446229900 >> v2:172.21.15.194:3300/0 conn(0x7fcc6c141d20 0x7fcc6c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- 172.21.15.97:0/3446229900 >> 172.21.15.97:0/3446229900 conn(0x7fcc6c0b1780 msgr2=0x7fcc6c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- 172.21.15.97:0/3446229900 shutdown_connections 2024-09-10T21:27:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- 172.21.15.97:0/3446229900 wait complete. 2024-09-10T21:27:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 Processor -- start 2024-09-10T21:27:04.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- start start 2024-09-10T21:27:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0ce5f0 0x7fcc6c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc6c141d20 0x7fcc6c13b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc6c15fc10 0x7fcc6c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c168a70 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c140cd0 con 0x7fcc6c15fc10 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc74200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c168bf0 con 0x7fcc6c141d20 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0ce5f0 0x7fcc6c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0ce5f0 0x7fcc6c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52620/0 (socket says 172.21.15.97:52620) 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 -- 172.21.15.97:0/3561429584 learned_addr learned my addr 172.21.15.97:0/3561429584 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc70e00640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc6c141d20 0x7fcc6c13b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc72200640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc6c15fc10 0x7fcc6c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 -- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc6c141d20 msgr2=0x7fcc6c13b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc6c141d20 0x7fcc6c13b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 -- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc6c15fc10 msgr2=0x7fcc6c0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc6c15fc10 0x7fcc6c0b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.298+0000 7fcc71800640 1 -- 172.21.15.97:0/3561429584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c15eaf0 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc71800640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0ce5f0 0x7fcc6c1393c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c500 tx=0x7fcc6000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc69600640 1 -- 172.21.15.97:0/3561429584 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60017070 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc69600640 1 -- 172.21.15.97:0/3561429584 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc600053c0 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c15edd0 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc69600640 1 -- 172.21.15.97:0/3561429584 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60012890 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c13b6c0 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6c15f6d0 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc69600640 1 -- 172.21.15.97:0/3561429584 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcc60003600 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc69600640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc4c0725e0 0x7fcc4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc69600640 1 -- 172.21.15.97:0/3561429584 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fcc60092980 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.302+0000 7fcc70e00640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc4c0725e0 0x7fcc4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.306+0000 7fcc70e00640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc4c0725e0 0x7fcc4c074aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0b3c40 tx=0x7fcc5800b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.306+0000 7fcc69600640 1 -- 172.21.15.97:0/3561429584 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc6005d7f0 con 0x7fcc6c0ce5f0 2024-09-10T21:27:04.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.474+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc6c141360 con 0x7fcc4c0725e0 2024-09-10T21:27:04.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.478+0000 7fcc69600640 1 -- 172.21.15.97:0/3561429584 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252010 (secure 0 0 0) 0x7fcc6c141360 con 0x7fcc4c0725e0 2024-09-10T21:27:04.486 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:04.488 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc4c0725e0 msgr2=0x7fcc4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc4c0725e0 0x7fcc4c074aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0b3c40 tx=0x7fcc5800b390 comp rx=0 tx=0).stop 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0ce5f0 msgr2=0x7fcc6c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0ce5f0 0x7fcc6c1393c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c500 tx=0x7fcc6000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 shutdown_connections 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc4c0725e0 0x7fcc4c074aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc6c15fc10 0x7fcc6c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc6c141d20 0x7fcc6c13b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 --2- 172.21.15.97:0/3561429584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0ce5f0 0x7fcc6c1393c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 >> 172.21.15.97:0/3561429584 conn(0x7fcc6c0b1780 msgr2=0x7fcc6c13e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 shutdown_connections 2024-09-10T21:27:04.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.482+0000 7fcc74200640 1 -- 172.21.15.97:0/3561429584 wait complete. 2024-09-10T21:27:04.505 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-10T21:27:04.111398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223652,"kb_used_data":7412,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751068,"statfs":{"total":773094113280,"available":772865093632,"internally_reserved":0,"allocated":7589888,"data_stored":4039920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.735361"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.422104+0000","last_change":"2024-09-10T21:27:00.417538+0000","last_active":"2024-09-10T21:27:00.422104+0000","last_peered":"2024-09-10T21:27:00.422104+0000","last_clean":"2024-09-10T21:27:00.422104+0000","last_became_active":"2024-09-10T21:27:00.417090+0000","last_became_peered":"2024-09-10T21:27:00.417090+0000","last_unstale":"2024-09-10T21:27:00.422104+0000","last_undegraded":"2024-09-10T21:27:00.422104+0000","last_fullsized":"2024-09-10T21:27:00.422104+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:34.790132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.415910+0000","last_change":"2024-09-10T21:27:00.409647+0000","last_active":"2024-09-10T21:27:00.415910+0000","last_peered":"2024-09-10T21:27:00.415910+0000","last_clean":"2024-09-10T21:27:00.415910+0000","last_became_active":"2024-09-10T21:27:00.409372+0000","last_became_peered":"2024-09-10T21:27:00.409372+0000","last_unstale":"2024-09-10T21:27:00.415910+0000","last_undegraded":"2024-09-10T21:27:00.415910+0000","last_fullsized":"2024-09-10T21:27:00.415910+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:52.367505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425466+0000","last_change":"2024-09-10T21:27:00.410734+0000","last_active":"2024-09-10T21:27:00.425466+0000","last_peered":"2024-09-10T21:27:00.425466+0000","last_clean":"2024-09-10T21:27:00.425466+0000","last_became_active":"2024-09-10T21:27:00.410320+0000","last_became_peered":"2024-09-10T21:27:00.410320+0000","last_unstale":"2024-09-10T21:27:00.425466+0000","last_undegraded":"2024-09-10T21:27:00.425466+0000","last_fullsized":"2024-09-10T21:27:00.425466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:48.170664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426942+0000","last_change":"2024-09-10T21:27:00.419743+0000","last_active":"2024-09-10T21:27:00.426942+0000","last_peered":"2024-09-10T21:27:00.426942+0000","last_clean":"2024-09-10T21:27:00.426942+0000","last_became_active":"2024-09-10T21:27:00.419466+0000","last_became_peered":"2024-09-10T21:27:00.419466+0000","last_unstale":"2024-09-10T21:27:00.426942+0000","last_undegraded":"2024-09-10T21:27:00.426942+0000","last_fullsized":"2024-09-10T21:27:00.426942+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.343005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425949+0000","last_change":"2024-09-10T21:27:00.414142+0000","last_active":"2024-09-10T21:27:00.425949+0000","last_peered":"2024-09-10T21:27:00.425949+0000","last_clean":"2024-09-10T21:27:00.425949+0000","last_became_active":"2024-09-10T21:27:00.413515+0000","last_became_peered":"2024-09-10T21:27:00.413515+0000","last_unstale":"2024-09-10T21:27:00.425949+0000","last_undegraded":"2024-09-10T21:27:00.425949+0000","last_fullsized":"2024-09-10T21:27:00.425949+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:32.574201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.419396+0000","last_change":"2024-09-10T21:27:00.414039+0000","last_active":"2024-09-10T21:27:00.419396+0000","last_peered":"2024-09-10T21:27:00.419396+0000","last_clean":"2024-09-10T21:27:00.419396+0000","last_became_active":"2024-09-10T21:27:00.413749+0000","last_became_peered":"2024-09-10T21:27:00.413749+0000","last_unstale":"2024-09-10T21:27:00.419396+0000","last_undegraded":"2024-09-10T21:27:00.419396+0000","last_fullsized":"2024-09-10T21:27:00.419396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:01.136669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.422213+0000","last_change":"2024-09-10T21:27:00.418330+0000","last_active":"2024-09-10T21:27:00.422213+0000","last_peered":"2024-09-10T21:27:00.422213+0000","last_clean":"2024-09-10T21:27:00.422213+0000","last_became_active":"2024-09-10T21:27:00.417957+0000","last_became_peered":"2024-09-10T21:27:00.417957+0000","last_unstale":"2024-09-10T21:27:00.422213+0000","last_undegraded":"2024-09-10T21:27:00.422213+0000","last_fullsized":"2024-09-10T21:27:00.422213+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:01.241843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.421547+0000","last_change":"2024-09-10T21:27:00.407145+0000","last_active":"2024-09-10T21:27:00.421547+0000","last_peered":"2024-09-10T21:27:00.421547+0000","last_clean":"2024-09-10T21:27:00.421547+0000","last_became_active":"2024-09-10T21:27:00.406896+0000","last_became_peered":"2024-09-10T21:27:00.406896+0000","last_unstale":"2024-09-10T21:27:00.421547+0000","last_undegraded":"2024-09-10T21:27:00.421547+0000","last_fullsized":"2024-09-10T21:27:00.421547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:04.929735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.427462+0000","last_change":"2024-09-10T21:27:00.423987+0000","last_active":"2024-09-10T21:27:00.427462+0000","last_peered":"2024-09-10T21:27:00.427462+0000","last_clean":"2024-09-10T21:27:00.427462+0000","last_became_active":"2024-09-10T21:27:00.422900+0000","last_became_peered":"2024-09-10T21:27:00.422900+0000","last_unstale":"2024-09-10T21:27:00.427462+0000","last_undegraded":"2024-09-10T21:27:00.427462+0000","last_fullsized":"2024-09-10T21:27:00.427462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:52.074995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.428089+0000","last_change":"2024-09-10T21:27:00.423473+0000","last_active":"2024-09-10T21:27:00.428089+0000","last_peered":"2024-09-10T21:27:00.428089+0000","last_clean":"2024-09-10T21:27:00.428089+0000","last_became_active":"2024-09-10T21:27:00.421864+0000","last_became_peered":"2024-09-10T21:27:00.421864+0000","last_unstale":"2024-09-10T21:27:00.428089+0000","last_undegraded":"2024-09-10T21:27:00.428089+0000","last_fullsized":"2024-09-10T21:27:00.428089+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:45.931525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426479+0000","last_change":"2024-09-10T21:27:00.418444+0000","last_active":"2024-09-10T21:27:00.426479+0000","last_peered":"2024-09-10T21:27:00.426479+0000","last_clean":"2024-09-10T21:27:00.426479+0000","last_became_active":"2024-09-10T21:27:00.415097+0000","last_became_peered":"2024-09-10T21:27:00.415097+0000","last_unstale":"2024-09-10T21:27:00.426479+0000","last_undegraded":"2024-09-10T21:27:00.426479+0000","last_fullsized":"2024-09-10T21:27:00.426479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:47.622969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426212+0000","last_change":"2024-09-10T21:27:00.415596+0000","last_active":"2024-09-10T21:27:00.426212+0000","last_peered":"2024-09-10T21:27:00.426212+0000","last_clean":"2024-09-10T21:27:00.426212+0000","last_became_active":"2024-09-10T21:27:00.415356+0000","last_became_peered":"2024-09-10T21:27:00.415356+0000","last_unstale":"2024-09-10T21:27:00.426212+0000","last_undegraded":"2024-09-10T21:27:00.426212+0000","last_fullsized":"2024-09-10T21:27:00.426212+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:47.798687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.411135+0000","last_change":"2024-09-10T21:27:00.409011+0000","last_active":"2024-09-10T21:27:00.411135+0000","last_peered":"2024-09-10T21:27:00.411135+0000","last_clean":"2024-09-10T21:27:00.411135+0000","last_became_active":"2024-09-10T21:27:00.408745+0000","last_became_peered":"2024-09-10T21:27:00.408745+0000","last_unstale":"2024-09-10T21:27:00.411135+0000","last_undegraded":"2024-09-10T21:27:00.411135+0000","last_fullsized":"2024-09-10T21:27:00.411135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:29.158533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.411011+0000","last_change":"2024-09-10T21:27:00.400889+0000","last_active":"2024-09-10T21:27:00.411011+0000","last_peered":"2024-09-10T21:27:00.411011+0000","last_clean":"2024-09-10T21:27:00.411011+0000","last_became_active":"2024-09-10T21:27:00.400624+0000","last_became_peered":"2024-09-10T21:27:00.400624+0000","last_unstale":"2024-09-10T21:27:00.411011+0000","last_undegraded":"2024-09-10T21:27:00.411011+0000","last_fullsized":"2024-09-10T21:27:00.411011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:55.167189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426288+0000","last_change":"2024-09-10T21:27:00.421139+0000","last_active":"2024-09-10T21:27:00.426288+0000","last_peered":"2024-09-10T21:27:00.426288+0000","last_clean":"2024-09-10T21:27:00.426288+0000","last_became_active":"2024-09-10T21:27:00.420387+0000","last_became_peered":"2024-09-10T21:27:00.420387+0000","last_unstale":"2024-09-10T21:27:00.426288+0000","last_undegraded":"2024-09-10T21:27:00.426288+0000","last_fullsized":"2024-09-10T21:27:00.426288+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:55.167189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.414174+0000","last_change":"2024-09-10T21:27:00.411650+0000","last_active":"2024-09-10T21:27:00.414174+0000","last_peered":"2024-09-10T21:27:00.414174+0000","last_clean":"2024-09-10T21:27:00.414174+0000","last_became_active":"2024-09-10T21:27:00.411333+0000","last_became_peered":"2024-09-10T21:27:00.411333+0000","last_unstale":"2024-09-10T21:27:00.414174+0000","last_undegraded":"2024-09-10T21:27:00.414174+0000","last_fullsized":"2024-09-10T21:27:00.414174+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:29.687670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395200+0000","last_change":"2024-09-10T21:27:00.405579+0000","last_active":"2024-09-10T21:27:01.395200+0000","last_peered":"2024-09-10T21:27:01.395200+0000","last_clean":"2024-09-10T21:27:01.395200+0000","last_became_active":"2024-09-10T21:27:00.404926+0000","last_became_peered":"2024-09-10T21:27:00.404926+0000","last_unstale":"2024-09-10T21:27:01.395200+0000","last_undegraded":"2024-09-10T21:27:01.395200+0000","last_fullsized":"2024-09-10T21:27:01.395200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:44.952921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425452+0000","last_change":"2024-09-10T21:27:00.411387+0000","last_active":"2024-09-10T21:27:00.425452+0000","last_peered":"2024-09-10T21:27:00.425452+0000","last_clean":"2024-09-10T21:27:00.425452+0000","last_became_active":"2024-09-10T21:27:00.411141+0000","last_became_peered":"2024-09-10T21:27:00.411141+0000","last_unstale":"2024-09-10T21:27:00.425452+0000","last_undegraded":"2024-09-10T21:27:00.425452+0000","last_fullsized":"2024-09-10T21:27:00.425452+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:04.885814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426122+0000","last_change":"2024-09-10T21:27:00.413655+0000","last_active":"2024-09-10T21:27:00.426122+0000","last_peered":"2024-09-10T21:27:00.426122+0000","last_clean":"2024-09-10T21:27:00.426122+0000","last_became_active":"2024-09-10T21:27:00.413382+0000","last_became_peered":"2024-09-10T21:27:00.413382+0000","last_unstale":"2024-09-10T21:27:00.426122+0000","last_undegraded":"2024-09-10T21:27:00.426122+0000","last_fullsized":"2024-09-10T21:27:00.426122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:22.582701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394722+0000","last_change":"2024-09-10T21:27:00.406759+0000","last_active":"2024-09-10T21:27:01.394722+0000","last_peered":"2024-09-10T21:27:01.394722+0000","last_clean":"2024-09-10T21:27:01.394722+0000","last_became_active":"2024-09-10T21:27:00.406470+0000","last_became_peered":"2024-09-10T21:27:00.406470+0000","last_unstale":"2024-09-10T21:27:01.394722+0000","last_undegraded":"2024-09-10T21:27:01.394722+0000","last_fullsized":"2024-09-10T21:27:01.394722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:25.025007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.408634+0000","last_change":"2024-09-10T21:27:00.401505+0000","last_active":"2024-09-10T21:27:00.408634+0000","last_peered":"2024-09-10T21:27:00.408634+0000","last_clean":"2024-09-10T21:27:00.408634+0000","last_became_active":"2024-09-10T21:27:00.401245+0000","last_became_peered":"2024-09-10T21:27:00.401245+0000","last_unstale":"2024-09-10T21:27:00.408634+0000","last_undegraded":"2024-09-10T21:27:00.408634+0000","last_fullsized":"2024-09-10T21:27:00.408634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:39.087323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.417166+0000","last_change":"2024-09-10T21:27:00.414314+0000","last_active":"2024-09-10T21:27:00.417166+0000","last_peered":"2024-09-10T21:27:00.417166+0000","last_clean":"2024-09-10T21:27:00.417166+0000","last_became_active":"2024-09-10T21:27:00.413524+0000","last_became_peered":"2024-09-10T21:27:00.413524+0000","last_unstale":"2024-09-10T21:27:00.417166+0000","last_undegraded":"2024-09-10T21:27:00.417166+0000","last_fullsized":"2024-09-10T21:27:00.417166+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:45.231553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.419140+0000","last_change":"2024-09-10T21:27:00.415626+0000","last_active":"2024-09-10T21:27:00.419140+0000","last_peered":"2024-09-10T21:27:00.419140+0000","last_clean":"2024-09-10T21:27:00.419140+0000","last_became_active":"2024-09-10T21:27:00.415370+0000","last_became_peered":"2024-09-10T21:27:00.415370+0000","last_unstale":"2024-09-10T21:27:00.419140+0000","last_undegraded":"2024-09-10T21:27:00.419140+0000","last_fullsized":"2024-09-10T21:27:00.419140+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:07.726401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.407048+0000","last_change":"2024-09-10T21:27:00.402787+0000","last_active":"2024-09-10T21:27:00.407048+0000","last_peered":"2024-09-10T21:27:00.407048+0000","last_clean":"2024-09-10T21:27:00.407048+0000","last_became_active":"2024-09-10T21:27:00.402565+0000","last_became_peered":"2024-09-10T21:27:00.402565+0000","last_unstale":"2024-09-10T21:27:00.407048+0000","last_undegraded":"2024-09-10T21:27:00.407048+0000","last_fullsized":"2024-09-10T21:27:00.407048+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:34.143667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.414451+0000","last_change":"2024-09-10T21:27:00.401602+0000","last_active":"2024-09-10T21:27:00.414451+0000","last_peered":"2024-09-10T21:27:00.414451+0000","last_clean":"2024-09-10T21:27:00.414451+0000","last_became_active":"2024-09-10T21:27:00.401316+0000","last_became_peered":"2024-09-10T21:27:00.401316+0000","last_unstale":"2024-09-10T21:27:00.414451+0000","last_undegraded":"2024-09-10T21:27:00.414451+0000","last_fullsized":"2024-09-10T21:27:00.414451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:45.931525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.401567+0000","last_change":"2024-09-10T21:27:00.399850+0000","last_active":"2024-09-10T21:27:00.401567+0000","last_peered":"2024-09-10T21:27:00.401567+0000","last_clean":"2024-09-10T21:27:00.401567+0000","last_became_active":"2024-09-10T21:27:00.399602+0000","last_became_peered":"2024-09-10T21:27:00.399602+0000","last_unstale":"2024-09-10T21:27:00.401567+0000","last_undegraded":"2024-09-10T21:27:00.401567+0000","last_fullsized":"2024-09-10T21:27:00.401567+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:46.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.421641+0000","last_change":"2024-09-10T21:27:00.412350+0000","last_active":"2024-09-10T21:27:00.421641+0000","last_peered":"2024-09-10T21:27:00.421641+0000","last_clean":"2024-09-10T21:27:00.421641+0000","last_became_active":"2024-09-10T21:27:00.411825+0000","last_became_peered":"2024-09-10T21:27:00.411825+0000","last_unstale":"2024-09-10T21:27:00.421641+0000","last_undegraded":"2024-09-10T21:27:00.421641+0000","last_fullsized":"2024-09-10T21:27:00.421641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:02.603528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.432778+0000","last_change":"2024-09-10T21:27:00.427324+0000","last_active":"2024-09-10T21:27:00.432778+0000","last_peered":"2024-09-10T21:27:00.432778+0000","last_clean":"2024-09-10T21:27:00.432778+0000","last_became_active":"2024-09-10T21:27:00.426596+0000","last_became_peered":"2024-09-10T21:27:00.426596+0000","last_unstale":"2024-09-10T21:27:00.432778+0000","last_undegraded":"2024-09-10T21:27:00.432778+0000","last_fullsized":"2024-09-10T21:27:00.432778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:04.885814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426501+0000","last_change":"2024-09-10T21:27:00.416352+0000","last_active":"2024-09-10T21:27:00.426501+0000","last_peered":"2024-09-10T21:27:00.426501+0000","last_clean":"2024-09-10T21:27:00.426501+0000","last_became_active":"2024-09-10T21:27:00.414024+0000","last_became_peered":"2024-09-10T21:27:00.414024+0000","last_unstale":"2024-09-10T21:27:00.426501+0000","last_undegraded":"2024-09-10T21:27:00.426501+0000","last_fullsized":"2024-09-10T21:27:00.426501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:04.960471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426972+0000","last_change":"2024-09-10T21:27:00.422316+0000","last_active":"2024-09-10T21:27:00.426972+0000","last_peered":"2024-09-10T21:27:00.426972+0000","last_clean":"2024-09-10T21:27:00.426972+0000","last_became_active":"2024-09-10T21:27:00.422027+0000","last_became_peered":"2024-09-10T21:27:00.422027+0000","last_unstale":"2024-09-10T21:27:00.426972+0000","last_undegraded":"2024-09-10T21:27:00.426972+0000","last_fullsized":"2024-09-10T21:27:00.426972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:39.690168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.420143+0000","last_change":"2024-09-10T21:27:00.405764+0000","last_active":"2024-09-10T21:27:00.420143+0000","last_peered":"2024-09-10T21:27:00.420143+0000","last_clean":"2024-09-10T21:27:00.420143+0000","last_became_active":"2024-09-10T21:27:00.405524+0000","last_became_peered":"2024-09-10T21:27:00.405524+0000","last_unstale":"2024-09-10T21:27:00.420143+0000","last_undegraded":"2024-09-10T21:27:00.420143+0000","last_fullsized":"2024-09-10T21:27:00.420143+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.205100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.415398+0000","last_change":"2024-09-10T21:27:00.410003+0000","last_active":"2024-09-10T21:27:00.415398+0000","last_peered":"2024-09-10T21:27:00.415398+0000","last_clean":"2024-09-10T21:27:00.415398+0000","last_became_active":"2024-09-10T21:27:00.409705+0000","last_became_peered":"2024-09-10T21:27:00.409705+0000","last_unstale":"2024-09-10T21:27:00.415398+0000","last_undegraded":"2024-09-10T21:27:00.415398+0000","last_fullsized":"2024-09-10T21:27:00.415398+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:49.974884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426105+0000","last_change":"2024-09-10T21:27:00.413588+0000","last_active":"2024-09-10T21:27:00.426105+0000","last_peered":"2024-09-10T21:27:00.426105+0000","last_clean":"2024-09-10T21:27:00.426105+0000","last_became_active":"2024-09-10T21:27:00.413325+0000","last_became_peered":"2024-09-10T21:27:00.413325+0000","last_unstale":"2024-09-10T21:27:00.426105+0000","last_undegraded":"2024-09-10T21:27:00.426105+0000","last_fullsized":"2024-09-10T21:27:00.426105+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:11.964526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.432889+0000","last_change":"2024-09-10T21:26:59.399802+0000","last_active":"2024-09-10T21:27:00.432889+0000","last_peered":"2024-09-10T21:27:00.432889+0000","last_clean":"2024-09-10T21:27:00.432889+0000","last_became_active":"2024-09-10T21:26:59.399546+0000","last_became_peered":"2024-09-10T21:26:59.399546+0000","last_unstale":"2024-09-10T21:27:00.432889+0000","last_undegraded":"2024-09-10T21:27:00.432889+0000","last_fullsized":"2024-09-10T21:27:00.432889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:01.596056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425444+0000","last_change":"2024-09-10T21:27:00.410357+0000","last_active":"2024-09-10T21:27:00.425444+0000","last_peered":"2024-09-10T21:27:00.425444+0000","last_clean":"2024-09-10T21:27:00.425444+0000","last_became_active":"2024-09-10T21:27:00.410004+0000","last_became_peered":"2024-09-10T21:27:00.410004+0000","last_unstale":"2024-09-10T21:27:00.425444+0000","last_undegraded":"2024-09-10T21:27:00.425444+0000","last_fullsized":"2024-09-10T21:27:00.425444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:32.954234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.434121+0000","last_change":"2024-09-10T21:26:59.401491+0000","last_active":"2024-09-10T21:27:00.434121+0000","last_peered":"2024-09-10T21:27:00.434121+0000","last_clean":"2024-09-10T21:27:00.434121+0000","last_became_active":"2024-09-10T21:26:59.400643+0000","last_became_peered":"2024-09-10T21:26:59.400643+0000","last_unstale":"2024-09-10T21:27:00.434121+0000","last_undegraded":"2024-09-10T21:27:00.434121+0000","last_fullsized":"2024-09-10T21:27:00.434121+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:38.637358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.408846+0000","last_change":"2024-09-10T21:27:00.405856+0000","last_active":"2024-09-10T21:27:00.408846+0000","last_peered":"2024-09-10T21:27:00.408846+0000","last_clean":"2024-09-10T21:27:00.408846+0000","last_became_active":"2024-09-10T21:27:00.405588+0000","last_became_peered":"2024-09-10T21:27:00.405588+0000","last_unstale":"2024-09-10T21:27:00.408846+0000","last_undegraded":"2024-09-10T21:27:00.408846+0000","last_fullsized":"2024-09-10T21:27:00.408846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:54.237250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.431491+0000","last_change":"2024-09-10T21:26:59.398906+0000","last_active":"2024-09-10T21:27:00.431491+0000","last_peered":"2024-09-10T21:27:00.431491+0000","last_clean":"2024-09-10T21:27:00.431491+0000","last_became_active":"2024-09-10T21:26:59.398638+0000","last_became_peered":"2024-09-10T21:26:59.398638+0000","last_unstale":"2024-09-10T21:27:00.431491+0000","last_undegraded":"2024-09-10T21:27:00.431491+0000","last_fullsized":"2024-09-10T21:27:00.431491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:16.192993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425949+0000","last_change":"2024-09-10T21:27:00.410657+0000","last_active":"2024-09-10T21:27:00.425949+0000","last_peered":"2024-09-10T21:27:00.425949+0000","last_clean":"2024-09-10T21:27:00.425949+0000","last_became_active":"2024-09-10T21:27:00.410410+0000","last_became_peered":"2024-09-10T21:27:00.410410+0000","last_unstale":"2024-09-10T21:27:00.425949+0000","last_undegraded":"2024-09-10T21:27:00.425949+0000","last_fullsized":"2024-09-10T21:27:00.425949+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:35.443620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.433471+0000","last_change":"2024-09-10T21:26:59.401767+0000","last_active":"2024-09-10T21:27:00.433471+0000","last_peered":"2024-09-10T21:27:00.433471+0000","last_clean":"2024-09-10T21:27:00.433471+0000","last_became_active":"2024-09-10T21:26:59.400887+0000","last_became_peered":"2024-09-10T21:26:59.400887+0000","last_unstale":"2024-09-10T21:27:00.433471+0000","last_undegraded":"2024-09-10T21:27:00.433471+0000","last_fullsized":"2024-09-10T21:27:00.433471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:04.180850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394000+0000","last_change":"2024-09-10T21:27:00.398718+0000","last_active":"2024-09-10T21:27:01.394000+0000","last_peered":"2024-09-10T21:27:01.394000+0000","last_clean":"2024-09-10T21:27:01.394000+0000","last_became_active":"2024-09-10T21:27:00.398401+0000","last_became_peered":"2024-09-10T21:27:00.398401+0000","last_unstale":"2024-09-10T21:27:01.394000+0000","last_undegraded":"2024-09-10T21:27:01.394000+0000","last_fullsized":"2024-09-10T21:27:01.394000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:06.312783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395053+0000","last_change":"2024-09-10T21:26:59.396906+0000","last_active":"2024-09-10T21:27:01.395053+0000","last_peered":"2024-09-10T21:27:01.395053+0000","last_clean":"2024-09-10T21:27:01.395053+0000","last_became_active":"2024-09-10T21:26:59.394994+0000","last_became_peered":"2024-09-10T21:26:59.394994+0000","last_unstale":"2024-09-10T21:27:01.395053+0000","last_undegraded":"2024-09-10T21:27:01.395053+0000","last_fullsized":"2024-09-10T21:27:01.395053+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:44.412678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.424782+0000","last_change":"2024-09-10T21:27:00.420108+0000","last_active":"2024-09-10T21:27:00.424782+0000","last_peered":"2024-09-10T21:27:00.424782+0000","last_clean":"2024-09-10T21:27:00.424782+0000","last_became_active":"2024-09-10T21:27:00.419819+0000","last_became_peered":"2024-09-10T21:27:00.419819+0000","last_unstale":"2024-09-10T21:27:00.424782+0000","last_undegraded":"2024-09-10T21:27:00.424782+0000","last_fullsized":"2024-09-10T21:27:00.424782+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:35.443620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.433376+0000","last_change":"2024-09-10T21:26:59.396168+0000","last_active":"2024-09-10T21:27:00.433376+0000","last_peered":"2024-09-10T21:27:00.433376+0000","last_clean":"2024-09-10T21:27:00.433376+0000","last_became_active":"2024-09-10T21:26:59.395311+0000","last_became_peered":"2024-09-10T21:26:59.395311+0000","last_unstale":"2024-09-10T21:27:00.433376+0000","last_undegraded":"2024-09-10T21:27:00.433376+0000","last_fullsized":"2024-09-10T21:27:00.433376+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:51.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.435153+0000","last_change":"2024-09-10T21:27:00.427703+0000","last_active":"2024-09-10T21:27:00.435153+0000","last_peered":"2024-09-10T21:27:00.435153+0000","last_clean":"2024-09-10T21:27:00.435153+0000","last_became_active":"2024-09-10T21:27:00.427393+0000","last_became_peered":"2024-09-10T21:27:00.427393+0000","last_unstale":"2024-09-10T21:27:00.435153+0000","last_undegraded":"2024-09-10T21:27:00.435153+0000","last_fullsized":"2024-09-10T21:27:00.435153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:41.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.402238+0000","last_change":"2024-09-10T21:26:59.402715+0000","last_active":"2024-09-10T21:27:00.402238+0000","last_peered":"2024-09-10T21:27:00.402238+0000","last_clean":"2024-09-10T21:27:00.402238+0000","last_became_active":"2024-09-10T21:26:59.402452+0000","last_became_peered":"2024-09-10T21:26:59.402452+0000","last_unstale":"2024-09-10T21:27:00.402238+0000","last_undegraded":"2024-09-10T21:27:00.402238+0000","last_fullsized":"2024-09-10T21:27:00.402238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:28.151060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394750+0000","last_change":"2024-09-10T21:27:00.405605+0000","last_active":"2024-09-10T21:27:01.394750+0000","last_peered":"2024-09-10T21:27:01.394750+0000","last_clean":"2024-09-10T21:27:01.394750+0000","last_became_active":"2024-09-10T21:27:00.404743+0000","last_became_peered":"2024-09-10T21:27:00.404743+0000","last_unstale":"2024-09-10T21:27:01.394750+0000","last_undegraded":"2024-09-10T21:27:01.394750+0000","last_fullsized":"2024-09-10T21:27:01.394750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:50.241177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.397390+0000","last_change":"2024-09-10T21:26:59.402712+0000","last_active":"2024-09-10T21:27:00.397390+0000","last_peered":"2024-09-10T21:27:00.397390+0000","last_clean":"2024-09-10T21:27:00.397390+0000","last_became_active":"2024-09-10T21:26:59.402449+0000","last_became_peered":"2024-09-10T21:26:59.402449+0000","last_unstale":"2024-09-10T21:27:00.397390+0000","last_undegraded":"2024-09-10T21:27:00.397390+0000","last_fullsized":"2024-09-10T21:27:00.397390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:11.704238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.416597+0000","last_change":"2024-09-10T21:27:00.413055+0000","last_active":"2024-09-10T21:27:00.416597+0000","last_peered":"2024-09-10T21:27:00.416597+0000","last_clean":"2024-09-10T21:27:00.416597+0000","last_became_active":"2024-09-10T21:27:00.412722+0000","last_became_peered":"2024-09-10T21:27:00.412722+0000","last_unstale":"2024-09-10T21:27:00.416597+0000","last_undegraded":"2024-09-10T21:27:00.416597+0000","last_fullsized":"2024-09-10T21:27:00.416597+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:27.616077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.398703+0000","last_change":"2024-09-10T21:26:59.397537+0000","last_active":"2024-09-10T21:27:00.398703+0000","last_peered":"2024-09-10T21:27:00.398703+0000","last_clean":"2024-09-10T21:27:00.398703+0000","last_became_active":"2024-09-10T21:26:59.396169+0000","last_became_peered":"2024-09-10T21:26:59.396169+0000","last_unstale":"2024-09-10T21:27:00.398703+0000","last_undegraded":"2024-09-10T21:27:00.398703+0000","last_fullsized":"2024-09-10T21:27:00.398703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:48.967412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395077+0000","last_change":"2024-09-10T21:27:00.408243+0000","last_active":"2024-09-10T21:27:01.395077+0000","last_peered":"2024-09-10T21:27:01.395077+0000","last_clean":"2024-09-10T21:27:01.395077+0000","last_became_active":"2024-09-10T21:27:00.407959+0000","last_became_peered":"2024-09-10T21:27:00.407959+0000","last_unstale":"2024-09-10T21:27:01.395077+0000","last_undegraded":"2024-09-10T21:27:01.395077+0000","last_fullsized":"2024-09-10T21:27:01.395077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:32.719778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.390863+0000","last_change":"2024-09-10T21:26:59.387728+0000","last_active":"2024-09-10T21:27:00.390863+0000","last_peered":"2024-09-10T21:27:00.390863+0000","last_clean":"2024-09-10T21:27:00.390863+0000","last_became_active":"2024-09-10T21:26:59.387436+0000","last_became_peered":"2024-09-10T21:26:59.387436+0000","last_unstale":"2024-09-10T21:27:00.390863+0000","last_undegraded":"2024-09-10T21:27:00.390863+0000","last_fullsized":"2024-09-10T21:27:00.390863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:02.312756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.413565+0000","last_change":"2024-09-10T21:27:00.410701+0000","last_active":"2024-09-10T21:27:00.413565+0000","last_peered":"2024-09-10T21:27:00.413565+0000","last_clean":"2024-09-10T21:27:00.413565+0000","last_became_active":"2024-09-10T21:27:00.410447+0000","last_became_peered":"2024-09-10T21:27:00.410447+0000","last_unstale":"2024-09-10T21:27:00.413565+0000","last_undegraded":"2024-09-10T21:27:00.413565+0000","last_fullsized":"2024-09-10T21:27:00.413565+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:39.644830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.434886+0000","last_change":"2024-09-10T21:26:59.400721+0000","last_active":"2024-09-10T21:27:00.434886+0000","last_peered":"2024-09-10T21:27:00.434886+0000","last_clean":"2024-09-10T21:27:00.434886+0000","last_became_active":"2024-09-10T21:26:59.400408+0000","last_became_peered":"2024-09-10T21:26:59.400408+0000","last_unstale":"2024-09-10T21:27:00.434886+0000","last_undegraded":"2024-09-10T21:27:00.434886+0000","last_fullsized":"2024-09-10T21:27:00.434886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:19.491413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.421133+0000","last_change":"2024-09-10T21:27:00.406332+0000","last_active":"2024-09-10T21:27:00.421133+0000","last_peered":"2024-09-10T21:27:00.421133+0000","last_clean":"2024-09-10T21:27:00.421133+0000","last_became_active":"2024-09-10T21:27:00.406103+0000","last_became_peered":"2024-09-10T21:27:00.406103+0000","last_unstale":"2024-09-10T21:27:00.421133+0000","last_undegraded":"2024-09-10T21:27:00.421133+0000","last_fullsized":"2024-09-10T21:27:00.421133+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.438613+0000","last_change":"2024-09-10T21:26:59.399005+0000","last_active":"2024-09-10T21:27:00.438613+0000","last_peered":"2024-09-10T21:27:00.438613+0000","last_clean":"2024-09-10T21:27:00.438613+0000","last_became_active":"2024-09-10T21:26:59.398736+0000","last_became_peered":"2024-09-10T21:26:59.398736+0000","last_unstale":"2024-09-10T21:27:00.438613+0000","last_undegraded":"2024-09-10T21:27:00.438613+0000","last_fullsized":"2024-09-10T21:27:00.438613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:43.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.423491+0000","last_change":"2024-09-10T21:27:00.412754+0000","last_active":"2024-09-10T21:27:00.423491+0000","last_peered":"2024-09-10T21:27:00.423491+0000","last_clean":"2024-09-10T21:27:00.423491+0000","last_became_active":"2024-09-10T21:27:00.412499+0000","last_became_peered":"2024-09-10T21:27:00.412499+0000","last_unstale":"2024-09-10T21:27:00.423491+0000","last_undegraded":"2024-09-10T21:27:00.423491+0000","last_fullsized":"2024-09-10T21:27:00.423491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:23.112085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.437168+0000","last_change":"2024-09-10T21:26:59.403472+0000","last_active":"2024-09-10T21:27:00.437168+0000","last_peered":"2024-09-10T21:27:00.437168+0000","last_clean":"2024-09-10T21:27:00.437168+0000","last_became_active":"2024-09-10T21:26:59.403227+0000","last_became_peered":"2024-09-10T21:26:59.403227+0000","last_unstale":"2024-09-10T21:27:00.437168+0000","last_undegraded":"2024-09-10T21:27:00.437168+0000","last_fullsized":"2024-09-10T21:27:00.437168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:01.596056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426254+0000","last_change":"2024-09-10T21:27:00.420841+0000","last_active":"2024-09-10T21:27:00.426254+0000","last_peered":"2024-09-10T21:27:00.426254+0000","last_clean":"2024-09-10T21:27:00.426254+0000","last_became_active":"2024-09-10T21:27:00.418373+0000","last_became_peered":"2024-09-10T21:27:00.418373+0000","last_unstale":"2024-09-10T21:27:00.426254+0000","last_undegraded":"2024-09-10T21:27:00.426254+0000","last_fullsized":"2024-09-10T21:27:00.426254+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:03.320228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.403641+0000","last_change":"2024-09-10T21:26:59.406666+0000","last_active":"2024-09-10T21:27:00.403641+0000","last_peered":"2024-09-10T21:27:00.403641+0000","last_clean":"2024-09-10T21:27:00.403641+0000","last_became_active":"2024-09-10T21:26:59.406398+0000","last_became_peered":"2024-09-10T21:26:59.406398+0000","last_unstale":"2024-09-10T21:27:00.403641+0000","last_undegraded":"2024-09-10T21:27:00.403641+0000","last_fullsized":"2024-09-10T21:27:00.403641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:38.205971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426420+0000","last_change":"2024-09-10T21:27:00.421487+0000","last_active":"2024-09-10T21:27:00.426420+0000","last_peered":"2024-09-10T21:27:00.426420+0000","last_clean":"2024-09-10T21:27:00.426420+0000","last_became_active":"2024-09-10T21:27:00.421239+0000","last_became_peered":"2024-09-10T21:27:00.421239+0000","last_unstale":"2024-09-10T21:27:00.426420+0000","last_undegraded":"2024-09-10T21:27:00.426420+0000","last_fullsized":"2024-09-10T21:27:00.426420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:39.087323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.396002+0000","last_change":"2024-09-10T21:26:59.401272+0000","last_active":"2024-09-10T21:27:00.396002+0000","last_peered":"2024-09-10T21:27:00.396002+0000","last_clean":"2024-09-10T21:27:00.396002+0000","last_became_active":"2024-09-10T21:26:59.400270+0000","last_became_peered":"2024-09-10T21:26:59.400270+0000","last_unstale":"2024-09-10T21:27:00.396002+0000","last_undegraded":"2024-09-10T21:27:00.396002+0000","last_fullsized":"2024-09-10T21:27:00.396002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:34.037894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.422134+0000","last_change":"2024-09-10T21:27:00.417956+0000","last_active":"2024-09-10T21:27:00.422134+0000","last_peered":"2024-09-10T21:27:00.422134+0000","last_clean":"2024-09-10T21:27:00.422134+0000","last_became_active":"2024-09-10T21:27:00.417604+0000","last_became_peered":"2024-09-10T21:27:00.417604+0000","last_unstale":"2024-09-10T21:27:00.422134+0000","last_undegraded":"2024-09-10T21:27:00.422134+0000","last_fullsized":"2024-09-10T21:27:00.422134+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:44.524688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":21,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.438089+0000","last_change":"2024-09-10T21:26:59.405239+0000","last_active":"2024-09-10T21:27:00.438089+0000","last_peered":"2024-09-10T21:27:00.438089+0000","last_clean":"2024-09-10T21:27:00.438089+0000","last_became_active":"2024-09-10T21:26:59.404989+0000","last_became_peered":"2024-09-10T21:26:59.404989+0000","last_unstale":"2024-09-10T21:27:00.438089+0000","last_undegraded":"2024-09-10T21:27:00.438089+0000","last_fullsized":"2024-09-10T21:27:00.438089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:23.639410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425155+0000","last_change":"2024-09-10T21:27:00.405340+0000","last_active":"2024-09-10T21:27:00.425155+0000","last_peered":"2024-09-10T21:27:00.425155+0000","last_clean":"2024-09-10T21:27:00.425155+0000","last_became_active":"2024-09-10T21:27:00.405080+0000","last_became_peered":"2024-09-10T21:27:00.405080+0000","last_unstale":"2024-09-10T21:27:00.425155+0000","last_undegraded":"2024-09-10T21:27:00.425155+0000","last_fullsized":"2024-09-10T21:27:00.425155+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:44.524688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.436957+0000","last_change":"2024-09-10T21:26:59.405550+0000","last_active":"2024-09-10T21:27:00.436957+0000","last_peered":"2024-09-10T21:27:00.436957+0000","last_clean":"2024-09-10T21:27:00.436957+0000","last_became_active":"2024-09-10T21:26:59.404895+0000","last_became_peered":"2024-09-10T21:26:59.404895+0000","last_unstale":"2024-09-10T21:27:00.436957+0000","last_undegraded":"2024-09-10T21:27:00.436957+0000","last_fullsized":"2024-09-10T21:27:00.436957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:26.608605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.411078+0000","last_change":"2024-09-10T21:27:00.409430+0000","last_active":"2024-09-10T21:27:00.411078+0000","last_peered":"2024-09-10T21:27:00.411078+0000","last_clean":"2024-09-10T21:27:00.411078+0000","last_became_active":"2024-09-10T21:27:00.409256+0000","last_became_peered":"2024-09-10T21:27:00.409256+0000","last_unstale":"2024-09-10T21:27:00.411078+0000","last_undegraded":"2024-09-10T21:27:00.411078+0000","last_fullsized":"2024-09-10T21:27:00.411078+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:12.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.436162+0000","last_change":"2024-09-10T21:26:59.399031+0000","last_active":"2024-09-10T21:27:00.436162+0000","last_peered":"2024-09-10T21:27:00.436162+0000","last_clean":"2024-09-10T21:27:00.436162+0000","last_became_active":"2024-09-10T21:26:59.398723+0000","last_became_peered":"2024-09-10T21:26:59.398723+0000","last_unstale":"2024-09-10T21:27:00.436162+0000","last_undegraded":"2024-09-10T21:27:00.436162+0000","last_fullsized":"2024-09-10T21:27:00.436162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:16.192993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.423067+0000","last_change":"2024-09-10T21:27:00.417858+0000","last_active":"2024-09-10T21:27:00.423067+0000","last_peered":"2024-09-10T21:27:00.423067+0000","last_clean":"2024-09-10T21:27:00.423067+0000","last_became_active":"2024-09-10T21:27:00.417600+0000","last_became_peered":"2024-09-10T21:27:00.417600+0000","last_unstale":"2024-09-10T21:27:00.423067+0000","last_undegraded":"2024-09-10T21:27:00.423067+0000","last_fullsized":"2024-09-10T21:27:00.423067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:49.974884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.434815+0000","last_change":"2024-09-10T21:26:59.400952+0000","last_active":"2024-09-10T21:27:00.434815+0000","last_peered":"2024-09-10T21:27:00.434815+0000","last_clean":"2024-09-10T21:27:00.434815+0000","last_became_active":"2024-09-10T21:26:59.400669+0000","last_became_peered":"2024-09-10T21:26:59.400669+0000","last_unstale":"2024-09-10T21:27:00.434815+0000","last_undegraded":"2024-09-10T21:27:00.434815+0000","last_fullsized":"2024-09-10T21:27:00.434815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:34.244246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425576+0000","last_change":"2024-09-10T21:27:00.412517+0000","last_active":"2024-09-10T21:27:00.425576+0000","last_peered":"2024-09-10T21:27:00.425576+0000","last_clean":"2024-09-10T21:27:00.425576+0000","last_became_active":"2024-09-10T21:27:00.411419+0000","last_became_peered":"2024-09-10T21:27:00.411419+0000","last_unstale":"2024-09-10T21:27:00.425576+0000","last_undegraded":"2024-09-10T21:27:00.425576+0000","last_fullsized":"2024-09-10T21:27:00.425576+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:59.090638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.397163+0000","last_change":"2024-09-10T21:26:59.400293+0000","last_active":"2024-09-10T21:27:00.397163+0000","last_peered":"2024-09-10T21:27:00.397163+0000","last_clean":"2024-09-10T21:27:00.397163+0000","last_became_active":"2024-09-10T21:26:59.400002+0000","last_became_peered":"2024-09-10T21:26:59.400002+0000","last_unstale":"2024-09-10T21:27:00.397163+0000","last_undegraded":"2024-09-10T21:27:00.397163+0000","last_fullsized":"2024-09-10T21:27:00.397163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:50.713206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.417194+0000","last_change":"2024-09-10T21:27:00.414254+0000","last_active":"2024-09-10T21:27:00.417194+0000","last_peered":"2024-09-10T21:27:00.417194+0000","last_clean":"2024-09-10T21:27:00.417194+0000","last_became_active":"2024-09-10T21:27:00.413853+0000","last_became_peered":"2024-09-10T21:27:00.413853+0000","last_unstale":"2024-09-10T21:27:00.417194+0000","last_undegraded":"2024-09-10T21:27:00.417194+0000","last_fullsized":"2024-09-10T21:27:00.417194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.343005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.433702+0000","last_change":"2024-09-10T21:26:59.399657+0000","last_active":"2024-09-10T21:27:00.433702+0000","last_peered":"2024-09-10T21:27:00.433702+0000","last_clean":"2024-09-10T21:27:00.433702+0000","last_became_active":"2024-09-10T21:26:59.399350+0000","last_became_peered":"2024-09-10T21:26:59.399350+0000","last_unstale":"2024-09-10T21:27:00.433702+0000","last_undegraded":"2024-09-10T21:27:00.433702+0000","last_fullsized":"2024-09-10T21:27:00.433702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:30.229587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.410916+0000","last_change":"2024-09-10T21:27:00.399775+0000","last_active":"2024-09-10T21:27:00.410916+0000","last_peered":"2024-09-10T21:27:00.410916+0000","last_clean":"2024-09-10T21:27:00.410916+0000","last_became_active":"2024-09-10T21:27:00.399514+0000","last_became_peered":"2024-09-10T21:27:00.399514+0000","last_unstale":"2024-09-10T21:27:00.410916+0000","last_undegraded":"2024-09-10T21:27:00.410916+0000","last_fullsized":"2024-09-10T21:27:00.410916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:41.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.403880+0000","last_change":"2024-09-10T21:26:59.402604+0000","last_active":"2024-09-10T21:27:00.403880+0000","last_peered":"2024-09-10T21:27:00.403880+0000","last_clean":"2024-09-10T21:27:00.403880+0000","last_became_active":"2024-09-10T21:26:59.402338+0000","last_became_peered":"2024-09-10T21:26:59.402338+0000","last_unstale":"2024-09-10T21:27:00.403880+0000","last_undegraded":"2024-09-10T21:27:00.403880+0000","last_fullsized":"2024-09-10T21:27:00.403880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:22.602159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.424924+0000","last_change":"2024-09-10T21:27:00.408300+0000","last_active":"2024-09-10T21:27:00.424924+0000","last_peered":"2024-09-10T21:27:00.424924+0000","last_clean":"2024-09-10T21:27:00.424924+0000","last_became_active":"2024-09-10T21:27:00.408003+0000","last_became_peered":"2024-09-10T21:27:00.408003+0000","last_unstale":"2024-09-10T21:27:00.424924+0000","last_undegraded":"2024-09-10T21:27:00.424924+0000","last_fullsized":"2024-09-10T21:27:00.424924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:01.241843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.401701+0000","last_change":"2024-09-10T21:26:59.400480+0000","last_active":"2024-09-10T21:27:00.401701+0000","last_peered":"2024-09-10T21:27:00.401701+0000","last_clean":"2024-09-10T21:27:00.401701+0000","last_became_active":"2024-09-10T21:26:59.399760+0000","last_became_peered":"2024-09-10T21:26:59.399760+0000","last_unstale":"2024-09-10T21:27:00.401701+0000","last_undegraded":"2024-09-10T21:27:00.401701+0000","last_fullsized":"2024-09-10T21:27:00.401701+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:47.320374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394998+0000","last_change":"2024-09-10T21:27:00.407556+0000","last_active":"2024-09-10T21:27:01.394998+0000","last_peered":"2024-09-10T21:27:01.394998+0000","last_clean":"2024-09-10T21:27:01.394998+0000","last_became_active":"2024-09-10T21:27:00.407265+0000","last_became_peered":"2024-09-10T21:27:00.407265+0000","last_unstale":"2024-09-10T21:27:01.394998+0000","last_undegraded":"2024-09-10T21:27:01.394998+0000","last_fullsized":"2024-09-10T21:27:01.394998+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:36.662564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.435079+0000","last_change":"2024-09-10T21:26:59.399989+0000","last_active":"2024-09-10T21:27:00.435079+0000","last_peered":"2024-09-10T21:27:00.435079+0000","last_clean":"2024-09-10T21:27:00.435079+0000","last_became_active":"2024-09-10T21:26:59.399719+0000","last_became_peered":"2024-09-10T21:26:59.399719+0000","last_unstale":"2024-09-10T21:27:00.435079+0000","last_undegraded":"2024-09-10T21:27:00.435079+0000","last_fullsized":"2024-09-10T21:27:00.435079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:33.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395226+0000","last_change":"2024-09-10T21:27:00.410547+0000","last_active":"2024-09-10T21:27:01.395226+0000","last_peered":"2024-09-10T21:27:01.395226+0000","last_clean":"2024-09-10T21:27:01.395226+0000","last_became_active":"2024-09-10T21:27:00.409850+0000","last_became_peered":"2024-09-10T21:27:00.409850+0000","last_unstale":"2024-09-10T21:27:01.395226+0000","last_undegraded":"2024-09-10T21:27:01.395226+0000","last_fullsized":"2024-09-10T21:27:01.395226+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:10.199959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.397993+0000","last_change":"2024-09-10T21:26:59.398904+0000","last_active":"2024-09-10T21:27:00.397993+0000","last_peered":"2024-09-10T21:27:00.397993+0000","last_clean":"2024-09-10T21:27:00.397993+0000","last_became_active":"2024-09-10T21:26:59.398595+0000","last_became_peered":"2024-09-10T21:26:59.398595+0000","last_unstale":"2024-09-10T21:27:00.397993+0000","last_undegraded":"2024-09-10T21:27:00.397993+0000","last_fullsized":"2024-09-10T21:27:00.397993+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:38.637358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.427374+0000","last_change":"2024-09-10T21:27:00.423788+0000","last_active":"2024-09-10T21:27:00.427374+0000","last_peered":"2024-09-10T21:27:00.427374+0000","last_clean":"2024-09-10T21:27:00.427374+0000","last_became_active":"2024-09-10T21:27:00.421501+0000","last_became_peered":"2024-09-10T21:27:00.421501+0000","last_unstale":"2024-09-10T21:27:00.427374+0000","last_undegraded":"2024-09-10T21:27:00.427374+0000","last_fullsized":"2024-09-10T21:27:00.427374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:09.592805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.401550+0000","last_change":"2024-09-10T21:26:59.402679+0000","last_active":"2024-09-10T21:27:00.401550+0000","last_peered":"2024-09-10T21:27:00.401550+0000","last_clean":"2024-09-10T21:27:00.401550+0000","last_became_active":"2024-09-10T21:26:59.402367+0000","last_became_peered":"2024-09-10T21:26:59.402367+0000","last_unstale":"2024-09-10T21:27:00.401550+0000","last_undegraded":"2024-09-10T21:27:00.401550+0000","last_fullsized":"2024-09-10T21:27:00.401550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:57.521015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425555+0000","last_change":"2024-09-10T21:27:00.411680+0000","last_active":"2024-09-10T21:27:00.425555+0000","last_peered":"2024-09-10T21:27:00.425555+0000","last_clean":"2024-09-10T21:27:00.425555+0000","last_became_active":"2024-09-10T21:27:00.411404+0000","last_became_peered":"2024-09-10T21:27:00.411404+0000","last_unstale":"2024-09-10T21:27:00.425555+0000","last_undegraded":"2024-09-10T21:27:00.425555+0000","last_fullsized":"2024-09-10T21:27:00.425555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:28.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.396307+0000","last_change":"2024-09-10T21:26:59.393197+0000","last_active":"2024-09-10T21:27:00.396307+0000","last_peered":"2024-09-10T21:27:00.396307+0000","last_clean":"2024-09-10T21:27:00.396307+0000","last_became_active":"2024-09-10T21:26:59.392888+0000","last_became_peered":"2024-09-10T21:26:59.392888+0000","last_unstale":"2024-09-10T21:27:00.396307+0000","last_undegraded":"2024-09-10T21:27:00.396307+0000","last_fullsized":"2024-09-10T21:27:00.396307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:39.983295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425487+0000","last_change":"2024-09-10T21:27:00.412032+0000","last_active":"2024-09-10T21:27:00.425487+0000","last_peered":"2024-09-10T21:27:00.425487+0000","last_clean":"2024-09-10T21:27:00.425487+0000","last_became_active":"2024-09-10T21:27:00.411758+0000","last_became_peered":"2024-09-10T21:27:00.411758+0000","last_unstale":"2024-09-10T21:27:00.425487+0000","last_undegraded":"2024-09-10T21:27:00.425487+0000","last_fullsized":"2024-09-10T21:27:00.425487+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:38.284949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.390350+0000","last_change":"2024-09-10T21:26:59.396950+0000","last_active":"2024-09-10T21:27:00.390350+0000","last_peered":"2024-09-10T21:27:00.390350+0000","last_clean":"2024-09-10T21:27:00.390350+0000","last_became_active":"2024-09-10T21:26:59.396659+0000","last_became_peered":"2024-09-10T21:26:59.396659+0000","last_unstale":"2024-09-10T21:27:00.390350+0000","last_undegraded":"2024-09-10T21:27:00.390350+0000","last_fullsized":"2024-09-10T21:27:00.390350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:02.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.417548+0000","last_change":"2024-09-10T21:27:00.411085+0000","last_active":"2024-09-10T21:27:00.417548+0000","last_peered":"2024-09-10T21:27:00.417548+0000","last_clean":"2024-09-10T21:27:00.417548+0000","last_became_active":"2024-09-10T21:27:00.410746+0000","last_became_peered":"2024-09-10T21:27:00.410746+0000","last_unstale":"2024-09-10T21:27:00.417548+0000","last_undegraded":"2024-09-10T21:27:00.417548+0000","last_fullsized":"2024-09-10T21:27:00.417548+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:17.200466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.402893+0000","last_change":"2024-09-10T21:26:59.398547+0000","last_active":"2024-09-10T21:27:00.402893+0000","last_peered":"2024-09-10T21:27:00.402893+0000","last_clean":"2024-09-10T21:27:00.402893+0000","last_became_active":"2024-09-10T21:26:59.398187+0000","last_became_peered":"2024-09-10T21:26:59.398187+0000","last_unstale":"2024-09-10T21:27:00.402893+0000","last_undegraded":"2024-09-10T21:27:00.402893+0000","last_fullsized":"2024-09-10T21:27:00.402893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:11.704238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.430681+0000","last_change":"2024-09-10T21:27:00.425585+0000","last_active":"2024-09-10T21:27:00.430681+0000","last_peered":"2024-09-10T21:27:00.430681+0000","last_clean":"2024-09-10T21:27:00.430681+0000","last_became_active":"2024-09-10T21:27:00.425274+0000","last_became_peered":"2024-09-10T21:27:00.425274+0000","last_unstale":"2024-09-10T21:27:00.430681+0000","last_undegraded":"2024-09-10T21:27:00.430681+0000","last_fullsized":"2024-09-10T21:27:00.430681+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:10.572804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.396074+0000","last_change":"2024-09-10T21:26:59.392982+0000","last_active":"2024-09-10T21:27:00.396074+0000","last_peered":"2024-09-10T21:27:00.396074+0000","last_clean":"2024-09-10T21:27:00.396074+0000","last_became_active":"2024-09-10T21:26:59.392672+0000","last_became_peered":"2024-09-10T21:26:59.392672+0000","last_unstale":"2024-09-10T21:27:00.396074+0000","last_undegraded":"2024-09-10T21:27:00.396074+0000","last_fullsized":"2024-09-10T21:27:00.396074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:54.508797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394940+0000","last_change":"2024-09-10T21:27:00.404292+0000","last_active":"2024-09-10T21:27:01.394940+0000","last_peered":"2024-09-10T21:27:01.394940+0000","last_clean":"2024-09-10T21:27:01.394940+0000","last_became_active":"2024-09-10T21:27:00.403868+0000","last_became_peered":"2024-09-10T21:27:00.403868+0000","last_unstale":"2024-09-10T21:27:01.394940+0000","last_undegraded":"2024-09-10T21:27:01.394940+0000","last_fullsized":"2024-09-10T21:27:01.394940+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:50.281693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.400948+0000","last_change":"2024-09-10T21:26:59.405102+0000","last_active":"2024-09-10T21:27:00.400948+0000","last_peered":"2024-09-10T21:27:00.400948+0000","last_clean":"2024-09-10T21:27:00.400948+0000","last_became_active":"2024-09-10T21:26:59.404841+0000","last_became_peered":"2024-09-10T21:26:59.404841+0000","last_unstale":"2024-09-10T21:27:00.400948+0000","last_undegraded":"2024-09-10T21:27:00.400948+0000","last_fullsized":"2024-09-10T21:27:00.400948+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:13.203314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.423284+0000","last_change":"2024-09-10T21:27:00.419630+0000","last_active":"2024-09-10T21:27:00.423284+0000","last_peered":"2024-09-10T21:27:00.423284+0000","last_clean":"2024-09-10T21:27:00.423284+0000","last_became_active":"2024-09-10T21:27:00.418133+0000","last_became_peered":"2024-09-10T21:27:00.418133+0000","last_unstale":"2024-09-10T21:27:00.423284+0000","last_undegraded":"2024-09-10T21:27:00.423284+0000","last_fullsized":"2024-09-10T21:27:00.423284+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:03.176928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.402253+0000","last_change":"2024-09-10T21:26:59.395944+0000","last_active":"2024-09-10T21:27:00.402253+0000","last_peered":"2024-09-10T21:27:00.402253+0000","last_clean":"2024-09-10T21:27:00.402253+0000","last_became_active":"2024-09-10T21:26:59.395685+0000","last_became_peered":"2024-09-10T21:26:59.395685+0000","last_unstale":"2024-09-10T21:27:00.402253+0000","last_undegraded":"2024-09-10T21:27:00.402253+0000","last_fullsized":"2024-09-10T21:27:00.402253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:23.835253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":194,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.406950+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:27:00.406950+0000","last_peered":"2024-09-10T21:27:00.406950+0000","last_clean":"2024-09-10T21:27:00.406950+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:27:00.406950+0000","last_undegraded":"2024-09-10T21:27:00.406950+0000","last_fullsized":"2024-09-10T21:27:00.406950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":355959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":358223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28404,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343436,"statfs":{"total":96636764160,"available":96607678464,"internally_reserved":0,"allocated":1404928,"data_stored":948051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":356119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":356015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":1388544,"data_stored":946361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":355959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":806912,"data_stored":363233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:27:04.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:27:04.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- 172.21.15.97:0/1025998566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40b1560 msgr2=0x7ff5c40b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/1025998566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40b1560 0x7ff5c40b1970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40735a0 tx=0x7ff5bc01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- 172.21.15.97:0/1025998566 shutdown_connections 2024-09-10T21:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/1025998566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40b1560 0x7ff5c40b1970 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/1025998566 >> v2:172.21.15.194:3300/0 conn(0x7ff5c40ce830 0x7ff5c40cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- 172.21.15.97:0/1025998566 >> 172.21.15.97:0/1025998566 conn(0x7ff5c40b37c0 msgr2=0x7ff5c40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- 172.21.15.97:0/1025998566 shutdown_connections 2024-09-10T21:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- 172.21.15.97:0/1025998566 wait complete. 2024-09-10T21:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 Processor -- start 2024-09-10T21:27:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- start start 2024-09-10T21:27:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5c40b1560 0x7ff5c415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40ce830 0x7ff5c41f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5c40cec10 0x7ff5c409f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40ce1a0 con 0x7ff5c40ce830 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c40cde60 con 0x7ff5c40b1560 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.686+0000 7ff5c8c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c40ce320 con 0x7ff5c40cec10 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40ce830 0x7ff5c41f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40ce830 0x7ff5c41f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52638/0 (socket says 172.21.15.97:52638) 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 -- 172.21.15.97:0/2601038370 learned_addr learned my addr 172.21.15.97:0/2601038370 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5c1e00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5c40b1560 0x7ff5c415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5c2800640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5c40cec10 0x7ff5c409f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 -- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5c40cec10 msgr2=0x7ff5c409f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5c40cec10 0x7ff5c409f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 -- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5c40b1560 msgr2=0x7ff5c415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5c40b1560 0x7ff5c415db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 -- 172.21.15.97:0/2601038370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c40b4e40 con 0x7ff5c40ce830 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5c1e00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5c40b1560 0x7ff5c415db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5c2800640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5c40cec10 0x7ff5c409f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5b9400640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40ce830 0x7ff5c41f4d50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001ec0 tx=0x7ff5ac00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5baa00640 1 -- 172.21.15.97:0/2601038370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac017070 con 0x7ff5c40ce830 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5baa00640 1 -- 172.21.15.97:0/2601038370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5ac0053c0 con 0x7ff5c40ce830 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5baa00640 1 -- 172.21.15.97:0/2601038370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac003860 con 0x7ff5c40ce830 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c40b5120 con 0x7ff5c40ce830 2024-09-10T21:27:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c40b4ac0 con 0x7ff5c40ce830 2024-09-10T21:27:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c40cf800 con 0x7ff5c40ce830 2024-09-10T21:27:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.690+0000 7ff5baa00640 1 -- 172.21.15.97:0/2601038370 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff5ac003a00 con 0x7ff5c40ce830 2024-09-10T21:27:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.694+0000 7ff5baa00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5a00725e0 0x7ff5a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.694+0000 7ff5baa00640 1 -- 172.21.15.97:0/2601038370 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff5ac092a20 con 0x7ff5c40ce830 2024-09-10T21:27:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.694+0000 7ff5c1e00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5a00725e0 0x7ff5a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.694+0000 7ff5c1e00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5a00725e0 0x7ff5a0074aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40af5f0 tx=0x7ff5b4009520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.694+0000 7ff5baa00640 1 -- 172.21.15.97:0/2601038370 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff5ac05e000 con 0x7ff5c40ce830 2024-09-10T21:27:04.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.858+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5c40760a0 con 0x7ff5a00725e0 2024-09-10T21:27:04.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.866+0000 7ff5baa00640 1 -- 172.21.15.97:0/2601038370 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252010 (secure 0 0 0) 0x7ff5c40760a0 con 0x7ff5a00725e0 2024-09-10T21:27:04.871 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:04.873 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:27:04.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.866+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5a00725e0 msgr2=0x7ff5a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.866+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5a00725e0 0x7ff5a0074aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40af5f0 tx=0x7ff5b4009520 comp rx=0 tx=0).stop 2024-09-10T21:27:04.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.866+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40ce830 msgr2=0x7ff5c41f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.866+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40ce830 0x7ff5c41f4d50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001ec0 tx=0x7ff5ac00c890 comp rx=0 tx=0).stop 2024-09-10T21:27:04.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 shutdown_connections 2024-09-10T21:27:04.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5a00725e0 0x7ff5a0074aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5c40cec10 0x7ff5c409f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c40ce830 0x7ff5c41f4d50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 --2- 172.21.15.97:0/2601038370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5c40b1560 0x7ff5c415db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 >> 172.21.15.97:0/2601038370 conn(0x7ff5c40b37c0 msgr2=0x7ff5c40aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:04.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 shutdown_connections 2024-09-10T21:27:04.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:04.870+0000 7ff5c8c00640 1 -- 172.21.15.97:0/2601038370 wait complete. 2024-09-10T21:27:04.891 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-10T21:27:04.111398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223652,"kb_used_data":7412,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751068,"statfs":{"total":773094113280,"available":772865093632,"internally_reserved":0,"allocated":7589888,"data_stored":4039920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.735361"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.422104+0000","last_change":"2024-09-10T21:27:00.417538+0000","last_active":"2024-09-10T21:27:00.422104+0000","last_peered":"2024-09-10T21:27:00.422104+0000","last_clean":"2024-09-10T21:27:00.422104+0000","last_became_active":"2024-09-10T21:27:00.417090+0000","last_became_peered":"2024-09-10T21:27:00.417090+0000","last_unstale":"2024-09-10T21:27:00.422104+0000","last_undegraded":"2024-09-10T21:27:00.422104+0000","last_fullsized":"2024-09-10T21:27:00.422104+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:34.790132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.415910+0000","last_change":"2024-09-10T21:27:00.409647+0000","last_active":"2024-09-10T21:27:00.415910+0000","last_peered":"2024-09-10T21:27:00.415910+0000","last_clean":"2024-09-10T21:27:00.415910+0000","last_became_active":"2024-09-10T21:27:00.409372+0000","last_became_peered":"2024-09-10T21:27:00.409372+0000","last_unstale":"2024-09-10T21:27:00.415910+0000","last_undegraded":"2024-09-10T21:27:00.415910+0000","last_fullsized":"2024-09-10T21:27:00.415910+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:52.367505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425466+0000","last_change":"2024-09-10T21:27:00.410734+0000","last_active":"2024-09-10T21:27:00.425466+0000","last_peered":"2024-09-10T21:27:00.425466+0000","last_clean":"2024-09-10T21:27:00.425466+0000","last_became_active":"2024-09-10T21:27:00.410320+0000","last_became_peered":"2024-09-10T21:27:00.410320+0000","last_unstale":"2024-09-10T21:27:00.425466+0000","last_undegraded":"2024-09-10T21:27:00.425466+0000","last_fullsized":"2024-09-10T21:27:00.425466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:48.170664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426942+0000","last_change":"2024-09-10T21:27:00.419743+0000","last_active":"2024-09-10T21:27:00.426942+0000","last_peered":"2024-09-10T21:27:00.426942+0000","last_clean":"2024-09-10T21:27:00.426942+0000","last_became_active":"2024-09-10T21:27:00.419466+0000","last_became_peered":"2024-09-10T21:27:00.419466+0000","last_unstale":"2024-09-10T21:27:00.426942+0000","last_undegraded":"2024-09-10T21:27:00.426942+0000","last_fullsized":"2024-09-10T21:27:00.426942+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.343005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425949+0000","last_change":"2024-09-10T21:27:00.414142+0000","last_active":"2024-09-10T21:27:00.425949+0000","last_peered":"2024-09-10T21:27:00.425949+0000","last_clean":"2024-09-10T21:27:00.425949+0000","last_became_active":"2024-09-10T21:27:00.413515+0000","last_became_peered":"2024-09-10T21:27:00.413515+0000","last_unstale":"2024-09-10T21:27:00.425949+0000","last_undegraded":"2024-09-10T21:27:00.425949+0000","last_fullsized":"2024-09-10T21:27:00.425949+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:32.574201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.419396+0000","last_change":"2024-09-10T21:27:00.414039+0000","last_active":"2024-09-10T21:27:00.419396+0000","last_peered":"2024-09-10T21:27:00.419396+0000","last_clean":"2024-09-10T21:27:00.419396+0000","last_became_active":"2024-09-10T21:27:00.413749+0000","last_became_peered":"2024-09-10T21:27:00.413749+0000","last_unstale":"2024-09-10T21:27:00.419396+0000","last_undegraded":"2024-09-10T21:27:00.419396+0000","last_fullsized":"2024-09-10T21:27:00.419396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:01.136669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.422213+0000","last_change":"2024-09-10T21:27:00.418330+0000","last_active":"2024-09-10T21:27:00.422213+0000","last_peered":"2024-09-10T21:27:00.422213+0000","last_clean":"2024-09-10T21:27:00.422213+0000","last_became_active":"2024-09-10T21:27:00.417957+0000","last_became_peered":"2024-09-10T21:27:00.417957+0000","last_unstale":"2024-09-10T21:27:00.422213+0000","last_undegraded":"2024-09-10T21:27:00.422213+0000","last_fullsized":"2024-09-10T21:27:00.422213+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:01.241843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.421547+0000","last_change":"2024-09-10T21:27:00.407145+0000","last_active":"2024-09-10T21:27:00.421547+0000","last_peered":"2024-09-10T21:27:00.421547+0000","last_clean":"2024-09-10T21:27:00.421547+0000","last_became_active":"2024-09-10T21:27:00.406896+0000","last_became_peered":"2024-09-10T21:27:00.406896+0000","last_unstale":"2024-09-10T21:27:00.421547+0000","last_undegraded":"2024-09-10T21:27:00.421547+0000","last_fullsized":"2024-09-10T21:27:00.421547+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:04.929735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.427462+0000","last_change":"2024-09-10T21:27:00.423987+0000","last_active":"2024-09-10T21:27:00.427462+0000","last_peered":"2024-09-10T21:27:00.427462+0000","last_clean":"2024-09-10T21:27:00.427462+0000","last_became_active":"2024-09-10T21:27:00.422900+0000","last_became_peered":"2024-09-10T21:27:00.422900+0000","last_unstale":"2024-09-10T21:27:00.427462+0000","last_undegraded":"2024-09-10T21:27:00.427462+0000","last_fullsized":"2024-09-10T21:27:00.427462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:52.074995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.428089+0000","last_change":"2024-09-10T21:27:00.423473+0000","last_active":"2024-09-10T21:27:00.428089+0000","last_peered":"2024-09-10T21:27:00.428089+0000","last_clean":"2024-09-10T21:27:00.428089+0000","last_became_active":"2024-09-10T21:27:00.421864+0000","last_became_peered":"2024-09-10T21:27:00.421864+0000","last_unstale":"2024-09-10T21:27:00.428089+0000","last_undegraded":"2024-09-10T21:27:00.428089+0000","last_fullsized":"2024-09-10T21:27:00.428089+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:45.931525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426479+0000","last_change":"2024-09-10T21:27:00.418444+0000","last_active":"2024-09-10T21:27:00.426479+0000","last_peered":"2024-09-10T21:27:00.426479+0000","last_clean":"2024-09-10T21:27:00.426479+0000","last_became_active":"2024-09-10T21:27:00.415097+0000","last_became_peered":"2024-09-10T21:27:00.415097+0000","last_unstale":"2024-09-10T21:27:00.426479+0000","last_undegraded":"2024-09-10T21:27:00.426479+0000","last_fullsized":"2024-09-10T21:27:00.426479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:47.622969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426212+0000","last_change":"2024-09-10T21:27:00.415596+0000","last_active":"2024-09-10T21:27:00.426212+0000","last_peered":"2024-09-10T21:27:00.426212+0000","last_clean":"2024-09-10T21:27:00.426212+0000","last_became_active":"2024-09-10T21:27:00.415356+0000","last_became_peered":"2024-09-10T21:27:00.415356+0000","last_unstale":"2024-09-10T21:27:00.426212+0000","last_undegraded":"2024-09-10T21:27:00.426212+0000","last_fullsized":"2024-09-10T21:27:00.426212+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:47.798687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.411135+0000","last_change":"2024-09-10T21:27:00.409011+0000","last_active":"2024-09-10T21:27:00.411135+0000","last_peered":"2024-09-10T21:27:00.411135+0000","last_clean":"2024-09-10T21:27:00.411135+0000","last_became_active":"2024-09-10T21:27:00.408745+0000","last_became_peered":"2024-09-10T21:27:00.408745+0000","last_unstale":"2024-09-10T21:27:00.411135+0000","last_undegraded":"2024-09-10T21:27:00.411135+0000","last_fullsized":"2024-09-10T21:27:00.411135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:29.158533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.411011+0000","last_change":"2024-09-10T21:27:00.400889+0000","last_active":"2024-09-10T21:27:00.411011+0000","last_peered":"2024-09-10T21:27:00.411011+0000","last_clean":"2024-09-10T21:27:00.411011+0000","last_became_active":"2024-09-10T21:27:00.400624+0000","last_became_peered":"2024-09-10T21:27:00.400624+0000","last_unstale":"2024-09-10T21:27:00.411011+0000","last_undegraded":"2024-09-10T21:27:00.411011+0000","last_fullsized":"2024-09-10T21:27:00.411011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:55.167189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426288+0000","last_change":"2024-09-10T21:27:00.421139+0000","last_active":"2024-09-10T21:27:00.426288+0000","last_peered":"2024-09-10T21:27:00.426288+0000","last_clean":"2024-09-10T21:27:00.426288+0000","last_became_active":"2024-09-10T21:27:00.420387+0000","last_became_peered":"2024-09-10T21:27:00.420387+0000","last_unstale":"2024-09-10T21:27:00.426288+0000","last_undegraded":"2024-09-10T21:27:00.426288+0000","last_fullsized":"2024-09-10T21:27:00.426288+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:55.167189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.414174+0000","last_change":"2024-09-10T21:27:00.411650+0000","last_active":"2024-09-10T21:27:00.414174+0000","last_peered":"2024-09-10T21:27:00.414174+0000","last_clean":"2024-09-10T21:27:00.414174+0000","last_became_active":"2024-09-10T21:27:00.411333+0000","last_became_peered":"2024-09-10T21:27:00.411333+0000","last_unstale":"2024-09-10T21:27:00.414174+0000","last_undegraded":"2024-09-10T21:27:00.414174+0000","last_fullsized":"2024-09-10T21:27:00.414174+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:29.687670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395200+0000","last_change":"2024-09-10T21:27:00.405579+0000","last_active":"2024-09-10T21:27:01.395200+0000","last_peered":"2024-09-10T21:27:01.395200+0000","last_clean":"2024-09-10T21:27:01.395200+0000","last_became_active":"2024-09-10T21:27:00.404926+0000","last_became_peered":"2024-09-10T21:27:00.404926+0000","last_unstale":"2024-09-10T21:27:01.395200+0000","last_undegraded":"2024-09-10T21:27:01.395200+0000","last_fullsized":"2024-09-10T21:27:01.395200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:44.952921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425452+0000","last_change":"2024-09-10T21:27:00.411387+0000","last_active":"2024-09-10T21:27:00.425452+0000","last_peered":"2024-09-10T21:27:00.425452+0000","last_clean":"2024-09-10T21:27:00.425452+0000","last_became_active":"2024-09-10T21:27:00.411141+0000","last_became_peered":"2024-09-10T21:27:00.411141+0000","last_unstale":"2024-09-10T21:27:00.425452+0000","last_undegraded":"2024-09-10T21:27:00.425452+0000","last_fullsized":"2024-09-10T21:27:00.425452+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:04.885814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426122+0000","last_change":"2024-09-10T21:27:00.413655+0000","last_active":"2024-09-10T21:27:00.426122+0000","last_peered":"2024-09-10T21:27:00.426122+0000","last_clean":"2024-09-10T21:27:00.426122+0000","last_became_active":"2024-09-10T21:27:00.413382+0000","last_became_peered":"2024-09-10T21:27:00.413382+0000","last_unstale":"2024-09-10T21:27:00.426122+0000","last_undegraded":"2024-09-10T21:27:00.426122+0000","last_fullsized":"2024-09-10T21:27:00.426122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:22.582701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394722+0000","last_change":"2024-09-10T21:27:00.406759+0000","last_active":"2024-09-10T21:27:01.394722+0000","last_peered":"2024-09-10T21:27:01.394722+0000","last_clean":"2024-09-10T21:27:01.394722+0000","last_became_active":"2024-09-10T21:27:00.406470+0000","last_became_peered":"2024-09-10T21:27:00.406470+0000","last_unstale":"2024-09-10T21:27:01.394722+0000","last_undegraded":"2024-09-10T21:27:01.394722+0000","last_fullsized":"2024-09-10T21:27:01.394722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:25.025007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.408634+0000","last_change":"2024-09-10T21:27:00.401505+0000","last_active":"2024-09-10T21:27:00.408634+0000","last_peered":"2024-09-10T21:27:00.408634+0000","last_clean":"2024-09-10T21:27:00.408634+0000","last_became_active":"2024-09-10T21:27:00.401245+0000","last_became_peered":"2024-09-10T21:27:00.401245+0000","last_unstale":"2024-09-10T21:27:00.408634+0000","last_undegraded":"2024-09-10T21:27:00.408634+0000","last_fullsized":"2024-09-10T21:27:00.408634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:39.087323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.417166+0000","last_change":"2024-09-10T21:27:00.414314+0000","last_active":"2024-09-10T21:27:00.417166+0000","last_peered":"2024-09-10T21:27:00.417166+0000","last_clean":"2024-09-10T21:27:00.417166+0000","last_became_active":"2024-09-10T21:27:00.413524+0000","last_became_peered":"2024-09-10T21:27:00.413524+0000","last_unstale":"2024-09-10T21:27:00.417166+0000","last_undegraded":"2024-09-10T21:27:00.417166+0000","last_fullsized":"2024-09-10T21:27:00.417166+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:45.231553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.419140+0000","last_change":"2024-09-10T21:27:00.415626+0000","last_active":"2024-09-10T21:27:00.419140+0000","last_peered":"2024-09-10T21:27:00.419140+0000","last_clean":"2024-09-10T21:27:00.419140+0000","last_became_active":"2024-09-10T21:27:00.415370+0000","last_became_peered":"2024-09-10T21:27:00.415370+0000","last_unstale":"2024-09-10T21:27:00.419140+0000","last_undegraded":"2024-09-10T21:27:00.419140+0000","last_fullsized":"2024-09-10T21:27:00.419140+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:07.726401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.407048+0000","last_change":"2024-09-10T21:27:00.402787+0000","last_active":"2024-09-10T21:27:00.407048+0000","last_peered":"2024-09-10T21:27:00.407048+0000","last_clean":"2024-09-10T21:27:00.407048+0000","last_became_active":"2024-09-10T21:27:00.402565+0000","last_became_peered":"2024-09-10T21:27:00.402565+0000","last_unstale":"2024-09-10T21:27:00.407048+0000","last_undegraded":"2024-09-10T21:27:00.407048+0000","last_fullsized":"2024-09-10T21:27:00.407048+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:34.143667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.414451+0000","last_change":"2024-09-10T21:27:00.401602+0000","last_active":"2024-09-10T21:27:00.414451+0000","last_peered":"2024-09-10T21:27:00.414451+0000","last_clean":"2024-09-10T21:27:00.414451+0000","last_became_active":"2024-09-10T21:27:00.401316+0000","last_became_peered":"2024-09-10T21:27:00.401316+0000","last_unstale":"2024-09-10T21:27:00.414451+0000","last_undegraded":"2024-09-10T21:27:00.414451+0000","last_fullsized":"2024-09-10T21:27:00.414451+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:45.931525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.401567+0000","last_change":"2024-09-10T21:27:00.399850+0000","last_active":"2024-09-10T21:27:00.401567+0000","last_peered":"2024-09-10T21:27:00.401567+0000","last_clean":"2024-09-10T21:27:00.401567+0000","last_became_active":"2024-09-10T21:27:00.399602+0000","last_became_peered":"2024-09-10T21:27:00.399602+0000","last_unstale":"2024-09-10T21:27:00.401567+0000","last_undegraded":"2024-09-10T21:27:00.401567+0000","last_fullsized":"2024-09-10T21:27:00.401567+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:46.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.421641+0000","last_change":"2024-09-10T21:27:00.412350+0000","last_active":"2024-09-10T21:27:00.421641+0000","last_peered":"2024-09-10T21:27:00.421641+0000","last_clean":"2024-09-10T21:27:00.421641+0000","last_became_active":"2024-09-10T21:27:00.411825+0000","last_became_peered":"2024-09-10T21:27:00.411825+0000","last_unstale":"2024-09-10T21:27:00.421641+0000","last_undegraded":"2024-09-10T21:27:00.421641+0000","last_fullsized":"2024-09-10T21:27:00.421641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:02.603528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.432778+0000","last_change":"2024-09-10T21:27:00.427324+0000","last_active":"2024-09-10T21:27:00.432778+0000","last_peered":"2024-09-10T21:27:00.432778+0000","last_clean":"2024-09-10T21:27:00.432778+0000","last_became_active":"2024-09-10T21:27:00.426596+0000","last_became_peered":"2024-09-10T21:27:00.426596+0000","last_unstale":"2024-09-10T21:27:00.432778+0000","last_undegraded":"2024-09-10T21:27:00.432778+0000","last_fullsized":"2024-09-10T21:27:00.432778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:04.885814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426501+0000","last_change":"2024-09-10T21:27:00.416352+0000","last_active":"2024-09-10T21:27:00.426501+0000","last_peered":"2024-09-10T21:27:00.426501+0000","last_clean":"2024-09-10T21:27:00.426501+0000","last_became_active":"2024-09-10T21:27:00.414024+0000","last_became_peered":"2024-09-10T21:27:00.414024+0000","last_unstale":"2024-09-10T21:27:00.426501+0000","last_undegraded":"2024-09-10T21:27:00.426501+0000","last_fullsized":"2024-09-10T21:27:00.426501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:04.960471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426972+0000","last_change":"2024-09-10T21:27:00.422316+0000","last_active":"2024-09-10T21:27:00.426972+0000","last_peered":"2024-09-10T21:27:00.426972+0000","last_clean":"2024-09-10T21:27:00.426972+0000","last_became_active":"2024-09-10T21:27:00.422027+0000","last_became_peered":"2024-09-10T21:27:00.422027+0000","last_unstale":"2024-09-10T21:27:00.426972+0000","last_undegraded":"2024-09-10T21:27:00.426972+0000","last_fullsized":"2024-09-10T21:27:00.426972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:39.690168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.420143+0000","last_change":"2024-09-10T21:27:00.405764+0000","last_active":"2024-09-10T21:27:00.420143+0000","last_peered":"2024-09-10T21:27:00.420143+0000","last_clean":"2024-09-10T21:27:00.420143+0000","last_became_active":"2024-09-10T21:27:00.405524+0000","last_became_peered":"2024-09-10T21:27:00.405524+0000","last_unstale":"2024-09-10T21:27:00.420143+0000","last_undegraded":"2024-09-10T21:27:00.420143+0000","last_fullsized":"2024-09-10T21:27:00.420143+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.205100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.415398+0000","last_change":"2024-09-10T21:27:00.410003+0000","last_active":"2024-09-10T21:27:00.415398+0000","last_peered":"2024-09-10T21:27:00.415398+0000","last_clean":"2024-09-10T21:27:00.415398+0000","last_became_active":"2024-09-10T21:27:00.409705+0000","last_became_peered":"2024-09-10T21:27:00.409705+0000","last_unstale":"2024-09-10T21:27:00.415398+0000","last_undegraded":"2024-09-10T21:27:00.415398+0000","last_fullsized":"2024-09-10T21:27:00.415398+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:49.974884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426105+0000","last_change":"2024-09-10T21:27:00.413588+0000","last_active":"2024-09-10T21:27:00.426105+0000","last_peered":"2024-09-10T21:27:00.426105+0000","last_clean":"2024-09-10T21:27:00.426105+0000","last_became_active":"2024-09-10T21:27:00.413325+0000","last_became_peered":"2024-09-10T21:27:00.413325+0000","last_unstale":"2024-09-10T21:27:00.426105+0000","last_undegraded":"2024-09-10T21:27:00.426105+0000","last_fullsized":"2024-09-10T21:27:00.426105+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:11.964526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.432889+0000","last_change":"2024-09-10T21:26:59.399802+0000","last_active":"2024-09-10T21:27:00.432889+0000","last_peered":"2024-09-10T21:27:00.432889+0000","last_clean":"2024-09-10T21:27:00.432889+0000","last_became_active":"2024-09-10T21:26:59.399546+0000","last_became_peered":"2024-09-10T21:26:59.399546+0000","last_unstale":"2024-09-10T21:27:00.432889+0000","last_undegraded":"2024-09-10T21:27:00.432889+0000","last_fullsized":"2024-09-10T21:27:00.432889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:01.596056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425444+0000","last_change":"2024-09-10T21:27:00.410357+0000","last_active":"2024-09-10T21:27:00.425444+0000","last_peered":"2024-09-10T21:27:00.425444+0000","last_clean":"2024-09-10T21:27:00.425444+0000","last_became_active":"2024-09-10T21:27:00.410004+0000","last_became_peered":"2024-09-10T21:27:00.410004+0000","last_unstale":"2024-09-10T21:27:00.425444+0000","last_undegraded":"2024-09-10T21:27:00.425444+0000","last_fullsized":"2024-09-10T21:27:00.425444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:32.954234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.434121+0000","last_change":"2024-09-10T21:26:59.401491+0000","last_active":"2024-09-10T21:27:00.434121+0000","last_peered":"2024-09-10T21:27:00.434121+0000","last_clean":"2024-09-10T21:27:00.434121+0000","last_became_active":"2024-09-10T21:26:59.400643+0000","last_became_peered":"2024-09-10T21:26:59.400643+0000","last_unstale":"2024-09-10T21:27:00.434121+0000","last_undegraded":"2024-09-10T21:27:00.434121+0000","last_fullsized":"2024-09-10T21:27:00.434121+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:38.637358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.408846+0000","last_change":"2024-09-10T21:27:00.405856+0000","last_active":"2024-09-10T21:27:00.408846+0000","last_peered":"2024-09-10T21:27:00.408846+0000","last_clean":"2024-09-10T21:27:00.408846+0000","last_became_active":"2024-09-10T21:27:00.405588+0000","last_became_peered":"2024-09-10T21:27:00.405588+0000","last_unstale":"2024-09-10T21:27:00.408846+0000","last_undegraded":"2024-09-10T21:27:00.408846+0000","last_fullsized":"2024-09-10T21:27:00.408846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:54.237250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.431491+0000","last_change":"2024-09-10T21:26:59.398906+0000","last_active":"2024-09-10T21:27:00.431491+0000","last_peered":"2024-09-10T21:27:00.431491+0000","last_clean":"2024-09-10T21:27:00.431491+0000","last_became_active":"2024-09-10T21:26:59.398638+0000","last_became_peered":"2024-09-10T21:26:59.398638+0000","last_unstale":"2024-09-10T21:27:00.431491+0000","last_undegraded":"2024-09-10T21:27:00.431491+0000","last_fullsized":"2024-09-10T21:27:00.431491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:16.192993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425949+0000","last_change":"2024-09-10T21:27:00.410657+0000","last_active":"2024-09-10T21:27:00.425949+0000","last_peered":"2024-09-10T21:27:00.425949+0000","last_clean":"2024-09-10T21:27:00.425949+0000","last_became_active":"2024-09-10T21:27:00.410410+0000","last_became_peered":"2024-09-10T21:27:00.410410+0000","last_unstale":"2024-09-10T21:27:00.425949+0000","last_undegraded":"2024-09-10T21:27:00.425949+0000","last_fullsized":"2024-09-10T21:27:00.425949+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:35.443620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.433471+0000","last_change":"2024-09-10T21:26:59.401767+0000","last_active":"2024-09-10T21:27:00.433471+0000","last_peered":"2024-09-10T21:27:00.433471+0000","last_clean":"2024-09-10T21:27:00.433471+0000","last_became_active":"2024-09-10T21:26:59.400887+0000","last_became_peered":"2024-09-10T21:26:59.400887+0000","last_unstale":"2024-09-10T21:27:00.433471+0000","last_undegraded":"2024-09-10T21:27:00.433471+0000","last_fullsized":"2024-09-10T21:27:00.433471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:04.180850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394000+0000","last_change":"2024-09-10T21:27:00.398718+0000","last_active":"2024-09-10T21:27:01.394000+0000","last_peered":"2024-09-10T21:27:01.394000+0000","last_clean":"2024-09-10T21:27:01.394000+0000","last_became_active":"2024-09-10T21:27:00.398401+0000","last_became_peered":"2024-09-10T21:27:00.398401+0000","last_unstale":"2024-09-10T21:27:01.394000+0000","last_undegraded":"2024-09-10T21:27:01.394000+0000","last_fullsized":"2024-09-10T21:27:01.394000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:06.312783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395053+0000","last_change":"2024-09-10T21:26:59.396906+0000","last_active":"2024-09-10T21:27:01.395053+0000","last_peered":"2024-09-10T21:27:01.395053+0000","last_clean":"2024-09-10T21:27:01.395053+0000","last_became_active":"2024-09-10T21:26:59.394994+0000","last_became_peered":"2024-09-10T21:26:59.394994+0000","last_unstale":"2024-09-10T21:27:01.395053+0000","last_undegraded":"2024-09-10T21:27:01.395053+0000","last_fullsized":"2024-09-10T21:27:01.395053+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:44.412678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.424782+0000","last_change":"2024-09-10T21:27:00.420108+0000","last_active":"2024-09-10T21:27:00.424782+0000","last_peered":"2024-09-10T21:27:00.424782+0000","last_clean":"2024-09-10T21:27:00.424782+0000","last_became_active":"2024-09-10T21:27:00.419819+0000","last_became_peered":"2024-09-10T21:27:00.419819+0000","last_unstale":"2024-09-10T21:27:00.424782+0000","last_undegraded":"2024-09-10T21:27:00.424782+0000","last_fullsized":"2024-09-10T21:27:00.424782+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:35.443620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.433376+0000","last_change":"2024-09-10T21:26:59.396168+0000","last_active":"2024-09-10T21:27:00.433376+0000","last_peered":"2024-09-10T21:27:00.433376+0000","last_clean":"2024-09-10T21:27:00.433376+0000","last_became_active":"2024-09-10T21:26:59.395311+0000","last_became_peered":"2024-09-10T21:26:59.395311+0000","last_unstale":"2024-09-10T21:27:00.433376+0000","last_undegraded":"2024-09-10T21:27:00.433376+0000","last_fullsized":"2024-09-10T21:27:00.433376+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:51.881719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.435153+0000","last_change":"2024-09-10T21:27:00.427703+0000","last_active":"2024-09-10T21:27:00.435153+0000","last_peered":"2024-09-10T21:27:00.435153+0000","last_clean":"2024-09-10T21:27:00.435153+0000","last_became_active":"2024-09-10T21:27:00.427393+0000","last_became_peered":"2024-09-10T21:27:00.427393+0000","last_unstale":"2024-09-10T21:27:00.435153+0000","last_undegraded":"2024-09-10T21:27:00.435153+0000","last_fullsized":"2024-09-10T21:27:00.435153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:41.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.402238+0000","last_change":"2024-09-10T21:26:59.402715+0000","last_active":"2024-09-10T21:27:00.402238+0000","last_peered":"2024-09-10T21:27:00.402238+0000","last_clean":"2024-09-10T21:27:00.402238+0000","last_became_active":"2024-09-10T21:26:59.402452+0000","last_became_peered":"2024-09-10T21:26:59.402452+0000","last_unstale":"2024-09-10T21:27:00.402238+0000","last_undegraded":"2024-09-10T21:27:00.402238+0000","last_fullsized":"2024-09-10T21:27:00.402238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:28.151060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394750+0000","last_change":"2024-09-10T21:27:00.405605+0000","last_active":"2024-09-10T21:27:01.394750+0000","last_peered":"2024-09-10T21:27:01.394750+0000","last_clean":"2024-09-10T21:27:01.394750+0000","last_became_active":"2024-09-10T21:27:00.404743+0000","last_became_peered":"2024-09-10T21:27:00.404743+0000","last_unstale":"2024-09-10T21:27:01.394750+0000","last_undegraded":"2024-09-10T21:27:01.394750+0000","last_fullsized":"2024-09-10T21:27:01.394750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:50.241177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.397390+0000","last_change":"2024-09-10T21:26:59.402712+0000","last_active":"2024-09-10T21:27:00.397390+0000","last_peered":"2024-09-10T21:27:00.397390+0000","last_clean":"2024-09-10T21:27:00.397390+0000","last_became_active":"2024-09-10T21:26:59.402449+0000","last_became_peered":"2024-09-10T21:26:59.402449+0000","last_unstale":"2024-09-10T21:27:00.397390+0000","last_undegraded":"2024-09-10T21:27:00.397390+0000","last_fullsized":"2024-09-10T21:27:00.397390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:11.704238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.416597+0000","last_change":"2024-09-10T21:27:00.413055+0000","last_active":"2024-09-10T21:27:00.416597+0000","last_peered":"2024-09-10T21:27:00.416597+0000","last_clean":"2024-09-10T21:27:00.416597+0000","last_became_active":"2024-09-10T21:27:00.412722+0000","last_became_peered":"2024-09-10T21:27:00.412722+0000","last_unstale":"2024-09-10T21:27:00.416597+0000","last_undegraded":"2024-09-10T21:27:00.416597+0000","last_fullsized":"2024-09-10T21:27:00.416597+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:27.616077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.398703+0000","last_change":"2024-09-10T21:26:59.397537+0000","last_active":"2024-09-10T21:27:00.398703+0000","last_peered":"2024-09-10T21:27:00.398703+0000","last_clean":"2024-09-10T21:27:00.398703+0000","last_became_active":"2024-09-10T21:26:59.396169+0000","last_became_peered":"2024-09-10T21:26:59.396169+0000","last_unstale":"2024-09-10T21:27:00.398703+0000","last_undegraded":"2024-09-10T21:27:00.398703+0000","last_fullsized":"2024-09-10T21:27:00.398703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:48.967412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395077+0000","last_change":"2024-09-10T21:27:00.408243+0000","last_active":"2024-09-10T21:27:01.395077+0000","last_peered":"2024-09-10T21:27:01.395077+0000","last_clean":"2024-09-10T21:27:01.395077+0000","last_became_active":"2024-09-10T21:27:00.407959+0000","last_became_peered":"2024-09-10T21:27:00.407959+0000","last_unstale":"2024-09-10T21:27:01.395077+0000","last_undegraded":"2024-09-10T21:27:01.395077+0000","last_fullsized":"2024-09-10T21:27:01.395077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:32.719778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.390863+0000","last_change":"2024-09-10T21:26:59.387728+0000","last_active":"2024-09-10T21:27:00.390863+0000","last_peered":"2024-09-10T21:27:00.390863+0000","last_clean":"2024-09-10T21:27:00.390863+0000","last_became_active":"2024-09-10T21:26:59.387436+0000","last_became_peered":"2024-09-10T21:26:59.387436+0000","last_unstale":"2024-09-10T21:27:00.390863+0000","last_undegraded":"2024-09-10T21:27:00.390863+0000","last_fullsized":"2024-09-10T21:27:00.390863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:02.312756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.413565+0000","last_change":"2024-09-10T21:27:00.410701+0000","last_active":"2024-09-10T21:27:00.413565+0000","last_peered":"2024-09-10T21:27:00.413565+0000","last_clean":"2024-09-10T21:27:00.413565+0000","last_became_active":"2024-09-10T21:27:00.410447+0000","last_became_peered":"2024-09-10T21:27:00.410447+0000","last_unstale":"2024-09-10T21:27:00.413565+0000","last_undegraded":"2024-09-10T21:27:00.413565+0000","last_fullsized":"2024-09-10T21:27:00.413565+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:39.644830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.434886+0000","last_change":"2024-09-10T21:26:59.400721+0000","last_active":"2024-09-10T21:27:00.434886+0000","last_peered":"2024-09-10T21:27:00.434886+0000","last_clean":"2024-09-10T21:27:00.434886+0000","last_became_active":"2024-09-10T21:26:59.400408+0000","last_became_peered":"2024-09-10T21:26:59.400408+0000","last_unstale":"2024-09-10T21:27:00.434886+0000","last_undegraded":"2024-09-10T21:27:00.434886+0000","last_fullsized":"2024-09-10T21:27:00.434886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:19.491413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.421133+0000","last_change":"2024-09-10T21:27:00.406332+0000","last_active":"2024-09-10T21:27:00.421133+0000","last_peered":"2024-09-10T21:27:00.421133+0000","last_clean":"2024-09-10T21:27:00.421133+0000","last_became_active":"2024-09-10T21:27:00.406103+0000","last_became_peered":"2024-09-10T21:27:00.406103+0000","last_unstale":"2024-09-10T21:27:00.421133+0000","last_undegraded":"2024-09-10T21:27:00.421133+0000","last_fullsized":"2024-09-10T21:27:00.421133+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.939382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.438613+0000","last_change":"2024-09-10T21:26:59.399005+0000","last_active":"2024-09-10T21:27:00.438613+0000","last_peered":"2024-09-10T21:27:00.438613+0000","last_clean":"2024-09-10T21:27:00.438613+0000","last_became_active":"2024-09-10T21:26:59.398736+0000","last_became_peered":"2024-09-10T21:26:59.398736+0000","last_unstale":"2024-09-10T21:27:00.438613+0000","last_undegraded":"2024-09-10T21:27:00.438613+0000","last_fullsized":"2024-09-10T21:27:00.438613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:43.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.423491+0000","last_change":"2024-09-10T21:27:00.412754+0000","last_active":"2024-09-10T21:27:00.423491+0000","last_peered":"2024-09-10T21:27:00.423491+0000","last_clean":"2024-09-10T21:27:00.423491+0000","last_became_active":"2024-09-10T21:27:00.412499+0000","last_became_peered":"2024-09-10T21:27:00.412499+0000","last_unstale":"2024-09-10T21:27:00.423491+0000","last_undegraded":"2024-09-10T21:27:00.423491+0000","last_fullsized":"2024-09-10T21:27:00.423491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:23.112085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.437168+0000","last_change":"2024-09-10T21:26:59.403472+0000","last_active":"2024-09-10T21:27:00.437168+0000","last_peered":"2024-09-10T21:27:00.437168+0000","last_clean":"2024-09-10T21:27:00.437168+0000","last_became_active":"2024-09-10T21:26:59.403227+0000","last_became_peered":"2024-09-10T21:26:59.403227+0000","last_unstale":"2024-09-10T21:27:00.437168+0000","last_undegraded":"2024-09-10T21:27:00.437168+0000","last_fullsized":"2024-09-10T21:27:00.437168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:01.596056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426254+0000","last_change":"2024-09-10T21:27:00.420841+0000","last_active":"2024-09-10T21:27:00.426254+0000","last_peered":"2024-09-10T21:27:00.426254+0000","last_clean":"2024-09-10T21:27:00.426254+0000","last_became_active":"2024-09-10T21:27:00.418373+0000","last_became_peered":"2024-09-10T21:27:00.418373+0000","last_unstale":"2024-09-10T21:27:00.426254+0000","last_undegraded":"2024-09-10T21:27:00.426254+0000","last_fullsized":"2024-09-10T21:27:00.426254+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:03.320228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.403641+0000","last_change":"2024-09-10T21:26:59.406666+0000","last_active":"2024-09-10T21:27:00.403641+0000","last_peered":"2024-09-10T21:27:00.403641+0000","last_clean":"2024-09-10T21:27:00.403641+0000","last_became_active":"2024-09-10T21:26:59.406398+0000","last_became_peered":"2024-09-10T21:26:59.406398+0000","last_unstale":"2024-09-10T21:27:00.403641+0000","last_undegraded":"2024-09-10T21:27:00.403641+0000","last_fullsized":"2024-09-10T21:27:00.403641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:38.205971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.426420+0000","last_change":"2024-09-10T21:27:00.421487+0000","last_active":"2024-09-10T21:27:00.426420+0000","last_peered":"2024-09-10T21:27:00.426420+0000","last_clean":"2024-09-10T21:27:00.426420+0000","last_became_active":"2024-09-10T21:27:00.421239+0000","last_became_peered":"2024-09-10T21:27:00.421239+0000","last_unstale":"2024-09-10T21:27:00.426420+0000","last_undegraded":"2024-09-10T21:27:00.426420+0000","last_fullsized":"2024-09-10T21:27:00.426420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:39.087323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.396002+0000","last_change":"2024-09-10T21:26:59.401272+0000","last_active":"2024-09-10T21:27:00.396002+0000","last_peered":"2024-09-10T21:27:00.396002+0000","last_clean":"2024-09-10T21:27:00.396002+0000","last_became_active":"2024-09-10T21:26:59.400270+0000","last_became_peered":"2024-09-10T21:26:59.400270+0000","last_unstale":"2024-09-10T21:27:00.396002+0000","last_undegraded":"2024-09-10T21:27:00.396002+0000","last_fullsized":"2024-09-10T21:27:00.396002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:34.037894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.422134+0000","last_change":"2024-09-10T21:27:00.417956+0000","last_active":"2024-09-10T21:27:00.422134+0000","last_peered":"2024-09-10T21:27:00.422134+0000","last_clean":"2024-09-10T21:27:00.422134+0000","last_became_active":"2024-09-10T21:27:00.417604+0000","last_became_peered":"2024-09-10T21:27:00.417604+0000","last_unstale":"2024-09-10T21:27:00.422134+0000","last_undegraded":"2024-09-10T21:27:00.422134+0000","last_fullsized":"2024-09-10T21:27:00.422134+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:44.524688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":21,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.438089+0000","last_change":"2024-09-10T21:26:59.405239+0000","last_active":"2024-09-10T21:27:00.438089+0000","last_peered":"2024-09-10T21:27:00.438089+0000","last_clean":"2024-09-10T21:27:00.438089+0000","last_became_active":"2024-09-10T21:26:59.404989+0000","last_became_peered":"2024-09-10T21:26:59.404989+0000","last_unstale":"2024-09-10T21:27:00.438089+0000","last_undegraded":"2024-09-10T21:27:00.438089+0000","last_fullsized":"2024-09-10T21:27:00.438089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:23.639410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425155+0000","last_change":"2024-09-10T21:27:00.405340+0000","last_active":"2024-09-10T21:27:00.425155+0000","last_peered":"2024-09-10T21:27:00.425155+0000","last_clean":"2024-09-10T21:27:00.425155+0000","last_became_active":"2024-09-10T21:27:00.405080+0000","last_became_peered":"2024-09-10T21:27:00.405080+0000","last_unstale":"2024-09-10T21:27:00.425155+0000","last_undegraded":"2024-09-10T21:27:00.425155+0000","last_fullsized":"2024-09-10T21:27:00.425155+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:44.524688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.436957+0000","last_change":"2024-09-10T21:26:59.405550+0000","last_active":"2024-09-10T21:27:00.436957+0000","last_peered":"2024-09-10T21:27:00.436957+0000","last_clean":"2024-09-10T21:27:00.436957+0000","last_became_active":"2024-09-10T21:26:59.404895+0000","last_became_peered":"2024-09-10T21:26:59.404895+0000","last_unstale":"2024-09-10T21:27:00.436957+0000","last_undegraded":"2024-09-10T21:27:00.436957+0000","last_fullsized":"2024-09-10T21:27:00.436957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:26.608605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.411078+0000","last_change":"2024-09-10T21:27:00.409430+0000","last_active":"2024-09-10T21:27:00.411078+0000","last_peered":"2024-09-10T21:27:00.411078+0000","last_clean":"2024-09-10T21:27:00.411078+0000","last_became_active":"2024-09-10T21:27:00.409256+0000","last_became_peered":"2024-09-10T21:27:00.409256+0000","last_unstale":"2024-09-10T21:27:00.411078+0000","last_undegraded":"2024-09-10T21:27:00.411078+0000","last_fullsized":"2024-09-10T21:27:00.411078+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:12.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.436162+0000","last_change":"2024-09-10T21:26:59.399031+0000","last_active":"2024-09-10T21:27:00.436162+0000","last_peered":"2024-09-10T21:27:00.436162+0000","last_clean":"2024-09-10T21:27:00.436162+0000","last_became_active":"2024-09-10T21:26:59.398723+0000","last_became_peered":"2024-09-10T21:26:59.398723+0000","last_unstale":"2024-09-10T21:27:00.436162+0000","last_undegraded":"2024-09-10T21:27:00.436162+0000","last_fullsized":"2024-09-10T21:27:00.436162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:16.192993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.423067+0000","last_change":"2024-09-10T21:27:00.417858+0000","last_active":"2024-09-10T21:27:00.423067+0000","last_peered":"2024-09-10T21:27:00.423067+0000","last_clean":"2024-09-10T21:27:00.423067+0000","last_became_active":"2024-09-10T21:27:00.417600+0000","last_became_peered":"2024-09-10T21:27:00.417600+0000","last_unstale":"2024-09-10T21:27:00.423067+0000","last_undegraded":"2024-09-10T21:27:00.423067+0000","last_fullsized":"2024-09-10T21:27:00.423067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:49.974884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.434815+0000","last_change":"2024-09-10T21:26:59.400952+0000","last_active":"2024-09-10T21:27:00.434815+0000","last_peered":"2024-09-10T21:27:00.434815+0000","last_clean":"2024-09-10T21:27:00.434815+0000","last_became_active":"2024-09-10T21:26:59.400669+0000","last_became_peered":"2024-09-10T21:26:59.400669+0000","last_unstale":"2024-09-10T21:27:00.434815+0000","last_undegraded":"2024-09-10T21:27:00.434815+0000","last_fullsized":"2024-09-10T21:27:00.434815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:34.244246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425576+0000","last_change":"2024-09-10T21:27:00.412517+0000","last_active":"2024-09-10T21:27:00.425576+0000","last_peered":"2024-09-10T21:27:00.425576+0000","last_clean":"2024-09-10T21:27:00.425576+0000","last_became_active":"2024-09-10T21:27:00.411419+0000","last_became_peered":"2024-09-10T21:27:00.411419+0000","last_unstale":"2024-09-10T21:27:00.425576+0000","last_undegraded":"2024-09-10T21:27:00.425576+0000","last_fullsized":"2024-09-10T21:27:00.425576+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:59.090638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.397163+0000","last_change":"2024-09-10T21:26:59.400293+0000","last_active":"2024-09-10T21:27:00.397163+0000","last_peered":"2024-09-10T21:27:00.397163+0000","last_clean":"2024-09-10T21:27:00.397163+0000","last_became_active":"2024-09-10T21:26:59.400002+0000","last_became_peered":"2024-09-10T21:26:59.400002+0000","last_unstale":"2024-09-10T21:27:00.397163+0000","last_undegraded":"2024-09-10T21:27:00.397163+0000","last_fullsized":"2024-09-10T21:27:00.397163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:50.713206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.417194+0000","last_change":"2024-09-10T21:27:00.414254+0000","last_active":"2024-09-10T21:27:00.417194+0000","last_peered":"2024-09-10T21:27:00.417194+0000","last_clean":"2024-09-10T21:27:00.417194+0000","last_became_active":"2024-09-10T21:27:00.413853+0000","last_became_peered":"2024-09-10T21:27:00.413853+0000","last_unstale":"2024-09-10T21:27:00.417194+0000","last_undegraded":"2024-09-10T21:27:00.417194+0000","last_fullsized":"2024-09-10T21:27:00.417194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:07.343005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.433702+0000","last_change":"2024-09-10T21:26:59.399657+0000","last_active":"2024-09-10T21:27:00.433702+0000","last_peered":"2024-09-10T21:27:00.433702+0000","last_clean":"2024-09-10T21:27:00.433702+0000","last_became_active":"2024-09-10T21:26:59.399350+0000","last_became_peered":"2024-09-10T21:26:59.399350+0000","last_unstale":"2024-09-10T21:27:00.433702+0000","last_undegraded":"2024-09-10T21:27:00.433702+0000","last_fullsized":"2024-09-10T21:27:00.433702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:30.229587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.410916+0000","last_change":"2024-09-10T21:27:00.399775+0000","last_active":"2024-09-10T21:27:00.410916+0000","last_peered":"2024-09-10T21:27:00.410916+0000","last_clean":"2024-09-10T21:27:00.410916+0000","last_became_active":"2024-09-10T21:27:00.399514+0000","last_became_peered":"2024-09-10T21:27:00.399514+0000","last_unstale":"2024-09-10T21:27:00.410916+0000","last_undegraded":"2024-09-10T21:27:00.410916+0000","last_fullsized":"2024-09-10T21:27:00.410916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:41.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.403880+0000","last_change":"2024-09-10T21:26:59.402604+0000","last_active":"2024-09-10T21:27:00.403880+0000","last_peered":"2024-09-10T21:27:00.403880+0000","last_clean":"2024-09-10T21:27:00.403880+0000","last_became_active":"2024-09-10T21:26:59.402338+0000","last_became_peered":"2024-09-10T21:26:59.402338+0000","last_unstale":"2024-09-10T21:27:00.403880+0000","last_undegraded":"2024-09-10T21:27:00.403880+0000","last_fullsized":"2024-09-10T21:27:00.403880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:22.602159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.424924+0000","last_change":"2024-09-10T21:27:00.408300+0000","last_active":"2024-09-10T21:27:00.424924+0000","last_peered":"2024-09-10T21:27:00.424924+0000","last_clean":"2024-09-10T21:27:00.424924+0000","last_became_active":"2024-09-10T21:27:00.408003+0000","last_became_peered":"2024-09-10T21:27:00.408003+0000","last_unstale":"2024-09-10T21:27:00.424924+0000","last_undegraded":"2024-09-10T21:27:00.424924+0000","last_fullsized":"2024-09-10T21:27:00.424924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:01.241843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.401701+0000","last_change":"2024-09-10T21:26:59.400480+0000","last_active":"2024-09-10T21:27:00.401701+0000","last_peered":"2024-09-10T21:27:00.401701+0000","last_clean":"2024-09-10T21:27:00.401701+0000","last_became_active":"2024-09-10T21:26:59.399760+0000","last_became_peered":"2024-09-10T21:26:59.399760+0000","last_unstale":"2024-09-10T21:27:00.401701+0000","last_undegraded":"2024-09-10T21:27:00.401701+0000","last_fullsized":"2024-09-10T21:27:00.401701+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:47.320374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394998+0000","last_change":"2024-09-10T21:27:00.407556+0000","last_active":"2024-09-10T21:27:01.394998+0000","last_peered":"2024-09-10T21:27:01.394998+0000","last_clean":"2024-09-10T21:27:01.394998+0000","last_became_active":"2024-09-10T21:27:00.407265+0000","last_became_peered":"2024-09-10T21:27:00.407265+0000","last_unstale":"2024-09-10T21:27:01.394998+0000","last_undegraded":"2024-09-10T21:27:01.394998+0000","last_fullsized":"2024-09-10T21:27:01.394998+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:36.662564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.435079+0000","last_change":"2024-09-10T21:26:59.399989+0000","last_active":"2024-09-10T21:27:00.435079+0000","last_peered":"2024-09-10T21:27:00.435079+0000","last_clean":"2024-09-10T21:27:00.435079+0000","last_became_active":"2024-09-10T21:26:59.399719+0000","last_became_peered":"2024-09-10T21:26:59.399719+0000","last_unstale":"2024-09-10T21:27:00.435079+0000","last_undegraded":"2024-09-10T21:27:00.435079+0000","last_fullsized":"2024-09-10T21:27:00.435079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:33.782660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.395226+0000","last_change":"2024-09-10T21:27:00.410547+0000","last_active":"2024-09-10T21:27:01.395226+0000","last_peered":"2024-09-10T21:27:01.395226+0000","last_clean":"2024-09-10T21:27:01.395226+0000","last_became_active":"2024-09-10T21:27:00.409850+0000","last_became_peered":"2024-09-10T21:27:00.409850+0000","last_unstale":"2024-09-10T21:27:01.395226+0000","last_undegraded":"2024-09-10T21:27:01.395226+0000","last_fullsized":"2024-09-10T21:27:01.395226+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:10.199959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.397993+0000","last_change":"2024-09-10T21:26:59.398904+0000","last_active":"2024-09-10T21:27:00.397993+0000","last_peered":"2024-09-10T21:27:00.397993+0000","last_clean":"2024-09-10T21:27:00.397993+0000","last_became_active":"2024-09-10T21:26:59.398595+0000","last_became_peered":"2024-09-10T21:26:59.398595+0000","last_unstale":"2024-09-10T21:27:00.397993+0000","last_undegraded":"2024-09-10T21:27:00.397993+0000","last_fullsized":"2024-09-10T21:27:00.397993+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:38.637358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.427374+0000","last_change":"2024-09-10T21:27:00.423788+0000","last_active":"2024-09-10T21:27:00.427374+0000","last_peered":"2024-09-10T21:27:00.427374+0000","last_clean":"2024-09-10T21:27:00.427374+0000","last_became_active":"2024-09-10T21:27:00.421501+0000","last_became_peered":"2024-09-10T21:27:00.421501+0000","last_unstale":"2024-09-10T21:27:00.427374+0000","last_undegraded":"2024-09-10T21:27:00.427374+0000","last_fullsized":"2024-09-10T21:27:00.427374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:09.592805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.401550+0000","last_change":"2024-09-10T21:26:59.402679+0000","last_active":"2024-09-10T21:27:00.401550+0000","last_peered":"2024-09-10T21:27:00.401550+0000","last_clean":"2024-09-10T21:27:00.401550+0000","last_became_active":"2024-09-10T21:26:59.402367+0000","last_became_peered":"2024-09-10T21:26:59.402367+0000","last_unstale":"2024-09-10T21:27:00.401550+0000","last_undegraded":"2024-09-10T21:27:00.401550+0000","last_fullsized":"2024-09-10T21:27:00.401550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:57.521015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425555+0000","last_change":"2024-09-10T21:27:00.411680+0000","last_active":"2024-09-10T21:27:00.425555+0000","last_peered":"2024-09-10T21:27:00.425555+0000","last_clean":"2024-09-10T21:27:00.425555+0000","last_became_active":"2024-09-10T21:27:00.411404+0000","last_became_peered":"2024-09-10T21:27:00.411404+0000","last_unstale":"2024-09-10T21:27:00.425555+0000","last_undegraded":"2024-09-10T21:27:00.425555+0000","last_fullsized":"2024-09-10T21:27:00.425555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:28.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.396307+0000","last_change":"2024-09-10T21:26:59.393197+0000","last_active":"2024-09-10T21:27:00.396307+0000","last_peered":"2024-09-10T21:27:00.396307+0000","last_clean":"2024-09-10T21:27:00.396307+0000","last_became_active":"2024-09-10T21:26:59.392888+0000","last_became_peered":"2024-09-10T21:26:59.392888+0000","last_unstale":"2024-09-10T21:27:00.396307+0000","last_undegraded":"2024-09-10T21:27:00.396307+0000","last_fullsized":"2024-09-10T21:27:00.396307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:39.983295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.425487+0000","last_change":"2024-09-10T21:27:00.412032+0000","last_active":"2024-09-10T21:27:00.425487+0000","last_peered":"2024-09-10T21:27:00.425487+0000","last_clean":"2024-09-10T21:27:00.425487+0000","last_became_active":"2024-09-10T21:27:00.411758+0000","last_became_peered":"2024-09-10T21:27:00.411758+0000","last_unstale":"2024-09-10T21:27:00.425487+0000","last_undegraded":"2024-09-10T21:27:00.425487+0000","last_fullsized":"2024-09-10T21:27:00.425487+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:38.284949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.390350+0000","last_change":"2024-09-10T21:26:59.396950+0000","last_active":"2024-09-10T21:27:00.390350+0000","last_peered":"2024-09-10T21:27:00.390350+0000","last_clean":"2024-09-10T21:27:00.390350+0000","last_became_active":"2024-09-10T21:26:59.396659+0000","last_became_peered":"2024-09-10T21:26:59.396659+0000","last_unstale":"2024-09-10T21:27:00.390350+0000","last_undegraded":"2024-09-10T21:27:00.390350+0000","last_fullsized":"2024-09-10T21:27:00.390350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:02.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.417548+0000","last_change":"2024-09-10T21:27:00.411085+0000","last_active":"2024-09-10T21:27:00.417548+0000","last_peered":"2024-09-10T21:27:00.417548+0000","last_clean":"2024-09-10T21:27:00.417548+0000","last_became_active":"2024-09-10T21:27:00.410746+0000","last_became_peered":"2024-09-10T21:27:00.410746+0000","last_unstale":"2024-09-10T21:27:00.417548+0000","last_undegraded":"2024-09-10T21:27:00.417548+0000","last_fullsized":"2024-09-10T21:27:00.417548+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:17.200466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.402893+0000","last_change":"2024-09-10T21:26:59.398547+0000","last_active":"2024-09-10T21:27:00.402893+0000","last_peered":"2024-09-10T21:27:00.402893+0000","last_clean":"2024-09-10T21:27:00.402893+0000","last_became_active":"2024-09-10T21:26:59.398187+0000","last_became_peered":"2024-09-10T21:26:59.398187+0000","last_unstale":"2024-09-10T21:27:00.402893+0000","last_undegraded":"2024-09-10T21:27:00.402893+0000","last_fullsized":"2024-09-10T21:27:00.402893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:11.704238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.430681+0000","last_change":"2024-09-10T21:27:00.425585+0000","last_active":"2024-09-10T21:27:00.430681+0000","last_peered":"2024-09-10T21:27:00.430681+0000","last_clean":"2024-09-10T21:27:00.430681+0000","last_became_active":"2024-09-10T21:27:00.425274+0000","last_became_peered":"2024-09-10T21:27:00.425274+0000","last_unstale":"2024-09-10T21:27:00.430681+0000","last_undegraded":"2024-09-10T21:27:00.430681+0000","last_fullsized":"2024-09-10T21:27:00.430681+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:10.572804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.396074+0000","last_change":"2024-09-10T21:26:59.392982+0000","last_active":"2024-09-10T21:27:00.396074+0000","last_peered":"2024-09-10T21:27:00.396074+0000","last_clean":"2024-09-10T21:27:00.396074+0000","last_became_active":"2024-09-10T21:26:59.392672+0000","last_became_peered":"2024-09-10T21:26:59.392672+0000","last_unstale":"2024-09-10T21:27:00.396074+0000","last_undegraded":"2024-09-10T21:27:00.396074+0000","last_fullsized":"2024-09-10T21:27:00.396074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:54.508797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T21:27:01.394940+0000","last_change":"2024-09-10T21:27:00.404292+0000","last_active":"2024-09-10T21:27:01.394940+0000","last_peered":"2024-09-10T21:27:01.394940+0000","last_clean":"2024-09-10T21:27:01.394940+0000","last_became_active":"2024-09-10T21:27:00.403868+0000","last_became_peered":"2024-09-10T21:27:00.403868+0000","last_unstale":"2024-09-10T21:27:01.394940+0000","last_undegraded":"2024-09-10T21:27:01.394940+0000","last_fullsized":"2024-09-10T21:27:01.394940+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:50.281693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.400948+0000","last_change":"2024-09-10T21:26:59.405102+0000","last_active":"2024-09-10T21:27:00.400948+0000","last_peered":"2024-09-10T21:27:00.400948+0000","last_clean":"2024-09-10T21:27:00.400948+0000","last_became_active":"2024-09-10T21:26:59.404841+0000","last_became_peered":"2024-09-10T21:26:59.404841+0000","last_unstale":"2024-09-10T21:27:00.400948+0000","last_undegraded":"2024-09-10T21:27:00.400948+0000","last_fullsized":"2024-09-10T21:27:00.400948+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:13.203314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.423284+0000","last_change":"2024-09-10T21:27:00.419630+0000","last_active":"2024-09-10T21:27:00.423284+0000","last_peered":"2024-09-10T21:27:00.423284+0000","last_clean":"2024-09-10T21:27:00.423284+0000","last_became_active":"2024-09-10T21:27:00.418133+0000","last_became_peered":"2024-09-10T21:27:00.418133+0000","last_unstale":"2024-09-10T21:27:00.423284+0000","last_undegraded":"2024-09-10T21:27:00.423284+0000","last_fullsized":"2024-09-10T21:27:00.423284+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:59.373749+0000","last_clean_scrub_stamp":"2024-09-10T21:26:59.373749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:03.176928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.402253+0000","last_change":"2024-09-10T21:26:59.395944+0000","last_active":"2024-09-10T21:27:00.402253+0000","last_peered":"2024-09-10T21:27:00.402253+0000","last_clean":"2024-09-10T21:27:00.402253+0000","last_became_active":"2024-09-10T21:26:59.395685+0000","last_became_peered":"2024-09-10T21:26:59.395685+0000","last_unstale":"2024-09-10T21:27:00.402253+0000","last_undegraded":"2024-09-10T21:27:00.402253+0000","last_fullsized":"2024-09-10T21:27:00.402253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:26:58.366276+0000","last_clean_scrub_stamp":"2024-09-10T21:26:58.366276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:23.835253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":194,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T21:27:00.406950+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:27:00.406950+0000","last_peered":"2024-09-10T21:27:00.406950+0000","last_clean":"2024-09-10T21:27:00.406950+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:27:00.406950+0000","last_undegraded":"2024-09-10T21:27:00.406950+0000","last_fullsized":"2024-09-10T21:27:00.406950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":355959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":358223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28404,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343436,"statfs":{"total":96636764160,"available":96607678464,"internally_reserved":0,"allocated":1404928,"data_stored":948051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":356119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":356015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":1388544,"data_stored":946361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":355959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":806912,"data_stored":363233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:27:04.953 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:27:04.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:27:05.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.098+0000 7f6264e00640 1 -- 172.21.15.97:0/399367924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244003a40 msgr2=0x7f6244003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.098+0000 7f6264e00640 1 --2- 172.21.15.97:0/399367924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244003a40 0x7f6244003e40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f625000c310 tx=0x7f625001ceb0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.098+0000 7f6264e00640 1 -- 172.21.15.97:0/399367924 shutdown_connections 2024-09-10T21:27:05.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.098+0000 7f6264e00640 1 --2- 172.21.15.97:0/399367924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6244003a40 0x7f6244003e40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.098+0000 7f6264e00640 1 -- 172.21.15.97:0/399367924 >> 172.21.15.97:0/399367924 conn(0x7f62600d3ea0 msgr2=0x7f62600d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:05.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.098+0000 7f6264e00640 1 -- 172.21.15.97:0/399367924 shutdown_connections 2024-09-10T21:27:05.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.098+0000 7f6264e00640 1 -- 172.21.15.97:0/399367924 wait complete. 2024-09-10T21:27:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 Processor -- start 2024-09-10T21:27:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 -- start start 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6244003a40 0x7f62600ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f626015d340 0x7f626014ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626014ef40 0x7f62600c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f62600c7650 con 0x7f626014ef40 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62600cfd50 con 0x7f6244003a40 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f62600c77d0 con 0x7f626015d340 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626014ef40 0x7f62600c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626014ef40 0x7f62600c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52644/0 (socket says 172.21.15.97:52644) 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 -- 172.21.15.97:0/285134769 learned_addr learned my addr 172.21.15.97:0/285134769 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625de00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6244003a40 0x7f62600ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625d400640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f626015d340 0x7f626014ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 -- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f626015d340 msgr2=0x7f626014ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f626015d340 0x7f626014ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 -- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6244003a40 msgr2=0x7f62600ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6244003a40 0x7f62600ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 -- 172.21.15.97:0/285134769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626015d7d0 con 0x7f626014ef40 2024-09-10T21:27:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f625e800640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626014ef40 0x7f62600c83e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f626008d770 tx=0x7f625800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6256000640 1 -- 172.21.15.97:0/285134769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258017070 con 0x7f626014ef40 2024-09-10T21:27:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6256000640 1 -- 172.21.15.97:0/285134769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f625800c940 con 0x7f626014ef40 2024-09-10T21:27:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6256000640 1 -- 172.21.15.97:0/285134769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625800cd90 con 0x7f626014ef40 2024-09-10T21:27:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626015dab0 con 0x7f626014ef40 2024-09-10T21:27:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.102+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626015e070 con 0x7f626014ef40 2024-09-10T21:27:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.106+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62600d2830 con 0x7f626014ef40 2024-09-10T21:27:05.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.106+0000 7f6256000640 1 -- 172.21.15.97:0/285134769 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6258011460 con 0x7f626014ef40 2024-09-10T21:27:05.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.106+0000 7f6256000640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62400725e0 0x7f6240074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.106+0000 7f6256000640 1 -- 172.21.15.97:0/285134769 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f62580938d0 con 0x7f626014ef40 2024-09-10T21:27:05.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.106+0000 7f625de00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62400725e0 0x7f6240074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.106+0000 7f625de00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62400725e0 0x7f6240074aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62500ab000 tx=0x7f6250019cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.110+0000 7f6256000640 1 -- 172.21.15.97:0/285134769 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f625805ee10 con 0x7f626014ef40 2024-09-10T21:27:05.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.334+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f62600b6420 con 0x7f626014ef40 2024-09-10T21:27:05.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.334+0000 7f6256000640 1 -- 172.21.15.97:0/285134769 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f6258003e20 con 0x7f626014ef40 2024-09-10T21:27:05.338 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-10T21:27:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:27:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-09-10T21:27:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-09-10T21:27:05.339 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:27:05.339 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-09-10T21:27:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62400725e0 msgr2=0x7f6240074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62400725e0 0x7f6240074aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62500ab000 tx=0x7f6250019cf0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626014ef40 msgr2=0x7f62600c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626014ef40 0x7f62600c83e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f626008d770 tx=0x7f625800eca0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 shutdown_connections 2024-09-10T21:27:05.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62400725e0 0x7f6240074aa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626014ef40 0x7f62600c83e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f626015d340 0x7f626014ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 --2- 172.21.15.97:0/285134769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6244003a40 0x7f62600ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 >> 172.21.15.97:0/285134769 conn(0x7f62600d3ea0 msgr2=0x7f62601ed7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:05.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 shutdown_connections 2024-09-10T21:27:05.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.338+0000 7f6264e00640 1 -- 172.21.15.97:0/285134769 wait complete. 2024-09-10T21:27:05.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:05.369 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:27:05.366+0000 7f0cdd000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:27:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 -- 172.21.15.97:0/2063959987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006d180 msgr2=0x7ff4b0070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/2063959987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006d180 0x7ff4b0070130 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff498009840 tx=0x7ff49801c7f0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 -- 172.21.15.97:0/2063959987 shutdown_connections 2024-09-10T21:27:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/2063959987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4b0161820 0x7ff4b00b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/2063959987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006d180 0x7ff4b0070130 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 -- 172.21.15.97:0/2063959987 >> 172.21.15.97:0/2063959987 conn(0x7ff4b00b2790 msgr2=0x7ff4b00b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 -- 172.21.15.97:0/2063959987 shutdown_connections 2024-09-10T21:27:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.498+0000 7ff4b5e00640 1 -- 172.21.15.97:0/2063959987 wait complete. 2024-09-10T21:27:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 Processor -- start 2024-09-10T21:27:05.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 -- start start 2024-09-10T21:27:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006cda0 0x7ff4b00ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4b006d180 0x7ff4b00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b0161820 0x7ff4b014c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b016b0c0 con 0x7ff4b006cda0 2024-09-10T21:27:05.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b016af40 con 0x7ff4b0161820 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b016b240 con 0x7ff4b006d180 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006cda0 0x7ff4b00ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006cda0 0x7ff4b00ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52656/0 (socket says 172.21.15.97:52656) 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 -- 172.21.15.97:0/3094866810 learned_addr learned my addr 172.21.15.97:0/3094866810 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4af400640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b0161820 0x7ff4b014c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4a7e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4b006d180 0x7ff4b00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 -- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4b006d180 msgr2=0x7ff4b00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4b006d180 0x7ff4b00b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 -- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b0161820 msgr2=0x7ff4b014c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b0161820 0x7ff4b014c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 -- 172.21.15.97:0/3094866810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b00afc10 con 0x7ff4b006cda0 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4af400640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b0161820 0x7ff4b014c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:05.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4aea00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006cda0 0x7ff4b00ae2c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c450 tx=0x7ff4a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4a7400640 1 -- 172.21.15.97:0/3094866810 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0016020 con 0x7ff4b006cda0 2024-09-10T21:27:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4a7400640 1 -- 172.21.15.97:0/3094866810 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4a0012920 con 0x7ff4b006cda0 2024-09-10T21:27:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4a7400640 1 -- 172.21.15.97:0/3094866810 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0012d70 con 0x7ff4b006cda0 2024-09-10T21:27:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b00af2e0 con 0x7ff4b006cda0 2024-09-10T21:27:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b00af8a0 con 0x7ff4b006cda0 2024-09-10T21:27:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.502+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b0070130 con 0x7ff4b006cda0 2024-09-10T21:27:05.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.506+0000 7ff4a7400640 1 -- 172.21.15.97:0/3094866810 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff4a001b400 con 0x7ff4b006cda0 2024-09-10T21:27:05.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.506+0000 7ff4a7400640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff48c0725e0 0x7ff48c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:05.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.506+0000 7ff4a7400640 1 -- 172.21.15.97:0/3094866810 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff4a00933a0 con 0x7ff4b006cda0 2024-09-10T21:27:05.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.506+0000 7ff4a7e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff48c0725e0 0x7ff48c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:05.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.506+0000 7ff4a7e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff48c0725e0 0x7ff48c074aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4980af000 tx=0x7ff4980b4040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:05.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.510+0000 7ff4a7400640 1 -- 172.21.15.97:0/3094866810 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4a0020030 con 0x7ff4b006cda0 2024-09-10T21:27:05.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4b00b1320 con 0x7ff4b006cda0 2024-09-10T21:27:05.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4a7400640 1 -- 172.21.15.97:0/3094866810 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5230 (secure 0 0 0) 0x7ff4b015e230 con 0x7ff4b006cda0 2024-09-10T21:27:05.713 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:05.714 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":48,"btime":"2024-09-10T21:27:03:415307+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:02.795427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/2609945029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2609945029},{"type":"v1","addr":"172.21.15.97:6835","nonce":2609945029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-10T21:27:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 48 2024-09-10T21:27:05.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff48c0725e0 msgr2=0x7ff48c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff48c0725e0 0x7ff48c074aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4980af000 tx=0x7ff4980b4040 comp rx=0 tx=0).stop 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006cda0 msgr2=0x7ff4b00ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006cda0 0x7ff4b00ae2c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c450 tx=0x7ff4a000c910 comp rx=0 tx=0).stop 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 shutdown_connections 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff48c0725e0 0x7ff48c074aa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b0161820 0x7ff4b014c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff4b006d180 0x7ff4b00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 --2- 172.21.15.97:0/3094866810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b006cda0 0x7ff4b00ae2c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:05.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 >> 172.21.15.97:0/3094866810 conn(0x7ff4b00b2790 msgr2=0x7ff4b00b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:05.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 shutdown_connections 2024-09-10T21:27:05.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:05.710+0000 7ff4b5e00640 1 -- 172.21.15.97:0/3094866810 wait complete. 2024-09-10T21:27:06.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:59.777372+0000', 'modified': '2024-09-10T21:27:02.795427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5964}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5964': {'gid': 5964, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6835/2609945029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 2609945029}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2609945029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5964, 'qdb_cluster': [5964]} 2024-09-10T21:27:06.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:27:06.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 -- 172.21.15.97:0/2332535175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 msgr2=0x7fe6740b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 --2- 172.21.15.97:0/2332535175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740b37b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe664009c80 tx=0x7fe66401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 -- 172.21.15.97:0/2332535175 shutdown_connections 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 --2- 172.21.15.97:0/2332535175 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe6740b3cf0 0x7fe67406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 --2- 172.21.15.97:0/2332535175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740b37b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 -- 172.21.15.97:0/2332535175 >> 172.21.15.97:0/2332535175 conn(0x7fe6740b1780 msgr2=0x7fe6740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 -- 172.21.15.97:0/2332535175 shutdown_connections 2024-09-10T21:27:06.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 -- 172.21.15.97:0/2332535175 wait complete. 2024-09-10T21:27:06.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 Processor -- start 2024-09-10T21:27:06.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.278+0000 7fe679000640 1 -- start start 2024-09-10T21:27:06.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe67406f180 0x7fe6741f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6740b3cf0 0x7fe6741f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe67406e740 con 0x7fe6741511f0 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe67406e5c0 con 0x7fe6740b3cf0 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe67406e8c0 con 0x7fe67406f180 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52674/0 (socket says 172.21.15.97:52674) 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 -- 172.21.15.97:0/4135865757 learned_addr learned my addr 172.21.15.97:0/4135865757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe669400640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6740b3cf0 0x7fe6741f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe671e00640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe67406f180 0x7fe6741f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 -- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe67406f180 msgr2=0x7fe6741f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe67406f180 0x7fe6741f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 -- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6740b3cf0 msgr2=0x7fe6741f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6740b3cf0 0x7fe6741f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 -- 172.21.15.97:0/4135865757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6741f5340 con 0x7fe6741511f0 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe672800640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740cd800 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b410 tx=0x7fe66c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe66aa00640 1 -- 172.21.15.97:0/4135865757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c015020 con 0x7fe6741511f0 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe66aa00640 1 -- 172.21.15.97:0/4135865757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe66c011920 con 0x7fe6741511f0 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe66aa00640 1 -- 172.21.15.97:0/4135865757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c011d70 con 0x7fe6741511f0 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6741f5620 con 0x7fe6741511f0 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.282+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6741e5a80 con 0x7fe6741511f0 2024-09-10T21:27:06.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe66aa00640 1 -- 172.21.15.97:0/4135865757 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe66c01a400 con 0x7fe6741511f0 2024-09-10T21:27:06.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe66aa00640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe650072610 0x7fe650074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe66aa00640 1 -- 172.21.15.97:0/4135865757 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fe66c093490 con 0x7fe6741511f0 2024-09-10T21:27:06.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 Processor -- start 2024-09-10T21:27:06.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 -- start start 2024-09-10T21:27:06.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe671e00640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe650072610 0x7fe650074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe671e00640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe650072610 0x7fe650074ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6640b3000 tx=0x7fe66401dde0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d09d0 0x55ee860d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee860848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee85fa53f0 con 0x55ee86084e10 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ee85ee2b20 con 0x55ee860d09d0 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ee85ed1f50 con 0x55ee860d12f0 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe671e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d09d0 0x55ee860d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860851f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52676/0 (socket says 172.21.15.97:52676) 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 -- 172.21.15.97:0/2704524966 learned_addr learned my addr 172.21.15.97:0/2704524966 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe669400640 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee860848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 -- 172.21.15.97:0/2704524966 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 msgr2=0x55ee860848d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee860848d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 -- 172.21.15.97:0/2704524966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d09d0 msgr2=0x55ee860d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d09d0 0x55ee860d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 -- 172.21.15.97:0/2704524966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee85e99590 con 0x55ee86084e10 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe672800640 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860851f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe66c013000 tx=0x7fe66c05db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b442dcda526b9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe662a00640 1 -- 172.21.15.97:0/2704524966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c067120 con 0x55ee86084e10 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe662a00640 1 -- 172.21.15.97:0/2704524966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe66c062db0 con 0x55ee86084e10 2024-09-10T21:27:06.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.286+0000 7fe662a00640 1 -- 172.21.15.97:0/2704524966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0677d0 con 0x55ee86084e10 2024-09-10T21:27:06.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- 172.21.15.97:0/2704524966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 msgr2=0x55ee860851f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860851f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe66c013000 tx=0x7fe66c05db70 comp rx=0 tx=0).stop 2024-09-10T21:27:06.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- 172.21.15.97:0/2704524966 shutdown_connections 2024-09-10T21:27:06.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860851f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee860848d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 --2- 172.21.15.97:0/2704524966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d09d0 0x55ee860d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- 172.21.15.97:0/2704524966 >> 172.21.15.97:0/2704524966 conn(0x55ee85f6e130 msgr2=0x55ee860b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- 172.21.15.97:0/2704524966 shutdown_connections 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- 172.21.15.97:0/2704524966 wait complete. 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- start start 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 Processor -- start 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d09d0 0x55ee860b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d12f0 0x55ee860b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ee85e6c5d0 con 0x55ee86084e10 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ee85ed5d60 con 0x55ee860d12f0 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ee85e78500 con 0x55ee860d09d0 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 10 client.0 _ll_register_callbacks cb 0x55ee85e7c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:06.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52682/0 (socket says 172.21.15.97:52682) 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 -- 172.21.15.97:0/2114973875 learned_addr learned my addr 172.21.15.97:0/2114973875 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe672800640 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d12f0 0x55ee860b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe669400640 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d09d0 0x55ee860b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 -- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d09d0 msgr2=0x55ee860b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d09d0 0x55ee860b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 -- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d12f0 msgr2=0x55ee860b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d12f0 0x55ee860b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 -- 172.21.15.97:0/2114973875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee85e70170 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe671e00640 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860b16a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe664026270 tx=0x7fe6640095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22108efc522ddd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe657400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe657400640 1 -- 172.21.15.97:0/2114973875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6640b6040 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe657400640 1 -- 172.21.15.97:0/2114973875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fe6640b1440 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee85e6a0f0 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe657400640 1 -- 172.21.15.97:0/2114973875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe6640d8050 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 10 client.6078 fetch_fsmap learned FSMap version 48 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x55ee85f56ef0 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.290+0000 7fe657400640 1 -- 172.21.15.97:0/2114973875 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ee85e6a0f0 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe657400640 1 -- 172.21.15.97:0/2114973875 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7fe6640db020 con 0x55ee86084e10 2024-09-10T21:27:06.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 10 client.6078 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 10 client.6078 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 20 client.6078 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860b30f0 0x55ee860e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 4 client.6078 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 --> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee85ee4a70 con 0x55ee860b30f0 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe669400640 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860b30f0 0x55ee860e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe669400640 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860b30f0 0x55ee860e2c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224bd19f362f9b70 server_cookie=e46122e4b8f2bff2 in_seq=0 out_seq=0 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe657400640 10 client.6078 ms_handle_connect on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe657400640 1 -- 172.21.15.97:0/2114973875 <== mds.0 v2:172.21.15.97:6834/2609945029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee85ee4a70 con 0x55ee860b30f0 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe657400640 10 client.6078 handle_command_reply: tid=0 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 client.6078 shutdown 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 msgr2=0x55ee860b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860b16a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe664026270 tx=0x7fe6640095c0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 shutdown_connections 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860b30f0 0x55ee860e2c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee860d12f0 0x55ee860b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d09d0 0x55ee860b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe657400640 0 client.6078 ms_handle_reset on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 --2- 172.21.15.97:0/2114973875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee86084e10 0x55ee860b16a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 >> 172.21.15.97:0/2114973875 conn(0x55ee85f6e130 msgr2=0x55ee86083fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 shutdown_connections 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 1 -- 172.21.15.97:0/2114973875 wait complete. 2024-09-10T21:27:06.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.294+0000 7fe67c478000 20 client.6078 trim_cache size 0 max 16384 2024-09-10T21:27:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 Processor -- start 2024-09-10T21:27:06.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- start start 2024-09-10T21:27:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86143310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe674201780 con 0x55ee860d09d0 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6742007b0 con 0x55ee86084e10 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe6741e4c00 con 0x55ee860d12f0 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52696/0 (socket says 172.21.15.97:52696) 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 -- 172.21.15.97:0/3596988772 learned_addr learned my addr 172.21.15.97:0/3596988772 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe672800640 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe671e00640 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 -- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 msgr2=0x55ee86143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 -- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 msgr2=0x55ee86142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 -- 172.21.15.97:0/3596988772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee85f2dd90 con 0x55ee860d09d0 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe672800640 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe671e00640 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe669400640 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86143310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55ee85e6a400 tx=0x7fe65c00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439e7bd1a52d4b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe662000640 1 -- 172.21.15.97:0/3596988772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c005220 con 0x55ee860d09d0 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe662000640 1 -- 172.21.15.97:0/3596988772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe65c0053c0 con 0x55ee860d09d0 2024-09-10T21:27:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe662000640 1 -- 172.21.15.97:0/3596988772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0047b0 con 0x55ee860d09d0 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- 172.21.15.97:0/3596988772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 msgr2=0x55ee86143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86143310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55ee85e6a400 tx=0x7fe65c00a2d0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- 172.21.15.97:0/3596988772 shutdown_connections 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86143310 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 --2- 172.21.15.97:0/3596988772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- 172.21.15.97:0/3596988772 >> 172.21.15.97:0/3596988772 conn(0x55ee85f6e130 msgr2=0x55ee860ddfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- 172.21.15.97:0/3596988772 shutdown_connections 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.314+0000 7fe67c478000 1 -- 172.21.15.97:0/3596988772 wait complete. 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 -- start start 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 Processor -- start 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86146c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86147140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe67414eb50 con 0x55ee860d09d0 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ee860e7ce0 con 0x55ee86084e10 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ee85ee18f0 con 0x55ee860d12f0 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 10 client.0 _ll_register_callbacks cb 0x7fe6640098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86147140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52708/0 (socket says 172.21.15.97:52708) 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 -- 172.21.15.97:0/712235211 learned_addr learned my addr 172.21.15.97:0/712235211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe671e00640 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe672800640 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 -- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 msgr2=0x55ee86147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 -- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 msgr2=0x55ee86146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86146c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 -- 172.21.15.97:0/712235211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee85e98a90 con 0x55ee860d09d0 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe669400640 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86147140 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe65c026040 tx=0x7fe65c005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e106224caf56c891 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe657400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe657400640 1 -- 172.21.15.97:0/712235211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c032020 con 0x55ee860d09d0 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe657400640 1 -- 172.21.15.97:0/712235211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fe65c0047b0 con 0x55ee860d09d0 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe65c0053c0 con 0x55ee860d09d0 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe657400640 1 -- 172.21.15.97:0/712235211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe65c0049f0 con 0x55ee860d09d0 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 10 client.6084 fetch_fsmap learned FSMap version 48 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x55ee85e77050 con 0x55ee860d09d0 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe657400640 1 -- 172.21.15.97:0/712235211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe65c0053c0 con 0x55ee860d09d0 2024-09-10T21:27:06.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe657400640 1 -- 172.21.15.97:0/712235211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7fe65c02e450 con 0x55ee860d09d0 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 10 client.6084 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 10 client.6084 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 20 client.6084 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860e5c60 0x55ee860e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 4 client.6084 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 --> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] -- command(tid 0: {"prefix": "status"}) -- 0x55ee85ee0520 con 0x55ee860e5c60 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe671e00640 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860e5c60 0x55ee860e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe671e00640 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860e5c60 0x55ee860e4d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c352d98bdcfb48 server_cookie=fba261c8d4fdf35a in_seq=0 out_seq=0 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.318+0000 7fe657400640 10 client.6084 ms_handle_connect on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:06.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe657400640 1 -- 172.21.15.97:0/712235211 <== mds.0 v2:172.21.15.97:6834/2609945029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ee85ee0520 con 0x55ee860e5c60 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe657400640 10 client.6084 handle_command_reply: tid=0 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 client.6084 shutdown 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 msgr2=0x55ee86147140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86147140 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe65c026040 tx=0x7fe65c005820 comp rx=0 tx=0).stop 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 shutdown_connections 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x55ee860e5c60 0x55ee860e4d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ee860d12f0 0x55ee86147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ee860d09d0 0x55ee86147140 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe657400640 0 client.6084 ms_handle_reset on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 --2- 172.21.15.97:0/712235211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ee86084e10 0x55ee86146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 >> 172.21.15.97:0/712235211 conn(0x55ee85f6e130 msgr2=0x55ee8608ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 shutdown_connections 2024-09-10T21:27:06.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 1 -- 172.21.15.97:0/712235211 wait complete. 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe67c478000 20 client.6084 trim_cache size 0 max 16384 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5964, 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:27:06.331 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:27:06.332 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.9193231879999999, 2024-09-10T21:27:06.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 48, 2024-09-10T21:27:06.332 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 67, 2024-09-10T21:27:06.332 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 67, 2024-09-10T21:27:06.332 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 11.506350947 2024-09-10T21:27:06.332 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:27:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe650072610 msgr2=0x7fe650074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe650072610 0x7fe650074ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6640b3000 tx=0x7fe66401dde0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 msgr2=0x7fe6740cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740cd800 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b410 tx=0x7fe66c00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 shutdown_connections 2024-09-10T21:27:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe650072610 0x7fe650074ad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6741511f0 0x7fe6740cd800 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6740b3cf0 0x7fe6741f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 --2- 172.21.15.97:0/4135865757 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe67406f180 0x7fe6741f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 >> 172.21.15.97:0/4135865757 conn(0x7fe6740b1780 msgr2=0x7fe6741506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 shutdown_connections 2024-09-10T21:27:06.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.322+0000 7fe679000640 1 -- 172.21.15.97:0/4135865757 wait complete. 2024-09-10T21:27:06.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:27:06.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 -- 172.21.15.97:0/4269404265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 msgr2=0x7f3ee80af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 --2- 172.21.15.97:0/4269404265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee80af4d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009900 tx=0x7f3ee001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 -- 172.21.15.97:0/4269404265 shutdown_connections 2024-09-10T21:27:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 --2- 172.21.15.97:0/4269404265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee80af4d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 --2- 172.21.15.97:0/4269404265 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ee80ce640 0x7f3ee80cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 -- 172.21.15.97:0/4269404265 >> 172.21.15.97:0/4269404265 conn(0x7f3ee80d1c30 msgr2=0x7f3ee80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 -- 172.21.15.97:0/4269404265 shutdown_connections 2024-09-10T21:27:06.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.482+0000 7f3eee400640 1 -- 172.21.15.97:0/4269404265 wait complete. 2024-09-10T21:27:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 Processor -- start 2024-09-10T21:27:06.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 -- start start 2024-09-10T21:27:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ee80b0d50 0x7f3ee80b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee814dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ee80ce640 0x7f3ee815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8168c70 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8168af0 con 0x7f3ee80ce640 2024-09-10T21:27:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee8168df0 con 0x7f3ee80b0d50 2024-09-10T21:27:06.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee814dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee814dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52728/0 (socket says 172.21.15.97:52728) 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 -- 172.21.15.97:0/2081680579 learned_addr learned my addr 172.21.15.97:0/2081680579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee7400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ee80b0d50 0x7f3ee80b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee7e00640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ee80ce640 0x7f3ee815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 -- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ee80b0d50 msgr2=0x7f3ee80b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ee80b0d50 0x7f3ee80b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 -- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ee80ce640 msgr2=0x7f3ee815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ee80ce640 0x7f3ee815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 -- 172.21.15.97:0/2081680579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee814e890 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3ee6a00640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee814dd90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c480 tx=0x7f3ed400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3edf400640 1 -- 172.21.15.97:0/2081680579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4017070 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3edf400640 1 -- 172.21.15.97:0/2081680579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ed4003600 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3edf400640 1 -- 172.21.15.97:0/2081680579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4003a50 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee814eb70 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee815b500 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.486+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee80cea20 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.490+0000 7f3edf400640 1 -- 172.21.15.97:0/2081680579 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ed4003bf0 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.490+0000 7f3edf400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ec80725e0 0x7f3ec8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:06.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.490+0000 7f3edf400640 1 -- 172.21.15.97:0/2081680579 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3ed4092a70 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.490+0000 7f3ee7400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ec80725e0 0x7f3ec8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:06.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.490+0000 7f3ee7400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ec80725e0 0x7f3ec8074aa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009e50 tx=0x7f3ed80095e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:06.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.494+0000 7f3edf400640 1 -- 172.21.15.97:0/2081680579 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ed405da20 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ee80d2350 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3edf400640 1 -- 172.21.15.97:0/2081680579 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7f3ed4062450 con 0x7f3ee80cd6e0 2024-09-10T21:27:06.701 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ec80725e0 msgr2=0x7f3ec8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ec80725e0 0x7f3ec8074aa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009e50 tx=0x7f3ed80095e0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 msgr2=0x7f3ee814dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee814dd90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c480 tx=0x7f3ed400c940 comp rx=0 tx=0).stop 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 shutdown_connections 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ec80725e0 0x7f3ec8074aa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ee80ce640 0x7f3ee815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee80cd6e0 0x7f3ee814dd90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 --2- 172.21.15.97:0/2081680579 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ee80b0d50 0x7f3ee80b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:06.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 >> 172.21.15.97:0/2081680579 conn(0x7f3ee80d1c30 msgr2=0x7f3ee80b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 shutdown_connections 2024-09-10T21:27:06.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06.698+0000 7f3eee400640 1 -- 172.21.15.97:0/2081680579 wait complete. 2024-09-10T21:27:06.722 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:27:06.722 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:27:06.722 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:27:06.722 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:27:06.722 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:06.723 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:27:06.723 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:27:06.723 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:27:06.723 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:27:06.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:06.723 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:27:06.733 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42267sec preferred_lft 42267sec 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.734 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:27:06.735 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:06.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:06.736 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:27:06.793 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:27:06.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:06.795 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:27:06.847 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:27:06.847 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:27:06.847 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:27:06.847 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:27:06.847 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:27:06.847 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:27:06.847 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:27:06.848 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:27:06.848 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:27:06.848 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:27:06.848 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:27:06.848 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:27:06.848 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:27:06.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:27:06.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:27:06.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:27:06.848 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:27:06.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:27:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:27:06.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:06.984 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:27:06.993 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:27:06.993 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:27:06.994 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:27:06.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:06.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:27:07.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:27:07.055 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:07.103 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:27:07.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:27:07.103 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:07.154 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:27:07.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:07.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:07.246 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:07.246 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:27:07.246 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:27:07.261 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:07.262 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:27:07.262 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:27:07.262 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:27:07.262 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:27:07.270 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:07.270 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:27:07.270 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:27:07.270 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:27:07.270 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:27:07.270 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:27:07.271 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:27:07.271 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:27:07.274 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:27:07.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:07.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:07.294 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:27:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:27:07.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:27:07.402 INFO:teuthology.orchestra.run.smithi097.stdout:client5492 2024-09-10T21:27:07.403 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:27:07.403 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:27:07.451 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d; sync .') 2024-09-10T21:27:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:27:07.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:27:07.565 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:27:07.565 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:27:07.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:27:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- 172.21.15.97:0/2915332994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc15fde0 msgr2=0x7f3ddc13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 --2- 172.21.15.97:0/2915332994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc15fde0 0x7f3ddc13f9b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc074ae0 tx=0x7f3dd401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- 172.21.15.97:0/2915332994 shutdown_connections 2024-09-10T21:27:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 --2- 172.21.15.97:0/2915332994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc15fde0 0x7f3ddc13f9b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 --2- 172.21.15.97:0/2915332994 >> v2:172.21.15.194:3300/0 conn(0x7f3ddc0ce5f0 0x7f3ddc141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- 172.21.15.97:0/2915332994 >> 172.21.15.97:0/2915332994 conn(0x7f3ddc0b1780 msgr2=0x7f3ddc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- 172.21.15.97:0/2915332994 shutdown_connections 2024-09-10T21:27:07.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- 172.21.15.97:0/2915332994 wait complete. 2024-09-10T21:27:07.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 Processor -- start 2024-09-10T21:27:07.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- start start 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ddc0ce5f0 0x7f3ddc15e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc0ce9d0 0x7f3ddc15ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ddc15fde0 0x7f3ddc164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc168a70 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc140cd0 con 0x7f3ddc15fde0 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de3c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc168bf0 con 0x7f3ddc0ce5f0 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc0ce9d0 0x7f3ddc15ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc0ce9d0 0x7f3ddc15ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45592/0 (socket says 172.21.15.97:45592) 2024-09-10T21:27:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 -- 172.21.15.97:0/4086878363 learned_addr learned my addr 172.21.15.97:0/4086878363 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de1c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ddc15fde0 0x7f3ddc164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3de1200640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ddc0ce5f0 0x7f3ddc15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 -- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ddc0ce5f0 msgr2=0x7f3ddc15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ddc0ce5f0 0x7f3ddc15e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 -- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ddc15fde0 msgr2=0x7f3ddc164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ddc15fde0 0x7f3ddc164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.754+0000 7f3ddbe00640 1 -- 172.21.15.97:0/4086878363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc0b4270 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3ddbe00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc0ce9d0 0x7f3ddc15ed40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c530 tx=0x7f3dc800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3dd8c00640 1 -- 172.21.15.97:0/4086878363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8017070 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3dd8c00640 1 -- 172.21.15.97:0/4086878363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dc8003600 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc13bce0 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3dd8c00640 1 -- 172.21.15.97:0/4086878363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8003a50 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc0b4a00 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc141d20 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3dd8c00640 1 -- 172.21.15.97:0/4086878363 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3dc8003bf0 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3dd8c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc072610 0x7f3dbc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3dd8c00640 1 -- 172.21.15.97:0/4086878363 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f3dc8092ad0 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.758+0000 7f3de1200640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc072610 0x7f3dbc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.762+0000 7f3de1200640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc072610 0x7f3dbc074ad0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc005c50 tx=0x7f3dcc00a3b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.762+0000 7f3dd8c00640 1 -- 172.21.15.97:0/4086878363 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3dc805e0d0 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.938+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ddc0cca70 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.938+0000 7f3dd8c00640 1 -- 172.21.15.97:0/4086878363 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7f3dc805d9d0 con 0x7f3ddc0ce9d0 2024-09-10T21:27:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:27:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc072610 msgr2=0x7f3dbc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc072610 0x7f3dbc074ad0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc005c50 tx=0x7f3dcc00a3b0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc0ce9d0 msgr2=0x7f3ddc15ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc0ce9d0 0x7f3ddc15ed40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c530 tx=0x7f3dc800c9f0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 shutdown_connections 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc072610 0x7f3dbc074ad0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ddc15fde0 0x7f3ddc164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ddc0ce9d0 0x7f3ddc15ed40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 --2- 172.21.15.97:0/4086878363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ddc0ce5f0 0x7f3ddc15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 >> 172.21.15.97:0/4086878363 conn(0x7f3ddc0b1780 msgr2=0x7f3ddc13df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 shutdown_connections 2024-09-10T21:27:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:07.942+0000 7f3de3c00640 1 -- 172.21.15.97:0/4086878363 wait complete. 2024-09-10T21:27:07.964 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:27:07.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:07.964 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:07.993 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:27:07.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:07.993 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:27:08.044 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 -- 172.21.15.97:0/112671434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180b2bb0 msgr2=0x7f87180b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 --2- 172.21.15.97:0/112671434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180b2bb0 0x7f87180b2f90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f87180cf190 tx=0x7f870801c830 comp rx=0 tx=0).stop 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 -- 172.21.15.97:0/112671434 shutdown_connections 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 --2- 172.21.15.97:0/112671434 >> v2:172.21.15.194:3300/0 conn(0x7f87180b1e00 0x7f871806dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 --2- 172.21.15.97:0/112671434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180b2bb0 0x7f87180b2f90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 -- 172.21.15.97:0/112671434 >> 172.21.15.97:0/112671434 conn(0x7f87180b37c0 msgr2=0x7f87180b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 -- 172.21.15.97:0/112671434 shutdown_connections 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.226+0000 7f871e400640 1 -- 172.21.15.97:0/112671434 wait complete. 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 Processor -- start 2024-09-10T21:27:08.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 -- start start 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180b2bb0 0x7f8718153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87180b42e0 0x7f87180b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8718153360 0x7f87180b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87180d1420 con 0x7f8718153360 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87180d12a0 con 0x7f87180b42e0 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f87180d15a0 con 0x7f87180b2bb0 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8718153360 0x7f87180b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8718153360 0x7f87180b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45610/0 (socket says 172.21.15.97:45610) 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 -- 172.21.15.97:0/1360845785 learned_addr learned my addr 172.21.15.97:0/1360845785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f870fe00640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87180b42e0 0x7f87180b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180b2bb0 0x7f8718153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 -- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180b2bb0 msgr2=0x7f8718153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180b2bb0 0x7f8718153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 -- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87180b42e0 msgr2=0x7f87180b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87180b42e0 0x7f87180b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 -- 172.21.15.97:0/1360845785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8718168760 con 0x7f8718153360 2024-09-10T21:27:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f8717e00640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8718153360 0x7f87180b69e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f871000b9a0 tx=0x7f871000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f870f400640 1 -- 172.21.15.97:0/1360845785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710019070 con 0x7f8718153360 2024-09-10T21:27:08.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f870f400640 1 -- 172.21.15.97:0/1360845785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f871000cc90 con 0x7f8718153360 2024-09-10T21:27:08.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f870f400640 1 -- 172.21.15.97:0/1360845785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100146a0 con 0x7f8718153360 2024-09-10T21:27:08.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8718156a70 con 0x7f8718153360 2024-09-10T21:27:08.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.230+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8718157030 con 0x7f8718153360 2024-09-10T21:27:08.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f870f400640 1 -- 172.21.15.97:0/1360845785 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f871001e440 con 0x7f8718153360 2024-09-10T21:27:08.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f870f400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f870f400640 1 -- 172.21.15.97:0/1360845785 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8710093340 con 0x7f8718153360 2024-09-10T21:27:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 Processor -- start 2024-09-10T21:27:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 -- start start 2024-09-10T21:27:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87080af000 tx=0x7f870801b070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd1283680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd1283fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560bd116bcb0 con 0x560bd1283bc0 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560bd1081310 con 0x560bd12cd1a0 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560bd10deb90 con 0x560bd12cdac0 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd1283fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd1283fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45616/0 (socket says 172.21.15.97:45616) 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 -- 172.21.15.97:0/3677246821 learned_addr learned my addr 172.21.15.97:0/3677246821 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f870fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd1283680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 -- 172.21.15.97:0/3677246821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 msgr2=0x560bd1283680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 --2- 172.21.15.97:0/3677246821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd1283680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 -- 172.21.15.97:0/3677246821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 msgr2=0x560bd12cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 --2- 172.21.15.97:0/3677246821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 -- 172.21.15.97:0/3677246821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd1082c60 con 0x560bd1283bc0 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.234+0000 7f8717e00640 1 --2- 172.21.15.97:0/3677246821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd1283fa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f871005e1c0 tx=0x7f871005dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1182df9d0a3d49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8707e00640 1 -- 172.21.15.97:0/3677246821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871005e230 con 0x560bd1283bc0 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8707e00640 1 -- 172.21.15.97:0/3677246821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8710062c60 con 0x560bd1283bc0 2024-09-10T21:27:08.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8707e00640 1 -- 172.21.15.97:0/3677246821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710067680 con 0x560bd1283bc0 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- 172.21.15.97:0/3677246821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 msgr2=0x560bd1283fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 --2- 172.21.15.97:0/3677246821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd1283fa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f871005e1c0 tx=0x7f871005dc40 comp rx=0 tx=0).stop 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- 172.21.15.97:0/3677246821 shutdown_connections 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 --2- 172.21.15.97:0/3677246821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd1283fa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 --2- 172.21.15.97:0/3677246821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd1283680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 --2- 172.21.15.97:0/3677246821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- 172.21.15.97:0/3677246821 >> 172.21.15.97:0/3677246821 conn(0x560bd1289d20 msgr2=0x560bd1281f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- 172.21.15.97:0/3677246821 shutdown_connections 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- 172.21.15.97:0/3677246821 wait complete. 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- start start 2024-09-10T21:27:08.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 Processor -- start 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12d4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560bd10d1dd0 con 0x560bd1283bc0 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560bd10cdfc0 con 0x560bd12cd1a0 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560bd10734b0 con 0x560bd12cdac0 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 10 client.0 _ll_register_callbacks cb 0x560bd10517d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45632/0 (socket says 172.21.15.97:45632) 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 -- 172.21.15.97:0/3614492919 learned_addr learned my addr 172.21.15.97:0/3614492919 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f870fe00640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717e00640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12d4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 -- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 msgr2=0x560bd12d4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12d4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 -- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 msgr2=0x560bd12d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12d4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 -- 172.21.15.97:0/3614492919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd106c1f0 con 0x560bd1283bc0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f870fe00640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8717400640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d4530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f870801d0d0 tx=0x7f870801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2da1a07ce7942e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8704c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8704c00640 1 -- 172.21.15.97:0/3614492919 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87080b2040 con 0x560bd1283bc0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8704c00640 1 -- 172.21.15.97:0/3614492919 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f87080c7c70 con 0x560bd1283bc0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bd1066330 con 0x560bd1283bc0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f8704c00640 1 -- 172.21.15.97:0/3614492919 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8708005610 con 0x560bd1283bc0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 10 client.6114 fetch_fsmap learned FSMap version 48 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.238+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x560bd1052fa0 con 0x560bd1283bc0 2024-09-10T21:27:08.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f8704c00640 1 -- 172.21.15.97:0/3614492919 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560bd1066330 con 0x560bd1283bc0 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f8704c00640 1 -- 172.21.15.97:0/3614492919 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7f87080b5030 con 0x560bd1283bc0 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 10 client.6114 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 10 client.6114 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 20 client.6114 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd12dc900 0x560bd12decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 4 client.6114 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 --> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd10e0ae0 con 0x560bd12dc900 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f870fe00640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd12dc900 0x560bd12decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f870fe00640 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd12dc900 0x560bd12decf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b25d15cd339cca server_cookie=34f6c93d98515de7 in_seq=0 out_seq=0 2024-09-10T21:27:08.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f8704c00640 10 client.6114 ms_handle_connect on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:08.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f8704c00640 1 -- 172.21.15.97:0/3614492919 <== mds.0 v2:172.21.15.97:6834/2609945029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560bd10e0ae0 con 0x560bd12dc900 2024-09-10T21:27:08.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f8704c00640 10 client.6114 handle_command_reply: tid=0 2024-09-10T21:27:08.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 client.6114 shutdown 2024-09-10T21:27:08.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 msgr2=0x560bd12d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d4530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f870801d0d0 tx=0x7f870801c830 comp rx=0 tx=0).stop 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 shutdown_connections 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd12dc900 0x560bd12decf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f8704c00640 0 client.6114 ms_handle_reset on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 --2- 172.21.15.97:0/3614492919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d4530 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 >> 172.21.15.97:0/3614492919 conn(0x560bd1289d20 msgr2=0x560bd1281f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 shutdown_connections 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 1 -- 172.21.15.97:0/3614492919 wait complete. 2024-09-10T21:27:08.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.242+0000 7f87218c2000 20 client.6114 trim_cache size 0 max 16384 2024-09-10T21:27:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 Processor -- start 2024-09-10T21:27:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 -- start start 2024-09-10T21:27:08.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd1283bc0 0x560bd1283070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cd1a0 0x560bd12835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd12cdac0 0x560bd12d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87182044b0 con 0x560bd12cdac0 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87182034b0 con 0x560bd1283bc0 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f871815ebc0 con 0x560bd12cd1a0 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd12cdac0 0x560bd12d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd12cdac0 0x560bd12d7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45638/0 (socket says 172.21.15.97:45638) 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 -- 172.21.15.97:0/689347820 learned_addr learned my addr 172.21.15.97:0/689347820 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f870fe00640 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cd1a0 0x560bd12835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717400640 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd1283bc0 0x560bd1283070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 -- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cd1a0 msgr2=0x560bd12835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cd1a0 0x560bd12835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 -- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd1283bc0 msgr2=0x560bd1283070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd1283bc0 0x560bd1283070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 -- 172.21.15.97:0/689347820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd112b660 con 0x560bd12cdac0 2024-09-10T21:27:08.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717400640 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd1283bc0 0x560bd1283070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:08.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8717e00640 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd12cdac0 0x560bd12d7310 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8710067150 tx=0x7f87100915a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d4a92c7a0b004e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8707400640 1 -- 172.21.15.97:0/689347820 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710077030 con 0x560bd12cdac0 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8707400640 1 -- 172.21.15.97:0/689347820 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8710062c60 con 0x560bd12cdac0 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.258+0000 7f8707400640 1 -- 172.21.15.97:0/689347820 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871005e230 con 0x560bd12cdac0 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/689347820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd12cdac0 msgr2=0x560bd12d7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd12cdac0 0x560bd12d7310 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8710067150 tx=0x7f87100915a0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/689347820 shutdown_connections 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd12cdac0 0x560bd12d7310 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cd1a0 0x560bd12835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- 172.21.15.97:0/689347820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd1283bc0 0x560bd1283070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/689347820 >> 172.21.15.97:0/689347820 conn(0x560bd1289d20 msgr2=0x560bd1282960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/689347820 shutdown_connections 2024-09-10T21:27:08.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/689347820 wait complete. 2024-09-10T21:27:08.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- start start 2024-09-10T21:27:08.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 Processor -- start 2024-09-10T21:27:08.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12da500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560bd128cd40 con 0x560bd1283bc0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560bd128a100 con 0x560bd12cd1a0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560bd10c8ec0 con 0x560bd12cdac0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 10 client.0 _ll_register_callbacks cb 0x7f87080051e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45650/0 (socket says 172.21.15.97:45650) 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 -- 172.21.15.97:0/1314188746 learned_addr learned my addr 172.21.15.97:0/1314188746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f870fe00640 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12da500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717e00640 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 -- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 msgr2=0x560bd12ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 -- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 msgr2=0x560bd12da500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12da500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 -- 172.21.15.97:0/1314188746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd1098090 con 0x560bd1283bc0 2024-09-10T21:27:08.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8717400640 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d9fc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f870801cd10 tx=0x7f8708004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7504dd85517274 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8704c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8704c00640 1 -- 172.21.15.97:0/1314188746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87080d3070 con 0x560bd1283bc0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8704c00640 1 -- 172.21.15.97:0/1314188746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8708004cb0 con 0x560bd1283bc0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8710062c60 con 0x560bd1283bc0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8704c00640 1 -- 172.21.15.97:0/1314188746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f87080dd050 con 0x560bd1283bc0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 10 client.6120 fetch_fsmap learned FSMap version 48 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x560bd10645f0 con 0x560bd1283bc0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8704c00640 1 -- 172.21.15.97:0/1314188746 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8710062c60 con 0x560bd1283bc0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f8704c00640 1 -- 172.21.15.97:0/1314188746 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7f87080e0020 con 0x560bd1283bc0 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 10 client.6120 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 10 client.6120 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 20 client.6120 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:08.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd1342640 0x560bd1342a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 4 client.6120 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:27:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 --> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bd10dc590 con 0x560bd1342640 2024-09-10T21:27:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.262+0000 7f870fe00640 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd1342640 0x560bd1342a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.266+0000 7f8704c00640 10 client.6120 ms_handle_connect on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.266+0000 7f870fe00640 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd1342640 0x560bd1342a50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad92db9652e87e8 server_cookie=e8192b3b7869b1e9 in_seq=0 out_seq=0 2024-09-10T21:27:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f8704c00640 1 -- 172.21.15.97:0/1314188746 <== mds.0 v2:172.21.15.97:6834/2609945029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bd10dc590 con 0x560bd1342640 2024-09-10T21:27:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f8704c00640 10 client.6120 handle_command_reply: tid=0 2024-09-10T21:27:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 client.6120 shutdown 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 msgr2=0x560bd12d9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d9fc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f870801cd10 tx=0x7f8708004830 comp rx=0 tx=0).stop 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 shutdown_connections 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:6834/2609945029,v1:172.21.15.97:6835/2609945029] conn(0x560bd1342640 0x560bd1342a50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560bd12cdac0 0x560bd12ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560bd12cd1a0 0x560bd12da500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 --2- 172.21.15.97:0/1314188746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560bd1283bc0 0x560bd12d9fc0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f8704c00640 0 client.6120 ms_handle_reset on v2:172.21.15.97:6834/2609945029 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 >> 172.21.15.97:0/1314188746 conn(0x560bd1289d20 msgr2=0x560bd1283180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 shutdown_connections 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 1 -- 172.21.15.97:0/1314188746 wait complete. 2024-09-10T21:27:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f87218c2000 20 client.6120 trim_cache size 0 max 16384 2024-09-10T21:27:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:27:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "message": "", 2024-09-10T21:27:08.282 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-09-10T21:27:08.282 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 msgr2=0x7f86f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87080af000 tx=0x7f870801b070 comp rx=0 tx=0).stop 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8718153360 msgr2=0x7f87180b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8718153360 0x7f87180b69e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f871000b9a0 tx=0x7f871000be60 comp rx=0 tx=0).stop 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 shutdown_connections 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8718153360 0x7f87180b69e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87180b42e0 0x7f87180b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 --2- 172.21.15.97:0/1360845785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180b2bb0 0x7f8718153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 >> 172.21.15.97:0/1360845785 conn(0x7f87180b37c0 msgr2=0x7f871806dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 shutdown_connections 2024-09-10T21:27:08.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.278+0000 7f871e400640 1 -- 172.21.15.97:0/1360845785 wait complete. 2024-09-10T21:27:08.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:27:08.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.442+0000 7f02f5c00640 1 -- 172.21.15.97:0/499429947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02d4004810 msgr2=0x7f02d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.442+0000 7f02f5c00640 1 --2- 172.21.15.97:0/499429947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02d4004810 0x7f02d4004c90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f02f0073b10 tx=0x7f02dc01d2a0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.442+0000 7f02f5200640 1 -- 172.21.15.97:0/499429947 reap_dead start 2024-09-10T21:27:08.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.442+0000 7f02f5c00640 1 -- 172.21.15.97:0/499429947 shutdown_connections 2024-09-10T21:27:08.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.442+0000 7f02f5c00640 1 -- 172.21.15.97:0/499429947 >> 172.21.15.97:0/499429947 conn(0x7f02f00b1dc0 msgr2=0x7f02f00b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.442+0000 7f02f5c00640 1 -- 172.21.15.97:0/499429947 shutdown_connections 2024-09-10T21:27:08.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.442+0000 7f02f5c00640 1 -- 172.21.15.97:0/499429947 wait complete. 2024-09-10T21:27:08.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 Processor -- start 2024-09-10T21:27:08.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 -- start start 2024-09-10T21:27:08.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4004810 0x7f02f01571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f02f0166450 0x7f02f00cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f00cba30 0x7f02f0158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0167710 con 0x7f02f00cba30 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00b19e0 con 0x7f02d4004810 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f02f0167890 con 0x7f02f0166450 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f00cba30 0x7f02f0158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f00cba30 0x7f02f0158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45668/0 (socket says 172.21.15.97:45668) 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02ef400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f02f0166450 0x7f02f00cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 -- 172.21.15.97:0/3136894113 learned_addr learned my addr 172.21.15.97:0/3136894113 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02efe00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4004810 0x7f02f01571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 -- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f02f0166450 msgr2=0x7f02f00cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f02f0166450 0x7f02f00cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 -- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4004810 msgr2=0x7f02f01571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4004810 0x7f02f01571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 -- 172.21.15.97:0/3136894113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f0161600 con 0x7f02f00cba30 2024-09-10T21:27:08.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5200640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f00cba30 0x7f02f0158300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f02e8006fd0 tx=0x7f02e8006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02e7e00640 1 -- 172.21.15.97:0/3136894113 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8015070 con 0x7f02f00cba30 2024-09-10T21:27:08.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02e7e00640 1 -- 172.21.15.97:0/3136894113 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02e8011900 con 0x7f02f00cba30 2024-09-10T21:27:08.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f01618e0 con 0x7f02f00cba30 2024-09-10T21:27:08.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02e7e00640 1 -- 172.21.15.97:0/3136894113 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8011d50 con 0x7f02f00cba30 2024-09-10T21:27:08.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.446+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f0161ea0 con 0x7f02f00cba30 2024-09-10T21:27:08.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.450+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f00ad0a0 con 0x7f02f00cba30 2024-09-10T21:27:08.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.450+0000 7f02e7e00640 1 -- 172.21.15.97:0/3136894113 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f02e80103c0 con 0x7f02f00cba30 2024-09-10T21:27:08.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.450+0000 7f02e7e00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02d00725e0 0x7f02d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.450+0000 7f02e7e00640 1 -- 172.21.15.97:0/3136894113 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f02e80943d0 con 0x7f02f00cba30 2024-09-10T21:27:08.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.450+0000 7f02efe00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02d00725e0 0x7f02d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.450+0000 7f02efe00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02d00725e0 0x7f02d0074aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f02e0001250 tx=0x7f02e0001df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.454+0000 7f02e7e00640 1 -- 172.21.15.97:0/3136894113 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02e805f2c0 con 0x7f02f00cba30 2024-09-10T21:27:08.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.654+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f02f00b1ba0 con 0x7f02f00cba30 2024-09-10T21:27:08.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02e7e00640 1 -- 172.21.15.97:0/3136894113 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7f02e8063cf0 con 0x7f02f00cba30 2024-09-10T21:27:08.660 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:27:08.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02d00725e0 msgr2=0x7f02d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02d00725e0 0x7f02d0074aa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f02e0001250 tx=0x7f02e0001df0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f00cba30 msgr2=0x7f02f0158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f00cba30 0x7f02f0158300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f02e8006fd0 tx=0x7f02e8006430 comp rx=0 tx=0).stop 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 shutdown_connections 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02d00725e0 0x7f02d0074aa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f00cba30 0x7f02f0158300 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f02f0166450 0x7f02f00cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 --2- 172.21.15.97:0/3136894113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4004810 0x7f02f01571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 >> 172.21.15.97:0/3136894113 conn(0x7f02f00b1dc0 msgr2=0x7f02f015adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 shutdown_connections 2024-09-10T21:27:08.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08.658+0000 7f02f5c00640 1 -- 172.21.15.97:0/3136894113 wait complete. 2024-09-10T21:27:08.681 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:27:08.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:27:08.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:27:08.681 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:27:08.681 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:08.681 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:27:08.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:27:08.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:27:08.682 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:27:08.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:08.682 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:27:08.691 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:27:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:27:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:27:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.691 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:27:08.692 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42265sec preferred_lft 42265sec 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:27:08.693 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:27:08.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:08.694 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:27:08.753 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:27:08.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:08.755 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:27:08.811 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:27:08.811 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:27:08.811 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:27:08.811 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:27:08.811 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:27:08.811 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:27:08.812 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:27:08.812 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:27:08.812 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:27:08.812 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:27:08.812 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:27:08.812 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:27:08.812 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:27:08.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:27:08.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:27:08.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:27:08.813 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:27:08.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:27:08.863 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:27:08.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:27:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:27:08.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:08.953 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:27:09.004 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:27:09.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:09.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:27:09.067 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:09.115 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:27:09.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:27:09.115 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:09.162 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:27:09.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:09.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:09.250 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:09.251 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:27:09.251 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:27:09.266 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:09.266 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:27:09.266 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:27:09.266 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:27:09.266 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:27:09.277 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:27:09.282 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:27:09.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:09.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:09.301 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:27:09.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:27:09.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:27:09.402 INFO:teuthology.orchestra.run.smithi097.stdout:client5827 2024-09-10T21:27:09.402 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:27:09.402 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:27:09.411 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 711 a/b/d; sync .') 2024-09-10T21:27:09.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:27:09.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:27:09.509 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:27:09.510 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:27:09.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- 172.21.15.97:0/2697973857 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f670815fc10 msgr2=0x7f670813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 --2- 172.21.15.97:0/2697973857 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f670815fc10 0x7f670813f9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6700009c80 tx=0x7f670001cda0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- 172.21.15.97:0/2697973857 shutdown_connections 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 --2- 172.21.15.97:0/2697973857 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f670815fc10 0x7f670813f9b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 --2- 172.21.15.97:0/2697973857 >> v2:172.21.15.194:3300/0 conn(0x7f6708141d20 0x7f670815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- 172.21.15.97:0/2697973857 >> 172.21.15.97:0/2697973857 conn(0x7f67080b1780 msgr2=0x7f67080b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- 172.21.15.97:0/2697973857 shutdown_connections 2024-09-10T21:27:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- 172.21.15.97:0/2697973857 wait complete. 2024-09-10T21:27:09.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 Processor -- start 2024-09-10T21:27:09.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- start start 2024-09-10T21:27:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67080ce5f0 0x7f670813cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6708141d20 0x7f67081faf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f670815fc10 0x7f67081fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6708168bf0 con 0x7f6708141d20 2024-09-10T21:27:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6708168a70 con 0x7f670815fc10 2024-09-10T21:27:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670f800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6708168d70 con 0x7f67080ce5f0 2024-09-10T21:27:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f6707e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6708141d20 0x7f67081faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f670ce00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67080ce5f0 0x7f670813cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.698+0000 7f6707e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6708141d20 0x7f67081faf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45682/0 (socket says 172.21.15.97:45682) 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6707e00640 1 -- 172.21.15.97:0/4206124982 learned_addr learned my addr 172.21.15.97:0/4206124982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f670d800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f670815fc10 0x7f67081fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6707e00640 1 -- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67080ce5f0 msgr2=0x7f670813cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6707e00640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67080ce5f0 0x7f670813cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6707e00640 1 -- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f670815fc10 msgr2=0x7f67081fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6707e00640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f670815fc10 0x7f67081fb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6707e00640 1 -- 172.21.15.97:0/4206124982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6708155260 con 0x7f6708141d20 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f670d800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f670815fc10 0x7f67081fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6707e00640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6708141d20 0x7f67081faf00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f66f400ea10 tx=0x7f66f400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6704c00640 1 -- 172.21.15.97:0/4206124982 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f400cde0 con 0x7f6708141d20 2024-09-10T21:27:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6704c00640 1 -- 172.21.15.97:0/4206124982 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66f4004020 con 0x7f6708141d20 2024-09-10T21:27:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6704c00640 1 -- 172.21.15.97:0/4206124982 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4004470 con 0x7f6708141d20 2024-09-10T21:27:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670815df00 con 0x7f6708141d20 2024-09-10T21:27:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670815dbd0 con 0x7f6708141d20 2024-09-10T21:27:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670815f6d0 con 0x7f6708141d20 2024-09-10T21:27:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.702+0000 7f6704c00640 1 -- 172.21.15.97:0/4206124982 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f66f4003600 con 0x7f6708141d20 2024-09-10T21:27:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.706+0000 7f6704c00640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66e80725e0 0x7f66e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.706+0000 7f6704c00640 1 -- 172.21.15.97:0/4206124982 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f66f4093c70 con 0x7f6708141d20 2024-09-10T21:27:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.706+0000 7f670ce00640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66e80725e0 0x7f66e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.706+0000 7f670ce00640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66e80725e0 0x7f66e8074aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6708145730 tx=0x7f66f8007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.706+0000 7f6704c00640 1 -- 172.21.15.97:0/4206124982 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66f405eb60 con 0x7f6708141d20 2024-09-10T21:27:09.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.886+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f670815e0d0 con 0x7f6708141d20 2024-09-10T21:27:09.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.886+0000 7f6704c00640 1 -- 172.21.15.97:0/4206124982 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7f66f4063590 con 0x7f6708141d20 2024-09-10T21:27:09.891 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66e80725e0 msgr2=0x7f66e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66e80725e0 0x7f66e8074aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6708145730 tx=0x7f66f8007e40 comp rx=0 tx=0).stop 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6708141d20 msgr2=0x7f67081faf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6708141d20 0x7f67081faf00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f66f400ea10 tx=0x7f66f400eed0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 shutdown_connections 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66e80725e0 0x7f66e8074aa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f670815fc10 0x7f67081fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6708141d20 0x7f67081faf00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 --2- 172.21.15.97:0/4206124982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67080ce5f0 0x7f670813cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 >> 172.21.15.97:0/4206124982 conn(0x7f67080b1780 msgr2=0x7f670806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 shutdown_connections 2024-09-10T21:27:09.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:09.890+0000 7f670f800640 1 -- 172.21.15.97:0/4206124982 wait complete. 2024-09-10T21:27:09.911 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:27:09.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:09.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:27:09.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:27:09.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:27:09.937 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:27:09.988 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 -- 172.21.15.97:0/3874486378 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c15b6b0 msgr2=0x7f6a0c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 --2- 172.21.15.97:0/3874486378 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c15b6b0 0x7f6a0c15ba90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009bc0 tx=0x7f69fc01cae0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 -- 172.21.15.97:0/3874486378 shutdown_connections 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 --2- 172.21.15.97:0/3874486378 >> v2:172.21.15.194:3300/0 conn(0x7f6a0c073e30 0x7f6a0c06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 --2- 172.21.15.97:0/3874486378 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c15b6b0 0x7f6a0c15ba90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 -- 172.21.15.97:0/3874486378 >> 172.21.15.97:0/3874486378 conn(0x7f6a0c0b2790 msgr2=0x7f6a0c0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 -- 172.21.15.97:0/3874486378 shutdown_connections 2024-09-10T21:27:10.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.170+0000 7f6a12e00640 1 -- 172.21.15.97:0/3874486378 wait complete. 2024-09-10T21:27:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 Processor -- start 2024-09-10T21:27:10.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 -- start start 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c073e30 0x7f6a0c0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c0b3a90 0x7f6a0c1f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a0c15b6b0 0x7f6a0c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c167ba0 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c167a20 con 0x7f6a0c15b6b0 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c167d20 con 0x7f6a0c073e30 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c0b3a90 0x7f6a0c1f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c0b3a90 0x7f6a0c1f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45684/0 (socket says 172.21.15.97:45684) 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 -- 172.21.15.97:0/881505075 learned_addr learned my addr 172.21.15.97:0/881505075 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a0be00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c073e30 0x7f6a0c0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a10e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a0c15b6b0 0x7f6a0c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 -- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c073e30 msgr2=0x7f6a0c0c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c073e30 0x7f6a0c0c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 -- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a0c15b6b0 msgr2=0x7f6a0c14cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a0c15b6b0 0x7f6a0c14cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 -- 172.21.15.97:0/881505075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c0c67b0 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a0be00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c073e30 0x7f6a0c0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a10e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a0c15b6b0 0x7f6a0c14cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a03400640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c0b3a90 0x7f6a0c1f8b20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f69f400c480 tx=0x7f69f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a08c00640 1 -- 172.21.15.97:0/881505075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4016020 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a08c00640 1 -- 172.21.15.97:0/881505075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f4012940 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c158300 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a08c00640 1 -- 172.21.15.97:0/881505075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4012d90 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1588c0 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.174+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c158ac0 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.178+0000 7f6a08c00640 1 -- 172.21.15.97:0/881505075 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f69f401b400 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.178+0000 7f6a08c00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69e80725e0 0x7f69e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.178+0000 7f6a08c00640 1 -- 172.21.15.97:0/881505075 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f69f4093730 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.178+0000 7f6a0be00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69e80725e0 0x7f69e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.178+0000 7f6a0be00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69e80725e0 0x7f69e8074aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0af000 tx=0x7f69fc01b320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.182+0000 7f6a08c00640 1 -- 172.21.15.97:0/881505075 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6a0c158ac0 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.378+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6a0c06fe90 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.383 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:27:10.382+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:27:10.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.382+0000 7f6a08c00640 1 -- 172.21.15.97:0/881505075 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7f69f4020020 con 0x7f6a0c0b3a90 2024-09-10T21:27:10.386 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:27:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69e80725e0 msgr2=0x7f69e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69e80725e0 0x7f69e8074aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0af000 tx=0x7f69fc01b320 comp rx=0 tx=0).stop 2024-09-10T21:27:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c0b3a90 msgr2=0x7f6a0c1f8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c0b3a90 0x7f6a0c1f8b20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f69f400c480 tx=0x7f69f400c940 comp rx=0 tx=0).stop 2024-09-10T21:27:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 shutdown_connections 2024-09-10T21:27:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f69e80725e0 0x7f69e8074aa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6a0c15b6b0 0x7f6a0c14cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c0b3a90 0x7f6a0c1f8b20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 --2- 172.21.15.97:0/881505075 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6a0c073e30 0x7f6a0c0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 >> 172.21.15.97:0/881505075 conn(0x7f6a0c0b2790 msgr2=0x7f6a0c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 shutdown_connections 2024-09-10T21:27:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.386+0000 7f6a12e00640 1 -- 172.21.15.97:0/881505075 wait complete. 2024-09-10T21:27:10.398 INFO:tasks.ceph.mds.a.smithi097.stderr: -5344> 2024-09-10T21:27:05.366+0000 7f0cdd000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:27:10.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2024-09-10T21:27:10.448 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:27:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.554+0000 7fc75fe00640 1 -- 172.21.15.97:0/3877581920 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 msgr2=0x7fc758166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.554+0000 7fc75fe00640 1 --2- 172.21.15.97:0/3877581920 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 0x7fc758166850 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc750009bf0 tx=0x7fc75001cbe0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- 172.21.15.97:0/3877581920 shutdown_connections 2024-09-10T21:27:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 --2- 172.21.15.97:0/3877581920 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 0x7fc758166850 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- 172.21.15.97:0/3877581920 >> 172.21.15.97:0/3877581920 conn(0x7fc7580b1780 msgr2=0x7fc7580b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- 172.21.15.97:0/3877581920 shutdown_connections 2024-09-10T21:27:10.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- 172.21.15.97:0/3877581920 wait complete. 2024-09-10T21:27:10.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 Processor -- start 2024-09-10T21:27:10.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- start start 2024-09-10T21:27:10.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580ce5f0 0x7fc7581534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc758141d20 0x7fc75815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 0x7fc7580b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc75816a180 con 0x7fc758141d20 2024-09-10T21:27:10.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc75816a000 con 0x7fc7580ce5f0 2024-09-10T21:27:10.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc75816a300 con 0x7fc75815fde0 2024-09-10T21:27:10.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc758141d20 0x7fc75815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc758141d20 0x7fc75815b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45702/0 (socket says 172.21.15.97:45702) 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 -- 172.21.15.97:0/1476560399 learned_addr learned my addr 172.21.15.97:0/1476560399 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75d400640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580ce5f0 0x7fc7581534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75de00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 0x7fc7580b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 -- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 msgr2=0x7fc7580b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 0x7fc7580b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 -- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580ce5f0 msgr2=0x7fc7581534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580ce5f0 0x7fc7581534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 -- 172.21.15.97:0/1476560399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc758144d60 con 0x7fc758141d20 2024-09-10T21:27:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc75ca00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc758141d20 0x7fc75815b690 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc74400c490 tx=0x7fc74400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:10.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.558+0000 7fc755600640 1 -- 172.21.15.97:0/1476560399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744016020 con 0x7fc758141d20 2024-09-10T21:27:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc755600640 1 -- 172.21.15.97:0/1476560399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc744012940 con 0x7fc758141d20 2024-09-10T21:27:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7580b4880 con 0x7fc758141d20 2024-09-10T21:27:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc755600640 1 -- 172.21.15.97:0/1476560399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744012d90 con 0x7fc758141d20 2024-09-10T21:27:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc758145000 con 0x7fc758141d20 2024-09-10T21:27:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7580ce9d0 con 0x7fc758141d20 2024-09-10T21:27:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc755600640 1 -- 172.21.15.97:0/1476560399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc74401b400 con 0x7fc758141d20 2024-09-10T21:27:10.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc755600640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7380725e0 0x7fc738074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc755600640 1 -- 172.21.15.97:0/1476560399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fc744093330 con 0x7fc758141d20 2024-09-10T21:27:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.562+0000 7fc75d400640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7380725e0 0x7fc738074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:10.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.566+0000 7fc75d400640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7380725e0 0x7fc738074aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc74c001820 tx=0x7fc74c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:10.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.566+0000 7fc755600640 1 -- 172.21.15.97:0/1476560399 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc744020030 con 0x7fc758141d20 2024-09-10T21:27:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.734+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) -- 0x7fc758141360 con 0x7fc758141d20 2024-09-10T21:27:10.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc755600640 1 -- 172.21.15.97:0/1476560399 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v36)=0 v36) ==== 139+0+0 (secure 0 0 0) 0x7fc74405e1d0 con 0x7fc758141d20 2024-09-10T21:27:10.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7380725e0 msgr2=0x7fc738074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7380725e0 0x7fc738074aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc74c001820 tx=0x7fc74c009040 comp rx=0 tx=0).stop 2024-09-10T21:27:10.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc758141d20 msgr2=0x7fc75815b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc758141d20 0x7fc75815b690 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc74400c490 tx=0x7fc74400c950 comp rx=0 tx=0).stop 2024-09-10T21:27:10.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 shutdown_connections 2024-09-10T21:27:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7380725e0 0x7fc738074aa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75815fde0 0x7fc7580b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc758141d20 0x7fc75815b690 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 --2- 172.21.15.97:0/1476560399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580ce5f0 0x7fc7581534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 >> 172.21.15.97:0/1476560399 conn(0x7fc7580b1780 msgr2=0x7fc75813e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 shutdown_connections 2024-09-10T21:27:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:10.738+0000 7fc75fe00640 1 -- 172.21.15.97:0/1476560399 wait complete. 2024-09-10T21:27:15.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:27:15.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- 172.21.15.97:0/3321710747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 msgr2=0x7fb3f4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:15.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 --2- 172.21.15.97:0/3321710747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 0x7fb3f4004c90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8001740 tx=0x7fb3f801c820 comp rx=0 tx=0).stop 2024-09-10T21:27:15.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb40e800640 1 -- 172.21.15.97:0/3321710747 reap_dead start 2024-09-10T21:27:15.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- 172.21.15.97:0/3321710747 shutdown_connections 2024-09-10T21:27:15.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- 172.21.15.97:0/3321710747 >> 172.21.15.97:0/3321710747 conn(0x7fb4100b3c70 msgr2=0x7fb4100b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:15.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- 172.21.15.97:0/3321710747 shutdown_connections 2024-09-10T21:27:15.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- 172.21.15.97:0/3321710747 wait complete. 2024-09-10T21:27:15.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 Processor -- start 2024-09-10T21:27:15.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- start start 2024-09-10T21:27:15.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 0x7fb410157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:15.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb410155840 0x7fb410159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:15.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4101598b0 0x7fb410150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:15.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4100b61c0 con 0x7fb3f4004810 2024-09-10T21:27:15.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4100b6040 con 0x7fb410155840 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb415200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb41015b720 con 0x7fb4101598b0 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb40de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 0x7fb410157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb40de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 0x7fb410157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45718/0 (socket says 172.21.15.97:45718) 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb40de00640 1 -- 172.21.15.97:0/1962759169 learned_addr learned my addr 172.21.15.97:0/1962759169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb40e800640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4101598b0 0x7fb410150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.918+0000 7fb405400640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb410155840 0x7fb410159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb40de00640 1 -- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4101598b0 msgr2=0x7fb410150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb40de00640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4101598b0 0x7fb410150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb40de00640 1 -- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb410155840 msgr2=0x7fb410159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb40de00640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb410155840 0x7fb410159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:15.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb40de00640 1 -- 172.21.15.97:0/1962759169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101579e0 con 0x7fb3f4004810 2024-09-10T21:27:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb405400640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb410155840 0x7fb410159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb40de00640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 0x7fb410157100 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb400012690 tx=0x7fb400012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb406a00640 1 -- 172.21.15.97:0/1962759169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400010b40 con 0x7fb3f4004810 2024-09-10T21:27:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb406a00640 1 -- 172.21.15.97:0/1962759169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb400017030 con 0x7fb3f4004810 2024-09-10T21:27:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb410157cc0 con 0x7fb3f4004810 2024-09-10T21:27:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb406a00640 1 -- 172.21.15.97:0/1962759169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400013450 con 0x7fb3f4004810 2024-09-10T21:27:15.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb410158280 con 0x7fb3f4004810 2024-09-10T21:27:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41015bc60 con 0x7fb3f4004810 2024-09-10T21:27:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb406a00640 1 -- 172.21.15.97:0/1962759169 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4000135f0 con 0x7fb3f4004810 2024-09-10T21:27:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb406a00640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3ec0725e0 0x7fb3ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.922+0000 7fb406a00640 1 -- 172.21.15.97:0/1962759169 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fb400097070 con 0x7fb3f4004810 2024-09-10T21:27:15.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.926+0000 7fb405400640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3ec0725e0 0x7fb3ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:15.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.926+0000 7fb405400640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3ec0725e0 0x7fb3ec074aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40057f0 tx=0x7fb3f80195c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:15.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:15.926+0000 7fb406a00640 1 -- 172.21.15.97:0/1962759169 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb400061f10 con 0x7fb3f4004810 2024-09-10T21:27:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.126+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb410074f40 con 0x7fb3f4004810 2024-09-10T21:27:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.562+0000 7fb406a00640 1 -- 172.21.15.97:0/1962759169 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v51) ==== 162+0+0 (secure 0 0 0) 0x7fb400066940 con 0x7fb3f4004810 2024-09-10T21:27:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:27:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3ec0725e0 msgr2=0x7fb3ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3ec0725e0 0x7fb3ec074aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40057f0 tx=0x7fb3f80195c0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 msgr2=0x7fb410157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 0x7fb410157100 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb400012690 tx=0x7fb400012b50 comp rx=0 tx=0).stop 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 shutdown_connections 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3ec0725e0 0x7fb3ec074aa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4101598b0 0x7fb410150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb410155840 0x7fb410159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 --2- 172.21.15.97:0/1962759169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3f4004810 0x7fb410157100 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 >> 172.21.15.97:0/1962759169 conn(0x7fb4100b3c70 msgr2=0x7fb4100ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 shutdown_connections 2024-09-10T21:27:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.566+0000 7fb415200640 1 -- 172.21.15.97:0/1962759169 wait complete. 2024-09-10T21:27:16.593 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 -- 172.21.15.97:0/4014796886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d815b110 msgr2=0x7ff0d80b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 --2- 172.21.15.97:0/4014796886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d815b110 0x7ff0d80b2a80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8072040 tx=0x7ff0d001cac0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 -- 172.21.15.97:0/4014796886 shutdown_connections 2024-09-10T21:27:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 --2- 172.21.15.97:0/4014796886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d815b110 0x7ff0d80b2a80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 --2- 172.21.15.97:0/4014796886 >> v2:172.21.15.194:3300/0 conn(0x7ff0d80cf0b0 0x7ff0d806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 -- 172.21.15.97:0/4014796886 >> 172.21.15.97:0/4014796886 conn(0x7ff0d80b1780 msgr2=0x7ff0d80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 -- 172.21.15.97:0/4014796886 shutdown_connections 2024-09-10T21:27:16.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.730+0000 7ff0dce00640 1 -- 172.21.15.97:0/4014796886 wait complete. 2024-09-10T21:27:16.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 Processor -- start 2024-09-10T21:27:16.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 -- start start 2024-09-10T21:27:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d80cf0b0 0x7ff0d80b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0d80cf490 0x7ff0d813c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d815b110 0x7ff0d81584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d80b3b60 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d80b39e0 con 0x7ff0d815b110 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d8070970 con 0x7ff0d80cf490 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d80cf0b0 0x7ff0d80b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d80cf0b0 0x7ff0d80b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45732/0 (socket says 172.21.15.97:45732) 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 -- 172.21.15.97:0/3277880865 learned_addr learned my addr 172.21.15.97:0/3277880865 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 -- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0d80cf490 msgr2=0x7ff0d813c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0d80cf490 0x7ff0d813c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 -- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d815b110 msgr2=0x7ff0d81584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d815b110 0x7ff0d81584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 -- 172.21.15.97:0/3277880865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d8156cd0 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0d5e00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d80cf0b0 0x7ff0d80b0100 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800ddf0 tx=0x7ff0c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0cea00640 1 -- 172.21.15.97:0/3277880865 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c801b070 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0cea00640 1 -- 172.21.15.97:0/3277880865 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0c800e540 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d8156fb0 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0cea00640 1 -- 172.21.15.97:0/3277880865 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800e990 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.734+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d8157570 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.738+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d806e3b0 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.738+0000 7ff0cea00640 1 -- 172.21.15.97:0/3277880865 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff0c800b040 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.738+0000 7ff0cea00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0b40725e0 0x7ff0b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.738+0000 7ff0cea00640 1 -- 172.21.15.97:0/3277880865 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7ff0c8093840 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.738+0000 7ff0cd400640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0b40725e0 0x7ff0b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.738+0000 7ff0cd400640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0b40725e0 0x7ff0b4074aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0000f70 tx=0x7ff0c00079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.742+0000 7ff0cea00640 1 -- 172.21.15.97:0/3277880865 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0c805e6e0 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0d8150bc0 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0cea00640 1 -- 172.21.15.97:0/3277880865 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5224 (secure 0 0 0) 0x7ff0c8063110 con 0x7ff0d80cf0b0 2024-09-10T21:27:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:16.948 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":51,"btime":"2024-09-10T21:27:16:564118+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:16.564095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T21:27:16.949 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 51 2024-09-10T21:27:16.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0b40725e0 msgr2=0x7ff0b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0b40725e0 0x7ff0b4074aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0000f70 tx=0x7ff0c00079c0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d80cf0b0 msgr2=0x7ff0d80b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d80cf0b0 0x7ff0d80b0100 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800ddf0 tx=0x7ff0c800a620 comp rx=0 tx=0).stop 2024-09-10T21:27:16.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 shutdown_connections 2024-09-10T21:27:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff0b40725e0 0x7ff0b4074aa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d815b110 0x7ff0d81584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff0d80cf490 0x7ff0d813c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 --2- 172.21.15.97:0/3277880865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d80cf0b0 0x7ff0d80b0100 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 >> 172.21.15.97:0/3277880865 conn(0x7ff0d80b1780 msgr2=0x7ff0d814f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 shutdown_connections 2024-09-10T21:27:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:16.946+0000 7ff0dce00640 1 -- 172.21.15.97:0/3277880865 wait complete. 2024-09-10T21:27:17.371 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:59.777372+0000', 'modified': '2024-09-10T21:27:16.564095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6144': {'gid': 6144, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/3391319282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3391319282}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3391319282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:27:17.372 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6144:up:replay 2024-09-10T21:27:18.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:18.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- 172.21.15.97:0/3113148179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 msgr2=0x7f1960003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 --2- 172.21.15.97:0/3113148179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 0x7f1960003e60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f196c001dc0 tx=0x7f196c01cb30 comp rx=0 tx=0).stop 2024-09-10T21:27:18.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f197be00640 1 -- 172.21.15.97:0/3113148179 reap_dead start 2024-09-10T21:27:18.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- 172.21.15.97:0/3113148179 shutdown_connections 2024-09-10T21:27:18.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- 172.21.15.97:0/3113148179 >> 172.21.15.97:0/3113148179 conn(0x7f197c0b1780 msgr2=0x7f197c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- 172.21.15.97:0/3113148179 shutdown_connections 2024-09-10T21:27:18.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- 172.21.15.97:0/3113148179 wait complete. 2024-09-10T21:27:18.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 Processor -- start 2024-09-10T21:27:18.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- start start 2024-09-10T21:27:18.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 0x7f197c1e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f197c159910 0x7f197c13cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f197c13f5c0 0x7f197c13fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f197c06f1e0 con 0x7f1960003a40 2024-09-10T21:27:18.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f197c06f060 con 0x7f197c159910 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f1982600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f197c06f360 con 0x7f197c13f5c0 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f197b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 0x7f197c1e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f197b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 0x7f197c1e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51322/0 (socket says 172.21.15.97:51322) 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f197b400640 1 -- 172.21.15.97:0/1218396345 learned_addr learned my addr 172.21.15.97:0/1218396345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f197aa00640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f197c159910 0x7f197c13cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.526+0000 7f197be00640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f197c13f5c0 0x7f197c13fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f197b400640 1 -- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f197c13f5c0 msgr2=0x7f197c13fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f197b400640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f197c13f5c0 0x7f197c13fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f197b400640 1 -- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f197c159910 msgr2=0x7f197c13cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f197b400640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f197c159910 0x7f197c13cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f197b400640 1 -- 172.21.15.97:0/1218396345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c1e7800 con 0x7f1960003a40 2024-09-10T21:27:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f197b400640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 0x7f197c1e72c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f196c01b230 tx=0x7f196c001300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1973400640 1 -- 172.21.15.97:0/1218396345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c003d90 con 0x7f1960003a40 2024-09-10T21:27:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1973400640 1 -- 172.21.15.97:0/1218396345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f196c0060d0 con 0x7f1960003a40 2024-09-10T21:27:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c159ed0 con 0x7f1960003a40 2024-09-10T21:27:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1973400640 1 -- 172.21.15.97:0/1218396345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c006500 con 0x7f1960003a40 2024-09-10T21:27:18.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c1e4dd0 con 0x7f1960003a40 2024-09-10T21:27:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197c0cf4e0 con 0x7f1960003a40 2024-09-10T21:27:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1973400640 1 -- 172.21.15.97:0/1218396345 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f196c0b6020 con 0x7f1960003a40 2024-09-10T21:27:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1973400640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f195c072610 0x7f195c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.530+0000 7f1973400640 1 -- 172.21.15.97:0/1218396345 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f196c12cfe0 con 0x7f1960003a40 2024-09-10T21:27:18.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.534+0000 7f197aa00640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f195c072610 0x7f195c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.534+0000 7f197aa00640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f195c072610 0x7f195c074ad0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f197c0cbf60 tx=0x7f1968010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.534+0000 7f1973400640 1 -- 172.21.15.97:0/1218396345 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f196c016890 con 0x7f1960003a40 2024-09-10T21:27:18.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.738+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f197c0d0250 con 0x7f1960003a40 2024-09-10T21:27:18.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.738+0000 7f1973400640 1 -- 172.21.15.97:0/1218396345 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5224 (secure 0 0 0) 0x7f196c0f8510 con 0x7f1960003a40 2024-09-10T21:27:18.743 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:18.743 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":53,"btime":"2024-09-10T21:27:18:568015+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:17.570441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T21:27:18.743 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 53 2024-09-10T21:27:18.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f195c072610 msgr2=0x7f195c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f195c072610 0x7f195c074ad0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f197c0cbf60 tx=0x7f1968010040 comp rx=0 tx=0).stop 2024-09-10T21:27:18.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 msgr2=0x7f197c1e72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 0x7f197c1e72c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f196c01b230 tx=0x7f196c001300 comp rx=0 tx=0).stop 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 shutdown_connections 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f195c072610 0x7f195c074ad0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f197c13f5c0 0x7f197c13fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f197c159910 0x7f197c13cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 --2- 172.21.15.97:0/1218396345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1960003a40 0x7f197c1e72c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 >> 172.21.15.97:0/1218396345 conn(0x7f197c0b1780 msgr2=0x7f197c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 shutdown_connections 2024-09-10T21:27:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:18.742+0000 7f1982600640 1 -- 172.21.15.97:0/1218396345 wait complete. 2024-09-10T21:27:19.168 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:59.777372+0000', 'modified': '2024-09-10T21:27:17.570441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6144': {'gid': 6144, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.97:6835/3391319282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3391319282}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3391319282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:27:19.169 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6144:up:rejoin 2024-09-10T21:27:20.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.318+0000 7f5262a00640 1 -- 172.21.15.97:0/3296883160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 msgr2=0x7f5240004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.318+0000 7f5262a00640 1 --2- 172.21.15.97:0/3296883160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f5240004c90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5244001860 tx=0x7f524401eff0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.318+0000 7f5262a00640 1 -- 172.21.15.97:0/3296883160 shutdown_connections 2024-09-10T21:27:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.318+0000 7f5262a00640 1 --2- 172.21.15.97:0/3296883160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f5240004c90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.318+0000 7f5262a00640 1 -- 172.21.15.97:0/3296883160 >> 172.21.15.97:0/3296883160 conn(0x7f525c0b28c0 msgr2=0x7f525c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.318+0000 7f5262a00640 1 -- 172.21.15.97:0/3296883160 shutdown_connections 2024-09-10T21:27:20.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.318+0000 7f5262a00640 1 -- 172.21.15.97:0/3296883160 wait complete. 2024-09-10T21:27:20.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 Processor -- start 2024-09-10T21:27:20.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 -- start start 2024-09-10T21:27:20.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f525c144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f525c153c80 0x7f525c144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f525c145490 0x7f525c15e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f525c0cff40 con 0x7f5240004810 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f525c15c2c0 con 0x7f525c153c80 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f525c0d00c0 con 0x7f525c145490 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f525c144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f525c144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51338/0 (socket says 172.21.15.97:51338) 2024-09-10T21:27:20.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 -- 172.21.15.97:0/2328652491 learned_addr learned my addr 172.21.15.97:0/2328652491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 -- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f525c145490 msgr2=0x7f525c15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f525c145490 0x7f525c15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 -- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f525c153c80 msgr2=0x7f525c144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5253400640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f525c153c80 0x7f525c144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f525c153c80 0x7f525c144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 -- 172.21.15.97:0/2328652491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c15d5c0 con 0x7f5240004810 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5253400640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f525c153c80 0x7f525c144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f525be00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f525c144680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f524c011a90 tx=0x7f524c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5258c00640 1 -- 172.21.15.97:0/2328652491 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c01e070 con 0x7f5240004810 2024-09-10T21:27:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5258c00640 1 -- 172.21.15.97:0/2328652491 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f524c019430 con 0x7f5240004810 2024-09-10T21:27:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5258c00640 1 -- 172.21.15.97:0/2328652491 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c019840 con 0x7f5240004810 2024-09-10T21:27:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c15d860 con 0x7f5240004810 2024-09-10T21:27:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c15de20 con 0x7f5240004810 2024-09-10T21:27:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.322+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525c074380 con 0x7f5240004810 2024-09-10T21:27:20.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.326+0000 7f5258c00640 1 -- 172.21.15.97:0/2328652491 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f524c0199e0 con 0x7f5240004810 2024-09-10T21:27:20.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.326+0000 7f5258c00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52380725e0 0x7f5238074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.326+0000 7f5258c00640 1 -- 172.21.15.97:0/2328652491 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f524c0941e0 con 0x7f5240004810 2024-09-10T21:27:20.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.326+0000 7f5253400640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52380725e0 0x7f5238074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.326+0000 7f5253400640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52380725e0 0x7f5238074aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5244001830 tx=0x7f52440aa040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.330+0000 7f5258c00640 1 -- 172.21.15.97:0/2328652491 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f524c05f6d0 con 0x7f5240004810 2024-09-10T21:27:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f525c09eeb0 con 0x7f5240004810 2024-09-10T21:27:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5258c00640 1 -- 172.21.15.97:0/2328652491 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5231 (secure 0 0 0) 0x7f524c05efd0 con 0x7f5240004810 2024-09-10T21:27:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:20.532 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"btime":"2024-09-10T21:27:19:570290+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:19.570283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6144,"qdb_cluster":[6144]},"id":4}]} 2024-09-10T21:27:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 54 2024-09-10T21:27:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52380725e0 msgr2=0x7f5238074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52380725e0 0x7f5238074aa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5244001830 tx=0x7f52440aa040 comp rx=0 tx=0).stop 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 msgr2=0x7f525c144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f525c144680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f524c011a90 tx=0x7f524c011f50 comp rx=0 tx=0).stop 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 shutdown_connections 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52380725e0 0x7f5238074aa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f525c145490 0x7f525c15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f525c153c80 0x7f525c144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 --2- 172.21.15.97:0/2328652491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5240004810 0x7f525c144680 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 >> 172.21.15.97:0/2328652491 conn(0x7f525c0b28c0 msgr2=0x7f525c0d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 shutdown_connections 2024-09-10T21:27:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:20.530+0000 7f5262a00640 1 -- 172.21.15.97:0/2328652491 wait complete. 2024-09-10T21:27:20.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:59.777372+0000', 'modified': '2024-09-10T21:27:19.570283+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6144}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6144': {'gid': 6144, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.97:6835/3391319282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3391319282}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3391319282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6144, 'qdb_cluster': [6144]} 2024-09-10T21:27:20.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:27:20.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- 172.21.15.97:0/3883546377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb80815bdd0 msgr2=0x7fb808074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 --2- 172.21.15.97:0/3883546377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb80815bdd0 0x7fb808074380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009900 tx=0x7fb7fc01c8b0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- 172.21.15.97:0/3883546377 shutdown_connections 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 --2- 172.21.15.97:0/3883546377 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 0x7fb8080cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 --2- 172.21.15.97:0/3883546377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb80815bdd0 0x7fb808074380 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- 172.21.15.97:0/3883546377 >> 172.21.15.97:0/3883546377 conn(0x7fb8080b28c0 msgr2=0x7fb8080b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- 172.21.15.97:0/3883546377 shutdown_connections 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- 172.21.15.97:0/3883546377 wait complete. 2024-09-10T21:27:21.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 Processor -- start 2024-09-10T21:27:21.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- start start 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 0x7fb80815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8080cfad0 0x7fb80814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb80815bdd0 0x7fb80815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb808168b30 con 0x7fb80815bdd0 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081689b0 con 0x7fb8080cfad0 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.098+0000 7fb810200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb808168cb0 con 0x7fb8080b0110 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb80815bdd0 0x7fb80815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 0x7fb80815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 0x7fb80815c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:33392/0 (socket says 172.21.15.97:33392) 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 -- 172.21.15.97:0/1947677424 learned_addr learned my addr 172.21.15.97:0/1947677424 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb80815bdd0 0x7fb80815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51352/0 (socket says 172.21.15.97:51352) 2024-09-10T21:27:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80ce00640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8080cfad0 0x7fb80814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 -- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8080cfad0 msgr2=0x7fb80814b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8080cfad0 0x7fb80814b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 -- 172.21.15.97:0/1947677424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb80815bdd0 msgr2=0x7fb80815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb80815bdd0 0x7fb80815a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 -- 172.21.15.97:0/1947677424 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081fb8f0 con 0x7fb8080b0110 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80e200640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb80815bdd0 0x7fb80815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb80d800640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 0x7fb80815c460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc01af60 tx=0x7fb7fc0a5690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb805600640 1 -- 172.21.15.97:0/1947677424 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0b2040 con 0x7fb8080b0110 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb805600640 1 -- 172.21.15.97:0/1947677424 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7fc0041b0 con 0x7fb8080b0110 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8081fb390 con 0x7fb8080b0110 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb805600640 1 -- 172.21.15.97:0/1947677424 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc005020 con 0x7fb8080b0110 2024-09-10T21:27:21.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb808070be0 con 0x7fb8080b0110 2024-09-10T21:27:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.102+0000 7fb805600640 1 -- 172.21.15.97:0/1947677424 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb7fc0051c0 con 0x7fb8080b0110 2024-09-10T21:27:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb805600640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7e80725e0 0x7fb7e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 Processor -- start 2024-09-10T21:27:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb805600640 1 -- 172.21.15.97:0/1947677424 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fb7fc12d6f0 con 0x7fb8080b0110 2024-09-10T21:27:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- start start 2024-09-10T21:27:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7e80725e0 0x7fb7e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7e80725e0 0x7fb7e8074aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb8081fa910 tx=0x7fb7f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc750db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc7051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5596cc6253f0 con 0x5596cc7512f0 2024-09-10T21:27:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5596cc562b20 con 0x5596cc7509d0 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5596cc551f50 con 0x5596cc704e10 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc750db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7048d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51356/0 (socket says 172.21.15.97:51356) 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 -- 172.21.15.97:0/160279769 learned_addr learned my addr 172.21.15.97:0/160279769 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80e200640 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc7051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 -- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 msgr2=0x5596cc7051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc7051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 -- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 msgr2=0x5596cc750db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc750db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 -- 172.21.15.97:0/160279769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cc519590 con 0x5596cc7512f0 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80d800640 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc750db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb80ce00640 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7048d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4001360 tx=0x7fb7f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da45408c8815612 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb7fa000640 1 -- 172.21.15.97:0/160279769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4025070 con 0x5596cc7512f0 2024-09-10T21:27:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb7fa000640 1 -- 172.21.15.97:0/160279769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7f401b050 con 0x5596cc7512f0 2024-09-10T21:27:21.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb7fa000640 1 -- 172.21.15.97:0/160279769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4014e00 con 0x5596cc7512f0 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- 172.21.15.97:0/160279769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 msgr2=0x5596cc7048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7048d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4001360 tx=0x7fb7f4012000 comp rx=0 tx=0).stop 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- 172.21.15.97:0/160279769 shutdown_connections 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc7051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7048d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 --2- 172.21.15.97:0/160279769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc750db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- 172.21.15.97:0/160279769 >> 172.21.15.97:0/160279769 conn(0x5596cc5ee130 msgr2=0x5596cc733e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- 172.21.15.97:0/160279769 shutdown_connections 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.106+0000 7fb8135cd000 1 -- 172.21.15.97:0/160279769 wait complete. 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 -- start start 2024-09-10T21:27:21.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 Processor -- start 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc706240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc706780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc706cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5596cc4ec5d0 con 0x5596cc7512f0 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5596cc555d60 con 0x5596cc704e10 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5596cc4f8500 con 0x5596cc7509d0 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 10 client.0 _ll_register_callbacks cb 0x5596cc4fc810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc706cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc706cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51362/0 (socket says 172.21.15.97:51362) 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 -- 172.21.15.97:0/2067937787 learned_addr learned my addr 172.21.15.97:0/2067937787 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80d800640 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc706240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80ce00640 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc706780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 -- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 msgr2=0x5596cc706780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc706780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 -- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 msgr2=0x5596cc706240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc706240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 -- 172.21.15.97:0/2067937787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cc4f0170 con 0x5596cc7512f0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80e200640 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc706cc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb80000fe20 tx=0x7fb800012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f51a10af31ac1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb7eea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb7eea00640 1 -- 172.21.15.97:0/2067937787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800003980 con 0x5596cc7512f0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596cc4ea0f0 con 0x5596cc7512f0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb7eea00640 1 -- 172.21.15.97:0/2067937787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fb800003b20 con 0x5596cc7512f0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb7eea00640 1 -- 172.21.15.97:0/2067937787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb800024050 con 0x5596cc7512f0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 10 client.6186 fetch_fsmap learned FSMap version 54 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5596cc5d6ef0 con 0x5596cc7512f0 2024-09-10T21:27:21.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb7eea00640 1 -- 172.21.15.97:0/2067937787 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5596cc4ea0f0 con 0x5596cc7512f0 2024-09-10T21:27:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb7eea00640 1 -- 172.21.15.97:0/2067937787 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7fb800027020 con 0x5596cc7512f0 2024-09-10T21:27:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 10 client.6186 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T21:27:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 10 client.6186 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:27:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 20 client.6186 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc75fc70 0x5596cc762060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 4 client.6186 mds_command: new command op to 6144 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:21.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 --> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596cc564a70 con 0x5596cc75fc70 2024-09-10T21:27:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.110+0000 7fb80d800640 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc75fc70 0x5596cc762060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb80d800640 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc75fc70 0x5596cc762060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ac235e59dbecb3 server_cookie=a8bf35749fa2cb75 in_seq=0 out_seq=0 2024-09-10T21:27:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb7eea00640 10 client.6186 ms_handle_connect on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb7eea00640 1 -- 172.21.15.97:0/2067937787 <== mds.0 v2:172.21.15.97:6834/3391319282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596cc564a70 con 0x5596cc75fc70 2024-09-10T21:27:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb7eea00640 10 client.6186 handle_command_reply: tid=0 2024-09-10T21:27:21.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 client.6186 shutdown 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 msgr2=0x5596cc706cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc706cc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb80000fe20 tx=0x7fb800012820 comp rx=0 tx=0).stop 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 shutdown_connections 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc75fc70 0x5596cc762060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc706cc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc706780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb7eea00640 0 client.6186 ms_handle_reset on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2067937787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc706240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 >> 172.21.15.97:0/2067937787 conn(0x5596cc5ee130 msgr2=0x5596cc703fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 shutdown_connections 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 1 -- 172.21.15.97:0/2067937787 wait complete. 2024-09-10T21:27:21.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.114+0000 7fb8135cd000 20 client.6186 trim_cache size 0 max 16384 2024-09-10T21:27:21.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.130+0000 7fb8135cd000 1 Processor -- start 2024-09-10T21:27:21.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.130+0000 7fb8135cd000 1 -- start start 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc7c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc7c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb80815f160 con 0x5596cc7512f0 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb808202ee0 con 0x5596cc704e10 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5596cc51dec0 con 0x5596cc7509d0 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7c3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51366/0 (socket says 172.21.15.97:51366) 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 -- 172.21.15.97:0/1994402992 learned_addr learned my addr 172.21.15.97:0/1994402992 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80d800640 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc7c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80ce00640 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc7c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 -- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 msgr2=0x5596cc7c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc7c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 -- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 msgr2=0x5596cc7c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc7c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 -- 172.21.15.97:0/1994402992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cc5add90 con 0x5596cc7512f0 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80d800640 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc7c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80ce00640 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc7c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb80e200640 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7c3680 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb800000f90 tx=0x7fb800009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffe71e7b3d76a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb7f9600640 1 -- 172.21.15.97:0/1994402992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80002d070 con 0x5596cc7512f0 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb7f9600640 1 -- 172.21.15.97:0/1994402992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb800003a70 con 0x5596cc7512f0 2024-09-10T21:27:21.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb7f9600640 1 -- 172.21.15.97:0/1994402992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800021040 con 0x5596cc7512f0 2024-09-10T21:27:21.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- 172.21.15.97:0/1994402992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 msgr2=0x5596cc7c3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7c3680 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb800000f90 tx=0x7fb800009fd0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- 172.21.15.97:0/1994402992 shutdown_connections 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7c3680 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc7509d0 0x5596cc7c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 --2- 172.21.15.97:0/1994402992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc704e10 0x5596cc7c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- 172.21.15.97:0/1994402992 >> 172.21.15.97:0/1994402992 conn(0x5596cc5ee130 msgr2=0x5596cc767e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- 172.21.15.97:0/1994402992 shutdown_connections 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- 172.21.15.97:0/1994402992 wait complete. 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 -- start start 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.134+0000 7fb8135cd000 1 Processor -- start 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc767160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc7aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5596cc555070 con 0x5596cc7512f0 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5596cc611730 con 0x5596cc7509d0 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5596cc5618f0 con 0x5596cc704e10 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 10 client.0 _ll_register_callbacks cb 0x7fb80000f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51378/0 (socket says 172.21.15.97:51378) 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 -- 172.21.15.97:0/2340933943 learned_addr learned my addr 172.21.15.97:0/2340933943 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80ce00640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc7aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80d800640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc767160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 -- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 msgr2=0x5596cc767160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc767160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 -- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 msgr2=0x5596cc7aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:21.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc7aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 -- 172.21.15.97:0/2340933943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cc518a90 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80ce00640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc7aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80e200640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7abf60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb800005cd0 tx=0x7fb800004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4754aceaf2f78cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 1 -- 172.21.15.97:0/2340933943 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80002d030 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 1 -- 172.21.15.97:0/2340933943 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fb800018070 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb800003a70 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 1 -- 172.21.15.97:0/2340933943 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb800036050 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 10 client.6192 fetch_fsmap learned FSMap version 54 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5596cc4f7050 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 1 -- 172.21.15.97:0/2340933943 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb800003a70 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 1 -- 172.21.15.97:0/2340933943 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7fb800039020 con 0x5596cc7512f0 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 10 client.6192 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T21:27:21.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 10 client.6192 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:27:21.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 20 client.6192 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:21.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc7ac4a0 0x5596cc7c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 4 client.6192 mds_command: new command op to 6144 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:27:21.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 --> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] -- command(tid 0: {"prefix": "status"}) -- 0x5596cc560520 con 0x5596cc7ac4a0 2024-09-10T21:27:21.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80d800640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc7ac4a0 0x5596cc7c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb80d800640 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc7ac4a0 0x5596cc7c6e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8193590d5b2bb536 server_cookie=7849ea359838d798 in_seq=0 out_seq=0 2024-09-10T21:27:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 10 client.6192 ms_handle_connect on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 1 -- 172.21.15.97:0/2340933943 <== mds.0 v2:172.21.15.97:6834/3391319282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5596cc560520 con 0x5596cc7ac4a0 2024-09-10T21:27:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 10 client.6192 handle_command_reply: tid=0 2024-09-10T21:27:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 client.6192 shutdown 2024-09-10T21:27:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 msgr2=0x5596cc7abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7abf60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb800005cd0 tx=0x7fb800004f30 comp rx=0 tx=0).stop 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 shutdown_connections 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x5596cc7ac4a0 0x5596cc7c6e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5596cc7512f0 0x5596cc7abf60 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596cc7509d0 0x5596cc7aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 --2- 172.21.15.97:0/2340933943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5596cc704e10 0x5596cc767160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb7eea00640 0 client.6192 ms_handle_reset on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 >> 172.21.15.97:0/2340933943 conn(0x5596cc5ee130 msgr2=0x5596cc762060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 shutdown_connections 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 1 -- 172.21.15.97:0/2340933943 wait complete. 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.138+0000 7fb8135cd000 20 client.6192 trim_cache size 0 max 16384 2024-09-10T21:27:21.147 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6144, 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 4.5760988339999997, 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 54, 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 68, 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 68, 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.690886395 2024-09-10T21:27:21.148 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:27:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7e80725e0 msgr2=0x7fb7e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7e80725e0 0x7fb7e8074aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb8081fa910 tx=0x7fb7f4009040 comp rx=0 tx=0).stop 2024-09-10T21:27:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 msgr2=0x7fb80815c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 0x7fb80815c460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc01af60 tx=0x7fb7fc0a5690 comp rx=0 tx=0).stop 2024-09-10T21:27:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 shutdown_connections 2024-09-10T21:27:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7e80725e0 0x7fb7e8074aa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb80815bdd0 0x7fb80815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8080cfad0 0x7fb80814b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 --2- 172.21.15.97:0/1947677424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb8080b0110 0x7fb80815c460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 >> 172.21.15.97:0/1947677424 conn(0x7fb8080b28c0 msgr2=0x7fb8080b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 shutdown_connections 2024-09-10T21:27:21.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.142+0000 7fb810200640 1 -- 172.21.15.97:0/1947677424 wait complete. 2024-09-10T21:27:21.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 -- 172.21.15.97:0/3755697361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 msgr2=0x7f14b00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 --2- 172.21.15.97:0/3755697361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b00b2a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f14b0073080 tx=0x7f14a001c800 comp rx=0 tx=0).stop 2024-09-10T21:27:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 -- 172.21.15.97:0/3755697361 shutdown_connections 2024-09-10T21:27:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 --2- 172.21.15.97:0/3755697361 >> v2:172.21.15.194:3300/0 conn(0x7f14b006fd50 0x7f14b0070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 --2- 172.21.15.97:0/3755697361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b00b2a80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 -- 172.21.15.97:0/3755697361 >> 172.21.15.97:0/3755697361 conn(0x7f14b00b1780 msgr2=0x7f14b00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 -- 172.21.15.97:0/3755697361 shutdown_connections 2024-09-10T21:27:21.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.298+0000 7f14b5800640 1 -- 172.21.15.97:0/3755697361 wait complete. 2024-09-10T21:27:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 Processor -- start 2024-09-10T21:27:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 -- start start 2024-09-10T21:27:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14b006fd50 0x7f14b01f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b0140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14b0144690 0x7f14b01410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0167bf0 con 0x7f14b0143f90 2024-09-10T21:27:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0167a70 con 0x7f14b006fd50 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f14b0167d70 con 0x7f14b0144690 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b0140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b0140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51400/0 (socket says 172.21.15.97:51400) 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 -- 172.21.15.97:0/2033595097 learned_addr learned my addr 172.21.15.97:0/2033595097 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14af400640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14b0144690 0x7f14b01410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14aea00640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14b006fd50 0x7f14b01f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 -- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14b0144690 msgr2=0x7f14b01410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14b0144690 0x7f14b01410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 -- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14b006fd50 msgr2=0x7f14b01f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14b006fd50 0x7f14b01f96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 -- 172.21.15.97:0/2033595097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b0136cf0 con 0x7f14b0143f90 2024-09-10T21:27:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14af400640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14b0144690 0x7f14b01410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14aea00640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14b006fd50 0x7f14b01f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a6000640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b0140270 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f149800c730 tx=0x7f149800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a7400640 1 -- 172.21.15.97:0/2033595097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498016020 con 0x7f14b0143f90 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a7400640 1 -- 172.21.15.97:0/2033595097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f149800a950 con 0x7f14b0143f90 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14a7400640 1 -- 172.21.15.97:0/2033595097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149800ada0 con 0x7f14b0143f90 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.302+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b0136fd0 con 0x7f14b0143f90 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.306+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b01f8ed0 con 0x7f14b0143f90 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.306+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b01f93c0 con 0x7f14b0143f90 2024-09-10T21:27:21.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.306+0000 7f14a7400640 1 -- 172.21.15.97:0/2033595097 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f149801b400 con 0x7f14b0143f90 2024-09-10T21:27:21.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.306+0000 7f14a7400640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f148c0725e0 0x7f148c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.306+0000 7f14a7400640 1 -- 172.21.15.97:0/2033595097 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f1498093670 con 0x7f14b0143f90 2024-09-10T21:27:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.306+0000 7f14aea00640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f148c0725e0 0x7f148c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.310+0000 7f14aea00640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f148c0725e0 0x7f148c074aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14a00af000 tx=0x7f14a001b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.310+0000 7f14a7400640 1 -- 172.21.15.97:0/2033595097 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1498020020 con 0x7f14b0143f90 2024-09-10T21:27:21.465 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:27:21.462+0000 7f56abc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:27:21.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.506+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b00cc4e0 con 0x7f14b0143f90 2024-09-10T21:27:21.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14a7400640 1 -- 172.21.15.97:0/2033595097 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5231 (secure 0 0 0) 0x7f14b00706a0 con 0x7f14b0143f90 2024-09-10T21:27:21.512 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:21.512 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"btime":"2024-09-10T21:27:19:570290+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:19.570283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6144,"qdb_cluster":[6144]},"id":4}]} 2024-09-10T21:27:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 54 2024-09-10T21:27:21.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f148c0725e0 msgr2=0x7f148c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f148c0725e0 0x7f148c074aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14a00af000 tx=0x7f14a001b040 comp rx=0 tx=0).stop 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 msgr2=0x7f14b0140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b0140270 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f149800c730 tx=0x7f149800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 shutdown_connections 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f148c0725e0 0x7f148c074aa0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14b0144690 0x7f14b01410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14b0143f90 0x7f14b0140270 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 --2- 172.21.15.97:0/2033595097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14b006fd50 0x7f14b01f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 >> 172.21.15.97:0/2033595097 conn(0x7f14b00b1780 msgr2=0x7f14b015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 shutdown_connections 2024-09-10T21:27:21.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.510+0000 7f14b5800640 1 -- 172.21.15.97:0/2033595097 wait complete. 2024-09-10T21:27:21.533 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2024-09-10T21:27:21.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 -- 172.21.15.97:0/341007724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 msgr2=0x7fe2000af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 --2- 172.21.15.97:0/341007724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000af7b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe200074d10 tx=0x7fe1f801cd80 comp rx=0 tx=0).stop 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 -- 172.21.15.97:0/341007724 shutdown_connections 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 --2- 172.21.15.97:0/341007724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000af7b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 --2- 172.21.15.97:0/341007724 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000b0d50 0x7fe2000aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 --2- 172.21.15.97:0/341007724 >> v2:172.21.15.194:3300/0 conn(0x7fe2000ce640 0x7fe2000cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 -- 172.21.15.97:0/341007724 >> 172.21.15.97:0/341007724 conn(0x7fe2000d1c30 msgr2=0x7fe2000d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 -- 172.21.15.97:0/341007724 shutdown_connections 2024-09-10T21:27:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.674+0000 7fe205200640 1 -- 172.21.15.97:0/341007724 wait complete. 2024-09-10T21:27:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 Processor -- start 2024-09-10T21:27:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 -- start start 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000b0d50 0x7fe2000ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000ce640 0x7fe2000cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2000cd2f0 con 0x7fe2000af300 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2000cd170 con 0x7fe2000b0d50 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe2000cd470 con 0x7fe2000ce640 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51418/0 (socket says 172.21.15.97:51418) 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 -- 172.21.15.97:0/3404922212 learned_addr learned my addr 172.21.15.97:0/3404922212 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1f7e00640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000b0d50 0x7fe2000ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fe800640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000ce640 0x7fe2000cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 -- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000ce640 msgr2=0x7fe2000cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000ce640 0x7fe2000cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 -- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000b0d50 msgr2=0x7fe2000ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000b0d50 0x7fe2000ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 -- 172.21.15.97:0/3404922212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2001f69d0 con 0x7fe2000af300 2024-09-10T21:27:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1fde00640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000b4a60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000ca20 tx=0x7fe1f000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1f6000640 1 -- 172.21.15.97:0/3404922212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0007c70 con 0x7fe2000af300 2024-09-10T21:27:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1f6000640 1 -- 172.21.15.97:0/3404922212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1f000f450 con 0x7fe2000af300 2024-09-10T21:27:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2000b7390 con 0x7fe2000af300 2024-09-10T21:27:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe1f6000640 1 -- 172.21.15.97:0/3404922212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000f8a0 con 0x7fe2000af300 2024-09-10T21:27:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.678+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2000b79a0 con 0x7fe2000af300 2024-09-10T21:27:21.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.682+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2000cd970 con 0x7fe2000af300 2024-09-10T21:27:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.682+0000 7fe1f6000640 1 -- 172.21.15.97:0/3404922212 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe1f0020030 con 0x7fe2000af300 2024-09-10T21:27:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.682+0000 7fe1f6000640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc0725e0 0x7fe1dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.682+0000 7fe1f6000640 1 -- 172.21.15.97:0/3404922212 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fe1f0093880 con 0x7fe2000af300 2024-09-10T21:27:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.682+0000 7fe1f7e00640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc0725e0 0x7fe1dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.682+0000 7fe1f7e00640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc0725e0 0x7fe1dc074aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe20009fe40 tx=0x7fe1e800a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.686+0000 7fe1f6000640 1 -- 172.21.15.97:0/3404922212 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1f0014030 con 0x7fe2000af300 2024-09-10T21:27:21.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:21.918+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) -- 0x7fe2001f66f0 con 0x7fe2000af300 2024-09-10T21:27:22.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe1f6000640 1 -- 172.21.15.97:0/3404922212 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.6144 v55) ==== 112+0+0 (secure 0 0 0) 0x7fe1f005e720 con 0x7fe2000af300 2024-09-10T21:27:22.588 INFO:teuthology.orchestra.run.smithi097.stderr:freezing mds.6144 2024-09-10T21:27:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc0725e0 msgr2=0x7fe1dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc0725e0 0x7fe1dc074aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe20009fe40 tx=0x7fe1e800a2d0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 msgr2=0x7fe2000b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000b4a60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000ca20 tx=0x7fe1f000cee0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 shutdown_connections 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe1dc0725e0 0x7fe1dc074aa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2000ce640 0x7fe2000cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2000b0d50 0x7fe2000ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 --2- 172.21.15.97:0/3404922212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2000af300 0x7fe2000b4a60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 >> 172.21.15.97:0/3404922212 conn(0x7fe2000d1c30 msgr2=0x7fe2000b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 shutdown_connections 2024-09-10T21:27:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.586+0000 7fe205200640 1 -- 172.21.15.97:0/3404922212 wait complete. 2024-09-10T21:27:22.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-10T21:27:22.615 DEBUG:teuthology.orchestra.run.smithi097:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2024-09-10T21:27:22.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-10T21:27:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 -- 172.21.15.97:0/1653837105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1cc0b0d50 msgr2=0x7fa1cc0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 --2- 172.21.15.97:0/1653837105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1cc0b0d50 0x7fa1cc0ccde0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009c80 tx=0x7fa1c401cda0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 -- 172.21.15.97:0/1653837105 shutdown_connections 2024-09-10T21:27:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 --2- 172.21.15.97:0/1653837105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1cc0b0d50 0x7fa1cc0ccde0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 -- 172.21.15.97:0/1653837105 >> 172.21.15.97:0/1653837105 conn(0x7fa1cc0d1c30 msgr2=0x7fa1cc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 -- 172.21.15.97:0/1653837105 shutdown_connections 2024-09-10T21:27:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 -- 172.21.15.97:0/1653837105 wait complete. 2024-09-10T21:27:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 Processor -- start 2024-09-10T21:27:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.762+0000 7fa1d3a00640 1 -- start start 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1cc0afdd0 0x7fa1cc1670f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1cc0b0d50 0x7fa1cc0b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1cc0ce640 0x7fa1cc15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc169fa0 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc169e20 con 0x7fa1cc0ce640 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc16a120 con 0x7fa1cc0b0d50 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1cc0afdd0 0x7fa1cc1670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1cc0afdd0 0x7fa1cc1670f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51436/0 (socket says 172.21.15.97:51436) 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 -- 172.21.15.97:0/2775383702 learned_addr learned my addr 172.21.15.97:0/2775383702 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1cc0b0d50 0x7fa1cc0b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1a00640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1cc0ce640 0x7fa1cc15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 -- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1cc0b0d50 msgr2=0x7fa1cc0b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1cc0b0d50 0x7fa1cc0b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 -- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1cc0ce640 msgr2=0x7fa1cc15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1cc0ce640 0x7fa1cc15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 -- 172.21.15.97:0/2775383702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc0b7a20 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d1000640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1cc0afdd0 0x7fa1cc1670f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000b440 tx=0x7fa1c000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1c8c00640 1 -- 172.21.15.97:0/2775383702 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0015020 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1c8c00640 1 -- 172.21.15.97:0/2775383702 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c0011980 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1c8c00640 1 -- 172.21.15.97:0/2775383702 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0011dd0 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 -- 172.21.15.97:0/2775383702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc0b6980 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.766+0000 7fa1d3a00640 1 -- 172.21.15.97:0/2775383702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc0b6f40 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.770+0000 7fa1c8c00640 1 -- 172.21.15.97:0/2775383702 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa1c001a400 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.770+0000 7fa1c8c00640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa1ac0725e0 0x7fa1ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.770+0000 7fa1c8c00640 1 -- 172.21.15.97:0/2775383702 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fa1c0093400 con 0x7fa1cc0afdd0 2024-09-10T21:27:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.770+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa1ac0725e0 0x7fa1ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.770+0000 7fa1d6dc9000 1 Processor -- start 2024-09-10T21:27:22.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.770+0000 7fa1d6dc9000 1 -- start start 2024-09-10T21:27:22.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/2775383702 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa1ac0725e0 0x7fa1ac074aa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0b42f0 tx=0x7fa1b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe2f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f5050 0x55f0fe2a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f0fe105cc0 con 0x55f0fe2a8920 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f0fe0f50f0 con 0x55f0fe2f5050 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f0fe0f8f00 con 0x55f0fe2f4730 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51442/0 (socket says 172.21.15.97:51442) 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1a00640 1 -- 172.21.15.97:0/2847077119 learned_addr learned my addr 172.21.15.97:0/2847077119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1000640 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe2f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f5050 0x55f0fe2a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1000640 1 -- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f5050 msgr2=0x55f0fe2a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1000640 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f5050 0x55f0fe2a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1000640 1 -- 172.21.15.97:0/2847077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 msgr2=0x55f0fe2a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1000640 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2a8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1000640 1 -- 172.21.15.97:0/2847077119 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0fe0934b0 con 0x55f0fe2f4730 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1a00640 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d1000640 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe2f4b10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0091830 tx=0x7fa1c005dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0de74b8f623461 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1bd600640 1 -- 172.21.15.97:0/2847077119 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0083020 con 0x55f0fe2f4730 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1bd600640 1 -- 172.21.15.97:0/2847077119 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c00670e0 con 0x55f0fe2f4730 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1bd600640 1 -- 172.21.15.97:0/2847077119 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0067790 con 0x55f0fe2f4730 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 msgr2=0x55f0fe2f4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe2f4b10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0091830 tx=0x7fa1c005dbe0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/2847077119 shutdown_connections 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f5050 0x55f0fe2a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/2847077119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe2f4b10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/2847077119 >> 172.21.15.97:0/2847077119 conn(0x55f0fe192b60 msgr2=0x55f0fe2d7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/2847077119 shutdown_connections 2024-09-10T21:27:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.774+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/2847077119 wait complete. 2024-09-10T21:27:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 -- start start 2024-09-10T21:27:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 Processor -- start 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f4730 0x55f0fe2b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f5050 0x55f0fe2b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f0fddba880 con 0x55f0fe2a8920 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f0fe08f900 con 0x55f0fe2f4730 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f0fe0aac60 con 0x55f0fe2f5050 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d6dc9000 10 client.0 _ll_register_callbacks cb 0x55f0fe296340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2d51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51454/0 (socket says 172.21.15.97:51454) 2024-09-10T21:27:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 -- 172.21.15.97:0/1608117558 learned_addr learned my addr 172.21.15.97:0/1608117558 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1a00640 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f5050 0x55f0fe2b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f4730 0x55f0fe2b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 -- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f5050 msgr2=0x55f0fe2b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f5050 0x55f0fe2b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 -- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f4730 msgr2=0x55f0fe2b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f4730 0x55f0fe2b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 -- 172.21.15.97:0/1608117558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0fe0b24b0 con 0x55f0fe2a8920 2024-09-10T21:27:22.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1d1000640 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2d51f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0091830 tx=0x7fa1c005a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da602401523a9d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.778+0000 7fa1b2000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 1 -- 172.21.15.97:0/1608117558 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c008f060 con 0x55f0fe2a8920 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f0fe08d430 con 0x55f0fe2a8920 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 1 -- 172.21.15.97:0/1608117558 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fa1c0067300 con 0x55f0fe2a8920 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 1 -- 172.21.15.97:0/1608117558 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa1c0088050 con 0x55f0fe2a8920 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 10 client.6222 fetch_fsmap learned FSMap version 55 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x55f0fe0b8dd0 con 0x55f0fe2a8920 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 1 -- 172.21.15.97:0/1608117558 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f0fe08d430 con 0x55f0fe2a8920 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 1 -- 172.21.15.97:0/1608117558 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7fa1c0091300 con 0x55f0fe2a8920 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 10 client.6222 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 10 client.6222 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 20 client.6222 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:22.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2d70f0 0x55f0fe2d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 4 client.6222 mds_command: new command op to 6144 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 --> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0fe107c10 con 0x55f0fe2d70f0 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2d70f0 0x55f0fe2d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 10 client.6222 ms_handle_connect on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2d70f0 0x55f0fe2d74d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f76c7d800d643d server_cookie=68f1353bc3f88996 in_seq=0 out_seq=0 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 1 -- 172.21.15.97:0/1608117558 <== mds.0 v2:172.21.15.97:6834/3391319282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0fe107c10 con 0x55f0fe2d70f0 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 10 client.6222 handle_command_reply: tid=0 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 client.6222 shutdown 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 msgr2=0x55f0fe2d51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2d51f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0091830 tx=0x7fa1c005a7e0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 shutdown_connections 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2d70f0 0x55f0fe2d74d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f5050 0x55f0fe2b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2f4730 0x55f0fe2b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1608117558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2a8920 0x55f0fe2d51f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1b2000640 0 client.6222 ms_handle_reset on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 >> 172.21.15.97:0/1608117558 conn(0x55f0fe192b60 msgr2=0x55f0fe2a7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 shutdown_connections 2024-09-10T21:27:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1608117558 wait complete. 2024-09-10T21:27:22.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.782+0000 7fa1d6dc9000 20 client.6222 trim_cache size 0 max 16384 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- 172.21.15.97:0/3973734136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac0ce510 msgr2=0x7fefac0b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 --2- 172.21.15.97:0/3973734136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac0ce510 0x7fefac0b49a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009900 tx=0x7fef9c01c850 comp rx=0 tx=0).stop 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- 172.21.15.97:0/3973734136 shutdown_connections 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 --2- 172.21.15.97:0/3973734136 >> v2:172.21.15.194:3300/0 conn(0x7fefac15f450 0x7fefac0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 --2- 172.21.15.97:0/3973734136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac0ce510 0x7fefac0b49a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- 172.21.15.97:0/3973734136 >> 172.21.15.97:0/3973734136 conn(0x7fefac0d1b70 msgr2=0x7fefac0d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- 172.21.15.97:0/3973734136 shutdown_connections 2024-09-10T21:27:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- 172.21.15.97:0/3973734136 wait complete. 2024-09-10T21:27:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 Processor -- start 2024-09-10T21:27:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- start start 2024-09-10T21:27:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce510 0x7fefac1662d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0cf1b0 0x7fefac1fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15f450 0x7fefac151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1694c0 con 0x7fefac0ce510 2024-09-10T21:27:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefac0cd900 con 0x7fefac0cf1b0 2024-09-10T21:27:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fefac169640 con 0x7fefac15f450 2024-09-10T21:27:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefabe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce510 0x7fefac1662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefabe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce510 0x7fefac1662d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51456/0 (socket says 172.21.15.97:51456) 2024-09-10T21:27:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefabe00640 1 -- 172.21.15.97:0/3518590423 learned_addr learned my addr 172.21.15.97:0/3518590423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefb1200640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15f450 0x7fefac151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.794+0000 7fefab400640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0cf1b0 0x7fefac1fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefabe00640 1 -- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15f450 msgr2=0x7fefac151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefabe00640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15f450 0x7fefac151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefabe00640 1 -- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0cf1b0 msgr2=0x7fefac1fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefabe00640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0cf1b0 0x7fefac1fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefabe00640 1 -- 172.21.15.97:0/3518590423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac0cb570 con 0x7fefac0ce510 2024-09-10T21:27:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefabe00640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce510 0x7fefac1662d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009850 tx=0x7fef9c0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0b2040 con 0x7fefac0ce510 2024-09-10T21:27:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef9c004400 con 0x7fefac0ce510 2024-09-10T21:27:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c005290 con 0x7fefac0ce510 2024-09-10T21:27:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefb3200640 1 -- 172.21.15.97:0/3518590423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac0cb850 con 0x7fefac0ce510 2024-09-10T21:27:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefb3200640 1 -- 172.21.15.97:0/3518590423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1fb3b0 con 0x7fefac0ce510 2024-09-10T21:27:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefb3200640 1 -- 172.21.15.97:0/3518590423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fefac0b49a0 con 0x7fefac0ce510 2024-09-10T21:27:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.798+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fef9c005430 con 0x7fefac0ce510 2024-09-10T21:27:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 Processor -- start 2024-09-10T21:27:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- start start 2024-09-10T21:27:22.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fefa3e00640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef8c0725e0 0x7fef8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fef9c12cde0 con 0x7fefac0ce510 2024-09-10T21:27:22.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+515 (secure 0 0 0) 0x7fef9c0164d0 con 0x7fefac0ce510 2024-09-10T21:27:22.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fefab400640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef8c0725e0 0x7fef8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe192b60 0x55f0fe3671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2a8920 0x55f0fe367710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fefb3200640 1 -- 172.21.15.97:0/3518590423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fefac15c4b0 con 0x7fefac0ce510 2024-09-10T21:27:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2f4730 0x55f0fe367c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc2036f0 con 0x55f0fe2f4730 2024-09-10T21:27:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc159830 con 0x55f0fe192b60 2024-09-10T21:27:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc0c6440 con 0x55f0fe2a8920 2024-09-10T21:27:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe192b60 0x55f0fe3671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2f4730 0x55f0fe367c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 7 ==== log(1 entries from seq 606 at 2024-09-10T21:27:21.920988+0000) ==== 308+0+0 (secure 0 0 0) 0x7fef9c12e220 con 0x7fefac0ce510 2024-09-10T21:27:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1cbe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2a8920 0x55f0fe367710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe192b60 0x55f0fe3671d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:51682/0 (socket says 172.21.15.97:51682) 2024-09-10T21:27:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2f4730 0x55f0fe367c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51466/0 (socket says 172.21.15.97:51466) 2024-09-10T21:27:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 -- 172.21.15.97:0/1636497272 learned_addr learned my addr 172.21.15.97:0/1636497272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:22.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fefab400640 1 --2- 172.21.15.97:0/3518590423 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef8c0725e0 0x7fef8c074aa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fefac09fab0 tx=0x7fef98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 -- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2a8920 msgr2=0x55f0fe367710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2a8920 0x55f0fe367710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 -- 172.21.15.97:0/1636497272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2f4730 msgr2=0x55f0fe367c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2f4730 0x55f0fe367c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 -- 172.21.15.97:0/1636497272 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0fe151b70 con 0x55f0fe192b60 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1a00640 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2f4730 0x55f0fe367c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2a8920 0x55f0fe367710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d1000640 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe192b60 0x55f0fe3671d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0065440 tx=0x7fa1c005daf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154223f7fd86a525 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1bcc00640 1 -- 172.21.15.97:0/1636497272 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c007a070 con 0x55f0fe192b60 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1bcc00640 1 -- 172.21.15.97:0/1636497272 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c0067300 con 0x55f0fe192b60 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1bcc00640 1 -- 172.21.15.97:0/1636497272 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0077070 con 0x55f0fe192b60 2024-09-10T21:27:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe192b60 msgr2=0x55f0fe3671d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe192b60 0x55f0fe3671d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0065440 tx=0x7fa1c005daf0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1636497272 shutdown_connections 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe2f4730 0x55f0fe367c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2a8920 0x55f0fe367710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/1636497272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe192b60 0x55f0fe3671d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1636497272 >> 172.21.15.97:0/1636497272 conn(0x55f0fe2b0bf0 msgr2=0x55f0fe2f35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1636497272 shutdown_connections 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.802+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/1636497272 wait complete. 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 -- start start 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 Processor -- start 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe192b60 0x55f0fe36ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2a8920 0x55f0fe36b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe36b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f0fe0bd8b0 con 0x55f0fe192b60 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f0fe0acdc0 con 0x55f0fe2a8920 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f0fe0f8210 con 0x55f0fe2f4730 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 10 client.0 _ll_register_callbacks cb 0x7fa1c40b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe192b60 0x55f0fe36ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe192b60 0x55f0fe36ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51480/0 (socket says 172.21.15.97:51480) 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 -- 172.21.15.97:0/662109799 learned_addr learned my addr 172.21.15.97:0/662109799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1a00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe36b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2a8920 0x55f0fe36b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 msgr2=0x55f0fe36b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f0fe2f4730 0x55f0fe36b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2a8920 msgr2=0x55f0fe36b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0fe2a8920 0x55f0fe36b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 -- 172.21.15.97:0/662109799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0fe07ead0 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d1000640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f0fe192b60 0x55f0fe36ac50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000be00 tx=0x7fa1c0083460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c852706c182390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1b2000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1b2000640 1 -- 172.21.15.97:0/662109799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0077070 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1b2000640 1 -- 172.21.15.97:0/662109799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fa1c0060e50 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/662109799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c0067300 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1b2000640 1 -- 172.21.15.97:0/662109799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa1c007a070 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 10 client.6231 fetch_fsmap learned FSMap version 55 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/662109799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x55f0fe09a330 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1b2000640 1 -- 172.21.15.97:0/662109799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c0067300 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1b2000640 1 -- 172.21.15.97:0/662109799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7fa1c005d5a0 con 0x55f0fe192b60 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 10 client.6231 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 10 client.6231 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:27:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 20 client.6231 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 4 client.6231 mds_command: new command op to 6144 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:27:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1d6dc9000 1 -- 172.21.15.97:0/662109799 --> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0fe1036c0 con 0x55f0fe2f5050 2024-09-10T21:27:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.806+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.810+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc0070c691f25e2 server_cookie=a2d9480842ad9e8b in_seq=0 out_seq=0 2024-09-10T21:27:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.810+0000 7fa1b2000640 10 client.6231 ms_handle_connect on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:22.818 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:27:22.810+0000 7f56a6c00640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55ea540e4f00] 2024-09-10T21:27:22.818 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:27:22.810+0000 7f56a6c00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55ea540e4f00] 2024-09-10T21:27:22.818 INFO:tasks.ceph.mds.a.smithi097.stderr:./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f56a6c00640 time 2024-09-10T21:27:22.815211+0000 2024-09-10T21:27:22.818 INFO:tasks.ceph.mds.a.smithi097.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:27:22.818 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.818 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f56b5f85968] 2024-09-10T21:27:22.818 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.819 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:27:22.814+0000 7f56a6c00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f56a6c00640 time 2024-09-10T21:27:22.815211+0000 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f56b5f85968] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.820 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.821 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr:*** Caught signal (Aborted) ** 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: in thread 7f56a6c00640 thread_name:MR_Finisher 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f56b5642520] 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: pthread_kill() 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: raise() 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: abort() 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f56b5f85a1e] 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.822 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 19: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.823 INFO:tasks.ceph.mds.a.smithi097.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 22: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:27:22.814+0000 7f56a6c00640 -1 *** Caught signal (Aborted) ** 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: in thread 7f56a6c00640 thread_name:MR_Finisher 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f56b5642520] 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: pthread_kill() 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: raise() 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: abort() 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f56b5f85a1e] 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.824 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 19: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.825 INFO:tasks.ceph.mds.a.smithi097.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.826 INFO:tasks.ceph.mds.a.smithi097.stderr: 22: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.826 INFO:tasks.ceph.mds.a.smithi097.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T21:27:22.826 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.827 INFO:teuthology.orchestra.run.smithi097.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-10T21:27:22.827 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-10T21:27:22.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:22.828 INFO:tasks.ceph.mds.a.smithi097.stderr: -197> 2024-09-10T21:27:21.462+0000 7f56abc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: -10> 2024-09-10T21:27:22.810+0000 7f56a6c00640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55ea540e4f00] 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: -8> 2024-09-10T21:27:22.810+0000 7f56a6c00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55ea540e4f00] 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: -1> 2024-09-10T21:27:22.814+0000 7f56a6c00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f56a6c00640 time 2024-09-10T21:27:22.815211+0000 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f56b5f85968] 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.829 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.830 INFO:tasks.ceph.mds.a.smithi097.stderr: 0> 2024-09-10T21:27:22.814+0000 7f56a6c00640 -1 *** Caught signal (Aborted) ** 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: in thread 7f56a6c00640 thread_name:MR_Finisher 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f56b5642520] 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: pthread_kill() 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: raise() 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: abort() 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f56b5f85a1e] 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.831 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 19: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 22: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T21:27:22.832 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.833 INFO:tasks.ceph.mds.a.smithi097.stderr: -199> 2024-09-10T21:27:21.462+0000 7f56abc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:27:22.833 INFO:tasks.ceph.mds.a.smithi097.stderr: -12> 2024-09-10T21:27:22.810+0000 7f56a6c00640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55ea540e4f00] 2024-09-10T21:27:22.833 INFO:tasks.ceph.mds.a.smithi097.stderr: -10> 2024-09-10T21:27:22.810+0000 7f56a6c00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55ea540e4f00] 2024-09-10T21:27:22.833 INFO:tasks.ceph.mds.a.smithi097.stderr: -3> 2024-09-10T21:27:22.814+0000 7f56a6c00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f56a6c00640 time 2024-09-10T21:27:22.815211+0000 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f56b5f85968] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.834 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: -2> 2024-09-10T21:27:22.814+0000 7f56a6c00640 -1 *** Caught signal (Aborted) ** 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: in thread 7f56a6c00640 thread_name:MR_Finisher 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:27:22.835 INFO:tasks.ceph.mds.a.smithi097.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f56b5642520] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 2: pthread_kill() 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 3: raise() 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 4: abort() 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f56b5f85a1e] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55ea35fc3de1] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x55ea361f2fe0] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 8: ceph-mds(+0x3c209c) [0x55ea3620b09c] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 9: (CDir::_omap_commit(int)+0x75b) [0x55ea3620c79b] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x12a) [0x55ea362001ca] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x2598) [0x55ea36012538] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55ea36297052] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 13: (MDLog::trim_all()+0x728) [0x55ea36297c38] 2024-09-10T21:27:22.836 INFO:tasks.ceph.mds.a.smithi097.stderr: 14: ceph-mds(+0x1b7018) [0x55ea36000018] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 15: ceph-mds(+0x14dc0d) [0x55ea35f96c0d] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 17: ceph-mds(+0x1580b2) [0x55ea35fa10b2] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55ea3628ef0c] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 19: (MDSIOContextBase::complete(int)+0x30c) [0x55ea3628f33c] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f56b6045625] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 21: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f56b5694b43] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 22: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f56b5726a00] 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T21:27:22.837 INFO:tasks.ceph.mds.a.smithi097.stderr: 2024-09-10T21:27:22.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.902+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:27:22.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.902+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:27:22.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.902+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:27:22.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.902+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-09-10T21:27:22.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.906+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk reading from fd=19 : -104 Unknown error -104 2024-09-10T21:27:22.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.906+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:27:22.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.906+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-09-10T21:27:22.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.906+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:27:22.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 -- 172.21.15.97:0/1158533805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 msgr2=0x7f3424073b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 --2- 172.21.15.97:0/1158533805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f3424073b50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f34240736b0 tx=0x7f341c01c890 comp rx=0 tx=0).stop 2024-09-10T21:27:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 -- 172.21.15.97:0/1158533805 shutdown_connections 2024-09-10T21:27:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 --2- 172.21.15.97:0/1158533805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f3424073b50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 --2- 172.21.15.97:0/1158533805 >> v2:172.21.15.194:3300/0 conn(0x7f34240ce7b0 0x7f3424074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 -- 172.21.15.97:0/1158533805 >> 172.21.15.97:0/1158533805 conn(0x7f34240cd680 msgr2=0x7f34240cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 -- 172.21.15.97:0/1158533805 shutdown_connections 2024-09-10T21:27:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 -- 172.21.15.97:0/1158533805 wait complete. 2024-09-10T21:27:22.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 Processor -- start 2024-09-10T21:27:22.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.974+0000 7f3428c00640 1 -- start start 2024-09-10T21:27:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f34241fba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34240ce7b0 0x7f34241f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f342415a8f0 0x7f342414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3424167ce0 con 0x7f3424073720 2024-09-10T21:27:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3424167b60 con 0x7f342415a8f0 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3424167e60 con 0x7f34240ce7b0 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f34241fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f34241fba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51492/0 (socket says 172.21.15.97:51492) 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 -- 172.21.15.97:0/2626020715 learned_addr learned my addr 172.21.15.97:0/2626020715 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3422800640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f342415a8f0 0x7f342414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3419400640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34240ce7b0 0x7f34241f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 -- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34240ce7b0 msgr2=0x7f34241f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34240ce7b0 0x7f34241f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 -- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f342415a8f0 msgr2=0x7f342414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f342415a8f0 0x7f342414a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 -- 172.21.15.97:0/2626020715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34241f89c0 con 0x7f3424073720 2024-09-10T21:27:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3419400640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34240ce7b0 0x7f34241f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3421e00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f34241fba70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f341400ba80 tx=0x7f341400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f341aa00640 1 -- 172.21.15.97:0/2626020715 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341400cba0 con 0x7f3424073720 2024-09-10T21:27:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f341aa00640 1 -- 172.21.15.97:0/2626020715 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f341400ce30 con 0x7f3424073720 2024-09-10T21:27:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f341aa00640 1 -- 172.21.15.97:0/2626020715 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414003be0 con 0x7f3424073720 2024-09-10T21:27:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34240b21d0 con 0x7f3424073720 2024-09-10T21:27:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241f7c80 con 0x7f3424073720 2024-09-10T21:27:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.978+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34241f7a70 con 0x7f3424073720 2024-09-10T21:27:22.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.982+0000 7f341aa00640 1 -- 172.21.15.97:0/2626020715 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3414014440 con 0x7f3424073720 2024-09-10T21:27:22.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.982+0000 7f341aa00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34000725e0 0x7f3400074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:22.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.982+0000 7f341aa00640 1 -- 172.21.15.97:0/2626020715 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f34140931d0 con 0x7f3424073720 2024-09-10T21:27:22.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.982+0000 7f3419400640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34000725e0 0x7f3400074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:22.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.982+0000 7f3419400640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34000725e0 0x7f3400074aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f340c0058d0 tx=0x7f340c005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:22.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:22.986+0000 7f341aa00640 1 -- 172.21.15.97:0/2626020715 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34241f89c0 con 0x7f3424073720 2024-09-10T21:27:23.018 INFO:tasks.ceph.mds.a.smithi097.stderr:daemon-helper: command crashed with signal 6 2024-09-10T21:27:23.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.110+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.110+0000 7fa1b2000640 1 client.6231 ms_handle_refused on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.110+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T21:27:23.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.182+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3424073b50 con 0x7f3424073720 2024-09-10T21:27:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f341aa00640 1 -- 172.21.15.97:0/2626020715 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f341405e110 con 0x7f3424073720 2024-09-10T21:27:23.188 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:23.188 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"btime":"2024-09-10T21:27:22:580181+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:21.921064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6144,"qdb_cluster":[6144]},"id":4}]} 2024-09-10T21:27:23.188 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 55 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34000725e0 msgr2=0x7f3400074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34000725e0 0x7f3400074aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f340c0058d0 tx=0x7f340c005860 comp rx=0 tx=0).stop 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 msgr2=0x7f34241fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f34241fba70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f341400ba80 tx=0x7f341400bf40 comp rx=0 tx=0).stop 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 shutdown_connections 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34000725e0 0x7f3400074aa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f342415a8f0 0x7f342414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34240ce7b0 0x7f34241f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 --2- 172.21.15.97:0/2626020715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3424073720 0x7f34241fba70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 >> 172.21.15.97:0/2626020715 conn(0x7f34240cd680 msgr2=0x7f34240b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 shutdown_connections 2024-09-10T21:27:23.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.186+0000 7f3428c00640 1 -- 172.21.15.97:0/2626020715 wait complete. 2024-09-10T21:27:23.208 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T21:27:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.514+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.514+0000 7fa1b2000640 1 client.6231 ms_handle_refused on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.514+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T21:27:23.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:23.586+0000 7fefa3e00640 1 -- 172.21.15.97:0/3518590423 <== mon.0 v2:172.21.15.97:3300/0 8 ==== log(2 entries from seq 607 at 2024-09-10T21:27:22.588018+0000) ==== 635+0+0 (secure 0 0 0) 0x7fef9c12b240 con 0x7fefac0ce510 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: cluster: 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: id: d1b4b680-9a7d-4173-b750-f3c8b85e2f7b 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: health: HEALTH_OK 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: services: 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: mon: 3 daemons, quorum a,b,c (age 3m) 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: mgr: x(active, since 3m), standbys: y 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: mds: 1/1 daemons up, 3 standby 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: data: 2024-09-10T21:27:23.591 INFO:teuthology.orchestra.run.smithi097.stdout: volumes: 1/1 healthy 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: pools: 3 pools, 97 pgs 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: objects: 29 objects, 601 KiB 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: usage: 219 MiB used, 720 GiB / 720 GiB avail 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: pgs: 97 active+clean 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: io: 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: client: 7.8 KiB/s rd, 3.3 KiB/s wr, 6 op/s rd, 3 op/s wr 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:23.592 INFO:teuthology.orchestra.run.smithi097.stdout:2024-09-10T21:27:22.815189+0000 mds.a [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55ea540e4f00] 2024-09-10T21:27:24.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:24.318+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:24.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:24.318+0000 7fa1b2000640 1 client.6231 ms_handle_refused on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:24.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:24.318+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T21:27:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:25.922+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:25.922+0000 7fa1b2000640 1 client.6231 ms_handle_refused on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:25.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:25.922+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=15 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T21:27:28.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.358+0000 7f6903a00640 1 -- 172.21.15.97:0/3288261955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e0004d50 msgr2=0x7f68e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.358+0000 7f6903a00640 1 --2- 172.21.15.97:0/3288261955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e0004d50 0x7f68e00051d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009e00 tx=0x7f68f401d130 comp rx=0 tx=0).stop 2024-09-10T21:27:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.358+0000 7f6901a00640 1 -- 172.21.15.97:0/3288261955 reap_dead start 2024-09-10T21:27:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.358+0000 7f6903a00640 1 -- 172.21.15.97:0/3288261955 shutdown_connections 2024-09-10T21:27:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.358+0000 7f6903a00640 1 -- 172.21.15.97:0/3288261955 >> 172.21.15.97:0/3288261955 conn(0x7f68fc0b1780 msgr2=0x7f68fc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.358+0000 7f6903a00640 1 -- 172.21.15.97:0/3288261955 shutdown_connections 2024-09-10T21:27:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.358+0000 7f6903a00640 1 -- 172.21.15.97:0/3288261955 wait complete. 2024-09-10T21:27:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 Processor -- start 2024-09-10T21:27:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 -- start start 2024-09-10T21:27:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68e0004d50 0x7f68fc1f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc14b2f0 0x7f68fc155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc14b750 0x7f68fc156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0cf700 con 0x7f68fc14b2f0 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0cf580 con 0x7f68fc14b750 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc1585f0 con 0x7f68e0004d50 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc14b2f0 0x7f68fc155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc14b2f0 0x7f68fc155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37220/0 (socket says 172.21.15.97:37220) 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 -- 172.21.15.97:0/1508884041 learned_addr learned my addr 172.21.15.97:0/1508884041 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6901a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc14b750 0x7f68fc156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6901000640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68e0004d50 0x7f68fc1f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 -- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68e0004d50 msgr2=0x7f68fc1f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68e0004d50 0x7f68fc1f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 -- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc14b750 msgr2=0x7f68fc156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc14b750 0x7f68fc156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 -- 172.21.15.97:0/1508884041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc1f6c80 con 0x7f68fc14b2f0 2024-09-10T21:27:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6901a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc14b750 0x7f68fc156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f3e00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc14b2f0 0x7f68fc155de0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f68e400fe70 tx=0x7f68e400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f9600640 1 -- 172.21.15.97:0/1508884041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4004230 con 0x7f68fc14b2f0 2024-09-10T21:27:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f9600640 1 -- 172.21.15.97:0/1508884041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e40043d0 con 0x7f68fc14b2f0 2024-09-10T21:27:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc154a70 con 0x7f68fc14b2f0 2024-09-10T21:27:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f68f9600640 1 -- 172.21.15.97:0/1508884041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4003860 con 0x7f68fc14b2f0 2024-09-10T21:27:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.362+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc157fa0 con 0x7f68fc14b2f0 2024-09-10T21:27:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.366+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc06e790 con 0x7f68fc14b2f0 2024-09-10T21:27:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.366+0000 7f68f9600640 1 -- 172.21.15.97:0/1508884041 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f68e4016070 con 0x7f68fc14b2f0 2024-09-10T21:27:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.366+0000 7f68f9600640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68d80725e0 0x7f68d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.366+0000 7f68f9600640 1 -- 172.21.15.97:0/1508884041 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7f68e4094380 con 0x7f68fc14b2f0 2024-09-10T21:27:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.366+0000 7f6901000640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68d80725e0 0x7f68d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.370+0000 7f6901000640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68d80725e0 0x7f68d8074aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006ba0 tx=0x7f68ec0067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.370+0000 7f68f9600640 1 -- 172.21.15.97:0/1508884041 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68fc1f6c80 con 0x7f68fc14b2f0 2024-09-10T21:27:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.570+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68fc06acf0 con 0x7f68fc14b2f0 2024-09-10T21:27:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.570+0000 7f68f9600640 1 -- 172.21.15.97:0/1508884041 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f68e4014070 con 0x7f68fc14b2f0 2024-09-10T21:27:28.575 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:28.575 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"btime":"2024-09-10T21:27:22:580181+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:21.921064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6144,"qdb_cluster":[6144]},"id":4}]} 2024-09-10T21:27:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 55 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68d80725e0 msgr2=0x7f68d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68d80725e0 0x7f68d8074aa0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006ba0 tx=0x7f68ec0067b0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc14b2f0 msgr2=0x7f68fc155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc14b2f0 0x7f68fc155de0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f68e400fe70 tx=0x7f68e400c7a0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 shutdown_connections 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68d80725e0 0x7f68d8074aa0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc14b750 0x7f68fc156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc14b2f0 0x7f68fc155de0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 --2- 172.21.15.97:0/1508884041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68e0004d50 0x7f68fc1f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 >> 172.21.15.97:0/1508884041 conn(0x7f68fc0b1780 msgr2=0x7f68fc15c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 shutdown_connections 2024-09-10T21:27:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:28.574+0000 7f6903a00640 1 -- 172.21.15.97:0/1508884041 wait complete. 2024-09-10T21:27:28.597 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T21:27:29.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-10T21:27:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:29.126+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:29.126+0000 7fa1b2000640 1 client.6231 ms_handle_refused on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:29.126+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=15 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-09-10T21:27:33.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.746+0000 7feaf9a00640 1 -- 172.21.15.97:0/2083560589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead8003a60 msgr2=0x7fead8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.746+0000 7feaf9a00640 1 --2- 172.21.15.97:0/2083560589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead8003a60 0x7fead8003e80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7feae4009f10 tx=0x7feae401d5d0 comp rx=0 tx=0).stop 2024-09-10T21:27:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.746+0000 7feaf3400640 1 -- 172.21.15.97:0/2083560589 reap_dead start 2024-09-10T21:27:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.746+0000 7feaf9a00640 1 -- 172.21.15.97:0/2083560589 shutdown_connections 2024-09-10T21:27:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.746+0000 7feaf9a00640 1 -- 172.21.15.97:0/2083560589 >> 172.21.15.97:0/2083560589 conn(0x7feaf40d1c30 msgr2=0x7feaf40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 -- 172.21.15.97:0/2083560589 shutdown_connections 2024-09-10T21:27:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 -- 172.21.15.97:0/2083560589 wait complete. 2024-09-10T21:27:33.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 Processor -- start 2024-09-10T21:27:33.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 -- start start 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fead8003a60 0x7feaf4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaf40b7340 0x7feaf4164a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf4164fc0 0x7feaf415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41670e0 con 0x7feaf4164fc0 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feaf40b00d0 con 0x7feaf40b7340 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7feaf4167260 con 0x7fead8003a60 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf4164fc0 0x7feaf415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf4164fc0 0x7feaf415c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37248/0 (socket says 172.21.15.97:37248) 2024-09-10T21:27:33.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 -- 172.21.15.97:0/3501657357 learned_addr learned my addr 172.21.15.97:0/3501657357 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf2a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fead8003a60 0x7feaf4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaea000640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaf40b7340 0x7feaf4164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 -- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fead8003a60 msgr2=0x7feaf4070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fead8003a60 0x7feaf4070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 -- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaf40b7340 msgr2=0x7feaf4164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaf40b7340 0x7feaf4164a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 -- 172.21.15.97:0/3501657357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf4165500 con 0x7feaf4164fc0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaea000640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaf40b7340 0x7feaf4164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf3400640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf4164fc0 0x7feaf415c360 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7feaf40d0f20 tx=0x7feaec00e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaeb400640 1 -- 172.21.15.97:0/3501657357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec0107d0 con 0x7feaf4164fc0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaeb400640 1 -- 172.21.15.97:0/3501657357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feaec016070 con 0x7feaf4164fc0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaeb400640 1 -- 172.21.15.97:0/3501657357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec010ce0 con 0x7feaf4164fc0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.750+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf40b6860 con 0x7feaf4164fc0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.754+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf40b6550 con 0x7feaf4164fc0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.754+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf40aedc0 con 0x7feaf4164fc0 2024-09-10T21:27:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.754+0000 7feaeb400640 1 -- 172.21.15.97:0/3501657357 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feaec01b460 con 0x7feaf4164fc0 2024-09-10T21:27:33.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.754+0000 7feaeb400640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fead0072610 0x7fead0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:33.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.754+0000 7feaeb400640 1 -- 172.21.15.97:0/3501657357 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7feaec093810 con 0x7feaf4164fc0 2024-09-10T21:27:33.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.754+0000 7feaf2a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fead0072610 0x7fead0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.758+0000 7feaf2a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fead0072610 0x7fead0074ad0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae40b0000 tx=0x7feae401a370 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.758+0000 7feaeb400640 1 -- 172.21.15.97:0/3501657357 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feaec05ec70 con 0x7feaf4164fc0 2024-09-10T21:27:33.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.954+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaf40b2f60 con 0x7feaf4164fc0 2024-09-10T21:27:33.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaeb400640 1 -- 172.21.15.97:0/3501657357 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7feaec067390 con 0x7feaf4164fc0 2024-09-10T21:27:33.960 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:33.960 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"btime":"2024-09-10T21:27:22:580181+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:21.921064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6144,"qdb_cluster":[6144]},"id":4}]} 2024-09-10T21:27:33.960 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 55 2024-09-10T21:27:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fead0072610 msgr2=0x7fead0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fead0072610 0x7fead0074ad0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae40b0000 tx=0x7feae401a370 comp rx=0 tx=0).stop 2024-09-10T21:27:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf4164fc0 msgr2=0x7feaf415c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf4164fc0 0x7feaf415c360 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7feaf40d0f20 tx=0x7feaec00e770 comp rx=0 tx=0).stop 2024-09-10T21:27:33.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 shutdown_connections 2024-09-10T21:27:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fead0072610 0x7fead0074ad0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaf4164fc0 0x7feaf415c360 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaf40b7340 0x7feaf4164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 --2- 172.21.15.97:0/3501657357 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fead8003a60 0x7feaf4070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 >> 172.21.15.97:0/3501657357 conn(0x7feaf40d1c30 msgr2=0x7feaf4072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 shutdown_connections 2024-09-10T21:27:33.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:33.958+0000 7feaf9a00640 1 -- 172.21.15.97:0/3501657357 wait complete. 2024-09-10T21:27:33.982 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T21:27:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:35.534+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:35.534+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-09-10T21:27:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:35.534+0000 7fa1b2000640 1 client.6231 ms_handle_refused on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:35.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-10T21:27:38.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:27:38.938 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:27:38.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:39.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 -- 172.21.15.97:0/2218689696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 msgr2=0x7fc75c0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 --2- 172.21.15.97:0/2218689696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009840 tx=0x7fc74c01c820 comp rx=0 tx=0).stop 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 -- 172.21.15.97:0/2218689696 shutdown_connections 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 --2- 172.21.15.97:0/2218689696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75c0d0490 0x7fc75c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 --2- 172.21.15.97:0/2218689696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf840 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 -- 172.21.15.97:0/2218689696 >> 172.21.15.97:0/2218689696 conn(0x7fc75c0d1a60 msgr2=0x7fc75c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 -- 172.21.15.97:0/2218689696 shutdown_connections 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 -- 172.21.15.97:0/2218689696 wait complete. 2024-09-10T21:27:39.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 Processor -- start 2024-09-10T21:27:39.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.130+0000 7fc761a00640 1 -- start start 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75c0cdcb0 0x7fc75c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc75c0d0490 0x7fc75c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c168930 con 0x7fc75c15efc0 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1687b0 con 0x7fc75c0d0490 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c168ab0 con 0x7fc75c0cdcb0 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:39.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35252/0 (socket says 172.21.15.97:35252) 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 -- 172.21.15.97:0/3618710056 learned_addr learned my addr 172.21.15.97:0/3618710056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc75be00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75c0cdcb0 0x7fc75c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc75b400640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc75c0d0490 0x7fc75c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 -- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75c0cdcb0 msgr2=0x7fc75c154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75c0cdcb0 0x7fc75c154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 -- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc75c0d0490 msgr2=0x7fc75c06dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc75c0d0490 0x7fc75c06dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 -- 172.21.15.97:0/3618710056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c1595f0 con 0x7fc75c15efc0 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761000640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf610 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ea20 tx=0x7fc75400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc753e00640 1 -- 172.21.15.97:0/3618710056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75400cde0 con 0x7fc75c15efc0 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc753e00640 1 -- 172.21.15.97:0/3618710056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc754004c40 con 0x7fc75c15efc0 2024-09-10T21:27:39.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc753e00640 1 -- 172.21.15.97:0/3618710056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754005280 con 0x7fc75c15efc0 2024-09-10T21:27:39.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c1598d0 con 0x7fc75c15efc0 2024-09-10T21:27:39.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c159e90 con 0x7fc75c15efc0 2024-09-10T21:27:39.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c0ae930 con 0x7fc75c15efc0 2024-09-10T21:27:39.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.134+0000 7fc753e00640 1 -- 172.21.15.97:0/3618710056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc75400f040 con 0x7fc75c15efc0 2024-09-10T21:27:39.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.138+0000 7fc753e00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc73c072610 0x7fc73c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:39.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.138+0000 7fc753e00640 1 -- 172.21.15.97:0/3618710056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fc754093b20 con 0x7fc75c15efc0 2024-09-10T21:27:39.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.138+0000 7fc75be00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc73c072610 0x7fc73c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:39.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.138+0000 7fc75be00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc73c072610 0x7fc73c074ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0af000 tx=0x7fc74c01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:39.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.142+0000 7fc753e00640 1 -- 172.21.15.97:0/3618710056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc754016030 con 0x7fc75c15efc0 2024-09-10T21:27:39.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.338+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc75c15a060 con 0x7fc75c15efc0 2024-09-10T21:27:39.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.338+0000 7fc753e00640 1 -- 172.21.15.97:0/3618710056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7fc75405e930 con 0x7fc75c15efc0 2024-09-10T21:27:39.343 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:39.343 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"btime":"2024-09-10T21:27:22:580181+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:21.921064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6144,"qdb_cluster":[6144]},"id":4}]} 2024-09-10T21:27:39.344 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 55 2024-09-10T21:27:39.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc73c072610 msgr2=0x7fc73c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:39.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc73c072610 0x7fc73c074ad0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0af000 tx=0x7fc74c01b060 comp rx=0 tx=0).stop 2024-09-10T21:27:39.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 msgr2=0x7fc75c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:39.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf610 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ea20 tx=0x7fc75400eee0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 shutdown_connections 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc73c072610 0x7fc73c074ad0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc75c15efc0 0x7fc75c0cf610 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc75c0d0490 0x7fc75c06dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 --2- 172.21.15.97:0/3618710056 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc75c0cdcb0 0x7fc75c154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 >> 172.21.15.97:0/3618710056 conn(0x7fc75c0d1a60 msgr2=0x7fc75c071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 shutdown_connections 2024-09-10T21:27:39.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:39.342+0000 7fc761a00640 1 -- 172.21.15.97:0/3618710056 wait complete. 2024-09-10T21:27:39.365 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T21:27:42.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-09-10T21:27:44.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- 172.21.15.97:0/2371046363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7080af520 msgr2=0x7fe708160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 --2- 172.21.15.97:0/2371046363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7080af520 0x7fe708160220 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009840 tx=0x7fe6f401c820 comp rx=0 tx=0).stop 2024-09-10T21:27:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- 172.21.15.97:0/2371046363 shutdown_connections 2024-09-10T21:27:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 --2- 172.21.15.97:0/2371046363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7080af520 0x7fe708160220 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- 172.21.15.97:0/2371046363 >> 172.21.15.97:0/2371046363 conn(0x7fe7080b3c70 msgr2=0x7fe7080b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- 172.21.15.97:0/2371046363 shutdown_connections 2024-09-10T21:27:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- 172.21.15.97:0/2371046363 wait complete. 2024-09-10T21:27:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 Processor -- start 2024-09-10T21:27:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- start start 2024-09-10T21:27:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70806cb40 0x7fe708159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7080af520 0x7fe708151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70815f120 0x7fe7080ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:44.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe70806d0d0 con 0x7fe70815f120 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe70806cf50 con 0x7fe7080af520 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70f000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe70806d250 con 0x7fe70806cb40 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70815f120 0x7fe7080ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70815f120 0x7fe7080ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35266/0 (socket says 172.21.15.97:35266) 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe70d000640 1 -- 172.21.15.97:0/3313064901 learned_addr learned my addr 172.21.15.97:0/3313064901 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe707400640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7080af520 0x7fe708151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.518+0000 7fe707e00640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70806cb40 0x7fe708159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70d000640 1 -- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70806cb40 msgr2=0x7fe708159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70d000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70806cb40 0x7fe708159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70d000640 1 -- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7080af520 msgr2=0x7fe708151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70d000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7080af520 0x7fe708151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70d000640 1 -- 172.21.15.97:0/3313064901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7080af100 con 0x7fe70815f120 2024-09-10T21:27:44.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe707e00640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70806cb40 0x7fe708159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:44.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe707400640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7080af520 0x7fe708151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:44.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70d000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70815f120 0x7fe7080ad0d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe70000cbc0 tx=0x7fe70000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:44.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe6ffe00640 1 -- 172.21.15.97:0/3313064901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700016020 con 0x7fe70815f120 2024-09-10T21:27:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe6ffe00640 1 -- 172.21.15.97:0/3313064901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe70000ab40 con 0x7fe70815f120 2024-09-10T21:27:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7080ae1f0 con 0x7fe70815f120 2024-09-10T21:27:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe6ffe00640 1 -- 172.21.15.97:0/3313064901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700012420 con 0x7fe70815f120 2024-09-10T21:27:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7081f86b0 con 0x7fe70815f120 2024-09-10T21:27:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe708160220 con 0x7fe70815f120 2024-09-10T21:27:44.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe6ffe00640 1 -- 172.21.15.97:0/3313064901 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe70001b400 con 0x7fe70815f120 2024-09-10T21:27:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe6ffe00640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.522+0000 7fe6ffe00640 1 -- 172.21.15.97:0/3313064901 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7fe700093ac0 con 0x7fe70815f120 2024-09-10T21:27:44.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.526+0000 7fe707e00640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:44.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.526+0000 7fe707e00640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80098b0 tx=0x7fe6f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:44.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.526+0000 7fe6ffe00640 1 -- 172.21.15.97:0/3313064901 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe700020020 con 0x7fe70815f120 2024-09-10T21:27:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7080d1080 con 0x7fe70815f120 2024-09-10T21:27:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe6ffe00640 1 -- 172.21.15.97:0/3313064901 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5279 (secure 0 0 0) 0x7fe7080cea30 con 0x7fe70815f120 2024-09-10T21:27:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:44.733 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"btime":"2024-09-10T21:27:41:568840+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:41.568797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6144},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6144":{"gid":6144,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3391319282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3391319282},{"type":"v1","addr":"172.21.15.97:6835","nonce":3391319282}]},"join_fscid":-1,"laggy_since":"2024-09-10T21:27:41.568797+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6144,"qdb_cluster":[6144]},"id":4}]} 2024-09-10T21:27:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2024-09-10T21:27:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 msgr2=0x7fe6e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80098b0 tx=0x7fe6f8008040 comp rx=0 tx=0).stop 2024-09-10T21:27:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70815f120 msgr2=0x7fe7080ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70815f120 0x7fe7080ad0d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe70000cbc0 tx=0x7fe70000ccb0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 shutdown_connections 2024-09-10T21:27:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70815f120 0x7fe7080ad0d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7080af520 0x7fe708151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 --2- 172.21.15.97:0/3313064901 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70806cb40 0x7fe708159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 >> 172.21.15.97:0/3313064901 conn(0x7fe7080b3c70 msgr2=0x7fe7081559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 shutdown_connections 2024-09-10T21:27:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.730+0000 7fe70f000640 1 -- 172.21.15.97:0/3313064901 wait complete. 2024-09-10T21:27:44.754 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-10T21:27:44.855 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-09-10T21:27:45.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 -- 172.21.15.97:0/1881168131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc0ae2c0 msgr2=0x7ff7bc15e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 --2- 172.21.15.97:0/1881168131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc15e350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009c80 tx=0x7ff7b401cda0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 -- 172.21.15.97:0/1881168131 shutdown_connections 2024-09-10T21:27:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 --2- 172.21.15.97:0/1881168131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc15e350 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 --2- 172.21.15.97:0/1881168131 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7bc1624c0 0x7ff7bc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 -- 172.21.15.97:0/1881168131 >> 172.21.15.97:0/1881168131 conn(0x7ff7bc06e560 msgr2=0x7ff7bc0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 -- 172.21.15.97:0/1881168131 shutdown_connections 2024-09-10T21:27:45.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:44.998+0000 7ff7c3400640 1 -- 172.21.15.97:0/1881168131 wait complete. 2024-09-10T21:27:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 Processor -- start 2024-09-10T21:27:45.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 -- start start 2024-09-10T21:27:45.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc06d260 0x7ff7bc153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7bc1624c0 0x7ff7bc0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc0cfec0 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc143da0 con 0x7ff7bc06d260 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff7bc0d0040 con 0x7ff7bc1624c0 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc13ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35274/0 (socket says 172.21.15.97:35274) 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 -- 172.21.15.97:0/464012880 learned_addr learned my addr 172.21.15.97:0/464012880 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c0a00640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc06d260 0x7ff7bc153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c1400640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7bc1624c0 0x7ff7bc0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 -- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7bc1624c0 msgr2=0x7ff7bc0b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7bc1624c0 0x7ff7bc0b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 -- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc06d260 msgr2=0x7ff7bc153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc06d260 0x7ff7bc153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 -- 172.21.15.97:0/464012880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc134d80 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7bbe00640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc13ea90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800e970 tx=0x7ff7a800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7b8c00640 1 -- 172.21.15.97:0/464012880 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a800cd80 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7b8c00640 1 -- 172.21.15.97:0/464012880 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7a8004020 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7b8c00640 1 -- 172.21.15.97:0/464012880 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8004470 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc135060 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.002+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc135620 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.006+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc15e350 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.006+0000 7ff7b8c00640 1 -- 172.21.15.97:0/464012880 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7a8003600 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.006+0000 7ff7b8c00640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff79c0725e0 0x7ff79c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.006+0000 7ff7b8c00640 1 -- 172.21.15.97:0/464012880 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7ff7a8093cf0 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.006+0000 7ff7c0a00640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff79c0725e0 0x7ff79c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.006+0000 7ff7c0a00640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff79c0725e0 0x7ff79c074aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc169aa0 tx=0x7ff7b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:45.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.010+0000 7ff7b8c00640 1 -- 172.21.15.97:0/464012880 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7a805eb00 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.174+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7ff7bc160780 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.178+0000 7ff7b8c00640 1 -- 172.21.15.97:0/464012880 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7ff7a8063530 con 0x7ff7bc0ae2c0 2024-09-10T21:27:45.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff79c0725e0 msgr2=0x7ff79c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff79c0725e0 0x7ff79c074aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc169aa0 tx=0x7ff7b0009040 comp rx=0 tx=0).stop 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7bc0ae2c0 msgr2=0x7ff7bc13ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc13ea90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800e970 tx=0x7ff7a800ee30 comp rx=0 tx=0).stop 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 shutdown_connections 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff79c0725e0 0x7ff79c074aa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7bc1624c0 0x7ff7bc0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7bc0ae2c0 0x7ff7bc13ea90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 --2- 172.21.15.97:0/464012880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7bc06d260 0x7ff7bc153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 >> 172.21.15.97:0/464012880 conn(0x7ff7bc06e560 msgr2=0x7ff7bc0d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 shutdown_connections 2024-09-10T21:27:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.182+0000 7ff7c3400640 1 -- 172.21.15.97:0/464012880 wait complete. 2024-09-10T21:27:45.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2024-09-10T21:27:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 -- 172.21.15.97:0/2574692638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee0003fe0 msgr2=0x7efee0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 --2- 172.21.15.97:0/2574692638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee0003fe0 0x7efee0004810 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efee8009b60 tx=0x7efee801d300 comp rx=0 tx=0).stop 2024-09-10T21:27:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7efefa800640 1 -- 172.21.15.97:0/2574692638 reap_dead start 2024-09-10T21:27:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 -- 172.21.15.97:0/2574692638 shutdown_connections 2024-09-10T21:27:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 -- 172.21.15.97:0/2574692638 >> 172.21.15.97:0/2574692638 conn(0x7efefc0b1780 msgr2=0x7efefc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 -- 172.21.15.97:0/2574692638 shutdown_connections 2024-09-10T21:27:45.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 -- 172.21.15.97:0/2574692638 wait complete. 2024-09-10T21:27:45.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 Processor -- start 2024-09-10T21:27:45.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.346+0000 7eff00c00640 1 -- start start 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efee0003fe0 0x7efefc1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1e4180 0x7efefc15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efefc1f4f00 0x7efefc13f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efefc06f3f0 con 0x7efefc1f4f00 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efefc06f270 con 0x7efefc1e4180 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efefc06e3b0 con 0x7efee0003fe0 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efefc1f4f00 0x7efefc13f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efefc1f4f00 0x7efefc13f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35300/0 (socket says 172.21.15.97:35300) 2024-09-10T21:27:45.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efef9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efee0003fe0 0x7efefc1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 -- 172.21.15.97:0/428390565 learned_addr learned my addr 172.21.15.97:0/428390565 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efef9400640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1e4180 0x7efefc15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 -- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efee0003fe0 msgr2=0x7efefc1f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efee0003fe0 0x7efefc1f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 -- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1e4180 msgr2=0x7efefc15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1e4180 0x7efefc15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 -- 172.21.15.97:0/428390565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc1f8510 con 0x7efefc1f4f00 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efef9400640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1e4180 0x7efefc15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:45.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efefa800640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efefc1f4f00 0x7efefc13f120 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efefc074190 tx=0x7efef400d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efef2000640 1 -- 172.21.15.97:0/428390565 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4003980 con 0x7efefc1f4f00 2024-09-10T21:27:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efef2000640 1 -- 172.21.15.97:0/428390565 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efef4003b40 con 0x7efefc1f4f00 2024-09-10T21:27:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc1f87f0 con 0x7efefc1f4f00 2024-09-10T21:27:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efef2000640 1 -- 172.21.15.97:0/428390565 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4004020 con 0x7efefc1f4f00 2024-09-10T21:27:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc1f46d0 con 0x7efefc1f4f00 2024-09-10T21:27:45.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc06e940 con 0x7efefc1f4f00 2024-09-10T21:27:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.350+0000 7efef2000640 1 -- 172.21.15.97:0/428390565 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efef400b040 con 0x7efefc1f4f00 2024-09-10T21:27:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.354+0000 7efef2000640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efedc0725e0 0x7efedc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:45.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.354+0000 7efef2000640 1 -- 172.21.15.97:0/428390565 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6187+0+0 (secure 0 0 0) 0x7efef4093b20 con 0x7efefc1f4f00 2024-09-10T21:27:45.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.354+0000 7efef9e00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efedc0725e0 0x7efedc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:45.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.354+0000 7efef9e00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efedc0725e0 0x7efedc074aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efeec000fc0 tx=0x7efeec006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:45.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.358+0000 7efef2000640 1 -- 172.21.15.97:0/428390565 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efef405f010 con 0x7efefc1f4f00 2024-09-10T21:27:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:45.582+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) -- 0x7efefc15b860 con 0x7efefc1f4f00 2024-09-10T21:27:46.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.178+0000 7efef2000640 1 -- 172.21.15.97:0/428390565 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.6144 v57) ==== 115+0+0 (secure 0 0 0) 0x7efef405e910 con 0x7efefc1f4f00 2024-09-10T21:27:46.183 INFO:teuthology.orchestra.run.smithi097.stderr:unfreezing mds.6144 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efedc0725e0 msgr2=0x7efedc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efedc0725e0 0x7efedc074aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efeec000fc0 tx=0x7efeec006040 comp rx=0 tx=0).stop 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efefc1f4f00 msgr2=0x7efefc13f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efefc1f4f00 0x7efefc13f120 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efefc074190 tx=0x7efef400d7b0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 shutdown_connections 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efedc0725e0 0x7efedc074aa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efefc1f4f00 0x7efefc13f120 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1e4180 0x7efefc15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 --2- 172.21.15.97:0/428390565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efee0003fe0 0x7efefc1f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 >> 172.21.15.97:0/428390565 conn(0x7efefc0b1780 msgr2=0x7efefc159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 shutdown_connections 2024-09-10T21:27:46.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.182+0000 7eff00c00640 1 -- 172.21.15.97:0/428390565 wait complete. 2024-09-10T21:27:46.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -n kernel.core_pattern 2024-09-10T21:27:46.230 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:27:46.231 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:27:46.232 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-10T21:27:46.311 INFO:teuthology.orchestra.run.smithi097.stdout:1726003642.41809.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-09-10T21:27:46.314 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726003642.41809.core'] 2024-09-10T21:27:46.314 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726003642.41809.core, deleting it 2024-09-10T21:27:46.314 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726003642.41809.core 2024-09-10T21:27:46.383 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:27:46.383 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:27:46.383 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:27:46.383 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:27:46.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:27:46.383 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi097 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-10T21:27:46.384 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:27:46.384 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:27:46.429 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:27:46.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:46.515 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 -- 172.21.15.97:0/3612810776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0ce640 msgr2=0x7f4a2c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 --2- 172.21.15.97:0/3612810776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0ce640 0x7f4a2c0cea20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0b36f0 tx=0x7f4a1c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 -- 172.21.15.97:0/3612810776 shutdown_connections 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 --2- 172.21.15.97:0/3612810776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a2c0af300 0x7f4a2c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 --2- 172.21.15.97:0/3612810776 >> v2:172.21.15.194:3300/0 conn(0x7f4a2c0b0d50 0x7f4a2c0aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 --2- 172.21.15.97:0/3612810776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0ce640 0x7f4a2c0cea20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 -- 172.21.15.97:0/3612810776 >> 172.21.15.97:0/3612810776 conn(0x7f4a2c0d1c30 msgr2=0x7f4a2c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 -- 172.21.15.97:0/3612810776 shutdown_connections 2024-09-10T21:27:46.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.570+0000 7f4a33000640 1 -- 172.21.15.97:0/3612810776 wait complete. 2024-09-10T21:27:46.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 Processor -- start 2024-09-10T21:27:46.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 -- start start 2024-09-10T21:27:46.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a2c0af300 0x7f4a2c0b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0b0d50 0x7f4a2c141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a2c0ce640 0x7f4a2c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c0cd270 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c0cd0f0 con 0x7f4a2c0ce640 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c0cd3f0 con 0x7f4a2c0af300 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0b0d50 0x7f4a2c141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0b0d50 0x7f4a2c141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35342/0 (socket says 172.21.15.97:35342) 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 -- 172.21.15.97:0/2351683973 learned_addr learned my addr 172.21.15.97:0/2351683973 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 -- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a2c0af300 msgr2=0x7f4a2c0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a2c0af300 0x7f4a2c0b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 -- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a2c0ce640 msgr2=0x7f4a2c0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:46.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a2c0ce640 0x7f4a2c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 -- 172.21.15.97:0/2351683973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c0b5ad0 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a23400640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0b0d50 0x7f4a2c141e80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400d950 tx=0x7f4a1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a28c00640 1 -- 172.21.15.97:0/2351683973 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14017070 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a28c00640 1 -- 172.21.15.97:0/2351683973 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a14009d20 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2c0b3fc0 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a28c00640 1 -- 172.21.15.97:0/2351683973 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a140106c0 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.574+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2c0cbaf0 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.578+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a2c0cb620 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.578+0000 7f4a28c00640 1 -- 172.21.15.97:0/2351683973 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a14010860 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.578+0000 7f4a28c00640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a08072630 0x7f4a08074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:46.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.578+0000 7f4a28c00640 1 -- 172.21.15.97:0/2351683973 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f4a14093800 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.578+0000 7f4a2be00640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a08072630 0x7f4a08074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:46.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.578+0000 7f4a2be00640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a08072630 0x7f4a08074af0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0af000 tx=0x7f4a1c019660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.582+0000 7f4a28c00640 1 -- 172.21.15.97:0/2351683973 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a1405ed10 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.786+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a2c0ce2e0 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.786+0000 7f4a28c00640 1 -- 172.21.15.97:0/2351683973 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+4399 (secure 0 0 0) 0x7f4a1405e610 con 0x7f4a2c0b0d50 2024-09-10T21:27:46.790 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:46.791 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":58,"btime":"2024-09-10T21:27:46:185739+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:46.185735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"d","rank":0,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T21:27:46.791 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 58 2024-09-10T21:27:46.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a08072630 msgr2=0x7f4a08074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a08072630 0x7f4a08074af0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0af000 tx=0x7f4a1c019660 comp rx=0 tx=0).stop 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0b0d50 msgr2=0x7f4a2c141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0b0d50 0x7f4a2c141e80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400d950 tx=0x7f4a1400de10 comp rx=0 tx=0).stop 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 shutdown_connections 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a08072630 0x7f4a08074af0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a2c0ce640 0x7f4a2c0b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0b0d50 0x7f4a2c141e80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 --2- 172.21.15.97:0/2351683973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a2c0af300 0x7f4a2c0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 >> 172.21.15.97:0/2351683973 conn(0x7f4a2c0d1c30 msgr2=0x7f4a2c0d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 shutdown_connections 2024-09-10T21:27:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:46.790+0000 7f4a33000640 1 -- 172.21.15.97:0/2351683973 wait complete. 2024-09-10T21:27:47.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:59.777372+0000', 'modified': '2024-09-10T21:27:46.185735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5955}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5955': {'gid': 5955, 'name': 'd', 'rank': 0, 'incarnation': 58, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.194:6837/1482860240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6835', 'nonce': 1482860240}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 1482860240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:27:47.215 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5955:up:replay 2024-09-10T21:27:48.215 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-09-10T21:27:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-09-10T21:27:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1b2000640 0 client.6231 ms_handle_remote_reset on v2:172.21.15.97:6834/3391319282 2024-09-10T21:27:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:27:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:27:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:27:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.346+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 -- 172.21.15.97:0/218207480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68415efc0 msgr2=0x7fb6840cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 --2- 172.21.15.97:0/218207480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68415efc0 0x7fb6840cf840 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb6840cf410 tx=0x7fb67401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 -- 172.21.15.97:0/218207480 shutdown_connections 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 --2- 172.21.15.97:0/218207480 >> v2:172.21.15.194:3300/0 conn(0x7fb6840d02c0 0x7fb6840ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 --2- 172.21.15.97:0/218207480 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6840cdcb0 0x7fb6840cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 --2- 172.21.15.97:0/218207480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68415efc0 0x7fb6840cf840 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 -- 172.21.15.97:0/218207480 >> 172.21.15.97:0/218207480 conn(0x7fb6840d1a60 msgr2=0x7fb6840d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 -- 172.21.15.97:0/218207480 shutdown_connections 2024-09-10T21:27:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.362+0000 7fb688a00640 1 -- 172.21.15.97:0/218207480 wait complete. 2024-09-10T21:27:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 Processor -- start 2024-09-10T21:27:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 -- start start 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6840cdcb0 0x7fb68414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6840d02c0 0x7fb6840c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68415efc0 0x7fb684153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb6840af070 con 0x7fb6840d02c0 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb6840aeef0 con 0x7fb68415efc0 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb6840af1f0 con 0x7fb6840cdcb0 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6840d02c0 0x7fb6840c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6840d02c0 0x7fb6840c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48328/0 (socket says 172.21.15.97:48328) 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 -- 172.21.15.97:0/3596418897 learned_addr learned my addr 172.21.15.97:0/3596418897 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb683400640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6840cdcb0 0x7fb68414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb683e00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68415efc0 0x7fb684153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 -- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6840cdcb0 msgr2=0x7fb68414e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6840cdcb0 0x7fb68414e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 -- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68415efc0 msgr2=0x7fb684153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68415efc0 0x7fb684153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 -- 172.21.15.97:0/3596418897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68414ede0 con 0x7fb6840d02c0 2024-09-10T21:27:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb682a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6840d02c0 0x7fb6840c7c10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d770 tx=0x7fb67000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb67b400640 1 -- 172.21.15.97:0/3596418897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670017070 con 0x7fb6840d02c0 2024-09-10T21:27:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb67b400640 1 -- 172.21.15.97:0/3596418897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb670009d20 con 0x7fb6840d02c0 2024-09-10T21:27:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6840c6bb0 con 0x7fb6840d02c0 2024-09-10T21:27:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb67b400640 1 -- 172.21.15.97:0/3596418897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700106c0 con 0x7fb6840d02c0 2024-09-10T21:27:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.366+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6840c7130 con 0x7fb6840d02c0 2024-09-10T21:27:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.370+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6840ce1e0 con 0x7fb6840d02c0 2024-09-10T21:27:48.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.370+0000 7fb67b400640 1 -- 172.21.15.97:0/3596418897 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb670010860 con 0x7fb6840d02c0 2024-09-10T21:27:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.370+0000 7fb67b400640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb664072610 0x7fb664074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.370+0000 7fb67b400640 1 -- 172.21.15.97:0/3596418897 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7fb670093920 con 0x7fb6840d02c0 2024-09-10T21:27:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.370+0000 7fb683400640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb664072610 0x7fb664074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.374+0000 7fb683400640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb664072610 0x7fb664074ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb6740af000 tx=0x7fb6740a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:48.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.374+0000 7fb67b400640 1 -- 172.21.15.97:0/3596418897 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb68414ede0 con 0x7fb6840d02c0 2024-09-10T21:27:48.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.578+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6841591e0 con 0x7fb6840d02c0 2024-09-10T21:27:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.578+0000 7fb67b400640 1 -- 172.21.15.97:0/3596418897 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5225 (secure 0 0 0) 0x7fb67005e6d0 con 0x7fb6840d02c0 2024-09-10T21:27:48.581 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:48.581 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"btime":"2024-09-10T21:27:48:194551+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2360657843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2360657843},{"type":"v1","addr":"172.21.15.97:6835","nonce":2360657843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:47.195962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"d","rank":0,"incarnation":58,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T21:27:48.581 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 60 2024-09-10T21:27:48.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb664072610 msgr2=0x7fb664074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb664072610 0x7fb664074ad0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb6740af000 tx=0x7fb6740a7040 comp rx=0 tx=0).stop 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6840d02c0 msgr2=0x7fb6840c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6840d02c0 0x7fb6840c7c10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d770 tx=0x7fb67000dc30 comp rx=0 tx=0).stop 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 shutdown_connections 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb664072610 0x7fb664074ad0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68415efc0 0x7fb684153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6840d02c0 0x7fb6840c7c10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 --2- 172.21.15.97:0/3596418897 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6840cdcb0 0x7fb68414e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 >> 172.21.15.97:0/3596418897 conn(0x7fb6840d1a60 msgr2=0x7fb6840aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 shutdown_connections 2024-09-10T21:27:48.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:48.582+0000 7fb688a00640 1 -- 172.21.15.97:0/3596418897 wait complete. 2024-09-10T21:27:49.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:59.777372+0000', 'modified': '2024-09-10T21:27:47.195962+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5955}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5955': {'gid': 5955, 'name': 'd', 'rank': 0, 'incarnation': 58, 'state': 'up:rejoin', 'state_seq': 17, 'addr': '172.21.15.194:6837/1482860240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6835', 'nonce': 1482860240}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 1482860240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:27:49.006 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5955:up:rejoin 2024-09-10T21:27:49.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-10T21:27:49.342 INFO:tasks.ceph.mds.d.smithi194.stderr:2024-09-10T21:27:49.340+0000 7f2411200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:27:50.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 -- 172.21.15.97:0/1147678085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99741511f0 msgr2=0x7f99740b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 --2- 172.21.15.97:0/1147678085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99741511f0 0x7f99740b37b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f99740cc590 tx=0x7f996401f0b0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 -- 172.21.15.97:0/1147678085 shutdown_connections 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 --2- 172.21.15.97:0/1147678085 >> v2:172.21.15.194:3300/0 conn(0x7f997406f180 0x7f99740ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 --2- 172.21.15.97:0/1147678085 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99740b3cf0 0x7f997406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 --2- 172.21.15.97:0/1147678085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99741511f0 0x7f99740b37b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 -- 172.21.15.97:0/1147678085 >> 172.21.15.97:0/1147678085 conn(0x7f99740b1780 msgr2=0x7f99740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 -- 172.21.15.97:0/1147678085 shutdown_connections 2024-09-10T21:27:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.154+0000 7f9979e00640 1 -- 172.21.15.97:0/1147678085 wait complete. 2024-09-10T21:27:50.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 Processor -- start 2024-09-10T21:27:50.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 -- start start 2024-09-10T21:27:50.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f997406f180 0x7f99741f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99740b3cf0 0x7f9974145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99741511f0 0x7f9974146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f997406e530 con 0x7f99740b3cf0 2024-09-10T21:27:50.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f997406e3b0 con 0x7f99741511f0 2024-09-10T21:27:50.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f997406e6b0 con 0x7f997406f180 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99740b3cf0 0x7f9974145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99740b3cf0 0x7f9974145350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48336/0 (socket says 172.21.15.97:48336) 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 -- 172.21.15.97:0/1415131166 learned_addr learned my addr 172.21.15.97:0/1415131166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9973400640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99741511f0 0x7f9974146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972a00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f997406f180 0x7f99741f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 -- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f997406f180 msgr2=0x7f99741f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f997406f180 0x7f99741f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 -- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99741511f0 msgr2=0x7f9974146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99741511f0 0x7f9974146750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 -- 172.21.15.97:0/1415131166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9974151c10 con 0x7f99740b3cf0 2024-09-10T21:27:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9972000640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99740b3cf0 0x7f9974145350 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f996000d950 tx=0x7f996000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f996b400640 1 -- 172.21.15.97:0/1415131166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960017070 con 0x7f99740b3cf0 2024-09-10T21:27:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f996b400640 1 -- 172.21.15.97:0/1415131166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9960009d20 con 0x7f99740b3cf0 2024-09-10T21:27:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f996b400640 1 -- 172.21.15.97:0/1415131166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600106c0 con 0x7f99740b3cf0 2024-09-10T21:27:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9974151ef0 con 0x7f99740b3cf0 2024-09-10T21:27:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.158+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99741524b0 con 0x7f99740b3cf0 2024-09-10T21:27:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.162+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99740ceee0 con 0x7f99740b3cf0 2024-09-10T21:27:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.162+0000 7f996b400640 1 -- 172.21.15.97:0/1415131166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9960010860 con 0x7f99740b3cf0 2024-09-10T21:27:50.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.162+0000 7f996b400640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99540725e0 0x7f9954074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.162+0000 7f996b400640 1 -- 172.21.15.97:0/1415131166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f99600937d0 con 0x7f99740b3cf0 2024-09-10T21:27:50.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.162+0000 7f9972a00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99540725e0 0x7f9954074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.162+0000 7f9972a00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99540725e0 0x7f9954074aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f99640b2000 tx=0x7f996401d8f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.166+0000 7f996b400640 1 -- 172.21.15.97:0/1415131166 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f996005e610 con 0x7f99740b3cf0 2024-09-10T21:27:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.366+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f997415c0e0 con 0x7f99740b3cf0 2024-09-10T21:27:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.366+0000 7f996b400640 1 -- 172.21.15.97:0/1415131166 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5232 (secure 0 0 0) 0x7f9960063040 con 0x7f99740b3cf0 2024-09-10T21:27:50.370 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:27:50.370 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"btime":"2024-09-10T21:27:49:202694+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2360657843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2360657843},{"type":"v1","addr":"172.21.15.97:6835","nonce":2360657843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:49.202684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":18,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5955,"qdb_cluster":[5955]},"id":4}]} 2024-09-10T21:27:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 61 2024-09-10T21:27:50.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99540725e0 msgr2=0x7f9954074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99540725e0 0x7f9954074aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f99640b2000 tx=0x7f996401d8f0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99740b3cf0 msgr2=0x7f9974145350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99740b3cf0 0x7f9974145350 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f996000d950 tx=0x7f996000de10 comp rx=0 tx=0).stop 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 shutdown_connections 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99540725e0 0x7f9954074aa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99741511f0 0x7f9974146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99740b3cf0 0x7f9974145350 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 --2- 172.21.15.97:0/1415131166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f997406f180 0x7f99741f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 >> 172.21.15.97:0/1415131166 conn(0x7f99740b1780 msgr2=0x7f997415c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 shutdown_connections 2024-09-10T21:27:50.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.370+0000 7f9979e00640 1 -- 172.21.15.97:0/1415131166 wait complete. 2024-09-10T21:27:50.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:26:59.777372+0000', 'modified': '2024-09-10T21:27:49.202684+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5955}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5955': {'gid': 5955, 'name': 'd', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.194:6837/1482860240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6835', 'nonce': 1482860240}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 1482860240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5955, 'qdb_cluster': [5955]} 2024-09-10T21:27:50.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:27:50.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:27:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.934+0000 7f36c6c00640 1 -- 172.21.15.97:0/1778533456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 msgr2=0x7f36c0168b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.934+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1778533456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 0x7f36c0168b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009bf0 tx=0x7f36b801d580 comp rx=0 tx=0).stop 2024-09-10T21:27:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.934+0000 7f36c6c00640 1 -- 172.21.15.97:0/1778533456 shutdown_connections 2024-09-10T21:27:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.934+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1778533456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 0x7f36c0168b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.934+0000 7f36c6c00640 1 -- 172.21.15.97:0/1778533456 >> 172.21.15.97:0/1778533456 conn(0x7f36c00d1c30 msgr2=0x7f36c00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.934+0000 7f36c6c00640 1 -- 172.21.15.97:0/1778533456 shutdown_connections 2024-09-10T21:27:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.934+0000 7f36c6c00640 1 -- 172.21.15.97:0/1778533456 wait complete. 2024-09-10T21:27:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 Processor -- start 2024-09-10T21:27:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 -- start start 2024-09-10T21:27:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 0x7f36c00b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c0161600 0x7f36c0154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c0155220 0x7f36c015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36c016c220 con 0x7f36c0155220 2024-09-10T21:27:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f36c016c0a0 con 0x7f36c0161600 2024-09-10T21:27:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f36c016c3a0 con 0x7f36c00b1590 2024-09-10T21:27:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c0155220 0x7f36c015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36bfe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 0x7f36c00b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c0155220 0x7f36c015b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48348/0 (socket says 172.21.15.97:48348) 2024-09-10T21:27:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 -- 172.21.15.97:0/1168908236 learned_addr learned my addr 172.21.15.97:0/1168908236 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36bfe00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 0x7f36c00b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:49388/0 (socket says 172.21.15.97:49388) 2024-09-10T21:27:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36b7400640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c0161600 0x7f36c0154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 -- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 msgr2=0x7f36c00b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 0x7f36c00b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 -- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c0161600 msgr2=0x7f36c0154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c0161600 0x7f36c0154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 -- 172.21.15.97:0/1168908236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c00c77e0 con 0x7f36c0155220 2024-09-10T21:27:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c4c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c0155220 0x7f36c015b8d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f36b8019730 tx=0x7f36b8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36bcc00640 1 -- 172.21.15.97:0/1168908236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80b2040 con 0x7f36c0155220 2024-09-10T21:27:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36bcc00640 1 -- 172.21.15.97:0/1168908236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36b8004040 con 0x7f36c0155220 2024-09-10T21:27:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c00c7ac0 con 0x7f36c0155220 2024-09-10T21:27:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36bcc00640 1 -- 172.21.15.97:0/1168908236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80044d0 con 0x7f36c0155220 2024-09-10T21:27:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.938+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c00b4c10 con 0x7f36c0155220 2024-09-10T21:27:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36bcc00640 1 -- 172.21.15.97:0/1168908236 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f36b80b5070 con 0x7f36c0155220 2024-09-10T21:27:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36bcc00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f369c072610 0x7f369c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36bcc00640 1 -- 172.21.15.97:0/1168908236 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f36b812d200 con 0x7f36c0155220 2024-09-10T21:27:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 Processor -- start 2024-09-10T21:27:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36bfe00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f369c072610 0x7f369c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 -- start start 2024-09-10T21:27:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36bfe00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f369c072610 0x7f369c074ad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f36b0001870 tx=0x7f36b0001380 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556434f4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556434f501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556434e703f0 con 0x556434f9c2f0 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556434dadb20 con 0x556434f9b9d0 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556434d9cf50 con 0x556434f4fe10 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.942+0000 7f36b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556434f4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556434f4f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48360/0 (socket says 172.21.15.97:48360) 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 -- 172.21.15.97:0/547967782 learned_addr learned my addr 172.21.15.97:0/547967782 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36bfe00640 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c4c00640 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556434f501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 -- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 msgr2=0x556434f501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556434f501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 -- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 msgr2=0x556434f9bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f9bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 -- 172.21.15.97:0/547967782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556434d64590 con 0x556434f9c2f0 2024-09-10T21:27:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556434f4f8d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f36a8001820 tx=0x7f36a8007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9ac18041c3cf4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b4c00640 1 -- 172.21.15.97:0/547967782 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a80052e0 con 0x556434f9c2f0 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b4c00640 1 -- 172.21.15.97:0/547967782 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a8018070 con 0x556434f9c2f0 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b4c00640 1 -- 172.21.15.97:0/547967782 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a800fa10 con 0x556434f9c2f0 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- 172.21.15.97:0/547967782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 msgr2=0x556434f4f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556434f4f8d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f36a8001820 tx=0x7f36a8007cf0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- 172.21.15.97:0/547967782 shutdown_connections 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556434f501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556434f4f8d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 --2- 172.21.15.97:0/547967782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f9bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- 172.21.15.97:0/547967782 >> 172.21.15.97:0/547967782 conn(0x556434e39130 msgr2=0x556434f7ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- 172.21.15.97:0/547967782 shutdown_connections 2024-09-10T21:27:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- 172.21.15.97:0/547967782 wait complete. 2024-09-10T21:27:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- start start 2024-09-10T21:27:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 Processor -- start 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f4fe10 0x556434f7c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f7ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f9c2f0 0x556434f7d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556434d375d0 con 0x556434f4fe10 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556434da0d60 con 0x556434f9b9d0 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556434d43500 con 0x556434f9c2f0 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c9f94000 10 client.0 _ll_register_callbacks cb 0x556434d47810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f4fe10 0x556434f7c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36bfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f4fe10 0x556434f7c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48364/0 (socket says 172.21.15.97:48364) 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36bfe00640 1 -- 172.21.15.97:0/1963990859 learned_addr learned my addr 172.21.15.97:0/1963990859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36c4c00640 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f9c2f0 0x556434f7d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.946+0000 7f36b7400640 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f7ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36bfe00640 1 -- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f9c2f0 msgr2=0x556434f7d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36bfe00640 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f9c2f0 0x556434f7d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36bfe00640 1 -- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 msgr2=0x556434f7ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36bfe00640 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f7ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36bfe00640 1 -- 172.21.15.97:0/1963990859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556434d3b170 con 0x556434f4fe10 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36bfe00640 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f4fe10 0x556434f7c6a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f36b00146d0 tx=0x7f36b0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8870428ee4355f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 1 -- 172.21.15.97:0/1963990859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0025070 con 0x556434f4fe10 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 1 -- 172.21.15.97:0/1963990859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f36b0015850 con 0x556434f4fe10 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556434d350f0 con 0x556434f4fe10 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 1 -- 172.21.15.97:0/1963990859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f36b0030050 con 0x556434f4fe10 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 10 client.6312 fetch_fsmap learned FSMap version 61 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x556434e21ef0 con 0x556434f4fe10 2024-09-10T21:27:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 1 -- 172.21.15.97:0/1963990859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556434d350f0 con 0x556434f4fe10 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 1 -- 172.21.15.97:0/1963990859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f36b0033020 con 0x556434f4fe10 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 10 client.6312 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 10 client.6312 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 20 client.6312 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434f7e0f0 0x556434fadc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 4 client.6312 mds_command: new command op to 5955 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 --> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556434dafa70 con 0x556434f7e0f0 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36b7400640 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434f7e0f0 0x556434fadc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36b7400640 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434f7e0f0 0x556434fadc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6525c5ff913eaa5a server_cookie=6ac3946df17ee78e in_seq=0 out_seq=0 2024-09-10T21:27:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 10 client.6312 ms_handle_connect on v2:172.21.15.194:6835/1482860240 2024-09-10T21:27:50.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 1 -- 172.21.15.97:0/1963990859 <== mds.0 v2:172.21.15.194:6835/1482860240 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556434dafa70 con 0x556434f7e0f0 2024-09-10T21:27:50.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 10 client.6312 handle_command_reply: tid=0 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 client.6312 shutdown 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f4fe10 msgr2=0x556434f7c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f4fe10 0x556434f7c6a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f36b00146d0 tx=0x7f36b0004630 comp rx=0 tx=0).stop 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 shutdown_connections 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434f7e0f0 0x556434fadc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f9c2f0 0x556434f7d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36ad600640 0 client.6312 ms_handle_reset on v2:172.21.15.194:6835/1482860240 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556434f7ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 --2- 172.21.15.97:0/1963990859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f4fe10 0x556434f7c6a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 >> 172.21.15.97:0/1963990859 conn(0x556434e39130 msgr2=0x556434f4efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 shutdown_connections 2024-09-10T21:27:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 1 -- 172.21.15.97:0/1963990859 wait complete. 2024-09-10T21:27:50.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.950+0000 7f36c9f94000 20 client.6312 trim_cache size 0 max 16384 2024-09-10T21:27:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 Processor -- start 2024-09-10T21:27:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 -- start start 2024-09-10T21:27:50.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x55643500dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9b9d0 0x55643500e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9c2f0 0x55643500ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36c0204590 con 0x556434f9b9d0 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f36c02035d0 con 0x556434f9c2f0 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f36c01540c0 con 0x556434f4fe10 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9b9d0 0x55643500e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9b9d0 0x55643500e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48374/0 (socket says 172.21.15.97:48374) 2024-09-10T21:27:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36b7400640 1 -- 172.21.15.97:0/3202191327 learned_addr learned my addr 172.21.15.97:0/3202191327 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36b7400640 1 -- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 msgr2=0x55643500dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36bfe00640 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x55643500dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.970+0000 7f36c4c00640 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9c2f0 0x55643500ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36b7400640 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x55643500dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36b7400640 1 -- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9c2f0 msgr2=0x55643500ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36b7400640 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9c2f0 0x55643500ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36b7400640 1 -- 172.21.15.97:0/3202191327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556434df8d90 con 0x556434f9b9d0 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36bfe00640 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x55643500dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9c2f0 0x55643500ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36b7400640 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9b9d0 0x55643500e520 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f36a8015040 tx=0x7f36a8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69475078eccce236 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36afe00640 1 -- 172.21.15.97:0/3202191327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a800f8b0 con 0x556434f9b9d0 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36afe00640 1 -- 172.21.15.97:0/3202191327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a802a020 con 0x556434f9b9d0 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36afe00640 1 -- 172.21.15.97:0/3202191327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a800fdc0 con 0x556434f9b9d0 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- 172.21.15.97:0/3202191327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9b9d0 msgr2=0x55643500e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9b9d0 0x55643500e520 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f36a8015040 tx=0x7f36a8000ee0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- 172.21.15.97:0/3202191327 shutdown_connections 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9c2f0 0x55643500ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9b9d0 0x55643500e520 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 --2- 172.21.15.97:0/3202191327 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x55643500dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- 172.21.15.97:0/3202191327 >> 172.21.15.97:0/3202191327 conn(0x556434e39130 msgr2=0x556434f57aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- 172.21.15.97:0/3202191327 shutdown_connections 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- 172.21.15.97:0/3202191327 wait complete. 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- start start 2024-09-10T21:27:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 Processor -- start 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556435011e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556435012350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556435012890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556434fb2ce0 con 0x556434f9c2f0 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556434da0070 con 0x556434f9b9d0 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556434e5c730 con 0x556434f4fe10 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c9f94000 10 client.0 _ll_register_callbacks cb 0x7f36b000f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556435012890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556435012890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48380/0 (socket says 172.21.15.97:48380) 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 -- 172.21.15.97:0/2060488320 learned_addr learned my addr 172.21.15.97:0/2060488320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36bfe00640 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556435011e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36b7400640 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556435012350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 -- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 msgr2=0x556435011e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556435011e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 -- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 msgr2=0x556435012350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556435012350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 -- 172.21.15.97:0/2060488320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556434d63a90 con 0x556434f9c2f0 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36c4c00640 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556435012890 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f36b8098730 tx=0x7f36b80f3e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1da9a41e62372c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.974+0000 7f36ad600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 1 -- 172.21.15.97:0/2060488320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80f7730 con 0x556434f9c2f0 2024-09-10T21:27:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 1 -- 172.21.15.97:0/2060488320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6273+0+0 (secure 0 0 0) 0x7f36b80a5b00 con 0x556434f9c2f0 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a802a020 con 0x556434f9c2f0 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 1 -- 172.21.15.97:0/2060488320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f36b8004870 con 0x556434f9c2f0 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 10 client.6318 fetch_fsmap learned FSMap version 61 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x556434d42050 con 0x556434f9c2f0 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 1 -- 172.21.15.97:0/2060488320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a802a020 con 0x556434f9c2f0 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 1 -- 172.21.15.97:0/2060488320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f36b80ffce0 con 0x556434f9c2f0 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 10 client.6318 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 10 client.6318 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 20 client.6318 populate_metadata read hostname 'smithi097' 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434fb1620 0x556434fb0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 4 client.6318 mds_command: new command op to 5955 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 --> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] -- command(tid 0: {"prefix": "status"}) -- 0x556434dab520 con 0x556434fb1620 2024-09-10T21:27:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36bfe00640 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434fb1620 0x556434fb0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36bfe00640 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434fb1620 0x556434fb0740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc20fad9e20fea3 server_cookie=4ad7ab697f0fdc5d in_seq=0 out_seq=0 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 10 client.6318 ms_handle_connect on v2:172.21.15.194:6835/1482860240 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 1 -- 172.21.15.97:0/2060488320 <== mds.0 v2:172.21.15.194:6835/1482860240 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x556434dab520 con 0x556434fb1620 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 10 client.6318 handle_command_reply: tid=0 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 client.6318 shutdown 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 msgr2=0x556435012890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556435012890 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f36b8098730 tx=0x7f36b80f3e60 comp rx=0 tx=0).stop 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 shutdown_connections 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x556434fb1620 0x556434fb0740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556434f9c2f0 0x556435012890 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556434f9b9d0 0x556435012350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36ad600640 0 client.6318 ms_handle_reset on v2:172.21.15.194:6835/1482860240 2024-09-10T21:27:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 --2- 172.21.15.97:0/2060488320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556434f4fe10 0x556435011e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 >> 172.21.15.97:0/2060488320 conn(0x556434e39130 msgr2=0x556434f9a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 shutdown_connections 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 1 -- 172.21.15.97:0/2060488320 wait complete. 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c9f94000 20 client.6318 trim_cache size 0 max 16384 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5955, 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:27:50.986 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:27:50.987 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 4.7915646799999996, 2024-09-10T21:27:50.987 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 61, 2024-09-10T21:27:50.987 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 69, 2024-09-10T21:27:50.987 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 69, 2024-09-10T21:27:50.987 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 62.122803701999999 2024-09-10T21:27:50.987 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f369c072610 msgr2=0x7f369c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f369c072610 0x7f369c074ad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f36b0001870 tx=0x7f36b0001380 comp rx=0 tx=0).stop 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c0155220 msgr2=0x7f36c015b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c0155220 0x7f36c015b8d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f36b8019730 tx=0x7f36b8003f60 comp rx=0 tx=0).stop 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 shutdown_connections 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f369c072610 0x7f369c074ad0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36c0155220 0x7f36c015b8d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c0161600 0x7f36c0154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 --2- 172.21.15.97:0/1168908236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c00b1590 0x7f36c00b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 >> 172.21.15.97:0/1168908236 conn(0x7f36c00d1c30 msgr2=0x7f36c00708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 shutdown_connections 2024-09-10T21:27:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:27:50.978+0000 7f36c6c00640 1 -- 172.21.15.97:0/1168908236 wait complete. 2024-09-10T21:28:03.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:03.362+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:03.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:03.362+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:28:03.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:03.362+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:28:03.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:03.362+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:28:03.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:03.362+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T21:28:09.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:09.017 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:18.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:18.378+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:18.378+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:28:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:18.378+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:28:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:18.378+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:28:18.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:18.378+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T21:28:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:33.394+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:33.394+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:28:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:33.394+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:28:33.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:33.394+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:28:33.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:33.394+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T21:28:39.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:39.074 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:48.397+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:48.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:48.397+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:28:48.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:48.397+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:28:48.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:48.397+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:28:48.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:28:48.397+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T21:29:03.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:03.413+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:03.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:03.413+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:29:03.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:03.413+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:29:03.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:03.413+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:03.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:03.413+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T21:29:09.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:09.124 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:18.429+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:18.429+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:18.429+0000 7fa1cbe00640 1 -- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 msgr2=0x55f0fe308180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:18.429+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:18.429+0000 7fa1cbe00640 1 --2- 172.21.15.97:0/662109799 >> [v2:172.21.15.97:6834/3391319282,v1:172.21.15.97:6835/3391319282] conn(0x55f0fe2f5050 0x55f0fe308180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T21:29:22.642 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-09-10T21:29:22.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:29:22.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.789+0000 7f8dd9200640 1 -- 172.21.15.97:0/3590286677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 msgr2=0x7f8dd40b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.789+0000 7f8dd9200640 1 --2- 172.21.15.97:0/3590286677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd40b2a80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40725c0 tx=0x7f8dbc01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.789+0000 7f8dd9200640 1 -- 172.21.15.97:0/3590286677 shutdown_connections 2024-09-10T21:29:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.789+0000 7f8dd9200640 1 --2- 172.21.15.97:0/3590286677 >> v2:172.21.15.194:3300/0 conn(0x7f8dd406fd50 0x7f8dd40701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.789+0000 7f8dd9200640 1 --2- 172.21.15.97:0/3590286677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd40b2a80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.789+0000 7f8dd9200640 1 -- 172.21.15.97:0/3590286677 >> 172.21.15.97:0/3590286677 conn(0x7f8dd40b1780 msgr2=0x7f8dd40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 -- 172.21.15.97:0/3590286677 shutdown_connections 2024-09-10T21:29:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 -- 172.21.15.97:0/3590286677 wait complete. 2024-09-10T21:29:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 Processor -- start 2024-09-10T21:29:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 -- start start 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8dd406fd50 0x7f8dd415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dd4143f90 0x7f8dd41fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd415e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd4167c40 con 0x7f8dd4144690 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd4167ac0 con 0x7f8dd4143f90 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd9200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd4167dc0 con 0x7f8dd406fd50 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42350/0 (socket says 172.21.15.97:42350) 2024-09-10T21:29:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 -- 172.21.15.97:0/172452015 learned_addr learned my addr 172.21.15.97:0/172452015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd1e00640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8dd406fd50 0x7f8dd415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dc9400640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dd4143f90 0x7f8dd41fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 -- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8dd406fd50 msgr2=0x7f8dd415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8dd406fd50 0x7f8dd415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 -- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dd4143f90 msgr2=0x7f8dd41fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dd4143f90 0x7f8dd41fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 -- 172.21.15.97:0/172452015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd415d7d0 con 0x7f8dd4144690 2024-09-10T21:29:22.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.793+0000 7f8dd2800640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd415e240 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00eb20 tx=0x7f8dcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dcaa00640 1 -- 172.21.15.97:0/172452015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc004b30 con 0x7f8dd4144690 2024-09-10T21:29:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dcaa00640 1 -- 172.21.15.97:0/172452015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dcc004cf0 con 0x7f8dd4144690 2024-09-10T21:29:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dcaa00640 1 -- 172.21.15.97:0/172452015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc005280 con 0x7f8dd4144690 2024-09-10T21:29:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd415bd30 con 0x7f8dd4144690 2024-09-10T21:29:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd415da40 con 0x7f8dd4144690 2024-09-10T21:29:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd40701a0 con 0x7f8dd4144690 2024-09-10T21:29:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dcaa00640 1 -- 172.21.15.97:0/172452015 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8dcc005420 con 0x7f8dd4144690 2024-09-10T21:29:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dcaa00640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8db00725e0 0x7f8db0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dcaa00640 1 -- 172.21.15.97:0/172452015 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f8dcc094120 con 0x7f8dd4144690 2024-09-10T21:29:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.797+0000 7f8dd1e00640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8db00725e0 0x7f8db0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.801+0000 7f8dd1e00640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8db00725e0 0x7f8db0074aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4007fd0 tx=0x7f8dc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:22.801+0000 7f8dcaa00640 1 -- 172.21.15.97:0/172452015 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8dcc05eea0 con 0x7f8dd4144690 2024-09-10T21:29:23.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8dd4165630 con 0x7f8dd4144690 2024-09-10T21:29:23.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dcaa00640 1 -- 172.21.15.97:0/172452015 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7f8dcc0638d0 con 0x7f8dd4144690 2024-09-10T21:29:23.008 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:29:23.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8db00725e0 msgr2=0x7f8db0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:23.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8db00725e0 0x7f8db0074aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4007fd0 tx=0x7f8dc4009040 comp rx=0 tx=0).stop 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 msgr2=0x7f8dd415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd415e240 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00eb20 tx=0x7f8dcc00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 shutdown_connections 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8db00725e0 0x7f8db0074aa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd4144690 0x7f8dd415e240 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8dd4143f90 0x7f8dd41fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 --2- 172.21.15.97:0/172452015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8dd406fd50 0x7f8dd415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.005+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 >> 172.21.15.97:0/172452015 conn(0x7f8dd40b1780 msgr2=0x7f8dd40b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.009+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 shutdown_connections 2024-09-10T21:29:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23.009+0000 7f8dd9200640 1 -- 172.21.15.97:0/172452015 wait complete. 2024-09-10T21:29:23.030 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:29:23.030 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:29:23.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:29:23.030 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:29:23.031 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:23.031 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:29:23.031 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:29:23.031 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:29:23.031 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:29:23.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:23.031 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:23.041 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.042 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:29:23.043 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:29:23.044 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.044 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:29:23.044 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:29:23.044 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:29:23.044 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:23.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:23.044 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:29:23.103 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:29:23.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:23.105 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:29:23.158 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:29:23.159 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:29:23.159 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:29:23.159 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:29:23.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:29:23.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:29:23.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:29:23.159 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:29:23.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:29:23.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:29:23.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:23.299 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:29:23.309 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:29:23.310 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:29:23.310 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:29:23.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:23.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:29:23.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:29:23.370 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:23.427 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:29:23.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:29:23.427 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:23.479 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:29:23.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:23.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:23.574 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:23.574 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:29:23.574 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:29:23.590 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:23.590 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:29:23.590 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:29:23.590 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:29:23.590 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:29:23.600 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:23.600 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:29:23.600 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:29:23.601 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:29:23.601 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:29:23.601 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:29:23.601 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:29:23.601 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:29:23.604 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:29:23.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:23.604 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:23.626 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:29:23.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:29:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:29:23.727 INFO:teuthology.orchestra.run.smithi097.stdout:client6327 2024-09-10T21:29:23.728 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:29:23.728 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:23.737 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-10T21:29:23.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:29:23.832 INFO:teuthology.orchestra.run.smithi097.stdout: File: a/ 2024-09-10T21:29:23.832 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-10T21:29:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 2ch/44d Inode: 1099511627776 Links: 4 2024-09-10T21:29:23.832 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2024-09-10T21:29:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-10 21:27:07.546867279 +0000 2024-09-10T21:29:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-10 21:27:07.550867239 +0000 2024-09-10T21:29:23.833 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-10 21:27:07.550867239 +0000 2024-09-10T21:29:23.833 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-10 21:27:07.546867279 +0000 2024-09-10T21:29:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:a/ 2024-09-10T21:29:23.839 INFO:teuthology.orchestra.run.smithi097.stdout:a/c 2024-09-10T21:29:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:a/c/d 2024-09-10T21:29:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:a/b 2024-09-10T21:29:23.840 INFO:teuthology.orchestra.run.smithi097.stdout:a/b/d 2024-09-10T21:29:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:29:23.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-10T21:29:24.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 -- 172.21.15.97:0/431428367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 msgr2=0x7f605006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 --2- 172.21.15.97:0/431428367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 0x7f605006f090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f603800a980 tx=0x7f6038020340 comp rx=0 tx=0).stop 2024-09-10T21:29:24.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 -- 172.21.15.97:0/431428367 shutdown_connections 2024-09-10T21:29:24.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 --2- 172.21.15.97:0/431428367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60500ceee0 0x7f60500cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 --2- 172.21.15.97:0/431428367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 0x7f605006f090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 -- 172.21.15.97:0/431428367 >> 172.21.15.97:0/431428367 conn(0x7f60500b1780 msgr2=0x7f60500b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 -- 172.21.15.97:0/431428367 shutdown_connections 2024-09-10T21:29:24.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.033+0000 7f6056200640 1 -- 172.21.15.97:0/431428367 wait complete. 2024-09-10T21:29:24.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 Processor -- start 2024-09-10T21:29:24.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 -- start start 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 0x7f6050154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60500ceee0 0x7f60501405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60501511f0 0x7f60500a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f605006e740 con 0x7f60501511f0 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f605006e5c0 con 0x7f605006ec40 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f605006e8c0 con 0x7f60500ceee0 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60501511f0 0x7f60500a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60501511f0 0x7f60500a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42364/0 (socket says 172.21.15.97:42364) 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 -- 172.21.15.97:0/1540776805 learned_addr learned my addr 172.21.15.97:0/1540776805 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604f400640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 0x7f6050154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6046a00640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60500ceee0 0x7f60501405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 -- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60500ceee0 msgr2=0x7f60501405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60500ceee0 0x7f60501405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 -- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 msgr2=0x7f6050154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 0x7f6050154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 -- 172.21.15.97:0/1540776805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605015b900 con 0x7f60501511f0 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f604fe00640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60501511f0 0x7f60500a94a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6048009fb0 tx=0x7f604800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6047e00640 1 -- 172.21.15.97:0/1540776805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048003980 con 0x7f60501511f0 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6047e00640 1 -- 172.21.15.97:0/1540776805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6048003c30 con 0x7f60501511f0 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6047e00640 1 -- 172.21.15.97:0/1540776805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048004280 con 0x7f60501511f0 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60500b0470 con 0x7f60501511f0 2024-09-10T21:29:24.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.037+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6050152370 con 0x7f60501511f0 2024-09-10T21:29:24.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6047e00640 1 -- 172.21.15.97:0/1540776805 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f604800b040 con 0x7f60501511f0 2024-09-10T21:29:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6047e00640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c0725e0 0x7f602c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6047e00640 1 -- 172.21.15.97:0/1540776805 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f6048093ae0 con 0x7f60501511f0 2024-09-10T21:29:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 Processor -- start 2024-09-10T21:29:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f604f400640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c0725e0 0x7f602c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 -- start start 2024-09-10T21:29:24.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f604f400640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c0725e0 0x7f602c074aa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6040007fd0 tx=0x7f6040009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e0861a0 0x55651e086580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e03c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e03cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55651df24cb0 con 0x55651e086ac0 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55651de3a310 con 0x55651e0861a0 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55651de97b90 con 0x55651e03cbc0 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e03c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f604f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e0861a0 0x55651e086580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e03c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42366/0 (socket says 172.21.15.97:42366) 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 -- 172.21.15.97:0/1063103507 learned_addr learned my addr 172.21.15.97:0/1063103507 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:24.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f604fe00640 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e03cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 -- 172.21.15.97:0/1063103507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 msgr2=0x55651e03cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e03cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 -- 172.21.15.97:0/1063103507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e0861a0 msgr2=0x55651e086580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e0861a0 0x55651e086580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.041+0000 7f6046a00640 1 -- 172.21.15.97:0/1063103507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651de3bc60 con 0x55651e086ac0 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f6046a00640 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e03c680 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f603800a8a0 tx=0x7f603801d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae51c5bebbd66490 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f603fe00640 1 -- 172.21.15.97:0/1063103507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60380b5040 con 0x55651e086ac0 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f603fe00640 1 -- 172.21.15.97:0/1063103507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60380b6040 con 0x55651e086ac0 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f603fe00640 1 -- 172.21.15.97:0/1063103507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038005290 con 0x55651e086ac0 2024-09-10T21:29:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- 172.21.15.97:0/1063103507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 msgr2=0x55651e03c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e03c680 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f603800a8a0 tx=0x7f603801d0e0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- 172.21.15.97:0/1063103507 shutdown_connections 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e03cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e03c680 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 --2- 172.21.15.97:0/1063103507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e0861a0 0x55651e086580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- 172.21.15.97:0/1063103507 >> 172.21.15.97:0/1063103507 conn(0x55651e042d20 msgr2=0x55651e03af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- 172.21.15.97:0/1063103507 shutdown_connections 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- 172.21.15.97:0/1063103507 wait complete. 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- start start 2024-09-10T21:29:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 Processor -- start 2024-09-10T21:29:24.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e03cbc0 0x55651e08d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e08da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e08dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55651de8add0 con 0x55651e03cbc0 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55651de86fc0 con 0x55651e086ac0 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55651de2c4b0 con 0x55651e0861a0 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f60596d5000 10 client.0 _ll_register_callbacks cb 0x55651de0a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e03cbc0 0x55651e08d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e03cbc0 0x55651e08d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42382/0 (socket says 172.21.15.97:42382) 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 -- 172.21.15.97:0/2909690567 learned_addr learned my addr 172.21.15.97:0/2909690567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604fe00640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e08dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f6046a00640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e08da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 -- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 msgr2=0x55651e08da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e08da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 -- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 msgr2=0x55651e08dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e08dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 -- 172.21.15.97:0/2909690567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651de251f0 con 0x55651e03cbc0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f6046a00640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e08da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604fe00640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e08dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f604f400640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e03cbc0 0x55651e08d530 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f60400146d0 tx=0x7f6040000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f54a4a8bebb2a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.045+0000 7f603cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 1 -- 172.21.15.97:0/2909690567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040025020 con 0x55651e03cbc0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 1 -- 172.21.15.97:0/2909690567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f6040013700 con 0x55651e03cbc0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55651de1f330 con 0x55651e03cbc0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 1 -- 172.21.15.97:0/2909690567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f604002f050 con 0x55651e03cbc0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 10 client.6336 fetch_fsmap learned FSMap version 61 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55651de0bfa0 con 0x55651e03cbc0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 1 -- 172.21.15.97:0/2909690567 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55651de1f330 con 0x55651e03cbc0 2024-09-10T21:29:24.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 1 -- 172.21.15.97:0/2909690567 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f6040032020 con 0x55651e03cbc0 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 10 client.6336 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 10 client.6336 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 20 client.6336 populate_metadata read hostname 'smithi097' 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e095900 0x55651e097cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 4 client.6336 mds_command: new command op to 5955 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 --> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55651de99ae0 con 0x55651e095900 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f6046a00640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e095900 0x55651e097cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f6046a00640 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e095900 0x55651e097cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f048792c8e3980 server_cookie=e69444e50620d7d9 in_seq=0 out_seq=0 2024-09-10T21:29:24.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 10 client.6336 ms_handle_connect on v2:172.21.15.194:6835/1482860240 2024-09-10T21:29:24.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 1 -- 172.21.15.97:0/2909690567 <== mds.0 v2:172.21.15.194:6835/1482860240 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55651de99ae0 con 0x55651e095900 2024-09-10T21:29:24.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 10 client.6336 handle_command_reply: tid=0 2024-09-10T21:29:24.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 client.6336 shutdown 2024-09-10T21:29:24.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e03cbc0 msgr2=0x55651e08d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e03cbc0 0x55651e08d530 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f60400146d0 tx=0x7f6040000ee0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 shutdown_connections 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e095900 0x55651e097cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e08dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e08da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 --2- 172.21.15.97:0/2909690567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e03cbc0 0x55651e08d530 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f603cc00640 0 client.6336 ms_handle_reset on v2:172.21.15.194:6835/1482860240 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 >> 172.21.15.97:0/2909690567 conn(0x55651e042d20 msgr2=0x55651e03af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 shutdown_connections 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 1 -- 172.21.15.97:0/2909690567 wait complete. 2024-09-10T21:29:24.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.049+0000 7f60596d5000 20 client.6336 trim_cache size 0 max 16384 2024-09-10T21:29:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 Processor -- start 2024-09-10T21:29:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 -- start start 2024-09-10T21:29:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e03cbc0 0x55651e03c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e03c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e090310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6050201c60 con 0x55651e086ac0 2024-09-10T21:29:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6050200c80 con 0x55651e03cbc0 2024-09-10T21:29:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f60501537e0 con 0x55651e0861a0 2024-09-10T21:29:24.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e090310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e090310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42390/0 (socket says 172.21.15.97:42390) 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 -- 172.21.15.97:0/1739098185 learned_addr learned my addr 172.21.15.97:0/1739098185 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604f400640 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e03cbc0 0x55651e03c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f6046a00640 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e03c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 -- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 msgr2=0x55651e03c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e03c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 -- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e03cbc0 msgr2=0x55651e03c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e03cbc0 0x55651e03c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f604fe00640 1 -- 172.21.15.97:0/1739098185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651dee4660 con 0x55651e086ac0 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.065+0000 7f6046a00640 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e03c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:24.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f604fe00640 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e090310 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6048009330 tx=0x7f6048063190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50d4452ffceef7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f603f400640 1 -- 172.21.15.97:0/1739098185 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604805e860 con 0x55651e086ac0 2024-09-10T21:29:24.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f603f400640 1 -- 172.21.15.97:0/1739098185 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6048063290 con 0x55651e086ac0 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f603f400640 1 -- 172.21.15.97:0/1739098185 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048067cb0 con 0x55651e086ac0 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- 172.21.15.97:0/1739098185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 msgr2=0x55651e090310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e090310 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6048009330 tx=0x7f6048063190 comp rx=0 tx=0).stop 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- 172.21.15.97:0/1739098185 shutdown_connections 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e086ac0 0x55651e090310 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e0861a0 0x55651e03c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 --2- 172.21.15.97:0/1739098185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e03cbc0 0x55651e03c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- 172.21.15.97:0/1739098185 >> 172.21.15.97:0/1739098185 conn(0x55651e042d20 msgr2=0x55651e03b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- 172.21.15.97:0/1739098185 shutdown_connections 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- 172.21.15.97:0/1739098185 wait complete. 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- start start 2024-09-10T21:29:24.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 Processor -- start 2024-09-10T21:29:24.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e092fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e0861a0 0x55651e093500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e0b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55651e045d40 con 0x55651e0861a0 2024-09-10T21:29:24.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55651e043100 con 0x55651e086ac0 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55651de81ec0 con 0x55651e03cbc0 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f60596d5000 10 client.0 _ll_register_callbacks cb 0x7f6040001450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e0861a0 0x55651e093500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e0861a0 0x55651e093500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42392/0 (socket says 172.21.15.97:42392) 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 -- 172.21.15.97:0/3923985194 learned_addr learned my addr 172.21.15.97:0/3923985194 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 -- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 msgr2=0x55651e092fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f604f400640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e092fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f604fe00640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e0b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e092fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 -- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 msgr2=0x55651e0b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e0b8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 -- 172.21.15.97:0/3923985194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651de51090 con 0x55651e0861a0 2024-09-10T21:29:24.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f604f400640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e092fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f604fe00640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e0b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f6046a00640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e0861a0 0x55651e093500 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6038027090 tx=0x7f603800a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bf17e0c6757476 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.069+0000 7f603cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f603cc00640 1 -- 172.21.15.97:0/3923985194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60380b5040 con 0x55651e0861a0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f603cc00640 1 -- 172.21.15.97:0/3923985194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f6038005230 con 0x55651e0861a0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6048063290 con 0x55651e0861a0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f603cc00640 1 -- 172.21.15.97:0/3923985194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f60380d9050 con 0x55651e0861a0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 10 client.6342 fetch_fsmap learned FSMap version 61 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55651de1d5f0 con 0x55651e0861a0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f603cc00640 1 -- 172.21.15.97:0/3923985194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6048063290 con 0x55651e0861a0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f603cc00640 1 -- 172.21.15.97:0/3923985194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f60380dc020 con 0x55651e0861a0 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 10 client.6342 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:29:24.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 10 client.6342 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-10T21:29:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 20 client.6342 populate_metadata read hostname 'smithi097' 2024-09-10T21:29:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e0b8c00 0x55651e103680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 4 client.6342 mds_command: new command op to 5955 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:29:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 --> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55651de95590 con 0x55651e0b8c00 2024-09-10T21:29:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f604f400640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e0b8c00 0x55651e103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f604f400640 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e0b8c00 0x55651e103680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b924b1e84ba831cf server_cookie=33d78bcec8da0f50 in_seq=0 out_seq=0 2024-09-10T21:29:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.073+0000 7f603cc00640 10 client.6342 ms_handle_connect on v2:172.21.15.194:6835/1482860240 2024-09-10T21:29:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f603cc00640 1 -- 172.21.15.97:0/3923985194 <== mds.0 v2:172.21.15.194:6835/1482860240 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55651de95590 con 0x55651e0b8c00 2024-09-10T21:29:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f603cc00640 10 client.6342 handle_command_reply: tid=0 2024-09-10T21:29:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 client.6342 shutdown 2024-09-10T21:29:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e0861a0 msgr2=0x55651e093500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e0861a0 0x55651e093500 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6038027090 tx=0x7f603800a2f0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 shutdown_connections 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:6835/1482860240,v1:172.21.15.194:6837/1482860240] conn(0x55651e0b8c00 0x55651e103680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651e086ac0 0x55651e0b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651e0861a0 0x55651e093500 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f603cc00640 0 client.6342 ms_handle_reset on v2:172.21.15.194:6835/1482860240 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 --2- 172.21.15.97:0/3923985194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651e03cbc0 0x55651e092fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 >> 172.21.15.97:0/3923985194 conn(0x55651e042d20 msgr2=0x55651e03c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 shutdown_connections 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 1 -- 172.21.15.97:0/3923985194 wait complete. 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f60596d5000 20 client.6342 trim_cache size 0 max 16384 2024-09-10T21:29:24.090 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:29:24.091 INFO:teuthology.orchestra.run.smithi097.stdout: "message": "", 2024-09-10T21:29:24.091 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-09-10T21:29:24.091 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:29:24.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c0725e0 msgr2=0x7f602c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c0725e0 0x7f602c074aa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6040007fd0 tx=0x7f6040009040 comp rx=0 tx=0).stop 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60501511f0 msgr2=0x7f60500a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60501511f0 0x7f60500a94a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6048009fb0 tx=0x7f604800c800 comp rx=0 tx=0).stop 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 shutdown_connections 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c0725e0 0x7f602c074aa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60501511f0 0x7f60500a94a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f60500ceee0 0x7f60501405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 --2- 172.21.15.97:0/1540776805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f605006ec40 0x7f6050154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 >> 172.21.15.97:0/1540776805 conn(0x7f60500b1780 msgr2=0x7f605015d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 shutdown_connections 2024-09-10T21:29:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.085+0000 7f6056200640 1 -- 172.21.15.97:0/1540776805 wait complete. 2024-09-10T21:29:24.108 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:29:24.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:29:24.173 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:29:24.269 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:29:24.269 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:29:24.269 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:29:24.269 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:24.278 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:29:24.278 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:24.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:29:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.517+0000 7f32ae400640 1 -- 172.21.15.97:0/4065647621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 msgr2=0x7f328c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.517+0000 7f32ae400640 1 --2- 172.21.15.97:0/4065647621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 0x7f328c004810 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3294009b60 tx=0x7f329401d270 comp rx=0 tx=0).stop 2024-09-10T21:29:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.517+0000 7f32a7e00640 1 -- 172.21.15.97:0/4065647621 reap_dead start 2024-09-10T21:29:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.517+0000 7f32ae400640 1 -- 172.21.15.97:0/4065647621 shutdown_connections 2024-09-10T21:29:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.517+0000 7f32ae400640 1 -- 172.21.15.97:0/4065647621 >> 172.21.15.97:0/4065647621 conn(0x7f32a80b1780 msgr2=0x7f32a80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.517+0000 7f32ae400640 1 -- 172.21.15.97:0/4065647621 shutdown_connections 2024-09-10T21:29:24.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.517+0000 7f32ae400640 1 -- 172.21.15.97:0/4065647621 wait complete. 2024-09-10T21:29:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 Processor -- start 2024-09-10T21:29:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 -- start start 2024-09-10T21:29:24.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 0x7f32a80ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32a813d2a0 0x7f32a813d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32a81407c0 0x7f32a8140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81444d0 con 0x7f328c003fe0 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8070670 con 0x7f32a81407c0 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f32a80b2a80 con 0x7f32a813d2a0 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 0x7f32a80ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 0x7f32a80ab100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42408/0 (socket says 172.21.15.97:42408) 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 -- 172.21.15.97:0/613142652 learned_addr learned my addr 172.21.15.97:0/613142652 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7e00640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32a81407c0 0x7f32a8140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a6a00640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32a813d2a0 0x7f32a813d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 -- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32a813d2a0 msgr2=0x7f32a813d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32a813d2a0 0x7f32a813d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 -- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32a81407c0 msgr2=0x7f32a8140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32a81407c0 0x7f32a8140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 -- 172.21.15.97:0/613142652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a81f2a10 con 0x7f328c003fe0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32a7400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 0x7f32a80ab100 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3298011ae0 tx=0x7f3298009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f329f400640 1 -- 172.21.15.97:0/613142652 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329801e070 con 0x7f328c003fe0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f329f400640 1 -- 172.21.15.97:0/613142652 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f329800fe40 con 0x7f328c003fe0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f329f400640 1 -- 172.21.15.97:0/613142652 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32980042b0 con 0x7f328c003fe0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a81f2cf0 con 0x7f328c003fe0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.521+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81f26e0 con 0x7f328c003fe0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.525+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a80b4120 con 0x7f328c003fe0 2024-09-10T21:29:24.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.525+0000 7f329f400640 1 -- 172.21.15.97:0/613142652 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3298003600 con 0x7f328c003fe0 2024-09-10T21:29:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.525+0000 7f329f400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32880725e0 0x7f3288074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.525+0000 7f329f400640 1 -- 172.21.15.97:0/613142652 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f32980942b0 con 0x7f328c003fe0 2024-09-10T21:29:24.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.525+0000 7f32a6a00640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32880725e0 0x7f3288074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:24.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.529+0000 7f32a6a00640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32880725e0 0x7f3288074aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32940ae000 tx=0x7f329401a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:24.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.529+0000 7f329f400640 1 -- 172.21.15.97:0/613142652 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f329805f060 con 0x7f328c003fe0 2024-09-10T21:29:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.705+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32a80b13f0 con 0x7f328c003fe0 2024-09-10T21:29:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.705+0000 7f329f400640 1 -- 172.21.15.97:0/613142652 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v71) ==== 80+0+244 (secure 0 0 0) 0x7f3298063a90 con 0x7f328c003fe0 2024-09-10T21:29:24.709 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3391319282 2024-09-11T21:27:46.185567+0000 2024-09-10T21:29:24.709 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3391319282 2024-09-11T21:27:46.185567+0000 2024-09-10T21:29:24.710 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2609945029 2024-09-11T21:27:10.383537+0000 2024-09-10T21:29:24.710 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2609945029 2024-09-11T21:27:10.383537+0000 2024-09-10T21:29:24.710 INFO:teuthology.orchestra.run.smithi097.stderr:listed 4 entries 2024-09-10T21:29:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32880725e0 msgr2=0x7f3288074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32880725e0 0x7f3288074aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32940ae000 tx=0x7f329401a010 comp rx=0 tx=0).stop 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 msgr2=0x7f32a80ab100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 0x7f32a80ab100 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3298011ae0 tx=0x7f3298009ea0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 shutdown_connections 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32880725e0 0x7f3288074aa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32a81407c0 0x7f32a8140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32a813d2a0 0x7f32a813d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 --2- 172.21.15.97:0/613142652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c003fe0 0x7f32a80ab100 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 >> 172.21.15.97:0/613142652 conn(0x7f32a80b1780 msgr2=0x7f32a80730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 shutdown_connections 2024-09-10T21:29:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:24.709+0000 7f32ae400640 1 -- 172.21.15.97:0/613142652 wait complete. 2024-09-10T21:29:24.730 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:29:24.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:24.731 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:24.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:29:24.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:24.756 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:29:24.810 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:29:24.810 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:29:24.810 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:24.820 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:29:24.820 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:29:24.820 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:29:24.820 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:24.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:29:24.866 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:29:24.866 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:29:24.866 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:24.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:29:24.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 -- 172.21.15.97:0/3999222210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40b1560 msgr2=0x7fb6c40b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 --2- 172.21.15.97:0/3999222210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40b1560 0x7fb6c40b19b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009c80 tx=0x7fb6bc01f5a0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 -- 172.21.15.97:0/3999222210 shutdown_connections 2024-09-10T21:29:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 --2- 172.21.15.97:0/3999222210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40b1560 0x7fb6c40b19b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 --2- 172.21.15.97:0/3999222210 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6c40cec10 0x7fb6c40cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 -- 172.21.15.97:0/3999222210 >> 172.21.15.97:0/3999222210 conn(0x7fb6c40b37c0 msgr2=0x7fb6c40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:25.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 -- 172.21.15.97:0/3999222210 shutdown_connections 2024-09-10T21:29:25.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.045+0000 7fb6c9600640 1 -- 172.21.15.97:0/3999222210 wait complete. 2024-09-10T21:29:25.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 Processor -- start 2024-09-10T21:29:25.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 -- start start 2024-09-10T21:29:25.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6c40b1560 0x7fb6c40b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6c40ce830 0x7fb6c41f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40cec10 0x7fb6c40b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c40ce1a0 con 0x7fb6c40cec10 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c40cde60 con 0x7fb6c40b1560 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c40ce320 con 0x7fb6c40ce830 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40cec10 0x7fb6c40b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40cec10 0x7fb6c40b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42420/0 (socket says 172.21.15.97:42420) 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 -- 172.21.15.97:0/488220925 learned_addr learned my addr 172.21.15.97:0/488220925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c2a00640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6c40b1560 0x7fb6c40b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6ba000640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6c40ce830 0x7fb6c41f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 -- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6c40ce830 msgr2=0x7fb6c41f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6c40ce830 0x7fb6c41f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 -- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6c40b1560 msgr2=0x7fb6c40b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6c40b1560 0x7fb6c40b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 -- 172.21.15.97:0/488220925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c41f2f00 con 0x7fb6c40cec10 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c3400640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40cec10 0x7fb6c40b71e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc01fe40 tx=0x7fb6bc0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6bb400640 1 -- 172.21.15.97:0/488220925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0af8a0 con 0x7fb6c40cec10 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6bb400640 1 -- 172.21.15.97:0/488220925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6bc0afa40 con 0x7fb6c40cec10 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6bb400640 1 -- 172.21.15.97:0/488220925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0afe30 con 0x7fb6c40cec10 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c41f31e0 con 0x7fb6c40cec10 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.049+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41e4a30 con 0x7fb6c40cec10 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.053+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c40cf800 con 0x7fb6c40cec10 2024-09-10T21:29:25.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.053+0000 7fb6bb400640 1 -- 172.21.15.97:0/488220925 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb6bc004040 con 0x7fb6c40cec10 2024-09-10T21:29:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.053+0000 7fb6bb400640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6a00725e0 0x7fb6a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.053+0000 7fb6bb400640 1 -- 172.21.15.97:0/488220925 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7fb6bc1322a0 con 0x7fb6c40cec10 2024-09-10T21:29:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.053+0000 7fb6c2a00640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6a00725e0 0x7fb6a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.057+0000 7fb6c2a00640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6a00725e0 0x7fb6a0074aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4007fd0 tx=0x7fb6b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.057+0000 7fb6bb400640 1 -- 172.21.15.97:0/488220925 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6bc0ac050 con 0x7fb6c40cec10 2024-09-10T21:29:25.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.257+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c40b5000 con 0x7fb6c40cec10 2024-09-10T21:29:25.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.257+0000 7fb6bb400640 1 -- 172.21.15.97:0/488220925 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5232 (secure 0 0 0) 0x7fb6bc019100 con 0x7fb6c40cec10 2024-09-10T21:29:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:25.263 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"btime":"2024-09-10T21:27:49:202694+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2360657843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2360657843},{"type":"v1","addr":"172.21.15.97:6835","nonce":2360657843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:49.202684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":18,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5955,"qdb_cluster":[5955]},"id":4}]} 2024-09-10T21:29:25.263 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 61 2024-09-10T21:29:25.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6a00725e0 msgr2=0x7fb6a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6a00725e0 0x7fb6a0074aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4007fd0 tx=0x7fb6b4009040 comp rx=0 tx=0).stop 2024-09-10T21:29:25.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40cec10 msgr2=0x7fb6c40b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40cec10 0x7fb6c40b71e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc01fe40 tx=0x7fb6bc0093d0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 shutdown_connections 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6a00725e0 0x7fb6a0074aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c40cec10 0x7fb6c40b71e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb6c40ce830 0x7fb6c41f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 --2- 172.21.15.97:0/488220925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6c40b1560 0x7fb6c40b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 >> 172.21.15.97:0/488220925 conn(0x7fb6c40b37c0 msgr2=0x7fb6c40cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 shutdown_connections 2024-09-10T21:29:25.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.261+0000 7fb6c9600640 1 -- 172.21.15.97:0/488220925 wait complete. 2024-09-10T21:29:25.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 -- 172.21.15.97:0/2368326550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400ce640 msgr2=0x7f62400cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 --2- 172.21.15.97:0/2368326550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400ce640 0x7f62400cea20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f62400c8c90 tx=0x7f623001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 -- 172.21.15.97:0/2368326550 shutdown_connections 2024-09-10T21:29:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 --2- 172.21.15.97:0/2368326550 >> v2:172.21.15.194:3300/0 conn(0x7f62400cd6e0 0x7f62400af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 --2- 172.21.15.97:0/2368326550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400ce640 0x7f62400cea20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 -- 172.21.15.97:0/2368326550 >> 172.21.15.97:0/2368326550 conn(0x7f62400d1c30 msgr2=0x7f62400d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 -- 172.21.15.97:0/2368326550 shutdown_connections 2024-09-10T21:29:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.421+0000 7f6246e00640 1 -- 172.21.15.97:0/2368326550 wait complete. 2024-09-10T21:29:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 Processor -- start 2024-09-10T21:29:25.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 -- start start 2024-09-10T21:29:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400b0d50 0x7f6240165720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62400cd6e0 0x7f6240157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62400ce640 0x7f62401641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6240168cd0 con 0x7f62400b0d50 2024-09-10T21:29:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6240168b50 con 0x7f62400cd6e0 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6246e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6240168e50 con 0x7f62400ce640 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400b0d50 0x7f6240165720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400b0d50 0x7f6240165720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42434/0 (socket says 172.21.15.97:42434) 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 -- 172.21.15.97:0/1953477290 learned_addr learned my addr 172.21.15.97:0/1953477290 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623f400640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62400cd6e0 0x7f6240157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6244e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62400ce640 0x7f62401641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 -- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62400ce640 msgr2=0x7f62401641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62400ce640 0x7f62401641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 -- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62400cd6e0 msgr2=0x7f6240157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62400cd6e0 0x7f6240157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 -- 172.21.15.97:0/1953477290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62401f7d40 con 0x7f62400b0d50 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f623fe00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400b0d50 0x7f6240165720 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f62300b0040 tx=0x7f62300a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6237e00640 1 -- 172.21.15.97:0/1953477290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300b2040 con 0x7f62400b0d50 2024-09-10T21:29:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6237e00640 1 -- 172.21.15.97:0/1953477290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6230004430 con 0x7f62400b0d50 2024-09-10T21:29:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.425+0000 7f6237e00640 1 -- 172.21.15.97:0/1953477290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230005280 con 0x7f62400b0d50 2024-09-10T21:29:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62401f8020 con 0x7f62400b0d50 2024-09-10T21:29:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401f85e0 con 0x7f62400b0d50 2024-09-10T21:29:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62400cea20 con 0x7f62400b0d50 2024-09-10T21:29:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f6237e00640 1 -- 172.21.15.97:0/1953477290 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f62300046d0 con 0x7f62400b0d50 2024-09-10T21:29:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f6237e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62200725e0 0x7f6220074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f6237e00640 1 -- 172.21.15.97:0/1953477290 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f623012cd70 con 0x7f62400b0d50 2024-09-10T21:29:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f623f400640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62200725e0 0x7f6220074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.429+0000 7f623f400640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62200725e0 0x7f6220074aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f62400cbf10 tx=0x7f622c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:25.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.433+0000 7f6237e00640 1 -- 172.21.15.97:0/1953477290 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f62300164b0 con 0x7f62400b0d50 2024-09-10T21:29:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62401f87b0 con 0x7f62400b0d50 2024-09-10T21:29:25.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6237e00640 1 -- 172.21.15.97:0/1953477290 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5232 (secure 0 0 0) 0x7f62300f7a60 con 0x7f62400b0d50 2024-09-10T21:29:25.636 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:25.636 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"btime":"2024-09-10T21:27:49:202694+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2360657843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2360657843},{"type":"v1","addr":"172.21.15.97:6835","nonce":2360657843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:49.202684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":18,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5955,"qdb_cluster":[5955]},"id":4}]} 2024-09-10T21:29:25.636 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 61 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62200725e0 msgr2=0x7f6220074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62200725e0 0x7f6220074aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f62400cbf10 tx=0x7f622c009040 comp rx=0 tx=0).stop 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400b0d50 msgr2=0x7f6240165720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400b0d50 0x7f6240165720 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f62300b0040 tx=0x7f62300a5780 comp rx=0 tx=0).stop 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 shutdown_connections 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f62200725e0 0x7f6220074aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f62400ce640 0x7f62401641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62400cd6e0 0x7f6240157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 --2- 172.21.15.97:0/1953477290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62400b0d50 0x7f6240165720 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 >> 172.21.15.97:0/1953477290 conn(0x7f62400d1c30 msgr2=0x7f624006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 shutdown_connections 2024-09-10T21:29:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.633+0000 7f6246e00640 1 -- 172.21.15.97:0/1953477290 wait complete. 2024-09-10T21:29:25.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:29:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- 172.21.15.97:0/2794798855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 msgr2=0x7f27ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 --2- 172.21.15.97:0/2794798855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27ac00b9d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c5c0 tx=0x7f27c001d2c0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- 172.21.15.97:0/2794798855 shutdown_connections 2024-09-10T21:29:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 --2- 172.21.15.97:0/2794798855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27ac00b9d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- 172.21.15.97:0/2794798855 >> 172.21.15.97:0/2794798855 conn(0x7f27c80b1780 msgr2=0x7f27c80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- 172.21.15.97:0/2794798855 shutdown_connections 2024-09-10T21:29:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- 172.21.15.97:0/2794798855 wait complete. 2024-09-10T21:29:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 Processor -- start 2024-09-10T21:29:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- start start 2024-09-10T21:29:25.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27c8131ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c80b3e30 0x7f27c8143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27c80b4290 0x7f27c8143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27c815fa60 con 0x7f27ac0051d0 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27c815f8e0 con 0x7f27c80b3e30 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cfc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f27c80ce9d0 con 0x7f27c80b4290 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27c8131ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27c8131ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42452/0 (socket says 172.21.15.97:42452) 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cd200640 1 -- 172.21.15.97:0/28663961 learned_addr learned my addr 172.21.15.97:0/28663961 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27c7e00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c80b3e30 0x7f27c8143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.797+0000 7f27cdc00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27c80b4290 0x7f27c8143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cd200640 1 -- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27c80b4290 msgr2=0x7f27c8143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cd200640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27c80b4290 0x7f27c8143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cd200640 1 -- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c80b3e30 msgr2=0x7f27c8143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cd200640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c80b3e30 0x7f27c8143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cd200640 1 -- 172.21.15.97:0/28663961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c81321e0 con 0x7f27ac0051d0 2024-09-10T21:29:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cd200640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27c8131ca0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27b8003ea0 tx=0x7f27b8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27c4c00640 1 -- 172.21.15.97:0/28663961 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8015070 con 0x7f27ac0051d0 2024-09-10T21:29:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27c4c00640 1 -- 172.21.15.97:0/28663961 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27b8011840 con 0x7f27ac0051d0 2024-09-10T21:29:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27c4c00640 1 -- 172.21.15.97:0/28663961 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8011c90 con 0x7f27ac0051d0 2024-09-10T21:29:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c81f4910 con 0x7f27ac0051d0 2024-09-10T21:29:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c815ca40 con 0x7f27ac0051d0 2024-09-10T21:29:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c8141360 con 0x7f27ac0051d0 2024-09-10T21:29:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27c4c00640 1 -- 172.21.15.97:0/28663961 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f27b80103c0 con 0x7f27ac0051d0 2024-09-10T21:29:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.801+0000 7f27c4c00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27a8072610 0x7f27a8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.805+0000 7f27c4c00640 1 -- 172.21.15.97:0/28663961 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f27b80943b0 con 0x7f27ac0051d0 2024-09-10T21:29:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.805+0000 7f27c7e00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27a8072610 0x7f27a8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:25.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.805+0000 7f27c7e00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27a8072610 0x7f27a8074ad0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f27c8145540 tx=0x7f27b40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:25.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.805+0000 7f27c4c00640 1 -- 172.21.15.97:0/28663961 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f27c81321e0 con 0x7f27ac0051d0 2024-09-10T21:29:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27c8070350 con 0x7f27ac0051d0 2024-09-10T21:29:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27c4c00640 1 -- 172.21.15.97:0/28663961 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17612 (secure 0 0 0) 0x7f27b805f130 con 0x7f27ac0051d0 2024-09-10T21:29:25.976 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:25.976 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:27:54.225089+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:57.469287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:58.747272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.5","mappings":[{"from":4,"to":7}]},{"pgid":"10.1a","mappings":[{"from":2,"to":5}]},{"pgid":"10.1c","mappings":[{"from":2,"to":1}]},{"pgid":"10.2c","mappings":[{"from":2,"to":1}]},{"pgid":"10.2e","mappings":[{"from":2,"to":7}]},{"pgid":"10.37","mappings":[{"from":2,"to":7}]},{"pgid":"10.3b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/3391319282":"2024-09-11T21:27:46.185567+0000","172.21.15.97:6834/3391319282":"2024-09-11T21:27:46.185567+0000","172.21.15.97:6835/2609945029":"2024-09-11T21:27:10.383537+0000","172.21.15.97:6834/2609945029":"2024-09-11T21:27:10.383537+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27a8072610 msgr2=0x7f27a8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27a8072610 0x7f27a8074ad0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f27c8145540 tx=0x7f27b40091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 msgr2=0x7f27c8131ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27c8131ca0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f27b8003ea0 tx=0x7f27b8003f90 comp rx=0 tx=0).stop 2024-09-10T21:29:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 shutdown_connections 2024-09-10T21:29:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27a8072610 0x7f27a8074ad0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27c80b4290 0x7f27c8143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27c80b3e30 0x7f27c8143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 --2- 172.21.15.97:0/28663961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ac0051d0 0x7f27c8131ca0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:25.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 >> 172.21.15.97:0/28663961 conn(0x7f27c80b1780 msgr2=0x7f27c81593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:25.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 shutdown_connections 2024-09-10T21:29:25.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:25.973+0000 7f27cfc00640 1 -- 172.21.15.97:0/28663961 wait complete. 2024-09-10T21:29:25.998 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:29:25.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 -- 172.21.15.97:0/3140197677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c070720 msgr2=0x7f241c072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 --2- 172.21.15.97:0/3140197677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c070720 0x7f241c072630 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f240800aa10 tx=0x7f240801dc30 comp rx=0 tx=0).stop 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 -- 172.21.15.97:0/3140197677 shutdown_connections 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 --2- 172.21.15.97:0/3140197677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c070720 0x7f241c072630 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 --2- 172.21.15.97:0/3140197677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f241c15b110 0x7f241c0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 -- 172.21.15.97:0/3140197677 >> 172.21.15.97:0/3140197677 conn(0x7f241c0b1780 msgr2=0x7f241c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 -- 172.21.15.97:0/3140197677 shutdown_connections 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.137+0000 7f2423200640 1 -- 172.21.15.97:0/3140197677 wait complete. 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 Processor -- start 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 -- start start 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f241c070720 0x7f241c14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c072b70 0x7f241c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c15b110 0x7f241c163f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f241c15e790 con 0x7f241c15b110 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f241c15e610 con 0x7f241c072b70 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f241c166b50 con 0x7f241c070720 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c15b110 0x7f241c163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f241b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c072b70 0x7f241c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f241be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f241c070720 0x7f241c14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c15b110 0x7f241c163f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42460/0 (socket says 172.21.15.97:42460) 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 -- 172.21.15.97:0/2473042278 learned_addr learned my addr 172.21.15.97:0/2473042278 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 -- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f241c070720 msgr2=0x7f241c14eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f241c070720 0x7f241c14eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 -- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c072b70 msgr2=0x7f241c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c072b70 0x7f241c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 -- 172.21.15.97:0/2473042278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c1f87a0 con 0x7f241c15b110 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f241b400640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c072b70 0x7f241c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2421200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c15b110 0x7f241c163f40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f241400c7d0 tx=0x7f241400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2413e00640 1 -- 172.21.15.97:0/2473042278 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414003b80 con 0x7f241c15b110 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2413e00640 1 -- 172.21.15.97:0/2473042278 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2414003d40 con 0x7f241c15b110 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2413e00640 1 -- 172.21.15.97:0/2473042278 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400aa50 con 0x7f241c15b110 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c14e2c0 con 0x7f241c15b110 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c14f2e0 con 0x7f241c15b110 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.141+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c150a50 con 0x7f241c15b110 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.145+0000 7f2413e00640 1 -- 172.21.15.97:0/2473042278 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f241400abf0 con 0x7f241c15b110 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.145+0000 7f2413e00640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23fc0725e0 0x7f23fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.145+0000 7f2413e00640 1 -- 172.21.15.97:0/2473042278 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f24140931e0 con 0x7f241c15b110 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.145+0000 7f241be00640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23fc0725e0 0x7f23fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.145+0000 7f241be00640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23fc0725e0 0x7f23fc074aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f240c009600 tx=0x7f240c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.149+0000 7f2413e00640 1 -- 172.21.15.97:0/2473042278 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f241405df60 con 0x7f241c15b110 2024-09-10T21:29:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f241c150770 con 0x7f241c15b110 2024-09-10T21:29:26.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2413e00640 1 -- 172.21.15.97:0/2473042278 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5232 (secure 0 0 0) 0x7f2414062990 con 0x7f241c15b110 2024-09-10T21:29:26.352 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:26.352 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"btime":"2024-09-10T21:27:49:202694+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2360657843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2360657843},{"type":"v1","addr":"172.21.15.97:6835","nonce":2360657843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:49.202684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":18,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5955,"qdb_cluster":[5955]},"id":4}]} 2024-09-10T21:29:26.352 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 61 2024-09-10T21:29:26.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23fc0725e0 msgr2=0x7f23fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2423200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23fc0725e0 0x7f23fc074aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f240c009600 tx=0x7f240c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c15b110 msgr2=0x7f241c163f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2423200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c15b110 0x7f241c163f40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f241400c7d0 tx=0x7f241400cc90 comp rx=0 tx=0).stop 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 shutdown_connections 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.349+0000 7f2423200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23fc0725e0 0x7f23fc074aa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.353+0000 7f2423200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c15b110 0x7f241c163f40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.353+0000 7f2423200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f241c072b70 0x7f241c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.353+0000 7f2423200640 1 --2- 172.21.15.97:0/2473042278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f241c070720 0x7f241c14eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.353+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 >> 172.21.15.97:0/2473042278 conn(0x7f241c0b1780 msgr2=0x7f241c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.353+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 shutdown_connections 2024-09-10T21:29:26.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.353+0000 7f2423200640 1 -- 172.21.15.97:0/2473042278 wait complete. 2024-09-10T21:29:26.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.513+0000 7fb5abe00640 1 -- 172.21.15.97:0/3205380394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5900051d0 msgr2=0x7fb59000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.513+0000 7fb5abe00640 1 --2- 172.21.15.97:0/3205380394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5900051d0 0x7fb59000b9d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c670 tx=0x7fb5a401d320 comp rx=0 tx=0).stop 2024-09-10T21:29:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.513+0000 7fb5ab400640 1 -- 172.21.15.97:0/3205380394 reap_dead start 2024-09-10T21:29:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.513+0000 7fb5abe00640 1 -- 172.21.15.97:0/3205380394 shutdown_connections 2024-09-10T21:29:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.513+0000 7fb5abe00640 1 -- 172.21.15.97:0/3205380394 >> 172.21.15.97:0/3205380394 conn(0x7fb5ac0d1a60 msgr2=0x7fb5ac0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.513+0000 7fb5abe00640 1 -- 172.21.15.97:0/3205380394 shutdown_connections 2024-09-10T21:29:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.513+0000 7fb5abe00640 1 -- 172.21.15.97:0/3205380394 wait complete. 2024-09-10T21:29:26.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 Processor -- start 2024-09-10T21:29:26.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 -- start start 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5900051d0 0x7fb5ac0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac150860 0x7fb5ac1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5ac153620 0x7fb5ac153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac15f510 con 0x7fb5ac153620 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ac0afb50 con 0x7fb5ac150860 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ac0cfd80 con 0x7fb5900051d0 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5ab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5ac153620 0x7fb5ac153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5ab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5ac153620 0x7fb5ac153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42476/0 (socket says 172.21.15.97:42476) 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aaa00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5900051d0 0x7fb5ac0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5ab400640 1 -- 172.21.15.97:0/3234262643 learned_addr learned my addr 172.21.15.97:0/3234262643 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aa000640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac150860 0x7fb5ac1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aaa00640 1 -- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac150860 msgr2=0x7fb5ac1530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aaa00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac150860 0x7fb5ac1530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aaa00640 1 -- 172.21.15.97:0/3234262643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5ac153620 msgr2=0x7fb5ac153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aaa00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5ac153620 0x7fb5ac153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aaa00640 1 -- 172.21.15.97:0/3234262643 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ac0caf10 con 0x7fb5900051d0 2024-09-10T21:29:26.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5ab400640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5ac153620 0x7fb5ac153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:26.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5aaa00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5900051d0 0x7fb5ac0ca8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb59c011ab0 tx=0x7fb59c011eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5a2a00640 1 -- 172.21.15.97:0/3234262643 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c01f070 con 0x7fb5900051d0 2024-09-10T21:29:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5a2a00640 1 -- 172.21.15.97:0/3234262643 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb59c012450 con 0x7fb5900051d0 2024-09-10T21:29:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ac0cb1f0 con 0x7fb5900051d0 2024-09-10T21:29:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5a2a00640 1 -- 172.21.15.97:0/3234262643 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0128c0 con 0x7fb5900051d0 2024-09-10T21:29:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.517+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ac0cb7b0 con 0x7fb5900051d0 2024-09-10T21:29:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.521+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ac0cf840 con 0x7fb5900051d0 2024-09-10T21:29:26.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.521+0000 7fb5a2a00640 1 -- 172.21.15.97:0/3234262643 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb59c012a60 con 0x7fb5900051d0 2024-09-10T21:29:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.521+0000 7fb5a2a00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb58c072610 0x7fb58c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.521+0000 7fb5a2a00640 1 -- 172.21.15.97:0/3234262643 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7fb59c094370 con 0x7fb5900051d0 2024-09-10T21:29:26.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.521+0000 7fb5aa000640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb58c072610 0x7fb58c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.521+0000 7fb5aa000640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb58c072610 0x7fb58c074ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0cc040 tx=0x7fb5980091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.525+0000 7fb5a2a00640 1 -- 172.21.15.97:0/3234262643 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb59c05f0f0 con 0x7fb5900051d0 2024-09-10T21:29:26.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.725+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5ac071150 con 0x7fb5900051d0 2024-09-10T21:29:26.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.725+0000 7fb5a2a00640 1 -- 172.21.15.97:0/3234262643 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5232 (secure 0 0 0) 0x7fb59c063b20 con 0x7fb5900051d0 2024-09-10T21:29:26.728 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:26.729 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"btime":"2024-09-10T21:27:49:202694+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2360657843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2360657843},{"type":"v1","addr":"172.21.15.97:6835","nonce":2360657843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:26:59.777372+0000","modified":"2024-09-10T21:27:49.202684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":18,"addr":"172.21.15.194:6837/1482860240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1482860240},{"type":"v1","addr":"172.21.15.194:6837","nonce":1482860240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5955,"qdb_cluster":[5955]},"id":4}]} 2024-09-10T21:29:26.729 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 61 2024-09-10T21:29:26.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb58c072610 msgr2=0x7fb58c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb58c072610 0x7fb58c074ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0cc040 tx=0x7fb5980091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5900051d0 msgr2=0x7fb5ac0ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5900051d0 0x7fb5ac0ca8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb59c011ab0 tx=0x7fb59c011eb0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 shutdown_connections 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb58c072610 0x7fb58c074ad0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5ac153620 0x7fb5ac153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac150860 0x7fb5ac1530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 --2- 172.21.15.97:0/3234262643 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5900051d0 0x7fb5ac0ca8c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 >> 172.21.15.97:0/3234262643 conn(0x7fb5ac0d1a60 msgr2=0x7fb5ac06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 shutdown_connections 2024-09-10T21:29:26.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.729+0000 7fb5abe00640 1 -- 172.21.15.97:0/3234262643 wait complete. 2024-09-10T21:29:26.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:29:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 -- 172.21.15.97:0/1005379778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c0ce5f0 msgr2=0x7fda7c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 --2- 172.21.15.97:0/1005379778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c0ce5f0 0x7fda7c141d20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fda7c074800 tx=0x7fda6c01cb80 comp rx=0 tx=0).stop 2024-09-10T21:29:26.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 -- 172.21.15.97:0/1005379778 shutdown_connections 2024-09-10T21:29:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 --2- 172.21.15.97:0/1005379778 >> v2:172.21.15.194:3300/0 conn(0x7fda7c0ce9d0 0x7fda7c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 --2- 172.21.15.97:0/1005379778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c0ce5f0 0x7fda7c141d20 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 -- 172.21.15.97:0/1005379778 >> 172.21.15.97:0/1005379778 conn(0x7fda7c0b1780 msgr2=0x7fda7c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 -- 172.21.15.97:0/1005379778 shutdown_connections 2024-09-10T21:29:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 -- 172.21.15.97:0/1005379778 wait complete. 2024-09-10T21:29:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 Processor -- start 2024-09-10T21:29:26.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.889+0000 7fda81e00640 1 -- start start 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda7c0ce5f0 0x7fda7c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda7c0ce9d0 0x7fda7c13c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c15fc10 0x7fda7c1f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c168fd0 con 0x7fda7c15fc10 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c168e50 con 0x7fda7c0ce5f0 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c169150 con 0x7fda7c0ce9d0 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c15fc10 0x7fda7c1f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c15fc10 0x7fda7c1f88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42506/0 (socket says 172.21.15.97:42506) 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 -- 172.21.15.97:0/1085738677 learned_addr learned my addr 172.21.15.97:0/1085738677 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:26.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda72000640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda7c0ce9d0 0x7fda7c13c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7aa00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda7c0ce5f0 0x7fda7c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 -- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda7c0ce9d0 msgr2=0x7fda7c13c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda7c0ce9d0 0x7fda7c13c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 -- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda7c0ce5f0 msgr2=0x7fda7c06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda7c0ce5f0 0x7fda7c06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 -- 172.21.15.97:0/1085738677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c1f80e0 con 0x7fda7c15fc10 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7aa00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda7c0ce5f0 0x7fda7c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda7b400640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c15fc10 0x7fda7c1f88a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c3c0 tx=0x7fda7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda73e00640 1 -- 172.21.15.97:0/1085738677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74011790 con 0x7fda7c15fc10 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda73e00640 1 -- 172.21.15.97:0/1085738677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda74017070 con 0x7fda7c15fc10 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda73e00640 1 -- 172.21.15.97:0/1085738677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74011ce0 con 0x7fda7c15fc10 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c142530 con 0x7fda7c15fc10 2024-09-10T21:29:26.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c1f8380 con 0x7fda7c15fc10 2024-09-10T21:29:26.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c1f8580 con 0x7fda7c15fc10 2024-09-10T21:29:26.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.893+0000 7fda73e00640 1 -- 172.21.15.97:0/1085738677 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fda7401c460 con 0x7fda7c15fc10 2024-09-10T21:29:26.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.897+0000 7fda73e00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda580725e0 0x7fda58074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.897+0000 7fda73e00640 1 -- 172.21.15.97:0/1085738677 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7fda740932e0 con 0x7fda7c15fc10 2024-09-10T21:29:26.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.897+0000 7fda7aa00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda580725e0 0x7fda58074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.897+0000 7fda7aa00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda580725e0 0x7fda58074aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0af000 tx=0x7fda6c01b3c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:26.901+0000 7fda73e00640 1 -- 172.21.15.97:0/1085738677 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fda7c1f8580 con 0x7fda7c15fc10 2024-09-10T21:29:27.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda7c141d20 con 0x7fda7c15fc10 2024-09-10T21:29:27.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda73e00640 1 -- 172.21.15.97:0/1085738677 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17612 (secure 0 0 0) 0x7fda7405e6b0 con 0x7fda7c15fc10 2024-09-10T21:29:27.072 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:27.072 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:27:54.225089+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:26:57.469287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T21:26:58.747272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.5","mappings":[{"from":4,"to":7}]},{"pgid":"10.1a","mappings":[{"from":2,"to":5}]},{"pgid":"10.1c","mappings":[{"from":2,"to":1}]},{"pgid":"10.2c","mappings":[{"from":2,"to":1}]},{"pgid":"10.2e","mappings":[{"from":2,"to":7}]},{"pgid":"10.37","mappings":[{"from":2,"to":7}]},{"pgid":"10.3b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/3391319282":"2024-09-11T21:27:46.185567+0000","172.21.15.97:6834/3391319282":"2024-09-11T21:27:46.185567+0000","172.21.15.97:6835/2609945029":"2024-09-11T21:27:10.383537+0000","172.21.15.97:6834/2609945029":"2024-09-11T21:27:10.383537+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda580725e0 msgr2=0x7fda58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda580725e0 0x7fda58074aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0af000 tx=0x7fda6c01b3c0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c15fc10 msgr2=0x7fda7c1f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c15fc10 0x7fda7c1f88a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c3c0 tx=0x7fda7400c880 comp rx=0 tx=0).stop 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 shutdown_connections 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda580725e0 0x7fda58074aa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda7c15fc10 0x7fda7c1f88a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fda7c0ce9d0 0x7fda7c13c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 --2- 172.21.15.97:0/1085738677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda7c0ce5f0 0x7fda7c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 >> 172.21.15.97:0/1085738677 conn(0x7fda7c0b1780 msgr2=0x7fda7c13f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 shutdown_connections 2024-09-10T21:29:27.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.069+0000 7fda81e00640 1 -- 172.21.15.97:0/1085738677 wait complete. 2024-09-10T21:29:27.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 -- 172.21.15.97:0/4236953349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600cec10 msgr2=0x7f76600cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 --2- 172.21.15.97:0/4236953349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600cec10 0x7f76600cfd40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f76600744b0 tx=0x7f764c01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 -- 172.21.15.97:0/4236953349 shutdown_connections 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 --2- 172.21.15.97:0/4236953349 >> v2:172.21.15.194:3300/0 conn(0x7f76600d0280 0x7f766015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 --2- 172.21.15.97:0/4236953349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600cec10 0x7f76600cfd40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 --2- 172.21.15.97:0/4236953349 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76600ce830 0x7f76600cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 -- 172.21.15.97:0/4236953349 >> 172.21.15.97:0/4236953349 conn(0x7f76600b37c0 msgr2=0x7f76600b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 -- 172.21.15.97:0/4236953349 shutdown_connections 2024-09-10T21:29:27.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.233+0000 7f7667c00640 1 -- 172.21.15.97:0/4236953349 wait complete. 2024-09-10T21:29:27.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 Processor -- start 2024-09-10T21:29:27.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 -- start start 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76600ce830 0x7f76601f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76600cec10 0x7f766015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600d0280 0x7f76600aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76600b4420 con 0x7f76600d0280 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f76600cde60 con 0x7f76600ce830 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f76600ce1a0 con 0x7f76600cec10 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600d0280 0x7f76600aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600d0280 0x7f76600aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42532/0 (socket says 172.21.15.97:42532) 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 -- 172.21.15.97:0/3661236104 learned_addr learned my addr 172.21.15.97:0/3661236104 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665200640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76600ce830 0x7f76601f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f765fe00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76600cec10 0x7f766015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 -- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76600cec10 msgr2=0x7f766015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76600cec10 0x7f766015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 -- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76600ce830 msgr2=0x7f76601f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76600ce830 0x7f76601f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 -- 172.21.15.97:0/3661236104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766015c570 con 0x7f76600d0280 2024-09-10T21:29:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7665c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600d0280 0x7f76600aff90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f765800ea80 tx=0x7f765800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f765cc00640 1 -- 172.21.15.97:0/3661236104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658004b30 con 0x7f76600d0280 2024-09-10T21:29:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f765cc00640 1 -- 172.21.15.97:0/3661236104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7658004cf0 con 0x7f76600d0280 2024-09-10T21:29:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f765cc00640 1 -- 172.21.15.97:0/3661236104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658005280 con 0x7f76600d0280 2024-09-10T21:29:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76600c98e0 con 0x7f76600d0280 2024-09-10T21:29:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76600c9ea0 con 0x7f76600d0280 2024-09-10T21:29:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.237+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f766015fd20 con 0x7f76600d0280 2024-09-10T21:29:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.241+0000 7f765cc00640 1 -- 172.21.15.97:0/3661236104 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7658005420 con 0x7f76600d0280 2024-09-10T21:29:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.241+0000 7f765cc00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76400725e0 0x7f7640074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.241+0000 7f765cc00640 1 -- 172.21.15.97:0/3661236104 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6476+0+0 (secure 0 0 0) 0x7f7658094250 con 0x7f76600d0280 2024-09-10T21:29:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.241+0000 7f7665200640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76400725e0 0x7f7640074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.241+0000 7f7665200640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76400725e0 0x7f7640074aa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7654009b70 tx=0x7f76540091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.245+0000 7f765cc00640 1 -- 172.21.15.97:0/3661236104 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f765805f590 con 0x7f76600d0280 2024-09-10T21:29:27.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.445+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f76600b7af0 con 0x7f76600d0280 2024-09-10T21:29:27.447 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:29:27.445+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:29:27.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.445+0000 7f765cc00640 1 -- 172.21.15.97:0/3661236104 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f765805ee90 con 0x7f76600d0280 2024-09-10T21:29:27.449 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:29:27.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76400725e0 msgr2=0x7f7640074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76400725e0 0x7f7640074aa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7654009b70 tx=0x7f76540091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600d0280 msgr2=0x7f76600aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600d0280 0x7f76600aff90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f765800ea80 tx=0x7f765800ef40 comp rx=0 tx=0).stop 2024-09-10T21:29:27.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 shutdown_connections 2024-09-10T21:29:27.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76400725e0 0x7f7640074aa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76600d0280 0x7f76600aff90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76600cec10 0x7f766015c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 --2- 172.21.15.97:0/3661236104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76600ce830 0x7f76601f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 >> 172.21.15.97:0/3661236104 conn(0x7f76600b37c0 msgr2=0x7f7660071540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 shutdown_connections 2024-09-10T21:29:27.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.449+0000 7f7667c00640 1 -- 172.21.15.97:0/3661236104 wait complete. 2024-09-10T21:29:27.464 INFO:tasks.ceph.mds.d.smithi194.stderr: -4331> 2024-09-10T21:27:49.340+0000 7f2411200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:29:27.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:29:27.512 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:29:27.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 -- 172.21.15.97:0/2372372532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20580b0110 msgr2=0x7f20580cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 --2- 172.21.15.97:0/2372372532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20580b0110 0x7f20580cff40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2050009bc0 tx=0x7f205001ca90 comp rx=0 tx=0).stop 2024-09-10T21:29:27.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 -- 172.21.15.97:0/2372372532 shutdown_connections 2024-09-10T21:29:27.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 --2- 172.21.15.97:0/2372372532 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20580b0110 0x7f20580cff40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 --2- 172.21.15.97:0/2372372532 >> v2:172.21.15.194:3300/0 conn(0x7f205815bdd0 0x7f2058074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 -- 172.21.15.97:0/2372372532 >> 172.21.15.97:0/2372372532 conn(0x7f20580b28c0 msgr2=0x7f20580b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 -- 172.21.15.97:0/2372372532 shutdown_connections 2024-09-10T21:29:27.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 -- 172.21.15.97:0/2372372532 wait complete. 2024-09-10T21:29:27.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 Processor -- start 2024-09-10T21:29:27.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.613+0000 7f2060000640 1 -- start start 2024-09-10T21:29:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20580b0110 0x7f2058150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20580cfad0 0x7f2058158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f205815bdd0 0x7f2058158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2058168b30 con 0x7f20580cfad0 2024-09-10T21:29:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f20581689b0 con 0x7f20580b0110 2024-09-10T21:29:27.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2058168cb0 con 0x7f205815bdd0 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20580cfad0 0x7f2058158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20580cfad0 0x7f2058158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49530/0 (socket says 172.21.15.97:49530) 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 -- 172.21.15.97:0/3929964376 learned_addr learned my addr 172.21.15.97:0/3929964376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205d600640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20580b0110 0x7f2058150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205e000640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f205815bdd0 0x7f2058158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 -- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f205815bdd0 msgr2=0x7f2058158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f205815bdd0 0x7f2058158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 -- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20580b0110 msgr2=0x7f2058150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20580b0110 0x7f2058150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 -- 172.21.15.97:0/3929964376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205806f620 con 0x7f20580cfad0 2024-09-10T21:29:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f205cc00640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20580cfad0 0x7f2058158050 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f204400c760 tx=0x7f204400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2055600640 1 -- 172.21.15.97:0/3929964376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044016020 con 0x7f20580cfad0 2024-09-10T21:29:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2055600640 1 -- 172.21.15.97:0/3929964376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f204400a8f0 con 0x7f20580cfad0 2024-09-10T21:29:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2055600640 1 -- 172.21.15.97:0/3929964376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400ad40 con 0x7f20580cfad0 2024-09-10T21:29:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205806f900 con 0x7f20580cfad0 2024-09-10T21:29:27.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205806fec0 con 0x7f20580cfad0 2024-09-10T21:29:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20580cff40 con 0x7f20580cfad0 2024-09-10T21:29:27.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.617+0000 7f2055600640 1 -- 172.21.15.97:0/3929964376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f204401b410 con 0x7f20580cfad0 2024-09-10T21:29:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.621+0000 7f2055600640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2038072610 0x7f2038074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.621+0000 7f2055600640 1 -- 172.21.15.97:0/3929964376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6562+0+0 (secure 0 0 0) 0x7f2044093330 con 0x7f20580cfad0 2024-09-10T21:29:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.621+0000 7f205d600640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2038072610 0x7f2038074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.621+0000 7f205d600640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2038072610 0x7f2038074ad0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f204c009600 tx=0x7f204c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.625+0000 7f2055600640 1 -- 172.21.15.97:0/3929964376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2044020070 con 0x7f20580cfad0 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:27.825+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f20580d0fe0 con 0x7f20580cfad0 2024-09-10T21:29:28.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.449+0000 7f2055600640 1 -- 172.21.15.97:0/3929964376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) ==== 104+0+0 (secure 0 0 0) 0x7f204405e050 con 0x7f20580cfad0 2024-09-10T21:29:28.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2038072610 msgr2=0x7f2038074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2038072610 0x7f2038074ad0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f204c009600 tx=0x7f204c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20580cfad0 msgr2=0x7f2058158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20580cfad0 0x7f2058158050 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f204400c760 tx=0x7f204400cc20 comp rx=0 tx=0).stop 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 shutdown_connections 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2038072610 0x7f2038074ad0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f205815bdd0 0x7f2058158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20580cfad0 0x7f2058158050 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 --2- 172.21.15.97:0/3929964376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20580b0110 0x7f2058150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 >> 172.21.15.97:0/3929964376 conn(0x7f20580b28c0 msgr2=0x7f2058152930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 shutdown_connections 2024-09-10T21:29:28.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.453+0000 7f2060000640 1 -- 172.21.15.97:0/3929964376 wait complete. 2024-09-10T21:29:28.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:29:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- 172.21.15.97:0/3581408514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8004220 msgr2=0x7f1de8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 --2- 172.21.15.97:0/3581408514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8004220 0x7f1de8004810 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1e0006fc10 tx=0x7f1dec01c870 comp rx=0 tx=0).stop 2024-09-10T21:29:28.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e06000640 1 -- 172.21.15.97:0/3581408514 reap_dead start 2024-09-10T21:29:28.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- 172.21.15.97:0/3581408514 shutdown_connections 2024-09-10T21:29:28.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- 172.21.15.97:0/3581408514 >> 172.21.15.97:0/3581408514 conn(0x7f1e000d1c30 msgr2=0x7f1e000d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:28.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- 172.21.15.97:0/3581408514 shutdown_connections 2024-09-10T21:29:28.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- 172.21.15.97:0/3581408514 wait complete. 2024-09-10T21:29:28.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 Processor -- start 2024-09-10T21:29:28.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- start start 2024-09-10T21:29:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1de8004220 0x7f1e000c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e001fbb30 0x7f1e000c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e000b3bd0 0x7f1e00159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00169c90 con 0x7f1e001fbb30 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00169b10 con 0x7f1e000b3bd0 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.625+0000 7f1e08000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1e00169e10 con 0x7f1de8004220 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e001fbb30 0x7f1e000c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e001fbb30 0x7f1e000c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49552/0 (socket says 172.21.15.97:49552) 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 -- 172.21.15.97:0/3911312106 learned_addr learned my addr 172.21.15.97:0/3911312106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e05600640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1de8004220 0x7f1e000c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e06000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e000b3bd0 0x7f1e00159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 -- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1de8004220 msgr2=0x7f1e000c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1de8004220 0x7f1e000c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 -- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e000b3bd0 msgr2=0x7f1e00159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e000b3bd0 0x7f1e00159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 -- 172.21.15.97:0/3911312106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e000c8270 con 0x7f1e001fbb30 2024-09-10T21:29:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e06000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e000b3bd0 0x7f1e00159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e04c00640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e001fbb30 0x7f1e000c7d30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1dec005040 tx=0x7f1dec0a5cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1dfd600640 1 -- 172.21.15.97:0/3911312106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0068d0 con 0x7f1e001fbb30 2024-09-10T21:29:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1dfd600640 1 -- 172.21.15.97:0/3911312106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dec0b4070 con 0x7f1e001fbb30 2024-09-10T21:29:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e000c8510 con 0x7f1e001fbb30 2024-09-10T21:29:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1dfd600640 1 -- 172.21.15.97:0/3911312106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec001420 con 0x7f1e001fbb30 2024-09-10T21:29:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001fb330 con 0x7f1e001fbb30 2024-09-10T21:29:28.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e000cfe30 con 0x7f1e001fbb30 2024-09-10T21:29:28.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.629+0000 7f1dfd600640 1 -- 172.21.15.97:0/3911312106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1dec0c0030 con 0x7f1e001fbb30 2024-09-10T21:29:28.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.633+0000 7f1dfd600640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1de0072610 0x7f1de0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.633+0000 7f1dfd600640 1 -- 172.21.15.97:0/3911312106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6562+0+0 (secure 0 0 0) 0x7f1dec12d470 con 0x7f1e001fbb30 2024-09-10T21:29:28.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.633+0000 7f1e05600640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1de0072610 0x7f1de0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:28.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.633+0000 7f1e05600640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1de0072610 0x7f1de0074ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1e00169a30 tx=0x7f1df40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:28.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.637+0000 7f1dfd600640 1 -- 172.21.15.97:0/3911312106 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1dec0165d0 con 0x7f1e001fbb30 2024-09-10T21:29:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:28.801+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e0015bc40 con 0x7f1e001fbb30 2024-09-10T21:29:29.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.453+0000 7f1dfd600640 1 -- 172.21.15.97:0/3911312106 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f1dec0f88f0 con 0x7f1e001fbb30 2024-09-10T21:29:29.457 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:29:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1de0072610 msgr2=0x7f1de0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1de0072610 0x7f1de0074ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1e00169a30 tx=0x7f1df40091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e001fbb30 msgr2=0x7f1e000c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e001fbb30 0x7f1e000c7d30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1dec005040 tx=0x7f1dec0a5cd0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 shutdown_connections 2024-09-10T21:29:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1de0072610 0x7f1de0074ad0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e000b3bd0 0x7f1e00159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e001fbb30 0x7f1e000c7d30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 --2- 172.21.15.97:0/3911312106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1de8004220 0x7f1e000c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 >> 172.21.15.97:0/3911312106 conn(0x7f1e000d1c30 msgr2=0x7f1e000b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 shutdown_connections 2024-09-10T21:29:29.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.457+0000 7f1e08000640 1 -- 172.21.15.97:0/3911312106 wait complete. 2024-09-10T21:29:29.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:29:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 -- 172.21.15.97:0/2710551348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234806cb40 msgr2=0x7f23480d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 --2- 172.21.15.97:0/2710551348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234806cb40 0x7f23480d0170 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f234809e420 tx=0x7f234001cb50 comp rx=0 tx=0).stop 2024-09-10T21:29:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 -- 172.21.15.97:0/2710551348 shutdown_connections 2024-09-10T21:29:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 --2- 172.21.15.97:0/2710551348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234806cb40 0x7f23480d0170 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 --2- 172.21.15.97:0/2710551348 >> v2:172.21.15.194:3300/0 conn(0x7f23480af520 0x7f2348160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 -- 172.21.15.97:0/2710551348 >> 172.21.15.97:0/2710551348 conn(0x7f23480b3c70 msgr2=0x7f23480b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 -- 172.21.15.97:0/2710551348 shutdown_connections 2024-09-10T21:29:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.625+0000 7f234ca00640 1 -- 172.21.15.97:0/2710551348 wait complete. 2024-09-10T21:29:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 Processor -- start 2024-09-10T21:29:29.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 -- start start 2024-09-10T21:29:29.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f234806cb40 0x7f23481494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23480af520 0x7f234815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f234815f120 0x7f2348158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2348169570 con 0x7f23480af520 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23480cdea0 con 0x7f234806cb40 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f23481696f0 con 0x7f234815f120 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23480af520 0x7f234815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23480af520 0x7f234815d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49582/0 (socket says 172.21.15.97:49582) 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 -- 172.21.15.97:0/873429630 learned_addr learned my addr 172.21.15.97:0/873429630 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345e00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f234806cb40 0x7f23481494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2346800640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f234815f120 0x7f2348158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 -- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f234815f120 msgr2=0x7f2348158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f234815f120 0x7f2348158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 -- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f234806cb40 msgr2=0x7f23481494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f234806cb40 0x7f23481494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 -- 172.21.15.97:0/873429630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23481f85a0 con 0x7f23480af520 2024-09-10T21:29:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2346800640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f234815f120 0x7f2348158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f2345400640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23480af520 0x7f234815d2a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f233400c970 tx=0x7f233400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f233e000640 1 -- 172.21.15.97:0/873429630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334007c70 con 0x7f23480af520 2024-09-10T21:29:29.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f233e000640 1 -- 172.21.15.97:0/873429630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f233400f450 con 0x7f23480af520 2024-09-10T21:29:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23481f8880 con 0x7f23480af520 2024-09-10T21:29:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.629+0000 7f233e000640 1 -- 172.21.15.97:0/873429630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233400f8a0 con 0x7f23480af520 2024-09-10T21:29:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.633+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23481f8e40 con 0x7f23480af520 2024-09-10T21:29:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.633+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23480d0170 con 0x7f23480af520 2024-09-10T21:29:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.633+0000 7f233e000640 1 -- 172.21.15.97:0/873429630 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2334020030 con 0x7f23480af520 2024-09-10T21:29:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.633+0000 7f233e000640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23280725e0 0x7f2328074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.633+0000 7f233e000640 1 -- 172.21.15.97:0/873429630 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6124+0+0 (secure 0 0 0) 0x7f23340938b0 con 0x7f23480af520 2024-09-10T21:29:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.633+0000 7f2345e00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23280725e0 0x7f2328074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:29.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.637+0000 7f2345e00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23280725e0 0x7f2328074aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2338007fd0 tx=0x7f2338009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.637+0000 7f233e000640 1 -- 172.21.15.97:0/873429630 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2334014030 con 0x7f23480af520 2024-09-10T21:29:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:29.805+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f234815c170 con 0x7f23480af520 2024-09-10T21:29:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.461+0000 7f233e000640 1 -- 172.21.15.97:0/873429630 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7f233405ede0 con 0x7f23480af520 2024-09-10T21:29:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23280725e0 msgr2=0x7f2328074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23280725e0 0x7f2328074aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2338007fd0 tx=0x7f2338009040 comp rx=0 tx=0).stop 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23480af520 msgr2=0x7f234815d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23480af520 0x7f234815d2a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f233400c970 tx=0x7f233400ce30 comp rx=0 tx=0).stop 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 shutdown_connections 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23280725e0 0x7f2328074aa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f234815f120 0x7f2348158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23480af520 0x7f234815d2a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 --2- 172.21.15.97:0/873429630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f234806cb40 0x7f23481494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 >> 172.21.15.97:0/873429630 conn(0x7f23480b3c70 msgr2=0x7f2348152cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 shutdown_connections 2024-09-10T21:29:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.465+0000 7f234ca00640 1 -- 172.21.15.97:0/873429630 wait complete. 2024-09-10T21:29:30.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:29:30.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.629+0000 7fac35200640 1 -- 172.21.15.97:0/3486439945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac14003a10 msgr2=0x7fac14003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.629+0000 7fac35200640 1 --2- 172.21.15.97:0/3486439945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac14003a10 0x7fac14003e30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fac20005fd0 tx=0x7fac2001cd30 comp rx=0 tx=0).stop 2024-09-10T21:29:30.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.629+0000 7fac2e800640 1 -- 172.21.15.97:0/3486439945 reap_dead start 2024-09-10T21:29:30.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.629+0000 7fac35200640 1 -- 172.21.15.97:0/3486439945 shutdown_connections 2024-09-10T21:29:30.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.629+0000 7fac35200640 1 -- 172.21.15.97:0/3486439945 >> 172.21.15.97:0/3486439945 conn(0x7fac300d1cd0 msgr2=0x7fac300d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:30.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.629+0000 7fac35200640 1 -- 172.21.15.97:0/3486439945 shutdown_connections 2024-09-10T21:29:30.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.629+0000 7fac35200640 1 -- 172.21.15.97:0/3486439945 wait complete. 2024-09-10T21:29:30.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 Processor -- start 2024-09-10T21:29:30.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 -- start start 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac14003a10 0x7fac300b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac3015e0b0 0x7fac301596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301fb390 0x7fac30159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac30166870 con 0x7fac301fb390 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fac301666f0 con 0x7fac14003a10 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fac301669f0 con 0x7fac3015e0b0 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301fb390 0x7fac30159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301fb390 0x7fac30159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49602/0 (socket says 172.21.15.97:49602) 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 -- 172.21.15.97:0/4150951965 learned_addr learned my addr 172.21.15.97:0/4150951965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2d400640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac3015e0b0 0x7fac301596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2de00640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac14003a10 0x7fac300b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 -- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac3015e0b0 msgr2=0x7fac301596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac3015e0b0 0x7fac301596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 -- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac14003a10 msgr2=0x7fac300b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac14003a10 0x7fac300b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 -- 172.21.15.97:0/4150951965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac301f8d70 con 0x7fac301fb390 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2de00640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac14003a10 0x7fac300b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac2e800640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301fb390 0x7fac30159c00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fac300d1600 tx=0x7fac2800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac26000640 1 -- 172.21.15.97:0/4150951965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28003d90 con 0x7fac301fb390 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac26000640 1 -- 172.21.15.97:0/4150951965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac2800e8d0 con 0x7fac301fb390 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac301f8160 con 0x7fac301fb390 2024-09-10T21:29:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac26000640 1 -- 172.21.15.97:0/4150951965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2800ed20 con 0x7fac301fb390 2024-09-10T21:29:30.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac301f8720 con 0x7fac301fb390 2024-09-10T21:29:30.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.633+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac300b0470 con 0x7fac301fb390 2024-09-10T21:29:30.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.637+0000 7fac26000640 1 -- 172.21.15.97:0/4150951965 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fac2800f040 con 0x7fac301fb390 2024-09-10T21:29:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.637+0000 7fac26000640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac100725e0 0x7fac10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.637+0000 7fac26000640 1 -- 172.21.15.97:0/4150951965 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7fac28093720 con 0x7fac301fb390 2024-09-10T21:29:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.637+0000 7fac2de00640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac100725e0 0x7fac10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.637+0000 7fac2de00640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac100725e0 0x7fac10074aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac200b0000 tx=0x7fac200a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:30.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.641+0000 7fac26000640 1 -- 172.21.15.97:0/4150951965 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fac28015030 con 0x7fac301fb390 2024-09-10T21:29:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.817+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fac3015a870 con 0x7fac301fb390 2024-09-10T21:29:30.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac26000640 1 -- 172.21.15.97:0/4150951965 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v38) ==== 71+0+0 (secure 0 0 0) 0x7fac2805e860 con 0x7fac301fb390 2024-09-10T21:29:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac100725e0 msgr2=0x7fac10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac100725e0 0x7fac10074aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac200b0000 tx=0x7fac200a7040 comp rx=0 tx=0).stop 2024-09-10T21:29:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301fb390 msgr2=0x7fac30159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301fb390 0x7fac30159c00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fac300d1600 tx=0x7fac2800fd90 comp rx=0 tx=0).stop 2024-09-10T21:29:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 shutdown_connections 2024-09-10T21:29:30.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac100725e0 0x7fac10074aa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301fb390 0x7fac30159c00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac3015e0b0 0x7fac301596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 --2- 172.21.15.97:0/4150951965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac14003a10 0x7fac300b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 >> 172.21.15.97:0/4150951965 conn(0x7fac300d1cd0 msgr2=0x7fac3015ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 shutdown_connections 2024-09-10T21:29:30.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.821+0000 7fac35200640 1 -- 172.21.15.97:0/4150951965 wait complete. 2024-09-10T21:29:30.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-10T21:29:30.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 -- 172.21.15.97:0/1796995390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff015fc10 msgr2=0x7fdff015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 --2- 172.21.15.97:0/1796995390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff015fc10 0x7fdff015c100 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdff0074ae0 tx=0x7fdfe801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 -- 172.21.15.97:0/1796995390 shutdown_connections 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 --2- 172.21.15.97:0/1796995390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff015fc10 0x7fdff015c100 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 --2- 172.21.15.97:0/1796995390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdff00ce9d0 0x7fdff015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 --2- 172.21.15.97:0/1796995390 >> v2:172.21.15.194:3300/0 conn(0x7fdff00ce5f0 0x7fdff0141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 -- 172.21.15.97:0/1796995390 >> 172.21.15.97:0/1796995390 conn(0x7fdff00b1780 msgr2=0x7fdff00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 -- 172.21.15.97:0/1796995390 shutdown_connections 2024-09-10T21:29:30.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.981+0000 7fdff5600640 1 -- 172.21.15.97:0/1796995390 wait complete. 2024-09-10T21:29:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 Processor -- start 2024-09-10T21:29:30.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 -- start start 2024-09-10T21:29:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff00ce5f0 0x7fdff015e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdff00ce9d0 0x7fdff01f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdff015fc10 0x7fdff0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0140180 con 0x7fdff00ce5f0 2024-09-10T21:29:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0140000 con 0x7fdff015fc10 2024-09-10T21:29:30.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdff0140300 con 0x7fdff00ce9d0 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff00ce5f0 0x7fdff015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff00ce5f0 0x7fdff015e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49628/0 (socket says 172.21.15.97:49628) 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 -- 172.21.15.97:0/2909200684 learned_addr learned my addr 172.21.15.97:0/2909200684 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfef400640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdff015fc10 0x7fdff0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfe6000640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdff00ce9d0 0x7fdff01f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 -- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdff00ce9d0 msgr2=0x7fdff01f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdff00ce9d0 0x7fdff01f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 -- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdff015fc10 msgr2=0x7fdff0158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdff015fc10 0x7fdff0158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 -- 172.21.15.97:0/2909200684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff015ec10 con 0x7fdff00ce5f0 2024-09-10T21:29:30.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfe6000640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdff00ce9d0 0x7fdff01f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfeea00640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff00ce5f0 0x7fdff015e550 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000eb00 tx=0x7fdfe000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfe7400640 1 -- 172.21.15.97:0/2909200684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0011470 con 0x7fdff00ce5f0 2024-09-10T21:29:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfe7400640 1 -- 172.21.15.97:0/2909200684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe0011700 con 0x7fdff00ce5f0 2024-09-10T21:29:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdfe7400640 1 -- 172.21.15.97:0/2909200684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0011b50 con 0x7fdff00ce5f0 2024-09-10T21:29:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff015eef0 con 0x7fdff00ce5f0 2024-09-10T21:29:30.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01f8fe0 con 0x7fdff00ce5f0 2024-09-10T21:29:30.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.985+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff015f6d0 con 0x7fdff00ce5f0 2024-09-10T21:29:30.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.989+0000 7fdfe7400640 1 -- 172.21.15.97:0/2909200684 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdfe000f040 con 0x7fdff00ce5f0 2024-09-10T21:29:30.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.989+0000 7fdfe7400640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfcc072610 0x7fdfcc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:30.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.989+0000 7fdfe7400640 1 -- 172.21.15.97:0/2909200684 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7fdfe0093ab0 con 0x7fdff00ce5f0 2024-09-10T21:29:30.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.989+0000 7fdfe6000640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfcc072610 0x7fdfcc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:30.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.989+0000 7fdfe6000640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfcc072610 0x7fdfcc074ad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80058d0 tx=0x7fdfd8005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:30.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:30.993+0000 7fdfe7400640 1 -- 172.21.15.97:0/2909200684 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdfe0016030 con 0x7fdff00ce5f0 2024-09-10T21:29:31.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.217+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7fdff0141360 con 0x7fdff00ce5f0 2024-09-10T21:29:31.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.461+0000 7fdfe7400640 1 -- 172.21.15.97:0/2909200684 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v256) ==== 142+0+0 (secure 0 0 0) 0x7fdfe005f1b0 con 0x7fdff00ce5f0 2024-09-10T21:29:31.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfcc072610 msgr2=0x7fdfcc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfcc072610 0x7fdfcc074ad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80058d0 tx=0x7fdfd8005860 comp rx=0 tx=0).stop 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff00ce5f0 msgr2=0x7fdff015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff00ce5f0 0x7fdff015e550 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000eb00 tx=0x7fdfe000c270 comp rx=0 tx=0).stop 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 shutdown_connections 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfcc072610 0x7fdfcc074ad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdff015fc10 0x7fdff0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdff00ce9d0 0x7fdff01f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 --2- 172.21.15.97:0/2909200684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff00ce5f0 0x7fdff015e550 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 >> 172.21.15.97:0/2909200684 conn(0x7fdff00b1780 msgr2=0x7fdff01391a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 shutdown_connections 2024-09-10T21:29:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.465+0000 7fdff5600640 1 -- 172.21.15.97:0/2909200684 wait complete. 2024-09-10T21:29:31.489 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-10T21:29:31.490 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-10T21:29:31.490 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2024-09-10T21:29:31.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:29:31.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- 172.21.15.97:0/1468524450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440cec10 msgr2=0x7fe4440cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 --2- 172.21.15.97:0/1468524450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440cec10 0x7fe4440cff10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe444069d60 tx=0x7fe42c01eff0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- 172.21.15.97:0/1468524450 shutdown_connections 2024-09-10T21:29:31.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 --2- 172.21.15.97:0/1468524450 >> v2:172.21.15.194:3300/0 conn(0x7fe4440b1560 0x7fe4440b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 --2- 172.21.15.97:0/1468524450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440cec10 0x7fe4440cff10 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- 172.21.15.97:0/1468524450 >> 172.21.15.97:0/1468524450 conn(0x7fe4440b37c0 msgr2=0x7fe4440b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:31.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- 172.21.15.97:0/1468524450 shutdown_connections 2024-09-10T21:29:31.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- 172.21.15.97:0/1468524450 wait complete. 2024-09-10T21:29:31.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 Processor -- start 2024-09-10T21:29:31.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- start start 2024-09-10T21:29:31.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4440b1560 0x7fe44415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440ce830 0x7fe44415dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4440cec10 0x7fe4441f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440ce1a0 con 0x7fe4440ce830 2024-09-10T21:29:31.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440cde60 con 0x7fe4440cec10 2024-09-10T21:29:31.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe44aa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440ce320 con 0x7fe4440b1560 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe43b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440ce830 0x7fe44415dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe43b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440ce830 0x7fe44415dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49650/0 (socket says 172.21.15.97:49650) 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe43b400640 1 -- 172.21.15.97:0/3516463124 learned_addr learned my addr 172.21.15.97:0/3516463124 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe448a00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4440cec10 0x7fe4441f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.629+0000 7fe443e00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4440b1560 0x7fe44415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe43b400640 1 -- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4440b1560 msgr2=0x7fe44415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe43b400640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4440b1560 0x7fe44415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe43b400640 1 -- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4440cec10 msgr2=0x7fe4441f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe43b400640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4440cec10 0x7fe4441f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe43b400640 1 -- 172.21.15.97:0/3516463124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4441f4b50 con 0x7fe4440ce830 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe443e00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4440b1560 0x7fe44415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:31.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe43b400640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440ce830 0x7fe44415dd30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000c10 tx=0x7fe42c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:31.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe440c00640 1 -- 172.21.15.97:0/3516463124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0be070 con 0x7fe4440ce830 2024-09-10T21:29:31.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe440c00640 1 -- 172.21.15.97:0/3516463124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe42c0053e0 con 0x7fe4440ce830 2024-09-10T21:29:31.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe440c00640 1 -- 172.21.15.97:0/3516463124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0b5040 con 0x7fe4440ce830 2024-09-10T21:29:31.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4441f4e30 con 0x7fe4440ce830 2024-09-10T21:29:31.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4441f53f0 con 0x7fe4440ce830 2024-09-10T21:29:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4440cffc0 con 0x7fe4440ce830 2024-09-10T21:29:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe440c00640 1 -- 172.21.15.97:0/3516463124 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe42c0b8030 con 0x7fe4440ce830 2024-09-10T21:29:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe440c00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4200725e0 0x7fe420074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.633+0000 7fe440c00640 1 -- 172.21.15.97:0/3516463124 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7fe42c12fcc0 con 0x7fe4440ce830 2024-09-10T21:29:31.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.637+0000 7fe443e00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4200725e0 0x7fe420074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.637+0000 7fe443e00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4200725e0 0x7fe420074aa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe434007fd0 tx=0x7fe434009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:31.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.637+0000 7fe440c00640 1 -- 172.21.15.97:0/3516463124 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe42c136080 con 0x7fe4440ce830 2024-09-10T21:29:31.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.805+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe44406f080 con 0x7fe4440ce830 2024-09-10T21:29:31.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.805+0000 7fe440c00640 1 -- 172.21.15.97:0/3516463124 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13592 (secure 0 0 0) 0x7fe42c018b50 con 0x7fe4440ce830 2024-09-10T21:29:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:31.810 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":74,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:29:30.458948+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6835/1482860240":"2024-09-11T21:29:27.448304+0000","172.21.15.97:6835/3391319282":"2024-09-11T21:27:46.185567+0000","172.21.15.97:6834/3391319282":"2024-09-11T21:27:46.185567+0000","172.21.15.97:6835/2609945029":"2024-09-11T21:27:10.383537+0000","172.21.15.194:6837/1482860240":"2024-09-11T21:29:27.448304+0000","172.21.15.97:6834/2609945029":"2024-09-11T21:27:10.383537+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:31.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4200725e0 msgr2=0x7fe420074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4200725e0 0x7fe420074aa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe434007fd0 tx=0x7fe434009040 comp rx=0 tx=0).stop 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440ce830 msgr2=0x7fe44415dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440ce830 0x7fe44415dd30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000c10 tx=0x7fe42c005dd0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 shutdown_connections 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe4200725e0 0x7fe420074aa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4440cec10 0x7fe4441f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440ce830 0x7fe44415dd30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 --2- 172.21.15.97:0/3516463124 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe4440b1560 0x7fe44415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 >> 172.21.15.97:0/3516463124 conn(0x7fe4440b37c0 msgr2=0x7fe4440b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 shutdown_connections 2024-09-10T21:29:31.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.809+0000 7fe44aa00640 1 -- 172.21.15.97:0/3516463124 wait complete. 2024-09-10T21:29:31.831 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:24:02.092708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:29:31.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:29:31.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.973+0000 7fb5dc000640 1 -- 172.21.15.97:0/4193915753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5bc004d50 msgr2=0x7fb5bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.973+0000 7fb5dc000640 1 --2- 172.21.15.97:0/4193915753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5bc004d50 0x7fb5bc0051d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c640 tx=0x7fb5cc01d320 comp rx=0 tx=0).stop 2024-09-10T21:29:31.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.973+0000 7fb5da000640 1 -- 172.21.15.97:0/4193915753 reap_dead start 2024-09-10T21:29:31.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.973+0000 7fb5dc000640 1 -- 172.21.15.97:0/4193915753 shutdown_connections 2024-09-10T21:29:31.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.973+0000 7fb5dc000640 1 -- 172.21.15.97:0/4193915753 >> 172.21.15.97:0/4193915753 conn(0x7fb5d40b1780 msgr2=0x7fb5d40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:31.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.973+0000 7fb5dc000640 1 -- 172.21.15.97:0/4193915753 shutdown_connections 2024-09-10T21:29:31.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.973+0000 7fb5dc000640 1 -- 172.21.15.97:0/4193915753 wait complete. 2024-09-10T21:29:31.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 Processor -- start 2024-09-10T21:29:31.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 -- start start 2024-09-10T21:29:31.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5bc004d50 0x7fb5d415de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5d41377e0 0x7fb5d415e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d4155350 0x7fb5d41f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4141d20 con 0x7fb5d4155350 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4140bd0 con 0x7fb5bc004d50 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d4141ea0 con 0x7fb5d41377e0 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d4155350 0x7fb5d41f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d4155350 0x7fb5d41f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49656/0 (socket says 172.21.15.97:49656) 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 -- 172.21.15.97:0/3730450805 learned_addr learned my addr 172.21.15.97:0/3730450805 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5d8c00640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5d41377e0 0x7fb5d415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5d9600640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5bc004d50 0x7fb5d415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 -- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5d41377e0 msgr2=0x7fb5d415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5d41377e0 0x7fb5d415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 -- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5bc004d50 msgr2=0x7fb5d415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5bc004d50 0x7fb5d415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 -- 172.21.15.97:0/3730450805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d40c99d0 con 0x7fb5d4155350 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5d9600640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5bc004d50 0x7fb5d415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5da000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d4155350 0x7fb5d41f7f30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc098ef0 tx=0x7fb5cc0a5fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5d1600640 1 -- 172.21.15.97:0/3730450805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc004e40 con 0x7fb5d4155350 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5d1600640 1 -- 172.21.15.97:0/3730450805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5cc0b3040 con 0x7fb5d4155350 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d40c9cb0 con 0x7fb5d4155350 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5d1600640 1 -- 172.21.15.97:0/3730450805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc003dd0 con 0x7fb5d4155350 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.977+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d40cc0b0 con 0x7fb5d4155350 2024-09-10T21:29:31.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.981+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d40c90c0 con 0x7fb5d4155350 2024-09-10T21:29:31.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.981+0000 7fb5d1600640 1 -- 172.21.15.97:0/3730450805 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb5cc006070 con 0x7fb5d4155350 2024-09-10T21:29:31.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.981+0000 7fb5d1600640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5b40725e0 0x7fb5b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.981+0000 7fb5d1600640 1 -- 172.21.15.97:0/3730450805 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7fb5cc12df40 con 0x7fb5d4155350 2024-09-10T21:29:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.981+0000 7fb5d9600640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5b40725e0 0x7fb5b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.981+0000 7fb5d9600640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5b40725e0 0x7fb5b4074aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009b70 tx=0x7fb5c80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:31.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:31.985+0000 7fb5d1600640 1 -- 172.21.15.97:0/3730450805 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb5cc0c0060 con 0x7fb5d4155350 2024-09-10T21:29:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.149+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5d40cf100 con 0x7fb5d4155350 2024-09-10T21:29:32.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5d1600640 1 -- 172.21.15.97:0/3730450805 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fb5cc0170d0 con 0x7fb5d4155350 2024-09-10T21:29:32.154 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5b40725e0 msgr2=0x7fb5b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5b40725e0 0x7fb5b4074aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009b70 tx=0x7fb5c80091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d4155350 msgr2=0x7fb5d41f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d4155350 0x7fb5d41f7f30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc098ef0 tx=0x7fb5cc0a5fd0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 shutdown_connections 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb5b40725e0 0x7fb5b4074aa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5d4155350 0x7fb5d41f7f30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb5d41377e0 0x7fb5d415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 --2- 172.21.15.97:0/3730450805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5bc004d50 0x7fb5d415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 >> 172.21.15.97:0/3730450805 conn(0x7fb5d40b1780 msgr2=0x7fb5d4155980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 shutdown_connections 2024-09-10T21:29:32.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.153+0000 7fb5dc000640 1 -- 172.21.15.97:0/3730450805 wait complete. 2024-09-10T21:29:32.176 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 -- 172.21.15.97:0/2507708039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40b0d50 msgr2=0x7f5ed40ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 --2- 172.21.15.97:0/2507708039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40b0d50 0x7f5ed40ccc10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009870 tx=0x7f5ecc01c820 comp rx=0 tx=0).stop 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 -- 172.21.15.97:0/2507708039 shutdown_connections 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 --2- 172.21.15.97:0/2507708039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40b0d50 0x7f5ed40ccc10 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 --2- 172.21.15.97:0/2507708039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ed40afdd0 0x7f5ed40cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 -- 172.21.15.97:0/2507708039 >> 172.21.15.97:0/2507708039 conn(0x7f5ed40d1c30 msgr2=0x7f5ed40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 -- 172.21.15.97:0/2507708039 shutdown_connections 2024-09-10T21:29:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.313+0000 7f5edc600640 1 -- 172.21.15.97:0/2507708039 wait complete. 2024-09-10T21:29:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 Processor -- start 2024-09-10T21:29:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 -- start start 2024-09-10T21:29:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40afdd0 0x7f5ed41f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ed40b0d50 0x7f5ed4154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ed40ce640 0x7f5ed4071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4168e90 con 0x7f5ed40afdd0 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4168d10 con 0x7f5ed40b0d50 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed4169010 con 0x7f5ed40ce640 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40afdd0 0x7f5ed41f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40afdd0 0x7f5ed41f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49676/0 (socket says 172.21.15.97:49676) 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 -- 172.21.15.97:0/3725513419 learned_addr learned my addr 172.21.15.97:0/3725513419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5eda600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ed40ce640 0x7f5ed4071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9200640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ed40b0d50 0x7f5ed4154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 -- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ed40ce640 msgr2=0x7f5ed4071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ed40ce640 0x7f5ed4071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 -- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ed40b0d50 msgr2=0x7f5ed4154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ed40b0d50 0x7f5ed4154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 -- 172.21.15.97:0/3725513419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41f8c30 con 0x7f5ed40afdd0 2024-09-10T21:29:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed9c00640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40afdd0 0x7f5ed41f7ad0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800cbc0 tx=0x7f5ec800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed1600640 1 -- 172.21.15.97:0/3725513419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8016020 con 0x7f5ed40afdd0 2024-09-10T21:29:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed1600640 1 -- 172.21.15.97:0/3725513419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec800ab40 con 0x7f5ed40afdd0 2024-09-10T21:29:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41f80b0 con 0x7f5ed40afdd0 2024-09-10T21:29:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5ed1600640 1 -- 172.21.15.97:0/3725513419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800ce40 con 0x7f5ed40afdd0 2024-09-10T21:29:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.317+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41f8670 con 0x7f5ed40afdd0 2024-09-10T21:29:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.321+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed40cea20 con 0x7f5ed40afdd0 2024-09-10T21:29:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.321+0000 7f5ed1600640 1 -- 172.21.15.97:0/3725513419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5ec801b400 con 0x7f5ed40afdd0 2024-09-10T21:29:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.321+0000 7f5ed1600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5eb40725e0 0x7f5eb4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.321+0000 7f5ed1600640 1 -- 172.21.15.97:0/3725513419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f5ec8093720 con 0x7f5ed40afdd0 2024-09-10T21:29:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.321+0000 7f5ed9200640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5eb40725e0 0x7f5eb4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.321+0000 7f5ed9200640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5eb40725e0 0x7f5eb4074aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00058d0 tx=0x7f5ec0005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.325+0000 7f5ed1600640 1 -- 172.21.15.97:0/3725513419 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5ec8020020 con 0x7f5ed40afdd0 2024-09-10T21:29:32.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:32.549+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7f5ed41597d0 con 0x7f5ed40afdd0 2024-09-10T21:29:33.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.469+0000 7f5ed1600640 1 -- 172.21.15.97:0/3725513419 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v258) ==== 144+0+0 (secure 0 0 0) 0x7f5ec805ee20 con 0x7f5ed40afdd0 2024-09-10T21:29:33.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5eb40725e0 msgr2=0x7f5eb4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5eb40725e0 0x7f5eb4074aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00058d0 tx=0x7f5ec0005860 comp rx=0 tx=0).stop 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40afdd0 msgr2=0x7f5ed41f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40afdd0 0x7f5ed41f7ad0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800cbc0 tx=0x7f5ec800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 shutdown_connections 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5eb40725e0 0x7f5eb4074aa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ed40ce640 0x7f5ed4071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ed40b0d50 0x7f5ed4154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 --2- 172.21.15.97:0/3725513419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed40afdd0 0x7f5ed41f7ad0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 >> 172.21.15.97:0/3725513419 conn(0x7f5ed40d1c30 msgr2=0x7f5ed40cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 shutdown_connections 2024-09-10T21:29:33.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.473+0000 7f5edc600640 1 -- 172.21.15.97:0/3725513419 wait complete. 2024-09-10T21:29:33.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:29:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.641+0000 7f8731e00640 1 -- 172.21.15.97:0/3981074526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 msgr2=0x7f872c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.641+0000 7f8731e00640 1 --2- 172.21.15.97:0/3981074526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0cea20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f871c009bc0 tx=0x7f871c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.641+0000 7f8731e00640 1 -- 172.21.15.97:0/3981074526 shutdown_connections 2024-09-10T21:29:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.641+0000 7f8731e00640 1 --2- 172.21.15.97:0/3981074526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0cea20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.641+0000 7f8731e00640 1 -- 172.21.15.97:0/3981074526 >> 172.21.15.97:0/3981074526 conn(0x7f872c0d1c30 msgr2=0x7f872c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.641+0000 7f8731e00640 1 -- 172.21.15.97:0/3981074526 shutdown_connections 2024-09-10T21:29:33.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.641+0000 7f8731e00640 1 -- 172.21.15.97:0/3981074526 wait complete. 2024-09-10T21:29:33.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 Processor -- start 2024-09-10T21:29:33.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 -- start start 2024-09-10T21:29:33.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f872c0b0d50 0x7f872c167180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f872c0cd6e0 0x7f872c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f872c16a8a0 con 0x7f872c0ce640 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f872c16a720 con 0x7f872c0cd6e0 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f872c16aa20 con 0x7f872c0b0d50 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49700/0 (socket says 172.21.15.97:49700) 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 -- 172.21.15.97:0/2887540799 learned_addr learned my addr 172.21.15.97:0/2887540799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872aa00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f872c0b0d50 0x7f872c167180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8722000640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f872c0cd6e0 0x7f872c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 -- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f872c0b0d50 msgr2=0x7f872c167180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f872c0b0d50 0x7f872c167180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 -- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f872c0cd6e0 msgr2=0x7f872c0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f872c0cd6e0 0x7f872c0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 -- 172.21.15.97:0/2887540799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c159380 con 0x7f872c0ce640 2024-09-10T21:29:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8722000640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f872c0cd6e0 0x7f872c0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f872b400640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0b22f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f872400c3c0 tx=0x7f872400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8723e00640 1 -- 172.21.15.97:0/2887540799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724017070 con 0x7f872c0ce640 2024-09-10T21:29:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8723e00640 1 -- 172.21.15.97:0/2887540799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87240053c0 con 0x7f872c0ce640 2024-09-10T21:29:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c1590a0 con 0x7f872c0ce640 2024-09-10T21:29:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8723e00640 1 -- 172.21.15.97:0/2887540799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724003860 con 0x7f872c0ce640 2024-09-10T21:29:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.645+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c0b15b0 con 0x7f872c0ce640 2024-09-10T21:29:33.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.649+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872c0cea20 con 0x7f872c0ce640 2024-09-10T21:29:33.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.649+0000 7f8723e00640 1 -- 172.21.15.97:0/2887540799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8724003a00 con 0x7f872c0ce640 2024-09-10T21:29:33.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.649+0000 7f8723e00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87080725e0 0x7f8708074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.649+0000 7f8723e00640 1 -- 172.21.15.97:0/2887540799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f8724092800 con 0x7f872c0ce640 2024-09-10T21:29:33.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.649+0000 7f872aa00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87080725e0 0x7f8708074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.649+0000 7f872aa00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87080725e0 0x7f8708074aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f871c0af000 tx=0x7f871c01b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:33.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.653+0000 7f8723e00640 1 -- 172.21.15.97:0/2887540799 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f872405df00 con 0x7f872c0ce640 2024-09-10T21:29:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.817+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f872c1570a0 con 0x7f872c0ce640 2024-09-10T21:29:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8723e00640 1 -- 172.21.15.97:0/2887540799 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v38) ==== 59+0+1998 (secure 0 0 0) 0x7f872405d800 con 0x7f872c0ce640 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3032.483965 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3656.533946 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3351.408335 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3363.485497 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 8164.284498 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4391.956543 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4391.204852 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4799.171341 2024-09-10T21:29:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds 20/20 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_ms 2/2 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_frag true 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_verify_scatter true 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced client_mount_timeout 600 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_client 20/20 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_ms 1/1 2024-09-10T21:29:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:29:33.825 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87080725e0 msgr2=0x7f8708074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87080725e0 0x7f8708074aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f871c0af000 tx=0x7f871c01b3e0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 msgr2=0x7f872c0b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0b22f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f872400c3c0 tx=0x7f872400c880 comp rx=0 tx=0).stop 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 shutdown_connections 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87080725e0 0x7f8708074aa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f872c0ce640 0x7f872c0b22f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f872c0cd6e0 0x7f872c0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 --2- 172.21.15.97:0/2887540799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f872c0b0d50 0x7f872c167180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 >> 172.21.15.97:0/2887540799 conn(0x7f872c0d1c30 msgr2=0x7f872c0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 shutdown_connections 2024-09-10T21:29:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.821+0000 7f8731e00640 1 -- 172.21.15.97:0/2887540799 wait complete. 2024-09-10T21:29:33.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:29:33.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 -- 172.21.15.97:0/2913138161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100cde30 msgr2=0x7f4f100cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2913138161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100cde30 0x7f4f100cf0d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4f10073600 tx=0x7f4f0801cb80 comp rx=0 tx=0).stop 2024-09-10T21:29:33.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 -- 172.21.15.97:0/2913138161 shutdown_connections 2024-09-10T21:29:33.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2913138161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100cde30 0x7f4f100cf0d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2913138161 >> v2:172.21.15.194:3300/0 conn(0x7f4f100ce5e0 0x7f4f100b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 -- 172.21.15.97:0/2913138161 >> 172.21.15.97:0/2913138161 conn(0x7f4f100d1ca0 msgr2=0x7f4f100d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:33.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 -- 172.21.15.97:0/2913138161 shutdown_connections 2024-09-10T21:29:33.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.985+0000 7f4f17a00640 1 -- 172.21.15.97:0/2913138161 wait complete. 2024-09-10T21:29:33.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 Processor -- start 2024-09-10T21:29:33.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 -- start start 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f100cde30 0x7f4f100b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f100ce5e0 0x7f4f1006f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100d0710 0x7f4f10150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f100cd350 con 0x7f4f100d0710 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4f100cd1d0 con 0x7f4f100cde30 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4f10169060 con 0x7f4f100ce5e0 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100d0710 0x7f4f10150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f100cde30 0x7f4f100b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100d0710 0x7f4f10150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49716/0 (socket says 172.21.15.97:49716) 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 -- 172.21.15.97:0/2953858250 learned_addr learned my addr 172.21.15.97:0/2953858250 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f07e00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f100ce5e0 0x7f4f1006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 -- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f100ce5e0 msgr2=0x7f4f1006f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f100ce5e0 0x7f4f1006f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 -- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f100cde30 msgr2=0x7f4f100b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f100cde30 0x7f4f100b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 -- 172.21.15.97:0/2953858250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f100b5200 con 0x7f4f100d0710 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f15a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100d0710 0x7f4f10150a10 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4f0801a7b0 tx=0x7f4f08009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f0d600640 1 -- 172.21.15.97:0/2953858250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080bb070 con 0x7f4f100d0710 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f0d600640 1 -- 172.21.15.97:0/2953858250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f08004d40 con 0x7f4f100d0710 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f0d600640 1 -- 172.21.15.97:0/2953858250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080b2040 con 0x7f4f100d0710 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f101569c0 con 0x7f4f100d0710 2024-09-10T21:29:33.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.989+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f101550f0 con 0x7f4f100d0710 2024-09-10T21:29:33.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.993+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f100b2010 con 0x7f4f100d0710 2024-09-10T21:29:33.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.993+0000 7f4f0d600640 1 -- 172.21.15.97:0/2953858250 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f080b5030 con 0x7f4f100d0710 2024-09-10T21:29:33.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.993+0000 7f4f0d600640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4eec0725e0 0x7f4eec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:33.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.993+0000 7f4f0d600640 1 -- 172.21.15.97:0/2953858250 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f4f0812cbf0 con 0x7f4f100d0710 2024-09-10T21:29:33.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.993+0000 7f4f15000640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4eec0725e0 0x7f4eec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:33.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.993+0000 7f4f15000640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4eec0725e0 0x7f4eec074aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009be0 tx=0x7f4f000091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:34.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:33.997+0000 7f4f0d600640 1 -- 172.21.15.97:0/2953858250 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4f080f1de0 con 0x7f4f100d0710 2024-09-10T21:29:34.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.165+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4f100d0060 con 0x7f4f100d0710 2024-09-10T21:29:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.169+0000 7f4f0d600640 1 -- 172.21.15.97:0/2953858250 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v39)=0 v39) ==== 122+0+0 (secure 0 0 0) 0x7f4f08016770 con 0x7f4f100d0710 2024-09-10T21:29:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4eec0725e0 msgr2=0x7f4eec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4eec0725e0 0x7f4eec074aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009be0 tx=0x7f4f000091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100d0710 msgr2=0x7f4f10150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100d0710 0x7f4f10150a10 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4f0801a7b0 tx=0x7f4f08009300 comp rx=0 tx=0).stop 2024-09-10T21:29:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 shutdown_connections 2024-09-10T21:29:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4eec0725e0 0x7f4eec074aa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f100d0710 0x7f4f10150a10 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f100ce5e0 0x7f4f1006f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 --2- 172.21.15.97:0/2953858250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f100cde30 0x7f4f100b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 >> 172.21.15.97:0/2953858250 conn(0x7f4f100d1ca0 msgr2=0x7f4f10070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 shutdown_connections 2024-09-10T21:29:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.173+0000 7f4f17a00640 1 -- 172.21.15.97:0/2953858250 wait complete. 2024-09-10T21:29:34.194 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:29:34.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:29:34.293 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:29:34.385 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:29:34.385 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:29:34.385 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:34.394 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:29:34.394 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:29:34.394 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:34.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:29:34.404 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:29:34.404 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:34.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:29:34.446 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:29:34.446 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:34.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:29:34.455 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:29:34.455 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:34.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 -- 172.21.15.97:0/2733631361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06540d18b0 msgr2=0x7f06540d22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:34.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 --2- 172.21.15.97:0/2733631361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06540d18b0 0x7f06540d22c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f064c009c80 tx=0x7f064c01cca0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 -- 172.21.15.97:0/2733631361 shutdown_connections 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 --2- 172.21.15.97:0/2733631361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06540d18b0 0x7f06540d22c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 --2- 172.21.15.97:0/2733631361 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06541614f0 0x7f06540cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 --2- 172.21.15.97:0/2733631361 >> v2:172.21.15.194:3300/0 conn(0x7f0654161110 0x7f06540d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 -- 172.21.15.97:0/2733631361 >> 172.21.15.97:0/2733631361 conn(0x7f06540d3ce0 msgr2=0x7f06540d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 -- 172.21.15.97:0/2733631361 shutdown_connections 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 -- 172.21.15.97:0/2733631361 wait complete. 2024-09-10T21:29:34.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 Processor -- start 2024-09-10T21:29:34.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.629+0000 7f0658a00640 1 -- start start 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06540d18b0 0x7f0654150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0654161110 0x7f06540b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06541614f0 0x7f06540b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0654161cd0 con 0x7f0654161110 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0654161b50 con 0x7f06540d18b0 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0654161e50 con 0x7f06541614f0 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0654161110 0x7f06540b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:34.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0654161110 0x7f06540b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49728/0 (socket says 172.21.15.97:49728) 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 -- 172.21.15.97:0/51617398 learned_addr learned my addr 172.21.15.97:0/51617398 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 -- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06541614f0 msgr2=0x7f06540b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06541614f0 0x7f06540b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 -- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06540d18b0 msgr2=0x7f0654150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0652800640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06540d18b0 0x7f0654150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06540d18b0 0x7f0654150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 -- 172.21.15.97:0/51617398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0654151140 con 0x7f0654161110 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0652800640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06540d18b0 0x7f0654150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0649e00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0654161110 0x7f06540b8130 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f063c00d650 tx=0x7f063c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f064b400640 1 -- 172.21.15.97:0/51617398 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c017070 con 0x7f0654161110 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f064b400640 1 -- 172.21.15.97:0/51617398 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f063c005310 con 0x7f0654161110 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f064b400640 1 -- 172.21.15.97:0/51617398 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c009820 con 0x7f0654161110 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065415c0e0 con 0x7f0654161110 2024-09-10T21:29:34.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06541514f0 con 0x7f0654161110 2024-09-10T21:29:34.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.633+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065406aea0 con 0x7f0654161110 2024-09-10T21:29:34.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.637+0000 7f064b400640 1 -- 172.21.15.97:0/51617398 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f063c0099c0 con 0x7f0654161110 2024-09-10T21:29:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.637+0000 7f064b400640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06300725e0 0x7f0630074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.637+0000 7f064b400640 1 -- 172.21.15.97:0/51617398 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7f063c0933a0 con 0x7f0654161110 2024-09-10T21:29:34.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.637+0000 7f0652800640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06300725e0 0x7f0630074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:34.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.637+0000 7f0652800640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06300725e0 0x7f0630074aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0644009ad0 tx=0x7f06440094d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:34.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.641+0000 7f064b400640 1 -- 172.21.15.97:0/51617398 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f063c05ea10 con 0x7f0654161110 2024-09-10T21:29:34.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.837+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f065415bed0 con 0x7f0654161110 2024-09-10T21:29:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.837+0000 7f064b400640 1 -- 172.21.15.97:0/51617398 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+3914 (secure 0 0 0) 0x7f063c0671c0 con 0x7f0654161110 2024-09-10T21:29:34.842 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:34.842 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":63,"btime":"2024-09-10T21:29:28:450776+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5689,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3050022485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3050022485},{"type":"v1","addr":"172.21.15.194:6836","nonce":3050022485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5949,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3018242820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3018242820},{"type":"v1","addr":"172.21.15.97:6833","nonce":3018242820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6073,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/315967985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":315967985},{"type":"v1","addr":"172.21.15.194:6837","nonce":315967985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6282,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2360657843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2360657843},{"type":"v1","addr":"172.21.15.97:6835","nonce":2360657843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-10T21:29:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 63 2024-09-10T21:29:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06300725e0 msgr2=0x7f0630074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06300725e0 0x7f0630074aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0644009ad0 tx=0x7f06440094d0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0654161110 msgr2=0x7f06540b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0654161110 0x7f06540b8130 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f063c00d650 tx=0x7f063c00db10 comp rx=0 tx=0).stop 2024-09-10T21:29:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 shutdown_connections 2024-09-10T21:29:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06300725e0 0x7f0630074aa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06541614f0 0x7f06540b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0654161110 0x7f06540b8130 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 --2- 172.21.15.97:0/51617398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06540d18b0 0x7f0654150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 >> 172.21.15.97:0/51617398 conn(0x7f06540d3ce0 msgr2=0x7f06540b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 shutdown_connections 2024-09-10T21:29:34.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:34.841+0000 7f0658a00640 1 -- 172.21.15.97:0/51617398 wait complete. 2024-09-10T21:29:34.863 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:29:34.863 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:29:34.863 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:29:34.863 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:29:34.864 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:29:34.864 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:29:34.864 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:29:34.864 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:29:34.864 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:29:34.864 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:29:34.864 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:29:34.864 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:29:34.865 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:29:34.868 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:29:34.869 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:29:34.869 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:29:34.876 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:29:34.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:29:34.877 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:29:34.877 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:29:34.879 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:29:34.879 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:29:34.882 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:29:34.883 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:29:34.921 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:29:34.921 DEBUG:teuthology.parallel:result is None 2024-09-10T21:29:34.922 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:29:34.922 DEBUG:teuthology.parallel:result is None 2024-09-10T21:29:34.928 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:29:34.928 DEBUG:teuthology.parallel:result is None 2024-09-10T21:29:34.930 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:29:34.930 DEBUG:teuthology.parallel:result is None 2024-09-10T21:29:34.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:29:35.008 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:29:35.008 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:29:35.021 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:29:35.021 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 -- 172.21.15.97:0/1088131612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 msgr2=0x7fafb80b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 --2- 172.21.15.97:0/1088131612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80b37b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009840 tx=0x7fafa801efd0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 -- 172.21.15.97:0/1088131612 shutdown_connections 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 --2- 172.21.15.97:0/1088131612 >> v2:172.21.15.194:3300/0 conn(0x7fafb80b3cf0 0x7fafb806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 --2- 172.21.15.97:0/1088131612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80b37b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 -- 172.21.15.97:0/1088131612 >> 172.21.15.97:0/1088131612 conn(0x7fafb80b1780 msgr2=0x7fafb80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 -- 172.21.15.97:0/1088131612 shutdown_connections 2024-09-10T21:29:35.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.077+0000 7fafbfa00640 1 -- 172.21.15.97:0/1088131612 wait complete. 2024-09-10T21:29:35.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 Processor -- start 2024-09-10T21:29:35.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 -- start start 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb806f180 0x7fafb81eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fafb80b3cf0 0x7fafb80a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fafb806e740 con 0x7fafb81511f0 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fafb806e5c0 con 0x7fafb806f180 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fafb806e8c0 con 0x7fafb80b3cf0 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80aa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49768/0 (socket says 172.21.15.97:49768) 2024-09-10T21:29:35.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 -- 172.21.15.97:0/2022459421 learned_addr learned my addr 172.21.15.97:0/2022459421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbd000640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb806f180 0x7fafb81eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafafe00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fafb80b3cf0 0x7fafb80a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 -- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fafb80b3cf0 msgr2=0x7fafb80a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fafb80b3cf0 0x7fafb80a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 -- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb806f180 msgr2=0x7fafb81eaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb806f180 0x7fafb81eaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 -- 172.21.15.97:0/2022459421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81eb3a0 con 0x7fafb81511f0 2024-09-10T21:29:35.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbda00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80aa3a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c730 tx=0x7fafb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafb5600640 1 -- 172.21.15.97:0/2022459421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003db0 con 0x7fafb81511f0 2024-09-10T21:29:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafb5600640 1 -- 172.21.15.97:0/2022459421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb000ce40 con 0x7fafb81511f0 2024-09-10T21:29:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81eb680 con 0x7fafb81511f0 2024-09-10T21:29:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafb5600640 1 -- 172.21.15.97:0/2022459421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00052d0 con 0x7fafb81511f0 2024-09-10T21:29:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.081+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb80aa8e0 con 0x7fafb81511f0 2024-09-10T21:29:35.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.085+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb81ebb30 con 0x7fafb81511f0 2024-09-10T21:29:35.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.085+0000 7fafb5600640 1 -- 172.21.15.97:0/2022459421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fafb0005470 con 0x7fafb81511f0 2024-09-10T21:29:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.085+0000 7fafb5600640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf940725e0 0x7faf94074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.085+0000 7fafb5600640 1 -- 172.21.15.97:0/2022459421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5507+0+0 (secure 0 0 0) 0x7fafb00938a0 con 0x7fafb81511f0 2024-09-10T21:29:35.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.085+0000 7fafbd000640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf940725e0 0x7faf94074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.085+0000 7fafbd000640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf940725e0 0x7faf94074aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fafa80b2000 tx=0x7fafa801d810 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:35.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.089+0000 7fafb5600640 1 -- 172.21.15.97:0/2022459421 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fafb005eb60 con 0x7fafb81511f0 2024-09-10T21:29:35.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.265+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fafb80b37b0 con 0x7fafb81511f0 2024-09-10T21:29:35.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.489+0000 7fafb5600640 1 -- 172.21.15.97:0/2022459421 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7fafb0063590 con 0x7fafb81511f0 2024-09-10T21:29:35.492 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf940725e0 msgr2=0x7faf94074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf940725e0 0x7faf94074aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fafa80b2000 tx=0x7fafa801d810 comp rx=0 tx=0).stop 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 msgr2=0x7fafb80aa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80aa3a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c730 tx=0x7fafb000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 shutdown_connections 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf940725e0 0x7faf94074aa0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb81511f0 0x7fafb80aa3a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fafb80b3cf0 0x7fafb80a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 --2- 172.21.15.97:0/2022459421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafb806f180 0x7fafb81eaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 >> 172.21.15.97:0/2022459421 conn(0x7fafb80b1780 msgr2=0x7fafb815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 shutdown_connections 2024-09-10T21:29:35.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.493+0000 7fafbfa00640 1 -- 172.21.15.97:0/2022459421 wait complete. 2024-09-10T21:29:35.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- 172.21.15.97:0/4282065277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 msgr2=0x7fe8540b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 --2- 172.21.15.97:0/4282065277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b42e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe854071550 tx=0x7fe84c01cac0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- 172.21.15.97:0/4282065277 shutdown_connections 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 --2- 172.21.15.97:0/4282065277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b42e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 --2- 172.21.15.97:0/4282065277 >> v2:172.21.15.194:3300/0 conn(0x7fe85406d180 0x7fe854070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- 172.21.15.97:0/4282065277 >> 172.21.15.97:0/4282065277 conn(0x7fe8540b2790 msgr2=0x7fe8540b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- 172.21.15.97:0/4282065277 shutdown_connections 2024-09-10T21:29:35.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- 172.21.15.97:0/4282065277 wait complete. 2024-09-10T21:29:35.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 Processor -- start 2024-09-10T21:29:35.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- start start 2024-09-10T21:29:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe85406cda0 0x7fe854159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe85406d180 0x7fe85413d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe85416b120 con 0x7fe854161820 2024-09-10T21:29:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe85416afa0 con 0x7fe85406cda0 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85c600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe85416b2a0 con 0x7fe85406d180 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49790/0 (socket says 172.21.15.97:49790) 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 -- 172.21.15.97:0/4291771732 learned_addr learned my addr 172.21.15.97:0/4291771732 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe859c00640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe85406cda0 0x7fe854159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe859200640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe85406d180 0x7fe85413d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 -- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe85406d180 msgr2=0x7fe85413d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe85406d180 0x7fe85413d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 -- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe85406cda0 msgr2=0x7fe854159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe85406cda0 0x7fe854159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 -- 172.21.15.97:0/4291771732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85413dad0 con 0x7fe854161820 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe85a600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b18f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe84c01a6f0 tx=0x7fe84c0a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.657+0000 7fe851600640 1 -- 172.21.15.97:0/4291771732 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0bc070 con 0x7fe854161820 2024-09-10T21:29:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe851600640 1 -- 172.21.15.97:0/4291771732 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe84c0ad730 con 0x7fe854161820 2024-09-10T21:29:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe851600640 1 -- 172.21.15.97:0/4291771732 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0b2040 con 0x7fe854161820 2024-09-10T21:29:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8541fb3b0 con 0x7fe854161820 2024-09-10T21:29:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541fb970 con 0x7fe854161820 2024-09-10T21:29:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe854070130 con 0x7fe854161820 2024-09-10T21:29:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe851600640 1 -- 172.21.15.97:0/4291771732 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe84c0b5070 con 0x7fe854161820 2024-09-10T21:29:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe851600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8340725e0 0x7fe834074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe851600640 1 -- 172.21.15.97:0/4291771732 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7fe84c12c680 con 0x7fe854161820 2024-09-10T21:29:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.661+0000 7fe859c00640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8340725e0 0x7fe834074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.665+0000 7fe859c00640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8340725e0 0x7fe834074aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe848000f70 tx=0x7fe848009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:35.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.665+0000 7fe851600640 1 -- 172.21.15.97:0/4291771732 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe84c016770 con 0x7fe854161820 2024-09-10T21:29:35.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe8540ad8b0 con 0x7fe854161820 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe851600640 1 -- 172.21.15.97:0/4291771732 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fe84c0f7830 con 0x7fe854161820 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA==", 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g==", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:29:35.893 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw==", 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA==", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q==", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q==", 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.896 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg==", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ==", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.897 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ==", 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-09-10T21:29:35.898 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw==", 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ==", 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q==", 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q==", 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.900 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ==", 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag==", 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA==", 2024-09-10T21:29:35.902 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmjCI0FRAATFUoCxw0bU6HSEUnlyrVJQ==", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.903 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA==", 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw==", 2024-09-10T21:29:35.904 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:29:35.905 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:29:35.905 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:29:35.905 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:29:35.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:35.905 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-10T21:29:35.905 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8340725e0 msgr2=0x7fe834074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8340725e0 0x7fe834074aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe848000f70 tx=0x7fe848009040 comp rx=0 tx=0).stop 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 msgr2=0x7fe8540b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b18f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe84c01a6f0 tx=0x7fe84c0a5670 comp rx=0 tx=0).stop 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 shutdown_connections 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8340725e0 0x7fe834074aa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe854161820 0x7fe8540b18f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe85406d180 0x7fe85413d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 --2- 172.21.15.97:0/4291771732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe85406cda0 0x7fe854159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.889+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 >> 172.21.15.97:0/4291771732 conn(0x7fe8540b2790 msgr2=0x7fe8540b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:35.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.893+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 shutdown_connections 2024-09-10T21:29:35.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:35.893+0000 7fe85c600640 1 -- 172.21.15.97:0/4291771732 wait complete. 2024-09-10T21:29:35.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:29:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- 172.21.15.97:0/2599988025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 msgr2=0x7f9a8400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 --2- 172.21.15.97:0/2599988025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 0x7f9a8400b9d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800c610 tx=0x7f9a9801d280 comp rx=0 tx=0).stop 2024-09-10T21:29:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa5a00640 1 -- 172.21.15.97:0/2599988025 reap_dead start 2024-09-10T21:29:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- 172.21.15.97:0/2599988025 shutdown_connections 2024-09-10T21:29:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- 172.21.15.97:0/2599988025 >> 172.21.15.97:0/2599988025 conn(0x7f9aa00b37c0 msgr2=0x7f9aa00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- 172.21.15.97:0/2599988025 shutdown_connections 2024-09-10T21:29:36.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- 172.21.15.97:0/2599988025 wait complete. 2024-09-10T21:29:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 Processor -- start 2024-09-10T21:29:36.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- start start 2024-09-10T21:29:36.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 0x7f9aa00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9aa01e5ac0 0x7f9aa01f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01e5f40 0x7f9aa006e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa00ce1a0 con 0x7f9a840051d0 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa00cde60 con 0x7f9aa01e5f40 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa7a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa00ce320 con 0x7f9aa01e5ac0 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.053+0000 7f9aa5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 0x7f9aa00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 0x7f9aa00aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49806/0 (socket says 172.21.15.97:49806) 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 -- 172.21.15.97:0/142887431 learned_addr learned my addr 172.21.15.97:0/142887431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5a00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01e5f40 0x7f9aa006e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9a97e00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9aa01e5ac0 0x7f9aa01f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 -- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9aa01e5ac0 msgr2=0x7f9aa01f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9aa01e5ac0 0x7f9aa01f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 -- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01e5f40 msgr2=0x7f9aa006e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01e5f40 0x7f9aa006e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 -- 172.21.15.97:0/142887431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa01f58a0 con 0x7f9a840051d0 2024-09-10T21:29:36.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa5000640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 0x7f9aa00aef80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9a90003f20 tx=0x7f9a900017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:36.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9a9d600640 1 -- 172.21.15.97:0/142887431 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90015070 con 0x7f9a840051d0 2024-09-10T21:29:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9a9d600640 1 -- 172.21.15.97:0/142887431 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a90011900 con 0x7f9a840051d0 2024-09-10T21:29:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9a9d600640 1 -- 172.21.15.97:0/142887431 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90011d50 con 0x7f9a840051d0 2024-09-10T21:29:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa01f5b80 con 0x7f9a840051d0 2024-09-10T21:29:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa01e55c0 con 0x7f9a840051d0 2024-09-10T21:29:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa00d0200 con 0x7f9a840051d0 2024-09-10T21:29:36.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.057+0000 7f9a9d600640 1 -- 172.21.15.97:0/142887431 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9a900103c0 con 0x7f9a840051d0 2024-09-10T21:29:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.061+0000 7f9a9d600640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a7c0725e0 0x7f9a7c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.061+0000 7f9a9d600640 1 -- 172.21.15.97:0/142887431 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f9a900940f0 con 0x7f9a840051d0 2024-09-10T21:29:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.061+0000 7f9a97e00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a7c0725e0 0x7f9a7c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.061+0000 7f9a97e00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a7c0725e0 0x7f9a7c074aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00c8770 tx=0x7f9a880091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.065+0000 7f9a9d600640 1 -- 172.21.15.97:0/142887431 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a9005f340 con 0x7f9a840051d0 2024-09-10T21:29:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9aa00c87a0 con 0x7f9a840051d0 2024-09-10T21:29:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9a9d600640 1 -- 172.21.15.97:0/142887431 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7f9a90063d70 con 0x7f9a840051d0 2024-09-10T21:29:36.235 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:36.235 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:29:36.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:29:36.236 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-09-10T21:29:36.236 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-09-10T21:29:36.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:29:36.236 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:29:36.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a7c0725e0 msgr2=0x7f9a7c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a7c0725e0 0x7f9a7c074aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00c8770 tx=0x7f9a880091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 msgr2=0x7f9aa00aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 0x7f9aa00aef80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9a90003f20 tx=0x7f9a900017b0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 shutdown_connections 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a7c0725e0 0x7f9a7c074aa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01e5f40 0x7f9aa006e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9aa01e5ac0 0x7f9aa01f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 --2- 172.21.15.97:0/142887431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a840051d0 0x7f9aa00aef80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 >> 172.21.15.97:0/142887431 conn(0x7f9aa00b37c0 msgr2=0x7f9aa00cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 shutdown_connections 2024-09-10T21:29:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.233+0000 7f9aa7a00640 1 -- 172.21.15.97:0/142887431 wait complete. 2024-09-10T21:29:36.257 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:29:36.257 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:29:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 -- 172.21.15.97:0/1295584222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 msgr2=0x7f06a80af980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 --2- 172.21.15.97:0/1295584222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80af980 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f06a8073690 tx=0x7f06a001c770 comp rx=0 tx=0).stop 2024-09-10T21:29:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 -- 172.21.15.97:0/1295584222 shutdown_connections 2024-09-10T21:29:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 --2- 172.21.15.97:0/1295584222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80af980 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 --2- 172.21.15.97:0/1295584222 >> v2:172.21.15.194:3300/0 conn(0x7f06a8161600 0x7f06a80d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 -- 172.21.15.97:0/1295584222 >> 172.21.15.97:0/1295584222 conn(0x7f06a80d1c30 msgr2=0x7f06a80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 -- 172.21.15.97:0/1295584222 shutdown_connections 2024-09-10T21:29:36.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.397+0000 7f06ae400640 1 -- 172.21.15.97:0/1295584222 wait complete. 2024-09-10T21:29:36.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 Processor -- start 2024-09-10T21:29:36.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 -- start start 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06a8161600 0x7f06a814f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06a815d7e0 0x7f06a81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06a816b3f0 con 0x7f06a80b1590 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f06a816b270 con 0x7f06a815d7e0 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f06a816b570 con 0x7f06a8161600 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49818/0 (socket says 172.21.15.97:49818) 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 -- 172.21.15.97:0/4212245207 learned_addr learned my addr 172.21.15.97:0/4212245207 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7e00640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06a815d7e0 0x7f06a81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f069ea00640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06a8161600 0x7f06a814f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 -- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06a8161600 msgr2=0x7f06a814f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06a8161600 0x7f06a814f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 -- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06a815d7e0 msgr2=0x7f06a81595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06a815d7e0 0x7f06a81595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 -- 172.21.15.97:0/4212245207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a80b68a0 con 0x7f06a80b1590 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7e00640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06a815d7e0 0x7f06a81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06a7400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80b5900 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f069800dac0 tx=0x7f069800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f069fe00640 1 -- 172.21.15.97:0/4212245207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698017070 con 0x7f06a80b1590 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f069fe00640 1 -- 172.21.15.97:0/4212245207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0698004b30 con 0x7f06a80b1590 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f069fe00640 1 -- 172.21.15.97:0/4212245207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698005020 con 0x7f06a80b1590 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a80b6b80 con 0x7f06a80b1590 2024-09-10T21:29:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.401+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a80b7140 con 0x7f06a80b1590 2024-09-10T21:29:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.405+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a80af980 con 0x7f06a80b1590 2024-09-10T21:29:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.405+0000 7f069fe00640 1 -- 172.21.15.97:0/4212245207 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f06980051c0 con 0x7f06a80b1590 2024-09-10T21:29:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.405+0000 7f069fe00640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06840725e0 0x7f0684074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.405+0000 7f069fe00640 1 -- 172.21.15.97:0/4212245207 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5249+0+0 (secure 0 0 0) 0x7f06980935e0 con 0x7f06a80b1590 2024-09-10T21:29:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.405+0000 7f069ea00640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06840725e0 0x7f0684074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.405+0000 7f069ea00640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06840725e0 0x7f0684074aa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06a81fb370 tx=0x7f0690009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:36.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.409+0000 7f069fe00640 1 -- 172.21.15.97:0/4212245207 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0698020070 con 0x7f06a80b1590 2024-09-10T21:29:36.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:36.577+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f06a8072520 con 0x7f06a80b1590 2024-09-10T21:29:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.509+0000 7f069fe00640 1 -- 172.21.15.97:0/4212245207 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f069805ee70 con 0x7f06a80b1590 2024-09-10T21:29:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06840725e0 msgr2=0x7f0684074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06840725e0 0x7f0684074aa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06a81fb370 tx=0x7f0690009040 comp rx=0 tx=0).stop 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 msgr2=0x7f06a80b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80b5900 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f069800dac0 tx=0x7f069800df80 comp rx=0 tx=0).stop 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 shutdown_connections 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06840725e0 0x7f0684074aa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06a815d7e0 0x7f06a81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06a8161600 0x7f06a814f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 --2- 172.21.15.97:0/4212245207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06a80b1590 0x7f06a80b5900 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 >> 172.21.15.97:0/4212245207 conn(0x7f06a80d1c30 msgr2=0x7f06a80b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 shutdown_connections 2024-09-10T21:29:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.513+0000 7f06ae400640 1 -- 172.21.15.97:0/4212245207 wait complete. 2024-09-10T21:29:37.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:29:37.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- 172.21.15.97:0/4274724549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00b1560 msgr2=0x7f14c00b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 --2- 172.21.15.97:0/4274724549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00b1560 0x7f14c00b19b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009bc0 tx=0x7f14b801f3b0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- 172.21.15.97:0/4274724549 shutdown_connections 2024-09-10T21:29:37.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 --2- 172.21.15.97:0/4274724549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00b1560 0x7f14c00b19b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 --2- 172.21.15.97:0/4274724549 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14c00cec10 0x7f14c00cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- 172.21.15.97:0/4274724549 >> 172.21.15.97:0/4274724549 conn(0x7f14c00b37c0 msgr2=0x7f14c00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:37.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- 172.21.15.97:0/4274724549 shutdown_connections 2024-09-10T21:29:37.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- 172.21.15.97:0/4274724549 wait complete. 2024-09-10T21:29:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 Processor -- start 2024-09-10T21:29:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- start start 2024-09-10T21:29:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14c00b1560 0x7f14c00b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00ce830 0x7f14c015d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c00cec10 0x7f14c015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00ce1a0 con 0x7f14c00ce830 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00cde60 con 0x7f14c00cec10 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14c4a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f14c00ce320 con 0x7f14c00b1560 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.693+0000 7f14b5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00ce830 0x7f14c015d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00ce830 0x7f14c015d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34964/0 (socket says 172.21.15.97:34964) 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 -- 172.21.15.97:0/1879793662 learned_addr learned my addr 172.21.15.97:0/1879793662 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14bde00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14c00b1560 0x7f14c00b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14be800640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c00cec10 0x7f14c015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 -- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14c00b1560 msgr2=0x7f14c00b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14c00b1560 0x7f14c00b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 -- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c00cec10 msgr2=0x7f14c015d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c00cec10 0x7f14c015d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 -- 172.21.15.97:0/1879793662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c00b66f0 con 0x7f14c00ce830 2024-09-10T21:29:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14be800640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c00cec10 0x7f14c015d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b5400640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00ce830 0x7f14c015d260 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b410 tx=0x7f14a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b6a00640 1 -- 172.21.15.97:0/1879793662 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8015020 con 0x7f14c00ce830 2024-09-10T21:29:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b6a00640 1 -- 172.21.15.97:0/1879793662 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14a8011920 con 0x7f14c00ce830 2024-09-10T21:29:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c015c4a0 con 0x7f14c00ce830 2024-09-10T21:29:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b6a00640 1 -- 172.21.15.97:0/1879793662 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8011d70 con 0x7f14c00ce830 2024-09-10T21:29:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c015ca60 con 0x7f14c00ce830 2024-09-10T21:29:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c00cf800 con 0x7f14c00ce830 2024-09-10T21:29:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.697+0000 7f14b6a00640 1 -- 172.21.15.97:0/1879793662 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f14a801a400 con 0x7f14c00ce830 2024-09-10T21:29:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.701+0000 7f14b6a00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f149c0725e0 0x7f149c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.701+0000 7f14b6a00640 1 -- 172.21.15.97:0/1879793662 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5619+0+0 (secure 0 0 0) 0x7f14a80938e0 con 0x7f14c00ce830 2024-09-10T21:29:37.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.701+0000 7f14bde00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f149c0725e0 0x7f149c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.701+0000 7f14bde00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f149c0725e0 0x7f149c074aa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f14b0007fd0 tx=0x7f14b0007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.705+0000 7f14b6a00640 1 -- 172.21.15.97:0/1879793662 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f14a801f030 con 0x7f14c00ce830 2024-09-10T21:29:37.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:37.869+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f14c0071ca0 con 0x7f14c00ce830 2024-09-10T21:29:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14b6a00640 1 -- 172.21.15.97:0/1879793662 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7f14a805f0f0 con 0x7f14c00ce830 2024-09-10T21:29:38.519 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-09-10T21:29:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f149c0725e0 msgr2=0x7f149c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f149c0725e0 0x7f149c074aa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f14b0007fd0 tx=0x7f14b0007ea0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00ce830 msgr2=0x7f14c015d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00ce830 0x7f14c015d260 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b410 tx=0x7f14a800b8d0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 shutdown_connections 2024-09-10T21:29:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f149c0725e0 0x7f149c074aa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c00cec10 0x7f14c015d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14c00ce830 0x7f14c015d260 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 --2- 172.21.15.97:0/1879793662 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f14c00b1560 0x7f14c00b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.517+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 >> 172.21.15.97:0/1879793662 conn(0x7f14c00b37c0 msgr2=0x7f14c00721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.521+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 shutdown_connections 2024-09-10T21:29:38.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.521+0000 7f14c4a00640 1 -- 172.21.15.97:0/1879793662 wait complete. 2024-09-10T21:29:38.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:29:38.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 -- 172.21.15.97:0/2195558467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80cd6e0 msgr2=0x7f4ad80af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:38.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 --2- 172.21.15.97:0/2195558467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80cd6e0 0x7f4ad80af4d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009bc0 tx=0x7f4ad001cac0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 -- 172.21.15.97:0/2195558467 shutdown_connections 2024-09-10T21:29:38.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 --2- 172.21.15.97:0/2195558467 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80cd6e0 0x7f4ad80af4d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 --2- 172.21.15.97:0/2195558467 >> v2:172.21.15.194:3300/0 conn(0x7f4ad80ce640 0x7f4ad80cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 -- 172.21.15.97:0/2195558467 >> 172.21.15.97:0/2195558467 conn(0x7f4ad80d1c30 msgr2=0x7f4ad80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:38.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 -- 172.21.15.97:0/2195558467 shutdown_connections 2024-09-10T21:29:38.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.705+0000 7f4add400640 1 -- 172.21.15.97:0/2195558467 wait complete. 2024-09-10T21:29:38.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 Processor -- start 2024-09-10T21:29:38.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 -- start start 2024-09-10T21:29:38.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80b0d50 0x7f4ad8155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:38.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad80cd6e0 0x7f4ad81fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:38.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad80ce640 0x7f4ad80708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:38.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8168cc0 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad8168b40 con 0x7f4ad80ce640 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad8168e40 con 0x7f4ad80b0d50 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad80cd6e0 0x7f4ad81fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad80cd6e0 0x7f4ad81fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34968/0 (socket says 172.21.15.97:34968) 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 -- 172.21.15.97:0/3411819514 learned_addr learned my addr 172.21.15.97:0/3411819514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4ad7200640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad80ce640 0x7f4ad80708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4ad6800640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80b0d50 0x7f4ad8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 -- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80b0d50 msgr2=0x7f4ad8155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80b0d50 0x7f4ad8155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 -- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad80ce640 msgr2=0x7f4ad80708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad80ce640 0x7f4ad80708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 -- 172.21.15.97:0/3411819514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad806dff0 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acde00640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad80cd6e0 0x7f4ad81fbca0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c760 tx=0x7f4ac000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:38.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acf400640 1 -- 172.21.15.97:0/3411819514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0016020 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acf400640 1 -- 172.21.15.97:0/3411819514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ac000a8f0 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad806e2d0 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4acf400640 1 -- 172.21.15.97:0/3411819514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000ad40 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad806e890 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.709+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad80cea20 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.713+0000 7f4acf400640 1 -- 172.21.15.97:0/3411819514 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4ac001b400 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.713+0000 7f4acf400640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ab40725e0 0x7f4ab4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:38.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.713+0000 7f4acf400640 1 -- 172.21.15.97:0/3411819514 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6001+0+0 (secure 0 0 0) 0x7f4ac0093760 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.713+0000 7f4ad6800640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ab40725e0 0x7f4ab4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:38.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.713+0000 7f4ad6800640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ab40725e0 0x7f4ab4074aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009600 tx=0x7f4ac80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:38.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.717+0000 7f4acf400640 1 -- 172.21.15.97:0/3411819514 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ac0020020 con 0x7f4ad80cd6e0 2024-09-10T21:29:38.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:38.917+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4ad8155850 con 0x7f4ad80cd6e0 2024-09-10T21:29:39.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:39.177 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.517+0000 7f4acf400640 1 -- 172.21.15.97:0/3411819514 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v66) ==== 174+0+0 (secure 0 0 0) 0x7f4ac0067310 con 0x7f4ad80cd6e0 2024-09-10T21:29:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 11 and data pool 12 2024-09-10T21:29:39.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ab40725e0 msgr2=0x7f4ab4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ab40725e0 0x7f4ab4074aa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009600 tx=0x7f4ac80091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad80cd6e0 msgr2=0x7f4ad81fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad80cd6e0 0x7f4ad81fbca0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c760 tx=0x7f4ac000cc20 comp rx=0 tx=0).stop 2024-09-10T21:29:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 shutdown_connections 2024-09-10T21:29:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ab40725e0 0x7f4ab4074aa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ad80ce640 0x7f4ad80708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad80cd6e0 0x7f4ad81fbca0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 --2- 172.21.15.97:0/3411819514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ad80b0d50 0x7f4ad8155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.521+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 >> 172.21.15.97:0/3411819514 conn(0x7f4ad80d1c30 msgr2=0x7f4ad80b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.525+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 shutdown_connections 2024-09-10T21:29:39.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.525+0000 7f4add400640 1 -- 172.21.15.97:0/3411819514 wait complete. 2024-09-10T21:29:39.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 -- 172.21.15.97:0/1841239613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b400b1e00 msgr2=0x7f4b40161db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 --2- 172.21.15.97:0/1841239613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b400b1e00 0x7f4b40161db0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4b40069d60 tx=0x7f4b2801c810 comp rx=0 tx=0).stop 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 -- 172.21.15.97:0/1841239613 shutdown_connections 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 --2- 172.21.15.97:0/1841239613 >> v2:172.21.15.194:3300/0 conn(0x7f4b400b42e0 0x7f4b401622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 --2- 172.21.15.97:0/1841239613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b400b1e00 0x7f4b40161db0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 -- 172.21.15.97:0/1841239613 >> 172.21.15.97:0/1841239613 conn(0x7f4b400b37c0 msgr2=0x7f4b400b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 -- 172.21.15.97:0/1841239613 shutdown_connections 2024-09-10T21:29:39.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 -- 172.21.15.97:0/1841239613 wait complete. 2024-09-10T21:29:39.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 Processor -- start 2024-09-10T21:29:39.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 -- start start 2024-09-10T21:29:39.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b400b2bb0 0x7f4b401fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b400b42e0 0x7f4b40152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.709+0000 7f4b47400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401f9350 0x7f4b401531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b47400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400d16a0 con 0x7f4b401f9350 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b47400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400d1520 con 0x7f4b400b42e0 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b47400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4016b080 con 0x7f4b400b2bb0 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401f9350 0x7f4b401531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b400b2bb0 0x7f4b401fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b37e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b400b42e0 0x7f4b40152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b400b2bb0 0x7f4b401fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:40964/0 (socket says 172.21.15.97:40964) 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b37e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b400b42e0 0x7f4b40152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:59342/0 (socket says 172.21.15.97:59342) 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b44a00640 1 -- 172.21.15.97:0/1519268983 learned_addr learned my addr 172.21.15.97:0/1519268983 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:39.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401f9350 0x7f4b401531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34992/0 (socket says 172.21.15.97:34992) 2024-09-10T21:29:39.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 -- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b400b2bb0 msgr2=0x7f4b401fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b400b2bb0 0x7f4b401fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 -- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b400b42e0 msgr2=0x7f4b40152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b400b42e0 0x7f4b40152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 -- 172.21.15.97:0/1519268983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b401fb140 con 0x7f4b401f9350 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b45400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401f9350 0x7f4b401531f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e990 tx=0x7f4b3800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b3d600640 1 -- 172.21.15.97:0/1519268983 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38004d30 con 0x7f4b401f9350 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b3d600640 1 -- 172.21.15.97:0/1519268983 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b3800caa0 con 0x7f4b401f9350 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b3d600640 1 -- 172.21.15.97:0/1519268983 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38005020 con 0x7f4b401f9350 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b40155bd0 con 0x7f4b401f9350 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b401597b0 con 0x7f4b401f9350 2024-09-10T21:29:39.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b400b2f90 con 0x7f4b401f9350 2024-09-10T21:29:39.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.713+0000 7f4b3d600640 1 -- 172.21.15.97:0/1519268983 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4b3800f040 con 0x7f4b401f9350 2024-09-10T21:29:39.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.717+0000 7f4b3d600640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b1c072610 0x7f4b1c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.717+0000 7f4b3d600640 1 -- 172.21.15.97:0/1519268983 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f4b380937f0 con 0x7f4b401f9350 2024-09-10T21:29:39.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.717+0000 7f4b44a00640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b1c072610 0x7f4b1c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.717+0000 7f4b44a00640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b1c072610 0x7f4b1c074ad0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b40072930 tx=0x7f4b30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.721+0000 7f4b3d600640 1 -- 172.21.15.97:0/1519268983 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b38016030 con 0x7f4b401f9350 2024-09-10T21:29:39.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b4006d7e0 con 0x7f4b401f9350 2024-09-10T21:29:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b3d600640 1 -- 172.21.15.97:0/1519268983 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16982 (secure 0 0 0) 0x7f4b3805e6f0 con 0x7f4b401f9350 2024-09-10T21:29:39.892 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:39.892 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:29:38.920726+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:29:36.590975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T21:29:37.883346+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b1c072610 msgr2=0x7f4b1c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b1c072610 0x7f4b1c074ad0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b40072930 tx=0x7f4b30009040 comp rx=0 tx=0).stop 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401f9350 msgr2=0x7f4b401531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401f9350 0x7f4b401531f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e990 tx=0x7f4b3800ee50 comp rx=0 tx=0).stop 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 shutdown_connections 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b1c072610 0x7f4b1c074ad0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401f9350 0x7f4b401531f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b400b42e0 0x7f4b40152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 --2- 172.21.15.97:0/1519268983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4b400b2bb0 0x7f4b401fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 >> 172.21.15.97:0/1519268983 conn(0x7f4b400b37c0 msgr2=0x7f4b400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 shutdown_connections 2024-09-10T21:29:39.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:39.889+0000 7f4b47400640 1 -- 172.21.15.97:0/1519268983 wait complete. 2024-09-10T21:29:39.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:29:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- 172.21.15.97:0/3237371129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b181511f0 msgr2=0x7f9b180b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/3237371129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b181511f0 0x7f9b180b37b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009870 tx=0x7f9b0801f050 comp rx=0 tx=0).stop 2024-09-10T21:29:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- 172.21.15.97:0/3237371129 shutdown_connections 2024-09-10T21:29:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/3237371129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b181511f0 0x7f9b180b37b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- 172.21.15.97:0/3237371129 >> 172.21.15.97:0/3237371129 conn(0x7f9b180b1780 msgr2=0x7f9b180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- 172.21.15.97:0/3237371129 shutdown_connections 2024-09-10T21:29:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- 172.21.15.97:0/3237371129 wait complete. 2024-09-10T21:29:40.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 Processor -- start 2024-09-10T21:29:40.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- start start 2024-09-10T21:29:40.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b180b3cf0 0x7f9b181e6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b180ceee0 0x7f9b181502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b181511f0 0x7f9b1814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b180cf390 con 0x7f9b181511f0 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1815b210 con 0x7f9b180b3cf0 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1fa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9b180cf510 con 0x7f9b180ceee0 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b181511f0 0x7f9b1814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b181511f0 0x7f9b1814dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35000/0 (socket says 172.21.15.97:35000) 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 -- 172.21.15.97:0/4045293870 learned_addr learned my addr 172.21.15.97:0/4045293870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b0fe00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b180ceee0 0x7f9b181502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1d000640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b180b3cf0 0x7f9b181e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 -- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b180ceee0 msgr2=0x7f9b181502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b180ceee0 0x7f9b181502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 -- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b180b3cf0 msgr2=0x7f9b181e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b180b3cf0 0x7f9b181e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.053+0000 7f9b1da00640 1 -- 172.21.15.97:0/4045293870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b18155070 con 0x7f9b181511f0 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b1da00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b181511f0 0x7f9b1814dc10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000caf0 tx=0x7f9b1000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b15600640 1 -- 172.21.15.97:0/4045293870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10016020 con 0x7f9b181511f0 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b15600640 1 -- 172.21.15.97:0/4045293870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b1000ab60 con 0x7f9b181511f0 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b18155350 con 0x7f9b181511f0 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b15600640 1 -- 172.21.15.97:0/4045293870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1000ce30 con 0x7f9b181511f0 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b18155bf0 con 0x7f9b181511f0 2024-09-10T21:29:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b18157fa0 con 0x7f9b181511f0 2024-09-10T21:29:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b15600640 1 -- 172.21.15.97:0/4045293870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9b1001b400 con 0x7f9b181511f0 2024-09-10T21:29:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b15600640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9af40725d0 0x7f9af4074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.057+0000 7f9b15600640 1 -- 172.21.15.97:0/4045293870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f9b10093dc0 con 0x7f9b181511f0 2024-09-10T21:29:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.061+0000 7f9b1d000640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9af40725d0 0x7f9af4074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.061+0000 7f9b1d000640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9af40725d0 0x7f9af4074a90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9b080b2000 tx=0x7f9b0801d890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.061+0000 7f9b15600640 1 -- 172.21.15.97:0/4045293870 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9b18157fa0 con 0x7f9b181511f0 2024-09-10T21:29:40.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.229+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b18155520 con 0x7f9b181511f0 2024-09-10T21:29:40.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.229+0000 7f9b15600640 1 -- 172.21.15.97:0/4045293870 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16982 (secure 0 0 0) 0x7f9b10020030 con 0x7f9b181511f0 2024-09-10T21:29:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:40.233 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:29:38.920726+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:29:36.590975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T21:29:37.883346+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9af40725d0 msgr2=0x7f9af4074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9af40725d0 0x7f9af4074a90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9b080b2000 tx=0x7f9b0801d890 comp rx=0 tx=0).stop 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b181511f0 msgr2=0x7f9b1814dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b181511f0 0x7f9b1814dc10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000caf0 tx=0x7f9b1000cbe0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 shutdown_connections 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9af40725d0 0x7f9af4074a90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b181511f0 0x7f9b1814dc10 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b180ceee0 0x7f9b181502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 --2- 172.21.15.97:0/4045293870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b180b3cf0 0x7f9b181e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 >> 172.21.15.97:0/4045293870 conn(0x7f9b180b1780 msgr2=0x7f9b18153980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 shutdown_connections 2024-09-10T21:29:40.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.233+0000 7f9b1fa00640 1 -- 172.21.15.97:0/4045293870 wait complete. 2024-09-10T21:29:40.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- 172.21.15.97:0/2063874602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0b1560 msgr2=0x7f799c0b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 --2- 172.21.15.97:0/2063874602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0b1560 0x7f799c0b1970 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f799c069d60 tx=0x7f799401f070 comp rx=0 tx=0).stop 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- 172.21.15.97:0/2063874602 shutdown_connections 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 --2- 172.21.15.97:0/2063874602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0b1560 0x7f799c0b1970 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 --2- 172.21.15.97:0/2063874602 >> v2:172.21.15.194:3300/0 conn(0x7f799c0cec10 0x7f799c0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- 172.21.15.97:0/2063874602 >> 172.21.15.97:0/2063874602 conn(0x7f799c0b37c0 msgr2=0x7f799c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- 172.21.15.97:0/2063874602 shutdown_connections 2024-09-10T21:29:40.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- 172.21.15.97:0/2063874602 wait complete. 2024-09-10T21:29:40.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 Processor -- start 2024-09-10T21:29:40.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- start start 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f799c0b1560 0x7f799c0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f799c0ce830 0x7f799c0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0cec10 0x7f799c0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f799c0ce1a0 con 0x7f799c0cec10 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f799c0cde60 con 0x7f799c0ce830 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f79a2000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f799c0ce320 con 0x7f799c0b1560 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0cec10 0x7f799c0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0cec10 0x7f799c0c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35022/0 (socket says 172.21.15.97:35022) 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 -- 172.21.15.97:0/2797521418 learned_addr learned my addr 172.21.15.97:0/2797521418 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799aa00640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f799c0ce830 0x7f799c0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799b400640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f799c0b1560 0x7f799c0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 -- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f799c0b1560 msgr2=0x7f799c0c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f799c0b1560 0x7f799c0c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 -- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f799c0ce830 msgr2=0x7f799c0c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f799c0ce830 0x7f799c0c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.397+0000 7f799be00640 1 -- 172.21.15.97:0/2797521418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c06ddf0 con 0x7f799c0cec10 2024-09-10T21:29:40.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f799be00640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0cec10 0x7f799c0c8d00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7994009850 tx=0x7f79940a8d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:40.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f7993400640 1 -- 172.21.15.97:0/2797521418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79940b5040 con 0x7f799c0cec10 2024-09-10T21:29:40.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f7993400640 1 -- 172.21.15.97:0/2797521418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7994005320 con 0x7f799c0cec10 2024-09-10T21:29:40.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c06e0d0 con 0x7f799c0cec10 2024-09-10T21:29:40.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f7993400640 1 -- 172.21.15.97:0/2797521418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994003700 con 0x7f799c0cec10 2024-09-10T21:29:40.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c0c6950 con 0x7f799c0cec10 2024-09-10T21:29:40.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c0cffc0 con 0x7f799c0cec10 2024-09-10T21:29:40.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f7993400640 1 -- 172.21.15.97:0/2797521418 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f79940038a0 con 0x7f799c0cec10 2024-09-10T21:29:40.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f7993400640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f797c0725e0 0x7f797c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:40.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f7993400640 1 -- 172.21.15.97:0/2797521418 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6101+0+0 (secure 0 0 0) 0x7f79941303b0 con 0x7f799c0cec10 2024-09-10T21:29:40.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.401+0000 7f799b400640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f797c0725e0 0x7f797c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:40.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.405+0000 7f799b400640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f797c0725e0 0x7f797c074aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f798c005530 tx=0x7f798c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:40.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.405+0000 7f7993400640 1 -- 172.21.15.97:0/2797521418 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7994018bd0 con 0x7f799c0cec10 2024-09-10T21:29:40.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:40.609+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f799c072030 con 0x7f799c0cec10 2024-09-10T21:29:41.504 INFO:tasks.ceph.mds.b.smithi194.stderr:2024-09-10T21:29:41.502+0000 7f48d9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:29:41.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.545+0000 7f7993400640 1 -- 172.21.15.97:0/2797521418 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) ==== 118+0+0 (secure 0 0 0) 0x7f79940fb860 con 0x7f799c0cec10 2024-09-10T21:29:41.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f797c0725e0 msgr2=0x7f797c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f797c0725e0 0x7f797c074aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f798c005530 tx=0x7f798c009040 comp rx=0 tx=0).stop 2024-09-10T21:29:41.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0cec10 msgr2=0x7f799c0c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0cec10 0x7f799c0c8d00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7994009850 tx=0x7f79940a8d30 comp rx=0 tx=0).stop 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 shutdown_connections 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f797c0725e0 0x7f797c074aa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799c0cec10 0x7f799c0c8d00 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f799c0ce830 0x7f799c0c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 --2- 172.21.15.97:0/2797521418 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f799c0b1560 0x7f799c0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 >> 172.21.15.97:0/2797521418 conn(0x7f799c0b37c0 msgr2=0x7f799c15e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 shutdown_connections 2024-09-10T21:29:41.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.549+0000 7f79a2000640 1 -- 172.21.15.97:0/2797521418 wait complete. 2024-09-10T21:29:41.575 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:29:41.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:29:41.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- 172.21.15.97:0/1407854140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 msgr2=0x7fbc400cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 --2- 172.21.15.97:0/1407854140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc400cfd80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009900 tx=0x7fbc2c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- 172.21.15.97:0/1407854140 shutdown_connections 2024-09-10T21:29:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 --2- 172.21.15.97:0/1407854140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc400cfd80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 --2- 172.21.15.97:0/1407854140 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc4015efc0 0x7fbc400cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- 172.21.15.97:0/1407854140 >> 172.21.15.97:0/1407854140 conn(0x7fbc400d1a60 msgr2=0x7fbc400d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- 172.21.15.97:0/1407854140 shutdown_connections 2024-09-10T21:29:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- 172.21.15.97:0/1407854140 wait complete. 2024-09-10T21:29:41.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 Processor -- start 2024-09-10T21:29:41.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- start start 2024-09-10T21:29:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc4015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc400d02c0 0x7fbc4006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc4015efc0 0x7fbc401582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40168f20 con 0x7fbc400cdcb0 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40168da0 con 0x7fbc4015efc0 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.721+0000 7fbc46800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbc401690a0 con 0x7fbc400d02c0 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc4015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc4015e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35038/0 (socket says 172.21.15.97:35038) 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 -- 172.21.15.97:0/1030441921 learned_addr learned my addr 172.21.15.97:0/1030441921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc44a00640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc400d02c0 0x7fbc4006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45e00640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc4015efc0 0x7fbc401582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 -- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc400d02c0 msgr2=0x7fbc4006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc400d02c0 0x7fbc4006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 -- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc4015efc0 msgr2=0x7fbc401582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc4015efc0 0x7fbc401582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 -- 172.21.15.97:0/1030441921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4015e840 con 0x7fbc400cdcb0 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc45400640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc4015e300 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400eaa0 tx=0x7fbc3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc3d600640 1 -- 172.21.15.97:0/1030441921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400ce10 con 0x7fbc400cdcb0 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc3d600640 1 -- 172.21.15.97:0/1030441921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc34011470 con 0x7fbc400cdcb0 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc400b56c0 con 0x7fbc400cdcb0 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc3d600640 1 -- 172.21.15.97:0/1030441921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc340118a0 con 0x7fbc400cdcb0 2024-09-10T21:29:41.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc40156780 con 0x7fbc400cdcb0 2024-09-10T21:29:41.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc400ce1e0 con 0x7fbc400cdcb0 2024-09-10T21:29:41.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.725+0000 7fbc3d600640 1 -- 172.21.15.97:0/1030441921 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbc34022030 con 0x7fbc400cdcb0 2024-09-10T21:29:41.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.729+0000 7fbc3d600640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc200725e0 0x7fbc20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.729+0000 7fbc3d600640 1 -- 172.21.15.97:0/1030441921 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fbc34093c40 con 0x7fbc400cdcb0 2024-09-10T21:29:41.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.729+0000 7fbc44a00640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc200725e0 0x7fbc20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.729+0000 7fbc44a00640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc200725e0 0x7fbc20074aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0ae000 tx=0x7fbc2c01b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.733+0000 7fbc3d600640 1 -- 172.21.15.97:0/1030441921 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbc34016030 con 0x7fbc400cdcb0 2024-09-10T21:29:41.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:41.933+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbc4015b9c0 con 0x7fbc400cdcb0 2024-09-10T21:29:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.549+0000 7fbc3d600640 1 -- 172.21.15.97:0/1030441921 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) ==== 122+0+0 (secure 0 0 0) 0x7fbc3405eb30 con 0x7fbc400cdcb0 2024-09-10T21:29:42.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc200725e0 msgr2=0x7fbc20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc200725e0 0x7fbc20074aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0ae000 tx=0x7fbc2c01b120 comp rx=0 tx=0).stop 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 msgr2=0x7fbc4015e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc4015e300 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400eaa0 tx=0x7fbc3400ef60 comp rx=0 tx=0).stop 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 shutdown_connections 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc200725e0 0x7fbc20074aa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc4015efc0 0x7fbc401582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc400d02c0 0x7fbc4006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 --2- 172.21.15.97:0/1030441921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc400cdcb0 0x7fbc4015e300 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.553+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 >> 172.21.15.97:0/1030441921 conn(0x7fbc400d1a60 msgr2=0x7fbc400b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.557+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 shutdown_connections 2024-09-10T21:29:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.557+0000 7fbc46800640 1 -- 172.21.15.97:0/1030441921 wait complete. 2024-09-10T21:29:42.581 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:29:42.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- 172.21.15.97:0/3897286607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 msgr2=0x7fc37415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 --2- 172.21.15.97:0/3897286607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37415f6d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009900 tx=0x7fc35c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- 172.21.15.97:0/3897286607 shutdown_connections 2024-09-10T21:29:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 --2- 172.21.15.97:0/3897286607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37415f6d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 --2- 172.21.15.97:0/3897286607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3740ce5f0 0x7fc374141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- 172.21.15.97:0/3897286607 >> 172.21.15.97:0/3897286607 conn(0x7fc3740b1780 msgr2=0x7fc3740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- 172.21.15.97:0/3897286607 shutdown_connections 2024-09-10T21:29:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- 172.21.15.97:0/3897286607 wait complete. 2024-09-10T21:29:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 Processor -- start 2024-09-10T21:29:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- start start 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3740ce5f0 0x7fc374157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37413f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc37415fc10 0x7fc37413fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741690c0 con 0x7fc3740ce9d0 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc374168f40 con 0x7fc37415fc10 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc37b600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc374169240 con 0x7fc3740ce5f0 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc36be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37413f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc36be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37413f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35064/0 (socket says 172.21.15.97:35064) 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc36be00640 1 -- 172.21.15.97:0/647163474 learned_addr learned my addr 172.21.15.97:0/647163474 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc378c00640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3740ce5f0 0x7fc374157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.725+0000 7fc379600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc37415fc10 0x7fc37413fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc36be00640 1 -- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3740ce5f0 msgr2=0x7fc374157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc36be00640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3740ce5f0 0x7fc374157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc36be00640 1 -- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc37415fc10 msgr2=0x7fc37413fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc36be00640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc37415fc10 0x7fc37413fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc36be00640 1 -- 172.21.15.97:0/647163474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3741f82f0 con 0x7fc3740ce9d0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc378c00640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3740ce5f0 0x7fc374157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc36be00640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37413f920 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009850 tx=0x7fc35c0a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc371600640 1 -- 172.21.15.97:0/647163474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0b2040 con 0x7fc3740ce9d0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc371600640 1 -- 172.21.15.97:0/647163474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc35c004470 con 0x7fc3740ce9d0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3741f85d0 con 0x7fc3740ce9d0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc371600640 1 -- 172.21.15.97:0/647163474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c005290 con 0x7fc3740ce9d0 2024-09-10T21:29:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3741f8b90 con 0x7fc3740ce9d0 2024-09-10T21:29:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc374141d20 con 0x7fc3740ce9d0 2024-09-10T21:29:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc371600640 1 -- 172.21.15.97:0/647163474 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc35c005430 con 0x7fc3740ce9d0 2024-09-10T21:29:42.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc371600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc3500725e0 0x7fc350074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.729+0000 7fc371600640 1 -- 172.21.15.97:0/647163474 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fc35c12c8d0 con 0x7fc3740ce9d0 2024-09-10T21:29:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.733+0000 7fc378c00640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc3500725e0 0x7fc350074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.733+0000 7fc378c00640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc3500725e0 0x7fc350074aa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc364009fd0 tx=0x7fc3640091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:42.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.733+0000 7fc371600640 1 -- 172.21.15.97:0/647163474 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc35c0bc060 con 0x7fc3740ce9d0 2024-09-10T21:29:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.933+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc374156f00 con 0x7fc3740ce9d0 2024-09-10T21:29:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.933+0000 7fc371600640 1 -- 172.21.15.97:0/647163474 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5231 (secure 0 0 0) 0x7fc35c0164d0 con 0x7fc3740ce9d0 2024-09-10T21:29:42.937 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:42.937 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"btime":"2024-09-10T21:29:42:550041+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6492,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:29:41.936668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6504},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6504":{"gid":6504,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6836/3597857813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3597857813},{"type":"v1","addr":"172.21.15.194:6836","nonce":3597857813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6504,"qdb_cluster":[6504]},"id":5}]} 2024-09-10T21:29:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 70 2024-09-10T21:29:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc3500725e0 msgr2=0x7fc350074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc3500725e0 0x7fc350074aa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc364009fd0 tx=0x7fc3640091e0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 msgr2=0x7fc37413f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37413f920 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009850 tx=0x7fc35c0a55a0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 shutdown_connections 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc3500725e0 0x7fc350074aa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc37415fc10 0x7fc37413fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3740ce9d0 0x7fc37413f920 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 --2- 172.21.15.97:0/647163474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc3740ce5f0 0x7fc374157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 >> 172.21.15.97:0/647163474 conn(0x7fc3740b1780 msgr2=0x7fc374072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 shutdown_connections 2024-09-10T21:29:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:42.937+0000 7fc37b600640 1 -- 172.21.15.97:0/647163474 wait complete. 2024-09-10T21:29:42.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:29:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.097+0000 7faed3600640 1 -- 172.21.15.97:0/1120595517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc15ac30 msgr2=0x7faecc0b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.097+0000 7faed3600640 1 --2- 172.21.15.97:0/1120595517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc15ac30 0x7faecc0b5d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- 172.21.15.97:0/1120595517 shutdown_connections 2024-09-10T21:29:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 --2- 172.21.15.97:0/1120595517 >> v2:172.21.15.194:3300/0 conn(0x7faecc074b20 0x7faecc0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 --2- 172.21.15.97:0/1120595517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc15ac30 0x7faecc0b5d20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- 172.21.15.97:0/1120595517 >> 172.21.15.97:0/1120595517 conn(0x7faecc0b3c70 msgr2=0x7faecc0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- 172.21.15.97:0/1120595517 shutdown_connections 2024-09-10T21:29:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- 172.21.15.97:0/1120595517 wait complete. 2024-09-10T21:29:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 Processor -- start 2024-09-10T21:29:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- start start 2024-09-10T21:29:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faecc0691b0 0x7faecc15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc074b20 0x7faecc0ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15ac30 0x7faecc0ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faecc06dee0 con 0x7faecc15ac30 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faecc06dd60 con 0x7faecc0691b0 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed3600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7faecc06e060 con 0x7faecc074b20 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15ac30 0x7faecc0ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15ac30 0x7faecc0ad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35080/0 (socket says 172.21.15.97:35080) 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 -- 172.21.15.97:0/728029184 learned_addr learned my addr 172.21.15.97:0/728029184 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed0c00640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faecc0691b0 0x7faecc15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faec3e00640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc074b20 0x7faecc0ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 -- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc074b20 msgr2=0x7faecc0ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc074b20 0x7faecc0ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 -- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faecc0691b0 msgr2=0x7faecc15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faecc0691b0 0x7faecc15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 -- 172.21.15.97:0/728029184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc152680 con 0x7faecc15ac30 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.101+0000 7faed1600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15ac30 0x7faecc0ad610 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7faec400c7a0 tx=0x7faec400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faec9600640 1 -- 172.21.15.97:0/728029184 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4003b80 con 0x7faecc15ac30 2024-09-10T21:29:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faec9600640 1 -- 172.21.15.97:0/728029184 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec4003d20 con 0x7faecc15ac30 2024-09-10T21:29:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faec9600640 1 -- 172.21.15.97:0/728029184 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400aab0 con 0x7faecc15ac30 2024-09-10T21:29:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc0aed00 con 0x7faecc15ac30 2024-09-10T21:29:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc157860 con 0x7faecc15ac30 2024-09-10T21:29:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faecc0cf5b0 con 0x7faecc15ac30 2024-09-10T21:29:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faec9600640 1 -- 172.21.15.97:0/728029184 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faec400ac50 con 0x7faecc15ac30 2024-09-10T21:29:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faec9600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea80725e0 0x7faea8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faec9600640 1 -- 172.21.15.97:0/728029184 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7faec4093130 con 0x7faecc15ac30 2024-09-10T21:29:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.105+0000 7faed0c00640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea80725e0 0x7faea8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.109+0000 7faed0c00640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea80725e0 0x7faea8074aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faebc0b2000 tx=0x7faebc01d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.109+0000 7faec9600640 1 -- 172.21.15.97:0/728029184 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faec405df90 con 0x7faecc15ac30 2024-09-10T21:29:43.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.277+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faecc0aeaf0 con 0x7faecc15ac30 2024-09-10T21:29:43.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faec9600640 1 -- 172.21.15.97:0/728029184 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16969 (secure 0 0 0) 0x7faec40629c0 con 0x7faecc15ac30 2024-09-10T21:29:43.283 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:43.283 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:29:40.535912+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:29:36.590975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T21:29:37.883346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea80725e0 msgr2=0x7faea8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea80725e0 0x7faea8074aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faebc0b2000 tx=0x7faebc01d910 comp rx=0 tx=0).stop 2024-09-10T21:29:43.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15ac30 msgr2=0x7faecc0ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15ac30 0x7faecc0ad610 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7faec400c7a0 tx=0x7faec400cc60 comp rx=0 tx=0).stop 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 shutdown_connections 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea80725e0 0x7faea8074aa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faecc15ac30 0x7faecc0ad610 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faecc074b20 0x7faecc0ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 --2- 172.21.15.97:0/728029184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faecc0691b0 0x7faecc15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 >> 172.21.15.97:0/728029184 conn(0x7faecc0b3c70 msgr2=0x7faecc0b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 shutdown_connections 2024-09-10T21:29:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.281+0000 7faed3600640 1 -- 172.21.15.97:0/728029184 wait complete. 2024-09-10T21:29:43.304 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:29:43.305 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 -- 172.21.15.97:0/501916388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015f150 msgr2=0x7f53200b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 --2- 172.21.15.97:0/501916388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015f150 0x7f53200b4380 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5310009c80 tx=0x7f531001cd10 comp rx=0 tx=0).stop 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 -- 172.21.15.97:0/501916388 shutdown_connections 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 --2- 172.21.15.97:0/501916388 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53200b48c0 0x7f532015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 --2- 172.21.15.97:0/501916388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015f150 0x7f53200b4380 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 -- 172.21.15.97:0/501916388 >> 172.21.15.97:0/501916388 conn(0x7f53200b28c0 msgr2=0x7f53200b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 -- 172.21.15.97:0/501916388 shutdown_connections 2024-09-10T21:29:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 -- 172.21.15.97:0/501916388 wait complete. 2024-09-10T21:29:43.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 Processor -- start 2024-09-10T21:29:43.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.445+0000 7f5325600640 1 -- start start 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532015b570 0x7f53200b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f532015f150 0x7f532015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015ab30 0x7f5320158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5320167c70 con 0x7f532015ab30 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5320167af0 con 0x7f532015b570 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5320167df0 con 0x7f532015f150 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015ab30 0x7f5320158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015ab30 0x7f5320158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35110/0 (socket says 172.21.15.97:35110) 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 -- 172.21.15.97:0/2773863191 learned_addr learned my addr 172.21.15.97:0/2773863191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5316000640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f532015f150 0x7f532015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531ea00640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532015b570 0x7f53200b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 -- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f532015f150 msgr2=0x7f532015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f532015f150 0x7f532015a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 -- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532015b570 msgr2=0x7f53200b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532015b570 0x7f53200b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 -- 172.21.15.97:0/2773863191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53201f7cf0 con 0x7f532015ab30 2024-09-10T21:29:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f531f400640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015ab30 0x7f5320158b60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f531800c390 tx=0x7f531800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5317400640 1 -- 172.21.15.97:0/2773863191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318011790 con 0x7f532015ab30 2024-09-10T21:29:43.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5317400640 1 -- 172.21.15.97:0/2773863191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5318017070 con 0x7f532015ab30 2024-09-10T21:29:43.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5317400640 1 -- 172.21.15.97:0/2773863191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318011d00 con 0x7f532015ab30 2024-09-10T21:29:43.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53201f7f90 con 0x7f532015ab30 2024-09-10T21:29:43.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201f8550 con 0x7f532015ab30 2024-09-10T21:29:43.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532015b030 con 0x7f532015ab30 2024-09-10T21:29:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.449+0000 7f5317400640 1 -- 172.21.15.97:0/2773863191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f531801c460 con 0x7f532015ab30 2024-09-10T21:29:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.453+0000 7f5317400640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52fc0725e0 0x7f52fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.453+0000 7f5317400640 1 -- 172.21.15.97:0/2773863191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f5318092e90 con 0x7f532015ab30 2024-09-10T21:29:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.453+0000 7f531ea00640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52fc0725e0 0x7f52fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.453+0000 7f531ea00640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52fc0725e0 0x7f52fc074aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f53100af000 tx=0x7f531001b550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.457+0000 7f5317400640 1 -- 172.21.15.97:0/2773863191 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5318021020 con 0x7f532015ab30 2024-09-10T21:29:43.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.621+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5320159d80 con 0x7f52fc0725e0 2024-09-10T21:29:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.629+0000 7f5317400640 1 -- 172.21.15.97:0/2773863191 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250862 (secure 0 0 0) 0x7f5320159d80 con 0x7f52fc0725e0 2024-09-10T21:29:43.634 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52fc0725e0 msgr2=0x7f52fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52fc0725e0 0x7f52fc074aa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f53100af000 tx=0x7f531001b550 comp rx=0 tx=0).stop 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015ab30 msgr2=0x7f5320158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015ab30 0x7f5320158b60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f531800c390 tx=0x7f531800c850 comp rx=0 tx=0).stop 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 shutdown_connections 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f52fc0725e0 0x7f52fc074aa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f532015ab30 0x7f5320158b60 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f532015f150 0x7f532015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 --2- 172.21.15.97:0/2773863191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532015b570 0x7f53200b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 >> 172.21.15.97:0/2773863191 conn(0x7f53200b28c0 msgr2=0x7f5320157aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 shutdown_connections 2024-09-10T21:29:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.633+0000 7f5325600640 1 -- 172.21.15.97:0/2773863191 wait complete. 2024-09-10T21:29:43.655 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-10T21:29:42.155481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241836,"kb_used_data":8948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754732884,"statfs":{"total":773094113280,"available":772846473216,"internally_reserved":0,"allocated":9162752,"data_stored":4823396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.640643"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768894+0000","last_change":"2024-09-10T21:29:39.539326+0000","last_active":"2024-09-10T21:29:40.768894+0000","last_peered":"2024-09-10T21:29:40.768894+0000","last_clean":"2024-09-10T21:29:40.768894+0000","last_became_active":"2024-09-10T21:29:39.539056+0000","last_became_peered":"2024-09-10T21:29:39.539056+0000","last_unstale":"2024-09-10T21:29:40.768894+0000","last_undegraded":"2024-09-10T21:29:40.768894+0000","last_fullsized":"2024-09-10T21:29:40.768894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:25.767146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769190+0000","last_change":"2024-09-10T21:29:39.564404+0000","last_active":"2024-09-10T21:29:40.769190+0000","last_peered":"2024-09-10T21:29:40.769190+0000","last_clean":"2024-09-10T21:29:40.769190+0000","last_became_active":"2024-09-10T21:29:39.563306+0000","last_became_peered":"2024-09-10T21:29:39.563306+0000","last_unstale":"2024-09-10T21:29:40.769190+0000","last_undegraded":"2024-09-10T21:29:40.769190+0000","last_fullsized":"2024-09-10T21:29:40.769190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:17.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769401+0000","last_change":"2024-09-10T21:29:39.561606+0000","last_active":"2024-09-10T21:29:40.769401+0000","last_peered":"2024-09-10T21:29:40.769401+0000","last_clean":"2024-09-10T21:29:40.769401+0000","last_became_active":"2024-09-10T21:29:39.560972+0000","last_became_peered":"2024-09-10T21:29:39.560972+0000","last_unstale":"2024-09-10T21:29:40.769401+0000","last_undegraded":"2024-09-10T21:29:40.769401+0000","last_fullsized":"2024-09-10T21:29:40.769401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:26.701898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544576+0000","last_change":"2024-09-10T21:29:39.537773+0000","last_active":"2024-09-10T21:29:40.544576+0000","last_peered":"2024-09-10T21:29:40.544576+0000","last_clean":"2024-09-10T21:29:40.544576+0000","last_became_active":"2024-09-10T21:29:39.537487+0000","last_became_peered":"2024-09-10T21:29:39.537487+0000","last_unstale":"2024-09-10T21:29:40.544576+0000","last_undegraded":"2024-09-10T21:29:40.544576+0000","last_fullsized":"2024-09-10T21:29:40.544576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:53.105160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546947+0000","last_change":"2024-09-10T21:29:39.546644+0000","last_active":"2024-09-10T21:29:40.546947+0000","last_peered":"2024-09-10T21:29:40.546947+0000","last_clean":"2024-09-10T21:29:40.546947+0000","last_became_active":"2024-09-10T21:29:39.546373+0000","last_became_peered":"2024-09-10T21:29:39.546373+0000","last_unstale":"2024-09-10T21:29:40.546947+0000","last_undegraded":"2024-09-10T21:29:40.546947+0000","last_fullsized":"2024-09-10T21:29:40.546947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.942938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546101+0000","last_change":"2024-09-10T21:29:39.558707+0000","last_active":"2024-09-10T21:29:40.546101+0000","last_peered":"2024-09-10T21:29:40.546101+0000","last_clean":"2024-09-10T21:29:40.546101+0000","last_became_active":"2024-09-10T21:29:39.554793+0000","last_became_peered":"2024-09-10T21:29:39.554793+0000","last_unstale":"2024-09-10T21:29:40.546101+0000","last_undegraded":"2024-09-10T21:29:40.546101+0000","last_fullsized":"2024-09-10T21:29:40.546101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544328+0000","last_change":"2024-09-10T21:29:39.540653+0000","last_active":"2024-09-10T21:29:40.544328+0000","last_peered":"2024-09-10T21:29:40.544328+0000","last_clean":"2024-09-10T21:29:40.544328+0000","last_became_active":"2024-09-10T21:29:39.540400+0000","last_became_peered":"2024-09-10T21:29:39.540400+0000","last_unstale":"2024-09-10T21:29:40.544328+0000","last_undegraded":"2024-09-10T21:29:40.544328+0000","last_fullsized":"2024-09-10T21:29:40.544328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:26.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545150+0000","last_change":"2024-09-10T21:29:39.556621+0000","last_active":"2024-09-10T21:29:40.545150+0000","last_peered":"2024-09-10T21:29:40.545150+0000","last_clean":"2024-09-10T21:29:40.545150+0000","last_became_active":"2024-09-10T21:29:39.555001+0000","last_became_peered":"2024-09-10T21:29:39.555001+0000","last_unstale":"2024-09-10T21:29:40.545150+0000","last_undegraded":"2024-09-10T21:29:40.545150+0000","last_fullsized":"2024-09-10T21:29:40.545150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769242+0000","last_change":"2024-09-10T21:29:39.536232+0000","last_active":"2024-09-10T21:29:40.769242+0000","last_peered":"2024-09-10T21:29:40.769242+0000","last_clean":"2024-09-10T21:29:40.769242+0000","last_became_active":"2024-09-10T21:29:39.535978+0000","last_became_peered":"2024-09-10T21:29:39.535978+0000","last_unstale":"2024-09-10T21:29:40.769242+0000","last_undegraded":"2024-09-10T21:29:40.769242+0000","last_fullsized":"2024-09-10T21:29:40.769242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:49.789897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769661+0000","last_change":"2024-09-10T21:29:39.562255+0000","last_active":"2024-09-10T21:29:40.769661+0000","last_peered":"2024-09-10T21:29:40.769661+0000","last_clean":"2024-09-10T21:29:40.769661+0000","last_became_active":"2024-09-10T21:29:39.559040+0000","last_became_peered":"2024-09-10T21:29:39.559040+0000","last_unstale":"2024-09-10T21:29:40.769661+0000","last_undegraded":"2024-09-10T21:29:40.769661+0000","last_fullsized":"2024-09-10T21:29:40.769661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:39.446682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769690+0000","last_change":"2024-09-10T21:29:39.550940+0000","last_active":"2024-09-10T21:29:40.769690+0000","last_peered":"2024-09-10T21:29:40.769690+0000","last_clean":"2024-09-10T21:29:40.769690+0000","last_became_active":"2024-09-10T21:29:39.550666+0000","last_became_peered":"2024-09-10T21:29:39.550666+0000","last_unstale":"2024-09-10T21:29:40.769690+0000","last_undegraded":"2024-09-10T21:29:40.769690+0000","last_fullsized":"2024-09-10T21:29:40.769690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:24.960891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768575+0000","last_change":"2024-09-10T21:29:39.555356+0000","last_active":"2024-09-10T21:29:40.768575+0000","last_peered":"2024-09-10T21:29:40.768575+0000","last_clean":"2024-09-10T21:29:40.768575+0000","last_became_active":"2024-09-10T21:29:39.554843+0000","last_became_peered":"2024-09-10T21:29:39.554843+0000","last_unstale":"2024-09-10T21:29:40.768575+0000","last_undegraded":"2024-09-10T21:29:40.768575+0000","last_fullsized":"2024-09-10T21:29:40.768575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:03.103988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546185+0000","last_change":"2024-09-10T21:29:39.563879+0000","last_active":"2024-09-10T21:29:40.546185+0000","last_peered":"2024-09-10T21:29:40.546185+0000","last_clean":"2024-09-10T21:29:40.546185+0000","last_became_active":"2024-09-10T21:29:39.563673+0000","last_became_peered":"2024-09-10T21:29:39.563673+0000","last_unstale":"2024-09-10T21:29:40.546185+0000","last_undegraded":"2024-09-10T21:29:40.546185+0000","last_fullsized":"2024-09-10T21:29:40.546185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:33.785842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544217+0000","last_change":"2024-09-10T21:29:39.552730+0000","last_active":"2024-09-10T21:29:40.544217+0000","last_peered":"2024-09-10T21:29:40.544217+0000","last_clean":"2024-09-10T21:29:40.544217+0000","last_became_active":"2024-09-10T21:29:39.552478+0000","last_became_peered":"2024-09-10T21:29:39.552478+0000","last_unstale":"2024-09-10T21:29:40.544217+0000","last_undegraded":"2024-09-10T21:29:40.544217+0000","last_fullsized":"2024-09-10T21:29:40.544217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:00.008953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769429+0000","last_change":"2024-09-10T21:29:39.558608+0000","last_active":"2024-09-10T21:29:40.769429+0000","last_peered":"2024-09-10T21:29:40.769429+0000","last_clean":"2024-09-10T21:29:40.769429+0000","last_became_active":"2024-09-10T21:29:39.558086+0000","last_became_peered":"2024-09-10T21:29:39.558086+0000","last_unstale":"2024-09-10T21:29:40.769429+0000","last_undegraded":"2024-09-10T21:29:40.769429+0000","last_fullsized":"2024-09-10T21:29:40.769429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:51.432263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546462+0000","last_change":"2024-09-10T21:29:39.560153+0000","last_active":"2024-09-10T21:29:40.546462+0000","last_peered":"2024-09-10T21:29:40.546462+0000","last_clean":"2024-09-10T21:29:40.546462+0000","last_became_active":"2024-09-10T21:29:39.559886+0000","last_became_peered":"2024-09-10T21:29:39.559886+0000","last_unstale":"2024-09-10T21:29:40.546462+0000","last_undegraded":"2024-09-10T21:29:40.546462+0000","last_fullsized":"2024-09-10T21:29:40.546462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768980+0000","last_change":"2024-09-10T21:29:39.547565+0000","last_active":"2024-09-10T21:29:40.768980+0000","last_peered":"2024-09-10T21:29:40.768980+0000","last_clean":"2024-09-10T21:29:40.768980+0000","last_became_active":"2024-09-10T21:29:39.547315+0000","last_became_peered":"2024-09-10T21:29:39.547315+0000","last_unstale":"2024-09-10T21:29:40.768980+0000","last_undegraded":"2024-09-10T21:29:40.768980+0000","last_fullsized":"2024-09-10T21:29:40.768980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:05.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769033+0000","last_change":"2024-09-10T21:29:39.562781+0000","last_active":"2024-09-10T21:29:40.769033+0000","last_peered":"2024-09-10T21:29:40.769033+0000","last_clean":"2024-09-10T21:29:40.769033+0000","last_became_active":"2024-09-10T21:29:39.562509+0000","last_became_peered":"2024-09-10T21:29:39.562509+0000","last_unstale":"2024-09-10T21:29:40.769033+0000","last_undegraded":"2024-09-10T21:29:40.769033+0000","last_fullsized":"2024-09-10T21:29:40.769033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:53.170104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768742+0000","last_change":"2024-09-10T21:29:39.540989+0000","last_active":"2024-09-10T21:29:40.768742+0000","last_peered":"2024-09-10T21:29:40.768742+0000","last_clean":"2024-09-10T21:29:40.768742+0000","last_became_active":"2024-09-10T21:29:39.540686+0000","last_became_peered":"2024-09-10T21:29:39.540686+0000","last_unstale":"2024-09-10T21:29:40.768742+0000","last_undegraded":"2024-09-10T21:29:40.768742+0000","last_fullsized":"2024-09-10T21:29:40.768742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:04.647939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768432+0000","last_change":"2024-09-10T21:29:39.555697+0000","last_active":"2024-09-10T21:29:40.768432+0000","last_peered":"2024-09-10T21:29:40.768432+0000","last_clean":"2024-09-10T21:29:40.768432+0000","last_became_active":"2024-09-10T21:29:39.555321+0000","last_became_peered":"2024-09-10T21:29:39.555321+0000","last_unstale":"2024-09-10T21:29:40.768432+0000","last_undegraded":"2024-09-10T21:29:40.768432+0000","last_fullsized":"2024-09-10T21:29:40.768432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:26.937688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547452+0000","last_change":"2024-09-10T21:29:39.566354+0000","last_active":"2024-09-10T21:29:40.547452+0000","last_peered":"2024-09-10T21:29:40.547452+0000","last_clean":"2024-09-10T21:29:40.547452+0000","last_became_active":"2024-09-10T21:29:39.566119+0000","last_became_peered":"2024-09-10T21:29:39.566119+0000","last_unstale":"2024-09-10T21:29:40.547452+0000","last_undegraded":"2024-09-10T21:29:40.547452+0000","last_fullsized":"2024-09-10T21:29:40.547452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544876+0000","last_change":"2024-09-10T21:29:39.555058+0000","last_active":"2024-09-10T21:29:40.544876+0000","last_peered":"2024-09-10T21:29:40.544876+0000","last_clean":"2024-09-10T21:29:40.544876+0000","last_became_active":"2024-09-10T21:29:39.554444+0000","last_became_peered":"2024-09-10T21:29:39.554444+0000","last_unstale":"2024-09-10T21:29:40.544876+0000","last_undegraded":"2024-09-10T21:29:40.544876+0000","last_fullsized":"2024-09-10T21:29:40.544876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544584+0000","last_change":"2024-09-10T21:29:39.553456+0000","last_active":"2024-09-10T21:29:40.544584+0000","last_peered":"2024-09-10T21:29:40.544584+0000","last_clean":"2024-09-10T21:29:40.544584+0000","last_became_active":"2024-09-10T21:29:39.553218+0000","last_became_peered":"2024-09-10T21:29:39.553218+0000","last_unstale":"2024-09-10T21:29:40.544584+0000","last_undegraded":"2024-09-10T21:29:40.544584+0000","last_fullsized":"2024-09-10T21:29:40.544584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547497+0000","last_change":"2024-09-10T21:29:39.551245+0000","last_active":"2024-09-10T21:29:40.547497+0000","last_peered":"2024-09-10T21:29:40.547497+0000","last_clean":"2024-09-10T21:29:40.547497+0000","last_became_active":"2024-09-10T21:29:39.550535+0000","last_became_peered":"2024-09-10T21:29:39.550535+0000","last_unstale":"2024-09-10T21:29:40.547497+0000","last_undegraded":"2024-09-10T21:29:40.547497+0000","last_fullsized":"2024-09-10T21:29:40.547497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:03.284635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545912+0000","last_change":"2024-09-10T21:29:39.558777+0000","last_active":"2024-09-10T21:29:40.545912+0000","last_peered":"2024-09-10T21:29:40.545912+0000","last_clean":"2024-09-10T21:29:40.545912+0000","last_became_active":"2024-09-10T21:29:39.555098+0000","last_became_peered":"2024-09-10T21:29:39.555098+0000","last_unstale":"2024-09-10T21:29:40.545912+0000","last_undegraded":"2024-09-10T21:29:40.545912+0000","last_fullsized":"2024-09-10T21:29:40.545912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769096+0000","last_change":"2024-09-10T21:29:39.564203+0000","last_active":"2024-09-10T21:29:40.769096+0000","last_peered":"2024-09-10T21:29:40.769096+0000","last_clean":"2024-09-10T21:29:40.769096+0000","last_became_active":"2024-09-10T21:29:39.563951+0000","last_became_peered":"2024-09-10T21:29:39.563951+0000","last_unstale":"2024-09-10T21:29:40.769096+0000","last_undegraded":"2024-09-10T21:29:40.769096+0000","last_fullsized":"2024-09-10T21:29:40.769096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:38.678603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768992+0000","last_change":"2024-09-10T21:29:39.564492+0000","last_active":"2024-09-10T21:29:40.768992+0000","last_peered":"2024-09-10T21:29:40.768992+0000","last_clean":"2024-09-10T21:29:40.768992+0000","last_became_active":"2024-09-10T21:29:39.563500+0000","last_became_peered":"2024-09-10T21:29:39.563500+0000","last_unstale":"2024-09-10T21:29:40.768992+0000","last_undegraded":"2024-09-10T21:29:40.768992+0000","last_fullsized":"2024-09-10T21:29:40.768992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:46.030917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544257+0000","last_change":"2024-09-10T21:29:39.555396+0000","last_active":"2024-09-10T21:29:40.544257+0000","last_peered":"2024-09-10T21:29:40.544257+0000","last_clean":"2024-09-10T21:29:40.544257+0000","last_became_active":"2024-09-10T21:29:39.555086+0000","last_became_peered":"2024-09-10T21:29:39.555086+0000","last_unstale":"2024-09-10T21:29:40.544257+0000","last_undegraded":"2024-09-10T21:29:40.544257+0000","last_fullsized":"2024-09-10T21:29:40.544257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:06.000345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546200+0000","last_change":"2024-09-10T21:29:39.548302+0000","last_active":"2024-09-10T21:29:40.546200+0000","last_peered":"2024-09-10T21:29:40.546200+0000","last_clean":"2024-09-10T21:29:40.546200+0000","last_became_active":"2024-09-10T21:29:39.547972+0000","last_became_peered":"2024-09-10T21:29:39.547972+0000","last_unstale":"2024-09-10T21:29:40.546200+0000","last_undegraded":"2024-09-10T21:29:40.546200+0000","last_fullsized":"2024-09-10T21:29:40.546200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:22.457526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547580+0000","last_change":"2024-09-10T21:29:38.549320+0000","last_active":"2024-09-10T21:29:40.547580+0000","last_peered":"2024-09-10T21:29:40.547580+0000","last_clean":"2024-09-10T21:29:40.547580+0000","last_became_active":"2024-09-10T21:29:38.549041+0000","last_became_peered":"2024-09-10T21:29:38.549041+0000","last_unstale":"2024-09-10T21:29:40.547580+0000","last_undegraded":"2024-09-10T21:29:40.547580+0000","last_fullsized":"2024-09-10T21:29:40.547580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:52.457273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546222+0000","last_change":"2024-09-10T21:29:39.542269+0000","last_active":"2024-09-10T21:29:40.546222+0000","last_peered":"2024-09-10T21:29:40.546222+0000","last_clean":"2024-09-10T21:29:40.546222+0000","last_became_active":"2024-09-10T21:29:39.541975+0000","last_became_peered":"2024-09-10T21:29:39.541975+0000","last_unstale":"2024-09-10T21:29:40.546222+0000","last_undegraded":"2024-09-10T21:29:40.546222+0000","last_fullsized":"2024-09-10T21:29:40.546222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:38.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769882+0000","last_change":"2024-09-10T21:29:38.544792+0000","last_active":"2024-09-10T21:29:40.769882+0000","last_peered":"2024-09-10T21:29:40.769882+0000","last_clean":"2024-09-10T21:29:40.769882+0000","last_became_active":"2024-09-10T21:29:38.544459+0000","last_became_peered":"2024-09-10T21:29:38.544459+0000","last_unstale":"2024-09-10T21:29:40.769882+0000","last_undegraded":"2024-09-10T21:29:40.769882+0000","last_fullsized":"2024-09-10T21:29:40.769882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:28.938255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769152+0000","last_change":"2024-09-10T21:29:39.546560+0000","last_active":"2024-09-10T21:29:40.769152+0000","last_peered":"2024-09-10T21:29:40.769152+0000","last_clean":"2024-09-10T21:29:40.769152+0000","last_became_active":"2024-09-10T21:29:39.545328+0000","last_became_peered":"2024-09-10T21:29:39.545328+0000","last_unstale":"2024-09-10T21:29:40.769152+0000","last_undegraded":"2024-09-10T21:29:40.769152+0000","last_fullsized":"2024-09-10T21:29:40.769152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:20.557881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768895+0000","last_change":"2024-09-10T21:29:38.538896+0000","last_active":"2024-09-10T21:29:40.768895+0000","last_peered":"2024-09-10T21:29:40.768895+0000","last_clean":"2024-09-10T21:29:40.768895+0000","last_became_active":"2024-09-10T21:29:38.538523+0000","last_became_peered":"2024-09-10T21:29:38.538523+0000","last_unstale":"2024-09-10T21:29:40.768895+0000","last_undegraded":"2024-09-10T21:29:40.768895+0000","last_fullsized":"2024-09-10T21:29:40.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:26.307429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544519+0000","last_change":"2024-09-10T21:29:39.538536+0000","last_active":"2024-09-10T21:29:40.544519+0000","last_peered":"2024-09-10T21:29:40.544519+0000","last_clean":"2024-09-10T21:29:40.544519+0000","last_became_active":"2024-09-10T21:29:39.538308+0000","last_became_peered":"2024-09-10T21:29:39.538308+0000","last_unstale":"2024-09-10T21:29:40.544519+0000","last_undegraded":"2024-09-10T21:29:40.544519+0000","last_fullsized":"2024-09-10T21:29:40.544519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:40.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769616+0000","last_change":"2024-09-10T21:29:38.536197+0000","last_active":"2024-09-10T21:29:40.769616+0000","last_peered":"2024-09-10T21:29:40.769616+0000","last_clean":"2024-09-10T21:29:40.769616+0000","last_became_active":"2024-09-10T21:29:38.535894+0000","last_became_peered":"2024-09-10T21:29:38.535894+0000","last_unstale":"2024-09-10T21:29:40.769616+0000","last_undegraded":"2024-09-10T21:29:40.769616+0000","last_fullsized":"2024-09-10T21:29:40.769616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546646+0000","last_change":"2024-09-10T21:29:39.554137+0000","last_active":"2024-09-10T21:29:40.546646+0000","last_peered":"2024-09-10T21:29:40.546646+0000","last_clean":"2024-09-10T21:29:40.546646+0000","last_became_active":"2024-09-10T21:29:39.553735+0000","last_became_peered":"2024-09-10T21:29:39.553735+0000","last_unstale":"2024-09-10T21:29:40.546646+0000","last_undegraded":"2024-09-10T21:29:40.546646+0000","last_fullsized":"2024-09-10T21:29:40.546646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769158+0000","last_change":"2024-09-10T21:29:38.535949+0000","last_active":"2024-09-10T21:29:40.769158+0000","last_peered":"2024-09-10T21:29:40.769158+0000","last_clean":"2024-09-10T21:29:40.769158+0000","last_became_active":"2024-09-10T21:29:38.535669+0000","last_became_peered":"2024-09-10T21:29:38.535669+0000","last_unstale":"2024-09-10T21:29:40.769158+0000","last_undegraded":"2024-09-10T21:29:40.769158+0000","last_fullsized":"2024-09-10T21:29:40.769158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769743+0000","last_change":"2024-09-10T21:29:39.561954+0000","last_active":"2024-09-10T21:29:40.769743+0000","last_peered":"2024-09-10T21:29:40.769743+0000","last_clean":"2024-09-10T21:29:40.769743+0000","last_became_active":"2024-09-10T21:29:39.561348+0000","last_became_peered":"2024-09-10T21:29:39.561348+0000","last_unstale":"2024-09-10T21:29:40.769743+0000","last_undegraded":"2024-09-10T21:29:40.769743+0000","last_fullsized":"2024-09-10T21:29:40.769743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:54.689918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769497+0000","last_change":"2024-09-10T21:29:38.543780+0000","last_active":"2024-09-10T21:29:40.769497+0000","last_peered":"2024-09-10T21:29:40.769497+0000","last_clean":"2024-09-10T21:29:40.769497+0000","last_became_active":"2024-09-10T21:29:38.543305+0000","last_became_peered":"2024-09-10T21:29:38.543305+0000","last_unstale":"2024-09-10T21:29:40.769497+0000","last_undegraded":"2024-09-10T21:29:40.769497+0000","last_fullsized":"2024-09-10T21:29:40.769497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.941433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768491+0000","last_change":"2024-09-10T21:29:39.549779+0000","last_active":"2024-09-10T21:29:40.768491+0000","last_peered":"2024-09-10T21:29:40.768491+0000","last_clean":"2024-09-10T21:29:40.768491+0000","last_became_active":"2024-09-10T21:29:39.549473+0000","last_became_peered":"2024-09-10T21:29:39.549473+0000","last_unstale":"2024-09-10T21:29:40.768491+0000","last_undegraded":"2024-09-10T21:29:40.768491+0000","last_fullsized":"2024-09-10T21:29:40.768491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":79,"state":"creating+peering","last_fresh":"2024-09-10T21:29:37.517765+0000","last_change":"2024-09-10T21:29:37.517765+0000","last_active":"2024-09-10T21:29:37.510514+0000","last_peered":"2024-09-10T21:29:37.510514+0000","last_clean":"2024-09-10T21:29:37.510514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:37.517765+0000","last_undegraded":"2024-09-10T21:29:37.517765+0000","last_fullsized":"2024-09-10T21:29:37.517765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769992+0000","last_change":"2024-09-10T21:29:39.558263+0000","last_active":"2024-09-10T21:29:40.769992+0000","last_peered":"2024-09-10T21:29:40.769992+0000","last_clean":"2024-09-10T21:29:40.769992+0000","last_became_active":"2024-09-10T21:29:39.557990+0000","last_became_peered":"2024-09-10T21:29:39.557990+0000","last_unstale":"2024-09-10T21:29:40.769992+0000","last_undegraded":"2024-09-10T21:29:40.769992+0000","last_fullsized":"2024-09-10T21:29:40.769992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.511263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768565+0000","last_change":"2024-09-10T21:29:38.545254+0000","last_active":"2024-09-10T21:29:40.768565+0000","last_peered":"2024-09-10T21:29:40.768565+0000","last_clean":"2024-09-10T21:29:40.768565+0000","last_became_active":"2024-09-10T21:29:38.544954+0000","last_became_peered":"2024-09-10T21:29:38.544954+0000","last_unstale":"2024-09-10T21:29:40.768565+0000","last_undegraded":"2024-09-10T21:29:40.768565+0000","last_fullsized":"2024-09-10T21:29:40.768565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768528+0000","last_change":"2024-09-10T21:29:39.555635+0000","last_active":"2024-09-10T21:29:40.768528+0000","last_peered":"2024-09-10T21:29:40.768528+0000","last_clean":"2024-09-10T21:29:40.768528+0000","last_became_active":"2024-09-10T21:29:39.555011+0000","last_became_peered":"2024-09-10T21:29:39.555011+0000","last_unstale":"2024-09-10T21:29:40.768528+0000","last_undegraded":"2024-09-10T21:29:40.768528+0000","last_fullsized":"2024-09-10T21:29:40.768528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:15.429757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545012+0000","last_change":"2024-09-10T21:29:38.549092+0000","last_active":"2024-09-10T21:29:40.545012+0000","last_peered":"2024-09-10T21:29:40.545012+0000","last_clean":"2024-09-10T21:29:40.545012+0000","last_became_active":"2024-09-10T21:29:38.548800+0000","last_became_peered":"2024-09-10T21:29:38.548800+0000","last_unstale":"2024-09-10T21:29:40.545012+0000","last_undegraded":"2024-09-10T21:29:40.545012+0000","last_fullsized":"2024-09-10T21:29:40.545012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:10.710966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768928+0000","last_change":"2024-09-10T21:29:39.554166+0000","last_active":"2024-09-10T21:29:40.768928+0000","last_peered":"2024-09-10T21:29:40.768928+0000","last_clean":"2024-09-10T21:29:40.768928+0000","last_became_active":"2024-09-10T21:29:39.551856+0000","last_became_peered":"2024-09-10T21:29:39.551856+0000","last_unstale":"2024-09-10T21:29:40.768928+0000","last_undegraded":"2024-09-10T21:29:40.768928+0000","last_fullsized":"2024-09-10T21:29:40.768928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.611740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769311+0000","last_change":"2024-09-10T21:29:38.545177+0000","last_active":"2024-09-10T21:29:40.769311+0000","last_peered":"2024-09-10T21:29:40.769311+0000","last_clean":"2024-09-10T21:29:40.769311+0000","last_became_active":"2024-09-10T21:29:38.544523+0000","last_became_peered":"2024-09-10T21:29:38.544523+0000","last_unstale":"2024-09-10T21:29:40.769311+0000","last_undegraded":"2024-09-10T21:29:40.769311+0000","last_fullsized":"2024-09-10T21:29:40.769311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:36.904722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546787+0000","last_change":"2024-09-10T21:29:38.542661+0000","last_active":"2024-09-10T21:29:40.546787+0000","last_peered":"2024-09-10T21:29:40.546787+0000","last_clean":"2024-09-10T21:29:40.546787+0000","last_became_active":"2024-09-10T21:29:38.542425+0000","last_became_peered":"2024-09-10T21:29:38.542425+0000","last_unstale":"2024-09-10T21:29:40.546787+0000","last_undegraded":"2024-09-10T21:29:40.546787+0000","last_fullsized":"2024-09-10T21:29:40.546787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:47.295559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545841+0000","last_change":"2024-09-10T21:29:39.548032+0000","last_active":"2024-09-10T21:29:40.545841+0000","last_peered":"2024-09-10T21:29:40.545841+0000","last_clean":"2024-09-10T21:29:40.545841+0000","last_became_active":"2024-09-10T21:29:39.547709+0000","last_became_peered":"2024-09-10T21:29:39.547709+0000","last_unstale":"2024-09-10T21:29:40.545841+0000","last_undegraded":"2024-09-10T21:29:40.545841+0000","last_fullsized":"2024-09-10T21:29:40.545841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769782+0000","last_change":"2024-09-10T21:29:38.532633+0000","last_active":"2024-09-10T21:29:40.769782+0000","last_peered":"2024-09-10T21:29:40.769782+0000","last_clean":"2024-09-10T21:29:40.769782+0000","last_became_active":"2024-09-10T21:29:38.532371+0000","last_became_peered":"2024-09-10T21:29:38.532371+0000","last_unstale":"2024-09-10T21:29:40.769782+0000","last_undegraded":"2024-09-10T21:29:40.769782+0000","last_fullsized":"2024-09-10T21:29:40.769782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:45.933789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544716+0000","last_change":"2024-09-10T21:29:39.554909+0000","last_active":"2024-09-10T21:29:40.544716+0000","last_peered":"2024-09-10T21:29:40.544716+0000","last_clean":"2024-09-10T21:29:40.544716+0000","last_became_active":"2024-09-10T21:29:39.554651+0000","last_became_peered":"2024-09-10T21:29:39.554651+0000","last_unstale":"2024-09-10T21:29:40.544716+0000","last_undegraded":"2024-09-10T21:29:40.544716+0000","last_fullsized":"2024-09-10T21:29:40.544716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768411+0000","last_change":"2024-09-10T21:29:38.535431+0000","last_active":"2024-09-10T21:29:40.768411+0000","last_peered":"2024-09-10T21:29:40.768411+0000","last_clean":"2024-09-10T21:29:40.768411+0000","last_became_active":"2024-09-10T21:29:38.535114+0000","last_became_peered":"2024-09-10T21:29:38.535114+0000","last_unstale":"2024-09-10T21:29:40.768411+0000","last_undegraded":"2024-09-10T21:29:40.768411+0000","last_fullsized":"2024-09-10T21:29:40.768411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:05.208145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768791+0000","last_change":"2024-09-10T21:29:39.563604+0000","last_active":"2024-09-10T21:29:40.768791+0000","last_peered":"2024-09-10T21:29:40.768791+0000","last_clean":"2024-09-10T21:29:40.768791+0000","last_became_active":"2024-09-10T21:29:39.563344+0000","last_became_peered":"2024-09-10T21:29:39.563344+0000","last_unstale":"2024-09-10T21:29:40.768791+0000","last_undegraded":"2024-09-10T21:29:40.768791+0000","last_fullsized":"2024-09-10T21:29:40.768791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:50.487501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769250+0000","last_change":"2024-09-10T21:29:38.544304+0000","last_active":"2024-09-10T21:29:40.769250+0000","last_peered":"2024-09-10T21:29:40.769250+0000","last_clean":"2024-09-10T21:29:40.769250+0000","last_became_active":"2024-09-10T21:29:38.544049+0000","last_became_peered":"2024-09-10T21:29:38.544049+0000","last_unstale":"2024-09-10T21:29:40.769250+0000","last_undegraded":"2024-09-10T21:29:40.769250+0000","last_fullsized":"2024-09-10T21:29:40.769250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768692+0000","last_change":"2024-09-10T21:29:39.544825+0000","last_active":"2024-09-10T21:29:40.768692+0000","last_peered":"2024-09-10T21:29:40.768692+0000","last_clean":"2024-09-10T21:29:40.768692+0000","last_became_active":"2024-09-10T21:29:39.544578+0000","last_became_peered":"2024-09-10T21:29:39.544578+0000","last_unstale":"2024-09-10T21:29:40.768692+0000","last_undegraded":"2024-09-10T21:29:40.768692+0000","last_fullsized":"2024-09-10T21:29:40.768692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:17.043812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546585+0000","last_change":"2024-09-10T21:29:38.549765+0000","last_active":"2024-09-10T21:29:40.546585+0000","last_peered":"2024-09-10T21:29:40.546585+0000","last_clean":"2024-09-10T21:29:40.546585+0000","last_became_active":"2024-09-10T21:29:38.548990+0000","last_became_peered":"2024-09-10T21:29:38.548990+0000","last_unstale":"2024-09-10T21:29:40.546585+0000","last_undegraded":"2024-09-10T21:29:40.546585+0000","last_fullsized":"2024-09-10T21:29:40.546585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:52.559734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768695+0000","last_change":"2024-09-10T21:29:39.547930+0000","last_active":"2024-09-10T21:29:40.768695+0000","last_peered":"2024-09-10T21:29:40.768695+0000","last_clean":"2024-09-10T21:29:40.768695+0000","last_became_active":"2024-09-10T21:29:39.547672+0000","last_became_peered":"2024-09-10T21:29:39.547672+0000","last_unstale":"2024-09-10T21:29:40.768695+0000","last_undegraded":"2024-09-10T21:29:40.768695+0000","last_fullsized":"2024-09-10T21:29:40.768695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:52.513550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545358+0000","last_change":"2024-09-10T21:29:38.544208+0000","last_active":"2024-09-10T21:29:40.545358+0000","last_peered":"2024-09-10T21:29:40.545358+0000","last_clean":"2024-09-10T21:29:40.545358+0000","last_became_active":"2024-09-10T21:29:38.543907+0000","last_became_peered":"2024-09-10T21:29:38.543907+0000","last_unstale":"2024-09-10T21:29:40.545358+0000","last_undegraded":"2024-09-10T21:29:40.545358+0000","last_fullsized":"2024-09-10T21:29:40.545358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:37.225392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546397+0000","last_change":"2024-09-10T21:29:39.557707+0000","last_active":"2024-09-10T21:29:40.546397+0000","last_peered":"2024-09-10T21:29:40.546397+0000","last_clean":"2024-09-10T21:29:40.546397+0000","last_became_active":"2024-09-10T21:29:39.557398+0000","last_became_peered":"2024-09-10T21:29:39.557398+0000","last_unstale":"2024-09-10T21:29:40.546397+0000","last_undegraded":"2024-09-10T21:29:40.546397+0000","last_fullsized":"2024-09-10T21:29:40.546397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:05.442260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545105+0000","last_change":"2024-09-10T21:29:38.549044+0000","last_active":"2024-09-10T21:29:40.545105+0000","last_peered":"2024-09-10T21:29:40.545105+0000","last_clean":"2024-09-10T21:29:40.545105+0000","last_became_active":"2024-09-10T21:29:38.548465+0000","last_became_peered":"2024-09-10T21:29:38.548465+0000","last_unstale":"2024-09-10T21:29:40.545105+0000","last_undegraded":"2024-09-10T21:29:40.545105+0000","last_fullsized":"2024-09-10T21:29:40.545105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:47.729570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769221+0000","last_change":"2024-09-10T21:29:39.555291+0000","last_active":"2024-09-10T21:29:40.769221+0000","last_peered":"2024-09-10T21:29:40.769221+0000","last_clean":"2024-09-10T21:29:40.769221+0000","last_became_active":"2024-09-10T21:29:39.554979+0000","last_became_peered":"2024-09-10T21:29:39.554979+0000","last_unstale":"2024-09-10T21:29:40.769221+0000","last_undegraded":"2024-09-10T21:29:40.769221+0000","last_fullsized":"2024-09-10T21:29:40.769221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:41.126386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544896+0000","last_change":"2024-09-10T21:29:38.545682+0000","last_active":"2024-09-10T21:29:40.544896+0000","last_peered":"2024-09-10T21:29:40.544896+0000","last_clean":"2024-09-10T21:29:40.544896+0000","last_became_active":"2024-09-10T21:29:38.545432+0000","last_became_peered":"2024-09-10T21:29:38.545432+0000","last_unstale":"2024-09-10T21:29:40.544896+0000","last_undegraded":"2024-09-10T21:29:40.544896+0000","last_fullsized":"2024-09-10T21:29:40.544896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546758+0000","last_change":"2024-09-10T21:29:38.550165+0000","last_active":"2024-09-10T21:29:40.546758+0000","last_peered":"2024-09-10T21:29:40.546758+0000","last_clean":"2024-09-10T21:29:40.546758+0000","last_became_active":"2024-09-10T21:29:38.549861+0000","last_became_peered":"2024-09-10T21:29:38.549861+0000","last_unstale":"2024-09-10T21:29:40.546758+0000","last_undegraded":"2024-09-10T21:29:40.546758+0000","last_fullsized":"2024-09-10T21:29:40.546758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:02.189449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547170+0000","last_change":"2024-09-10T21:29:39.545985+0000","last_active":"2024-09-10T21:29:40.547170+0000","last_peered":"2024-09-10T21:29:40.547170+0000","last_clean":"2024-09-10T21:29:40.547170+0000","last_became_active":"2024-09-10T21:29:39.545349+0000","last_became_peered":"2024-09-10T21:29:39.545349+0000","last_unstale":"2024-09-10T21:29:40.547170+0000","last_undegraded":"2024-09-10T21:29:40.547170+0000","last_fullsized":"2024-09-10T21:29:40.547170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:14.688355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547653+0000","last_change":"2024-09-10T21:29:38.546910+0000","last_active":"2024-09-10T21:29:40.547653+0000","last_peered":"2024-09-10T21:29:40.547653+0000","last_clean":"2024-09-10T21:29:40.547653+0000","last_became_active":"2024-09-10T21:29:38.546159+0000","last_became_peered":"2024-09-10T21:29:38.546159+0000","last_unstale":"2024-09-10T21:29:40.547653+0000","last_undegraded":"2024-09-10T21:29:40.547653+0000","last_fullsized":"2024-09-10T21:29:40.547653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.324328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547733+0000","last_change":"2024-09-10T21:29:39.551802+0000","last_active":"2024-09-10T21:29:40.547733+0000","last_peered":"2024-09-10T21:29:40.547733+0000","last_clean":"2024-09-10T21:29:40.547733+0000","last_became_active":"2024-09-10T21:29:39.550894+0000","last_became_peered":"2024-09-10T21:29:39.550894+0000","last_unstale":"2024-09-10T21:29:40.547733+0000","last_undegraded":"2024-09-10T21:29:40.547733+0000","last_fullsized":"2024-09-10T21:29:40.547733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:35.976956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768722+0000","last_change":"2024-09-10T21:29:38.549752+0000","last_active":"2024-09-10T21:29:40.768722+0000","last_peered":"2024-09-10T21:29:40.768722+0000","last_clean":"2024-09-10T21:29:40.768722+0000","last_became_active":"2024-09-10T21:29:38.549380+0000","last_became_peered":"2024-09-10T21:29:38.549380+0000","last_unstale":"2024-09-10T21:29:40.768722+0000","last_undegraded":"2024-09-10T21:29:40.768722+0000","last_fullsized":"2024-09-10T21:29:40.768722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547195+0000","last_change":"2024-09-10T21:29:39.544277+0000","last_active":"2024-09-10T21:29:40.547195+0000","last_peered":"2024-09-10T21:29:40.547195+0000","last_clean":"2024-09-10T21:29:40.547195+0000","last_became_active":"2024-09-10T21:29:39.544011+0000","last_became_peered":"2024-09-10T21:29:39.544011+0000","last_unstale":"2024-09-10T21:29:40.547195+0000","last_undegraded":"2024-09-10T21:29:40.547195+0000","last_fullsized":"2024-09-10T21:29:40.547195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:05.999146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":79,"state":"creating+peering","last_fresh":"2024-09-10T21:29:37.517236+0000","last_change":"2024-09-10T21:29:37.517236+0000","last_active":"2024-09-10T21:29:37.510514+0000","last_peered":"2024-09-10T21:29:37.510514+0000","last_clean":"2024-09-10T21:29:37.510514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:37.517236+0000","last_undegraded":"2024-09-10T21:29:37.517236+0000","last_fullsized":"2024-09-10T21:29:37.517236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.770023+0000","last_change":"2024-09-10T21:29:39.545983+0000","last_active":"2024-09-10T21:29:40.770023+0000","last_peered":"2024-09-10T21:29:40.770023+0000","last_clean":"2024-09-10T21:29:40.770023+0000","last_became_active":"2024-09-10T21:29:39.545699+0000","last_became_peered":"2024-09-10T21:29:39.545699+0000","last_unstale":"2024-09-10T21:29:40.770023+0000","last_undegraded":"2024-09-10T21:29:40.770023+0000","last_fullsized":"2024-09-10T21:29:40.770023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.110951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768649+0000","last_change":"2024-09-10T21:29:38.546518+0000","last_active":"2024-09-10T21:29:40.768649+0000","last_peered":"2024-09-10T21:29:40.768649+0000","last_clean":"2024-09-10T21:29:40.768649+0000","last_became_active":"2024-09-10T21:29:38.546262+0000","last_became_peered":"2024-09-10T21:29:38.546262+0000","last_unstale":"2024-09-10T21:29:40.768649+0000","last_undegraded":"2024-09-10T21:29:40.768649+0000","last_fullsized":"2024-09-10T21:29:40.768649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544409+0000","last_change":"2024-09-10T21:29:39.554112+0000","last_active":"2024-09-10T21:29:40.544409+0000","last_peered":"2024-09-10T21:29:40.544409+0000","last_clean":"2024-09-10T21:29:40.544409+0000","last_became_active":"2024-09-10T21:29:39.552077+0000","last_became_peered":"2024-09-10T21:29:39.552077+0000","last_unstale":"2024-09-10T21:29:40.544409+0000","last_undegraded":"2024-09-10T21:29:40.544409+0000","last_fullsized":"2024-09-10T21:29:40.544409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:18.545317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769044+0000","last_change":"2024-09-10T21:29:38.544002+0000","last_active":"2024-09-10T21:29:40.769044+0000","last_peered":"2024-09-10T21:29:40.769044+0000","last_clean":"2024-09-10T21:29:40.769044+0000","last_became_active":"2024-09-10T21:29:38.543699+0000","last_became_peered":"2024-09-10T21:29:38.543699+0000","last_unstale":"2024-09-10T21:29:40.769044+0000","last_undegraded":"2024-09-10T21:29:40.769044+0000","last_fullsized":"2024-09-10T21:29:40.769044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:21.454494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768948+0000","last_change":"2024-09-10T21:29:39.558189+0000","last_active":"2024-09-10T21:29:40.768948+0000","last_peered":"2024-09-10T21:29:40.768948+0000","last_clean":"2024-09-10T21:29:40.768948+0000","last_became_active":"2024-09-10T21:29:39.557549+0000","last_became_peered":"2024-09-10T21:29:39.557549+0000","last_unstale":"2024-09-10T21:29:40.768948+0000","last_undegraded":"2024-09-10T21:29:40.768948+0000","last_fullsized":"2024-09-10T21:29:40.768948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:48.581722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547028+0000","last_change":"2024-09-10T21:29:38.544536+0000","last_active":"2024-09-10T21:29:40.547028+0000","last_peered":"2024-09-10T21:29:40.547028+0000","last_clean":"2024-09-10T21:29:40.547028+0000","last_became_active":"2024-09-10T21:29:38.544212+0000","last_became_peered":"2024-09-10T21:29:38.544212+0000","last_unstale":"2024-09-10T21:29:40.547028+0000","last_undegraded":"2024-09-10T21:29:40.547028+0000","last_fullsized":"2024-09-10T21:29:40.547028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:53.514589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769300+0000","last_change":"2024-09-10T21:29:39.546810+0000","last_active":"2024-09-10T21:29:40.769300+0000","last_peered":"2024-09-10T21:29:40.769300+0000","last_clean":"2024-09-10T21:29:40.769300+0000","last_became_active":"2024-09-10T21:29:39.546317+0000","last_became_peered":"2024-09-10T21:29:39.546317+0000","last_unstale":"2024-09-10T21:29:40.769300+0000","last_undegraded":"2024-09-10T21:29:40.769300+0000","last_fullsized":"2024-09-10T21:29:40.769300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:46.343210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768987+0000","last_change":"2024-09-10T21:29:38.533647+0000","last_active":"2024-09-10T21:29:40.768987+0000","last_peered":"2024-09-10T21:29:40.768987+0000","last_clean":"2024-09-10T21:29:40.768987+0000","last_became_active":"2024-09-10T21:29:38.533361+0000","last_became_peered":"2024-09-10T21:29:38.533361+0000","last_unstale":"2024-09-10T21:29:40.768987+0000","last_undegraded":"2024-09-10T21:29:40.768987+0000","last_fullsized":"2024-09-10T21:29:40.768987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:32.782811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544676+0000","last_change":"2024-09-10T21:29:38.547087+0000","last_active":"2024-09-10T21:29:40.544676+0000","last_peered":"2024-09-10T21:29:40.544676+0000","last_clean":"2024-09-10T21:29:40.544676+0000","last_became_active":"2024-09-10T21:29:38.546829+0000","last_became_peered":"2024-09-10T21:29:38.546829+0000","last_unstale":"2024-09-10T21:29:40.544676+0000","last_undegraded":"2024-09-10T21:29:40.544676+0000","last_fullsized":"2024-09-10T21:29:40.544676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769511+0000","last_change":"2024-09-10T21:29:39.550392+0000","last_active":"2024-09-10T21:29:40.769511+0000","last_peered":"2024-09-10T21:29:40.769511+0000","last_clean":"2024-09-10T21:29:40.769511+0000","last_became_active":"2024-09-10T21:29:39.550131+0000","last_became_peered":"2024-09-10T21:29:39.550131+0000","last_unstale":"2024-09-10T21:29:40.769511+0000","last_undegraded":"2024-09-10T21:29:40.769511+0000","last_fullsized":"2024-09-10T21:29:40.769511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:15.885819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769553+0000","last_change":"2024-09-10T21:29:38.549285+0000","last_active":"2024-09-10T21:29:40.769553+0000","last_peered":"2024-09-10T21:29:40.769553+0000","last_clean":"2024-09-10T21:29:40.769553+0000","last_became_active":"2024-09-10T21:29:38.549014+0000","last_became_peered":"2024-09-10T21:29:38.549014+0000","last_unstale":"2024-09-10T21:29:40.769553+0000","last_undegraded":"2024-09-10T21:29:40.769553+0000","last_fullsized":"2024-09-10T21:29:40.769553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:58.754314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544681+0000","last_change":"2024-09-10T21:29:39.562581+0000","last_active":"2024-09-10T21:29:40.544681+0000","last_peered":"2024-09-10T21:29:40.544681+0000","last_clean":"2024-09-10T21:29:40.544681+0000","last_became_active":"2024-09-10T21:29:39.562287+0000","last_became_peered":"2024-09-10T21:29:39.562287+0000","last_unstale":"2024-09-10T21:29:40.544681+0000","last_undegraded":"2024-09-10T21:29:40.544681+0000","last_fullsized":"2024-09-10T21:29:40.544681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545978+0000","last_change":"2024-09-10T21:29:38.546345+0000","last_active":"2024-09-10T21:29:40.545978+0000","last_peered":"2024-09-10T21:29:40.545978+0000","last_clean":"2024-09-10T21:29:40.545978+0000","last_became_active":"2024-09-10T21:29:38.546094+0000","last_became_peered":"2024-09-10T21:29:38.546094+0000","last_unstale":"2024-09-10T21:29:40.545978+0000","last_undegraded":"2024-09-10T21:29:40.545978+0000","last_fullsized":"2024-09-10T21:29:40.545978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769644+0000","last_change":"2024-09-10T21:29:39.558347+0000","last_active":"2024-09-10T21:29:40.769644+0000","last_peered":"2024-09-10T21:29:40.769644+0000","last_clean":"2024-09-10T21:29:40.769644+0000","last_became_active":"2024-09-10T21:29:39.557760+0000","last_became_peered":"2024-09-10T21:29:39.557760+0000","last_unstale":"2024-09-10T21:29:40.769644+0000","last_undegraded":"2024-09-10T21:29:40.769644+0000","last_fullsized":"2024-09-10T21:29:40.769644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:06.975636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544238+0000","last_change":"2024-09-10T21:29:38.535077+0000","last_active":"2024-09-10T21:29:40.544238+0000","last_peered":"2024-09-10T21:29:40.544238+0000","last_clean":"2024-09-10T21:29:40.544238+0000","last_became_active":"2024-09-10T21:29:38.534747+0000","last_became_peered":"2024-09-10T21:29:38.534747+0000","last_unstale":"2024-09-10T21:29:40.544238+0000","last_undegraded":"2024-09-10T21:29:40.544238+0000","last_fullsized":"2024-09-10T21:29:40.544238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547454+0000","last_change":"2024-09-10T21:29:39.548356+0000","last_active":"2024-09-10T21:29:40.547454+0000","last_peered":"2024-09-10T21:29:40.547454+0000","last_clean":"2024-09-10T21:29:40.547454+0000","last_became_active":"2024-09-10T21:29:39.548096+0000","last_became_peered":"2024-09-10T21:29:39.548096+0000","last_unstale":"2024-09-10T21:29:40.547454+0000","last_undegraded":"2024-09-10T21:29:40.547454+0000","last_fullsized":"2024-09-10T21:29:40.547454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:50.836505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768672+0000","last_change":"2024-09-10T21:29:38.544063+0000","last_active":"2024-09-10T21:29:40.768672+0000","last_peered":"2024-09-10T21:29:40.768672+0000","last_clean":"2024-09-10T21:29:40.768672+0000","last_became_active":"2024-09-10T21:29:38.543508+0000","last_became_peered":"2024-09-10T21:29:38.543508+0000","last_unstale":"2024-09-10T21:29:40.768672+0000","last_undegraded":"2024-09-10T21:29:40.768672+0000","last_fullsized":"2024-09-10T21:29:40.768672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:02.927923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769972+0000","last_change":"2024-09-10T21:29:39.552135+0000","last_active":"2024-09-10T21:29:40.769972+0000","last_peered":"2024-09-10T21:29:40.769972+0000","last_clean":"2024-09-10T21:29:40.769972+0000","last_became_active":"2024-09-10T21:29:39.551649+0000","last_became_peered":"2024-09-10T21:29:39.551649+0000","last_unstale":"2024-09-10T21:29:40.769972+0000","last_undegraded":"2024-09-10T21:29:40.769972+0000","last_fullsized":"2024-09-10T21:29:40.769972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:26.827601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545785+0000","last_change":"2024-09-10T21:29:38.547451+0000","last_active":"2024-09-10T21:29:40.545785+0000","last_peered":"2024-09-10T21:29:40.545785+0000","last_clean":"2024-09-10T21:29:40.545785+0000","last_became_active":"2024-09-10T21:29:38.546360+0000","last_became_peered":"2024-09-10T21:29:38.546360+0000","last_unstale":"2024-09-10T21:29:40.545785+0000","last_undegraded":"2024-09-10T21:29:40.545785+0000","last_fullsized":"2024-09-10T21:29:40.545785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768220+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:29:40.768220+0000","last_peered":"2024-09-10T21:29:40.768220+0000","last_clean":"2024-09-10T21:29:40.768220+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:29:40.768220+0000","last_undegraded":"2024-09-10T21:29:40.768220+0000","last_fullsized":"2024-09-10T21:29:40.768220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":458807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":1048593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":453159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":458135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1605632,"data_stored":1050149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":958464,"data_stored":437609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:29:43.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:29:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 -- 172.21.15.97:0/3793887932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df81511f0 msgr2=0x7f6df80b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 --2- 172.21.15.97:0/3793887932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df81511f0 0x7f6df80b37b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6de800aa40 tx=0x7f6de8020400 comp rx=0 tx=0).stop 2024-09-10T21:29:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 -- 172.21.15.97:0/3793887932 shutdown_connections 2024-09-10T21:29:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 --2- 172.21.15.97:0/3793887932 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6df80ceee0 0x7f6df80cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 --2- 172.21.15.97:0/3793887932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df81511f0 0x7f6df80b37b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 -- 172.21.15.97:0/3793887932 >> 172.21.15.97:0/3793887932 conn(0x7f6df80b1780 msgr2=0x7f6df80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 -- 172.21.15.97:0/3793887932 shutdown_connections 2024-09-10T21:29:43.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.837+0000 7f6dff200640 1 -- 172.21.15.97:0/3793887932 wait complete. 2024-09-10T21:29:43.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 Processor -- start 2024-09-10T21:29:43.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 -- start start 2024-09-10T21:29:43.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6df80b3cf0 0x7f6df8159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df80ceee0 0x7f6df81f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81511f0 0x7f6df8154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6df806e740 con 0x7f6df80ceee0 2024-09-10T21:29:43.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6df806e5c0 con 0x7f6df81511f0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6df806e8c0 con 0x7f6df80b3cf0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df80ceee0 0x7f6df81f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df80ceee0 0x7f6df81f82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35128/0 (socket says 172.21.15.97:35128) 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 -- 172.21.15.97:0/2021793710 learned_addr learned my addr 172.21.15.97:0/2021793710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dfd200640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81511f0 0x7f6df8154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6df7e00640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6df80b3cf0 0x7f6df8159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 -- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6df80b3cf0 msgr2=0x7f6df8159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6df80b3cf0 0x7f6df8159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 -- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81511f0 msgr2=0x7f6df8154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81511f0 0x7f6df8154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 -- 172.21.15.97:0/2021793710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df81f89b0 con 0x7f6df80ceee0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6def400640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df80ceee0 0x7f6df81f82d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6de000c450 tx=0x7f6de000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6df4c00640 1 -- 172.21.15.97:0/2021793710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0012830 con 0x7f6df80ceee0 2024-09-10T21:29:43.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6df4c00640 1 -- 172.21.15.97:0/2021793710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6de0017070 con 0x7f6df80ceee0 2024-09-10T21:29:43.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df81f8c90 con 0x7f6df80ceee0 2024-09-10T21:29:43.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6df4c00640 1 -- 172.21.15.97:0/2021793710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0012d80 con 0x7f6df80ceee0 2024-09-10T21:29:43.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.841+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df81f4e80 con 0x7f6df80ceee0 2024-09-10T21:29:43.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.845+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df80cf370 con 0x7f6df80ceee0 2024-09-10T21:29:43.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.845+0000 7f6df4c00640 1 -- 172.21.15.97:0/2021793710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6de001c460 con 0x7f6df80ceee0 2024-09-10T21:29:43.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.845+0000 7f6df4c00640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6dd40725e0 0x7f6dd4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.845+0000 7f6df4c00640 1 -- 172.21.15.97:0/2021793710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f6de00931d0 con 0x7f6df80ceee0 2024-09-10T21:29:43.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.845+0000 7f6df7e00640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6dd40725e0 0x7f6dd4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.845+0000 7f6df7e00640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6dd40725e0 0x7f6dd4074aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6de80b2000 tx=0x7f6de801ec40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:43.849+0000 7f6df4c00640 1 -- 172.21.15.97:0/2021793710 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6de005e710 con 0x7f6df80ceee0 2024-09-10T21:29:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.013+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6df80cbf00 con 0x7f6dd40725e0 2024-09-10T21:29:44.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.017+0000 7f6df4c00640 1 -- 172.21.15.97:0/2021793710 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250862 (secure 0 0 0) 0x7f6df80cbf00 con 0x7f6dd40725e0 2024-09-10T21:29:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:44.025 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:29:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6dd40725e0 msgr2=0x7f6dd4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6dd40725e0 0x7f6dd4074aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6de80b2000 tx=0x7f6de801ec40 comp rx=0 tx=0).stop 2024-09-10T21:29:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df80ceee0 msgr2=0x7f6df81f82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df80ceee0 0x7f6df81f82d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6de000c450 tx=0x7f6de000c910 comp rx=0 tx=0).stop 2024-09-10T21:29:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 shutdown_connections 2024-09-10T21:29:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6dd40725e0 0x7f6dd4074aa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df81511f0 0x7f6df8154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6df80ceee0 0x7f6df81f82d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 --2- 172.21.15.97:0/2021793710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6df80b3cf0 0x7f6df8159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 >> 172.21.15.97:0/2021793710 conn(0x7f6df80b1780 msgr2=0x7f6df80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 shutdown_connections 2024-09-10T21:29:44.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.021+0000 7f6dff200640 1 -- 172.21.15.97:0/2021793710 wait complete. 2024-09-10T21:29:44.044 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-10T21:29:42.155481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241836,"kb_used_data":8948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754732884,"statfs":{"total":773094113280,"available":772846473216,"internally_reserved":0,"allocated":9162752,"data_stored":4823396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.640643"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768894+0000","last_change":"2024-09-10T21:29:39.539326+0000","last_active":"2024-09-10T21:29:40.768894+0000","last_peered":"2024-09-10T21:29:40.768894+0000","last_clean":"2024-09-10T21:29:40.768894+0000","last_became_active":"2024-09-10T21:29:39.539056+0000","last_became_peered":"2024-09-10T21:29:39.539056+0000","last_unstale":"2024-09-10T21:29:40.768894+0000","last_undegraded":"2024-09-10T21:29:40.768894+0000","last_fullsized":"2024-09-10T21:29:40.768894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:25.767146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769190+0000","last_change":"2024-09-10T21:29:39.564404+0000","last_active":"2024-09-10T21:29:40.769190+0000","last_peered":"2024-09-10T21:29:40.769190+0000","last_clean":"2024-09-10T21:29:40.769190+0000","last_became_active":"2024-09-10T21:29:39.563306+0000","last_became_peered":"2024-09-10T21:29:39.563306+0000","last_unstale":"2024-09-10T21:29:40.769190+0000","last_undegraded":"2024-09-10T21:29:40.769190+0000","last_fullsized":"2024-09-10T21:29:40.769190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:17.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769401+0000","last_change":"2024-09-10T21:29:39.561606+0000","last_active":"2024-09-10T21:29:40.769401+0000","last_peered":"2024-09-10T21:29:40.769401+0000","last_clean":"2024-09-10T21:29:40.769401+0000","last_became_active":"2024-09-10T21:29:39.560972+0000","last_became_peered":"2024-09-10T21:29:39.560972+0000","last_unstale":"2024-09-10T21:29:40.769401+0000","last_undegraded":"2024-09-10T21:29:40.769401+0000","last_fullsized":"2024-09-10T21:29:40.769401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:26.701898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544576+0000","last_change":"2024-09-10T21:29:39.537773+0000","last_active":"2024-09-10T21:29:40.544576+0000","last_peered":"2024-09-10T21:29:40.544576+0000","last_clean":"2024-09-10T21:29:40.544576+0000","last_became_active":"2024-09-10T21:29:39.537487+0000","last_became_peered":"2024-09-10T21:29:39.537487+0000","last_unstale":"2024-09-10T21:29:40.544576+0000","last_undegraded":"2024-09-10T21:29:40.544576+0000","last_fullsized":"2024-09-10T21:29:40.544576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:53.105160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546947+0000","last_change":"2024-09-10T21:29:39.546644+0000","last_active":"2024-09-10T21:29:40.546947+0000","last_peered":"2024-09-10T21:29:40.546947+0000","last_clean":"2024-09-10T21:29:40.546947+0000","last_became_active":"2024-09-10T21:29:39.546373+0000","last_became_peered":"2024-09-10T21:29:39.546373+0000","last_unstale":"2024-09-10T21:29:40.546947+0000","last_undegraded":"2024-09-10T21:29:40.546947+0000","last_fullsized":"2024-09-10T21:29:40.546947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.942938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546101+0000","last_change":"2024-09-10T21:29:39.558707+0000","last_active":"2024-09-10T21:29:40.546101+0000","last_peered":"2024-09-10T21:29:40.546101+0000","last_clean":"2024-09-10T21:29:40.546101+0000","last_became_active":"2024-09-10T21:29:39.554793+0000","last_became_peered":"2024-09-10T21:29:39.554793+0000","last_unstale":"2024-09-10T21:29:40.546101+0000","last_undegraded":"2024-09-10T21:29:40.546101+0000","last_fullsized":"2024-09-10T21:29:40.546101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544328+0000","last_change":"2024-09-10T21:29:39.540653+0000","last_active":"2024-09-10T21:29:40.544328+0000","last_peered":"2024-09-10T21:29:40.544328+0000","last_clean":"2024-09-10T21:29:40.544328+0000","last_became_active":"2024-09-10T21:29:39.540400+0000","last_became_peered":"2024-09-10T21:29:39.540400+0000","last_unstale":"2024-09-10T21:29:40.544328+0000","last_undegraded":"2024-09-10T21:29:40.544328+0000","last_fullsized":"2024-09-10T21:29:40.544328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:26.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545150+0000","last_change":"2024-09-10T21:29:39.556621+0000","last_active":"2024-09-10T21:29:40.545150+0000","last_peered":"2024-09-10T21:29:40.545150+0000","last_clean":"2024-09-10T21:29:40.545150+0000","last_became_active":"2024-09-10T21:29:39.555001+0000","last_became_peered":"2024-09-10T21:29:39.555001+0000","last_unstale":"2024-09-10T21:29:40.545150+0000","last_undegraded":"2024-09-10T21:29:40.545150+0000","last_fullsized":"2024-09-10T21:29:40.545150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769242+0000","last_change":"2024-09-10T21:29:39.536232+0000","last_active":"2024-09-10T21:29:40.769242+0000","last_peered":"2024-09-10T21:29:40.769242+0000","last_clean":"2024-09-10T21:29:40.769242+0000","last_became_active":"2024-09-10T21:29:39.535978+0000","last_became_peered":"2024-09-10T21:29:39.535978+0000","last_unstale":"2024-09-10T21:29:40.769242+0000","last_undegraded":"2024-09-10T21:29:40.769242+0000","last_fullsized":"2024-09-10T21:29:40.769242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:49.789897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769661+0000","last_change":"2024-09-10T21:29:39.562255+0000","last_active":"2024-09-10T21:29:40.769661+0000","last_peered":"2024-09-10T21:29:40.769661+0000","last_clean":"2024-09-10T21:29:40.769661+0000","last_became_active":"2024-09-10T21:29:39.559040+0000","last_became_peered":"2024-09-10T21:29:39.559040+0000","last_unstale":"2024-09-10T21:29:40.769661+0000","last_undegraded":"2024-09-10T21:29:40.769661+0000","last_fullsized":"2024-09-10T21:29:40.769661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:39.446682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769690+0000","last_change":"2024-09-10T21:29:39.550940+0000","last_active":"2024-09-10T21:29:40.769690+0000","last_peered":"2024-09-10T21:29:40.769690+0000","last_clean":"2024-09-10T21:29:40.769690+0000","last_became_active":"2024-09-10T21:29:39.550666+0000","last_became_peered":"2024-09-10T21:29:39.550666+0000","last_unstale":"2024-09-10T21:29:40.769690+0000","last_undegraded":"2024-09-10T21:29:40.769690+0000","last_fullsized":"2024-09-10T21:29:40.769690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:24.960891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768575+0000","last_change":"2024-09-10T21:29:39.555356+0000","last_active":"2024-09-10T21:29:40.768575+0000","last_peered":"2024-09-10T21:29:40.768575+0000","last_clean":"2024-09-10T21:29:40.768575+0000","last_became_active":"2024-09-10T21:29:39.554843+0000","last_became_peered":"2024-09-10T21:29:39.554843+0000","last_unstale":"2024-09-10T21:29:40.768575+0000","last_undegraded":"2024-09-10T21:29:40.768575+0000","last_fullsized":"2024-09-10T21:29:40.768575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:03.103988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546185+0000","last_change":"2024-09-10T21:29:39.563879+0000","last_active":"2024-09-10T21:29:40.546185+0000","last_peered":"2024-09-10T21:29:40.546185+0000","last_clean":"2024-09-10T21:29:40.546185+0000","last_became_active":"2024-09-10T21:29:39.563673+0000","last_became_peered":"2024-09-10T21:29:39.563673+0000","last_unstale":"2024-09-10T21:29:40.546185+0000","last_undegraded":"2024-09-10T21:29:40.546185+0000","last_fullsized":"2024-09-10T21:29:40.546185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:33.785842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544217+0000","last_change":"2024-09-10T21:29:39.552730+0000","last_active":"2024-09-10T21:29:40.544217+0000","last_peered":"2024-09-10T21:29:40.544217+0000","last_clean":"2024-09-10T21:29:40.544217+0000","last_became_active":"2024-09-10T21:29:39.552478+0000","last_became_peered":"2024-09-10T21:29:39.552478+0000","last_unstale":"2024-09-10T21:29:40.544217+0000","last_undegraded":"2024-09-10T21:29:40.544217+0000","last_fullsized":"2024-09-10T21:29:40.544217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:00.008953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769429+0000","last_change":"2024-09-10T21:29:39.558608+0000","last_active":"2024-09-10T21:29:40.769429+0000","last_peered":"2024-09-10T21:29:40.769429+0000","last_clean":"2024-09-10T21:29:40.769429+0000","last_became_active":"2024-09-10T21:29:39.558086+0000","last_became_peered":"2024-09-10T21:29:39.558086+0000","last_unstale":"2024-09-10T21:29:40.769429+0000","last_undegraded":"2024-09-10T21:29:40.769429+0000","last_fullsized":"2024-09-10T21:29:40.769429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:51.432263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546462+0000","last_change":"2024-09-10T21:29:39.560153+0000","last_active":"2024-09-10T21:29:40.546462+0000","last_peered":"2024-09-10T21:29:40.546462+0000","last_clean":"2024-09-10T21:29:40.546462+0000","last_became_active":"2024-09-10T21:29:39.559886+0000","last_became_peered":"2024-09-10T21:29:39.559886+0000","last_unstale":"2024-09-10T21:29:40.546462+0000","last_undegraded":"2024-09-10T21:29:40.546462+0000","last_fullsized":"2024-09-10T21:29:40.546462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768980+0000","last_change":"2024-09-10T21:29:39.547565+0000","last_active":"2024-09-10T21:29:40.768980+0000","last_peered":"2024-09-10T21:29:40.768980+0000","last_clean":"2024-09-10T21:29:40.768980+0000","last_became_active":"2024-09-10T21:29:39.547315+0000","last_became_peered":"2024-09-10T21:29:39.547315+0000","last_unstale":"2024-09-10T21:29:40.768980+0000","last_undegraded":"2024-09-10T21:29:40.768980+0000","last_fullsized":"2024-09-10T21:29:40.768980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:05.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769033+0000","last_change":"2024-09-10T21:29:39.562781+0000","last_active":"2024-09-10T21:29:40.769033+0000","last_peered":"2024-09-10T21:29:40.769033+0000","last_clean":"2024-09-10T21:29:40.769033+0000","last_became_active":"2024-09-10T21:29:39.562509+0000","last_became_peered":"2024-09-10T21:29:39.562509+0000","last_unstale":"2024-09-10T21:29:40.769033+0000","last_undegraded":"2024-09-10T21:29:40.769033+0000","last_fullsized":"2024-09-10T21:29:40.769033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:53.170104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768742+0000","last_change":"2024-09-10T21:29:39.540989+0000","last_active":"2024-09-10T21:29:40.768742+0000","last_peered":"2024-09-10T21:29:40.768742+0000","last_clean":"2024-09-10T21:29:40.768742+0000","last_became_active":"2024-09-10T21:29:39.540686+0000","last_became_peered":"2024-09-10T21:29:39.540686+0000","last_unstale":"2024-09-10T21:29:40.768742+0000","last_undegraded":"2024-09-10T21:29:40.768742+0000","last_fullsized":"2024-09-10T21:29:40.768742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:04.647939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768432+0000","last_change":"2024-09-10T21:29:39.555697+0000","last_active":"2024-09-10T21:29:40.768432+0000","last_peered":"2024-09-10T21:29:40.768432+0000","last_clean":"2024-09-10T21:29:40.768432+0000","last_became_active":"2024-09-10T21:29:39.555321+0000","last_became_peered":"2024-09-10T21:29:39.555321+0000","last_unstale":"2024-09-10T21:29:40.768432+0000","last_undegraded":"2024-09-10T21:29:40.768432+0000","last_fullsized":"2024-09-10T21:29:40.768432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:26.937688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547452+0000","last_change":"2024-09-10T21:29:39.566354+0000","last_active":"2024-09-10T21:29:40.547452+0000","last_peered":"2024-09-10T21:29:40.547452+0000","last_clean":"2024-09-10T21:29:40.547452+0000","last_became_active":"2024-09-10T21:29:39.566119+0000","last_became_peered":"2024-09-10T21:29:39.566119+0000","last_unstale":"2024-09-10T21:29:40.547452+0000","last_undegraded":"2024-09-10T21:29:40.547452+0000","last_fullsized":"2024-09-10T21:29:40.547452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544876+0000","last_change":"2024-09-10T21:29:39.555058+0000","last_active":"2024-09-10T21:29:40.544876+0000","last_peered":"2024-09-10T21:29:40.544876+0000","last_clean":"2024-09-10T21:29:40.544876+0000","last_became_active":"2024-09-10T21:29:39.554444+0000","last_became_peered":"2024-09-10T21:29:39.554444+0000","last_unstale":"2024-09-10T21:29:40.544876+0000","last_undegraded":"2024-09-10T21:29:40.544876+0000","last_fullsized":"2024-09-10T21:29:40.544876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544584+0000","last_change":"2024-09-10T21:29:39.553456+0000","last_active":"2024-09-10T21:29:40.544584+0000","last_peered":"2024-09-10T21:29:40.544584+0000","last_clean":"2024-09-10T21:29:40.544584+0000","last_became_active":"2024-09-10T21:29:39.553218+0000","last_became_peered":"2024-09-10T21:29:39.553218+0000","last_unstale":"2024-09-10T21:29:40.544584+0000","last_undegraded":"2024-09-10T21:29:40.544584+0000","last_fullsized":"2024-09-10T21:29:40.544584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547497+0000","last_change":"2024-09-10T21:29:39.551245+0000","last_active":"2024-09-10T21:29:40.547497+0000","last_peered":"2024-09-10T21:29:40.547497+0000","last_clean":"2024-09-10T21:29:40.547497+0000","last_became_active":"2024-09-10T21:29:39.550535+0000","last_became_peered":"2024-09-10T21:29:39.550535+0000","last_unstale":"2024-09-10T21:29:40.547497+0000","last_undegraded":"2024-09-10T21:29:40.547497+0000","last_fullsized":"2024-09-10T21:29:40.547497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:03.284635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545912+0000","last_change":"2024-09-10T21:29:39.558777+0000","last_active":"2024-09-10T21:29:40.545912+0000","last_peered":"2024-09-10T21:29:40.545912+0000","last_clean":"2024-09-10T21:29:40.545912+0000","last_became_active":"2024-09-10T21:29:39.555098+0000","last_became_peered":"2024-09-10T21:29:39.555098+0000","last_unstale":"2024-09-10T21:29:40.545912+0000","last_undegraded":"2024-09-10T21:29:40.545912+0000","last_fullsized":"2024-09-10T21:29:40.545912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769096+0000","last_change":"2024-09-10T21:29:39.564203+0000","last_active":"2024-09-10T21:29:40.769096+0000","last_peered":"2024-09-10T21:29:40.769096+0000","last_clean":"2024-09-10T21:29:40.769096+0000","last_became_active":"2024-09-10T21:29:39.563951+0000","last_became_peered":"2024-09-10T21:29:39.563951+0000","last_unstale":"2024-09-10T21:29:40.769096+0000","last_undegraded":"2024-09-10T21:29:40.769096+0000","last_fullsized":"2024-09-10T21:29:40.769096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:38.678603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768992+0000","last_change":"2024-09-10T21:29:39.564492+0000","last_active":"2024-09-10T21:29:40.768992+0000","last_peered":"2024-09-10T21:29:40.768992+0000","last_clean":"2024-09-10T21:29:40.768992+0000","last_became_active":"2024-09-10T21:29:39.563500+0000","last_became_peered":"2024-09-10T21:29:39.563500+0000","last_unstale":"2024-09-10T21:29:40.768992+0000","last_undegraded":"2024-09-10T21:29:40.768992+0000","last_fullsized":"2024-09-10T21:29:40.768992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:46.030917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544257+0000","last_change":"2024-09-10T21:29:39.555396+0000","last_active":"2024-09-10T21:29:40.544257+0000","last_peered":"2024-09-10T21:29:40.544257+0000","last_clean":"2024-09-10T21:29:40.544257+0000","last_became_active":"2024-09-10T21:29:39.555086+0000","last_became_peered":"2024-09-10T21:29:39.555086+0000","last_unstale":"2024-09-10T21:29:40.544257+0000","last_undegraded":"2024-09-10T21:29:40.544257+0000","last_fullsized":"2024-09-10T21:29:40.544257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:06.000345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546200+0000","last_change":"2024-09-10T21:29:39.548302+0000","last_active":"2024-09-10T21:29:40.546200+0000","last_peered":"2024-09-10T21:29:40.546200+0000","last_clean":"2024-09-10T21:29:40.546200+0000","last_became_active":"2024-09-10T21:29:39.547972+0000","last_became_peered":"2024-09-10T21:29:39.547972+0000","last_unstale":"2024-09-10T21:29:40.546200+0000","last_undegraded":"2024-09-10T21:29:40.546200+0000","last_fullsized":"2024-09-10T21:29:40.546200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:22.457526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547580+0000","last_change":"2024-09-10T21:29:38.549320+0000","last_active":"2024-09-10T21:29:40.547580+0000","last_peered":"2024-09-10T21:29:40.547580+0000","last_clean":"2024-09-10T21:29:40.547580+0000","last_became_active":"2024-09-10T21:29:38.549041+0000","last_became_peered":"2024-09-10T21:29:38.549041+0000","last_unstale":"2024-09-10T21:29:40.547580+0000","last_undegraded":"2024-09-10T21:29:40.547580+0000","last_fullsized":"2024-09-10T21:29:40.547580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:52.457273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546222+0000","last_change":"2024-09-10T21:29:39.542269+0000","last_active":"2024-09-10T21:29:40.546222+0000","last_peered":"2024-09-10T21:29:40.546222+0000","last_clean":"2024-09-10T21:29:40.546222+0000","last_became_active":"2024-09-10T21:29:39.541975+0000","last_became_peered":"2024-09-10T21:29:39.541975+0000","last_unstale":"2024-09-10T21:29:40.546222+0000","last_undegraded":"2024-09-10T21:29:40.546222+0000","last_fullsized":"2024-09-10T21:29:40.546222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:38.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769882+0000","last_change":"2024-09-10T21:29:38.544792+0000","last_active":"2024-09-10T21:29:40.769882+0000","last_peered":"2024-09-10T21:29:40.769882+0000","last_clean":"2024-09-10T21:29:40.769882+0000","last_became_active":"2024-09-10T21:29:38.544459+0000","last_became_peered":"2024-09-10T21:29:38.544459+0000","last_unstale":"2024-09-10T21:29:40.769882+0000","last_undegraded":"2024-09-10T21:29:40.769882+0000","last_fullsized":"2024-09-10T21:29:40.769882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:28.938255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769152+0000","last_change":"2024-09-10T21:29:39.546560+0000","last_active":"2024-09-10T21:29:40.769152+0000","last_peered":"2024-09-10T21:29:40.769152+0000","last_clean":"2024-09-10T21:29:40.769152+0000","last_became_active":"2024-09-10T21:29:39.545328+0000","last_became_peered":"2024-09-10T21:29:39.545328+0000","last_unstale":"2024-09-10T21:29:40.769152+0000","last_undegraded":"2024-09-10T21:29:40.769152+0000","last_fullsized":"2024-09-10T21:29:40.769152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:20.557881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768895+0000","last_change":"2024-09-10T21:29:38.538896+0000","last_active":"2024-09-10T21:29:40.768895+0000","last_peered":"2024-09-10T21:29:40.768895+0000","last_clean":"2024-09-10T21:29:40.768895+0000","last_became_active":"2024-09-10T21:29:38.538523+0000","last_became_peered":"2024-09-10T21:29:38.538523+0000","last_unstale":"2024-09-10T21:29:40.768895+0000","last_undegraded":"2024-09-10T21:29:40.768895+0000","last_fullsized":"2024-09-10T21:29:40.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:26.307429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544519+0000","last_change":"2024-09-10T21:29:39.538536+0000","last_active":"2024-09-10T21:29:40.544519+0000","last_peered":"2024-09-10T21:29:40.544519+0000","last_clean":"2024-09-10T21:29:40.544519+0000","last_became_active":"2024-09-10T21:29:39.538308+0000","last_became_peered":"2024-09-10T21:29:39.538308+0000","last_unstale":"2024-09-10T21:29:40.544519+0000","last_undegraded":"2024-09-10T21:29:40.544519+0000","last_fullsized":"2024-09-10T21:29:40.544519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:40.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769616+0000","last_change":"2024-09-10T21:29:38.536197+0000","last_active":"2024-09-10T21:29:40.769616+0000","last_peered":"2024-09-10T21:29:40.769616+0000","last_clean":"2024-09-10T21:29:40.769616+0000","last_became_active":"2024-09-10T21:29:38.535894+0000","last_became_peered":"2024-09-10T21:29:38.535894+0000","last_unstale":"2024-09-10T21:29:40.769616+0000","last_undegraded":"2024-09-10T21:29:40.769616+0000","last_fullsized":"2024-09-10T21:29:40.769616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546646+0000","last_change":"2024-09-10T21:29:39.554137+0000","last_active":"2024-09-10T21:29:40.546646+0000","last_peered":"2024-09-10T21:29:40.546646+0000","last_clean":"2024-09-10T21:29:40.546646+0000","last_became_active":"2024-09-10T21:29:39.553735+0000","last_became_peered":"2024-09-10T21:29:39.553735+0000","last_unstale":"2024-09-10T21:29:40.546646+0000","last_undegraded":"2024-09-10T21:29:40.546646+0000","last_fullsized":"2024-09-10T21:29:40.546646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769158+0000","last_change":"2024-09-10T21:29:38.535949+0000","last_active":"2024-09-10T21:29:40.769158+0000","last_peered":"2024-09-10T21:29:40.769158+0000","last_clean":"2024-09-10T21:29:40.769158+0000","last_became_active":"2024-09-10T21:29:38.535669+0000","last_became_peered":"2024-09-10T21:29:38.535669+0000","last_unstale":"2024-09-10T21:29:40.769158+0000","last_undegraded":"2024-09-10T21:29:40.769158+0000","last_fullsized":"2024-09-10T21:29:40.769158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769743+0000","last_change":"2024-09-10T21:29:39.561954+0000","last_active":"2024-09-10T21:29:40.769743+0000","last_peered":"2024-09-10T21:29:40.769743+0000","last_clean":"2024-09-10T21:29:40.769743+0000","last_became_active":"2024-09-10T21:29:39.561348+0000","last_became_peered":"2024-09-10T21:29:39.561348+0000","last_unstale":"2024-09-10T21:29:40.769743+0000","last_undegraded":"2024-09-10T21:29:40.769743+0000","last_fullsized":"2024-09-10T21:29:40.769743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:54.689918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769497+0000","last_change":"2024-09-10T21:29:38.543780+0000","last_active":"2024-09-10T21:29:40.769497+0000","last_peered":"2024-09-10T21:29:40.769497+0000","last_clean":"2024-09-10T21:29:40.769497+0000","last_became_active":"2024-09-10T21:29:38.543305+0000","last_became_peered":"2024-09-10T21:29:38.543305+0000","last_unstale":"2024-09-10T21:29:40.769497+0000","last_undegraded":"2024-09-10T21:29:40.769497+0000","last_fullsized":"2024-09-10T21:29:40.769497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.941433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768491+0000","last_change":"2024-09-10T21:29:39.549779+0000","last_active":"2024-09-10T21:29:40.768491+0000","last_peered":"2024-09-10T21:29:40.768491+0000","last_clean":"2024-09-10T21:29:40.768491+0000","last_became_active":"2024-09-10T21:29:39.549473+0000","last_became_peered":"2024-09-10T21:29:39.549473+0000","last_unstale":"2024-09-10T21:29:40.768491+0000","last_undegraded":"2024-09-10T21:29:40.768491+0000","last_fullsized":"2024-09-10T21:29:40.768491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":79,"state":"creating+peering","last_fresh":"2024-09-10T21:29:37.517765+0000","last_change":"2024-09-10T21:29:37.517765+0000","last_active":"2024-09-10T21:29:37.510514+0000","last_peered":"2024-09-10T21:29:37.510514+0000","last_clean":"2024-09-10T21:29:37.510514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:37.517765+0000","last_undegraded":"2024-09-10T21:29:37.517765+0000","last_fullsized":"2024-09-10T21:29:37.517765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769992+0000","last_change":"2024-09-10T21:29:39.558263+0000","last_active":"2024-09-10T21:29:40.769992+0000","last_peered":"2024-09-10T21:29:40.769992+0000","last_clean":"2024-09-10T21:29:40.769992+0000","last_became_active":"2024-09-10T21:29:39.557990+0000","last_became_peered":"2024-09-10T21:29:39.557990+0000","last_unstale":"2024-09-10T21:29:40.769992+0000","last_undegraded":"2024-09-10T21:29:40.769992+0000","last_fullsized":"2024-09-10T21:29:40.769992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.511263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768565+0000","last_change":"2024-09-10T21:29:38.545254+0000","last_active":"2024-09-10T21:29:40.768565+0000","last_peered":"2024-09-10T21:29:40.768565+0000","last_clean":"2024-09-10T21:29:40.768565+0000","last_became_active":"2024-09-10T21:29:38.544954+0000","last_became_peered":"2024-09-10T21:29:38.544954+0000","last_unstale":"2024-09-10T21:29:40.768565+0000","last_undegraded":"2024-09-10T21:29:40.768565+0000","last_fullsized":"2024-09-10T21:29:40.768565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768528+0000","last_change":"2024-09-10T21:29:39.555635+0000","last_active":"2024-09-10T21:29:40.768528+0000","last_peered":"2024-09-10T21:29:40.768528+0000","last_clean":"2024-09-10T21:29:40.768528+0000","last_became_active":"2024-09-10T21:29:39.555011+0000","last_became_peered":"2024-09-10T21:29:39.555011+0000","last_unstale":"2024-09-10T21:29:40.768528+0000","last_undegraded":"2024-09-10T21:29:40.768528+0000","last_fullsized":"2024-09-10T21:29:40.768528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:15.429757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545012+0000","last_change":"2024-09-10T21:29:38.549092+0000","last_active":"2024-09-10T21:29:40.545012+0000","last_peered":"2024-09-10T21:29:40.545012+0000","last_clean":"2024-09-10T21:29:40.545012+0000","last_became_active":"2024-09-10T21:29:38.548800+0000","last_became_peered":"2024-09-10T21:29:38.548800+0000","last_unstale":"2024-09-10T21:29:40.545012+0000","last_undegraded":"2024-09-10T21:29:40.545012+0000","last_fullsized":"2024-09-10T21:29:40.545012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:10.710966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768928+0000","last_change":"2024-09-10T21:29:39.554166+0000","last_active":"2024-09-10T21:29:40.768928+0000","last_peered":"2024-09-10T21:29:40.768928+0000","last_clean":"2024-09-10T21:29:40.768928+0000","last_became_active":"2024-09-10T21:29:39.551856+0000","last_became_peered":"2024-09-10T21:29:39.551856+0000","last_unstale":"2024-09-10T21:29:40.768928+0000","last_undegraded":"2024-09-10T21:29:40.768928+0000","last_fullsized":"2024-09-10T21:29:40.768928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.611740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769311+0000","last_change":"2024-09-10T21:29:38.545177+0000","last_active":"2024-09-10T21:29:40.769311+0000","last_peered":"2024-09-10T21:29:40.769311+0000","last_clean":"2024-09-10T21:29:40.769311+0000","last_became_active":"2024-09-10T21:29:38.544523+0000","last_became_peered":"2024-09-10T21:29:38.544523+0000","last_unstale":"2024-09-10T21:29:40.769311+0000","last_undegraded":"2024-09-10T21:29:40.769311+0000","last_fullsized":"2024-09-10T21:29:40.769311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:36.904722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546787+0000","last_change":"2024-09-10T21:29:38.542661+0000","last_active":"2024-09-10T21:29:40.546787+0000","last_peered":"2024-09-10T21:29:40.546787+0000","last_clean":"2024-09-10T21:29:40.546787+0000","last_became_active":"2024-09-10T21:29:38.542425+0000","last_became_peered":"2024-09-10T21:29:38.542425+0000","last_unstale":"2024-09-10T21:29:40.546787+0000","last_undegraded":"2024-09-10T21:29:40.546787+0000","last_fullsized":"2024-09-10T21:29:40.546787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:47.295559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545841+0000","last_change":"2024-09-10T21:29:39.548032+0000","last_active":"2024-09-10T21:29:40.545841+0000","last_peered":"2024-09-10T21:29:40.545841+0000","last_clean":"2024-09-10T21:29:40.545841+0000","last_became_active":"2024-09-10T21:29:39.547709+0000","last_became_peered":"2024-09-10T21:29:39.547709+0000","last_unstale":"2024-09-10T21:29:40.545841+0000","last_undegraded":"2024-09-10T21:29:40.545841+0000","last_fullsized":"2024-09-10T21:29:40.545841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769782+0000","last_change":"2024-09-10T21:29:38.532633+0000","last_active":"2024-09-10T21:29:40.769782+0000","last_peered":"2024-09-10T21:29:40.769782+0000","last_clean":"2024-09-10T21:29:40.769782+0000","last_became_active":"2024-09-10T21:29:38.532371+0000","last_became_peered":"2024-09-10T21:29:38.532371+0000","last_unstale":"2024-09-10T21:29:40.769782+0000","last_undegraded":"2024-09-10T21:29:40.769782+0000","last_fullsized":"2024-09-10T21:29:40.769782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:45.933789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544716+0000","last_change":"2024-09-10T21:29:39.554909+0000","last_active":"2024-09-10T21:29:40.544716+0000","last_peered":"2024-09-10T21:29:40.544716+0000","last_clean":"2024-09-10T21:29:40.544716+0000","last_became_active":"2024-09-10T21:29:39.554651+0000","last_became_peered":"2024-09-10T21:29:39.554651+0000","last_unstale":"2024-09-10T21:29:40.544716+0000","last_undegraded":"2024-09-10T21:29:40.544716+0000","last_fullsized":"2024-09-10T21:29:40.544716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768411+0000","last_change":"2024-09-10T21:29:38.535431+0000","last_active":"2024-09-10T21:29:40.768411+0000","last_peered":"2024-09-10T21:29:40.768411+0000","last_clean":"2024-09-10T21:29:40.768411+0000","last_became_active":"2024-09-10T21:29:38.535114+0000","last_became_peered":"2024-09-10T21:29:38.535114+0000","last_unstale":"2024-09-10T21:29:40.768411+0000","last_undegraded":"2024-09-10T21:29:40.768411+0000","last_fullsized":"2024-09-10T21:29:40.768411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:05.208145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768791+0000","last_change":"2024-09-10T21:29:39.563604+0000","last_active":"2024-09-10T21:29:40.768791+0000","last_peered":"2024-09-10T21:29:40.768791+0000","last_clean":"2024-09-10T21:29:40.768791+0000","last_became_active":"2024-09-10T21:29:39.563344+0000","last_became_peered":"2024-09-10T21:29:39.563344+0000","last_unstale":"2024-09-10T21:29:40.768791+0000","last_undegraded":"2024-09-10T21:29:40.768791+0000","last_fullsized":"2024-09-10T21:29:40.768791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:50.487501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769250+0000","last_change":"2024-09-10T21:29:38.544304+0000","last_active":"2024-09-10T21:29:40.769250+0000","last_peered":"2024-09-10T21:29:40.769250+0000","last_clean":"2024-09-10T21:29:40.769250+0000","last_became_active":"2024-09-10T21:29:38.544049+0000","last_became_peered":"2024-09-10T21:29:38.544049+0000","last_unstale":"2024-09-10T21:29:40.769250+0000","last_undegraded":"2024-09-10T21:29:40.769250+0000","last_fullsized":"2024-09-10T21:29:40.769250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768692+0000","last_change":"2024-09-10T21:29:39.544825+0000","last_active":"2024-09-10T21:29:40.768692+0000","last_peered":"2024-09-10T21:29:40.768692+0000","last_clean":"2024-09-10T21:29:40.768692+0000","last_became_active":"2024-09-10T21:29:39.544578+0000","last_became_peered":"2024-09-10T21:29:39.544578+0000","last_unstale":"2024-09-10T21:29:40.768692+0000","last_undegraded":"2024-09-10T21:29:40.768692+0000","last_fullsized":"2024-09-10T21:29:40.768692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:17.043812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546585+0000","last_change":"2024-09-10T21:29:38.549765+0000","last_active":"2024-09-10T21:29:40.546585+0000","last_peered":"2024-09-10T21:29:40.546585+0000","last_clean":"2024-09-10T21:29:40.546585+0000","last_became_active":"2024-09-10T21:29:38.548990+0000","last_became_peered":"2024-09-10T21:29:38.548990+0000","last_unstale":"2024-09-10T21:29:40.546585+0000","last_undegraded":"2024-09-10T21:29:40.546585+0000","last_fullsized":"2024-09-10T21:29:40.546585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:52.559734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768695+0000","last_change":"2024-09-10T21:29:39.547930+0000","last_active":"2024-09-10T21:29:40.768695+0000","last_peered":"2024-09-10T21:29:40.768695+0000","last_clean":"2024-09-10T21:29:40.768695+0000","last_became_active":"2024-09-10T21:29:39.547672+0000","last_became_peered":"2024-09-10T21:29:39.547672+0000","last_unstale":"2024-09-10T21:29:40.768695+0000","last_undegraded":"2024-09-10T21:29:40.768695+0000","last_fullsized":"2024-09-10T21:29:40.768695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:52.513550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545358+0000","last_change":"2024-09-10T21:29:38.544208+0000","last_active":"2024-09-10T21:29:40.545358+0000","last_peered":"2024-09-10T21:29:40.545358+0000","last_clean":"2024-09-10T21:29:40.545358+0000","last_became_active":"2024-09-10T21:29:38.543907+0000","last_became_peered":"2024-09-10T21:29:38.543907+0000","last_unstale":"2024-09-10T21:29:40.545358+0000","last_undegraded":"2024-09-10T21:29:40.545358+0000","last_fullsized":"2024-09-10T21:29:40.545358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:37.225392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546397+0000","last_change":"2024-09-10T21:29:39.557707+0000","last_active":"2024-09-10T21:29:40.546397+0000","last_peered":"2024-09-10T21:29:40.546397+0000","last_clean":"2024-09-10T21:29:40.546397+0000","last_became_active":"2024-09-10T21:29:39.557398+0000","last_became_peered":"2024-09-10T21:29:39.557398+0000","last_unstale":"2024-09-10T21:29:40.546397+0000","last_undegraded":"2024-09-10T21:29:40.546397+0000","last_fullsized":"2024-09-10T21:29:40.546397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:05.442260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545105+0000","last_change":"2024-09-10T21:29:38.549044+0000","last_active":"2024-09-10T21:29:40.545105+0000","last_peered":"2024-09-10T21:29:40.545105+0000","last_clean":"2024-09-10T21:29:40.545105+0000","last_became_active":"2024-09-10T21:29:38.548465+0000","last_became_peered":"2024-09-10T21:29:38.548465+0000","last_unstale":"2024-09-10T21:29:40.545105+0000","last_undegraded":"2024-09-10T21:29:40.545105+0000","last_fullsized":"2024-09-10T21:29:40.545105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:47.729570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769221+0000","last_change":"2024-09-10T21:29:39.555291+0000","last_active":"2024-09-10T21:29:40.769221+0000","last_peered":"2024-09-10T21:29:40.769221+0000","last_clean":"2024-09-10T21:29:40.769221+0000","last_became_active":"2024-09-10T21:29:39.554979+0000","last_became_peered":"2024-09-10T21:29:39.554979+0000","last_unstale":"2024-09-10T21:29:40.769221+0000","last_undegraded":"2024-09-10T21:29:40.769221+0000","last_fullsized":"2024-09-10T21:29:40.769221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:41.126386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544896+0000","last_change":"2024-09-10T21:29:38.545682+0000","last_active":"2024-09-10T21:29:40.544896+0000","last_peered":"2024-09-10T21:29:40.544896+0000","last_clean":"2024-09-10T21:29:40.544896+0000","last_became_active":"2024-09-10T21:29:38.545432+0000","last_became_peered":"2024-09-10T21:29:38.545432+0000","last_unstale":"2024-09-10T21:29:40.544896+0000","last_undegraded":"2024-09-10T21:29:40.544896+0000","last_fullsized":"2024-09-10T21:29:40.544896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546758+0000","last_change":"2024-09-10T21:29:38.550165+0000","last_active":"2024-09-10T21:29:40.546758+0000","last_peered":"2024-09-10T21:29:40.546758+0000","last_clean":"2024-09-10T21:29:40.546758+0000","last_became_active":"2024-09-10T21:29:38.549861+0000","last_became_peered":"2024-09-10T21:29:38.549861+0000","last_unstale":"2024-09-10T21:29:40.546758+0000","last_undegraded":"2024-09-10T21:29:40.546758+0000","last_fullsized":"2024-09-10T21:29:40.546758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:02.189449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547170+0000","last_change":"2024-09-10T21:29:39.545985+0000","last_active":"2024-09-10T21:29:40.547170+0000","last_peered":"2024-09-10T21:29:40.547170+0000","last_clean":"2024-09-10T21:29:40.547170+0000","last_became_active":"2024-09-10T21:29:39.545349+0000","last_became_peered":"2024-09-10T21:29:39.545349+0000","last_unstale":"2024-09-10T21:29:40.547170+0000","last_undegraded":"2024-09-10T21:29:40.547170+0000","last_fullsized":"2024-09-10T21:29:40.547170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:14.688355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547653+0000","last_change":"2024-09-10T21:29:38.546910+0000","last_active":"2024-09-10T21:29:40.547653+0000","last_peered":"2024-09-10T21:29:40.547653+0000","last_clean":"2024-09-10T21:29:40.547653+0000","last_became_active":"2024-09-10T21:29:38.546159+0000","last_became_peered":"2024-09-10T21:29:38.546159+0000","last_unstale":"2024-09-10T21:29:40.547653+0000","last_undegraded":"2024-09-10T21:29:40.547653+0000","last_fullsized":"2024-09-10T21:29:40.547653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.324328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547733+0000","last_change":"2024-09-10T21:29:39.551802+0000","last_active":"2024-09-10T21:29:40.547733+0000","last_peered":"2024-09-10T21:29:40.547733+0000","last_clean":"2024-09-10T21:29:40.547733+0000","last_became_active":"2024-09-10T21:29:39.550894+0000","last_became_peered":"2024-09-10T21:29:39.550894+0000","last_unstale":"2024-09-10T21:29:40.547733+0000","last_undegraded":"2024-09-10T21:29:40.547733+0000","last_fullsized":"2024-09-10T21:29:40.547733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:35.976956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768722+0000","last_change":"2024-09-10T21:29:38.549752+0000","last_active":"2024-09-10T21:29:40.768722+0000","last_peered":"2024-09-10T21:29:40.768722+0000","last_clean":"2024-09-10T21:29:40.768722+0000","last_became_active":"2024-09-10T21:29:38.549380+0000","last_became_peered":"2024-09-10T21:29:38.549380+0000","last_unstale":"2024-09-10T21:29:40.768722+0000","last_undegraded":"2024-09-10T21:29:40.768722+0000","last_fullsized":"2024-09-10T21:29:40.768722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547195+0000","last_change":"2024-09-10T21:29:39.544277+0000","last_active":"2024-09-10T21:29:40.547195+0000","last_peered":"2024-09-10T21:29:40.547195+0000","last_clean":"2024-09-10T21:29:40.547195+0000","last_became_active":"2024-09-10T21:29:39.544011+0000","last_became_peered":"2024-09-10T21:29:39.544011+0000","last_unstale":"2024-09-10T21:29:40.547195+0000","last_undegraded":"2024-09-10T21:29:40.547195+0000","last_fullsized":"2024-09-10T21:29:40.547195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:05.999146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":79,"state":"creating+peering","last_fresh":"2024-09-10T21:29:37.517236+0000","last_change":"2024-09-10T21:29:37.517236+0000","last_active":"2024-09-10T21:29:37.510514+0000","last_peered":"2024-09-10T21:29:37.510514+0000","last_clean":"2024-09-10T21:29:37.510514+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:37.517236+0000","last_undegraded":"2024-09-10T21:29:37.517236+0000","last_fullsized":"2024-09-10T21:29:37.517236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.770023+0000","last_change":"2024-09-10T21:29:39.545983+0000","last_active":"2024-09-10T21:29:40.770023+0000","last_peered":"2024-09-10T21:29:40.770023+0000","last_clean":"2024-09-10T21:29:40.770023+0000","last_became_active":"2024-09-10T21:29:39.545699+0000","last_became_peered":"2024-09-10T21:29:39.545699+0000","last_unstale":"2024-09-10T21:29:40.770023+0000","last_undegraded":"2024-09-10T21:29:40.770023+0000","last_fullsized":"2024-09-10T21:29:40.770023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.110951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768649+0000","last_change":"2024-09-10T21:29:38.546518+0000","last_active":"2024-09-10T21:29:40.768649+0000","last_peered":"2024-09-10T21:29:40.768649+0000","last_clean":"2024-09-10T21:29:40.768649+0000","last_became_active":"2024-09-10T21:29:38.546262+0000","last_became_peered":"2024-09-10T21:29:38.546262+0000","last_unstale":"2024-09-10T21:29:40.768649+0000","last_undegraded":"2024-09-10T21:29:40.768649+0000","last_fullsized":"2024-09-10T21:29:40.768649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544409+0000","last_change":"2024-09-10T21:29:39.554112+0000","last_active":"2024-09-10T21:29:40.544409+0000","last_peered":"2024-09-10T21:29:40.544409+0000","last_clean":"2024-09-10T21:29:40.544409+0000","last_became_active":"2024-09-10T21:29:39.552077+0000","last_became_peered":"2024-09-10T21:29:39.552077+0000","last_unstale":"2024-09-10T21:29:40.544409+0000","last_undegraded":"2024-09-10T21:29:40.544409+0000","last_fullsized":"2024-09-10T21:29:40.544409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:18.545317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769044+0000","last_change":"2024-09-10T21:29:38.544002+0000","last_active":"2024-09-10T21:29:40.769044+0000","last_peered":"2024-09-10T21:29:40.769044+0000","last_clean":"2024-09-10T21:29:40.769044+0000","last_became_active":"2024-09-10T21:29:38.543699+0000","last_became_peered":"2024-09-10T21:29:38.543699+0000","last_unstale":"2024-09-10T21:29:40.769044+0000","last_undegraded":"2024-09-10T21:29:40.769044+0000","last_fullsized":"2024-09-10T21:29:40.769044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:21.454494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768948+0000","last_change":"2024-09-10T21:29:39.558189+0000","last_active":"2024-09-10T21:29:40.768948+0000","last_peered":"2024-09-10T21:29:40.768948+0000","last_clean":"2024-09-10T21:29:40.768948+0000","last_became_active":"2024-09-10T21:29:39.557549+0000","last_became_peered":"2024-09-10T21:29:39.557549+0000","last_unstale":"2024-09-10T21:29:40.768948+0000","last_undegraded":"2024-09-10T21:29:40.768948+0000","last_fullsized":"2024-09-10T21:29:40.768948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:48.581722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547028+0000","last_change":"2024-09-10T21:29:38.544536+0000","last_active":"2024-09-10T21:29:40.547028+0000","last_peered":"2024-09-10T21:29:40.547028+0000","last_clean":"2024-09-10T21:29:40.547028+0000","last_became_active":"2024-09-10T21:29:38.544212+0000","last_became_peered":"2024-09-10T21:29:38.544212+0000","last_unstale":"2024-09-10T21:29:40.547028+0000","last_undegraded":"2024-09-10T21:29:40.547028+0000","last_fullsized":"2024-09-10T21:29:40.547028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:53.514589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769300+0000","last_change":"2024-09-10T21:29:39.546810+0000","last_active":"2024-09-10T21:29:40.769300+0000","last_peered":"2024-09-10T21:29:40.769300+0000","last_clean":"2024-09-10T21:29:40.769300+0000","last_became_active":"2024-09-10T21:29:39.546317+0000","last_became_peered":"2024-09-10T21:29:39.546317+0000","last_unstale":"2024-09-10T21:29:40.769300+0000","last_undegraded":"2024-09-10T21:29:40.769300+0000","last_fullsized":"2024-09-10T21:29:40.769300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:46.343210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768987+0000","last_change":"2024-09-10T21:29:38.533647+0000","last_active":"2024-09-10T21:29:40.768987+0000","last_peered":"2024-09-10T21:29:40.768987+0000","last_clean":"2024-09-10T21:29:40.768987+0000","last_became_active":"2024-09-10T21:29:38.533361+0000","last_became_peered":"2024-09-10T21:29:38.533361+0000","last_unstale":"2024-09-10T21:29:40.768987+0000","last_undegraded":"2024-09-10T21:29:40.768987+0000","last_fullsized":"2024-09-10T21:29:40.768987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:32.782811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:29:38.513546+0000","last_change":"2024-09-10T21:29:38.513546+0000","last_active":"2024-09-10T21:29:38.513546+0000","last_peered":"2024-09-10T21:29:38.513546+0000","last_clean":"2024-09-10T21:29:38.513546+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:29:38.513546+0000","last_undegraded":"2024-09-10T21:29:38.513546+0000","last_fullsized":"2024-09-10T21:29:38.513546+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544676+0000","last_change":"2024-09-10T21:29:38.547087+0000","last_active":"2024-09-10T21:29:40.544676+0000","last_peered":"2024-09-10T21:29:40.544676+0000","last_clean":"2024-09-10T21:29:40.544676+0000","last_became_active":"2024-09-10T21:29:38.546829+0000","last_became_peered":"2024-09-10T21:29:38.546829+0000","last_unstale":"2024-09-10T21:29:40.544676+0000","last_undegraded":"2024-09-10T21:29:40.544676+0000","last_fullsized":"2024-09-10T21:29:40.544676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769511+0000","last_change":"2024-09-10T21:29:39.550392+0000","last_active":"2024-09-10T21:29:40.769511+0000","last_peered":"2024-09-10T21:29:40.769511+0000","last_clean":"2024-09-10T21:29:40.769511+0000","last_became_active":"2024-09-10T21:29:39.550131+0000","last_became_peered":"2024-09-10T21:29:39.550131+0000","last_unstale":"2024-09-10T21:29:40.769511+0000","last_undegraded":"2024-09-10T21:29:40.769511+0000","last_fullsized":"2024-09-10T21:29:40.769511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:15.885819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769553+0000","last_change":"2024-09-10T21:29:38.549285+0000","last_active":"2024-09-10T21:29:40.769553+0000","last_peered":"2024-09-10T21:29:40.769553+0000","last_clean":"2024-09-10T21:29:40.769553+0000","last_became_active":"2024-09-10T21:29:38.549014+0000","last_became_peered":"2024-09-10T21:29:38.549014+0000","last_unstale":"2024-09-10T21:29:40.769553+0000","last_undegraded":"2024-09-10T21:29:40.769553+0000","last_fullsized":"2024-09-10T21:29:40.769553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:58.754314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544681+0000","last_change":"2024-09-10T21:29:39.562581+0000","last_active":"2024-09-10T21:29:40.544681+0000","last_peered":"2024-09-10T21:29:40.544681+0000","last_clean":"2024-09-10T21:29:40.544681+0000","last_became_active":"2024-09-10T21:29:39.562287+0000","last_became_peered":"2024-09-10T21:29:39.562287+0000","last_unstale":"2024-09-10T21:29:40.544681+0000","last_undegraded":"2024-09-10T21:29:40.544681+0000","last_fullsized":"2024-09-10T21:29:40.544681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545978+0000","last_change":"2024-09-10T21:29:38.546345+0000","last_active":"2024-09-10T21:29:40.545978+0000","last_peered":"2024-09-10T21:29:40.545978+0000","last_clean":"2024-09-10T21:29:40.545978+0000","last_became_active":"2024-09-10T21:29:38.546094+0000","last_became_peered":"2024-09-10T21:29:38.546094+0000","last_unstale":"2024-09-10T21:29:40.545978+0000","last_undegraded":"2024-09-10T21:29:40.545978+0000","last_fullsized":"2024-09-10T21:29:40.545978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769644+0000","last_change":"2024-09-10T21:29:39.558347+0000","last_active":"2024-09-10T21:29:40.769644+0000","last_peered":"2024-09-10T21:29:40.769644+0000","last_clean":"2024-09-10T21:29:40.769644+0000","last_became_active":"2024-09-10T21:29:39.557760+0000","last_became_peered":"2024-09-10T21:29:39.557760+0000","last_unstale":"2024-09-10T21:29:40.769644+0000","last_undegraded":"2024-09-10T21:29:40.769644+0000","last_fullsized":"2024-09-10T21:29:40.769644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:06.975636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544238+0000","last_change":"2024-09-10T21:29:38.535077+0000","last_active":"2024-09-10T21:29:40.544238+0000","last_peered":"2024-09-10T21:29:40.544238+0000","last_clean":"2024-09-10T21:29:40.544238+0000","last_became_active":"2024-09-10T21:29:38.534747+0000","last_became_peered":"2024-09-10T21:29:38.534747+0000","last_unstale":"2024-09-10T21:29:40.544238+0000","last_undegraded":"2024-09-10T21:29:40.544238+0000","last_fullsized":"2024-09-10T21:29:40.544238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547454+0000","last_change":"2024-09-10T21:29:39.548356+0000","last_active":"2024-09-10T21:29:40.547454+0000","last_peered":"2024-09-10T21:29:40.547454+0000","last_clean":"2024-09-10T21:29:40.547454+0000","last_became_active":"2024-09-10T21:29:39.548096+0000","last_became_peered":"2024-09-10T21:29:39.548096+0000","last_unstale":"2024-09-10T21:29:40.547454+0000","last_undegraded":"2024-09-10T21:29:40.547454+0000","last_fullsized":"2024-09-10T21:29:40.547454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:50.836505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768672+0000","last_change":"2024-09-10T21:29:38.544063+0000","last_active":"2024-09-10T21:29:40.768672+0000","last_peered":"2024-09-10T21:29:40.768672+0000","last_clean":"2024-09-10T21:29:40.768672+0000","last_became_active":"2024-09-10T21:29:38.543508+0000","last_became_peered":"2024-09-10T21:29:38.543508+0000","last_unstale":"2024-09-10T21:29:40.768672+0000","last_undegraded":"2024-09-10T21:29:40.768672+0000","last_fullsized":"2024-09-10T21:29:40.768672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:02.927923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769972+0000","last_change":"2024-09-10T21:29:39.552135+0000","last_active":"2024-09-10T21:29:40.769972+0000","last_peered":"2024-09-10T21:29:40.769972+0000","last_clean":"2024-09-10T21:29:40.769972+0000","last_became_active":"2024-09-10T21:29:39.551649+0000","last_became_peered":"2024-09-10T21:29:39.551649+0000","last_unstale":"2024-09-10T21:29:40.769972+0000","last_undegraded":"2024-09-10T21:29:40.769972+0000","last_fullsized":"2024-09-10T21:29:40.769972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:26.827601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545785+0000","last_change":"2024-09-10T21:29:38.547451+0000","last_active":"2024-09-10T21:29:40.545785+0000","last_peered":"2024-09-10T21:29:40.545785+0000","last_clean":"2024-09-10T21:29:40.545785+0000","last_became_active":"2024-09-10T21:29:38.546360+0000","last_became_peered":"2024-09-10T21:29:38.546360+0000","last_unstale":"2024-09-10T21:29:40.545785+0000","last_undegraded":"2024-09-10T21:29:40.545785+0000","last_fullsized":"2024-09-10T21:29:40.545785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768220+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:29:40.768220+0000","last_peered":"2024-09-10T21:29:40.768220+0000","last_clean":"2024-09-10T21:29:40.768220+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:29:40.768220+0000","last_undegraded":"2024-09-10T21:29:40.768220+0000","last_fullsized":"2024-09-10T21:29:40.768220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":458807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":1048593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":453159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":458135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1605632,"data_stored":1050149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":958464,"data_stored":437609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:29:44.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:29:44.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 -- 172.21.15.97:0/2839999274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc29c005e60 msgr2=0x7fc29c0062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 --2- 172.21.15.97:0/2839999274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc29c005e60 0x7fc29c0062e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c640 tx=0x7fc2ac01d260 comp rx=0 tx=0).stop 2024-09-10T21:29:44.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2ba600640 1 -- 172.21.15.97:0/2839999274 reap_dead start 2024-09-10T21:29:44.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 -- 172.21.15.97:0/2839999274 shutdown_connections 2024-09-10T21:29:44.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 -- 172.21.15.97:0/2839999274 >> 172.21.15.97:0/2839999274 conn(0x7fc2b40b1780 msgr2=0x7fc2b40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:44.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 -- 172.21.15.97:0/2839999274 shutdown_connections 2024-09-10T21:29:44.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 -- 172.21.15.97:0/2839999274 wait complete. 2024-09-10T21:29:44.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 Processor -- start 2024-09-10T21:29:44.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.225+0000 7fc2bc600640 1 -- start start 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc29c005e60 0x7fc2b41f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b4150680 0x7fc2b40ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2b414ffb0 0x7fc2b4159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b406f060 con 0x7fc2b4150680 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b415b270 con 0x7fc29c005e60 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b406f1e0 con 0x7fc2b414ffb0 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b4150680 0x7fc2b40ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:44.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b4150680 0x7fc2b40ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35140/0 (socket says 172.21.15.97:35140) 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 -- 172.21.15.97:0/2047023959 learned_addr learned my addr 172.21.15.97:0/2047023959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9c00640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc29c005e60 0x7fc2b41f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2ba600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2b414ffb0 0x7fc2b4159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 -- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2b414ffb0 msgr2=0x7fc2b4159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2b414ffb0 0x7fc2b4159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 -- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc29c005e60 msgr2=0x7fc2b41f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc29c005e60 0x7fc2b41f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 -- 172.21.15.97:0/2047023959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b41f4990 con 0x7fc2b4150680 2024-09-10T21:29:44.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b9200640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b4150680 0x7fc2b40ce0f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0011b10 tx=0x7fc2a0009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:44.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b1600640 1 -- 172.21.15.97:0/2047023959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a001e070 con 0x7fc2b4150680 2024-09-10T21:29:44.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b1600640 1 -- 172.21.15.97:0/2047023959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2a0004050 con 0x7fc2b4150680 2024-09-10T21:29:44.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2b1600640 1 -- 172.21.15.97:0/2047023959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000fea0 con 0x7fc2b4150680 2024-09-10T21:29:44.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b4150460 con 0x7fc2b4150680 2024-09-10T21:29:44.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41f4c30 con 0x7fc2b4150680 2024-09-10T21:29:44.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.229+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b415a8d0 con 0x7fc2b4150680 2024-09-10T21:29:44.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.233+0000 7fc2b1600640 1 -- 172.21.15.97:0/2047023959 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc2a0003600 con 0x7fc2b4150680 2024-09-10T21:29:44.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.233+0000 7fc2b1600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc294072520 0x7fc2940749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.233+0000 7fc2b1600640 1 -- 172.21.15.97:0/2047023959 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fc2a0094230 con 0x7fc2b4150680 2024-09-10T21:29:44.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.233+0000 7fc2b9c00640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc294072520 0x7fc2940749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:44.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.233+0000 7fc2b9c00640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc294072520 0x7fc2940749e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8006fd0 tx=0x7fc2a8006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:44.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.237+0000 7fc2b1600640 1 -- 172.21.15.97:0/2047023959 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc2a005f1e0 con 0x7fc2b4150680 2024-09-10T21:29:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.401+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2b41f5010 con 0x7fc294072520 2024-09-10T21:29:44.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.405+0000 7fc2b1600640 1 -- 172.21.15.97:0/2047023959 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252061 (secure 0 0 0) 0x7fc2b41f5010 con 0x7fc294072520 2024-09-10T21:29:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:44.412 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:29:44.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc294072520 msgr2=0x7fc2940749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc294072520 0x7fc2940749e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8006fd0 tx=0x7fc2a8006b20 comp rx=0 tx=0).stop 2024-09-10T21:29:44.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b4150680 msgr2=0x7fc2b40ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b4150680 0x7fc2b40ce0f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0011b10 tx=0x7fc2a0009ea0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 shutdown_connections 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc294072520 0x7fc2940749e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2b414ffb0 0x7fc2b4159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b4150680 0x7fc2b40ce0f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 --2- 172.21.15.97:0/2047023959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc29c005e60 0x7fc2b41f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 >> 172.21.15.97:0/2047023959 conn(0x7fc2b40b1780 msgr2=0x7fc2b415caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 shutdown_connections 2024-09-10T21:29:44.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.409+0000 7fc2bc600640 1 -- 172.21.15.97:0/2047023959 wait complete. 2024-09-10T21:29:44.432 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-10T21:29:44.156157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241876,"kb_used_data":8988,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754732844,"statfs":{"total":773094113280,"available":772846432256,"internally_reserved":0,"allocated":9203712,"data_stored":4843944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001568"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768894+0000","last_change":"2024-09-10T21:29:39.539326+0000","last_active":"2024-09-10T21:29:40.768894+0000","last_peered":"2024-09-10T21:29:40.768894+0000","last_clean":"2024-09-10T21:29:40.768894+0000","last_became_active":"2024-09-10T21:29:39.539056+0000","last_became_peered":"2024-09-10T21:29:39.539056+0000","last_unstale":"2024-09-10T21:29:40.768894+0000","last_undegraded":"2024-09-10T21:29:40.768894+0000","last_fullsized":"2024-09-10T21:29:40.768894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:25.767146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769190+0000","last_change":"2024-09-10T21:29:39.564404+0000","last_active":"2024-09-10T21:29:40.769190+0000","last_peered":"2024-09-10T21:29:40.769190+0000","last_clean":"2024-09-10T21:29:40.769190+0000","last_became_active":"2024-09-10T21:29:39.563306+0000","last_became_peered":"2024-09-10T21:29:39.563306+0000","last_unstale":"2024-09-10T21:29:40.769190+0000","last_undegraded":"2024-09-10T21:29:40.769190+0000","last_fullsized":"2024-09-10T21:29:40.769190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:17.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769401+0000","last_change":"2024-09-10T21:29:39.561606+0000","last_active":"2024-09-10T21:29:40.769401+0000","last_peered":"2024-09-10T21:29:40.769401+0000","last_clean":"2024-09-10T21:29:40.769401+0000","last_became_active":"2024-09-10T21:29:39.560972+0000","last_became_peered":"2024-09-10T21:29:39.560972+0000","last_unstale":"2024-09-10T21:29:40.769401+0000","last_undegraded":"2024-09-10T21:29:40.769401+0000","last_fullsized":"2024-09-10T21:29:40.769401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:26.701898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544576+0000","last_change":"2024-09-10T21:29:39.537773+0000","last_active":"2024-09-10T21:29:40.544576+0000","last_peered":"2024-09-10T21:29:40.544576+0000","last_clean":"2024-09-10T21:29:40.544576+0000","last_became_active":"2024-09-10T21:29:39.537487+0000","last_became_peered":"2024-09-10T21:29:39.537487+0000","last_unstale":"2024-09-10T21:29:40.544576+0000","last_undegraded":"2024-09-10T21:29:40.544576+0000","last_fullsized":"2024-09-10T21:29:40.544576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:53.105160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546947+0000","last_change":"2024-09-10T21:29:39.546644+0000","last_active":"2024-09-10T21:29:40.546947+0000","last_peered":"2024-09-10T21:29:40.546947+0000","last_clean":"2024-09-10T21:29:40.546947+0000","last_became_active":"2024-09-10T21:29:39.546373+0000","last_became_peered":"2024-09-10T21:29:39.546373+0000","last_unstale":"2024-09-10T21:29:40.546947+0000","last_undegraded":"2024-09-10T21:29:40.546947+0000","last_fullsized":"2024-09-10T21:29:40.546947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.942938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546101+0000","last_change":"2024-09-10T21:29:39.558707+0000","last_active":"2024-09-10T21:29:40.546101+0000","last_peered":"2024-09-10T21:29:40.546101+0000","last_clean":"2024-09-10T21:29:40.546101+0000","last_became_active":"2024-09-10T21:29:39.554793+0000","last_became_peered":"2024-09-10T21:29:39.554793+0000","last_unstale":"2024-09-10T21:29:40.546101+0000","last_undegraded":"2024-09-10T21:29:40.546101+0000","last_fullsized":"2024-09-10T21:29:40.546101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544328+0000","last_change":"2024-09-10T21:29:39.540653+0000","last_active":"2024-09-10T21:29:40.544328+0000","last_peered":"2024-09-10T21:29:40.544328+0000","last_clean":"2024-09-10T21:29:40.544328+0000","last_became_active":"2024-09-10T21:29:39.540400+0000","last_became_peered":"2024-09-10T21:29:39.540400+0000","last_unstale":"2024-09-10T21:29:40.544328+0000","last_undegraded":"2024-09-10T21:29:40.544328+0000","last_fullsized":"2024-09-10T21:29:40.544328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:26.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768468+0000","last_change":"2024-09-10T21:29:39.535592+0000","last_active":"2024-09-10T21:29:40.768468+0000","last_peered":"2024-09-10T21:29:40.768468+0000","last_clean":"2024-09-10T21:29:40.768468+0000","last_became_active":"2024-09-10T21:29:39.535236+0000","last_became_peered":"2024-09-10T21:29:39.535236+0000","last_unstale":"2024-09-10T21:29:40.768468+0000","last_undegraded":"2024-09-10T21:29:40.768468+0000","last_fullsized":"2024-09-10T21:29:40.768468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:54.259155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545150+0000","last_change":"2024-09-10T21:29:39.556621+0000","last_active":"2024-09-10T21:29:40.545150+0000","last_peered":"2024-09-10T21:29:40.545150+0000","last_clean":"2024-09-10T21:29:40.545150+0000","last_became_active":"2024-09-10T21:29:39.555001+0000","last_became_peered":"2024-09-10T21:29:39.555001+0000","last_unstale":"2024-09-10T21:29:40.545150+0000","last_undegraded":"2024-09-10T21:29:40.545150+0000","last_fullsized":"2024-09-10T21:29:40.545150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769242+0000","last_change":"2024-09-10T21:29:39.536232+0000","last_active":"2024-09-10T21:29:40.769242+0000","last_peered":"2024-09-10T21:29:40.769242+0000","last_clean":"2024-09-10T21:29:40.769242+0000","last_became_active":"2024-09-10T21:29:39.535978+0000","last_became_peered":"2024-09-10T21:29:39.535978+0000","last_unstale":"2024-09-10T21:29:40.769242+0000","last_undegraded":"2024-09-10T21:29:40.769242+0000","last_fullsized":"2024-09-10T21:29:40.769242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:49.789897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769661+0000","last_change":"2024-09-10T21:29:39.562255+0000","last_active":"2024-09-10T21:29:40.769661+0000","last_peered":"2024-09-10T21:29:40.769661+0000","last_clean":"2024-09-10T21:29:40.769661+0000","last_became_active":"2024-09-10T21:29:39.559040+0000","last_became_peered":"2024-09-10T21:29:39.559040+0000","last_unstale":"2024-09-10T21:29:40.769661+0000","last_undegraded":"2024-09-10T21:29:40.769661+0000","last_fullsized":"2024-09-10T21:29:40.769661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:39.446682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769690+0000","last_change":"2024-09-10T21:29:39.550940+0000","last_active":"2024-09-10T21:29:40.769690+0000","last_peered":"2024-09-10T21:29:40.769690+0000","last_clean":"2024-09-10T21:29:40.769690+0000","last_became_active":"2024-09-10T21:29:39.550666+0000","last_became_peered":"2024-09-10T21:29:39.550666+0000","last_unstale":"2024-09-10T21:29:40.769690+0000","last_undegraded":"2024-09-10T21:29:40.769690+0000","last_fullsized":"2024-09-10T21:29:40.769690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:24.960891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768575+0000","last_change":"2024-09-10T21:29:39.555356+0000","last_active":"2024-09-10T21:29:40.768575+0000","last_peered":"2024-09-10T21:29:40.768575+0000","last_clean":"2024-09-10T21:29:40.768575+0000","last_became_active":"2024-09-10T21:29:39.554843+0000","last_became_peered":"2024-09-10T21:29:39.554843+0000","last_unstale":"2024-09-10T21:29:40.768575+0000","last_undegraded":"2024-09-10T21:29:40.768575+0000","last_fullsized":"2024-09-10T21:29:40.768575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:03.103988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769008+0000","last_change":"2024-09-10T21:29:39.531161+0000","last_active":"2024-09-10T21:29:40.769008+0000","last_peered":"2024-09-10T21:29:40.769008+0000","last_clean":"2024-09-10T21:29:40.769008+0000","last_became_active":"2024-09-10T21:29:39.530873+0000","last_became_peered":"2024-09-10T21:29:39.530873+0000","last_unstale":"2024-09-10T21:29:40.769008+0000","last_undegraded":"2024-09-10T21:29:40.769008+0000","last_fullsized":"2024-09-10T21:29:40.769008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:25.754412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546185+0000","last_change":"2024-09-10T21:29:39.563879+0000","last_active":"2024-09-10T21:29:40.546185+0000","last_peered":"2024-09-10T21:29:40.546185+0000","last_clean":"2024-09-10T21:29:40.546185+0000","last_became_active":"2024-09-10T21:29:39.563673+0000","last_became_peered":"2024-09-10T21:29:39.563673+0000","last_unstale":"2024-09-10T21:29:40.546185+0000","last_undegraded":"2024-09-10T21:29:40.546185+0000","last_fullsized":"2024-09-10T21:29:40.546185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:33.785842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544217+0000","last_change":"2024-09-10T21:29:39.552730+0000","last_active":"2024-09-10T21:29:40.544217+0000","last_peered":"2024-09-10T21:29:40.544217+0000","last_clean":"2024-09-10T21:29:40.544217+0000","last_became_active":"2024-09-10T21:29:39.552478+0000","last_became_peered":"2024-09-10T21:29:39.552478+0000","last_unstale":"2024-09-10T21:29:40.544217+0000","last_undegraded":"2024-09-10T21:29:40.544217+0000","last_fullsized":"2024-09-10T21:29:40.544217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:00.008953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768952+0000","last_change":"2024-09-10T21:29:39.541920+0000","last_active":"2024-09-10T21:29:40.768952+0000","last_peered":"2024-09-10T21:29:40.768952+0000","last_clean":"2024-09-10T21:29:40.768952+0000","last_became_active":"2024-09-10T21:29:39.541604+0000","last_became_peered":"2024-09-10T21:29:39.541604+0000","last_unstale":"2024-09-10T21:29:40.768952+0000","last_undegraded":"2024-09-10T21:29:40.768952+0000","last_fullsized":"2024-09-10T21:29:40.768952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:24.814305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769429+0000","last_change":"2024-09-10T21:29:39.558608+0000","last_active":"2024-09-10T21:29:40.769429+0000","last_peered":"2024-09-10T21:29:40.769429+0000","last_clean":"2024-09-10T21:29:40.769429+0000","last_became_active":"2024-09-10T21:29:39.558086+0000","last_became_peered":"2024-09-10T21:29:39.558086+0000","last_unstale":"2024-09-10T21:29:40.769429+0000","last_undegraded":"2024-09-10T21:29:40.769429+0000","last_fullsized":"2024-09-10T21:29:40.769429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:51.432263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769280+0000","last_change":"2024-09-10T21:29:39.539085+0000","last_active":"2024-09-10T21:29:40.769280+0000","last_peered":"2024-09-10T21:29:40.769280+0000","last_clean":"2024-09-10T21:29:40.769280+0000","last_became_active":"2024-09-10T21:29:39.536918+0000","last_became_peered":"2024-09-10T21:29:39.536918+0000","last_unstale":"2024-09-10T21:29:40.769280+0000","last_undegraded":"2024-09-10T21:29:40.769280+0000","last_fullsized":"2024-09-10T21:29:40.769280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:56.499531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546462+0000","last_change":"2024-09-10T21:29:39.560153+0000","last_active":"2024-09-10T21:29:40.546462+0000","last_peered":"2024-09-10T21:29:40.546462+0000","last_clean":"2024-09-10T21:29:40.546462+0000","last_became_active":"2024-09-10T21:29:39.559886+0000","last_became_peered":"2024-09-10T21:29:39.559886+0000","last_unstale":"2024-09-10T21:29:40.546462+0000","last_undegraded":"2024-09-10T21:29:40.546462+0000","last_fullsized":"2024-09-10T21:29:40.546462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768980+0000","last_change":"2024-09-10T21:29:39.547565+0000","last_active":"2024-09-10T21:29:40.768980+0000","last_peered":"2024-09-10T21:29:40.768980+0000","last_clean":"2024-09-10T21:29:40.768980+0000","last_became_active":"2024-09-10T21:29:39.547315+0000","last_became_peered":"2024-09-10T21:29:39.547315+0000","last_unstale":"2024-09-10T21:29:40.768980+0000","last_undegraded":"2024-09-10T21:29:40.768980+0000","last_fullsized":"2024-09-10T21:29:40.768980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:05.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769033+0000","last_change":"2024-09-10T21:29:39.562781+0000","last_active":"2024-09-10T21:29:40.769033+0000","last_peered":"2024-09-10T21:29:40.769033+0000","last_clean":"2024-09-10T21:29:40.769033+0000","last_became_active":"2024-09-10T21:29:39.562509+0000","last_became_peered":"2024-09-10T21:29:39.562509+0000","last_unstale":"2024-09-10T21:29:40.769033+0000","last_undegraded":"2024-09-10T21:29:40.769033+0000","last_fullsized":"2024-09-10T21:29:40.769033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:53.170104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768742+0000","last_change":"2024-09-10T21:29:39.540989+0000","last_active":"2024-09-10T21:29:40.768742+0000","last_peered":"2024-09-10T21:29:40.768742+0000","last_clean":"2024-09-10T21:29:40.768742+0000","last_became_active":"2024-09-10T21:29:39.540686+0000","last_became_peered":"2024-09-10T21:29:39.540686+0000","last_unstale":"2024-09-10T21:29:40.768742+0000","last_undegraded":"2024-09-10T21:29:40.768742+0000","last_fullsized":"2024-09-10T21:29:40.768742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:04.647939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768432+0000","last_change":"2024-09-10T21:29:39.555697+0000","last_active":"2024-09-10T21:29:40.768432+0000","last_peered":"2024-09-10T21:29:40.768432+0000","last_clean":"2024-09-10T21:29:40.768432+0000","last_became_active":"2024-09-10T21:29:39.555321+0000","last_became_peered":"2024-09-10T21:29:39.555321+0000","last_unstale":"2024-09-10T21:29:40.768432+0000","last_undegraded":"2024-09-10T21:29:40.768432+0000","last_fullsized":"2024-09-10T21:29:40.768432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:26.937688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547452+0000","last_change":"2024-09-10T21:29:39.566354+0000","last_active":"2024-09-10T21:29:40.547452+0000","last_peered":"2024-09-10T21:29:40.547452+0000","last_clean":"2024-09-10T21:29:40.547452+0000","last_became_active":"2024-09-10T21:29:39.566119+0000","last_became_peered":"2024-09-10T21:29:39.566119+0000","last_unstale":"2024-09-10T21:29:40.547452+0000","last_undegraded":"2024-09-10T21:29:40.547452+0000","last_fullsized":"2024-09-10T21:29:40.547452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544876+0000","last_change":"2024-09-10T21:29:39.555058+0000","last_active":"2024-09-10T21:29:40.544876+0000","last_peered":"2024-09-10T21:29:40.544876+0000","last_clean":"2024-09-10T21:29:40.544876+0000","last_became_active":"2024-09-10T21:29:39.554444+0000","last_became_peered":"2024-09-10T21:29:39.554444+0000","last_unstale":"2024-09-10T21:29:40.544876+0000","last_undegraded":"2024-09-10T21:29:40.544876+0000","last_fullsized":"2024-09-10T21:29:40.544876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544584+0000","last_change":"2024-09-10T21:29:39.553456+0000","last_active":"2024-09-10T21:29:40.544584+0000","last_peered":"2024-09-10T21:29:40.544584+0000","last_clean":"2024-09-10T21:29:40.544584+0000","last_became_active":"2024-09-10T21:29:39.553218+0000","last_became_peered":"2024-09-10T21:29:39.553218+0000","last_unstale":"2024-09-10T21:29:40.544584+0000","last_undegraded":"2024-09-10T21:29:40.544584+0000","last_fullsized":"2024-09-10T21:29:40.544584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547497+0000","last_change":"2024-09-10T21:29:39.551245+0000","last_active":"2024-09-10T21:29:40.547497+0000","last_peered":"2024-09-10T21:29:40.547497+0000","last_clean":"2024-09-10T21:29:40.547497+0000","last_became_active":"2024-09-10T21:29:39.550535+0000","last_became_peered":"2024-09-10T21:29:39.550535+0000","last_unstale":"2024-09-10T21:29:40.547497+0000","last_undegraded":"2024-09-10T21:29:40.547497+0000","last_fullsized":"2024-09-10T21:29:40.547497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:03.284635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545912+0000","last_change":"2024-09-10T21:29:39.558777+0000","last_active":"2024-09-10T21:29:40.545912+0000","last_peered":"2024-09-10T21:29:40.545912+0000","last_clean":"2024-09-10T21:29:40.545912+0000","last_became_active":"2024-09-10T21:29:39.555098+0000","last_became_peered":"2024-09-10T21:29:39.555098+0000","last_unstale":"2024-09-10T21:29:40.545912+0000","last_undegraded":"2024-09-10T21:29:40.545912+0000","last_fullsized":"2024-09-10T21:29:40.545912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769096+0000","last_change":"2024-09-10T21:29:39.564203+0000","last_active":"2024-09-10T21:29:40.769096+0000","last_peered":"2024-09-10T21:29:40.769096+0000","last_clean":"2024-09-10T21:29:40.769096+0000","last_became_active":"2024-09-10T21:29:39.563951+0000","last_became_peered":"2024-09-10T21:29:39.563951+0000","last_unstale":"2024-09-10T21:29:40.769096+0000","last_undegraded":"2024-09-10T21:29:40.769096+0000","last_fullsized":"2024-09-10T21:29:40.769096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:38.678603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768992+0000","last_change":"2024-09-10T21:29:39.564492+0000","last_active":"2024-09-10T21:29:40.768992+0000","last_peered":"2024-09-10T21:29:40.768992+0000","last_clean":"2024-09-10T21:29:40.768992+0000","last_became_active":"2024-09-10T21:29:39.563500+0000","last_became_peered":"2024-09-10T21:29:39.563500+0000","last_unstale":"2024-09-10T21:29:40.768992+0000","last_undegraded":"2024-09-10T21:29:40.768992+0000","last_fullsized":"2024-09-10T21:29:40.768992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:46.030917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544257+0000","last_change":"2024-09-10T21:29:39.555396+0000","last_active":"2024-09-10T21:29:40.544257+0000","last_peered":"2024-09-10T21:29:40.544257+0000","last_clean":"2024-09-10T21:29:40.544257+0000","last_became_active":"2024-09-10T21:29:39.555086+0000","last_became_peered":"2024-09-10T21:29:39.555086+0000","last_unstale":"2024-09-10T21:29:40.544257+0000","last_undegraded":"2024-09-10T21:29:40.544257+0000","last_fullsized":"2024-09-10T21:29:40.544257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:06.000345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546200+0000","last_change":"2024-09-10T21:29:39.548302+0000","last_active":"2024-09-10T21:29:40.546200+0000","last_peered":"2024-09-10T21:29:40.546200+0000","last_clean":"2024-09-10T21:29:40.546200+0000","last_became_active":"2024-09-10T21:29:39.547972+0000","last_became_peered":"2024-09-10T21:29:39.547972+0000","last_unstale":"2024-09-10T21:29:40.546200+0000","last_undegraded":"2024-09-10T21:29:40.546200+0000","last_fullsized":"2024-09-10T21:29:40.546200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:22.457526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547580+0000","last_change":"2024-09-10T21:29:38.549320+0000","last_active":"2024-09-10T21:29:40.547580+0000","last_peered":"2024-09-10T21:29:40.547580+0000","last_clean":"2024-09-10T21:29:40.547580+0000","last_became_active":"2024-09-10T21:29:38.549041+0000","last_became_peered":"2024-09-10T21:29:38.549041+0000","last_unstale":"2024-09-10T21:29:40.547580+0000","last_undegraded":"2024-09-10T21:29:40.547580+0000","last_fullsized":"2024-09-10T21:29:40.547580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:52.457273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546222+0000","last_change":"2024-09-10T21:29:39.542269+0000","last_active":"2024-09-10T21:29:40.546222+0000","last_peered":"2024-09-10T21:29:40.546222+0000","last_clean":"2024-09-10T21:29:40.546222+0000","last_became_active":"2024-09-10T21:29:39.541975+0000","last_became_peered":"2024-09-10T21:29:39.541975+0000","last_unstale":"2024-09-10T21:29:40.546222+0000","last_undegraded":"2024-09-10T21:29:40.546222+0000","last_fullsized":"2024-09-10T21:29:40.546222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:38.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769882+0000","last_change":"2024-09-10T21:29:38.544792+0000","last_active":"2024-09-10T21:29:40.769882+0000","last_peered":"2024-09-10T21:29:40.769882+0000","last_clean":"2024-09-10T21:29:40.769882+0000","last_became_active":"2024-09-10T21:29:38.544459+0000","last_became_peered":"2024-09-10T21:29:38.544459+0000","last_unstale":"2024-09-10T21:29:40.769882+0000","last_undegraded":"2024-09-10T21:29:40.769882+0000","last_fullsized":"2024-09-10T21:29:40.769882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:28.938255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769152+0000","last_change":"2024-09-10T21:29:39.546560+0000","last_active":"2024-09-10T21:29:40.769152+0000","last_peered":"2024-09-10T21:29:40.769152+0000","last_clean":"2024-09-10T21:29:40.769152+0000","last_became_active":"2024-09-10T21:29:39.545328+0000","last_became_peered":"2024-09-10T21:29:39.545328+0000","last_unstale":"2024-09-10T21:29:40.769152+0000","last_undegraded":"2024-09-10T21:29:40.769152+0000","last_fullsized":"2024-09-10T21:29:40.769152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:20.557881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768895+0000","last_change":"2024-09-10T21:29:38.538896+0000","last_active":"2024-09-10T21:29:40.768895+0000","last_peered":"2024-09-10T21:29:40.768895+0000","last_clean":"2024-09-10T21:29:40.768895+0000","last_became_active":"2024-09-10T21:29:38.538523+0000","last_became_peered":"2024-09-10T21:29:38.538523+0000","last_unstale":"2024-09-10T21:29:40.768895+0000","last_undegraded":"2024-09-10T21:29:40.768895+0000","last_fullsized":"2024-09-10T21:29:40.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:26.307429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544519+0000","last_change":"2024-09-10T21:29:39.538536+0000","last_active":"2024-09-10T21:29:40.544519+0000","last_peered":"2024-09-10T21:29:40.544519+0000","last_clean":"2024-09-10T21:29:40.544519+0000","last_became_active":"2024-09-10T21:29:39.538308+0000","last_became_peered":"2024-09-10T21:29:39.538308+0000","last_unstale":"2024-09-10T21:29:40.544519+0000","last_undegraded":"2024-09-10T21:29:40.544519+0000","last_fullsized":"2024-09-10T21:29:40.544519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:40.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769616+0000","last_change":"2024-09-10T21:29:38.536197+0000","last_active":"2024-09-10T21:29:40.769616+0000","last_peered":"2024-09-10T21:29:40.769616+0000","last_clean":"2024-09-10T21:29:40.769616+0000","last_became_active":"2024-09-10T21:29:38.535894+0000","last_became_peered":"2024-09-10T21:29:38.535894+0000","last_unstale":"2024-09-10T21:29:40.769616+0000","last_undegraded":"2024-09-10T21:29:40.769616+0000","last_fullsized":"2024-09-10T21:29:40.769616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546646+0000","last_change":"2024-09-10T21:29:39.554137+0000","last_active":"2024-09-10T21:29:40.546646+0000","last_peered":"2024-09-10T21:29:40.546646+0000","last_clean":"2024-09-10T21:29:40.546646+0000","last_became_active":"2024-09-10T21:29:39.553735+0000","last_became_peered":"2024-09-10T21:29:39.553735+0000","last_unstale":"2024-09-10T21:29:40.546646+0000","last_undegraded":"2024-09-10T21:29:40.546646+0000","last_fullsized":"2024-09-10T21:29:40.546646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769158+0000","last_change":"2024-09-10T21:29:38.535949+0000","last_active":"2024-09-10T21:29:40.769158+0000","last_peered":"2024-09-10T21:29:40.769158+0000","last_clean":"2024-09-10T21:29:40.769158+0000","last_became_active":"2024-09-10T21:29:38.535669+0000","last_became_peered":"2024-09-10T21:29:38.535669+0000","last_unstale":"2024-09-10T21:29:40.769158+0000","last_undegraded":"2024-09-10T21:29:40.769158+0000","last_fullsized":"2024-09-10T21:29:40.769158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769743+0000","last_change":"2024-09-10T21:29:39.561954+0000","last_active":"2024-09-10T21:29:40.769743+0000","last_peered":"2024-09-10T21:29:40.769743+0000","last_clean":"2024-09-10T21:29:40.769743+0000","last_became_active":"2024-09-10T21:29:39.561348+0000","last_became_peered":"2024-09-10T21:29:39.561348+0000","last_unstale":"2024-09-10T21:29:40.769743+0000","last_undegraded":"2024-09-10T21:29:40.769743+0000","last_fullsized":"2024-09-10T21:29:40.769743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:54.689918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769497+0000","last_change":"2024-09-10T21:29:38.543780+0000","last_active":"2024-09-10T21:29:40.769497+0000","last_peered":"2024-09-10T21:29:40.769497+0000","last_clean":"2024-09-10T21:29:40.769497+0000","last_became_active":"2024-09-10T21:29:38.543305+0000","last_became_peered":"2024-09-10T21:29:38.543305+0000","last_unstale":"2024-09-10T21:29:40.769497+0000","last_undegraded":"2024-09-10T21:29:40.769497+0000","last_fullsized":"2024-09-10T21:29:40.769497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.941433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768491+0000","last_change":"2024-09-10T21:29:39.549779+0000","last_active":"2024-09-10T21:29:40.768491+0000","last_peered":"2024-09-10T21:29:40.768491+0000","last_clean":"2024-09-10T21:29:40.768491+0000","last_became_active":"2024-09-10T21:29:39.549473+0000","last_became_peered":"2024-09-10T21:29:39.549473+0000","last_unstale":"2024-09-10T21:29:40.768491+0000","last_undegraded":"2024-09-10T21:29:40.768491+0000","last_fullsized":"2024-09-10T21:29:40.768491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768418+0000","last_change":"2024-09-10T21:29:38.546344+0000","last_active":"2024-09-10T21:29:40.768418+0000","last_peered":"2024-09-10T21:29:40.768418+0000","last_clean":"2024-09-10T21:29:40.768418+0000","last_became_active":"2024-09-10T21:29:38.545808+0000","last_became_peered":"2024-09-10T21:29:38.545808+0000","last_unstale":"2024-09-10T21:29:40.768418+0000","last_undegraded":"2024-09-10T21:29:40.768418+0000","last_fullsized":"2024-09-10T21:29:40.768418+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:12.358215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769992+0000","last_change":"2024-09-10T21:29:39.558263+0000","last_active":"2024-09-10T21:29:40.769992+0000","last_peered":"2024-09-10T21:29:40.769992+0000","last_clean":"2024-09-10T21:29:40.769992+0000","last_became_active":"2024-09-10T21:29:39.557990+0000","last_became_peered":"2024-09-10T21:29:39.557990+0000","last_unstale":"2024-09-10T21:29:40.769992+0000","last_undegraded":"2024-09-10T21:29:40.769992+0000","last_fullsized":"2024-09-10T21:29:40.769992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.511263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768565+0000","last_change":"2024-09-10T21:29:38.545254+0000","last_active":"2024-09-10T21:29:40.768565+0000","last_peered":"2024-09-10T21:29:40.768565+0000","last_clean":"2024-09-10T21:29:40.768565+0000","last_became_active":"2024-09-10T21:29:38.544954+0000","last_became_peered":"2024-09-10T21:29:38.544954+0000","last_unstale":"2024-09-10T21:29:40.768565+0000","last_undegraded":"2024-09-10T21:29:40.768565+0000","last_fullsized":"2024-09-10T21:29:40.768565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768528+0000","last_change":"2024-09-10T21:29:39.555635+0000","last_active":"2024-09-10T21:29:40.768528+0000","last_peered":"2024-09-10T21:29:40.768528+0000","last_clean":"2024-09-10T21:29:40.768528+0000","last_became_active":"2024-09-10T21:29:39.555011+0000","last_became_peered":"2024-09-10T21:29:39.555011+0000","last_unstale":"2024-09-10T21:29:40.768528+0000","last_undegraded":"2024-09-10T21:29:40.768528+0000","last_fullsized":"2024-09-10T21:29:40.768528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:15.429757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545012+0000","last_change":"2024-09-10T21:29:38.549092+0000","last_active":"2024-09-10T21:29:40.545012+0000","last_peered":"2024-09-10T21:29:40.545012+0000","last_clean":"2024-09-10T21:29:40.545012+0000","last_became_active":"2024-09-10T21:29:38.548800+0000","last_became_peered":"2024-09-10T21:29:38.548800+0000","last_unstale":"2024-09-10T21:29:40.545012+0000","last_undegraded":"2024-09-10T21:29:40.545012+0000","last_fullsized":"2024-09-10T21:29:40.545012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:10.710966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768928+0000","last_change":"2024-09-10T21:29:39.554166+0000","last_active":"2024-09-10T21:29:40.768928+0000","last_peered":"2024-09-10T21:29:40.768928+0000","last_clean":"2024-09-10T21:29:40.768928+0000","last_became_active":"2024-09-10T21:29:39.551856+0000","last_became_peered":"2024-09-10T21:29:39.551856+0000","last_unstale":"2024-09-10T21:29:40.768928+0000","last_undegraded":"2024-09-10T21:29:40.768928+0000","last_fullsized":"2024-09-10T21:29:40.768928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.611740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769311+0000","last_change":"2024-09-10T21:29:38.545177+0000","last_active":"2024-09-10T21:29:40.769311+0000","last_peered":"2024-09-10T21:29:40.769311+0000","last_clean":"2024-09-10T21:29:40.769311+0000","last_became_active":"2024-09-10T21:29:38.544523+0000","last_became_peered":"2024-09-10T21:29:38.544523+0000","last_unstale":"2024-09-10T21:29:40.769311+0000","last_undegraded":"2024-09-10T21:29:40.769311+0000","last_fullsized":"2024-09-10T21:29:40.769311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:36.904722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769325+0000","last_change":"2024-09-10T21:29:39.536154+0000","last_active":"2024-09-10T21:29:40.769325+0000","last_peered":"2024-09-10T21:29:40.769325+0000","last_clean":"2024-09-10T21:29:40.769325+0000","last_became_active":"2024-09-10T21:29:39.535816+0000","last_became_peered":"2024-09-10T21:29:39.535816+0000","last_unstale":"2024-09-10T21:29:40.769325+0000","last_undegraded":"2024-09-10T21:29:40.769325+0000","last_fullsized":"2024-09-10T21:29:40.769325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:41.370702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546787+0000","last_change":"2024-09-10T21:29:38.542661+0000","last_active":"2024-09-10T21:29:40.546787+0000","last_peered":"2024-09-10T21:29:40.546787+0000","last_clean":"2024-09-10T21:29:40.546787+0000","last_became_active":"2024-09-10T21:29:38.542425+0000","last_became_peered":"2024-09-10T21:29:38.542425+0000","last_unstale":"2024-09-10T21:29:40.546787+0000","last_undegraded":"2024-09-10T21:29:40.546787+0000","last_fullsized":"2024-09-10T21:29:40.546787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:47.295559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545841+0000","last_change":"2024-09-10T21:29:39.548032+0000","last_active":"2024-09-10T21:29:40.545841+0000","last_peered":"2024-09-10T21:29:40.545841+0000","last_clean":"2024-09-10T21:29:40.545841+0000","last_became_active":"2024-09-10T21:29:39.547709+0000","last_became_peered":"2024-09-10T21:29:39.547709+0000","last_unstale":"2024-09-10T21:29:40.545841+0000","last_undegraded":"2024-09-10T21:29:40.545841+0000","last_fullsized":"2024-09-10T21:29:40.545841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769782+0000","last_change":"2024-09-10T21:29:38.532633+0000","last_active":"2024-09-10T21:29:40.769782+0000","last_peered":"2024-09-10T21:29:40.769782+0000","last_clean":"2024-09-10T21:29:40.769782+0000","last_became_active":"2024-09-10T21:29:38.532371+0000","last_became_peered":"2024-09-10T21:29:38.532371+0000","last_unstale":"2024-09-10T21:29:40.769782+0000","last_undegraded":"2024-09-10T21:29:40.769782+0000","last_fullsized":"2024-09-10T21:29:40.769782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:45.933789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544716+0000","last_change":"2024-09-10T21:29:39.554909+0000","last_active":"2024-09-10T21:29:40.544716+0000","last_peered":"2024-09-10T21:29:40.544716+0000","last_clean":"2024-09-10T21:29:40.544716+0000","last_became_active":"2024-09-10T21:29:39.554651+0000","last_became_peered":"2024-09-10T21:29:39.554651+0000","last_unstale":"2024-09-10T21:29:40.544716+0000","last_undegraded":"2024-09-10T21:29:40.544716+0000","last_fullsized":"2024-09-10T21:29:40.544716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768411+0000","last_change":"2024-09-10T21:29:38.535431+0000","last_active":"2024-09-10T21:29:40.768411+0000","last_peered":"2024-09-10T21:29:40.768411+0000","last_clean":"2024-09-10T21:29:40.768411+0000","last_became_active":"2024-09-10T21:29:38.535114+0000","last_became_peered":"2024-09-10T21:29:38.535114+0000","last_unstale":"2024-09-10T21:29:40.768411+0000","last_undegraded":"2024-09-10T21:29:40.768411+0000","last_fullsized":"2024-09-10T21:29:40.768411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:05.208145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768791+0000","last_change":"2024-09-10T21:29:39.563604+0000","last_active":"2024-09-10T21:29:40.768791+0000","last_peered":"2024-09-10T21:29:40.768791+0000","last_clean":"2024-09-10T21:29:40.768791+0000","last_became_active":"2024-09-10T21:29:39.563344+0000","last_became_peered":"2024-09-10T21:29:39.563344+0000","last_unstale":"2024-09-10T21:29:40.768791+0000","last_undegraded":"2024-09-10T21:29:40.768791+0000","last_fullsized":"2024-09-10T21:29:40.768791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:50.487501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769250+0000","last_change":"2024-09-10T21:29:38.544304+0000","last_active":"2024-09-10T21:29:40.769250+0000","last_peered":"2024-09-10T21:29:40.769250+0000","last_clean":"2024-09-10T21:29:40.769250+0000","last_became_active":"2024-09-10T21:29:38.544049+0000","last_became_peered":"2024-09-10T21:29:38.544049+0000","last_unstale":"2024-09-10T21:29:40.769250+0000","last_undegraded":"2024-09-10T21:29:40.769250+0000","last_fullsized":"2024-09-10T21:29:40.769250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768692+0000","last_change":"2024-09-10T21:29:39.544825+0000","last_active":"2024-09-10T21:29:40.768692+0000","last_peered":"2024-09-10T21:29:40.768692+0000","last_clean":"2024-09-10T21:29:40.768692+0000","last_became_active":"2024-09-10T21:29:39.544578+0000","last_became_peered":"2024-09-10T21:29:39.544578+0000","last_unstale":"2024-09-10T21:29:40.768692+0000","last_undegraded":"2024-09-10T21:29:40.768692+0000","last_fullsized":"2024-09-10T21:29:40.768692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:17.043812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546585+0000","last_change":"2024-09-10T21:29:38.549765+0000","last_active":"2024-09-10T21:29:40.546585+0000","last_peered":"2024-09-10T21:29:40.546585+0000","last_clean":"2024-09-10T21:29:40.546585+0000","last_became_active":"2024-09-10T21:29:38.548990+0000","last_became_peered":"2024-09-10T21:29:38.548990+0000","last_unstale":"2024-09-10T21:29:40.546585+0000","last_undegraded":"2024-09-10T21:29:40.546585+0000","last_fullsized":"2024-09-10T21:29:40.546585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:52.559734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768695+0000","last_change":"2024-09-10T21:29:39.547930+0000","last_active":"2024-09-10T21:29:40.768695+0000","last_peered":"2024-09-10T21:29:40.768695+0000","last_clean":"2024-09-10T21:29:40.768695+0000","last_became_active":"2024-09-10T21:29:39.547672+0000","last_became_peered":"2024-09-10T21:29:39.547672+0000","last_unstale":"2024-09-10T21:29:40.768695+0000","last_undegraded":"2024-09-10T21:29:40.768695+0000","last_fullsized":"2024-09-10T21:29:40.768695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:52.513550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545358+0000","last_change":"2024-09-10T21:29:38.544208+0000","last_active":"2024-09-10T21:29:40.545358+0000","last_peered":"2024-09-10T21:29:40.545358+0000","last_clean":"2024-09-10T21:29:40.545358+0000","last_became_active":"2024-09-10T21:29:38.543907+0000","last_became_peered":"2024-09-10T21:29:38.543907+0000","last_unstale":"2024-09-10T21:29:40.545358+0000","last_undegraded":"2024-09-10T21:29:40.545358+0000","last_fullsized":"2024-09-10T21:29:40.545358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:37.225392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546397+0000","last_change":"2024-09-10T21:29:39.557707+0000","last_active":"2024-09-10T21:29:40.546397+0000","last_peered":"2024-09-10T21:29:40.546397+0000","last_clean":"2024-09-10T21:29:40.546397+0000","last_became_active":"2024-09-10T21:29:39.557398+0000","last_became_peered":"2024-09-10T21:29:39.557398+0000","last_unstale":"2024-09-10T21:29:40.546397+0000","last_undegraded":"2024-09-10T21:29:40.546397+0000","last_fullsized":"2024-09-10T21:29:40.546397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:05.442260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545105+0000","last_change":"2024-09-10T21:29:38.549044+0000","last_active":"2024-09-10T21:29:40.545105+0000","last_peered":"2024-09-10T21:29:40.545105+0000","last_clean":"2024-09-10T21:29:40.545105+0000","last_became_active":"2024-09-10T21:29:38.548465+0000","last_became_peered":"2024-09-10T21:29:38.548465+0000","last_unstale":"2024-09-10T21:29:40.545105+0000","last_undegraded":"2024-09-10T21:29:40.545105+0000","last_fullsized":"2024-09-10T21:29:40.545105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:47.729570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769221+0000","last_change":"2024-09-10T21:29:39.555291+0000","last_active":"2024-09-10T21:29:40.769221+0000","last_peered":"2024-09-10T21:29:40.769221+0000","last_clean":"2024-09-10T21:29:40.769221+0000","last_became_active":"2024-09-10T21:29:39.554979+0000","last_became_peered":"2024-09-10T21:29:39.554979+0000","last_unstale":"2024-09-10T21:29:40.769221+0000","last_undegraded":"2024-09-10T21:29:40.769221+0000","last_fullsized":"2024-09-10T21:29:40.769221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:41.126386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544896+0000","last_change":"2024-09-10T21:29:38.545682+0000","last_active":"2024-09-10T21:29:40.544896+0000","last_peered":"2024-09-10T21:29:40.544896+0000","last_clean":"2024-09-10T21:29:40.544896+0000","last_became_active":"2024-09-10T21:29:38.545432+0000","last_became_peered":"2024-09-10T21:29:38.545432+0000","last_unstale":"2024-09-10T21:29:40.544896+0000","last_undegraded":"2024-09-10T21:29:40.544896+0000","last_fullsized":"2024-09-10T21:29:40.544896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769268+0000","last_change":"2024-09-10T21:29:39.560953+0000","last_active":"2024-09-10T21:29:40.769268+0000","last_peered":"2024-09-10T21:29:40.769268+0000","last_clean":"2024-09-10T21:29:40.769268+0000","last_became_active":"2024-09-10T21:29:39.560702+0000","last_became_peered":"2024-09-10T21:29:39.560702+0000","last_unstale":"2024-09-10T21:29:40.769268+0000","last_undegraded":"2024-09-10T21:29:40.769268+0000","last_fullsized":"2024-09-10T21:29:40.769268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:05.821463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546758+0000","last_change":"2024-09-10T21:29:38.550165+0000","last_active":"2024-09-10T21:29:40.546758+0000","last_peered":"2024-09-10T21:29:40.546758+0000","last_clean":"2024-09-10T21:29:40.546758+0000","last_became_active":"2024-09-10T21:29:38.549861+0000","last_became_peered":"2024-09-10T21:29:38.549861+0000","last_unstale":"2024-09-10T21:29:40.546758+0000","last_undegraded":"2024-09-10T21:29:40.546758+0000","last_fullsized":"2024-09-10T21:29:40.546758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:02.189449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547170+0000","last_change":"2024-09-10T21:29:39.545985+0000","last_active":"2024-09-10T21:29:40.547170+0000","last_peered":"2024-09-10T21:29:40.547170+0000","last_clean":"2024-09-10T21:29:40.547170+0000","last_became_active":"2024-09-10T21:29:39.545349+0000","last_became_peered":"2024-09-10T21:29:39.545349+0000","last_unstale":"2024-09-10T21:29:40.547170+0000","last_undegraded":"2024-09-10T21:29:40.547170+0000","last_fullsized":"2024-09-10T21:29:40.547170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:14.688355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547653+0000","last_change":"2024-09-10T21:29:38.546910+0000","last_active":"2024-09-10T21:29:40.547653+0000","last_peered":"2024-09-10T21:29:40.547653+0000","last_clean":"2024-09-10T21:29:40.547653+0000","last_became_active":"2024-09-10T21:29:38.546159+0000","last_became_peered":"2024-09-10T21:29:38.546159+0000","last_unstale":"2024-09-10T21:29:40.547653+0000","last_undegraded":"2024-09-10T21:29:40.547653+0000","last_fullsized":"2024-09-10T21:29:40.547653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.324328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547733+0000","last_change":"2024-09-10T21:29:39.551802+0000","last_active":"2024-09-10T21:29:40.547733+0000","last_peered":"2024-09-10T21:29:40.547733+0000","last_clean":"2024-09-10T21:29:40.547733+0000","last_became_active":"2024-09-10T21:29:39.550894+0000","last_became_peered":"2024-09-10T21:29:39.550894+0000","last_unstale":"2024-09-10T21:29:40.547733+0000","last_undegraded":"2024-09-10T21:29:40.547733+0000","last_fullsized":"2024-09-10T21:29:40.547733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:35.976956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768722+0000","last_change":"2024-09-10T21:29:38.549752+0000","last_active":"2024-09-10T21:29:40.768722+0000","last_peered":"2024-09-10T21:29:40.768722+0000","last_clean":"2024-09-10T21:29:40.768722+0000","last_became_active":"2024-09-10T21:29:38.549380+0000","last_became_peered":"2024-09-10T21:29:38.549380+0000","last_unstale":"2024-09-10T21:29:40.768722+0000","last_undegraded":"2024-09-10T21:29:40.768722+0000","last_fullsized":"2024-09-10T21:29:40.768722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547195+0000","last_change":"2024-09-10T21:29:39.544277+0000","last_active":"2024-09-10T21:29:40.547195+0000","last_peered":"2024-09-10T21:29:40.547195+0000","last_clean":"2024-09-10T21:29:40.547195+0000","last_became_active":"2024-09-10T21:29:39.544011+0000","last_became_peered":"2024-09-10T21:29:39.544011+0000","last_unstale":"2024-09-10T21:29:40.547195+0000","last_undegraded":"2024-09-10T21:29:40.547195+0000","last_fullsized":"2024-09-10T21:29:40.547195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:05.999146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768535+0000","last_change":"2024-09-10T21:29:38.544149+0000","last_active":"2024-09-10T21:29:40.768535+0000","last_peered":"2024-09-10T21:29:40.768535+0000","last_clean":"2024-09-10T21:29:40.768535+0000","last_became_active":"2024-09-10T21:29:38.543790+0000","last_became_peered":"2024-09-10T21:29:38.543790+0000","last_unstale":"2024-09-10T21:29:40.768535+0000","last_undegraded":"2024-09-10T21:29:40.768535+0000","last_fullsized":"2024-09-10T21:29:40.768535+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:28.591046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.770023+0000","last_change":"2024-09-10T21:29:39.545983+0000","last_active":"2024-09-10T21:29:40.770023+0000","last_peered":"2024-09-10T21:29:40.770023+0000","last_clean":"2024-09-10T21:29:40.770023+0000","last_became_active":"2024-09-10T21:29:39.545699+0000","last_became_peered":"2024-09-10T21:29:39.545699+0000","last_unstale":"2024-09-10T21:29:40.770023+0000","last_undegraded":"2024-09-10T21:29:40.770023+0000","last_fullsized":"2024-09-10T21:29:40.770023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.110951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768649+0000","last_change":"2024-09-10T21:29:38.546518+0000","last_active":"2024-09-10T21:29:40.768649+0000","last_peered":"2024-09-10T21:29:40.768649+0000","last_clean":"2024-09-10T21:29:40.768649+0000","last_became_active":"2024-09-10T21:29:38.546262+0000","last_became_peered":"2024-09-10T21:29:38.546262+0000","last_unstale":"2024-09-10T21:29:40.768649+0000","last_undegraded":"2024-09-10T21:29:40.768649+0000","last_fullsized":"2024-09-10T21:29:40.768649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544409+0000","last_change":"2024-09-10T21:29:39.554112+0000","last_active":"2024-09-10T21:29:40.544409+0000","last_peered":"2024-09-10T21:29:40.544409+0000","last_clean":"2024-09-10T21:29:40.544409+0000","last_became_active":"2024-09-10T21:29:39.552077+0000","last_became_peered":"2024-09-10T21:29:39.552077+0000","last_unstale":"2024-09-10T21:29:40.544409+0000","last_undegraded":"2024-09-10T21:29:40.544409+0000","last_fullsized":"2024-09-10T21:29:40.544409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:18.545317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769044+0000","last_change":"2024-09-10T21:29:38.544002+0000","last_active":"2024-09-10T21:29:40.769044+0000","last_peered":"2024-09-10T21:29:40.769044+0000","last_clean":"2024-09-10T21:29:40.769044+0000","last_became_active":"2024-09-10T21:29:38.543699+0000","last_became_peered":"2024-09-10T21:29:38.543699+0000","last_unstale":"2024-09-10T21:29:40.769044+0000","last_undegraded":"2024-09-10T21:29:40.769044+0000","last_fullsized":"2024-09-10T21:29:40.769044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:21.454494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768948+0000","last_change":"2024-09-10T21:29:39.558189+0000","last_active":"2024-09-10T21:29:40.768948+0000","last_peered":"2024-09-10T21:29:40.768948+0000","last_clean":"2024-09-10T21:29:40.768948+0000","last_became_active":"2024-09-10T21:29:39.557549+0000","last_became_peered":"2024-09-10T21:29:39.557549+0000","last_unstale":"2024-09-10T21:29:40.768948+0000","last_undegraded":"2024-09-10T21:29:40.768948+0000","last_fullsized":"2024-09-10T21:29:40.768948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:48.581722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547028+0000","last_change":"2024-09-10T21:29:38.544536+0000","last_active":"2024-09-10T21:29:40.547028+0000","last_peered":"2024-09-10T21:29:40.547028+0000","last_clean":"2024-09-10T21:29:40.547028+0000","last_became_active":"2024-09-10T21:29:38.544212+0000","last_became_peered":"2024-09-10T21:29:38.544212+0000","last_unstale":"2024-09-10T21:29:40.547028+0000","last_undegraded":"2024-09-10T21:29:40.547028+0000","last_fullsized":"2024-09-10T21:29:40.547028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:53.514589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769300+0000","last_change":"2024-09-10T21:29:39.546810+0000","last_active":"2024-09-10T21:29:40.769300+0000","last_peered":"2024-09-10T21:29:40.769300+0000","last_clean":"2024-09-10T21:29:40.769300+0000","last_became_active":"2024-09-10T21:29:39.546317+0000","last_became_peered":"2024-09-10T21:29:39.546317+0000","last_unstale":"2024-09-10T21:29:40.769300+0000","last_undegraded":"2024-09-10T21:29:40.769300+0000","last_fullsized":"2024-09-10T21:29:40.769300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:46.343210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768987+0000","last_change":"2024-09-10T21:29:38.533647+0000","last_active":"2024-09-10T21:29:40.768987+0000","last_peered":"2024-09-10T21:29:40.768987+0000","last_clean":"2024-09-10T21:29:40.768987+0000","last_became_active":"2024-09-10T21:29:38.533361+0000","last_became_peered":"2024-09-10T21:29:38.533361+0000","last_unstale":"2024-09-10T21:29:40.768987+0000","last_undegraded":"2024-09-10T21:29:40.768987+0000","last_fullsized":"2024-09-10T21:29:40.768987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:32.782811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768884+0000","last_change":"2024-09-10T21:29:39.539314+0000","last_active":"2024-09-10T21:29:40.768884+0000","last_peered":"2024-09-10T21:29:40.768884+0000","last_clean":"2024-09-10T21:29:40.768884+0000","last_became_active":"2024-09-10T21:29:39.539015+0000","last_became_peered":"2024-09-10T21:29:39.539015+0000","last_unstale":"2024-09-10T21:29:40.768884+0000","last_undegraded":"2024-09-10T21:29:40.768884+0000","last_fullsized":"2024-09-10T21:29:40.768884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:19.770402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544676+0000","last_change":"2024-09-10T21:29:38.547087+0000","last_active":"2024-09-10T21:29:40.544676+0000","last_peered":"2024-09-10T21:29:40.544676+0000","last_clean":"2024-09-10T21:29:40.544676+0000","last_became_active":"2024-09-10T21:29:38.546829+0000","last_became_peered":"2024-09-10T21:29:38.546829+0000","last_unstale":"2024-09-10T21:29:40.544676+0000","last_undegraded":"2024-09-10T21:29:40.544676+0000","last_fullsized":"2024-09-10T21:29:40.544676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769511+0000","last_change":"2024-09-10T21:29:39.550392+0000","last_active":"2024-09-10T21:29:40.769511+0000","last_peered":"2024-09-10T21:29:40.769511+0000","last_clean":"2024-09-10T21:29:40.769511+0000","last_became_active":"2024-09-10T21:29:39.550131+0000","last_became_peered":"2024-09-10T21:29:39.550131+0000","last_unstale":"2024-09-10T21:29:40.769511+0000","last_undegraded":"2024-09-10T21:29:40.769511+0000","last_fullsized":"2024-09-10T21:29:40.769511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:15.885819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769553+0000","last_change":"2024-09-10T21:29:38.549285+0000","last_active":"2024-09-10T21:29:40.769553+0000","last_peered":"2024-09-10T21:29:40.769553+0000","last_clean":"2024-09-10T21:29:40.769553+0000","last_became_active":"2024-09-10T21:29:38.549014+0000","last_became_peered":"2024-09-10T21:29:38.549014+0000","last_unstale":"2024-09-10T21:29:40.769553+0000","last_undegraded":"2024-09-10T21:29:40.769553+0000","last_fullsized":"2024-09-10T21:29:40.769553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:58.754314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544681+0000","last_change":"2024-09-10T21:29:39.562581+0000","last_active":"2024-09-10T21:29:40.544681+0000","last_peered":"2024-09-10T21:29:40.544681+0000","last_clean":"2024-09-10T21:29:40.544681+0000","last_became_active":"2024-09-10T21:29:39.562287+0000","last_became_peered":"2024-09-10T21:29:39.562287+0000","last_unstale":"2024-09-10T21:29:40.544681+0000","last_undegraded":"2024-09-10T21:29:40.544681+0000","last_fullsized":"2024-09-10T21:29:40.544681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545978+0000","last_change":"2024-09-10T21:29:38.546345+0000","last_active":"2024-09-10T21:29:40.545978+0000","last_peered":"2024-09-10T21:29:40.545978+0000","last_clean":"2024-09-10T21:29:40.545978+0000","last_became_active":"2024-09-10T21:29:38.546094+0000","last_became_peered":"2024-09-10T21:29:38.546094+0000","last_unstale":"2024-09-10T21:29:40.545978+0000","last_undegraded":"2024-09-10T21:29:40.545978+0000","last_fullsized":"2024-09-10T21:29:40.545978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769644+0000","last_change":"2024-09-10T21:29:39.558347+0000","last_active":"2024-09-10T21:29:40.769644+0000","last_peered":"2024-09-10T21:29:40.769644+0000","last_clean":"2024-09-10T21:29:40.769644+0000","last_became_active":"2024-09-10T21:29:39.557760+0000","last_became_peered":"2024-09-10T21:29:39.557760+0000","last_unstale":"2024-09-10T21:29:40.769644+0000","last_undegraded":"2024-09-10T21:29:40.769644+0000","last_fullsized":"2024-09-10T21:29:40.769644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:06.975636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544238+0000","last_change":"2024-09-10T21:29:38.535077+0000","last_active":"2024-09-10T21:29:40.544238+0000","last_peered":"2024-09-10T21:29:40.544238+0000","last_clean":"2024-09-10T21:29:40.544238+0000","last_became_active":"2024-09-10T21:29:38.534747+0000","last_became_peered":"2024-09-10T21:29:38.534747+0000","last_unstale":"2024-09-10T21:29:40.544238+0000","last_undegraded":"2024-09-10T21:29:40.544238+0000","last_fullsized":"2024-09-10T21:29:40.544238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547454+0000","last_change":"2024-09-10T21:29:39.548356+0000","last_active":"2024-09-10T21:29:40.547454+0000","last_peered":"2024-09-10T21:29:40.547454+0000","last_clean":"2024-09-10T21:29:40.547454+0000","last_became_active":"2024-09-10T21:29:39.548096+0000","last_became_peered":"2024-09-10T21:29:39.548096+0000","last_unstale":"2024-09-10T21:29:40.547454+0000","last_undegraded":"2024-09-10T21:29:40.547454+0000","last_fullsized":"2024-09-10T21:29:40.547454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:50.836505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768672+0000","last_change":"2024-09-10T21:29:38.544063+0000","last_active":"2024-09-10T21:29:40.768672+0000","last_peered":"2024-09-10T21:29:40.768672+0000","last_clean":"2024-09-10T21:29:40.768672+0000","last_became_active":"2024-09-10T21:29:38.543508+0000","last_became_peered":"2024-09-10T21:29:38.543508+0000","last_unstale":"2024-09-10T21:29:40.768672+0000","last_undegraded":"2024-09-10T21:29:40.768672+0000","last_fullsized":"2024-09-10T21:29:40.768672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:02.927923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769972+0000","last_change":"2024-09-10T21:29:39.552135+0000","last_active":"2024-09-10T21:29:40.769972+0000","last_peered":"2024-09-10T21:29:40.769972+0000","last_clean":"2024-09-10T21:29:40.769972+0000","last_became_active":"2024-09-10T21:29:39.551649+0000","last_became_peered":"2024-09-10T21:29:39.551649+0000","last_unstale":"2024-09-10T21:29:40.769972+0000","last_undegraded":"2024-09-10T21:29:40.769972+0000","last_fullsized":"2024-09-10T21:29:40.769972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:26.827601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545785+0000","last_change":"2024-09-10T21:29:38.547451+0000","last_active":"2024-09-10T21:29:40.545785+0000","last_peered":"2024-09-10T21:29:40.545785+0000","last_clean":"2024-09-10T21:29:40.545785+0000","last_became_active":"2024-09-10T21:29:38.546360+0000","last_became_peered":"2024-09-10T21:29:38.546360+0000","last_unstale":"2024-09-10T21:29:40.545785+0000","last_undegraded":"2024-09-10T21:29:40.545785+0000","last_fullsized":"2024-09-10T21:29:40.545785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768220+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:29:40.768220+0000","last_peered":"2024-09-10T21:29:40.768220+0000","last_clean":"2024-09-10T21:29:40.768220+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:29:40.768220+0000","last_undegraded":"2024-09-10T21:29:40.768220+0000","last_fullsized":"2024-09-10T21:29:40.768220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":458807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":1048593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":453159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":458135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1605632,"data_stored":1050149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:29:44.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:29:44.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- 172.21.15.97:0/2120090727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 msgr2=0x7fa4cc072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 --2- 172.21.15.97:0/2120090727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc072a80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc154a40 tx=0x7fa4c401cb30 comp rx=0 tx=0).stop 2024-09-10T21:29:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- 172.21.15.97:0/2120090727 shutdown_connections 2024-09-10T21:29:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 --2- 172.21.15.97:0/2120090727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc072a80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 --2- 172.21.15.97:0/2120090727 >> v2:172.21.15.194:3300/0 conn(0x7fa4cc070720 0x7fa4cc06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- 172.21.15.97:0/2120090727 >> 172.21.15.97:0/2120090727 conn(0x7fa4cc0b1780 msgr2=0x7fa4cc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- 172.21.15.97:0/2120090727 shutdown_connections 2024-09-10T21:29:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- 172.21.15.97:0/2120090727 wait complete. 2024-09-10T21:29:44.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 Processor -- start 2024-09-10T21:29:44.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- start start 2024-09-10T21:29:44.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4cc070720 0x7fa4cc152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc1fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4cc15b110 0x7fa4cc1415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc167830 con 0x7fa4cc072630 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc1676b0 con 0x7fa4cc15b110 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d3600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc1679b0 con 0x7fa4cc070720 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc1fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc1fb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35156/0 (socket says 172.21.15.97:35156) 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 -- 172.21.15.97:0/3319749202 learned_addr learned my addr 172.21.15.97:0/3319749202 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 -- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4cc070720 msgr2=0x7fa4cc152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4cc070720 0x7fa4cc152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 -- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4cc15b110 msgr2=0x7fa4cc1415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d1600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4cc15b110 0x7fa4cc1415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:44.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4cc15b110 0x7fa4cc1415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 -- 172.21.15.97:0/3319749202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc151650 con 0x7fa4cc072630 2024-09-10T21:29:44.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4d1600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4cc15b110 0x7fa4cc1415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:44.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.613+0000 7fa4c3e00640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc1fb030 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400c970 tx=0x7fa4b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:44.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4c9600640 1 -- 172.21.15.97:0/3319749202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4007c70 con 0x7fa4cc072630 2024-09-10T21:29:44.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4c9600640 1 -- 172.21.15.97:0/3319749202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4b400f450 con 0x7fa4cc072630 2024-09-10T21:29:44.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc1fb570 con 0x7fa4cc072630 2024-09-10T21:29:44.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4c9600640 1 -- 172.21.15.97:0/3319749202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b400f8a0 con 0x7fa4cc072630 2024-09-10T21:29:44.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc151b90 con 0x7fa4cc072630 2024-09-10T21:29:44.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc06e3b0 con 0x7fa4cc072630 2024-09-10T21:29:44.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4c9600640 1 -- 172.21.15.97:0/3319749202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa4b4020030 con 0x7fa4cc072630 2024-09-10T21:29:44.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4c9600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4a8072610 0x7fa4a8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:44.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4c9600640 1 -- 172.21.15.97:0/3319749202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fa4b4093930 con 0x7fa4cc072630 2024-09-10T21:29:44.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.617+0000 7fa4d0c00640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4a8072610 0x7fa4a8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:44.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.621+0000 7fa4d0c00640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4a8072610 0x7fa4a8074ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc007fd0 tx=0x7fa4bc007940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:44.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.621+0000 7fa4c9600640 1 -- 172.21.15.97:0/3319749202 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa4cc1fb570 con 0x7fa4cc072630 2024-09-10T21:29:44.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.845+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa4cc0cf1c0 con 0x7fa4cc072630 2024-09-10T21:29:44.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.845+0000 7fa4c9600640 1 -- 172.21.15.97:0/3319749202 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1056 (secure 0 0 0) 0x7fa4cc151650 con 0x7fa4cc072630 2024-09-10T21:29:44.850 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:44.850 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":358,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":8,"num_up_osds":8,"osd_up_since":1726003441,"num_in_osds":8,"osd_in_since":1726003430,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":247681024,"bytes_avail":772846432256,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":70,"btime":"2024-09-10T21:29:42:550041+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":6504}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T21:28:50.142056+0000","services":{}},"progress_events":{}} 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4a8072610 msgr2=0x7fa4a8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4a8072610 0x7fa4a8074ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc007fd0 tx=0x7fa4bc007940 comp rx=0 tx=0).stop 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 msgr2=0x7fa4cc1fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc1fb030 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400c970 tx=0x7fa4b400ce30 comp rx=0 tx=0).stop 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 shutdown_connections 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4a8072610 0x7fa4a8074ad0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4cc15b110 0x7fa4cc1415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4cc072630 0x7fa4cc1fb030 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 --2- 172.21.15.97:0/3319749202 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4cc070720 0x7fa4cc152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 >> 172.21.15.97:0/3319749202 conn(0x7fa4cc0b1780 msgr2=0x7fa4cc153560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 shutdown_connections 2024-09-10T21:29:44.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:44.849+0000 7fa4d3600640 1 -- 172.21.15.97:0/3319749202 wait complete. 2024-09-10T21:29:44.870 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:29:44.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:29:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- 172.21.15.97:0/640481943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb00ce9d0 msgr2=0x7fdfb015f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/640481943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb00ce9d0 0x7fdfb015f6d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0074ae0 tx=0x7fdf9801cd80 comp rx=0 tx=0).stop 2024-09-10T21:29:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- 172.21.15.97:0/640481943 shutdown_connections 2024-09-10T21:29:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/640481943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb00ce9d0 0x7fdfb015f6d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/640481943 >> v2:172.21.15.194:3300/0 conn(0x7fdfb00ce5f0 0x7fdfb0141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- 172.21.15.97:0/640481943 >> 172.21.15.97:0/640481943 conn(0x7fdfb00b1780 msgr2=0x7fdfb00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- 172.21.15.97:0/640481943 shutdown_connections 2024-09-10T21:29:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- 172.21.15.97:0/640481943 wait complete. 2024-09-10T21:29:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 Processor -- start 2024-09-10T21:29:45.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- start start 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfb00ce5f0 0x7fdfb01f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfb00ce9d0 0x7fdfb01f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb015fc10 0x7fdfb00b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0169110 con 0x7fdfb015fc10 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0168f90 con 0x7fdfb00ce5f0 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfb4e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb0169290 con 0x7fdfb00ce9d0 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfae800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb015fc10 0x7fdfb00b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfae800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb015fc10 0x7fdfb00b33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35174/0 (socket says 172.21.15.97:35174) 2024-09-10T21:29:45.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfae800640 1 -- 172.21.15.97:0/3460409305 learned_addr learned my addr 172.21.15.97:0/3460409305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.009+0000 7fdfade00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfb00ce5f0 0x7fdfb01f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfa5400640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfb00ce9d0 0x7fdfb01f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfae800640 1 -- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfb00ce9d0 msgr2=0x7fdfb01f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfae800640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfb00ce9d0 0x7fdfb01f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfae800640 1 -- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfb00ce5f0 msgr2=0x7fdfb01f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfae800640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfb00ce5f0 0x7fdfb01f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfae800640 1 -- 172.21.15.97:0/3460409305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb01f8fd0 con 0x7fdfb015fc10 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfade00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfb00ce5f0 0x7fdfb01f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfae800640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb015fc10 0x7fdfb00b33f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800b410 tx=0x7fdfa800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfa6a00640 1 -- 172.21.15.97:0/3460409305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8015020 con 0x7fdfb015fc10 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfa6a00640 1 -- 172.21.15.97:0/3460409305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa8011960 con 0x7fdfb015fc10 2024-09-10T21:29:45.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfa6a00640 1 -- 172.21.15.97:0/3460409305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8011db0 con 0x7fdfb015fc10 2024-09-10T21:29:45.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb015e520 con 0x7fdfb015fc10 2024-09-10T21:29:45.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb015eae0 con 0x7fdfb015fc10 2024-09-10T21:29:45.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb0141d20 con 0x7fdfb015fc10 2024-09-10T21:29:45.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfa6a00640 1 -- 172.21.15.97:0/3460409305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdfa80193b0 con 0x7fdfb015fc10 2024-09-10T21:29:45.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.013+0000 7fdfa6a00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf8c0725e0 0x7fdf8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.017+0000 7fdfa6a00640 1 -- 172.21.15.97:0/3460409305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fdfa8093260 con 0x7fdfb015fc10 2024-09-10T21:29:45.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.017+0000 7fdfade00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf8c0725e0 0x7fdf8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.017+0000 7fdfade00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf8c0725e0 0x7fdf8c074aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0154740 tx=0x7fdfa0005f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:45.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.017+0000 7fdfa6a00640 1 -- 172.21.15.97:0/3460409305 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdfa801f030 con 0x7fdfb015fc10 2024-09-10T21:29:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.181+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfb0141360 con 0x7fdf8c0725e0 2024-09-10T21:29:45.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfa6a00640 1 -- 172.21.15.97:0/3460409305 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252061 (secure 0 0 0) 0x7fdfb0141360 con 0x7fdf8c0725e0 2024-09-10T21:29:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:45.192 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:29:45.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf8c0725e0 msgr2=0x7fdf8c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf8c0725e0 0x7fdf8c074aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0154740 tx=0x7fdfa0005f50 comp rx=0 tx=0).stop 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb015fc10 msgr2=0x7fdfb00b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb015fc10 0x7fdfb00b33f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800b410 tx=0x7fdfa800b8d0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 shutdown_connections 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf8c0725e0 0x7fdf8c074aa0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb015fc10 0x7fdfb00b33f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfb00ce9d0 0x7fdfb01f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 --2- 172.21.15.97:0/3460409305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfb00ce5f0 0x7fdfb01f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 >> 172.21.15.97:0/3460409305 conn(0x7fdfb00b1780 msgr2=0x7fdfb013d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 shutdown_connections 2024-09-10T21:29:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:45.189+0000 7fdfb4e00640 1 -- 172.21.15.97:0/3460409305 wait complete. 2024-09-10T21:29:45.211 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-10T21:29:44.156157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241876,"kb_used_data":8988,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754732844,"statfs":{"total":773094113280,"available":772846432256,"internally_reserved":0,"allocated":9203712,"data_stored":4843944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001568"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768894+0000","last_change":"2024-09-10T21:29:39.539326+0000","last_active":"2024-09-10T21:29:40.768894+0000","last_peered":"2024-09-10T21:29:40.768894+0000","last_clean":"2024-09-10T21:29:40.768894+0000","last_became_active":"2024-09-10T21:29:39.539056+0000","last_became_peered":"2024-09-10T21:29:39.539056+0000","last_unstale":"2024-09-10T21:29:40.768894+0000","last_undegraded":"2024-09-10T21:29:40.768894+0000","last_fullsized":"2024-09-10T21:29:40.768894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:25.767146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769190+0000","last_change":"2024-09-10T21:29:39.564404+0000","last_active":"2024-09-10T21:29:40.769190+0000","last_peered":"2024-09-10T21:29:40.769190+0000","last_clean":"2024-09-10T21:29:40.769190+0000","last_became_active":"2024-09-10T21:29:39.563306+0000","last_became_peered":"2024-09-10T21:29:39.563306+0000","last_unstale":"2024-09-10T21:29:40.769190+0000","last_undegraded":"2024-09-10T21:29:40.769190+0000","last_fullsized":"2024-09-10T21:29:40.769190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:17.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769401+0000","last_change":"2024-09-10T21:29:39.561606+0000","last_active":"2024-09-10T21:29:40.769401+0000","last_peered":"2024-09-10T21:29:40.769401+0000","last_clean":"2024-09-10T21:29:40.769401+0000","last_became_active":"2024-09-10T21:29:39.560972+0000","last_became_peered":"2024-09-10T21:29:39.560972+0000","last_unstale":"2024-09-10T21:29:40.769401+0000","last_undegraded":"2024-09-10T21:29:40.769401+0000","last_fullsized":"2024-09-10T21:29:40.769401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:26.701898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544576+0000","last_change":"2024-09-10T21:29:39.537773+0000","last_active":"2024-09-10T21:29:40.544576+0000","last_peered":"2024-09-10T21:29:40.544576+0000","last_clean":"2024-09-10T21:29:40.544576+0000","last_became_active":"2024-09-10T21:29:39.537487+0000","last_became_peered":"2024-09-10T21:29:39.537487+0000","last_unstale":"2024-09-10T21:29:40.544576+0000","last_undegraded":"2024-09-10T21:29:40.544576+0000","last_fullsized":"2024-09-10T21:29:40.544576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:53.105160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546947+0000","last_change":"2024-09-10T21:29:39.546644+0000","last_active":"2024-09-10T21:29:40.546947+0000","last_peered":"2024-09-10T21:29:40.546947+0000","last_clean":"2024-09-10T21:29:40.546947+0000","last_became_active":"2024-09-10T21:29:39.546373+0000","last_became_peered":"2024-09-10T21:29:39.546373+0000","last_unstale":"2024-09-10T21:29:40.546947+0000","last_undegraded":"2024-09-10T21:29:40.546947+0000","last_fullsized":"2024-09-10T21:29:40.546947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.942938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546101+0000","last_change":"2024-09-10T21:29:39.558707+0000","last_active":"2024-09-10T21:29:40.546101+0000","last_peered":"2024-09-10T21:29:40.546101+0000","last_clean":"2024-09-10T21:29:40.546101+0000","last_became_active":"2024-09-10T21:29:39.554793+0000","last_became_peered":"2024-09-10T21:29:39.554793+0000","last_unstale":"2024-09-10T21:29:40.546101+0000","last_undegraded":"2024-09-10T21:29:40.546101+0000","last_fullsized":"2024-09-10T21:29:40.546101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544328+0000","last_change":"2024-09-10T21:29:39.540653+0000","last_active":"2024-09-10T21:29:40.544328+0000","last_peered":"2024-09-10T21:29:40.544328+0000","last_clean":"2024-09-10T21:29:40.544328+0000","last_became_active":"2024-09-10T21:29:39.540400+0000","last_became_peered":"2024-09-10T21:29:39.540400+0000","last_unstale":"2024-09-10T21:29:40.544328+0000","last_undegraded":"2024-09-10T21:29:40.544328+0000","last_fullsized":"2024-09-10T21:29:40.544328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:26.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768468+0000","last_change":"2024-09-10T21:29:39.535592+0000","last_active":"2024-09-10T21:29:40.768468+0000","last_peered":"2024-09-10T21:29:40.768468+0000","last_clean":"2024-09-10T21:29:40.768468+0000","last_became_active":"2024-09-10T21:29:39.535236+0000","last_became_peered":"2024-09-10T21:29:39.535236+0000","last_unstale":"2024-09-10T21:29:40.768468+0000","last_undegraded":"2024-09-10T21:29:40.768468+0000","last_fullsized":"2024-09-10T21:29:40.768468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:54.259155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545150+0000","last_change":"2024-09-10T21:29:39.556621+0000","last_active":"2024-09-10T21:29:40.545150+0000","last_peered":"2024-09-10T21:29:40.545150+0000","last_clean":"2024-09-10T21:29:40.545150+0000","last_became_active":"2024-09-10T21:29:39.555001+0000","last_became_peered":"2024-09-10T21:29:39.555001+0000","last_unstale":"2024-09-10T21:29:40.545150+0000","last_undegraded":"2024-09-10T21:29:40.545150+0000","last_fullsized":"2024-09-10T21:29:40.545150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769242+0000","last_change":"2024-09-10T21:29:39.536232+0000","last_active":"2024-09-10T21:29:40.769242+0000","last_peered":"2024-09-10T21:29:40.769242+0000","last_clean":"2024-09-10T21:29:40.769242+0000","last_became_active":"2024-09-10T21:29:39.535978+0000","last_became_peered":"2024-09-10T21:29:39.535978+0000","last_unstale":"2024-09-10T21:29:40.769242+0000","last_undegraded":"2024-09-10T21:29:40.769242+0000","last_fullsized":"2024-09-10T21:29:40.769242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:49.789897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769661+0000","last_change":"2024-09-10T21:29:39.562255+0000","last_active":"2024-09-10T21:29:40.769661+0000","last_peered":"2024-09-10T21:29:40.769661+0000","last_clean":"2024-09-10T21:29:40.769661+0000","last_became_active":"2024-09-10T21:29:39.559040+0000","last_became_peered":"2024-09-10T21:29:39.559040+0000","last_unstale":"2024-09-10T21:29:40.769661+0000","last_undegraded":"2024-09-10T21:29:40.769661+0000","last_fullsized":"2024-09-10T21:29:40.769661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:39.446682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769690+0000","last_change":"2024-09-10T21:29:39.550940+0000","last_active":"2024-09-10T21:29:40.769690+0000","last_peered":"2024-09-10T21:29:40.769690+0000","last_clean":"2024-09-10T21:29:40.769690+0000","last_became_active":"2024-09-10T21:29:39.550666+0000","last_became_peered":"2024-09-10T21:29:39.550666+0000","last_unstale":"2024-09-10T21:29:40.769690+0000","last_undegraded":"2024-09-10T21:29:40.769690+0000","last_fullsized":"2024-09-10T21:29:40.769690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:24.960891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768575+0000","last_change":"2024-09-10T21:29:39.555356+0000","last_active":"2024-09-10T21:29:40.768575+0000","last_peered":"2024-09-10T21:29:40.768575+0000","last_clean":"2024-09-10T21:29:40.768575+0000","last_became_active":"2024-09-10T21:29:39.554843+0000","last_became_peered":"2024-09-10T21:29:39.554843+0000","last_unstale":"2024-09-10T21:29:40.768575+0000","last_undegraded":"2024-09-10T21:29:40.768575+0000","last_fullsized":"2024-09-10T21:29:40.768575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:03.103988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769008+0000","last_change":"2024-09-10T21:29:39.531161+0000","last_active":"2024-09-10T21:29:40.769008+0000","last_peered":"2024-09-10T21:29:40.769008+0000","last_clean":"2024-09-10T21:29:40.769008+0000","last_became_active":"2024-09-10T21:29:39.530873+0000","last_became_peered":"2024-09-10T21:29:39.530873+0000","last_unstale":"2024-09-10T21:29:40.769008+0000","last_undegraded":"2024-09-10T21:29:40.769008+0000","last_fullsized":"2024-09-10T21:29:40.769008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:25.754412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546185+0000","last_change":"2024-09-10T21:29:39.563879+0000","last_active":"2024-09-10T21:29:40.546185+0000","last_peered":"2024-09-10T21:29:40.546185+0000","last_clean":"2024-09-10T21:29:40.546185+0000","last_became_active":"2024-09-10T21:29:39.563673+0000","last_became_peered":"2024-09-10T21:29:39.563673+0000","last_unstale":"2024-09-10T21:29:40.546185+0000","last_undegraded":"2024-09-10T21:29:40.546185+0000","last_fullsized":"2024-09-10T21:29:40.546185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:33.785842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544217+0000","last_change":"2024-09-10T21:29:39.552730+0000","last_active":"2024-09-10T21:29:40.544217+0000","last_peered":"2024-09-10T21:29:40.544217+0000","last_clean":"2024-09-10T21:29:40.544217+0000","last_became_active":"2024-09-10T21:29:39.552478+0000","last_became_peered":"2024-09-10T21:29:39.552478+0000","last_unstale":"2024-09-10T21:29:40.544217+0000","last_undegraded":"2024-09-10T21:29:40.544217+0000","last_fullsized":"2024-09-10T21:29:40.544217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:00.008953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768952+0000","last_change":"2024-09-10T21:29:39.541920+0000","last_active":"2024-09-10T21:29:40.768952+0000","last_peered":"2024-09-10T21:29:40.768952+0000","last_clean":"2024-09-10T21:29:40.768952+0000","last_became_active":"2024-09-10T21:29:39.541604+0000","last_became_peered":"2024-09-10T21:29:39.541604+0000","last_unstale":"2024-09-10T21:29:40.768952+0000","last_undegraded":"2024-09-10T21:29:40.768952+0000","last_fullsized":"2024-09-10T21:29:40.768952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:24.814305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769429+0000","last_change":"2024-09-10T21:29:39.558608+0000","last_active":"2024-09-10T21:29:40.769429+0000","last_peered":"2024-09-10T21:29:40.769429+0000","last_clean":"2024-09-10T21:29:40.769429+0000","last_became_active":"2024-09-10T21:29:39.558086+0000","last_became_peered":"2024-09-10T21:29:39.558086+0000","last_unstale":"2024-09-10T21:29:40.769429+0000","last_undegraded":"2024-09-10T21:29:40.769429+0000","last_fullsized":"2024-09-10T21:29:40.769429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:51.432263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769280+0000","last_change":"2024-09-10T21:29:39.539085+0000","last_active":"2024-09-10T21:29:40.769280+0000","last_peered":"2024-09-10T21:29:40.769280+0000","last_clean":"2024-09-10T21:29:40.769280+0000","last_became_active":"2024-09-10T21:29:39.536918+0000","last_became_peered":"2024-09-10T21:29:39.536918+0000","last_unstale":"2024-09-10T21:29:40.769280+0000","last_undegraded":"2024-09-10T21:29:40.769280+0000","last_fullsized":"2024-09-10T21:29:40.769280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:56.499531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546462+0000","last_change":"2024-09-10T21:29:39.560153+0000","last_active":"2024-09-10T21:29:40.546462+0000","last_peered":"2024-09-10T21:29:40.546462+0000","last_clean":"2024-09-10T21:29:40.546462+0000","last_became_active":"2024-09-10T21:29:39.559886+0000","last_became_peered":"2024-09-10T21:29:39.559886+0000","last_unstale":"2024-09-10T21:29:40.546462+0000","last_undegraded":"2024-09-10T21:29:40.546462+0000","last_fullsized":"2024-09-10T21:29:40.546462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768980+0000","last_change":"2024-09-10T21:29:39.547565+0000","last_active":"2024-09-10T21:29:40.768980+0000","last_peered":"2024-09-10T21:29:40.768980+0000","last_clean":"2024-09-10T21:29:40.768980+0000","last_became_active":"2024-09-10T21:29:39.547315+0000","last_became_peered":"2024-09-10T21:29:39.547315+0000","last_unstale":"2024-09-10T21:29:40.768980+0000","last_undegraded":"2024-09-10T21:29:40.768980+0000","last_fullsized":"2024-09-10T21:29:40.768980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:05.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769033+0000","last_change":"2024-09-10T21:29:39.562781+0000","last_active":"2024-09-10T21:29:40.769033+0000","last_peered":"2024-09-10T21:29:40.769033+0000","last_clean":"2024-09-10T21:29:40.769033+0000","last_became_active":"2024-09-10T21:29:39.562509+0000","last_became_peered":"2024-09-10T21:29:39.562509+0000","last_unstale":"2024-09-10T21:29:40.769033+0000","last_undegraded":"2024-09-10T21:29:40.769033+0000","last_fullsized":"2024-09-10T21:29:40.769033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:53.170104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768742+0000","last_change":"2024-09-10T21:29:39.540989+0000","last_active":"2024-09-10T21:29:40.768742+0000","last_peered":"2024-09-10T21:29:40.768742+0000","last_clean":"2024-09-10T21:29:40.768742+0000","last_became_active":"2024-09-10T21:29:39.540686+0000","last_became_peered":"2024-09-10T21:29:39.540686+0000","last_unstale":"2024-09-10T21:29:40.768742+0000","last_undegraded":"2024-09-10T21:29:40.768742+0000","last_fullsized":"2024-09-10T21:29:40.768742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:04.647939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768432+0000","last_change":"2024-09-10T21:29:39.555697+0000","last_active":"2024-09-10T21:29:40.768432+0000","last_peered":"2024-09-10T21:29:40.768432+0000","last_clean":"2024-09-10T21:29:40.768432+0000","last_became_active":"2024-09-10T21:29:39.555321+0000","last_became_peered":"2024-09-10T21:29:39.555321+0000","last_unstale":"2024-09-10T21:29:40.768432+0000","last_undegraded":"2024-09-10T21:29:40.768432+0000","last_fullsized":"2024-09-10T21:29:40.768432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:26.937688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547452+0000","last_change":"2024-09-10T21:29:39.566354+0000","last_active":"2024-09-10T21:29:40.547452+0000","last_peered":"2024-09-10T21:29:40.547452+0000","last_clean":"2024-09-10T21:29:40.547452+0000","last_became_active":"2024-09-10T21:29:39.566119+0000","last_became_peered":"2024-09-10T21:29:39.566119+0000","last_unstale":"2024-09-10T21:29:40.547452+0000","last_undegraded":"2024-09-10T21:29:40.547452+0000","last_fullsized":"2024-09-10T21:29:40.547452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544876+0000","last_change":"2024-09-10T21:29:39.555058+0000","last_active":"2024-09-10T21:29:40.544876+0000","last_peered":"2024-09-10T21:29:40.544876+0000","last_clean":"2024-09-10T21:29:40.544876+0000","last_became_active":"2024-09-10T21:29:39.554444+0000","last_became_peered":"2024-09-10T21:29:39.554444+0000","last_unstale":"2024-09-10T21:29:40.544876+0000","last_undegraded":"2024-09-10T21:29:40.544876+0000","last_fullsized":"2024-09-10T21:29:40.544876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544584+0000","last_change":"2024-09-10T21:29:39.553456+0000","last_active":"2024-09-10T21:29:40.544584+0000","last_peered":"2024-09-10T21:29:40.544584+0000","last_clean":"2024-09-10T21:29:40.544584+0000","last_became_active":"2024-09-10T21:29:39.553218+0000","last_became_peered":"2024-09-10T21:29:39.553218+0000","last_unstale":"2024-09-10T21:29:40.544584+0000","last_undegraded":"2024-09-10T21:29:40.544584+0000","last_fullsized":"2024-09-10T21:29:40.544584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547497+0000","last_change":"2024-09-10T21:29:39.551245+0000","last_active":"2024-09-10T21:29:40.547497+0000","last_peered":"2024-09-10T21:29:40.547497+0000","last_clean":"2024-09-10T21:29:40.547497+0000","last_became_active":"2024-09-10T21:29:39.550535+0000","last_became_peered":"2024-09-10T21:29:39.550535+0000","last_unstale":"2024-09-10T21:29:40.547497+0000","last_undegraded":"2024-09-10T21:29:40.547497+0000","last_fullsized":"2024-09-10T21:29:40.547497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:03.284635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545912+0000","last_change":"2024-09-10T21:29:39.558777+0000","last_active":"2024-09-10T21:29:40.545912+0000","last_peered":"2024-09-10T21:29:40.545912+0000","last_clean":"2024-09-10T21:29:40.545912+0000","last_became_active":"2024-09-10T21:29:39.555098+0000","last_became_peered":"2024-09-10T21:29:39.555098+0000","last_unstale":"2024-09-10T21:29:40.545912+0000","last_undegraded":"2024-09-10T21:29:40.545912+0000","last_fullsized":"2024-09-10T21:29:40.545912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769096+0000","last_change":"2024-09-10T21:29:39.564203+0000","last_active":"2024-09-10T21:29:40.769096+0000","last_peered":"2024-09-10T21:29:40.769096+0000","last_clean":"2024-09-10T21:29:40.769096+0000","last_became_active":"2024-09-10T21:29:39.563951+0000","last_became_peered":"2024-09-10T21:29:39.563951+0000","last_unstale":"2024-09-10T21:29:40.769096+0000","last_undegraded":"2024-09-10T21:29:40.769096+0000","last_fullsized":"2024-09-10T21:29:40.769096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:38.678603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768992+0000","last_change":"2024-09-10T21:29:39.564492+0000","last_active":"2024-09-10T21:29:40.768992+0000","last_peered":"2024-09-10T21:29:40.768992+0000","last_clean":"2024-09-10T21:29:40.768992+0000","last_became_active":"2024-09-10T21:29:39.563500+0000","last_became_peered":"2024-09-10T21:29:39.563500+0000","last_unstale":"2024-09-10T21:29:40.768992+0000","last_undegraded":"2024-09-10T21:29:40.768992+0000","last_fullsized":"2024-09-10T21:29:40.768992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:46.030917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544257+0000","last_change":"2024-09-10T21:29:39.555396+0000","last_active":"2024-09-10T21:29:40.544257+0000","last_peered":"2024-09-10T21:29:40.544257+0000","last_clean":"2024-09-10T21:29:40.544257+0000","last_became_active":"2024-09-10T21:29:39.555086+0000","last_became_peered":"2024-09-10T21:29:39.555086+0000","last_unstale":"2024-09-10T21:29:40.544257+0000","last_undegraded":"2024-09-10T21:29:40.544257+0000","last_fullsized":"2024-09-10T21:29:40.544257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:06.000345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546200+0000","last_change":"2024-09-10T21:29:39.548302+0000","last_active":"2024-09-10T21:29:40.546200+0000","last_peered":"2024-09-10T21:29:40.546200+0000","last_clean":"2024-09-10T21:29:40.546200+0000","last_became_active":"2024-09-10T21:29:39.547972+0000","last_became_peered":"2024-09-10T21:29:39.547972+0000","last_unstale":"2024-09-10T21:29:40.546200+0000","last_undegraded":"2024-09-10T21:29:40.546200+0000","last_fullsized":"2024-09-10T21:29:40.546200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:22.457526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547580+0000","last_change":"2024-09-10T21:29:38.549320+0000","last_active":"2024-09-10T21:29:40.547580+0000","last_peered":"2024-09-10T21:29:40.547580+0000","last_clean":"2024-09-10T21:29:40.547580+0000","last_became_active":"2024-09-10T21:29:38.549041+0000","last_became_peered":"2024-09-10T21:29:38.549041+0000","last_unstale":"2024-09-10T21:29:40.547580+0000","last_undegraded":"2024-09-10T21:29:40.547580+0000","last_fullsized":"2024-09-10T21:29:40.547580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:52.457273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546222+0000","last_change":"2024-09-10T21:29:39.542269+0000","last_active":"2024-09-10T21:29:40.546222+0000","last_peered":"2024-09-10T21:29:40.546222+0000","last_clean":"2024-09-10T21:29:40.546222+0000","last_became_active":"2024-09-10T21:29:39.541975+0000","last_became_peered":"2024-09-10T21:29:39.541975+0000","last_unstale":"2024-09-10T21:29:40.546222+0000","last_undegraded":"2024-09-10T21:29:40.546222+0000","last_fullsized":"2024-09-10T21:29:40.546222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:38.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769882+0000","last_change":"2024-09-10T21:29:38.544792+0000","last_active":"2024-09-10T21:29:40.769882+0000","last_peered":"2024-09-10T21:29:40.769882+0000","last_clean":"2024-09-10T21:29:40.769882+0000","last_became_active":"2024-09-10T21:29:38.544459+0000","last_became_peered":"2024-09-10T21:29:38.544459+0000","last_unstale":"2024-09-10T21:29:40.769882+0000","last_undegraded":"2024-09-10T21:29:40.769882+0000","last_fullsized":"2024-09-10T21:29:40.769882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:28.938255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769152+0000","last_change":"2024-09-10T21:29:39.546560+0000","last_active":"2024-09-10T21:29:40.769152+0000","last_peered":"2024-09-10T21:29:40.769152+0000","last_clean":"2024-09-10T21:29:40.769152+0000","last_became_active":"2024-09-10T21:29:39.545328+0000","last_became_peered":"2024-09-10T21:29:39.545328+0000","last_unstale":"2024-09-10T21:29:40.769152+0000","last_undegraded":"2024-09-10T21:29:40.769152+0000","last_fullsized":"2024-09-10T21:29:40.769152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:20.557881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768895+0000","last_change":"2024-09-10T21:29:38.538896+0000","last_active":"2024-09-10T21:29:40.768895+0000","last_peered":"2024-09-10T21:29:40.768895+0000","last_clean":"2024-09-10T21:29:40.768895+0000","last_became_active":"2024-09-10T21:29:38.538523+0000","last_became_peered":"2024-09-10T21:29:38.538523+0000","last_unstale":"2024-09-10T21:29:40.768895+0000","last_undegraded":"2024-09-10T21:29:40.768895+0000","last_fullsized":"2024-09-10T21:29:40.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:26.307429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544519+0000","last_change":"2024-09-10T21:29:39.538536+0000","last_active":"2024-09-10T21:29:40.544519+0000","last_peered":"2024-09-10T21:29:40.544519+0000","last_clean":"2024-09-10T21:29:40.544519+0000","last_became_active":"2024-09-10T21:29:39.538308+0000","last_became_peered":"2024-09-10T21:29:39.538308+0000","last_unstale":"2024-09-10T21:29:40.544519+0000","last_undegraded":"2024-09-10T21:29:40.544519+0000","last_fullsized":"2024-09-10T21:29:40.544519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:40.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769616+0000","last_change":"2024-09-10T21:29:38.536197+0000","last_active":"2024-09-10T21:29:40.769616+0000","last_peered":"2024-09-10T21:29:40.769616+0000","last_clean":"2024-09-10T21:29:40.769616+0000","last_became_active":"2024-09-10T21:29:38.535894+0000","last_became_peered":"2024-09-10T21:29:38.535894+0000","last_unstale":"2024-09-10T21:29:40.769616+0000","last_undegraded":"2024-09-10T21:29:40.769616+0000","last_fullsized":"2024-09-10T21:29:40.769616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546646+0000","last_change":"2024-09-10T21:29:39.554137+0000","last_active":"2024-09-10T21:29:40.546646+0000","last_peered":"2024-09-10T21:29:40.546646+0000","last_clean":"2024-09-10T21:29:40.546646+0000","last_became_active":"2024-09-10T21:29:39.553735+0000","last_became_peered":"2024-09-10T21:29:39.553735+0000","last_unstale":"2024-09-10T21:29:40.546646+0000","last_undegraded":"2024-09-10T21:29:40.546646+0000","last_fullsized":"2024-09-10T21:29:40.546646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769158+0000","last_change":"2024-09-10T21:29:38.535949+0000","last_active":"2024-09-10T21:29:40.769158+0000","last_peered":"2024-09-10T21:29:40.769158+0000","last_clean":"2024-09-10T21:29:40.769158+0000","last_became_active":"2024-09-10T21:29:38.535669+0000","last_became_peered":"2024-09-10T21:29:38.535669+0000","last_unstale":"2024-09-10T21:29:40.769158+0000","last_undegraded":"2024-09-10T21:29:40.769158+0000","last_fullsized":"2024-09-10T21:29:40.769158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769743+0000","last_change":"2024-09-10T21:29:39.561954+0000","last_active":"2024-09-10T21:29:40.769743+0000","last_peered":"2024-09-10T21:29:40.769743+0000","last_clean":"2024-09-10T21:29:40.769743+0000","last_became_active":"2024-09-10T21:29:39.561348+0000","last_became_peered":"2024-09-10T21:29:39.561348+0000","last_unstale":"2024-09-10T21:29:40.769743+0000","last_undegraded":"2024-09-10T21:29:40.769743+0000","last_fullsized":"2024-09-10T21:29:40.769743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:54.689918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769497+0000","last_change":"2024-09-10T21:29:38.543780+0000","last_active":"2024-09-10T21:29:40.769497+0000","last_peered":"2024-09-10T21:29:40.769497+0000","last_clean":"2024-09-10T21:29:40.769497+0000","last_became_active":"2024-09-10T21:29:38.543305+0000","last_became_peered":"2024-09-10T21:29:38.543305+0000","last_unstale":"2024-09-10T21:29:40.769497+0000","last_undegraded":"2024-09-10T21:29:40.769497+0000","last_fullsized":"2024-09-10T21:29:40.769497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.941433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768491+0000","last_change":"2024-09-10T21:29:39.549779+0000","last_active":"2024-09-10T21:29:40.768491+0000","last_peered":"2024-09-10T21:29:40.768491+0000","last_clean":"2024-09-10T21:29:40.768491+0000","last_became_active":"2024-09-10T21:29:39.549473+0000","last_became_peered":"2024-09-10T21:29:39.549473+0000","last_unstale":"2024-09-10T21:29:40.768491+0000","last_undegraded":"2024-09-10T21:29:40.768491+0000","last_fullsized":"2024-09-10T21:29:40.768491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768418+0000","last_change":"2024-09-10T21:29:38.546344+0000","last_active":"2024-09-10T21:29:40.768418+0000","last_peered":"2024-09-10T21:29:40.768418+0000","last_clean":"2024-09-10T21:29:40.768418+0000","last_became_active":"2024-09-10T21:29:38.545808+0000","last_became_peered":"2024-09-10T21:29:38.545808+0000","last_unstale":"2024-09-10T21:29:40.768418+0000","last_undegraded":"2024-09-10T21:29:40.768418+0000","last_fullsized":"2024-09-10T21:29:40.768418+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:12.358215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769992+0000","last_change":"2024-09-10T21:29:39.558263+0000","last_active":"2024-09-10T21:29:40.769992+0000","last_peered":"2024-09-10T21:29:40.769992+0000","last_clean":"2024-09-10T21:29:40.769992+0000","last_became_active":"2024-09-10T21:29:39.557990+0000","last_became_peered":"2024-09-10T21:29:39.557990+0000","last_unstale":"2024-09-10T21:29:40.769992+0000","last_undegraded":"2024-09-10T21:29:40.769992+0000","last_fullsized":"2024-09-10T21:29:40.769992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.511263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768565+0000","last_change":"2024-09-10T21:29:38.545254+0000","last_active":"2024-09-10T21:29:40.768565+0000","last_peered":"2024-09-10T21:29:40.768565+0000","last_clean":"2024-09-10T21:29:40.768565+0000","last_became_active":"2024-09-10T21:29:38.544954+0000","last_became_peered":"2024-09-10T21:29:38.544954+0000","last_unstale":"2024-09-10T21:29:40.768565+0000","last_undegraded":"2024-09-10T21:29:40.768565+0000","last_fullsized":"2024-09-10T21:29:40.768565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768528+0000","last_change":"2024-09-10T21:29:39.555635+0000","last_active":"2024-09-10T21:29:40.768528+0000","last_peered":"2024-09-10T21:29:40.768528+0000","last_clean":"2024-09-10T21:29:40.768528+0000","last_became_active":"2024-09-10T21:29:39.555011+0000","last_became_peered":"2024-09-10T21:29:39.555011+0000","last_unstale":"2024-09-10T21:29:40.768528+0000","last_undegraded":"2024-09-10T21:29:40.768528+0000","last_fullsized":"2024-09-10T21:29:40.768528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:15.429757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545012+0000","last_change":"2024-09-10T21:29:38.549092+0000","last_active":"2024-09-10T21:29:40.545012+0000","last_peered":"2024-09-10T21:29:40.545012+0000","last_clean":"2024-09-10T21:29:40.545012+0000","last_became_active":"2024-09-10T21:29:38.548800+0000","last_became_peered":"2024-09-10T21:29:38.548800+0000","last_unstale":"2024-09-10T21:29:40.545012+0000","last_undegraded":"2024-09-10T21:29:40.545012+0000","last_fullsized":"2024-09-10T21:29:40.545012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:10.710966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768928+0000","last_change":"2024-09-10T21:29:39.554166+0000","last_active":"2024-09-10T21:29:40.768928+0000","last_peered":"2024-09-10T21:29:40.768928+0000","last_clean":"2024-09-10T21:29:40.768928+0000","last_became_active":"2024-09-10T21:29:39.551856+0000","last_became_peered":"2024-09-10T21:29:39.551856+0000","last_unstale":"2024-09-10T21:29:40.768928+0000","last_undegraded":"2024-09-10T21:29:40.768928+0000","last_fullsized":"2024-09-10T21:29:40.768928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.611740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769311+0000","last_change":"2024-09-10T21:29:38.545177+0000","last_active":"2024-09-10T21:29:40.769311+0000","last_peered":"2024-09-10T21:29:40.769311+0000","last_clean":"2024-09-10T21:29:40.769311+0000","last_became_active":"2024-09-10T21:29:38.544523+0000","last_became_peered":"2024-09-10T21:29:38.544523+0000","last_unstale":"2024-09-10T21:29:40.769311+0000","last_undegraded":"2024-09-10T21:29:40.769311+0000","last_fullsized":"2024-09-10T21:29:40.769311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:36.904722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769325+0000","last_change":"2024-09-10T21:29:39.536154+0000","last_active":"2024-09-10T21:29:40.769325+0000","last_peered":"2024-09-10T21:29:40.769325+0000","last_clean":"2024-09-10T21:29:40.769325+0000","last_became_active":"2024-09-10T21:29:39.535816+0000","last_became_peered":"2024-09-10T21:29:39.535816+0000","last_unstale":"2024-09-10T21:29:40.769325+0000","last_undegraded":"2024-09-10T21:29:40.769325+0000","last_fullsized":"2024-09-10T21:29:40.769325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:41.370702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546787+0000","last_change":"2024-09-10T21:29:38.542661+0000","last_active":"2024-09-10T21:29:40.546787+0000","last_peered":"2024-09-10T21:29:40.546787+0000","last_clean":"2024-09-10T21:29:40.546787+0000","last_became_active":"2024-09-10T21:29:38.542425+0000","last_became_peered":"2024-09-10T21:29:38.542425+0000","last_unstale":"2024-09-10T21:29:40.546787+0000","last_undegraded":"2024-09-10T21:29:40.546787+0000","last_fullsized":"2024-09-10T21:29:40.546787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:47.295559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545841+0000","last_change":"2024-09-10T21:29:39.548032+0000","last_active":"2024-09-10T21:29:40.545841+0000","last_peered":"2024-09-10T21:29:40.545841+0000","last_clean":"2024-09-10T21:29:40.545841+0000","last_became_active":"2024-09-10T21:29:39.547709+0000","last_became_peered":"2024-09-10T21:29:39.547709+0000","last_unstale":"2024-09-10T21:29:40.545841+0000","last_undegraded":"2024-09-10T21:29:40.545841+0000","last_fullsized":"2024-09-10T21:29:40.545841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769782+0000","last_change":"2024-09-10T21:29:38.532633+0000","last_active":"2024-09-10T21:29:40.769782+0000","last_peered":"2024-09-10T21:29:40.769782+0000","last_clean":"2024-09-10T21:29:40.769782+0000","last_became_active":"2024-09-10T21:29:38.532371+0000","last_became_peered":"2024-09-10T21:29:38.532371+0000","last_unstale":"2024-09-10T21:29:40.769782+0000","last_undegraded":"2024-09-10T21:29:40.769782+0000","last_fullsized":"2024-09-10T21:29:40.769782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:45.933789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544716+0000","last_change":"2024-09-10T21:29:39.554909+0000","last_active":"2024-09-10T21:29:40.544716+0000","last_peered":"2024-09-10T21:29:40.544716+0000","last_clean":"2024-09-10T21:29:40.544716+0000","last_became_active":"2024-09-10T21:29:39.554651+0000","last_became_peered":"2024-09-10T21:29:39.554651+0000","last_unstale":"2024-09-10T21:29:40.544716+0000","last_undegraded":"2024-09-10T21:29:40.544716+0000","last_fullsized":"2024-09-10T21:29:40.544716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768411+0000","last_change":"2024-09-10T21:29:38.535431+0000","last_active":"2024-09-10T21:29:40.768411+0000","last_peered":"2024-09-10T21:29:40.768411+0000","last_clean":"2024-09-10T21:29:40.768411+0000","last_became_active":"2024-09-10T21:29:38.535114+0000","last_became_peered":"2024-09-10T21:29:38.535114+0000","last_unstale":"2024-09-10T21:29:40.768411+0000","last_undegraded":"2024-09-10T21:29:40.768411+0000","last_fullsized":"2024-09-10T21:29:40.768411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:05.208145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768791+0000","last_change":"2024-09-10T21:29:39.563604+0000","last_active":"2024-09-10T21:29:40.768791+0000","last_peered":"2024-09-10T21:29:40.768791+0000","last_clean":"2024-09-10T21:29:40.768791+0000","last_became_active":"2024-09-10T21:29:39.563344+0000","last_became_peered":"2024-09-10T21:29:39.563344+0000","last_unstale":"2024-09-10T21:29:40.768791+0000","last_undegraded":"2024-09-10T21:29:40.768791+0000","last_fullsized":"2024-09-10T21:29:40.768791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:50.487501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769250+0000","last_change":"2024-09-10T21:29:38.544304+0000","last_active":"2024-09-10T21:29:40.769250+0000","last_peered":"2024-09-10T21:29:40.769250+0000","last_clean":"2024-09-10T21:29:40.769250+0000","last_became_active":"2024-09-10T21:29:38.544049+0000","last_became_peered":"2024-09-10T21:29:38.544049+0000","last_unstale":"2024-09-10T21:29:40.769250+0000","last_undegraded":"2024-09-10T21:29:40.769250+0000","last_fullsized":"2024-09-10T21:29:40.769250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768692+0000","last_change":"2024-09-10T21:29:39.544825+0000","last_active":"2024-09-10T21:29:40.768692+0000","last_peered":"2024-09-10T21:29:40.768692+0000","last_clean":"2024-09-10T21:29:40.768692+0000","last_became_active":"2024-09-10T21:29:39.544578+0000","last_became_peered":"2024-09-10T21:29:39.544578+0000","last_unstale":"2024-09-10T21:29:40.768692+0000","last_undegraded":"2024-09-10T21:29:40.768692+0000","last_fullsized":"2024-09-10T21:29:40.768692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:17.043812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546585+0000","last_change":"2024-09-10T21:29:38.549765+0000","last_active":"2024-09-10T21:29:40.546585+0000","last_peered":"2024-09-10T21:29:40.546585+0000","last_clean":"2024-09-10T21:29:40.546585+0000","last_became_active":"2024-09-10T21:29:38.548990+0000","last_became_peered":"2024-09-10T21:29:38.548990+0000","last_unstale":"2024-09-10T21:29:40.546585+0000","last_undegraded":"2024-09-10T21:29:40.546585+0000","last_fullsized":"2024-09-10T21:29:40.546585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:52.559734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768695+0000","last_change":"2024-09-10T21:29:39.547930+0000","last_active":"2024-09-10T21:29:40.768695+0000","last_peered":"2024-09-10T21:29:40.768695+0000","last_clean":"2024-09-10T21:29:40.768695+0000","last_became_active":"2024-09-10T21:29:39.547672+0000","last_became_peered":"2024-09-10T21:29:39.547672+0000","last_unstale":"2024-09-10T21:29:40.768695+0000","last_undegraded":"2024-09-10T21:29:40.768695+0000","last_fullsized":"2024-09-10T21:29:40.768695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:52.513550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545358+0000","last_change":"2024-09-10T21:29:38.544208+0000","last_active":"2024-09-10T21:29:40.545358+0000","last_peered":"2024-09-10T21:29:40.545358+0000","last_clean":"2024-09-10T21:29:40.545358+0000","last_became_active":"2024-09-10T21:29:38.543907+0000","last_became_peered":"2024-09-10T21:29:38.543907+0000","last_unstale":"2024-09-10T21:29:40.545358+0000","last_undegraded":"2024-09-10T21:29:40.545358+0000","last_fullsized":"2024-09-10T21:29:40.545358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:37.225392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546397+0000","last_change":"2024-09-10T21:29:39.557707+0000","last_active":"2024-09-10T21:29:40.546397+0000","last_peered":"2024-09-10T21:29:40.546397+0000","last_clean":"2024-09-10T21:29:40.546397+0000","last_became_active":"2024-09-10T21:29:39.557398+0000","last_became_peered":"2024-09-10T21:29:39.557398+0000","last_unstale":"2024-09-10T21:29:40.546397+0000","last_undegraded":"2024-09-10T21:29:40.546397+0000","last_fullsized":"2024-09-10T21:29:40.546397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:05.442260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545105+0000","last_change":"2024-09-10T21:29:38.549044+0000","last_active":"2024-09-10T21:29:40.545105+0000","last_peered":"2024-09-10T21:29:40.545105+0000","last_clean":"2024-09-10T21:29:40.545105+0000","last_became_active":"2024-09-10T21:29:38.548465+0000","last_became_peered":"2024-09-10T21:29:38.548465+0000","last_unstale":"2024-09-10T21:29:40.545105+0000","last_undegraded":"2024-09-10T21:29:40.545105+0000","last_fullsized":"2024-09-10T21:29:40.545105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:47.729570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769221+0000","last_change":"2024-09-10T21:29:39.555291+0000","last_active":"2024-09-10T21:29:40.769221+0000","last_peered":"2024-09-10T21:29:40.769221+0000","last_clean":"2024-09-10T21:29:40.769221+0000","last_became_active":"2024-09-10T21:29:39.554979+0000","last_became_peered":"2024-09-10T21:29:39.554979+0000","last_unstale":"2024-09-10T21:29:40.769221+0000","last_undegraded":"2024-09-10T21:29:40.769221+0000","last_fullsized":"2024-09-10T21:29:40.769221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:41.126386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544896+0000","last_change":"2024-09-10T21:29:38.545682+0000","last_active":"2024-09-10T21:29:40.544896+0000","last_peered":"2024-09-10T21:29:40.544896+0000","last_clean":"2024-09-10T21:29:40.544896+0000","last_became_active":"2024-09-10T21:29:38.545432+0000","last_became_peered":"2024-09-10T21:29:38.545432+0000","last_unstale":"2024-09-10T21:29:40.544896+0000","last_undegraded":"2024-09-10T21:29:40.544896+0000","last_fullsized":"2024-09-10T21:29:40.544896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769268+0000","last_change":"2024-09-10T21:29:39.560953+0000","last_active":"2024-09-10T21:29:40.769268+0000","last_peered":"2024-09-10T21:29:40.769268+0000","last_clean":"2024-09-10T21:29:40.769268+0000","last_became_active":"2024-09-10T21:29:39.560702+0000","last_became_peered":"2024-09-10T21:29:39.560702+0000","last_unstale":"2024-09-10T21:29:40.769268+0000","last_undegraded":"2024-09-10T21:29:40.769268+0000","last_fullsized":"2024-09-10T21:29:40.769268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:05.821463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546758+0000","last_change":"2024-09-10T21:29:38.550165+0000","last_active":"2024-09-10T21:29:40.546758+0000","last_peered":"2024-09-10T21:29:40.546758+0000","last_clean":"2024-09-10T21:29:40.546758+0000","last_became_active":"2024-09-10T21:29:38.549861+0000","last_became_peered":"2024-09-10T21:29:38.549861+0000","last_unstale":"2024-09-10T21:29:40.546758+0000","last_undegraded":"2024-09-10T21:29:40.546758+0000","last_fullsized":"2024-09-10T21:29:40.546758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:02.189449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547170+0000","last_change":"2024-09-10T21:29:39.545985+0000","last_active":"2024-09-10T21:29:40.547170+0000","last_peered":"2024-09-10T21:29:40.547170+0000","last_clean":"2024-09-10T21:29:40.547170+0000","last_became_active":"2024-09-10T21:29:39.545349+0000","last_became_peered":"2024-09-10T21:29:39.545349+0000","last_unstale":"2024-09-10T21:29:40.547170+0000","last_undegraded":"2024-09-10T21:29:40.547170+0000","last_fullsized":"2024-09-10T21:29:40.547170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:14.688355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547653+0000","last_change":"2024-09-10T21:29:38.546910+0000","last_active":"2024-09-10T21:29:40.547653+0000","last_peered":"2024-09-10T21:29:40.547653+0000","last_clean":"2024-09-10T21:29:40.547653+0000","last_became_active":"2024-09-10T21:29:38.546159+0000","last_became_peered":"2024-09-10T21:29:38.546159+0000","last_unstale":"2024-09-10T21:29:40.547653+0000","last_undegraded":"2024-09-10T21:29:40.547653+0000","last_fullsized":"2024-09-10T21:29:40.547653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.324328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547733+0000","last_change":"2024-09-10T21:29:39.551802+0000","last_active":"2024-09-10T21:29:40.547733+0000","last_peered":"2024-09-10T21:29:40.547733+0000","last_clean":"2024-09-10T21:29:40.547733+0000","last_became_active":"2024-09-10T21:29:39.550894+0000","last_became_peered":"2024-09-10T21:29:39.550894+0000","last_unstale":"2024-09-10T21:29:40.547733+0000","last_undegraded":"2024-09-10T21:29:40.547733+0000","last_fullsized":"2024-09-10T21:29:40.547733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:35.976956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768722+0000","last_change":"2024-09-10T21:29:38.549752+0000","last_active":"2024-09-10T21:29:40.768722+0000","last_peered":"2024-09-10T21:29:40.768722+0000","last_clean":"2024-09-10T21:29:40.768722+0000","last_became_active":"2024-09-10T21:29:38.549380+0000","last_became_peered":"2024-09-10T21:29:38.549380+0000","last_unstale":"2024-09-10T21:29:40.768722+0000","last_undegraded":"2024-09-10T21:29:40.768722+0000","last_fullsized":"2024-09-10T21:29:40.768722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547195+0000","last_change":"2024-09-10T21:29:39.544277+0000","last_active":"2024-09-10T21:29:40.547195+0000","last_peered":"2024-09-10T21:29:40.547195+0000","last_clean":"2024-09-10T21:29:40.547195+0000","last_became_active":"2024-09-10T21:29:39.544011+0000","last_became_peered":"2024-09-10T21:29:39.544011+0000","last_unstale":"2024-09-10T21:29:40.547195+0000","last_undegraded":"2024-09-10T21:29:40.547195+0000","last_fullsized":"2024-09-10T21:29:40.547195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:05.999146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768535+0000","last_change":"2024-09-10T21:29:38.544149+0000","last_active":"2024-09-10T21:29:40.768535+0000","last_peered":"2024-09-10T21:29:40.768535+0000","last_clean":"2024-09-10T21:29:40.768535+0000","last_became_active":"2024-09-10T21:29:38.543790+0000","last_became_peered":"2024-09-10T21:29:38.543790+0000","last_unstale":"2024-09-10T21:29:40.768535+0000","last_undegraded":"2024-09-10T21:29:40.768535+0000","last_fullsized":"2024-09-10T21:29:40.768535+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:28.591046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.770023+0000","last_change":"2024-09-10T21:29:39.545983+0000","last_active":"2024-09-10T21:29:40.770023+0000","last_peered":"2024-09-10T21:29:40.770023+0000","last_clean":"2024-09-10T21:29:40.770023+0000","last_became_active":"2024-09-10T21:29:39.545699+0000","last_became_peered":"2024-09-10T21:29:39.545699+0000","last_unstale":"2024-09-10T21:29:40.770023+0000","last_undegraded":"2024-09-10T21:29:40.770023+0000","last_fullsized":"2024-09-10T21:29:40.770023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.110951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768649+0000","last_change":"2024-09-10T21:29:38.546518+0000","last_active":"2024-09-10T21:29:40.768649+0000","last_peered":"2024-09-10T21:29:40.768649+0000","last_clean":"2024-09-10T21:29:40.768649+0000","last_became_active":"2024-09-10T21:29:38.546262+0000","last_became_peered":"2024-09-10T21:29:38.546262+0000","last_unstale":"2024-09-10T21:29:40.768649+0000","last_undegraded":"2024-09-10T21:29:40.768649+0000","last_fullsized":"2024-09-10T21:29:40.768649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544409+0000","last_change":"2024-09-10T21:29:39.554112+0000","last_active":"2024-09-10T21:29:40.544409+0000","last_peered":"2024-09-10T21:29:40.544409+0000","last_clean":"2024-09-10T21:29:40.544409+0000","last_became_active":"2024-09-10T21:29:39.552077+0000","last_became_peered":"2024-09-10T21:29:39.552077+0000","last_unstale":"2024-09-10T21:29:40.544409+0000","last_undegraded":"2024-09-10T21:29:40.544409+0000","last_fullsized":"2024-09-10T21:29:40.544409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:18.545317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769044+0000","last_change":"2024-09-10T21:29:38.544002+0000","last_active":"2024-09-10T21:29:40.769044+0000","last_peered":"2024-09-10T21:29:40.769044+0000","last_clean":"2024-09-10T21:29:40.769044+0000","last_became_active":"2024-09-10T21:29:38.543699+0000","last_became_peered":"2024-09-10T21:29:38.543699+0000","last_unstale":"2024-09-10T21:29:40.769044+0000","last_undegraded":"2024-09-10T21:29:40.769044+0000","last_fullsized":"2024-09-10T21:29:40.769044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:21.454494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768948+0000","last_change":"2024-09-10T21:29:39.558189+0000","last_active":"2024-09-10T21:29:40.768948+0000","last_peered":"2024-09-10T21:29:40.768948+0000","last_clean":"2024-09-10T21:29:40.768948+0000","last_became_active":"2024-09-10T21:29:39.557549+0000","last_became_peered":"2024-09-10T21:29:39.557549+0000","last_unstale":"2024-09-10T21:29:40.768948+0000","last_undegraded":"2024-09-10T21:29:40.768948+0000","last_fullsized":"2024-09-10T21:29:40.768948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:48.581722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547028+0000","last_change":"2024-09-10T21:29:38.544536+0000","last_active":"2024-09-10T21:29:40.547028+0000","last_peered":"2024-09-10T21:29:40.547028+0000","last_clean":"2024-09-10T21:29:40.547028+0000","last_became_active":"2024-09-10T21:29:38.544212+0000","last_became_peered":"2024-09-10T21:29:38.544212+0000","last_unstale":"2024-09-10T21:29:40.547028+0000","last_undegraded":"2024-09-10T21:29:40.547028+0000","last_fullsized":"2024-09-10T21:29:40.547028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:53.514589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769300+0000","last_change":"2024-09-10T21:29:39.546810+0000","last_active":"2024-09-10T21:29:40.769300+0000","last_peered":"2024-09-10T21:29:40.769300+0000","last_clean":"2024-09-10T21:29:40.769300+0000","last_became_active":"2024-09-10T21:29:39.546317+0000","last_became_peered":"2024-09-10T21:29:39.546317+0000","last_unstale":"2024-09-10T21:29:40.769300+0000","last_undegraded":"2024-09-10T21:29:40.769300+0000","last_fullsized":"2024-09-10T21:29:40.769300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:46.343210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768987+0000","last_change":"2024-09-10T21:29:38.533647+0000","last_active":"2024-09-10T21:29:40.768987+0000","last_peered":"2024-09-10T21:29:40.768987+0000","last_clean":"2024-09-10T21:29:40.768987+0000","last_became_active":"2024-09-10T21:29:38.533361+0000","last_became_peered":"2024-09-10T21:29:38.533361+0000","last_unstale":"2024-09-10T21:29:40.768987+0000","last_undegraded":"2024-09-10T21:29:40.768987+0000","last_fullsized":"2024-09-10T21:29:40.768987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:32.782811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768884+0000","last_change":"2024-09-10T21:29:39.539314+0000","last_active":"2024-09-10T21:29:40.768884+0000","last_peered":"2024-09-10T21:29:40.768884+0000","last_clean":"2024-09-10T21:29:40.768884+0000","last_became_active":"2024-09-10T21:29:39.539015+0000","last_became_peered":"2024-09-10T21:29:39.539015+0000","last_unstale":"2024-09-10T21:29:40.768884+0000","last_undegraded":"2024-09-10T21:29:40.768884+0000","last_fullsized":"2024-09-10T21:29:40.768884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:19.770402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544676+0000","last_change":"2024-09-10T21:29:38.547087+0000","last_active":"2024-09-10T21:29:40.544676+0000","last_peered":"2024-09-10T21:29:40.544676+0000","last_clean":"2024-09-10T21:29:40.544676+0000","last_became_active":"2024-09-10T21:29:38.546829+0000","last_became_peered":"2024-09-10T21:29:38.546829+0000","last_unstale":"2024-09-10T21:29:40.544676+0000","last_undegraded":"2024-09-10T21:29:40.544676+0000","last_fullsized":"2024-09-10T21:29:40.544676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769511+0000","last_change":"2024-09-10T21:29:39.550392+0000","last_active":"2024-09-10T21:29:40.769511+0000","last_peered":"2024-09-10T21:29:40.769511+0000","last_clean":"2024-09-10T21:29:40.769511+0000","last_became_active":"2024-09-10T21:29:39.550131+0000","last_became_peered":"2024-09-10T21:29:39.550131+0000","last_unstale":"2024-09-10T21:29:40.769511+0000","last_undegraded":"2024-09-10T21:29:40.769511+0000","last_fullsized":"2024-09-10T21:29:40.769511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:15.885819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769553+0000","last_change":"2024-09-10T21:29:38.549285+0000","last_active":"2024-09-10T21:29:40.769553+0000","last_peered":"2024-09-10T21:29:40.769553+0000","last_clean":"2024-09-10T21:29:40.769553+0000","last_became_active":"2024-09-10T21:29:38.549014+0000","last_became_peered":"2024-09-10T21:29:38.549014+0000","last_unstale":"2024-09-10T21:29:40.769553+0000","last_undegraded":"2024-09-10T21:29:40.769553+0000","last_fullsized":"2024-09-10T21:29:40.769553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:58.754314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544681+0000","last_change":"2024-09-10T21:29:39.562581+0000","last_active":"2024-09-10T21:29:40.544681+0000","last_peered":"2024-09-10T21:29:40.544681+0000","last_clean":"2024-09-10T21:29:40.544681+0000","last_became_active":"2024-09-10T21:29:39.562287+0000","last_became_peered":"2024-09-10T21:29:39.562287+0000","last_unstale":"2024-09-10T21:29:40.544681+0000","last_undegraded":"2024-09-10T21:29:40.544681+0000","last_fullsized":"2024-09-10T21:29:40.544681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545978+0000","last_change":"2024-09-10T21:29:38.546345+0000","last_active":"2024-09-10T21:29:40.545978+0000","last_peered":"2024-09-10T21:29:40.545978+0000","last_clean":"2024-09-10T21:29:40.545978+0000","last_became_active":"2024-09-10T21:29:38.546094+0000","last_became_peered":"2024-09-10T21:29:38.546094+0000","last_unstale":"2024-09-10T21:29:40.545978+0000","last_undegraded":"2024-09-10T21:29:40.545978+0000","last_fullsized":"2024-09-10T21:29:40.545978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769644+0000","last_change":"2024-09-10T21:29:39.558347+0000","last_active":"2024-09-10T21:29:40.769644+0000","last_peered":"2024-09-10T21:29:40.769644+0000","last_clean":"2024-09-10T21:29:40.769644+0000","last_became_active":"2024-09-10T21:29:39.557760+0000","last_became_peered":"2024-09-10T21:29:39.557760+0000","last_unstale":"2024-09-10T21:29:40.769644+0000","last_undegraded":"2024-09-10T21:29:40.769644+0000","last_fullsized":"2024-09-10T21:29:40.769644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:06.975636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544238+0000","last_change":"2024-09-10T21:29:38.535077+0000","last_active":"2024-09-10T21:29:40.544238+0000","last_peered":"2024-09-10T21:29:40.544238+0000","last_clean":"2024-09-10T21:29:40.544238+0000","last_became_active":"2024-09-10T21:29:38.534747+0000","last_became_peered":"2024-09-10T21:29:38.534747+0000","last_unstale":"2024-09-10T21:29:40.544238+0000","last_undegraded":"2024-09-10T21:29:40.544238+0000","last_fullsized":"2024-09-10T21:29:40.544238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547454+0000","last_change":"2024-09-10T21:29:39.548356+0000","last_active":"2024-09-10T21:29:40.547454+0000","last_peered":"2024-09-10T21:29:40.547454+0000","last_clean":"2024-09-10T21:29:40.547454+0000","last_became_active":"2024-09-10T21:29:39.548096+0000","last_became_peered":"2024-09-10T21:29:39.548096+0000","last_unstale":"2024-09-10T21:29:40.547454+0000","last_undegraded":"2024-09-10T21:29:40.547454+0000","last_fullsized":"2024-09-10T21:29:40.547454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:50.836505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768672+0000","last_change":"2024-09-10T21:29:38.544063+0000","last_active":"2024-09-10T21:29:40.768672+0000","last_peered":"2024-09-10T21:29:40.768672+0000","last_clean":"2024-09-10T21:29:40.768672+0000","last_became_active":"2024-09-10T21:29:38.543508+0000","last_became_peered":"2024-09-10T21:29:38.543508+0000","last_unstale":"2024-09-10T21:29:40.768672+0000","last_undegraded":"2024-09-10T21:29:40.768672+0000","last_fullsized":"2024-09-10T21:29:40.768672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:02.927923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769972+0000","last_change":"2024-09-10T21:29:39.552135+0000","last_active":"2024-09-10T21:29:40.769972+0000","last_peered":"2024-09-10T21:29:40.769972+0000","last_clean":"2024-09-10T21:29:40.769972+0000","last_became_active":"2024-09-10T21:29:39.551649+0000","last_became_peered":"2024-09-10T21:29:39.551649+0000","last_unstale":"2024-09-10T21:29:40.769972+0000","last_undegraded":"2024-09-10T21:29:40.769972+0000","last_fullsized":"2024-09-10T21:29:40.769972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:26.827601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545785+0000","last_change":"2024-09-10T21:29:38.547451+0000","last_active":"2024-09-10T21:29:40.545785+0000","last_peered":"2024-09-10T21:29:40.545785+0000","last_clean":"2024-09-10T21:29:40.545785+0000","last_became_active":"2024-09-10T21:29:38.546360+0000","last_became_peered":"2024-09-10T21:29:38.546360+0000","last_unstale":"2024-09-10T21:29:40.545785+0000","last_undegraded":"2024-09-10T21:29:40.545785+0000","last_fullsized":"2024-09-10T21:29:40.545785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768220+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:29:40.768220+0000","last_peered":"2024-09-10T21:29:40.768220+0000","last_clean":"2024-09-10T21:29:40.768220+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:29:40.768220+0000","last_undegraded":"2024-09-10T21:29:40.768220+0000","last_fullsized":"2024-09-10T21:29:40.768220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":458807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":1048593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":453159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":458135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1605632,"data_stored":1050149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:29:48.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:29:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.365+0000 7f37a1a00640 1 -- 172.21.15.97:0/3942852710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 msgr2=0x7f378000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.365+0000 7f37a1a00640 1 --2- 172.21.15.97:0/3942852710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f378000b9d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3794006fd0 tx=0x7f379401caf0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.365+0000 7f37a1a00640 1 -- 172.21.15.97:0/3942852710 shutdown_connections 2024-09-10T21:29:48.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.365+0000 7f37a1a00640 1 --2- 172.21.15.97:0/3942852710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f378000b9d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.365+0000 7f37a1a00640 1 -- 172.21.15.97:0/3942852710 >> 172.21.15.97:0/3942852710 conn(0x7f379c0d0090 msgr2=0x7f379c0d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.365+0000 7f37a1a00640 1 -- 172.21.15.97:0/3942852710 shutdown_connections 2024-09-10T21:29:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 -- 172.21.15.97:0/3942852710 wait complete. 2024-09-10T21:29:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 Processor -- start 2024-09-10T21:29:48.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 -- start start 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f379c156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f379c0c9d10 0x7f379c0c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f379c0ca190 0x7f379c0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f379c15bde0 con 0x7f3780005220 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f379c15bc60 con 0x7f379c0c9d10 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f379c06cb40 con 0x7f379c0ca190 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f379c156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f379c156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45162/0 (socket says 172.21.15.97:45162) 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 -- 172.21.15.97:0/611364164 learned_addr learned my addr 172.21.15.97:0/611364164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f3793400640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f379c0c9d10 0x7f379c0c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1000640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f379c0ca190 0x7f379c0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 -- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f379c0ca190 msgr2=0x7f379c0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f379c0ca190 0x7f379c0c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 -- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f379c0c9d10 msgr2=0x7f379c0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f379c0c9d10 0x7f379c0c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 -- 172.21.15.97:0/611364164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1f27f0 con 0x7f3780005220 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1000640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f379c0ca190 0x7f379c0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f379be00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f379c156350 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f378c011ab0 tx=0x7f378c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f3798c00640 1 -- 172.21.15.97:0/611364164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c01e070 con 0x7f3780005220 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f3798c00640 1 -- 172.21.15.97:0/611364164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f378c00fe70 con 0x7f3780005220 2024-09-10T21:29:48.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f3798c00640 1 -- 172.21.15.97:0/611364164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0042b0 con 0x7f3780005220 2024-09-10T21:29:48.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.369+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c1f2ad0 con 0x7f3780005220 2024-09-10T21:29:48.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.373+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c1f3090 con 0x7f3780005220 2024-09-10T21:29:48.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.373+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c06d040 con 0x7f3780005220 2024-09-10T21:29:48.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.373+0000 7f3798c00640 1 -- 172.21.15.97:0/611364164 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f378c003600 con 0x7f3780005220 2024-09-10T21:29:48.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.373+0000 7f3798c00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37780725e0 0x7f3778074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.373+0000 7f3798c00640 1 -- 172.21.15.97:0/611364164 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f378c094180 con 0x7f3780005220 2024-09-10T21:29:48.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.373+0000 7f3793400640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37780725e0 0x7f3778074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.373+0000 7f3793400640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37780725e0 0x7f3778074aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f379c15a0a0 tx=0x7f3784006730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:48.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.377+0000 7f3798c00640 1 -- 172.21.15.97:0/611364164 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f378c05efe0 con 0x7f3780005220 2024-09-10T21:29:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.541+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f379c0b5ee0 con 0x7f37780725e0 2024-09-10T21:29:48.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f3798c00640 1 -- 172.21.15.97:0/611364164 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251975 (secure 0 0 0) 0x7f379c0b5ee0 con 0x7f37780725e0 2024-09-10T21:29:48.553 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:48.555 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:29:48.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37780725e0 msgr2=0x7f3778074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37780725e0 0x7f3778074aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f379c15a0a0 tx=0x7f3784006730 comp rx=0 tx=0).stop 2024-09-10T21:29:48.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 msgr2=0x7f379c156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f379c156350 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f378c011ab0 tx=0x7f378c011f70 comp rx=0 tx=0).stop 2024-09-10T21:29:48.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 shutdown_connections 2024-09-10T21:29:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37780725e0 0x7f3778074aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f379c0ca190 0x7f379c0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f379c0c9d10 0x7f379c0c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 --2- 172.21.15.97:0/611364164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780005220 0x7f379c156350 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 >> 172.21.15.97:0/611364164 conn(0x7f379c0d0090 msgr2=0x7f379c1582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 shutdown_connections 2024-09-10T21:29:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.549+0000 7f37a1a00640 1 -- 172.21.15.97:0/611364164 wait complete. 2024-09-10T21:29:48.573 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-10T21:29:48.157103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241892,"kb_used_data":9000,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754732828,"statfs":{"total":773094113280,"available":772846415872,"internally_reserved":0,"allocated":9216000,"data_stored":4850636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":238276147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.631843"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768894+0000","last_change":"2024-09-10T21:29:39.539326+0000","last_active":"2024-09-10T21:29:40.768894+0000","last_peered":"2024-09-10T21:29:40.768894+0000","last_clean":"2024-09-10T21:29:40.768894+0000","last_became_active":"2024-09-10T21:29:39.539056+0000","last_became_peered":"2024-09-10T21:29:39.539056+0000","last_unstale":"2024-09-10T21:29:40.768894+0000","last_undegraded":"2024-09-10T21:29:40.768894+0000","last_fullsized":"2024-09-10T21:29:40.768894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:25.767146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769190+0000","last_change":"2024-09-10T21:29:39.564404+0000","last_active":"2024-09-10T21:29:40.769190+0000","last_peered":"2024-09-10T21:29:40.769190+0000","last_clean":"2024-09-10T21:29:40.769190+0000","last_became_active":"2024-09-10T21:29:39.563306+0000","last_became_peered":"2024-09-10T21:29:39.563306+0000","last_unstale":"2024-09-10T21:29:40.769190+0000","last_undegraded":"2024-09-10T21:29:40.769190+0000","last_fullsized":"2024-09-10T21:29:40.769190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:17.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769401+0000","last_change":"2024-09-10T21:29:39.561606+0000","last_active":"2024-09-10T21:29:40.769401+0000","last_peered":"2024-09-10T21:29:40.769401+0000","last_clean":"2024-09-10T21:29:40.769401+0000","last_became_active":"2024-09-10T21:29:39.560972+0000","last_became_peered":"2024-09-10T21:29:39.560972+0000","last_unstale":"2024-09-10T21:29:40.769401+0000","last_undegraded":"2024-09-10T21:29:40.769401+0000","last_fullsized":"2024-09-10T21:29:40.769401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:26.701898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544576+0000","last_change":"2024-09-10T21:29:39.537773+0000","last_active":"2024-09-10T21:29:40.544576+0000","last_peered":"2024-09-10T21:29:40.544576+0000","last_clean":"2024-09-10T21:29:40.544576+0000","last_became_active":"2024-09-10T21:29:39.537487+0000","last_became_peered":"2024-09-10T21:29:39.537487+0000","last_unstale":"2024-09-10T21:29:40.544576+0000","last_undegraded":"2024-09-10T21:29:40.544576+0000","last_fullsized":"2024-09-10T21:29:40.544576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:53.105160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546947+0000","last_change":"2024-09-10T21:29:39.546644+0000","last_active":"2024-09-10T21:29:40.546947+0000","last_peered":"2024-09-10T21:29:40.546947+0000","last_clean":"2024-09-10T21:29:40.546947+0000","last_became_active":"2024-09-10T21:29:39.546373+0000","last_became_peered":"2024-09-10T21:29:39.546373+0000","last_unstale":"2024-09-10T21:29:40.546947+0000","last_undegraded":"2024-09-10T21:29:40.546947+0000","last_fullsized":"2024-09-10T21:29:40.546947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.942938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546101+0000","last_change":"2024-09-10T21:29:39.558707+0000","last_active":"2024-09-10T21:29:40.546101+0000","last_peered":"2024-09-10T21:29:40.546101+0000","last_clean":"2024-09-10T21:29:40.546101+0000","last_became_active":"2024-09-10T21:29:39.554793+0000","last_became_peered":"2024-09-10T21:29:39.554793+0000","last_unstale":"2024-09-10T21:29:40.546101+0000","last_undegraded":"2024-09-10T21:29:40.546101+0000","last_fullsized":"2024-09-10T21:29:40.546101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544328+0000","last_change":"2024-09-10T21:29:39.540653+0000","last_active":"2024-09-10T21:29:40.544328+0000","last_peered":"2024-09-10T21:29:40.544328+0000","last_clean":"2024-09-10T21:29:40.544328+0000","last_became_active":"2024-09-10T21:29:39.540400+0000","last_became_peered":"2024-09-10T21:29:39.540400+0000","last_unstale":"2024-09-10T21:29:40.544328+0000","last_undegraded":"2024-09-10T21:29:40.544328+0000","last_fullsized":"2024-09-10T21:29:40.544328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:26.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768468+0000","last_change":"2024-09-10T21:29:39.535592+0000","last_active":"2024-09-10T21:29:40.768468+0000","last_peered":"2024-09-10T21:29:40.768468+0000","last_clean":"2024-09-10T21:29:40.768468+0000","last_became_active":"2024-09-10T21:29:39.535236+0000","last_became_peered":"2024-09-10T21:29:39.535236+0000","last_unstale":"2024-09-10T21:29:40.768468+0000","last_undegraded":"2024-09-10T21:29:40.768468+0000","last_fullsized":"2024-09-10T21:29:40.768468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:54.259155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545150+0000","last_change":"2024-09-10T21:29:39.556621+0000","last_active":"2024-09-10T21:29:40.545150+0000","last_peered":"2024-09-10T21:29:40.545150+0000","last_clean":"2024-09-10T21:29:40.545150+0000","last_became_active":"2024-09-10T21:29:39.555001+0000","last_became_peered":"2024-09-10T21:29:39.555001+0000","last_unstale":"2024-09-10T21:29:40.545150+0000","last_undegraded":"2024-09-10T21:29:40.545150+0000","last_fullsized":"2024-09-10T21:29:40.545150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769242+0000","last_change":"2024-09-10T21:29:39.536232+0000","last_active":"2024-09-10T21:29:40.769242+0000","last_peered":"2024-09-10T21:29:40.769242+0000","last_clean":"2024-09-10T21:29:40.769242+0000","last_became_active":"2024-09-10T21:29:39.535978+0000","last_became_peered":"2024-09-10T21:29:39.535978+0000","last_unstale":"2024-09-10T21:29:40.769242+0000","last_undegraded":"2024-09-10T21:29:40.769242+0000","last_fullsized":"2024-09-10T21:29:40.769242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:49.789897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769661+0000","last_change":"2024-09-10T21:29:39.562255+0000","last_active":"2024-09-10T21:29:40.769661+0000","last_peered":"2024-09-10T21:29:40.769661+0000","last_clean":"2024-09-10T21:29:40.769661+0000","last_became_active":"2024-09-10T21:29:39.559040+0000","last_became_peered":"2024-09-10T21:29:39.559040+0000","last_unstale":"2024-09-10T21:29:40.769661+0000","last_undegraded":"2024-09-10T21:29:40.769661+0000","last_fullsized":"2024-09-10T21:29:40.769661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:39.446682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769690+0000","last_change":"2024-09-10T21:29:39.550940+0000","last_active":"2024-09-10T21:29:40.769690+0000","last_peered":"2024-09-10T21:29:40.769690+0000","last_clean":"2024-09-10T21:29:40.769690+0000","last_became_active":"2024-09-10T21:29:39.550666+0000","last_became_peered":"2024-09-10T21:29:39.550666+0000","last_unstale":"2024-09-10T21:29:40.769690+0000","last_undegraded":"2024-09-10T21:29:40.769690+0000","last_fullsized":"2024-09-10T21:29:40.769690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:24.960891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768575+0000","last_change":"2024-09-10T21:29:39.555356+0000","last_active":"2024-09-10T21:29:40.768575+0000","last_peered":"2024-09-10T21:29:40.768575+0000","last_clean":"2024-09-10T21:29:40.768575+0000","last_became_active":"2024-09-10T21:29:39.554843+0000","last_became_peered":"2024-09-10T21:29:39.554843+0000","last_unstale":"2024-09-10T21:29:40.768575+0000","last_undegraded":"2024-09-10T21:29:40.768575+0000","last_fullsized":"2024-09-10T21:29:40.768575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:03.103988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769008+0000","last_change":"2024-09-10T21:29:39.531161+0000","last_active":"2024-09-10T21:29:40.769008+0000","last_peered":"2024-09-10T21:29:40.769008+0000","last_clean":"2024-09-10T21:29:40.769008+0000","last_became_active":"2024-09-10T21:29:39.530873+0000","last_became_peered":"2024-09-10T21:29:39.530873+0000","last_unstale":"2024-09-10T21:29:40.769008+0000","last_undegraded":"2024-09-10T21:29:40.769008+0000","last_fullsized":"2024-09-10T21:29:40.769008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:25.754412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546185+0000","last_change":"2024-09-10T21:29:39.563879+0000","last_active":"2024-09-10T21:29:40.546185+0000","last_peered":"2024-09-10T21:29:40.546185+0000","last_clean":"2024-09-10T21:29:40.546185+0000","last_became_active":"2024-09-10T21:29:39.563673+0000","last_became_peered":"2024-09-10T21:29:39.563673+0000","last_unstale":"2024-09-10T21:29:40.546185+0000","last_undegraded":"2024-09-10T21:29:40.546185+0000","last_fullsized":"2024-09-10T21:29:40.546185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:33.785842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544217+0000","last_change":"2024-09-10T21:29:39.552730+0000","last_active":"2024-09-10T21:29:40.544217+0000","last_peered":"2024-09-10T21:29:40.544217+0000","last_clean":"2024-09-10T21:29:40.544217+0000","last_became_active":"2024-09-10T21:29:39.552478+0000","last_became_peered":"2024-09-10T21:29:39.552478+0000","last_unstale":"2024-09-10T21:29:40.544217+0000","last_undegraded":"2024-09-10T21:29:40.544217+0000","last_fullsized":"2024-09-10T21:29:40.544217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:00.008953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768952+0000","last_change":"2024-09-10T21:29:39.541920+0000","last_active":"2024-09-10T21:29:40.768952+0000","last_peered":"2024-09-10T21:29:40.768952+0000","last_clean":"2024-09-10T21:29:40.768952+0000","last_became_active":"2024-09-10T21:29:39.541604+0000","last_became_peered":"2024-09-10T21:29:39.541604+0000","last_unstale":"2024-09-10T21:29:40.768952+0000","last_undegraded":"2024-09-10T21:29:40.768952+0000","last_fullsized":"2024-09-10T21:29:40.768952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:24.814305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769429+0000","last_change":"2024-09-10T21:29:39.558608+0000","last_active":"2024-09-10T21:29:40.769429+0000","last_peered":"2024-09-10T21:29:40.769429+0000","last_clean":"2024-09-10T21:29:40.769429+0000","last_became_active":"2024-09-10T21:29:39.558086+0000","last_became_peered":"2024-09-10T21:29:39.558086+0000","last_unstale":"2024-09-10T21:29:40.769429+0000","last_undegraded":"2024-09-10T21:29:40.769429+0000","last_fullsized":"2024-09-10T21:29:40.769429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:51.432263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769280+0000","last_change":"2024-09-10T21:29:39.539085+0000","last_active":"2024-09-10T21:29:40.769280+0000","last_peered":"2024-09-10T21:29:40.769280+0000","last_clean":"2024-09-10T21:29:40.769280+0000","last_became_active":"2024-09-10T21:29:39.536918+0000","last_became_peered":"2024-09-10T21:29:39.536918+0000","last_unstale":"2024-09-10T21:29:40.769280+0000","last_undegraded":"2024-09-10T21:29:40.769280+0000","last_fullsized":"2024-09-10T21:29:40.769280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:56.499531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546462+0000","last_change":"2024-09-10T21:29:39.560153+0000","last_active":"2024-09-10T21:29:40.546462+0000","last_peered":"2024-09-10T21:29:40.546462+0000","last_clean":"2024-09-10T21:29:40.546462+0000","last_became_active":"2024-09-10T21:29:39.559886+0000","last_became_peered":"2024-09-10T21:29:39.559886+0000","last_unstale":"2024-09-10T21:29:40.546462+0000","last_undegraded":"2024-09-10T21:29:40.546462+0000","last_fullsized":"2024-09-10T21:29:40.546462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768980+0000","last_change":"2024-09-10T21:29:39.547565+0000","last_active":"2024-09-10T21:29:40.768980+0000","last_peered":"2024-09-10T21:29:40.768980+0000","last_clean":"2024-09-10T21:29:40.768980+0000","last_became_active":"2024-09-10T21:29:39.547315+0000","last_became_peered":"2024-09-10T21:29:39.547315+0000","last_unstale":"2024-09-10T21:29:40.768980+0000","last_undegraded":"2024-09-10T21:29:40.768980+0000","last_fullsized":"2024-09-10T21:29:40.768980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:05.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769033+0000","last_change":"2024-09-10T21:29:39.562781+0000","last_active":"2024-09-10T21:29:40.769033+0000","last_peered":"2024-09-10T21:29:40.769033+0000","last_clean":"2024-09-10T21:29:40.769033+0000","last_became_active":"2024-09-10T21:29:39.562509+0000","last_became_peered":"2024-09-10T21:29:39.562509+0000","last_unstale":"2024-09-10T21:29:40.769033+0000","last_undegraded":"2024-09-10T21:29:40.769033+0000","last_fullsized":"2024-09-10T21:29:40.769033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:53.170104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768742+0000","last_change":"2024-09-10T21:29:39.540989+0000","last_active":"2024-09-10T21:29:40.768742+0000","last_peered":"2024-09-10T21:29:40.768742+0000","last_clean":"2024-09-10T21:29:40.768742+0000","last_became_active":"2024-09-10T21:29:39.540686+0000","last_became_peered":"2024-09-10T21:29:39.540686+0000","last_unstale":"2024-09-10T21:29:40.768742+0000","last_undegraded":"2024-09-10T21:29:40.768742+0000","last_fullsized":"2024-09-10T21:29:40.768742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:04.647939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768432+0000","last_change":"2024-09-10T21:29:39.555697+0000","last_active":"2024-09-10T21:29:40.768432+0000","last_peered":"2024-09-10T21:29:40.768432+0000","last_clean":"2024-09-10T21:29:40.768432+0000","last_became_active":"2024-09-10T21:29:39.555321+0000","last_became_peered":"2024-09-10T21:29:39.555321+0000","last_unstale":"2024-09-10T21:29:40.768432+0000","last_undegraded":"2024-09-10T21:29:40.768432+0000","last_fullsized":"2024-09-10T21:29:40.768432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:26.937688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547452+0000","last_change":"2024-09-10T21:29:39.566354+0000","last_active":"2024-09-10T21:29:40.547452+0000","last_peered":"2024-09-10T21:29:40.547452+0000","last_clean":"2024-09-10T21:29:40.547452+0000","last_became_active":"2024-09-10T21:29:39.566119+0000","last_became_peered":"2024-09-10T21:29:39.566119+0000","last_unstale":"2024-09-10T21:29:40.547452+0000","last_undegraded":"2024-09-10T21:29:40.547452+0000","last_fullsized":"2024-09-10T21:29:40.547452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544876+0000","last_change":"2024-09-10T21:29:39.555058+0000","last_active":"2024-09-10T21:29:40.544876+0000","last_peered":"2024-09-10T21:29:40.544876+0000","last_clean":"2024-09-10T21:29:40.544876+0000","last_became_active":"2024-09-10T21:29:39.554444+0000","last_became_peered":"2024-09-10T21:29:39.554444+0000","last_unstale":"2024-09-10T21:29:40.544876+0000","last_undegraded":"2024-09-10T21:29:40.544876+0000","last_fullsized":"2024-09-10T21:29:40.544876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:07.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544584+0000","last_change":"2024-09-10T21:29:39.553456+0000","last_active":"2024-09-10T21:29:40.544584+0000","last_peered":"2024-09-10T21:29:40.544584+0000","last_clean":"2024-09-10T21:29:40.544584+0000","last_became_active":"2024-09-10T21:29:39.553218+0000","last_became_peered":"2024-09-10T21:29:39.553218+0000","last_unstale":"2024-09-10T21:29:40.544584+0000","last_undegraded":"2024-09-10T21:29:40.544584+0000","last_fullsized":"2024-09-10T21:29:40.544584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547497+0000","last_change":"2024-09-10T21:29:39.551245+0000","last_active":"2024-09-10T21:29:40.547497+0000","last_peered":"2024-09-10T21:29:40.547497+0000","last_clean":"2024-09-10T21:29:40.547497+0000","last_became_active":"2024-09-10T21:29:39.550535+0000","last_became_peered":"2024-09-10T21:29:39.550535+0000","last_unstale":"2024-09-10T21:29:40.547497+0000","last_undegraded":"2024-09-10T21:29:40.547497+0000","last_fullsized":"2024-09-10T21:29:40.547497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:03.284635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545912+0000","last_change":"2024-09-10T21:29:39.558777+0000","last_active":"2024-09-10T21:29:40.545912+0000","last_peered":"2024-09-10T21:29:40.545912+0000","last_clean":"2024-09-10T21:29:40.545912+0000","last_became_active":"2024-09-10T21:29:39.555098+0000","last_became_peered":"2024-09-10T21:29:39.555098+0000","last_unstale":"2024-09-10T21:29:40.545912+0000","last_undegraded":"2024-09-10T21:29:40.545912+0000","last_fullsized":"2024-09-10T21:29:40.545912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769096+0000","last_change":"2024-09-10T21:29:39.564203+0000","last_active":"2024-09-10T21:29:40.769096+0000","last_peered":"2024-09-10T21:29:40.769096+0000","last_clean":"2024-09-10T21:29:40.769096+0000","last_became_active":"2024-09-10T21:29:39.563951+0000","last_became_peered":"2024-09-10T21:29:39.563951+0000","last_unstale":"2024-09-10T21:29:40.769096+0000","last_undegraded":"2024-09-10T21:29:40.769096+0000","last_fullsized":"2024-09-10T21:29:40.769096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:38.678603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768992+0000","last_change":"2024-09-10T21:29:39.564492+0000","last_active":"2024-09-10T21:29:40.768992+0000","last_peered":"2024-09-10T21:29:40.768992+0000","last_clean":"2024-09-10T21:29:40.768992+0000","last_became_active":"2024-09-10T21:29:39.563500+0000","last_became_peered":"2024-09-10T21:29:39.563500+0000","last_unstale":"2024-09-10T21:29:40.768992+0000","last_undegraded":"2024-09-10T21:29:40.768992+0000","last_fullsized":"2024-09-10T21:29:40.768992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:46.030917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544257+0000","last_change":"2024-09-10T21:29:39.555396+0000","last_active":"2024-09-10T21:29:40.544257+0000","last_peered":"2024-09-10T21:29:40.544257+0000","last_clean":"2024-09-10T21:29:40.544257+0000","last_became_active":"2024-09-10T21:29:39.555086+0000","last_became_peered":"2024-09-10T21:29:39.555086+0000","last_unstale":"2024-09-10T21:29:40.544257+0000","last_undegraded":"2024-09-10T21:29:40.544257+0000","last_fullsized":"2024-09-10T21:29:40.544257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:06.000345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546200+0000","last_change":"2024-09-10T21:29:39.548302+0000","last_active":"2024-09-10T21:29:40.546200+0000","last_peered":"2024-09-10T21:29:40.546200+0000","last_clean":"2024-09-10T21:29:40.546200+0000","last_became_active":"2024-09-10T21:29:39.547972+0000","last_became_peered":"2024-09-10T21:29:39.547972+0000","last_unstale":"2024-09-10T21:29:40.546200+0000","last_undegraded":"2024-09-10T21:29:40.546200+0000","last_fullsized":"2024-09-10T21:29:40.546200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:22.457526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547580+0000","last_change":"2024-09-10T21:29:38.549320+0000","last_active":"2024-09-10T21:29:40.547580+0000","last_peered":"2024-09-10T21:29:40.547580+0000","last_clean":"2024-09-10T21:29:40.547580+0000","last_became_active":"2024-09-10T21:29:38.549041+0000","last_became_peered":"2024-09-10T21:29:38.549041+0000","last_unstale":"2024-09-10T21:29:40.547580+0000","last_undegraded":"2024-09-10T21:29:40.547580+0000","last_fullsized":"2024-09-10T21:29:40.547580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:52.457273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546222+0000","last_change":"2024-09-10T21:29:39.542269+0000","last_active":"2024-09-10T21:29:40.546222+0000","last_peered":"2024-09-10T21:29:40.546222+0000","last_clean":"2024-09-10T21:29:40.546222+0000","last_became_active":"2024-09-10T21:29:39.541975+0000","last_became_peered":"2024-09-10T21:29:39.541975+0000","last_unstale":"2024-09-10T21:29:40.546222+0000","last_undegraded":"2024-09-10T21:29:40.546222+0000","last_fullsized":"2024-09-10T21:29:40.546222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:38.513998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769882+0000","last_change":"2024-09-10T21:29:38.544792+0000","last_active":"2024-09-10T21:29:40.769882+0000","last_peered":"2024-09-10T21:29:40.769882+0000","last_clean":"2024-09-10T21:29:40.769882+0000","last_became_active":"2024-09-10T21:29:38.544459+0000","last_became_peered":"2024-09-10T21:29:38.544459+0000","last_unstale":"2024-09-10T21:29:40.769882+0000","last_undegraded":"2024-09-10T21:29:40.769882+0000","last_fullsized":"2024-09-10T21:29:40.769882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:28.938255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769152+0000","last_change":"2024-09-10T21:29:39.546560+0000","last_active":"2024-09-10T21:29:40.769152+0000","last_peered":"2024-09-10T21:29:40.769152+0000","last_clean":"2024-09-10T21:29:40.769152+0000","last_became_active":"2024-09-10T21:29:39.545328+0000","last_became_peered":"2024-09-10T21:29:39.545328+0000","last_unstale":"2024-09-10T21:29:40.769152+0000","last_undegraded":"2024-09-10T21:29:40.769152+0000","last_fullsized":"2024-09-10T21:29:40.769152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:20.557881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768895+0000","last_change":"2024-09-10T21:29:38.538896+0000","last_active":"2024-09-10T21:29:40.768895+0000","last_peered":"2024-09-10T21:29:40.768895+0000","last_clean":"2024-09-10T21:29:40.768895+0000","last_became_active":"2024-09-10T21:29:38.538523+0000","last_became_peered":"2024-09-10T21:29:38.538523+0000","last_unstale":"2024-09-10T21:29:40.768895+0000","last_undegraded":"2024-09-10T21:29:40.768895+0000","last_fullsized":"2024-09-10T21:29:40.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:26.307429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544519+0000","last_change":"2024-09-10T21:29:39.538536+0000","last_active":"2024-09-10T21:29:40.544519+0000","last_peered":"2024-09-10T21:29:40.544519+0000","last_clean":"2024-09-10T21:29:40.544519+0000","last_became_active":"2024-09-10T21:29:39.538308+0000","last_became_peered":"2024-09-10T21:29:39.538308+0000","last_unstale":"2024-09-10T21:29:40.544519+0000","last_undegraded":"2024-09-10T21:29:40.544519+0000","last_fullsized":"2024-09-10T21:29:40.544519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:40.847145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769616+0000","last_change":"2024-09-10T21:29:38.536197+0000","last_active":"2024-09-10T21:29:40.769616+0000","last_peered":"2024-09-10T21:29:40.769616+0000","last_clean":"2024-09-10T21:29:40.769616+0000","last_became_active":"2024-09-10T21:29:38.535894+0000","last_became_peered":"2024-09-10T21:29:38.535894+0000","last_unstale":"2024-09-10T21:29:40.769616+0000","last_undegraded":"2024-09-10T21:29:40.769616+0000","last_fullsized":"2024-09-10T21:29:40.769616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546646+0000","last_change":"2024-09-10T21:29:39.554137+0000","last_active":"2024-09-10T21:29:40.546646+0000","last_peered":"2024-09-10T21:29:40.546646+0000","last_clean":"2024-09-10T21:29:40.546646+0000","last_became_active":"2024-09-10T21:29:39.553735+0000","last_became_peered":"2024-09-10T21:29:39.553735+0000","last_unstale":"2024-09-10T21:29:40.546646+0000","last_undegraded":"2024-09-10T21:29:40.546646+0000","last_fullsized":"2024-09-10T21:29:40.546646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:38.675833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769158+0000","last_change":"2024-09-10T21:29:38.535949+0000","last_active":"2024-09-10T21:29:40.769158+0000","last_peered":"2024-09-10T21:29:40.769158+0000","last_clean":"2024-09-10T21:29:40.769158+0000","last_became_active":"2024-09-10T21:29:38.535669+0000","last_became_peered":"2024-09-10T21:29:38.535669+0000","last_unstale":"2024-09-10T21:29:40.769158+0000","last_undegraded":"2024-09-10T21:29:40.769158+0000","last_fullsized":"2024-09-10T21:29:40.769158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769743+0000","last_change":"2024-09-10T21:29:39.561954+0000","last_active":"2024-09-10T21:29:40.769743+0000","last_peered":"2024-09-10T21:29:40.769743+0000","last_clean":"2024-09-10T21:29:40.769743+0000","last_became_active":"2024-09-10T21:29:39.561348+0000","last_became_peered":"2024-09-10T21:29:39.561348+0000","last_unstale":"2024-09-10T21:29:40.769743+0000","last_undegraded":"2024-09-10T21:29:40.769743+0000","last_fullsized":"2024-09-10T21:29:40.769743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:54.689918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769497+0000","last_change":"2024-09-10T21:29:38.543780+0000","last_active":"2024-09-10T21:29:40.769497+0000","last_peered":"2024-09-10T21:29:40.769497+0000","last_clean":"2024-09-10T21:29:40.769497+0000","last_became_active":"2024-09-10T21:29:38.543305+0000","last_became_peered":"2024-09-10T21:29:38.543305+0000","last_unstale":"2024-09-10T21:29:40.769497+0000","last_undegraded":"2024-09-10T21:29:40.769497+0000","last_fullsized":"2024-09-10T21:29:40.769497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.941433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768491+0000","last_change":"2024-09-10T21:29:39.549779+0000","last_active":"2024-09-10T21:29:40.768491+0000","last_peered":"2024-09-10T21:29:40.768491+0000","last_clean":"2024-09-10T21:29:40.768491+0000","last_became_active":"2024-09-10T21:29:39.549473+0000","last_became_peered":"2024-09-10T21:29:39.549473+0000","last_unstale":"2024-09-10T21:29:40.768491+0000","last_undegraded":"2024-09-10T21:29:40.768491+0000","last_fullsized":"2024-09-10T21:29:40.768491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:38.685843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768418+0000","last_change":"2024-09-10T21:29:38.546344+0000","last_active":"2024-09-10T21:29:40.768418+0000","last_peered":"2024-09-10T21:29:40.768418+0000","last_clean":"2024-09-10T21:29:40.768418+0000","last_became_active":"2024-09-10T21:29:38.545808+0000","last_became_peered":"2024-09-10T21:29:38.545808+0000","last_unstale":"2024-09-10T21:29:40.768418+0000","last_undegraded":"2024-09-10T21:29:40.768418+0000","last_fullsized":"2024-09-10T21:29:40.768418+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:12.358215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769992+0000","last_change":"2024-09-10T21:29:39.558263+0000","last_active":"2024-09-10T21:29:40.769992+0000","last_peered":"2024-09-10T21:29:40.769992+0000","last_clean":"2024-09-10T21:29:40.769992+0000","last_became_active":"2024-09-10T21:29:39.557990+0000","last_became_peered":"2024-09-10T21:29:39.557990+0000","last_unstale":"2024-09-10T21:29:40.769992+0000","last_undegraded":"2024-09-10T21:29:40.769992+0000","last_fullsized":"2024-09-10T21:29:40.769992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.511263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768565+0000","last_change":"2024-09-10T21:29:38.545254+0000","last_active":"2024-09-10T21:29:40.768565+0000","last_peered":"2024-09-10T21:29:40.768565+0000","last_clean":"2024-09-10T21:29:40.768565+0000","last_became_active":"2024-09-10T21:29:38.544954+0000","last_became_peered":"2024-09-10T21:29:38.544954+0000","last_unstale":"2024-09-10T21:29:40.768565+0000","last_undegraded":"2024-09-10T21:29:40.768565+0000","last_fullsized":"2024-09-10T21:29:40.768565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768528+0000","last_change":"2024-09-10T21:29:39.555635+0000","last_active":"2024-09-10T21:29:40.768528+0000","last_peered":"2024-09-10T21:29:40.768528+0000","last_clean":"2024-09-10T21:29:40.768528+0000","last_became_active":"2024-09-10T21:29:39.555011+0000","last_became_peered":"2024-09-10T21:29:39.555011+0000","last_unstale":"2024-09-10T21:29:40.768528+0000","last_undegraded":"2024-09-10T21:29:40.768528+0000","last_fullsized":"2024-09-10T21:29:40.768528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:15.429757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545012+0000","last_change":"2024-09-10T21:29:38.549092+0000","last_active":"2024-09-10T21:29:40.545012+0000","last_peered":"2024-09-10T21:29:40.545012+0000","last_clean":"2024-09-10T21:29:40.545012+0000","last_became_active":"2024-09-10T21:29:38.548800+0000","last_became_peered":"2024-09-10T21:29:38.548800+0000","last_unstale":"2024-09-10T21:29:40.545012+0000","last_undegraded":"2024-09-10T21:29:40.545012+0000","last_fullsized":"2024-09-10T21:29:40.545012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:10.710966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768928+0000","last_change":"2024-09-10T21:29:39.554166+0000","last_active":"2024-09-10T21:29:40.768928+0000","last_peered":"2024-09-10T21:29:40.768928+0000","last_clean":"2024-09-10T21:29:40.768928+0000","last_became_active":"2024-09-10T21:29:39.551856+0000","last_became_peered":"2024-09-10T21:29:39.551856+0000","last_unstale":"2024-09-10T21:29:40.768928+0000","last_undegraded":"2024-09-10T21:29:40.768928+0000","last_fullsized":"2024-09-10T21:29:40.768928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:39.611740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769311+0000","last_change":"2024-09-10T21:29:38.545177+0000","last_active":"2024-09-10T21:29:40.769311+0000","last_peered":"2024-09-10T21:29:40.769311+0000","last_clean":"2024-09-10T21:29:40.769311+0000","last_became_active":"2024-09-10T21:29:38.544523+0000","last_became_peered":"2024-09-10T21:29:38.544523+0000","last_unstale":"2024-09-10T21:29:40.769311+0000","last_undegraded":"2024-09-10T21:29:40.769311+0000","last_fullsized":"2024-09-10T21:29:40.769311+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:36.904722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769325+0000","last_change":"2024-09-10T21:29:39.536154+0000","last_active":"2024-09-10T21:29:40.769325+0000","last_peered":"2024-09-10T21:29:40.769325+0000","last_clean":"2024-09-10T21:29:40.769325+0000","last_became_active":"2024-09-10T21:29:39.535816+0000","last_became_peered":"2024-09-10T21:29:39.535816+0000","last_unstale":"2024-09-10T21:29:40.769325+0000","last_undegraded":"2024-09-10T21:29:40.769325+0000","last_fullsized":"2024-09-10T21:29:40.769325+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:41.370702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546787+0000","last_change":"2024-09-10T21:29:38.542661+0000","last_active":"2024-09-10T21:29:40.546787+0000","last_peered":"2024-09-10T21:29:40.546787+0000","last_clean":"2024-09-10T21:29:40.546787+0000","last_became_active":"2024-09-10T21:29:38.542425+0000","last_became_peered":"2024-09-10T21:29:38.542425+0000","last_unstale":"2024-09-10T21:29:40.546787+0000","last_undegraded":"2024-09-10T21:29:40.546787+0000","last_fullsized":"2024-09-10T21:29:40.546787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:47.295559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545841+0000","last_change":"2024-09-10T21:29:39.548032+0000","last_active":"2024-09-10T21:29:40.545841+0000","last_peered":"2024-09-10T21:29:40.545841+0000","last_clean":"2024-09-10T21:29:40.545841+0000","last_became_active":"2024-09-10T21:29:39.547709+0000","last_became_peered":"2024-09-10T21:29:39.547709+0000","last_unstale":"2024-09-10T21:29:40.545841+0000","last_undegraded":"2024-09-10T21:29:40.545841+0000","last_fullsized":"2024-09-10T21:29:40.545841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:43.662976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769782+0000","last_change":"2024-09-10T21:29:38.532633+0000","last_active":"2024-09-10T21:29:40.769782+0000","last_peered":"2024-09-10T21:29:40.769782+0000","last_clean":"2024-09-10T21:29:40.769782+0000","last_became_active":"2024-09-10T21:29:38.532371+0000","last_became_peered":"2024-09-10T21:29:38.532371+0000","last_unstale":"2024-09-10T21:29:40.769782+0000","last_undegraded":"2024-09-10T21:29:40.769782+0000","last_fullsized":"2024-09-10T21:29:40.769782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:45.933789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544716+0000","last_change":"2024-09-10T21:29:39.554909+0000","last_active":"2024-09-10T21:29:40.544716+0000","last_peered":"2024-09-10T21:29:40.544716+0000","last_clean":"2024-09-10T21:29:40.544716+0000","last_became_active":"2024-09-10T21:29:39.554651+0000","last_became_peered":"2024-09-10T21:29:39.554651+0000","last_unstale":"2024-09-10T21:29:40.544716+0000","last_undegraded":"2024-09-10T21:29:40.544716+0000","last_fullsized":"2024-09-10T21:29:40.544716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:31.876273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768411+0000","last_change":"2024-09-10T21:29:38.535431+0000","last_active":"2024-09-10T21:29:40.768411+0000","last_peered":"2024-09-10T21:29:40.768411+0000","last_clean":"2024-09-10T21:29:40.768411+0000","last_became_active":"2024-09-10T21:29:38.535114+0000","last_became_peered":"2024-09-10T21:29:38.535114+0000","last_unstale":"2024-09-10T21:29:40.768411+0000","last_undegraded":"2024-09-10T21:29:40.768411+0000","last_fullsized":"2024-09-10T21:29:40.768411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:05.208145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768791+0000","last_change":"2024-09-10T21:29:39.563604+0000","last_active":"2024-09-10T21:29:40.768791+0000","last_peered":"2024-09-10T21:29:40.768791+0000","last_clean":"2024-09-10T21:29:40.768791+0000","last_became_active":"2024-09-10T21:29:39.563344+0000","last_became_peered":"2024-09-10T21:29:39.563344+0000","last_unstale":"2024-09-10T21:29:40.768791+0000","last_undegraded":"2024-09-10T21:29:40.768791+0000","last_fullsized":"2024-09-10T21:29:40.768791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:50.487501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769250+0000","last_change":"2024-09-10T21:29:38.544304+0000","last_active":"2024-09-10T21:29:40.769250+0000","last_peered":"2024-09-10T21:29:40.769250+0000","last_clean":"2024-09-10T21:29:40.769250+0000","last_became_active":"2024-09-10T21:29:38.544049+0000","last_became_peered":"2024-09-10T21:29:38.544049+0000","last_unstale":"2024-09-10T21:29:40.769250+0000","last_undegraded":"2024-09-10T21:29:40.769250+0000","last_fullsized":"2024-09-10T21:29:40.769250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768692+0000","last_change":"2024-09-10T21:29:39.544825+0000","last_active":"2024-09-10T21:29:40.768692+0000","last_peered":"2024-09-10T21:29:40.768692+0000","last_clean":"2024-09-10T21:29:40.768692+0000","last_became_active":"2024-09-10T21:29:39.544578+0000","last_became_peered":"2024-09-10T21:29:39.544578+0000","last_unstale":"2024-09-10T21:29:40.768692+0000","last_undegraded":"2024-09-10T21:29:40.768692+0000","last_fullsized":"2024-09-10T21:29:40.768692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:17.043812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546585+0000","last_change":"2024-09-10T21:29:38.549765+0000","last_active":"2024-09-10T21:29:40.546585+0000","last_peered":"2024-09-10T21:29:40.546585+0000","last_clean":"2024-09-10T21:29:40.546585+0000","last_became_active":"2024-09-10T21:29:38.548990+0000","last_became_peered":"2024-09-10T21:29:38.548990+0000","last_unstale":"2024-09-10T21:29:40.546585+0000","last_undegraded":"2024-09-10T21:29:40.546585+0000","last_fullsized":"2024-09-10T21:29:40.546585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:52.559734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768695+0000","last_change":"2024-09-10T21:29:39.547930+0000","last_active":"2024-09-10T21:29:40.768695+0000","last_peered":"2024-09-10T21:29:40.768695+0000","last_clean":"2024-09-10T21:29:40.768695+0000","last_became_active":"2024-09-10T21:29:39.547672+0000","last_became_peered":"2024-09-10T21:29:39.547672+0000","last_unstale":"2024-09-10T21:29:40.768695+0000","last_undegraded":"2024-09-10T21:29:40.768695+0000","last_fullsized":"2024-09-10T21:29:40.768695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:52.513550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545358+0000","last_change":"2024-09-10T21:29:38.544208+0000","last_active":"2024-09-10T21:29:40.545358+0000","last_peered":"2024-09-10T21:29:40.545358+0000","last_clean":"2024-09-10T21:29:40.545358+0000","last_became_active":"2024-09-10T21:29:38.543907+0000","last_became_peered":"2024-09-10T21:29:38.543907+0000","last_unstale":"2024-09-10T21:29:40.545358+0000","last_undegraded":"2024-09-10T21:29:40.545358+0000","last_fullsized":"2024-09-10T21:29:40.545358+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:37.225392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546397+0000","last_change":"2024-09-10T21:29:39.557707+0000","last_active":"2024-09-10T21:29:40.546397+0000","last_peered":"2024-09-10T21:29:40.546397+0000","last_clean":"2024-09-10T21:29:40.546397+0000","last_became_active":"2024-09-10T21:29:39.557398+0000","last_became_peered":"2024-09-10T21:29:39.557398+0000","last_unstale":"2024-09-10T21:29:40.546397+0000","last_undegraded":"2024-09-10T21:29:40.546397+0000","last_fullsized":"2024-09-10T21:29:40.546397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:05.442260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545105+0000","last_change":"2024-09-10T21:29:38.549044+0000","last_active":"2024-09-10T21:29:40.545105+0000","last_peered":"2024-09-10T21:29:40.545105+0000","last_clean":"2024-09-10T21:29:40.545105+0000","last_became_active":"2024-09-10T21:29:38.548465+0000","last_became_peered":"2024-09-10T21:29:38.548465+0000","last_unstale":"2024-09-10T21:29:40.545105+0000","last_undegraded":"2024-09-10T21:29:40.545105+0000","last_fullsized":"2024-09-10T21:29:40.545105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:47.729570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769221+0000","last_change":"2024-09-10T21:29:39.555291+0000","last_active":"2024-09-10T21:29:40.769221+0000","last_peered":"2024-09-10T21:29:40.769221+0000","last_clean":"2024-09-10T21:29:40.769221+0000","last_became_active":"2024-09-10T21:29:39.554979+0000","last_became_peered":"2024-09-10T21:29:39.554979+0000","last_unstale":"2024-09-10T21:29:40.769221+0000","last_undegraded":"2024-09-10T21:29:40.769221+0000","last_fullsized":"2024-09-10T21:29:40.769221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:41.126386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544896+0000","last_change":"2024-09-10T21:29:38.545682+0000","last_active":"2024-09-10T21:29:40.544896+0000","last_peered":"2024-09-10T21:29:40.544896+0000","last_clean":"2024-09-10T21:29:40.544896+0000","last_became_active":"2024-09-10T21:29:38.545432+0000","last_became_peered":"2024-09-10T21:29:38.545432+0000","last_unstale":"2024-09-10T21:29:40.544896+0000","last_undegraded":"2024-09-10T21:29:40.544896+0000","last_fullsized":"2024-09-10T21:29:40.544896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769268+0000","last_change":"2024-09-10T21:29:39.560953+0000","last_active":"2024-09-10T21:29:40.769268+0000","last_peered":"2024-09-10T21:29:40.769268+0000","last_clean":"2024-09-10T21:29:40.769268+0000","last_became_active":"2024-09-10T21:29:39.560702+0000","last_became_peered":"2024-09-10T21:29:39.560702+0000","last_unstale":"2024-09-10T21:29:40.769268+0000","last_undegraded":"2024-09-10T21:29:40.769268+0000","last_fullsized":"2024-09-10T21:29:40.769268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:05.821463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.546758+0000","last_change":"2024-09-10T21:29:38.550165+0000","last_active":"2024-09-10T21:29:40.546758+0000","last_peered":"2024-09-10T21:29:40.546758+0000","last_clean":"2024-09-10T21:29:40.546758+0000","last_became_active":"2024-09-10T21:29:38.549861+0000","last_became_peered":"2024-09-10T21:29:38.549861+0000","last_unstale":"2024-09-10T21:29:40.546758+0000","last_undegraded":"2024-09-10T21:29:40.546758+0000","last_fullsized":"2024-09-10T21:29:40.546758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:02.189449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547170+0000","last_change":"2024-09-10T21:29:39.545985+0000","last_active":"2024-09-10T21:29:40.547170+0000","last_peered":"2024-09-10T21:29:40.547170+0000","last_clean":"2024-09-10T21:29:40.547170+0000","last_became_active":"2024-09-10T21:29:39.545349+0000","last_became_peered":"2024-09-10T21:29:39.545349+0000","last_unstale":"2024-09-10T21:29:40.547170+0000","last_undegraded":"2024-09-10T21:29:40.547170+0000","last_fullsized":"2024-09-10T21:29:40.547170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:14.688355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547653+0000","last_change":"2024-09-10T21:29:38.546910+0000","last_active":"2024-09-10T21:29:40.547653+0000","last_peered":"2024-09-10T21:29:40.547653+0000","last_clean":"2024-09-10T21:29:40.547653+0000","last_became_active":"2024-09-10T21:29:38.546159+0000","last_became_peered":"2024-09-10T21:29:38.546159+0000","last_unstale":"2024-09-10T21:29:40.547653+0000","last_undegraded":"2024-09-10T21:29:40.547653+0000","last_fullsized":"2024-09-10T21:29:40.547653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.324328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547733+0000","last_change":"2024-09-10T21:29:39.551802+0000","last_active":"2024-09-10T21:29:40.547733+0000","last_peered":"2024-09-10T21:29:40.547733+0000","last_clean":"2024-09-10T21:29:40.547733+0000","last_became_active":"2024-09-10T21:29:39.550894+0000","last_became_peered":"2024-09-10T21:29:39.550894+0000","last_unstale":"2024-09-10T21:29:40.547733+0000","last_undegraded":"2024-09-10T21:29:40.547733+0000","last_fullsized":"2024-09-10T21:29:40.547733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:35.976956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768722+0000","last_change":"2024-09-10T21:29:38.549752+0000","last_active":"2024-09-10T21:29:40.768722+0000","last_peered":"2024-09-10T21:29:40.768722+0000","last_clean":"2024-09-10T21:29:40.768722+0000","last_became_active":"2024-09-10T21:29:38.549380+0000","last_became_peered":"2024-09-10T21:29:38.549380+0000","last_unstale":"2024-09-10T21:29:40.768722+0000","last_undegraded":"2024-09-10T21:29:40.768722+0000","last_fullsized":"2024-09-10T21:29:40.768722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547195+0000","last_change":"2024-09-10T21:29:39.544277+0000","last_active":"2024-09-10T21:29:40.547195+0000","last_peered":"2024-09-10T21:29:40.547195+0000","last_clean":"2024-09-10T21:29:40.547195+0000","last_became_active":"2024-09-10T21:29:39.544011+0000","last_became_peered":"2024-09-10T21:29:39.544011+0000","last_unstale":"2024-09-10T21:29:40.547195+0000","last_undegraded":"2024-09-10T21:29:40.547195+0000","last_fullsized":"2024-09-10T21:29:40.547195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:05.999146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768535+0000","last_change":"2024-09-10T21:29:38.544149+0000","last_active":"2024-09-10T21:29:40.768535+0000","last_peered":"2024-09-10T21:29:40.768535+0000","last_clean":"2024-09-10T21:29:40.768535+0000","last_became_active":"2024-09-10T21:29:38.543790+0000","last_became_peered":"2024-09-10T21:29:38.543790+0000","last_unstale":"2024-09-10T21:29:40.768535+0000","last_undegraded":"2024-09-10T21:29:40.768535+0000","last_fullsized":"2024-09-10T21:29:40.768535+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:28.591046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.770023+0000","last_change":"2024-09-10T21:29:39.545983+0000","last_active":"2024-09-10T21:29:40.770023+0000","last_peered":"2024-09-10T21:29:40.770023+0000","last_clean":"2024-09-10T21:29:40.770023+0000","last_became_active":"2024-09-10T21:29:39.545699+0000","last_became_peered":"2024-09-10T21:29:39.545699+0000","last_unstale":"2024-09-10T21:29:40.770023+0000","last_undegraded":"2024-09-10T21:29:40.770023+0000","last_fullsized":"2024-09-10T21:29:40.770023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:27.110951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768649+0000","last_change":"2024-09-10T21:29:38.546518+0000","last_active":"2024-09-10T21:29:40.768649+0000","last_peered":"2024-09-10T21:29:40.768649+0000","last_clean":"2024-09-10T21:29:40.768649+0000","last_became_active":"2024-09-10T21:29:38.546262+0000","last_became_peered":"2024-09-10T21:29:38.546262+0000","last_unstale":"2024-09-10T21:29:40.768649+0000","last_undegraded":"2024-09-10T21:29:40.768649+0000","last_fullsized":"2024-09-10T21:29:40.768649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:52.560186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544409+0000","last_change":"2024-09-10T21:29:39.554112+0000","last_active":"2024-09-10T21:29:40.544409+0000","last_peered":"2024-09-10T21:29:40.544409+0000","last_clean":"2024-09-10T21:29:40.544409+0000","last_became_active":"2024-09-10T21:29:39.552077+0000","last_became_peered":"2024-09-10T21:29:39.552077+0000","last_unstale":"2024-09-10T21:29:40.544409+0000","last_undegraded":"2024-09-10T21:29:40.544409+0000","last_fullsized":"2024-09-10T21:29:40.544409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:18.545317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769044+0000","last_change":"2024-09-10T21:29:38.544002+0000","last_active":"2024-09-10T21:29:40.769044+0000","last_peered":"2024-09-10T21:29:40.769044+0000","last_clean":"2024-09-10T21:29:40.769044+0000","last_became_active":"2024-09-10T21:29:38.543699+0000","last_became_peered":"2024-09-10T21:29:38.543699+0000","last_unstale":"2024-09-10T21:29:40.769044+0000","last_undegraded":"2024-09-10T21:29:40.769044+0000","last_fullsized":"2024-09-10T21:29:40.769044+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:21.454494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768948+0000","last_change":"2024-09-10T21:29:39.558189+0000","last_active":"2024-09-10T21:29:40.768948+0000","last_peered":"2024-09-10T21:29:40.768948+0000","last_clean":"2024-09-10T21:29:40.768948+0000","last_became_active":"2024-09-10T21:29:39.557549+0000","last_became_peered":"2024-09-10T21:29:39.557549+0000","last_unstale":"2024-09-10T21:29:40.768948+0000","last_undegraded":"2024-09-10T21:29:40.768948+0000","last_fullsized":"2024-09-10T21:29:40.768948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:48.581722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:41.321787+0000","last_change":"2024-09-10T21:29:38.544536+0000","last_active":"2024-09-10T21:29:41.321787+0000","last_peered":"2024-09-10T21:29:41.321787+0000","last_clean":"2024-09-10T21:29:41.321787+0000","last_became_active":"2024-09-10T21:29:38.544212+0000","last_became_peered":"2024-09-10T21:29:38.544212+0000","last_unstale":"2024-09-10T21:29:41.321787+0000","last_undegraded":"2024-09-10T21:29:41.321787+0000","last_fullsized":"2024-09-10T21:29:41.321787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:53.514589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769300+0000","last_change":"2024-09-10T21:29:39.546810+0000","last_active":"2024-09-10T21:29:40.769300+0000","last_peered":"2024-09-10T21:29:40.769300+0000","last_clean":"2024-09-10T21:29:40.769300+0000","last_became_active":"2024-09-10T21:29:39.546317+0000","last_became_peered":"2024-09-10T21:29:39.546317+0000","last_unstale":"2024-09-10T21:29:40.769300+0000","last_undegraded":"2024-09-10T21:29:40.769300+0000","last_fullsized":"2024-09-10T21:29:40.769300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:46.343210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768987+0000","last_change":"2024-09-10T21:29:38.533647+0000","last_active":"2024-09-10T21:29:40.768987+0000","last_peered":"2024-09-10T21:29:40.768987+0000","last_clean":"2024-09-10T21:29:40.768987+0000","last_became_active":"2024-09-10T21:29:38.533361+0000","last_became_peered":"2024-09-10T21:29:38.533361+0000","last_unstale":"2024-09-10T21:29:40.768987+0000","last_undegraded":"2024-09-10T21:29:40.768987+0000","last_fullsized":"2024-09-10T21:29:40.768987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:32.782811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768884+0000","last_change":"2024-09-10T21:29:39.539314+0000","last_active":"2024-09-10T21:29:40.768884+0000","last_peered":"2024-09-10T21:29:40.768884+0000","last_clean":"2024-09-10T21:29:40.768884+0000","last_became_active":"2024-09-10T21:29:39.539015+0000","last_became_peered":"2024-09-10T21:29:39.539015+0000","last_unstale":"2024-09-10T21:29:40.768884+0000","last_undegraded":"2024-09-10T21:29:40.768884+0000","last_fullsized":"2024-09-10T21:29:40.768884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:19.770402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544676+0000","last_change":"2024-09-10T21:29:38.547087+0000","last_active":"2024-09-10T21:29:40.544676+0000","last_peered":"2024-09-10T21:29:40.544676+0000","last_clean":"2024-09-10T21:29:40.544676+0000","last_became_active":"2024-09-10T21:29:38.546829+0000","last_became_peered":"2024-09-10T21:29:38.546829+0000","last_unstale":"2024-09-10T21:29:40.544676+0000","last_undegraded":"2024-09-10T21:29:40.544676+0000","last_fullsized":"2024-09-10T21:29:40.544676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:29.118144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769511+0000","last_change":"2024-09-10T21:29:39.550392+0000","last_active":"2024-09-10T21:29:40.769511+0000","last_peered":"2024-09-10T21:29:40.769511+0000","last_clean":"2024-09-10T21:29:40.769511+0000","last_became_active":"2024-09-10T21:29:39.550131+0000","last_became_peered":"2024-09-10T21:29:39.550131+0000","last_unstale":"2024-09-10T21:29:40.769511+0000","last_undegraded":"2024-09-10T21:29:40.769511+0000","last_fullsized":"2024-09-10T21:29:40.769511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:15.885819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769553+0000","last_change":"2024-09-10T21:29:38.549285+0000","last_active":"2024-09-10T21:29:40.769553+0000","last_peered":"2024-09-10T21:29:40.769553+0000","last_clean":"2024-09-10T21:29:40.769553+0000","last_became_active":"2024-09-10T21:29:38.549014+0000","last_became_peered":"2024-09-10T21:29:38.549014+0000","last_unstale":"2024-09-10T21:29:40.769553+0000","last_undegraded":"2024-09-10T21:29:40.769553+0000","last_fullsized":"2024-09-10T21:29:40.769553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:58.754314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544681+0000","last_change":"2024-09-10T21:29:39.562581+0000","last_active":"2024-09-10T21:29:40.544681+0000","last_peered":"2024-09-10T21:29:40.544681+0000","last_clean":"2024-09-10T21:29:40.544681+0000","last_became_active":"2024-09-10T21:29:39.562287+0000","last_became_peered":"2024-09-10T21:29:39.562287+0000","last_unstale":"2024-09-10T21:29:40.544681+0000","last_undegraded":"2024-09-10T21:29:40.544681+0000","last_fullsized":"2024-09-10T21:29:40.544681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:38.153165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545978+0000","last_change":"2024-09-10T21:29:38.546345+0000","last_active":"2024-09-10T21:29:40.545978+0000","last_peered":"2024-09-10T21:29:40.545978+0000","last_clean":"2024-09-10T21:29:40.545978+0000","last_became_active":"2024-09-10T21:29:38.546094+0000","last_became_peered":"2024-09-10T21:29:38.546094+0000","last_unstale":"2024-09-10T21:29:40.545978+0000","last_undegraded":"2024-09-10T21:29:40.545978+0000","last_fullsized":"2024-09-10T21:29:40.545978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:04.997313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769644+0000","last_change":"2024-09-10T21:29:39.558347+0000","last_active":"2024-09-10T21:29:40.769644+0000","last_peered":"2024-09-10T21:29:40.769644+0000","last_clean":"2024-09-10T21:29:40.769644+0000","last_became_active":"2024-09-10T21:29:39.557760+0000","last_became_peered":"2024-09-10T21:29:39.557760+0000","last_unstale":"2024-09-10T21:29:40.769644+0000","last_undegraded":"2024-09-10T21:29:40.769644+0000","last_fullsized":"2024-09-10T21:29:40.769644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:06.975636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.544238+0000","last_change":"2024-09-10T21:29:38.535077+0000","last_active":"2024-09-10T21:29:40.544238+0000","last_peered":"2024-09-10T21:29:40.544238+0000","last_clean":"2024-09-10T21:29:40.544238+0000","last_became_active":"2024-09-10T21:29:38.534747+0000","last_became_peered":"2024-09-10T21:29:38.534747+0000","last_unstale":"2024-09-10T21:29:40.544238+0000","last_undegraded":"2024-09-10T21:29:40.544238+0000","last_fullsized":"2024-09-10T21:29:40.544238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:11.268308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.547454+0000","last_change":"2024-09-10T21:29:39.548356+0000","last_active":"2024-09-10T21:29:40.547454+0000","last_peered":"2024-09-10T21:29:40.547454+0000","last_clean":"2024-09-10T21:29:40.547454+0000","last_became_active":"2024-09-10T21:29:39.548096+0000","last_became_peered":"2024-09-10T21:29:39.548096+0000","last_unstale":"2024-09-10T21:29:40.547454+0000","last_undegraded":"2024-09-10T21:29:40.547454+0000","last_fullsized":"2024-09-10T21:29:40.547454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:50.836505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768672+0000","last_change":"2024-09-10T21:29:38.544063+0000","last_active":"2024-09-10T21:29:40.768672+0000","last_peered":"2024-09-10T21:29:40.768672+0000","last_clean":"2024-09-10T21:29:40.768672+0000","last_became_active":"2024-09-10T21:29:38.543508+0000","last_became_peered":"2024-09-10T21:29:38.543508+0000","last_unstale":"2024-09-10T21:29:40.768672+0000","last_undegraded":"2024-09-10T21:29:40.768672+0000","last_fullsized":"2024-09-10T21:29:40.768672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:02.927923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.769972+0000","last_change":"2024-09-10T21:29:39.552135+0000","last_active":"2024-09-10T21:29:40.769972+0000","last_peered":"2024-09-10T21:29:40.769972+0000","last_clean":"2024-09-10T21:29:40.769972+0000","last_became_active":"2024-09-10T21:29:39.551649+0000","last_became_peered":"2024-09-10T21:29:39.551649+0000","last_unstale":"2024-09-10T21:29:40.769972+0000","last_undegraded":"2024-09-10T21:29:40.769972+0000","last_fullsized":"2024-09-10T21:29:40.769972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:38.513546+0000","last_clean_scrub_stamp":"2024-09-10T21:29:38.513546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:26.827601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.545785+0000","last_change":"2024-09-10T21:29:38.547451+0000","last_active":"2024-09-10T21:29:40.545785+0000","last_peered":"2024-09-10T21:29:40.545785+0000","last_clean":"2024-09-10T21:29:40.545785+0000","last_became_active":"2024-09-10T21:29:38.546360+0000","last_became_peered":"2024-09-10T21:29:38.546360+0000","last_unstale":"2024-09-10T21:29:40.545785+0000","last_undegraded":"2024-09-10T21:29:40.545785+0000","last_fullsized":"2024-09-10T21:29:40.545785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:29:37.510514+0000","last_clean_scrub_stamp":"2024-09-10T21:29:37.510514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:59.005921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":225,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T21:29:40.768220+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:29:40.768220+0000","last_peered":"2024-09-10T21:29:40.768220+0000","last_clean":"2024-09-10T21:29:40.768220+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:29:40.768220+0000","last_undegraded":"2024-09-10T21:29:40.768220+0000","last_fullsized":"2024-09-10T21:29:40.768220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":458787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509511,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1003520,"data_stored":458807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1593344,"data_stored":1048593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1015808,"data_stored":459851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":458135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509513,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1605632,"data_stored":1050149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":999424,"data_stored":458157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:29:48.577 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:29:48.577 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:29:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- 172.21.15.97:0/2130483218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 msgr2=0x7f9e740b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 --2- 172.21.15.97:0/2130483218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e740b2a80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009bc0 tx=0x7f9e6401cba0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- 172.21.15.97:0/2130483218 shutdown_connections 2024-09-10T21:29:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 --2- 172.21.15.97:0/2130483218 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e7406fd50 0x7f9e740701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 --2- 172.21.15.97:0/2130483218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e740b2a80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- 172.21.15.97:0/2130483218 >> 172.21.15.97:0/2130483218 conn(0x7f9e740b1780 msgr2=0x7f9e740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- 172.21.15.97:0/2130483218 shutdown_connections 2024-09-10T21:29:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- 172.21.15.97:0/2130483218 wait complete. 2024-09-10T21:29:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 Processor -- start 2024-09-10T21:29:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- start start 2024-09-10T21:29:48.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e7406fd50 0x7f9e740aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e7415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e74144690 0x7f9e7415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74167c50 con 0x7f9e74143f90 2024-09-10T21:29:48.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74167ad0 con 0x7f9e7406fd50 2024-09-10T21:29:48.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e7a600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9e74167dd0 con 0x7f9e74144690 2024-09-10T21:29:48.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e7415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e7415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45174/0 (socket says 172.21.15.97:45174) 2024-09-10T21:29:48.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 -- 172.21.15.97:0/732965807 learned_addr learned my addr 172.21.15.97:0/732965807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e73e00640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e74144690 0x7f9e7415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 -- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e74144690 msgr2=0x7f9e7415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e74144690 0x7f9e7415ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e73400640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e7406fd50 0x7f9e740aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 -- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e7406fd50 msgr2=0x7f9e740aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e7406fd50 0x7f9e740aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e6aa00640 1 -- 172.21.15.97:0/732965807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7414d540 con 0x7f9e74143f90 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e73e00640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e74144690 0x7f9e7415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:48.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.761+0000 7f9e73400640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e7406fd50 0x7f9e740aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e6aa00640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e7415e680 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00c460 tx=0x7f9e5c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e6be00640 1 -- 172.21.15.97:0/732965807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c016020 con 0x7f9e74143f90 2024-09-10T21:29:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e6be00640 1 -- 172.21.15.97:0/732965807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e5c012940 con 0x7f9e74143f90 2024-09-10T21:29:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e6be00640 1 -- 172.21.15.97:0/732965807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c012d90 con 0x7f9e74143f90 2024-09-10T21:29:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7414d820 con 0x7f9e74143f90 2024-09-10T21:29:48.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7414dde0 con 0x7f9e74143f90 2024-09-10T21:29:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74136dc0 con 0x7f9e74143f90 2024-09-10T21:29:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e6be00640 1 -- 172.21.15.97:0/732965807 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e5c01b400 con 0x7f9e74143f90 2024-09-10T21:29:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e6be00640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e500725e0 0x7f9e50074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:48.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e6be00640 1 -- 172.21.15.97:0/732965807 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f9e5c093690 con 0x7f9e74143f90 2024-09-10T21:29:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.765+0000 7f9e73400640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e500725e0 0x7f9e50074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:48.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.769+0000 7f9e73400640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e500725e0 0x7f9e50074aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e640af000 tx=0x7f9e6401b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:48.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.769+0000 7f9e6be00640 1 -- 172.21.15.97:0/732965807 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e74136dc0 con 0x7f9e74143f90 2024-09-10T21:29:48.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.993+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9e740b2da0 con 0x7f9e74143f90 2024-09-10T21:29:48.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:48.997+0000 7f9e6be00640 1 -- 172.21.15.97:0/732965807 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f9e5c020020 con 0x7f9e74143f90 2024-09-10T21:29:49.000 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-10T21:29:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:29:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-09-10T21:29:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-09-10T21:29:49.000 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:29:49.000 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-09-10T21:29:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e500725e0 msgr2=0x7f9e50074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e500725e0 0x7f9e50074aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9e640af000 tx=0x7f9e6401b3e0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 msgr2=0x7f9e7415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e7415e680 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00c460 tx=0x7f9e5c00c920 comp rx=0 tx=0).stop 2024-09-10T21:29:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 shutdown_connections 2024-09-10T21:29:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e500725e0 0x7f9e50074aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e74144690 0x7f9e7415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e74143f90 0x7f9e7415e680 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 --2- 172.21.15.97:0/732965807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e7406fd50 0x7f9e740aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 >> 172.21.15.97:0/732965807 conn(0x7f9e740b1780 msgr2=0x7f9e740cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 shutdown_connections 2024-09-10T21:29:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.001+0000 7f9e7a600640 1 -- 172.21.15.97:0/732965807 wait complete. 2024-09-10T21:29:49.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 -- 172.21.15.97:0/136794851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 msgr2=0x7f1ed40d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 --2- 172.21.15.97:0/136794851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed40d11f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009840 tx=0x7f1ec001c700 comp rx=0 tx=0).stop 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 -- 172.21.15.97:0/136794851 shutdown_connections 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 --2- 172.21.15.97:0/136794851 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ed40d29d0 0x7f1ed40cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 --2- 172.21.15.97:0/136794851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed40d11f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 -- 172.21.15.97:0/136794851 >> 172.21.15.97:0/136794851 conn(0x7f1ed40d3ce0 msgr2=0x7f1ed40d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 -- 172.21.15.97:0/136794851 shutdown_connections 2024-09-10T21:29:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 -- 172.21.15.97:0/136794851 wait complete. 2024-09-10T21:29:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 Processor -- start 2024-09-10T21:29:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.161+0000 7f1ed9000640 1 -- start start 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ed40d29d0 0x7f1ed40c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ed4161110 0x7f1ed41560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed416b060 con 0x7f1ed41614f0 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed416aee0 con 0x7f1ed4161110 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed416b1e0 con 0x7f1ed40d29d0 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed4156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45196/0 (socket says 172.21.15.97:45196) 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 -- 172.21.15.97:0/2386080273 learned_addr learned my addr 172.21.15.97:0/2386080273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed2a00640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ed4161110 0x7f1ed41560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3400640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ed40d29d0 0x7f1ed40c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 -- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ed40d29d0 msgr2=0x7f1ed40c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ed40d29d0 0x7f1ed40c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 -- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ed4161110 msgr2=0x7f1ed41560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ed4161110 0x7f1ed41560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 -- 172.21.15.97:0/2386080273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed41f7a10 con 0x7f1ed41614f0 2024-09-10T21:29:49.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed3e00640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed4156d10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00da40 tx=0x7f1ecc00df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ecb400640 1 -- 172.21.15.97:0/2386080273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc017070 con 0x7f1ed41614f0 2024-09-10T21:29:49.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ecb400640 1 -- 172.21.15.97:0/2386080273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ecc003980 con 0x7f1ed41614f0 2024-09-10T21:29:49.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ecb400640 1 -- 172.21.15.97:0/2386080273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003db0 con 0x7f1ed41614f0 2024-09-10T21:29:49.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed41fb480 con 0x7f1ed41614f0 2024-09-10T21:29:49.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41fbd20 con 0x7f1ed41614f0 2024-09-10T21:29:49.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed41f7be0 con 0x7f1ed41614f0 2024-09-10T21:29:49.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.165+0000 7f1ecb400640 1 -- 172.21.15.97:0/2386080273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ecc003f50 con 0x7f1ed41614f0 2024-09-10T21:29:49.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.169+0000 7f1ecb400640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1eb40725e0 0x7f1eb4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.169+0000 7f1ecb400640 1 -- 172.21.15.97:0/2386080273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f1ecc0937a0 con 0x7f1ed41614f0 2024-09-10T21:29:49.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.169+0000 7f1ed3400640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1eb40725e0 0x7f1eb4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.169+0000 7f1ed3400640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1eb40725e0 0x7f1eb4074aa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4000f70 tx=0x7f1ec4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.173+0000 7f1ecb400640 1 -- 172.21.15.97:0/2386080273 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ecc05e6a0 con 0x7f1ed41614f0 2024-09-10T21:29:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.369+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ed40b9c60 con 0x7f1ed41614f0 2024-09-10T21:29:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.369+0000 7f1ecb400640 1 -- 172.21.15.97:0/2386080273 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5231 (secure 0 0 0) 0x7f1ecc0630d0 con 0x7f1ed41614f0 2024-09-10T21:29:49.374 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:49.374 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"btime":"2024-09-10T21:29:42:550041+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6492,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:29:41.936668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6504},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6504":{"gid":6504,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6836/3597857813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3597857813},{"type":"v1","addr":"172.21.15.194:6836","nonce":3597857813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6504,"qdb_cluster":[6504]},"id":5}]} 2024-09-10T21:29:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 70 2024-09-10T21:29:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1eb40725e0 msgr2=0x7f1eb4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1eb40725e0 0x7f1eb4074aa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4000f70 tx=0x7f1ec4009040 comp rx=0 tx=0).stop 2024-09-10T21:29:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 msgr2=0x7f1ed4156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed4156d10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00da40 tx=0x7f1ecc00df00 comp rx=0 tx=0).stop 2024-09-10T21:29:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 shutdown_connections 2024-09-10T21:29:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1eb40725e0 0x7f1eb4074aa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed41614f0 0x7f1ed4156d10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ed4161110 0x7f1ed41560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 --2- 172.21.15.97:0/2386080273 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ed40d29d0 0x7f1ed40c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 >> 172.21.15.97:0/2386080273 conn(0x7f1ed40d3ce0 msgr2=0x7f1ed40d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 shutdown_connections 2024-09-10T21:29:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.373+0000 7f1ed9000640 1 -- 172.21.15.97:0/2386080273 wait complete. 2024-09-10T21:29:49.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:29:38.920694+0000', 'modified': '2024-09-10T21:29:41.936668+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6504}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6504': {'gid': 6504, 'name': 'b', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.194:6836/3597857813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 3597857813}, {'type': 'v1', 'addr': '172.21.15.194:6836', 'nonce': 3597857813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6504, 'qdb_cluster': [6504]} 2024-09-10T21:29:49.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:29:49.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:29:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.941+0000 7f04c9200640 1 -- 172.21.15.97:0/2945916464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 msgr2=0x7f04a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.941+0000 7f04c9200640 1 --2- 172.21.15.97:0/2945916464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 0x7f04a80051d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009eb0 tx=0x7f04bc01d210 comp rx=0 tx=0).stop 2024-09-10T21:29:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.941+0000 7f04c3e00640 1 -- 172.21.15.97:0/2945916464 reap_dead start 2024-09-10T21:29:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.941+0000 7f04c9200640 1 -- 172.21.15.97:0/2945916464 shutdown_connections 2024-09-10T21:29:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.941+0000 7f04c9200640 1 -- 172.21.15.97:0/2945916464 >> 172.21.15.97:0/2945916464 conn(0x7f04c40d1a60 msgr2=0x7f04c40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.941+0000 7f04c9200640 1 -- 172.21.15.97:0/2945916464 shutdown_connections 2024-09-10T21:29:49.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.941+0000 7f04c9200640 1 -- 172.21.15.97:0/2945916464 wait complete. 2024-09-10T21:29:49.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 Processor -- start 2024-09-10T21:29:49.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 -- start start 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 0x7f04c40c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04c40c68c0 0x7f04c40b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04c40b6310 0x7f04c40b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40cfbd0 con 0x7f04a8004d50 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f04c40cfa50 con 0x7f04c40c68c0 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40aee70 con 0x7f04c40b6310 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 0x7f04c40c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 0x7f04c40c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45210/0 (socket says 172.21.15.97:45210) 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 -- 172.21.15.97:0/3497637669 learned_addr learned my addr 172.21.15.97:0/3497637669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04baa00640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04c40c68c0 0x7f04c40b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3e00640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04c40b6310 0x7f04c40b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 -- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04c40b6310 msgr2=0x7f04c40b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04c40b6310 0x7f04c40b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 -- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04c40c68c0 msgr2=0x7f04c40b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04c40c68c0 0x7f04c40b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 -- 172.21.15.97:0/3497637669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c40b6820 con 0x7f04a8004d50 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c3400640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 0x7f04c40c8010 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f04b4011ab0 tx=0x7f04b4011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04bbe00640 1 -- 172.21.15.97:0/3497637669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b401e070 con 0x7f04a8004d50 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04bbe00640 1 -- 172.21.15.97:0/3497637669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b400fe70 con 0x7f04a8004d50 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04bbe00640 1 -- 172.21.15.97:0/3497637669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40042b0 con 0x7f04a8004d50 2024-09-10T21:29:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c40b6b00 con 0x7f04a8004d50 2024-09-10T21:29:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.945+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c40c7b10 con 0x7f04a8004d50 2024-09-10T21:29:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04bbe00640 1 -- 172.21.15.97:0/3497637669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04b4003600 con 0x7f04a8004d50 2024-09-10T21:29:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04bbe00640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04a00725e0 0x7f04a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04bbe00640 1 -- 172.21.15.97:0/3497637669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f04b4094170 con 0x7f04a8004d50 2024-09-10T21:29:49.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04baa00640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04a00725e0 0x7f04a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 Processor -- start 2024-09-10T21:29:49.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 -- start start 2024-09-10T21:29:49.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04baa00640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04a00725e0 0x7f04a0074aa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001870 tx=0x7f04ac001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e10349d0 0x5623e1034db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e0fe88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e0fe8e10 0x5623e0fe91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5623e0f093f0 con 0x5623e10352f0 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5623e0e46b20 con 0x5623e0fe8e10 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5623e0e35f50 con 0x5623e10349d0 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04baa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e0fe88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04baa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e0fe88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45220/0 (socket says 172.21.15.97:45220) 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04baa00640 1 -- 172.21.15.97:0/2563324714 learned_addr learned my addr 172.21.15.97:0/2563324714 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04c3400640 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e10349d0 0x5623e1034db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.949+0000 7f04c3e00640 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e0fe8e10 0x5623e0fe91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 -- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e10349d0 msgr2=0x5623e1034db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e10349d0 0x5623e1034db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 -- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e0fe8e10 msgr2=0x5623e0fe91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e0fe8e10 0x5623e0fe91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 -- 172.21.15.97:0/2563324714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e0dfd590 con 0x5623e10352f0 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04c3e00640 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e0fe8e10 0x5623e0fe91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e0fe88d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001820 tx=0x7f04ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8821719b76cfc0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04b3e00640 1 -- 172.21.15.97:0/2563324714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac012800 con 0x5623e10352f0 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04b3e00640 1 -- 172.21.15.97:0/2563324714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04ac022030 con 0x5623e10352f0 2024-09-10T21:29:49.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04b3e00640 1 -- 172.21.15.97:0/2563324714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac012be0 con 0x5623e10352f0 2024-09-10T21:29:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- 172.21.15.97:0/2563324714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 msgr2=0x5623e0fe88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e0fe88d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001820 tx=0x7f04ac014000 comp rx=0 tx=0).stop 2024-09-10T21:29:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- 172.21.15.97:0/2563324714 shutdown_connections 2024-09-10T21:29:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e0fe8e10 0x5623e0fe91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e0fe88d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2563324714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e10349d0 0x5623e1034db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- 172.21.15.97:0/2563324714 >> 172.21.15.97:0/2563324714 conn(0x5623e0ed2130 msgr2=0x5623e1017e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- 172.21.15.97:0/2563324714 shutdown_connections 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- 172.21.15.97:0/2563324714 wait complete. 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- start start 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 Processor -- start 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e103bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10349d0 0x5623e103c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10352f0 0x5623e103c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5623e0dd05d0 con 0x5623e10349d0 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5623e0e39d60 con 0x5623e10352f0 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5623e0ddc500 con 0x5623e0fe8e10 2024-09-10T21:29:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04cdb47000 10 client.0 _ll_register_callbacks cb 0x5623e0de0810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10349d0 0x5623e103c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10349d0 0x5623e103c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45222/0 (socket says 172.21.15.97:45222) 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 -- 172.21.15.97:0/2363956141 learned_addr learned my addr 172.21.15.97:0/2363956141 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04c3400640 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e103bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04c3e00640 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10352f0 0x5623e103c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 -- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 msgr2=0x5623e103bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e103bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 -- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10352f0 msgr2=0x5623e103c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10352f0 0x5623e103c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.953+0000 7f04baa00640 1 -- 172.21.15.97:0/2363956141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e0dd4170 con 0x5623e10349d0 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04baa00640 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10349d0 0x5623e103c2a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5623e0de7130 tx=0x7f04ac012af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d2618a2abc8939 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:29:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 1 -- 172.21.15.97:0/2363956141 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac036060 con 0x5623e10349d0 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 1 -- 172.21.15.97:0/2363956141 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f04ac012ce0 con 0x5623e10349d0 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623e0dce0f0 con 0x5623e10349d0 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 1 -- 172.21.15.97:0/2363956141 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f04ac027420 con 0x5623e10349d0 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 10 client.6657 fetch_fsmap learned FSMap version 70 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5623e0ebaef0 con 0x5623e10349d0 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 1 -- 172.21.15.97:0/2363956141 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5623e0dce0f0 con 0x5623e10349d0 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 1 -- 172.21.15.97:0/2363956141 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f04ac0275c0 con 0x5623e10349d0 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 10 client.6657 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 10 client.6657 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 20 client.6657 populate_metadata read hostname 'smithi097' 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e1044130 0x5623e1046520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 4 client.6657 mds_command: new command op to 6504 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:29:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 --> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623e0e48a70 con 0x5623e1044130 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04c3400640 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e1044130 0x5623e1046520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04c3400640 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e1044130 0x5623e1046520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c651e6373c918a2 server_cookie=f6bef1030889c18 in_seq=0 out_seq=0 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 10 client.6657 ms_handle_connect on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 1 -- 172.21.15.97:0/2363956141 <== mds.0 v2:172.21.15.194:6834/3597857813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5623e0e48a70 con 0x5623e1044130 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 10 client.6657 handle_command_reply: tid=0 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 client.6657 shutdown 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10349d0 msgr2=0x5623e103c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10349d0 0x5623e103c2a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5623e0de7130 tx=0x7f04ac012af0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 shutdown_connections 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e1044130 0x5623e1046520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10352f0 0x5623e103c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10349d0 0x5623e103c2a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2363956141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e103bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04b0c00640 0 client.6657 ms_handle_reset on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.957+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 >> 172.21.15.97:0/2363956141 conn(0x5623e0ed2130 msgr2=0x5623e0fe7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.961+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 shutdown_connections 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.961+0000 7f04cdb47000 1 -- 172.21.15.97:0/2363956141 wait complete. 2024-09-10T21:29:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.961+0000 7f04cdb47000 20 client.6657 trim_cache size 0 max 16384 2024-09-10T21:29:49.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 Processor -- start 2024-09-10T21:29:49.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 -- start start 2024-09-10T21:29:49.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e10a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4201a40 con 0x5623e10352f0 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4200aa0 con 0x5623e10349d0 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f04c41ffe80 con 0x5623e0fe8e10 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10a78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45228/0 (socket says 172.21.15.97:45228) 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 -- 172.21.15.97:0/2713504190 learned_addr learned my addr 172.21.15.97:0/2713504190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04baa00640 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3400640 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e10a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 -- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 msgr2=0x5623e10a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e10a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 -- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 msgr2=0x5623e10a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 -- 172.21.15.97:0/2713504190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e0e91d90 con 0x5623e10352f0 2024-09-10T21:29:49.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04baa00640 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:49.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.977+0000 7f04c3e00640 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10a78b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f04bc01d040 tx=0x7f04a8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493944caf8c31563 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04b3400640 1 -- 172.21.15.97:0/2713504190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0b1040 con 0x5623e10352f0 2024-09-10T21:29:49.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04b3400640 1 -- 172.21.15.97:0/2713504190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04bc004a50 con 0x5623e10352f0 2024-09-10T21:29:49.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04b3400640 1 -- 172.21.15.97:0/2713504190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc004e40 con 0x5623e10352f0 2024-09-10T21:29:49.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- 172.21.15.97:0/2713504190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 msgr2=0x5623e10a78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10a78b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f04bc01d040 tx=0x7f04a8005840 comp rx=0 tx=0).stop 2024-09-10T21:29:49.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- 172.21.15.97:0/2713504190 shutdown_connections 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10a78b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 --2- 172.21.15.97:0/2713504190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e10a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- 172.21.15.97:0/2713504190 >> 172.21.15.97:0/2713504190 conn(0x5623e0ed2130 msgr2=0x5623e0fe70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- 172.21.15.97:0/2713504190 shutdown_connections 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- 172.21.15.97:0/2713504190 wait complete. 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- start start 2024-09-10T21:29:49.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 Processor -- start 2024-09-10T21:29:49.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e10aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5623e0e01ec0 con 0x5623e10352f0 2024-09-10T21:29:49.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4200450 con 0x5623e10349d0 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5623e0e3e390 con 0x5623e0fe8e10 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 10 client.0 _ll_register_callbacks cb 0x7f04b405b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10ab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45242/0 (socket says 172.21.15.97:45242) 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 -- 172.21.15.97:0/3736839244 learned_addr learned my addr 172.21.15.97:0/3736839244 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 -- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 msgr2=0x5623e10aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e10aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 -- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 msgr2=0x5623e10ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 -- 172.21.15.97:0/3736839244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e0dfca90 con 0x5623e10352f0 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04c3e00640 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10ab540 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009eb0 tx=0x7f04bc005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922ac507a3322c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:29:49.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04b0c00640 1 -- 172.21.15.97:0/3736839244 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0b1040 con 0x5623e10352f0 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04b0c00640 1 -- 172.21.15.97:0/3736839244 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f04bc0c7450 con 0x5623e10352f0 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.981+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04bc004a50 con 0x5623e10352f0 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04b0c00640 1 -- 172.21.15.97:0/3736839244 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f04bc0cf050 con 0x5623e10352f0 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 10 client.6663 fetch_fsmap learned FSMap version 70 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5623e0ddb050 con 0x5623e10352f0 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04b0c00640 1 -- 172.21.15.97:0/3736839244 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04bc004a50 con 0x5623e10352f0 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04b0c00640 1 -- 172.21.15.97:0/3736839244 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f04bc0d2020 con 0x5623e10352f0 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 10 client.6663 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 10 client.6663 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 20 client.6663 populate_metadata read hostname 'smithi097' 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e104a0f0 0x5623e104a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 4 client.6663 mds_command: new command op to 6504 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:29:49.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 --> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] -- command(tid 0: {"prefix": "status"}) -- 0x5623e0e44520 con 0x5623e104a0f0 2024-09-10T21:29:49.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c3400640 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e104a0f0 0x5623e104a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c3400640 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e104a0f0 0x5623e104a4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d281e2255047f7c2 server_cookie=b78f4f666511309b in_seq=0 out_seq=0 2024-09-10T21:29:49.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04b0c00640 10 client.6663 ms_handle_connect on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04b0c00640 1 -- 172.21.15.97:0/3736839244 <== mds.0 v2:172.21.15.194:6834/3597857813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5623e0e44520 con 0x5623e104a0f0 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04b0c00640 10 client.6663 handle_command_reply: tid=0 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 client.6663 shutdown 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 msgr2=0x5623e10ab540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10ab540 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009eb0 tx=0x7f04bc005ed0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 shutdown_connections 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x5623e104a0f0 0x5623e104a4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623e10352f0 0x5623e10ab540 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5623e10349d0 0x5623e10ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 --2- 172.21.15.97:0/3736839244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5623e0fe8e10 0x5623e10aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 >> 172.21.15.97:0/3736839244 conn(0x5623e0ed2130 msgr2=0x5623e0fe78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04b0c00640 0 client.6663 ms_handle_reset on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 shutdown_connections 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 1 -- 172.21.15.97:0/3736839244 wait complete. 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04cdb47000 20 client.6663 trim_cache size 0 max 16384 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6504, 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:29:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:29:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 10.466347315, 2024-09-10T21:29:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 70, 2024-09-10T21:29:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 82, 2024-09-10T21:29:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 82, 2024-09-10T21:29:49.993 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 14.962625327 2024-09-10T21:29:49.993 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04a00725e0 msgr2=0x7f04a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04a00725e0 0x7f04a0074aa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001870 tx=0x7f04ac001520 comp rx=0 tx=0).stop 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 msgr2=0x7f04c40c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 0x7f04c40c8010 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f04b4011ab0 tx=0x7f04b4011f70 comp rx=0 tx=0).stop 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 shutdown_connections 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04a00725e0 0x7f04a0074aa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04c40b6310 0x7f04c40b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04c40c68c0 0x7f04c40b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 --2- 172.21.15.97:0/3497637669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04a8004d50 0x7f04c40c8010 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 >> 172.21.15.97:0/3497637669 conn(0x7f04c40d1a60 msgr2=0x7f04c40cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 shutdown_connections 2024-09-10T21:29:49.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:49.985+0000 7f04c9200640 1 -- 172.21.15.97:0/3497637669 wait complete. 2024-09-10T21:29:50.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:29:50.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 -- 172.21.15.97:0/3132599000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c0051d0 msgr2=0x7f1d7c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:50.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3132599000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c0051d0 0x7f1d7c00b9d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f1d90001dc0 tx=0x7f1d9001cb30 comp rx=0 tx=0).stop 2024-09-10T21:29:50.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 -- 172.21.15.97:0/3132599000 shutdown_connections 2024-09-10T21:29:50.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3132599000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d7c0051d0 0x7f1d7c00b9d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 -- 172.21.15.97:0/3132599000 >> 172.21.15.97:0/3132599000 conn(0x7f1d940b28c0 msgr2=0x7f1d940b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:50.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 -- 172.21.15.97:0/3132599000 shutdown_connections 2024-09-10T21:29:50.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 -- 172.21.15.97:0/3132599000 wait complete. 2024-09-10T21:29:50.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 Processor -- start 2024-09-10T21:29:50.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 -- start start 2024-09-10T21:29:50.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d7c0051d0 0x7f1d941609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:50.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d94155b10 0x7f1d94155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.149+0000 7f1d9c800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d941f8310 0x7f1d941f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9c800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9406d350 con 0x7f1d941f8310 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9c800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9406d1d0 con 0x7f1d7c0051d0 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9c800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9406d4d0 con 0x7f1d94155b10 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d941f8310 0x7f1d941f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d941f8310 0x7f1d941f8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45250/0 (socket says 172.21.15.97:45250) 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 -- 172.21.15.97:0/3512370637 learned_addr learned my addr 172.21.15.97:0/3512370637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d99e00640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d7c0051d0 0x7f1d941609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d99400640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d94155b10 0x7f1d94155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 -- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d94155b10 msgr2=0x7f1d94155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d94155b10 0x7f1d94155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 -- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d7c0051d0 msgr2=0x7f1d941609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d7c0051d0 0x7f1d941609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 -- 172.21.15.97:0/3512370637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d94156700 con 0x7f1d941f8310 2024-09-10T21:29:50.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9a800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d941f8310 0x7f1d941f8790 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1d9001b230 tx=0x7f1d900012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:50.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d8e000640 1 -- 172.21.15.97:0/3512370637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900019f0 con 0x7f1d941f8310 2024-09-10T21:29:50.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d8e000640 1 -- 172.21.15.97:0/3512370637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d900b5070 con 0x7f1d941f8310 2024-09-10T21:29:50.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d8e000640 1 -- 172.21.15.97:0/3512370637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90006610 con 0x7f1d941f8310 2024-09-10T21:29:50.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d941f4c70 con 0x7f1d941f8310 2024-09-10T21:29:50.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941f5230 con 0x7f1d941f8310 2024-09-10T21:29:50.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d94162750 con 0x7f1d941f8310 2024-09-10T21:29:50.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.153+0000 7f1d8e000640 1 -- 172.21.15.97:0/3512370637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d900ba470 con 0x7f1d941f8310 2024-09-10T21:29:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.157+0000 7f1d8e000640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d78072610 0x7f1d78074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.157+0000 7f1d8e000640 1 -- 172.21.15.97:0/3512370637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f1d9012ded0 con 0x7f1d941f8310 2024-09-10T21:29:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.157+0000 7f1d99e00640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d78072610 0x7f1d78074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:50.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.157+0000 7f1d99e00640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d78072610 0x7f1d78074ad0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1d88006cc0 tx=0x7f1d880067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.161+0000 7f1d8e000640 1 -- 172.21.15.97:0/3512370637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d900c1030 con 0x7f1d941f8310 2024-09-10T21:29:50.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1d9415c110 con 0x7f1d941f8310 2024-09-10T21:29:50.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d8e000640 1 -- 172.21.15.97:0/3512370637 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7f1d90016890 con 0x7f1d941f8310 2024-09-10T21:29:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:29:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d78072610 msgr2=0x7f1d78074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d78072610 0x7f1d78074ad0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1d88006cc0 tx=0x7f1d880067b0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d941f8310 msgr2=0x7f1d941f8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d941f8310 0x7f1d941f8790 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1d9001b230 tx=0x7f1d900012e0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 shutdown_connections 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d78072610 0x7f1d78074ad0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d941f8310 0x7f1d941f8790 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d94155b10 0x7f1d94155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 --2- 172.21.15.97:0/3512370637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d7c0051d0 0x7f1d941609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 >> 172.21.15.97:0/3512370637 conn(0x7f1d940b28c0 msgr2=0x7f1d940b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 shutdown_connections 2024-09-10T21:29:50.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50.365+0000 7f1d9c800640 1 -- 172.21.15.97:0/3512370637 wait complete. 2024-09-10T21:29:50.389 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:29:50.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:29:50.390 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:29:50.390 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:29:50.390 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:50.390 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:29:50.390 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:29:50.390 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:29:50.390 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:29:50.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:50.390 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:50.399 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42103sec preferred_lft 42103sec 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:29:50.400 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:29:50.401 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:29:50.402 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:29:50.402 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.402 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:29:50.402 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:29:50.402 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:29:50.402 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:29:50.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:50.403 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:29:50.463 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:29:50.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:50.465 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:29:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:29:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:29:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:29:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:29:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:29:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:29:50.516 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:29:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:29:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:29:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:29:50.517 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:29:50.517 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:29:50.517 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:29:50.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:29:50.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:29:50.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:29:50.518 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:29:50.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:29:50.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:29:50.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:50.652 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:29:50.663 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:29:50.663 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:29:50.663 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:29:50.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:50.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:29:50.725 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:29:50.726 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:50.773 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:29:50.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:29:50.774 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:50.825 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:29:50.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:50.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:50.914 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:50.914 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:29:50.914 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:29:50.929 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:50.929 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:29:50.929 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:29:50.929 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:29:50.930 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:29:50.939 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:29:50.943 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:29:50.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:50.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:50.963 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:29:51.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:29:51.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:29:51.076 INFO:teuthology.orchestra.run.smithi097.stdout:client6672 2024-09-10T21:29:51.076 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:29:51.076 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:51.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- 172.21.15.97:0/1220061457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 msgr2=0x7f8d9800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 --2- 172.21.15.97:0/1220061457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8d9800b9d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8da80016f0 tx=0x7f8da801c770 comp rx=0 tx=0).stop 2024-09-10T21:29:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- 172.21.15.97:0/1220061457 shutdown_connections 2024-09-10T21:29:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 --2- 172.21.15.97:0/1220061457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8d9800b9d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- 172.21.15.97:0/1220061457 >> 172.21.15.97:0/1220061457 conn(0x7f8db00b1780 msgr2=0x7f8db00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- 172.21.15.97:0/1220061457 shutdown_connections 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- 172.21.15.97:0/1220061457 wait complete. 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 Processor -- start 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- start start 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8db015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8db01f7da0 0x7f8db01e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db01e58e0 0x7f8db01f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8db00b2e10 con 0x7f8d980052f0 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8db00b2c90 con 0x7f8db01e58e0 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db8400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8db01444d0 con 0x7f8db01f7da0 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8db015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8db015bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45266/0 (socket says 172.21.15.97:45266) 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db5a00640 1 -- 172.21.15.97:0/1724463801 learned_addr learned my addr 172.21.15.97:0/1724463801 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db5000640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8db01f7da0 0x7f8db01e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.265+0000 7f8db6400640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db01e58e0 0x7f8db01f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db5a00640 1 -- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8db01f7da0 msgr2=0x7f8db01e7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db5a00640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8db01f7da0 0x7f8db01e7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db5a00640 1 -- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db01e58e0 msgr2=0x7f8db01f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db5a00640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db01e58e0 0x7f8db01f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db5a00640 1 -- 172.21.15.97:0/1724463801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db01f8310 con 0x7f8d980052f0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db5a00640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8db015bc00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006310 tx=0x7f8da40067d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8dad600640 1 -- 172.21.15.97:0/1724463801 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4015070 con 0x7f8d980052f0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8dad600640 1 -- 172.21.15.97:0/1724463801 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8da400fd40 con 0x7f8d980052f0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8dad600640 1 -- 172.21.15.97:0/1724463801 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4011690 con 0x7f8d980052f0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db01f85f0 con 0x7f8d980052f0 2024-09-10T21:29:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db01f8bb0 con 0x7f8d980052f0 2024-09-10T21:29:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db0070580 con 0x7f8d980052f0 2024-09-10T21:29:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8dad600640 1 -- 172.21.15.97:0/1724463801 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8da4011830 con 0x7f8d980052f0 2024-09-10T21:29:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8dad600640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d90072610 0x7f8d90074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.269+0000 7f8dad600640 1 -- 172.21.15.97:0/1724463801 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f8da40943c0 con 0x7f8d980052f0 2024-09-10T21:29:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.273+0000 7f8db5000640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d90072610 0x7f8d90074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.273+0000 7f8db5000640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d90072610 0x7f8d90074ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8db015db30 tx=0x7f8d9c00b260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.273+0000 7f8dad600640 1 -- 172.21.15.97:0/1724463801 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8da405f870 con 0x7f8d980052f0 2024-09-10T21:29:51.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.481+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8db00ae520 con 0x7f8d980052f0 2024-09-10T21:29:51.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8dad600640 1 -- 172.21.15.97:0/1724463801 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5231 (secure 0 0 0) 0x7f8da405f170 con 0x7f8d980052f0 2024-09-10T21:29:51.486 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:51.487 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":70,"btime":"2024-09-10T21:29:42:550041+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6492,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:29:41.936668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6504},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6504":{"gid":6504,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6836/3597857813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3597857813},{"type":"v1","addr":"172.21.15.194:6836","nonce":3597857813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6504,"qdb_cluster":[6504]},"id":5}]} 2024-09-10T21:29:51.487 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 70 2024-09-10T21:29:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d90072610 msgr2=0x7f8d90074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d90072610 0x7f8d90074ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8db015db30 tx=0x7f8d9c00b260 comp rx=0 tx=0).stop 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 msgr2=0x7f8db015bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8db015bc00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006310 tx=0x7f8da40067d0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 shutdown_connections 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d90072610 0x7f8d90074ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db01e58e0 0x7f8db01f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8db01f7da0 0x7f8db01e7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 --2- 172.21.15.97:0/1724463801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d980052f0 0x7f8db015bc00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 >> 172.21.15.97:0/1724463801 conn(0x7f8db00b1780 msgr2=0x7f8db00af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 shutdown_connections 2024-09-10T21:29:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.485+0000 7f8db8400640 1 -- 172.21.15.97:0/1724463801 wait complete. 2024-09-10T21:29:51.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- 172.21.15.97:0/3463932597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 msgr2=0x7f06400b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 --2- 172.21.15.97:0/3463932597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f06400b0490 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0638009840 tx=0x7f063801c7b0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- 172.21.15.97:0/3463932597 shutdown_connections 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 --2- 172.21.15.97:0/3463932597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f06400b0490 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 --2- 172.21.15.97:0/3463932597 >> v2:172.21.15.194:3300/0 conn(0x7f064015eed0 0x7f06400d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- 172.21.15.97:0/3463932597 >> 172.21.15.97:0/3463932597 conn(0x7f06400d1cd0 msgr2=0x7f06400d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- 172.21.15.97:0/3463932597 shutdown_connections 2024-09-10T21:29:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- 172.21.15.97:0/3463932597 wait complete. 2024-09-10T21:29:51.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 Processor -- start 2024-09-10T21:29:51.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- start start 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f0640158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06400cdd40 0x7f064006e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f064015eed0 0x7f064015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0640168880 con 0x7f06400ccc60 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0640168700 con 0x7f064015eed0 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f0646200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0640168a00 con 0x7f06400cdd40 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f0640158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f0640158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45290/0 (socket says 172.21.15.97:45290) 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 -- 172.21.15.97:0/2089975346 learned_addr learned my addr 172.21.15.97:0/2089975346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063fe00640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f064015eed0 0x7f064015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063ea00640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06400cdd40 0x7f064006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 -- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06400cdd40 msgr2=0x7f064006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06400cdd40 0x7f064006e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 -- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f064015eed0 msgr2=0x7f064015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f064015eed0 0x7f064015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063f400640 1 -- 172.21.15.97:0/2089975346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06401fb4c0 con 0x7f06400ccc60 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.653+0000 7f063fe00640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f064015eed0 0x7f064015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:51.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f063f400640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f0640158bb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f063000ead0 tx=0x7f063000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:51.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0637400640 1 -- 172.21.15.97:0/2089975346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063000cde0 con 0x7f06400ccc60 2024-09-10T21:29:51.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0637400640 1 -- 172.21.15.97:0/2089975346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0630011470 con 0x7f06400ccc60 2024-09-10T21:29:51.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0637400640 1 -- 172.21.15.97:0/2089975346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06300118a0 con 0x7f06400ccc60 2024-09-10T21:29:51.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06401fb7a0 con 0x7f06400ccc60 2024-09-10T21:29:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06401fbd60 con 0x7f06400ccc60 2024-09-10T21:29:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06400b0490 con 0x7f06400ccc60 2024-09-10T21:29:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0637400640 1 -- 172.21.15.97:0/2089975346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0630022030 con 0x7f06400ccc60 2024-09-10T21:29:51.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0637400640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06200725e0 0x7f0620074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:51.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.657+0000 7f0637400640 1 -- 172.21.15.97:0/2089975346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f0630093d00 con 0x7f06400ccc60 2024-09-10T21:29:51.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.661+0000 7f063ea00640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06200725e0 0x7f0620074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:51.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.661+0000 7f063ea00640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06200725e0 0x7f0620074aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f06400cb5d0 tx=0x7f062c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:51.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.661+0000 7f0637400640 1 -- 172.21.15.97:0/2089975346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0630016030 con 0x7f06400ccc60 2024-09-10T21:29:51.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:51.893+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) -- 0x7f064015d3b0 con 0x7f06400ccc60 2024-09-10T21:29:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.585+0000 7f0637400640 1 -- 172.21.15.97:0/2089975346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.6504 v71) ==== 112+0+0 (secure 0 0 0) 0x7f063005f240 con 0x7f06400ccc60 2024-09-10T21:29:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:freezing mds.6504 2024-09-10T21:29:52.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06200725e0 msgr2=0x7f0620074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06200725e0 0x7f0620074aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f06400cb5d0 tx=0x7f062c009040 comp rx=0 tx=0).stop 2024-09-10T21:29:52.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 msgr2=0x7f0640158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f0640158bb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f063000ead0 tx=0x7f063000ef90 comp rx=0 tx=0).stop 2024-09-10T21:29:52.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 shutdown_connections 2024-09-10T21:29:52.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f06200725e0 0x7f0620074aa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f064015eed0 0x7f064015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f06400cdd40 0x7f064006e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 --2- 172.21.15.97:0/2089975346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06400ccc60 0x7f0640158bb0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 >> 172.21.15.97:0/2089975346 conn(0x7f06400d1cd0 msgr2=0x7f06400b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:52.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 shutdown_connections 2024-09-10T21:29:52.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.589+0000 7f0646200640 1 -- 172.21.15.97:0/2089975346 wait complete. 2024-09-10T21:29:52.619 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d') 2024-09-10T21:29:52.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:29:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:29:52.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-10T21:29:52.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 -- 172.21.15.97:0/4289129542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 msgr2=0x7fb1540b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 --2- 172.21.15.97:0/4289129542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb1540b3070 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb144009900 tx=0x7fb14401c850 comp rx=0 tx=0).stop 2024-09-10T21:29:52.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 -- 172.21.15.97:0/4289129542 shutdown_connections 2024-09-10T21:29:52.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 --2- 172.21.15.97:0/4289129542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb1540b3070 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 -- 172.21.15.97:0/4289129542 >> 172.21.15.97:0/4289129542 conn(0x7fb1540d1c60 msgr2=0x7fb1540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:52.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 -- 172.21.15.97:0/4289129542 shutdown_connections 2024-09-10T21:29:52.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 -- 172.21.15.97:0/4289129542 wait complete. 2024-09-10T21:29:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 Processor -- start 2024-09-10T21:29:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.837+0000 7fb158a00640 1 -- start start 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1540d0150 0x7fb15414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1540d05e0 0x7fb15415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb154156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb154169dc0 con 0x7fb15415f350 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb154169c40 con 0x7fb1540d05e0 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb154169f40 con 0x7fb1540d0150 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb154156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb154156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45306/0 (socket says 172.21.15.97:45306) 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 -- 172.21.15.97:0/970408000 learned_addr learned my addr 172.21.15.97:0/970408000 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb14be00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1540d05e0 0x7fb15415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb151e00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1540d0150 0x7fb15414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 -- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1540d0150 msgr2=0x7fb15414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1540d0150 0x7fb15414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 -- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1540d05e0 msgr2=0x7fb15415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1540d05e0 0x7fb15415e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 -- 172.21.15.97:0/970408000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15415c7d0 con 0x7fb15415f350 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb152800640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb154156070 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c7b0 tx=0x7fb14c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb14a000640 1 -- 172.21.15.97:0/970408000 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c003b80 con 0x7fb15415f350 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb14a000640 1 -- 172.21.15.97:0/970408000 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb14c003d20 con 0x7fb15415f350 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb14a000640 1 -- 172.21.15.97:0/970408000 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00aab0 con 0x7fb15415f350 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15415cab0 con 0x7fb15415f350 2024-09-10T21:29:52.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1540b5e50 con 0x7fb15415f350 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.841+0000 7fb14a000640 1 -- 172.21.15.97:0/970408000 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb14c00ac50 con 0x7fb15415f350 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb14a000640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb1300725e0 0x7fb130074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb14a000640 1 -- 172.21.15.97:0/970408000 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb14c093130 con 0x7fb15415f350 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 Processor -- start 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- start start 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb1300725e0 0x7fb130074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e037a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e037afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0262cb0 con 0x55d2e03c41a0 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d2e0178310 con 0x55d2e03c4ac0 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d2e01d5b90 con 0x55d2e037abc0 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb1300725e0 0x7fb130074aa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1440af000 tx=0x7fb14401b090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45310/0 (socket says 172.21.15.97:45310) 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 -- 172.21.15.97:0/2719762246 learned_addr learned my addr 172.21.15.97:0/2719762246 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb152800640 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e037afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb14be00640 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e037a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 -- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 msgr2=0x55d2e037afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e037afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 -- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 msgr2=0x55d2e037a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e037a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 -- 172.21.15.97:0/2719762246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e0179c60 con 0x55d2e03c41a0 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb152800640 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e037afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb151e00640 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03c4580 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb1440b0040 tx=0x7fb1440b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761c907c42333ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb142a00640 1 -- 172.21.15.97:0/2719762246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440bd050 con 0x55d2e03c41a0 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb142a00640 1 -- 172.21.15.97:0/2719762246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1440be040 con 0x55d2e03c41a0 2024-09-10T21:29:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb142a00640 1 -- 172.21.15.97:0/2719762246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440046d0 con 0x55d2e03c41a0 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- 172.21.15.97:0/2719762246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 msgr2=0x55d2e03c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03c4580 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb1440b0040 tx=0x7fb1440b5000 comp rx=0 tx=0).stop 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- 172.21.15.97:0/2719762246 shutdown_connections 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e037afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e037a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2719762246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03c4580 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- 172.21.15.97:0/2719762246 >> 172.21.15.97:0/2719762246 conn(0x55d2e0380d20 msgr2=0x55d2e0378f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- 172.21.15.97:0/2719762246 shutdown_connections 2024-09-10T21:29:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.845+0000 7fb15bde7000 1 -- 172.21.15.97:0/2719762246 wait complete. 2024-09-10T21:29:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 -- start start 2024-09-10T21:29:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 Processor -- start 2024-09-10T21:29:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e037abc0 0x55d2e03c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c41a0 0x55d2e0389990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e0389ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d2e01c8dd0 con 0x55d2e037abc0 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d2e01c4fc0 con 0x55d2e03c4ac0 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d2e016a4b0 con 0x55d2e03c41a0 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 10 client.0 _ll_register_callbacks cb 0x55d2e01487d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e037abc0 0x55d2e03c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e037abc0 0x55d2e03c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45312/0 (socket says 172.21.15.97:45312) 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 -- 172.21.15.97:0/3481145891 learned_addr learned my addr 172.21.15.97:0/3481145891 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb152800640 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e0389ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb14be00640 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c41a0 0x55d2e0389990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 -- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c41a0 msgr2=0x55d2e0389990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c41a0 0x55d2e0389990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 -- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 msgr2=0x55d2e0389ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e0389ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 -- 172.21.15.97:0/3481145891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e01631f0 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb151e00640 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e037abc0 0x55d2e03c3ad0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb14401cd30 tx=0x7fb144004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d355e6760bdf8af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb137400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb137400640 1 -- 172.21.15.97:0/3481145891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144005510 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2e015d330 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb137400640 1 -- 172.21.15.97:0/3481145891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb144004d30 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb137400640 1 -- 172.21.15.97:0/3481145891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb1440bd020 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 10 client.6702 fetch_fsmap learned FSMap version 71 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55d2e0149fa0 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb137400640 1 -- 172.21.15.97:0/3481145891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d2e015d330 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb137400640 1 -- 172.21.15.97:0/3481145891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7fb1440a5600 con 0x55d2e037abc0 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 10 client.6702 fetch_fsmap finished waiting for FSMap version 71 2024-09-10T21:29:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 10 client.6702 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 20 client.6702 populate_metadata read hostname 'smithi097' 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e038ac10 0x55d2e03da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 4 client.6702 mds_command: new command op to 6504 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 --> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2e01d7ae0 con 0x55d2e038ac10 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.849+0000 7fb14be00640 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e038ac10 0x55d2e03da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb14be00640 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e038ac10 0x55d2e03da4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d803ac8d525fb778 server_cookie=8a34e8bf06675d85 in_seq=0 out_seq=0 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb137400640 10 client.6702 ms_handle_connect on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb137400640 1 -- 172.21.15.97:0/3481145891 <== mds.0 v2:172.21.15.194:6834/3597857813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d2e01d7ae0 con 0x55d2e038ac10 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb137400640 10 client.6702 handle_command_reply: tid=0 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 client.6702 shutdown 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e037abc0 msgr2=0x55d2e03c3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e037abc0 0x55d2e03c3ad0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb14401cd30 tx=0x7fb144004bc0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 shutdown_connections 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e038ac10 0x55d2e03da4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e0389ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c41a0 0x55d2e0389990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb137400640 0 client.6702 ms_handle_reset on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 --2- 172.21.15.97:0/3481145891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e037abc0 0x55d2e03c3ad0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 >> 172.21.15.97:0/3481145891 conn(0x55d2e0380d20 msgr2=0x55d2e0378f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 shutdown_connections 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 1 -- 172.21.15.97:0/3481145891 wait complete. 2024-09-10T21:29:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.853+0000 7fb15bde7000 20 client.6702 trim_cache size 0 max 16384 2024-09-10T21:29:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 Processor -- start 2024-09-10T21:29:52.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- start start 2024-09-10T21:29:52.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e043a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e043ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e043b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d2e037aa40 con 0x55d2e03c41a0 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541fffa0 con 0x55d2e03c4ac0 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb15406f650 con 0x55d2e037abc0 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e043ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e043ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45322/0 (socket says 172.21.15.97:45322) 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 -- 172.21.15.97:0/2240553633 learned_addr learned my addr 172.21.15.97:0/2240553633 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb152800640 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e043b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb151e00640 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e043a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 -- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 msgr2=0x55d2e043a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e043a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 -- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 msgr2=0x55d2e043b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e043b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 -- 172.21.15.97:0/2240553633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e0222660 con 0x55d2e03c41a0 2024-09-10T21:29:52.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb14be00640 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e043ad40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007a40 tx=0x7fb13c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790f6875e0ea5888 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb142000640 1 -- 172.21.15.97:0/2240553633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c019900 con 0x55d2e03c41a0 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb142000640 1 -- 172.21.15.97:0/2240553633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb13c022070 con 0x55d2e03c41a0 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb142000640 1 -- 172.21.15.97:0/2240553633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c019e50 con 0x55d2e03c41a0 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- 172.21.15.97:0/2240553633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 msgr2=0x55d2e043ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e043ad40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007a40 tx=0x7fb13c009040 comp rx=0 tx=0).stop 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- 172.21.15.97:0/2240553633 shutdown_connections 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e03c4ac0 0x55d2e043b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e043ad40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 --2- 172.21.15.97:0/2240553633 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e037abc0 0x55d2e043a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- 172.21.15.97:0/2240553633 >> 172.21.15.97:0/2240553633 conn(0x55d2e0380d20 msgr2=0x55d2e03d5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:52.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- 172.21.15.97:0/2240553633 shutdown_connections 2024-09-10T21:29:52.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.869+0000 7fb15bde7000 1 -- 172.21.15.97:0/2240553633 wait complete. 2024-09-10T21:29:52.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 -- start start 2024-09-10T21:29:52.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 Processor -- start 2024-09-10T21:29:52.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e037abc0 0x55d2e03dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c4ac0 0x55d2e03fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb15406f000 con 0x55d2e03c41a0 2024-09-10T21:29:52.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb1540b75f0 con 0x55d2e037abc0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d2dfe8a680 con 0x55d2e03c4ac0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 10 client.0 _ll_register_callbacks cb 0x7fb144004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03dd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45334/0 (socket says 172.21.15.97:45334) 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 -- 172.21.15.97:0/1283848631 learned_addr learned my addr 172.21.15.97:0/1283848631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb151e00640 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e037abc0 0x55d2e03dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb152800640 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c4ac0 0x55d2e03fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 -- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c4ac0 msgr2=0x55d2e03fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c4ac0 0x55d2e03fa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 -- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e037abc0 msgr2=0x55d2e03dd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e037abc0 0x55d2e03dd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 -- 172.21.15.97:0/1283848631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e018f090 con 0x55d2e03c41a0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb14be00640 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03dd8f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007a10 tx=0x7fb13c019b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ef5d41d1eb2d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:52.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb137400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb137400640 1 -- 172.21.15.97:0/1283848631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c02a070 con 0x55d2e03c41a0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb13c022070 con 0x55d2e03c41a0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb137400640 1 -- 172.21.15.97:0/1283848631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fb13c026620 con 0x55d2e03c41a0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb137400640 1 -- 172.21.15.97:0/1283848631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb13c035050 con 0x55d2e03c41a0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 10 client.6708 fetch_fsmap learned FSMap version 71 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55d2e015b5f0 con 0x55d2e03c41a0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb137400640 1 -- 172.21.15.97:0/1283848631 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb13c022070 con 0x55d2e03c41a0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb137400640 1 -- 172.21.15.97:0/1283848631 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7fb13c038020 con 0x55d2e03c41a0 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 10 client.6708 fetch_fsmap finished waiting for FSMap version 71 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 10 client.6708 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 20 client.6708 populate_metadata read hostname 'smithi097' 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e043e420 0x55d2e043e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 4 client.6708 mds_command: new command op to 6504 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:29:52.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.873+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 --> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2e01d3590 con 0x55d2e043e420 2024-09-10T21:29:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.877+0000 7fb151e00640 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e043e420 0x55d2e043e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.877+0000 7fb137400640 10 client.6708 ms_handle_connect on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:52.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.877+0000 7fb151e00640 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e043e420 0x55d2e043e830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7d8c9e07413526 server_cookie=aeb37bb1d83097d8 in_seq=0 out_seq=0 2024-09-10T21:29:52.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb137400640 1 -- 172.21.15.97:0/1283848631 <== mds.0 v2:172.21.15.194:6834/3597857813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2e01d3590 con 0x55d2e043e420 2024-09-10T21:29:52.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb137400640 10 client.6708 handle_command_reply: tid=0 2024-09-10T21:29:52.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 client.6708 shutdown 2024-09-10T21:29:52.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 msgr2=0x55d2e03dd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03dd8f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007a10 tx=0x7fb13c019b30 comp rx=0 tx=0).stop 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 shutdown_connections 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:6834/3597857813,v1:172.21.15.194:6836/3597857813] conn(0x55d2e043e420 0x55d2e043e830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2e03c4ac0 0x55d2e03fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2e03c41a0 0x55d2e03dd8f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb137400640 0 client.6708 ms_handle_reset on v2:172.21.15.194:6834/3597857813 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 --2- 172.21.15.97:0/1283848631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d2e037abc0 0x55d2e03dd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 >> 172.21.15.97:0/1283848631 conn(0x55d2e0380d20 msgr2=0x55d2e03d8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 shutdown_connections 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 1 -- 172.21.15.97:0/1283848631 wait complete. 2024-09-10T21:29:52.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.889+0000 7fb15bde7000 20 client.6708 trim_cache size 0 max 16384 2024-09-10T21:29:52.896 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:29:52.896 INFO:teuthology.orchestra.run.smithi097.stdout: "message": "", 2024-09-10T21:29:52.896 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-09-10T21:29:52.897 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:29:52.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb1300725e0 msgr2=0x7fb130074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb1300725e0 0x7fb130074aa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1440af000 tx=0x7fb14401b090 comp rx=0 tx=0).stop 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 msgr2=0x7fb154156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb154156070 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c7b0 tx=0x7fb14c00cc70 comp rx=0 tx=0).stop 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 shutdown_connections 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb1300725e0 0x7fb130074aa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb15415f350 0x7fb154156070 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1540d05e0 0x7fb15415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 --2- 172.21.15.97:0/970408000 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb1540d0150 0x7fb15414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 >> 172.21.15.97:0/970408000 conn(0x7fb1540d1c60 msgr2=0x7fb1540b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 shutdown_connections 2024-09-10T21:29:52.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:52.893+0000 7fb158a00640 1 -- 172.21.15.97:0/970408000 wait complete. 2024-09-10T21:29:52.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-10T21:29:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 -- 172.21.15.97:0/2476473881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 msgr2=0x7fdcbc1624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/2476473881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc1624b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc06f590 tx=0x7fdcac01cbc0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 -- 172.21.15.97:0/2476473881 shutdown_connections 2024-09-10T21:29:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/2476473881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcbc06cb40 0x7fdcbc06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/2476473881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc1624b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 -- 172.21.15.97:0/2476473881 >> 172.21.15.97:0/2476473881 conn(0x7fdcbc0b28c0 msgr2=0x7fdcbc0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:53.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 -- 172.21.15.97:0/2476473881 shutdown_connections 2024-09-10T21:29:53.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.053+0000 7fdcc1c00640 1 -- 172.21.15.97:0/2476473881 wait complete. 2024-09-10T21:29:53.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 Processor -- start 2024-09-10T21:29:53.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 -- start start 2024-09-10T21:29:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcbc06cb40 0x7fdcbc13f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcbc073a70 0x7fdcbc154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc16afc0 con 0x7fdcbc0d0060 2024-09-10T21:29:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc16ae40 con 0x7fdcbc06cb40 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdcbc16b140 con 0x7fdcbc073a70 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbaa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcbc06cb40 0x7fdcbc13f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45352/0 (socket says 172.21.15.97:45352) 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 -- 172.21.15.97:0/3732687544 learned_addr learned my addr 172.21.15.97:0/3732687544 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcba000640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcbc073a70 0x7fdcbc154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 -- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcbc073a70 msgr2=0x7fdcbc154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcbc073a70 0x7fdcbc154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 -- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcbc06cb40 msgr2=0x7fdcbc13f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcbc06cb40 0x7fdcbc13f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 -- 172.21.15.97:0/3732687544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc13fb90 con 0x7fdcbc0d0060 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcba000640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcbc073a70 0x7fdcbc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcbb400640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc150cd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b3c0 tx=0x7fdcb400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcb2a00640 1 -- 172.21.15.97:0/3732687544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4015020 con 0x7fdcbc0d0060 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcb2a00640 1 -- 172.21.15.97:0/3732687544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcb4011920 con 0x7fdcbc0d0060 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc160de0 con 0x7fdcbc0d0060 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.057+0000 7fdcb2a00640 1 -- 172.21.15.97:0/3732687544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4011d70 con 0x7fdcbc0d0060 2024-09-10T21:29:53.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.061+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc154270 con 0x7fdcbc0d0060 2024-09-10T21:29:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.061+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc06cfc0 con 0x7fdcbc0d0060 2024-09-10T21:29:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.061+0000 7fdcb2a00640 1 -- 172.21.15.97:0/3732687544 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdcb401a400 con 0x7fdcbc0d0060 2024-09-10T21:29:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.061+0000 7fdcb2a00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c072610 0x7fdc9c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.061+0000 7fdcb2a00640 1 -- 172.21.15.97:0/3732687544 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fdcb40934a0 con 0x7fdcbc0d0060 2024-09-10T21:29:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.061+0000 7fdcbaa00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c072610 0x7fdc9c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.065+0000 7fdcbaa00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c072610 0x7fdc9c074ad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0af000 tx=0x7fdcac01b400 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:53.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.065+0000 7fdcb2a00640 1 -- 172.21.15.97:0/3732687544 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdcb401f020 con 0x7fdcbc0d0060 2024-09-10T21:29:53.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.229+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7fdcbc15d540 con 0x7fdcbc0d0060 2024-09-10T21:29:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.233+0000 7fdcb2a00640 1 -- 172.21.15.97:0/3732687544 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v40)=0 v40) ==== 138+0+0 (secure 0 0 0) 0x7fdcb405e9e0 con 0x7fdcbc0d0060 2024-09-10T21:29:53.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c072610 msgr2=0x7fdc9c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:53.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c072610 0x7fdc9c074ad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0af000 tx=0x7fdcac01b400 comp rx=0 tx=0).stop 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 msgr2=0x7fdcbc150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc150cd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b3c0 tx=0x7fdcb400b880 comp rx=0 tx=0).stop 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 shutdown_connections 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c072610 0x7fdc9c074ad0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcbc0d0060 0x7fdcbc150cd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcbc073a70 0x7fdcbc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 --2- 172.21.15.97:0/3732687544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcbc06cb40 0x7fdcbc13f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 >> 172.21.15.97:0/3732687544 conn(0x7fdcbc0b28c0 msgr2=0x7fdcbc15bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 shutdown_connections 2024-09-10T21:29:53.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:53.237+0000 7fdcc1c00640 1 -- 172.21.15.97:0/3732687544 wait complete. 2024-09-10T21:29:58.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-10T21:29:58.262 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'timeout 60 mv a/b a/z') 2024-09-10T21:29:58.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-10T21:29:58.325 INFO:tasks.ceph.mds.b.smithi194.stderr:2024-09-10T21:29:58.322+0000 7f48dce00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6672:47 nref=7 cr=0x562fcb654700)) (dversion lock w=1 last_client=6672) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562fcb69d180] 2024-09-10T21:29:58.325 INFO:tasks.ceph.mds.b.smithi194.stderr:2024-09-10T21:29:58.322+0000 7f48dce00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6672:47 nref=7 cr=0x562fcb654700)) (dversion lock w=1 last_client=6672) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562fcb69d180] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr:./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f48dce00640 time 2024-09-10T21:29:58.327115+0000 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f48e4185968] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.326 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr:*** Caught signal (Aborted) ** 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: in thread 7f48dce00640 thread_name:ms_dispatch 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr:2024-09-10T21:29:58.322+0000 7f48dce00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f48dce00640 time 2024-09-10T21:29:58.327115+0000 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:29:58.327 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f48e4185968] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.328 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.329 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.329 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.329 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.329 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.329 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.329 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f48e3842520] 2024-09-10T21:29:58.329 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: pthread_kill() 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: raise() 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: abort() 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f48e4185a1e] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.330 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 16: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 19: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr:2024-09-10T21:29:58.326+0000 7f48dce00640 -1 *** Caught signal (Aborted) ** 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: in thread 7f48dce00640 thread_name:ms_dispatch 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f48e3842520] 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: pthread_kill() 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: raise() 2024-09-10T21:29:58.331 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: abort() 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f48e4185a1e] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 16: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.332 INFO:tasks.ceph.mds.b.smithi194.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.333 INFO:tasks.ceph.mds.b.smithi194.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.333 INFO:tasks.ceph.mds.b.smithi194.stderr: 19: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.333 INFO:tasks.ceph.mds.b.smithi194.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T21:29:58.333 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.339 INFO:tasks.ceph.mds.b.smithi194.stderr: -5213> 2024-09-10T21:29:41.502+0000 7f48d9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:29:58.347 INFO:tasks.ceph.mds.b.smithi194.stderr: -9> 2024-09-10T21:29:58.322+0000 7f48dce00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6672:47 nref=7 cr=0x562fcb654700)) (dversion lock w=1 last_client=6672) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562fcb69d180] 2024-09-10T21:29:58.347 INFO:tasks.ceph.mds.b.smithi194.stderr: -7> 2024-09-10T21:29:58.322+0000 7f48dce00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6672:47 nref=7 cr=0x562fcb654700)) (dversion lock w=1 last_client=6672) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562fcb69d180] 2024-09-10T21:29:58.347 INFO:tasks.ceph.mds.b.smithi194.stderr: -1> 2024-09-10T21:29:58.322+0000 7f48dce00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f48dce00640 time 2024-09-10T21:29:58.327115+0000 2024-09-10T21:29:58.347 INFO:tasks.ceph.mds.b.smithi194.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:29:58.347 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.347 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.348 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f48e4185968] 2024-09-10T21:29:58.348 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.348 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.348 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.348 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.348 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: 0> 2024-09-10T21:29:58.326+0000 7f48dce00640 -1 *** Caught signal (Aborted) ** 2024-09-10T21:29:58.349 INFO:tasks.ceph.mds.b.smithi194.stderr: in thread 7f48dce00640 thread_name:ms_dispatch 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f48e3842520] 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: pthread_kill() 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: raise() 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: abort() 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f48e4185a1e] 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.350 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 16: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 19: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T21:29:58.351 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.352 INFO:tasks.ceph.mds.b.smithi194.stderr: -5213> 2024-09-10T21:29:41.502+0000 7f48d9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:29:58.355 INFO:tasks.ceph.mds.b.smithi194.stderr: -9> 2024-09-10T21:29:58.322+0000 7f48dce00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6672:47 nref=7 cr=0x562fcb654700)) (dversion lock w=1 last_client=6672) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562fcb69d180] 2024-09-10T21:29:58.355 INFO:tasks.ceph.mds.b.smithi194.stderr: -7> 2024-09-10T21:29:58.322+0000 7f48dce00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6672:47 nref=7 cr=0x562fcb654700)) (dversion lock w=1 last_client=6672) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562fcb69d180] 2024-09-10T21:29:58.355 INFO:tasks.ceph.mds.b.smithi194.stderr: -1> 2024-09-10T21:29:58.322+0000 7f48dce00640 -1 ./src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f48dce00640 time 2024-09-10T21:29:58.327115+0000 2024-09-10T21:29:58.355 INFO:tasks.ceph.mds.b.smithi194.stderr:./src/mds/MDSRank.cc: 955: ceph_abort_msg("abort() called") 2024-09-10T21:29:58.355 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.355 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.355 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f48e4185968] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.356 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 0> 2024-09-10T21:29:58.326+0000 7f48dce00640 -1 *** Caught signal (Aborted) ** 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: in thread 7f48dce00640 thread_name:ms_dispatch 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f48e3842520] 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 2: pthread_kill() 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 3: raise() 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 4: abort() 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x189) [0x7f48e4185a1e] 2024-09-10T21:29:58.357 INFO:tasks.ceph.mds.b.smithi194.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x562fa0111de1] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 7: (CDentry::check_corruption(bool)+0x4f0) [0x562fa0340fe0] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 8: ceph-mds(+0x1c5869) [0x562fa015c869] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x1aff) [0x562fa01e3f3f] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x22a3) [0x562fa01df3d3] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3cf) [0x562fa01ae2af] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x562fa019ac8c] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4db) [0x562fa011a1db] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562fa011a68c] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x562fa00ffd56] 2024-09-10T21:29:58.358 INFO:tasks.ceph.mds.b.smithi194.stderr: 16: (DispatchQueue::entry()+0x58a) [0x7f48e438eaaa] 2024-09-10T21:29:58.359 INFO:tasks.ceph.mds.b.smithi194.stderr: 17: /usr/lib/ceph/libceph-common.so.2(+0x42def1) [0x7f48e442def1] 2024-09-10T21:29:58.359 INFO:tasks.ceph.mds.b.smithi194.stderr: 18: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f48e3894b43] 2024-09-10T21:29:58.359 INFO:tasks.ceph.mds.b.smithi194.stderr: 19: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f48e3926a00] 2024-09-10T21:29:58.359 INFO:tasks.ceph.mds.b.smithi194.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T21:29:58.359 INFO:tasks.ceph.mds.b.smithi194.stderr: 2024-09-10T21:29:58.426 INFO:teuthology.orchestra.run.smithi097.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-10T21:29:58.426 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-10T21:29:58.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:29:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.441+0000 7f7f0ae00640 1 -- 172.21.15.97:0/1359746022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8003a10 msgr2=0x7f7ee8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.441+0000 7f7f0ae00640 1 --2- 172.21.15.97:0/1359746022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8003a10 0x7f7ee8003e30 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009b60 tx=0x7f7ef401d2b0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.441+0000 7f7f08e00640 1 -- 172.21.15.97:0/1359746022 reap_dead start 2024-09-10T21:29:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.441+0000 7f7f0ae00640 1 -- 172.21.15.97:0/1359746022 shutdown_connections 2024-09-10T21:29:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.441+0000 7f7f0ae00640 1 -- 172.21.15.97:0/1359746022 >> 172.21.15.97:0/1359746022 conn(0x7f7f040b1780 msgr2=0x7f7f040b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.441+0000 7f7f0ae00640 1 -- 172.21.15.97:0/1359746022 shutdown_connections 2024-09-10T21:29:58.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.441+0000 7f7f0ae00640 1 -- 172.21.15.97:0/1359746022 wait complete. 2024-09-10T21:29:58.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 Processor -- start 2024-09-10T21:29:58.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 -- start start 2024-09-10T21:29:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8003a10 0x7f7f04152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f041f7220 0x7f7f04153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f041f8410 0x7f7f040cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0406fcf0 con 0x7f7ee8003a10 2024-09-10T21:29:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0406fb70 con 0x7f7f041f8410 2024-09-10T21:29:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0415c500 con 0x7f7f041f7220 2024-09-10T21:29:58.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8003a10 0x7f7f04152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8003a10 0x7f7f04152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33416/0 (socket says 172.21.15.97:33416) 2024-09-10T21:29:58.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 -- 172.21.15.97:0/3270816896 learned_addr learned my addr 172.21.15.97:0/3270816896 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:58.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03400640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f041f7220 0x7f7f04153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f08e00640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f041f8410 0x7f7f040cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 -- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f041f7220 msgr2=0x7f7f04153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f041f7220 0x7f7f04153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 -- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f041f8410 msgr2=0x7f7f040cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f041f8410 0x7f7f040cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 -- 172.21.15.97:0/3270816896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f04153f50 con 0x7f7ee8003a10 2024-09-10T21:29:58.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f03e00640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee8003a10 0x7f7f04152680 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40af040 tx=0x7f7ef40055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40b1040 con 0x7f7ee8003a10 2024-09-10T21:29:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ef40bb070 con 0x7f7ee8003a10 2024-09-10T21:29:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40a5e60 con 0x7f7ee8003a10 2024-09-10T21:29:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 -- 172.21.15.97:0/3270816896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f041f7710 con 0x7f7ee8003a10 2024-09-10T21:29:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.445+0000 7f7f0ae00640 1 -- 172.21.15.97:0/3270816896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f041f7cd0 con 0x7f7ee8003a10 2024-09-10T21:29:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7f0ae00640 1 -- 172.21.15.97:0/3270816896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7f040cc150 con 0x7f7ee8003a10 2024-09-10T21:29:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ef40046d0 con 0x7f7ee8003a10 2024-09-10T21:29:58.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7efbe00640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ee40725e0 0x7f7ee4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f7ef412dff0 con 0x7f7ee8003a10 2024-09-10T21:29:58.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+498 (secure 0 0 0) 0x7f7ef4016fc0 con 0x7f7ee8003a10 2024-09-10T21:29:58.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7f03400640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ee40725e0 0x7f7ee4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7f0ae00640 1 -- 172.21.15.97:0/3270816896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f7f040cf7b0 con 0x7f7ee8003a10 2024-09-10T21:29:58.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.449+0000 7f7f03400640 1 --2- 172.21.15.97:0/3270816896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ee40725e0 0x7f7ee4074aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0001250 tx=0x7f7ef0001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:58.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- 172.21.15.97:0/1917878348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 msgr2=0x7fa6980cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 --2- 172.21.15.97:0/1917878348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa6980cf0b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa6980af6c0 tx=0x7fa68801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- 172.21.15.97:0/1917878348 shutdown_connections 2024-09-10T21:29:58.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 --2- 172.21.15.97:0/1917878348 >> v2:172.21.15.194:3300/0 conn(0x7fa698072b70 0x7fa69815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 --2- 172.21.15.97:0/1917878348 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa698070720 0x7fa698072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 --2- 172.21.15.97:0/1917878348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa6980cf0b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- 172.21.15.97:0/1917878348 >> 172.21.15.97:0/1917878348 conn(0x7fa6980b1780 msgr2=0x7fa6980b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:58.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- 172.21.15.97:0/1917878348 shutdown_connections 2024-09-10T21:29:58.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- 172.21.15.97:0/1917878348 wait complete. 2024-09-10T21:29:58.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 Processor -- start 2024-09-10T21:29:58.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- start start 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa698070720 0x7fa698157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa698072b70 0x7fa69814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa69814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa69815e7c0 con 0x7fa69815b110 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa69815e640 con 0x7fa698070720 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa69d000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa6980b2a80 con 0x7fa698072b70 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa696800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa69814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa696800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa69814f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33430/0 (socket says 172.21.15.97:33430) 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa696800640 1 -- 172.21.15.97:0/3875353491 learned_addr learned my addr 172.21.15.97:0/3875353491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa695e00640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa698070720 0x7fa698157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.569+0000 7fa68d400640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa698072b70 0x7fa69814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa696800640 1 -- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa698072b70 msgr2=0x7fa69814c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa696800640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa698072b70 0x7fa69814c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa696800640 1 -- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa698070720 msgr2=0x7fa698157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa696800640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa698070720 0x7fa698157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa696800640 1 -- 172.21.15.97:0/3875353491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa698150370 con 0x7fa69815b110 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa695e00640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa698070720 0x7fa698157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa696800640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa69814f790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa69000eb20 tx=0x7fa69000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa68ea00640 1 -- 172.21.15.97:0/3875353491 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690004b30 con 0x7fa69815b110 2024-09-10T21:29:58.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa68ea00640 1 -- 172.21.15.97:0/3875353491 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa690004cf0 con 0x7fa69815b110 2024-09-10T21:29:58.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa68ea00640 1 -- 172.21.15.97:0/3875353491 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690005280 con 0x7fa69815b110 2024-09-10T21:29:58.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69814d0e0 con 0x7fa69815b110 2024-09-10T21:29:58.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69814d6a0 con 0x7fa69815b110 2024-09-10T21:29:58.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69815d9c0 con 0x7fa69815b110 2024-09-10T21:29:58.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.573+0000 7fa68ea00640 1 -- 172.21.15.97:0/3875353491 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa690005420 con 0x7fa69815b110 2024-09-10T21:29:58.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.577+0000 7fa68ea00640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674072610 0x7fa674074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:58.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.577+0000 7fa68ea00640 1 -- 172.21.15.97:0/3875353491 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fa690093f70 con 0x7fa69815b110 2024-09-10T21:29:58.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.577+0000 7fa695e00640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674072610 0x7fa674074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:58.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.577+0000 7fa695e00640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674072610 0x7fa674074ad0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa6880af000 tx=0x7fa68801b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:58.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.581+0000 7fa68ea00640 1 -- 172.21.15.97:0/3875353491 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa69005ee60 con 0x7fa69815b110 2024-09-10T21:29:58.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.605+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 7 ==== log(1 entries from seq 1 at 2024-09-10T21:29:58.327087+0000) ==== 468+0+0 (secure 0 0 0) 0x7f7ef40043a0 con 0x7f7ee8003a10 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: cluster: 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: id: d1b4b680-9a7d-4173-b750-f3c8b85e2f7b 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: health: HEALTH_OK 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: services: 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: mgr: x(active, since 6m), standbys: y 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: mds: 1/1 daemons up, 3 standby 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: osd: 8 osds: 8 up (since 5m), 8 in (since 6m) 2024-09-10T21:29:58.607 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: data: 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: volumes: 1/1 healthy 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: pools: 3 pools, 97 pgs 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: objects: 24 objects, 579 KiB 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: usage: 236 MiB used, 720 GiB / 720 GiB avail 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: pgs: 97 active+clean 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: io: 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: client: 85 B/s wr, 0 op/s rd, 0 op/s wr 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:58.608 INFO:teuthology.orchestra.run.smithi097.stdout:2024-09-10T21:29:58.327087+0000 mds.b [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6672:47 nref=7 cr=0x562fcb654700)) (dversion lock w=1 last_client=6672) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562fcb69d180] 2024-09-10T21:29:58.623 INFO:tasks.ceph.mds.b.smithi194.stderr:daemon-helper: command crashed with signal 6 2024-09-10T21:29:58.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa69814caa0 con 0x7fa69815b110 2024-09-10T21:29:58.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa68ea00640 1 -- 172.21.15.97:0/3875353491 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5231 (secure 0 0 0) 0x7fa690063890 con 0x7fa69815b110 2024-09-10T21:29:58.783 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:29:58.783 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"btime":"2024-09-10T21:29:52:583747+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6492,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:29:51.896989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6504},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6504":{"gid":6504,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6836/3597857813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3597857813},{"type":"v1","addr":"172.21.15.194:6836","nonce":3597857813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6504,"qdb_cluster":[6504]},"id":5}]} 2024-09-10T21:29:58.783 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 71 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674072610 msgr2=0x7fa674074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674072610 0x7fa674074ad0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa6880af000 tx=0x7fa68801b100 comp rx=0 tx=0).stop 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 msgr2=0x7fa69814f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa69814f790 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa69000eb20 tx=0x7fa69000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 shutdown_connections 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674072610 0x7fa674074ad0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69815b110 0x7fa69814f790 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa698072b70 0x7fa69814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 --2- 172.21.15.97:0/3875353491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa698070720 0x7fa698157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:58.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 >> 172.21.15.97:0/3875353491 conn(0x7fa6980b1780 msgr2=0x7fa6980b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:58.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 shutdown_connections 2024-09-10T21:29:58.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:29:58.781+0000 7fa69d000640 1 -- 172.21.15.97:0/3875353491 wait complete. 2024-09-10T21:29:58.804 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T21:30:00.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:00.613+0000 7f7efbe00640 1 -- 172.21.15.97:0/3270816896 <== mon.0 v2:172.21.15.97:3300/0 8 ==== log(1 entries from seq 751 at 2024-09-10T21:30:00.000153+0000) ==== 187+0+0 (secure 0 0 0) 0x7f7ef40f5110 con 0x7f7ee8003a10 2024-09-10T21:30:00.616 INFO:teuthology.orchestra.run.smithi097.stdout:2024-09-10T21:30:00.000153+0000 mon.a [INF] overall HEALTH_OK 2024-09-10T21:30:03.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T21:30:03.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 -- 172.21.15.97:0/3427531338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 msgr2=0x7f09ec0b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3427531338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b1f50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f09ec073490 tx=0x7f09d801c770 comp rx=0 tx=0).stop 2024-09-10T21:30:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 -- 172.21.15.97:0/3427531338 shutdown_connections 2024-09-10T21:30:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3427531338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b1f50 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3427531338 >> v2:172.21.15.194:3300/0 conn(0x7f09ec161290 0x7f09ec0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 -- 172.21.15.97:0/3427531338 >> 172.21.15.97:0/3427531338 conn(0x7f09ec0d1a60 msgr2=0x7f09ec0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 -- 172.21.15.97:0/3427531338 shutdown_connections 2024-09-10T21:30:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 -- 172.21.15.97:0/3427531338 wait complete. 2024-09-10T21:30:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 Processor -- start 2024-09-10T21:30:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.953+0000 7f09f1c00640 1 -- start start 2024-09-10T21:30:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09ec0b2490 0x7f09ec0c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09ec161290 0x7f09ec0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec16aac0 con 0x7f09ec0b29b0 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec16a940 con 0x7f09ec0b2490 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec16ac40 con 0x7f09ec161290 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33438/0 (socket says 172.21.15.97:33438) 2024-09-10T21:30:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 -- 172.21.15.97:0/3637463812 learned_addr learned my addr 172.21.15.97:0/3637463812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:03.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1200640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09ec161290 0x7f09ec0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 -- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09ec161290 msgr2=0x7f09ec0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09ec161290 0x7f09ec0b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09ebe00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09ec0b2490 0x7f09ec0c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 -- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09ec0b2490 msgr2=0x7f09ec0c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09ec0b2490 0x7f09ec0c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 -- 172.21.15.97:0/3637463812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec0c6130 con 0x7f09ec0b29b0 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1200640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09ec161290 0x7f09ec0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09ebe00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09ec0b2490 0x7f09ec0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09eb400640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b0320 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f09d80b0040 tx=0x7f09d8004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09e3e00640 1 -- 172.21.15.97:0/3637463812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80b2040 con 0x7f09ec0b29b0 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09e3e00640 1 -- 172.21.15.97:0/3637463812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09d8005160 con 0x7f09ec0b29b0 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec0c6410 con 0x7f09ec0b29b0 2024-09-10T21:30:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09e3e00640 1 -- 172.21.15.97:0/3637463812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80055b0 con 0x7f09ec0b29b0 2024-09-10T21:30:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec158460 con 0x7f09ec0b29b0 2024-09-10T21:30:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.957+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec0b1f50 con 0x7f09ec0b29b0 2024-09-10T21:30:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.961+0000 7f09e3e00640 1 -- 172.21.15.97:0/3637463812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f09d80046d0 con 0x7f09ec0b29b0 2024-09-10T21:30:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.961+0000 7f09e3e00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09cc0725e0 0x7f09cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.961+0000 7f09e3e00640 1 -- 172.21.15.97:0/3637463812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f09d812cd10 con 0x7f09ec0b29b0 2024-09-10T21:30:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.961+0000 7f09ebe00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09cc0725e0 0x7f09cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.961+0000 7f09ebe00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09cc0725e0 0x7f09cc074aa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00a870 tx=0x7f09dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:03.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:03.965+0000 7f09e3e00640 1 -- 172.21.15.97:0/3637463812 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09d80164b0 con 0x7f09ec0b29b0 2024-09-10T21:30:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.161+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09ec0d2730 con 0x7f09ec0b29b0 2024-09-10T21:30:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.161+0000 7f09e3e00640 1 -- 172.21.15.97:0/3637463812 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5231 (secure 0 0 0) 0x7f09d80f8250 con 0x7f09ec0b29b0 2024-09-10T21:30:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:04.165 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"btime":"2024-09-10T21:29:52:583747+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6492,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:29:51.896989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6504},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6504":{"gid":6504,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6836/3597857813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3597857813},{"type":"v1","addr":"172.21.15.194:6836","nonce":3597857813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6504,"qdb_cluster":[6504]},"id":5}]} 2024-09-10T21:30:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 71 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09cc0725e0 msgr2=0x7f09cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09cc0725e0 0x7f09cc074aa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00a870 tx=0x7f09dc008040 comp rx=0 tx=0).stop 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 msgr2=0x7f09ec0b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b0320 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f09d80b0040 tx=0x7f09d8004220 comp rx=0 tx=0).stop 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 shutdown_connections 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09cc0725e0 0x7f09cc074aa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09ec161290 0x7f09ec0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09ec0b29b0 0x7f09ec0b0320 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 --2- 172.21.15.97:0/3637463812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09ec0b2490 0x7f09ec0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 >> 172.21.15.97:0/3637463812 conn(0x7f09ec0d1a60 msgr2=0x7f09ec15c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 shutdown_connections 2024-09-10T21:30:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:04.165+0000 7f09f1c00640 1 -- 172.21.15.97:0/3637463812 wait complete. 2024-09-10T21:30:04.187 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T21:30:09.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:09.229 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:09.240 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- 172.21.15.97:0/72548492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 msgr2=0x7f484c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 --2- 172.21.15.97:0/72548492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 0x7f484c004c90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f485000c310 tx=0x7f485001f6e0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4866800640 1 -- 172.21.15.97:0/72548492 reap_dead start 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- 172.21.15.97:0/72548492 shutdown_connections 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- 172.21.15.97:0/72548492 >> 172.21.15.97:0/72548492 conn(0x7f48680b1780 msgr2=0x7f48680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- 172.21.15.97:0/72548492 shutdown_connections 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- 172.21.15.97:0/72548492 wait complete. 2024-09-10T21:30:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 Processor -- start 2024-09-10T21:30:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- start start 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 0x7f4868133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48681569d0 0x7f4868156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48681f7c70 0x7f48681f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f486813fef0 con 0x7f484c004810 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4868141f80 con 0x7f48681f7c70 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f486cc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4868140070 con 0x7f48681569d0 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 0x7f4868133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 0x7f4868133a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46338/0 (socket says 172.21.15.97:46338) 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 -- 172.21.15.97:0/4020037621 learned_addr learned my addr 172.21.15.97:0/4020037621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f485d400640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48681569d0 0x7f4868156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4866800640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48681f7c70 0x7f48681f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 -- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48681569d0 msgr2=0x7f4868156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48681569d0 0x7f4868156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 -- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48681f7c70 msgr2=0x7f48681f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48681f7c70 0x7f48681f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.341+0000 7f4865e00640 1 -- 172.21.15.97:0/4020037621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4868158610 con 0x7f484c004810 2024-09-10T21:30:09.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f4865e00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 0x7f4868133a00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4858011840 tx=0x7f4858011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:09.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f485ea00640 1 -- 172.21.15.97:0/4020037621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485800fd50 con 0x7f484c004810 2024-09-10T21:30:09.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f485ea00640 1 -- 172.21.15.97:0/4020037621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4858016070 con 0x7f484c004810 2024-09-10T21:30:09.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48681f8770 con 0x7f484c004810 2024-09-10T21:30:09.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f485ea00640 1 -- 172.21.15.97:0/4020037621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580126b0 con 0x7f484c004810 2024-09-10T21:30:09.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4868133200 con 0x7f484c004810 2024-09-10T21:30:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486813f9b0 con 0x7f484c004810 2024-09-10T21:30:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f485ea00640 1 -- 172.21.15.97:0/4020037621 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4858012850 con 0x7f484c004810 2024-09-10T21:30:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f485ea00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4844072610 0x7f4844074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.345+0000 7f485ea00640 1 -- 172.21.15.97:0/4020037621 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f4858093e00 con 0x7f484c004810 2024-09-10T21:30:09.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.349+0000 7f485d400640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4844072610 0x7f4844074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.349+0000 7f485d400640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4844072610 0x7f4844074ad0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f48500b0000 tx=0x7f485001c520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:09.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.349+0000 7f485ea00640 1 -- 172.21.15.97:0/4020037621 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f485805ec60 con 0x7f484c004810 2024-09-10T21:30:09.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4868141360 con 0x7f484c004810 2024-09-10T21:30:09.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f485ea00640 1 -- 172.21.15.97:0/4020037621 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5231 (secure 0 0 0) 0x7f4858063690 con 0x7f484c004810 2024-09-10T21:30:09.552 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:09.552 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"btime":"2024-09-10T21:29:52:583747+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6492,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:29:51.896989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6504},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6504":{"gid":6504,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6836/3597857813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3597857813},{"type":"v1","addr":"172.21.15.194:6836","nonce":3597857813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6504,"qdb_cluster":[6504]},"id":5}]} 2024-09-10T21:30:09.552 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 71 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4844072610 msgr2=0x7f4844074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4844072610 0x7f4844074ad0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f48500b0000 tx=0x7f485001c520 comp rx=0 tx=0).stop 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 msgr2=0x7f4868133a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 0x7f4868133a00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4858011840 tx=0x7f4858011d00 comp rx=0 tx=0).stop 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 shutdown_connections 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4844072610 0x7f4844074ad0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48681f7c70 0x7f48681f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48681569d0 0x7f4868156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 --2- 172.21.15.97:0/4020037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f484c004810 0x7f4868133a00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 >> 172.21.15.97:0/4020037621 conn(0x7f48680b1780 msgr2=0x7f48681405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:09.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 shutdown_connections 2024-09-10T21:30:09.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:09.549+0000 7f486cc00640 1 -- 172.21.15.97:0/4020037621 wait complete. 2024-09-10T21:30:09.573 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T21:30:09.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-09-10T21:30:14.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- 172.21.15.97:0/3381296147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac15fde0 msgr2=0x7fefac13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 --2- 172.21.15.97:0/3381296147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac15fde0 0x7fefac13f9b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009bc0 tx=0x7fefa401cba0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- 172.21.15.97:0/3381296147 shutdown_connections 2024-09-10T21:30:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 --2- 172.21.15.97:0/3381296147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac15fde0 0x7fefac13f9b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 --2- 172.21.15.97:0/3381296147 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac0ce5f0 0x7fefac141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- 172.21.15.97:0/3381296147 >> 172.21.15.97:0/3381296147 conn(0x7fefac0b1780 msgr2=0x7fefac0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- 172.21.15.97:0/3381296147 shutdown_connections 2024-09-10T21:30:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- 172.21.15.97:0/3381296147 wait complete. 2024-09-10T21:30:14.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 Processor -- start 2024-09-10T21:30:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- start start 2024-09-10T21:30:14.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0ce5f0 0x7fefac1662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce9d0 0x7fefac1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15fde0 0x7fefac156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fefac168bf0 con 0x7fefac0ce9d0 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefac168a70 con 0x7fefac0ce5f0 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.725+0000 7fefb3600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fefac168d70 con 0x7fefac15fde0 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce9d0 0x7fefac1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce9d0 0x7fefac1541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46362/0 (socket says 172.21.15.97:46362) 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 -- 172.21.15.97:0/1076048833 learned_addr learned my addr 172.21.15.97:0/1076048833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefb1600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15fde0 0x7fefac156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefb0c00640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0ce5f0 0x7fefac1662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 -- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15fde0 msgr2=0x7fefac156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15fde0 0x7fefac156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 -- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0ce5f0 msgr2=0x7fefac1662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0ce5f0 0x7fefac1662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 -- 172.21.15.97:0/1076048833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac072b10 con 0x7fefac0ce9d0 2024-09-10T21:30:14.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefb0c00640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0ce5f0 0x7fefac1662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:14.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa3e00640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce9d0 0x7fefac1541a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fef9400c450 tx=0x7fef9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:14.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa9600640 1 -- 172.21.15.97:0/1076048833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94017070 con 0x7fefac0ce9d0 2024-09-10T21:30:14.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa9600640 1 -- 172.21.15.97:0/1076048833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef94003600 con 0x7fefac0ce9d0 2024-09-10T21:30:14.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac072df0 con 0x7fefac0ce9d0 2024-09-10T21:30:14.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa9600640 1 -- 172.21.15.97:0/1076048833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94003a50 con 0x7fefac0ce9d0 2024-09-10T21:30:14.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac0733b0 con 0x7fefac0ce9d0 2024-09-10T21:30:14.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefac141d20 con 0x7fefac0ce9d0 2024-09-10T21:30:14.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.729+0000 7fefa9600640 1 -- 172.21.15.97:0/1076048833 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fef94003bf0 con 0x7fefac0ce9d0 2024-09-10T21:30:14.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.733+0000 7fefa9600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef880725e0 0x7fef88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.733+0000 7fefa9600640 1 -- 172.21.15.97:0/1076048833 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fef94092a40 con 0x7fefac0ce9d0 2024-09-10T21:30:14.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.733+0000 7fefb0c00640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef880725e0 0x7fef88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.733+0000 7fefb0c00640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef880725e0 0x7fef88074aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005fd0 tx=0x7fef9c005dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.737+0000 7fefa9600640 1 -- 172.21.15.97:0/1076048833 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fef9405e040 con 0x7fefac0ce9d0 2024-09-10T21:30:14.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.933+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefac1418f0 con 0x7fefac0ce9d0 2024-09-10T21:30:14.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefa9600640 1 -- 172.21.15.97:0/1076048833 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5279 (secure 0 0 0) 0x7fef9405d940 con 0x7fefac0ce9d0 2024-09-10T21:30:14.938 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:14.938 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":72,"btime":"2024-09-10T21:30:11:597862+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6492,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:11.597822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6504},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6504":{"gid":6504,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6836/3597857813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3597857813},{"type":"v1","addr":"172.21.15.194:6836","nonce":3597857813}]},"join_fscid":-1,"laggy_since":"2024-09-10T21:30:11.597822+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6504,"qdb_cluster":[6504]},"id":5}]} 2024-09-10T21:30:14.938 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 72 2024-09-10T21:30:14.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef880725e0 msgr2=0x7fef88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef880725e0 0x7fef88074aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005fd0 tx=0x7fef9c005dc0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce9d0 msgr2=0x7fefac1541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce9d0 0x7fefac1541a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fef9400c450 tx=0x7fef9400c910 comp rx=0 tx=0).stop 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 shutdown_connections 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef880725e0 0x7fef88074aa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefac15fde0 0x7fefac156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefac0ce9d0 0x7fefac1541a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 --2- 172.21.15.97:0/1076048833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac0ce5f0 0x7fefac1662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 >> 172.21.15.97:0/1076048833 conn(0x7fefac0b1780 msgr2=0x7fefac15c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 shutdown_connections 2024-09-10T21:30:14.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:14.937+0000 7fefb3600640 1 -- 172.21.15.97:0/1076048833 wait complete. 2024-09-10T21:30:14.959 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-10T21:30:15.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 -- 172.21.15.97:0/3487314479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b1e00 msgr2=0x7f8ea806dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 --2- 172.21.15.97:0/3487314479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b1e00 0x7f8ea806dc50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009900 tx=0x7f8e9001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 -- 172.21.15.97:0/3487314479 shutdown_connections 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 --2- 172.21.15.97:0/3487314479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b1e00 0x7f8ea806dc50 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 --2- 172.21.15.97:0/3487314479 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ea80b2bb0 0x7f8ea80b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 -- 172.21.15.97:0/3487314479 >> 172.21.15.97:0/3487314479 conn(0x7f8ea80b37c0 msgr2=0x7f8ea80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 -- 172.21.15.97:0/3487314479 shutdown_connections 2024-09-10T21:30:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.201+0000 7f8eaca00640 1 -- 172.21.15.97:0/3487314479 wait complete. 2024-09-10T21:30:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 Processor -- start 2024-09-10T21:30:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 -- start start 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b2bb0 0x7f8ea81588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ea80b42e0 0x7f8ea8150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ea8151750 0x7f8ea815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea80d12c0 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea806eae0 con 0x7f8ea8151750 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea80d1440 con 0x7f8ea80b42e0 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b2bb0 0x7f8ea81588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b2bb0 0x7f8ea81588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46372/0 (socket says 172.21.15.97:46372) 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 -- 172.21.15.97:0/453037840 learned_addr learned my addr 172.21.15.97:0/453037840 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8e9d400640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ea80b42e0 0x7f8ea8150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea6800640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ea8151750 0x7f8ea815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 -- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ea80b42e0 msgr2=0x7f8ea8150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ea80b42e0 0x7f8ea8150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 -- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ea8151750 msgr2=0x7f8ea815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ea8151750 0x7f8ea815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 -- 172.21.15.97:0/453037840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea81558f0 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8ea5e00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b2bb0 0x7f8ea81588b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800d980 tx=0x7f8e9800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8e9ea00640 1 -- 172.21.15.97:0/453037840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98017070 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8e9ea00640 1 -- 172.21.15.97:0/453037840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e98009d60 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8e9ea00640 1 -- 172.21.15.97:0/453037840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98005280 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea8155bd0 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea815cf80 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.205+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea80b2f90 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.209+0000 7f8e9ea00640 1 -- 172.21.15.97:0/453037840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8e98005420 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.209+0000 7f8e9ea00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e840725e0 0x7f8e84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.209+0000 7f8e9ea00640 1 -- 172.21.15.97:0/453037840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7f8e98093630 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.209+0000 7f8e9d400640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e840725e0 0x7f8e84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.209+0000 7f8e9d400640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e840725e0 0x7f8e84074aa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8e900af000 tx=0x7f8e9001b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.213+0000 7f8e9ea00640 1 -- 172.21.15.97:0/453037840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e9805e400 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.377+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f8ea806d7e0 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.381+0000 7f8e9ea00640 1 -- 172.21.15.97:0/453037840 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v41) ==== 121+0+0 (secure 0 0 0) 0x7f8e98067020 con 0x7f8ea80b2bb0 2024-09-10T21:30:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e840725e0 msgr2=0x7f8e84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e840725e0 0x7f8e84074aa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8e900af000 tx=0x7f8e9001b100 comp rx=0 tx=0).stop 2024-09-10T21:30:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b2bb0 msgr2=0x7f8ea81588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b2bb0 0x7f8ea81588b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800d980 tx=0x7f8e9800de40 comp rx=0 tx=0).stop 2024-09-10T21:30:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 shutdown_connections 2024-09-10T21:30:15.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e840725e0 0x7f8e84074aa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ea8151750 0x7f8ea815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ea80b42e0 0x7f8ea8150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 --2- 172.21.15.97:0/453037840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea80b2bb0 0x7f8ea81588b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 >> 172.21.15.97:0/453037840 conn(0x7f8ea80b37c0 msgr2=0x7f8ea80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 shutdown_connections 2024-09-10T21:30:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.385+0000 7f8eaca00640 1 -- 172.21.15.97:0/453037840 wait complete. 2024-09-10T21:30:15.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- 172.21.15.97:0/23345391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 msgr2=0x7fefc406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 --2- 172.21.15.97:0/23345391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc406fd50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fefc4069e70 tx=0x7fefbc01cd80 comp rx=0 tx=0).stop 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- 172.21.15.97:0/23345391 shutdown_connections 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 --2- 172.21.15.97:0/23345391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc406fd50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 --2- 172.21.15.97:0/23345391 >> v2:172.21.15.194:3300/0 conn(0x7fefc4144690 0x7fefc40b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 --2- 172.21.15.97:0/23345391 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefc4143f90 0x7fefc40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- 172.21.15.97:0/23345391 >> 172.21.15.97:0/23345391 conn(0x7fefc40b1780 msgr2=0x7fefc40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:15.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- 172.21.15.97:0/23345391 shutdown_connections 2024-09-10T21:30:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- 172.21.15.97:0/23345391 wait complete. 2024-09-10T21:30:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 Processor -- start 2024-09-10T21:30:15.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- start start 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc40a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc4143f90 0x7fefc4133a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefc4144690 0x7fefc40cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fefc415ede0 con 0x7fefc40b4660 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefc415ec60 con 0x7fefc4143f90 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefcb400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fefc415ef60 con 0x7fefc4144690 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc40a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc40a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46388/0 (socket says 172.21.15.97:46388) 2024-09-10T21:30:15.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc8a00640 1 -- 172.21.15.97:0/4102332470 learned_addr learned my addr 172.21.15.97:0/4102332470 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefbbe00640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc4143f90 0x7fefc4133a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc9400640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefc4144690 0x7fefc40cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc8a00640 1 -- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefc4144690 msgr2=0x7fefc40cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc8a00640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefc4144690 0x7fefc40cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc8a00640 1 -- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc4143f90 msgr2=0x7fefc4133a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.549+0000 7fefc8a00640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc4143f90 0x7fefc4133a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc8a00640 1 -- 172.21.15.97:0/4102332470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40cc510 con 0x7fefc40b4660 2024-09-10T21:30:15.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc8a00640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc40a94a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fefb400c970 tx=0x7fefb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc1600640 1 -- 172.21.15.97:0/4102332470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4007c10 con 0x7fefc40b4660 2024-09-10T21:30:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc1600640 1 -- 172.21.15.97:0/4102332470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefb400f450 con 0x7fefc40b4660 2024-09-10T21:30:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc1600640 1 -- 172.21.15.97:0/4102332470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb400f8a0 con 0x7fefc40b4660 2024-09-10T21:30:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc40a9b60 con 0x7fefc40b4660 2024-09-10T21:30:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc40aa120 con 0x7fefc40b4660 2024-09-10T21:30:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc415f4f0 con 0x7fefc40b4660 2024-09-10T21:30:15.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc1600640 1 -- 172.21.15.97:0/4102332470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fefb400fa40 con 0x7fefc40b4660 2024-09-10T21:30:15.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc1600640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefa00725e0 0x7fefa0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:15.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.553+0000 7fefc1600640 1 -- 172.21.15.97:0/4102332470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6101+0+0 (secure 0 0 0) 0x7fefb4093a60 con 0x7fefc40b4660 2024-09-10T21:30:15.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.557+0000 7fefbbe00640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefa00725e0 0x7fefa0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:15.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.557+0000 7fefbbe00640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefa00725e0 0x7fefa0074aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fefc40cf7d0 tx=0x7fefac00a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:15.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.557+0000 7fefc1600640 1 -- 172.21.15.97:0/4102332470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fefb4014030 con 0x7fefc40b4660 2024-09-10T21:30:15.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:15.785+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) -- 0x7fefc40738f0 con 0x7fefc40b4660 2024-09-10T21:30:16.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2024-09-10T21:30:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.597+0000 7fefc1600640 1 -- 172.21.15.97:0/4102332470 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.6504 v73) ==== 115+0+0 (secure 0 0 0) 0x7fefb405e8c0 con 0x7fefc40b4660 2024-09-10T21:30:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:unfreezing mds.6504 2024-09-10T21:30:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefa00725e0 msgr2=0x7fefa0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefa00725e0 0x7fefa0074aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fefc40cf7d0 tx=0x7fefac00a2d0 comp rx=0 tx=0).stop 2024-09-10T21:30:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 msgr2=0x7fefc40a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc40a94a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fefb400c970 tx=0x7fefb400ce30 comp rx=0 tx=0).stop 2024-09-10T21:30:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 shutdown_connections 2024-09-10T21:30:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefa00725e0 0x7fefa0074aa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefc4144690 0x7fefc40cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc4143f90 0x7fefc4133a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 --2- 172.21.15.97:0/4102332470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefc40b4660 0x7fefc40a94a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 >> 172.21.15.97:0/4102332470 conn(0x7fefc40b1780 msgr2=0x7fefc40b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 shutdown_connections 2024-09-10T21:30:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:16.601+0000 7fefcb400640 1 -- 172.21.15.97:0/4102332470 wait complete. 2024-09-10T21:30:16.632 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -n kernel.core_pattern 2024-09-10T21:30:16.651 INFO:teuthology.orchestra.run.smithi194.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:30:16.653 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:30:16.653 DEBUG:teuthology.orchestra.run.smithi194:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-10T21:30:16.733 INFO:teuthology.orchestra.run.smithi194.stdout:1726003798.35053.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/ceph-mds', platform: 'x86_64' 2024-09-10T21:30:16.736 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726003798.35053.core'] 2024-09-10T21:30:16.736 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726003798.35053.core, deleting it 2024-09-10T21:30:16.736 DEBUG:teuthology.orchestra.run.smithi194:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726003798.35053.core 2024-09-10T21:30:16.813 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:30:16.813 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:30:16.813 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:30:16.813 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:30:16.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:30:16.814 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi194 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-09-10T21:30:16.814 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:30:16.814 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:30:16.860 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:30:16.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:16.944 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:30:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 -- 172.21.15.97:0/2594386963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c15fde0 msgr2=0x7fe70c166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 --2- 172.21.15.97:0/2594386963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c15fde0 0x7fe70c166850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe704009b70 tx=0x7fe70401c930 comp rx=0 tx=0).stop 2024-09-10T21:30:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 -- 172.21.15.97:0/2594386963 shutdown_connections 2024-09-10T21:30:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 --2- 172.21.15.97:0/2594386963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c15fde0 0x7fe70c166850 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 -- 172.21.15.97:0/2594386963 >> 172.21.15.97:0/2594386963 conn(0x7fe70c0b1780 msgr2=0x7fe70c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 -- 172.21.15.97:0/2594386963 shutdown_connections 2024-09-10T21:30:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 -- 172.21.15.97:0/2594386963 wait complete. 2024-09-10T21:30:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 Processor -- start 2024-09-10T21:30:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.009+0000 7fe713a00640 1 -- start start 2024-09-10T21:30:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c0ce5f0 0x7fe70c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70c141d20 0x7fe70c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe70c15fde0 0x7fe70c13dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c16a180 con 0x7fe70c141d20 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c16a000 con 0x7fe70c15fde0 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe70c16a300 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe703e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70c141d20 0x7fe70c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe703e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70c141d20 0x7fe70c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46396/0 (socket says 172.21.15.97:46396) 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe703e00640 1 -- 172.21.15.97:0/3885641417 learned_addr learned my addr 172.21.15.97:0/3885641417 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711000640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c0ce5f0 0x7fe70c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711a00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe70c15fde0 0x7fe70c13dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711000640 1 -- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe70c15fde0 msgr2=0x7fe70c13dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711000640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe70c15fde0 0x7fe70c13dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711000640 1 -- 172.21.15.97:0/3885641417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70c141d20 msgr2=0x7fe70c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711000640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70c141d20 0x7fe70c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711000640 1 -- 172.21.15.97:0/3885641417 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c1faf60 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe703e00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70c141d20 0x7fe70c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe711000640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c0ce5f0 0x7fe70c152be0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc000f70 tx=0x7fe6fc00cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe709600640 1 -- 172.21.15.97:0/3885641417 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc017070 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe709600640 1 -- 172.21.15.97:0/3885641417 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6fc005360 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c1fb240 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe709600640 1 -- 172.21.15.97:0/3885641417 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00a8a0 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1fb800 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.013+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c0ce9d0 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.017+0000 7fe709600640 1 -- 172.21.15.97:0/3885641417 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe6fc00aa40 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.017+0000 7fe709600640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.017+0000 7fe709600640 1 -- 172.21.15.97:0/3885641417 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fe6fc092b90 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.017+0000 7fe703e00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.017+0000 7fe703e00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe70c1fbec0 tx=0x7fe6f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.021+0000 7fe709600640 1 -- 172.21.15.97:0/3885641417 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe6fc05da30 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.221+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe70c1410c0 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.221+0000 7fe709600640 1 -- 172.21.15.97:0/3885641417 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+4397 (secure 0 0 0) 0x7fe6fc062460 con 0x7fe70c0ce5f0 2024-09-10T21:30:17.225 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:17.225 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":73,"btime":"2024-09-10T21:30:16:599622+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:16.599616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"c","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-10T21:30:17.225 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 73 2024-09-10T21:30:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 msgr2=0x7fe6e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe70c1fbec0 tx=0x7fe6f4009040 comp rx=0 tx=0).stop 2024-09-10T21:30:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c0ce5f0 msgr2=0x7fe70c152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c0ce5f0 0x7fe70c152be0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc000f70 tx=0x7fe6fc00cb90 comp rx=0 tx=0).stop 2024-09-10T21:30:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 shutdown_connections 2024-09-10T21:30:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe6e80725e0 0x7fe6e8074aa0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:17.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe70c15fde0 0x7fe70c13dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe70c141d20 0x7fe70c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 --2- 172.21.15.97:0/3885641417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe70c0ce5f0 0x7fe70c152be0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 >> 172.21.15.97:0/3885641417 conn(0x7fe70c0b1780 msgr2=0x7fe70c15b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 shutdown_connections 2024-09-10T21:30:17.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:17.225+0000 7fe713a00640 1 -- 172.21.15.97:0/3885641417 wait complete. 2024-09-10T21:30:17.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:29:38.920694+0000', 'modified': '2024-09-10T21:30:16.599616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6492}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6492': {'gid': 6492, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6834/2931568595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2931568595}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 2931568595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:30:17.648 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6492:up:replay 2024-09-10T21:30:18.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2272174290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 msgr2=0x7f7ef8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 --2- 172.21.15.97:0/2272174290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 0x7f7ef8003eb0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c640 tx=0x7f7f0401d260 comp rx=0 tx=0).stop 2024-09-10T21:30:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f18e00640 1 -- 172.21.15.97:0/2272174290 reap_dead start 2024-09-10T21:30:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2272174290 shutdown_connections 2024-09-10T21:30:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2272174290 >> 172.21.15.97:0/2272174290 conn(0x7f7f140b1780 msgr2=0x7f7f140b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:18.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2272174290 shutdown_connections 2024-09-10T21:30:18.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2272174290 wait complete. 2024-09-10T21:30:18.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 Processor -- start 2024-09-10T21:30:18.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- start start 2024-09-10T21:30:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 0x7f7f1413c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f141f83e0 0x7f7f14157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f141f8860 0x7f7f141582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14167b40 con 0x7f7ef8003a90 2024-09-10T21:30:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7f141679c0 con 0x7f7f141f83e0 2024-09-10T21:30:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.797+0000 7f7f1ae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7f14167cc0 con 0x7f7f141f8860 2024-09-10T21:30:18.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f13e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 0x7f7f1413c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f13e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 0x7f7f1413c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46230/0 (socket says 172.21.15.97:46230) 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f0b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f141f83e0 0x7f7f14157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f18e00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f141f8860 0x7f7f141582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f13e00640 1 -- 172.21.15.97:0/2506247250 learned_addr learned my addr 172.21.15.97:0/2506247250 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f0b400640 1 -- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f141f8860 msgr2=0x7f7f141582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f0b400640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f141f8860 0x7f7f141582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f0b400640 1 -- 172.21.15.97:0/2506247250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 msgr2=0x7f7f1413c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f0b400640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 0x7f7f1413c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f0b400640 1 -- 172.21.15.97:0/2506247250 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f141fbc90 con 0x7f7f141f83e0 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f13e00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 0x7f7f1413c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f18e00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f141f8860 0x7f7f141582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f0b400640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f141f83e0 0x7f7f14157d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7efc006fd0 tx=0x7f7efc001680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:18.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f10c00640 1 -- 172.21.15.97:0/2506247250 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc015070 con 0x7f7f141f83e0 2024-09-10T21:30:18.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f10c00640 1 -- 172.21.15.97:0/2506247250 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7efc011840 con 0x7f7f141f83e0 2024-09-10T21:30:18.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f10c00640 1 -- 172.21.15.97:0/2506247250 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc011c70 con 0x7f7f141f83e0 2024-09-10T21:30:18.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f141fbf70 con 0x7f7f141f83e0 2024-09-10T21:30:18.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f141f7ee0 con 0x7f7f141f83e0 2024-09-10T21:30:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1415f6d0 con 0x7f7f141f83e0 2024-09-10T21:30:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.801+0000 7f7f10c00640 1 -- 172.21.15.97:0/2506247250 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7efc0103c0 con 0x7f7f141f83e0 2024-09-10T21:30:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.805+0000 7f7f10c00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ef00725e0 0x7f7ef0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.805+0000 7f7f10c00640 1 -- 172.21.15.97:0/2506247250 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f7efc094250 con 0x7f7f141f83e0 2024-09-10T21:30:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.805+0000 7f7f13e00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ef00725e0 0x7f7ef0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:18.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.805+0000 7f7f13e00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ef00725e0 0x7f7ef0074aa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7f0401bb30 tx=0x7f7f0401a0a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:18.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:18.809+0000 7f7f10c00640 1 -- 172.21.15.97:0/2506247250 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7efc05f070 con 0x7f7f141f83e0 2024-09-10T21:30:19.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f14141360 con 0x7f7f141f83e0 2024-09-10T21:30:19.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f10c00640 1 -- 172.21.15.97:0/2506247250 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5226 (secure 0 0 0) 0x7f7efc063aa0 con 0x7f7f141f83e0 2024-09-10T21:30:19.012 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:19.012 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":75,"btime":"2024-09-10T21:30:18:604078+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/2912272065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2912272065},{"type":"v1","addr":"172.21.15.194:6836","nonce":2912272065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:17.606461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"c","rank":0,"incarnation":73,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-10T21:30:19.012 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 75 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ef00725e0 msgr2=0x7f7ef0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ef00725e0 0x7f7ef0074aa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7f0401bb30 tx=0x7f7f0401a0a0 comp rx=0 tx=0).stop 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f141f83e0 msgr2=0x7f7f14157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f141f83e0 0x7f7f14157d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7efc006fd0 tx=0x7f7efc001680 comp rx=0 tx=0).stop 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 shutdown_connections 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ef00725e0 0x7f7ef0074aa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7f141f8860 0x7f7f141582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f141f83e0 0x7f7f14157d90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 --2- 172.21.15.97:0/2506247250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef8003a90 0x7f7f1413c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 >> 172.21.15.97:0/2506247250 conn(0x7f7f140b1780 msgr2=0x7f7f1415c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 shutdown_connections 2024-09-10T21:30:19.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19.009+0000 7f7f1ae00640 1 -- 172.21.15.97:0/2506247250 wait complete. 2024-09-10T21:30:19.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:29:38.920694+0000', 'modified': '2024-09-10T21:30:17.606461+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6492}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6492': {'gid': 6492, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:rejoin', 'state_seq': 13, 'addr': '172.21.15.97:6834/2931568595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2931568595}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 2931568595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:30:19.436 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6492:up:rejoin 2024-09-10T21:30:19.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:30:20.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:20.492 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:30:20.489+0000 7f03fea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:30:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- 172.21.15.97:0/244652453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 msgr2=0x7f6ca80cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 --2- 172.21.15.97:0/244652453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca80cff40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80ce4d0 tx=0x7f6c9001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- 172.21.15.97:0/244652453 shutdown_connections 2024-09-10T21:30:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 --2- 172.21.15.97:0/244652453 >> v2:172.21.15.194:3300/0 conn(0x7f6ca80af100 0x7f6ca80af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 --2- 172.21.15.97:0/244652453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca80cff40 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- 172.21.15.97:0/244652453 >> 172.21.15.97:0/244652453 conn(0x7f6ca80b28c0 msgr2=0x7f6ca80b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- 172.21.15.97:0/244652453 shutdown_connections 2024-09-10T21:30:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- 172.21.15.97:0/244652453 wait complete. 2024-09-10T21:30:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 Processor -- start 2024-09-10T21:30:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- start start 2024-09-10T21:30:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ca80af100 0x7f6ca80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca8153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ca815bdd0 0x7f6ca81fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8168e80 con 0x7f6ca80cfad0 2024-09-10T21:30:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8168d00 con 0x7f6ca80af100 2024-09-10T21:30:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.589+0000 7f6cad600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca8169000 con 0x7f6ca815bdd0 2024-09-10T21:30:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca8153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca8153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46260/0 (socket says 172.21.15.97:46260) 2024-09-10T21:30:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 -- 172.21.15.97:0/2541006162 learned_addr learned my addr 172.21.15.97:0/2541006162 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6ca6a00640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ca80af100 0x7f6ca80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6ca7400640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ca815bdd0 0x7f6ca81fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 -- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ca815bdd0 msgr2=0x7f6ca81fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ca815bdd0 0x7f6ca81fbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 -- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ca80af100 msgr2=0x7f6ca80b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ca80af100 0x7f6ca80b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 -- 172.21.15.97:0/2541006162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca813b490 con 0x7f6ca80cfad0 2024-09-10T21:30:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6ca7400640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ca815bdd0 0x7f6ca81fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6ca6a00640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ca80af100 0x7f6ca80b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9e000640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca8153cd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6c900b0040 tx=0x7f6c900a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9f400640 1 -- 172.21.15.97:0/2541006162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c900b2040 con 0x7f6ca80cfad0 2024-09-10T21:30:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9f400640 1 -- 172.21.15.97:0/2541006162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c90004420 con 0x7f6ca80cfad0 2024-09-10T21:30:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9f400640 1 -- 172.21.15.97:0/2541006162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90005290 con 0x7f6ca80cfad0 2024-09-10T21:30:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca813b770 con 0x7f6ca80cfad0 2024-09-10T21:30:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca813bd30 con 0x7f6ca80cfad0 2024-09-10T21:30:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca80af530 con 0x7f6ca80cfad0 2024-09-10T21:30:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.593+0000 7f6c9f400640 1 -- 172.21.15.97:0/2541006162 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6c90005430 con 0x7f6ca80cfad0 2024-09-10T21:30:20.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.597+0000 7f6c9f400640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c840725e0 0x7f6c84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.597+0000 7f6c9f400640 1 -- 172.21.15.97:0/2541006162 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f6c9012cff0 con 0x7f6ca80cfad0 2024-09-10T21:30:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.597+0000 7f6ca6a00640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c840725e0 0x7f6c84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.597+0000 7f6ca6a00640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c840725e0 0x7f6c84074aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009fc0 tx=0x7f6c98009550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.601+0000 7f6c9f400640 1 -- 172.21.15.97:0/2541006162 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c900164b0 con 0x7f6ca80cfad0 2024-09-10T21:30:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ca80d1470 con 0x7f6ca80cfad0 2024-09-10T21:30:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6c9f400640 1 -- 172.21.15.97:0/2541006162 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5233 (secure 0 0 0) 0x7f6c900f84e0 con 0x7f6ca80cfad0 2024-09-10T21:30:20.803 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:20.803 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":76,"btime":"2024-09-10T21:30:19:606108+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/2912272065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2912272065},{"type":"v1","addr":"172.21.15.194:6836","nonce":2912272065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:19.606099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":14,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6492,"qdb_cluster":[6492]},"id":5}]} 2024-09-10T21:30:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 76 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c840725e0 msgr2=0x7f6c84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c840725e0 0x7f6c84074aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009fc0 tx=0x7f6c98009550 comp rx=0 tx=0).stop 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 msgr2=0x7f6ca8153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca8153cd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6c900b0040 tx=0x7f6c900a5740 comp rx=0 tx=0).stop 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 shutdown_connections 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c840725e0 0x7f6c84074aa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ca815bdd0 0x7f6ca81fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca80cfad0 0x7f6ca8153cd0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 --2- 172.21.15.97:0/2541006162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ca80af100 0x7f6ca80b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:20.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 >> 172.21.15.97:0/2541006162 conn(0x7f6ca80b28c0 msgr2=0x7f6ca80b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 shutdown_connections 2024-09-10T21:30:20.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:20.800+0000 7f6cad600640 1 -- 172.21.15.97:0/2541006162 wait complete. 2024-09-10T21:30:21.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:29:38.920694+0000', 'modified': '2024-09-10T21:30:19.606099+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6492}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6492': {'gid': 6492, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.97:6834/2931568595', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2931568595}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 2931568595}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6492, 'qdb_cluster': [6492]} 2024-09-10T21:30:21.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:30:21.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:30:21.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 -- 172.21.15.97:0/1175946812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 msgr2=0x7f7a280b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 --2- 172.21.15.97:0/1175946812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a280b37b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009c80 tx=0x7f7a1801f560 comp rx=0 tx=0).stop 2024-09-10T21:30:21.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 -- 172.21.15.97:0/1175946812 shutdown_connections 2024-09-10T21:30:21.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 --2- 172.21.15.97:0/1175946812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a2806f180 0x7f7a280ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 --2- 172.21.15.97:0/1175946812 >> v2:172.21.15.194:3300/0 conn(0x7f7a280b3cf0 0x7f7a2806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 --2- 172.21.15.97:0/1175946812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a280b37b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 -- 172.21.15.97:0/1175946812 >> 172.21.15.97:0/1175946812 conn(0x7f7a280b1780 msgr2=0x7f7a280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 -- 172.21.15.97:0/1175946812 shutdown_connections 2024-09-10T21:30:21.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 -- 172.21.15.97:0/1175946812 wait complete. 2024-09-10T21:30:21.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 Processor -- start 2024-09-10T21:30:21.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.368+0000 7f7a2d200640 1 -- start start 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a2806f180 0x7f7a2814b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a280b3cf0 0x7f7a281f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a281f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2806e3b0 con 0x7f7a281511f0 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7a280cf7d0 con 0x7f7a2806f180 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2806e530 con 0x7f7a280b3cf0 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a281f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a281f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46282/0 (socket says 172.21.15.97:46282) 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 -- 172.21.15.97:0/2597228886 learned_addr learned my addr 172.21.15.97:0/2597228886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a1d400640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a280b3cf0 0x7f7a281f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a25e00640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a2806f180 0x7f7a2814b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 -- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a280b3cf0 msgr2=0x7f7a281f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a280b3cf0 0x7f7a281f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 -- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a2806f180 msgr2=0x7f7a2814b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a2806f180 0x7f7a2814b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 -- 172.21.15.97:0/2597228886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a28158cb0 con 0x7f7a281511f0 2024-09-10T21:30:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a26800640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a281f6060 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c940 tx=0x7f7a2000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a1ea00640 1 -- 172.21.15.97:0/2597228886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20004460 con 0x7f7a281511f0 2024-09-10T21:30:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a1ea00640 1 -- 172.21.15.97:0/2597228886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a20007890 con 0x7f7a281511f0 2024-09-10T21:30:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a1ea00640 1 -- 172.21.15.97:0/2597228886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20007ce0 con 0x7f7a281511f0 2024-09-10T21:30:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a28158f90 con 0x7f7a281511f0 2024-09-10T21:30:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.372+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a28151860 con 0x7f7a281511f0 2024-09-10T21:30:21.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a1ea00640 1 -- 172.21.15.97:0/2597228886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7a20003600 con 0x7f7a281511f0 2024-09-10T21:30:21.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a1ea00640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a040725e0 0x7f7a04074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a1ea00640 1 -- 172.21.15.97:0/2597228886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f7a20093a70 con 0x7f7a281511f0 2024-09-10T21:30:21.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 Processor -- start 2024-09-10T21:30:21.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a040725e0 0x7f7a04074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 -- start start 2024-09-10T21:30:21.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a040725e0 0x7f7a04074aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a180b3000 tx=0x7f7a1801dda0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac741f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac741aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac741ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac740cb3f0 con 0x55ac741f69d0 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ac74008b20 con 0x55ac741aae10 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ac73ff7f50 con 0x55ac741f72f0 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac741f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac741f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46284/0 (socket says 172.21.15.97:46284) 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 -- 172.21.15.97:0/3910604359 learned_addr learned my addr 172.21.15.97:0/3910604359 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a26800640 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac741ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a1d400640 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac741aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 -- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 msgr2=0x55ac741aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac741aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 -- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 msgr2=0x55ac741ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac741ab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 -- 172.21.15.97:0/3910604359 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac73fbf590 con 0x55ac741f69d0 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a25e00640 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac741f6db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7a1801fe00 tx=0x7f7a180bb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30c396fa2f0014a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a16a00640 1 -- 172.21.15.97:0/3910604359 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a180b6040 con 0x55ac741f69d0 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a16a00640 1 -- 172.21.15.97:0/3910604359 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a180cd070 con 0x55ac741f69d0 2024-09-10T21:30:21.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.376+0000 7f7a16a00640 1 -- 172.21.15.97:0/3910604359 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18004730 con 0x55ac741f69d0 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- 172.21.15.97:0/3910604359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 msgr2=0x55ac741f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac741f6db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7a1801fe00 tx=0x7f7a180bb000 comp rx=0 tx=0).stop 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- 172.21.15.97:0/3910604359 shutdown_connections 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac741ab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac741aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 --2- 172.21.15.97:0/3910604359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac741f6db0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- 172.21.15.97:0/3910604359 >> 172.21.15.97:0/3910604359 conn(0x55ac74094130 msgr2=0x55ac741d9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- 172.21.15.97:0/3910604359 shutdown_connections 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- 172.21.15.97:0/3910604359 wait complete. 2024-09-10T21:30:21.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- start start 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 Processor -- start 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741aae10 0x55ac741ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f69d0 0x55ac741ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741f72f0 0x55ac741accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac73f925d0 con 0x55ac741aae10 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ac73ffbd60 con 0x55ac741f72f0 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ac73f9e500 con 0x55ac741f69d0 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 10 client.0 _ll_register_callbacks cb 0x55ac73fa2810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741aae10 0x55ac741ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741aae10 0x55ac741ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46292/0 (socket says 172.21.15.97:46292) 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 -- 172.21.15.97:0/1368108363 learned_addr learned my addr 172.21.15.97:0/1368108363 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 -- 172.21.15.97:0/1368108363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f69d0 msgr2=0x55ac741ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f69d0 0x55ac741ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 -- 172.21.15.97:0/1368108363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741f72f0 msgr2=0x55ac741accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741f72f0 0x55ac741accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 -- 172.21.15.97:0/1368108363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac73f96170 con 0x55ac741aae10 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a25e00640 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741aae10 0x55ac741ac240 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7a18026270 tx=0x7f7a18009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bea3685cb9c6e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a0b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a0b400640 1 -- 172.21.15.97:0/1368108363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a180d7030 con 0x55ac741aae10 2024-09-10T21:30:21.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a0b400640 1 -- 172.21.15.97:0/1368108363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f7a18004710 con 0x55ac741aae10 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac73f900f0 con 0x55ac741aae10 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a0b400640 1 -- 172.21.15.97:0/1368108363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7a180e3050 con 0x55ac741aae10 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.380+0000 7f7a3053a000 10 client.6798 fetch_fsmap learned FSMap version 76 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ac7407cef0 con 0x55ac741aae10 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a0b400640 1 -- 172.21.15.97:0/1368108363 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ac73f900f0 con 0x55ac741aae10 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a0b400640 1 -- 172.21.15.97:0/1368108363 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f7a180e6020 con 0x55ac741aae10 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 10 client.6798 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 10 client.6798 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 20 client.6798 populate_metadata read hostname 'smithi097' 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74205c70 0x55ac74208060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 4 client.6798 mds_command: new command op to 6492 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 --> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac7400aa70 con 0x55ac74205c70 2024-09-10T21:30:21.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a1d400640 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74205c70 0x55ac74208060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a1d400640 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74205c70 0x55ac74208060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c682d8485695b2f server_cookie=4702bddf6df5908b in_seq=0 out_seq=0 2024-09-10T21:30:21.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a0b400640 10 client.6798 ms_handle_connect on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a0b400640 1 -- 172.21.15.97:0/1368108363 <== mds.0 v2:172.21.15.97:6833/2931568595 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac7400aa70 con 0x55ac74205c70 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a0b400640 10 client.6798 handle_command_reply: tid=0 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 client.6798 shutdown 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741aae10 msgr2=0x55ac741ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741aae10 0x55ac741ac240 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7a18026270 tx=0x7f7a18009380 comp rx=0 tx=0).stop 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 shutdown_connections 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74205c70 0x55ac74208060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741f72f0 0x55ac741accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f69d0 0x55ac741ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 --2- 172.21.15.97:0/1368108363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741aae10 0x55ac741ac240 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a0b400640 0 client.6798 ms_handle_reset on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 >> 172.21.15.97:0/1368108363 conn(0x55ac74094130 msgr2=0x55ac741a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 shutdown_connections 2024-09-10T21:30:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 1 -- 172.21.15.97:0/1368108363 wait complete. 2024-09-10T21:30:21.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.384+0000 7f7a3053a000 20 client.6798 trim_cache size 0 max 16384 2024-09-10T21:30:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 Processor -- start 2024-09-10T21:30:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 -- start start 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac742689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74268f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74269470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a280cd8c0 con 0x55ac741f69d0 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ac74000390 con 0x55ac741aae10 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ac73f925d0 con 0x55ac741f72f0 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74268f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74268f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46298/0 (socket says 172.21.15.97:46298) 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 -- 172.21.15.97:0/517736958 learned_addr learned my addr 172.21.15.97:0/517736958 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a26800640 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74269470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a25e00640 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac742689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 -- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 msgr2=0x55ac74269470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74269470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 -- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 msgr2=0x55ac742689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac742689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 -- 172.21.15.97:0/517736958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac74053d90 con 0x55ac741f69d0 2024-09-10T21:30:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a1d400640 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74268f30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7a100058d0 tx=0x7f7a10012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ad6dd1215782b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a16000640 1 -- 172.21.15.97:0/517736958 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1001f040 con 0x55ac741f69d0 2024-09-10T21:30:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a16000640 1 -- 172.21.15.97:0/517736958 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a10012110 con 0x55ac741f69d0 2024-09-10T21:30:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.404+0000 7f7a16000640 1 -- 172.21.15.97:0/517736958 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10005250 con 0x55ac741f69d0 2024-09-10T21:30:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/517736958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 msgr2=0x55ac74268f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74268f30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7a100058d0 tx=0x7f7a10012820 comp rx=0 tx=0).stop 2024-09-10T21:30:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/517736958 shutdown_connections 2024-09-10T21:30:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74269470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74268f30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- 172.21.15.97:0/517736958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac742689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/517736958 >> 172.21.15.97:0/517736958 conn(0x55ac74094130 msgr2=0x55ac74208060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/517736958 shutdown_connections 2024-09-10T21:30:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/517736958 wait complete. 2024-09-10T21:30:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- start start 2024-09-10T21:30:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 Processor -- start 2024-09-10T21:30:21.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac7420d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74228580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74228ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac73ffb070 con 0x55ac741f69d0 2024-09-10T21:30:21.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ac73ffbd60 con 0x55ac741aae10 2024-09-10T21:30:21.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ac73fc3ec0 con 0x55ac741f72f0 2024-09-10T21:30:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 10 client.0 _ll_register_callbacks cb 0x7f7a200056e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:30:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74228580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74228580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46314/0 (socket says 172.21.15.97:46314) 2024-09-10T21:30:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 -- 172.21.15.97:0/2970811437 learned_addr learned my addr 172.21.15.97:0/2970811437 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a25e00640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac7420d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a26800640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74228ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 -- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 msgr2=0x55ac74228ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74228ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 -- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 msgr2=0x55ac7420d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac7420d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 -- 172.21.15.97:0/2970811437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac73fbea90 con 0x55ac741f69d0 2024-09-10T21:30:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a26800640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74228ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:21.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a1d400640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74228580 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7a2815d320 tx=0x7f7a10012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3680b9c3a04743d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a0b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a0b400640 1 -- 172.21.15.97:0/2970811437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1001f070 con 0x55ac741f69d0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a0b400640 1 -- 172.21.15.97:0/2970811437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f7a1001d630 con 0x55ac741f69d0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a10012110 con 0x55ac741f69d0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a0b400640 1 -- 172.21.15.97:0/2970811437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7a10036050 con 0x55ac741f69d0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 10 client.6804 fetch_fsmap learned FSMap version 76 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ac73f9d050 con 0x55ac741f69d0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a0b400640 1 -- 172.21.15.97:0/2970811437 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a10012110 con 0x55ac741f69d0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a0b400640 1 -- 172.21.15.97:0/2970811437 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f7a10039020 con 0x55ac741f69d0 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 10 client.6804 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 10 client.6804 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:30:21.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 20 client.6804 populate_metadata read hostname 'smithi097' 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74229000 0x55ac7426ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 4 client.6804 mds_command: new command op to 6492 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.408+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 --> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] -- command(tid 0: {"prefix": "status"}) -- 0x55ac74006520 con 0x55ac74229000 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a25e00640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74229000 0x55ac7426ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a25e00640 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74229000 0x55ac7426ca30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6343ff12fd30228f server_cookie=4ac0ea5c53b225de in_seq=0 out_seq=0 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a0b400640 10 client.6804 ms_handle_connect on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a0b400640 1 -- 172.21.15.97:0/2970811437 <== mds.0 v2:172.21.15.97:6833/2931568595 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ac74006520 con 0x55ac74229000 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a0b400640 10 client.6804 handle_command_reply: tid=0 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 client.6804 shutdown 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 msgr2=0x55ac74228580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74228580 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7a2815d320 tx=0x7f7a10012390 comp rx=0 tx=0).stop 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 shutdown_connections 2024-09-10T21:30:21.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x55ac74229000 0x55ac7426ca30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac741f72f0 0x55ac74228ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac741f69d0 0x55ac74228580 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 --2- 172.21.15.97:0/2970811437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac741aae10 0x55ac7420d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a0b400640 0 client.6804 ms_handle_reset on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 >> 172.21.15.97:0/2970811437 conn(0x55ac74094130 msgr2=0x55ac74208880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 shutdown_connections 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 1 -- 172.21.15.97:0/2970811437 wait complete. 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a3053a000 20 client.6804 trim_cache size 0 max 16384 2024-09-10T21:30:21.421 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6492, 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 4.8136927710000004, 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 76, 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 83, 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 83, 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 46.403711811000001 2024-09-10T21:30:21.422 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:30:21.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a040725e0 msgr2=0x7f7a04074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a040725e0 0x7f7a04074aa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7a180b3000 tx=0x7f7a1801dda0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 msgr2=0x7f7a281f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a281f6060 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c940 tx=0x7f7a2000ce00 comp rx=0 tx=0).stop 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 shutdown_connections 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a040725e0 0x7f7a04074aa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a281511f0 0x7f7a281f6060 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a280b3cf0 0x7f7a281f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 --2- 172.21.15.97:0/2597228886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a2806f180 0x7f7a2814b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 >> 172.21.15.97:0/2597228886 conn(0x7f7a280b1780 msgr2=0x7f7a2814ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 shutdown_connections 2024-09-10T21:30:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.412+0000 7f7a2d200640 1 -- 172.21.15.97:0/2597228886 wait complete. 2024-09-10T21:30:21.435 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-10T21:30:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout: File: a/ 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 2ch/44d Inode: 1099511627776 Links: 4 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-10 21:29:52.677264989 +0000 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-10 21:29:58.321211425 +0000 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-10 21:29:58.321211425 +0000 2024-09-10T21:30:21.486 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-10 21:29:52.677264989 +0000 2024-09-10T21:30:21.491 INFO:teuthology.orchestra.run.smithi097.stdout:a/ 2024-09-10T21:30:21.491 INFO:teuthology.orchestra.run.smithi097.stdout:a/z 2024-09-10T21:30:21.491 INFO:teuthology.orchestra.run.smithi097.stdout:a/z/d 2024-09-10T21:30:21.491 INFO:teuthology.orchestra.run.smithi097.stdout:a/c 2024-09-10T21:30:21.491 INFO:teuthology.orchestra.run.smithi097.stdout:a/c/d 2024-09-10T21:30:21.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:30:21.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- 172.21.15.97:0/3375442948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826006fd50 msgr2=0x7f82600701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 --2- 172.21.15.97:0/3375442948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826006fd50 0x7f82600701e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f825c009c80 tx=0x7f825c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- 172.21.15.97:0/3375442948 shutdown_connections 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 --2- 172.21.15.97:0/3375442948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826006fd50 0x7f82600701e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 --2- 172.21.15.97:0/3375442948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8260144690 0x7f82600b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- 172.21.15.97:0/3375442948 >> 172.21.15.97:0/3375442948 conn(0x7f82600b1780 msgr2=0x7f82600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- 172.21.15.97:0/3375442948 shutdown_connections 2024-09-10T21:30:21.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- 172.21.15.97:0/3375442948 wait complete. 2024-09-10T21:30:21.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 Processor -- start 2024-09-10T21:30:21.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- start start 2024-09-10T21:30:21.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f826006fd50 0x7f82601faf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8260143f90 0x7f826015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8260144690 0x7f8260159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8260167d50 con 0x7f8260144690 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8260167bd0 con 0x7f8260143f90 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8268800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8260167ed0 con 0x7f826006fd50 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8265e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f826006fd50 0x7f82601faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8266800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8260144690 0x7f8260159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8265e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f826006fd50 0x7f82601faf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:57346/0 (socket says 172.21.15.97:57346) 2024-09-10T21:30:21.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8265e00640 1 -- 172.21.15.97:0/4167005099 learned_addr learned my addr 172.21.15.97:0/4167005099 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8265400640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8260143f90 0x7f826015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8266800640 1 -- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f826006fd50 msgr2=0x7f82601faf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8266800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f826006fd50 0x7f82601faf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8266800640 1 -- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8260143f90 msgr2=0x7f826015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8266800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8260143f90 0x7f826015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.640+0000 7f8266800640 1 -- 172.21.15.97:0/4167005099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82601fb860 con 0x7f8260144690 2024-09-10T21:30:21.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f8265e00640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f826006fd50 0x7f82601faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:21.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f8266800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8260144690 0x7f8260159d30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ac000 tx=0x7f825c009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f825a000640 1 -- 172.21.15.97:0/4167005099 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0bb070 con 0x7f8260144690 2024-09-10T21:30:21.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f825a000640 1 -- 172.21.15.97:0/4167005099 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c004c50 con 0x7f8260144690 2024-09-10T21:30:21.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f825a000640 1 -- 172.21.15.97:0/4167005099 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0b2040 con 0x7f8260144690 2024-09-10T21:30:21.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82601fbb40 con 0x7f8260144690 2024-09-10T21:30:21.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8260138f70 con 0x7f8260144690 2024-09-10T21:30:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.644+0000 7f825a000640 1 -- 172.21.15.97:0/4167005099 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f825c0b5030 con 0x7f8260144690 2024-09-10T21:30:21.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f825a000640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f82440725e0 0x7f8244074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 Processor -- start 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f825a000640 1 -- 172.21.15.97:0/4167005099 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f825c12cd30 con 0x7f8260144690 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- start start 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8265e00640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f82440725e0 0x7f8244074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8265e00640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f82440725e0 0x7f8244074aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8254001d00 tx=0x7f8254009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bda580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bdaac0 0x564b59b90680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59b90fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564b59a78cb0 con 0x564b59b90bc0 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564b5998e310 con 0x564b59bda1a0 2024-09-10T21:30:21.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564b599ebb90 con 0x564b59bdaac0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59b90fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59b90fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46346/0 (socket says 172.21.15.97:46346) 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 -- 172.21.15.97:0/87334710 learned_addr learned my addr 172.21.15.97:0/87334710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8265e00640 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8265400640 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bdaac0 0x564b59b90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 -- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bdaac0 msgr2=0x564b59b90680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bdaac0 0x564b59b90680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 -- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 msgr2=0x564b59bda580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bda580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 -- 172.21.15.97:0/87334710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5998fc60 con 0x564b59b90bc0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f8266800640 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59b90fa0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f825c0f7840 tx=0x7f825c0f7910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46c00cc0ba375b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f824ea00640 1 -- 172.21.15.97:0/87334710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0f7650 con 0x564b59b90bc0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f824ea00640 1 -- 172.21.15.97:0/87334710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c004e20 con 0x564b59b90bc0 2024-09-10T21:30:21.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f824ea00640 1 -- 172.21.15.97:0/87334710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c0f7b40 con 0x564b59b90bc0 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- 172.21.15.97:0/87334710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 msgr2=0x564b59b90fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59b90fa0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f825c0f7840 tx=0x7f825c0f7910 comp rx=0 tx=0).stop 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- 172.21.15.97:0/87334710 shutdown_connections 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59b90fa0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bdaac0 0x564b59b90680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 --2- 172.21.15.97:0/87334710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bda580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- 172.21.15.97:0/87334710 >> 172.21.15.97:0/87334710 conn(0x564b59b96d20 msgr2=0x564b59b8ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- 172.21.15.97:0/87334710 shutdown_connections 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- 172.21.15.97:0/87334710 wait complete. 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.648+0000 7f826bcfa000 1 -- start start 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 Processor -- start 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59be1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bda1a0 0x564b59be1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bdaac0 0x564b59be1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564b599dedd0 con 0x564b59b90bc0 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564b599dafc0 con 0x564b59bdaac0 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564b599804b0 con 0x564b59bda1a0 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 10 client.0 _ll_register_callbacks cb 0x564b5995e7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59be1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59be1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46356/0 (socket says 172.21.15.97:46356) 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 -- 172.21.15.97:0/332275556 learned_addr learned my addr 172.21.15.97:0/332275556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265400640 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bda1a0 0x564b59be1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 -- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bda1a0 msgr2=0x564b59be1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bda1a0 0x564b59be1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 -- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bdaac0 msgr2=0x564b59be1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bdaac0 0x564b59be1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 -- 172.21.15.97:0/332275556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b599791f0 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265400640 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bda1a0 0x564b59be1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265e00640 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59be1530 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8254007640 tx=0x7f8254000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f087272eb021e7cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8243400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8243400640 1 -- 172.21.15.97:0/332275556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254014980 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b59973330 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8243400640 1 -- 172.21.15.97:0/332275556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f8254010040 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8243400640 1 -- 172.21.15.97:0/332275556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f825402e050 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 10 client.6816 fetch_fsmap learned FSMap version 76 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x564b5995ffa0 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8243400640 1 -- 172.21.15.97:0/332275556 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x564b59973330 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8243400640 1 -- 172.21.15.97:0/332275556 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f8254031020 con 0x564b59b90bc0 2024-09-10T21:30:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 10 client.6816 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 10 client.6816 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 20 client.6816 populate_metadata read hostname 'smithi097' 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59be9900 0x564b59bebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 4 client.6816 mds_command: new command op to 6492 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 --> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b599edae0 con 0x564b59be9900 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265400640 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59be9900 0x564b59bebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8243400640 10 client.6816 ms_handle_connect on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.652+0000 7f8265400640 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59be9900 0x564b59bebcf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1ee9887e710976 server_cookie=fcabc62e7014ea41 in_seq=0 out_seq=0 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f8243400640 1 -- 172.21.15.97:0/332275556 <== mds.0 v2:172.21.15.97:6833/2931568595 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564b599edae0 con 0x564b59be9900 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f8243400640 10 client.6816 handle_command_reply: tid=0 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 client.6816 shutdown 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 msgr2=0x564b59be1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59be1530 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8254007640 tx=0x7f8254000bd0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 shutdown_connections 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59be9900 0x564b59bebcf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bdaac0 0x564b59be1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59bda1a0 0x564b59be1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 --2- 172.21.15.97:0/332275556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59b90bc0 0x564b59be1530 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f8243400640 0 client.6816 ms_handle_reset on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 >> 172.21.15.97:0/332275556 conn(0x564b59b96d20 msgr2=0x564b59b8ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 shutdown_connections 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 1 -- 172.21.15.97:0/332275556 wait complete. 2024-09-10T21:30:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.656+0000 7f826bcfa000 20 client.6816 trim_cache size 0 max 16384 2024-09-10T21:30:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 Processor -- start 2024-09-10T21:30:21.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 -- start start 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59c4ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59c4bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c4c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82602047d0 con 0x564b59bdaac0 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82600afbc0 con 0x564b59bda1a0 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f82601398c0 con 0x564b59b90bc0 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c4c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c4c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46366/0 (socket says 172.21.15.97:46366) 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 -- 172.21.15.97:0/1559168944 learned_addr learned my addr 172.21.15.97:0/1559168944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8265e00640 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59c4ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8265400640 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59c4bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 -- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 msgr2=0x564b59c4ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59c4ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 -- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 msgr2=0x564b59c4bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59c4bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8266800640 1 -- 172.21.15.97:0/1559168944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b59a38660 con 0x564b59bdaac0 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.672+0000 7f8265400640 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59c4bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c4c4a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f825c01d640 tx=0x7f825c0f3ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf3e773adf3b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f824e000640 1 -- 172.21.15.97:0/1559168944 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c004e20 con 0x564b59bdaac0 2024-09-10T21:30:21.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f824e000640 1 -- 172.21.15.97:0/1559168944 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c0fc570 con 0x564b59bdaac0 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f824e000640 1 -- 172.21.15.97:0/1559168944 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c100f90 con 0x564b59bdaac0 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- 172.21.15.97:0/1559168944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 msgr2=0x564b59c4c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c4c4a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f825c01d640 tx=0x7f825c0f3ba0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- 172.21.15.97:0/1559168944 shutdown_connections 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c4c4a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59c4bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1559168944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59c4ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- 172.21.15.97:0/1559168944 >> 172.21.15.97:0/1559168944 conn(0x564b59b96d20 msgr2=0x564b59b92080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- 172.21.15.97:0/1559168944 shutdown_connections 2024-09-10T21:30:21.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- 172.21.15.97:0/1559168944 wait complete. 2024-09-10T21:30:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- start start 2024-09-10T21:30:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 Processor -- start 2024-09-10T21:30:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59bf0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bf0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c0c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564b599dedd0 con 0x564b59bdaac0 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564b59b99d40 con 0x564b59bda1a0 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564b59b97100 con 0x564b59b90bc0 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 10 client.0 _ll_register_callbacks cb 0x7f82540072f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c0c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c0c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46370/0 (socket says 172.21.15.97:46370) 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 -- 172.21.15.97:0/1142470753 learned_addr learned my addr 172.21.15.97:0/1142470753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8265400640 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bf0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8265e00640 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59bf0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 -- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 msgr2=0x564b59bf0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59bf0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 -- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 msgr2=0x564b59bf0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bf0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 -- 172.21.15.97:0/1142470753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b599a5090 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8266800640 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c0c210 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f825c0f7ad0 tx=0x7f825c0ab570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2d35a885881405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8243400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8243400640 1 -- 172.21.15.97:0/1142470753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c004e20 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f8243400640 1 -- 172.21.15.97:0/1142470753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f825c0c0c80 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.676+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f825c0fc570 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f8243400640 1 -- 172.21.15.97:0/1142470753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f825c0f7b40 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 10 client.6822 fetch_fsmap learned FSMap version 76 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x564b599715f0 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f8243400640 1 -- 172.21.15.97:0/1142470753 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c125080 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f8243400640 1 -- 172.21.15.97:0/1142470753 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f825c0a5af0 con 0x564b59bdaac0 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 10 client.6822 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 10 client.6822 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 20 client.6822 populate_metadata read hostname 'smithi097' 2024-09-10T21:30:21.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59c0cbd0 0x564b59c4f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 4 client.6822 mds_command: new command op to 6492 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:30:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 --> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b599e9590 con 0x564b59c0cbd0 2024-09-10T21:30:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f8265e00640 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59c0cbd0 0x564b59c4f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f8265e00640 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59c0cbd0 0x564b59c4f640 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d86de9ee4e7e18ef server_cookie=44dd5982accaf475 in_seq=0 out_seq=0 2024-09-10T21:30:21.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.680+0000 7f8243400640 10 client.6822 ms_handle_connect on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f8243400640 1 -- 172.21.15.97:0/1142470753 <== mds.0 v2:172.21.15.97:6833/2931568595 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b599e9590 con 0x564b59c0cbd0 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f8243400640 10 client.6822 handle_command_reply: tid=0 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 client.6822 shutdown 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 msgr2=0x564b59c0c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c0c210 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f825c0f7ad0 tx=0x7f825c0ab570 comp rx=0 tx=0).stop 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 shutdown_connections 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:6833/2931568595,v1:172.21.15.97:6834/2931568595] conn(0x564b59c0cbd0 0x564b59c4f640 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b59bdaac0 0x564b59c0c210 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564b59bda1a0 0x564b59bf0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 --2- 172.21.15.97:0/1142470753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564b59b90bc0 0x564b59bf0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f8243400640 0 client.6822 ms_handle_reset on v2:172.21.15.97:6833/2931568595 2024-09-10T21:30:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 >> 172.21.15.97:0/1142470753 conn(0x564b59b96d20 msgr2=0x564b59bebcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 shutdown_connections 2024-09-10T21:30:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.692+0000 7f826bcfa000 1 -- 172.21.15.97:0/1142470753 wait complete. 2024-09-10T21:30:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f826bcfa000 20 client.6822 trim_cache size 0 max 16384 2024-09-10T21:30:21.699 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:30:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "message": "", 2024-09-10T21:30:21.699 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0 2024-09-10T21:30:21.699 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:30:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f82440725e0 msgr2=0x7f8244074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f82440725e0 0x7f8244074aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8254001d00 tx=0x7f8254009040 comp rx=0 tx=0).stop 2024-09-10T21:30:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8260144690 msgr2=0x7f8260159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8260144690 0x7f8260159d30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ac000 tx=0x7f825c009330 comp rx=0 tx=0).stop 2024-09-10T21:30:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 shutdown_connections 2024-09-10T21:30:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f82440725e0 0x7f8244074aa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8260144690 0x7f8260159d30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8260143f90 0x7f826015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 --2- 172.21.15.97:0/4167005099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f826006fd50 0x7f82601faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 >> 172.21.15.97:0/4167005099 conn(0x7f82600b1780 msgr2=0x7f82600b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 shutdown_connections 2024-09-10T21:30:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:21.696+0000 7f8268800640 1 -- 172.21.15.97:0/4167005099 wait complete. 2024-09-10T21:30:21.717 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:30:21.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:30:21.808 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:30:21.888 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:30:21.888 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:30:21.889 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:21.889 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:21.898 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:21.898 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:21.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:30:22.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 -- 172.21.15.97:0/4003298448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 msgr2=0x7fd0780cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 --2- 172.21.15.97:0/4003298448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 0x7fd0780cf800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009840 tx=0x7fd06c01f010 comp rx=0 tx=0).stop 2024-09-10T21:30:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 -- 172.21.15.97:0/4003298448 shutdown_connections 2024-09-10T21:30:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 --2- 172.21.15.97:0/4003298448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd0780b1560 0x7fd0780b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 --2- 172.21.15.97:0/4003298448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 0x7fd0780cf800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 -- 172.21.15.97:0/4003298448 >> 172.21.15.97:0/4003298448 conn(0x7fd0780b37c0 msgr2=0x7fd0780b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 -- 172.21.15.97:0/4003298448 shutdown_connections 2024-09-10T21:30:22.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 -- 172.21.15.97:0/4003298448 wait complete. 2024-09-10T21:30:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 Processor -- start 2024-09-10T21:30:22.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.136+0000 7fd080800640 1 -- start start 2024-09-10T21:30:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd0780b1560 0x7fd0780b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 0x7fd0780ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0780cec10 0x7fd0781f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780ce320 con 0x7fd0780cec10 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780ce1a0 con 0x7fd0780ce830 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd0780d0450 con 0x7fd0780b1560 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0780cec10 0x7fd0781f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd0780b1560 0x7fd0780b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0780cec10 0x7fd0781f8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46380/0 (socket says 172.21.15.97:46380) 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07e800640 1 -- 172.21.15.97:0/2812852777 learned_addr learned my addr 172.21.15.97:0/2812852777 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07d400640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 0x7fd0780ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07de00640 1 -- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 msgr2=0x7fd0780ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07de00640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 0x7fd0780ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07de00640 1 -- 172.21.15.97:0/2812852777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0780cec10 msgr2=0x7fd0781f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07de00640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0780cec10 0x7fd0781f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07de00640 1 -- 172.21.15.97:0/2812852777 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0780cb2b0 con 0x7fd0780b1560 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07e800640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0780cec10 0x7fd0781f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd07de00640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd0780b1560 0x7fd0780b7470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0af000 tx=0x7fd06c0a8c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd072000640 1 -- 172.21.15.97:0/2812852777 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0bf070 con 0x7fd0780b1560 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd072000640 1 -- 172.21.15.97:0/2812852777 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd06c0052f0 con 0x7fd0780b1560 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd072000640 1 -- 172.21.15.97:0/2812852777 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0b5040 con 0x7fd0780b1560 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0780cb590 con 0x7fd0780b1560 2024-09-10T21:30:22.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0780cbb50 con 0x7fd0780b1560 2024-09-10T21:30:22.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.140+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0780b1970 con 0x7fd0780b1560 2024-09-10T21:30:22.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.144+0000 7fd072000640 1 -- 172.21.15.97:0/2812852777 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd06c0b8070 con 0x7fd0780b1560 2024-09-10T21:30:22.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.144+0000 7fd072000640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd05c0725e0 0x7fd05c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.144+0000 7fd072000640 1 -- 172.21.15.97:0/2812852777 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fd06c130090 con 0x7fd0780b1560 2024-09-10T21:30:22.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.144+0000 7fd07d400640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd05c0725e0 0x7fd05c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.144+0000 7fd07d400640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd05c0725e0 0x7fd05c074aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd078159db0 tx=0x7fd068007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:22.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.148+0000 7fd072000640 1 -- 172.21.15.97:0/2812852777 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd06c018b70 con 0x7fd0780b1560 2024-09-10T21:30:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd078071300 con 0x7fd0780b1560 2024-09-10T21:30:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd072000640 1 -- 172.21.15.97:0/2812852777 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) ==== 80+0+124 (secure 0 0 0) 0x7fd06c0faf30 con 0x7fd0780b1560 2024-09-10T21:30:22.330 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.194:6836/3597857813 2024-09-11T21:30:16.599378+0000 2024-09-10T21:30:22.330 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.194:6834/3597857813 2024-09-11T21:30:16.599378+0000 2024-09-10T21:30:22.331 INFO:teuthology.orchestra.run.smithi097.stderr:listed 2 entries 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd05c0725e0 msgr2=0x7fd05c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd05c0725e0 0x7fd05c074aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd078159db0 tx=0x7fd068007ec0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd0780b1560 msgr2=0x7fd0780b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd0780b1560 0x7fd0780b7470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0af000 tx=0x7fd06c0a8c20 comp rx=0 tx=0).stop 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 shutdown_connections 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd05c0725e0 0x7fd05c074aa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0780cec10 0x7fd0781f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0780ce830 0x7fd0780ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 --2- 172.21.15.97:0/2812852777 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd0780b1560 0x7fd0780b7470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 >> 172.21.15.97:0/2812852777 conn(0x7fd0780b37c0 msgr2=0x7fd0780b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:22.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 shutdown_connections 2024-09-10T21:30:22.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.324+0000 7fd080800640 1 -- 172.21.15.97:0/2812852777 wait complete. 2024-09-10T21:30:22.347 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:30:22.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:22.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:30:22.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:30:22.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:22.373 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:30:22.422 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:30:22.423 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:30:22.423 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:22.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:30:22.433 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:30:22.433 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:22.433 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:22.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:30:22.474 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:30:22.474 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:30:22.474 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:22.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:30:22.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 -- 172.21.15.97:0/1684368657 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00b0d50 msgr2=0x7fb0f00aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/1684368657 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00b0d50 0x7fb0f00aedc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 -- 172.21.15.97:0/1684368657 shutdown_connections 2024-09-10T21:30:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/1684368657 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00b0d50 0x7fb0f00aedc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/1684368657 >> v2:172.21.15.194:3300/0 conn(0x7fb0f00ce640 0x7fb0f00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 -- 172.21.15.97:0/1684368657 >> 172.21.15.97:0/1684368657 conn(0x7fb0f00d1c30 msgr2=0x7fb0f00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 -- 172.21.15.97:0/1684368657 shutdown_connections 2024-09-10T21:30:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 -- 172.21.15.97:0/1684368657 wait complete. 2024-09-10T21:30:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.656+0000 7fb0f5a00640 1 Processor -- start 2024-09-10T21:30:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 -- start start 2024-09-10T21:30:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00af300 0x7fb0f0165290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00b0d50 0x7fb0f00cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0f00ce640 0x7fb0f0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0169270 con 0x7fb0f00b0d50 2024-09-10T21:30:22.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f01690f0 con 0x7fb0f00ce640 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f01693f0 con 0x7fb0f00af300 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00b0d50 0x7fb0f00cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00b0d50 0x7fb0f00cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46382/0 (socket says 172.21.15.97:46382) 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 -- 172.21.15.97:0/2698291237 learned_addr learned my addr 172.21.15.97:0/2698291237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0eea00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00af300 0x7fb0f0165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ef400640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0f00ce640 0x7fb0f0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 -- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00af300 msgr2=0x7fb0f0165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00af300 0x7fb0f0165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 -- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0f00ce640 msgr2=0x7fb0f0158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0f00ce640 0x7fb0f0158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 -- 172.21.15.97:0/2698291237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f01f8080 con 0x7fb0f00b0d50 2024-09-10T21:30:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0ee000640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00b0d50 0x7fb0f00cadd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009850 tx=0x7fb0dc0a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2698291237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0b2040 con 0x7fb0f00b0d50 2024-09-10T21:30:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2698291237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0dc005030 con 0x7fb0f00b0d50 2024-09-10T21:30:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2698291237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0044f0 con 0x7fb0f00b0d50 2024-09-10T21:30:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f01f8360 con 0x7fb0f00b0d50 2024-09-10T21:30:22.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f01f8920 con 0x7fb0f00b0d50 2024-09-10T21:30:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.660+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f00aedc0 con 0x7fb0f00b0d50 2024-09-10T21:30:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.664+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2698291237 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb0dc0bf060 con 0x7fb0f00b0d50 2024-09-10T21:30:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.664+0000 7fb0e6a00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb0d0072610 0x7fb0d0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:22.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.664+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2698291237 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fb0dc12c3e0 con 0x7fb0f00b0d50 2024-09-10T21:30:22.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.664+0000 7fb0eea00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb0d0072610 0x7fb0d0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:22.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.664+0000 7fb0eea00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb0d0072610 0x7fb0d0074ad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009980 tx=0x7fb0e0009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:22.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.668+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2698291237 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb0dc0164d0 con 0x7fb0f00b0d50 2024-09-10T21:30:22.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.868+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0f006ddf0 con 0x7fb0f00b0d50 2024-09-10T21:30:22.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2698291237 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5233 (secure 0 0 0) 0x7fb0dc0f7170 con 0x7fb0f00b0d50 2024-09-10T21:30:22.874 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:22.874 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":76,"btime":"2024-09-10T21:30:19:606108+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/2912272065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2912272065},{"type":"v1","addr":"172.21.15.194:6836","nonce":2912272065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:19.606099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":14,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6492,"qdb_cluster":[6492]},"id":5}]} 2024-09-10T21:30:22.875 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 76 2024-09-10T21:30:22.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb0d0072610 msgr2=0x7fb0d0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb0d0072610 0x7fb0d0074ad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009980 tx=0x7fb0e0009500 comp rx=0 tx=0).stop 2024-09-10T21:30:22.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00b0d50 msgr2=0x7fb0f00cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:22.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00b0d50 0x7fb0f00cadd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009850 tx=0x7fb0dc0a57c0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 shutdown_connections 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb0d0072610 0x7fb0d0074ad0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0f00ce640 0x7fb0f0158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00b0d50 0x7fb0f00cadd0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 --2- 172.21.15.97:0/2698291237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb0f00af300 0x7fb0f0165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 >> 172.21.15.97:0/2698291237 conn(0x7fb0f00d1c30 msgr2=0x7fb0f00d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 shutdown_connections 2024-09-10T21:30:22.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:22.872+0000 7fb0f5a00640 1 -- 172.21.15.97:0/2698291237 wait complete. 2024-09-10T21:30:22.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- 172.21.15.97:0/3141930910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0b1560 msgr2=0x7f1acc0b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/3141930910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0b1560 0x7f1acc0b19b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009c80 tx=0x7f1ac401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- 172.21.15.97:0/3141930910 shutdown_connections 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/3141930910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0b1560 0x7f1acc0b19b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/3141930910 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1acc0cec10 0x7f1acc0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- 172.21.15.97:0/3141930910 >> 172.21.15.97:0/3141930910 conn(0x7f1acc0b37c0 msgr2=0x7f1acc0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- 172.21.15.97:0/3141930910 shutdown_connections 2024-09-10T21:30:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- 172.21.15.97:0/3141930910 wait complete. 2024-09-10T21:30:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 Processor -- start 2024-09-10T21:30:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- start start 2024-09-10T21:30:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1acc0b1560 0x7f1acc15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1acc0ce830 0x7f1acc0b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0cec10 0x7f1acc0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc0ce1a0 con 0x7f1acc0cec10 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc0cde60 con 0x7f1acc0ce830 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ad0e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1acc0ce320 con 0x7f1acc0b1560 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0cec10 0x7f1acc0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0cec10 0x7f1acc0afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46406/0 (socket says 172.21.15.97:46406) 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 -- 172.21.15.97:0/640525607 learned_addr learned my addr 172.21.15.97:0/640525607 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ac9e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1acc0b1560 0x7f1acc15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1ac9400640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1acc0ce830 0x7f1acc0b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 -- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1acc0b1560 msgr2=0x7f1acc15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1acc0b1560 0x7f1acc15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 -- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1acc0ce830 msgr2=0x7f1acc0b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1acc0ce830 0x7f1acc0b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.036+0000 7f1aca800640 1 -- 172.21.15.97:0/640525607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc1e7150 con 0x7f1acc0cec10 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1aca800640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0cec10 0x7f1acc0afaf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009bd0 tx=0x7f1ac40af410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ac2000640 1 -- 172.21.15.97:0/640525607 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4004e60 con 0x7f1acc0cec10 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ac2000640 1 -- 172.21.15.97:0/640525607 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ac40be030 con 0x7f1acc0cec10 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc1e76c0 con 0x7f1acc0cec10 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ac2000640 1 -- 172.21.15.97:0/640525607 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40afb00 con 0x7f1acc0cec10 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1e7c80 con 0x7f1acc0cec10 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc0cf800 con 0x7f1acc0cec10 2024-09-10T21:30:23.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ac2000640 1 -- 172.21.15.97:0/640525607 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ac40cb060 con 0x7f1acc0cec10 2024-09-10T21:30:23.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ac2000640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1aac0725e0 0x7f1aac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.040+0000 7f1ac2000640 1 -- 172.21.15.97:0/640525607 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f1ac41319e0 con 0x7f1acc0cec10 2024-09-10T21:30:23.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.044+0000 7f1ac9e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1aac0725e0 0x7f1aac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.044+0000 7f1ac9e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1aac0725e0 0x7f1aac074aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1abc007fd0 tx=0x7f1abc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:23.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.044+0000 7f1ac2000640 1 -- 172.21.15.97:0/640525607 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ac40ac050 con 0x7f1acc0cec10 2024-09-10T21:30:23.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T21:30:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.240+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1acc0b52b0 con 0x7f1acc0cec10 2024-09-10T21:30:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ac2000640 1 -- 172.21.15.97:0/640525607 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5233 (secure 0 0 0) 0x7f1ac4019100 con 0x7f1acc0cec10 2024-09-10T21:30:23.246 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:23.246 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":76,"btime":"2024-09-10T21:30:19:606108+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/2912272065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2912272065},{"type":"v1","addr":"172.21.15.194:6836","nonce":2912272065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:19.606099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":14,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6492,"qdb_cluster":[6492]},"id":5}]} 2024-09-10T21:30:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 76 2024-09-10T21:30:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1aac0725e0 msgr2=0x7f1aac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1aac0725e0 0x7f1aac074aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1abc007fd0 tx=0x7f1abc009040 comp rx=0 tx=0).stop 2024-09-10T21:30:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0cec10 msgr2=0x7f1acc0afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0cec10 0x7f1acc0afaf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009bd0 tx=0x7f1ac40af410 comp rx=0 tx=0).stop 2024-09-10T21:30:23.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 shutdown_connections 2024-09-10T21:30:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1aac0725e0 0x7f1aac074aa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1acc0cec10 0x7f1acc0afaf0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1acc0ce830 0x7f1acc0b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 --2- 172.21.15.97:0/640525607 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1acc0b1560 0x7f1acc15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 >> 172.21.15.97:0/640525607 conn(0x7f1acc0b37c0 msgr2=0x7f1acc071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 shutdown_connections 2024-09-10T21:30:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.244+0000 7f1ad0e00640 1 -- 172.21.15.97:0/640525607 wait complete. 2024-09-10T21:30:23.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:30:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 -- 172.21.15.97:0/1898797733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 msgr2=0x7fc6640cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 --2- 172.21.15.97:0/1898797733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 0x7fc6640cff50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc650009840 tx=0x7fc65001c6b0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 -- 172.21.15.97:0/1898797733 shutdown_connections 2024-09-10T21:30:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 --2- 172.21.15.97:0/1898797733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 0x7fc6640cff50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 -- 172.21.15.97:0/1898797733 >> 172.21.15.97:0/1898797733 conn(0x7fc6640d1a60 msgr2=0x7fc6640d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 -- 172.21.15.97:0/1898797733 shutdown_connections 2024-09-10T21:30:23.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 -- 172.21.15.97:0/1898797733 wait complete. 2024-09-10T21:30:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 Processor -- start 2024-09-10T21:30:23.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.404+0000 7fc669200640 1 -- start start 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 0x7fc66406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6640d0490 0x7fc66414d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc66415efc0 0x7fc6640b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc664169b50 con 0x7fc6640d0490 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641699d0 con 0x7fc6640cdcb0 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc664169cd0 con 0x7fc66415efc0 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6640d0490 0x7fc66414d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6640d0490 0x7fc66414d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46414/0 (socket says 172.21.15.97:46414) 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 -- 172.21.15.97:0/1179752849 learned_addr learned my addr 172.21.15.97:0/1179752849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc663400640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 0x7fc66406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc663e00640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc66415efc0 0x7fc6640b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 -- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc66415efc0 msgr2=0x7fc6640b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc66415efc0 0x7fc6640b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 -- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 msgr2=0x7fc66406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 0x7fc66406ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 -- 172.21.15.97:0/1179752849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66406f030 con 0x7fc6640d0490 2024-09-10T21:30:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc662a00640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6640d0490 0x7fc66414d610 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc650009790 tx=0x7fc650005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc65b400640 1 -- 172.21.15.97:0/1179752849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6500047d0 con 0x7fc6640d0490 2024-09-10T21:30:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc65b400640 1 -- 172.21.15.97:0/1179752849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc650004970 con 0x7fc6640d0490 2024-09-10T21:30:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66406f310 con 0x7fc6640d0490 2024-09-10T21:30:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc65b400640 1 -- 172.21.15.97:0/1179752849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650004d60 con 0x7fc6640d0490 2024-09-10T21:30:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6640c6500 con 0x7fc6640d0490 2024-09-10T21:30:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.408+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6640ae930 con 0x7fc6640d0490 2024-09-10T21:30:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.412+0000 7fc65b400640 1 -- 172.21.15.97:0/1179752849 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc6500a8450 con 0x7fc6640d0490 2024-09-10T21:30:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.412+0000 7fc65b400640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc644072640 0x7fc644074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.412+0000 7fc65b400640 1 -- 172.21.15.97:0/1179752849 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fc65012d590 con 0x7fc6640d0490 2024-09-10T21:30:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.412+0000 7fc663400640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc644072640 0x7fc644074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.412+0000 7fc663400640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc644072640 0x7fc644074b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc654001ec0 tx=0x7fc6540019e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.416+0000 7fc65b400640 1 -- 172.21.15.97:0/1179752849 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6500c2030 con 0x7fc6640d0490 2024-09-10T21:30:23.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.580+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc66414db50 con 0x7fc6640d0490 2024-09-10T21:30:23.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc65b400640 1 -- 172.21.15.97:0/1179752849 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+17100 (secure 0 0 0) 0x7fc6500ac030 con 0x7fc6640d0490 2024-09-10T21:30:23.587 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:23.587 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":83,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:30:16.599447+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:29:36.590975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T21:29:37.883346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6836/3597857813":"2024-09-11T21:30:16.599378+0000","172.21.15.194:6834/3597857813":"2024-09-11T21:30:16.599378+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:30:23.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc644072640 msgr2=0x7fc644074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc644072640 0x7fc644074b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc654001ec0 tx=0x7fc6540019e0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6640d0490 msgr2=0x7fc66414d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6640d0490 0x7fc66414d610 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc650009790 tx=0x7fc650005dc0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 shutdown_connections 2024-09-10T21:30:23.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc644072640 0x7fc644074b00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc66415efc0 0x7fc6640b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6640d0490 0x7fc66414d610 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 --2- 172.21.15.97:0/1179752849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6640cdcb0 0x7fc66406ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 >> 172.21.15.97:0/1179752849 conn(0x7fc6640d1a60 msgr2=0x7fc664156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:23.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 shutdown_connections 2024-09-10T21:30:23.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.584+0000 7fc669200640 1 -- 172.21.15.97:0/1179752849 wait complete. 2024-09-10T21:30:23.608 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:30:23.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- 172.21.15.97:0/2044830860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c7015f350 msgr2=0x7f8c700b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 --2- 172.21.15.97:0/2044830860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c7015f350 0x7f8c700b3070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009840 tx=0x7f8c6001c790 comp rx=0 tx=0).stop 2024-09-10T21:30:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- 172.21.15.97:0/2044830860 shutdown_connections 2024-09-10T21:30:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 --2- 172.21.15.97:0/2044830860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700d05e0 0x7f8c700cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 --2- 172.21.15.97:0/2044830860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c7015f350 0x7f8c700b3070 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- 172.21.15.97:0/2044830860 >> 172.21.15.97:0/2044830860 conn(0x7f8c700d1c60 msgr2=0x7f8c700d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- 172.21.15.97:0/2044830860 shutdown_connections 2024-09-10T21:30:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- 172.21.15.97:0/2044830860 wait complete. 2024-09-10T21:30:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 Processor -- start 2024-09-10T21:30:23.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- start start 2024-09-10T21:30:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700cff80 0x7f8c700cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c700d05e0 0x7f8c701f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c7015f350 0x7f8c7015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70169180 con 0x7f8c7015f350 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70169000 con 0x7f8c700d05e0 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c77000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8c70169300 con 0x7f8c700cff80 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c75000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c7015f350 0x7f8c7015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c6fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700cff80 0x7f8c700cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c6fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700cff80 0x7f8c700cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:57442/0 (socket says 172.21.15.97:57442) 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c6fe00640 1 -- 172.21.15.97:0/2681334476 learned_addr learned my addr 172.21.15.97:0/2681334476 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c75000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c7015f350 0x7f8c7015e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46418/0 (socket says 172.21.15.97:46418) 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.748+0000 7f8c67400640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c700d05e0 0x7f8c701f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6fe00640 1 -- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c700d05e0 msgr2=0x7f8c701f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6fe00640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c700d05e0 0x7f8c701f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6fe00640 1 -- 172.21.15.97:0/2681334476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c7015f350 msgr2=0x7f8c7015e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6fe00640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c7015f350 0x7f8c7015e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6fe00640 1 -- 172.21.15.97:0/2681334476 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c701f9010 con 0x7f8c700cff80 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6fe00640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700cff80 0x7f8c700cba80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009790 tx=0x7f8c600a6460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6cc00640 1 -- 172.21.15.97:0/2681334476 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600ac030 con 0x7f8c700cff80 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6cc00640 1 -- 172.21.15.97:0/2681334476 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c600a6b80 con 0x7f8c700cff80 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6cc00640 1 -- 172.21.15.97:0/2681334476 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c600046d0 con 0x7f8c700cff80 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c700b5ff0 con 0x7f8c700cff80 2024-09-10T21:30:23.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c700b51c0 con 0x7f8c700cff80 2024-09-10T21:30:23.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c700cfa40 con 0x7f8c700cff80 2024-09-10T21:30:23.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.752+0000 7f8c6cc00640 1 -- 172.21.15.97:0/2681334476 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8c60004870 con 0x7f8c700cff80 2024-09-10T21:30:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.756+0000 7f8c6cc00640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c4c0725e0 0x7f8c4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:23.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.756+0000 7f8c6cc00640 1 -- 172.21.15.97:0/2681334476 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f8c6012dc00 con 0x7f8c700cff80 2024-09-10T21:30:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.756+0000 7f8c67400640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c4c0725e0 0x7f8c4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.756+0000 7f8c67400640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c4c0725e0 0x7f8c4c074aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009a40 tx=0x7f8c58010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.756+0000 7f8c6cc00640 1 -- 172.21.15.97:0/2681334476 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8c60016410 con 0x7f8c700cff80 2024-09-10T21:30:23.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.952+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c700b13a0 con 0x7f8c700cff80 2024-09-10T21:30:23.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c6cc00640 1 -- 172.21.15.97:0/2681334476 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5233 (secure 0 0 0) 0x7f8c600f8b20 con 0x7f8c700cff80 2024-09-10T21:30:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:23.958 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":76,"btime":"2024-09-10T21:30:19:606108+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/2912272065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2912272065},{"type":"v1","addr":"172.21.15.194:6836","nonce":2912272065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:19.606099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":14,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6492,"qdb_cluster":[6492]},"id":5}]} 2024-09-10T21:30:23.959 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 76 2024-09-10T21:30:23.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c4c0725e0 msgr2=0x7f8c4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c4c0725e0 0x7f8c4c074aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009a40 tx=0x7f8c58010040 comp rx=0 tx=0).stop 2024-09-10T21:30:23.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700cff80 msgr2=0x7f8c700cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700cff80 0x7f8c700cba80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009790 tx=0x7f8c600a6460 comp rx=0 tx=0).stop 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 shutdown_connections 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8c4c0725e0 0x7f8c4c074aa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c7015f350 0x7f8c7015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c700d05e0 0x7f8c701f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 --2- 172.21.15.97:0/2681334476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8c700cff80 0x7f8c700cba80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 >> 172.21.15.97:0/2681334476 conn(0x7f8c700d1c60 msgr2=0x7f8c7006e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 shutdown_connections 2024-09-10T21:30:23.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:23.956+0000 7f8c77000640 1 -- 172.21.15.97:0/2681334476 wait complete. 2024-09-10T21:30:23.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 -- 172.21.15.97:0/3176055759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b900708e0 msgr2=0x7f7b90070d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 --2- 172.21.15.97:0/3176055759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b900708e0 0x7f7b90070d90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7b88009900 tx=0x7f7b8801c7f0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 -- 172.21.15.97:0/3176055759 shutdown_connections 2024-09-10T21:30:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 --2- 172.21.15.97:0/3176055759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b900708e0 0x7f7b90070d90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 --2- 172.21.15.97:0/3176055759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b9015f020 0x7f7b9015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 -- 172.21.15.97:0/3176055759 >> 172.21.15.97:0/3176055759 conn(0x7f7b900d1b30 msgr2=0x7f7b900d1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 -- 172.21.15.97:0/3176055759 shutdown_connections 2024-09-10T21:30:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.116+0000 7f7b95e00640 1 -- 172.21.15.97:0/3176055759 wait complete. 2024-09-10T21:30:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 Processor -- start 2024-09-10T21:30:24.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 -- start start 2024-09-10T21:30:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b900708e0 0x7f7b900ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b90071cb0 0x7f7b90153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9015f020 0x7f7b90158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90168f00 con 0x7f7b9015f020 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90168d80 con 0x7f7b900708e0 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7b90169080 con 0x7f7b90071cb0 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9015f020 0x7f7b90158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9015f020 0x7f7b90158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46434/0 (socket says 172.21.15.97:46434) 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 -- 172.21.15.97:0/1764737291 learned_addr learned my addr 172.21.15.97:0/1764737291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b94a00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b900708e0 0x7f7b900ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b8fe00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b90071cb0 0x7f7b90153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 -- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b90071cb0 msgr2=0x7f7b90153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b90071cb0 0x7f7b90153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 -- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b900708e0 msgr2=0x7f7b900ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b900708e0 0x7f7b900ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 -- 172.21.15.97:0/1764737291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b900b78e0 con 0x7f7b9015f020 2024-09-10T21:30:24.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95400640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9015f020 0x7f7b90158fe0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7b8801af00 tx=0x7f7b88005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b8cc00640 1 -- 172.21.15.97:0/1764737291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b880be070 con 0x7f7b9015f020 2024-09-10T21:30:24.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b8cc00640 1 -- 172.21.15.97:0/1764737291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b880a8590 con 0x7f7b9015f020 2024-09-10T21:30:24.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b8cc00640 1 -- 172.21.15.97:0/1764737291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b880042a0 con 0x7f7b9015f020 2024-09-10T21:30:24.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b900aef70 con 0x7f7b9015f020 2024-09-10T21:30:24.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.120+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b900af530 con 0x7f7b9015f020 2024-09-10T21:30:24.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.124+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b90070d90 con 0x7f7b9015f020 2024-09-10T21:30:24.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.124+0000 7f7b8cc00640 1 -- 172.21.15.97:0/1764737291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7b880a7070 con 0x7f7b9015f020 2024-09-10T21:30:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.124+0000 7f7b8cc00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7b700725e0 0x7f7b70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.124+0000 7f7b8cc00640 1 -- 172.21.15.97:0/1764737291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f7b8812cff0 con 0x7f7b9015f020 2024-09-10T21:30:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.124+0000 7f7b94a00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7b700725e0 0x7f7b70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.124+0000 7f7b94a00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7b700725e0 0x7f7b70074aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009ec0 tx=0x7f7b80009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.128+0000 7f7b8cc00640 1 -- 172.21.15.97:0/1764737291 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7b880ac030 con 0x7f7b9015f020 2024-09-10T21:30:24.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.328+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b900b7ab0 con 0x7f7b9015f020 2024-09-10T21:30:24.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.328+0000 7f7b8cc00640 1 -- 172.21.15.97:0/1764737291 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5233 (secure 0 0 0) 0x7f7b880164d0 con 0x7f7b9015f020 2024-09-10T21:30:24.333 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:24.333 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":76,"btime":"2024-09-10T21:30:19:606108+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/2912272065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2912272065},{"type":"v1","addr":"172.21.15.194:6836","nonce":2912272065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:29:38.920694+0000","modified":"2024-09-10T21:30:19.606099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6492":{"gid":6492,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":14,"addr":"172.21.15.97:6834/2931568595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2931568595},{"type":"v1","addr":"172.21.15.97:6834","nonce":2931568595}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6492,"qdb_cluster":[6492]},"id":5}]} 2024-09-10T21:30:24.333 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 76 2024-09-10T21:30:24.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7b700725e0 msgr2=0x7f7b70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7b700725e0 0x7f7b70074aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009ec0 tx=0x7f7b80009500 comp rx=0 tx=0).stop 2024-09-10T21:30:24.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9015f020 msgr2=0x7f7b90158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9015f020 0x7f7b90158fe0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7b8801af00 tx=0x7f7b88005760 comp rx=0 tx=0).stop 2024-09-10T21:30:24.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 shutdown_connections 2024-09-10T21:30:24.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7b700725e0 0x7f7b70074aa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9015f020 0x7f7b90158fe0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7b90071cb0 0x7f7b90153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 --2- 172.21.15.97:0/1764737291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b900708e0 0x7f7b900ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 >> 172.21.15.97:0/1764737291 conn(0x7f7b900d1b30 msgr2=0x7f7b9015f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:24.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 shutdown_connections 2024-09-10T21:30:24.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.332+0000 7f7b95e00640 1 -- 172.21.15.97:0/1764737291 wait complete. 2024-09-10T21:30:24.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 -- 172.21.15.97:0/179870192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000cf5f0 msgr2=0x7fc00015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 --2- 172.21.15.97:0/179870192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000cf5f0 0x7fc00015a730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009c80 tx=0x7fbff801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 -- 172.21.15.97:0/179870192 shutdown_connections 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 --2- 172.21.15.97:0/179870192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000cf5f0 0x7fc00015a730 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 --2- 172.21.15.97:0/179870192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0001511f0 0x7fc0000b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 -- 172.21.15.97:0/179870192 >> 172.21.15.97:0/179870192 conn(0x7fc0000b1780 msgr2=0x7fc0000b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 -- 172.21.15.97:0/179870192 shutdown_connections 2024-09-10T21:30:24.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.492+0000 7fc006a00640 1 -- 172.21.15.97:0/179870192 wait complete. 2024-09-10T21:30:24.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 Processor -- start 2024-09-10T21:30:24.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 -- start start 2024-09-10T21:30:24.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000b3cf0 0x7fc0001593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0000cf5f0 0x7fc0000cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0001511f0 0x7fc0001450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc00006e530 con 0x7fc0001511f0 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc00006e3b0 con 0x7fc0000b3cf0 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc00006e6b0 con 0x7fc0000cf5f0 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0001511f0 0x7fc0001450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0001511f0 0x7fc0001450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46442/0 (socket says 172.21.15.97:46442) 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 -- 172.21.15.97:0/1768214986 learned_addr learned my addr 172.21.15.97:0/1768214986 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fbfffe00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000b3cf0 0x7fc0001593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fbfff400640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0000cf5f0 0x7fc0000cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 -- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0000cf5f0 msgr2=0x7fc0000cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0000cf5f0 0x7fc0000cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 -- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000b3cf0 msgr2=0x7fc0001593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000b3cf0 0x7fc0001593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 -- 172.21.15.97:0/1768214986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0001f8150 con 0x7fc0001511f0 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fbfffe00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000b3cf0 0x7fc0001593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc004a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0001511f0 0x7fc0001450d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbff801dc50 tx=0x7fbff8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fbff7e00640 1 -- 172.21.15.97:0/1768214986 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8004e60 con 0x7fc0001511f0 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fbff7e00640 1 -- 172.21.15.97:0/1768214986 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbff80af8a0 con 0x7fc0001511f0 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0001f8430 con 0x7fc0001511f0 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fbff7e00640 1 -- 172.21.15.97:0/1768214986 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80afcd0 con 0x7fc0001511f0 2024-09-10T21:30:24.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.496+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001f89f0 con 0x7fc0001511f0 2024-09-10T21:30:24.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.500+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0001f8ee0 con 0x7fc0001511f0 2024-09-10T21:30:24.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.500+0000 7fbff7e00640 1 -- 172.21.15.97:0/1768214986 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbff8004000 con 0x7fc0001511f0 2024-09-10T21:30:24.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.500+0000 7fbff7e00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbfe00725e0 0x7fbfe0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.500+0000 7fbff7e00640 1 -- 172.21.15.97:0/1768214986 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7fbff8131cd0 con 0x7fc0001511f0 2024-09-10T21:30:24.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.500+0000 7fbfffe00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbfe00725e0 0x7fbfe0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.500+0000 7fbfffe00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbfe00725e0 0x7fbfe0074aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009f30 tx=0x7fbff00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.504+0000 7fbff7e00640 1 -- 172.21.15.97:0/1768214986 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbff80ac050 con 0x7fc0001511f0 2024-09-10T21:30:24.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.680+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0000cbd70 con 0x7fc0001511f0 2024-09-10T21:30:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.680+0000 7fbff7e00640 1 -- 172.21.15.97:0/1768214986 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+17100 (secure 0 0 0) 0x7fbff8019100 con 0x7fc0001511f0 2024-09-10T21:30:24.686 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:24.686 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":83,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:30:16.599447+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:29:36.590975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T21:29:37.883346+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6836/3597857813":"2024-09-11T21:30:16.599378+0000","172.21.15.194:6834/3597857813":"2024-09-11T21:30:16.599378+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbfe00725e0 msgr2=0x7fbfe0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbfe00725e0 0x7fbfe0074aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009f30 tx=0x7fbff00091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0001511f0 msgr2=0x7fc0001450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0001511f0 0x7fc0001450d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbff801dc50 tx=0x7fbff8009390 comp rx=0 tx=0).stop 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 shutdown_connections 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbfe00725e0 0x7fbfe0074aa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0001511f0 0x7fc0001450d0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0000cf5f0 0x7fc0000cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 --2- 172.21.15.97:0/1768214986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0000b3cf0 0x7fc0001593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 >> 172.21.15.97:0/1768214986 conn(0x7fc0000b1780 msgr2=0x7fc0001474a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 shutdown_connections 2024-09-10T21:30:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.684+0000 7fc006a00640 1 -- 172.21.15.97:0/1768214986 wait complete. 2024-09-10T21:30:24.707 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:30:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 -- 172.21.15.97:0/1802064789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d406fd50 msgr2=0x7f23d40701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 --2- 172.21.15.97:0/1802064789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d406fd50 0x7f23d40701a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f23d415e650 tx=0x7f23cc01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 -- 172.21.15.97:0/1802064789 shutdown_connections 2024-09-10T21:30:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 --2- 172.21.15.97:0/1802064789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d406fd50 0x7f23d40701a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 --2- 172.21.15.97:0/1802064789 >> v2:172.21.15.194:3300/0 conn(0x7f23d4144690 0x7f23d40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 -- 172.21.15.97:0/1802064789 >> 172.21.15.97:0/1802064789 conn(0x7f23d40b1780 msgr2=0x7f23d40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 -- 172.21.15.97:0/1802064789 shutdown_connections 2024-09-10T21:30:24.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.844+0000 7f23d9400640 1 -- 172.21.15.97:0/1802064789 wait complete. 2024-09-10T21:30:24.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 Processor -- start 2024-09-10T21:30:24.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 -- start start 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23d406fd50 0x7f23d413d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d4143f90 0x7f23d41f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23d4144690 0x7f23d4136940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4167c40 con 0x7f23d4143f90 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4167ac0 con 0x7f23d4144690 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f23d4167dc0 con 0x7f23d406fd50 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d4143f90 0x7f23d41f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d4143f90 0x7f23d41f9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46462/0 (socket says 172.21.15.97:46462) 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 -- 172.21.15.97:0/2522412259 learned_addr learned my addr 172.21.15.97:0/2522412259 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d3200640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23d4144690 0x7f23d4136940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d2800640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23d406fd50 0x7f23d413d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 -- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23d406fd50 msgr2=0x7f23d413d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23d406fd50 0x7f23d413d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 -- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23d4144690 msgr2=0x7f23d4136940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23d4144690 0x7f23d4136940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 -- 172.21.15.97:0/2522412259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d413d970 con 0x7f23d4143f90 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d1e00640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d4143f90 0x7f23d41f9a40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f23c000b440 tx=0x7f23c000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23caa00640 1 -- 172.21.15.97:0/2522412259 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0015020 con 0x7f23d4143f90 2024-09-10T21:30:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23caa00640 1 -- 172.21.15.97:0/2522412259 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23c00119a0 con 0x7f23d4143f90 2024-09-10T21:30:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d413dc50 con 0x7f23d4143f90 2024-09-10T21:30:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23caa00640 1 -- 172.21.15.97:0/2522412259 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0011df0 con 0x7f23d4143f90 2024-09-10T21:30:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.848+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d413d660 con 0x7f23d4143f90 2024-09-10T21:30:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.852+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d40701a0 con 0x7f23d4143f90 2024-09-10T21:30:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.852+0000 7f23caa00640 1 -- 172.21.15.97:0/2522412259 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f23c001a400 con 0x7f23d4143f90 2024-09-10T21:30:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.852+0000 7f23caa00640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23b40725e0 0x7f23b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.852+0000 7f23caa00640 1 -- 172.21.15.97:0/2522412259 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6187+0+0 (secure 0 0 0) 0x7f23c00938e0 con 0x7f23d4143f90 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.852+0000 7f23d2800640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23b40725e0 0x7f23b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.852+0000 7f23d2800640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23b40725e0 0x7f23b4074aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f23c4001dc0 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:24.856+0000 7f23caa00640 1 -- 172.21.15.97:0/2522412259 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f23c001f020 con 0x7f23d4143f90 2024-09-10T21:30:25.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.068+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f23d4143470 con 0x7f23d4143f90 2024-09-10T21:30:25.071 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:30:25.068+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:30:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.068+0000 7f23caa00640 1 -- 172.21.15.97:0/2522412259 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7f23c000d040 con 0x7f23d4143f90 2024-09-10T21:30:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:30:25.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23b40725e0 msgr2=0x7f23b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23b40725e0 0x7f23b4074aa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f23c4001dc0 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d4143f90 msgr2=0x7f23d41f9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d4143f90 0x7f23d41f9a40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f23c000b440 tx=0x7f23c000b900 comp rx=0 tx=0).stop 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 shutdown_connections 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f23b40725e0 0x7f23b4074aa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23d4144690 0x7f23d4136940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d4143f90 0x7f23d41f9a40 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 --2- 172.21.15.97:0/2522412259 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23d406fd50 0x7f23d413d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 >> 172.21.15.97:0/2522412259 conn(0x7f23d40b1780 msgr2=0x7f23d40b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 shutdown_connections 2024-09-10T21:30:25.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.072+0000 7f23d9400640 1 -- 172.21.15.97:0/2522412259 wait complete. 2024-09-10T21:30:25.084 INFO:tasks.ceph.mds.c.smithi097.stderr: -1438> 2024-09-10T21:30:20.489+0000 7f03fea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:30:25.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:30:25.129 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:30:25.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 -- 172.21.15.97:0/985120787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c15fc10 msgr2=0x7fde6c13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 --2- 172.21.15.97:0/985120787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c15fc10 0x7fde6c13f9b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fde6c074800 tx=0x7fde6401cd60 comp rx=0 tx=0).stop 2024-09-10T21:30:25.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 -- 172.21.15.97:0/985120787 shutdown_connections 2024-09-10T21:30:25.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 --2- 172.21.15.97:0/985120787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c15fc10 0x7fde6c13f9b0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 --2- 172.21.15.97:0/985120787 >> v2:172.21.15.194:3300/0 conn(0x7fde6c141d20 0x7fde6c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 -- 172.21.15.97:0/985120787 >> 172.21.15.97:0/985120787 conn(0x7fde6c0b1780 msgr2=0x7fde6c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 -- 172.21.15.97:0/985120787 shutdown_connections 2024-09-10T21:30:25.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 -- 172.21.15.97:0/985120787 wait complete. 2024-09-10T21:30:25.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.240+0000 7fde72600640 1 Processor -- start 2024-09-10T21:30:25.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 -- start start 2024-09-10T21:30:25.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0ce5f0 0x7fde6c13c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c141d20 0x7fde6c139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde6c15fc10 0x7fde6c13a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1689e0 con 0x7fde6c141d20 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c168860 con 0x7fde6c0ce5f0 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fde6c168b60 con 0x7fde6c15fc10 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c141d20 0x7fde6c139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c141d20 0x7fde6c139e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46514/0 (socket says 172.21.15.97:46514) 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 -- 172.21.15.97:0/1898661362 learned_addr learned my addr 172.21.15.97:0/1898661362 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde6b400640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0ce5f0 0x7fde6c13c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde6be00640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde6c15fc10 0x7fde6c13a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 -- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde6c15fc10 msgr2=0x7fde6c13a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde6c15fc10 0x7fde6c13a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 -- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0ce5f0 msgr2=0x7fde6c13c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0ce5f0 0x7fde6c13c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 -- 172.21.15.97:0/1898661362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c074400 con 0x7fde6c141d20 2024-09-10T21:30:25.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde62a00640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c141d20 0x7fde6c139e90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fde5400d980 tx=0x7fde5400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde63e00640 1 -- 172.21.15.97:0/1898661362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54017070 con 0x7fde6c141d20 2024-09-10T21:30:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde63e00640 1 -- 172.21.15.97:0/1898661362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde54009d20 con 0x7fde6c141d20 2024-09-10T21:30:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde63e00640 1 -- 172.21.15.97:0/1898661362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde540106c0 con 0x7fde6c141d20 2024-09-10T21:30:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c13b9d0 con 0x7fde6c141d20 2024-09-10T21:30:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c13bf90 con 0x7fde6c141d20 2024-09-10T21:30:25.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.244+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6c15f6d0 con 0x7fde6c141d20 2024-09-10T21:30:25.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.248+0000 7fde63e00640 1 -- 172.21.15.97:0/1898661362 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fde54010860 con 0x7fde6c141d20 2024-09-10T21:30:25.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.248+0000 7fde63e00640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde480725e0 0x7fde48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.248+0000 7fde63e00640 1 -- 172.21.15.97:0/1898661362 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6273+0+0 (secure 0 0 0) 0x7fde54093950 con 0x7fde6c141d20 2024-09-10T21:30:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.248+0000 7fde6b400640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde480725e0 0x7fde48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.248+0000 7fde6b400640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde480725e0 0x7fde48074aa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fde5c000f70 tx=0x7fde5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.252+0000 7fde63e00640 1 -- 172.21.15.97:0/1898661362 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde5405ed50 con 0x7fde6c141d20 2024-09-10T21:30:25.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:25.448+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fde6c1599d0 con 0x7fde6c141d20 2024-09-10T21:30:26.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde63e00640 1 -- 172.21.15.97:0/1898661362 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7fde5405e650 con 0x7fde6c141d20 2024-09-10T21:30:26.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde480725e0 msgr2=0x7fde48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:26.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde480725e0 0x7fde48074aa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fde5c000f70 tx=0x7fde5c009040 comp rx=0 tx=0).stop 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c141d20 msgr2=0x7fde6c139e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c141d20 0x7fde6c139e90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fde5400d980 tx=0x7fde5400de40 comp rx=0 tx=0).stop 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 shutdown_connections 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde480725e0 0x7fde48074aa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fde6c15fc10 0x7fde6c13a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6c141d20 0x7fde6c139e90 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 --2- 172.21.15.97:0/1898661362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0ce5f0 0x7fde6c13c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 >> 172.21.15.97:0/1898661362 conn(0x7fde6c0b1780 msgr2=0x7fde6c13d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 shutdown_connections 2024-09-10T21:30:26.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.072+0000 7fde72600640 1 -- 172.21.15.97:0/1898661362 wait complete. 2024-09-10T21:30:26.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- 172.21.15.97:0/1862828206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8160220 msgr2=0x7f8df80ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 --2- 172.21.15.97:0/1862828206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8160220 0x7f8df80ce320 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8df8069990 tx=0x7f8de001c870 comp rx=0 tx=0).stop 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- 172.21.15.97:0/1862828206 shutdown_connections 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 --2- 172.21.15.97:0/1862828206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8160220 0x7f8df80ce320 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 --2- 172.21.15.97:0/1862828206 >> v2:172.21.15.194:3300/0 conn(0x7f8df815f120 0x7f8df80af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- 172.21.15.97:0/1862828206 >> 172.21.15.97:0/1862828206 conn(0x7f8df80b3c70 msgr2=0x7f8df80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- 172.21.15.97:0/1862828206 shutdown_connections 2024-09-10T21:30:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- 172.21.15.97:0/1862828206 wait complete. 2024-09-10T21:30:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 Processor -- start 2024-09-10T21:30:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- start start 2024-09-10T21:30:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8df806cb40 0x7f8df815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df815f120 0x7f8df814b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8df8160220 0x7f8df8164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8df80cea50 con 0x7f8df815f120 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8df80d0a90 con 0x7f8df8160220 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dff400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8df806cfd0 con 0x7f8df806cb40 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8defe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df815f120 0x7f8df814b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8defe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df815f120 0x7f8df814b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46526/0 (socket says 172.21.15.97:46526) 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8defe00640 1 -- 172.21.15.97:0/1072428586 learned_addr learned my addr 172.21.15.97:0/1072428586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dfd400640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8df8160220 0x7f8df8164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.284+0000 7f8dfca00640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8df806cb40 0x7f8df815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8defe00640 1 -- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8df806cb40 msgr2=0x7f8df815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8defe00640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8df806cb40 0x7f8df815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8defe00640 1 -- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8df8160220 msgr2=0x7f8df8164ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8defe00640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8df8160220 0x7f8df8164ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8defe00640 1 -- 172.21.15.97:0/1072428586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df815cc00 con 0x7f8df815f120 2024-09-10T21:30:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8dfca00640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8df806cb40 0x7f8df815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8defe00640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df815f120 0x7f8df814b610 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8de00b0040 tx=0x7f8de0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8df5600640 1 -- 172.21.15.97:0/1072428586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de00b2040 con 0x7f8df815f120 2024-09-10T21:30:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8df5600640 1 -- 172.21.15.97:0/1072428586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8de0003600 con 0x7f8df815f120 2024-09-10T21:30:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df81fb770 con 0x7f8df815f120 2024-09-10T21:30:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8df5600640 1 -- 172.21.15.97:0/1072428586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003a30 con 0x7f8df815f120 2024-09-10T21:30:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df81fbd30 con 0x7f8df815f120 2024-09-10T21:30:26.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df81fb560 con 0x7f8df815f120 2024-09-10T21:30:26.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8df5600640 1 -- 172.21.15.97:0/1072428586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8de0003bd0 con 0x7f8df815f120 2024-09-10T21:30:26.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8df5600640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dd40725e0 0x7f8dd4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:26.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8df5600640 1 -- 172.21.15.97:0/1072428586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6273+0+0 (secure 0 0 0) 0x7f8de012d400 con 0x7f8df815f120 2024-09-10T21:30:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.288+0000 7f8dfca00640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dd40725e0 0x7f8dd4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.292+0000 7f8dfca00640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dd40725e0 0x7f8dd4074aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8de800a8b0 tx=0x7f8de8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.292+0000 7f8df5600640 1 -- 172.21.15.97:0/1072428586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8de00bd060 con 0x7f8df815f120 2024-09-10T21:30:26.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:26.460+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8df80ce320 con 0x7f8df815f120 2024-09-10T21:30:27.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.076+0000 7f8df5600640 1 -- 172.21.15.97:0/1072428586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f8de0016490 con 0x7f8df815f120 2024-09-10T21:30:27.081 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:30:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dd40725e0 msgr2=0x7f8dd4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dd40725e0 0x7f8dd4074aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8de800a8b0 tx=0x7f8de8008040 comp rx=0 tx=0).stop 2024-09-10T21:30:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df815f120 msgr2=0x7f8df814b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df815f120 0x7f8df814b610 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8de00b0040 tx=0x7f8de0004040 comp rx=0 tx=0).stop 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 shutdown_connections 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dd40725e0 0x7f8dd4074aa0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8df8160220 0x7f8df8164ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df815f120 0x7f8df814b610 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 --2- 172.21.15.97:0/1072428586 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8df806cb40 0x7f8df815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 >> 172.21.15.97:0/1072428586 conn(0x7f8df80b3c70 msgr2=0x7f8df80aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 shutdown_connections 2024-09-10T21:30:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.080+0000 7f8dff400640 1 -- 172.21.15.97:0/1072428586 wait complete. 2024-09-10T21:30:27.106 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:30:27.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.248+0000 7f9a5b200640 1 -- 172.21.15.97:0/624531180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 msgr2=0x7f9a540b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:27.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.248+0000 7f9a5b200640 1 --2- 172.21.15.97:0/624531180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a540b37b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009870 tx=0x7f9a4401f010 comp rx=0 tx=0).stop 2024-09-10T21:30:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.248+0000 7f9a5b200640 1 -- 172.21.15.97:0/624531180 shutdown_connections 2024-09-10T21:30:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.248+0000 7f9a5b200640 1 --2- 172.21.15.97:0/624531180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a540b37b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.248+0000 7f9a5b200640 1 -- 172.21.15.97:0/624531180 >> 172.21.15.97:0/624531180 conn(0x7f9a540b1780 msgr2=0x7f9a540b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.248+0000 7f9a5b200640 1 -- 172.21.15.97:0/624531180 shutdown_connections 2024-09-10T21:30:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.248+0000 7f9a5b200640 1 -- 172.21.15.97:0/624531180 wait complete. 2024-09-10T21:30:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 Processor -- start 2024-09-10T21:30:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 -- start start 2024-09-10T21:30:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a540b3cf0 0x7f9a54149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a540ceee0 0x7f9a5414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a5414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5415b1d0 con 0x7f9a541511f0 2024-09-10T21:30:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5415b050 con 0x7f9a540b3cf0 2024-09-10T21:30:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9a540cf350 con 0x7f9a540ceee0 2024-09-10T21:30:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a5414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a5414ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46546/0 (socket says 172.21.15.97:46546) 2024-09-10T21:30:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 -- 172.21.15.97:0/3616998973 learned_addr learned my addr 172.21.15.97:0/3616998973 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a53e00640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a540b3cf0 0x7f9a54149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a53400640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a540ceee0 0x7f9a5414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 -- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a540ceee0 msgr2=0x7f9a5414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a540ceee0 0x7f9a5414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 -- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a540b3cf0 msgr2=0x7f9a54149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a540b3cf0 0x7f9a54149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 -- 172.21.15.97:0/3616998973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5414f1d0 con 0x7f9a541511f0 2024-09-10T21:30:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a53400640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a540ceee0 0x7f9a5414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a59200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a5414ea70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c780 tx=0x7f9a4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a4be00640 1 -- 172.21.15.97:0/3616998973 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c004b70 con 0x7f9a541511f0 2024-09-10T21:30:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a4be00640 1 -- 172.21.15.97:0/3616998973 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a4c004e00 con 0x7f9a541511f0 2024-09-10T21:30:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5414f4b0 con 0x7f9a541511f0 2024-09-10T21:30:27.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a4be00640 1 -- 172.21.15.97:0/3616998973 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00aa90 con 0x7f9a541511f0 2024-09-10T21:30:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5414fa70 con 0x7f9a541511f0 2024-09-10T21:30:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.252+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a540b37b0 con 0x7f9a541511f0 2024-09-10T21:30:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.256+0000 7f9a4be00640 1 -- 172.21.15.97:0/3616998973 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9a4c00ac30 con 0x7f9a541511f0 2024-09-10T21:30:27.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.256+0000 7f9a4be00640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a34072610 0x7f9a34074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:27.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.256+0000 7f9a4be00640 1 -- 172.21.15.97:0/3616998973 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5835+0+0 (secure 0 0 0) 0x7f9a4c0935f0 con 0x7f9a541511f0 2024-09-10T21:30:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.256+0000 7f9a53e00640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a34072610 0x7f9a34074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.256+0000 7f9a53e00640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a34072610 0x7f9a34074ad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9a440b2000 tx=0x7f9a4401d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.260+0000 7f9a4be00640 1 -- 172.21.15.97:0/3616998973 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a4c020070 con 0x7f9a541511f0 2024-09-10T21:30:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:27.424+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9a5414e120 con 0x7f9a541511f0 2024-09-10T21:30:28.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.080+0000 7f9a4be00640 1 -- 172.21.15.97:0/3616998973 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f9a4c091e20 con 0x7f9a541511f0 2024-09-10T21:30:28.083 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-09-10T21:30:28.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a34072610 msgr2=0x7f9a34074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a34072610 0x7f9a34074ad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9a440b2000 tx=0x7f9a4401d850 comp rx=0 tx=0).stop 2024-09-10T21:30:28.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 msgr2=0x7f9a5414ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a5414ea70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c780 tx=0x7f9a4c00cc40 comp rx=0 tx=0).stop 2024-09-10T21:30:28.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 shutdown_connections 2024-09-10T21:30:28.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9a34072610 0x7f9a34074ad0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a541511f0 0x7f9a5414ea70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9a540ceee0 0x7f9a5414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 --2- 172.21.15.97:0/3616998973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a540b3cf0 0x7f9a54149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 >> 172.21.15.97:0/3616998973 conn(0x7f9a540b1780 msgr2=0x7f9a5415a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:28.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 shutdown_connections 2024-09-10T21:30:28.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.084+0000 7f9a5b200640 1 -- 172.21.15.97:0/3616998973 wait complete. 2024-09-10T21:30:28.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:30:28.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.256+0000 7f2585c00640 1 -- 172.21.15.97:0/1253322743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 msgr2=0x7f256400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.256+0000 7f2585c00640 1 --2- 172.21.15.97:0/1253322743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f256400b9d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2578009e00 tx=0x7f257801d170 comp rx=0 tx=0).stop 2024-09-10T21:30:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.256+0000 7f2585c00640 1 -- 172.21.15.97:0/1253322743 shutdown_connections 2024-09-10T21:30:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.256+0000 7f2585c00640 1 --2- 172.21.15.97:0/1253322743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f256400b9d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.256+0000 7f2585c00640 1 -- 172.21.15.97:0/1253322743 >> 172.21.15.97:0/1253322743 conn(0x7f25800d1a60 msgr2=0x7f25800d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.256+0000 7f2585c00640 1 -- 172.21.15.97:0/1253322743 shutdown_connections 2024-09-10T21:30:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.256+0000 7f2585c00640 1 -- 172.21.15.97:0/1253322743 wait complete. 2024-09-10T21:30:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 Processor -- start 2024-09-10T21:30:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 -- start start 2024-09-10T21:30:28.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f25800c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25800c8b30 0x7f25800b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25801f81b0 0x7f258015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25800cf840 con 0x7f2564005220 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f25800cd9e0 con 0x7f25801f81b0 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f25800cf9c0 con 0x7f25800c8b30 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f25800c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f25800c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42902/0 (socket says 172.21.15.97:42902) 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 -- 172.21.15.97:0/1609840870 learned_addr learned my addr 172.21.15.97:0/1609840870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585200640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25801f81b0 0x7f258015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257f400640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25800c8b30 0x7f25800b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 -- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25800c8b30 msgr2=0x7f25800b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25800c8b30 0x7f25800b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 -- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25801f81b0 msgr2=0x7f258015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25801f81b0 0x7f258015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 -- 172.21.15.97:0/1609840870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25801f7a20 con 0x7f2564005220 2024-09-10T21:30:28.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f257fe00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f25800c84e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2570006630 tx=0x7f25700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:28.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2577e00640 1 -- 172.21.15.97:0/1609840870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570015070 con 0x7f2564005220 2024-09-10T21:30:28.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2577e00640 1 -- 172.21.15.97:0/1609840870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2570011920 con 0x7f2564005220 2024-09-10T21:30:28.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25800c94a0 con 0x7f2564005220 2024-09-10T21:30:28.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2577e00640 1 -- 172.21.15.97:0/1609840870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570011d70 con 0x7f2564005220 2024-09-10T21:30:28.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25800c9a60 con 0x7f2564005220 2024-09-10T21:30:28.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.260+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25800c8f10 con 0x7f2564005220 2024-09-10T21:30:28.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.264+0000 7f2577e00640 1 -- 172.21.15.97:0/1609840870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f25700103c0 con 0x7f2564005220 2024-09-10T21:30:28.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.264+0000 7f2577e00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25600725e0 0x7f2560074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.264+0000 7f2577e00640 1 -- 172.21.15.97:0/1609840870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f2570094180 con 0x7f2564005220 2024-09-10T21:30:28.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.264+0000 7f257f400640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25600725e0 0x7f2560074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.264+0000 7f257f400640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25600725e0 0x7f2560074aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f25800706a0 tx=0x7f256c006b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.268+0000 7f2577e00640 1 -- 172.21.15.97:0/1609840870 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f257005f320 con 0x7f2564005220 2024-09-10T21:30:28.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.432+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f25800cfeb0 con 0x7f2564005220 2024-09-10T21:30:28.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.436+0000 7f2577e00640 1 -- 172.21.15.97:0/1609840870 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v42) ==== 71+0+0 (secure 0 0 0) 0x7f2570063d50 con 0x7f2564005220 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.436+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25600725e0 msgr2=0x7f2560074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.436+0000 7f2585c00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25600725e0 0x7f2560074aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f25800706a0 tx=0x7f256c006b80 comp rx=0 tx=0).stop 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 msgr2=0x7f25800c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f25800c84e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2570006630 tx=0x7f25700039d0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 shutdown_connections 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25600725e0 0x7f2560074aa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25801f81b0 0x7f258015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25800c8b30 0x7f25800b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 --2- 172.21.15.97:0/1609840870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2564005220 0x7f25800c84e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 >> 172.21.15.97:0/1609840870 conn(0x7f25800d1a60 msgr2=0x7f25800afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:28.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 shutdown_connections 2024-09-10T21:30:28.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.440+0000 7f2585c00640 1 -- 172.21.15.97:0/1609840870 wait complete. 2024-09-10T21:30:28.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-10T21:30:28.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.600+0000 7f43b3e00640 1 -- 172.21.15.97:0/2477134512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b415efc0 msgr2=0x7f43b40cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.600+0000 7f43b3e00640 1 --2- 172.21.15.97:0/2477134512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b415efc0 0x7f43b40cf840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009870 tx=0x7f43a401c860 comp rx=0 tx=0).stop 2024-09-10T21:30:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.600+0000 7f43b3e00640 1 -- 172.21.15.97:0/2477134512 shutdown_connections 2024-09-10T21:30:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.600+0000 7f43b3e00640 1 --2- 172.21.15.97:0/2477134512 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b415efc0 0x7f43b40cf840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.600+0000 7f43b3e00640 1 -- 172.21.15.97:0/2477134512 >> 172.21.15.97:0/2477134512 conn(0x7f43b40d1a60 msgr2=0x7f43b40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.600+0000 7f43b3e00640 1 -- 172.21.15.97:0/2477134512 shutdown_connections 2024-09-10T21:30:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.600+0000 7f43b3e00640 1 -- 172.21.15.97:0/2477134512 wait complete. 2024-09-10T21:30:28.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 Processor -- start 2024-09-10T21:30:28.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 -- start start 2024-09-10T21:30:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43b40cdcb0 0x7f43b415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b40d0490 0x7f43b414d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43b415efc0 0x7f43b4155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43b4169c00 con 0x7f43b415efc0 2024-09-10T21:30:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f43b4169a80 con 0x7f43b40cdcb0 2024-09-10T21:30:28.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f43b4169d80 con 0x7f43b40d0490 2024-09-10T21:30:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43b415efc0 0x7f43b4155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43b415efc0 0x7f43b4155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42910/0 (socket says 172.21.15.97:42910) 2024-09-10T21:30:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 -- 172.21.15.97:0/1703771518 learned_addr learned my addr 172.21.15.97:0/1703771518 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b2a00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43b40cdcb0 0x7f43b415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b2000640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b40d0490 0x7f43b414d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 -- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b40d0490 msgr2=0x7f43b414d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b40d0490 0x7f43b414d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 -- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43b40cdcb0 msgr2=0x7f43b415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43b40cdcb0 0x7f43b415e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 -- 172.21.15.97:0/1703771518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b4152af0 con 0x7f43b415efc0 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3400640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43b415efc0 0x7f43b4155aa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00cae0 tx=0x7f43ac00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43aaa00640 1 -- 172.21.15.97:0/1703771518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac016020 con 0x7f43b415efc0 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43aaa00640 1 -- 172.21.15.97:0/1703771518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43ac00ab40 con 0x7f43b415efc0 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43aaa00640 1 -- 172.21.15.97:0/1703771518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac00ce20 con 0x7f43b415efc0 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b415c2f0 con 0x7f43b415efc0 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b415c8b0 con 0x7f43b415efc0 2024-09-10T21:30:28.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.604+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b40ae930 con 0x7f43b415efc0 2024-09-10T21:30:28.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.608+0000 7f43aaa00640 1 -- 172.21.15.97:0/1703771518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f43ac01a3b0 con 0x7f43b415efc0 2024-09-10T21:30:28.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.608+0000 7f43aaa00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43940725e0 0x7f4394074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:28.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.608+0000 7f43aaa00640 1 -- 172.21.15.97:0/1703771518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f43ac093260 con 0x7f43b415efc0 2024-09-10T21:30:28.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.608+0000 7f43b2a00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43940725e0 0x7f4394074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:28.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.608+0000 7f43b2a00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43940725e0 0x7f4394074aa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f43a40af000 tx=0x7f43a401b0a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:28.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.612+0000 7f43aaa00640 1 -- 172.21.15.97:0/1703771518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f43ac020030 con 0x7f43b415efc0 2024-09-10T21:30:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:28.836+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7f43b4157bb0 con 0x7f43b415efc0 2024-09-10T21:30:29.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.084+0000 7f43aaa00640 1 -- 172.21.15.97:0/1703771518 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v303) ==== 141+0+0 (secure 0 0 0) 0x7f43ac067170 con 0x7f43b415efc0 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43940725e0 msgr2=0x7f4394074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43940725e0 0x7f4394074aa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f43a40af000 tx=0x7f43a401b0a0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43b415efc0 msgr2=0x7f43b4155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43b415efc0 0x7f43b4155aa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00cae0 tx=0x7f43ac00cbd0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 shutdown_connections 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43940725e0 0x7f4394074aa0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43b415efc0 0x7f43b4155aa0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f43b40d0490 0x7f43b414d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 --2- 172.21.15.97:0/1703771518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43b40cdcb0 0x7f43b415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 >> 172.21.15.97:0/1703771518 conn(0x7f43b40d1a60 msgr2=0x7f43b40b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 shutdown_connections 2024-09-10T21:30:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.088+0000 7f43b3e00640 1 -- 172.21.15.97:0/1703771518 wait complete. 2024-09-10T21:30:29.111 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-10T21:30:29.111 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2024-09-10T21:30:29.111 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2024-09-10T21:30:29.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:30:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.248+0000 7fab85000640 1 -- 172.21.15.97:0/2622970681 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800cf5f0 msgr2=0x7fab8015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.248+0000 7fab85000640 1 --2- 172.21.15.97:0/2622970681 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800cf5f0 0x7fab8015a730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fab78009cb0 tx=0x7fab7801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- 172.21.15.97:0/2622970681 shutdown_connections 2024-09-10T21:30:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 --2- 172.21.15.97:0/2622970681 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800cf5f0 0x7fab8015a730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 --2- 172.21.15.97:0/2622970681 >> v2:172.21.15.194:3300/0 conn(0x7fab801511f0 0x7fab800b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- 172.21.15.97:0/2622970681 >> 172.21.15.97:0/2622970681 conn(0x7fab800b1780 msgr2=0x7fab800b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- 172.21.15.97:0/2622970681 shutdown_connections 2024-09-10T21:30:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- 172.21.15.97:0/2622970681 wait complete. 2024-09-10T21:30:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 Processor -- start 2024-09-10T21:30:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- start start 2024-09-10T21:30:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800b3cf0 0x7fab801517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab800cf5f0 0x7fab801f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab801511f0 0x7fab80159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab8006e3b0 con 0x7fab800cf5f0 2024-09-10T21:30:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab8015b290 con 0x7fab801511f0 2024-09-10T21:30:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab85000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fab8006e530 con 0x7fab800b3cf0 2024-09-10T21:30:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab7d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab800cf5f0 0x7fab801f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab7e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab801511f0 0x7fab80159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab7e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab801511f0 0x7fab80159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:60118/0 (socket says 172.21.15.97:60118) 2024-09-10T21:30:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab7e800640 1 -- 172.21.15.97:0/3584250223 learned_addr learned my addr 172.21.15.97:0/3584250223 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.252+0000 7fab7de00640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800b3cf0 0x7fab801517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab7e800640 1 -- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800b3cf0 msgr2=0x7fab801517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab7e800640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800b3cf0 0x7fab801517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab7e800640 1 -- 172.21.15.97:0/3584250223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab800cf5f0 msgr2=0x7fab801f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab7e800640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab800cf5f0 0x7fab801f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab7e800640 1 -- 172.21.15.97:0/3584250223 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab800cc740 con 0x7fab801511f0 2024-09-10T21:30:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab7e800640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab801511f0 0x7fab80159170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fab78009bd0 tx=0x7fab78009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab76000640 1 -- 172.21.15.97:0/3584250223 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78004e60 con 0x7fab801511f0 2024-09-10T21:30:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab800cca20 con 0x7fab801511f0 2024-09-10T21:30:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab76000640 1 -- 172.21.15.97:0/3584250223 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab780be030 con 0x7fab801511f0 2024-09-10T21:30:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab76000640 1 -- 172.21.15.97:0/3584250223 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab780afb00 con 0x7fab801511f0 2024-09-10T21:30:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab800ccfe0 con 0x7fab801511f0 2024-09-10T21:30:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8015a730 con 0x7fab801511f0 2024-09-10T21:30:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab76000640 1 -- 172.21.15.97:0/3584250223 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fab780afca0 con 0x7fab801511f0 2024-09-10T21:30:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab76000640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab600725e0 0x7fab60074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.256+0000 7fab76000640 1 -- 172.21.15.97:0/3584250223 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7fab78132130 con 0x7fab801511f0 2024-09-10T21:30:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.260+0000 7fab7de00640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab600725e0 0x7fab60074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.260+0000 7fab7de00640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab600725e0 0x7fab60074aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fab70009b40 tx=0x7fab700091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:29.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.260+0000 7fab76000640 1 -- 172.21.15.97:0/3584250223 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fab780ac050 con 0x7fab801511f0 2024-09-10T21:30:29.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab8006ed10 con 0x7fab801511f0 2024-09-10T21:30:29.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab76000640 1 -- 172.21.15.97:0/3584250223 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+13462 (secure 0 0 0) 0x7fab78019100 con 0x7fab801511f0 2024-09-10T21:30:29.431 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:29.431 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:30:28.082539+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6833/2931568595":"2024-09-11T21:30:25.072814+0000","172.21.15.194:6836/3597857813":"2024-09-11T21:30:16.599378+0000","172.21.15.97:6834/2931568595":"2024-09-11T21:30:25.072814+0000","172.21.15.194:6834/3597857813":"2024-09-11T21:30:16.599378+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:30:29.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab600725e0 msgr2=0x7fab60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab600725e0 0x7fab60074aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fab70009b40 tx=0x7fab700091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab801511f0 msgr2=0x7fab80159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab801511f0 0x7fab80159170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fab78009bd0 tx=0x7fab78009390 comp rx=0 tx=0).stop 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 shutdown_connections 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fab600725e0 0x7fab60074aa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab801511f0 0x7fab80159170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab800cf5f0 0x7fab801f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 --2- 172.21.15.97:0/3584250223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fab800b3cf0 0x7fab801517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 >> 172.21.15.97:0/3584250223 conn(0x7fab800b1780 msgr2=0x7fab8015c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 shutdown_connections 2024-09-10T21:30:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.428+0000 7fab85000640 1 -- 172.21.15.97:0/3584250223 wait complete. 2024-09-10T21:30:29.453 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:24:02.092708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:30:29.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:30:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- 172.21.15.97:0/2758553578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 msgr2=0x7f8a8415abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 --2- 172.21.15.97:0/2758553578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 0x7f8a8415abc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009bf0 tx=0x7f8a7c01f390 comp rx=0 tx=0).stop 2024-09-10T21:30:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- 172.21.15.97:0/2758553578 shutdown_connections 2024-09-10T21:30:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 --2- 172.21.15.97:0/2758553578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 0x7f8a8415abc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 --2- 172.21.15.97:0/2758553578 >> v2:172.21.15.194:3300/0 conn(0x7f8a840ceee0 0x7f8a840cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- 172.21.15.97:0/2758553578 >> 172.21.15.97:0/2758553578 conn(0x7f8a840b1780 msgr2=0x7f8a840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- 172.21.15.97:0/2758553578 shutdown_connections 2024-09-10T21:30:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- 172.21.15.97:0/2758553578 wait complete. 2024-09-10T21:30:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 Processor -- start 2024-09-10T21:30:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- start start 2024-09-10T21:30:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a840ceee0 0x7f8a840cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a841511f0 0x7f8a840b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 0x7f8a84146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8406e530 con 0x7f8a841511f0 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8406e3b0 con 0x7f8a840ceee0 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a8b400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8406e6b0 con 0x7f8a8415a730 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a841511f0 0x7f8a840b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a841511f0 0x7f8a840b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42920/0 (socket says 172.21.15.97:42920) 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 -- 172.21.15.97:0/1638254918 learned_addr learned my addr 172.21.15.97:0/1638254918 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a88a00640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a840ceee0 0x7f8a840cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a89400640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 0x7f8a84146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 -- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 msgr2=0x7f8a84146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 0x7f8a84146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 -- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a840ceee0 msgr2=0x7f8a840cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a840ceee0 0x7f8a840cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 -- 172.21.15.97:0/1638254918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a841e5840 con 0x7f8a841511f0 2024-09-10T21:30:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.596+0000 7f8a7be00640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a841511f0 0x7f8a840b0ca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00e9f0 tx=0x7f8a6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a81600640 1 -- 172.21.15.97:0/1638254918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00cd80 con 0x7f8a841511f0 2024-09-10T21:30:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a81600640 1 -- 172.21.15.97:0/1638254918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a6c010430 con 0x7f8a841511f0 2024-09-10T21:30:29.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a81600640 1 -- 172.21.15.97:0/1638254918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c010860 con 0x7f8a841511f0 2024-09-10T21:30:29.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a841e5b20 con 0x7f8a841511f0 2024-09-10T21:30:29.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a84147200 con 0x7f8a841511f0 2024-09-10T21:30:29.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a840cf330 con 0x7f8a841511f0 2024-09-10T21:30:29.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a81600640 1 -- 172.21.15.97:0/1638254918 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8a6c010a00 con 0x7f8a841511f0 2024-09-10T21:30:29.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a81600640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a600725e0 0x7f8a60074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a81600640 1 -- 172.21.15.97:0/1638254918 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f8a6c093cf0 con 0x7f8a841511f0 2024-09-10T21:30:29.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.600+0000 7f8a88a00640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a600725e0 0x7f8a60074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.604+0000 7f8a88a00640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a600725e0 0x7f8a60074aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a74001910 tx=0x7f8a74001420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.604+0000 7f8a81600640 1 -- 172.21.15.97:0/1638254918 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8a6c05ef50 con 0x7f8a841511f0 2024-09-10T21:30:29.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.772+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a840af6b0 con 0x7f8a841511f0 2024-09-10T21:30:29.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.772+0000 7f8a81600640 1 -- 172.21.15.97:0/1638254918 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f8a6c063980 con 0x7f8a841511f0 2024-09-10T21:30:29.776 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-09-10T21:30:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a600725e0 msgr2=0x7f8a60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a600725e0 0x7f8a60074aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a74001910 tx=0x7f8a74001420 comp rx=0 tx=0).stop 2024-09-10T21:30:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a841511f0 msgr2=0x7f8a840b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a841511f0 0x7f8a840b0ca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00e9f0 tx=0x7f8a6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 shutdown_connections 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a600725e0 0x7f8a60074aa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a8415a730 0x7f8a84146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a841511f0 0x7f8a840b0ca0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 --2- 172.21.15.97:0/1638254918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a840ceee0 0x7f8a840cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 >> 172.21.15.97:0/1638254918 conn(0x7f8a840b1780 msgr2=0x7f8a840b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 shutdown_connections 2024-09-10T21:30:29.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.776+0000 7f8a8b400640 1 -- 172.21.15.97:0/1638254918 wait complete. 2024-09-10T21:30:29.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-10T21:30:29.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 -- 172.21.15.97:0/2587858659 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c7015b9b0 msgr2=0x7f7c7006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 --2- 172.21.15.97:0/2587858659 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c7015b9b0 0x7f7c7006cb40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009840 tx=0x7f7c6401c790 comp rx=0 tx=0).stop 2024-09-10T21:30:29.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 -- 172.21.15.97:0/2587858659 shutdown_connections 2024-09-10T21:30:29.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 --2- 172.21.15.97:0/2587858659 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c700b00a0 0x7f7c700b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 --2- 172.21.15.97:0/2587858659 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c7015b9b0 0x7f7c7006cb40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 -- 172.21.15.97:0/2587858659 >> 172.21.15.97:0/2587858659 conn(0x7f7c700cd680 msgr2=0x7f7c700cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:29.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 -- 172.21.15.97:0/2587858659 shutdown_connections 2024-09-10T21:30:29.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.936+0000 7f7c78400640 1 -- 172.21.15.97:0/2587858659 wait complete. 2024-09-10T21:30:29.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 Processor -- start 2024-09-10T21:30:29.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 -- start start 2024-09-10T21:30:29.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c700b00a0 0x7f7c70156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c7015b9b0 0x7f7c700b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c7015bd90 0x7f7c700cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701692a0 con 0x7f7c7015b9b0 2024-09-10T21:30:29.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70169120 con 0x7f7c700b00a0 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7c70169420 con 0x7f7c7015bd90 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c7015b9b0 0x7f7c700b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c7015b9b0 0x7f7c700b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42924/0 (socket says 172.21.15.97:42924) 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 -- 172.21.15.97:0/2781570487 learned_addr learned my addr 172.21.15.97:0/2781570487 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 -- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c7015bd90 msgr2=0x7f7c700cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c7015bd90 0x7f7c700cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 -- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c700b00a0 msgr2=0x7f7c70156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c700b00a0 0x7f7c70156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 -- 172.21.15.97:0/2781570487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c701fb590 con 0x7f7c7015b9b0 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c75000640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c7015b9b0 0x7f7c700b1c00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c460 tx=0x7f7c5c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c6d600640 1 -- 172.21.15.97:0/2781570487 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c016020 con 0x7f7c7015b9b0 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c6d600640 1 -- 172.21.15.97:0/2781570487 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c5c012940 con 0x7f7c7015b9b0 2024-09-10T21:30:29.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c6d600640 1 -- 172.21.15.97:0/2781570487 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c012d90 con 0x7f7c7015b9b0 2024-09-10T21:30:29.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c701fb870 con 0x7f7c7015b9b0 2024-09-10T21:30:29.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.940+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c701fbe30 con 0x7f7c7015b9b0 2024-09-10T21:30:29.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.944+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7006cb40 con 0x7f7c7015b9b0 2024-09-10T21:30:29.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.944+0000 7f7c6d600640 1 -- 172.21.15.97:0/2781570487 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7c5c01b400 con 0x7f7c7015b9b0 2024-09-10T21:30:29.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.944+0000 7f7c6d600640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c50072610 0x7f7c50074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:29.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.944+0000 7f7c6d600640 1 -- 172.21.15.97:0/2781570487 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f7c5c0932f0 con 0x7f7c7015b9b0 2024-09-10T21:30:29.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.944+0000 7f7c75a00640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c50072610 0x7f7c50074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:29.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.944+0000 7f7c75a00640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c50072610 0x7f7c50074ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7c640af000 tx=0x7f7c64019530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:29.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:29.948+0000 7f7c6d600640 1 -- 172.21.15.97:0/2781570487 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7c5c020020 con 0x7f7c7015b9b0 2024-09-10T21:30:30.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:30.176+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7f7c700b0680 con 0x7f7c7015b9b0 2024-09-10T21:30:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.092+0000 7f7c6d600640 1 -- 172.21.15.97:0/2781570487 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v305) ==== 138+0+0 (secure 0 0 0) 0x7f7c5c065070 con 0x7f7c7015b9b0 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c50072610 msgr2=0x7f7c50074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c50072610 0x7f7c50074ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7c640af000 tx=0x7f7c64019530 comp rx=0 tx=0).stop 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c7015b9b0 msgr2=0x7f7c700b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c7015b9b0 0x7f7c700b1c00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c460 tx=0x7f7c5c00c920 comp rx=0 tx=0).stop 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 shutdown_connections 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c50072610 0x7f7c50074ad0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c7015bd90 0x7f7c700cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c7015b9b0 0x7f7c700b1c00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 --2- 172.21.15.97:0/2781570487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c700b00a0 0x7f7c70156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 >> 172.21.15.97:0/2781570487 conn(0x7f7c700cd680 msgr2=0x7f7c700b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 shutdown_connections 2024-09-10T21:30:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.096+0000 7f7c78400640 1 -- 172.21.15.97:0/2781570487 wait complete. 2024-09-10T21:30:31.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:30:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- 172.21.15.97:0/1771914630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80cec10 msgr2=0x7fd6c80cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/1771914630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80cec10 0x7fd6c80cff10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009900 tx=0x7fd6b001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- 172.21.15.97:0/1771914630 shutdown_connections 2024-09-10T21:30:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/1771914630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80cec10 0x7fd6c80cff10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- 172.21.15.97:0/1771914630 >> 172.21.15.97:0/1771914630 conn(0x7fd6c80b37c0 msgr2=0x7fd6c80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- 172.21.15.97:0/1771914630 shutdown_connections 2024-09-10T21:30:31.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- 172.21.15.97:0/1771914630 wait complete. 2024-09-10T21:30:31.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 Processor -- start 2024-09-10T21:30:31.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- start start 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80b1560 0x7fd6c80b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce830 0x7fd6c80afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c80cec10 0x7fd6c815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c80ce320 con 0x7fd6c80b1560 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c80ce1a0 con 0x7fd6c80cec10 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cfa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c80d0450 con 0x7fd6c80ce830 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80b1560 0x7fd6c80b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80b1560 0x7fd6c80b7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42938/0 (socket says 172.21.15.97:42938) 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.268+0000 7fd6cd000640 1 -- 172.21.15.97:0/2844918867 learned_addr learned my addr 172.21.15.97:0/2844918867 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6bfe00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce830 0x7fd6c80afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cda00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c80cec10 0x7fd6c815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cd000640 1 -- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce830 msgr2=0x7fd6c80afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cd000640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce830 0x7fd6c80afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cd000640 1 -- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c80cec10 msgr2=0x7fd6c815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cd000640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c80cec10 0x7fd6c815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cd000640 1 -- 172.21.15.97:0/2844918867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c80b5980 con 0x7fd6c80b1560 2024-09-10T21:30:31.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cd000640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80b1560 0x7fd6c80b7410 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c430 tx=0x7fd6b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6c5600640 1 -- 172.21.15.97:0/2844918867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8017070 con 0x7fd6c80b1560 2024-09-10T21:30:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6c5600640 1 -- 172.21.15.97:0/2844918867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6b80053c0 con 0x7fd6c80b1560 2024-09-10T21:30:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6c5600640 1 -- 172.21.15.97:0/2844918867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8003880 con 0x7fd6c80b1560 2024-09-10T21:30:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c81f5a70 con 0x7fd6c80b1560 2024-09-10T21:30:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c80b5f40 con 0x7fd6c80b1560 2024-09-10T21:30:31.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c80b1970 con 0x7fd6c80b1560 2024-09-10T21:30:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6c5600640 1 -- 172.21.15.97:0/2844918867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd6b8003a20 con 0x7fd6c80b1560 2024-09-10T21:30:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6c5600640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a40725e0 0x7fd6a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.272+0000 7fd6c5600640 1 -- 172.21.15.97:0/2844918867 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7fd6b8092800 con 0x7fd6c80b1560 2024-09-10T21:30:31.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.276+0000 7fd6bfe00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a40725e0 0x7fd6a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.276+0000 7fd6bfe00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a40725e0 0x7fd6a4074aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00b2000 tx=0x7fd6b001d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:31.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.276+0000 7fd6c5600640 1 -- 172.21.15.97:0/2844918867 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6b8001130 con 0x7fd6c80b1560 2024-09-10T21:30:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fd6c80d0080 con 0x7fd6c80b1560 2024-09-10T21:30:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6c5600640 1 -- 172.21.15.97:0/2844918867 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v42) ==== 59+0+1998 (secure 0 0 0) 0x7fd6b805dff0 con 0x7fd6c80b1560 2024-09-10T21:30:31.447 INFO:teuthology.orchestra.run.smithi097.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3032.483965 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3656.533946 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3351.408335 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3363.485497 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 8164.284498 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4391.956543 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4391.204852 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4799.171341 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds 20/20 2024-09-10T21:30:31.448 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_ms 2/2 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_frag true 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_verify_scatter true 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced client_mount_timeout 600 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_client 20/20 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_ms 1/1 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:30:31.449 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:30:31.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a40725e0 msgr2=0x7fd6a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a40725e0 0x7fd6a4074aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00b2000 tx=0x7fd6b001d910 comp rx=0 tx=0).stop 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80b1560 msgr2=0x7fd6c80b7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80b1560 0x7fd6c80b7410 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c430 tx=0x7fd6b800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 shutdown_connections 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a40725e0 0x7fd6a4074aa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c80cec10 0x7fd6c815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce830 0x7fd6c80afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 --2- 172.21.15.97:0/2844918867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80b1560 0x7fd6c80b7410 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 >> 172.21.15.97:0/2844918867 conn(0x7fd6c80b37c0 msgr2=0x7fd6c815f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 shutdown_connections 2024-09-10T21:30:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.444+0000 7fd6cfa00640 1 -- 172.21.15.97:0/2844918867 wait complete. 2024-09-10T21:30:31.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:30:31.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- 172.21.15.97:0/2409319441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd7c003a10 msgr2=0x7fbd7c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 --2- 172.21.15.97:0/2409319441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd7c003a10 0x7fbd7c003e30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009b60 tx=0x7fbd8801d2b0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 -- 172.21.15.97:0/2409319441 reap_dead start 2024-09-10T21:30:31.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- 172.21.15.97:0/2409319441 shutdown_connections 2024-09-10T21:30:31.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- 172.21.15.97:0/2409319441 >> 172.21.15.97:0/2409319441 conn(0x7fbd980d1a60 msgr2=0x7fbd980d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:31.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- 172.21.15.97:0/2409319441 shutdown_connections 2024-09-10T21:30:31.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- 172.21.15.97:0/2409319441 wait complete. 2024-09-10T21:30:31.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 Processor -- start 2024-09-10T21:30:31.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- start start 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd7c003a10 0x7fbd981f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd9815cc50 0x7fbd98157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9815dd40 0x7fbd9815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98166c90 con 0x7fbd9815dd40 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98166b10 con 0x7fbd7c003a10 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd9ca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbd98166e10 con 0x7fbd9815cc50 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9815dd40 0x7fbd9815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9815dd40 0x7fbd9815e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42956/0 (socket says 172.21.15.97:42956) 2024-09-10T21:30:31.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 -- 172.21.15.97:0/1952251267 learned_addr learned my addr 172.21.15.97:0/1952251267 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd96800640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd7c003a10 0x7fbd981f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd95e00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd9815cc50 0x7fbd98157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 -- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd9815cc50 msgr2=0x7fbd98157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd9815cc50 0x7fbd98157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 -- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd7c003a10 msgr2=0x7fbd981f9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd7c003a10 0x7fbd981f9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd97200640 1 -- 172.21.15.97:0/1952251267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9815b900 con 0x7fbd9815dd40 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.612+0000 7fbd95e00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd9815cc50 0x7fbd98157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd97200640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9815dd40 0x7fbd9815e1b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbd9806daf0 tx=0x7fbd9000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd8ea00640 1 -- 172.21.15.97:0/1952251267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90015020 con 0x7fbd9815dd40 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd8ea00640 1 -- 172.21.15.97:0/1952251267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd9000cad0 con 0x7fbd9815dd40 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd8ea00640 1 -- 172.21.15.97:0/1952251267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90011460 con 0x7fbd9815dd40 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9815bbe0 con 0x7fbd9815dd40 2024-09-10T21:30:31.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9815c1a0 con 0x7fbd9815dd40 2024-09-10T21:30:31.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd980cf840 con 0x7fbd9815dd40 2024-09-10T21:30:31.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.616+0000 7fbd8ea00640 1 -- 172.21.15.97:0/1952251267 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbd90011810 con 0x7fbd9815dd40 2024-09-10T21:30:31.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.620+0000 7fbd8ea00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd780725e0 0x7fbd78074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:31.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.620+0000 7fbd8ea00640 1 -- 172.21.15.97:0/1952251267 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7fbd90093c70 con 0x7fbd9815dd40 2024-09-10T21:30:31.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.620+0000 7fbd96800640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd780725e0 0x7fbd78074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:31.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.620+0000 7fbd96800640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd780725e0 0x7fbd78074aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbd880ae000 tx=0x7fbd8801a050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:31.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.624+0000 7fbd8ea00640 1 -- 172.21.15.97:0/1952251267 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd9001f020 con 0x7fbd9815dd40 2024-09-10T21:30:31.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.784+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbd980706e0 con 0x7fbd9815dd40 2024-09-10T21:30:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd8ea00640 1 -- 172.21.15.97:0/1952251267 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fbd9005f460 con 0x7fbd9815dd40 2024-09-10T21:30:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd780725e0 msgr2=0x7fbd78074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd780725e0 0x7fbd78074aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbd880ae000 tx=0x7fbd8801a050 comp rx=0 tx=0).stop 2024-09-10T21:30:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9815dd40 msgr2=0x7fbd9815e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9815dd40 0x7fbd9815e1b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbd9806daf0 tx=0x7fbd9000ecf0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 shutdown_connections 2024-09-10T21:30:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd780725e0 0x7fbd78074aa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9815dd40 0x7fbd9815e1b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd9815cc50 0x7fbd98157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 --2- 172.21.15.97:0/1952251267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd7c003a10 0x7fbd981f9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 >> 172.21.15.97:0/1952251267 conn(0x7fbd980d1a60 msgr2=0x7fbd981650a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 shutdown_connections 2024-09-10T21:30:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:31.792+0000 7fbd9ca00640 1 -- 172.21.15.97:0/1952251267 wait complete. 2024-09-10T21:30:31.815 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:30:31.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:30:31.896 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:30:31.976 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:30:31.977 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:31.977 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:31.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:30:31.985 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:30:31.985 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:31.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:30:31.994 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:31.994 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:32.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:30:32.034 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:30:32.034 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:32.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:30:32.046 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:30:32.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:32.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.216+0000 7fbf71c00640 1 -- 172.21.15.97:0/530175008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf50004810 msgr2=0x7fbf50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.216+0000 7fbf71c00640 1 --2- 172.21.15.97:0/530175008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf50004810 0x7fbf50004c90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbf540016c0 tx=0x7fbf5401c730 comp rx=0 tx=0).stop 2024-09-10T21:30:32.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.216+0000 7fbf71c00640 1 -- 172.21.15.97:0/530175008 shutdown_connections 2024-09-10T21:30:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.216+0000 7fbf71c00640 1 --2- 172.21.15.97:0/530175008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf50004810 0x7fbf50004c90 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.216+0000 7fbf71c00640 1 -- 172.21.15.97:0/530175008 >> 172.21.15.97:0/530175008 conn(0x7fbf6c0d1cd0 msgr2=0x7fbf6c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.216+0000 7fbf71c00640 1 -- 172.21.15.97:0/530175008 shutdown_connections 2024-09-10T21:30:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.216+0000 7fbf71c00640 1 -- 172.21.15.97:0/530175008 wait complete. 2024-09-10T21:30:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 Processor -- start 2024-09-10T21:30:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 -- start start 2024-09-10T21:30:32.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf50004810 0x7fbf6c1f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c0b2e90 0x7fbf6c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf6c146290 0x7fbf6c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0cc550 con 0x7fbf6c146290 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0d2b60 con 0x7fbf6c0b2e90 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c0cc6d0 con 0x7fbf50004810 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf6c146290 0x7fbf6c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf6c146290 0x7fbf6c1466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42974/0 (socket says 172.21.15.97:42974) 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 -- 172.21.15.97:0/346002166 learned_addr learned my addr 172.21.15.97:0/346002166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6aa00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf50004810 0x7fbf6c1f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 -- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf50004810 msgr2=0x7fbf6c1f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf50004810 0x7fbf6c1f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf62000640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c0b2e90 0x7fbf6c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 -- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c0b2e90 msgr2=0x7fbf6c0b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c0b2e90 0x7fbf6c0b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 -- 172.21.15.97:0/346002166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c146c30 con 0x7fbf6c146290 2024-09-10T21:30:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6aa00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf50004810 0x7fbf6c1f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf62000640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c0b2e90 0x7fbf6c0b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf6b400640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf6c146290 0x7fbf6c1466f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbf64003a30 tx=0x7fbf64011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf63400640 1 -- 172.21.15.97:0/346002166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64017070 con 0x7fbf6c146290 2024-09-10T21:30:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf63400640 1 -- 172.21.15.97:0/346002166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf64013af0 con 0x7fbf6c146290 2024-09-10T21:30:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf63400640 1 -- 172.21.15.97:0/346002166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf640123e0 con 0x7fbf6c146290 2024-09-10T21:30:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c0b1df0 con 0x7fbf6c146290 2024-09-10T21:30:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c0b23b0 con 0x7fbf6c146290 2024-09-10T21:30:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.220+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0aec80 con 0x7fbf6c146290 2024-09-10T21:30:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.224+0000 7fbf63400640 1 -- 172.21.15.97:0/346002166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf64010040 con 0x7fbf6c146290 2024-09-10T21:30:32.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.224+0000 7fbf63400640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf480725e0 0x7fbf48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.224+0000 7fbf63400640 1 -- 172.21.15.97:0/346002166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7fbf64096af0 con 0x7fbf6c146290 2024-09-10T21:30:32.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.224+0000 7fbf6aa00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf480725e0 0x7fbf48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.224+0000 7fbf6aa00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf480725e0 0x7fbf48074aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c073910 tx=0x7fbf5c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:32.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.228+0000 7fbf63400640 1 -- 172.21.15.97:0/346002166 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf640622e0 con 0x7fbf6c146290 2024-09-10T21:30:32.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.428+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf6c0cfd60 con 0x7fbf6c146290 2024-09-10T21:30:32.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.428+0000 7fbf63400640 1 -- 172.21.15.97:0/346002166 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3917 (secure 0 0 0) 0x7fbf64061be0 con 0x7fbf6c146290 2024-09-10T21:30:32.434 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:32.434 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":78,"btime":"2024-09-10T21:30:26:074897+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1600816522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1600816522},{"type":"v1","addr":"172.21.15.97:6835","nonce":1600816522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1275667825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":1275667825},{"type":"v1","addr":"172.21.15.194:6837","nonce":1275667825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5933,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/2912272065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2912272065},{"type":"v1","addr":"172.21.15.194:6836","nonce":2912272065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3853880093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3853880093},{"type":"v1","addr":"172.21.15.97:6834","nonce":3853880093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[]} 2024-09-10T21:30:32.434 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 78 2024-09-10T21:30:32.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf480725e0 msgr2=0x7fbf48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf480725e0 0x7fbf48074aa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c073910 tx=0x7fbf5c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf6c146290 msgr2=0x7fbf6c1466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf6c146290 0x7fbf6c1466f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbf64003a30 tx=0x7fbf64011a30 comp rx=0 tx=0).stop 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 shutdown_connections 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf480725e0 0x7fbf48074aa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf6c146290 0x7fbf6c1466f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf6c0b2e90 0x7fbf6c0b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 --2- 172.21.15.97:0/346002166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf50004810 0x7fbf6c1f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 >> 172.21.15.97:0/346002166 conn(0x7fbf6c0d1cd0 msgr2=0x7fbf6c0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 shutdown_connections 2024-09-10T21:30:32.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.432+0000 7fbf71c00640 1 -- 172.21.15.97:0/346002166 wait complete. 2024-09-10T21:30:32.454 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:30:32.454 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:30:32.454 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:30:32.454 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:30:32.455 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:30:32.455 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:30:32.455 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:30:32.455 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:30:32.455 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:30:32.456 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:30:32.456 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:30:32.456 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:30:32.457 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:30:32.457 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:30:32.457 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:30:32.457 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:30:32.466 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:30:32.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:30:32.468 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:30:32.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:30:32.472 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:30:32.472 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:30:32.473 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:30:32.473 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:30:32.511 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:30:32.512 DEBUG:teuthology.parallel:result is None 2024-09-10T21:30:32.515 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:30:32.515 DEBUG:teuthology.parallel:result is None 2024-09-10T21:30:32.517 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:30:32.517 DEBUG:teuthology.parallel:result is None 2024-09-10T21:30:32.519 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:30:32.519 DEBUG:teuthology.parallel:result is None 2024-09-10T21:30:32.519 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:30:32.600 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:30:32.601 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:30:32.610 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:30:32.611 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:30:32.611 INFO:tasks.ceph.mds.d.smithi194.stderr:2024-09-10T21:30:32.610+0000 7f843c000640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-10T21:30:32.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 -- 172.21.15.97:0/1841213123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 msgr2=0x7f3df800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 --2- 172.21.15.97:0/1841213123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3df800b9d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009e00 tx=0x7f3e0801d170 comp rx=0 tx=0).stop 2024-09-10T21:30:32.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 -- 172.21.15.97:0/1841213123 shutdown_connections 2024-09-10T21:30:32.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 --2- 172.21.15.97:0/1841213123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3df800b9d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 -- 172.21.15.97:0/1841213123 >> 172.21.15.97:0/1841213123 conn(0x7f3e100d1a60 msgr2=0x7f3e100d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:32.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 -- 172.21.15.97:0/1841213123 shutdown_connections 2024-09-10T21:30:32.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 -- 172.21.15.97:0/1841213123 wait complete. 2024-09-10T21:30:32.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 Processor -- start 2024-09-10T21:30:32.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.668+0000 7f3e16400640 1 -- start start 2024-09-10T21:30:32.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3e1009f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e100b22a0 0x7f3e100b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e1014f9b0 0x7f3e1014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e100cf840 con 0x7f3df8005220 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3e100cd9e0 con 0x7f3e100b22a0 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3e100cf9c0 con 0x7f3e1014f9b0 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3e1009f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3e1009f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43020/0 (socket says 172.21.15.97:43020) 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 -- 172.21.15.97:0/1503389223 learned_addr learned my addr 172.21.15.97:0/1503389223 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15a00640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e1014f9b0 0x7f3e1014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e0fe00640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e100b22a0 0x7f3e100b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 -- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e1014f9b0 msgr2=0x7f3e1014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e1014f9b0 0x7f3e1014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 -- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e100b22a0 msgr2=0x7f3e100b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:32.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e100b22a0 0x7f3e100b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 -- 172.21.15.97:0/1503389223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e101507f0 con 0x7f3df8005220 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e0fe00640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e100b22a0 0x7f3e100b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e15000640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3e1009f200 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3e04006630 tx=0x7f3e040039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e0cc00640 1 -- 172.21.15.97:0/1503389223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04015070 con 0x7f3df8005220 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e0cc00640 1 -- 172.21.15.97:0/1503389223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e04011920 con 0x7f3df8005220 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e101f2fb0 con 0x7f3df8005220 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e0cc00640 1 -- 172.21.15.97:0/1503389223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04011d70 con 0x7f3df8005220 2024-09-10T21:30:32.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e101f24d0 con 0x7f3df8005220 2024-09-10T21:30:32.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e100b2c60 con 0x7f3df8005220 2024-09-10T21:30:32.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.672+0000 7f3e0cc00640 1 -- 172.21.15.97:0/1503389223 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3e040103c0 con 0x7f3df8005220 2024-09-10T21:30:32.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.676+0000 7f3e0cc00640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3df0072610 0x7f3df0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:32.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.676+0000 7f3e0cc00640 1 -- 172.21.15.97:0/1503389223 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7f3e04094280 con 0x7f3df8005220 2024-09-10T21:30:32.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.676+0000 7f3e0fe00640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3df0072610 0x7f3df0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:32.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.676+0000 7f3e0fe00640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3df0072610 0x7f3df0074ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b3160 tx=0x7f3dfc006b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:32.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.680+0000 7f3e0cc00640 1 -- 172.21.15.97:0/1503389223 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3e0405fa70 con 0x7f3df8005220 2024-09-10T21:30:32.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:32.856+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3e1009f740 con 0x7f3df8005220 2024-09-10T21:30:33.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e0cc00640 1 -- 172.21.15.97:0/1503389223 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7f3e0405f370 con 0x7f3df8005220 2024-09-10T21:30:33.115 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3df0072610 msgr2=0x7f3df0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3df0072610 0x7f3df0074ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b3160 tx=0x7f3dfc006b80 comp rx=0 tx=0).stop 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 msgr2=0x7f3e1009f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3e1009f200 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3e04006630 tx=0x7f3e040039d0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 shutdown_connections 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3df0072610 0x7f3df0074ad0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3e1014f9b0 0x7f3e1014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e100b22a0 0x7f3e100b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 --2- 172.21.15.97:0/1503389223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df8005220 0x7f3e1009f200 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.112+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 >> 172.21.15.97:0/1503389223 conn(0x7f3e100d1a60 msgr2=0x7f3e1015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:33.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.116+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 shutdown_connections 2024-09-10T21:30:33.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.116+0000 7f3e16400640 1 -- 172.21.15.97:0/1503389223 wait complete. 2024-09-10T21:30:33.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:30:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 -- 172.21.15.97:0/1871104610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840af300 msgr2=0x7fd1840af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 --2- 172.21.15.97:0/1871104610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840af300 0x7fd1840af7b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd184073590 tx=0x7fd17c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 -- 172.21.15.97:0/1871104610 shutdown_connections 2024-09-10T21:30:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 --2- 172.21.15.97:0/1871104610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840af300 0x7fd1840af7b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 --2- 172.21.15.97:0/1871104610 >> v2:172.21.15.194:3300/0 conn(0x7fd1840b0d50 0x7fd1840aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 --2- 172.21.15.97:0/1871104610 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1840ce640 0x7fd1840cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 -- 172.21.15.97:0/1871104610 >> 172.21.15.97:0/1871104610 conn(0x7fd1840d1c30 msgr2=0x7fd1840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 -- 172.21.15.97:0/1871104610 shutdown_connections 2024-09-10T21:30:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.276+0000 7fd18ae00640 1 -- 172.21.15.97:0/1871104610 wait complete. 2024-09-10T21:30:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 Processor -- start 2024-09-10T21:30:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 -- start start 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1840af300 0x7fd184157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1840b0d50 0x7fd18415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840ce640 0x7fd1840cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840cd270 con 0x7fd1840ce640 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840cd0f0 con 0x7fd1840af300 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd1840cd3f0 con 0x7fd1840b0d50 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840ce640 0x7fd1840cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840ce640 0x7fd1840cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43046/0 (socket says 172.21.15.97:43046) 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 -- 172.21.15.97:0/2407731780 learned_addr learned my addr 172.21.15.97:0/2407731780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd183e00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1840af300 0x7fd184157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd183400640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1840b0d50 0x7fd18415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 -- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1840b0d50 msgr2=0x7fd18415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1840b0d50 0x7fd18415d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 -- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1840af300 msgr2=0x7fd184157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1840af300 0x7fd184157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 -- 172.21.15.97:0/2407731780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd184164200 con 0x7fd1840ce640 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd183e00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1840af300 0x7fd184157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:33.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd188e00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840ce640 0x7fd1840cb7f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009850 tx=0x7fd17c0a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:33.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd17be00640 1 -- 172.21.15.97:0/2407731780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0b2040 con 0x7fd1840ce640 2024-09-10T21:30:33.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd17be00640 1 -- 172.21.15.97:0/2407731780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd17c0042c0 con 0x7fd1840ce640 2024-09-10T21:30:33.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd17be00640 1 -- 172.21.15.97:0/2407731780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c003860 con 0x7fd1840ce640 2024-09-10T21:30:33.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1841644e0 con 0x7fd1840ce640 2024-09-10T21:30:33.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd184158210 con 0x7fd1840ce640 2024-09-10T21:30:33.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.280+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1840cd950 con 0x7fd1840ce640 2024-09-10T21:30:33.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.284+0000 7fd17be00640 1 -- 172.21.15.97:0/2407731780 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd17c003a00 con 0x7fd1840ce640 2024-09-10T21:30:33.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.284+0000 7fd17be00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1640725e0 0x7fd164074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.284+0000 7fd17be00640 1 -- 172.21.15.97:0/2407731780 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7fd17c12d140 con 0x7fd1840ce640 2024-09-10T21:30:33.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.284+0000 7fd183e00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1640725e0 0x7fd164074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:33.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.284+0000 7fd183e00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1640725e0 0x7fd164074aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd174009e70 tx=0x7fd174009850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:33.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.288+0000 7fd17be00640 1 -- 172.21.15.97:0/2407731780 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd17c0164b0 con 0x7fd1840ce640 2024-09-10T21:30:33.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.512+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd1840aed30 con 0x7fd1840ce640 2024-09-10T21:30:33.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.512+0000 7fd17be00640 1 -- 172.21.15.97:0/2407731780 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fd17c0f8a70 con 0x7fd1840ce640 2024-09-10T21:30:33.517 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:33.517 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:30:33.517 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-09-10T21:30:33.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA==", 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g==", 2024-09-10T21:30:33.518 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw==", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:30:33.519 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA==", 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q==", 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q==", 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg==", 2024-09-10T21:30:33.522 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ==", 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ==", 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw==", 2024-09-10T21:30:33.524 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ==", 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q==", 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q==", 2024-09-10T21:30:33.526 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ==", 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:30:33.527 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag==", 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-09-10T21:30:33.528 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA==", 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmjCI0FRAATFUoCxw0bU6HSEUnlyrVJQ==", 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:30:33.529 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA==", 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.530 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw==", 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-10T21:30:33.531 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:30:33.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1640725e0 msgr2=0x7fd164074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1640725e0 0x7fd164074aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd174009e70 tx=0x7fd174009850 comp rx=0 tx=0).stop 2024-09-10T21:30:33.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840ce640 msgr2=0x7fd1840cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840ce640 0x7fd1840cb7f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009850 tx=0x7fd17c0a56e0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 shutdown_connections 2024-09-10T21:30:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1640725e0 0x7fd164074aa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1840ce640 0x7fd1840cb7f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1840b0d50 0x7fd18415d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 --2- 172.21.15.97:0/2407731780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1840af300 0x7fd184157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 >> 172.21.15.97:0/2407731780 conn(0x7fd1840d1c30 msgr2=0x7fd18406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 shutdown_connections 2024-09-10T21:30:33.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.516+0000 7fd18ae00640 1 -- 172.21.15.97:0/2407731780 wait complete. 2024-09-10T21:30:33.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:30:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 -- 172.21.15.97:0/308154870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400cd6e0 msgr2=0x7fdb400af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 --2- 172.21.15.97:0/308154870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400cd6e0 0x7fdb400af4d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009c80 tx=0x7fdb3801cda0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 -- 172.21.15.97:0/308154870 shutdown_connections 2024-09-10T21:30:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 --2- 172.21.15.97:0/308154870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400cd6e0 0x7fdb400af4d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 --2- 172.21.15.97:0/308154870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb400ce640 0x7fdb400cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 -- 172.21.15.97:0/308154870 >> 172.21.15.97:0/308154870 conn(0x7fdb400d1c30 msgr2=0x7fdb400d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:33.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 -- 172.21.15.97:0/308154870 shutdown_connections 2024-09-10T21:30:33.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 -- 172.21.15.97:0/308154870 wait complete. 2024-09-10T21:30:33.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 Processor -- start 2024-09-10T21:30:33.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.676+0000 7fdb45600640 1 -- start start 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400b0d50 0x7fdb401481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb400cd6e0 0x7fdb400c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb400ce640 0x7fdb400b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40168d80 con 0x7fdb400b0d50 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40168c00 con 0x7fdb400ce640 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdb40168f00 con 0x7fdb400cd6e0 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400b0d50 0x7fdb401481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:33.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400b0d50 0x7fdb401481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43062/0 (socket says 172.21.15.97:43062) 2024-09-10T21:30:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 -- 172.21.15.97:0/2740758987 learned_addr learned my addr 172.21.15.97:0/2740758987 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 -- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb400cd6e0 msgr2=0x7fdb400c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb400cd6e0 0x7fdb400c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 -- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb400ce640 msgr2=0x7fdb400b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb400ce640 0x7fdb400b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 -- 172.21.15.97:0/2740758987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb400b4d90 con 0x7fdb400b0d50 2024-09-10T21:30:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb3ea00640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400b0d50 0x7fdb401481b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000b440 tx=0x7fdb3000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:33.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb36a00640 1 -- 172.21.15.97:0/2740758987 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30015020 con 0x7fdb400b0d50 2024-09-10T21:30:33.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb36a00640 1 -- 172.21.15.97:0/2740758987 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb300119a0 con 0x7fdb400b0d50 2024-09-10T21:30:33.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb36a00640 1 -- 172.21.15.97:0/2740758987 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30011df0 con 0x7fdb400b0d50 2024-09-10T21:30:33.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb400b5070 con 0x7fdb400b0d50 2024-09-10T21:30:33.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb400b5630 con 0x7fdb400b0d50 2024-09-10T21:30:33.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb400cea20 con 0x7fdb400b0d50 2024-09-10T21:30:33.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.680+0000 7fdb36a00640 1 -- 172.21.15.97:0/2740758987 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb3001a400 con 0x7fdb400b0d50 2024-09-10T21:30:33.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.684+0000 7fdb36a00640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb20072610 0x7fdb20074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:33.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.684+0000 7fdb36a00640 1 -- 172.21.15.97:0/2740758987 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7fdb300935a0 con 0x7fdb400b0d50 2024-09-10T21:30:33.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.684+0000 7fdb3e000640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb20072610 0x7fdb20074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:33.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.684+0000 7fdb3e000640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb20072610 0x7fdb20074ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c007fd0 tx=0x7fdb2c0079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.688+0000 7fdb36a00640 1 -- 172.21.15.97:0/2740758987 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb3001f020 con 0x7fdb400b0d50 2024-09-10T21:30:33.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.852+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdb400d22d0 con 0x7fdb400b0d50 2024-09-10T21:30:33.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.852+0000 7fdb36a00640 1 -- 172.21.15.97:0/2740758987 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v90) ==== 84+0+65 (secure 0 0 0) 0x7fdb3000d040 con 0x7fdb400b0d50 2024-09-10T21:30:33.857 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:33.858 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:30:33.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:30:33.858 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-09-10T21:30:33.858 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-09-10T21:30:33.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:30:33.858 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb20072610 msgr2=0x7fdb20074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb20072610 0x7fdb20074ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c007fd0 tx=0x7fdb2c0079a0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400b0d50 msgr2=0x7fdb401481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400b0d50 0x7fdb401481b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000b440 tx=0x7fdb3000b900 comp rx=0 tx=0).stop 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 shutdown_connections 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb20072610 0x7fdb20074ad0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb400ce640 0x7fdb400b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb400cd6e0 0x7fdb400c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 --2- 172.21.15.97:0/2740758987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb400b0d50 0x7fdb401481b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 >> 172.21.15.97:0/2740758987 conn(0x7fdb400d1c30 msgr2=0x7fdb400720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 shutdown_connections 2024-09-10T21:30:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:33.856+0000 7fdb45600640 1 -- 172.21.15.97:0/2740758987 wait complete. 2024-09-10T21:30:33.879 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:30:33.879 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:30:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 -- 172.21.15.97:0/3404352507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb900d0150 msgr2=0x7feb900b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:34.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 --2- 172.21.15.97:0/3404352507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb900d0150 0x7feb900b3070 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009900 tx=0x7feb7c01c850 comp rx=0 tx=0).stop 2024-09-10T21:30:34.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 -- 172.21.15.97:0/3404352507 shutdown_connections 2024-09-10T21:30:34.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 --2- 172.21.15.97:0/3404352507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb900d05e0 0x7feb900afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:34.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 --2- 172.21.15.97:0/3404352507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb900d0150 0x7feb900b3070 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:34.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 -- 172.21.15.97:0/3404352507 >> 172.21.15.97:0/3404352507 conn(0x7feb900d1c60 msgr2=0x7feb900d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:34.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 -- 172.21.15.97:0/3404352507 shutdown_connections 2024-09-10T21:30:34.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.020+0000 7feb97800640 1 -- 172.21.15.97:0/3404352507 wait complete. 2024-09-10T21:30:34.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 Processor -- start 2024-09-10T21:30:34.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 -- start start 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb900d0150 0x7feb90149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900d05e0 0x7feb900c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb9015f350 0x7feb900cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb90168ba0 con 0x7feb9015f350 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb90168a20 con 0x7feb900d05e0 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7feb90168d20 con 0x7feb900d0150 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb9015f350 0x7feb900cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb9015f350 0x7feb900cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43084/0 (socket says 172.21.15.97:43084) 2024-09-10T21:30:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 -- 172.21.15.97:0/730303179 learned_addr learned my addr 172.21.15.97:0/730303179 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb94e00640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb900d0150 0x7feb90149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb8fe00640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900d05e0 0x7feb900c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 -- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb900d0150 msgr2=0x7feb90149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb900d0150 0x7feb90149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 -- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900d05e0 msgr2=0x7feb900c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900d05e0 0x7feb900c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 -- 172.21.15.97:0/730303179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9015bb90 con 0x7feb9015f350 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb95800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb9015f350 0x7feb900cd660 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7feb8800e9f0 tx=0x7feb8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb8cc00640 1 -- 172.21.15.97:0/730303179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8800cd80 con 0x7feb9015f350 2024-09-10T21:30:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb8cc00640 1 -- 172.21.15.97:0/730303179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb88004c20 con 0x7feb9015f350 2024-09-10T21:30:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9015be70 con 0x7feb9015f350 2024-09-10T21:30:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb8cc00640 1 -- 172.21.15.97:0/730303179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88005020 con 0x7feb9015f350 2024-09-10T21:30:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.024+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb900c5cb0 con 0x7feb9015f350 2024-09-10T21:30:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.028+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb900b3070 con 0x7feb9015f350 2024-09-10T21:30:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.028+0000 7feb8cc00640 1 -- 172.21.15.97:0/730303179 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feb8800f040 con 0x7feb9015f350 2024-09-10T21:30:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.028+0000 7feb8cc00640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb700725e0 0x7feb70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.028+0000 7feb8cc00640 1 -- 172.21.15.97:0/730303179 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5249+0+0 (secure 0 0 0) 0x7feb88093650 con 0x7feb9015f350 2024-09-10T21:30:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.028+0000 7feb94e00640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb700725e0 0x7feb70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.028+0000 7feb94e00640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb700725e0 0x7feb70074aa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb840098e0 tx=0x7feb84009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:34.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.032+0000 7feb8cc00640 1 -- 172.21.15.97:0/730303179 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feb88016030 con 0x7feb9015f350 2024-09-10T21:30:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:34.200+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feb900b1b60 con 0x7feb9015f350 2024-09-10T21:30:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.116+0000 7feb8cc00640 1 -- 172.21.15.97:0/730303179 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v91) ==== 138+0+0 (secure 0 0 0) 0x7feb8805e890 con 0x7feb9015f350 2024-09-10T21:30:35.119 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb700725e0 msgr2=0x7feb70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb700725e0 0x7feb70074aa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7feb840098e0 tx=0x7feb84009060 comp rx=0 tx=0).stop 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb9015f350 msgr2=0x7feb900cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb9015f350 0x7feb900cd660 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7feb8800e9f0 tx=0x7feb8800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 shutdown_connections 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feb700725e0 0x7feb70074aa0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb9015f350 0x7feb900cd660 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900d05e0 0x7feb900c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 --2- 172.21.15.97:0/730303179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feb900d0150 0x7feb90149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 >> 172.21.15.97:0/730303179 conn(0x7feb900d1c60 msgr2=0x7feb90071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:35.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 shutdown_connections 2024-09-10T21:30:35.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.120+0000 7feb97800640 1 -- 172.21.15.97:0/730303179 wait complete. 2024-09-10T21:30:35.152 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:30:35.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 -- 172.21.15.97:0/3888837633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00af300 msgr2=0x7f6cf00af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:35.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 --2- 172.21.15.97:0/3888837633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00af300 0x7f6cf00af7b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0074d10 tx=0x7f6ce801cd80 comp rx=0 tx=0).stop 2024-09-10T21:30:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 -- 172.21.15.97:0/3888837633 shutdown_connections 2024-09-10T21:30:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 --2- 172.21.15.97:0/3888837633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00af300 0x7f6cf00af7b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 --2- 172.21.15.97:0/3888837633 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6cf00b0d50 0x7f6cf00aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 --2- 172.21.15.97:0/3888837633 >> v2:172.21.15.194:3300/0 conn(0x7f6cf00ce640 0x7f6cf00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 -- 172.21.15.97:0/3888837633 >> 172.21.15.97:0/3888837633 conn(0x7f6cf00d1c30 msgr2=0x7f6cf00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 -- 172.21.15.97:0/3888837633 shutdown_connections 2024-09-10T21:30:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 -- 172.21.15.97:0/3888837633 wait complete. 2024-09-10T21:30:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 Processor -- start 2024-09-10T21:30:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.300+0000 7f6cf8400640 1 -- start start 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6cf00af300 0x7f6cf0154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00b0d50 0x7f6cf015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cf00ce640 0x7f6cf0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf00cd350 con 0x7f6cf00b0d50 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf00cd1d0 con 0x7f6cf00ce640 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf00cd4d0 con 0x7f6cf00af300 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00b0d50 0x7f6cf015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00b0d50 0x7f6cf015dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43100/0 (socket says 172.21.15.97:43100) 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 -- 172.21.15.97:0/1245264601 learned_addr learned my addr 172.21.15.97:0/1245264601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf6400640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cf00ce640 0x7f6cf0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5a00640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6cf00af300 0x7f6cf0154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 -- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6cf00af300 msgr2=0x7f6cf0154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6cf00af300 0x7f6cf0154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 -- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cf00ce640 msgr2=0x7f6cf0154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cf00ce640 0x7f6cf0154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 -- 172.21.15.97:0/1245264601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf015ce40 con 0x7f6cf00b0d50 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf5000640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00b0d50 0x7f6cf015dee0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b480 tx=0x7f6cdc00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6ced600640 1 -- 172.21.15.97:0/1245264601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc014070 con 0x7f6cf00b0d50 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6ced600640 1 -- 172.21.15.97:0/1245264601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cdc010980 con 0x7f6cf00b0d50 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6ced600640 1 -- 172.21.15.97:0/1245264601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc010dd0 con 0x7f6cf00b0d50 2024-09-10T21:30:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf015d120 con 0x7f6cf00b0d50 2024-09-10T21:30:35.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf015d6e0 con 0x7f6cf00b0d50 2024-09-10T21:30:35.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.304+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf00aedc0 con 0x7f6cf00b0d50 2024-09-10T21:30:35.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.308+0000 7f6ced600640 1 -- 172.21.15.97:0/1245264601 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6cdc00f3c0 con 0x7f6cf00b0d50 2024-09-10T21:30:35.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.308+0000 7f6ced600640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cd0072610 0x7f6cd0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:35.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.308+0000 7f6ced600640 1 -- 172.21.15.97:0/1245264601 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5619+0+0 (secure 0 0 0) 0x7f6cdc0932b0 con 0x7f6cf00b0d50 2024-09-10T21:30:35.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.308+0000 7f6cf5a00640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cd0072610 0x7f6cd0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:35.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.308+0000 7f6cf5a00640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cd0072610 0x7f6cd0074ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4006fd0 tx=0x7f6ce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.312+0000 7f6ced600640 1 -- 172.21.15.97:0/1245264601 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6cdc05e380 con 0x7f6cf00b0d50 2024-09-10T21:30:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:35.480+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6cf0075f90 con 0x7f6cf00b0d50 2024-09-10T21:30:36.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.124+0000 7f6ced600640 1 -- 172.21.15.97:0/1245264601 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v92) ==== 170+0+0 (secure 0 0 0) 0x7f6cdc062db0 con 0x7f6cf00b0d50 2024-09-10T21:30:36.127 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cd0072610 msgr2=0x7f6cd0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cd0072610 0x7f6cd0074ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4006fd0 tx=0x7f6ce4008040 comp rx=0 tx=0).stop 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00b0d50 msgr2=0x7f6cf015dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00b0d50 0x7f6cf015dee0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b480 tx=0x7f6cdc00b940 comp rx=0 tx=0).stop 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 shutdown_connections 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cd0072610 0x7f6cd0074ad0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cf00ce640 0x7f6cf0154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00b0d50 0x7f6cf015dee0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:36.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 --2- 172.21.15.97:0/1245264601 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6cf00af300 0x7f6cf0154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 >> 172.21.15.97:0/1245264601 conn(0x7f6cf00d1c30 msgr2=0x7f6cf00d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 shutdown_connections 2024-09-10T21:30:36.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.128+0000 7f6cf8400640 1 -- 172.21.15.97:0/1245264601 wait complete. 2024-09-10T21:30:36.167 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:30:36.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 -- 172.21.15.97:0/1109157863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7ac004d50 msgr2=0x7fe7ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 --2- 172.21.15.97:0/1109157863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7ac004d50 0x7fe7ac0051d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000c640 tx=0x7fe7c001d320 comp rx=0 tx=0).stop 2024-09-10T21:30:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7c7e00640 1 -- 172.21.15.97:0/1109157863 reap_dead start 2024-09-10T21:30:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 -- 172.21.15.97:0/1109157863 shutdown_connections 2024-09-10T21:30:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 -- 172.21.15.97:0/1109157863 >> 172.21.15.97:0/1109157863 conn(0x7fe7c80b1780 msgr2=0x7fe7c80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 -- 172.21.15.97:0/1109157863 shutdown_connections 2024-09-10T21:30:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 -- 172.21.15.97:0/1109157863 wait complete. 2024-09-10T21:30:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 Processor -- start 2024-09-10T21:30:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.312+0000 7fe7ce600640 1 -- start start 2024-09-10T21:30:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7ac004d50 0x7fe7c813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7c81f6460 0x7fe7c80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7c80cecd0 0x7fe7c815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81444d0 con 0x7fe7c80cecd0 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8070670 con 0x7fe7ac004d50 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c80b2a80 con 0x7fe7c81f6460 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7c80cecd0 0x7fe7c815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7c80cecd0 0x7fe7c815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43106/0 (socket says 172.21.15.97:43106) 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 -- 172.21.15.97:0/3515047674 learned_addr learned my addr 172.21.15.97:0/3515047674 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7bea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7c81f6460 0x7fe7c80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7ac004d50 0x7fe7c813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 -- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7c81f6460 msgr2=0x7fe7c80ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7c81f6460 0x7fe7c80ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 -- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7ac004d50 msgr2=0x7fe7c813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7ac004d50 0x7fe7c813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 -- 172.21.15.97:0/3515047674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c80cf1e0 con 0x7fe7c80cecd0 2024-09-10T21:30:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7c7e00640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7c80cecd0 0x7fe7c815a300 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe7c001d800 tx=0x7fe7c0001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7bfe00640 1 -- 172.21.15.97:0/3515047674 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00bc070 con 0x7fe7c80cecd0 2024-09-10T21:30:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7bfe00640 1 -- 172.21.15.97:0/3515047674 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7c0003bc0 con 0x7fe7c80cecd0 2024-09-10T21:30:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7bfe00640 1 -- 172.21.15.97:0/3515047674 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00b2040 con 0x7fe7c80cecd0 2024-09-10T21:30:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c81f5980 con 0x7fe7c80cecd0 2024-09-10T21:30:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81f5f40 con 0x7fe7c80cecd0 2024-09-10T21:30:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c80b2fe0 con 0x7fe7c80cecd0 2024-09-10T21:30:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.316+0000 7fe7bfe00640 1 -- 172.21.15.97:0/3515047674 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe7c00b7410 con 0x7fe7c80cecd0 2024-09-10T21:30:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.320+0000 7fe7bfe00640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7a40725e0 0x7fe7a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.320+0000 7fe7bfe00640 1 -- 172.21.15.97:0/3515047674 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6001+0+0 (secure 0 0 0) 0x7fe7c012d480 con 0x7fe7c80cecd0 2024-09-10T21:30:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.320+0000 7fe7c7400640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7a40725e0 0x7fe7a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.320+0000 7fe7c7400640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7a40725e0 0x7fe7a4074aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009c60 tx=0x7fe7b80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:36.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.324+0000 7fe7bfe00640 1 -- 172.21.15.97:0/3515047674 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe7c00170d0 con 0x7fe7c80cecd0 2024-09-10T21:30:36.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:36.520+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe7c81f6110 con 0x7fe7c80cecd0 2024-09-10T21:30:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 13 and data pool 14 2024-09-10T21:30:37.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.136+0000 7fe7bfe00640 1 -- 172.21.15.97:0/3515047674 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v81) ==== 174+0+0 (secure 0 0 0) 0x7fe7c00f8340 con 0x7fe7c80cecd0 2024-09-10T21:30:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.140+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7a40725e0 msgr2=0x7fe7a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.140+0000 7fe7ce600640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7a40725e0 0x7fe7a4074aa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009c60 tx=0x7fe7b80091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.140+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7c80cecd0 msgr2=0x7fe7c815a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.140+0000 7fe7ce600640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7c80cecd0 0x7fe7c815a300 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe7c001d800 tx=0x7fe7c0001cd0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 shutdown_connections 2024-09-10T21:30:37.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7a40725e0 0x7fe7a4074aa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7c80cecd0 0x7fe7c815a300 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7c81f6460 0x7fe7c80ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 --2- 172.21.15.97:0/3515047674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7ac004d50 0x7fe7c813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 >> 172.21.15.97:0/3515047674 conn(0x7fe7c80b1780 msgr2=0x7fe7c8142230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 shutdown_connections 2024-09-10T21:30:37.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.144+0000 7fe7ce600640 1 -- 172.21.15.97:0/3515047674 wait complete. 2024-09-10T21:30:37.185 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:30:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- 172.21.15.97:0/157439894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b29b0 msgr2=0x7f74b80b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 --2- 172.21.15.97:0/157439894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b29b0 0x7f74b80b1f50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f74b8073490 tx=0x7f74a401c770 comp rx=0 tx=0).stop 2024-09-10T21:30:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- 172.21.15.97:0/157439894 shutdown_connections 2024-09-10T21:30:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 --2- 172.21.15.97:0/157439894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b29b0 0x7f74b80b1f50 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 --2- 172.21.15.97:0/157439894 >> v2:172.21.15.194:3300/0 conn(0x7f74b8161290 0x7f74b80b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- 172.21.15.97:0/157439894 >> 172.21.15.97:0/157439894 conn(0x7f74b80d1a60 msgr2=0x7f74b80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:37.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- 172.21.15.97:0/157439894 shutdown_connections 2024-09-10T21:30:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- 172.21.15.97:0/157439894 wait complete. 2024-09-10T21:30:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 Processor -- start 2024-09-10T21:30:37.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- start start 2024-09-10T21:30:37.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b2490 0x7f74b8150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b80b29b0 0x7f74b815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74b8161290 0x7f74b815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74b816ab50 con 0x7f74b80b2490 2024-09-10T21:30:37.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f74b806e840 con 0x7f74b80b29b0 2024-09-10T21:30:37.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b7e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f74b816acd0 con 0x7f74b8161290 2024-09-10T21:30:37.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.324+0000 7f74b6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b2490 0x7f74b8150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b2490 0x7f74b8150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48518/0 (socket says 172.21.15.97:48518) 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 -- 172.21.15.97:0/1988791983 learned_addr learned my addr 172.21.15.97:0/1988791983 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6000640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b80b29b0 0x7f74b815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b7400640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74b8161290 0x7f74b815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 -- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74b8161290 msgr2=0x7f74b815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74b8161290 0x7f74b815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 -- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b80b29b0 msgr2=0x7f74b815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b80b29b0 0x7f74b815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 -- 172.21.15.97:0/1988791983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b80c57e0 con 0x7f74b80b2490 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b7400640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74b8161290 0x7f74b815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:37.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b6a00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b2490 0x7f74b8150ec0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f74a800cce0 tx=0x7f74a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74aea00640 1 -- 172.21.15.97:0/1988791983 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a800f450 con 0x7f74b80b2490 2024-09-10T21:30:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74aea00640 1 -- 172.21.15.97:0/1988791983 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74a800f6e0 con 0x7f74b80b2490 2024-09-10T21:30:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74aea00640 1 -- 172.21.15.97:0/1988791983 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a800fb10 con 0x7f74b80b2490 2024-09-10T21:30:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b80c5ac0 con 0x7f74b80b2490 2024-09-10T21:30:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b80c6080 con 0x7f74b80b2490 2024-09-10T21:30:37.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b80b1f50 con 0x7f74b80b2490 2024-09-10T21:30:37.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.328+0000 7f74aea00640 1 -- 172.21.15.97:0/1988791983 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f74a8020030 con 0x7f74b80b2490 2024-09-10T21:30:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.332+0000 7f74aea00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74980725e0 0x7f7498074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.332+0000 7f74aea00640 1 -- 172.21.15.97:0/1988791983 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7f74a8093bb0 con 0x7f74b80b2490 2024-09-10T21:30:37.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.332+0000 7f74b6000640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74980725e0 0x7f7498074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.332+0000 7f74b6000640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74980725e0 0x7f7498074aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74a40af000 tx=0x7f74a401afb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:37.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.336+0000 7f74aea00640 1 -- 172.21.15.97:0/1988791983 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f74a8014030 con 0x7f74b80b2490 2024-09-10T21:30:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.500+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74b8071f20 con 0x7f74b80b2490 2024-09-10T21:30:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.500+0000 7f74aea00640 1 -- 172.21.15.97:0/1988791983 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17030 (secure 0 0 0) 0x7f74a805f0f0 con 0x7f74b80b2490 2024-09-10T21:30:37.505 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:37.505 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":93,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:30:36.527212+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:30:34.216652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T21:30:35.493513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:30:37.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74980725e0 msgr2=0x7f7498074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74980725e0 0x7f7498074aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74a40af000 tx=0x7f74a401afb0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b2490 msgr2=0x7f74b8150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b2490 0x7f74b8150ec0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f74a800cce0 tx=0x7f74a800a620 comp rx=0 tx=0).stop 2024-09-10T21:30:37.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 shutdown_connections 2024-09-10T21:30:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74980725e0 0x7f7498074aa0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74b8161290 0x7f74b815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b80b29b0 0x7f74b815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 --2- 172.21.15.97:0/1988791983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74b80b2490 0x7f74b8150ec0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 >> 172.21.15.97:0/1988791983 conn(0x7f74b80d1a60 msgr2=0x7f74b80b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 shutdown_connections 2024-09-10T21:30:37.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.504+0000 7f74b7e00640 1 -- 172.21.15.97:0/1988791983 wait complete. 2024-09-10T21:30:37.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:30:37.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 -- 172.21.15.97:0/4248703265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 msgr2=0x7febd80b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 --2- 172.21.15.97:0/4248703265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd80b2a80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7febd815e650 tx=0x7febc801cd80 comp rx=0 tx=0).stop 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 -- 172.21.15.97:0/4248703265 shutdown_connections 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 --2- 172.21.15.97:0/4248703265 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febd80b4660 0x7febd806fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 --2- 172.21.15.97:0/4248703265 >> v2:172.21.15.194:3300/0 conn(0x7febd8144690 0x7febd80b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 --2- 172.21.15.97:0/4248703265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd80b2a80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 -- 172.21.15.97:0/4248703265 >> 172.21.15.97:0/4248703265 conn(0x7febd80b1780 msgr2=0x7febd80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 -- 172.21.15.97:0/4248703265 shutdown_connections 2024-09-10T21:30:37.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.664+0000 7febdd000640 1 -- 172.21.15.97:0/4248703265 wait complete. 2024-09-10T21:30:37.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 Processor -- start 2024-09-10T21:30:37.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 -- start start 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febd80b4660 0x7febd813efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd8159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febd8144690 0x7febd80cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7febd815edb0 con 0x7febd8143f90 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febd815ec30 con 0x7febd80b4660 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7febd815ef30 con 0x7febd8144690 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd8159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd8159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48524/0 (socket says 172.21.15.97:48524) 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 -- 172.21.15.97:0/3042065339 learned_addr learned my addr 172.21.15.97:0/3042065339 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 -- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febd8144690 msgr2=0x7febd80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5e00640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febd80b4660 0x7febd813efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd6800640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febd8144690 0x7febd80cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febd8144690 0x7febd80cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 -- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febd80b4660 msgr2=0x7febd813efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febd80b4660 0x7febd813efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 -- 172.21.15.97:0/3042065339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd813f510 con 0x7febd8143f90 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5e00640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febd80b4660 0x7febd813efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd6800640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febd8144690 0x7febd80cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febd5400640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd8159d30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7febc400c9f0 tx=0x7febc400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febce000640 1 -- 172.21.15.97:0/3042065339 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4007bb0 con 0x7febd8143f90 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febce000640 1 -- 172.21.15.97:0/3042065339 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc4007e40 con 0x7febd8143f90 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febce000640 1 -- 172.21.15.97:0/3042065339 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc400f690 con 0x7febd8143f90 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.668+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd813f7e0 con 0x7febd8143f90 2024-09-10T21:30:37.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.672+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd813fda0 con 0x7febd8143f90 2024-09-10T21:30:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.672+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd806a010 con 0x7febd8143f90 2024-09-10T21:30:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.672+0000 7febce000640 1 -- 172.21.15.97:0/3042065339 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7febc4020030 con 0x7febd8143f90 2024-09-10T21:30:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.672+0000 7febce000640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febb80725e0 0x7febb8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.672+0000 7febce000640 1 -- 172.21.15.97:0/3042065339 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7febc4093870 con 0x7febd8143f90 2024-09-10T21:30:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.672+0000 7febd5e00640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febb80725e0 0x7febb8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.676+0000 7febd5e00640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febb80725e0 0x7febb8074aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7febc80af000 tx=0x7febc801b5c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:37.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.676+0000 7febce000640 1 -- 172.21.15.97:0/3042065339 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7febc4014030 con 0x7febd8143f90 2024-09-10T21:30:37.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.848+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febd815f4e0 con 0x7febd8143f90 2024-09-10T21:30:37.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.848+0000 7febce000640 1 -- 172.21.15.97:0/3042065339 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17030 (secure 0 0 0) 0x7febc405edb0 con 0x7febd8143f90 2024-09-10T21:30:37.853 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:37.853 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":93,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:30:36.527212+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:30:34.216652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T21:30:35.493513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:30:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febb80725e0 msgr2=0x7febb8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febb80725e0 0x7febb8074aa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7febc80af000 tx=0x7febc801b5c0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 msgr2=0x7febd8159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd8159d30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7febc400c9f0 tx=0x7febc400ceb0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 shutdown_connections 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7febb80725e0 0x7febb8074aa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7febd8144690 0x7febd80cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febd8143f90 0x7febd8159d30 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 --2- 172.21.15.97:0/3042065339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febd80b4660 0x7febd813efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 >> 172.21.15.97:0/3042065339 conn(0x7febd80b1780 msgr2=0x7febd80730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 shutdown_connections 2024-09-10T21:30:37.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:37.852+0000 7febdd000640 1 -- 172.21.15.97:0/3042065339 wait complete. 2024-09-10T21:30:37.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:30:38.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 -- 172.21.15.97:0/1951354494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7640b48c0 msgr2=0x7fd76415b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:38.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 --2- 172.21.15.97:0/1951354494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7640b48c0 0x7fd76415b030 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd750009900 tx=0x7fd75001c850 comp rx=0 tx=0).stop 2024-09-10T21:30:38.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 -- 172.21.15.97:0/1951354494 shutdown_connections 2024-09-10T21:30:38.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 --2- 172.21.15.97:0/1951354494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7640b48c0 0x7fd76415b030 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:38.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 --2- 172.21.15.97:0/1951354494 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd76415f150 0x7fd7640b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:38.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 -- 172.21.15.97:0/1951354494 >> 172.21.15.97:0/1951354494 conn(0x7fd7640b28c0 msgr2=0x7fd7640b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:38.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 -- 172.21.15.97:0/1951354494 shutdown_connections 2024-09-10T21:30:38.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 -- 172.21.15.97:0/1951354494 wait complete. 2024-09-10T21:30:38.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.016+0000 7fd76a000640 1 Processor -- start 2024-09-10T21:30:38.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 -- start start 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76415b570 0x7fd7640ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76415f150 0x7fd76413f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7641552e0 0x7fd76406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd764167af0 con 0x7fd76415f150 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd76415ea70 con 0x7fd76415b570 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd764167c70 con 0x7fd7641552e0 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76415f150 0x7fd76413f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76415f150 0x7fd76413f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48540/0 (socket says 172.21.15.97:48540) 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 -- 172.21.15.97:0/3386842692 learned_addr learned my addr 172.21.15.97:0/3386842692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd763400640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76415b570 0x7fd7640ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:38.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd763e00640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7641552e0 0x7fd76406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 -- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7641552e0 msgr2=0x7fd76406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7641552e0 0x7fd76406cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 -- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76415b570 msgr2=0x7fd7640ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76415b570 0x7fd7640ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 -- 172.21.15.97:0/3386842692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7641faeb0 con 0x7fd76415f150 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd762a00640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76415f150 0x7fd76413f170 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd750009850 tx=0x7fd7500a65b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd75b400640 1 -- 172.21.15.97:0/3386842692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500ac070 con 0x7fd76415f150 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd75b400640 1 -- 172.21.15.97:0/3386842692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd750005140 con 0x7fd76415f150 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7641fb190 con 0x7fd76415f150 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd75b400640 1 -- 172.21.15.97:0/3386842692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750005570 con 0x7fd76415f150 2024-09-10T21:30:38.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7641fb750 con 0x7fd76415f150 2024-09-10T21:30:38.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.020+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7640ce7b0 con 0x7fd76415f150 2024-09-10T21:30:38.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.024+0000 7fd75b400640 1 -- 172.21.15.97:0/3386842692 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd7500046d0 con 0x7fd76415f150 2024-09-10T21:30:38.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.024+0000 7fd75b400640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd744072610 0x7fd744074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:38.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.024+0000 7fd75b400640 1 -- 172.21.15.97:0/3386842692 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6101+0+0 (secure 0 0 0) 0x7fd75012daa0 con 0x7fd76415f150 2024-09-10T21:30:38.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.024+0000 7fd763400640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd744072610 0x7fd744074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:38.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.024+0000 7fd763400640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd744072610 0x7fd744074ad0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd75400ab40 tx=0x7fd75400a250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:38.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.028+0000 7fd75b400640 1 -- 172.21.15.97:0/3386842692 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd7500164d0 con 0x7fd76415f150 2024-09-10T21:30:38.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:38.232+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd764157cb0 con 0x7fd76415f150 2024-09-10T21:30:39.119 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:30:39.116+0000 7faf71e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:30:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd75b400640 1 -- 172.21.15.97:0/3386842692 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) ==== 118+0+0 (secure 0 0 0) 0x7fd7500f8990 con 0x7fd76415f150 2024-09-10T21:30:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd744072610 msgr2=0x7fd744074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd744072610 0x7fd744074ad0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd75400ab40 tx=0x7fd75400a250 comp rx=0 tx=0).stop 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76415f150 msgr2=0x7fd76413f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76415f150 0x7fd76413f170 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd750009850 tx=0x7fd7500a65b0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 shutdown_connections 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd744072610 0x7fd744074ad0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7641552e0 0x7fd76406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd76415f150 0x7fd76413f170 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 --2- 172.21.15.97:0/3386842692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd76415b570 0x7fd7640ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 >> 172.21.15.97:0/3386842692 conn(0x7fd7640b28c0 msgr2=0x7fd7641584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 shutdown_connections 2024-09-10T21:30:39.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.148+0000 7fd76a000640 1 -- 172.21.15.97:0/3386842692 wait complete. 2024-09-10T21:30:39.177 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:30:39.177 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:30:39.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:39.296 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:39.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- 172.21.15.97:0/33006528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 msgr2=0x7f481006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:39.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 --2- 172.21.15.97:0/33006528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f481006cb40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c80 tx=0x7f480001cce0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- 172.21.15.97:0/33006528 shutdown_connections 2024-09-10T21:30:39.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 --2- 172.21.15.97:0/33006528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f481006cb40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- 172.21.15.97:0/33006528 >> 172.21.15.97:0/33006528 conn(0x7f48100cd680 msgr2=0x7f48100cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:39.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- 172.21.15.97:0/33006528 shutdown_connections 2024-09-10T21:30:39.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- 172.21.15.97:0/33006528 wait complete. 2024-09-10T21:30:39.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 Processor -- start 2024-09-10T21:30:39.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- start start 2024-09-10T21:30:39.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48100b00a0 0x7f481014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:39.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f48100ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:39.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481015bd90 0x7f48100ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:39.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f481016a500 con 0x7f481015b9b0 2024-09-10T21:30:39.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f481016a380 con 0x7f48100b00a0 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4817000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f481016a680 con 0x7f481015bd90 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f480f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f48100ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f480f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f48100ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48562/0 (socket says 172.21.15.97:48562) 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f480f400640 1 -- 172.21.15.97:0/2592248153 learned_addr learned my addr 172.21.15.97:0/2592248153 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f480fe00640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48100b00a0 0x7f481014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.324+0000 7f4815000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481015bd90 0x7f48100ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f480f400640 1 -- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481015bd90 msgr2=0x7f48100ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f480f400640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481015bd90 0x7f48100ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f480f400640 1 -- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48100b00a0 msgr2=0x7f481014a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f480f400640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48100b00a0 0x7f481014a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f480f400640 1 -- 172.21.15.97:0/2592248153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48101fb2f0 con 0x7f481015b9b0 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4815000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481015bd90 0x7f48100ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:39.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f480f400640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f48100ce960 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00c3c0 tx=0x7f47fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:39.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4807e00640 1 -- 172.21.15.97:0/2592248153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc017070 con 0x7f481015b9b0 2024-09-10T21:30:39.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4807e00640 1 -- 172.21.15.97:0/2592248153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47fc003600 con 0x7f481015b9b0 2024-09-10T21:30:39.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4810161d80 con 0x7f481015b9b0 2024-09-10T21:30:39.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4807e00640 1 -- 172.21.15.97:0/2592248153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc003a50 con 0x7f481015b9b0 2024-09-10T21:30:39.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48101fb890 con 0x7f481015b9b0 2024-09-10T21:30:39.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481006cb40 con 0x7f481015b9b0 2024-09-10T21:30:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4807e00640 1 -- 172.21.15.97:0/2592248153 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f47fc003bf0 con 0x7f481015b9b0 2024-09-10T21:30:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.328+0000 7f4807e00640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.332+0000 7f4807e00640 1 -- 172.21.15.97:0/2592248153 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f47fc092900 con 0x7f481015b9b0 2024-09-10T21:30:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.332+0000 7f480fe00640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.332+0000 7f480fe00640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f48000af000 tx=0x7f480001b520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:39.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.332+0000 7f4807e00640 1 -- 172.21.15.97:0/2592248153 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47fc05d760 con 0x7f481015b9b0 2024-09-10T21:30:39.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:39.532+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4810159510 con 0x7f481015b9b0 2024-09-10T21:30:40.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.148+0000 7f4807e00640 1 -- 172.21.15.97:0/2592248153 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) ==== 122+0+0 (secure 0 0 0) 0x7f47fc062190 con 0x7f481015b9b0 2024-09-10T21:30:40.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 msgr2=0x7f47f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f48000af000 tx=0x7f480001b520 comp rx=0 tx=0).stop 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 msgr2=0x7f48100ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f48100ce960 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00c3c0 tx=0x7f47fc00c880 comp rx=0 tx=0).stop 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 shutdown_connections 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f481015bd90 0x7f48100ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481015b9b0 0x7f48100ce960 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 --2- 172.21.15.97:0/2592248153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48100b00a0 0x7f481014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 >> 172.21.15.97:0/2592248153 conn(0x7f48100cd680 msgr2=0x7f4810159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 shutdown_connections 2024-09-10T21:30:40.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.152+0000 7f4817000640 1 -- 172.21.15.97:0/2592248153 wait complete. 2024-09-10T21:30:40.178 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:30:40.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.316+0000 7fa71ec00640 1 -- 172.21.15.97:0/3941654655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815b570 msgr2=0x7fa7180cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.316+0000 7fa71ec00640 1 --2- 172.21.15.97:0/3941654655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815b570 0x7fa7180cd580 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa71806f5c0 tx=0x7fa71001ccf0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.316+0000 7fa71ec00640 1 -- 172.21.15.97:0/3941654655 shutdown_connections 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.316+0000 7fa71ec00640 1 --2- 172.21.15.97:0/3941654655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815b570 0x7fa7180cd580 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.316+0000 7fa71ec00640 1 --2- 172.21.15.97:0/3941654655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa7180b48c0 0x7fa71815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.316+0000 7fa71ec00640 1 --2- 172.21.15.97:0/3941654655 >> v2:172.21.15.194:3300/0 conn(0x7fa71815f150 0x7fa7180b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.316+0000 7fa71ec00640 1 -- 172.21.15.97:0/3941654655 >> 172.21.15.97:0/3941654655 conn(0x7fa7180b28c0 msgr2=0x7fa7180b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 -- 172.21.15.97:0/3941654655 shutdown_connections 2024-09-10T21:30:40.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 -- 172.21.15.97:0/3941654655 wait complete. 2024-09-10T21:30:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 Processor -- start 2024-09-10T21:30:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 -- start start 2024-09-10T21:30:40.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa71815b570 0x7fa7180ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815f150 0x7fa718152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa718155fa0 0x7fa718164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7180cf7e0 con 0x7fa71815f150 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa7180cf660 con 0x7fa718155fa0 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71ec00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa7180cf960 con 0x7fa71815b570 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815f150 0x7fa718152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815f150 0x7fa718152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48584/0 (socket says 172.21.15.97:48584) 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 -- 172.21.15.97:0/669225685 learned_addr learned my addr 172.21.15.97:0/669225685 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa717e00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa71815b570 0x7fa7180ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa71cc00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa718155fa0 0x7fa718164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 -- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa71815b570 msgr2=0x7fa7180ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa71815b570 0x7fa7180ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 -- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa718155fa0 msgr2=0x7fa718164ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa718155fa0 0x7fa718164ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 -- 172.21.15.97:0/669225685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7180ae350 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.320+0000 7fa70f400640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815f150 0x7fa718152650 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa70000d750 tx=0x7fa70000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa714c00640 1 -- 172.21.15.97:0/669225685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700017070 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa714c00640 1 -- 172.21.15.97:0/669225685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa700009d20 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa714c00640 1 -- 172.21.15.97:0/669225685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70000de60 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7181fbd30 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181fc2f0 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7180cd580 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa714c00640 1 -- 172.21.15.97:0/669225685 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa7000106c0 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa714c00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6f40725e0 0x7fa6f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa714c00640 1 -- 172.21.15.97:0/669225685 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fa700093740 con 0x7fa71815f150 2024-09-10T21:30:40.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.324+0000 7fa717e00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6f40725e0 0x7fa6f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.328+0000 7fa717e00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6f40725e0 0x7fa6f4074aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa70800bb50 tx=0x7fa70800b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.328+0000 7fa714c00640 1 -- 172.21.15.97:0/669225685 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa70005e630 con 0x7fa71815f150 2024-09-10T21:30:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7180ce7b0 con 0x7fa71815f150 2024-09-10T21:30:40.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa714c00640 1 -- 172.21.15.97:0/669225685 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5225 (secure 0 0 0) 0x7fa700063060 con 0x7fa71815f150 2024-09-10T21:30:40.637 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:40.637 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"btime":"2024-09-10T21:30:40:152516+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:30:39.538691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/873879253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":873879253},{"type":"v1","addr":"172.21.15.97:6835","nonce":873879253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6957,"qdb_cluster":[6957]},"id":6}]} 2024-09-10T21:30:40.638 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 85 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6f40725e0 msgr2=0x7fa6f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6f40725e0 0x7fa6f4074aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa70800bb50 tx=0x7fa70800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815f150 msgr2=0x7fa718152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815f150 0x7fa718152650 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa70000d750 tx=0x7fa70000dc10 comp rx=0 tx=0).stop 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 shutdown_connections 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6f40725e0 0x7fa6f4074aa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa718155fa0 0x7fa718164ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa71815f150 0x7fa718152650 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 --2- 172.21.15.97:0/669225685 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa71815b570 0x7fa7180ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 >> 172.21.15.97:0/669225685 conn(0x7fa7180b28c0 msgr2=0x7fa718159390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.528+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 shutdown_connections 2024-09-10T21:30:40.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.532+0000 7fa71ec00640 1 -- 172.21.15.97:0/669225685 wait complete. 2024-09-10T21:30:40.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:30:40.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.776+0000 7f67b3200640 1 -- 172.21.15.97:0/3966081955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 msgr2=0x7f67ac13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.776+0000 7f67b3200640 1 --2- 172.21.15.97:0/3966081955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac13f9b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f67ac074800 tx=0x7f67a401c8a0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- 172.21.15.97:0/3966081955 shutdown_connections 2024-09-10T21:30:40.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 --2- 172.21.15.97:0/3966081955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac13f9b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 --2- 172.21.15.97:0/3966081955 >> v2:172.21.15.194:3300/0 conn(0x7f67ac141d20 0x7f67ac15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- 172.21.15.97:0/3966081955 >> 172.21.15.97:0/3966081955 conn(0x7f67ac0b1780 msgr2=0x7f67ac0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:40.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- 172.21.15.97:0/3966081955 shutdown_connections 2024-09-10T21:30:40.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- 172.21.15.97:0/3966081955 wait complete. 2024-09-10T21:30:40.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 Processor -- start 2024-09-10T21:30:40.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- start start 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67ac0ce5f0 0x7f67ac136e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67ac141d20 0x7f67ac13bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac168a70 con 0x7f67ac15fc10 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac140c80 con 0x7f67ac141d20 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f67ac168bf0 con 0x7f67ac0ce5f0 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac073e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48608/0 (socket says 172.21.15.97:48608) 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 -- 172.21.15.97:0/3038345111 learned_addr learned my addr 172.21.15.97:0/3038345111 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:40.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67ab400640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67ac141d20 0x7f67ac13bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67abe00640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67ac0ce5f0 0x7f67ac136e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 -- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67ac0ce5f0 msgr2=0x7f67ac136e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67ac0ce5f0 0x7f67ac136e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 -- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67ac141d20 msgr2=0x7f67ac13bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67ac141d20 0x7f67ac13bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 -- 172.21.15.97:0/3038345111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac165f50 con 0x7f67ac15fc10 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67ab400640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67ac141d20 0x7f67ac13bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b1200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac073e80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f67a40194b0 tx=0x7f67a40a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67a3e00640 1 -- 172.21.15.97:0/3038345111 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40b2040 con 0x7f67ac15fc10 2024-09-10T21:30:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67a3e00640 1 -- 172.21.15.97:0/3038345111 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67a40042c0 con 0x7f67ac15fc10 2024-09-10T21:30:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67a3e00640 1 -- 172.21.15.97:0/3038345111 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4005030 con 0x7f67ac15fc10 2024-09-10T21:30:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.780+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac13a320 con 0x7f67ac15fc10 2024-09-10T21:30:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.784+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac073680 con 0x7f67ac15fc10 2024-09-10T21:30:40.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.784+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ac15f6d0 con 0x7f67ac15fc10 2024-09-10T21:30:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.784+0000 7f67a3e00640 1 -- 172.21.15.97:0/3038345111 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f67a40051f0 con 0x7f67ac15fc10 2024-09-10T21:30:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.784+0000 7f67a3e00640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f678c0725e0 0x7f678c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.784+0000 7f67a3e00640 1 -- 172.21.15.97:0/3038345111 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f67a412d420 con 0x7f67ac15fc10 2024-09-10T21:30:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.784+0000 7f67abe00640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f678c0725e0 0x7f678c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.784+0000 7f67abe00640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f678c0725e0 0x7f678c074aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f679c001d90 tx=0x7f679c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:40.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.788+0000 7f67a3e00640 1 -- 172.21.15.97:0/3038345111 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f67a40c4020 con 0x7f67ac15fc10 2024-09-10T21:30:40.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.956+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67ac1599d0 con 0x7f67ac15fc10 2024-09-10T21:30:40.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.956+0000 7f67a3e00640 1 -- 172.21.15.97:0/3038345111 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17017 (secure 0 0 0) 0x7f67a4016490 con 0x7f67ac15fc10 2024-09-10T21:30:40.960 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:40.960 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:30:38.139730+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:30:34.216652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T21:30:35.493513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f678c0725e0 msgr2=0x7f678c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f678c0725e0 0x7f678c074aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f679c001d90 tx=0x7f679c009040 comp rx=0 tx=0).stop 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 msgr2=0x7f67ac073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac073e80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f67a40194b0 tx=0x7f67a40a57b0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 shutdown_connections 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f678c0725e0 0x7f678c074aa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67ac15fc10 0x7f67ac073e80 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67ac141d20 0x7f67ac13bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 --2- 172.21.15.97:0/3038345111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f67ac0ce5f0 0x7f67ac136e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:40.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 >> 172.21.15.97:0/3038345111 conn(0x7f67ac0b1780 msgr2=0x7f67ac0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:40.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 shutdown_connections 2024-09-10T21:30:40.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:40.960+0000 7f67b3200640 1 -- 172.21.15.97:0/3038345111 wait complete. 2024-09-10T21:30:40.982 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:30:40.982 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:30:41.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 -- 172.21.15.97:0/1121403140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f601015fde0 msgr2=0x7f6010166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 --2- 172.21.15.97:0/1121403140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f601015fde0 0x7f6010166850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6008009bc0 tx=0x7f600801cac0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 -- 172.21.15.97:0/1121403140 shutdown_connections 2024-09-10T21:30:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 --2- 172.21.15.97:0/1121403140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f601015fde0 0x7f6010166850 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 -- 172.21.15.97:0/1121403140 >> 172.21.15.97:0/1121403140 conn(0x7f60100b1780 msgr2=0x7f60100b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 -- 172.21.15.97:0/1121403140 shutdown_connections 2024-09-10T21:30:41.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 -- 172.21.15.97:0/1121403140 wait complete. 2024-09-10T21:30:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 Processor -- start 2024-09-10T21:30:41.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.120+0000 7f6016800640 1 -- start start 2024-09-10T21:30:41.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60100ce5f0 0x7f601013e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6010141d20 0x7f601013d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601015fde0 0x7f6010144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f601016a130 con 0x7f601015fde0 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6010169fb0 con 0x7f60100ce5f0 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f601016a2b0 con 0x7f6010141d20 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601015fde0 0x7f6010144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601015fde0 0x7f6010144a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48622/0 (socket says 172.21.15.97:48622) 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 -- 172.21.15.97:0/1841676753 learned_addr learned my addr 172.21.15.97:0/1841676753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600f400640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60100ce5f0 0x7f601013e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6006a00640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6010141d20 0x7f601013d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 -- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6010141d20 msgr2=0x7f601013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6010141d20 0x7f601013d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 -- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60100ce5f0 msgr2=0x7f601013e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60100ce5f0 0x7f601013e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 -- 172.21.15.97:0/1841676753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601013d130 con 0x7f601015fde0 2024-09-10T21:30:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600f400640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60100ce5f0 0x7f601013e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6006a00640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6010141d20 0x7f601013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:41.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f600fe00640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601015fde0 0x7f6010144a50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f60080b0040 tx=0x7f6008005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6007e00640 1 -- 172.21.15.97:0/1841676753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60080b2040 con 0x7f601015fde0 2024-09-10T21:30:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6007e00640 1 -- 172.21.15.97:0/1841676753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60080bb070 con 0x7f601015fde0 2024-09-10T21:30:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601013ea60 con 0x7f601015fde0 2024-09-10T21:30:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6007e00640 1 -- 172.21.15.97:0/1841676753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008004040 con 0x7f601015fde0 2024-09-10T21:30:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601013be00 con 0x7f601015fde0 2024-09-10T21:30:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60100ce9d0 con 0x7f601015fde0 2024-09-10T21:30:41.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.124+0000 7f6007e00640 1 -- 172.21.15.97:0/1841676753 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f60080b5070 con 0x7f601015fde0 2024-09-10T21:30:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.128+0000 7f6007e00640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fec0725e0 0x7f5fec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.128+0000 7f6007e00640 1 -- 172.21.15.97:0/1841676753 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f600812e0f0 con 0x7f601015fde0 2024-09-10T21:30:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.128+0000 7f600f400640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fec0725e0 0x7f5fec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.128+0000 7f600f400640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fec0725e0 0x7f5fec074aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6000001870 tx=0x7f6000009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:41.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.128+0000 7f6007e00640 1 -- 172.21.15.97:0/1841676753 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f60080166b0 con 0x7f601015fde0 2024-09-10T21:30:41.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.296+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60101404f0 con 0x7f5fec0725e0 2024-09-10T21:30:41.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.300+0000 7f6007e00640 1 -- 172.21.15.97:0/1841676753 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247758 (secure 0 0 0) 0x7f60101404f0 con 0x7f5fec0725e0 2024-09-10T21:30:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:41.308 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fec0725e0 msgr2=0x7f5fec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fec0725e0 0x7f5fec074aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6000001870 tx=0x7f6000009040 comp rx=0 tx=0).stop 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601015fde0 msgr2=0x7f6010144a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601015fde0 0x7f6010144a50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f60080b0040 tx=0x7f6008005af0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 shutdown_connections 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fec0725e0 0x7f5fec074aa0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f601015fde0 0x7f6010144a50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6010141d20 0x7f601013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 --2- 172.21.15.97:0/1841676753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60100ce5f0 0x7f601013e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 >> 172.21.15.97:0/1841676753 conn(0x7f60100b1780 msgr2=0x7f601013f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:41.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 shutdown_connections 2024-09-10T21:30:41.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.304+0000 7f6016800640 1 -- 172.21.15.97:0/1841676753 wait complete. 2024-09-10T21:30:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T21:30:40.168976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255260,"kb_used_data":9888,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719460,"statfs":{"total":773094113280,"available":772832727040,"internally_reserved":0,"allocated":10125312,"data_stored":5290634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.045881"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141245+0000","last_change":"2024-09-10T21:30:36.141245+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141245+0000","last_undegraded":"2024-09-10T21:30:36.141245+0000","last_fullsized":"2024-09-10T21:30:36.141245+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.146230+0000","last_change":"2024-09-10T21:30:36.146230+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.146230+0000","last_undegraded":"2024-09-10T21:30:36.146230+0000","last_fullsized":"2024-09-10T21:30:36.146230+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136865+0000","last_change":"2024-09-10T21:30:36.136865+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136865+0000","last_undegraded":"2024-09-10T21:30:36.136865+0000","last_fullsized":"2024-09-10T21:30:36.136865+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135307+0000","last_change":"2024-09-10T21:30:36.135307+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135307+0000","last_undegraded":"2024-09-10T21:30:36.135307+0000","last_fullsized":"2024-09-10T21:30:36.135307+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139784+0000","last_change":"2024-09-10T21:30:36.139784+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139784+0000","last_undegraded":"2024-09-10T21:30:36.139784+0000","last_fullsized":"2024-09-10T21:30:36.139784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287577+0000","last_change":"2024-09-10T21:30:37.156704+0000","last_active":"2024-09-10T21:30:38.287577+0000","last_peered":"2024-09-10T21:30:38.287577+0000","last_clean":"2024-09-10T21:30:38.287577+0000","last_became_active":"2024-09-10T21:30:37.154734+0000","last_became_peered":"2024-09-10T21:30:37.154734+0000","last_unstale":"2024-09-10T21:30:38.287577+0000","last_undegraded":"2024-09-10T21:30:38.287577+0000","last_fullsized":"2024-09-10T21:30:38.287577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:59.951655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137207+0000","last_change":"2024-09-10T21:30:36.137207+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137207+0000","last_undegraded":"2024-09-10T21:30:36.137207+0000","last_fullsized":"2024-09-10T21:30:36.137207+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287144+0000","last_change":"2024-09-10T21:30:37.146717+0000","last_active":"2024-09-10T21:30:38.287144+0000","last_peered":"2024-09-10T21:30:38.287144+0000","last_clean":"2024-09-10T21:30:38.287144+0000","last_became_active":"2024-09-10T21:30:37.146428+0000","last_became_peered":"2024-09-10T21:30:37.146428+0000","last_unstale":"2024-09-10T21:30:38.287144+0000","last_undegraded":"2024-09-10T21:30:38.287144+0000","last_fullsized":"2024-09-10T21:30:38.287144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:18.307373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137459+0000","last_change":"2024-09-10T21:30:36.137459+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137459+0000","last_undegraded":"2024-09-10T21:30:36.137459+0000","last_fullsized":"2024-09-10T21:30:36.137459+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137009+0000","last_change":"2024-09-10T21:30:36.137009+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137009+0000","last_undegraded":"2024-09-10T21:30:36.137009+0000","last_fullsized":"2024-09-10T21:30:36.137009+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137328+0000","last_change":"2024-09-10T21:30:36.137328+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137328+0000","last_undegraded":"2024-09-10T21:30:36.137328+0000","last_fullsized":"2024-09-10T21:30:36.137328+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134793+0000","last_change":"2024-09-10T21:30:36.134793+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134793+0000","last_undegraded":"2024-09-10T21:30:36.134793+0000","last_fullsized":"2024-09-10T21:30:36.134793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141795+0000","last_change":"2024-09-10T21:30:36.141795+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141795+0000","last_undegraded":"2024-09-10T21:30:36.141795+0000","last_fullsized":"2024-09-10T21:30:36.141795+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136360+0000","last_change":"2024-09-10T21:30:36.136360+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136360+0000","last_undegraded":"2024-09-10T21:30:36.136360+0000","last_fullsized":"2024-09-10T21:30:36.136360+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134234+0000","last_change":"2024-09-10T21:30:36.134234+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134234+0000","last_undegraded":"2024-09-10T21:30:36.134234+0000","last_fullsized":"2024-09-10T21:30:36.134234+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287182+0000","last_change":"2024-09-10T21:30:37.152682+0000","last_active":"2024-09-10T21:30:38.287182+0000","last_peered":"2024-09-10T21:30:38.287182+0000","last_clean":"2024-09-10T21:30:38.287182+0000","last_became_active":"2024-09-10T21:30:37.152308+0000","last_became_peered":"2024-09-10T21:30:37.152308+0000","last_unstale":"2024-09-10T21:30:38.287182+0000","last_undegraded":"2024-09-10T21:30:38.287182+0000","last_fullsized":"2024-09-10T21:30:38.287182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:39.533064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288396+0000","last_change":"2024-09-10T21:30:37.155784+0000","last_active":"2024-09-10T21:30:38.288396+0000","last_peered":"2024-09-10T21:30:38.288396+0000","last_clean":"2024-09-10T21:30:38.288396+0000","last_became_active":"2024-09-10T21:30:37.155514+0000","last_became_peered":"2024-09-10T21:30:37.155514+0000","last_unstale":"2024-09-10T21:30:38.288396+0000","last_undegraded":"2024-09-10T21:30:38.288396+0000","last_fullsized":"2024-09-10T21:30:38.288396+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:10.496991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135754+0000","last_change":"2024-09-10T21:30:36.135754+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135754+0000","last_undegraded":"2024-09-10T21:30:36.135754+0000","last_fullsized":"2024-09-10T21:30:36.135754+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144667+0000","last_change":"2024-09-10T21:30:36.144667+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144667+0000","last_undegraded":"2024-09-10T21:30:36.144667+0000","last_fullsized":"2024-09-10T21:30:36.144667+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287734+0000","last_change":"2024-09-10T21:30:37.161159+0000","last_active":"2024-09-10T21:30:38.287734+0000","last_peered":"2024-09-10T21:30:38.287734+0000","last_clean":"2024-09-10T21:30:38.287734+0000","last_became_active":"2024-09-10T21:30:37.160818+0000","last_became_peered":"2024-09-10T21:30:37.160818+0000","last_unstale":"2024-09-10T21:30:38.287734+0000","last_undegraded":"2024-09-10T21:30:38.287734+0000","last_fullsized":"2024-09-10T21:30:38.287734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:10.536055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139341+0000","last_change":"2024-09-10T21:30:36.139341+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139341+0000","last_undegraded":"2024-09-10T21:30:36.139341+0000","last_fullsized":"2024-09-10T21:30:36.139341+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144558+0000","last_change":"2024-09-10T21:30:36.144558+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144558+0000","last_undegraded":"2024-09-10T21:30:36.144558+0000","last_fullsized":"2024-09-10T21:30:36.144558+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139205+0000","last_change":"2024-09-10T21:30:36.139205+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139205+0000","last_undegraded":"2024-09-10T21:30:36.139205+0000","last_fullsized":"2024-09-10T21:30:36.139205+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143144+0000","last_change":"2024-09-10T21:30:36.143144+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143144+0000","last_undegraded":"2024-09-10T21:30:36.143144+0000","last_fullsized":"2024-09-10T21:30:36.143144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142060+0000","last_change":"2024-09-10T21:30:36.142060+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142060+0000","last_undegraded":"2024-09-10T21:30:36.142060+0000","last_fullsized":"2024-09-10T21:30:36.142060+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139633+0000","last_change":"2024-09-10T21:30:36.139633+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139633+0000","last_undegraded":"2024-09-10T21:30:36.139633+0000","last_fullsized":"2024-09-10T21:30:36.139633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135362+0000","last_change":"2024-09-10T21:30:36.135362+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135362+0000","last_undegraded":"2024-09-10T21:30:36.135362+0000","last_fullsized":"2024-09-10T21:30:36.135362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142361+0000","last_change":"2024-09-10T21:30:36.142361+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142361+0000","last_undegraded":"2024-09-10T21:30:36.142361+0000","last_fullsized":"2024-09-10T21:30:36.142361+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138287+0000","last_change":"2024-09-10T21:30:36.138287+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138287+0000","last_undegraded":"2024-09-10T21:30:36.138287+0000","last_fullsized":"2024-09-10T21:30:36.138287+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137350+0000","last_change":"2024-09-10T21:30:36.137350+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137350+0000","last_undegraded":"2024-09-10T21:30:36.137350+0000","last_fullsized":"2024-09-10T21:30:36.137350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137784+0000","last_change":"2024-09-10T21:30:36.137784+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137784+0000","last_undegraded":"2024-09-10T21:30:36.137784+0000","last_fullsized":"2024-09-10T21:30:36.137784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287384+0000","last_change":"2024-09-10T21:30:37.142844+0000","last_active":"2024-09-10T21:30:38.287384+0000","last_peered":"2024-09-10T21:30:38.287384+0000","last_clean":"2024-09-10T21:30:38.287384+0000","last_became_active":"2024-09-10T21:30:37.142487+0000","last_became_peered":"2024-09-10T21:30:37.142487+0000","last_unstale":"2024-09-10T21:30:38.287384+0000","last_undegraded":"2024-09-10T21:30:38.287384+0000","last_fullsized":"2024-09-10T21:30:38.287384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:49.728530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143651+0000","last_change":"2024-09-10T21:30:36.143651+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143651+0000","last_undegraded":"2024-09-10T21:30:36.143651+0000","last_fullsized":"2024-09-10T21:30:36.143651+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159664+0000","last_change":"2024-09-10T21:30:36.156733+0000","last_active":"2024-09-10T21:30:36.159664+0000","last_peered":"2024-09-10T21:30:36.159664+0000","last_clean":"2024-09-10T21:30:36.159664+0000","last_became_active":"2024-09-10T21:30:36.156360+0000","last_became_peered":"2024-09-10T21:30:36.156360+0000","last_unstale":"2024-09-10T21:30:36.159664+0000","last_undegraded":"2024-09-10T21:30:36.159664+0000","last_fullsized":"2024-09-10T21:30:36.159664+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:23.429032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287825+0000","last_change":"2024-09-10T21:30:37.161240+0000","last_active":"2024-09-10T21:30:38.287825+0000","last_peered":"2024-09-10T21:30:38.287825+0000","last_clean":"2024-09-10T21:30:38.287825+0000","last_became_active":"2024-09-10T21:30:37.160927+0000","last_became_peered":"2024-09-10T21:30:37.160927+0000","last_unstale":"2024-09-10T21:30:38.287825+0000","last_undegraded":"2024-09-10T21:30:38.287825+0000","last_fullsized":"2024-09-10T21:30:38.287825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:57.275653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158055+0000","last_change":"2024-09-10T21:30:36.152982+0000","last_active":"2024-09-10T21:30:36.158055+0000","last_peered":"2024-09-10T21:30:36.158055+0000","last_clean":"2024-09-10T21:30:36.158055+0000","last_became_active":"2024-09-10T21:30:36.151294+0000","last_became_peered":"2024-09-10T21:30:36.151294+0000","last_unstale":"2024-09-10T21:30:36.158055+0000","last_undegraded":"2024-09-10T21:30:36.158055+0000","last_fullsized":"2024-09-10T21:30:36.158055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135162+0000","last_change":"2024-09-10T21:30:36.135162+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135162+0000","last_undegraded":"2024-09-10T21:30:36.135162+0000","last_fullsized":"2024-09-10T21:30:36.135162+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158290+0000","last_change":"2024-09-10T21:30:36.151783+0000","last_active":"2024-09-10T21:30:36.158290+0000","last_peered":"2024-09-10T21:30:36.158290+0000","last_clean":"2024-09-10T21:30:36.158290+0000","last_became_active":"2024-09-10T21:30:36.151113+0000","last_became_peered":"2024-09-10T21:30:36.151113+0000","last_unstale":"2024-09-10T21:30:36.158290+0000","last_undegraded":"2024-09-10T21:30:36.158290+0000","last_fullsized":"2024-09-10T21:30:36.158290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:48.032419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135977+0000","last_change":"2024-09-10T21:30:36.135977+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135977+0000","last_undegraded":"2024-09-10T21:30:36.135977+0000","last_fullsized":"2024-09-10T21:30:36.135977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.150307+0000","last_change":"2024-09-10T21:30:36.147985+0000","last_active":"2024-09-10T21:30:36.150307+0000","last_peered":"2024-09-10T21:30:36.150307+0000","last_clean":"2024-09-10T21:30:36.150307+0000","last_became_active":"2024-09-10T21:30:36.147685+0000","last_became_peered":"2024-09-10T21:30:36.147685+0000","last_unstale":"2024-09-10T21:30:36.150307+0000","last_undegraded":"2024-09-10T21:30:36.150307+0000","last_fullsized":"2024-09-10T21:30:36.150307+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:23.894646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139414+0000","last_change":"2024-09-10T21:30:36.139414+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139414+0000","last_undegraded":"2024-09-10T21:30:36.139414+0000","last_fullsized":"2024-09-10T21:30:36.139414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158555+0000","last_change":"2024-09-10T21:30:36.152597+0000","last_active":"2024-09-10T21:30:36.158555+0000","last_peered":"2024-09-10T21:30:36.158555+0000","last_clean":"2024-09-10T21:30:36.158555+0000","last_became_active":"2024-09-10T21:30:36.152336+0000","last_became_peered":"2024-09-10T21:30:36.152336+0000","last_unstale":"2024-09-10T21:30:36.158555+0000","last_undegraded":"2024-09-10T21:30:36.158555+0000","last_fullsized":"2024-09-10T21:30:36.158555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135003+0000","last_change":"2024-09-10T21:30:36.135003+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135003+0000","last_undegraded":"2024-09-10T21:30:36.135003+0000","last_fullsized":"2024-09-10T21:30:36.135003+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.160752+0000","last_change":"2024-09-10T21:30:36.156510+0000","last_active":"2024-09-10T21:30:36.160752+0000","last_peered":"2024-09-10T21:30:36.160752+0000","last_clean":"2024-09-10T21:30:36.160752+0000","last_became_active":"2024-09-10T21:30:36.156197+0000","last_became_peered":"2024-09-10T21:30:36.156197+0000","last_unstale":"2024-09-10T21:30:36.160752+0000","last_undegraded":"2024-09-10T21:30:36.160752+0000","last_fullsized":"2024-09-10T21:30:36.160752+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:38.551184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287564+0000","last_change":"2024-09-10T21:30:37.143279+0000","last_active":"2024-09-10T21:30:38.287564+0000","last_peered":"2024-09-10T21:30:38.287564+0000","last_clean":"2024-09-10T21:30:38.287564+0000","last_became_active":"2024-09-10T21:30:37.142984+0000","last_became_peered":"2024-09-10T21:30:37.142984+0000","last_unstale":"2024-09-10T21:30:38.287564+0000","last_undegraded":"2024-09-10T21:30:38.287564+0000","last_fullsized":"2024-09-10T21:30:38.287564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:12.128103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.152766+0000","last_change":"2024-09-10T21:30:36.150338+0000","last_active":"2024-09-10T21:30:36.152766+0000","last_peered":"2024-09-10T21:30:36.152766+0000","last_clean":"2024-09-10T21:30:36.152766+0000","last_became_active":"2024-09-10T21:30:36.149500+0000","last_became_peered":"2024-09-10T21:30:36.149500+0000","last_unstale":"2024-09-10T21:30:36.152766+0000","last_undegraded":"2024-09-10T21:30:36.152766+0000","last_fullsized":"2024-09-10T21:30:36.152766+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:23.529335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.145566+0000","last_change":"2024-09-10T21:30:36.145566+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.145566+0000","last_undegraded":"2024-09-10T21:30:36.145566+0000","last_fullsized":"2024-09-10T21:30:36.145566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158416+0000","last_change":"2024-09-10T21:30:36.152065+0000","last_active":"2024-09-10T21:30:36.158416+0000","last_peered":"2024-09-10T21:30:36.158416+0000","last_clean":"2024-09-10T21:30:36.158416+0000","last_became_active":"2024-09-10T21:30:36.151761+0000","last_became_peered":"2024-09-10T21:30:36.151761+0000","last_unstale":"2024-09-10T21:30:36.158416+0000","last_undegraded":"2024-09-10T21:30:36.158416+0000","last_fullsized":"2024-09-10T21:30:36.158416+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:03.575792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140059+0000","last_change":"2024-09-10T21:30:36.140059+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140059+0000","last_undegraded":"2024-09-10T21:30:36.140059+0000","last_fullsized":"2024-09-10T21:30:36.140059+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158268+0000","last_change":"2024-09-10T21:30:36.152474+0000","last_active":"2024-09-10T21:30:36.158268+0000","last_peered":"2024-09-10T21:30:36.158268+0000","last_clean":"2024-09-10T21:30:36.158268+0000","last_became_active":"2024-09-10T21:30:36.152180+0000","last_became_peered":"2024-09-10T21:30:36.152180+0000","last_unstale":"2024-09-10T21:30:36.158268+0000","last_undegraded":"2024-09-10T21:30:36.158268+0000","last_fullsized":"2024-09-10T21:30:36.158268+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:36.046838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135087+0000","last_change":"2024-09-10T21:30:36.135087+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135087+0000","last_undegraded":"2024-09-10T21:30:36.135087+0000","last_fullsized":"2024-09-10T21:30:36.135087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.152696+0000","last_change":"2024-09-10T21:30:36.149165+0000","last_active":"2024-09-10T21:30:36.152696+0000","last_peered":"2024-09-10T21:30:36.152696+0000","last_clean":"2024-09-10T21:30:36.152696+0000","last_became_active":"2024-09-10T21:30:36.148849+0000","last_became_peered":"2024-09-10T21:30:36.148849+0000","last_unstale":"2024-09-10T21:30:36.152696+0000","last_undegraded":"2024-09-10T21:30:36.152696+0000","last_fullsized":"2024-09-10T21:30:36.152696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:02.600501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140912+0000","last_change":"2024-09-10T21:30:36.140912+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140912+0000","last_undegraded":"2024-09-10T21:30:36.140912+0000","last_fullsized":"2024-09-10T21:30:36.140912+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157397+0000","last_change":"2024-09-10T21:30:36.149184+0000","last_active":"2024-09-10T21:30:36.157397+0000","last_peered":"2024-09-10T21:30:36.157397+0000","last_clean":"2024-09-10T21:30:36.157397+0000","last_became_active":"2024-09-10T21:30:36.148915+0000","last_became_peered":"2024-09-10T21:30:36.148915+0000","last_unstale":"2024-09-10T21:30:36.157397+0000","last_undegraded":"2024-09-10T21:30:36.157397+0000","last_fullsized":"2024-09-10T21:30:36.157397+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:46.052870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288504+0000","last_change":"2024-09-10T21:30:37.147434+0000","last_active":"2024-09-10T21:30:38.288504+0000","last_peered":"2024-09-10T21:30:38.288504+0000","last_clean":"2024-09-10T21:30:38.288504+0000","last_became_active":"2024-09-10T21:30:37.147160+0000","last_became_peered":"2024-09-10T21:30:37.147160+0000","last_unstale":"2024-09-10T21:30:38.288504+0000","last_undegraded":"2024-09-10T21:30:38.288504+0000","last_fullsized":"2024-09-10T21:30:38.288504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:07.679232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.161172+0000","last_change":"2024-09-10T21:30:36.158339+0000","last_active":"2024-09-10T21:30:36.161172+0000","last_peered":"2024-09-10T21:30:36.161172+0000","last_clean":"2024-09-10T21:30:36.161172+0000","last_became_active":"2024-09-10T21:30:36.158050+0000","last_became_peered":"2024-09-10T21:30:36.158050+0000","last_unstale":"2024-09-10T21:30:36.161172+0000","last_undegraded":"2024-09-10T21:30:36.161172+0000","last_fullsized":"2024-09-10T21:30:36.161172+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:36.544620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142693+0000","last_change":"2024-09-10T21:30:36.142693+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142693+0000","last_undegraded":"2024-09-10T21:30:36.142693+0000","last_fullsized":"2024-09-10T21:30:36.142693+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159535+0000","last_change":"2024-09-10T21:30:36.155028+0000","last_active":"2024-09-10T21:30:36.159535+0000","last_peered":"2024-09-10T21:30:36.159535+0000","last_clean":"2024-09-10T21:30:36.159535+0000","last_became_active":"2024-09-10T21:30:36.154105+0000","last_became_peered":"2024-09-10T21:30:36.154105+0000","last_unstale":"2024-09-10T21:30:36.159535+0000","last_undegraded":"2024-09-10T21:30:36.159535+0000","last_fullsized":"2024-09-10T21:30:36.159535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:51.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138771+0000","last_change":"2024-09-10T21:30:36.138771+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138771+0000","last_undegraded":"2024-09-10T21:30:36.138771+0000","last_fullsized":"2024-09-10T21:30:36.138771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155516+0000","last_change":"2024-09-10T21:30:36.149196+0000","last_active":"2024-09-10T21:30:36.155516+0000","last_peered":"2024-09-10T21:30:36.155516+0000","last_clean":"2024-09-10T21:30:36.155516+0000","last_became_active":"2024-09-10T21:30:36.148907+0000","last_became_peered":"2024-09-10T21:30:36.148907+0000","last_unstale":"2024-09-10T21:30:36.155516+0000","last_undegraded":"2024-09-10T21:30:36.155516+0000","last_fullsized":"2024-09-10T21:30:36.155516+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.146320+0000","last_change":"2024-09-10T21:30:36.146320+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.146320+0000","last_undegraded":"2024-09-10T21:30:36.146320+0000","last_fullsized":"2024-09-10T21:30:36.146320+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157943+0000","last_change":"2024-09-10T21:30:36.152984+0000","last_active":"2024-09-10T21:30:36.157943+0000","last_peered":"2024-09-10T21:30:36.157943+0000","last_clean":"2024-09-10T21:30:36.157943+0000","last_became_active":"2024-09-10T21:30:36.152704+0000","last_became_peered":"2024-09-10T21:30:36.152704+0000","last_unstale":"2024-09-10T21:30:36.157943+0000","last_undegraded":"2024-09-10T21:30:36.157943+0000","last_fullsized":"2024-09-10T21:30:36.157943+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138596+0000","last_change":"2024-09-10T21:30:36.138596+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138596+0000","last_undegraded":"2024-09-10T21:30:36.138596+0000","last_fullsized":"2024-09-10T21:30:36.138596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155485+0000","last_change":"2024-09-10T21:30:36.150376+0000","last_active":"2024-09-10T21:30:36.155485+0000","last_peered":"2024-09-10T21:30:36.155485+0000","last_clean":"2024-09-10T21:30:36.155485+0000","last_became_active":"2024-09-10T21:30:36.150090+0000","last_became_peered":"2024-09-10T21:30:36.150090+0000","last_unstale":"2024-09-10T21:30:36.155485+0000","last_undegraded":"2024-09-10T21:30:36.155485+0000","last_fullsized":"2024-09-10T21:30:36.155485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:15.145473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143402+0000","last_change":"2024-09-10T21:30:36.143402+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143402+0000","last_undegraded":"2024-09-10T21:30:36.143402+0000","last_fullsized":"2024-09-10T21:30:36.143402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158775+0000","last_change":"2024-09-10T21:30:36.153213+0000","last_active":"2024-09-10T21:30:36.158775+0000","last_peered":"2024-09-10T21:30:36.158775+0000","last_clean":"2024-09-10T21:30:36.158775+0000","last_became_active":"2024-09-10T21:30:36.152918+0000","last_became_peered":"2024-09-10T21:30:36.152918+0000","last_unstale":"2024-09-10T21:30:36.158775+0000","last_undegraded":"2024-09-10T21:30:36.158775+0000","last_fullsized":"2024-09-10T21:30:36.158775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:39.986638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134560+0000","last_change":"2024-09-10T21:30:36.134560+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134560+0000","last_undegraded":"2024-09-10T21:30:36.134560+0000","last_fullsized":"2024-09-10T21:30:36.134560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158805+0000","last_change":"2024-09-10T21:30:36.155934+0000","last_active":"2024-09-10T21:30:36.158805+0000","last_peered":"2024-09-10T21:30:36.158805+0000","last_clean":"2024-09-10T21:30:36.158805+0000","last_became_active":"2024-09-10T21:30:36.155669+0000","last_became_peered":"2024-09-10T21:30:36.155669+0000","last_unstale":"2024-09-10T21:30:36.158805+0000","last_undegraded":"2024-09-10T21:30:36.158805+0000","last_fullsized":"2024-09-10T21:30:36.158805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144601+0000","last_change":"2024-09-10T21:30:36.144601+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144601+0000","last_undegraded":"2024-09-10T21:30:36.144601+0000","last_fullsized":"2024-09-10T21:30:36.144601+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158093+0000","last_change":"2024-09-10T21:30:36.152792+0000","last_active":"2024-09-10T21:30:36.158093+0000","last_peered":"2024-09-10T21:30:36.158093+0000","last_clean":"2024-09-10T21:30:36.158093+0000","last_became_active":"2024-09-10T21:30:36.152526+0000","last_became_peered":"2024-09-10T21:30:36.152526+0000","last_unstale":"2024-09-10T21:30:36.158093+0000","last_undegraded":"2024-09-10T21:30:36.158093+0000","last_fullsized":"2024-09-10T21:30:36.158093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.541056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287996+0000","last_change":"2024-09-10T21:30:37.163752+0000","last_active":"2024-09-10T21:30:38.287996+0000","last_peered":"2024-09-10T21:30:38.287996+0000","last_clean":"2024-09-10T21:30:38.287996+0000","last_became_active":"2024-09-10T21:30:37.162710+0000","last_became_peered":"2024-09-10T21:30:37.162710+0000","last_unstale":"2024-09-10T21:30:38.287996+0000","last_undegraded":"2024-09-10T21:30:38.287996+0000","last_fullsized":"2024-09-10T21:30:38.287996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:33.852578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159112+0000","last_change":"2024-09-10T21:30:36.153508+0000","last_active":"2024-09-10T21:30:36.159112+0000","last_peered":"2024-09-10T21:30:36.159112+0000","last_clean":"2024-09-10T21:30:36.159112+0000","last_became_active":"2024-09-10T21:30:36.153255+0000","last_became_peered":"2024-09-10T21:30:36.153255+0000","last_unstale":"2024-09-10T21:30:36.159112+0000","last_undegraded":"2024-09-10T21:30:36.159112+0000","last_fullsized":"2024-09-10T21:30:36.159112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:44.746855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136690+0000","last_change":"2024-09-10T21:30:36.136690+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136690+0000","last_undegraded":"2024-09-10T21:30:36.136690+0000","last_fullsized":"2024-09-10T21:30:36.136690+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287713+0000","last_change":"2024-09-10T21:30:36.153764+0000","last_active":"2024-09-10T21:30:38.287713+0000","last_peered":"2024-09-10T21:30:38.287713+0000","last_clean":"2024-09-10T21:30:38.287713+0000","last_became_active":"2024-09-10T21:30:36.153453+0000","last_became_peered":"2024-09-10T21:30:36.153453+0000","last_unstale":"2024-09-10T21:30:38.287713+0000","last_undegraded":"2024-09-10T21:30:38.287713+0000","last_fullsized":"2024-09-10T21:30:38.287713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:28.260842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144090+0000","last_change":"2024-09-10T21:30:36.144090+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144090+0000","last_undegraded":"2024-09-10T21:30:36.144090+0000","last_fullsized":"2024-09-10T21:30:36.144090+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.147411+0000","last_change":"2024-09-10T21:30:36.145884+0000","last_active":"2024-09-10T21:30:36.147411+0000","last_peered":"2024-09-10T21:30:36.147411+0000","last_clean":"2024-09-10T21:30:36.147411+0000","last_became_active":"2024-09-10T21:30:36.145592+0000","last_became_peered":"2024-09-10T21:30:36.145592+0000","last_unstale":"2024-09-10T21:30:36.147411+0000","last_undegraded":"2024-09-10T21:30:36.147411+0000","last_fullsized":"2024-09-10T21:30:36.147411+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:52.517766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.145154+0000","last_change":"2024-09-10T21:30:36.145154+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.145154+0000","last_undegraded":"2024-09-10T21:30:36.145154+0000","last_fullsized":"2024-09-10T21:30:36.145154+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158363+0000","last_change":"2024-09-10T21:30:36.153170+0000","last_active":"2024-09-10T21:30:36.158363+0000","last_peered":"2024-09-10T21:30:36.158363+0000","last_clean":"2024-09-10T21:30:36.158363+0000","last_became_active":"2024-09-10T21:30:36.152223+0000","last_became_peered":"2024-09-10T21:30:36.152223+0000","last_unstale":"2024-09-10T21:30:36.158363+0000","last_undegraded":"2024-09-10T21:30:36.158363+0000","last_fullsized":"2024-09-10T21:30:36.158363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:00.035137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143977+0000","last_change":"2024-09-10T21:30:36.143977+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143977+0000","last_undegraded":"2024-09-10T21:30:36.143977+0000","last_fullsized":"2024-09-10T21:30:36.143977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157622+0000","last_change":"2024-09-10T21:30:36.149909+0000","last_active":"2024-09-10T21:30:36.157622+0000","last_peered":"2024-09-10T21:30:36.157622+0000","last_clean":"2024-09-10T21:30:36.157622+0000","last_became_active":"2024-09-10T21:30:36.149596+0000","last_became_peered":"2024-09-10T21:30:36.149596+0000","last_unstale":"2024-09-10T21:30:36.157622+0000","last_undegraded":"2024-09-10T21:30:36.157622+0000","last_fullsized":"2024-09-10T21:30:36.157622+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:27.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140295+0000","last_change":"2024-09-10T21:30:36.140295+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140295+0000","last_undegraded":"2024-09-10T21:30:36.140295+0000","last_fullsized":"2024-09-10T21:30:36.140295+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159054+0000","last_change":"2024-09-10T21:30:36.151988+0000","last_active":"2024-09-10T21:30:36.159054+0000","last_peered":"2024-09-10T21:30:36.159054+0000","last_clean":"2024-09-10T21:30:36.159054+0000","last_became_active":"2024-09-10T21:30:36.151626+0000","last_became_peered":"2024-09-10T21:30:36.151626+0000","last_unstale":"2024-09-10T21:30:36.159054+0000","last_undegraded":"2024-09-10T21:30:36.159054+0000","last_fullsized":"2024-09-10T21:30:36.159054+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:01.606529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138904+0000","last_change":"2024-09-10T21:30:36.138904+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138904+0000","last_undegraded":"2024-09-10T21:30:36.138904+0000","last_fullsized":"2024-09-10T21:30:36.138904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159234+0000","last_change":"2024-09-10T21:30:36.156216+0000","last_active":"2024-09-10T21:30:36.159234+0000","last_peered":"2024-09-10T21:30:36.159234+0000","last_clean":"2024-09-10T21:30:36.159234+0000","last_became_active":"2024-09-10T21:30:36.155962+0000","last_became_peered":"2024-09-10T21:30:36.155962+0000","last_unstale":"2024-09-10T21:30:36.159234+0000","last_undegraded":"2024-09-10T21:30:36.159234+0000","last_fullsized":"2024-09-10T21:30:36.159234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:13.896623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141644+0000","last_change":"2024-09-10T21:30:36.141644+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141644+0000","last_undegraded":"2024-09-10T21:30:36.141644+0000","last_fullsized":"2024-09-10T21:30:36.141644+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157051+0000","last_change":"2024-09-10T21:30:36.146558+0000","last_active":"2024-09-10T21:30:36.157051+0000","last_peered":"2024-09-10T21:30:36.157051+0000","last_clean":"2024-09-10T21:30:36.157051+0000","last_became_active":"2024-09-10T21:30:36.144327+0000","last_became_peered":"2024-09-10T21:30:36.144327+0000","last_unstale":"2024-09-10T21:30:36.157051+0000","last_undegraded":"2024-09-10T21:30:36.157051+0000","last_fullsized":"2024-09-10T21:30:36.157051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:40.224703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287947+0000","last_change":"2024-09-10T21:30:37.162963+0000","last_active":"2024-09-10T21:30:38.287947+0000","last_peered":"2024-09-10T21:30:38.287947+0000","last_clean":"2024-09-10T21:30:38.287947+0000","last_became_active":"2024-09-10T21:30:37.162366+0000","last_became_peered":"2024-09-10T21:30:37.162366+0000","last_unstale":"2024-09-10T21:30:38.287947+0000","last_undegraded":"2024-09-10T21:30:38.287947+0000","last_fullsized":"2024-09-10T21:30:38.287947+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:37.376166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.161192+0000","last_change":"2024-09-10T21:30:36.159592+0000","last_active":"2024-09-10T21:30:36.161192+0000","last_peered":"2024-09-10T21:30:36.161192+0000","last_clean":"2024-09-10T21:30:36.161192+0000","last_became_active":"2024-09-10T21:30:36.159354+0000","last_became_peered":"2024-09-10T21:30:36.159354+0000","last_unstale":"2024-09-10T21:30:36.161192+0000","last_undegraded":"2024-09-10T21:30:36.161192+0000","last_fullsized":"2024-09-10T21:30:36.161192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:26.541443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.133612+0000","last_change":"2024-09-10T21:30:36.133612+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.133612+0000","last_undegraded":"2024-09-10T21:30:36.133612+0000","last_fullsized":"2024-09-10T21:30:36.133612+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159701+0000","last_change":"2024-09-10T21:30:36.157140+0000","last_active":"2024-09-10T21:30:36.159701+0000","last_peered":"2024-09-10T21:30:36.159701+0000","last_clean":"2024-09-10T21:30:36.159701+0000","last_became_active":"2024-09-10T21:30:36.156861+0000","last_became_peered":"2024-09-10T21:30:36.156861+0000","last_unstale":"2024-09-10T21:30:36.159701+0000","last_undegraded":"2024-09-10T21:30:36.159701+0000","last_fullsized":"2024-09-10T21:30:36.159701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:09.577405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137154+0000","last_change":"2024-09-10T21:30:36.137154+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137154+0000","last_undegraded":"2024-09-10T21:30:36.137154+0000","last_fullsized":"2024-09-10T21:30:36.137154+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.154650+0000","last_change":"2024-09-10T21:30:36.147814+0000","last_active":"2024-09-10T21:30:36.154650+0000","last_peered":"2024-09-10T21:30:36.154650+0000","last_clean":"2024-09-10T21:30:36.154650+0000","last_became_active":"2024-09-10T21:30:36.147488+0000","last_became_peered":"2024-09-10T21:30:36.147488+0000","last_unstale":"2024-09-10T21:30:36.154650+0000","last_undegraded":"2024-09-10T21:30:36.154650+0000","last_fullsized":"2024-09-10T21:30:36.154650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:23.302054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287689+0000","last_change":"2024-09-10T21:30:37.164739+0000","last_active":"2024-09-10T21:30:38.287689+0000","last_peered":"2024-09-10T21:30:38.287689+0000","last_clean":"2024-09-10T21:30:38.287689+0000","last_became_active":"2024-09-10T21:30:37.164473+0000","last_became_peered":"2024-09-10T21:30:37.164473+0000","last_unstale":"2024-09-10T21:30:38.287689+0000","last_undegraded":"2024-09-10T21:30:38.287689+0000","last_fullsized":"2024-09-10T21:30:38.287689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:36.726703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159717+0000","last_change":"2024-09-10T21:30:36.156585+0000","last_active":"2024-09-10T21:30:36.159717+0000","last_peered":"2024-09-10T21:30:36.159717+0000","last_clean":"2024-09-10T21:30:36.159717+0000","last_became_active":"2024-09-10T21:30:36.156288+0000","last_became_peered":"2024-09-10T21:30:36.156288+0000","last_unstale":"2024-09-10T21:30:36.159717+0000","last_undegraded":"2024-09-10T21:30:36.159717+0000","last_fullsized":"2024-09-10T21:30:36.159717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:24.061853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141257+0000","last_change":"2024-09-10T21:30:36.141257+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141257+0000","last_undegraded":"2024-09-10T21:30:36.141257+0000","last_fullsized":"2024-09-10T21:30:36.141257+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155869+0000","last_change":"2024-09-10T21:30:36.151215+0000","last_active":"2024-09-10T21:30:36.155869+0000","last_peered":"2024-09-10T21:30:36.155869+0000","last_clean":"2024-09-10T21:30:36.155869+0000","last_became_active":"2024-09-10T21:30:36.150699+0000","last_became_peered":"2024-09-10T21:30:36.150699+0000","last_unstale":"2024-09-10T21:30:36.155869+0000","last_undegraded":"2024-09-10T21:30:36.155869+0000","last_fullsized":"2024-09-10T21:30:36.155869+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.150455+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:30:36.150455+0000","last_peered":"2024-09-10T21:30:36.150455+0000","last_clean":"2024-09-10T21:30:36.150455+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:30:36.150455+0000","last_undegraded":"2024-09-10T21:30:36.150455+0000","last_fullsized":"2024-09-10T21:30:36.150455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1102376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1102376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1138688,"data_stored":525842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:30:41.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:30:41.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- 172.21.15.97:0/1155120316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 msgr2=0x7f71cc166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 --2- 172.21.15.97:0/1155120316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc166850 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009c80 tx=0x7f71c401cda0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- 172.21.15.97:0/1155120316 shutdown_connections 2024-09-10T21:30:41.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 --2- 172.21.15.97:0/1155120316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc166850 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- 172.21.15.97:0/1155120316 >> 172.21.15.97:0/1155120316 conn(0x7f71cc0b1780 msgr2=0x7f71cc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- 172.21.15.97:0/1155120316 shutdown_connections 2024-09-10T21:30:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- 172.21.15.97:0/1155120316 wait complete. 2024-09-10T21:30:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 Processor -- start 2024-09-10T21:30:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- start start 2024-09-10T21:30:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71cc0ce5f0 0x7f71cc156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71cc141d20 0x7f71cc15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc16a130 con 0x7f71cc15fde0 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc169fb0 con 0x7f71cc141d20 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71d2000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc16a2b0 con 0x7f71cc0ce5f0 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc15d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48642/0 (socket says 172.21.15.97:48642) 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 -- 172.21.15.97:0/3843143774 learned_addr learned my addr 172.21.15.97:0/3843143774 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71c2a00640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71cc141d20 0x7f71cc15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cb400640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71cc0ce5f0 0x7f71cc156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 -- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71cc0ce5f0 msgr2=0x7f71cc156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71cc0ce5f0 0x7f71cc156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 -- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71cc141d20 msgr2=0x7f71cc15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71cc141d20 0x7f71cc15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.512+0000 7f71cbe00640 1 -- 172.21.15.97:0/3843143774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc159580 con 0x7f71cc15fde0 2024-09-10T21:30:41.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71cbe00640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc15d440 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f71c401a9d0 tx=0x7f71c40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:41.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71c3e00640 1 -- 172.21.15.97:0/3843143774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40bc070 con 0x7f71cc15fde0 2024-09-10T21:30:41.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71c3e00640 1 -- 172.21.15.97:0/3843143774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71c40042f0 con 0x7f71cc15fde0 2024-09-10T21:30:41.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71c3e00640 1 -- 172.21.15.97:0/3843143774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40b2040 con 0x7f71cc15fde0 2024-09-10T21:30:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc159860 con 0x7f71cc15fde0 2024-09-10T21:30:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc156c50 con 0x7f71cc15fde0 2024-09-10T21:30:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc0ce9d0 con 0x7f71cc15fde0 2024-09-10T21:30:41.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71c3e00640 1 -- 172.21.15.97:0/3843143774 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71c40b5070 con 0x7f71cc15fde0 2024-09-10T21:30:41.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71c3e00640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71a80725e0 0x7f71a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71c3e00640 1 -- 172.21.15.97:0/3843143774 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f71c412cff0 con 0x7f71cc15fde0 2024-09-10T21:30:41.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.516+0000 7f71cb400640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71a80725e0 0x7f71a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.520+0000 7f71cb400640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71a80725e0 0x7f71a8074aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000f70 tx=0x7f71bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.520+0000 7f71c3e00640 1 -- 172.21.15.97:0/3843143774 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71c4016990 con 0x7f71cc15fde0 2024-09-10T21:30:41.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.684+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71cc141360 con 0x7f71a80725e0 2024-09-10T21:30:41.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.688+0000 7f71c3e00640 1 -- 172.21.15.97:0/3843143774 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247758 (secure 0 0 0) 0x7f71cc141360 con 0x7f71a80725e0 2024-09-10T21:30:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:41.695 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71a80725e0 msgr2=0x7f71a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71a80725e0 0x7f71a8074aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000f70 tx=0x7f71bc009040 comp rx=0 tx=0).stop 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 msgr2=0x7f71cc15d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc15d440 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f71c401a9d0 tx=0x7f71c40093d0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 shutdown_connections 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71a80725e0 0x7f71a8074aa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71cc15fde0 0x7f71cc15d440 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71cc141d20 0x7f71cc15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 --2- 172.21.15.97:0/3843143774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71cc0ce5f0 0x7f71cc156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 >> 172.21.15.97:0/3843143774 conn(0x7f71cc0b1780 msgr2=0x7f71cc06eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:41.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 shutdown_connections 2024-09-10T21:30:41.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.692+0000 7f71d2000640 1 -- 172.21.15.97:0/3843143774 wait complete. 2024-09-10T21:30:41.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T21:30:40.168976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255260,"kb_used_data":9888,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719460,"statfs":{"total":773094113280,"available":772832727040,"internally_reserved":0,"allocated":10125312,"data_stored":5290634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.045881"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141245+0000","last_change":"2024-09-10T21:30:36.141245+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141245+0000","last_undegraded":"2024-09-10T21:30:36.141245+0000","last_fullsized":"2024-09-10T21:30:36.141245+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.146230+0000","last_change":"2024-09-10T21:30:36.146230+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.146230+0000","last_undegraded":"2024-09-10T21:30:36.146230+0000","last_fullsized":"2024-09-10T21:30:36.146230+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136865+0000","last_change":"2024-09-10T21:30:36.136865+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136865+0000","last_undegraded":"2024-09-10T21:30:36.136865+0000","last_fullsized":"2024-09-10T21:30:36.136865+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135307+0000","last_change":"2024-09-10T21:30:36.135307+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135307+0000","last_undegraded":"2024-09-10T21:30:36.135307+0000","last_fullsized":"2024-09-10T21:30:36.135307+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139784+0000","last_change":"2024-09-10T21:30:36.139784+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139784+0000","last_undegraded":"2024-09-10T21:30:36.139784+0000","last_fullsized":"2024-09-10T21:30:36.139784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287577+0000","last_change":"2024-09-10T21:30:37.156704+0000","last_active":"2024-09-10T21:30:38.287577+0000","last_peered":"2024-09-10T21:30:38.287577+0000","last_clean":"2024-09-10T21:30:38.287577+0000","last_became_active":"2024-09-10T21:30:37.154734+0000","last_became_peered":"2024-09-10T21:30:37.154734+0000","last_unstale":"2024-09-10T21:30:38.287577+0000","last_undegraded":"2024-09-10T21:30:38.287577+0000","last_fullsized":"2024-09-10T21:30:38.287577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:59.951655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137207+0000","last_change":"2024-09-10T21:30:36.137207+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137207+0000","last_undegraded":"2024-09-10T21:30:36.137207+0000","last_fullsized":"2024-09-10T21:30:36.137207+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287144+0000","last_change":"2024-09-10T21:30:37.146717+0000","last_active":"2024-09-10T21:30:38.287144+0000","last_peered":"2024-09-10T21:30:38.287144+0000","last_clean":"2024-09-10T21:30:38.287144+0000","last_became_active":"2024-09-10T21:30:37.146428+0000","last_became_peered":"2024-09-10T21:30:37.146428+0000","last_unstale":"2024-09-10T21:30:38.287144+0000","last_undegraded":"2024-09-10T21:30:38.287144+0000","last_fullsized":"2024-09-10T21:30:38.287144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:18.307373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137459+0000","last_change":"2024-09-10T21:30:36.137459+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137459+0000","last_undegraded":"2024-09-10T21:30:36.137459+0000","last_fullsized":"2024-09-10T21:30:36.137459+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137009+0000","last_change":"2024-09-10T21:30:36.137009+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137009+0000","last_undegraded":"2024-09-10T21:30:36.137009+0000","last_fullsized":"2024-09-10T21:30:36.137009+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137328+0000","last_change":"2024-09-10T21:30:36.137328+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137328+0000","last_undegraded":"2024-09-10T21:30:36.137328+0000","last_fullsized":"2024-09-10T21:30:36.137328+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134793+0000","last_change":"2024-09-10T21:30:36.134793+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134793+0000","last_undegraded":"2024-09-10T21:30:36.134793+0000","last_fullsized":"2024-09-10T21:30:36.134793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141795+0000","last_change":"2024-09-10T21:30:36.141795+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141795+0000","last_undegraded":"2024-09-10T21:30:36.141795+0000","last_fullsized":"2024-09-10T21:30:36.141795+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136360+0000","last_change":"2024-09-10T21:30:36.136360+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136360+0000","last_undegraded":"2024-09-10T21:30:36.136360+0000","last_fullsized":"2024-09-10T21:30:36.136360+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134234+0000","last_change":"2024-09-10T21:30:36.134234+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134234+0000","last_undegraded":"2024-09-10T21:30:36.134234+0000","last_fullsized":"2024-09-10T21:30:36.134234+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287182+0000","last_change":"2024-09-10T21:30:37.152682+0000","last_active":"2024-09-10T21:30:38.287182+0000","last_peered":"2024-09-10T21:30:38.287182+0000","last_clean":"2024-09-10T21:30:38.287182+0000","last_became_active":"2024-09-10T21:30:37.152308+0000","last_became_peered":"2024-09-10T21:30:37.152308+0000","last_unstale":"2024-09-10T21:30:38.287182+0000","last_undegraded":"2024-09-10T21:30:38.287182+0000","last_fullsized":"2024-09-10T21:30:38.287182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:39.533064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288396+0000","last_change":"2024-09-10T21:30:37.155784+0000","last_active":"2024-09-10T21:30:38.288396+0000","last_peered":"2024-09-10T21:30:38.288396+0000","last_clean":"2024-09-10T21:30:38.288396+0000","last_became_active":"2024-09-10T21:30:37.155514+0000","last_became_peered":"2024-09-10T21:30:37.155514+0000","last_unstale":"2024-09-10T21:30:38.288396+0000","last_undegraded":"2024-09-10T21:30:38.288396+0000","last_fullsized":"2024-09-10T21:30:38.288396+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:10.496991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135754+0000","last_change":"2024-09-10T21:30:36.135754+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135754+0000","last_undegraded":"2024-09-10T21:30:36.135754+0000","last_fullsized":"2024-09-10T21:30:36.135754+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144667+0000","last_change":"2024-09-10T21:30:36.144667+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144667+0000","last_undegraded":"2024-09-10T21:30:36.144667+0000","last_fullsized":"2024-09-10T21:30:36.144667+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287734+0000","last_change":"2024-09-10T21:30:37.161159+0000","last_active":"2024-09-10T21:30:38.287734+0000","last_peered":"2024-09-10T21:30:38.287734+0000","last_clean":"2024-09-10T21:30:38.287734+0000","last_became_active":"2024-09-10T21:30:37.160818+0000","last_became_peered":"2024-09-10T21:30:37.160818+0000","last_unstale":"2024-09-10T21:30:38.287734+0000","last_undegraded":"2024-09-10T21:30:38.287734+0000","last_fullsized":"2024-09-10T21:30:38.287734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:10.536055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139341+0000","last_change":"2024-09-10T21:30:36.139341+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139341+0000","last_undegraded":"2024-09-10T21:30:36.139341+0000","last_fullsized":"2024-09-10T21:30:36.139341+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144558+0000","last_change":"2024-09-10T21:30:36.144558+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144558+0000","last_undegraded":"2024-09-10T21:30:36.144558+0000","last_fullsized":"2024-09-10T21:30:36.144558+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139205+0000","last_change":"2024-09-10T21:30:36.139205+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139205+0000","last_undegraded":"2024-09-10T21:30:36.139205+0000","last_fullsized":"2024-09-10T21:30:36.139205+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143144+0000","last_change":"2024-09-10T21:30:36.143144+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143144+0000","last_undegraded":"2024-09-10T21:30:36.143144+0000","last_fullsized":"2024-09-10T21:30:36.143144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142060+0000","last_change":"2024-09-10T21:30:36.142060+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142060+0000","last_undegraded":"2024-09-10T21:30:36.142060+0000","last_fullsized":"2024-09-10T21:30:36.142060+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139633+0000","last_change":"2024-09-10T21:30:36.139633+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139633+0000","last_undegraded":"2024-09-10T21:30:36.139633+0000","last_fullsized":"2024-09-10T21:30:36.139633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135362+0000","last_change":"2024-09-10T21:30:36.135362+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135362+0000","last_undegraded":"2024-09-10T21:30:36.135362+0000","last_fullsized":"2024-09-10T21:30:36.135362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142361+0000","last_change":"2024-09-10T21:30:36.142361+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142361+0000","last_undegraded":"2024-09-10T21:30:36.142361+0000","last_fullsized":"2024-09-10T21:30:36.142361+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138287+0000","last_change":"2024-09-10T21:30:36.138287+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138287+0000","last_undegraded":"2024-09-10T21:30:36.138287+0000","last_fullsized":"2024-09-10T21:30:36.138287+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137350+0000","last_change":"2024-09-10T21:30:36.137350+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137350+0000","last_undegraded":"2024-09-10T21:30:36.137350+0000","last_fullsized":"2024-09-10T21:30:36.137350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137784+0000","last_change":"2024-09-10T21:30:36.137784+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137784+0000","last_undegraded":"2024-09-10T21:30:36.137784+0000","last_fullsized":"2024-09-10T21:30:36.137784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287384+0000","last_change":"2024-09-10T21:30:37.142844+0000","last_active":"2024-09-10T21:30:38.287384+0000","last_peered":"2024-09-10T21:30:38.287384+0000","last_clean":"2024-09-10T21:30:38.287384+0000","last_became_active":"2024-09-10T21:30:37.142487+0000","last_became_peered":"2024-09-10T21:30:37.142487+0000","last_unstale":"2024-09-10T21:30:38.287384+0000","last_undegraded":"2024-09-10T21:30:38.287384+0000","last_fullsized":"2024-09-10T21:30:38.287384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:49.728530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143651+0000","last_change":"2024-09-10T21:30:36.143651+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143651+0000","last_undegraded":"2024-09-10T21:30:36.143651+0000","last_fullsized":"2024-09-10T21:30:36.143651+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159664+0000","last_change":"2024-09-10T21:30:36.156733+0000","last_active":"2024-09-10T21:30:36.159664+0000","last_peered":"2024-09-10T21:30:36.159664+0000","last_clean":"2024-09-10T21:30:36.159664+0000","last_became_active":"2024-09-10T21:30:36.156360+0000","last_became_peered":"2024-09-10T21:30:36.156360+0000","last_unstale":"2024-09-10T21:30:36.159664+0000","last_undegraded":"2024-09-10T21:30:36.159664+0000","last_fullsized":"2024-09-10T21:30:36.159664+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:23.429032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287825+0000","last_change":"2024-09-10T21:30:37.161240+0000","last_active":"2024-09-10T21:30:38.287825+0000","last_peered":"2024-09-10T21:30:38.287825+0000","last_clean":"2024-09-10T21:30:38.287825+0000","last_became_active":"2024-09-10T21:30:37.160927+0000","last_became_peered":"2024-09-10T21:30:37.160927+0000","last_unstale":"2024-09-10T21:30:38.287825+0000","last_undegraded":"2024-09-10T21:30:38.287825+0000","last_fullsized":"2024-09-10T21:30:38.287825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:57.275653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158055+0000","last_change":"2024-09-10T21:30:36.152982+0000","last_active":"2024-09-10T21:30:36.158055+0000","last_peered":"2024-09-10T21:30:36.158055+0000","last_clean":"2024-09-10T21:30:36.158055+0000","last_became_active":"2024-09-10T21:30:36.151294+0000","last_became_peered":"2024-09-10T21:30:36.151294+0000","last_unstale":"2024-09-10T21:30:36.158055+0000","last_undegraded":"2024-09-10T21:30:36.158055+0000","last_fullsized":"2024-09-10T21:30:36.158055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135162+0000","last_change":"2024-09-10T21:30:36.135162+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135162+0000","last_undegraded":"2024-09-10T21:30:36.135162+0000","last_fullsized":"2024-09-10T21:30:36.135162+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158290+0000","last_change":"2024-09-10T21:30:36.151783+0000","last_active":"2024-09-10T21:30:36.158290+0000","last_peered":"2024-09-10T21:30:36.158290+0000","last_clean":"2024-09-10T21:30:36.158290+0000","last_became_active":"2024-09-10T21:30:36.151113+0000","last_became_peered":"2024-09-10T21:30:36.151113+0000","last_unstale":"2024-09-10T21:30:36.158290+0000","last_undegraded":"2024-09-10T21:30:36.158290+0000","last_fullsized":"2024-09-10T21:30:36.158290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:48.032419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135977+0000","last_change":"2024-09-10T21:30:36.135977+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135977+0000","last_undegraded":"2024-09-10T21:30:36.135977+0000","last_fullsized":"2024-09-10T21:30:36.135977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.150307+0000","last_change":"2024-09-10T21:30:36.147985+0000","last_active":"2024-09-10T21:30:36.150307+0000","last_peered":"2024-09-10T21:30:36.150307+0000","last_clean":"2024-09-10T21:30:36.150307+0000","last_became_active":"2024-09-10T21:30:36.147685+0000","last_became_peered":"2024-09-10T21:30:36.147685+0000","last_unstale":"2024-09-10T21:30:36.150307+0000","last_undegraded":"2024-09-10T21:30:36.150307+0000","last_fullsized":"2024-09-10T21:30:36.150307+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:23.894646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139414+0000","last_change":"2024-09-10T21:30:36.139414+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139414+0000","last_undegraded":"2024-09-10T21:30:36.139414+0000","last_fullsized":"2024-09-10T21:30:36.139414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158555+0000","last_change":"2024-09-10T21:30:36.152597+0000","last_active":"2024-09-10T21:30:36.158555+0000","last_peered":"2024-09-10T21:30:36.158555+0000","last_clean":"2024-09-10T21:30:36.158555+0000","last_became_active":"2024-09-10T21:30:36.152336+0000","last_became_peered":"2024-09-10T21:30:36.152336+0000","last_unstale":"2024-09-10T21:30:36.158555+0000","last_undegraded":"2024-09-10T21:30:36.158555+0000","last_fullsized":"2024-09-10T21:30:36.158555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135003+0000","last_change":"2024-09-10T21:30:36.135003+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135003+0000","last_undegraded":"2024-09-10T21:30:36.135003+0000","last_fullsized":"2024-09-10T21:30:36.135003+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.160752+0000","last_change":"2024-09-10T21:30:36.156510+0000","last_active":"2024-09-10T21:30:36.160752+0000","last_peered":"2024-09-10T21:30:36.160752+0000","last_clean":"2024-09-10T21:30:36.160752+0000","last_became_active":"2024-09-10T21:30:36.156197+0000","last_became_peered":"2024-09-10T21:30:36.156197+0000","last_unstale":"2024-09-10T21:30:36.160752+0000","last_undegraded":"2024-09-10T21:30:36.160752+0000","last_fullsized":"2024-09-10T21:30:36.160752+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:38.551184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287564+0000","last_change":"2024-09-10T21:30:37.143279+0000","last_active":"2024-09-10T21:30:38.287564+0000","last_peered":"2024-09-10T21:30:38.287564+0000","last_clean":"2024-09-10T21:30:38.287564+0000","last_became_active":"2024-09-10T21:30:37.142984+0000","last_became_peered":"2024-09-10T21:30:37.142984+0000","last_unstale":"2024-09-10T21:30:38.287564+0000","last_undegraded":"2024-09-10T21:30:38.287564+0000","last_fullsized":"2024-09-10T21:30:38.287564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:12.128103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.152766+0000","last_change":"2024-09-10T21:30:36.150338+0000","last_active":"2024-09-10T21:30:36.152766+0000","last_peered":"2024-09-10T21:30:36.152766+0000","last_clean":"2024-09-10T21:30:36.152766+0000","last_became_active":"2024-09-10T21:30:36.149500+0000","last_became_peered":"2024-09-10T21:30:36.149500+0000","last_unstale":"2024-09-10T21:30:36.152766+0000","last_undegraded":"2024-09-10T21:30:36.152766+0000","last_fullsized":"2024-09-10T21:30:36.152766+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:23.529335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.145566+0000","last_change":"2024-09-10T21:30:36.145566+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.145566+0000","last_undegraded":"2024-09-10T21:30:36.145566+0000","last_fullsized":"2024-09-10T21:30:36.145566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158416+0000","last_change":"2024-09-10T21:30:36.152065+0000","last_active":"2024-09-10T21:30:36.158416+0000","last_peered":"2024-09-10T21:30:36.158416+0000","last_clean":"2024-09-10T21:30:36.158416+0000","last_became_active":"2024-09-10T21:30:36.151761+0000","last_became_peered":"2024-09-10T21:30:36.151761+0000","last_unstale":"2024-09-10T21:30:36.158416+0000","last_undegraded":"2024-09-10T21:30:36.158416+0000","last_fullsized":"2024-09-10T21:30:36.158416+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:03.575792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140059+0000","last_change":"2024-09-10T21:30:36.140059+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140059+0000","last_undegraded":"2024-09-10T21:30:36.140059+0000","last_fullsized":"2024-09-10T21:30:36.140059+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158268+0000","last_change":"2024-09-10T21:30:36.152474+0000","last_active":"2024-09-10T21:30:36.158268+0000","last_peered":"2024-09-10T21:30:36.158268+0000","last_clean":"2024-09-10T21:30:36.158268+0000","last_became_active":"2024-09-10T21:30:36.152180+0000","last_became_peered":"2024-09-10T21:30:36.152180+0000","last_unstale":"2024-09-10T21:30:36.158268+0000","last_undegraded":"2024-09-10T21:30:36.158268+0000","last_fullsized":"2024-09-10T21:30:36.158268+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:36.046838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135087+0000","last_change":"2024-09-10T21:30:36.135087+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135087+0000","last_undegraded":"2024-09-10T21:30:36.135087+0000","last_fullsized":"2024-09-10T21:30:36.135087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.152696+0000","last_change":"2024-09-10T21:30:36.149165+0000","last_active":"2024-09-10T21:30:36.152696+0000","last_peered":"2024-09-10T21:30:36.152696+0000","last_clean":"2024-09-10T21:30:36.152696+0000","last_became_active":"2024-09-10T21:30:36.148849+0000","last_became_peered":"2024-09-10T21:30:36.148849+0000","last_unstale":"2024-09-10T21:30:36.152696+0000","last_undegraded":"2024-09-10T21:30:36.152696+0000","last_fullsized":"2024-09-10T21:30:36.152696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:02.600501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140912+0000","last_change":"2024-09-10T21:30:36.140912+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140912+0000","last_undegraded":"2024-09-10T21:30:36.140912+0000","last_fullsized":"2024-09-10T21:30:36.140912+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157397+0000","last_change":"2024-09-10T21:30:36.149184+0000","last_active":"2024-09-10T21:30:36.157397+0000","last_peered":"2024-09-10T21:30:36.157397+0000","last_clean":"2024-09-10T21:30:36.157397+0000","last_became_active":"2024-09-10T21:30:36.148915+0000","last_became_peered":"2024-09-10T21:30:36.148915+0000","last_unstale":"2024-09-10T21:30:36.157397+0000","last_undegraded":"2024-09-10T21:30:36.157397+0000","last_fullsized":"2024-09-10T21:30:36.157397+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:46.052870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288504+0000","last_change":"2024-09-10T21:30:37.147434+0000","last_active":"2024-09-10T21:30:38.288504+0000","last_peered":"2024-09-10T21:30:38.288504+0000","last_clean":"2024-09-10T21:30:38.288504+0000","last_became_active":"2024-09-10T21:30:37.147160+0000","last_became_peered":"2024-09-10T21:30:37.147160+0000","last_unstale":"2024-09-10T21:30:38.288504+0000","last_undegraded":"2024-09-10T21:30:38.288504+0000","last_fullsized":"2024-09-10T21:30:38.288504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:07.679232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.161172+0000","last_change":"2024-09-10T21:30:36.158339+0000","last_active":"2024-09-10T21:30:36.161172+0000","last_peered":"2024-09-10T21:30:36.161172+0000","last_clean":"2024-09-10T21:30:36.161172+0000","last_became_active":"2024-09-10T21:30:36.158050+0000","last_became_peered":"2024-09-10T21:30:36.158050+0000","last_unstale":"2024-09-10T21:30:36.161172+0000","last_undegraded":"2024-09-10T21:30:36.161172+0000","last_fullsized":"2024-09-10T21:30:36.161172+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:36.544620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142693+0000","last_change":"2024-09-10T21:30:36.142693+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142693+0000","last_undegraded":"2024-09-10T21:30:36.142693+0000","last_fullsized":"2024-09-10T21:30:36.142693+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159535+0000","last_change":"2024-09-10T21:30:36.155028+0000","last_active":"2024-09-10T21:30:36.159535+0000","last_peered":"2024-09-10T21:30:36.159535+0000","last_clean":"2024-09-10T21:30:36.159535+0000","last_became_active":"2024-09-10T21:30:36.154105+0000","last_became_peered":"2024-09-10T21:30:36.154105+0000","last_unstale":"2024-09-10T21:30:36.159535+0000","last_undegraded":"2024-09-10T21:30:36.159535+0000","last_fullsized":"2024-09-10T21:30:36.159535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:51.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138771+0000","last_change":"2024-09-10T21:30:36.138771+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138771+0000","last_undegraded":"2024-09-10T21:30:36.138771+0000","last_fullsized":"2024-09-10T21:30:36.138771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155516+0000","last_change":"2024-09-10T21:30:36.149196+0000","last_active":"2024-09-10T21:30:36.155516+0000","last_peered":"2024-09-10T21:30:36.155516+0000","last_clean":"2024-09-10T21:30:36.155516+0000","last_became_active":"2024-09-10T21:30:36.148907+0000","last_became_peered":"2024-09-10T21:30:36.148907+0000","last_unstale":"2024-09-10T21:30:36.155516+0000","last_undegraded":"2024-09-10T21:30:36.155516+0000","last_fullsized":"2024-09-10T21:30:36.155516+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.146320+0000","last_change":"2024-09-10T21:30:36.146320+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.146320+0000","last_undegraded":"2024-09-10T21:30:36.146320+0000","last_fullsized":"2024-09-10T21:30:36.146320+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157943+0000","last_change":"2024-09-10T21:30:36.152984+0000","last_active":"2024-09-10T21:30:36.157943+0000","last_peered":"2024-09-10T21:30:36.157943+0000","last_clean":"2024-09-10T21:30:36.157943+0000","last_became_active":"2024-09-10T21:30:36.152704+0000","last_became_peered":"2024-09-10T21:30:36.152704+0000","last_unstale":"2024-09-10T21:30:36.157943+0000","last_undegraded":"2024-09-10T21:30:36.157943+0000","last_fullsized":"2024-09-10T21:30:36.157943+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138596+0000","last_change":"2024-09-10T21:30:36.138596+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138596+0000","last_undegraded":"2024-09-10T21:30:36.138596+0000","last_fullsized":"2024-09-10T21:30:36.138596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155485+0000","last_change":"2024-09-10T21:30:36.150376+0000","last_active":"2024-09-10T21:30:36.155485+0000","last_peered":"2024-09-10T21:30:36.155485+0000","last_clean":"2024-09-10T21:30:36.155485+0000","last_became_active":"2024-09-10T21:30:36.150090+0000","last_became_peered":"2024-09-10T21:30:36.150090+0000","last_unstale":"2024-09-10T21:30:36.155485+0000","last_undegraded":"2024-09-10T21:30:36.155485+0000","last_fullsized":"2024-09-10T21:30:36.155485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:15.145473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143402+0000","last_change":"2024-09-10T21:30:36.143402+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143402+0000","last_undegraded":"2024-09-10T21:30:36.143402+0000","last_fullsized":"2024-09-10T21:30:36.143402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158775+0000","last_change":"2024-09-10T21:30:36.153213+0000","last_active":"2024-09-10T21:30:36.158775+0000","last_peered":"2024-09-10T21:30:36.158775+0000","last_clean":"2024-09-10T21:30:36.158775+0000","last_became_active":"2024-09-10T21:30:36.152918+0000","last_became_peered":"2024-09-10T21:30:36.152918+0000","last_unstale":"2024-09-10T21:30:36.158775+0000","last_undegraded":"2024-09-10T21:30:36.158775+0000","last_fullsized":"2024-09-10T21:30:36.158775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:39.986638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134560+0000","last_change":"2024-09-10T21:30:36.134560+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134560+0000","last_undegraded":"2024-09-10T21:30:36.134560+0000","last_fullsized":"2024-09-10T21:30:36.134560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158805+0000","last_change":"2024-09-10T21:30:36.155934+0000","last_active":"2024-09-10T21:30:36.158805+0000","last_peered":"2024-09-10T21:30:36.158805+0000","last_clean":"2024-09-10T21:30:36.158805+0000","last_became_active":"2024-09-10T21:30:36.155669+0000","last_became_peered":"2024-09-10T21:30:36.155669+0000","last_unstale":"2024-09-10T21:30:36.158805+0000","last_undegraded":"2024-09-10T21:30:36.158805+0000","last_fullsized":"2024-09-10T21:30:36.158805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144601+0000","last_change":"2024-09-10T21:30:36.144601+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144601+0000","last_undegraded":"2024-09-10T21:30:36.144601+0000","last_fullsized":"2024-09-10T21:30:36.144601+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158093+0000","last_change":"2024-09-10T21:30:36.152792+0000","last_active":"2024-09-10T21:30:36.158093+0000","last_peered":"2024-09-10T21:30:36.158093+0000","last_clean":"2024-09-10T21:30:36.158093+0000","last_became_active":"2024-09-10T21:30:36.152526+0000","last_became_peered":"2024-09-10T21:30:36.152526+0000","last_unstale":"2024-09-10T21:30:36.158093+0000","last_undegraded":"2024-09-10T21:30:36.158093+0000","last_fullsized":"2024-09-10T21:30:36.158093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.541056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287996+0000","last_change":"2024-09-10T21:30:37.163752+0000","last_active":"2024-09-10T21:30:38.287996+0000","last_peered":"2024-09-10T21:30:38.287996+0000","last_clean":"2024-09-10T21:30:38.287996+0000","last_became_active":"2024-09-10T21:30:37.162710+0000","last_became_peered":"2024-09-10T21:30:37.162710+0000","last_unstale":"2024-09-10T21:30:38.287996+0000","last_undegraded":"2024-09-10T21:30:38.287996+0000","last_fullsized":"2024-09-10T21:30:38.287996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:33.852578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159112+0000","last_change":"2024-09-10T21:30:36.153508+0000","last_active":"2024-09-10T21:30:36.159112+0000","last_peered":"2024-09-10T21:30:36.159112+0000","last_clean":"2024-09-10T21:30:36.159112+0000","last_became_active":"2024-09-10T21:30:36.153255+0000","last_became_peered":"2024-09-10T21:30:36.153255+0000","last_unstale":"2024-09-10T21:30:36.159112+0000","last_undegraded":"2024-09-10T21:30:36.159112+0000","last_fullsized":"2024-09-10T21:30:36.159112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:44.746855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136690+0000","last_change":"2024-09-10T21:30:36.136690+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136690+0000","last_undegraded":"2024-09-10T21:30:36.136690+0000","last_fullsized":"2024-09-10T21:30:36.136690+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287713+0000","last_change":"2024-09-10T21:30:36.153764+0000","last_active":"2024-09-10T21:30:38.287713+0000","last_peered":"2024-09-10T21:30:38.287713+0000","last_clean":"2024-09-10T21:30:38.287713+0000","last_became_active":"2024-09-10T21:30:36.153453+0000","last_became_peered":"2024-09-10T21:30:36.153453+0000","last_unstale":"2024-09-10T21:30:38.287713+0000","last_undegraded":"2024-09-10T21:30:38.287713+0000","last_fullsized":"2024-09-10T21:30:38.287713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:28.260842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144090+0000","last_change":"2024-09-10T21:30:36.144090+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144090+0000","last_undegraded":"2024-09-10T21:30:36.144090+0000","last_fullsized":"2024-09-10T21:30:36.144090+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.147411+0000","last_change":"2024-09-10T21:30:36.145884+0000","last_active":"2024-09-10T21:30:36.147411+0000","last_peered":"2024-09-10T21:30:36.147411+0000","last_clean":"2024-09-10T21:30:36.147411+0000","last_became_active":"2024-09-10T21:30:36.145592+0000","last_became_peered":"2024-09-10T21:30:36.145592+0000","last_unstale":"2024-09-10T21:30:36.147411+0000","last_undegraded":"2024-09-10T21:30:36.147411+0000","last_fullsized":"2024-09-10T21:30:36.147411+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:52.517766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.145154+0000","last_change":"2024-09-10T21:30:36.145154+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.145154+0000","last_undegraded":"2024-09-10T21:30:36.145154+0000","last_fullsized":"2024-09-10T21:30:36.145154+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158363+0000","last_change":"2024-09-10T21:30:36.153170+0000","last_active":"2024-09-10T21:30:36.158363+0000","last_peered":"2024-09-10T21:30:36.158363+0000","last_clean":"2024-09-10T21:30:36.158363+0000","last_became_active":"2024-09-10T21:30:36.152223+0000","last_became_peered":"2024-09-10T21:30:36.152223+0000","last_unstale":"2024-09-10T21:30:36.158363+0000","last_undegraded":"2024-09-10T21:30:36.158363+0000","last_fullsized":"2024-09-10T21:30:36.158363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:00.035137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143977+0000","last_change":"2024-09-10T21:30:36.143977+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143977+0000","last_undegraded":"2024-09-10T21:30:36.143977+0000","last_fullsized":"2024-09-10T21:30:36.143977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157622+0000","last_change":"2024-09-10T21:30:36.149909+0000","last_active":"2024-09-10T21:30:36.157622+0000","last_peered":"2024-09-10T21:30:36.157622+0000","last_clean":"2024-09-10T21:30:36.157622+0000","last_became_active":"2024-09-10T21:30:36.149596+0000","last_became_peered":"2024-09-10T21:30:36.149596+0000","last_unstale":"2024-09-10T21:30:36.157622+0000","last_undegraded":"2024-09-10T21:30:36.157622+0000","last_fullsized":"2024-09-10T21:30:36.157622+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:27.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140295+0000","last_change":"2024-09-10T21:30:36.140295+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140295+0000","last_undegraded":"2024-09-10T21:30:36.140295+0000","last_fullsized":"2024-09-10T21:30:36.140295+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159054+0000","last_change":"2024-09-10T21:30:36.151988+0000","last_active":"2024-09-10T21:30:36.159054+0000","last_peered":"2024-09-10T21:30:36.159054+0000","last_clean":"2024-09-10T21:30:36.159054+0000","last_became_active":"2024-09-10T21:30:36.151626+0000","last_became_peered":"2024-09-10T21:30:36.151626+0000","last_unstale":"2024-09-10T21:30:36.159054+0000","last_undegraded":"2024-09-10T21:30:36.159054+0000","last_fullsized":"2024-09-10T21:30:36.159054+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:01.606529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138904+0000","last_change":"2024-09-10T21:30:36.138904+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138904+0000","last_undegraded":"2024-09-10T21:30:36.138904+0000","last_fullsized":"2024-09-10T21:30:36.138904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159234+0000","last_change":"2024-09-10T21:30:36.156216+0000","last_active":"2024-09-10T21:30:36.159234+0000","last_peered":"2024-09-10T21:30:36.159234+0000","last_clean":"2024-09-10T21:30:36.159234+0000","last_became_active":"2024-09-10T21:30:36.155962+0000","last_became_peered":"2024-09-10T21:30:36.155962+0000","last_unstale":"2024-09-10T21:30:36.159234+0000","last_undegraded":"2024-09-10T21:30:36.159234+0000","last_fullsized":"2024-09-10T21:30:36.159234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:13.896623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141644+0000","last_change":"2024-09-10T21:30:36.141644+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141644+0000","last_undegraded":"2024-09-10T21:30:36.141644+0000","last_fullsized":"2024-09-10T21:30:36.141644+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157051+0000","last_change":"2024-09-10T21:30:36.146558+0000","last_active":"2024-09-10T21:30:36.157051+0000","last_peered":"2024-09-10T21:30:36.157051+0000","last_clean":"2024-09-10T21:30:36.157051+0000","last_became_active":"2024-09-10T21:30:36.144327+0000","last_became_peered":"2024-09-10T21:30:36.144327+0000","last_unstale":"2024-09-10T21:30:36.157051+0000","last_undegraded":"2024-09-10T21:30:36.157051+0000","last_fullsized":"2024-09-10T21:30:36.157051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:40.224703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287947+0000","last_change":"2024-09-10T21:30:37.162963+0000","last_active":"2024-09-10T21:30:38.287947+0000","last_peered":"2024-09-10T21:30:38.287947+0000","last_clean":"2024-09-10T21:30:38.287947+0000","last_became_active":"2024-09-10T21:30:37.162366+0000","last_became_peered":"2024-09-10T21:30:37.162366+0000","last_unstale":"2024-09-10T21:30:38.287947+0000","last_undegraded":"2024-09-10T21:30:38.287947+0000","last_fullsized":"2024-09-10T21:30:38.287947+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:37.376166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.161192+0000","last_change":"2024-09-10T21:30:36.159592+0000","last_active":"2024-09-10T21:30:36.161192+0000","last_peered":"2024-09-10T21:30:36.161192+0000","last_clean":"2024-09-10T21:30:36.161192+0000","last_became_active":"2024-09-10T21:30:36.159354+0000","last_became_peered":"2024-09-10T21:30:36.159354+0000","last_unstale":"2024-09-10T21:30:36.161192+0000","last_undegraded":"2024-09-10T21:30:36.161192+0000","last_fullsized":"2024-09-10T21:30:36.161192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:26.541443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.133612+0000","last_change":"2024-09-10T21:30:36.133612+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.133612+0000","last_undegraded":"2024-09-10T21:30:36.133612+0000","last_fullsized":"2024-09-10T21:30:36.133612+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159701+0000","last_change":"2024-09-10T21:30:36.157140+0000","last_active":"2024-09-10T21:30:36.159701+0000","last_peered":"2024-09-10T21:30:36.159701+0000","last_clean":"2024-09-10T21:30:36.159701+0000","last_became_active":"2024-09-10T21:30:36.156861+0000","last_became_peered":"2024-09-10T21:30:36.156861+0000","last_unstale":"2024-09-10T21:30:36.159701+0000","last_undegraded":"2024-09-10T21:30:36.159701+0000","last_fullsized":"2024-09-10T21:30:36.159701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:09.577405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137154+0000","last_change":"2024-09-10T21:30:36.137154+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137154+0000","last_undegraded":"2024-09-10T21:30:36.137154+0000","last_fullsized":"2024-09-10T21:30:36.137154+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.154650+0000","last_change":"2024-09-10T21:30:36.147814+0000","last_active":"2024-09-10T21:30:36.154650+0000","last_peered":"2024-09-10T21:30:36.154650+0000","last_clean":"2024-09-10T21:30:36.154650+0000","last_became_active":"2024-09-10T21:30:36.147488+0000","last_became_peered":"2024-09-10T21:30:36.147488+0000","last_unstale":"2024-09-10T21:30:36.154650+0000","last_undegraded":"2024-09-10T21:30:36.154650+0000","last_fullsized":"2024-09-10T21:30:36.154650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:23.302054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287689+0000","last_change":"2024-09-10T21:30:37.164739+0000","last_active":"2024-09-10T21:30:38.287689+0000","last_peered":"2024-09-10T21:30:38.287689+0000","last_clean":"2024-09-10T21:30:38.287689+0000","last_became_active":"2024-09-10T21:30:37.164473+0000","last_became_peered":"2024-09-10T21:30:37.164473+0000","last_unstale":"2024-09-10T21:30:38.287689+0000","last_undegraded":"2024-09-10T21:30:38.287689+0000","last_fullsized":"2024-09-10T21:30:38.287689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:36.726703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159717+0000","last_change":"2024-09-10T21:30:36.156585+0000","last_active":"2024-09-10T21:30:36.159717+0000","last_peered":"2024-09-10T21:30:36.159717+0000","last_clean":"2024-09-10T21:30:36.159717+0000","last_became_active":"2024-09-10T21:30:36.156288+0000","last_became_peered":"2024-09-10T21:30:36.156288+0000","last_unstale":"2024-09-10T21:30:36.159717+0000","last_undegraded":"2024-09-10T21:30:36.159717+0000","last_fullsized":"2024-09-10T21:30:36.159717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:24.061853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141257+0000","last_change":"2024-09-10T21:30:36.141257+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141257+0000","last_undegraded":"2024-09-10T21:30:36.141257+0000","last_fullsized":"2024-09-10T21:30:36.141257+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155869+0000","last_change":"2024-09-10T21:30:36.151215+0000","last_active":"2024-09-10T21:30:36.155869+0000","last_peered":"2024-09-10T21:30:36.155869+0000","last_clean":"2024-09-10T21:30:36.155869+0000","last_became_active":"2024-09-10T21:30:36.150699+0000","last_became_peered":"2024-09-10T21:30:36.150699+0000","last_unstale":"2024-09-10T21:30:36.155869+0000","last_undegraded":"2024-09-10T21:30:36.155869+0000","last_fullsized":"2024-09-10T21:30:36.155869+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.150455+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:30:36.150455+0000","last_peered":"2024-09-10T21:30:36.150455+0000","last_clean":"2024-09-10T21:30:36.150455+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:30:36.150455+0000","last_undegraded":"2024-09-10T21:30:36.150455+0000","last_fullsized":"2024-09-10T21:30:36.150455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1102376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1102376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1138688,"data_stored":525842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:30:41.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:30:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 -- 172.21.15.97:0/150582366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 msgr2=0x7f5b9c152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 --2- 172.21.15.97:0/150582366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c152760 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009c80 tx=0x7f5b8c01cd70 comp rx=0 tx=0).stop 2024-09-10T21:30:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 -- 172.21.15.97:0/150582366 shutdown_connections 2024-09-10T21:30:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 --2- 172.21.15.97:0/150582366 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b9c06d180 0x7f5b9c152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 --2- 172.21.15.97:0/150582366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c152760 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 -- 172.21.15.97:0/150582366 >> 172.21.15.97:0/150582366 conn(0x7f5b9c06e2b0 msgr2=0x7f5b9c0b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 -- 172.21.15.97:0/150582366 shutdown_connections 2024-09-10T21:30:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 -- 172.21.15.97:0/150582366 wait complete. 2024-09-10T21:30:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.900+0000 7f5ba3800640 1 Processor -- start 2024-09-10T21:30:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 -- start start 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c06d180 0x7f5b9c143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b9c0d01a0 0x7f5b9c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c168c90 con 0x7f5b9c06cda0 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c168b10 con 0x7f5b9c06d180 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c168e10 con 0x7f5b9c0d01a0 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48660/0 (socket says 172.21.15.97:48660) 2024-09-10T21:30:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 -- 172.21.15.97:0/2958958701 learned_addr learned my addr 172.21.15.97:0/2958958701 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5b9be00640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c06d180 0x7f5b9c143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba1800640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b9c0d01a0 0x7f5b9c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 -- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b9c0d01a0 msgr2=0x7f5b9c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b9c0d01a0 0x7f5b9c151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 -- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c06d180 msgr2=0x7f5b9c143b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c06d180 0x7f5b9c143b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 -- 172.21.15.97:0/2958958701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1440a0 con 0x7f5b9c06cda0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba0e00640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c155b10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c000f90 tx=0x7f5b8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5b98c00640 1 -- 172.21.15.97:0/2958958701 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0bb070 con 0x7f5b9c06cda0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5b98c00640 1 -- 172.21.15.97:0/2958958701 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b8c0045d0 con 0x7f5b9c06cda0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5b98c00640 1 -- 172.21.15.97:0/2958958701 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0b2040 con 0x7f5b9c06cda0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1561b0 con 0x7f5b9c06cda0 2024-09-10T21:30:41.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c143360 con 0x7f5b9c06cda0 2024-09-10T21:30:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.904+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c15c260 con 0x7f5b9c06cda0 2024-09-10T21:30:41.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.908+0000 7f5b98c00640 1 -- 172.21.15.97:0/2958958701 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5b8c0b5030 con 0x7f5b9c06cda0 2024-09-10T21:30:41.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.908+0000 7f5b98c00640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5b7c072610 0x7f5b7c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.908+0000 7f5b98c00640 1 -- 172.21.15.97:0/2958958701 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f5b8c12d340 con 0x7f5b9c06cda0 2024-09-10T21:30:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.908+0000 7f5b9be00640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5b7c072610 0x7f5b7c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:41.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.908+0000 7f5b9be00640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5b7c072610 0x7f5b7c074ad0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b880058d0 tx=0x7f5b88005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:41.912+0000 7f5b98c00640 1 -- 172.21.15.97:0/2958958701 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5b8c016990 con 0x7f5b9c06cda0 2024-09-10T21:30:42.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.076+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b9c0b2c50 con 0x7f5b7c072610 2024-09-10T21:30:42.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.080+0000 7f5b98c00640 1 -- 172.21.15.97:0/2958958701 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247758 (secure 0 0 0) 0x7f5b9c0b2c50 con 0x7f5b7c072610 2024-09-10T21:30:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:42.089 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:30:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5b7c072610 msgr2=0x7f5b7c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5b7c072610 0x7f5b7c074ad0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b880058d0 tx=0x7f5b88005860 comp rx=0 tx=0).stop 2024-09-10T21:30:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 msgr2=0x7f5b9c155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c155b10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c000f90 tx=0x7f5b8c0042f0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 shutdown_connections 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5b7c072610 0x7f5b7c074ad0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5b9c0d01a0 0x7f5b9c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c06d180 0x7f5b9c143b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 --2- 172.21.15.97:0/2958958701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9c06cda0 0x7f5b9c155b10 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 >> 172.21.15.97:0/2958958701 conn(0x7f5b9c06e2b0 msgr2=0x7f5b9c145b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 shutdown_connections 2024-09-10T21:30:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.084+0000 7f5ba3800640 1 -- 172.21.15.97:0/2958958701 wait complete. 2024-09-10T21:30:42.108 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T21:30:40.168976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255260,"kb_used_data":9888,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719460,"statfs":{"total":773094113280,"available":772832727040,"internally_reserved":0,"allocated":10125312,"data_stored":5290634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.045881"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141245+0000","last_change":"2024-09-10T21:30:36.141245+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141245+0000","last_undegraded":"2024-09-10T21:30:36.141245+0000","last_fullsized":"2024-09-10T21:30:36.141245+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.146230+0000","last_change":"2024-09-10T21:30:36.146230+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.146230+0000","last_undegraded":"2024-09-10T21:30:36.146230+0000","last_fullsized":"2024-09-10T21:30:36.146230+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136865+0000","last_change":"2024-09-10T21:30:36.136865+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136865+0000","last_undegraded":"2024-09-10T21:30:36.136865+0000","last_fullsized":"2024-09-10T21:30:36.136865+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135307+0000","last_change":"2024-09-10T21:30:36.135307+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135307+0000","last_undegraded":"2024-09-10T21:30:36.135307+0000","last_fullsized":"2024-09-10T21:30:36.135307+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139784+0000","last_change":"2024-09-10T21:30:36.139784+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139784+0000","last_undegraded":"2024-09-10T21:30:36.139784+0000","last_fullsized":"2024-09-10T21:30:36.139784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287577+0000","last_change":"2024-09-10T21:30:37.156704+0000","last_active":"2024-09-10T21:30:38.287577+0000","last_peered":"2024-09-10T21:30:38.287577+0000","last_clean":"2024-09-10T21:30:38.287577+0000","last_became_active":"2024-09-10T21:30:37.154734+0000","last_became_peered":"2024-09-10T21:30:37.154734+0000","last_unstale":"2024-09-10T21:30:38.287577+0000","last_undegraded":"2024-09-10T21:30:38.287577+0000","last_fullsized":"2024-09-10T21:30:38.287577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:59.951655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137207+0000","last_change":"2024-09-10T21:30:36.137207+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137207+0000","last_undegraded":"2024-09-10T21:30:36.137207+0000","last_fullsized":"2024-09-10T21:30:36.137207+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287144+0000","last_change":"2024-09-10T21:30:37.146717+0000","last_active":"2024-09-10T21:30:38.287144+0000","last_peered":"2024-09-10T21:30:38.287144+0000","last_clean":"2024-09-10T21:30:38.287144+0000","last_became_active":"2024-09-10T21:30:37.146428+0000","last_became_peered":"2024-09-10T21:30:37.146428+0000","last_unstale":"2024-09-10T21:30:38.287144+0000","last_undegraded":"2024-09-10T21:30:38.287144+0000","last_fullsized":"2024-09-10T21:30:38.287144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:18.307373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137459+0000","last_change":"2024-09-10T21:30:36.137459+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137459+0000","last_undegraded":"2024-09-10T21:30:36.137459+0000","last_fullsized":"2024-09-10T21:30:36.137459+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137009+0000","last_change":"2024-09-10T21:30:36.137009+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137009+0000","last_undegraded":"2024-09-10T21:30:36.137009+0000","last_fullsized":"2024-09-10T21:30:36.137009+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137328+0000","last_change":"2024-09-10T21:30:36.137328+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137328+0000","last_undegraded":"2024-09-10T21:30:36.137328+0000","last_fullsized":"2024-09-10T21:30:36.137328+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134793+0000","last_change":"2024-09-10T21:30:36.134793+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134793+0000","last_undegraded":"2024-09-10T21:30:36.134793+0000","last_fullsized":"2024-09-10T21:30:36.134793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141795+0000","last_change":"2024-09-10T21:30:36.141795+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141795+0000","last_undegraded":"2024-09-10T21:30:36.141795+0000","last_fullsized":"2024-09-10T21:30:36.141795+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136360+0000","last_change":"2024-09-10T21:30:36.136360+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136360+0000","last_undegraded":"2024-09-10T21:30:36.136360+0000","last_fullsized":"2024-09-10T21:30:36.136360+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134234+0000","last_change":"2024-09-10T21:30:36.134234+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134234+0000","last_undegraded":"2024-09-10T21:30:36.134234+0000","last_fullsized":"2024-09-10T21:30:36.134234+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287182+0000","last_change":"2024-09-10T21:30:37.152682+0000","last_active":"2024-09-10T21:30:38.287182+0000","last_peered":"2024-09-10T21:30:38.287182+0000","last_clean":"2024-09-10T21:30:38.287182+0000","last_became_active":"2024-09-10T21:30:37.152308+0000","last_became_peered":"2024-09-10T21:30:37.152308+0000","last_unstale":"2024-09-10T21:30:38.287182+0000","last_undegraded":"2024-09-10T21:30:38.287182+0000","last_fullsized":"2024-09-10T21:30:38.287182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:39.533064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288396+0000","last_change":"2024-09-10T21:30:37.155784+0000","last_active":"2024-09-10T21:30:38.288396+0000","last_peered":"2024-09-10T21:30:38.288396+0000","last_clean":"2024-09-10T21:30:38.288396+0000","last_became_active":"2024-09-10T21:30:37.155514+0000","last_became_peered":"2024-09-10T21:30:37.155514+0000","last_unstale":"2024-09-10T21:30:38.288396+0000","last_undegraded":"2024-09-10T21:30:38.288396+0000","last_fullsized":"2024-09-10T21:30:38.288396+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:10.496991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135754+0000","last_change":"2024-09-10T21:30:36.135754+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135754+0000","last_undegraded":"2024-09-10T21:30:36.135754+0000","last_fullsized":"2024-09-10T21:30:36.135754+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144667+0000","last_change":"2024-09-10T21:30:36.144667+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144667+0000","last_undegraded":"2024-09-10T21:30:36.144667+0000","last_fullsized":"2024-09-10T21:30:36.144667+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287734+0000","last_change":"2024-09-10T21:30:37.161159+0000","last_active":"2024-09-10T21:30:38.287734+0000","last_peered":"2024-09-10T21:30:38.287734+0000","last_clean":"2024-09-10T21:30:38.287734+0000","last_became_active":"2024-09-10T21:30:37.160818+0000","last_became_peered":"2024-09-10T21:30:37.160818+0000","last_unstale":"2024-09-10T21:30:38.287734+0000","last_undegraded":"2024-09-10T21:30:38.287734+0000","last_fullsized":"2024-09-10T21:30:38.287734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:10.536055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139341+0000","last_change":"2024-09-10T21:30:36.139341+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139341+0000","last_undegraded":"2024-09-10T21:30:36.139341+0000","last_fullsized":"2024-09-10T21:30:36.139341+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144558+0000","last_change":"2024-09-10T21:30:36.144558+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144558+0000","last_undegraded":"2024-09-10T21:30:36.144558+0000","last_fullsized":"2024-09-10T21:30:36.144558+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139205+0000","last_change":"2024-09-10T21:30:36.139205+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139205+0000","last_undegraded":"2024-09-10T21:30:36.139205+0000","last_fullsized":"2024-09-10T21:30:36.139205+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143144+0000","last_change":"2024-09-10T21:30:36.143144+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143144+0000","last_undegraded":"2024-09-10T21:30:36.143144+0000","last_fullsized":"2024-09-10T21:30:36.143144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142060+0000","last_change":"2024-09-10T21:30:36.142060+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142060+0000","last_undegraded":"2024-09-10T21:30:36.142060+0000","last_fullsized":"2024-09-10T21:30:36.142060+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139633+0000","last_change":"2024-09-10T21:30:36.139633+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139633+0000","last_undegraded":"2024-09-10T21:30:36.139633+0000","last_fullsized":"2024-09-10T21:30:36.139633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135362+0000","last_change":"2024-09-10T21:30:36.135362+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135362+0000","last_undegraded":"2024-09-10T21:30:36.135362+0000","last_fullsized":"2024-09-10T21:30:36.135362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142361+0000","last_change":"2024-09-10T21:30:36.142361+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142361+0000","last_undegraded":"2024-09-10T21:30:36.142361+0000","last_fullsized":"2024-09-10T21:30:36.142361+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138287+0000","last_change":"2024-09-10T21:30:36.138287+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138287+0000","last_undegraded":"2024-09-10T21:30:36.138287+0000","last_fullsized":"2024-09-10T21:30:36.138287+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137350+0000","last_change":"2024-09-10T21:30:36.137350+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137350+0000","last_undegraded":"2024-09-10T21:30:36.137350+0000","last_fullsized":"2024-09-10T21:30:36.137350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137784+0000","last_change":"2024-09-10T21:30:36.137784+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137784+0000","last_undegraded":"2024-09-10T21:30:36.137784+0000","last_fullsized":"2024-09-10T21:30:36.137784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287384+0000","last_change":"2024-09-10T21:30:37.142844+0000","last_active":"2024-09-10T21:30:38.287384+0000","last_peered":"2024-09-10T21:30:38.287384+0000","last_clean":"2024-09-10T21:30:38.287384+0000","last_became_active":"2024-09-10T21:30:37.142487+0000","last_became_peered":"2024-09-10T21:30:37.142487+0000","last_unstale":"2024-09-10T21:30:38.287384+0000","last_undegraded":"2024-09-10T21:30:38.287384+0000","last_fullsized":"2024-09-10T21:30:38.287384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:49.728530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143651+0000","last_change":"2024-09-10T21:30:36.143651+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143651+0000","last_undegraded":"2024-09-10T21:30:36.143651+0000","last_fullsized":"2024-09-10T21:30:36.143651+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159664+0000","last_change":"2024-09-10T21:30:36.156733+0000","last_active":"2024-09-10T21:30:36.159664+0000","last_peered":"2024-09-10T21:30:36.159664+0000","last_clean":"2024-09-10T21:30:36.159664+0000","last_became_active":"2024-09-10T21:30:36.156360+0000","last_became_peered":"2024-09-10T21:30:36.156360+0000","last_unstale":"2024-09-10T21:30:36.159664+0000","last_undegraded":"2024-09-10T21:30:36.159664+0000","last_fullsized":"2024-09-10T21:30:36.159664+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:23.429032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287825+0000","last_change":"2024-09-10T21:30:37.161240+0000","last_active":"2024-09-10T21:30:38.287825+0000","last_peered":"2024-09-10T21:30:38.287825+0000","last_clean":"2024-09-10T21:30:38.287825+0000","last_became_active":"2024-09-10T21:30:37.160927+0000","last_became_peered":"2024-09-10T21:30:37.160927+0000","last_unstale":"2024-09-10T21:30:38.287825+0000","last_undegraded":"2024-09-10T21:30:38.287825+0000","last_fullsized":"2024-09-10T21:30:38.287825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:57.275653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158055+0000","last_change":"2024-09-10T21:30:36.152982+0000","last_active":"2024-09-10T21:30:36.158055+0000","last_peered":"2024-09-10T21:30:36.158055+0000","last_clean":"2024-09-10T21:30:36.158055+0000","last_became_active":"2024-09-10T21:30:36.151294+0000","last_became_peered":"2024-09-10T21:30:36.151294+0000","last_unstale":"2024-09-10T21:30:36.158055+0000","last_undegraded":"2024-09-10T21:30:36.158055+0000","last_fullsized":"2024-09-10T21:30:36.158055+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135162+0000","last_change":"2024-09-10T21:30:36.135162+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135162+0000","last_undegraded":"2024-09-10T21:30:36.135162+0000","last_fullsized":"2024-09-10T21:30:36.135162+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158290+0000","last_change":"2024-09-10T21:30:36.151783+0000","last_active":"2024-09-10T21:30:36.158290+0000","last_peered":"2024-09-10T21:30:36.158290+0000","last_clean":"2024-09-10T21:30:36.158290+0000","last_became_active":"2024-09-10T21:30:36.151113+0000","last_became_peered":"2024-09-10T21:30:36.151113+0000","last_unstale":"2024-09-10T21:30:36.158290+0000","last_undegraded":"2024-09-10T21:30:36.158290+0000","last_fullsized":"2024-09-10T21:30:36.158290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:48.032419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135977+0000","last_change":"2024-09-10T21:30:36.135977+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135977+0000","last_undegraded":"2024-09-10T21:30:36.135977+0000","last_fullsized":"2024-09-10T21:30:36.135977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.150307+0000","last_change":"2024-09-10T21:30:36.147985+0000","last_active":"2024-09-10T21:30:36.150307+0000","last_peered":"2024-09-10T21:30:36.150307+0000","last_clean":"2024-09-10T21:30:36.150307+0000","last_became_active":"2024-09-10T21:30:36.147685+0000","last_became_peered":"2024-09-10T21:30:36.147685+0000","last_unstale":"2024-09-10T21:30:36.150307+0000","last_undegraded":"2024-09-10T21:30:36.150307+0000","last_fullsized":"2024-09-10T21:30:36.150307+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:23.894646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.139414+0000","last_change":"2024-09-10T21:30:36.139414+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.139414+0000","last_undegraded":"2024-09-10T21:30:36.139414+0000","last_fullsized":"2024-09-10T21:30:36.139414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158555+0000","last_change":"2024-09-10T21:30:36.152597+0000","last_active":"2024-09-10T21:30:36.158555+0000","last_peered":"2024-09-10T21:30:36.158555+0000","last_clean":"2024-09-10T21:30:36.158555+0000","last_became_active":"2024-09-10T21:30:36.152336+0000","last_became_peered":"2024-09-10T21:30:36.152336+0000","last_unstale":"2024-09-10T21:30:36.158555+0000","last_undegraded":"2024-09-10T21:30:36.158555+0000","last_fullsized":"2024-09-10T21:30:36.158555+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135003+0000","last_change":"2024-09-10T21:30:36.135003+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135003+0000","last_undegraded":"2024-09-10T21:30:36.135003+0000","last_fullsized":"2024-09-10T21:30:36.135003+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.160752+0000","last_change":"2024-09-10T21:30:36.156510+0000","last_active":"2024-09-10T21:30:36.160752+0000","last_peered":"2024-09-10T21:30:36.160752+0000","last_clean":"2024-09-10T21:30:36.160752+0000","last_became_active":"2024-09-10T21:30:36.156197+0000","last_became_peered":"2024-09-10T21:30:36.156197+0000","last_unstale":"2024-09-10T21:30:36.160752+0000","last_undegraded":"2024-09-10T21:30:36.160752+0000","last_fullsized":"2024-09-10T21:30:36.160752+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:38.551184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287564+0000","last_change":"2024-09-10T21:30:37.143279+0000","last_active":"2024-09-10T21:30:38.287564+0000","last_peered":"2024-09-10T21:30:38.287564+0000","last_clean":"2024-09-10T21:30:38.287564+0000","last_became_active":"2024-09-10T21:30:37.142984+0000","last_became_peered":"2024-09-10T21:30:37.142984+0000","last_unstale":"2024-09-10T21:30:38.287564+0000","last_undegraded":"2024-09-10T21:30:38.287564+0000","last_fullsized":"2024-09-10T21:30:38.287564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:12.128103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.152766+0000","last_change":"2024-09-10T21:30:36.150338+0000","last_active":"2024-09-10T21:30:36.152766+0000","last_peered":"2024-09-10T21:30:36.152766+0000","last_clean":"2024-09-10T21:30:36.152766+0000","last_became_active":"2024-09-10T21:30:36.149500+0000","last_became_peered":"2024-09-10T21:30:36.149500+0000","last_unstale":"2024-09-10T21:30:36.152766+0000","last_undegraded":"2024-09-10T21:30:36.152766+0000","last_fullsized":"2024-09-10T21:30:36.152766+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:23.529335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.145566+0000","last_change":"2024-09-10T21:30:36.145566+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.145566+0000","last_undegraded":"2024-09-10T21:30:36.145566+0000","last_fullsized":"2024-09-10T21:30:36.145566+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158416+0000","last_change":"2024-09-10T21:30:36.152065+0000","last_active":"2024-09-10T21:30:36.158416+0000","last_peered":"2024-09-10T21:30:36.158416+0000","last_clean":"2024-09-10T21:30:36.158416+0000","last_became_active":"2024-09-10T21:30:36.151761+0000","last_became_peered":"2024-09-10T21:30:36.151761+0000","last_unstale":"2024-09-10T21:30:36.158416+0000","last_undegraded":"2024-09-10T21:30:36.158416+0000","last_fullsized":"2024-09-10T21:30:36.158416+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:03.575792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140059+0000","last_change":"2024-09-10T21:30:36.140059+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140059+0000","last_undegraded":"2024-09-10T21:30:36.140059+0000","last_fullsized":"2024-09-10T21:30:36.140059+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158268+0000","last_change":"2024-09-10T21:30:36.152474+0000","last_active":"2024-09-10T21:30:36.158268+0000","last_peered":"2024-09-10T21:30:36.158268+0000","last_clean":"2024-09-10T21:30:36.158268+0000","last_became_active":"2024-09-10T21:30:36.152180+0000","last_became_peered":"2024-09-10T21:30:36.152180+0000","last_unstale":"2024-09-10T21:30:36.158268+0000","last_undegraded":"2024-09-10T21:30:36.158268+0000","last_fullsized":"2024-09-10T21:30:36.158268+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:36.046838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.135087+0000","last_change":"2024-09-10T21:30:36.135087+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.135087+0000","last_undegraded":"2024-09-10T21:30:36.135087+0000","last_fullsized":"2024-09-10T21:30:36.135087+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.152696+0000","last_change":"2024-09-10T21:30:36.149165+0000","last_active":"2024-09-10T21:30:36.152696+0000","last_peered":"2024-09-10T21:30:36.152696+0000","last_clean":"2024-09-10T21:30:36.152696+0000","last_became_active":"2024-09-10T21:30:36.148849+0000","last_became_peered":"2024-09-10T21:30:36.148849+0000","last_unstale":"2024-09-10T21:30:36.152696+0000","last_undegraded":"2024-09-10T21:30:36.152696+0000","last_fullsized":"2024-09-10T21:30:36.152696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:02.600501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140912+0000","last_change":"2024-09-10T21:30:36.140912+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140912+0000","last_undegraded":"2024-09-10T21:30:36.140912+0000","last_fullsized":"2024-09-10T21:30:36.140912+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157397+0000","last_change":"2024-09-10T21:30:36.149184+0000","last_active":"2024-09-10T21:30:36.157397+0000","last_peered":"2024-09-10T21:30:36.157397+0000","last_clean":"2024-09-10T21:30:36.157397+0000","last_became_active":"2024-09-10T21:30:36.148915+0000","last_became_peered":"2024-09-10T21:30:36.148915+0000","last_unstale":"2024-09-10T21:30:36.157397+0000","last_undegraded":"2024-09-10T21:30:36.157397+0000","last_fullsized":"2024-09-10T21:30:36.157397+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:46.052870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288504+0000","last_change":"2024-09-10T21:30:37.147434+0000","last_active":"2024-09-10T21:30:38.288504+0000","last_peered":"2024-09-10T21:30:38.288504+0000","last_clean":"2024-09-10T21:30:38.288504+0000","last_became_active":"2024-09-10T21:30:37.147160+0000","last_became_peered":"2024-09-10T21:30:37.147160+0000","last_unstale":"2024-09-10T21:30:38.288504+0000","last_undegraded":"2024-09-10T21:30:38.288504+0000","last_fullsized":"2024-09-10T21:30:38.288504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:07.679232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.161172+0000","last_change":"2024-09-10T21:30:36.158339+0000","last_active":"2024-09-10T21:30:36.161172+0000","last_peered":"2024-09-10T21:30:36.161172+0000","last_clean":"2024-09-10T21:30:36.161172+0000","last_became_active":"2024-09-10T21:30:36.158050+0000","last_became_peered":"2024-09-10T21:30:36.158050+0000","last_unstale":"2024-09-10T21:30:36.161172+0000","last_undegraded":"2024-09-10T21:30:36.161172+0000","last_fullsized":"2024-09-10T21:30:36.161172+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:36.544620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.142693+0000","last_change":"2024-09-10T21:30:36.142693+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.142693+0000","last_undegraded":"2024-09-10T21:30:36.142693+0000","last_fullsized":"2024-09-10T21:30:36.142693+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159535+0000","last_change":"2024-09-10T21:30:36.155028+0000","last_active":"2024-09-10T21:30:36.159535+0000","last_peered":"2024-09-10T21:30:36.159535+0000","last_clean":"2024-09-10T21:30:36.159535+0000","last_became_active":"2024-09-10T21:30:36.154105+0000","last_became_peered":"2024-09-10T21:30:36.154105+0000","last_unstale":"2024-09-10T21:30:36.159535+0000","last_undegraded":"2024-09-10T21:30:36.159535+0000","last_fullsized":"2024-09-10T21:30:36.159535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:51.290074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138771+0000","last_change":"2024-09-10T21:30:36.138771+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138771+0000","last_undegraded":"2024-09-10T21:30:36.138771+0000","last_fullsized":"2024-09-10T21:30:36.138771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155516+0000","last_change":"2024-09-10T21:30:36.149196+0000","last_active":"2024-09-10T21:30:36.155516+0000","last_peered":"2024-09-10T21:30:36.155516+0000","last_clean":"2024-09-10T21:30:36.155516+0000","last_became_active":"2024-09-10T21:30:36.148907+0000","last_became_peered":"2024-09-10T21:30:36.148907+0000","last_unstale":"2024-09-10T21:30:36.155516+0000","last_undegraded":"2024-09-10T21:30:36.155516+0000","last_fullsized":"2024-09-10T21:30:36.155516+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.146320+0000","last_change":"2024-09-10T21:30:36.146320+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.146320+0000","last_undegraded":"2024-09-10T21:30:36.146320+0000","last_fullsized":"2024-09-10T21:30:36.146320+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157943+0000","last_change":"2024-09-10T21:30:36.152984+0000","last_active":"2024-09-10T21:30:36.157943+0000","last_peered":"2024-09-10T21:30:36.157943+0000","last_clean":"2024-09-10T21:30:36.157943+0000","last_became_active":"2024-09-10T21:30:36.152704+0000","last_became_peered":"2024-09-10T21:30:36.152704+0000","last_unstale":"2024-09-10T21:30:36.157943+0000","last_undegraded":"2024-09-10T21:30:36.157943+0000","last_fullsized":"2024-09-10T21:30:36.157943+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138596+0000","last_change":"2024-09-10T21:30:36.138596+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138596+0000","last_undegraded":"2024-09-10T21:30:36.138596+0000","last_fullsized":"2024-09-10T21:30:36.138596+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155485+0000","last_change":"2024-09-10T21:30:36.150376+0000","last_active":"2024-09-10T21:30:36.155485+0000","last_peered":"2024-09-10T21:30:36.155485+0000","last_clean":"2024-09-10T21:30:36.155485+0000","last_became_active":"2024-09-10T21:30:36.150090+0000","last_became_peered":"2024-09-10T21:30:36.150090+0000","last_unstale":"2024-09-10T21:30:36.155485+0000","last_undegraded":"2024-09-10T21:30:36.155485+0000","last_fullsized":"2024-09-10T21:30:36.155485+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:15.145473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143402+0000","last_change":"2024-09-10T21:30:36.143402+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143402+0000","last_undegraded":"2024-09-10T21:30:36.143402+0000","last_fullsized":"2024-09-10T21:30:36.143402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158775+0000","last_change":"2024-09-10T21:30:36.153213+0000","last_active":"2024-09-10T21:30:36.158775+0000","last_peered":"2024-09-10T21:30:36.158775+0000","last_clean":"2024-09-10T21:30:36.158775+0000","last_became_active":"2024-09-10T21:30:36.152918+0000","last_became_peered":"2024-09-10T21:30:36.152918+0000","last_unstale":"2024-09-10T21:30:36.158775+0000","last_undegraded":"2024-09-10T21:30:36.158775+0000","last_fullsized":"2024-09-10T21:30:36.158775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:39.986638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.134560+0000","last_change":"2024-09-10T21:30:36.134560+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.134560+0000","last_undegraded":"2024-09-10T21:30:36.134560+0000","last_fullsized":"2024-09-10T21:30:36.134560+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158805+0000","last_change":"2024-09-10T21:30:36.155934+0000","last_active":"2024-09-10T21:30:36.158805+0000","last_peered":"2024-09-10T21:30:36.158805+0000","last_clean":"2024-09-10T21:30:36.158805+0000","last_became_active":"2024-09-10T21:30:36.155669+0000","last_became_peered":"2024-09-10T21:30:36.155669+0000","last_unstale":"2024-09-10T21:30:36.158805+0000","last_undegraded":"2024-09-10T21:30:36.158805+0000","last_fullsized":"2024-09-10T21:30:36.158805+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144601+0000","last_change":"2024-09-10T21:30:36.144601+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144601+0000","last_undegraded":"2024-09-10T21:30:36.144601+0000","last_fullsized":"2024-09-10T21:30:36.144601+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158093+0000","last_change":"2024-09-10T21:30:36.152792+0000","last_active":"2024-09-10T21:30:36.158093+0000","last_peered":"2024-09-10T21:30:36.158093+0000","last_clean":"2024-09-10T21:30:36.158093+0000","last_became_active":"2024-09-10T21:30:36.152526+0000","last_became_peered":"2024-09-10T21:30:36.152526+0000","last_unstale":"2024-09-10T21:30:36.158093+0000","last_undegraded":"2024-09-10T21:30:36.158093+0000","last_fullsized":"2024-09-10T21:30:36.158093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.541056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287996+0000","last_change":"2024-09-10T21:30:37.163752+0000","last_active":"2024-09-10T21:30:38.287996+0000","last_peered":"2024-09-10T21:30:38.287996+0000","last_clean":"2024-09-10T21:30:38.287996+0000","last_became_active":"2024-09-10T21:30:37.162710+0000","last_became_peered":"2024-09-10T21:30:37.162710+0000","last_unstale":"2024-09-10T21:30:38.287996+0000","last_undegraded":"2024-09-10T21:30:38.287996+0000","last_fullsized":"2024-09-10T21:30:38.287996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:33.852578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159112+0000","last_change":"2024-09-10T21:30:36.153508+0000","last_active":"2024-09-10T21:30:36.159112+0000","last_peered":"2024-09-10T21:30:36.159112+0000","last_clean":"2024-09-10T21:30:36.159112+0000","last_became_active":"2024-09-10T21:30:36.153255+0000","last_became_peered":"2024-09-10T21:30:36.153255+0000","last_unstale":"2024-09-10T21:30:36.159112+0000","last_undegraded":"2024-09-10T21:30:36.159112+0000","last_fullsized":"2024-09-10T21:30:36.159112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:44.746855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.136690+0000","last_change":"2024-09-10T21:30:36.136690+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.136690+0000","last_undegraded":"2024-09-10T21:30:36.136690+0000","last_fullsized":"2024-09-10T21:30:36.136690+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287713+0000","last_change":"2024-09-10T21:30:36.153764+0000","last_active":"2024-09-10T21:30:38.287713+0000","last_peered":"2024-09-10T21:30:38.287713+0000","last_clean":"2024-09-10T21:30:38.287713+0000","last_became_active":"2024-09-10T21:30:36.153453+0000","last_became_peered":"2024-09-10T21:30:36.153453+0000","last_unstale":"2024-09-10T21:30:38.287713+0000","last_undegraded":"2024-09-10T21:30:38.287713+0000","last_fullsized":"2024-09-10T21:30:38.287713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:28.260842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.144090+0000","last_change":"2024-09-10T21:30:36.144090+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.144090+0000","last_undegraded":"2024-09-10T21:30:36.144090+0000","last_fullsized":"2024-09-10T21:30:36.144090+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.147411+0000","last_change":"2024-09-10T21:30:36.145884+0000","last_active":"2024-09-10T21:30:36.147411+0000","last_peered":"2024-09-10T21:30:36.147411+0000","last_clean":"2024-09-10T21:30:36.147411+0000","last_became_active":"2024-09-10T21:30:36.145592+0000","last_became_peered":"2024-09-10T21:30:36.145592+0000","last_unstale":"2024-09-10T21:30:36.147411+0000","last_undegraded":"2024-09-10T21:30:36.147411+0000","last_fullsized":"2024-09-10T21:30:36.147411+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:52.517766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.145154+0000","last_change":"2024-09-10T21:30:36.145154+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.145154+0000","last_undegraded":"2024-09-10T21:30:36.145154+0000","last_fullsized":"2024-09-10T21:30:36.145154+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.158363+0000","last_change":"2024-09-10T21:30:36.153170+0000","last_active":"2024-09-10T21:30:36.158363+0000","last_peered":"2024-09-10T21:30:36.158363+0000","last_clean":"2024-09-10T21:30:36.158363+0000","last_became_active":"2024-09-10T21:30:36.152223+0000","last_became_peered":"2024-09-10T21:30:36.152223+0000","last_unstale":"2024-09-10T21:30:36.158363+0000","last_undegraded":"2024-09-10T21:30:36.158363+0000","last_fullsized":"2024-09-10T21:30:36.158363+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:00.035137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.143977+0000","last_change":"2024-09-10T21:30:36.143977+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.143977+0000","last_undegraded":"2024-09-10T21:30:36.143977+0000","last_fullsized":"2024-09-10T21:30:36.143977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157622+0000","last_change":"2024-09-10T21:30:36.149909+0000","last_active":"2024-09-10T21:30:36.157622+0000","last_peered":"2024-09-10T21:30:36.157622+0000","last_clean":"2024-09-10T21:30:36.157622+0000","last_became_active":"2024-09-10T21:30:36.149596+0000","last_became_peered":"2024-09-10T21:30:36.149596+0000","last_unstale":"2024-09-10T21:30:36.157622+0000","last_undegraded":"2024-09-10T21:30:36.157622+0000","last_fullsized":"2024-09-10T21:30:36.157622+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:27.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.140295+0000","last_change":"2024-09-10T21:30:36.140295+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.140295+0000","last_undegraded":"2024-09-10T21:30:36.140295+0000","last_fullsized":"2024-09-10T21:30:36.140295+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159054+0000","last_change":"2024-09-10T21:30:36.151988+0000","last_active":"2024-09-10T21:30:36.159054+0000","last_peered":"2024-09-10T21:30:36.159054+0000","last_clean":"2024-09-10T21:30:36.159054+0000","last_became_active":"2024-09-10T21:30:36.151626+0000","last_became_peered":"2024-09-10T21:30:36.151626+0000","last_unstale":"2024-09-10T21:30:36.159054+0000","last_undegraded":"2024-09-10T21:30:36.159054+0000","last_fullsized":"2024-09-10T21:30:36.159054+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:01.606529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.138904+0000","last_change":"2024-09-10T21:30:36.138904+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.138904+0000","last_undegraded":"2024-09-10T21:30:36.138904+0000","last_fullsized":"2024-09-10T21:30:36.138904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159234+0000","last_change":"2024-09-10T21:30:36.156216+0000","last_active":"2024-09-10T21:30:36.159234+0000","last_peered":"2024-09-10T21:30:36.159234+0000","last_clean":"2024-09-10T21:30:36.159234+0000","last_became_active":"2024-09-10T21:30:36.155962+0000","last_became_peered":"2024-09-10T21:30:36.155962+0000","last_unstale":"2024-09-10T21:30:36.159234+0000","last_undegraded":"2024-09-10T21:30:36.159234+0000","last_fullsized":"2024-09-10T21:30:36.159234+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:13.896623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141644+0000","last_change":"2024-09-10T21:30:36.141644+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141644+0000","last_undegraded":"2024-09-10T21:30:36.141644+0000","last_fullsized":"2024-09-10T21:30:36.141644+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.157051+0000","last_change":"2024-09-10T21:30:36.146558+0000","last_active":"2024-09-10T21:30:36.157051+0000","last_peered":"2024-09-10T21:30:36.157051+0000","last_clean":"2024-09-10T21:30:36.157051+0000","last_became_active":"2024-09-10T21:30:36.144327+0000","last_became_peered":"2024-09-10T21:30:36.144327+0000","last_unstale":"2024-09-10T21:30:36.157051+0000","last_undegraded":"2024-09-10T21:30:36.157051+0000","last_fullsized":"2024-09-10T21:30:36.157051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:40.224703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287947+0000","last_change":"2024-09-10T21:30:37.162963+0000","last_active":"2024-09-10T21:30:38.287947+0000","last_peered":"2024-09-10T21:30:38.287947+0000","last_clean":"2024-09-10T21:30:38.287947+0000","last_became_active":"2024-09-10T21:30:37.162366+0000","last_became_peered":"2024-09-10T21:30:37.162366+0000","last_unstale":"2024-09-10T21:30:38.287947+0000","last_undegraded":"2024-09-10T21:30:38.287947+0000","last_fullsized":"2024-09-10T21:30:38.287947+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:37.376166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.161192+0000","last_change":"2024-09-10T21:30:36.159592+0000","last_active":"2024-09-10T21:30:36.161192+0000","last_peered":"2024-09-10T21:30:36.161192+0000","last_clean":"2024-09-10T21:30:36.161192+0000","last_became_active":"2024-09-10T21:30:36.159354+0000","last_became_peered":"2024-09-10T21:30:36.159354+0000","last_unstale":"2024-09-10T21:30:36.161192+0000","last_undegraded":"2024-09-10T21:30:36.161192+0000","last_fullsized":"2024-09-10T21:30:36.161192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:26.541443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.133612+0000","last_change":"2024-09-10T21:30:36.133612+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.133612+0000","last_undegraded":"2024-09-10T21:30:36.133612+0000","last_fullsized":"2024-09-10T21:30:36.133612+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159701+0000","last_change":"2024-09-10T21:30:36.157140+0000","last_active":"2024-09-10T21:30:36.159701+0000","last_peered":"2024-09-10T21:30:36.159701+0000","last_clean":"2024-09-10T21:30:36.159701+0000","last_became_active":"2024-09-10T21:30:36.156861+0000","last_became_peered":"2024-09-10T21:30:36.156861+0000","last_unstale":"2024-09-10T21:30:36.159701+0000","last_undegraded":"2024-09-10T21:30:36.159701+0000","last_fullsized":"2024-09-10T21:30:36.159701+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:09.577405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.137154+0000","last_change":"2024-09-10T21:30:36.137154+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.137154+0000","last_undegraded":"2024-09-10T21:30:36.137154+0000","last_fullsized":"2024-09-10T21:30:36.137154+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.154650+0000","last_change":"2024-09-10T21:30:36.147814+0000","last_active":"2024-09-10T21:30:36.154650+0000","last_peered":"2024-09-10T21:30:36.154650+0000","last_clean":"2024-09-10T21:30:36.154650+0000","last_became_active":"2024-09-10T21:30:36.147488+0000","last_became_peered":"2024-09-10T21:30:36.147488+0000","last_unstale":"2024-09-10T21:30:36.154650+0000","last_undegraded":"2024-09-10T21:30:36.154650+0000","last_fullsized":"2024-09-10T21:30:36.154650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:23.302054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287689+0000","last_change":"2024-09-10T21:30:37.164739+0000","last_active":"2024-09-10T21:30:38.287689+0000","last_peered":"2024-09-10T21:30:38.287689+0000","last_clean":"2024-09-10T21:30:38.287689+0000","last_became_active":"2024-09-10T21:30:37.164473+0000","last_became_peered":"2024-09-10T21:30:37.164473+0000","last_unstale":"2024-09-10T21:30:38.287689+0000","last_undegraded":"2024-09-10T21:30:38.287689+0000","last_fullsized":"2024-09-10T21:30:38.287689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:36.726703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.159717+0000","last_change":"2024-09-10T21:30:36.156585+0000","last_active":"2024-09-10T21:30:36.159717+0000","last_peered":"2024-09-10T21:30:36.159717+0000","last_clean":"2024-09-10T21:30:36.159717+0000","last_became_active":"2024-09-10T21:30:36.156288+0000","last_became_peered":"2024-09-10T21:30:36.156288+0000","last_unstale":"2024-09-10T21:30:36.159717+0000","last_undegraded":"2024-09-10T21:30:36.159717+0000","last_fullsized":"2024-09-10T21:30:36.159717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:09:24.061853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2024-09-10T21:30:36.141257+0000","last_change":"2024-09-10T21:30:36.141257+0000","last_active":"2024-09-10T21:30:36.121501+0000","last_peered":"2024-09-10T21:30:36.121501+0000","last_clean":"2024-09-10T21:30:36.121501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:30:36.141257+0000","last_undegraded":"2024-09-10T21:30:36.141257+0000","last_fullsized":"2024-09-10T21:30:36.141257+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.155869+0000","last_change":"2024-09-10T21:30:36.151215+0000","last_active":"2024-09-10T21:30:36.155869+0000","last_peered":"2024-09-10T21:30:36.155869+0000","last_clean":"2024-09-10T21:30:36.155869+0000","last_became_active":"2024-09-10T21:30:36.150699+0000","last_became_peered":"2024-09-10T21:30:36.150699+0000","last_unstale":"2024-09-10T21:30:36.155869+0000","last_undegraded":"2024-09-10T21:30:36.155869+0000","last_fullsized":"2024-09-10T21:30:36.155869+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":244,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T21:30:36.150455+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:30:36.150455+0000","last_peered":"2024-09-10T21:30:36.150455+0000","last_clean":"2024-09-10T21:30:36.150455+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:30:36.150455+0000","last_undegraded":"2024-09-10T21:30:36.150455+0000","last_fullsized":"2024-09-10T21:30:36.150455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509521,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1102376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1708032,"data_stored":1102376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1114112,"data_stored":512008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1138688,"data_stored":525842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:30:42.112 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3f is not active+clean 2024-09-10T21:30:42.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.141245+0000', 'last_change': '2024-09-10T21:30:36.141245+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.141245+0000', 'last_undegraded': '2024-09-10T21:30:36.141245+0000', 'last_fullsized': '2024-09-10T21:30:36.141245+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.112 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3e is not active+clean 2024-09-10T21:30:42.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.146230+0000', 'last_change': '2024-09-10T21:30:36.146230+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.146230+0000', 'last_undegraded': '2024-09-10T21:30:36.146230+0000', 'last_fullsized': '2024-09-10T21:30:36.146230+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.112 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3d is not active+clean 2024-09-10T21:30:42.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.136865+0000', 'last_change': '2024-09-10T21:30:36.136865+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.136865+0000', 'last_undegraded': '2024-09-10T21:30:36.136865+0000', 'last_fullsized': '2024-09-10T21:30:36.136865+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.112 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3c is not active+clean 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.135307+0000', 'last_change': '2024-09-10T21:30:36.135307+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.135307+0000', 'last_undegraded': '2024-09-10T21:30:36.135307+0000', 'last_fullsized': '2024-09-10T21:30:36.135307+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3b is not active+clean 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.139784+0000', 'last_change': '2024-09-10T21:30:36.139784+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.139784+0000', 'last_undegraded': '2024-09-10T21:30:36.139784+0000', 'last_fullsized': '2024-09-10T21:30:36.139784+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:PG 14.39 is not active+clean 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.137207+0000', 'last_change': '2024-09-10T21:30:36.137207+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.137207+0000', 'last_undegraded': '2024-09-10T21:30:36.137207+0000', 'last_fullsized': '2024-09-10T21:30:36.137207+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:PG 14.37 is not active+clean 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.137459+0000', 'last_change': '2024-09-10T21:30:36.137459+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.137459+0000', 'last_undegraded': '2024-09-10T21:30:36.137459+0000', 'last_fullsized': '2024-09-10T21:30:36.137459+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:PG 14.36 is not active+clean 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.137009+0000', 'last_change': '2024-09-10T21:30:36.137009+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.137009+0000', 'last_undegraded': '2024-09-10T21:30:36.137009+0000', 'last_fullsized': '2024-09-10T21:30:36.137009+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.113 INFO:tasks.ceph.ceph_manager.ceph:PG 14.35 is not active+clean 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.137328+0000', 'last_change': '2024-09-10T21:30:36.137328+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.137328+0000', 'last_undegraded': '2024-09-10T21:30:36.137328+0000', 'last_fullsized': '2024-09-10T21:30:36.137328+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:PG 14.34 is not active+clean 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.134793+0000', 'last_change': '2024-09-10T21:30:36.134793+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.134793+0000', 'last_undegraded': '2024-09-10T21:30:36.134793+0000', 'last_fullsized': '2024-09-10T21:30:36.134793+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:PG 14.33 is not active+clean 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.141795+0000', 'last_change': '2024-09-10T21:30:36.141795+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.141795+0000', 'last_undegraded': '2024-09-10T21:30:36.141795+0000', 'last_fullsized': '2024-09-10T21:30:36.141795+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:PG 14.32 is not active+clean 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.136360+0000', 'last_change': '2024-09-10T21:30:36.136360+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.136360+0000', 'last_undegraded': '2024-09-10T21:30:36.136360+0000', 'last_fullsized': '2024-09-10T21:30:36.136360+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:PG 14.31 is not active+clean 2024-09-10T21:30:42.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.134234+0000', 'last_change': '2024-09-10T21:30:36.134234+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.134234+0000', 'last_undegraded': '2024-09-10T21:30:36.134234+0000', 'last_fullsized': '2024-09-10T21:30:36.134234+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2e is not active+clean 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.135754+0000', 'last_change': '2024-09-10T21:30:36.135754+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.135754+0000', 'last_undegraded': '2024-09-10T21:30:36.135754+0000', 'last_fullsized': '2024-09-10T21:30:36.135754+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2d is not active+clean 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.144667+0000', 'last_change': '2024-09-10T21:30:36.144667+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.144667+0000', 'last_undegraded': '2024-09-10T21:30:36.144667+0000', 'last_fullsized': '2024-09-10T21:30:36.144667+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2b is not active+clean 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.139341+0000', 'last_change': '2024-09-10T21:30:36.139341+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.139341+0000', 'last_undegraded': '2024-09-10T21:30:36.139341+0000', 'last_fullsized': '2024-09-10T21:30:36.139341+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2a is not active+clean 2024-09-10T21:30:42.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.144558+0000', 'last_change': '2024-09-10T21:30:36.144558+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.144558+0000', 'last_undegraded': '2024-09-10T21:30:36.144558+0000', 'last_fullsized': '2024-09-10T21:30:36.144558+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:PG 14.29 is not active+clean 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.139205+0000', 'last_change': '2024-09-10T21:30:36.139205+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.139205+0000', 'last_undegraded': '2024-09-10T21:30:36.139205+0000', 'last_fullsized': '2024-09-10T21:30:36.139205+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:PG 14.28 is not active+clean 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.143144+0000', 'last_change': '2024-09-10T21:30:36.143144+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.143144+0000', 'last_undegraded': '2024-09-10T21:30:36.143144+0000', 'last_fullsized': '2024-09-10T21:30:36.143144+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:PG 14.27 is not active+clean 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.142060+0000', 'last_change': '2024-09-10T21:30:36.142060+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.142060+0000', 'last_undegraded': '2024-09-10T21:30:36.142060+0000', 'last_fullsized': '2024-09-10T21:30:36.142060+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:PG 14.26 is not active+clean 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.139633+0000', 'last_change': '2024-09-10T21:30:36.139633+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.139633+0000', 'last_undegraded': '2024-09-10T21:30:36.139633+0000', 'last_fullsized': '2024-09-10T21:30:36.139633+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:PG 14.25 is not active+clean 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.135362+0000', 'last_change': '2024-09-10T21:30:36.135362+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.135362+0000', 'last_undegraded': '2024-09-10T21:30:36.135362+0000', 'last_fullsized': '2024-09-10T21:30:36.135362+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.116 INFO:tasks.ceph.ceph_manager.ceph:PG 14.24 is not active+clean 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.142361+0000', 'last_change': '2024-09-10T21:30:36.142361+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.142361+0000', 'last_undegraded': '2024-09-10T21:30:36.142361+0000', 'last_fullsized': '2024-09-10T21:30:36.142361+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:PG 14.23 is not active+clean 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.138287+0000', 'last_change': '2024-09-10T21:30:36.138287+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.138287+0000', 'last_undegraded': '2024-09-10T21:30:36.138287+0000', 'last_fullsized': '2024-09-10T21:30:36.138287+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:PG 14.22 is not active+clean 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.137350+0000', 'last_change': '2024-09-10T21:30:36.137350+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.137350+0000', 'last_undegraded': '2024-09-10T21:30:36.137350+0000', 'last_fullsized': '2024-09-10T21:30:36.137350+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:PG 14.21 is not active+clean 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.137784+0000', 'last_change': '2024-09-10T21:30:36.137784+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.137784+0000', 'last_undegraded': '2024-09-10T21:30:36.137784+0000', 'last_fullsized': '2024-09-10T21:30:36.137784+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:30:42.117 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1c is not active+clean 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.143651+0000', 'last_change': '2024-09-10T21:30:36.143651+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.143651+0000', 'last_undegraded': '2024-09-10T21:30:36.143651+0000', 'last_fullsized': '2024-09-10T21:30:36.143651+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1e is not active+clean 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.135162+0000', 'last_change': '2024-09-10T21:30:36.135162+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.135162+0000', 'last_undegraded': '2024-09-10T21:30:36.135162+0000', 'last_fullsized': '2024-09-10T21:30:36.135162+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1f is not active+clean 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.135977+0000', 'last_change': '2024-09-10T21:30:36.135977+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.135977+0000', 'last_undegraded': '2024-09-10T21:30:36.135977+0000', 'last_fullsized': '2024-09-10T21:30:36.135977+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:PG 14.18 is not active+clean 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.139414+0000', 'last_change': '2024-09-10T21:30:36.139414+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.139414+0000', 'last_undegraded': '2024-09-10T21:30:36.139414+0000', 'last_fullsized': '2024-09-10T21:30:36.139414+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:PG 14.19 is not active+clean 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.135003+0000', 'last_change': '2024-09-10T21:30:36.135003+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.135003+0000', 'last_undegraded': '2024-09-10T21:30:36.135003+0000', 'last_fullsized': '2024-09-10T21:30:36.135003+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1b is not active+clean 2024-09-10T21:30:42.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.145566+0000', 'last_change': '2024-09-10T21:30:36.145566+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.145566+0000', 'last_undegraded': '2024-09-10T21:30:36.145566+0000', 'last_fullsized': '2024-09-10T21:30:36.145566+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:PG 14.14 is not active+clean 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.140059+0000', 'last_change': '2024-09-10T21:30:36.140059+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.140059+0000', 'last_undegraded': '2024-09-10T21:30:36.140059+0000', 'last_fullsized': '2024-09-10T21:30:36.140059+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:PG 14.15 is not active+clean 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.135087+0000', 'last_change': '2024-09-10T21:30:36.135087+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.135087+0000', 'last_undegraded': '2024-09-10T21:30:36.135087+0000', 'last_fullsized': '2024-09-10T21:30:36.135087+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:PG 14.16 is not active+clean 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.140912+0000', 'last_change': '2024-09-10T21:30:36.140912+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.140912+0000', 'last_undegraded': '2024-09-10T21:30:36.140912+0000', 'last_fullsized': '2024-09-10T21:30:36.140912+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:PG 14.10 is not active+clean 2024-09-10T21:30:42.119 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.142693+0000', 'last_change': '2024-09-10T21:30:36.142693+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.142693+0000', 'last_undegraded': '2024-09-10T21:30:36.142693+0000', 'last_fullsized': '2024-09-10T21:30:36.142693+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:PG 14.11 is not active+clean 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.138771+0000', 'last_change': '2024-09-10T21:30:36.138771+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.138771+0000', 'last_undegraded': '2024-09-10T21:30:36.138771+0000', 'last_fullsized': '2024-09-10T21:30:36.138771+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:PG 14.12 is not active+clean 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.146320+0000', 'last_change': '2024-09-10T21:30:36.146320+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.146320+0000', 'last_undegraded': '2024-09-10T21:30:36.146320+0000', 'last_fullsized': '2024-09-10T21:30:36.146320+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:PG 14.13 is not active+clean 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.138596+0000', 'last_change': '2024-09-10T21:30:36.138596+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.138596+0000', 'last_undegraded': '2024-09-10T21:30:36.138596+0000', 'last_fullsized': '2024-09-10T21:30:36.138596+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:PG 14.c is not active+clean 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.143402+0000', 'last_change': '2024-09-10T21:30:36.143402+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.143402+0000', 'last_undegraded': '2024-09-10T21:30:36.143402+0000', 'last_fullsized': '2024-09-10T21:30:36.143402+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:PG 14.d is not active+clean 2024-09-10T21:30:42.120 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.134560+0000', 'last_change': '2024-09-10T21:30:36.134560+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.134560+0000', 'last_undegraded': '2024-09-10T21:30:36.134560+0000', 'last_fullsized': '2024-09-10T21:30:36.134560+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:PG 14.e is not active+clean 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.144601+0000', 'last_change': '2024-09-10T21:30:36.144601+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.144601+0000', 'last_undegraded': '2024-09-10T21:30:36.144601+0000', 'last_fullsized': '2024-09-10T21:30:36.144601+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:PG 14.9 is not active+clean 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.136690+0000', 'last_change': '2024-09-10T21:30:36.136690+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.136690+0000', 'last_undegraded': '2024-09-10T21:30:36.136690+0000', 'last_fullsized': '2024-09-10T21:30:36.136690+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:PG 14.a is not active+clean 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.144090+0000', 'last_change': '2024-09-10T21:30:36.144090+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.144090+0000', 'last_undegraded': '2024-09-10T21:30:36.144090+0000', 'last_fullsized': '2024-09-10T21:30:36.144090+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:PG 14.b is not active+clean 2024-09-10T21:30:42.121 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.145154+0000', 'last_change': '2024-09-10T21:30:36.145154+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.145154+0000', 'last_undegraded': '2024-09-10T21:30:36.145154+0000', 'last_fullsized': '2024-09-10T21:30:36.145154+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:PG 14.4 is not active+clean 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.143977+0000', 'last_change': '2024-09-10T21:30:36.143977+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.143977+0000', 'last_undegraded': '2024-09-10T21:30:36.143977+0000', 'last_fullsized': '2024-09-10T21:30:36.143977+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:PG 14.5 is not active+clean 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.140295+0000', 'last_change': '2024-09-10T21:30:36.140295+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.140295+0000', 'last_undegraded': '2024-09-10T21:30:36.140295+0000', 'last_fullsized': '2024-09-10T21:30:36.140295+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:PG 14.6 is not active+clean 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.138904+0000', 'last_change': '2024-09-10T21:30:36.138904+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.138904+0000', 'last_undegraded': '2024-09-10T21:30:36.138904+0000', 'last_fullsized': '2024-09-10T21:30:36.138904+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:PG 14.7 is not active+clean 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.141644+0000', 'last_change': '2024-09-10T21:30:36.141644+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.141644+0000', 'last_undegraded': '2024-09-10T21:30:36.141644+0000', 'last_fullsized': '2024-09-10T21:30:36.141644+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1 is not active+clean 2024-09-10T21:30:42.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.133612+0000', 'last_change': '2024-09-10T21:30:36.133612+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.133612+0000', 'last_undegraded': '2024-09-10T21:30:36.133612+0000', 'last_fullsized': '2024-09-10T21:30:36.133612+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T21:30:42.123 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2 is not active+clean 2024-09-10T21:30:42.123 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.137154+0000', 'last_change': '2024-09-10T21:30:36.137154+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.137154+0000', 'last_undegraded': '2024-09-10T21:30:36.137154+0000', 'last_fullsized': '2024-09-10T21:30:36.137154+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:30:42.123 INFO:tasks.ceph.ceph_manager.ceph:PG 14.f is not active+clean 2024-09-10T21:30:42.123 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 92, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:30:36.141257+0000', 'last_change': '2024-09-10T21:30:36.141257+0000', 'last_active': '2024-09-10T21:30:36.121501+0000', 'last_peered': '2024-09-10T21:30:36.121501+0000', 'last_clean': '2024-09-10T21:30:36.121501+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:30:36.141257+0000', 'last_undegraded': '2024-09-10T21:30:36.141257+0000', 'last_fullsized': '2024-09-10T21:30:36.141257+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 92, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'last_clean_scrub_stamp': '2024-09-10T21:30:36.121501+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:30:42.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:30:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 -- 172.21.15.97:0/4099271202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 msgr2=0x7f1388072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 --2- 172.21.15.97:0/4099271202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388072aa0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1380009c80 tx=0x7f138001cda0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 -- 172.21.15.97:0/4099271202 shutdown_connections 2024-09-10T21:30:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 --2- 172.21.15.97:0/4099271202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388072aa0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 --2- 172.21.15.97:0/4099271202 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f138815b110 0x7f13880cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 -- 172.21.15.97:0/4099271202 >> 172.21.15.97:0/4099271202 conn(0x7f13880b1780 msgr2=0x7f13880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 -- 172.21.15.97:0/4099271202 shutdown_connections 2024-09-10T21:30:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.292+0000 7f138fc00640 1 -- 172.21.15.97:0/4099271202 wait complete. 2024-09-10T21:30:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 Processor -- start 2024-09-10T21:30:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 -- start start 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1388070720 0x7f138814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f138815b110 0x7f13881470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1388167990 con 0x7f1388072630 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1388167810 con 0x7f1388070720 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1388167b10 con 0x7f138815b110 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48688/0 (socket says 172.21.15.97:48688) 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 -- 172.21.15.97:0/2759016704 learned_addr learned my addr 172.21.15.97:0/2759016704 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138d200640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1388070720 0x7f138814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138dc00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f138815b110 0x7f13881470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 -- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f138815b110 msgr2=0x7f13881470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f138815b110 0x7f13881470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 -- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1388070720 msgr2=0x7f138814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1388070720 0x7f138814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 -- 172.21.15.97:0/2759016704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388158440 con 0x7f1388072630 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f137fe00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388144e70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f137000c500 tx=0x7f137000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f1385600640 1 -- 172.21.15.97:0/2759016704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370017070 con 0x7f1388072630 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f1385600640 1 -- 172.21.15.97:0/2759016704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13700053c0 con 0x7f1388072630 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13881453b0 con 0x7f1388072630 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f1385600640 1 -- 172.21.15.97:0/2759016704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370012890 con 0x7f1388072630 2024-09-10T21:30:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.296+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1388158ca0 con 0x7f1388072630 2024-09-10T21:30:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.300+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1388072aa0 con 0x7f1388072630 2024-09-10T21:30:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.300+0000 7f1385600640 1 -- 172.21.15.97:0/2759016704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1370003600 con 0x7f1388072630 2024-09-10T21:30:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.300+0000 7f1385600640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13640725e0 0x7f1364074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.300+0000 7f1385600640 1 -- 172.21.15.97:0/2759016704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f1370092d20 con 0x7f1388072630 2024-09-10T21:30:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.300+0000 7f138d200640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13640725e0 0x7f1364074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.304+0000 7f138d200640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13640725e0 0x7f1364074aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13780058d0 tx=0x7f1378005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.304+0000 7f1385600640 1 -- 172.21.15.97:0/2759016704 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1370021080 con 0x7f1388072630 2024-09-10T21:30:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.528+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f138814f1f0 con 0x7f1388072630 2024-09-10T21:30:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.528+0000 7f1385600640 1 -- 172.21.15.97:0/2759016704 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1077 (secure 0 0 0) 0x7f137005e260 con 0x7f1388072630 2024-09-10T21:30:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:42.532 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":415,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":94,"num_osds":8,"num_up_osds":8,"osd_up_since":1726003441,"num_in_osds":8,"osd_in_since":1726003430,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":52},{"state_name":"active+clean","count":45}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":261386240,"bytes_avail":772832727040,"bytes_total":773094113280,"inactive_pgs_ratio":0.53608244657516479},"fsmap":{"epoch":85,"btime":"2024-09-10T21:30:40:152516+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":6957}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T21:28:50.142056+0000","services":{}},"progress_events":{}} 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13640725e0 msgr2=0x7f1364074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13640725e0 0x7f1364074aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13780058d0 tx=0x7f1378005860 comp rx=0 tx=0).stop 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 msgr2=0x7f1388144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388144e70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f137000c500 tx=0x7f137000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 shutdown_connections 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f13640725e0 0x7f1364074aa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f138815b110 0x7f13881470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1388072630 0x7f1388144e70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 --2- 172.21.15.97:0/2759016704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1388070720 0x7f138814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 >> 172.21.15.97:0/2759016704 conn(0x7f13880b1780 msgr2=0x7f13880b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 shutdown_connections 2024-09-10T21:30:42.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.532+0000 7f138fc00640 1 -- 172.21.15.97:0/2759016704 wait complete. 2024-09-10T21:30:42.553 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:30:42.553 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:30:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719c00640 1 -- 172.21.15.97:0/2062948232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f8003f80 msgr2=0x7ff6f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719c00640 1 --2- 172.21.15.97:0/2062948232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6f8003f80 0x7ff6f8004810 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc001740 tx=0x7ff6fc01c810 comp rx=0 tx=0).stop 2024-09-10T21:30:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719200640 1 -- 172.21.15.97:0/2062948232 reap_dead start 2024-09-10T21:30:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719c00640 1 -- 172.21.15.97:0/2062948232 shutdown_connections 2024-09-10T21:30:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719c00640 1 -- 172.21.15.97:0/2062948232 >> 172.21.15.97:0/2062948232 conn(0x7ff7140ccac0 msgr2=0x7ff7140cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719c00640 1 -- 172.21.15.97:0/2062948232 shutdown_connections 2024-09-10T21:30:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719c00640 1 -- 172.21.15.97:0/2062948232 wait complete. 2024-09-10T21:30:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.692+0000 7ff719c00640 1 Processor -- start 2024-09-10T21:30:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 -- start start 2024-09-10T21:30:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f8003f80 0x7ff7140b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff71415b150 0x7ff714149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714146d70 0x7ff7141471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff714167d80 con 0x7ff71415b150 2024-09-10T21:30:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff714167c00 con 0x7ff714146d70 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff714167f00 con 0x7ff6f8003f80 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff71415b150 0x7ff714149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff71415b150 0x7ff714149130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48704/0 (socket says 172.21.15.97:48704) 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 -- 172.21.15.97:0/1626908041 learned_addr learned my addr 172.21.15.97:0/1626908041 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719200640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714146d70 0x7ff7141471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff713e00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f8003f80 0x7ff7140b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 -- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f8003f80 msgr2=0x7ff7140b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f8003f80 0x7ff7140b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 -- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714146d70 msgr2=0x7ff7141471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714146d70 0x7ff7141471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 -- 172.21.15.97:0/1626908041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7140b5b50 con 0x7ff71415b150 2024-09-10T21:30:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff70be00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff71415b150 0x7ff714149130 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc01ccf0 tx=0x7ff6fc006620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff710c00640 1 -- 172.21.15.97:0/1626908041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0a56d0 con 0x7ff71415b150 2024-09-10T21:30:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff710c00640 1 -- 172.21.15.97:0/1626908041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6fc0a5960 con 0x7ff71415b150 2024-09-10T21:30:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71415a670 con 0x7ff71415b150 2024-09-10T21:30:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff710c00640 1 -- 172.21.15.97:0/1626908041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0a5db0 con 0x7ff71415b150 2024-09-10T21:30:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff714149670 con 0x7ff71415b150 2024-09-10T21:30:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.696+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71406df70 con 0x7ff71415b150 2024-09-10T21:30:42.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.700+0000 7ff710c00640 1 -- 172.21.15.97:0/1626908041 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff6fc003600 con 0x7ff71415b150 2024-09-10T21:30:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.700+0000 7ff710c00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6f00725e0 0x7ff6f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.700+0000 7ff710c00640 1 -- 172.21.15.97:0/1626908041 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7ff6fc12cfe0 con 0x7ff71415b150 2024-09-10T21:30:42.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.700+0000 7ff713e00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6f00725e0 0x7ff6f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:42.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.700+0000 7ff713e00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6f00725e0 0x7ff6f0074aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff704009ba0 tx=0x7ff7040091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:42.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.704+0000 7ff710c00640 1 -- 172.21.15.97:0/1626908041 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff6fc016550 con 0x7ff71415b150 2024-09-10T21:30:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.868+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff71415a4b0 con 0x7ff6f00725e0 2024-09-10T21:30:42.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.872+0000 7ff710c00640 1 -- 172.21.15.97:0/1626908041 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252070 (secure 0 0 0) 0x7ff704012bd0 con 0x7ff6f00725e0 2024-09-10T21:30:42.879 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:42.881 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:30:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6f00725e0 msgr2=0x7ff6f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6f00725e0 0x7ff6f0074aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff704009ba0 tx=0x7ff7040091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff71415b150 msgr2=0x7ff714149130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff71415b150 0x7ff714149130 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc01ccf0 tx=0x7ff6fc006620 comp rx=0 tx=0).stop 2024-09-10T21:30:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 shutdown_connections 2024-09-10T21:30:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6f00725e0 0x7ff6f0074aa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff714146d70 0x7ff7141471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff71415b150 0x7ff714149130 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 --2- 172.21.15.97:0/1626908041 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6f8003f80 0x7ff7140b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 >> 172.21.15.97:0/1626908041 conn(0x7ff7140ccac0 msgr2=0x7ff7140ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 shutdown_connections 2024-09-10T21:30:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:42.876+0000 7ff719c00640 1 -- 172.21.15.97:0/1626908041 wait complete. 2024-09-10T21:30:42.899 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T21:30:42.169344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255588,"kb_used_data":10204,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719132,"statfs":{"total":773094113280,"available":772832391168,"internally_reserved":0,"allocated":10448896,"data_stored":5439136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.039084"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287886+0000","last_change":"2024-09-10T21:30:37.159556+0000","last_active":"2024-09-10T21:30:38.287886+0000","last_peered":"2024-09-10T21:30:38.287886+0000","last_clean":"2024-09-10T21:30:38.287886+0000","last_became_active":"2024-09-10T21:30:37.159215+0000","last_became_peered":"2024-09-10T21:30:37.159215+0000","last_unstale":"2024-09-10T21:30:38.287886+0000","last_undegraded":"2024-09-10T21:30:38.287886+0000","last_fullsized":"2024-09-10T21:30:38.287886+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:47.060669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154534+0000","last_change":"2024-09-10T21:30:37.159414+0000","last_active":"2024-09-10T21:30:38.154534+0000","last_peered":"2024-09-10T21:30:38.154534+0000","last_clean":"2024-09-10T21:30:38.154534+0000","last_became_active":"2024-09-10T21:30:37.159157+0000","last_became_peered":"2024-09-10T21:30:37.159157+0000","last_unstale":"2024-09-10T21:30:38.154534+0000","last_undegraded":"2024-09-10T21:30:38.154534+0000","last_fullsized":"2024-09-10T21:30:38.154534+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:09.640497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152689+0000","last_change":"2024-09-10T21:30:37.157235+0000","last_active":"2024-09-10T21:30:38.152689+0000","last_peered":"2024-09-10T21:30:38.152689+0000","last_clean":"2024-09-10T21:30:38.152689+0000","last_became_active":"2024-09-10T21:30:37.156065+0000","last_became_peered":"2024-09-10T21:30:37.156065+0000","last_unstale":"2024-09-10T21:30:38.152689+0000","last_undegraded":"2024-09-10T21:30:38.152689+0000","last_fullsized":"2024-09-10T21:30:38.152689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:57.193003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151359+0000","last_change":"2024-09-10T21:30:37.159075+0000","last_active":"2024-09-10T21:30:38.151359+0000","last_peered":"2024-09-10T21:30:38.151359+0000","last_clean":"2024-09-10T21:30:38.151359+0000","last_became_active":"2024-09-10T21:30:37.156300+0000","last_became_peered":"2024-09-10T21:30:37.156300+0000","last_unstale":"2024-09-10T21:30:38.151359+0000","last_undegraded":"2024-09-10T21:30:38.151359+0000","last_fullsized":"2024-09-10T21:30:38.151359+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:24.902445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154550+0000","last_change":"2024-09-10T21:30:37.153572+0000","last_active":"2024-09-10T21:30:38.154550+0000","last_peered":"2024-09-10T21:30:38.154550+0000","last_clean":"2024-09-10T21:30:38.154550+0000","last_became_active":"2024-09-10T21:30:37.152445+0000","last_became_peered":"2024-09-10T21:30:37.152445+0000","last_unstale":"2024-09-10T21:30:38.154550+0000","last_undegraded":"2024-09-10T21:30:38.154550+0000","last_fullsized":"2024-09-10T21:30:38.154550+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:53.825751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287577+0000","last_change":"2024-09-10T21:30:37.156704+0000","last_active":"2024-09-10T21:30:38.287577+0000","last_peered":"2024-09-10T21:30:38.287577+0000","last_clean":"2024-09-10T21:30:38.287577+0000","last_became_active":"2024-09-10T21:30:37.154734+0000","last_became_peered":"2024-09-10T21:30:37.154734+0000","last_unstale":"2024-09-10T21:30:38.287577+0000","last_undegraded":"2024-09-10T21:30:38.287577+0000","last_fullsized":"2024-09-10T21:30:38.287577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:59.951655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151748+0000","last_change":"2024-09-10T21:30:37.157302+0000","last_active":"2024-09-10T21:30:38.151748+0000","last_peered":"2024-09-10T21:30:38.151748+0000","last_clean":"2024-09-10T21:30:38.151748+0000","last_became_active":"2024-09-10T21:30:37.156825+0000","last_became_peered":"2024-09-10T21:30:37.156825+0000","last_unstale":"2024-09-10T21:30:38.151748+0000","last_undegraded":"2024-09-10T21:30:38.151748+0000","last_fullsized":"2024-09-10T21:30:38.151748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:13.008263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287144+0000","last_change":"2024-09-10T21:30:37.146717+0000","last_active":"2024-09-10T21:30:38.287144+0000","last_peered":"2024-09-10T21:30:38.287144+0000","last_clean":"2024-09-10T21:30:38.287144+0000","last_became_active":"2024-09-10T21:30:37.146428+0000","last_became_peered":"2024-09-10T21:30:37.146428+0000","last_unstale":"2024-09-10T21:30:38.287144+0000","last_undegraded":"2024-09-10T21:30:38.287144+0000","last_fullsized":"2024-09-10T21:30:38.287144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:18.307373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152277+0000","last_change":"2024-09-10T21:30:37.155234+0000","last_active":"2024-09-10T21:30:38.152277+0000","last_peered":"2024-09-10T21:30:38.152277+0000","last_clean":"2024-09-10T21:30:38.152277+0000","last_became_active":"2024-09-10T21:30:37.154993+0000","last_became_peered":"2024-09-10T21:30:37.154993+0000","last_unstale":"2024-09-10T21:30:38.152277+0000","last_undegraded":"2024-09-10T21:30:38.152277+0000","last_fullsized":"2024-09-10T21:30:38.152277+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:17.183416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151444+0000","last_change":"2024-09-10T21:30:37.155994+0000","last_active":"2024-09-10T21:30:38.151444+0000","last_peered":"2024-09-10T21:30:38.151444+0000","last_clean":"2024-09-10T21:30:38.151444+0000","last_became_active":"2024-09-10T21:30:37.155283+0000","last_became_peered":"2024-09-10T21:30:37.155283+0000","last_unstale":"2024-09-10T21:30:38.151444+0000","last_undegraded":"2024-09-10T21:30:38.151444+0000","last_fullsized":"2024-09-10T21:30:38.151444+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:11.548855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152003+0000","last_change":"2024-09-10T21:30:37.157683+0000","last_active":"2024-09-10T21:30:38.152003+0000","last_peered":"2024-09-10T21:30:38.152003+0000","last_clean":"2024-09-10T21:30:38.152003+0000","last_became_active":"2024-09-10T21:30:37.156106+0000","last_became_peered":"2024-09-10T21:30:37.156106+0000","last_unstale":"2024-09-10T21:30:38.152003+0000","last_undegraded":"2024-09-10T21:30:38.152003+0000","last_fullsized":"2024-09-10T21:30:38.152003+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:27.891796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151246+0000","last_change":"2024-09-10T21:30:37.159806+0000","last_active":"2024-09-10T21:30:38.151246+0000","last_peered":"2024-09-10T21:30:38.151246+0000","last_clean":"2024-09-10T21:30:38.151246+0000","last_became_active":"2024-09-10T21:30:37.159533+0000","last_became_peered":"2024-09-10T21:30:37.159533+0000","last_unstale":"2024-09-10T21:30:38.151246+0000","last_undegraded":"2024-09-10T21:30:38.151246+0000","last_fullsized":"2024-09-10T21:30:38.151246+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:56.957241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151498+0000","last_change":"2024-09-10T21:30:37.158510+0000","last_active":"2024-09-10T21:30:38.151498+0000","last_peered":"2024-09-10T21:30:38.151498+0000","last_clean":"2024-09-10T21:30:38.151498+0000","last_became_active":"2024-09-10T21:30:37.158191+0000","last_became_peered":"2024-09-10T21:30:37.158191+0000","last_unstale":"2024-09-10T21:30:38.151498+0000","last_undegraded":"2024-09-10T21:30:38.151498+0000","last_fullsized":"2024-09-10T21:30:38.151498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:07.314862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148326+0000","last_change":"2024-09-10T21:30:37.146477+0000","last_active":"2024-09-10T21:30:38.148326+0000","last_peered":"2024-09-10T21:30:38.148326+0000","last_clean":"2024-09-10T21:30:38.148326+0000","last_became_active":"2024-09-10T21:30:37.143113+0000","last_became_peered":"2024-09-10T21:30:37.143113+0000","last_unstale":"2024-09-10T21:30:38.148326+0000","last_undegraded":"2024-09-10T21:30:38.148326+0000","last_fullsized":"2024-09-10T21:30:38.148326+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:11.873940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150735+0000","last_change":"2024-09-10T21:30:37.146196+0000","last_active":"2024-09-10T21:30:38.150735+0000","last_peered":"2024-09-10T21:30:38.150735+0000","last_clean":"2024-09-10T21:30:38.150735+0000","last_became_active":"2024-09-10T21:30:37.145480+0000","last_became_peered":"2024-09-10T21:30:37.145480+0000","last_unstale":"2024-09-10T21:30:38.150735+0000","last_undegraded":"2024-09-10T21:30:38.150735+0000","last_fullsized":"2024-09-10T21:30:38.150735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:13.493774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287182+0000","last_change":"2024-09-10T21:30:37.152682+0000","last_active":"2024-09-10T21:30:38.287182+0000","last_peered":"2024-09-10T21:30:38.287182+0000","last_clean":"2024-09-10T21:30:38.287182+0000","last_became_active":"2024-09-10T21:30:37.152308+0000","last_became_peered":"2024-09-10T21:30:37.152308+0000","last_unstale":"2024-09-10T21:30:38.287182+0000","last_undegraded":"2024-09-10T21:30:38.287182+0000","last_fullsized":"2024-09-10T21:30:38.287182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:39.533064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288396+0000","last_change":"2024-09-10T21:30:37.155784+0000","last_active":"2024-09-10T21:30:38.288396+0000","last_peered":"2024-09-10T21:30:38.288396+0000","last_clean":"2024-09-10T21:30:38.288396+0000","last_became_active":"2024-09-10T21:30:37.155514+0000","last_became_peered":"2024-09-10T21:30:37.155514+0000","last_unstale":"2024-09-10T21:30:38.288396+0000","last_undegraded":"2024-09-10T21:30:38.288396+0000","last_fullsized":"2024-09-10T21:30:38.288396+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:10.496991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151661+0000","last_change":"2024-09-10T21:30:37.157022+0000","last_active":"2024-09-10T21:30:38.151661+0000","last_peered":"2024-09-10T21:30:38.151661+0000","last_clean":"2024-09-10T21:30:38.151661+0000","last_became_active":"2024-09-10T21:30:37.155667+0000","last_became_peered":"2024-09-10T21:30:37.155667+0000","last_unstale":"2024-09-10T21:30:38.151661+0000","last_undegraded":"2024-09-10T21:30:38.151661+0000","last_fullsized":"2024-09-10T21:30:38.151661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:08.805186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286189+0000","last_change":"2024-09-10T21:30:37.158088+0000","last_active":"2024-09-10T21:30:38.286189+0000","last_peered":"2024-09-10T21:30:38.286189+0000","last_clean":"2024-09-10T21:30:38.286189+0000","last_became_active":"2024-09-10T21:30:37.157730+0000","last_became_peered":"2024-09-10T21:30:37.157730+0000","last_unstale":"2024-09-10T21:30:38.286189+0000","last_undegraded":"2024-09-10T21:30:38.286189+0000","last_fullsized":"2024-09-10T21:30:38.286189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:28.880581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287734+0000","last_change":"2024-09-10T21:30:37.161159+0000","last_active":"2024-09-10T21:30:38.287734+0000","last_peered":"2024-09-10T21:30:38.287734+0000","last_clean":"2024-09-10T21:30:38.287734+0000","last_became_active":"2024-09-10T21:30:37.160818+0000","last_became_peered":"2024-09-10T21:30:37.160818+0000","last_unstale":"2024-09-10T21:30:38.287734+0000","last_undegraded":"2024-09-10T21:30:38.287734+0000","last_fullsized":"2024-09-10T21:30:38.287734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:10.536055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154426+0000","last_change":"2024-09-10T21:30:37.153112+0000","last_active":"2024-09-10T21:30:38.154426+0000","last_peered":"2024-09-10T21:30:38.154426+0000","last_clean":"2024-09-10T21:30:38.154426+0000","last_became_active":"2024-09-10T21:30:37.152856+0000","last_became_peered":"2024-09-10T21:30:37.152856+0000","last_unstale":"2024-09-10T21:30:38.154426+0000","last_undegraded":"2024-09-10T21:30:38.154426+0000","last_fullsized":"2024-09-10T21:30:38.154426+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286004+0000","last_change":"2024-09-10T21:30:37.156498+0000","last_active":"2024-09-10T21:30:38.286004+0000","last_peered":"2024-09-10T21:30:38.286004+0000","last_clean":"2024-09-10T21:30:38.286004+0000","last_became_active":"2024-09-10T21:30:37.156217+0000","last_became_peered":"2024-09-10T21:30:37.156217+0000","last_unstale":"2024-09-10T21:30:38.286004+0000","last_undegraded":"2024-09-10T21:30:38.286004+0000","last_fullsized":"2024-09-10T21:30:38.286004+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:23.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154370+0000","last_change":"2024-09-10T21:30:37.160341+0000","last_active":"2024-09-10T21:30:38.154370+0000","last_peered":"2024-09-10T21:30:38.154370+0000","last_clean":"2024-09-10T21:30:38.154370+0000","last_became_active":"2024-09-10T21:30:37.160065+0000","last_became_peered":"2024-09-10T21:30:37.160065+0000","last_unstale":"2024-09-10T21:30:38.154370+0000","last_undegraded":"2024-09-10T21:30:38.154370+0000","last_fullsized":"2024-09-10T21:30:38.154370+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:52.394833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286884+0000","last_change":"2024-09-10T21:30:37.155875+0000","last_active":"2024-09-10T21:30:38.286884+0000","last_peered":"2024-09-10T21:30:38.286884+0000","last_clean":"2024-09-10T21:30:38.286884+0000","last_became_active":"2024-09-10T21:30:37.155291+0000","last_became_peered":"2024-09-10T21:30:37.155291+0000","last_unstale":"2024-09-10T21:30:38.286884+0000","last_undegraded":"2024-09-10T21:30:38.286884+0000","last_fullsized":"2024-09-10T21:30:38.286884+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:23.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154496+0000","last_change":"2024-09-10T21:30:37.152802+0000","last_active":"2024-09-10T21:30:38.154496+0000","last_peered":"2024-09-10T21:30:38.154496+0000","last_clean":"2024-09-10T21:30:38.154496+0000","last_became_active":"2024-09-10T21:30:37.152505+0000","last_became_peered":"2024-09-10T21:30:37.152505+0000","last_unstale":"2024-09-10T21:30:38.154496+0000","last_undegraded":"2024-09-10T21:30:38.154496+0000","last_fullsized":"2024-09-10T21:30:38.154496+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:57.443883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154923+0000","last_change":"2024-09-10T21:30:37.158604+0000","last_active":"2024-09-10T21:30:38.154923+0000","last_peered":"2024-09-10T21:30:38.154923+0000","last_clean":"2024-09-10T21:30:38.154923+0000","last_became_active":"2024-09-10T21:30:37.158029+0000","last_became_peered":"2024-09-10T21:30:37.158029+0000","last_unstale":"2024-09-10T21:30:38.154923+0000","last_undegraded":"2024-09-10T21:30:38.154923+0000","last_fullsized":"2024-09-10T21:30:38.154923+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:10.364464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152305+0000","last_change":"2024-09-10T21:30:37.155691+0000","last_active":"2024-09-10T21:30:38.152305+0000","last_peered":"2024-09-10T21:30:38.152305+0000","last_clean":"2024-09-10T21:30:38.152305+0000","last_became_active":"2024-09-10T21:30:37.155382+0000","last_became_peered":"2024-09-10T21:30:37.155382+0000","last_unstale":"2024-09-10T21:30:38.152305+0000","last_undegraded":"2024-09-10T21:30:38.152305+0000","last_fullsized":"2024-09-10T21:30:38.152305+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:27.386955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152312+0000","last_change":"2024-09-10T21:30:37.167891+0000","last_active":"2024-09-10T21:30:38.152312+0000","last_peered":"2024-09-10T21:30:38.152312+0000","last_clean":"2024-09-10T21:30:38.152312+0000","last_became_active":"2024-09-10T21:30:37.163511+0000","last_became_peered":"2024-09-10T21:30:37.163511+0000","last_unstale":"2024-09-10T21:30:38.152312+0000","last_undegraded":"2024-09-10T21:30:38.152312+0000","last_fullsized":"2024-09-10T21:30:38.152312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:49.640342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154685+0000","last_change":"2024-09-10T21:30:37.160305+0000","last_active":"2024-09-10T21:30:38.154685+0000","last_peered":"2024-09-10T21:30:38.154685+0000","last_clean":"2024-09-10T21:30:38.154685+0000","last_became_active":"2024-09-10T21:30:37.160011+0000","last_became_peered":"2024-09-10T21:30:37.160011+0000","last_unstale":"2024-09-10T21:30:38.154685+0000","last_undegraded":"2024-09-10T21:30:38.154685+0000","last_fullsized":"2024-09-10T21:30:38.154685+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:28.880581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154345+0000","last_change":"2024-09-10T21:30:37.151678+0000","last_active":"2024-09-10T21:30:38.154345+0000","last_peered":"2024-09-10T21:30:38.154345+0000","last_clean":"2024-09-10T21:30:38.154345+0000","last_became_active":"2024-09-10T21:30:37.151090+0000","last_became_peered":"2024-09-10T21:30:37.151090+0000","last_unstale":"2024-09-10T21:30:38.154345+0000","last_undegraded":"2024-09-10T21:30:38.154345+0000","last_fullsized":"2024-09-10T21:30:38.154345+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:19.172370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.285885+0000","last_change":"2024-09-10T21:30:37.153740+0000","last_active":"2024-09-10T21:30:38.285885+0000","last_peered":"2024-09-10T21:30:38.285885+0000","last_clean":"2024-09-10T21:30:38.285885+0000","last_became_active":"2024-09-10T21:30:37.153498+0000","last_became_peered":"2024-09-10T21:30:37.153498+0000","last_unstale":"2024-09-10T21:30:38.285885+0000","last_undegraded":"2024-09-10T21:30:38.285885+0000","last_fullsized":"2024-09-10T21:30:38.285885+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:48.396677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287384+0000","last_change":"2024-09-10T21:30:37.142844+0000","last_active":"2024-09-10T21:30:38.287384+0000","last_peered":"2024-09-10T21:30:38.287384+0000","last_clean":"2024-09-10T21:30:38.287384+0000","last_became_active":"2024-09-10T21:30:37.142487+0000","last_became_peered":"2024-09-10T21:30:37.142487+0000","last_unstale":"2024-09-10T21:30:38.287384+0000","last_undegraded":"2024-09-10T21:30:38.287384+0000","last_fullsized":"2024-09-10T21:30:38.287384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:49.728530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288237+0000","last_change":"2024-09-10T21:30:37.165261+0000","last_active":"2024-09-10T21:30:38.288237+0000","last_peered":"2024-09-10T21:30:38.288237+0000","last_clean":"2024-09-10T21:30:38.288237+0000","last_became_active":"2024-09-10T21:30:37.156273+0000","last_became_peered":"2024-09-10T21:30:37.156273+0000","last_unstale":"2024-09-10T21:30:38.288237+0000","last_undegraded":"2024-09-10T21:30:38.288237+0000","last_fullsized":"2024-09-10T21:30:38.288237+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:02.614328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.155022+0000","last_change":"2024-09-10T21:30:36.156733+0000","last_active":"2024-09-10T21:30:38.155022+0000","last_peered":"2024-09-10T21:30:38.155022+0000","last_clean":"2024-09-10T21:30:38.155022+0000","last_became_active":"2024-09-10T21:30:36.156360+0000","last_became_peered":"2024-09-10T21:30:36.156360+0000","last_unstale":"2024-09-10T21:30:38.155022+0000","last_undegraded":"2024-09-10T21:30:38.155022+0000","last_fullsized":"2024-09-10T21:30:38.155022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287825+0000","last_change":"2024-09-10T21:30:37.161240+0000","last_active":"2024-09-10T21:30:38.287825+0000","last_peered":"2024-09-10T21:30:38.287825+0000","last_clean":"2024-09-10T21:30:38.287825+0000","last_became_active":"2024-09-10T21:30:37.160927+0000","last_became_peered":"2024-09-10T21:30:37.160927+0000","last_unstale":"2024-09-10T21:30:38.287825+0000","last_undegraded":"2024-09-10T21:30:38.287825+0000","last_fullsized":"2024-09-10T21:30:38.287825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:57.275653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148672+0000","last_change":"2024-09-10T21:30:36.152982+0000","last_active":"2024-09-10T21:30:38.148672+0000","last_peered":"2024-09-10T21:30:38.148672+0000","last_clean":"2024-09-10T21:30:38.148672+0000","last_became_active":"2024-09-10T21:30:36.151294+0000","last_became_peered":"2024-09-10T21:30:36.151294+0000","last_unstale":"2024-09-10T21:30:38.148672+0000","last_undegraded":"2024-09-10T21:30:38.148672+0000","last_fullsized":"2024-09-10T21:30:38.148672+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:26.541443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152043+0000","last_change":"2024-09-10T21:30:37.161761+0000","last_active":"2024-09-10T21:30:38.152043+0000","last_peered":"2024-09-10T21:30:38.152043+0000","last_clean":"2024-09-10T21:30:38.152043+0000","last_became_active":"2024-09-10T21:30:37.161491+0000","last_became_peered":"2024-09-10T21:30:37.161491+0000","last_unstale":"2024-09-10T21:30:38.152043+0000","last_undegraded":"2024-09-10T21:30:38.152043+0000","last_fullsized":"2024-09-10T21:30:38.152043+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:08.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154849+0000","last_change":"2024-09-10T21:30:36.151783+0000","last_active":"2024-09-10T21:30:38.154849+0000","last_peered":"2024-09-10T21:30:38.154849+0000","last_clean":"2024-09-10T21:30:38.154849+0000","last_became_active":"2024-09-10T21:30:36.151113+0000","last_became_peered":"2024-09-10T21:30:36.151113+0000","last_unstale":"2024-09-10T21:30:38.154849+0000","last_undegraded":"2024-09-10T21:30:38.154849+0000","last_fullsized":"2024-09-10T21:30:38.154849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151436+0000","last_change":"2024-09-10T21:30:37.156034+0000","last_active":"2024-09-10T21:30:38.151436+0000","last_peered":"2024-09-10T21:30:38.151436+0000","last_clean":"2024-09-10T21:30:38.151436+0000","last_became_active":"2024-09-10T21:30:37.155732+0000","last_became_peered":"2024-09-10T21:30:37.155732+0000","last_unstale":"2024-09-10T21:30:38.151436+0000","last_undegraded":"2024-09-10T21:30:38.151436+0000","last_fullsized":"2024-09-10T21:30:38.151436+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:06.192346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148525+0000","last_change":"2024-09-10T21:30:36.147985+0000","last_active":"2024-09-10T21:30:38.148525+0000","last_peered":"2024-09-10T21:30:38.148525+0000","last_clean":"2024-09-10T21:30:38.148525+0000","last_became_active":"2024-09-10T21:30:36.147685+0000","last_became_peered":"2024-09-10T21:30:36.147685+0000","last_unstale":"2024-09-10T21:30:38.148525+0000","last_undegraded":"2024-09-10T21:30:38.148525+0000","last_fullsized":"2024-09-10T21:30:38.148525+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:43.536977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150759+0000","last_change":"2024-09-10T21:30:37.157486+0000","last_active":"2024-09-10T21:30:38.150759+0000","last_peered":"2024-09-10T21:30:38.150759+0000","last_clean":"2024-09-10T21:30:38.150759+0000","last_became_active":"2024-09-10T21:30:37.157180+0000","last_became_peered":"2024-09-10T21:30:37.157180+0000","last_unstale":"2024-09-10T21:30:38.150759+0000","last_undegraded":"2024-09-10T21:30:38.150759+0000","last_fullsized":"2024-09-10T21:30:38.150759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:11.873940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.285998+0000","last_change":"2024-09-10T21:30:36.152597+0000","last_active":"2024-09-10T21:30:38.285998+0000","last_peered":"2024-09-10T21:30:38.285998+0000","last_clean":"2024-09-10T21:30:38.285998+0000","last_became_active":"2024-09-10T21:30:36.152336+0000","last_became_peered":"2024-09-10T21:30:36.152336+0000","last_unstale":"2024-09-10T21:30:38.285998+0000","last_undegraded":"2024-09-10T21:30:38.285998+0000","last_fullsized":"2024-09-10T21:30:38.285998+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:10.866141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151824+0000","last_change":"2024-09-10T21:30:37.157054+0000","last_active":"2024-09-10T21:30:38.151824+0000","last_peered":"2024-09-10T21:30:38.151824+0000","last_clean":"2024-09-10T21:30:38.151824+0000","last_became_active":"2024-09-10T21:30:37.156528+0000","last_became_peered":"2024-09-10T21:30:37.156528+0000","last_unstale":"2024-09-10T21:30:38.151824+0000","last_undegraded":"2024-09-10T21:30:38.151824+0000","last_fullsized":"2024-09-10T21:30:38.151824+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:55.564342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151415+0000","last_change":"2024-09-10T21:30:36.156510+0000","last_active":"2024-09-10T21:30:38.151415+0000","last_peered":"2024-09-10T21:30:38.151415+0000","last_clean":"2024-09-10T21:30:38.151415+0000","last_became_active":"2024-09-10T21:30:36.156197+0000","last_became_peered":"2024-09-10T21:30:36.156197+0000","last_unstale":"2024-09-10T21:30:38.151415+0000","last_undegraded":"2024-09-10T21:30:38.151415+0000","last_fullsized":"2024-09-10T21:30:38.151415+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287564+0000","last_change":"2024-09-10T21:30:37.143279+0000","last_active":"2024-09-10T21:30:38.287564+0000","last_peered":"2024-09-10T21:30:38.287564+0000","last_clean":"2024-09-10T21:30:38.287564+0000","last_became_active":"2024-09-10T21:30:37.142984+0000","last_became_peered":"2024-09-10T21:30:37.142984+0000","last_unstale":"2024-09-10T21:30:38.287564+0000","last_undegraded":"2024-09-10T21:30:38.287564+0000","last_fullsized":"2024-09-10T21:30:38.287564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:12.128103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151074+0000","last_change":"2024-09-10T21:30:36.150338+0000","last_active":"2024-09-10T21:30:38.151074+0000","last_peered":"2024-09-10T21:30:38.151074+0000","last_clean":"2024-09-10T21:30:38.151074+0000","last_became_active":"2024-09-10T21:30:36.149500+0000","last_became_peered":"2024-09-10T21:30:36.149500+0000","last_unstale":"2024-09-10T21:30:38.151074+0000","last_undegraded":"2024-09-10T21:30:38.151074+0000","last_fullsized":"2024-09-10T21:30:38.151074+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:40.263132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288013+0000","last_change":"2024-09-10T21:30:37.156836+0000","last_active":"2024-09-10T21:30:38.288013+0000","last_peered":"2024-09-10T21:30:38.288013+0000","last_clean":"2024-09-10T21:30:38.288013+0000","last_became_active":"2024-09-10T21:30:37.156585+0000","last_became_peered":"2024-09-10T21:30:37.156585+0000","last_unstale":"2024-09-10T21:30:38.288013+0000","last_undegraded":"2024-09-10T21:30:38.288013+0000","last_fullsized":"2024-09-10T21:30:38.288013+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:28.880581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154183+0000","last_change":"2024-09-10T21:30:36.152065+0000","last_active":"2024-09-10T21:30:38.154183+0000","last_peered":"2024-09-10T21:30:38.154183+0000","last_clean":"2024-09-10T21:30:38.154183+0000","last_became_active":"2024-09-10T21:30:36.151761+0000","last_became_peered":"2024-09-10T21:30:36.151761+0000","last_unstale":"2024-09-10T21:30:38.154183+0000","last_undegraded":"2024-09-10T21:30:38.154183+0000","last_fullsized":"2024-09-10T21:30:38.154183+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:52.517766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154496+0000","last_change":"2024-09-10T21:30:37.161916+0000","last_active":"2024-09-10T21:30:38.154496+0000","last_peered":"2024-09-10T21:30:38.154496+0000","last_clean":"2024-09-10T21:30:38.154496+0000","last_became_active":"2024-09-10T21:30:37.161659+0000","last_became_peered":"2024-09-10T21:30:37.161659+0000","last_unstale":"2024-09-10T21:30:38.154496+0000","last_undegraded":"2024-09-10T21:30:38.154496+0000","last_fullsized":"2024-09-10T21:30:38.154496+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:23.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.155064+0000","last_change":"2024-09-10T21:30:36.152474+0000","last_active":"2024-09-10T21:30:38.155064+0000","last_peered":"2024-09-10T21:30:38.155064+0000","last_clean":"2024-09-10T21:30:38.155064+0000","last_became_active":"2024-09-10T21:30:36.152180+0000","last_became_peered":"2024-09-10T21:30:36.152180+0000","last_unstale":"2024-09-10T21:30:38.155064+0000","last_undegraded":"2024-09-10T21:30:38.155064+0000","last_fullsized":"2024-09-10T21:30:38.155064+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152034+0000","last_change":"2024-09-10T21:30:37.154771+0000","last_active":"2024-09-10T21:30:38.152034+0000","last_peered":"2024-09-10T21:30:38.152034+0000","last_clean":"2024-09-10T21:30:38.152034+0000","last_became_active":"2024-09-10T21:30:37.154432+0000","last_became_peered":"2024-09-10T21:30:37.154432+0000","last_unstale":"2024-09-10T21:30:38.152034+0000","last_undegraded":"2024-09-10T21:30:38.152034+0000","last_fullsized":"2024-09-10T21:30:38.152034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:04.406280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151914+0000","last_change":"2024-09-10T21:30:36.149165+0000","last_active":"2024-09-10T21:30:38.151914+0000","last_peered":"2024-09-10T21:30:38.151914+0000","last_clean":"2024-09-10T21:30:38.151914+0000","last_became_active":"2024-09-10T21:30:36.148849+0000","last_became_peered":"2024-09-10T21:30:36.148849+0000","last_unstale":"2024-09-10T21:30:38.151914+0000","last_undegraded":"2024-09-10T21:30:38.151914+0000","last_fullsized":"2024-09-10T21:30:38.151914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:34.753321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287758+0000","last_change":"2024-09-10T21:30:37.160123+0000","last_active":"2024-09-10T21:30:38.287758+0000","last_peered":"2024-09-10T21:30:38.287758+0000","last_clean":"2024-09-10T21:30:38.287758+0000","last_became_active":"2024-09-10T21:30:37.159833+0000","last_became_peered":"2024-09-10T21:30:37.159833+0000","last_unstale":"2024-09-10T21:30:38.287758+0000","last_undegraded":"2024-09-10T21:30:38.287758+0000","last_fullsized":"2024-09-10T21:30:38.287758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:41.232502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152185+0000","last_change":"2024-09-10T21:30:36.149184+0000","last_active":"2024-09-10T21:30:38.152185+0000","last_peered":"2024-09-10T21:30:38.152185+0000","last_clean":"2024-09-10T21:30:38.152185+0000","last_became_active":"2024-09-10T21:30:36.148915+0000","last_became_peered":"2024-09-10T21:30:36.148915+0000","last_unstale":"2024-09-10T21:30:38.152185+0000","last_undegraded":"2024-09-10T21:30:38.152185+0000","last_fullsized":"2024-09-10T21:30:38.152185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:04.537601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288504+0000","last_change":"2024-09-10T21:30:37.147434+0000","last_active":"2024-09-10T21:30:38.288504+0000","last_peered":"2024-09-10T21:30:38.288504+0000","last_clean":"2024-09-10T21:30:38.288504+0000","last_became_active":"2024-09-10T21:30:37.147160+0000","last_became_peered":"2024-09-10T21:30:37.147160+0000","last_unstale":"2024-09-10T21:30:38.288504+0000","last_undegraded":"2024-09-10T21:30:38.288504+0000","last_fullsized":"2024-09-10T21:30:38.288504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:07.679232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288314+0000","last_change":"2024-09-10T21:30:36.158339+0000","last_active":"2024-09-10T21:30:38.288314+0000","last_peered":"2024-09-10T21:30:38.288314+0000","last_clean":"2024-09-10T21:30:38.288314+0000","last_became_active":"2024-09-10T21:30:36.158050+0000","last_became_peered":"2024-09-10T21:30:36.158050+0000","last_unstale":"2024-09-10T21:30:38.288314+0000","last_undegraded":"2024-09-10T21:30:38.288314+0000","last_fullsized":"2024-09-10T21:30:38.288314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:39.986638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286957+0000","last_change":"2024-09-10T21:30:37.155599+0000","last_active":"2024-09-10T21:30:38.286957+0000","last_peered":"2024-09-10T21:30:38.286957+0000","last_clean":"2024-09-10T21:30:38.286957+0000","last_became_active":"2024-09-10T21:30:37.154937+0000","last_became_peered":"2024-09-10T21:30:37.154937+0000","last_unstale":"2024-09-10T21:30:38.286957+0000","last_undegraded":"2024-09-10T21:30:38.286957+0000","last_fullsized":"2024-09-10T21:30:38.286957+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:48.396677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154829+0000","last_change":"2024-09-10T21:30:36.155028+0000","last_active":"2024-09-10T21:30:38.154829+0000","last_peered":"2024-09-10T21:30:38.154829+0000","last_clean":"2024-09-10T21:30:38.154829+0000","last_became_active":"2024-09-10T21:30:36.154105+0000","last_became_peered":"2024-09-10T21:30:36.154105+0000","last_unstale":"2024-09-10T21:30:38.154829+0000","last_undegraded":"2024-09-10T21:30:38.154829+0000","last_fullsized":"2024-09-10T21:30:38.154829+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:23.894646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154240+0000","last_change":"2024-09-10T21:30:37.147903+0000","last_active":"2024-09-10T21:30:38.154240+0000","last_peered":"2024-09-10T21:30:38.154240+0000","last_clean":"2024-09-10T21:30:38.154240+0000","last_became_active":"2024-09-10T21:30:37.147347+0000","last_became_peered":"2024-09-10T21:30:37.147347+0000","last_unstale":"2024-09-10T21:30:38.154240+0000","last_undegraded":"2024-09-10T21:30:38.154240+0000","last_fullsized":"2024-09-10T21:30:38.154240+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:47.060669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288548+0000","last_change":"2024-09-10T21:30:36.149196+0000","last_active":"2024-09-10T21:30:38.288548+0000","last_peered":"2024-09-10T21:30:38.288548+0000","last_clean":"2024-09-10T21:30:38.288548+0000","last_became_active":"2024-09-10T21:30:36.148907+0000","last_became_peered":"2024-09-10T21:30:36.148907+0000","last_unstale":"2024-09-10T21:30:38.288548+0000","last_undegraded":"2024-09-10T21:30:38.288548+0000","last_fullsized":"2024-09-10T21:30:38.288548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288482+0000","last_change":"2024-09-10T21:30:37.160607+0000","last_active":"2024-09-10T21:30:38.288482+0000","last_peered":"2024-09-10T21:30:38.288482+0000","last_clean":"2024-09-10T21:30:38.288482+0000","last_became_active":"2024-09-10T21:30:37.159663+0000","last_became_peered":"2024-09-10T21:30:37.159663+0000","last_unstale":"2024-09-10T21:30:38.288482+0000","last_undegraded":"2024-09-10T21:30:38.288482+0000","last_fullsized":"2024-09-10T21:30:38.288482+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:55.977517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287012+0000","last_change":"2024-09-10T21:30:36.152984+0000","last_active":"2024-09-10T21:30:38.287012+0000","last_peered":"2024-09-10T21:30:38.287012+0000","last_clean":"2024-09-10T21:30:38.287012+0000","last_became_active":"2024-09-10T21:30:36.152704+0000","last_became_peered":"2024-09-10T21:30:36.152704+0000","last_unstale":"2024-09-10T21:30:38.287012+0000","last_undegraded":"2024-09-10T21:30:38.287012+0000","last_fullsized":"2024-09-10T21:30:38.287012+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:43.536977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151820+0000","last_change":"2024-09-10T21:30:37.163498+0000","last_active":"2024-09-10T21:30:38.151820+0000","last_peered":"2024-09-10T21:30:38.151820+0000","last_clean":"2024-09-10T21:30:38.151820+0000","last_became_active":"2024-09-10T21:30:37.163181+0000","last_became_peered":"2024-09-10T21:30:37.163181+0000","last_unstale":"2024-09-10T21:30:38.151820+0000","last_undegraded":"2024-09-10T21:30:38.151820+0000","last_fullsized":"2024-09-10T21:30:38.151820+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:10.882901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150470+0000","last_change":"2024-09-10T21:30:36.150376+0000","last_active":"2024-09-10T21:30:38.150470+0000","last_peered":"2024-09-10T21:30:38.150470+0000","last_clean":"2024-09-10T21:30:38.150470+0000","last_became_active":"2024-09-10T21:30:36.150090+0000","last_became_peered":"2024-09-10T21:30:36.150090+0000","last_unstale":"2024-09-10T21:30:38.150470+0000","last_undegraded":"2024-09-10T21:30:38.150470+0000","last_fullsized":"2024-09-10T21:30:38.150470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:49.705316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151920+0000","last_change":"2024-09-10T21:30:37.166907+0000","last_active":"2024-09-10T21:30:38.151920+0000","last_peered":"2024-09-10T21:30:38.151920+0000","last_clean":"2024-09-10T21:30:38.151920+0000","last_became_active":"2024-09-10T21:30:37.164943+0000","last_became_peered":"2024-09-10T21:30:37.164943+0000","last_unstale":"2024-09-10T21:30:38.151920+0000","last_undegraded":"2024-09-10T21:30:38.151920+0000","last_fullsized":"2024-09-10T21:30:38.151920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:28.954938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286112+0000","last_change":"2024-09-10T21:30:36.153213+0000","last_active":"2024-09-10T21:30:38.286112+0000","last_peered":"2024-09-10T21:30:38.286112+0000","last_clean":"2024-09-10T21:30:38.286112+0000","last_became_active":"2024-09-10T21:30:36.152918+0000","last_became_peered":"2024-09-10T21:30:36.152918+0000","last_unstale":"2024-09-10T21:30:38.286112+0000","last_undegraded":"2024-09-10T21:30:38.286112+0000","last_fullsized":"2024-09-10T21:30:38.286112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:09.875102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152202+0000","last_change":"2024-09-10T21:30:37.147332+0000","last_active":"2024-09-10T21:30:38.152202+0000","last_peered":"2024-09-10T21:30:38.152202+0000","last_clean":"2024-09-10T21:30:38.152202+0000","last_became_active":"2024-09-10T21:30:37.146975+0000","last_became_peered":"2024-09-10T21:30:37.146975+0000","last_unstale":"2024-09-10T21:30:38.152202+0000","last_undegraded":"2024-09-10T21:30:38.152202+0000","last_fullsized":"2024-09-10T21:30:38.152202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:38.142189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148744+0000","last_change":"2024-09-10T21:30:36.155934+0000","last_active":"2024-09-10T21:30:38.148744+0000","last_peered":"2024-09-10T21:30:38.148744+0000","last_clean":"2024-09-10T21:30:38.148744+0000","last_became_active":"2024-09-10T21:30:36.155669+0000","last_became_peered":"2024-09-10T21:30:36.155669+0000","last_unstale":"2024-09-10T21:30:38.148744+0000","last_undegraded":"2024-09-10T21:30:38.148744+0000","last_fullsized":"2024-09-10T21:30:38.148744+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:56.357501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154965+0000","last_change":"2024-09-10T21:30:37.158812+0000","last_active":"2024-09-10T21:30:38.154965+0000","last_peered":"2024-09-10T21:30:38.154965+0000","last_clean":"2024-09-10T21:30:38.154965+0000","last_became_active":"2024-09-10T21:30:37.158370+0000","last_became_peered":"2024-09-10T21:30:37.158370+0000","last_unstale":"2024-09-10T21:30:38.154965+0000","last_undegraded":"2024-09-10T21:30:38.154965+0000","last_fullsized":"2024-09-10T21:30:38.154965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:05.545400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148656+0000","last_change":"2024-09-10T21:30:36.152792+0000","last_active":"2024-09-10T21:30:38.148656+0000","last_peered":"2024-09-10T21:30:38.148656+0000","last_clean":"2024-09-10T21:30:38.148656+0000","last_became_active":"2024-09-10T21:30:36.152526+0000","last_became_peered":"2024-09-10T21:30:36.152526+0000","last_unstale":"2024-09-10T21:30:38.148656+0000","last_undegraded":"2024-09-10T21:30:38.148656+0000","last_fullsized":"2024-09-10T21:30:38.148656+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287996+0000","last_change":"2024-09-10T21:30:37.163752+0000","last_active":"2024-09-10T21:30:38.287996+0000","last_peered":"2024-09-10T21:30:38.287996+0000","last_clean":"2024-09-10T21:30:38.287996+0000","last_became_active":"2024-09-10T21:30:37.162710+0000","last_became_peered":"2024-09-10T21:30:37.162710+0000","last_unstale":"2024-09-10T21:30:38.287996+0000","last_undegraded":"2024-09-10T21:30:38.287996+0000","last_fullsized":"2024-09-10T21:30:38.287996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:33.852578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151847+0000","last_change":"2024-09-10T21:30:36.153508+0000","last_active":"2024-09-10T21:30:38.151847+0000","last_peered":"2024-09-10T21:30:38.151847+0000","last_clean":"2024-09-10T21:30:38.151847+0000","last_became_active":"2024-09-10T21:30:36.153255+0000","last_became_peered":"2024-09-10T21:30:36.153255+0000","last_unstale":"2024-09-10T21:30:38.151847+0000","last_undegraded":"2024-09-10T21:30:38.151847+0000","last_fullsized":"2024-09-10T21:30:38.151847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:30.450914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148273+0000","last_change":"2024-09-10T21:30:37.146511+0000","last_active":"2024-09-10T21:30:38.148273+0000","last_peered":"2024-09-10T21:30:38.148273+0000","last_clean":"2024-09-10T21:30:38.148273+0000","last_became_active":"2024-09-10T21:30:37.143002+0000","last_became_peered":"2024-09-10T21:30:37.143002+0000","last_unstale":"2024-09-10T21:30:38.148273+0000","last_undegraded":"2024-09-10T21:30:38.148273+0000","last_fullsized":"2024-09-10T21:30:38.148273+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:14.904422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287713+0000","last_change":"2024-09-10T21:30:36.153764+0000","last_active":"2024-09-10T21:30:38.287713+0000","last_peered":"2024-09-10T21:30:38.287713+0000","last_clean":"2024-09-10T21:30:38.287713+0000","last_became_active":"2024-09-10T21:30:36.153453+0000","last_became_peered":"2024-09-10T21:30:36.153453+0000","last_unstale":"2024-09-10T21:30:38.287713+0000","last_undegraded":"2024-09-10T21:30:38.287713+0000","last_fullsized":"2024-09-10T21:30:38.287713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:28.260842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286121+0000","last_change":"2024-09-10T21:30:37.160469+0000","last_active":"2024-09-10T21:30:38.286121+0000","last_peered":"2024-09-10T21:30:38.286121+0000","last_clean":"2024-09-10T21:30:38.286121+0000","last_became_active":"2024-09-10T21:30:37.159860+0000","last_became_peered":"2024-09-10T21:30:37.159860+0000","last_unstale":"2024-09-10T21:30:38.286121+0000","last_undegraded":"2024-09-10T21:30:38.286121+0000","last_fullsized":"2024-09-10T21:30:38.286121+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:47.060669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148223+0000","last_change":"2024-09-10T21:30:36.145884+0000","last_active":"2024-09-10T21:30:38.148223+0000","last_peered":"2024-09-10T21:30:38.148223+0000","last_clean":"2024-09-10T21:30:38.148223+0000","last_became_active":"2024-09-10T21:30:36.145592+0000","last_became_peered":"2024-09-10T21:30:36.145592+0000","last_unstale":"2024-09-10T21:30:38.148223+0000","last_undegraded":"2024-09-10T21:30:38.148223+0000","last_fullsized":"2024-09-10T21:30:38.148223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:36.544620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288227+0000","last_change":"2024-09-10T21:30:37.164650+0000","last_active":"2024-09-10T21:30:38.288227+0000","last_peered":"2024-09-10T21:30:38.288227+0000","last_clean":"2024-09-10T21:30:38.288227+0000","last_became_active":"2024-09-10T21:30:37.164223+0000","last_became_peered":"2024-09-10T21:30:37.164223+0000","last_unstale":"2024-09-10T21:30:38.288227+0000","last_undegraded":"2024-09-10T21:30:38.288227+0000","last_fullsized":"2024-09-10T21:30:38.288227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:07.663482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151192+0000","last_change":"2024-09-10T21:30:36.153170+0000","last_active":"2024-09-10T21:30:38.151192+0000","last_peered":"2024-09-10T21:30:38.151192+0000","last_clean":"2024-09-10T21:30:38.151192+0000","last_became_active":"2024-09-10T21:30:36.152223+0000","last_became_peered":"2024-09-10T21:30:36.152223+0000","last_unstale":"2024-09-10T21:30:38.151192+0000","last_undegraded":"2024-09-10T21:30:38.151192+0000","last_fullsized":"2024-09-10T21:30:38.151192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286078+0000","last_change":"2024-09-10T21:30:37.156348+0000","last_active":"2024-09-10T21:30:38.286078+0000","last_peered":"2024-09-10T21:30:38.286078+0000","last_clean":"2024-09-10T21:30:38.286078+0000","last_became_active":"2024-09-10T21:30:37.156039+0000","last_became_peered":"2024-09-10T21:30:37.156039+0000","last_unstale":"2024-09-10T21:30:38.286078+0000","last_undegraded":"2024-09-10T21:30:38.286078+0000","last_fullsized":"2024-09-10T21:30:38.286078+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:41.232502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152262+0000","last_change":"2024-09-10T21:30:36.149909+0000","last_active":"2024-09-10T21:30:38.152262+0000","last_peered":"2024-09-10T21:30:38.152262+0000","last_clean":"2024-09-10T21:30:38.152262+0000","last_became_active":"2024-09-10T21:30:36.149596+0000","last_became_peered":"2024-09-10T21:30:36.149596+0000","last_unstale":"2024-09-10T21:30:38.152262+0000","last_undegraded":"2024-09-10T21:30:38.152262+0000","last_fullsized":"2024-09-10T21:30:38.152262+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:09.356665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152558+0000","last_change":"2024-09-10T21:30:37.158088+0000","last_active":"2024-09-10T21:30:38.152558+0000","last_peered":"2024-09-10T21:30:38.152558+0000","last_clean":"2024-09-10T21:30:38.152558+0000","last_became_active":"2024-09-10T21:30:37.157676+0000","last_became_peered":"2024-09-10T21:30:37.157676+0000","last_unstale":"2024-09-10T21:30:38.152558+0000","last_undegraded":"2024-09-10T21:30:38.152558+0000","last_fullsized":"2024-09-10T21:30:38.152558+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:19.172370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152097+0000","last_change":"2024-09-10T21:30:36.151988+0000","last_active":"2024-09-10T21:30:38.152097+0000","last_peered":"2024-09-10T21:30:38.152097+0000","last_clean":"2024-09-10T21:30:38.152097+0000","last_became_active":"2024-09-10T21:30:36.151626+0000","last_became_peered":"2024-09-10T21:30:36.151626+0000","last_unstale":"2024-09-10T21:30:38.152097+0000","last_undegraded":"2024-09-10T21:30:38.152097+0000","last_fullsized":"2024-09-10T21:30:38.152097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:53.273442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152467+0000","last_change":"2024-09-10T21:30:37.166893+0000","last_active":"2024-09-10T21:30:38.152467+0000","last_peered":"2024-09-10T21:30:38.152467+0000","last_clean":"2024-09-10T21:30:38.152467+0000","last_became_active":"2024-09-10T21:30:37.166457+0000","last_became_peered":"2024-09-10T21:30:37.166457+0000","last_unstale":"2024-09-10T21:30:38.152467+0000","last_undegraded":"2024-09-10T21:30:38.152467+0000","last_fullsized":"2024-09-10T21:30:38.152467+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:03.101308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286142+0000","last_change":"2024-09-10T21:30:36.156216+0000","last_active":"2024-09-10T21:30:38.286142+0000","last_peered":"2024-09-10T21:30:38.286142+0000","last_clean":"2024-09-10T21:30:38.286142+0000","last_became_active":"2024-09-10T21:30:36.155962+0000","last_became_peered":"2024-09-10T21:30:36.155962+0000","last_unstale":"2024-09-10T21:30:38.286142+0000","last_undegraded":"2024-09-10T21:30:38.286142+0000","last_fullsized":"2024-09-10T21:30:38.286142+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:06.307063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154748+0000","last_change":"2024-09-10T21:30:37.156724+0000","last_active":"2024-09-10T21:30:38.154748+0000","last_peered":"2024-09-10T21:30:38.154748+0000","last_clean":"2024-09-10T21:30:38.154748+0000","last_became_active":"2024-09-10T21:30:37.156047+0000","last_became_peered":"2024-09-10T21:30:37.156047+0000","last_unstale":"2024-09-10T21:30:38.154748+0000","last_undegraded":"2024-09-10T21:30:38.154748+0000","last_fullsized":"2024-09-10T21:30:38.154748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:02.614328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152110+0000","last_change":"2024-09-10T21:30:36.146558+0000","last_active":"2024-09-10T21:30:38.152110+0000","last_peered":"2024-09-10T21:30:38.152110+0000","last_clean":"2024-09-10T21:30:38.152110+0000","last_became_active":"2024-09-10T21:30:36.144327+0000","last_became_peered":"2024-09-10T21:30:36.144327+0000","last_unstale":"2024-09-10T21:30:38.152110+0000","last_undegraded":"2024-09-10T21:30:38.152110+0000","last_fullsized":"2024-09-10T21:30:38.152110+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:54.969718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287947+0000","last_change":"2024-09-10T21:30:37.162963+0000","last_active":"2024-09-10T21:30:38.287947+0000","last_peered":"2024-09-10T21:30:38.287947+0000","last_clean":"2024-09-10T21:30:38.287947+0000","last_became_active":"2024-09-10T21:30:37.162366+0000","last_became_peered":"2024-09-10T21:30:37.162366+0000","last_unstale":"2024-09-10T21:30:38.287947+0000","last_undegraded":"2024-09-10T21:30:38.287947+0000","last_fullsized":"2024-09-10T21:30:38.287947+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:37.376166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288447+0000","last_change":"2024-09-10T21:30:36.159592+0000","last_active":"2024-09-10T21:30:38.288447+0000","last_peered":"2024-09-10T21:30:38.288447+0000","last_clean":"2024-09-10T21:30:38.288447+0000","last_became_active":"2024-09-10T21:30:36.159354+0000","last_became_peered":"2024-09-10T21:30:36.159354+0000","last_unstale":"2024-09-10T21:30:38.288447+0000","last_undegraded":"2024-09-10T21:30:38.288447+0000","last_fullsized":"2024-09-10T21:30:38.288447+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:10.866141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150731+0000","last_change":"2024-09-10T21:30:37.158517+0000","last_active":"2024-09-10T21:30:38.150731+0000","last_peered":"2024-09-10T21:30:38.150731+0000","last_clean":"2024-09-10T21:30:38.150731+0000","last_became_active":"2024-09-10T21:30:37.153692+0000","last_became_peered":"2024-09-10T21:30:37.153692+0000","last_unstale":"2024-09-10T21:30:38.150731+0000","last_undegraded":"2024-09-10T21:30:38.150731+0000","last_fullsized":"2024-09-10T21:30:38.150731+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:32.366799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154881+0000","last_change":"2024-09-10T21:30:36.157140+0000","last_active":"2024-09-10T21:30:38.154881+0000","last_peered":"2024-09-10T21:30:38.154881+0000","last_clean":"2024-09-10T21:30:38.154881+0000","last_became_active":"2024-09-10T21:30:36.156861+0000","last_became_peered":"2024-09-10T21:30:36.156861+0000","last_unstale":"2024-09-10T21:30:38.154881+0000","last_undegraded":"2024-09-10T21:30:38.154881+0000","last_fullsized":"2024-09-10T21:30:38.154881+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.541056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286243+0000","last_change":"2024-09-10T21:30:37.160667+0000","last_active":"2024-09-10T21:30:38.286243+0000","last_peered":"2024-09-10T21:30:38.286243+0000","last_clean":"2024-09-10T21:30:38.286243+0000","last_became_active":"2024-09-10T21:30:37.160187+0000","last_became_peered":"2024-09-10T21:30:37.160187+0000","last_unstale":"2024-09-10T21:30:38.286243+0000","last_undegraded":"2024-09-10T21:30:38.286243+0000","last_fullsized":"2024-09-10T21:30:38.286243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:02.614328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154526+0000","last_change":"2024-09-10T21:30:36.147814+0000","last_active":"2024-09-10T21:30:38.154526+0000","last_peered":"2024-09-10T21:30:38.154526+0000","last_clean":"2024-09-10T21:30:38.154526+0000","last_became_active":"2024-09-10T21:30:36.147488+0000","last_became_peered":"2024-09-10T21:30:36.147488+0000","last_unstale":"2024-09-10T21:30:38.154526+0000","last_undegraded":"2024-09-10T21:30:38.154526+0000","last_fullsized":"2024-09-10T21:30:38.154526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:31.359000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287689+0000","last_change":"2024-09-10T21:30:37.164739+0000","last_active":"2024-09-10T21:30:38.287689+0000","last_peered":"2024-09-10T21:30:38.287689+0000","last_clean":"2024-09-10T21:30:38.287689+0000","last_became_active":"2024-09-10T21:30:37.164473+0000","last_became_peered":"2024-09-10T21:30:37.164473+0000","last_unstale":"2024-09-10T21:30:38.287689+0000","last_undegraded":"2024-09-10T21:30:38.287689+0000","last_fullsized":"2024-09-10T21:30:38.287689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:36.726703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154255+0000","last_change":"2024-09-10T21:30:36.156585+0000","last_active":"2024-09-10T21:30:38.154255+0000","last_peered":"2024-09-10T21:30:38.154255+0000","last_clean":"2024-09-10T21:30:38.154255+0000","last_became_active":"2024-09-10T21:30:36.156288+0000","last_became_peered":"2024-09-10T21:30:36.156288+0000","last_unstale":"2024-09-10T21:30:38.154255+0000","last_undegraded":"2024-09-10T21:30:38.154255+0000","last_fullsized":"2024-09-10T21:30:38.154255+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:44.644836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154765+0000","last_change":"2024-09-10T21:30:37.156412+0000","last_active":"2024-09-10T21:30:38.154765+0000","last_peered":"2024-09-10T21:30:38.154765+0000","last_clean":"2024-09-10T21:30:38.154765+0000","last_became_active":"2024-09-10T21:30:37.155638+0000","last_became_peered":"2024-09-10T21:30:37.155638+0000","last_unstale":"2024-09-10T21:30:38.154765+0000","last_undegraded":"2024-09-10T21:30:38.154765+0000","last_fullsized":"2024-09-10T21:30:38.154765+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:38.366747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288471+0000","last_change":"2024-09-10T21:30:36.151215+0000","last_active":"2024-09-10T21:30:38.288471+0000","last_peered":"2024-09-10T21:30:38.288471+0000","last_clean":"2024-09-10T21:30:38.288471+0000","last_became_active":"2024-09-10T21:30:36.150699+0000","last_became_peered":"2024-09-10T21:30:36.150699+0000","last_unstale":"2024-09-10T21:30:38.288471+0000","last_undegraded":"2024-09-10T21:30:38.288471+0000","last_fullsized":"2024-09-10T21:30:38.288471+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:13.896623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":248,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154288+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:30:38.154288+0000","last_peered":"2024-09-10T21:30:38.154288+0000","last_clean":"2024-09-10T21:30:38.154288+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:30:38.154288+0000","last_undegraded":"2024-09-10T21:30:38.154288+0000","last_fullsized":"2024-09-10T21:30:38.154288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1150976,"data_stored":532534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1171456,"data_stored":534270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1744896,"data_stored":1122902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1171456,"data_stored":534258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509523,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1159168,"data_stored":533138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509524,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1757184,"data_stored":1123588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1155072,"data_stored":532604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1138688,"data_stored":525842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:30:45.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:30:46.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 -- 172.21.15.97:0/3444048908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4880ceee0 msgr2=0x7fc4880cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 --2- 172.21.15.97:0/3444048908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4880ceee0 0x7fc4880cf370 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc480009bc0 tx=0x7fc48001f3b0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 -- 172.21.15.97:0/3444048908 shutdown_connections 2024-09-10T21:30:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 --2- 172.21.15.97:0/3444048908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4880ceee0 0x7fc4880cf370 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 --2- 172.21.15.97:0/3444048908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc48806ec40 0x7fc48806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 -- 172.21.15.97:0/3444048908 >> 172.21.15.97:0/3444048908 conn(0x7fc4880b1780 msgr2=0x7fc4880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 -- 172.21.15.97:0/3444048908 shutdown_connections 2024-09-10T21:30:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 -- 172.21.15.97:0/3444048908 wait complete. 2024-09-10T21:30:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 Processor -- start 2024-09-10T21:30:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 -- start start 2024-09-10T21:30:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc48806ec40 0x7fc48806f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4880ceee0 0x7fc48806fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.052+0000 7fc48d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4881511f0 0x7fc488135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc48d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc48806e740 con 0x7fc48806ec40 2024-09-10T21:30:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc48d400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc48806e5c0 con 0x7fc4880ceee0 2024-09-10T21:30:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc48d400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc48806e8c0 con 0x7fc4881511f0 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc48806ec40 0x7fc48806f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc48806ec40 0x7fc48806f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48726/0 (socket says 172.21.15.97:48726) 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 -- 172.21.15.97:0/1016047110 learned_addr learned my addr 172.21.15.97:0/1016047110 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc487200640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4881511f0 0x7fc488135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc47de00640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4880ceee0 0x7fc48806fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 -- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4881511f0 msgr2=0x7fc488135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4881511f0 0x7fc488135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 -- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4880ceee0 msgr2=0x7fc48806fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4880ceee0 0x7fc48806fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 -- 172.21.15.97:0/1016047110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4881f8170 con 0x7fc48806ec40 2024-09-10T21:30:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc486800640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc48806ec40 0x7fc48806f690 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c450 tx=0x7fc47800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc47f400640 1 -- 172.21.15.97:0/1016047110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478016020 con 0x7fc48806ec40 2024-09-10T21:30:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc47f400640 1 -- 172.21.15.97:0/1016047110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc478012920 con 0x7fc48806ec40 2024-09-10T21:30:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4881f8450 con 0x7fc48806ec40 2024-09-10T21:30:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc47f400640 1 -- 172.21.15.97:0/1016047110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478012d70 con 0x7fc48806ec40 2024-09-10T21:30:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4881f8a10 con 0x7fc48806ec40 2024-09-10T21:30:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48806f060 con 0x7fc48806ec40 2024-09-10T21:30:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.056+0000 7fc47f400640 1 -- 172.21.15.97:0/1016047110 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc47801b400 con 0x7fc48806ec40 2024-09-10T21:30:46.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.060+0000 7fc47f400640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4640725e0 0x7fc464074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.060+0000 7fc47f400640 1 -- 172.21.15.97:0/1016047110 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fc478093520 con 0x7fc48806ec40 2024-09-10T21:30:46.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.060+0000 7fc47de00640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4640725e0 0x7fc464074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.060+0000 7fc47de00640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4640725e0 0x7fc464074aa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc4880b2120 tx=0x7fc47000b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.064+0000 7fc47f400640 1 -- 172.21.15.97:0/1016047110 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc478020020 con 0x7fc48806ec40 2024-09-10T21:30:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.228+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4880cbee0 con 0x7fc4640725e0 2024-09-10T21:30:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.232+0000 7fc47f400640 1 -- 172.21.15.97:0/1016047110 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fc4880cbee0 con 0x7fc4640725e0 2024-09-10T21:30:46.241 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4640725e0 msgr2=0x7fc464074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4640725e0 0x7fc464074aa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc4880b2120 tx=0x7fc47000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc48806ec40 msgr2=0x7fc48806f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc48806ec40 0x7fc48806f690 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c450 tx=0x7fc47800c910 comp rx=0 tx=0).stop 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 shutdown_connections 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc4640725e0 0x7fc464074aa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc4881511f0 0x7fc488135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4880ceee0 0x7fc48806fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 --2- 172.21.15.97:0/1016047110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc48806ec40 0x7fc48806f690 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 >> 172.21.15.97:0/1016047110 conn(0x7fc4880b1780 msgr2=0x7fc48814b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 shutdown_connections 2024-09-10T21:30:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.236+0000 7fc48d400640 1 -- 172.21.15.97:0/1016047110 wait complete. 2024-09-10T21:30:46.259 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-10T21:30:46.170566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255604,"kb_used_data":10216,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719116,"statfs":{"total":773094113280,"available":772832374784,"internally_reserved":0,"allocated":10461184,"data_stored":5445828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.037876"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287886+0000","last_change":"2024-09-10T21:30:37.159556+0000","last_active":"2024-09-10T21:30:38.287886+0000","last_peered":"2024-09-10T21:30:38.287886+0000","last_clean":"2024-09-10T21:30:38.287886+0000","last_became_active":"2024-09-10T21:30:37.159215+0000","last_became_peered":"2024-09-10T21:30:37.159215+0000","last_unstale":"2024-09-10T21:30:38.287886+0000","last_undegraded":"2024-09-10T21:30:38.287886+0000","last_fullsized":"2024-09-10T21:30:38.287886+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:47.060669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154534+0000","last_change":"2024-09-10T21:30:37.159414+0000","last_active":"2024-09-10T21:30:38.154534+0000","last_peered":"2024-09-10T21:30:38.154534+0000","last_clean":"2024-09-10T21:30:38.154534+0000","last_became_active":"2024-09-10T21:30:37.159157+0000","last_became_peered":"2024-09-10T21:30:37.159157+0000","last_unstale":"2024-09-10T21:30:38.154534+0000","last_undegraded":"2024-09-10T21:30:38.154534+0000","last_fullsized":"2024-09-10T21:30:38.154534+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:09.640497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152689+0000","last_change":"2024-09-10T21:30:37.157235+0000","last_active":"2024-09-10T21:30:38.152689+0000","last_peered":"2024-09-10T21:30:38.152689+0000","last_clean":"2024-09-10T21:30:38.152689+0000","last_became_active":"2024-09-10T21:30:37.156065+0000","last_became_peered":"2024-09-10T21:30:37.156065+0000","last_unstale":"2024-09-10T21:30:38.152689+0000","last_undegraded":"2024-09-10T21:30:38.152689+0000","last_fullsized":"2024-09-10T21:30:38.152689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:57.193003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151359+0000","last_change":"2024-09-10T21:30:37.159075+0000","last_active":"2024-09-10T21:30:38.151359+0000","last_peered":"2024-09-10T21:30:38.151359+0000","last_clean":"2024-09-10T21:30:38.151359+0000","last_became_active":"2024-09-10T21:30:37.156300+0000","last_became_peered":"2024-09-10T21:30:37.156300+0000","last_unstale":"2024-09-10T21:30:38.151359+0000","last_undegraded":"2024-09-10T21:30:38.151359+0000","last_fullsized":"2024-09-10T21:30:38.151359+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:24.902445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154550+0000","last_change":"2024-09-10T21:30:37.153572+0000","last_active":"2024-09-10T21:30:38.154550+0000","last_peered":"2024-09-10T21:30:38.154550+0000","last_clean":"2024-09-10T21:30:38.154550+0000","last_became_active":"2024-09-10T21:30:37.152445+0000","last_became_peered":"2024-09-10T21:30:37.152445+0000","last_unstale":"2024-09-10T21:30:38.154550+0000","last_undegraded":"2024-09-10T21:30:38.154550+0000","last_fullsized":"2024-09-10T21:30:38.154550+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:53.825751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287577+0000","last_change":"2024-09-10T21:30:37.156704+0000","last_active":"2024-09-10T21:30:38.287577+0000","last_peered":"2024-09-10T21:30:38.287577+0000","last_clean":"2024-09-10T21:30:38.287577+0000","last_became_active":"2024-09-10T21:30:37.154734+0000","last_became_peered":"2024-09-10T21:30:37.154734+0000","last_unstale":"2024-09-10T21:30:38.287577+0000","last_undegraded":"2024-09-10T21:30:38.287577+0000","last_fullsized":"2024-09-10T21:30:38.287577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:59.951655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151748+0000","last_change":"2024-09-10T21:30:37.157302+0000","last_active":"2024-09-10T21:30:38.151748+0000","last_peered":"2024-09-10T21:30:38.151748+0000","last_clean":"2024-09-10T21:30:38.151748+0000","last_became_active":"2024-09-10T21:30:37.156825+0000","last_became_peered":"2024-09-10T21:30:37.156825+0000","last_unstale":"2024-09-10T21:30:38.151748+0000","last_undegraded":"2024-09-10T21:30:38.151748+0000","last_fullsized":"2024-09-10T21:30:38.151748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:13.008263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287144+0000","last_change":"2024-09-10T21:30:37.146717+0000","last_active":"2024-09-10T21:30:38.287144+0000","last_peered":"2024-09-10T21:30:38.287144+0000","last_clean":"2024-09-10T21:30:38.287144+0000","last_became_active":"2024-09-10T21:30:37.146428+0000","last_became_peered":"2024-09-10T21:30:37.146428+0000","last_unstale":"2024-09-10T21:30:38.287144+0000","last_undegraded":"2024-09-10T21:30:38.287144+0000","last_fullsized":"2024-09-10T21:30:38.287144+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:18.307373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152277+0000","last_change":"2024-09-10T21:30:37.155234+0000","last_active":"2024-09-10T21:30:38.152277+0000","last_peered":"2024-09-10T21:30:38.152277+0000","last_clean":"2024-09-10T21:30:38.152277+0000","last_became_active":"2024-09-10T21:30:37.154993+0000","last_became_peered":"2024-09-10T21:30:37.154993+0000","last_unstale":"2024-09-10T21:30:38.152277+0000","last_undegraded":"2024-09-10T21:30:38.152277+0000","last_fullsized":"2024-09-10T21:30:38.152277+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:17.183416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151444+0000","last_change":"2024-09-10T21:30:37.155994+0000","last_active":"2024-09-10T21:30:38.151444+0000","last_peered":"2024-09-10T21:30:38.151444+0000","last_clean":"2024-09-10T21:30:38.151444+0000","last_became_active":"2024-09-10T21:30:37.155283+0000","last_became_peered":"2024-09-10T21:30:37.155283+0000","last_unstale":"2024-09-10T21:30:38.151444+0000","last_undegraded":"2024-09-10T21:30:38.151444+0000","last_fullsized":"2024-09-10T21:30:38.151444+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:11.548855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152003+0000","last_change":"2024-09-10T21:30:37.157683+0000","last_active":"2024-09-10T21:30:38.152003+0000","last_peered":"2024-09-10T21:30:38.152003+0000","last_clean":"2024-09-10T21:30:38.152003+0000","last_became_active":"2024-09-10T21:30:37.156106+0000","last_became_peered":"2024-09-10T21:30:37.156106+0000","last_unstale":"2024-09-10T21:30:38.152003+0000","last_undegraded":"2024-09-10T21:30:38.152003+0000","last_fullsized":"2024-09-10T21:30:38.152003+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:27.891796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151246+0000","last_change":"2024-09-10T21:30:37.159806+0000","last_active":"2024-09-10T21:30:38.151246+0000","last_peered":"2024-09-10T21:30:38.151246+0000","last_clean":"2024-09-10T21:30:38.151246+0000","last_became_active":"2024-09-10T21:30:37.159533+0000","last_became_peered":"2024-09-10T21:30:37.159533+0000","last_unstale":"2024-09-10T21:30:38.151246+0000","last_undegraded":"2024-09-10T21:30:38.151246+0000","last_fullsized":"2024-09-10T21:30:38.151246+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:56.957241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151498+0000","last_change":"2024-09-10T21:30:37.158510+0000","last_active":"2024-09-10T21:30:38.151498+0000","last_peered":"2024-09-10T21:30:38.151498+0000","last_clean":"2024-09-10T21:30:38.151498+0000","last_became_active":"2024-09-10T21:30:37.158191+0000","last_became_peered":"2024-09-10T21:30:37.158191+0000","last_unstale":"2024-09-10T21:30:38.151498+0000","last_undegraded":"2024-09-10T21:30:38.151498+0000","last_fullsized":"2024-09-10T21:30:38.151498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:07.314862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148326+0000","last_change":"2024-09-10T21:30:37.146477+0000","last_active":"2024-09-10T21:30:38.148326+0000","last_peered":"2024-09-10T21:30:38.148326+0000","last_clean":"2024-09-10T21:30:38.148326+0000","last_became_active":"2024-09-10T21:30:37.143113+0000","last_became_peered":"2024-09-10T21:30:37.143113+0000","last_unstale":"2024-09-10T21:30:38.148326+0000","last_undegraded":"2024-09-10T21:30:38.148326+0000","last_fullsized":"2024-09-10T21:30:38.148326+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:11.873940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150735+0000","last_change":"2024-09-10T21:30:37.146196+0000","last_active":"2024-09-10T21:30:38.150735+0000","last_peered":"2024-09-10T21:30:38.150735+0000","last_clean":"2024-09-10T21:30:38.150735+0000","last_became_active":"2024-09-10T21:30:37.145480+0000","last_became_peered":"2024-09-10T21:30:37.145480+0000","last_unstale":"2024-09-10T21:30:38.150735+0000","last_undegraded":"2024-09-10T21:30:38.150735+0000","last_fullsized":"2024-09-10T21:30:38.150735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:13.493774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287182+0000","last_change":"2024-09-10T21:30:37.152682+0000","last_active":"2024-09-10T21:30:38.287182+0000","last_peered":"2024-09-10T21:30:38.287182+0000","last_clean":"2024-09-10T21:30:38.287182+0000","last_became_active":"2024-09-10T21:30:37.152308+0000","last_became_peered":"2024-09-10T21:30:37.152308+0000","last_unstale":"2024-09-10T21:30:38.287182+0000","last_undegraded":"2024-09-10T21:30:38.287182+0000","last_fullsized":"2024-09-10T21:30:38.287182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:39.533064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288396+0000","last_change":"2024-09-10T21:30:37.155784+0000","last_active":"2024-09-10T21:30:38.288396+0000","last_peered":"2024-09-10T21:30:38.288396+0000","last_clean":"2024-09-10T21:30:38.288396+0000","last_became_active":"2024-09-10T21:30:37.155514+0000","last_became_peered":"2024-09-10T21:30:37.155514+0000","last_unstale":"2024-09-10T21:30:38.288396+0000","last_undegraded":"2024-09-10T21:30:38.288396+0000","last_fullsized":"2024-09-10T21:30:38.288396+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:10.496991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151661+0000","last_change":"2024-09-10T21:30:37.157022+0000","last_active":"2024-09-10T21:30:38.151661+0000","last_peered":"2024-09-10T21:30:38.151661+0000","last_clean":"2024-09-10T21:30:38.151661+0000","last_became_active":"2024-09-10T21:30:37.155667+0000","last_became_peered":"2024-09-10T21:30:37.155667+0000","last_unstale":"2024-09-10T21:30:38.151661+0000","last_undegraded":"2024-09-10T21:30:38.151661+0000","last_fullsized":"2024-09-10T21:30:38.151661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:08.805186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286189+0000","last_change":"2024-09-10T21:30:37.158088+0000","last_active":"2024-09-10T21:30:38.286189+0000","last_peered":"2024-09-10T21:30:38.286189+0000","last_clean":"2024-09-10T21:30:38.286189+0000","last_became_active":"2024-09-10T21:30:37.157730+0000","last_became_peered":"2024-09-10T21:30:37.157730+0000","last_unstale":"2024-09-10T21:30:38.286189+0000","last_undegraded":"2024-09-10T21:30:38.286189+0000","last_fullsized":"2024-09-10T21:30:38.286189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:28.880581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287734+0000","last_change":"2024-09-10T21:30:37.161159+0000","last_active":"2024-09-10T21:30:38.287734+0000","last_peered":"2024-09-10T21:30:38.287734+0000","last_clean":"2024-09-10T21:30:38.287734+0000","last_became_active":"2024-09-10T21:30:37.160818+0000","last_became_peered":"2024-09-10T21:30:37.160818+0000","last_unstale":"2024-09-10T21:30:38.287734+0000","last_undegraded":"2024-09-10T21:30:38.287734+0000","last_fullsized":"2024-09-10T21:30:38.287734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:10.536055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154426+0000","last_change":"2024-09-10T21:30:37.153112+0000","last_active":"2024-09-10T21:30:38.154426+0000","last_peered":"2024-09-10T21:30:38.154426+0000","last_clean":"2024-09-10T21:30:38.154426+0000","last_became_active":"2024-09-10T21:30:37.152856+0000","last_became_peered":"2024-09-10T21:30:37.152856+0000","last_unstale":"2024-09-10T21:30:38.154426+0000","last_undegraded":"2024-09-10T21:30:38.154426+0000","last_fullsized":"2024-09-10T21:30:38.154426+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286004+0000","last_change":"2024-09-10T21:30:37.156498+0000","last_active":"2024-09-10T21:30:38.286004+0000","last_peered":"2024-09-10T21:30:38.286004+0000","last_clean":"2024-09-10T21:30:38.286004+0000","last_became_active":"2024-09-10T21:30:37.156217+0000","last_became_peered":"2024-09-10T21:30:37.156217+0000","last_unstale":"2024-09-10T21:30:38.286004+0000","last_undegraded":"2024-09-10T21:30:38.286004+0000","last_fullsized":"2024-09-10T21:30:38.286004+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:23.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154370+0000","last_change":"2024-09-10T21:30:37.160341+0000","last_active":"2024-09-10T21:30:38.154370+0000","last_peered":"2024-09-10T21:30:38.154370+0000","last_clean":"2024-09-10T21:30:38.154370+0000","last_became_active":"2024-09-10T21:30:37.160065+0000","last_became_peered":"2024-09-10T21:30:37.160065+0000","last_unstale":"2024-09-10T21:30:38.154370+0000","last_undegraded":"2024-09-10T21:30:38.154370+0000","last_fullsized":"2024-09-10T21:30:38.154370+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:52.394833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286884+0000","last_change":"2024-09-10T21:30:37.155875+0000","last_active":"2024-09-10T21:30:38.286884+0000","last_peered":"2024-09-10T21:30:38.286884+0000","last_clean":"2024-09-10T21:30:38.286884+0000","last_became_active":"2024-09-10T21:30:37.155291+0000","last_became_peered":"2024-09-10T21:30:37.155291+0000","last_unstale":"2024-09-10T21:30:38.286884+0000","last_undegraded":"2024-09-10T21:30:38.286884+0000","last_fullsized":"2024-09-10T21:30:38.286884+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:23.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154496+0000","last_change":"2024-09-10T21:30:37.152802+0000","last_active":"2024-09-10T21:30:38.154496+0000","last_peered":"2024-09-10T21:30:38.154496+0000","last_clean":"2024-09-10T21:30:38.154496+0000","last_became_active":"2024-09-10T21:30:37.152505+0000","last_became_peered":"2024-09-10T21:30:37.152505+0000","last_unstale":"2024-09-10T21:30:38.154496+0000","last_undegraded":"2024-09-10T21:30:38.154496+0000","last_fullsized":"2024-09-10T21:30:38.154496+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:57.443883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154923+0000","last_change":"2024-09-10T21:30:37.158604+0000","last_active":"2024-09-10T21:30:38.154923+0000","last_peered":"2024-09-10T21:30:38.154923+0000","last_clean":"2024-09-10T21:30:38.154923+0000","last_became_active":"2024-09-10T21:30:37.158029+0000","last_became_peered":"2024-09-10T21:30:37.158029+0000","last_unstale":"2024-09-10T21:30:38.154923+0000","last_undegraded":"2024-09-10T21:30:38.154923+0000","last_fullsized":"2024-09-10T21:30:38.154923+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:10.364464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152305+0000","last_change":"2024-09-10T21:30:37.155691+0000","last_active":"2024-09-10T21:30:38.152305+0000","last_peered":"2024-09-10T21:30:38.152305+0000","last_clean":"2024-09-10T21:30:38.152305+0000","last_became_active":"2024-09-10T21:30:37.155382+0000","last_became_peered":"2024-09-10T21:30:37.155382+0000","last_unstale":"2024-09-10T21:30:38.152305+0000","last_undegraded":"2024-09-10T21:30:38.152305+0000","last_fullsized":"2024-09-10T21:30:38.152305+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:27.386955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152312+0000","last_change":"2024-09-10T21:30:37.167891+0000","last_active":"2024-09-10T21:30:38.152312+0000","last_peered":"2024-09-10T21:30:38.152312+0000","last_clean":"2024-09-10T21:30:38.152312+0000","last_became_active":"2024-09-10T21:30:37.163511+0000","last_became_peered":"2024-09-10T21:30:37.163511+0000","last_unstale":"2024-09-10T21:30:38.152312+0000","last_undegraded":"2024-09-10T21:30:38.152312+0000","last_fullsized":"2024-09-10T21:30:38.152312+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:49.640342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154685+0000","last_change":"2024-09-10T21:30:37.160305+0000","last_active":"2024-09-10T21:30:38.154685+0000","last_peered":"2024-09-10T21:30:38.154685+0000","last_clean":"2024-09-10T21:30:38.154685+0000","last_became_active":"2024-09-10T21:30:37.160011+0000","last_became_peered":"2024-09-10T21:30:37.160011+0000","last_unstale":"2024-09-10T21:30:38.154685+0000","last_undegraded":"2024-09-10T21:30:38.154685+0000","last_fullsized":"2024-09-10T21:30:38.154685+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:28.880581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154345+0000","last_change":"2024-09-10T21:30:37.151678+0000","last_active":"2024-09-10T21:30:38.154345+0000","last_peered":"2024-09-10T21:30:38.154345+0000","last_clean":"2024-09-10T21:30:38.154345+0000","last_became_active":"2024-09-10T21:30:37.151090+0000","last_became_peered":"2024-09-10T21:30:37.151090+0000","last_unstale":"2024-09-10T21:30:38.154345+0000","last_undegraded":"2024-09-10T21:30:38.154345+0000","last_fullsized":"2024-09-10T21:30:38.154345+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:19.172370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.285885+0000","last_change":"2024-09-10T21:30:37.153740+0000","last_active":"2024-09-10T21:30:38.285885+0000","last_peered":"2024-09-10T21:30:38.285885+0000","last_clean":"2024-09-10T21:30:38.285885+0000","last_became_active":"2024-09-10T21:30:37.153498+0000","last_became_peered":"2024-09-10T21:30:37.153498+0000","last_unstale":"2024-09-10T21:30:38.285885+0000","last_undegraded":"2024-09-10T21:30:38.285885+0000","last_fullsized":"2024-09-10T21:30:38.285885+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:48.396677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287384+0000","last_change":"2024-09-10T21:30:37.142844+0000","last_active":"2024-09-10T21:30:38.287384+0000","last_peered":"2024-09-10T21:30:38.287384+0000","last_clean":"2024-09-10T21:30:38.287384+0000","last_became_active":"2024-09-10T21:30:37.142487+0000","last_became_peered":"2024-09-10T21:30:37.142487+0000","last_unstale":"2024-09-10T21:30:38.287384+0000","last_undegraded":"2024-09-10T21:30:38.287384+0000","last_fullsized":"2024-09-10T21:30:38.287384+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:49.728530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288237+0000","last_change":"2024-09-10T21:30:37.165261+0000","last_active":"2024-09-10T21:30:38.288237+0000","last_peered":"2024-09-10T21:30:38.288237+0000","last_clean":"2024-09-10T21:30:38.288237+0000","last_became_active":"2024-09-10T21:30:37.156273+0000","last_became_peered":"2024-09-10T21:30:37.156273+0000","last_unstale":"2024-09-10T21:30:38.288237+0000","last_undegraded":"2024-09-10T21:30:38.288237+0000","last_fullsized":"2024-09-10T21:30:38.288237+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:02.614328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.155022+0000","last_change":"2024-09-10T21:30:36.156733+0000","last_active":"2024-09-10T21:30:38.155022+0000","last_peered":"2024-09-10T21:30:38.155022+0000","last_clean":"2024-09-10T21:30:38.155022+0000","last_became_active":"2024-09-10T21:30:36.156360+0000","last_became_peered":"2024-09-10T21:30:36.156360+0000","last_unstale":"2024-09-10T21:30:38.155022+0000","last_undegraded":"2024-09-10T21:30:38.155022+0000","last_fullsized":"2024-09-10T21:30:38.155022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287825+0000","last_change":"2024-09-10T21:30:37.161240+0000","last_active":"2024-09-10T21:30:38.287825+0000","last_peered":"2024-09-10T21:30:38.287825+0000","last_clean":"2024-09-10T21:30:38.287825+0000","last_became_active":"2024-09-10T21:30:37.160927+0000","last_became_peered":"2024-09-10T21:30:37.160927+0000","last_unstale":"2024-09-10T21:30:38.287825+0000","last_undegraded":"2024-09-10T21:30:38.287825+0000","last_fullsized":"2024-09-10T21:30:38.287825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:57.275653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148672+0000","last_change":"2024-09-10T21:30:36.152982+0000","last_active":"2024-09-10T21:30:38.148672+0000","last_peered":"2024-09-10T21:30:38.148672+0000","last_clean":"2024-09-10T21:30:38.148672+0000","last_became_active":"2024-09-10T21:30:36.151294+0000","last_became_peered":"2024-09-10T21:30:36.151294+0000","last_unstale":"2024-09-10T21:30:38.148672+0000","last_undegraded":"2024-09-10T21:30:38.148672+0000","last_fullsized":"2024-09-10T21:30:38.148672+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:26.541443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152043+0000","last_change":"2024-09-10T21:30:37.161761+0000","last_active":"2024-09-10T21:30:38.152043+0000","last_peered":"2024-09-10T21:30:38.152043+0000","last_clean":"2024-09-10T21:30:38.152043+0000","last_became_active":"2024-09-10T21:30:37.161491+0000","last_became_peered":"2024-09-10T21:30:37.161491+0000","last_unstale":"2024-09-10T21:30:38.152043+0000","last_undegraded":"2024-09-10T21:30:38.152043+0000","last_fullsized":"2024-09-10T21:30:38.152043+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:08.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154849+0000","last_change":"2024-09-10T21:30:36.151783+0000","last_active":"2024-09-10T21:30:38.154849+0000","last_peered":"2024-09-10T21:30:38.154849+0000","last_clean":"2024-09-10T21:30:38.154849+0000","last_became_active":"2024-09-10T21:30:36.151113+0000","last_became_peered":"2024-09-10T21:30:36.151113+0000","last_unstale":"2024-09-10T21:30:38.154849+0000","last_undegraded":"2024-09-10T21:30:38.154849+0000","last_fullsized":"2024-09-10T21:30:38.154849+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:05.184547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151436+0000","last_change":"2024-09-10T21:30:37.156034+0000","last_active":"2024-09-10T21:30:38.151436+0000","last_peered":"2024-09-10T21:30:38.151436+0000","last_clean":"2024-09-10T21:30:38.151436+0000","last_became_active":"2024-09-10T21:30:37.155732+0000","last_became_peered":"2024-09-10T21:30:37.155732+0000","last_unstale":"2024-09-10T21:30:38.151436+0000","last_undegraded":"2024-09-10T21:30:38.151436+0000","last_fullsized":"2024-09-10T21:30:38.151436+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:06.192346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148525+0000","last_change":"2024-09-10T21:30:36.147985+0000","last_active":"2024-09-10T21:30:38.148525+0000","last_peered":"2024-09-10T21:30:38.148525+0000","last_clean":"2024-09-10T21:30:38.148525+0000","last_became_active":"2024-09-10T21:30:36.147685+0000","last_became_peered":"2024-09-10T21:30:36.147685+0000","last_unstale":"2024-09-10T21:30:38.148525+0000","last_undegraded":"2024-09-10T21:30:38.148525+0000","last_fullsized":"2024-09-10T21:30:38.148525+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:43.536977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150759+0000","last_change":"2024-09-10T21:30:37.157486+0000","last_active":"2024-09-10T21:30:38.150759+0000","last_peered":"2024-09-10T21:30:38.150759+0000","last_clean":"2024-09-10T21:30:38.150759+0000","last_became_active":"2024-09-10T21:30:37.157180+0000","last_became_peered":"2024-09-10T21:30:37.157180+0000","last_unstale":"2024-09-10T21:30:38.150759+0000","last_undegraded":"2024-09-10T21:30:38.150759+0000","last_fullsized":"2024-09-10T21:30:38.150759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:11.873940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.285998+0000","last_change":"2024-09-10T21:30:36.152597+0000","last_active":"2024-09-10T21:30:38.285998+0000","last_peered":"2024-09-10T21:30:38.285998+0000","last_clean":"2024-09-10T21:30:38.285998+0000","last_became_active":"2024-09-10T21:30:36.152336+0000","last_became_peered":"2024-09-10T21:30:36.152336+0000","last_unstale":"2024-09-10T21:30:38.285998+0000","last_undegraded":"2024-09-10T21:30:38.285998+0000","last_fullsized":"2024-09-10T21:30:38.285998+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:10.866141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151824+0000","last_change":"2024-09-10T21:30:37.157054+0000","last_active":"2024-09-10T21:30:38.151824+0000","last_peered":"2024-09-10T21:30:38.151824+0000","last_clean":"2024-09-10T21:30:38.151824+0000","last_became_active":"2024-09-10T21:30:37.156528+0000","last_became_peered":"2024-09-10T21:30:37.156528+0000","last_unstale":"2024-09-10T21:30:38.151824+0000","last_undegraded":"2024-09-10T21:30:38.151824+0000","last_fullsized":"2024-09-10T21:30:38.151824+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:55.564342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151415+0000","last_change":"2024-09-10T21:30:36.156510+0000","last_active":"2024-09-10T21:30:38.151415+0000","last_peered":"2024-09-10T21:30:38.151415+0000","last_clean":"2024-09-10T21:30:38.151415+0000","last_became_active":"2024-09-10T21:30:36.156197+0000","last_became_peered":"2024-09-10T21:30:36.156197+0000","last_unstale":"2024-09-10T21:30:38.151415+0000","last_undegraded":"2024-09-10T21:30:38.151415+0000","last_fullsized":"2024-09-10T21:30:38.151415+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287564+0000","last_change":"2024-09-10T21:30:37.143279+0000","last_active":"2024-09-10T21:30:38.287564+0000","last_peered":"2024-09-10T21:30:38.287564+0000","last_clean":"2024-09-10T21:30:38.287564+0000","last_became_active":"2024-09-10T21:30:37.142984+0000","last_became_peered":"2024-09-10T21:30:37.142984+0000","last_unstale":"2024-09-10T21:30:38.287564+0000","last_undegraded":"2024-09-10T21:30:38.287564+0000","last_fullsized":"2024-09-10T21:30:38.287564+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:12.128103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151074+0000","last_change":"2024-09-10T21:30:36.150338+0000","last_active":"2024-09-10T21:30:38.151074+0000","last_peered":"2024-09-10T21:30:38.151074+0000","last_clean":"2024-09-10T21:30:38.151074+0000","last_became_active":"2024-09-10T21:30:36.149500+0000","last_became_peered":"2024-09-10T21:30:36.149500+0000","last_unstale":"2024-09-10T21:30:38.151074+0000","last_undegraded":"2024-09-10T21:30:38.151074+0000","last_fullsized":"2024-09-10T21:30:38.151074+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:40.263132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288013+0000","last_change":"2024-09-10T21:30:37.156836+0000","last_active":"2024-09-10T21:30:38.288013+0000","last_peered":"2024-09-10T21:30:38.288013+0000","last_clean":"2024-09-10T21:30:38.288013+0000","last_became_active":"2024-09-10T21:30:37.156585+0000","last_became_peered":"2024-09-10T21:30:37.156585+0000","last_unstale":"2024-09-10T21:30:38.288013+0000","last_undegraded":"2024-09-10T21:30:38.288013+0000","last_fullsized":"2024-09-10T21:30:38.288013+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:28.880581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154183+0000","last_change":"2024-09-10T21:30:36.152065+0000","last_active":"2024-09-10T21:30:38.154183+0000","last_peered":"2024-09-10T21:30:38.154183+0000","last_clean":"2024-09-10T21:30:38.154183+0000","last_became_active":"2024-09-10T21:30:36.151761+0000","last_became_peered":"2024-09-10T21:30:36.151761+0000","last_unstale":"2024-09-10T21:30:38.154183+0000","last_undegraded":"2024-09-10T21:30:38.154183+0000","last_fullsized":"2024-09-10T21:30:38.154183+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:52.517766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154496+0000","last_change":"2024-09-10T21:30:37.161916+0000","last_active":"2024-09-10T21:30:38.154496+0000","last_peered":"2024-09-10T21:30:38.154496+0000","last_clean":"2024-09-10T21:30:38.154496+0000","last_became_active":"2024-09-10T21:30:37.161659+0000","last_became_peered":"2024-09-10T21:30:37.161659+0000","last_unstale":"2024-09-10T21:30:38.154496+0000","last_undegraded":"2024-09-10T21:30:38.154496+0000","last_fullsized":"2024-09-10T21:30:38.154496+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:23.718740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.155064+0000","last_change":"2024-09-10T21:30:36.152474+0000","last_active":"2024-09-10T21:30:38.155064+0000","last_peered":"2024-09-10T21:30:38.155064+0000","last_clean":"2024-09-10T21:30:38.155064+0000","last_became_active":"2024-09-10T21:30:36.152180+0000","last_became_peered":"2024-09-10T21:30:36.152180+0000","last_unstale":"2024-09-10T21:30:38.155064+0000","last_undegraded":"2024-09-10T21:30:38.155064+0000","last_fullsized":"2024-09-10T21:30:38.155064+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152034+0000","last_change":"2024-09-10T21:30:37.154771+0000","last_active":"2024-09-10T21:30:38.152034+0000","last_peered":"2024-09-10T21:30:38.152034+0000","last_clean":"2024-09-10T21:30:38.152034+0000","last_became_active":"2024-09-10T21:30:37.154432+0000","last_became_peered":"2024-09-10T21:30:37.154432+0000","last_unstale":"2024-09-10T21:30:38.152034+0000","last_undegraded":"2024-09-10T21:30:38.152034+0000","last_fullsized":"2024-09-10T21:30:38.152034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:04.406280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151914+0000","last_change":"2024-09-10T21:30:36.149165+0000","last_active":"2024-09-10T21:30:38.151914+0000","last_peered":"2024-09-10T21:30:38.151914+0000","last_clean":"2024-09-10T21:30:38.151914+0000","last_became_active":"2024-09-10T21:30:36.148849+0000","last_became_peered":"2024-09-10T21:30:36.148849+0000","last_unstale":"2024-09-10T21:30:38.151914+0000","last_undegraded":"2024-09-10T21:30:38.151914+0000","last_fullsized":"2024-09-10T21:30:38.151914+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:34.753321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287758+0000","last_change":"2024-09-10T21:30:37.160123+0000","last_active":"2024-09-10T21:30:38.287758+0000","last_peered":"2024-09-10T21:30:38.287758+0000","last_clean":"2024-09-10T21:30:38.287758+0000","last_became_active":"2024-09-10T21:30:37.159833+0000","last_became_peered":"2024-09-10T21:30:37.159833+0000","last_unstale":"2024-09-10T21:30:38.287758+0000","last_undegraded":"2024-09-10T21:30:38.287758+0000","last_fullsized":"2024-09-10T21:30:38.287758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:41.232502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152185+0000","last_change":"2024-09-10T21:30:36.149184+0000","last_active":"2024-09-10T21:30:38.152185+0000","last_peered":"2024-09-10T21:30:38.152185+0000","last_clean":"2024-09-10T21:30:38.152185+0000","last_became_active":"2024-09-10T21:30:36.148915+0000","last_became_peered":"2024-09-10T21:30:36.148915+0000","last_unstale":"2024-09-10T21:30:38.152185+0000","last_undegraded":"2024-09-10T21:30:38.152185+0000","last_fullsized":"2024-09-10T21:30:38.152185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:04.537601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288504+0000","last_change":"2024-09-10T21:30:37.147434+0000","last_active":"2024-09-10T21:30:38.288504+0000","last_peered":"2024-09-10T21:30:38.288504+0000","last_clean":"2024-09-10T21:30:38.288504+0000","last_became_active":"2024-09-10T21:30:37.147160+0000","last_became_peered":"2024-09-10T21:30:37.147160+0000","last_unstale":"2024-09-10T21:30:38.288504+0000","last_undegraded":"2024-09-10T21:30:38.288504+0000","last_fullsized":"2024-09-10T21:30:38.288504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:07.679232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288314+0000","last_change":"2024-09-10T21:30:36.158339+0000","last_active":"2024-09-10T21:30:38.288314+0000","last_peered":"2024-09-10T21:30:38.288314+0000","last_clean":"2024-09-10T21:30:38.288314+0000","last_became_active":"2024-09-10T21:30:36.158050+0000","last_became_peered":"2024-09-10T21:30:36.158050+0000","last_unstale":"2024-09-10T21:30:38.288314+0000","last_undegraded":"2024-09-10T21:30:38.288314+0000","last_fullsized":"2024-09-10T21:30:38.288314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:39.986638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286957+0000","last_change":"2024-09-10T21:30:37.155599+0000","last_active":"2024-09-10T21:30:38.286957+0000","last_peered":"2024-09-10T21:30:38.286957+0000","last_clean":"2024-09-10T21:30:38.286957+0000","last_became_active":"2024-09-10T21:30:37.154937+0000","last_became_peered":"2024-09-10T21:30:37.154937+0000","last_unstale":"2024-09-10T21:30:38.286957+0000","last_undegraded":"2024-09-10T21:30:38.286957+0000","last_fullsized":"2024-09-10T21:30:38.286957+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:48.396677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154829+0000","last_change":"2024-09-10T21:30:36.155028+0000","last_active":"2024-09-10T21:30:38.154829+0000","last_peered":"2024-09-10T21:30:38.154829+0000","last_clean":"2024-09-10T21:30:38.154829+0000","last_became_active":"2024-09-10T21:30:36.154105+0000","last_became_peered":"2024-09-10T21:30:36.154105+0000","last_unstale":"2024-09-10T21:30:38.154829+0000","last_undegraded":"2024-09-10T21:30:38.154829+0000","last_fullsized":"2024-09-10T21:30:38.154829+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:23.894646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154240+0000","last_change":"2024-09-10T21:30:37.147903+0000","last_active":"2024-09-10T21:30:38.154240+0000","last_peered":"2024-09-10T21:30:38.154240+0000","last_clean":"2024-09-10T21:30:38.154240+0000","last_became_active":"2024-09-10T21:30:37.147347+0000","last_became_peered":"2024-09-10T21:30:37.147347+0000","last_unstale":"2024-09-10T21:30:38.154240+0000","last_undegraded":"2024-09-10T21:30:38.154240+0000","last_fullsized":"2024-09-10T21:30:38.154240+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:47.060669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288548+0000","last_change":"2024-09-10T21:30:36.149196+0000","last_active":"2024-09-10T21:30:38.288548+0000","last_peered":"2024-09-10T21:30:38.288548+0000","last_clean":"2024-09-10T21:30:38.288548+0000","last_became_active":"2024-09-10T21:30:36.148907+0000","last_became_peered":"2024-09-10T21:30:36.148907+0000","last_unstale":"2024-09-10T21:30:38.288548+0000","last_undegraded":"2024-09-10T21:30:38.288548+0000","last_fullsized":"2024-09-10T21:30:38.288548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:28.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288482+0000","last_change":"2024-09-10T21:30:37.160607+0000","last_active":"2024-09-10T21:30:38.288482+0000","last_peered":"2024-09-10T21:30:38.288482+0000","last_clean":"2024-09-10T21:30:38.288482+0000","last_became_active":"2024-09-10T21:30:37.159663+0000","last_became_peered":"2024-09-10T21:30:37.159663+0000","last_unstale":"2024-09-10T21:30:38.288482+0000","last_undegraded":"2024-09-10T21:30:38.288482+0000","last_fullsized":"2024-09-10T21:30:38.288482+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:55.977517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287012+0000","last_change":"2024-09-10T21:30:36.152984+0000","last_active":"2024-09-10T21:30:38.287012+0000","last_peered":"2024-09-10T21:30:38.287012+0000","last_clean":"2024-09-10T21:30:38.287012+0000","last_became_active":"2024-09-10T21:30:36.152704+0000","last_became_peered":"2024-09-10T21:30:36.152704+0000","last_unstale":"2024-09-10T21:30:38.287012+0000","last_undegraded":"2024-09-10T21:30:38.287012+0000","last_fullsized":"2024-09-10T21:30:38.287012+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:43.536977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151820+0000","last_change":"2024-09-10T21:30:37.163498+0000","last_active":"2024-09-10T21:30:38.151820+0000","last_peered":"2024-09-10T21:30:38.151820+0000","last_clean":"2024-09-10T21:30:38.151820+0000","last_became_active":"2024-09-10T21:30:37.163181+0000","last_became_peered":"2024-09-10T21:30:37.163181+0000","last_unstale":"2024-09-10T21:30:38.151820+0000","last_undegraded":"2024-09-10T21:30:38.151820+0000","last_fullsized":"2024-09-10T21:30:38.151820+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:10.882901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150470+0000","last_change":"2024-09-10T21:30:36.150376+0000","last_active":"2024-09-10T21:30:38.150470+0000","last_peered":"2024-09-10T21:30:38.150470+0000","last_clean":"2024-09-10T21:30:38.150470+0000","last_became_active":"2024-09-10T21:30:36.150090+0000","last_became_peered":"2024-09-10T21:30:36.150090+0000","last_unstale":"2024-09-10T21:30:38.150470+0000","last_undegraded":"2024-09-10T21:30:38.150470+0000","last_fullsized":"2024-09-10T21:30:38.150470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:49.705316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151920+0000","last_change":"2024-09-10T21:30:37.166907+0000","last_active":"2024-09-10T21:30:38.151920+0000","last_peered":"2024-09-10T21:30:38.151920+0000","last_clean":"2024-09-10T21:30:38.151920+0000","last_became_active":"2024-09-10T21:30:37.164943+0000","last_became_peered":"2024-09-10T21:30:37.164943+0000","last_unstale":"2024-09-10T21:30:38.151920+0000","last_undegraded":"2024-09-10T21:30:38.151920+0000","last_fullsized":"2024-09-10T21:30:38.151920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:28.954938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286112+0000","last_change":"2024-09-10T21:30:36.153213+0000","last_active":"2024-09-10T21:30:38.286112+0000","last_peered":"2024-09-10T21:30:38.286112+0000","last_clean":"2024-09-10T21:30:38.286112+0000","last_became_active":"2024-09-10T21:30:36.152918+0000","last_became_peered":"2024-09-10T21:30:36.152918+0000","last_unstale":"2024-09-10T21:30:38.286112+0000","last_undegraded":"2024-09-10T21:30:38.286112+0000","last_fullsized":"2024-09-10T21:30:38.286112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:09.875102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152202+0000","last_change":"2024-09-10T21:30:37.147332+0000","last_active":"2024-09-10T21:30:38.152202+0000","last_peered":"2024-09-10T21:30:38.152202+0000","last_clean":"2024-09-10T21:30:38.152202+0000","last_became_active":"2024-09-10T21:30:37.146975+0000","last_became_peered":"2024-09-10T21:30:37.146975+0000","last_unstale":"2024-09-10T21:30:38.152202+0000","last_undegraded":"2024-09-10T21:30:38.152202+0000","last_fullsized":"2024-09-10T21:30:38.152202+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:38.142189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148744+0000","last_change":"2024-09-10T21:30:36.155934+0000","last_active":"2024-09-10T21:30:38.148744+0000","last_peered":"2024-09-10T21:30:38.148744+0000","last_clean":"2024-09-10T21:30:38.148744+0000","last_became_active":"2024-09-10T21:30:36.155669+0000","last_became_peered":"2024-09-10T21:30:36.155669+0000","last_unstale":"2024-09-10T21:30:38.148744+0000","last_undegraded":"2024-09-10T21:30:38.148744+0000","last_fullsized":"2024-09-10T21:30:38.148744+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:56.357501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154965+0000","last_change":"2024-09-10T21:30:37.158812+0000","last_active":"2024-09-10T21:30:38.154965+0000","last_peered":"2024-09-10T21:30:38.154965+0000","last_clean":"2024-09-10T21:30:38.154965+0000","last_became_active":"2024-09-10T21:30:37.158370+0000","last_became_peered":"2024-09-10T21:30:37.158370+0000","last_unstale":"2024-09-10T21:30:38.154965+0000","last_undegraded":"2024-09-10T21:30:38.154965+0000","last_fullsized":"2024-09-10T21:30:38.154965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:05.545400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148656+0000","last_change":"2024-09-10T21:30:36.152792+0000","last_active":"2024-09-10T21:30:38.148656+0000","last_peered":"2024-09-10T21:30:38.148656+0000","last_clean":"2024-09-10T21:30:38.148656+0000","last_became_active":"2024-09-10T21:30:36.152526+0000","last_became_peered":"2024-09-10T21:30:36.152526+0000","last_unstale":"2024-09-10T21:30:38.148656+0000","last_undegraded":"2024-09-10T21:30:38.148656+0000","last_fullsized":"2024-09-10T21:30:38.148656+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:43.266507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287996+0000","last_change":"2024-09-10T21:30:37.163752+0000","last_active":"2024-09-10T21:30:38.287996+0000","last_peered":"2024-09-10T21:30:38.287996+0000","last_clean":"2024-09-10T21:30:38.287996+0000","last_became_active":"2024-09-10T21:30:37.162710+0000","last_became_peered":"2024-09-10T21:30:37.162710+0000","last_unstale":"2024-09-10T21:30:38.287996+0000","last_undegraded":"2024-09-10T21:30:38.287996+0000","last_fullsized":"2024-09-10T21:30:38.287996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:33.852578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151847+0000","last_change":"2024-09-10T21:30:36.153508+0000","last_active":"2024-09-10T21:30:38.151847+0000","last_peered":"2024-09-10T21:30:38.151847+0000","last_clean":"2024-09-10T21:30:38.151847+0000","last_became_active":"2024-09-10T21:30:36.153255+0000","last_became_peered":"2024-09-10T21:30:36.153255+0000","last_unstale":"2024-09-10T21:30:38.151847+0000","last_undegraded":"2024-09-10T21:30:38.151847+0000","last_fullsized":"2024-09-10T21:30:38.151847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:30.450914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148273+0000","last_change":"2024-09-10T21:30:37.146511+0000","last_active":"2024-09-10T21:30:38.148273+0000","last_peered":"2024-09-10T21:30:38.148273+0000","last_clean":"2024-09-10T21:30:38.148273+0000","last_became_active":"2024-09-10T21:30:37.143002+0000","last_became_peered":"2024-09-10T21:30:37.143002+0000","last_unstale":"2024-09-10T21:30:38.148273+0000","last_undegraded":"2024-09-10T21:30:38.148273+0000","last_fullsized":"2024-09-10T21:30:38.148273+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:14.904422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287713+0000","last_change":"2024-09-10T21:30:36.153764+0000","last_active":"2024-09-10T21:30:38.287713+0000","last_peered":"2024-09-10T21:30:38.287713+0000","last_clean":"2024-09-10T21:30:38.287713+0000","last_became_active":"2024-09-10T21:30:36.153453+0000","last_became_peered":"2024-09-10T21:30:36.153453+0000","last_unstale":"2024-09-10T21:30:38.287713+0000","last_undegraded":"2024-09-10T21:30:38.287713+0000","last_fullsized":"2024-09-10T21:30:38.287713+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:28.260842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286121+0000","last_change":"2024-09-10T21:30:37.160469+0000","last_active":"2024-09-10T21:30:38.286121+0000","last_peered":"2024-09-10T21:30:38.286121+0000","last_clean":"2024-09-10T21:30:38.286121+0000","last_became_active":"2024-09-10T21:30:37.159860+0000","last_became_peered":"2024-09-10T21:30:37.159860+0000","last_unstale":"2024-09-10T21:30:38.286121+0000","last_undegraded":"2024-09-10T21:30:38.286121+0000","last_fullsized":"2024-09-10T21:30:38.286121+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:47.060669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.148223+0000","last_change":"2024-09-10T21:30:36.145884+0000","last_active":"2024-09-10T21:30:38.148223+0000","last_peered":"2024-09-10T21:30:38.148223+0000","last_clean":"2024-09-10T21:30:38.148223+0000","last_became_active":"2024-09-10T21:30:36.145592+0000","last_became_peered":"2024-09-10T21:30:36.145592+0000","last_unstale":"2024-09-10T21:30:38.148223+0000","last_undegraded":"2024-09-10T21:30:38.148223+0000","last_fullsized":"2024-09-10T21:30:38.148223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:36.544620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288227+0000","last_change":"2024-09-10T21:30:37.164650+0000","last_active":"2024-09-10T21:30:38.288227+0000","last_peered":"2024-09-10T21:30:38.288227+0000","last_clean":"2024-09-10T21:30:38.288227+0000","last_became_active":"2024-09-10T21:30:37.164223+0000","last_became_peered":"2024-09-10T21:30:37.164223+0000","last_unstale":"2024-09-10T21:30:38.288227+0000","last_undegraded":"2024-09-10T21:30:38.288227+0000","last_fullsized":"2024-09-10T21:30:38.288227+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:07.663482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.151192+0000","last_change":"2024-09-10T21:30:36.153170+0000","last_active":"2024-09-10T21:30:38.151192+0000","last_peered":"2024-09-10T21:30:38.151192+0000","last_clean":"2024-09-10T21:30:38.151192+0000","last_became_active":"2024-09-10T21:30:36.152223+0000","last_became_peered":"2024-09-10T21:30:36.152223+0000","last_unstale":"2024-09-10T21:30:38.151192+0000","last_undegraded":"2024-09-10T21:30:38.151192+0000","last_fullsized":"2024-09-10T21:30:38.151192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:38.190823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286078+0000","last_change":"2024-09-10T21:30:37.156348+0000","last_active":"2024-09-10T21:30:38.286078+0000","last_peered":"2024-09-10T21:30:38.286078+0000","last_clean":"2024-09-10T21:30:38.286078+0000","last_became_active":"2024-09-10T21:30:37.156039+0000","last_became_peered":"2024-09-10T21:30:37.156039+0000","last_unstale":"2024-09-10T21:30:38.286078+0000","last_undegraded":"2024-09-10T21:30:38.286078+0000","last_fullsized":"2024-09-10T21:30:38.286078+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:41.232502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152262+0000","last_change":"2024-09-10T21:30:36.149909+0000","last_active":"2024-09-10T21:30:38.152262+0000","last_peered":"2024-09-10T21:30:38.152262+0000","last_clean":"2024-09-10T21:30:38.152262+0000","last_became_active":"2024-09-10T21:30:36.149596+0000","last_became_peered":"2024-09-10T21:30:36.149596+0000","last_unstale":"2024-09-10T21:30:38.152262+0000","last_undegraded":"2024-09-10T21:30:38.152262+0000","last_fullsized":"2024-09-10T21:30:38.152262+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:09.356665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152558+0000","last_change":"2024-09-10T21:30:37.158088+0000","last_active":"2024-09-10T21:30:38.152558+0000","last_peered":"2024-09-10T21:30:38.152558+0000","last_clean":"2024-09-10T21:30:38.152558+0000","last_became_active":"2024-09-10T21:30:37.157676+0000","last_became_peered":"2024-09-10T21:30:37.157676+0000","last_unstale":"2024-09-10T21:30:38.152558+0000","last_undegraded":"2024-09-10T21:30:38.152558+0000","last_fullsized":"2024-09-10T21:30:38.152558+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:19.172370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"94'1","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152097+0000","last_change":"2024-09-10T21:30:36.151988+0000","last_active":"2024-09-10T21:30:38.152097+0000","last_peered":"2024-09-10T21:30:38.152097+0000","last_clean":"2024-09-10T21:30:38.152097+0000","last_became_active":"2024-09-10T21:30:36.151626+0000","last_became_peered":"2024-09-10T21:30:36.151626+0000","last_unstale":"2024-09-10T21:30:38.152097+0000","last_undegraded":"2024-09-10T21:30:38.152097+0000","last_fullsized":"2024-09-10T21:30:38.152097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:53.273442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152467+0000","last_change":"2024-09-10T21:30:37.166893+0000","last_active":"2024-09-10T21:30:38.152467+0000","last_peered":"2024-09-10T21:30:38.152467+0000","last_clean":"2024-09-10T21:30:38.152467+0000","last_became_active":"2024-09-10T21:30:37.166457+0000","last_became_peered":"2024-09-10T21:30:37.166457+0000","last_unstale":"2024-09-10T21:30:38.152467+0000","last_undegraded":"2024-09-10T21:30:38.152467+0000","last_fullsized":"2024-09-10T21:30:38.152467+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:03.101308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286142+0000","last_change":"2024-09-10T21:30:36.156216+0000","last_active":"2024-09-10T21:30:38.286142+0000","last_peered":"2024-09-10T21:30:38.286142+0000","last_clean":"2024-09-10T21:30:38.286142+0000","last_became_active":"2024-09-10T21:30:36.155962+0000","last_became_peered":"2024-09-10T21:30:36.155962+0000","last_unstale":"2024-09-10T21:30:38.286142+0000","last_undegraded":"2024-09-10T21:30:38.286142+0000","last_fullsized":"2024-09-10T21:30:38.286142+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:06.307063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154748+0000","last_change":"2024-09-10T21:30:37.156724+0000","last_active":"2024-09-10T21:30:38.154748+0000","last_peered":"2024-09-10T21:30:38.154748+0000","last_clean":"2024-09-10T21:30:38.154748+0000","last_became_active":"2024-09-10T21:30:37.156047+0000","last_became_peered":"2024-09-10T21:30:37.156047+0000","last_unstale":"2024-09-10T21:30:38.154748+0000","last_undegraded":"2024-09-10T21:30:38.154748+0000","last_fullsized":"2024-09-10T21:30:38.154748+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:02.614328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.152110+0000","last_change":"2024-09-10T21:30:36.146558+0000","last_active":"2024-09-10T21:30:38.152110+0000","last_peered":"2024-09-10T21:30:38.152110+0000","last_clean":"2024-09-10T21:30:38.152110+0000","last_became_active":"2024-09-10T21:30:36.144327+0000","last_became_peered":"2024-09-10T21:30:36.144327+0000","last_unstale":"2024-09-10T21:30:38.152110+0000","last_undegraded":"2024-09-10T21:30:38.152110+0000","last_fullsized":"2024-09-10T21:30:38.152110+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:54.969718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287947+0000","last_change":"2024-09-10T21:30:37.162963+0000","last_active":"2024-09-10T21:30:38.287947+0000","last_peered":"2024-09-10T21:30:38.287947+0000","last_clean":"2024-09-10T21:30:38.287947+0000","last_became_active":"2024-09-10T21:30:37.162366+0000","last_became_peered":"2024-09-10T21:30:37.162366+0000","last_unstale":"2024-09-10T21:30:38.287947+0000","last_undegraded":"2024-09-10T21:30:38.287947+0000","last_fullsized":"2024-09-10T21:30:38.287947+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:37.376166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288447+0000","last_change":"2024-09-10T21:30:36.159592+0000","last_active":"2024-09-10T21:30:38.288447+0000","last_peered":"2024-09-10T21:30:38.288447+0000","last_clean":"2024-09-10T21:30:38.288447+0000","last_became_active":"2024-09-10T21:30:36.159354+0000","last_became_peered":"2024-09-10T21:30:36.159354+0000","last_unstale":"2024-09-10T21:30:38.288447+0000","last_undegraded":"2024-09-10T21:30:38.288447+0000","last_fullsized":"2024-09-10T21:30:38.288447+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:10.866141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.150731+0000","last_change":"2024-09-10T21:30:37.158517+0000","last_active":"2024-09-10T21:30:38.150731+0000","last_peered":"2024-09-10T21:30:38.150731+0000","last_clean":"2024-09-10T21:30:38.150731+0000","last_became_active":"2024-09-10T21:30:37.153692+0000","last_became_peered":"2024-09-10T21:30:37.153692+0000","last_unstale":"2024-09-10T21:30:38.150731+0000","last_undegraded":"2024-09-10T21:30:38.150731+0000","last_fullsized":"2024-09-10T21:30:38.150731+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:32.366799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154881+0000","last_change":"2024-09-10T21:30:36.157140+0000","last_active":"2024-09-10T21:30:38.154881+0000","last_peered":"2024-09-10T21:30:38.154881+0000","last_clean":"2024-09-10T21:30:38.154881+0000","last_became_active":"2024-09-10T21:30:36.156861+0000","last_became_peered":"2024-09-10T21:30:36.156861+0000","last_unstale":"2024-09-10T21:30:38.154881+0000","last_undegraded":"2024-09-10T21:30:38.154881+0000","last_fullsized":"2024-09-10T21:30:38.154881+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:10.541056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.286243+0000","last_change":"2024-09-10T21:30:37.160667+0000","last_active":"2024-09-10T21:30:38.286243+0000","last_peered":"2024-09-10T21:30:38.286243+0000","last_clean":"2024-09-10T21:30:38.286243+0000","last_became_active":"2024-09-10T21:30:37.160187+0000","last_became_peered":"2024-09-10T21:30:37.160187+0000","last_unstale":"2024-09-10T21:30:38.286243+0000","last_undegraded":"2024-09-10T21:30:38.286243+0000","last_fullsized":"2024-09-10T21:30:38.286243+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:02.614328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154526+0000","last_change":"2024-09-10T21:30:36.147814+0000","last_active":"2024-09-10T21:30:38.154526+0000","last_peered":"2024-09-10T21:30:38.154526+0000","last_clean":"2024-09-10T21:30:38.154526+0000","last_became_active":"2024-09-10T21:30:36.147488+0000","last_became_peered":"2024-09-10T21:30:36.147488+0000","last_unstale":"2024-09-10T21:30:38.154526+0000","last_undegraded":"2024-09-10T21:30:38.154526+0000","last_fullsized":"2024-09-10T21:30:38.154526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:31.359000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.287689+0000","last_change":"2024-09-10T21:30:37.164739+0000","last_active":"2024-09-10T21:30:38.287689+0000","last_peered":"2024-09-10T21:30:38.287689+0000","last_clean":"2024-09-10T21:30:38.287689+0000","last_became_active":"2024-09-10T21:30:37.164473+0000","last_became_peered":"2024-09-10T21:30:37.164473+0000","last_unstale":"2024-09-10T21:30:38.287689+0000","last_undegraded":"2024-09-10T21:30:38.287689+0000","last_fullsized":"2024-09-10T21:30:38.287689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:36.726703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154255+0000","last_change":"2024-09-10T21:30:36.156585+0000","last_active":"2024-09-10T21:30:38.154255+0000","last_peered":"2024-09-10T21:30:38.154255+0000","last_clean":"2024-09-10T21:30:38.154255+0000","last_became_active":"2024-09-10T21:30:36.156288+0000","last_became_peered":"2024-09-10T21:30:36.156288+0000","last_unstale":"2024-09-10T21:30:38.154255+0000","last_undegraded":"2024-09-10T21:30:38.154255+0000","last_fullsized":"2024-09-10T21:30:38.154255+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:44.644836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154765+0000","last_change":"2024-09-10T21:30:37.156412+0000","last_active":"2024-09-10T21:30:38.154765+0000","last_peered":"2024-09-10T21:30:38.154765+0000","last_clean":"2024-09-10T21:30:38.154765+0000","last_became_active":"2024-09-10T21:30:37.155638+0000","last_became_peered":"2024-09-10T21:30:37.155638+0000","last_unstale":"2024-09-10T21:30:38.154765+0000","last_undegraded":"2024-09-10T21:30:38.154765+0000","last_fullsized":"2024-09-10T21:30:38.154765+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:36.121501+0000","last_clean_scrub_stamp":"2024-09-10T21:30:36.121501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:38.366747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.288471+0000","last_change":"2024-09-10T21:30:36.151215+0000","last_active":"2024-09-10T21:30:38.288471+0000","last_peered":"2024-09-10T21:30:38.288471+0000","last_clean":"2024-09-10T21:30:38.288471+0000","last_became_active":"2024-09-10T21:30:36.150699+0000","last_became_peered":"2024-09-10T21:30:36.150699+0000","last_unstale":"2024-09-10T21:30:38.288471+0000","last_undegraded":"2024-09-10T21:30:38.288471+0000","last_fullsized":"2024-09-10T21:30:38.288471+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:30:35.113702+0000","last_clean_scrub_stamp":"2024-09-10T21:30:35.113702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:13.896623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":248,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T21:30:38.154288+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:30:38.154288+0000","last_peered":"2024-09-10T21:30:38.154288+0000","last_clean":"2024-09-10T21:30:38.154288+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:30:38.154288+0000","last_undegraded":"2024-09-10T21:30:38.154288+0000","last_fullsized":"2024-09-10T21:30:38.154288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1150976,"data_stored":532534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1171456,"data_stored":534270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1744896,"data_stored":1122902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1171456,"data_stored":534258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509523,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1159168,"data_stored":533138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509524,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1757184,"data_stored":1123588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1155072,"data_stored":532604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1150976,"data_stored":532534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:30:46.263 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:30:46.263 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:30:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 -- 172.21.15.97:0/2624061114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e41511f0 msgr2=0x7f25e40b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 --2- 172.21.15.97:0/2624061114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e41511f0 0x7f25e40b37b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009c80 tx=0x7f25d401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 -- 172.21.15.97:0/2624061114 shutdown_connections 2024-09-10T21:30:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 --2- 172.21.15.97:0/2624061114 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25e40b3cf0 0x7f25e406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 --2- 172.21.15.97:0/2624061114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e41511f0 0x7f25e40b37b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 -- 172.21.15.97:0/2624061114 >> 172.21.15.97:0/2624061114 conn(0x7f25e40b1780 msgr2=0x7f25e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 -- 172.21.15.97:0/2624061114 shutdown_connections 2024-09-10T21:30:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.448+0000 7f25ea400640 1 -- 172.21.15.97:0/2624061114 wait complete. 2024-09-10T21:30:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 Processor -- start 2024-09-10T21:30:46.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 -- start start 2024-09-10T21:30:46.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e406f180 0x7f25e414e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e40b3cf0 0x7f25e40cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25e41511f0 0x7f25e4159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f25e406e740 con 0x7f25e406f180 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f25e406e5c0 con 0x7f25e40b3cf0 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f25e406e8c0 con 0x7f25e41511f0 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e406f180 0x7f25e414e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e406f180 0x7f25e414e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48746/0 (socket says 172.21.15.97:48746) 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 -- 172.21.15.97:0/411821376 learned_addr learned my addr 172.21.15.97:0/411821376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3e00640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25e41511f0 0x7f25e4159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25daa00640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e40b3cf0 0x7f25e40cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 -- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25e41511f0 msgr2=0x7f25e4159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25e41511f0 0x7f25e4159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 -- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e40b3cf0 msgr2=0x7f25e40cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e40b3cf0 0x7f25e40cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 -- 172.21.15.97:0/411821376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e40cbf10 con 0x7f25e406f180 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25e3400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e406f180 0x7f25e414e7a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000f90 tx=0x7f25d40af410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25dbe00640 1 -- 172.21.15.97:0/411821376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004e60 con 0x7f25e406f180 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25dbe00640 1 -- 172.21.15.97:0/411821376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25d40be070 con 0x7f25e406f180 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e40cdf70 con 0x7f25e406f180 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.452+0000 7f25dbe00640 1 -- 172.21.15.97:0/411821376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d40afaa0 con 0x7f25e406f180 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.456+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e40c4b70 con 0x7f25e406f180 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.456+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e40b37b0 con 0x7f25e406f180 2024-09-10T21:30:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.456+0000 7f25dbe00640 1 -- 172.21.15.97:0/411821376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f25d40afc40 con 0x7f25e406f180 2024-09-10T21:30:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.456+0000 7f25dbe00640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c00725e0 0x7f25c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.456+0000 7f25dbe00640 1 -- 172.21.15.97:0/411821376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f25d4132280 con 0x7f25e406f180 2024-09-10T21:30:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.456+0000 7f25daa00640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c00725e0 0x7f25c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.456+0000 7f25daa00640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c00725e0 0x7f25c0074aa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f25e4150c60 tx=0x7f25cc00b490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.460+0000 7f25dbe00640 1 -- 172.21.15.97:0/411821376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f25d40ac050 con 0x7f25e406f180 2024-09-10T21:30:46.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.684+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f25e415bfa0 con 0x7f25e406f180 2024-09-10T21:30:46.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25dbe00640 1 -- 172.21.15.97:0/411821376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f25d4019100 con 0x7f25e406f180 2024-09-10T21:30:46.691 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-10T21:30:46.692 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:30:46.692 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-09-10T21:30:46.692 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-09-10T21:30:46.692 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:30:46.692 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-09-10T21:30:46.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c00725e0 msgr2=0x7f25c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c00725e0 0x7f25c0074aa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f25e4150c60 tx=0x7f25cc00b490 comp rx=0 tx=0).stop 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e406f180 msgr2=0x7f25e414e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e406f180 0x7f25e414e7a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000f90 tx=0x7f25d40af410 comp rx=0 tx=0).stop 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 shutdown_connections 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f25c00725e0 0x7f25c0074aa0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f25e41511f0 0x7f25e4159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e40b3cf0 0x7f25e40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 --2- 172.21.15.97:0/411821376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25e406f180 0x7f25e414e7a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 >> 172.21.15.97:0/411821376 conn(0x7f25e40b1780 msgr2=0x7f25e40cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 shutdown_connections 2024-09-10T21:30:46.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.688+0000 7f25ea400640 1 -- 172.21.15.97:0/411821376 wait complete. 2024-09-10T21:30:46.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:46.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 -- 172.21.15.97:0/2600206881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec144690 msgr2=0x7f51ec0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 --2- 172.21.15.97:0/2600206881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec144690 0x7f51ec0b2a80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009840 tx=0x7f51d801c820 comp rx=0 tx=0).stop 2024-09-10T21:30:46.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 -- 172.21.15.97:0/2600206881 shutdown_connections 2024-09-10T21:30:46.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 --2- 172.21.15.97:0/2600206881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec144690 0x7f51ec0b2a80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 -- 172.21.15.97:0/2600206881 >> 172.21.15.97:0/2600206881 conn(0x7f51ec0b1780 msgr2=0x7f51ec0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:46.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 -- 172.21.15.97:0/2600206881 shutdown_connections 2024-09-10T21:30:46.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 -- 172.21.15.97:0/2600206881 wait complete. 2024-09-10T21:30:46.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 Processor -- start 2024-09-10T21:30:46.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 -- start start 2024-09-10T21:30:46.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51ec06fd50 0x7f51ec134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec143f90 0x7f51ec13d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.852+0000 7f51f1e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51ec144690 0x7f51ec13c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51f1e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec169410 con 0x7f51ec143f90 2024-09-10T21:30:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51f1e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec169290 con 0x7f51ec06fd50 2024-09-10T21:30:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51f1e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec169590 con 0x7f51ec144690 2024-09-10T21:30:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec143f90 0x7f51ec13d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec143f90 0x7f51ec13d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48762/0 (socket says 172.21.15.97:48762) 2024-09-10T21:30:46.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 -- 172.21.15.97:0/1060598404 learned_addr learned my addr 172.21.15.97:0/1060598404 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:46.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51eb400640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51ec144690 0x7f51ec13c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51eaa00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51ec06fd50 0x7f51ec134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 -- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51ec144690 msgr2=0x7f51ec13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51ec144690 0x7f51ec13c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 -- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51ec06fd50 msgr2=0x7f51ec134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51ec06fd50 0x7f51ec134170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 -- 172.21.15.97:0/1060598404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec1f5d80 con 0x7f51ec143f90 2024-09-10T21:30:46.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51eb400640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51ec144690 0x7f51ec13c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:46.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51eaa00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51ec06fd50 0x7f51ec134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:46.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51ea000640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec143f90 0x7f51ec13d320 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009790 tx=0x7f51d80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51e3400640 1 -- 172.21.15.97:0/1060598404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004370 con 0x7f51ec143f90 2024-09-10T21:30:46.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51e3400640 1 -- 172.21.15.97:0/1060598404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51d8005070 con 0x7f51ec143f90 2024-09-10T21:30:46.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec1f6040 con 0x7f51ec143f90 2024-09-10T21:30:46.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51e3400640 1 -- 172.21.15.97:0/1060598404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8005460 con 0x7f51ec143f90 2024-09-10T21:30:46.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec1f6600 con 0x7f51ec143f90 2024-09-10T21:30:46.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec0b2a80 con 0x7f51ec143f90 2024-09-10T21:30:46.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.856+0000 7f51e3400640 1 -- 172.21.15.97:0/1060598404 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f51d80046d0 con 0x7f51ec143f90 2024-09-10T21:30:46.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.860+0000 7f51e3400640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51cc072610 0x7f51cc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.860+0000 7f51e3400640 1 -- 172.21.15.97:0/1060598404 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f51d812dd40 con 0x7f51ec143f90 2024-09-10T21:30:46.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.860+0000 7f51eaa00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51cc072610 0x7f51cc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.860+0000 7f51eaa00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51cc072610 0x7f51cc074ad0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0045c0 tx=0x7f51dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:46.864+0000 7f51e3400640 1 -- 172.21.15.97:0/1060598404 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f51d80c5020 con 0x7f51ec143f90 2024-09-10T21:30:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.060+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51ec136030 con 0x7f51ec143f90 2024-09-10T21:30:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.060+0000 7f51e3400640 1 -- 172.21.15.97:0/1060598404 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5225 (secure 0 0 0) 0x7f51d8016410 con 0x7f51ec143f90 2024-09-10T21:30:47.064 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:47.064 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"btime":"2024-09-10T21:30:40:152516+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:30:39.538691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/873879253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":873879253},{"type":"v1","addr":"172.21.15.97:6835","nonce":873879253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6957,"qdb_cluster":[6957]},"id":6}]} 2024-09-10T21:30:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 85 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51cc072610 msgr2=0x7f51cc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51cc072610 0x7f51cc074ad0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0045c0 tx=0x7f51dc009040 comp rx=0 tx=0).stop 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec143f90 msgr2=0x7f51ec13d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec143f90 0x7f51ec13d320 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009790 tx=0x7f51d80040c0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 shutdown_connections 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51cc072610 0x7f51cc074ad0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51ec144690 0x7f51ec13c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec143f90 0x7f51ec13d320 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 --2- 172.21.15.97:0/1060598404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51ec06fd50 0x7f51ec134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 >> 172.21.15.97:0/1060598404 conn(0x7f51ec0b1780 msgr2=0x7f51ec0730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 shutdown_connections 2024-09-10T21:30:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.064+0000 7f51f1e00640 1 -- 172.21.15.97:0/1060598404 wait complete. 2024-09-10T21:30:47.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:30:39.538691+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6957}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6957': {'gid': 6957, 'name': 'c', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6835/873879253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 873879253}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 873879253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6957, 'qdb_cluster': [6957]} 2024-09-10T21:30:47.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:30:47.489 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:30:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.628+0000 7f292e600640 1 -- 172.21.15.97:0/946160147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 msgr2=0x7f29281667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.628+0000 7f292e600640 1 --2- 172.21.15.97:0/946160147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281667a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2920009bc0 tx=0x7f292001cba0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.628+0000 7f292e600640 1 -- 172.21.15.97:0/946160147 shutdown_connections 2024-09-10T21:30:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.628+0000 7f292e600640 1 --2- 172.21.15.97:0/946160147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281667a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.628+0000 7f292e600640 1 -- 172.21.15.97:0/946160147 >> 172.21.15.97:0/946160147 conn(0x7f29280d1a60 msgr2=0x7f29280d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.628+0000 7f292e600640 1 -- 172.21.15.97:0/946160147 shutdown_connections 2024-09-10T21:30:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.628+0000 7f292e600640 1 -- 172.21.15.97:0/946160147 wait complete. 2024-09-10T21:30:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 Processor -- start 2024-09-10T21:30:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 -- start start 2024-09-10T21:30:47.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281666e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29280cdcb0 0x7f29281561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f292815efc0 0x7f292815b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2928169aa0 con 0x7f29280af940 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2928169920 con 0x7f292815efc0 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2928169c20 con 0x7f29280cdcb0 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281666e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60136/0 (socket says 172.21.15.97:60136) 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 -- 172.21.15.97:0/452741871 learned_addr learned my addr 172.21.15.97:0/452741871 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f291fe00640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29280cdcb0 0x7f29281561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292dc00640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f292815efc0 0x7f292815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 -- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29280cdcb0 msgr2=0x7f29281561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29280cdcb0 0x7f29281561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 -- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f292815efc0 msgr2=0x7f292815b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f292815efc0 0x7f292815b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 -- 172.21.15.97:0/452741871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292815c3c0 con 0x7f29280af940 2024-09-10T21:30:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292d200640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281666e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f291800c3f0 tx=0x7f291800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f2925600640 1 -- 172.21.15.97:0/452741871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918017070 con 0x7f29280af940 2024-09-10T21:30:47.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f2925600640 1 -- 172.21.15.97:0/452741871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29180053c0 con 0x7f29280af940 2024-09-10T21:30:47.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f2925600640 1 -- 172.21.15.97:0/452741871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918003860 con 0x7f29280af940 2024-09-10T21:30:47.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292815c6a0 con 0x7f29280af940 2024-09-10T21:30:47.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.632+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292815dc90 con 0x7f29280af940 2024-09-10T21:30:47.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.636+0000 7f2925600640 1 -- 172.21.15.97:0/452741871 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2918003a00 con 0x7f29280af940 2024-09-10T21:30:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.636+0000 7f2925600640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2904072610 0x7f2904074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.636+0000 7f2925600640 1 -- 172.21.15.97:0/452741871 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f2918092a30 con 0x7f29280af940 2024-09-10T21:30:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.636+0000 7f2933075000 1 Processor -- start 2024-09-10T21:30:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.636+0000 7f291fe00640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2904072610 0x7f2904074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.636+0000 7f2933075000 1 -- start start 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2904072610 0x7f2904074ad0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2928072450 tx=0x7f2910009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d813ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d80f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d80f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646d80123f0 con 0x5646d813e2f0 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5646d7f4fb20 con 0x5646d813d9d0 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5646d7f3ef50 con 0x5646d80f1e10 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d80f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d80f18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60142/0 (socket says 172.21.15.97:60142) 2024-09-10T21:30:47.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 -- 172.21.15.97:0/563848708 learned_addr learned my addr 172.21.15.97:0/563848708 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f292d200640 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d813ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f292dc00640 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d80f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 -- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 msgr2=0x5646d80f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d80f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 -- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 msgr2=0x5646d813ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d813ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 -- 172.21.15.97:0/563848708 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d7f06590 con 0x5646d813e2f0 2024-09-10T21:30:47.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291fe00640 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d80f18d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2910001580 tx=0x7f2910011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a8e0e43bfee09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291d600640 1 -- 172.21.15.97:0/563848708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910025070 con 0x5646d813e2f0 2024-09-10T21:30:47.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291d600640 1 -- 172.21.15.97:0/563848708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29100159b0 con 0x5646d813e2f0 2024-09-10T21:30:47.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f291d600640 1 -- 172.21.15.97:0/563848708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910015d60 con 0x5646d813e2f0 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- 172.21.15.97:0/563848708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 msgr2=0x5646d80f18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d80f18d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2910001580 tx=0x7f2910011000 comp rx=0 tx=0).stop 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- 172.21.15.97:0/563848708 shutdown_connections 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d80f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d80f18d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 --2- 172.21.15.97:0/563848708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d813ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- 172.21.15.97:0/563848708 >> 172.21.15.97:0/563848708 conn(0x5646d7fdb130 msgr2=0x5646d8120e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- 172.21.15.97:0/563848708 shutdown_connections 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- 172.21.15.97:0/563848708 wait complete. 2024-09-10T21:30:47.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 -- start start 2024-09-10T21:30:47.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.640+0000 7f2933075000 1 Processor -- start 2024-09-10T21:30:47.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d811e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d811ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d811f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646d7ed95d0 con 0x5646d813e2f0 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5646d7f42d60 con 0x5646d813d9d0 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5646d7ee5500 con 0x5646d80f1e10 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 10 client.0 _ll_register_callbacks cb 0x5646d7ee9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d811f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d811f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60156/0 (socket says 172.21.15.97:60156) 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 -- 172.21.15.97:0/1931804561 learned_addr learned my addr 172.21.15.97:0/1931804561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292d200640 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d811e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f291fe00640 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d811ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 -- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 msgr2=0x5646d811e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d811e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 -- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 msgr2=0x5646d811ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d811ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 -- 172.21.15.97:0/1931804561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d7edd170 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292dc00640 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d811f3b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2920009980 tx=0x7f2920004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b354a36a262fd0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2916000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2916000640 1 -- 172.21.15.97:0/1931804561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200be060 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646d7ed70f0 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2916000640 1 -- 172.21.15.97:0/1931804561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f2920003980 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2916000640 1 -- 172.21.15.97:0/1931804561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f29200c4050 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 10 client.7122 fetch_fsmap learned FSMap version 85 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5646d7fc3ef0 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2916000640 1 -- 172.21.15.97:0/1931804561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5646d7ed70f0 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2916000640 1 -- 172.21.15.97:0/1931804561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f29200c7020 con 0x5646d813e2f0 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 10 client.7122 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 10 client.7122 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:30:47.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 20 client.7122 populate_metadata read hostname 'smithi097' 2024-09-10T21:30:47.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8120120 0x5646d814fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 4 client.7122 mds_command: new command op to 6957 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:30:47.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 --> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646d7f51a70 con 0x5646d8120120 2024-09-10T21:30:47.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292d200640 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8120120 0x5646d814fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f2916000640 10 client.7122 ms_handle_connect on v2:172.21.15.97:6833/873879253 2024-09-10T21:30:47.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.644+0000 7f292d200640 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8120120 0x5646d814fc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527890a788e0e21c server_cookie=4aacb93ef8ba1c68 in_seq=0 out_seq=0 2024-09-10T21:30:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2916000640 1 -- 172.21.15.97:0/1931804561 <== mds.0 v2:172.21.15.97:6833/873879253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646d7f51a70 con 0x5646d8120120 2024-09-10T21:30:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2916000640 10 client.7122 handle_command_reply: tid=0 2024-09-10T21:30:47.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 client.7122 shutdown 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 msgr2=0x5646d811f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d811f3b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2920009980 tx=0x7f2920004c80 comp rx=0 tx=0).stop 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 shutdown_connections 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8120120 0x5646d814fc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d811f3b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813d9d0 0x5646d811ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 --2- 172.21.15.97:0/1931804561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d80f1e10 0x5646d811e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2916000640 0 client.7122 ms_handle_reset on v2:172.21.15.97:6833/873879253 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 >> 172.21.15.97:0/1931804561 conn(0x5646d7fdb130 msgr2=0x5646d80f0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 shutdown_connections 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 1 -- 172.21.15.97:0/1931804561 wait complete. 2024-09-10T21:30:47.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.648+0000 7f2933075000 20 client.7122 trim_cache size 0 max 16384 2024-09-10T21:30:47.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.664+0000 7f2933075000 1 Processor -- start 2024-09-10T21:30:47.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.664+0000 7f2933075000 1 -- start start 2024-09-10T21:30:47.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d80f1e10 0x5646d81afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d81b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2928204270 con 0x5646d813e2f0 2024-09-10T21:30:47.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f29282032d0 con 0x5646d80f1e10 2024-09-10T21:30:47.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2928165f80 con 0x5646d813d9d0 2024-09-10T21:30:47.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d81b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d81b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60160/0 (socket says 172.21.15.97:60160) 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 -- 172.21.15.97:0/2622139974 learned_addr learned my addr 172.21.15.97:0/2622139974 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f291fe00640 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292d200640 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d80f1e10 0x5646d81afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 -- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 msgr2=0x5646d81b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 -- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d80f1e10 msgr2=0x5646d81afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d80f1e10 0x5646d81afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 -- 172.21.15.97:0/2622139974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646d7f9ad90 con 0x5646d813e2f0 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f292dc00640 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d81b0640 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2920009b10 tx=0x7f2920004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0251121e27f224b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f291cc00640 1 -- 172.21.15.97:0/2622139974 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200be070 con 0x5646d813e2f0 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f291cc00640 1 -- 172.21.15.97:0/2622139974 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2920003a70 con 0x5646d813e2f0 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f291cc00640 1 -- 172.21.15.97:0/2622139974 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200b2040 con 0x5646d813e2f0 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- 172.21.15.97:0/2622139974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 msgr2=0x5646d81b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d81b0640 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2920009b10 tx=0x7f2920004a50 comp rx=0 tx=0).stop 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- 172.21.15.97:0/2622139974 shutdown_connections 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d813e2f0 0x5646d81b0640 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 --2- 172.21.15.97:0/2622139974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d80f1e10 0x5646d81afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- 172.21.15.97:0/2622139974 >> 172.21.15.97:0/2622139974 conn(0x5646d7fdb130 msgr2=0x5646d814afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- 172.21.15.97:0/2622139974 shutdown_connections 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- 172.21.15.97:0/2622139974 wait complete. 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 -- start start 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.668+0000 7f2933075000 1 Processor -- start 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d80f1e10 0x5646d81b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813e2f0 0x5646d81b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646d7f4e8f0 con 0x5646d80f1e10 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5646d7f42070 con 0x5646d813e2f0 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5646d7ffe730 con 0x5646d813d9d0 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 10 client.0 _ll_register_callbacks cb 0x7f2920009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d80f1e10 0x5646d81b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d80f1e10 0x5646d81b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60174/0 (socket says 172.21.15.97:60174) 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 -- 172.21.15.97:0/1381698283 learned_addr learned my addr 172.21.15.97:0/1381698283 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f291fe00640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292dc00640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813e2f0 0x5646d81b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 -- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 msgr2=0x5646d81b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 -- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813e2f0 msgr2=0x5646d81b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813e2f0 0x5646d81b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 -- 172.21.15.97:0/1381698283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646d7f05a90 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292dc00640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813e2f0 0x5646d81b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292d200640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d80f1e10 0x5646d81b37e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2918090d30 tx=0x7f2918000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2072b5fa1472be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 1 -- 172.21.15.97:0/1381698283 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29180622c0 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 1 -- 172.21.15.97:0/1381698283 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f291805ca40 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2920003a70 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 1 -- 172.21.15.97:0/1381698283 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f2918066ce0 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 10 client.7128 fetch_fsmap learned FSMap version 85 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5646d7ee4050 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 1 -- 172.21.15.97:0/1381698283 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2920003a70 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 1 -- 172.21.15.97:0/1381698283 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f2918082ba0 con 0x5646d80f1e10 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 10 client.7128 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:30:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 10 client.7128 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:30:47.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 20 client.7128 populate_metadata read hostname 'smithi097' 2024-09-10T21:30:47.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8151b80 0x5646d8151f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 4 client.7128 mds_command: new command op to 6957 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:30:47.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 --> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] -- command(tid 0: {"prefix": "status"}) -- 0x5646d7f4d520 con 0x5646d8151b80 2024-09-10T21:30:47.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f291fe00640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8151b80 0x5646d8151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f291fe00640 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8151b80 0x5646d8151f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d1bb351f6df015 server_cookie=9ec466cce5630ca8 in_seq=0 out_seq=0 2024-09-10T21:30:47.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 10 client.7128 ms_handle_connect on v2:172.21.15.97:6833/873879253 2024-09-10T21:30:47.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 1 -- 172.21.15.97:0/1381698283 <== mds.0 v2:172.21.15.97:6833/873879253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5646d7f4d520 con 0x5646d8151b80 2024-09-10T21:30:47.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 10 client.7128 handle_command_reply: tid=0 2024-09-10T21:30:47.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 client.7128 shutdown 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d80f1e10 msgr2=0x5646d81b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d80f1e10 0x5646d81b37e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2918090d30 tx=0x7f2918000bd0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 shutdown_connections 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:6833/873879253,v1:172.21.15.97:6835/873879253] conn(0x5646d8151b80 0x5646d8151f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5646d813e2f0 0x5646d81b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5646d813d9d0 0x5646d81b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 --2- 172.21.15.97:0/1381698283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646d80f1e10 0x5646d81b37e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2916000640 0 client.7128 ms_handle_reset on v2:172.21.15.97:6833/873879253 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 >> 172.21.15.97:0/1381698283 conn(0x5646d7fdb130 msgr2=0x5646d80f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 shutdown_connections 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 1 -- 172.21.15.97:0/1381698283 wait complete. 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f2933075000 20 client.7128 trim_cache size 0 max 16384 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6957, 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:30:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 10.537345139999999, 2024-09-10T21:30:47.684 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 85, 2024-09-10T21:30:47.684 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 94, 2024-09-10T21:30:47.684 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T21:30:47.684 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 15.072600451 2024-09-10T21:30:47.684 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:30:47.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2904072610 msgr2=0x7f2904074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292e600640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2904072610 0x7f2904074ad0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2928072450 tx=0x7f2910009040 comp rx=0 tx=0).stop 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 msgr2=0x7f29281666e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.672+0000 7f292e600640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281666e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f291800c3f0 tx=0x7f291800c8b0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 shutdown_connections 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2904072610 0x7f2904074ad0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f292815efc0 0x7f292815b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29280cdcb0 0x7f29281561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 --2- 172.21.15.97:0/452741871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29280af940 0x7f29281666e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 >> 172.21.15.97:0/452741871 conn(0x7f29280d1a60 msgr2=0x7f29280b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 shutdown_connections 2024-09-10T21:30:47.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.676+0000 7f292e600640 1 -- 172.21.15.97:0/452741871 wait complete. 2024-09-10T21:30:47.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- 172.21.15.97:0/2562766770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64144690 msgr2=0x7f5c640b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 --2- 172.21.15.97:0/2562766770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64144690 0x7f5c640b2a80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009840 tx=0x7f5c5001c820 comp rx=0 tx=0).stop 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- 172.21.15.97:0/2562766770 shutdown_connections 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 --2- 172.21.15.97:0/2562766770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c6406fd50 0x7f5c640701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 --2- 172.21.15.97:0/2562766770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64144690 0x7f5c640b2a80 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- 172.21.15.97:0/2562766770 >> 172.21.15.97:0/2562766770 conn(0x7f5c640b1780 msgr2=0x7f5c640b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- 172.21.15.97:0/2562766770 shutdown_connections 2024-09-10T21:30:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- 172.21.15.97:0/2562766770 wait complete. 2024-09-10T21:30:47.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 Processor -- start 2024-09-10T21:30:47.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- start start 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c6406fd50 0x7f5c641f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64143f90 0x7f5c6415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c64144690 0x7f5c641fb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64167c50 con 0x7f5c64143f90 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64167ad0 con 0x7f5c64144690 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.836+0000 7f5c6be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5c64167dd0 con 0x7f5c6406fd50 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64143f90 0x7f5c6415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64143f90 0x7f5c6415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60196/0 (socket says 172.21.15.97:60196) 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 -- 172.21.15.97:0/3628729065 learned_addr learned my addr 172.21.15.97:0/3628729065 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c69e00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c64144690 0x7f5c641fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c69400640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c6406fd50 0x7f5c641f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 -- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c6406fd50 msgr2=0x7f5c641f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c6406fd50 0x7f5c641f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 -- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c64144690 msgr2=0x7f5c641fb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c64144690 0x7f5c641fb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 -- 172.21.15.97:0/3628729065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c641fbd50 con 0x7f5c64143f90 2024-09-10T21:30:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c68a00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64143f90 0x7f5c6415a1c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009790 tx=0x7f5c500a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c61600640 1 -- 172.21.15.97:0/3628729065 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500be070 con 0x7f5c64143f90 2024-09-10T21:30:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c61600640 1 -- 172.21.15.97:0/3628729065 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c50004470 con 0x7f5c64143f90 2024-09-10T21:30:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c61600640 1 -- 172.21.15.97:0/3628729065 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500052d0 con 0x7f5c64143f90 2024-09-10T21:30:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c641f7ea0 con 0x7f5c64143f90 2024-09-10T21:30:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c641f8460 con 0x7f5c64143f90 2024-09-10T21:30:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c640cade0 con 0x7f5c64143f90 2024-09-10T21:30:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.840+0000 7f5c61600640 1 -- 172.21.15.97:0/3628729065 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5c50005470 con 0x7f5c64143f90 2024-09-10T21:30:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.844+0000 7f5c61600640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c440725e0 0x7f5c44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.844+0000 7f5c61600640 1 -- 172.21.15.97:0/3628729065 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f5c5012d940 con 0x7f5c64143f90 2024-09-10T21:30:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.844+0000 7f5c69400640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c440725e0 0x7f5c44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.844+0000 7f5c69400640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c440725e0 0x7f5c44074aa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5c58001690 tx=0x7f5c58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:47.844+0000 7f5c61600640 1 -- 172.21.15.97:0/3628729065 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c641fbd50 con 0x7f5c64143f90 2024-09-10T21:30:48.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.044+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5c640701e0 con 0x7f5c64143f90 2024-09-10T21:30:48.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c61600640 1 -- 172.21.15.97:0/3628729065 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7f5c50016410 con 0x7f5c64143f90 2024-09-10T21:30:48.050 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c440725e0 msgr2=0x7f5c44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c440725e0 0x7f5c44074aa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5c58001690 tx=0x7f5c58009040 comp rx=0 tx=0).stop 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64143f90 msgr2=0x7f5c6415a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64143f90 0x7f5c6415a1c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009790 tx=0x7f5c500a66f0 comp rx=0 tx=0).stop 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 shutdown_connections 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c440725e0 0x7f5c44074aa0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c64144690 0x7f5c641fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64143f90 0x7f5c6415a1c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:48.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 --2- 172.21.15.97:0/3628729065 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c6406fd50 0x7f5c641f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:48.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 >> 172.21.15.97:0/3628729065 conn(0x7f5c640b1780 msgr2=0x7f5c640b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:48.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 shutdown_connections 2024-09-10T21:30:48.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48.048+0000 7f5c6be00640 1 -- 172.21.15.97:0/3628729065 wait complete. 2024-09-10T21:30:48.070 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:30:48.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:30:48.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:30:48.070 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:30:48.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:30:48.070 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:30:48.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:30:48.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:30:48.071 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:30:48.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.071 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:30:48.080 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:30:48.081 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42045sec preferred_lft 42045sec 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:30:48.082 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:30:48.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.084 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:30:48.144 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:30:48.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.146 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:30:48.202 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:30:48.203 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:30:48.203 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:30:48.203 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:30:48.203 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:30:48.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:30:48.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:30:48.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:30:48.203 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:30:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:30:48.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:30:48.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.339 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:30:48.349 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:30:48.349 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:30:48.349 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:30:48.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:30:48.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:30:48.413 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:30:48.465 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:30:48.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:30:48.466 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:30:48.517 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:30:48.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:30:48.610 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:30:48.610 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:30:48.610 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:30:48.626 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:30:48.626 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:30:48.626 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:30:48.626 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:30:48.626 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:30:48.635 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:30:48.635 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:30:48.635 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:30:48.635 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:30:48.635 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:30:48.635 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:30:48.635 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:30:48.636 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:30:48.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:30:48.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:30:48.660 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:30:48.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:30:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:30:48.764 INFO:teuthology.orchestra.run.smithi097.stdout:client6176 2024-09-10T21:30:48.764 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:48.764 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:48.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.774 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-09-10T21:30:48.832 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:48.832 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:48.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:48.881 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-10T21:30:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:6+0 records in 2024-09-10T21:30:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:6+0 records out 2024-09-10T21:30:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.100206 s, 62.8 MB/s 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> import json 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(e.errno) 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> print(json.dumps( 2024-09-10T21:30:49.036 DEBUG:teuthology.orchestra.run.smithi097:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T21:30:49.037 DEBUG:teuthology.orchestra.run.smithi097:> indent=2)) 2024-09-10T21:30:49.037 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:30:49.105 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:30:49.105 INFO:teuthology.orchestra.run.smithi097.stdout: "st_mode": 33204, 2024-09-10T21:30:49.105 INFO:teuthology.orchestra.run.smithi097.stdout: "st_ino": 1099511627777, 2024-09-10T21:30:49.105 INFO:teuthology.orchestra.run.smithi097.stdout: "st_dev": 44, 2024-09-10T21:30:49.105 INFO:teuthology.orchestra.run.smithi097.stdout: "st_nlink": 1, 2024-09-10T21:30:49.105 INFO:teuthology.orchestra.run.smithi097.stdout: "st_uid": 1000, 2024-09-10T21:30:49.106 INFO:teuthology.orchestra.run.smithi097.stdout: "st_gid": 1271, 2024-09-10T21:30:49.106 INFO:teuthology.orchestra.run.smithi097.stdout: "st_size": 6291456, 2024-09-10T21:30:49.106 INFO:teuthology.orchestra.run.smithi097.stdout: "st_atime": 1726003848.9287338, 2024-09-10T21:30:49.106 INFO:teuthology.orchestra.run.smithi097.stdout: "st_mtime": 1726003848.9647334, 2024-09-10T21:30:49.106 INFO:teuthology.orchestra.run.smithi097.stdout: "st_ctime": 1726003848.9647334 2024-09-10T21:30:49.106 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:30:49.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:30:49.114 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:49.114 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:49.170 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:49.170 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:49.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:30:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 -- 172.21.15.97:0/3191838690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0af300 msgr2=0x7fca4c0af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 --2- 172.21.15.97:0/3191838690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0af300 0x7fca4c0af7b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fca4c073590 tx=0x7fca4401cd80 comp rx=0 tx=0).stop 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 -- 172.21.15.97:0/3191838690 shutdown_connections 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 --2- 172.21.15.97:0/3191838690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0af300 0x7fca4c0af7b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 --2- 172.21.15.97:0/3191838690 >> v2:172.21.15.194:3300/0 conn(0x7fca4c0b0d50 0x7fca4c0aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 --2- 172.21.15.97:0/3191838690 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fca4c0ce640 0x7fca4c0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 -- 172.21.15.97:0/3191838690 >> 172.21.15.97:0/3191838690 conn(0x7fca4c0d1c30 msgr2=0x7fca4c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 -- 172.21.15.97:0/3191838690 shutdown_connections 2024-09-10T21:30:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 -- 172.21.15.97:0/3191838690 wait complete. 2024-09-10T21:30:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.404+0000 7fca53400640 1 Processor -- start 2024-09-10T21:30:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 -- start start 2024-09-10T21:30:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca4c0af300 0x7fca4c0b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fca4c0b0d50 0x7fca4c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0ce640 0x7fca4c154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c0cd2f0 con 0x7fca4c0ce640 2024-09-10T21:30:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c0cd170 con 0x7fca4c0af300 2024-09-10T21:30:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c0cd470 con 0x7fca4c0b0d50 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0ce640 0x7fca4c154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0ce640 0x7fca4c154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60212/0 (socket says 172.21.15.97:60212) 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 -- 172.21.15.97:0/1041973700 learned_addr learned my addr 172.21.15.97:0/1041973700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca4be00640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fca4c0b0d50 0x7fca4c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca50a00640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca4c0af300 0x7fca4c0b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 -- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fca4c0b0d50 msgr2=0x7fca4c14f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fca4c0b0d50 0x7fca4c14f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 -- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca4c0af300 msgr2=0x7fca4c0b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca4c0af300 0x7fca4c0b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 -- 172.21.15.97:0/1041973700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c0b7170 con 0x7fca4c0ce640 2024-09-10T21:30:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca51400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0ce640 0x7fca4c154c70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fca4401b430 tx=0x7fca44003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:49.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca48c00640 1 -- 172.21.15.97:0/1041973700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca440bb070 con 0x7fca4c0ce640 2024-09-10T21:30:49.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca48c00640 1 -- 172.21.15.97:0/1041973700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca44004d40 con 0x7fca4c0ce640 2024-09-10T21:30:49.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca48c00640 1 -- 172.21.15.97:0/1041973700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca440b2040 con 0x7fca4c0ce640 2024-09-10T21:30:49.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c0b7440 con 0x7fca4c0ce640 2024-09-10T21:30:49.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c0c7fc0 con 0x7fca4c0ce640 2024-09-10T21:30:49.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.408+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c0cd970 con 0x7fca4c0ce640 2024-09-10T21:30:49.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.412+0000 7fca48c00640 1 -- 172.21.15.97:0/1041973700 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fca440b5030 con 0x7fca4c0ce640 2024-09-10T21:30:49.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.412+0000 7fca48c00640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fca2c0725e0 0x7fca2c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.412+0000 7fca48c00640 1 -- 172.21.15.97:0/1041973700 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7fca4412ccd0 con 0x7fca4c0ce640 2024-09-10T21:30:49.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.412+0000 7fca50a00640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fca2c0725e0 0x7fca2c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.412+0000 7fca50a00640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fca2c0725e0 0x7fca2c074aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0b7ba0 tx=0x7fca400091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:49.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.416+0000 7fca48c00640 1 -- 172.21.15.97:0/1041973700 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fca440f1de0 con 0x7fca4c0ce640 2024-09-10T21:30:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fca4c075f90 con 0x7fca4c0ce640 2024-09-10T21:30:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca48c00640 1 -- 172.21.15.97:0/1041973700 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v94) ==== 80+0+0 (secure 0 0 0) 0x7fca44016970 con 0x7fca4c0ce640 2024-09-10T21:30:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fca2c0725e0 msgr2=0x7fca2c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fca2c0725e0 0x7fca2c074aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0b7ba0 tx=0x7fca400091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0ce640 msgr2=0x7fca4c154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0ce640 0x7fca4c154c70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fca4401b430 tx=0x7fca44003f60 comp rx=0 tx=0).stop 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 shutdown_connections 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fca2c0725e0 0x7fca2c074aa0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca4c0ce640 0x7fca4c154c70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fca4c0b0d50 0x7fca4c14f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 --2- 172.21.15.97:0/1041973700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca4c0af300 0x7fca4c0b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 >> 172.21.15.97:0/1041973700 conn(0x7fca4c0d1c30 msgr2=0x7fca4c14fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 shutdown_connections 2024-09-10T21:30:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.592+0000 7fca53400640 1 -- 172.21.15.97:0/1041973700 wait complete. 2024-09-10T21:30:49.616 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:30:49.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:49.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:30:49.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:30:49.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:49.676 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:30:49.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- 172.21.15.97:0/4260832229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9464003a90 msgr2=0x7f9464003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 --2- 172.21.15.97:0/4260832229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9464003a90 0x7f9464003eb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f947000c640 tx=0x7f947001f9c0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947e800640 1 -- 172.21.15.97:0/4260832229 reap_dead start 2024-09-10T21:30:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- 172.21.15.97:0/4260832229 shutdown_connections 2024-09-10T21:30:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- 172.21.15.97:0/4260832229 >> 172.21.15.97:0/4260832229 conn(0x7f948006cb40 msgr2=0x7f948006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- 172.21.15.97:0/4260832229 shutdown_connections 2024-09-10T21:30:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- 172.21.15.97:0/4260832229 wait complete. 2024-09-10T21:30:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 Processor -- start 2024-09-10T21:30:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- start start 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464003a90 0x7f9480157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f948015be40 0x7f948015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9480143d80 0x7f9480144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9480146dd0 con 0x7f9480143d80 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9480146c50 con 0x7f9464003a90 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.868+0000 7f947f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9480146f50 con 0x7f948015be40 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9480143d80 0x7f9480144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464003a90 0x7f9480157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9480143d80 0x7f9480144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60230/0 (socket says 172.21.15.97:60230) 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 -- 172.21.15.97:0/1848179331 learned_addr learned my addr 172.21.15.97:0/1848179331 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947d400640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f948015be40 0x7f948015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 -- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f948015be40 msgr2=0x7f948015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f948015be40 0x7f948015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 -- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464003a90 msgr2=0x7f9480157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464003a90 0x7f9480157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 -- 172.21.15.97:0/1848179331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801f5380 con 0x7f9480143d80 2024-09-10T21:30:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947e800640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9480143d80 0x7f9480144230 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9480162610 tx=0x7f9478011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f9476000640 1 -- 172.21.15.97:0/1848179331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478004230 con 0x7f9480143d80 2024-09-10T21:30:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f9476000640 1 -- 172.21.15.97:0/1848179331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9478016070 con 0x7f9480143d80 2024-09-10T21:30:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9480157ac0 con 0x7f9480143d80 2024-09-10T21:30:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f9476000640 1 -- 172.21.15.97:0/1848179331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947800faf0 con 0x7f9480143d80 2024-09-10T21:30:49.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801f5900 con 0x7f9480143d80 2024-09-10T21:30:49.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94801474e0 con 0x7f9480143d80 2024-09-10T21:30:49.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f9476000640 1 -- 172.21.15.97:0/1848179331 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9478023030 con 0x7f9480143d80 2024-09-10T21:30:49.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.872+0000 7f9476000640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9460072610 0x7f9460074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:49.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.876+0000 7f9476000640 1 -- 172.21.15.97:0/1848179331 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f9478093b60 con 0x7f9480143d80 2024-09-10T21:30:49.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.876+0000 7f947de00640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9460072610 0x7f9460074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:49.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.876+0000 7f947de00640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9460072610 0x7f9460074ad0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f94700af000 tx=0x7f947001c800 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:49.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:49.876+0000 7f9476000640 1 -- 172.21.15.97:0/1848179331 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f947805e9c0 con 0x7f9480143d80 2024-09-10T21:30:50.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f948006fb90 con 0x7f9480143d80 2024-09-10T21:30:50.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f9476000640 1 -- 172.21.15.97:0/1848179331 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5225 (secure 0 0 0) 0x7f94780633f0 con 0x7f9480143d80 2024-09-10T21:30:50.078 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:50.078 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"btime":"2024-09-10T21:30:40:152516+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:30:39.538691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/873879253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":873879253},{"type":"v1","addr":"172.21.15.97:6835","nonce":873879253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6957,"qdb_cluster":[6957]},"id":6}]} 2024-09-10T21:30:50.079 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 85 2024-09-10T21:30:50.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9460072610 msgr2=0x7f9460074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9460072610 0x7f9460074ad0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f94700af000 tx=0x7f947001c800 comp rx=0 tx=0).stop 2024-09-10T21:30:50.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9480143d80 msgr2=0x7f9480144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9480143d80 0x7f9480144230 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9480162610 tx=0x7f9478011b10 comp rx=0 tx=0).stop 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 shutdown_connections 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9460072610 0x7f9460074ad0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9480143d80 0x7f9480144230 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f948015be40 0x7f948015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 --2- 172.21.15.97:0/1848179331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9464003a90 0x7f9480157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 >> 172.21.15.97:0/1848179331 conn(0x7f948006cb40 msgr2=0x7f94800b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 shutdown_connections 2024-09-10T21:30:50.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.076+0000 7f947f200640 1 -- 172.21.15.97:0/1848179331 wait complete. 2024-09-10T21:30:50.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-10T21:30:50.269 INFO:teuthology.orchestra.run.smithi097.stdout:{"message":"","return_code":0} 2024-09-10T21:30:50.270 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:30:50.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:30:50.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 -- 172.21.15.97:0/3588902221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 msgr2=0x7f71e40cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 --2- 172.21.15.97:0/3588902221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 0x7f71e40cff50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009840 tx=0x7f71cc01c820 comp rx=0 tx=0).stop 2024-09-10T21:30:50.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 -- 172.21.15.97:0/3588902221 shutdown_connections 2024-09-10T21:30:50.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 --2- 172.21.15.97:0/3588902221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71e40d0490 0x7f71e40ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 --2- 172.21.15.97:0/3588902221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 0x7f71e40cff50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 -- 172.21.15.97:0/3588902221 >> 172.21.15.97:0/3588902221 conn(0x7f71e40d1a60 msgr2=0x7f71e40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 -- 172.21.15.97:0/3588902221 shutdown_connections 2024-09-10T21:30:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.408+0000 7f71e9e00640 1 -- 172.21.15.97:0/3588902221 wait complete. 2024-09-10T21:30:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 Processor -- start 2024-09-10T21:30:50.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 -- start start 2024-09-10T21:30:50.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 0x7f71e415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71e40d0490 0x7f71e4154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415efc0 0x7f71e4153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4168930 con 0x7f71e415efc0 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41687b0 con 0x7f71e40cdcb0 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f71e4168ab0 con 0x7f71e40d0490 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415efc0 0x7f71e4153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415efc0 0x7f71e4153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60240/0 (socket says 172.21.15.97:60240) 2024-09-10T21:30:50.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 -- 172.21.15.97:0/4252732240 learned_addr learned my addr 172.21.15.97:0/4252732240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71dbe00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71e40d0490 0x7f71e4154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e8a00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 0x7f71e415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 -- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71e40d0490 msgr2=0x7f71e4154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71e40d0490 0x7f71e4154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 -- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 msgr2=0x7f71e415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 0x7f71e415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 -- 172.21.15.97:0/4252732240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e415bd60 con 0x7f71e415efc0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e8a00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 0x7f71e415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9400640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415efc0 0x7f71e4153870 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00efd0 tx=0x7f71dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e1600640 1 -- 172.21.15.97:0/4252732240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc003b80 con 0x7f71e415efc0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e1600640 1 -- 172.21.15.97:0/4252732240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71dc003d20 con 0x7f71e415efc0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e415c040 con 0x7f71e415efc0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e1600640 1 -- 172.21.15.97:0/4252732240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc004280 con 0x7f71e415efc0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.412+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e415c600 con 0x7f71e415efc0 2024-09-10T21:30:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.416+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e40ae930 con 0x7f71e415efc0 2024-09-10T21:30:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.416+0000 7f71e1600640 1 -- 172.21.15.97:0/4252732240 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71dc004420 con 0x7f71e415efc0 2024-09-10T21:30:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.416+0000 7f71e1600640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c0072610 0x7f71c0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.416+0000 7f71e1600640 1 -- 172.21.15.97:0/4252732240 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6101+0+0 (secure 0 0 0) 0x7f71dc0941e0 con 0x7f71e415efc0 2024-09-10T21:30:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.416+0000 7f71e8a00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c0072610 0x7f71c0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.416+0000 7f71e8a00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c0072610 0x7f71c0074ad0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f71d4001230 tx=0x7f71d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.420+0000 7f71e1600640 1 -- 172.21.15.97:0/4252732240 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71dc05f690 con 0x7f71e415efc0 2024-09-10T21:30:50.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.620+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f71e40b3960 con 0x7f71e415efc0 2024-09-10T21:30:50.625 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:30:50.620+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:30:50.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.624+0000 7f71e1600640 1 -- 172.21.15.97:0/4252732240 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7f71dc05ef90 con 0x7f71e415efc0 2024-09-10T21:30:50.627 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:30:50.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.624+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c0072610 msgr2=0x7f71c0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.624+0000 7f71e9e00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c0072610 0x7f71c0074ad0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f71d4001230 tx=0x7f71d4009040 comp rx=0 tx=0).stop 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.624+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415efc0 msgr2=0x7f71e4153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.624+0000 7f71e9e00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415efc0 0x7f71e4153870 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00efd0 tx=0x7f71dc00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 shutdown_connections 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71c0072610 0x7f71c0074ad0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71e415efc0 0x7f71e4153870 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71e40d0490 0x7f71e4154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 --2- 172.21.15.97:0/4252732240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e40cdcb0 0x7f71e415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 >> 172.21.15.97:0/4252732240 conn(0x7f71e40d1a60 msgr2=0x7f71e406e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:50.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 shutdown_connections 2024-09-10T21:30:50.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.628+0000 7f71e9e00640 1 -- 172.21.15.97:0/4252732240 wait complete. 2024-09-10T21:30:50.638 INFO:tasks.ceph.mds.c.smithi097.stderr: -5151> 2024-09-10T21:30:39.116+0000 7faf71e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:30:50.651 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2024-09-10T21:30:50.688 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:30:50.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 Processor -- start 2024-09-10T21:30:50.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 -- start start 2024-09-10T21:30:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f345c80 0x55cc2f3f5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f346060 0x55cc2f3f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f349430 0x55cc2f3f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc2f1d9a00 con 0x55cc2f349430 2024-09-10T21:30:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cc2f237f40 con 0x55cc2f345c80 2024-09-10T21:30:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.692+0000 7f1cbe92d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cc2f347c90 con 0x55cc2f346060 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f349430 0x55cc2f3f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f349430 0x55cc2f3f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60294/0 (socket says 172.21.15.97:60294) 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 -- 172.21.15.97:0/2056960693 learned_addr learned my addr 172.21.15.97:0/2056960693 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f345c80 0x55cc2f3f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbb200640 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f346060 0x55cc2f3f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 -- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f346060 msgr2=0x55cc2f3f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f346060 0x55cc2f3f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 -- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f345c80 msgr2=0x55cc2f3f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f345c80 0x55cc2f3f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 -- 172.21.15.97:0/2056960693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2f1b2c10 con 0x55cc2f349430 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f349430 0x55cc2f3f5df0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40ad040 tx=0x7f1cb4018170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d765e19265e49948 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cb9e00640 1 -- 172.21.15.97:0/2056960693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40ba020 con 0x55cc2f349430 2024-09-10T21:30:50.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cb9e00640 1 -- 172.21.15.97:0/2056960693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb40aaa00 con 0x55cc2f349430 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cb9e00640 1 -- 172.21.15.97:0/2056960693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40af070 con 0x55cc2f349430 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- 172.21.15.97:0/2056960693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f349430 msgr2=0x55cc2f3f5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f349430 0x55cc2f3f5df0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40ad040 tx=0x7f1cb4018170 comp rx=0 tx=0).stop 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- 172.21.15.97:0/2056960693 shutdown_connections 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f349430 0x55cc2f3f5df0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f346060 0x55cc2f3f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/2056960693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f345c80 0x55cc2f3f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- 172.21.15.97:0/2056960693 >> 172.21.15.97:0/2056960693 conn(0x55cc2f28a140 msgr2=0x55cc2f28ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- 172.21.15.97:0/2056960693 shutdown_connections 2024-09-10T21:30:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- 172.21.15.97:0/2056960693 wait complete. 2024-09-10T21:30:50.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 Processor -- start 2024-09-10T21:30:50.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- start start 2024-09-10T21:30:50.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f345c80 0x55cc2f344310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f346060 0x55cc2f3fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f349430 0x55cc2f3fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc2f2b7250 con 0x55cc2f345c80 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cc2f2b7790 con 0x55cc2f346060 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbe92d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cc2f2b74f0 con 0x55cc2f349430 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f345c80 0x55cc2f344310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f345c80 0x55cc2f344310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60296/0 (socket says 172.21.15.97:60296) 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 learned_addr learned my addr 172.21.15.97:0/1905991765 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbb200640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f346060 0x55cc2f3fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.696+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f349430 0x55cc2f3fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f349430 msgr2=0x55cc2f3fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc2f349430 0x55cc2f3fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f346060 msgr2=0x55cc2f3fc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc2f346060 0x55cc2f3fc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2f36a130 con 0x55cc2f345c80 2024-09-10T21:30:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f345c80 0x55cc2f344310 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0087b0 tx=0x7f1cac008c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cb8a00640 1 -- 172.21.15.97:0/1905991765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac01a070 con 0x55cc2f345c80 2024-09-10T21:30:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cb8a00640 1 -- 172.21.15.97:0/1905991765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cac00abc0 con 0x55cc2f345c80 2024-09-10T21:30:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cc2f364dd0 con 0x55cc2f345c80 2024-09-10T21:30:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cb8a00640 1 -- 172.21.15.97:0/1905991765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac015420 con 0x55cc2f345c80 2024-09-10T21:30:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2f365310 con 0x55cc2f345c80 2024-09-10T21:30:50.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cb8a00640 1 -- 172.21.15.97:0/1905991765 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1cac0155c0 con 0x55cc2f345c80 2024-09-10T21:30:50.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cb8a00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c9c070fc0 0x7f1c9c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cb8a00640 1 -- 172.21.15.97:0/1905991765 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f1cac0925c0 con 0x55cc2f345c80 2024-09-10T21:30:50.706 INFO:teuthology.orchestra.run.smithi097.stdout:���� 2024-09-10T21:30:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: �� 2024-09-10T21:30:50.706 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:50.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbb200640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c9c070fc0 0x7f1c9c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cc2f344850 0x55cc2f368ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.700+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.0 13:00000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.704+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cc2f344850 0x55cc2f368ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.704+0000 7f1cbb200640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c9c070fc0 0x7f1c9c073480 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8003f00 tx=0x7f1ca8003960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.704+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cc2f344850 0x55cc2f368ba0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.704+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.704+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f1cb40bf040 0x7f1cb40b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.704+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:08000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40bf040 2024-09-10T21:30:50.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.704+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f1cb40bf040 0x7f1cb40b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.708+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f1cb40bf040 0x7f1cb40b63f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.708+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v94'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.708+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f1cac01f060 0x7f1cac05c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.708+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- osd_op(unknown.0.0:3 13.8 13:10000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac01f060 2024-09-10T21:30:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.708+0000 7f1cbb200640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f1cac01f060 0x7f1cac05c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.708+0000 7f1cbb200640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f1cac01f060 0x7f1cac05c140 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.5 v2:172.21.15.194:6818/3659294892 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac01f060 2024-09-10T21:30:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:4 13.18 13:18000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f1cb40b5390 0x7f1cb40b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:5 13.4 13:20000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40b5390 2024-09-10T21:30:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f1cb40b5390 0x7f1cb40b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f1cb40b5390 0x7f1cb40b57f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f1cb40be9c0 0x7f1cb40cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.712+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:6 13.14 13:28000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40be9c0 2024-09-10T21:30:50.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.716+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f1cb40be9c0 0x7f1cb40cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.716+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f1cb40be9c0 0x7f1cb40cf410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.716+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.716+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:7 13.c 13:30000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40be9c0 2024-09-10T21:30:50.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.716+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.716+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f1cac008040 0x7f1cac082110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.716+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:8 13.1c 13:38000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac008040 2024-09-10T21:30:50.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbb200640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f1cac008040 0x7f1cac082110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbb200640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f1cac008040 0x7f1cac082110 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:9 13.2 13:40000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:10 13.12 13:48000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40be9c0 2024-09-10T21:30:50.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f1cac0640a0 0x7f1cac080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- osd_op(unknown.0.0:11 13.a 13:50000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac0640a0 2024-09-10T21:30:50.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.720+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f1cac0640a0 0x7f1cac080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbc600640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f1cac0640a0 0x7f1cac080890 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.4 v2:172.21.15.194:6800/4073109273 1 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cac0640a0 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- osd_op(unknown.0.0:12 13.1a 13:58000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac01f060 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.5 v2:172.21.15.194:6818/3659294892 2 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac01f060 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:13 13.6 13:60000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40b5390 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'4 uv4 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.16 13:68000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40bf040 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:15 13.e 13:70000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac008040 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:16 13.1e 13:78000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac008040 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:17 13.1 13:80000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:18 13.11 13:88000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40be9c0 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:19 13.9 13:90000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac008040 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.724+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.19 13:98000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40bf040 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f1cac084790 0x7f1cac091d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:21 13.5 13:a0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac084790 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f1cac084790 0x7f1cac091d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f1cac084790 0x7f1cac091d40 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cac084790 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:22 13.15 13:a8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40b5390 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v93'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:23 13.d 13:b0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac008040 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v93'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:24 13.1d 13:b8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:25 13.3 13:c0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40be9c0 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:26 13.13 13:c8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:27 13.b 13:d0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40b5390 2024-09-10T21:30:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:28 13.1b 13:d8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac084790 2024-09-10T21:30:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cac084790 2024-09-10T21:30:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.728+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:29 13.7 13:e0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cb40b5390 2024-09-10T21:30:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:30 13.17 13:e8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:31 13.f 13:f0000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x7f1cac084790 2024-09-10T21:30:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v93'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cac084790 2024-09-10T21:30:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:32 13.1f 13:f8000000:%01:::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366a60 con 0x55cc2f344850 2024-09-10T21:30:50.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f3690e0 con 0x7f1cb40bf040 2024-09-10T21:30:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(33 601.00000000 [read 0~0] v0'0 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:34 13.10 13:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f369480 con 0x7f1cb40bf040 2024-09-10T21:30:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(34 601.00000000 [getxattrs out=124b] v0'0 uv5 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:35 13.10 13:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f369870 con 0x7f1cb40bf040 2024-09-10T21:30:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(35 601.00000000 [omap-get-header out=274b] v0'0 uv5 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:36 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f369c90 con 0x7f1cb40bf040 2024-09-10T21:30:50.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(36 601.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:37 13.10 13:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f365a50 con 0x7f1cb40bf040 2024-09-10T21:30:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(37 602.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:38 13.10 13:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f365e70 con 0x7f1cb40bf040 2024-09-10T21:30:50.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(38 602.00000000 [getxattrs out=124b] v0'0 uv6 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:39 13.10 13:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366290 con 0x7f1cb40bf040 2024-09-10T21:30:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(39 602.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:40 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366680 con 0x7f1cb40bf040 2024-09-10T21:30:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(40 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.732+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:41 13.10 13:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366680 con 0x7f1cb40bf040 2024-09-10T21:30:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(41 10000000000.00000000 [read 0~0] v0'0 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:42 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(42 10000000000.00000000 [getxattrs out=124b] v0'0 uv10 ondisk = 0) ==== 164+0+124 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:43 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(43 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:44 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.747 INFO:teuthology.orchestra.run.smithi097.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1 ���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2 ���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir ���� ��  (���fjZ[7(���fjZ[78`���f��98`���f��9���� �� sixmegs_head�i������f��9����@@`�����������f��9���fjZ[7(8`���f��98`���f��9 2024-09-10T21:30:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals out=533b] v0'0 uv10 ondisk = 0) ==== 164+0+533 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:45 13.10 13:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(45 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:46 13.10 13:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(46 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:47 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(47 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:48 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(48 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:49 13.10 13:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(49 603.00000000 [read 0~0] v0'0 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:50 13.10 13:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(50 603.00000000 [getxattrs out=124b] v0'0 uv8 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:51 13.10 13:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(51 603.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:52 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40bf040 2024-09-10T21:30:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(52 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40bf040 2024-09-10T21:30:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:53 13.14 13:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(53 1.00000000.inode [read 0~582 out=582b] v0'0 uv15 ondisk = 0) ==== 160+0+582 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:54 13.14 13:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.754 INFO:teuthology.orchestra.run.smithi097.stdout:�����������fjZ[7����`������������ �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0 ���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3 ���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)����f;t(1�C@@�����������f;t(1(���f;t(18���f;t(18}��f���}��f�������NH}��f��������������� �� 2024-09-10T21:30:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(54 1.00000000.inode [getxattrs out=4b] v0'0 uv15 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.736+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:55 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.755 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(55 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:56 13.14 13:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(56 1.00000000.inode [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:57 13.14 13:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(57 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:58 13.14 13:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.757 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@�9@�9@�A@@@ ���� �� 2024-09-10T21:30:50.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(58 200.00000000 [getxattrs out=4b] v0'0 uv19 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:59 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.758 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(59 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:60 13.14 13:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(60 200.00000000 [omap-get-vals out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:61 13.14 13:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.760 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ���A�A�A�A��XS&YA1e@��XS&YA1h^T}��f�r�2 ((88 2 ( 2024-09-10T21:30:50.760 INFO:teuthology.orchestra.run.smithi097.stdout:( 2024-09-10T21:30:50.760 INFO:teuthology.orchestra.run.smithi097.stdout:8 2024-09-10T21:30:50.760 INFO:teuthology.orchestra.run.smithi097.stdout:}��fu�8 2024-09-10T21:30:50.760 INFO:teuthology.orchestra.run.smithi097.stdout:}��fu� ��������@0@��XS&YA1�� 2024-09-10T21:30:50.760 INFO:teuthology.orchestra.run.smithi097.stdout:����f�% �K�'�a� entity_id0hostname smithi097kernel_version6.11.0-rc5-g2ca44a2f35caroot/��7 ����@��XS&YA1������f50W'n���������@��XS&YA1���f�Y'setattr� C������������ff%'�C@@��������}��f���(8���ff%'8}��f���}��f�������NH}��f����������� ��������[@��XS&YA1������f�u1mkdir�3 (���f;t(1(���f;t(18���f;t(18���f;t(1  �subdir������������f;t(1�A���������������������f;t(1���f;t(1(88�����������f;t(1������������2 ((88  C������������f;t(1�C@@�����������f;t(1(���f;t(18���f;t(18}��f���}��f�������NH}��f������������  ���������@��XS&YA1������f���7opencL3 (���f;t(1(���f;t(18���fjZ[78���f;t(1  �subdir������������fjZ[7�A���������������������fjZ[7���f;t(1(���fjZ[78���fjZ[78���fjZ[7�����������f;t(1������������V (���fjZ[7(���fjZ[78���fjZ[78���fjZ[70 sixmegs������������fjZ[7����@@�����������fjZ[7���fjZ[7 @(88�����������fjZ[7������������  !!��������h@��XS&YA1������f��8 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:cap updateT3 (���f;t(1(���f;t(18 ���f�}O88���f;t(1  �subdir������������fjZ[7�A���������������������fjZ[7���f;t(1(���fjZ[78 ���f�}O88 ���f�}O8�����������f;t(1������������^ (���fjZ[7(���fjZ[78 ���f�}O88 ���f�}O88 &sixmegs������������f�}O8����@@ �����������f�}O8���fjZ[7 �(8 ���f�}O88 ���f�}O8�����������fjZ[7���� ���������������� @��XS&YA1������fR��9 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:cap updateT3  (���f;t(1(���f;t(18P���f=�C98���f;t(1  �subdir������������fjZ[7�A���������������������fjZ[7���f;t(1(���fjZ[78P���f=�C98P���f=�C9�����������f;t(1������������^ (���fjZ[7(���fjZ[78P���f=�C98P���f=�C98 &sixmegs������������f=�C9����@@P�����������f=�C9���fjZ[7 �(8P���f=�C98P���f=�C9�����������fjZ[7����P���������������� �@��XS&YA1������f�x& 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:cap updateT3  (���f;t(1(���f;t(18`���f��98���f;t(1  �subdir�������� 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:����fjZ[7�A���������������������fjZ[7���f;t(1(���fjZ[78`���f��98`���f��9 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:�����������f;t(1������������^  (���fjZ[7(���fjZ[78`���f��98`���f��98 &sixmegs������������f��9����@@`�����������f��9���fjZ[7 (8`���f��98`���f��9�����������fjZ[7����`���������������� }'@��XS&YA1�� 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:����f\�`& �K�'�a���3/@��XS&YA1s i _ ���fg�'check_inode_max_size 3  (���f;t(1(���f;t(18`���f��98���f;t(1  �subdir�������� ����fjZ[7�A���������������������fjZ[7���f;t(1(���fjZ[78`���f��98`���f��9 �����������f;t(1������������>  (���fjZ[7(���fjZ[78`���f��98`���f��9 sixmegs�������� 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:����f��9����@@`�����������f��9���fjZ[7(8`���f��98`���f��9 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:�����������fjZ[7����`�������� sixmegs�������� 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:����f��9����@@`�����������f��9���fjZ[7(8`���f��98`���f��9 2024-09-10T21:30:50.761 INFO:teuthology.orchestra.run.smithi097.stdout:�����������fjZ[7����`�����������������/@��XS&YA18.$�g9@��XS&YA1h^T���f��x2  (���f;t(1(���f;t(18`���f��98���f;t(1 2 ( 2024-09-10T21:30:50.762 INFO:teuthology.orchestra.run.smithi097.stdout:( 2024-09-10T21:30:50.762 INFO:teuthology.orchestra.run.smithi097.stdout:8 2024-09-10T21:30:50.762 INFO:teuthology.orchestra.run.smithi097.stdout:}��fu�8 2024-09-10T21:30:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(61 200.00000001 [read 0~16801 out=16801b] v0'0 uv16 ondisk = 0) ==== 156+0+16801 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:62 13.14 13:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.763 INFO:teuthology.orchestra.run.smithi097.stdout:}��fu� ��������@�9@��XS&YA1^TJ���f�ߥscatter_writebehind2 (���f;t(1(���f;t(18`���f��98`���f��9  C������������f;t(1�C@@�����������f;t(1(���f;t(18`���f��98}��f���}��f�������NH}��f�������������������/=@���� �� 2024-09-10T21:30:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(62 200.00000001 [getxattrs out=4b] v0'0 uv16 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:63 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.763 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(63 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:64 13.14 13:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(64 200.00000001 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:65 13.14 13:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(65 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:66 13.14 13:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(66 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.740+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:67 13.14 13:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(67 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:68 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cb40be9c0 2024-09-10T21:30:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(68 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cb40be9c0 2024-09-10T21:30:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:69 13.1c 13:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f366630 con 0x7f1cac008040 2024-09-10T21:30:50.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(69 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:70 13.1c 13:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f405b80 con 0x7f1cac008040 2024-09-10T21:30:50.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(70 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:71 13.1c 13:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f405f20 con 0x7f1cac008040 2024-09-10T21:30:50.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(71 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:72 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f407320 con 0x7f1cac008040 2024-09-10T21:30:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(72 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.6 13:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f406310 con 0x7f1cb40b5390 2024-09-10T21:30:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(73 mds0_openfiles.0 [read 0~0] v0'0 uv4 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.6 13:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f4067b0 con 0x7f1cb40b5390 2024-09-10T21:30:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(74 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv4 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.771 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6 ���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7 ���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2024-09-10T21:30:50.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f406bf0 con 0x7f1cb40b5390 2024-09-10T21:30:50.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(75 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv4 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408750 con 0x7f1cb40b5390 2024-09-10T21:30:50.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(76 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:77 13.e 13:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f407740 con 0x7f1cac008040 2024-09-10T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(77 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:78 13.e 13:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f407b30 con 0x7f1cac008040 2024-09-10T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(78 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:79 13.e 13:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f407ed0 con 0x7f1cac008040 2024-09-10T21:30:50.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(79 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.744+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408270 con 0x7f1cac008040 2024-09-10T21:30:50.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:81 13.15 13:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408270 con 0x7f1cb40b5390 2024-09-10T21:30:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(81 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:82 13.15 13:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408af0 con 0x7f1cb40b5390 2024-09-10T21:30:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(82 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:83 13.15 13:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(83 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:84 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x7f1cb40b5390 2024-09-10T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(84 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:85 13.15 13:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(85 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:86 13.15 13:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.778 INFO:teuthology.orchestra.run.smithi097.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8 ���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4 ���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@ ���� �� 2024-09-10T21:30:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(86 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:87 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.779 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(87 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:88 13.15 13:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x7f1cb40b5390 2024-09-10T21:30:50.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(88 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:89 13.d 13:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cac008040 2024-09-10T21:30:50.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(89 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:90 13.d 13:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cac008040 2024-09-10T21:30:50.781 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2024-09-10T21:30:50.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(90 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:91 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cac008040 2024-09-10T21:30:50.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(91 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.782 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:92 13.d 13:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x7f1cac008040 2024-09-10T21:30:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbb200640 1 -- 172.21.15.97:0/1905991765 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(92 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f1ca801b040 con 0x7f1cac008040 2024-09-10T21:30:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.748+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.1d 13:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.1d 13:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.1d 13:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x55cc2f344850 2024-09-10T21:30:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.787 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5 ���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2024-09-10T21:30:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 mds0_inotable [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.787 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x55cc2f344850 2024-09-10T21:30:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 mds0_inotable [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:101 13.13 13:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(101 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:102 13.13 13:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.789 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2024-09-10T21:30:50.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(102 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:103 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.790 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ( 2024-09-10T21:30:50.790 INFO:teuthology.orchestra.run.smithi097.stdout:( 2024-09-10T21:30:50.790 INFO:teuthology.orchestra.run.smithi097.stdout:8 2024-09-10T21:30:50.791 INFO:teuthology.orchestra.run.smithi097.stdout:}��fu�8 2024-09-10T21:30:50.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(103 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:104 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x55cc2f344850 2024-09-10T21:30:50.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(104 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.792 INFO:teuthology.orchestra.run.smithi097.stdout:}��fu����� ��PJ 2024-09-10T21:30:50.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.b 13:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409340 con 0x7f1cb40b5390 2024-09-10T21:30:50.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(105 mds0_sessionmap [read 0~0] v0'0 uv3 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.b 13:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409340 con 0x7f1cb40b5390 2024-09-10T21:30:50.793 INFO:teuthology.orchestra.run.smithi097.stdout: stray0_head�i���}��f���@A����������������}��f���(8}��f���8}��f���}��f��������������� stray1_head�i���}��f*�@A����������������}��f*�(8}��f*�8}��f*�}��f*������������� stray2_head�i���}��fy7�@A����������������}��fy7�(8}��fy7�8}��fy7�}��fy7������������� stray3_head�i���}��f��@A����������������}��f��(8}��f��8}��f��}��f�������������� stray4_head�i���}��f���@A����������������}��f���(8}��f���8}��f���}��f��������������� stray5_head�i���}��f� �@A����������������}��f� �(8}��f� �8}��f� �}��f� ������������� stray6_head�i���}��ff/�@A����������������}��ff/�(8}��ff/�8}��ff/�}��ff/������������� stray7_head�i���}��f��@A����������������}��f��(8}��f��8}��f��}��f�������������� stray8_head�i���}��f�[�@A����������������}��f�[�(8}��f�[�8}��f�[�}��f�[������������� stray9_head�i��� }��fu�@A����������������}��fu�(8}��fu�8}��fu�}��fu����������������� �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2024-09-10T21:30:50.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(106 mds0_sessionmap [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 159+0+4 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.752+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(107 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x7f1cb40b5390 2024-09-10T21:30:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(108 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(109 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(110 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:111 13.7 13:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cb40b5390 2024-09-10T21:30:50.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(111 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:112 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x7f1cb40b5390 2024-09-10T21:30:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(112 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x7f1cb40b5390 2024-09-10T21:30:50.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cac084790 2024-09-10T21:30:50.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(113 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cac084790 2024-09-10T21:30:50.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cac084790 2024-09-10T21:30:50.798 INFO:teuthology.orchestra.run.smithi097.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9 ���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2024-09-10T21:30:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(114 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cac084790 2024-09-10T21:30:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x7f1cac084790 2024-09-10T21:30:50.799 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.2 v2:172.21.15.97:6808/3581819336 6 ==== osd_op_reply(115 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cac084790 2024-09-10T21:30:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:116 13.f 13:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x7f1cac084790 2024-09-10T21:30:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbbc00640 1 -- 172.21.15.97:0/1905991765 <== osd.2 v2:172.21.15.97:6808/3581819336 7 ==== osd_op_reply(116 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1cac01c030 con 0x7f1cac084790 2024-09-10T21:30:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.800 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)�}��f"�@A����������������}��f"�( 2024-09-10T21:30:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(117 100.00000000.inode [read 0~582 out=582b] v0'0 uv3 ondisk = 0) ==== 162+0+582 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.801 INFO:teuthology.orchestra.run.smithi097.stdout:8 }��fu�8 }��fu�}��f"�����NH������������ �� 2024-09-10T21:30:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(118 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 162+0+4 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.802 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� 2024-09-10T21:30:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(119 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.756+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x55cc2f344850 2024-09-10T21:30:50.803 INFO:teuthology.orchestra.run.smithi097.stdout:���� �� ��("+ 2024-09-10T21:30:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(120 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(121 1.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(122 1.00000000 [getxattrs out=92b] v0'0 uv6 ondisk = 0) ==== 154+0+92 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f408e90 con 0x55cc2f344850 2024-09-10T21:30:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(123 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc2f409ea0 con 0x55cc2f344850 2024-09-10T21:30:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(124 1.00000000 [omap-get-vals out=524b] v0'0 uv6 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x7f1cb40bc040 con 0x55cc2f344850 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f1cac008040 msgr2=0x7f1cac082110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f1cac008040 0x7f1cac082110 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cc2f344850 msgr2=0x55cc2f368ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cc2f344850 0x55cc2f368ba0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f1cac084790 msgr2=0x7f1cac091d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f1cac084790 0x7f1cac091d40 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f1cb40b5390 msgr2=0x7f1cb40b57f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f1cb40b5390 0x7f1cb40b57f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f1cac0640a0 msgr2=0x7f1cac080890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f1cac0640a0 0x7f1cac080890 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f1cac01f060 msgr2=0x7f1cac05c140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f1cac01f060 0x7f1cac05c140 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f1cb40be9c0 msgr2=0x7f1cb40cf410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f1cb40be9c0 0x7f1cb40cf410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f1cb40bf040 msgr2=0x7f1cb40b63f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f1cb40bf040 0x7f1cb40b63f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 reap_dead start 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 reap_dead start 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 reap_dead start 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbc600640 1 -- 172.21.15.97:0/1905991765 reap_dead start 2024-09-10T21:30:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c9c070fc0 msgr2=0x7f1c9c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c9c070fc0 0x7f1c9c073480 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8003f00 tx=0x7f1ca8003960 comp rx=0 tx=0).stop 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f345c80 msgr2=0x55cc2f344310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f345c80 0x55cc2f344310 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0087b0 tx=0x7f1cac008c70 comp rx=0 tx=0).stop 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 shutdown_connections 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c9c070fc0 0x7f1c9c073480 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 --2- 172.21.15.97:0/1905991765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc2f345c80 0x55cc2f344310 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 >> 172.21.15.97:0/1905991765 conn(0x55cc2f28a140 msgr2=0x55cc2f360960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 shutdown_connections 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.760+0000 7f1cbe92d080 1 -- 172.21.15.97:0/1905991765 wait complete. 2024-09-10T21:30:50.810 INFO:teuthology.orchestra.run.smithi097.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@_parent ���� ��  (���f;t(1(���f;t(18`���f��98���f;t(1���� ��  subdir_head�i������fjZ[7�A���������������������fjZ[7���f;t(1(���fjZ[78`���f��98`���f��9 �����������f;t(1���������������� �� �� 2024-09-10T21:30:50.811 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2024-09-10T21:30:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 Processor -- start 2024-09-10T21:30:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- start start 2024-09-10T21:30:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dcb1aa0 0x55c51dd88230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dd6aca0 0x55c51dd85d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 0x55c51dd862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c51dbffa00 con 0x55c51dd6aca0 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c51dc5df40 con 0x55c51dcb1aa0 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c51dce0da0 con 0x55c51dd6b080 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dd6aca0 0x55c51dd85d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dd6aca0 0x55c51dd85d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60316/0 (socket says 172.21.15.97:60316) 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 -- 172.21.15.97:0/768606277 learned_addr learned my addr 172.21.15.97:0/768606277 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c99600640 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 0x55c51dd862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c98c00640 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dcb1aa0 0x55c51dd88230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 -- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 msgr2=0x55c51dd862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 0x55c51dd862d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 -- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dcb1aa0 msgr2=0x55c51dd88230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dcb1aa0 0x55c51dd88230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 -- 172.21.15.97:0/768606277 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51dbd8c10 con 0x55c51dd6aca0 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c93e00640 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dd6aca0 0x55c51dd85d90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3c84018e10 tx=0x7f3c84017f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6a367a1098233 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c92a00640 1 -- 172.21.15.97:0/768606277 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840ab5f0 con 0x55c51dd6aca0 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c92a00640 1 -- 172.21.15.97:0/768606277 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c840bb070 con 0x55c51dd6aca0 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c92a00640 1 -- 172.21.15.97:0/768606277 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840abc70 con 0x55c51dd6aca0 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/768606277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dd6aca0 msgr2=0x55c51dd85d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dd6aca0 0x55c51dd85d90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3c84018e10 tx=0x7f3c84017f80 comp rx=0 tx=0).stop 2024-09-10T21:30:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/768606277 shutdown_connections 2024-09-10T21:30:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 0x55c51dd862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dd6aca0 0x55c51dd85d90 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/768606277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dcb1aa0 0x55c51dd88230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/768606277 >> 172.21.15.97:0/768606277 conn(0x55c51dcb0140 msgr2=0x55c51dcaf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/768606277 shutdown_connections 2024-09-10T21:30:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/768606277 wait complete. 2024-09-10T21:30:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 Processor -- start 2024-09-10T21:30:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.892+0000 7f3c9b9ad080 1 -- start start 2024-09-10T21:30:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dcb1aa0 0x55c51dd70ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dd6aca0 0x55c51dcf3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 0x55c51de1e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c51dd88cb0 con 0x55c51dcb1aa0 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c51dd846f0 con 0x55c51dd6aca0 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c51dd84990 con 0x55c51dd6b080 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dcb1aa0 0x55c51dd70ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dcb1aa0 0x55c51dd70ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60322/0 (socket says 172.21.15.97:60322) 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 learned_addr learned my addr 172.21.15.97:0/4284735499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c93e00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dd6aca0 0x55c51dcf3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 0x55c51de1e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 msgr2=0x55c51de1e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c51dd6b080 0x55c51de1e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dd6aca0 msgr2=0x55c51dcf3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c51dd6aca0 0x55c51dcf3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51dd715e0 con 0x55c51dcb1aa0 2024-09-10T21:30:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dcb1aa0 0x55c51dd70ff0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c002fa0 tx=0x7f3c8c00c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c91600640 1 -- 172.21.15.97:0/4284735499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c001c90 con 0x55c51dcb1aa0 2024-09-10T21:30:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c91600640 1 -- 172.21.15.97:0/4284735499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c8c001e30 con 0x55c51dcb1aa0 2024-09-10T21:30:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c51dd71880 con 0x55c51dcb1aa0 2024-09-10T21:30:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c91600640 1 -- 172.21.15.97:0/4284735499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c002050 con 0x55c51dcb1aa0 2024-09-10T21:30:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.896+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51dd71dc0 con 0x55c51dcb1aa0 2024-09-10T21:30:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c91600640 1 -- 172.21.15.97:0/4284735499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3c8c0021f0 con 0x55c51dcb1aa0 2024-09-10T21:30:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c91600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3c78070fc0 0x7f3c78073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c91600640 1 -- 172.21.15.97:0/4284735499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f3c8c092200 con 0x55c51dcb1aa0 2024-09-10T21:30:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c93e00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3c78070fc0 0x7f3c78073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c51dcf5890 0x55c51dcf5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c51dcf5890 0x55c51dcf5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c93e00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3c78070fc0 0x7f3c78073480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3c84019890 tx=0x7f3c84004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c51dcf5890 0x55c51dcf5c70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c94002040 con 0x55c51dcf5890 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f3c9400f940 0x7f3c9400d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400f940 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f3c9400f940 0x7f3c9400d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f3c9400f940 0x7f3c9400d3b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v94'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c9400f940 2024-09-10T21:30:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f3c8c062be0 0x7f3c8c05bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.900+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c062be0 2024-09-10T21:30:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c93e00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f3c8c062be0 0x7f3c8c05bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c93e00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f3c8c062be0 0x7f3c8c05bcf0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 <== osd.5 v2:172.21.15.194:6818/3659294892 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c840b3070 con 0x7f3c8c062be0 2024-09-10T21:30:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c94002040 con 0x55c51dcf5890 2024-09-10T21:30:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f3c9400c340 0x7f3c9400c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400c340 2024-09-10T21:30:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f3c9400c340 0x7f3c9400c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f3c9400c340 0x7f3c9400c7a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c94002040 con 0x7f3c9400c340 2024-09-10T21:30:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f3c94022a60 0x7f3c940243f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c94022a60 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f3c94022a60 0x7f3c940243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f3c94022a60 0x7f3c940243f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c94022a60 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.904+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c94022a60 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c94022a60 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f3c8c065fe0 0x7f3c8c0613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c065fe0 2024-09-10T21:30:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c93e00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f3c8c065fe0 0x7f3c8c0613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c93e00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f3c8c065fe0 0x7f3c8c0613f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f3c840b3070 con 0x7f3c8c065fe0 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c94002040 con 0x55c51dcf5890 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c94022a60 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c94022a60 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f3c8c07cb40 0x7f3c8c090910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c07cb40 2024-09-10T21:30:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f3c8c07cb40 0x7f3c8c090910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c99600640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f3c8c07cb40 0x7f3c8c090910 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.4 v2:172.21.15.194:6800/4073109273 1 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c94031040 con 0x7f3c8c07cb40 2024-09-10T21:30:50.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c062be0 2024-09-10T21:30:50.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 <== osd.5 v2:172.21.15.194:6818/3659294892 2 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c840b3070 con 0x7f3c8c062be0 2024-09-10T21:30:50.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.908+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400c340 2024-09-10T21:30:50.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'4 uv4 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f3c94031040 con 0x7f3c9400c340 2024-09-10T21:30:50.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400f940 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c9400f940 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c065fe0 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f3c840b3070 con 0x7f3c8c065fe0 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c065fe0 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c840b3070 con 0x7f3c8c065fe0 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c94031040 con 0x55c51dcf5890 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c94022a60 2024-09-10T21:30:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c94022a60 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c065fe0 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c840b3070 con 0x7f3c8c065fe0 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400f940 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c9400f940 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f3c8c0896e0 0x7f3c8c07c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c0896e0 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f3c8c0896e0 0x7f3c8c07c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f3c8c0896e0 0x7f3c8c07c0d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c8c0896e0 2024-09-10T21:30:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400c340 2024-09-10T21:30:50.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.912+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v93'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f3c94031040 con 0x7f3c9400c340 2024-09-10T21:30:50.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c065fe0 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v93'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3c840b3070 con 0x7f3c8c065fe0 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c93e00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f3c94031040 con 0x55c51dcf5890 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c94022a60 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c94022a60 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f3c94031040 con 0x55c51dcf5890 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400c340 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f3c94031040 con 0x7f3c9400c340 2024-09-10T21:30:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c0896e0 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v93'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c8c0896e0 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c9400c340 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v93'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f3c94031040 con 0x7f3c9400c340 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c94031040 con 0x55c51dcf5890 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x7f3c8c0896e0 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v93'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f3c8c059290 con 0x7f3c8c0896e0 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c98c00640 1 -- 172.21.15.97:0/4284735499 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) -- 0x55c51dcf61b0 con 0x55c51dcf5890 2024-09-10T21:30:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f3c94031040 con 0x55c51dcf5890 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:601.00000000 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:602.00000000 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:10000000000.00000000 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:600.00000000 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:603.00000000 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:1.00000000.inode 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:200.00000000 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:200.00000001 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:606.00000000 2024-09-10T21:30:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:607.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:mds0_openfiles.0 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:608.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:604.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:500.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:mds_snaptable 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:605.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:mds0_inotable 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:100.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:mds0_sessionmap 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:609.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:400.00000000 2024-09-10T21:30:50.926 INFO:teuthology.orchestra.run.smithi097.stdout:100.00000000.inode 2024-09-10T21:30:50.927 INFO:teuthology.orchestra.run.smithi097.stdout:1.00000000 2024-09-10T21:30:50.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f3c8c065fe0 msgr2=0x7f3c8c0613f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f3c8c065fe0 0x7f3c8c0613f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c51dcf5890 msgr2=0x55c51dcf5c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c51dcf5890 0x55c51dcf5c70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f3c8c0896e0 msgr2=0x7f3c8c07c0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f3c8c0896e0 0x7f3c8c07c0d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f3c9400c340 msgr2=0x7f3c9400c7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f3c9400c340 0x7f3c9400c7a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f3c8c07cb40 msgr2=0x7f3c8c090910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f3c8c07cb40 0x7f3c8c090910 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f3c8c062be0 msgr2=0x7f3c8c05bcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7f3c8c062be0 0x7f3c8c05bcf0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f3c94022a60 msgr2=0x7f3c940243f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f3c94022a60 0x7f3c940243f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f3c9400f940 msgr2=0x7f3c9400d3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7f3c9400f940 0x7f3c9400d3b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 reap_dead start 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 reap_dead start 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 reap_dead start 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c99600640 1 -- 172.21.15.97:0/4284735499 reap_dead start 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3c78070fc0 msgr2=0x7f3c78073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3c78070fc0 0x7f3c78073480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3c84019890 tx=0x7f3c84004890 comp rx=0 tx=0).stop 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dcb1aa0 msgr2=0x55c51dd70ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.916+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dcb1aa0 0x55c51dd70ff0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c002fa0 tx=0x7f3c8c00c6d0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.920+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 shutdown_connections 2024-09-10T21:30:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.920+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3c78070fc0 0x7f3c78073480 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.920+0000 7f3c9b9ad080 1 --2- 172.21.15.97:0/4284735499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c51dcb1aa0 0x55c51dd70ff0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.920+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 >> 172.21.15.97:0/4284735499 conn(0x55c51dcb0140 msgr2=0x55c51dd898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.920+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 shutdown_connections 2024-09-10T21:30:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:50.920+0000 7f3c9b9ad080 1 -- 172.21.15.97:0/4284735499 wait complete. 2024-09-10T21:30:50.936 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2024-09-10T21:30:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 Processor -- start 2024-09-10T21:30:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- start start 2024-09-10T21:30:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e108211cc0 0x55e1081a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108213a00 0x55e1081a3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1082171d0 0x55e10822ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e108105f40 con 0x55e108213a00 2024-09-10T21:30:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e108112d00 con 0x55e1082171d0 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55e1080a7a00 con 0x55e108211cc0 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108213a00 0x55e1081a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108213a00 0x55e1081a3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60338/0 (socket says 172.21.15.97:60338) 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 -- 172.21.15.97:0/1111811669 learned_addr learned my addr 172.21.15.97:0/1111811669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 -- 172.21.15.97:0/1111811669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e108211cc0 msgr2=0x55e1081a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 --2- 172.21.15.97:0/1111811669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e108211cc0 0x55e1081a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 -- 172.21.15.97:0/1111811669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1082171d0 msgr2=0x55e10822ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 --2- 172.21.15.97:0/1111811669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1082171d0 0x55e10822ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 -- 172.21.15.97:0/1111811669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e108230da0 con 0x55e108213a00 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ce000640 1 --2- 172.21.15.97:0/1111811669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108213a00 0x55e1081a3810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00b350 tx=0x7f35bc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64ce66d2c27be67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ccc00640 1 -- 172.21.15.97:0/1111811669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00a040 con 0x55e108213a00 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ccc00640 1 -- 172.21.15.97:0/1111811669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35bc00f620 con 0x55e108213a00 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35ccc00640 1 -- 172.21.15.97:0/1111811669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00fa50 con 0x55e108213a00 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- 172.21.15.97:0/1111811669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108213a00 msgr2=0x55e1081a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 --2- 172.21.15.97:0/1111811669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108213a00 0x55e1081a3810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00b350 tx=0x7f35bc00b810 comp rx=0 tx=0).stop 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- 172.21.15.97:0/1111811669 shutdown_connections 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 --2- 172.21.15.97:0/1111811669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1082171d0 0x55e10822ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 --2- 172.21.15.97:0/1111811669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108213a00 0x55e1081a3810 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 --2- 172.21.15.97:0/1111811669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e108211cc0 0x55e1081a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- 172.21.15.97:0/1111811669 >> 172.21.15.97:0/1111811669 conn(0x55e1081a10d0 msgr2=0x55e108189d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- 172.21.15.97:0/1111811669 shutdown_connections 2024-09-10T21:30:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- 172.21.15.97:0/1111811669 wait complete. 2024-09-10T21:30:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 Processor -- start 2024-09-10T21:30:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.016+0000 7f35d224d080 1 -- start start 2024-09-10T21:30:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108211cc0 0x55e1082c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e108213a00 0x55e10822a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e1082171d0 0x55e10822a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e1081897b0 con 0x55e108211cc0 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e108210880 con 0x55e108213a00 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55e108189a50 con 0x55e1082171d0 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108211cc0 0x55e1082c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108211cc0 0x55e1082c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60354/0 (socket says 172.21.15.97:60354) 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 -- 172.21.15.97:0/1330016759 learned_addr learned my addr 172.21.15.97:0/1330016759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35ce000640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e108213a00 0x55e10822a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cf400640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e1082171d0 0x55e10822a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 -- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e1082171d0 msgr2=0x55e10822a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e1082171d0 0x55e10822a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 -- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e108213a00 msgr2=0x55e10822a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e108213a00 0x55e10822a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 -- 172.21.15.97:0/1330016759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1082106b0 con 0x55e108211cc0 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35cea00640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108211cc0 0x55e1082c8820 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f35c001b7a0 tx=0x7f35c00ab6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35c7400640 1 -- 172.21.15.97:0/1330016759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c00b0040 con 0x55e108211cc0 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35c7400640 1 -- 172.21.15.97:0/1330016759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35c00acdf0 con 0x55e108211cc0 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e10822e750 con 0x55e108211cc0 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35c7400640 1 -- 172.21.15.97:0/1330016759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c00b5670 con 0x55e108211cc0 2024-09-10T21:30:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e10822b180 con 0x55e108211cc0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.020+0000 7f35c7400640 1 -- 172.21.15.97:0/1330016759 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f35c00be3f0 con 0x55e108211cc0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35c7400640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35b0070fc0 0x7f35b0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35c7400640 1 -- 172.21.15.97:0/1330016759 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f35c012bde0 con 0x55e108211cc0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55e1082cc4e0 0x55e1082cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35ce000640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35b0070fc0 0x7f35b0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55e1082cce00 con 0x55e1082cc4e0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35cf400640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55e1082cc4e0 0x55e1082cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35ce000640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35b0070fc0 0x7f35b0073480 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000c10 tx=0x7f35bc001130 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35cf400640 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55e1082cc4e0 0x55e1082cc8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35cf400640 1 -- 172.21.15.97:0/1330016759 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f35c80017b0 con 0x55e1082cc4e0 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:header (274 bytes) : 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:00000000 04 03 0c 01 00 00 0b 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:00000020 00 00 00 00 88 ba e0 66 6a 5a 5b 37 01 00 00 00 |.......fjZ[7....| 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:00000050 00 00 88 ba e0 66 6a 5a 5b 37 01 00 00 00 00 00 |.....fjZ[7......| 2024-09-10T21:30:51.029 INFO:teuthology.orchestra.run.smithi097.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:000000a0 00 00 00 00 00 00 00 00 88 ba e0 66 18 aa 80 39 |...........f...9| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:000000e0 00 00 00 00 00 00 88 ba e0 66 18 aa 80 39 00 00 |.........f...9..| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:* 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:00000110 00 00 |..| 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout:00000112 2024-09-10T21:30:51.030 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55e1082cc4e0 msgr2=0x55e1082cc8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55e1082cc4e0 0x55e1082cc8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35b0070fc0 msgr2=0x7f35b0073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35b0070fc0 0x7f35b0073480 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000c10 tx=0x7f35bc001130 comp rx=0 tx=0).stop 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108211cc0 msgr2=0x55e1082c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 --2- 172.21.15.97:0/1330016759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e108211cc0 0x55e1082c8820 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f35c001b7a0 tx=0x7f35c00ab6f0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35cf400640 1 -- 172.21.15.97:0/1330016759 reap_dead start 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 shutdown_connections 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 >> 172.21.15.97:0/1330016759 conn(0x55e1081a10d0 msgr2=0x55e10818a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 shutdown_connections 2024-09-10T21:30:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.024+0000 7f35d224d080 1 -- 172.21.15.97:0/1330016759 wait complete. 2024-09-10T21:30:51.034 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2024-09-10T21:30:51.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.120+0000 7fbeaa479080 1 Processor -- start 2024-09-10T21:30:51.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- start start 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 0x55df62df1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dd9b00 0x55df62df2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dda2a0 0x55df62dddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55df62c6ea00 con 0x55df62dd9b00 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55df62cccf40 con 0x55df62dda2a0 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55df62ddbe70 con 0x55df62d4f940 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dd9b00 0x55df62df2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dd9b00 0x55df62df2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60372/0 (socket says 172.21.15.97:60372) 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 -- 172.21.15.97:0/3061100490 learned_addr learned my addr 172.21.15.97:0/3061100490 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbea8000640 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dda2a0 0x55df62dddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbea7600640 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 0x55df62df1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 -- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 msgr2=0x55df62df1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 0x55df62df1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 -- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dda2a0 msgr2=0x55df62dddf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dda2a0 0x55df62dddf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 -- 172.21.15.97:0/3061100490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df62c47c10 con 0x55df62dd9b00 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbea8000640 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dda2a0 0x55df62dddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:51.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbe9fe00640 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dd9b00 0x55df62df2b90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000b280 tx=0x7fbe90001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d2153cb999cab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbea6200640 1 -- 172.21.15.97:0/3061100490 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90016070 con 0x55df62dd9b00 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbea6200640 1 -- 172.21.15.97:0/3061100490 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe9000d4c0 con 0x55df62dd9b00 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbea6200640 1 -- 172.21.15.97:0/3061100490 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000d6f0 con 0x55df62dd9b00 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- 172.21.15.97:0/3061100490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dd9b00 msgr2=0x55df62df2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dd9b00 0x55df62df2b90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000b280 tx=0x7fbe90001e80 comp rx=0 tx=0).stop 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- 172.21.15.97:0/3061100490 shutdown_connections 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dda2a0 0x55df62dddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dd9b00 0x55df62df2b90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3061100490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 0x55df62df1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- 172.21.15.97:0/3061100490 >> 172.21.15.97:0/3061100490 conn(0x55df62d4e520 msgr2=0x55df62d1fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- 172.21.15.97:0/3061100490 shutdown_connections 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- 172.21.15.97:0/3061100490 wait complete. 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 Processor -- start 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.124+0000 7fbeaa479080 1 -- start start 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 0x55df62dd8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dd9b00 0x55df62df5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dda2a0 0x55df62df62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55df62df8300 con 0x55df62dda2a0 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55df62df8060 con 0x55df62dd9b00 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55df62d527d0 con 0x55df62d4f940 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dda2a0 0x55df62df62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dda2a0 0x55df62df62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60384/0 (socket says 172.21.15.97:60384) 2024-09-10T21:30:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 -- 172.21.15.97:0/3139129037 learned_addr learned my addr 172.21.15.97:0/3139129037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbe9fe00640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dd9b00 0x55df62df5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea7600640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 0x55df62dd8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 -- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 msgr2=0x55df62dd8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55df62d4f940 0x55df62dd8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 -- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dd9b00 msgr2=0x55df62df5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dd9b00 0x55df62df5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 -- 172.21.15.97:0/3139129037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df62df4c30 con 0x55df62dda2a0 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbe9fe00640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55df62dd9b00 0x55df62df5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea8000640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dda2a0 0x55df62df62a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fbea000b700 tx=0x7fbea000bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea4e00640 1 -- 172.21.15.97:0/3139129037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0004220 con 0x55df62dda2a0 2024-09-10T21:30:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea4e00640 1 -- 172.21.15.97:0/3139129037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbea0009aa0 con 0x55df62dda2a0 2024-09-10T21:30:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df62df4ed0 con 0x55df62dda2a0 2024-09-10T21:30:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbea4e00640 1 -- 172.21.15.97:0/3139129037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0017450 con 0x55df62dda2a0 2024-09-10T21:30:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.128+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df62dde2b0 con 0x55df62dda2a0 2024-09-10T21:30:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbea4e00640 1 -- 172.21.15.97:0/3139129037 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbea00175f0 con 0x55df62dda2a0 2024-09-10T21:30:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbea4e00640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe84070fc0 0x7fbe84073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbea4e00640 1 -- 172.21.15.97:0/3139129037 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fbea0092ff0 con 0x55df62dda2a0 2024-09-10T21:30:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbea7600640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe84070fc0 0x7fbe84073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55df62df58b0 0x55df62e903c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55df62e90900 con 0x55df62df58b0 2024-09-10T21:30:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbe9fe00640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55df62df58b0 0x55df62e903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbea7600640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe84070fc0 0x7fbe84073480 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ab000 tx=0x7fbe98019920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbe9fe00640 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55df62df58b0 0x55df62e903c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbe9fe00640 1 -- 172.21.15.97:0/3139129037 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7fbe90019040 con 0x55df62df58b0 2024-09-10T21:30:51.137 INFO:teuthology.orchestra.run.smithi097.stdout:header (274 bytes) : 2024-09-10T21:30:51.137 INFO:teuthology.orchestra.run.smithi097.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.137 INFO:teuthology.orchestra.run.smithi097.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:* 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:* 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:* 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:* 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:00000110 00 00 |..| 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout:00000112 2024-09-10T21:30:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55df62df58b0 msgr2=0x55df62e903c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55df62df58b0 0x55df62e903c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe84070fc0 msgr2=0x7fbe84073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe84070fc0 0x7fbe84073480 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ab000 tx=0x7fbe98019920 comp rx=0 tx=0).stop 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dda2a0 msgr2=0x55df62df62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 --2- 172.21.15.97:0/3139129037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df62dda2a0 0x55df62df62a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fbea000b700 tx=0x7fbea000bbc0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbea8000640 1 -- 172.21.15.97:0/3139129037 reap_dead start 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 shutdown_connections 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 >> 172.21.15.97:0/3139129037 conn(0x55df62d4e520 msgr2=0x55df62d1fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 shutdown_connections 2024-09-10T21:30:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.132+0000 7fbeaa479080 1 -- 172.21.15.97:0/3139129037 wait complete. 2024-09-10T21:30:51.141 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2024-09-10T21:30:51.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.180+0000 7f237c82b080 1 Processor -- start 2024-09-10T21:30:51.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- start start 2024-09-10T21:30:51.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 0x558193c476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cd6060 0x558193cd4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cf03c0 0x558193cd4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558193b69a00 con 0x558193cf03c0 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558193bc7f40 con 0x558193cd5c80 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558193cd7c90 con 0x558193cd6060 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cf03c0 0x558193cd4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cf03c0 0x558193cd4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60414/0 (socket says 172.21.15.97:60414) 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 -- 172.21.15.97:0/3068260485 learned_addr learned my addr 172.21.15.97:0/3068260485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2378c00640 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 0x558193c476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2373e00640 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cd6060 0x558193cd4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 -- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cd6060 msgr2=0x558193cd4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cd6060 0x558193cd4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 -- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 msgr2=0x558193c476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 0x558193c476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 -- 172.21.15.97:0/3068260485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558193b42c10 con 0x558193cf03c0 2024-09-10T21:30:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2379600640 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cf03c0 0x558193cd4940 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f237400b7d0 tx=0x7f237400bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d568dfce5c506b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2372a00640 1 -- 172.21.15.97:0/3068260485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237400db30 con 0x558193cf03c0 2024-09-10T21:30:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2372a00640 1 -- 172.21.15.97:0/3068260485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f237400ddc0 con 0x558193cf03c0 2024-09-10T21:30:51.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f2372a00640 1 -- 172.21.15.97:0/3068260485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237400f690 con 0x558193cf03c0 2024-09-10T21:30:51.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- 172.21.15.97:0/3068260485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cf03c0 msgr2=0x558193cd4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cf03c0 0x558193cd4940 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f237400b7d0 tx=0x7f237400bc90 comp rx=0 tx=0).stop 2024-09-10T21:30:51.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- 172.21.15.97:0/3068260485 shutdown_connections 2024-09-10T21:30:51.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cf03c0 0x558193cd4940 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cd6060 0x558193cd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 --2- 172.21.15.97:0/3068260485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 0x558193c476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- 172.21.15.97:0/3068260485 >> 172.21.15.97:0/3068260485 conn(0x558193c1a140 msgr2=0x558193c1ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- 172.21.15.97:0/3068260485 shutdown_connections 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- 172.21.15.97:0/3068260485 wait complete. 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 Processor -- start 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.184+0000 7f237c82b080 1 -- start start 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 0x558193cd2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cd6060 0x558193cf0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cf03c0 0x558193cf0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558193cf26d0 con 0x558193cd6060 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558193cf41d0 con 0x558193cd5c80 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558193cf3f30 con 0x558193cf03c0 2024-09-10T21:30:51.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cd6060 0x558193cf0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cd6060 0x558193cf0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60422/0 (socket says 172.21.15.97:60422) 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 -- 172.21.15.97:0/696413119 learned_addr learned my addr 172.21.15.97:0/696413119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2378c00640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 0x558193cd2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2379600640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cf03c0 0x558193cf0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 -- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cf03c0 msgr2=0x558193cf0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558193cf03c0 0x558193cf0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 -- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 msgr2=0x558193cd2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558193cd5c80 0x558193cd2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 -- 172.21.15.97:0/696413119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558193cd3320 con 0x558193cd6060 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2373e00640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cd6060 0x558193cf0960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2364017d60 tx=0x7f23640ac8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2371600640 1 -- 172.21.15.97:0/696413119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640acd60 con 0x558193cd6060 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2371600640 1 -- 172.21.15.97:0/696413119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23640b8440 con 0x558193cd6060 2024-09-10T21:30:51.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558193cf91f0 con 0x558193cd6060 2024-09-10T21:30:51.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2371600640 1 -- 172.21.15.97:0/696413119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640b8850 con 0x558193cd6060 2024-09-10T21:30:51.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558193cf9730 con 0x558193cd6060 2024-09-10T21:30:51.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.188+0000 7f2371600640 1 -- 172.21.15.97:0/696413119 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f23640b89f0 con 0x558193cd6060 2024-09-10T21:30:51.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2371600640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2358070fc0 0x7f2358073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2371600640 1 -- 172.21.15.97:0/696413119 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f236412dfe0 con 0x558193cd6060 2024-09-10T21:30:51.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2378c00640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2358070fc0 0x7f2358073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558193cf1710 0x558193c3f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x558193c3f9c0 con 0x558193cf1710 2024-09-10T21:30:51.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2379600640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558193cf1710 0x558193c3f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2378c00640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2358070fc0 0x7f2358073480 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f236c000c10 tx=0x7f236c001820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2379600640 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558193cf1710 0x558193c3f480 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2379600640 1 -- 172.21.15.97:0/696413119 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2374002030 con 0x558193cf1710 2024-09-10T21:30:51.196 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:51.196 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558193cf1710 msgr2=0x558193c3f480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558193cf1710 0x558193c3f480 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2358070fc0 msgr2=0x7f2358073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2358070fc0 0x7f2358073480 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f236c000c10 tx=0x7f236c001820 comp rx=0 tx=0).stop 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cd6060 msgr2=0x558193cf0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 --2- 172.21.15.97:0/696413119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558193cd6060 0x558193cf0960 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2364017d60 tx=0x7f23640ac8a0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f2379600640 1 -- 172.21.15.97:0/696413119 reap_dead start 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 shutdown_connections 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 >> 172.21.15.97:0/696413119 conn(0x558193c1a140 msgr2=0x558193c49ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 shutdown_connections 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.192+0000 7f237c82b080 1 -- 172.21.15.97:0/696413119 wait complete. 2024-09-10T21:30:51.201 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2024-09-10T21:30:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.288+0000 7fbad4479080 1 Processor -- start 2024-09-10T21:30:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.288+0000 7fbad4479080 1 -- start start 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e62cc0 0x55f2b8daabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e64a00 0x55f2b8e7e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e681d0 0x55f2b8defe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8cf8a00 con 0x55f2b8e62cc0 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8d56f40 con 0x55f2b8e64a00 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f2b8e66710 con 0x55f2b8e681d0 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e62cc0 0x55f2b8daabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e62cc0 0x55f2b8daabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60440/0 (socket says 172.21.15.97:60440) 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 -- 172.21.15.97:0/2309706595 learned_addr learned my addr 172.21.15.97:0/2309706595 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad2000640 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e681d0 0x55f2b8defe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad0c00640 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e64a00 0x55f2b8e7e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 -- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e681d0 msgr2=0x55f2b8defe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e681d0 0x55f2b8defe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 -- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e64a00 msgr2=0x55f2b8e7e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e64a00 0x55f2b8e7e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 -- 172.21.15.97:0/2309706595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b8cd1c10 con 0x55f2b8e62cc0 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad1600640 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e62cc0 0x55f2b8daabf0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbac400b2b0 tx=0x7fbac4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd55d7aba8302b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbacb400640 1 -- 172.21.15.97:0/2309706595 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4016070 con 0x55f2b8e62cc0 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbacb400640 1 -- 172.21.15.97:0/2309706595 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbac400d4c0 con 0x55f2b8e62cc0 2024-09-10T21:30:51.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbacb400640 1 -- 172.21.15.97:0/2309706595 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac400d6f0 con 0x55f2b8e62cc0 2024-09-10T21:30:51.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- 172.21.15.97:0/2309706595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e62cc0 msgr2=0x55f2b8daabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e62cc0 0x55f2b8daabf0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbac400b2b0 tx=0x7fbac4001e80 comp rx=0 tx=0).stop 2024-09-10T21:30:51.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- 172.21.15.97:0/2309706595 shutdown_connections 2024-09-10T21:30:51.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e681d0 0x55f2b8defe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e64a00 0x55f2b8e7e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- 172.21.15.97:0/2309706595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e62cc0 0x55f2b8daabf0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- 172.21.15.97:0/2309706595 >> 172.21.15.97:0/2309706595 conn(0x55f2b8df20d0 msgr2=0x55f2b8e623c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- 172.21.15.97:0/2309706595 shutdown_connections 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- 172.21.15.97:0/2309706595 wait complete. 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 Processor -- start 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- start start 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e62cc0 0x55f2b8e7d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e64a00 0x55f2b8e834e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e681d0 0x55f2b8e7e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8dd9240 con 0x55f2b8e64a00 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f2b8e81e80 con 0x55f2b8e681d0 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.292+0000 7fbad4479080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f2b8dd94e0 con 0x55f2b8e62cc0 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e64a00 0x55f2b8e834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e64a00 0x55f2b8e834e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60442/0 (socket says 172.21.15.97:60442) 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 -- 172.21.15.97:0/767373634 learned_addr learned my addr 172.21.15.97:0/767373634 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad1600640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e62cc0 0x55f2b8e7d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad2000640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e681d0 0x55f2b8e7e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 -- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e62cc0 msgr2=0x55f2b8e7d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f2b8e62cc0 0x55f2b8e7d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 -- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e681d0 msgr2=0x55f2b8e7e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2b8e681d0 0x55f2b8e7e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 -- 172.21.15.97:0/767373634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b8e814d0 con 0x55f2b8e64a00 2024-09-10T21:30:51.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad0c00640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e64a00 0x55f2b8e834e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbac0018ed0 tx=0x7fbac00ab610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbaca000640 1 -- 172.21.15.97:0/767373634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac00acad0 con 0x55f2b8e64a00 2024-09-10T21:30:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbaca000640 1 -- 172.21.15.97:0/767373634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbac00be070 con 0x55f2b8e64a00 2024-09-10T21:30:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbaca000640 1 -- 172.21.15.97:0/767373634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac00b8470 con 0x55f2b8e64a00 2024-09-10T21:30:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f2b8e81770 con 0x55f2b8e64a00 2024-09-10T21:30:51.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b8e83040 con 0x55f2b8e64a00 2024-09-10T21:30:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.296+0000 7fbaca000640 1 -- 172.21.15.97:0/767373634 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbac00b6040 con 0x55f2b8e64a00 2024-09-10T21:30:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbaca000640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab4070ff0 0x7fbab40734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbaca000640 1 -- 172.21.15.97:0/767373634 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fbac012d640 con 0x55f2b8e64a00 2024-09-10T21:30:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad1600640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab4070ff0 0x7fbab40734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f2b8e83a20 0x55f2b8e84ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f2b8e7daa0 con 0x55f2b8e83a20 2024-09-10T21:30:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad2000640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f2b8e83a20 0x55f2b8e84ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad1600640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab4070ff0 0x7fbab40734b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbac4000c10 tx=0x7fbac4001090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad2000640 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f2b8e83a20 0x55f2b8e84ee0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad2000640 1 -- 172.21.15.97:0/767373634 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbacc00b040 con 0x55f2b8e83a20 2024-09-10T21:30:51.304 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:51.304 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f2b8e83a20 msgr2=0x55f2b8e84ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f2b8e83a20 0x55f2b8e84ee0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab4070ff0 msgr2=0x7fbab40734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab4070ff0 0x7fbab40734b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbac4000c10 tx=0x7fbac4001090 comp rx=0 tx=0).stop 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e64a00 msgr2=0x55f2b8e834e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 --2- 172.21.15.97:0/767373634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f2b8e64a00 0x55f2b8e834e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbac0018ed0 tx=0x7fbac00ab610 comp rx=0 tx=0).stop 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad2000640 1 -- 172.21.15.97:0/767373634 reap_dead start 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 shutdown_connections 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 >> 172.21.15.97:0/767373634 conn(0x55f2b8df20d0 msgr2=0x55f2b8df4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 shutdown_connections 2024-09-10T21:30:51.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.300+0000 7fbad4479080 1 -- 172.21.15.97:0/767373634 wait complete. 2024-09-10T21:30:51.308 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2024-09-10T21:30:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.388+0000 7f7474d13080 1 Processor -- start 2024-09-10T21:30:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- start start 2024-09-10T21:30:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8fec0 0x561154ea4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 0x561154f1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154f1eb50 0x561154f1aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561154e0bf40 con 0x561154e8fec0 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561154e18d00 con 0x561154f1eb50 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561154dada00 con 0x561154f1c2a0 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8fec0 0x561154ea4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8fec0 0x561154ea4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60444/0 (socket says 172.21.15.97:60444) 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 -- 172.21.15.97:0/2464916505 learned_addr learned my addr 172.21.15.97:0/2464916505 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472a00640 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154f1eb50 0x561154f1aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7471600640 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 0x561154f1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 -- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 msgr2=0x561154f1aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 0x561154f1aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 -- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154f1eb50 msgr2=0x561154f1aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154f1eb50 0x561154f1aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 -- 172.21.15.97:0/2464916505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561154f18620 con 0x561154e8fec0 2024-09-10T21:30:51.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7472000640 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8fec0 0x561154ea4380 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f746400b3e0 tx=0x7f7464002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97fd3d9a85ee88c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f746be00640 1 -- 172.21.15.97:0/2464916505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464012070 con 0x561154e8fec0 2024-09-10T21:30:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f746be00640 1 -- 172.21.15.97:0/2464916505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f746400d950 con 0x561154e8fec0 2024-09-10T21:30:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f746be00640 1 -- 172.21.15.97:0/2464916505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746400dd80 con 0x561154e8fec0 2024-09-10T21:30:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- 172.21.15.97:0/2464916505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8fec0 msgr2=0x561154ea4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8fec0 0x561154ea4380 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f746400b3e0 tx=0x7f7464002f70 comp rx=0 tx=0).stop 2024-09-10T21:30:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- 172.21.15.97:0/2464916505 shutdown_connections 2024-09-10T21:30:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154f1eb50 0x561154f1aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 0x561154f1aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 --2- 172.21.15.97:0/2464916505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8fec0 0x561154ea4380 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- 172.21.15.97:0/2464916505 >> 172.21.15.97:0/2464916505 conn(0x561154ea70d0 msgr2=0x561154e5f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- 172.21.15.97:0/2464916505 shutdown_connections 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- 172.21.15.97:0/2464916505 wait complete. 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 Processor -- start 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.392+0000 7f7474d13080 1 -- start start 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154e8fec0 0x561154ea4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 0x561154f3ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154f1eb50 0x561154e82760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561154f18b60 con 0x561154f1eb50 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561154f188c0 con 0x561154e8fec0 2024-09-10T21:30:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561154f1a610 con 0x561154f1c2a0 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154f1eb50 0x561154e82760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154f1eb50 0x561154e82760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60454/0 (socket says 172.21.15.97:60454) 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 -- 172.21.15.97:0/3602541835 learned_addr learned my addr 172.21.15.97:0/3602541835 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472000640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154e8fec0 0x561154ea4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7471600640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 0x561154f3ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 -- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 msgr2=0x561154f3ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561154f1c2a0 0x561154f3ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 -- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154e8fec0 msgr2=0x561154ea4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561154e8fec0 0x561154ea4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 -- 172.21.15.97:0/3602541835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561154ea5710 con 0x561154f1eb50 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7472a00640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154f1eb50 0x561154e82760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ad040 tx=0x7f746c0aa730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f746aa00640 1 -- 172.21.15.97:0/3602541835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c0af040 con 0x561154f1eb50 2024-09-10T21:30:51.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f746aa00640 1 -- 172.21.15.97:0/3602541835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f746c004110 con 0x561154f1eb50 2024-09-10T21:30:51.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561154ea59b0 con 0x561154f1eb50 2024-09-10T21:30:51.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f746aa00640 1 -- 172.21.15.97:0/3602541835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c0aaba0 con 0x561154f1eb50 2024-09-10T21:30:51.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561154ea5ef0 con 0x561154f1eb50 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.396+0000 7f746aa00640 1 -- 172.21.15.97:0/3602541835 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f746c0b7410 con 0x561154f1eb50 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f746aa00640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7454070ff0 0x7f74540734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f746aa00640 1 -- 172.21.15.97:0/3602541835 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f746c12ce80 con 0x561154f1eb50 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7472000640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7454070ff0 0x7f74540734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561154f3f140 0x561154fd2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x561154ea52d0 con 0x561154f3f140 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7471600640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561154f3f140 0x561154fd2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7472000640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7454070ff0 0x7f74540734b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7464001180 tx=0x7f7464019040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7471600640 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561154f3f140 0x561154fd2d00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7471600640 1 -- 172.21.15.97:0/3602541835 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f746000d040 con 0x561154f3f140 2024-09-10T21:30:51.404 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:51.404 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561154f3f140 msgr2=0x561154fd2d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561154f3f140 0x561154fd2d00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7454070ff0 msgr2=0x7f74540734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7454070ff0 0x7f74540734b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7464001180 tx=0x7f7464019040 comp rx=0 tx=0).stop 2024-09-10T21:30:51.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154f1eb50 msgr2=0x561154e82760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 --2- 172.21.15.97:0/3602541835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154f1eb50 0x561154e82760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ad040 tx=0x7f746c0aa730 comp rx=0 tx=0).stop 2024-09-10T21:30:51.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7472a00640 1 -- 172.21.15.97:0/3602541835 reap_dead start 2024-09-10T21:30:51.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 shutdown_connections 2024-09-10T21:30:51.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 >> 172.21.15.97:0/3602541835 conn(0x561154ea70d0 msgr2=0x561154f1aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 shutdown_connections 2024-09-10T21:30:51.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.400+0000 7f7474d13080 1 -- 172.21.15.97:0/3602541835 wait complete. 2024-09-10T21:30:51.408 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2024-09-10T21:30:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 Processor -- start 2024-09-10T21:30:51.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- start start 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561425514cc0 0x5614255303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 0x5614255171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56142551a1d0 0x56142552cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614253aaa00 con 0x56142551a1d0 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561425408f40 con 0x561425514cc0 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56142553d440 con 0x561425516830 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56142551a1d0 0x56142552cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56142551a1d0 0x56142552cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60478/0 (socket says 172.21.15.97:60478) 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 -- 172.21.15.97:0/2203280314 learned_addr learned my addr 172.21.15.97:0/2203280314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb8a00640 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561425514cc0 0x5614255303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb3e00640 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 0x5614255171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 -- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 msgr2=0x5614255171d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 0x5614255171d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 -- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561425514cc0 msgr2=0x5614255303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561425514cc0 0x5614255303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 -- 172.21.15.97:0/2203280314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561425383c10 con 0x56142551a1d0 2024-09-10T21:30:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb9400640 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56142551a1d0 0x56142552cc60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b25ec700119142 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb2a00640 1 -- 172.21.15.97:0/2203280314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4017070 con 0x56142551a1d0 2024-09-10T21:30:51.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb2a00640 1 -- 172.21.15.97:0/2203280314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fb40027b0 con 0x56142551a1d0 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fb2a00640 1 -- 172.21.15.97:0/2203280314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4002c40 con 0x56142551a1d0 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/2203280314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56142551a1d0 msgr2=0x56142552cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56142551a1d0 0x56142552cc60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4006fd0 tx=0x7f9fb4001b10 comp rx=0 tx=0).stop 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/2203280314 shutdown_connections 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56142551a1d0 0x56142552cc60 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 0x5614255171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/2203280314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561425514cc0 0x5614255303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/2203280314 >> 172.21.15.97:0/2203280314 conn(0x5614254a40d0 msgr2=0x56142548cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/2203280314 shutdown_connections 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/2203280314 wait complete. 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 Processor -- start 2024-09-10T21:30:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.496+0000 7f9fbb7c6080 1 -- start start 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561425514cc0 0x561425513fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 0x56142549eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56142551a1d0 0x56142549f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614255183f0 con 0x561425514cc0 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561425518150 con 0x56142551a1d0 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56142548a820 con 0x561425516830 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561425514cc0 0x561425513fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561425514cc0 0x561425513fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60482/0 (socket says 172.21.15.97:60482) 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 -- 172.21.15.97:0/74589268 learned_addr learned my addr 172.21.15.97:0/74589268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb9400640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56142551a1d0 0x56142549f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb3e00640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 0x56142549eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 -- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 msgr2=0x56142549eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561425516830 0x56142549eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 -- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56142551a1d0 msgr2=0x56142549f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56142551a1d0 0x56142549f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 -- 172.21.15.97:0/74589268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614254a0930 con 0x561425514cc0 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb8a00640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561425514cc0 0x561425513fc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f9fac017eb0 tx=0x7f9fac0ab590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb1600640 1 -- 172.21.15.97:0/74589268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0b0040 con 0x561425514cc0 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb1600640 1 -- 172.21.15.97:0/74589268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fac0aba40 con 0x561425514cc0 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5614254a0bd0 con 0x561425514cc0 2024-09-10T21:30:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb1600640 1 -- 172.21.15.97:0/74589268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0abe30 con 0x561425514cc0 2024-09-10T21:30:51.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614254a1110 con 0x561425514cc0 2024-09-10T21:30:51.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.500+0000 7f9fb1600640 1 -- 172.21.15.97:0/74589268 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9fac0b7460 con 0x561425514cc0 2024-09-10T21:30:51.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb1600640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f98070ff0 0x7f9f980734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb1600640 1 -- 172.21.15.97:0/74589268 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f9fac12cd80 con 0x561425514cc0 2024-09-10T21:30:51.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561425514500 0x5614255cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x561425530940 con 0x561425514500 2024-09-10T21:30:51.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb3e00640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f98070ff0 0x7f9f980734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb9400640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561425514500 0x5614255cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb3e00640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f98070ff0 0x7f9f980734b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400b8e0 tx=0x7f9fa4004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb9400640 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561425514500 0x5614255cd850 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb9400640 1 -- 172.21.15.97:0/74589268 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv4 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f9fb400f070 con 0x561425514500 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stdout:header (42 bytes) : 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 04 00 00 |me v011.........| 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stdout:0000002a 2024-09-10T21:30:51.508 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561425514500 msgr2=0x5614255cd850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561425514500 0x5614255cd850 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f98070ff0 msgr2=0x7f9f980734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f98070ff0 0x7f9f980734b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400b8e0 tx=0x7f9fa4004890 comp rx=0 tx=0).stop 2024-09-10T21:30:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561425514cc0 msgr2=0x561425513fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 --2- 172.21.15.97:0/74589268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561425514cc0 0x561425513fc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f9fac017eb0 tx=0x7f9fac0ab590 comp rx=0 tx=0).stop 2024-09-10T21:30:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fb9400640 1 -- 172.21.15.97:0/74589268 reap_dead start 2024-09-10T21:30:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 shutdown_connections 2024-09-10T21:30:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 >> 172.21.15.97:0/74589268 conn(0x5614254a40d0 msgr2=0x5614255131a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 shutdown_connections 2024-09-10T21:30:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.504+0000 7f9fbb7c6080 1 -- 172.21.15.97:0/74589268 wait complete. 2024-09-10T21:30:51.513 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2024-09-10T21:30:51.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 Processor -- start 2024-09-10T21:30:51.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- start start 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb4002970 0x55efe4d692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c370 0x55efe4df36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55efe4e0c750 0x55efe4d39dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55efe4c86a00 con 0x55efe4e0c370 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55efe4ce4f40 con 0x7fdbb4002970 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55efe4df10a0 con 0x55efe4e0c750 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c370 0x55efe4df36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c370 0x55efe4df36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60496/0 (socket says 172.21.15.97:60496) 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 -- 172.21.15.97:0/1636534783 learned_addr learned my addr 172.21.15.97:0/1636534783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd0600640 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55efe4e0c750 0x55efe4d39dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcfc00640 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb4002970 0x55efe4d692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 -- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55efe4e0c750 msgr2=0x55efe4d39dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55efe4e0c750 0x55efe4d39dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 -- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb4002970 msgr2=0x55efe4d692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb4002970 0x55efe4d692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 -- 172.21.15.97:0/1636534783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe4c5fc10 con 0x55efe4e0c370 2024-09-10T21:30:51.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcf200640 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c370 0x55efe4df36a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc002040 tx=0x7fdbbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba81683421e9a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcde00640 1 -- 172.21.15.97:0/1636534783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0a4590 con 0x55efe4e0c370 2024-09-10T21:30:51.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcde00640 1 -- 172.21.15.97:0/1636534783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbbc0a4940 con 0x55efe4e0c370 2024-09-10T21:30:51.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbcde00640 1 -- 172.21.15.97:0/1636534783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0a4b10 con 0x55efe4e0c370 2024-09-10T21:30:51.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- 172.21.15.97:0/1636534783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c370 msgr2=0x55efe4df36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c370 0x55efe4df36a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc002040 tx=0x7fdbbc0046c0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- 172.21.15.97:0/1636534783 shutdown_connections 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55efe4e0c750 0x55efe4d39dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c370 0x55efe4df36a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 --2- 172.21.15.97:0/1636534783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb4002970 0x55efe4d692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- 172.21.15.97:0/1636534783 >> 172.21.15.97:0/1636534783 conn(0x55efe4d800d0 msgr2=0x55efe4ded5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- 172.21.15.97:0/1636534783 shutdown_connections 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.604+0000 7fdbd297e080 1 -- 172.21.15.97:0/1636534783 wait complete. 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 Processor -- start 2024-09-10T21:30:51.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 -- start start 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55efe4e0c370 0x55efe4e0b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c750 0x55efe4ea4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb4002970 0x55efe4dee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55efe4dee3c0 con 0x55efe4e0c750 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55efe4dee120 con 0x55efe4e0c370 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55efe4d68bb0 con 0x7fdbb4002970 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c750 0x55efe4ea4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c750 0x55efe4ea4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60498/0 (socket says 172.21.15.97:60498) 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 -- 172.21.15.97:0/3035724288 learned_addr learned my addr 172.21.15.97:0/3035724288 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcfc00640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55efe4e0c370 0x55efe4e0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd0600640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb4002970 0x55efe4dee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 -- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb4002970 msgr2=0x55efe4dee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdbb4002970 0x55efe4dee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 -- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55efe4e0c370 msgr2=0x55efe4e0b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55efe4e0c370 0x55efe4e0b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 -- 172.21.15.97:0/3035724288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe4ea56f0 con 0x55efe4e0c750 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcf200640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c750 0x55efe4ea4c90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc01c030 tx=0x7fdbbc0a5ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcca00640 1 -- 172.21.15.97:0/3035724288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0a4590 con 0x55efe4e0c750 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcca00640 1 -- 172.21.15.97:0/3035724288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbbc0c3070 con 0x55efe4e0c750 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbcca00640 1 -- 172.21.15.97:0/3035724288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0b47a0 con 0x55efe4e0c750 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55efe4ea5990 con 0x55efe4e0c750 2024-09-10T21:30:51.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.608+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe4ea5ed0 con 0x55efe4e0c750 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbcca00640 1 -- 172.21.15.97:0/3035724288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdbbc0b4940 con 0x55efe4e0c750 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbcca00640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdbb0070ff0 0x7fdbb00734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbcca00640 1 -- 172.21.15.97:0/3035724288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fdbbc12dff0 con 0x55efe4e0c750 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbcfc00640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdbb0070ff0 0x7fdbb00734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55efe4ea51d0 0x55efe4ea36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55efe4e0bac0 con 0x55efe4ea51d0 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd0600640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55efe4ea51d0 0x55efe4ea36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbcfc00640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdbb0070ff0 0x7fdbb00734b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00044c0 tx=0x7fdbc000f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd0600640 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55efe4ea51d0 0x55efe4ea36e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd0600640 1 -- 172.21.15.97:0/3035724288 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbc8009a80 con 0x55efe4ea51d0 2024-09-10T21:30:51.617 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:51.617 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55efe4ea51d0 msgr2=0x55efe4ea36e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55efe4ea51d0 0x55efe4ea36e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdbb0070ff0 msgr2=0x7fdbb00734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdbb0070ff0 0x7fdbb00734b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00044c0 tx=0x7fdbc000f040 comp rx=0 tx=0).stop 2024-09-10T21:30:51.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c750 msgr2=0x55efe4ea4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 --2- 172.21.15.97:0/3035724288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efe4e0c750 0x55efe4ea4c90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc01c030 tx=0x7fdbbc0a5ee0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd0600640 1 -- 172.21.15.97:0/3035724288 reap_dead start 2024-09-10T21:30:51.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 shutdown_connections 2024-09-10T21:30:51.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 >> 172.21.15.97:0/3035724288 conn(0x55efe4d800d0 msgr2=0x55efe4df36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 shutdown_connections 2024-09-10T21:30:51.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.612+0000 7fdbd297e080 1 -- 172.21.15.97:0/3035724288 wait complete. 2024-09-10T21:30:51.621 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2024-09-10T21:30:51.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 Processor -- start 2024-09-10T21:30:51.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 -- start start 2024-09-10T21:30:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1000 0x55ec1fcf13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1920 0x55ec1fcf23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec1fc9ff40 con 0x7f029c003100 2024-09-10T21:30:51.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ec1fcacd00 con 0x55ec1fcf1000 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02ba29b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ec1fc41a00 con 0x55ec1fcf1920 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60522/0 (socket says 172.21.15.97:60522) 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 -- 172.21.15.97:0/2080586929 learned_addr learned my addr 172.21.15.97:0/2080586929 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7e00640 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1920 0x55ec1fcf23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b6a00640 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1000 0x55ec1fcf13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 -- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1920 msgr2=0x55ec1fcf23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1920 0x55ec1fcf23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 -- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1000 msgr2=0x55ec1fcf13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1000 0x55ec1fcf13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 -- 172.21.15.97:0/2080586929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1fe56640 con 0x7f029c003100 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b6a00640 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1000 0x55ec1fcf13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b7400640 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc94b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b770 tx=0x7f02a800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e668856419f717 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b5600640 1 -- 172.21.15.97:0/2080586929 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a800ad70 con 0x7f029c003100 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b5600640 1 -- 172.21.15.97:0/2080586929 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02a8017430 con 0x7f029c003100 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.712+0000 7f02b5600640 1 -- 172.21.15.97:0/2080586929 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8017820 con 0x7f029c003100 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- 172.21.15.97:0/2080586929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 msgr2=0x55ec1fdc94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc94b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b770 tx=0x7f02a800d040 comp rx=0 tx=0).stop 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- 172.21.15.97:0/2080586929 shutdown_connections 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1920 0x55ec1fcf23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1000 0x55ec1fcf13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 --2- 172.21.15.97:0/2080586929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc94b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- 172.21.15.97:0/2080586929 >> 172.21.15.97:0/2080586929 conn(0x55ec1fd21520 msgr2=0x55ec1fdaf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- 172.21.15.97:0/2080586929 shutdown_connections 2024-09-10T21:30:51.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- 172.21.15.97:0/2080586929 wait complete. 2024-09-10T21:30:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 Processor -- start 2024-09-10T21:30:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- start start 2024-09-10T21:30:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1000 0x55ec1fdaaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1920 0x55ec1fdc7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec1fe568e0 con 0x7f029c003100 2024-09-10T21:30:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ec1fe56b80 con 0x55ec1fcf1920 2024-09-10T21:30:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ec1fdcc0e0 con 0x55ec1fcf1000 2024-09-10T21:30:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1000 0x55ec1fdaaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60532/0 (socket says 172.21.15.97:60532) 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 -- 172.21.15.97:0/185144017 learned_addr learned my addr 172.21.15.97:0/185144017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b6a00640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1920 0x55ec1fdc7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 -- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1000 msgr2=0x55ec1fdaaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1000 0x55ec1fdaaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 -- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1920 msgr2=0x55ec1fdc7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ec1fcf1920 0x55ec1fdc7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 -- 172.21.15.97:0/185144017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1fdcc380 con 0x7f029c003100 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7400640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ec1fcf1000 0x55ec1fdaaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02b7e00640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc8380 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f02b0002240 tx=0x7f02b00059c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02afe00640 1 -- 172.21.15.97:0/185144017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b001c070 con 0x7f029c003100 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02afe00640 1 -- 172.21.15.97:0/185144017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02b0007590 con 0x7f029c003100 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02afe00640 1 -- 172.21.15.97:0/185144017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b00079c0 con 0x7f029c003100 2024-09-10T21:30:51.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec1fdcc620 con 0x7f029c003100 2024-09-10T21:30:51.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.716+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec1fdcb3d0 con 0x7f029c003100 2024-09-10T21:30:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02afe00640 1 -- 172.21.15.97:0/185144017 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f02b0007b60 con 0x7f029c003100 2024-09-10T21:30:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02afe00640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0298070ff0 0x7f02980734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02afe00640 1 -- 172.21.15.97:0/185144017 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f02b0091da0 con 0x7f029c003100 2024-09-10T21:30:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02b7400640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0298070ff0 0x7f02980734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02ba29b080 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ec1fdc6710 0x55ec1fdc6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55ec1fdcbc80 con 0x55ec1fdc6710 2024-09-10T21:30:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02b6a00640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ec1fdc6710 0x55ec1fdc6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.720+0000 7f02b7400640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0298070ff0 0x7f02980734b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f02a80040c0 tx=0x7f02a8002fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02b6a00640 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ec1fdc6710 0x55ec1fdc6af0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02b6a00640 1 -- 172.21.15.97:0/185144017 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f02a400b040 con 0x55ec1fdc6710 2024-09-10T21:30:51.727 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:51.727 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ec1fdc6710 msgr2=0x55ec1fdc6af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ec1fdc6710 0x55ec1fdc6af0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0298070ff0 msgr2=0x7f02980734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0298070ff0 0x7f02980734b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f02a80040c0 tx=0x7f02a8002fc0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 msgr2=0x55ec1fdc8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 --2- 172.21.15.97:0/185144017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f029c003100 0x55ec1fdc8380 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f02b0002240 tx=0x7f02b00059c0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02b7e00640 1 -- 172.21.15.97:0/185144017 reap_dead start 2024-09-10T21:30:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 shutdown_connections 2024-09-10T21:30:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 >> 172.21.15.97:0/185144017 conn(0x55ec1fd21520 msgr2=0x55ec1fdc8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 shutdown_connections 2024-09-10T21:30:51.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.724+0000 7f02ba29b080 1 -- 172.21.15.97:0/185144017 wait complete. 2024-09-10T21:30:51.731 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2024-09-10T21:30:51.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 Processor -- start 2024-09-10T21:30:51.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 -- start start 2024-09-10T21:30:51.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc1c80 0x56125cdbf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc2060 0x56125cdbfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc5430 0x56125cdc0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56125cc55a00 con 0x56125cdc2060 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56125ccb3f40 con 0x56125cdc5430 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56125cde8460 con 0x56125cdc1c80 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc2060 0x56125cdbfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc2060 0x56125cdbfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60542/0 (socket says 172.21.15.97:60542) 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 -- 172.21.15.97:0/244692205 learned_addr learned my addr 172.21.15.97:0/244692205 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c8800640 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc1c80 0x56125cdbf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c9200640 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc5430 0x56125cdc0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 -- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc1c80 msgr2=0x56125cdbf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc1c80 0x56125cdbf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 -- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc5430 msgr2=0x56125cdc0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc5430 0x56125cdc0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 -- 172.21.15.97:0/244692205 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56125cc2ec10 con 0x56125cdc2060 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c9200640 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc5430 0x56125cdc0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c7e00640 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc2060 0x56125cdbfc30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f12b40197f0 tx=0x7f12b4017ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c4bd1f03a83d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c6a00640 1 -- 172.21.15.97:0/244692205 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b40ab630 con 0x56125cdc2060 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c6a00640 1 -- 172.21.15.97:0/244692205 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b40bb070 con 0x56125cdc2060 2024-09-10T21:30:51.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12c6a00640 1 -- 172.21.15.97:0/244692205 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b40abd30 con 0x56125cdc2060 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 -- 172.21.15.97:0/244692205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc2060 msgr2=0x56125cdbfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc2060 0x56125cdbfc30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f12b40197f0 tx=0x7f12b4017ee0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 -- 172.21.15.97:0/244692205 shutdown_connections 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc5430 0x56125cdc0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc2060 0x56125cdbfc30 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/244692205 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc1c80 0x56125cdbf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.816+0000 7f12cb5d9080 1 -- 172.21.15.97:0/244692205 >> 172.21.15.97:0/244692205 conn(0x56125cd06140 msgr2=0x56125cd06d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- 172.21.15.97:0/244692205 shutdown_connections 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- 172.21.15.97:0/244692205 wait complete. 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 Processor -- start 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- start start 2024-09-10T21:30:51.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc1c80 0x56125cde30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc2060 0x56125cde5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc5430 0x56125cdd9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56125cdc0980 con 0x56125cdc5430 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56125cd33480 con 0x56125cdc1c80 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56125cd336c0 con 0x56125cdc2060 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc5430 0x56125cdd9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc5430 0x56125cdd9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60548/0 (socket says 172.21.15.97:60548) 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 -- 172.21.15.97:0/18484908 learned_addr learned my addr 172.21.15.97:0/18484908 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c8800640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc1c80 0x56125cde30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c7e00640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc2060 0x56125cde5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 -- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc2060 msgr2=0x56125cde5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56125cdc2060 0x56125cde5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 -- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc1c80 msgr2=0x56125cde30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56125cdc1c80 0x56125cde30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 -- 172.21.15.97:0/18484908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56125cde5a60 con 0x56125cdc5430 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c9200640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc5430 0x56125cdd9190 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c180 tx=0x7f12c000c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c5600640 1 -- 172.21.15.97:0/18484908 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0015030 con 0x56125cdc5430 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c5600640 1 -- 172.21.15.97:0/18484908 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12c00113f0 con 0x56125cdc5430 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56125cde5d00 con 0x56125cdc5430 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12c5600640 1 -- 172.21.15.97:0/18484908 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0011800 con 0x56125cdc5430 2024-09-10T21:30:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.820+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56125cddfa90 con 0x56125cdc5430 2024-09-10T21:30:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c5600640 1 -- 172.21.15.97:0/18484908 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f12c0022060 con 0x56125cdc5430 2024-09-10T21:30:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c5600640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12a8070ff0 0x7f12a80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c5600640 1 -- 172.21.15.97:0/18484908 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f12c0092070 con 0x56125cdc5430 2024-09-10T21:30:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56125cdd96d0 0x56125cde1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56125cde24c0 con 0x56125cdd96d0 2024-09-10T21:30:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c8800640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12a8070ff0 0x7f12a80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c7e00640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56125cdd96d0 0x56125cde1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c8800640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12a8070ff0 0x7f12a80734b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c650 tx=0x7f12bc004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c7e00640 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56125cdd96d0 0x56125cde1f80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12c7e00640 1 -- 172.21.15.97:0/18484908 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f12b40b3070 con 0x56125cdd96d0 2024-09-10T21:30:51.830 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:51.830 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56125cdd96d0 msgr2=0x56125cde1f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56125cdd96d0 0x56125cde1f80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12a8070ff0 msgr2=0x7f12a80734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12a8070ff0 0x7f12a80734b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c650 tx=0x7f12bc004890 comp rx=0 tx=0).stop 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc5430 msgr2=0x56125cdd9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.824+0000 7f12cb5d9080 1 --2- 172.21.15.97:0/18484908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56125cdc5430 0x56125cdd9190 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c180 tx=0x7f12c000c640 comp rx=0 tx=0).stop 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.828+0000 7f12c9200640 1 -- 172.21.15.97:0/18484908 reap_dead start 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.828+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 shutdown_connections 2024-09-10T21:30:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.828+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 >> 172.21.15.97:0/18484908 conn(0x56125cd06140 msgr2=0x56125cddd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.828+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 shutdown_connections 2024-09-10T21:30:51.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.828+0000 7f12cb5d9080 1 -- 172.21.15.97:0/18484908 wait complete. 2024-09-10T21:30:51.835 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2024-09-10T21:30:51.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 Processor -- start 2024-09-10T21:30:51.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- start start 2024-09-10T21:30:51.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f5c80 0x559ea48f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f6060 0x559ea4911970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f9430 0x559ea4911eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ea4839030 con 0x559ea48f9430 2024-09-10T21:30:51.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559ea4789a00 con 0x559ea48f6060 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559ea48391b0 con 0x559ea48f5c80 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f9430 0x559ea4911eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f9430 0x559ea4911eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60562/0 (socket says 172.21.15.97:60562) 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 -- 172.21.15.97:0/2264000321 learned_addr learned my addr 172.21.15.97:0/2264000321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c8e00640 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f5c80 0x559ea48f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c0400640 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f6060 0x559ea4911970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 -- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f5c80 msgr2=0x559ea48f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f5c80 0x559ea48f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 -- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f6060 msgr2=0x559ea4911970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f6060 0x559ea4911970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 -- 172.21.15.97:0/2264000321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea4788a20 con 0x559ea48f9430 2024-09-10T21:30:51.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c0400640 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f6060 0x559ea4911970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c9800640 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f9430 0x559ea4911eb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c720 tx=0x7f54c400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c615416ea2a3d11d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c3400640 1 -- 172.21.15.97:0/2264000321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4004020 con 0x559ea48f9430 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c3400640 1 -- 172.21.15.97:0/2264000321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54c40042b0 con 0x559ea48f9430 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54c3400640 1 -- 172.21.15.97:0/2264000321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c400aa80 con 0x559ea48f9430 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- 172.21.15.97:0/2264000321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f9430 msgr2=0x559ea4911eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f9430 0x559ea4911eb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c720 tx=0x7f54c400cbe0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- 172.21.15.97:0/2264000321 shutdown_connections 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f9430 0x559ea4911eb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f6060 0x559ea4911970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/2264000321 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f5c80 0x559ea48f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- 172.21.15.97:0/2264000321 >> 172.21.15.97:0/2264000321 conn(0x559ea483a140 msgr2=0x559ea49133a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- 172.21.15.97:0/2264000321 shutdown_connections 2024-09-10T21:30:51.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- 172.21.15.97:0/2264000321 wait complete. 2024-09-10T21:30:51.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 Processor -- start 2024-09-10T21:30:51.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.924+0000 7f54cbbe8080 1 -- start start 2024-09-10T21:30:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f5c80 0x559ea49a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f6060 0x559ea490f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f9430 0x559ea49a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ea491a170 con 0x559ea48f6060 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559ea48f2da0 con 0x559ea48f5c80 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559ea4919ed0 con 0x559ea48f9430 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f6060 0x559ea490f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f6060 0x559ea490f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60570/0 (socket says 172.21.15.97:60570) 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 -- 172.21.15.97:0/3146561157 learned_addr learned my addr 172.21.15.97:0/3146561157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c8e00640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f5c80 0x559ea49a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c9800640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f9430 0x559ea49a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 -- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f9430 msgr2=0x559ea49a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559ea48f9430 0x559ea49a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 -- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f5c80 msgr2=0x559ea49a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559ea48f5c80 0x559ea49a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 -- 172.21.15.97:0/3146561157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea4762c10 con 0x559ea48f6060 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c0400640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f6060 0x559ea490f450 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f54b000a6c0 tx=0x7f54b000ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c2000640 1 -- 172.21.15.97:0/3146561157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b00089a0 con 0x559ea48f6060 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c2000640 1 -- 172.21.15.97:0/3146561157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54b0008b40 con 0x559ea48f6060 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559ea49a2490 con 0x559ea48f6060 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54c2000640 1 -- 172.21.15.97:0/3146561157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0016420 con 0x559ea48f6060 2024-09-10T21:30:51.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.928+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea49a29d0 con 0x559ea48f6060 2024-09-10T21:30:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c2000640 1 -- 172.21.15.97:0/3146561157 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f54b001f440 con 0x559ea48f6060 2024-09-10T21:30:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c2000640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54a4070fc0 0x7f54a4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c2000640 1 -- 172.21.15.97:0/3146561157 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f54b00921b0 con 0x559ea48f6060 2024-09-10T21:30:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559ea49138b0 0x559ea49ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c8e00640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54a4070fc0 0x7f54a4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x559ea49abcc0 con 0x559ea49138b0 2024-09-10T21:30:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c9800640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559ea49138b0 0x559ea49ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c8e00640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54a4070fc0 0x7f54a4073480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f54b80ab000 tx=0x7f54b8019a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c9800640 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559ea49138b0 0x559ea49ab780 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:51.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c9800640 1 -- 172.21.15.97:0/3146561157 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f54c4002030 con 0x559ea49138b0 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:header (274 bytes) : 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:000000a0 00 00 00 00 00 00 00 00 7d ba e0 66 75 14 e2 08 |........}..fu...| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:000000e0 00 00 00 00 00 00 7d ba e0 66 75 14 e2 08 00 00 |......}..fu.....| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:* 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:00000110 00 00 |..| 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout:00000112 2024-09-10T21:30:51.938 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:51.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559ea49138b0 msgr2=0x559ea49ab780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559ea49138b0 0x559ea49ab780 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:51.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54a4070fc0 msgr2=0x7f54a4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54a4070fc0 0x7f54a4073480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f54b80ab000 tx=0x7f54b8019a50 comp rx=0 tx=0).stop 2024-09-10T21:30:51.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f6060 msgr2=0x559ea490f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 --2- 172.21.15.97:0/3146561157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ea48f6060 0x559ea490f450 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f54b000a6c0 tx=0x7f54b000ab80 comp rx=0 tx=0).stop 2024-09-10T21:30:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54c9800640 1 -- 172.21.15.97:0/3146561157 reap_dead start 2024-09-10T21:30:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 shutdown_connections 2024-09-10T21:30:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 >> 172.21.15.97:0/3146561157 conn(0x559ea483a140 msgr2=0x559ea4911b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 shutdown_connections 2024-09-10T21:30:51.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:51.932+0000 7f54cbbe8080 1 -- 172.21.15.97:0/3146561157 wait complete. 2024-09-10T21:30:51.941 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2024-09-10T21:30:52.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 Processor -- start 2024-09-10T21:30:52.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 -- start start 2024-09-10T21:30:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d96d0 0x55cc6d7da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d4c70 0x55cc6d7d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d64ea00 con 0x7f007c003640 2024-09-10T21:30:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d6acf40 con 0x55cc6d7d4c70 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0099a48080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cc6d7bb920 con 0x55cc6d7d96d0 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60588/0 (socket says 172.21.15.97:60588) 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 -- 172.21.15.97:0/2154303413 learned_addr learned my addr 172.21.15.97:0/2154303413 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 -- 172.21.15.97:0/2154303413 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d96d0 msgr2=0x55cc6d7da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 --2- 172.21.15.97:0/2154303413 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d96d0 0x55cc6d7da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 -- 172.21.15.97:0/2154303413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d4c70 msgr2=0x55cc6d7d5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 --2- 172.21.15.97:0/2154303413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d4c70 0x55cc6d7d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 -- 172.21.15.97:0/2154303413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6d627c10 con 0x7f007c003640 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f0096000640 1 --2- 172.21.15.97:0/2154303413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d6e70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f008800b7a0 tx=0x7f008800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8537465f4aff48a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f008fe00640 1 -- 172.21.15.97:0/2154303413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008801a070 con 0x7f007c003640 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f008fe00640 1 -- 172.21.15.97:0/2154303413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f008800adf0 con 0x7f007c003640 2024-09-10T21:30:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.028+0000 7f008fe00640 1 -- 172.21.15.97:0/2154303413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088008680 con 0x7f007c003640 2024-09-10T21:30:52.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- 172.21.15.97:0/2154303413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 msgr2=0x55cc6d7d6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 --2- 172.21.15.97:0/2154303413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d6e70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f008800b7a0 tx=0x7f008800bc60 comp rx=0 tx=0).stop 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- 172.21.15.97:0/2154303413 shutdown_connections 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 --2- 172.21.15.97:0/2154303413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d4c70 0x55cc6d7d5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 --2- 172.21.15.97:0/2154303413 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d96d0 0x55cc6d7da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 --2- 172.21.15.97:0/2154303413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d6e70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- 172.21.15.97:0/2154303413 >> 172.21.15.97:0/2154303413 conn(0x55cc6d72e520 msgr2=0x55cc6d7d6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- 172.21.15.97:0/2154303413 shutdown_connections 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- 172.21.15.97:0/2154303413 wait complete. 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 Processor -- start 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- start start 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d4c70 0x55cc6d7332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d96d0 0x55cc6d7b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d7000c0 con 0x7f007c003640 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d7d8500 con 0x55cc6d7d96d0 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cc6d700360 con 0x55cc6d7d4c70 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60590/0 (socket says 172.21.15.97:60590) 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 -- 172.21.15.97:0/2092753083 learned_addr learned my addr 172.21.15.97:0/2092753083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096000640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d4c70 0x55cc6d7332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0095600640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d96d0 0x55cc6d7b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 -- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d4c70 msgr2=0x55cc6d7332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cc6d7d4c70 0x55cc6d7332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 -- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d96d0 msgr2=0x55cc6d7b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cc6d7d96d0 0x55cc6d7b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 -- 172.21.15.97:0/2092753083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6d7dc420 con 0x7f007c003640 2024-09-10T21:30:52.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0096a00640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d44d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0090009040 tx=0x7f0090009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f008ea00640 1 -- 172.21.15.97:0/2092753083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084004e40 con 0x7f007c003640 2024-09-10T21:30:52.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f008ea00640 1 -- 172.21.15.97:0/2092753083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00900b8070 con 0x7f007c003640 2024-09-10T21:30:52.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cc6d7dc6c0 con 0x7f007c003640 2024-09-10T21:30:52.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f008ea00640 1 -- 172.21.15.97:0/2092753083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090001fe0 con 0x7f007c003640 2024-09-10T21:30:52.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.032+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6d7dcc00 con 0x7f007c003640 2024-09-10T21:30:52.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f008ea00640 1 -- 172.21.15.97:0/2092753083 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0090007590 con 0x7f007c003640 2024-09-10T21:30:52.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f008ea00640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0078070fc0 0x7f0078073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f008ea00640 1 -- 172.21.15.97:0/2092753083 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f009012c950 con 0x7f007c003640 2024-09-10T21:30:52.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f0096000640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0078070fc0 0x7f0078073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f0099a48080 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cc6d7db170 0x55cc6d7db550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55cc6d7dba90 con 0x55cc6d7db170 2024-09-10T21:30:52.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f0095600640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cc6d7db170 0x55cc6d7db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f0096000640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0078070fc0 0x7f0078073480 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0088005010 tx=0x7f0088003030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.036+0000 7f0095600640 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cc6d7db170 0x55cc6d7db550 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0095600640 1 -- 172.21.15.97:0/2092753083 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f008400baf0 con 0x55cc6d7db170 2024-09-10T21:30:52.042 INFO:teuthology.orchestra.run.smithi097.stdout:header (14 bytes) : 2024-09-10T21:30:52.042 INFO:teuthology.orchestra.run.smithi097.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2024-09-10T21:30:52.042 INFO:teuthology.orchestra.run.smithi097.stdout:0000000e 2024-09-10T21:30:52.042 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:52.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cc6d7db170 msgr2=0x55cc6d7db550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cc6d7db170 0x55cc6d7db550 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0078070fc0 msgr2=0x7f0078073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0078070fc0 0x7f0078073480 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0088005010 tx=0x7f0088003030 comp rx=0 tx=0).stop 2024-09-10T21:30:52.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 msgr2=0x55cc6d7d44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 --2- 172.21.15.97:0/2092753083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007c003640 0x55cc6d7d44d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0090009040 tx=0x7f0090009a60 comp rx=0 tx=0).stop 2024-09-10T21:30:52.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0096a00640 1 -- 172.21.15.97:0/2092753083 reap_dead start 2024-09-10T21:30:52.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 shutdown_connections 2024-09-10T21:30:52.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 >> 172.21.15.97:0/2092753083 conn(0x55cc6d72e520 msgr2=0x55cc6d7b84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 shutdown_connections 2024-09-10T21:30:52.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.040+0000 7f0099a48080 1 -- 172.21.15.97:0/2092753083 wait complete. 2024-09-10T21:30:52.047 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2024-09-10T21:30:52.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 Processor -- start 2024-09-10T21:30:52.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- start start 2024-09-10T21:30:52.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 0x55923b573ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b556b00 0x55923b574c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b5572a0 0x55923b55af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55923b3eba00 con 0x55923b5572a0 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55923b449f40 con 0x55923b556b00 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55923b559080 con 0x55923b4cc940 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b5572a0 0x55923b55af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b5572a0 0x55923b55af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60594/0 (socket says 172.21.15.97:60594) 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 -- 172.21.15.97:0/426361486 learned_addr learned my addr 172.21.15.97:0/426361486 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cec00640 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 0x55923b573ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83ce200640 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b556b00 0x55923b574c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 -- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 msgr2=0x55923b573ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 0x55923b573ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 -- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b556b00 msgr2=0x55923b574c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b556b00 0x55923b574c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 -- 172.21.15.97:0/426361486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55923b3c4c10 con 0x55923b5572a0 2024-09-10T21:30:52.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83ce200640 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b556b00 0x55923b574c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:52.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cf600640 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b5572a0 0x55923b55af80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f83c800a550 tx=0x7f83c80181c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f6b57fa6341c30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cce00640 1 -- 172.21.15.97:0/426361486 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80ba020 con 0x55923b5572a0 2024-09-10T21:30:52.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cce00640 1 -- 172.21.15.97:0/426361486 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83c80b0040 con 0x55923b5572a0 2024-09-10T21:30:52.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83cce00640 1 -- 172.21.15.97:0/426361486 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80af070 con 0x55923b5572a0 2024-09-10T21:30:52.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- 172.21.15.97:0/426361486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b5572a0 msgr2=0x55923b55af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b5572a0 0x55923b55af80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f83c800a550 tx=0x7f83c80181c0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- 172.21.15.97:0/426361486 shutdown_connections 2024-09-10T21:30:52.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b5572a0 0x55923b55af80 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b556b00 0x55923b574c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 --2- 172.21.15.97:0/426361486 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 0x55923b573ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- 172.21.15.97:0/426361486 >> 172.21.15.97:0/426361486 conn(0x55923b4cb520 msgr2=0x55923b5733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.136+0000 7f83d1a47080 1 -- 172.21.15.97:0/426361486 shutdown_connections 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 -- 172.21.15.97:0/426361486 wait complete. 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 Processor -- start 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 -- start start 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 0x55923b572290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b556b00 0x55923b55a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b5572a0 0x55923b55a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55923b4cd940 con 0x55923b556b00 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55923b49d4b0 con 0x55923b5572a0 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55923b4cdbe0 con 0x55923b4cc940 2024-09-10T21:30:52.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b556b00 0x55923b55a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b556b00 0x55923b55a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60600/0 (socket says 172.21.15.97:60600) 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 -- 172.21.15.97:0/2577270832 learned_addr learned my addr 172.21.15.97:0/2577270832 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83cf600640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b5572a0 0x55923b55a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83cec00640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 0x55923b572290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 -- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 msgr2=0x55923b572290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55923b4cc940 0x55923b572290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 -- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b5572a0 msgr2=0x55923b55a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55923b5572a0 0x55923b55a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 -- 172.21.15.97:0/2577270832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55923b54da60 con 0x55923b556b00 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83ce200640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b556b00 0x55923b55a060 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00df90 tx=0x7f83bc00b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83c7400640 1 -- 172.21.15.97:0/2577270832 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc014020 con 0x55923b556b00 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83c7400640 1 -- 172.21.15.97:0/2577270832 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83bc0015e0 con 0x55923b556b00 2024-09-10T21:30:52.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83c7400640 1 -- 172.21.15.97:0/2577270832 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc00a040 con 0x55923b556b00 2024-09-10T21:30:52.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55923b54dd00 con 0x55923b556b00 2024-09-10T21:30:52.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.140+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55923b54e240 con 0x55923b556b00 2024-09-10T21:30:52.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83c7400640 1 -- 172.21.15.97:0/2577270832 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f83bc0019d0 con 0x55923b556b00 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83c7400640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f83b0070fc0 0x7f83b0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83c7400640 1 -- 172.21.15.97:0/2577270832 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f83bc092280 con 0x55923b556b00 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83d1a47080 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55923b5727d0 0x55923b49c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83cec00640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f83b0070fc0 0x7f83b0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55923b54d550 con 0x55923b5727d0 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83cf600640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55923b5727d0 0x55923b49c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83cec00640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f83b0070fc0 0x7f83b0073480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f83c00044c0 tx=0x7f83c000f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.144+0000 7f83cf600640 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55923b5727d0 0x55923b49c2f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.150 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:52.150 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83cf600640 1 -- 172.21.15.97:0/2577270832 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f83c8002050 con 0x55923b5727d0 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55923b5727d0 msgr2=0x55923b49c2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55923b5727d0 0x55923b49c2f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f83b0070fc0 msgr2=0x7f83b0073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f83b0070fc0 0x7f83b0073480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f83c00044c0 tx=0x7f83c000f040 comp rx=0 tx=0).stop 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b556b00 msgr2=0x55923b55a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 --2- 172.21.15.97:0/2577270832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55923b556b00 0x55923b55a060 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00df90 tx=0x7f83bc00b680 comp rx=0 tx=0).stop 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83cf600640 1 -- 172.21.15.97:0/2577270832 reap_dead start 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 shutdown_connections 2024-09-10T21:30:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 >> 172.21.15.97:0/2577270832 conn(0x55923b4cb520 msgr2=0x55923b54f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 shutdown_connections 2024-09-10T21:30:52.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.148+0000 7f83d1a47080 1 -- 172.21.15.97:0/2577270832 wait complete. 2024-09-10T21:30:52.155 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2024-09-10T21:30:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 Processor -- start 2024-09-10T21:30:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 -- start start 2024-09-10T21:30:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 0x55c759c87540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 0x55c759d134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c759b8ba00 con 0x55c759cf5cc0 2024-09-10T21:30:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c759be9f40 con 0x55c759cfb1d0 2024-09-10T21:30:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2d643080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c759cf9710 con 0x55c759cf7a00 2024-09-10T21:30:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60612/0 (socket says 172.21.15.97:60612) 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 -- 172.21.15.97:0/727067671 learned_addr learned my addr 172.21.15.97:0/727067671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2b200640 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 0x55c759d134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c29e00640 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 0x55c759c87540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 -- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 msgr2=0x55c759c87540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 0x55c759c87540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 -- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 msgr2=0x55c759d134a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 0x55c759d134a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.196+0000 7f9c2a800640 1 -- 172.21.15.97:0/727067671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c759b64c10 con 0x55c759cf5cc0 2024-09-10T21:30:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3c140 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c006f10 tx=0x7f9c1c00b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae40f3eb9b9ac481 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c28a00640 1 -- 172.21.15.97:0/727067671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00e840 con 0x55c759cf5cc0 2024-09-10T21:30:52.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c28a00640 1 -- 172.21.15.97:0/727067671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c1c00ead0 con 0x55c759cf5cc0 2024-09-10T21:30:52.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c28a00640 1 -- 172.21.15.97:0/727067671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00d390 con 0x55c759cf5cc0 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- 172.21.15.97:0/727067671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 msgr2=0x55c759c3c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3c140 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c006f10 tx=0x7f9c1c00b680 comp rx=0 tx=0).stop 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- 172.21.15.97:0/727067671 shutdown_connections 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 0x55c759d134a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 0x55c759c87540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 --2- 172.21.15.97:0/727067671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3c140 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- 172.21.15.97:0/727067671 >> 172.21.15.97:0/727067671 conn(0x55c759c850d0 msgr2=0x55c759d112f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- 172.21.15.97:0/727067671 shutdown_connections 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- 172.21.15.97:0/727067671 wait complete. 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 Processor -- start 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- start start 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 0x55c759dac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 0x55c759d11210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c759c88050 con 0x55c759cf5cc0 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c759d14cc0 con 0x55c759cfb1d0 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2d643080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c759c6ca20 con 0x55c759cf7a00 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60622/0 (socket says 172.21.15.97:60622) 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 -- 172.21.15.97:0/6656167 learned_addr learned my addr 172.21.15.97:0/6656167 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c29e00640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 0x55c759dac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2b200640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 0x55c759d11210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 -- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 msgr2=0x55c759dac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c759cf7a00 0x55c759dac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 -- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 msgr2=0x55c759d11210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c759cfb1d0 0x55c759d11210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 -- 172.21.15.97:0/6656167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c759c3f5d0 con 0x55c759cf5cc0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c2a800640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3f090 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c017620 tx=0x7f9c1c017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.200+0000 7f9c23400640 1 -- 172.21.15.97:0/6656167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c017980 con 0x55c759cf5cc0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c23400640 1 -- 172.21.15.97:0/6656167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c1c017b20 con 0x55c759cf5cc0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c23400640 1 -- 172.21.15.97:0/6656167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c021420 con 0x55c759cf5cc0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c759c3f840 con 0x55c759cf5cc0 2024-09-10T21:30:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c759c3fd80 con 0x55c759cf5cc0 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c23400640 1 -- 172.21.15.97:0/6656167 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9c1c02a440 con 0x55c759cf5cc0 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c23400640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9c0c070fc0 0x7f9c0c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c23400640 1 -- 172.21.15.97:0/6656167 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f9c1c093710 con 0x55c759cf5cc0 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c2d643080 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c759dacc50 0x55c759dad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55c759dadd00 con 0x55c759dacc50 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c29e00640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9c0c070fc0 0x7f9c0c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.204+0000 7f9c2b200640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c759dacc50 0x55c759dad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2b200640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c759dacc50 0x55c759dad7c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c29e00640 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9c0c070fc0 0x7f9c0c073480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9c180ab000 tx=0x7f9c18019a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2b200640 1 -- 172.21.15.97:0/6656167 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c24002040 con 0x55c759dacc50 2024-09-10T21:30:52.211 INFO:teuthology.orchestra.run.smithi097.stdout:header (0 bytes) : 2024-09-10T21:30:52.212 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:52.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c759dacc50 msgr2=0x55c759dad7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c759dacc50 0x55c759dad7c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9c0c070fc0 msgr2=0x7f9c0c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9c0c070fc0 0x7f9c0c073480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9c180ab000 tx=0x7f9c18019a20 comp rx=0 tx=0).stop 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 msgr2=0x55c759c3f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 --2- 172.21.15.97:0/6656167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c759cf5cc0 0x55c759c3f090 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c017620 tx=0x7f9c1c017650 comp rx=0 tx=0).stop 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2b200640 1 -- 172.21.15.97:0/6656167 reap_dead start 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 shutdown_connections 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 >> 172.21.15.97:0/6656167 conn(0x55c759c850d0 msgr2=0x55c759cf7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 shutdown_connections 2024-09-10T21:30:52.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.208+0000 7f9c2d643080 1 -- 172.21.15.97:0/6656167 wait complete. 2024-09-10T21:30:52.215 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2024-09-10T21:30:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 Processor -- start 2024-09-10T21:30:52.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- start start 2024-09-10T21:30:52.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c002ac0 0x55a8303d5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303d9110 0x55a8303d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a8303b74d0 0x55a8303d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a8302abf40 con 0x7f229c002ac0 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a8302b8d00 con 0x55a8303b74d0 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55a83024da00 con 0x55a8303d9110 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c002ac0 0x55a8303d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c002ac0 0x55a8303d5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60636/0 (socket says 172.21.15.97:60636) 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 -- 172.21.15.97:0/918221614 learned_addr learned my addr 172.21.15.97:0/918221614 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b6600640 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303d9110 0x55a8303d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7a00640 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a8303b74d0 0x55a8303d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 -- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303d9110 msgr2=0x55a8303d4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303d9110 0x55a8303d4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 -- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a8303b74d0 msgr2=0x55a8303d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a8303b74d0 0x55a8303d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 -- 172.21.15.97:0/918221614 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a830349450 con 0x7f229c002ac0 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7a00640 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a8303b74d0 0x55a8303d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b7000640 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c002ac0 0x55a8303d5790 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f22a8001760 tx=0x7f22a8009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77876497d59e9eca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b5200640 1 -- 172.21.15.97:0/918221614 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8013070 con 0x7f229c002ac0 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b5200640 1 -- 172.21.15.97:0/918221614 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22a800f8e0 con 0x7f229c002ac0 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b5200640 1 -- 172.21.15.97:0/918221614 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a800fd30 con 0x7f229c002ac0 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- 172.21.15.97:0/918221614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c002ac0 msgr2=0x55a8303d5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c002ac0 0x55a8303d5790 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f22a8001760 tx=0x7f22a8009970 comp rx=0 tx=0).stop 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- 172.21.15.97:0/918221614 shutdown_connections 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a8303b74d0 0x55a8303d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303d9110 0x55a8303d4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 --2- 172.21.15.97:0/918221614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f229c002ac0 0x55a8303d5790 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- 172.21.15.97:0/918221614 >> 172.21.15.97:0/918221614 conn(0x55a8303470d0 msgr2=0x55a8303d9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- 172.21.15.97:0/918221614 shutdown_connections 2024-09-10T21:30:52.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.304+0000 7f22b9d19080 1 -- 172.21.15.97:0/918221614 wait complete. 2024-09-10T21:30:52.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 Processor -- start 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 -- start start 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303b74d0 0x55a830346320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8303d9110 0x55a8303d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229c002ac0 0x55a8303d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a8303d5050 con 0x55a8303d9110 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a8303496f0 con 0x7f229c002ac0 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55a8303b6ad0 con 0x55a8303b74d0 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8303d9110 0x55a8303d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8303d9110 0x55a8303d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60644/0 (socket says 172.21.15.97:60644) 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 -- 172.21.15.97:0/1438909067 learned_addr learned my addr 172.21.15.97:0/1438909067 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b7a00640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229c002ac0 0x55a8303d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b7000640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303b74d0 0x55a830346320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 -- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303b74d0 msgr2=0x55a830346320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a8303b74d0 0x55a830346320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 -- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229c002ac0 msgr2=0x55a8303d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f229c002ac0 0x55a8303d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 -- 172.21.15.97:0/1438909067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a83046c4c0 con 0x55a8303d9110 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b6600640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8303d9110 0x55a8303d1ca0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f22a401f990 tx=0x7f22a40043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22af400640 1 -- 172.21.15.97:0/1438909067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a40c0070 con 0x55a8303d9110 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22af400640 1 -- 172.21.15.97:0/1438909067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22a40bc480 con 0x55a8303d9110 2024-09-10T21:30:52.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a83046c760 con 0x55a8303d9110 2024-09-10T21:30:52.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22af400640 1 -- 172.21.15.97:0/1438909067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a40bc8d0 con 0x55a8303d9110 2024-09-10T21:30:52.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.308+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83046cca0 con 0x55a8303d9110 2024-09-10T21:30:52.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22af400640 1 -- 172.21.15.97:0/1438909067 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f22a40bca70 con 0x55a8303d9110 2024-09-10T21:30:52.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22af400640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2298070ff0 0x7f22980734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22af400640 1 -- 172.21.15.97:0/1438909067 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f22a4131100 con 0x55a8303d9110 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a830346860 0x55a83046ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b7000640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2298070ff0 0x7f22980734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55a83046f2c0 con 0x55a830346860 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b7a00640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a830346860 0x55a83046ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b7000640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2298070ff0 0x7f22980734b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f22a8001ec0 tx=0x7f22a8003040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b7a00640 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a830346860 0x55a83046ed80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b7a00640 1 -- 172.21.15.97:0/1438909067 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f22b0008aa0 con 0x55a830346860 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:header (274 bytes) : 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000000 04 03 0c 01 00 00 0d 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000020 00 00 00 00 88 ba e0 66 3b 74 28 31 00 00 00 00 |.......f;t(1....| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000050 00 00 88 ba e0 66 3b 74 28 31 00 00 00 00 00 00 |.....f;t(1......| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:52.319 INFO:teuthology.orchestra.run.smithi097.stdout:000000a0 00 00 00 00 00 00 00 00 88 ba e0 66 18 aa 80 39 |...........f...9| 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 |................| 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:000000e0 00 00 00 00 00 00 88 ba e0 66 3b 74 28 31 00 00 |.........f;t(1..| 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:* 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:00000110 00 00 |..| 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout:00000112 2024-09-10T21:30:52.320 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a830346860 msgr2=0x55a83046ed80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a830346860 0x55a83046ed80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2298070ff0 msgr2=0x7f22980734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2298070ff0 0x7f22980734b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f22a8001ec0 tx=0x7f22a8003040 comp rx=0 tx=0).stop 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8303d9110 msgr2=0x55a8303d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 --2- 172.21.15.97:0/1438909067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8303d9110 0x55a8303d1ca0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f22a401f990 tx=0x7f22a40043f0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b7a00640 1 -- 172.21.15.97:0/1438909067 reap_dead start 2024-09-10T21:30:52.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 shutdown_connections 2024-09-10T21:30:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 >> 172.21.15.97:0/1438909067 conn(0x55a8303470d0 msgr2=0x55a8303d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 shutdown_connections 2024-09-10T21:30:52.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.312+0000 7f22b9d19080 1 -- 172.21.15.97:0/1438909067 wait complete. 2024-09-10T21:30:52.323 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-09-10T21:30:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 Processor -- start 2024-09-10T21:30:52.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- start start 2024-09-10T21:30:52.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366efcc360 0x56366f0a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effaec0 0x56366f0aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366effb2a0 0x56366f0aa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56366ef18a00 con 0x56366effb2a0 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56366ef76f40 con 0x56366efcc360 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56366f089560 con 0x56366effaec0 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366effb2a0 0x56366f0aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366effb2a0 0x56366f0aa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60652/0 (socket says 172.21.15.97:60652) 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 -- 172.21.15.97:0/3218866906 learned_addr learned my addr 172.21.15.97:0/3218866906 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1cc00640 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366efcc360 0x56366f0a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d17e00640 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effaec0 0x56366f0aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 -- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effaec0 msgr2=0x56366f0aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effaec0 0x56366f0aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 -- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366efcc360 msgr2=0x56366f0a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366efcc360 0x56366f0a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 -- 172.21.15.97:0/3218866906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366eef1c10 con 0x56366effb2a0 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1d600640 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366effb2a0 0x56366f0aa750 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9d18001680 tx=0x7f9d18001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f67b6530667cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d16a00640 1 -- 172.21.15.97:0/3218866906 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18017070 con 0x56366effb2a0 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d16a00640 1 -- 172.21.15.97:0/3218866906 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d18002790 con 0x56366effb2a0 2024-09-10T21:30:52.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d16a00640 1 -- 172.21.15.97:0/3218866906 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18002be0 con 0x56366effb2a0 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- 172.21.15.97:0/3218866906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366effb2a0 msgr2=0x56366f0aa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366effb2a0 0x56366f0aa750 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9d18001680 tx=0x7f9d18001b40 comp rx=0 tx=0).stop 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- 172.21.15.97:0/3218866906 shutdown_connections 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366effb2a0 0x56366f0aa750 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effaec0 0x56366f0aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 --2- 172.21.15.97:0/3218866906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366efcc360 0x56366f0a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- 172.21.15.97:0/3218866906 >> 172.21.15.97:0/3218866906 conn(0x56366f0120d0 msgr2=0x56366eff9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.412+0000 7f9d1f970080 1 -- 172.21.15.97:0/3218866906 shutdown_connections 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 -- 172.21.15.97:0/3218866906 wait complete. 2024-09-10T21:30:52.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 Processor -- start 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 -- start start 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366efcc360 0x56366f0914e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366effaec0 0x56366f010a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effb2a0 0x56366f010f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56366f085300 con 0x56366efcc360 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56366f00f690 con 0x56366effaec0 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56366f0855a0 con 0x56366effb2a0 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366efcc360 0x56366f0914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366efcc360 0x56366f0914e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60662/0 (socket says 172.21.15.97:60662) 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 -- 172.21.15.97:0/2361907942 learned_addr learned my addr 172.21.15.97:0/2361907942 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d17e00640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366effaec0 0x56366f010a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1d600640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effb2a0 0x56366f010f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 -- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effb2a0 msgr2=0x56366f010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56366effb2a0 0x56366f010f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 -- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366effaec0 msgr2=0x56366f010a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56366effaec0 0x56366f010a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 -- 172.21.15.97:0/2361907942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366f0a3f40 con 0x56366efcc360 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1cc00640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366efcc360 0x56366f0914e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c10 tx=0x7f9d100acbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d15600640 1 -- 172.21.15.97:0/2361907942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100b8440 con 0x56366efcc360 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d15600640 1 -- 172.21.15.97:0/2361907942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d100b86d0 con 0x56366efcc360 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56366f0a41e0 con 0x56366efcc360 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d15600640 1 -- 172.21.15.97:0/2361907942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d100b88c0 con 0x56366efcc360 2024-09-10T21:30:52.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.416+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366f0a4720 con 0x56366efcc360 2024-09-10T21:30:52.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d15600640 1 -- 172.21.15.97:0/2361907942 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d100b8a60 con 0x56366efcc360 2024-09-10T21:30:52.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d15600640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cfc070fc0 0x7f9cfc073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d15600640 1 -- 172.21.15.97:0/2361907942 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f9d1012da60 con 0x56366efcc360 2024-09-10T21:30:52.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d1f970080 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56366f0a4e60 0x56366f0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d17e00640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cfc070fc0 0x7f9cfc073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x56366f0a5780 con 0x56366f0a4e60 2024-09-10T21:30:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d1d600640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56366f0a4e60 0x56366f0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d17e00640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cfc070fc0 0x7f9cfc073480 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b8c0 tx=0x7f9d08004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.420+0000 7f9d1d600640 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56366f0a4e60 0x56366f0a5240 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1d600640 1 -- 172.21.15.97:0/2361907942 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv10 ondisk = 0) ==== 164+0+21 (crc 0 0 0) 0x7f9d1800f070 con 0x56366f0a4e60 2024-09-10T21:30:52.426 INFO:teuthology.orchestra.run.smithi097.stdout:sixmegs_head 2024-09-10T21:30:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56366f0a4e60 msgr2=0x56366f0a5240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56366f0a4e60 0x56366f0a5240 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cfc070fc0 msgr2=0x7f9cfc073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cfc070fc0 0x7f9cfc073480 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b8c0 tx=0x7f9d08004890 comp rx=0 tx=0).stop 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366efcc360 msgr2=0x56366f0914e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 --2- 172.21.15.97:0/2361907942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56366efcc360 0x56366f0914e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c10 tx=0x7f9d100acbc0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1d600640 1 -- 172.21.15.97:0/2361907942 reap_dead start 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 shutdown_connections 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 >> 172.21.15.97:0/2361907942 conn(0x56366f0120d0 msgr2=0x56366f0ab030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 shutdown_connections 2024-09-10T21:30:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.424+0000 7f9d1f970080 1 -- 172.21.15.97:0/2361907942 wait complete. 2024-09-10T21:30:52.430 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2024-09-10T21:30:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 Processor -- start 2024-09-10T21:30:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 -- start start 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb939940 0x5626bb9e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c3b00 0x5626bb9c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb9c42a0 0x5626bb9c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5626bb858a00 con 0x5626bb939940 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5626bb8b6f40 con 0x5626bb9c42a0 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f410182b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5626bb9c6080 con 0x5626bb9c3b00 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb939940 0x5626bb9e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb939940 0x5626bb9e00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60670/0 (socket says 172.21.15.97:60670) 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fd000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c3b00 0x5626bb9c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fda00640 1 -- 172.21.15.97:0/4246208235 learned_addr learned my addr 172.21.15.97:0/4246208235 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fe400640 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb9c42a0 0x5626bb9c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fd000640 1 -- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb9c42a0 msgr2=0x5626bb9c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fd000640 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb9c42a0 0x5626bb9c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fd000640 1 -- 172.21.15.97:0/4246208235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb939940 msgr2=0x5626bb9e00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fd000640 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb939940 0x5626bb9e00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fd000640 1 -- 172.21.15.97:0/4246208235 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bb831c10 con 0x5626bb9c3b00 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.472+0000 7f40fda00640 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb939940 0x5626bb9e00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c3b00 0x5626bb9c7f80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f40ec001d80 tx=0x7f40ec00b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e64f475724831e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40f7400640 1 -- 172.21.15.97:0/4246208235 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec012070 con 0x5626bb9c3b00 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40f7400640 1 -- 172.21.15.97:0/4246208235 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40ec00e930 con 0x5626bb9c3b00 2024-09-10T21:30:52.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40f7400640 1 -- 172.21.15.97:0/4246208235 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00ed80 con 0x5626bb9c3b00 2024-09-10T21:30:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c3b00 msgr2=0x5626bb9c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c3b00 0x5626bb9c7f80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f40ec001d80 tx=0x7f40ec00b680 comp rx=0 tx=0).stop 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- 172.21.15.97:0/4246208235 shutdown_connections 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb9c42a0 0x5626bb9c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c3b00 0x5626bb9c7f80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 --2- 172.21.15.97:0/4246208235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb939940 0x5626bb9e00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- 172.21.15.97:0/4246208235 >> 172.21.15.97:0/4246208235 conn(0x5626bb938520 msgr2=0x5626bb909da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- 172.21.15.97:0/4246208235 shutdown_connections 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- 172.21.15.97:0/4246208235 wait complete. 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 Processor -- start 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- start start 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb939940 0x5626bb93d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb9c3b00 0x5626bb9db6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c42a0 0x5626bb93c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5626bb9e36c0 con 0x5626bb9c3b00 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5626bb9e3960 con 0x5626bb939940 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5626bb9e1f80 con 0x5626bb9c42a0 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb9c3b00 0x5626bb9db6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb9c3b00 0x5626bb9db6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60674/0 (socket says 172.21.15.97:60674) 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 -- 172.21.15.97:0/3107654733 learned_addr learned my addr 172.21.15.97:0/3107654733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 -- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c42a0 msgr2=0x5626bb93c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626bb9c42a0 0x5626bb93c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 -- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb939940 msgr2=0x5626bb93d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626bb939940 0x5626bb93d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 -- 172.21.15.97:0/3107654733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bb9ddca0 con 0x5626bb9c3b00 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40fd000640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb9c3b00 0x5626bb9db6d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00bb80 tx=0x7f40ec010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40f6000640 1 -- 172.21.15.97:0/3107654733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec012040 con 0x5626bb9c3b00 2024-09-10T21:30:52.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f40f6000640 1 -- 172.21.15.97:0/3107654733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40ec00ed30 con 0x5626bb9c3b00 2024-09-10T21:30:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.476+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5626bb9ddf40 con 0x5626bb9c3b00 2024-09-10T21:30:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f40f6000640 1 -- 172.21.15.97:0/3107654733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0102d0 con 0x5626bb9c3b00 2024-09-10T21:30:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626bb9dd9b0 con 0x5626bb9c3b00 2024-09-10T21:30:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f40f6000640 1 -- 172.21.15.97:0/3107654733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f40ec010470 con 0x5626bb9c3b00 2024-09-10T21:30:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f40f6000640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f40e0070ff0 0x7f40e00734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f40f6000640 1 -- 172.21.15.97:0/3107654733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f40ec093420 con 0x5626bb9c3b00 2024-09-10T21:30:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f40fda00640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f40e0070ff0 0x7f40e00734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f410182b080 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5626bb9decd0 0x5626bb9c1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5626bb9c17c0 con 0x5626bb9decd0 2024-09-10T21:30:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.480+0000 7f40fe400640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5626bb9decd0 0x5626bb9c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f40fda00640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f40e0070ff0 0x7f40e00734b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000c10 tx=0x7f40f0017e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f40fe400640 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5626bb9decd0 0x5626bb9c1280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f40fe400640 1 -- 172.21.15.97:0/3107654733 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f40f8002040 con 0x5626bb9decd0 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5626bb9decd0 msgr2=0x5626bb9c1280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5626bb9decd0 0x5626bb9c1280 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f40e0070ff0 msgr2=0x7f40e00734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f40e0070ff0 0x7f40e00734b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000c10 tx=0x7f40f0017e80 comp rx=0 tx=0).stop 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb9c3b00 msgr2=0x5626bb9db6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 --2- 172.21.15.97:0/3107654733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626bb9c3b00 0x5626bb9db6d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00bb80 tx=0x7f40ec010d70 comp rx=0 tx=0).stop 2024-09-10T21:30:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f40fe400640 1 -- 172.21.15.97:0/3107654733 reap_dead start 2024-09-10T21:30:52.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 shutdown_connections 2024-09-10T21:30:52.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 >> 172.21.15.97:0/3107654733 conn(0x5626bb938520 msgr2=0x5626bb9e24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 shutdown_connections 2024-09-10T21:30:52.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.484+0000 7f410182b080 1 -- 172.21.15.97:0/3107654733 wait complete. 2024-09-10T21:30:52.491 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2024-09-10T21:30:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 Processor -- start 2024-09-10T21:30:52.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- start start 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd1c002380 0x560079fb8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560079fdc990 0x560079f00d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560079eaff40 con 0x560079f02e60 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560079ebcd00 con 0x7efd1c002380 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560079e51a00 con 0x560079fdc990 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60688/0 (socket says 172.21.15.97:60688) 2024-09-10T21:30:52.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 -- 172.21.15.97:0/4211586944 learned_addr learned my addr 172.21.15.97:0/4211586944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd36800640 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd1c002380 0x560079fb8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd35e00640 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560079fdc990 0x560079f00d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 -- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560079fdc990 msgr2=0x560079f00d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560079fdc990 0x560079f00d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 -- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd1c002380 msgr2=0x560079fb8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd1c002380 0x560079fb8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 -- 172.21.15.97:0/4211586944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560079fd9ac0 con 0x560079f02e60 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd37200640 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f012d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efd30004fd0 tx=0x7efd30010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad3f7a439269e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd34a00640 1 -- 172.21.15.97:0/4211586944 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000bb40 con 0x560079f02e60 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd34a00640 1 -- 172.21.15.97:0/4211586944 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd3000bce0 con 0x560079f02e60 2024-09-10T21:30:52.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd34a00640 1 -- 172.21.15.97:0/4211586944 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30019650 con 0x560079f02e60 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- 172.21.15.97:0/4211586944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 msgr2=0x560079f012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f012d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efd30004fd0 tx=0x7efd30010ba0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- 172.21.15.97:0/4211586944 shutdown_connections 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f012d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560079fdc990 0x560079f00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 --2- 172.21.15.97:0/4211586944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd1c002380 0x560079fb8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- 172.21.15.97:0/4211586944 >> 172.21.15.97:0/4211586944 conn(0x560079f31520 msgr2=0x560079fd9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- 172.21.15.97:0/4211586944 shutdown_connections 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.576+0000 7efd395fc080 1 -- 172.21.15.97:0/4211586944 wait complete. 2024-09-10T21:30:52.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 Processor -- start 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 -- start start 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f36400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560079fdc990 0x560079fe2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efd1c002380 0x560079fd8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560079fdb2b0 con 0x560079f02e60 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560079f03420 con 0x560079fdc990 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560079fd9580 con 0x7efd1c002380 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f36400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f36400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60690/0 (socket says 172.21.15.97:60690) 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 -- 172.21.15.97:0/943680620 learned_addr learned my addr 172.21.15.97:0/943680620 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd37200640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efd1c002380 0x560079fd8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd35e00640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560079fdc990 0x560079fe2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 -- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efd1c002380 msgr2=0x560079fd8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efd1c002380 0x560079fd8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 -- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560079fdc990 msgr2=0x560079fe2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560079fdc990 0x560079fe2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 -- 172.21.15.97:0/943680620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560079f354e0 con 0x560079f02e60 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd37200640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efd1c002380 0x560079fd8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd35e00640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560079fdc990 0x560079fe2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd36800640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f36400 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efd28004040 tx=0x7efd28009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd2f400640 1 -- 172.21.15.97:0/943680620 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28001640 con 0x560079f02e60 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd2f400640 1 -- 172.21.15.97:0/943680620 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd280a49d0 con 0x560079f02e60 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd2f400640 1 -- 172.21.15.97:0/943680620 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd280a4e00 con 0x560079f02e60 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560079f35780 con 0x560079f02e60 2024-09-10T21:30:52.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.580+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560079f35cc0 con 0x560079f02e60 2024-09-10T21:30:52.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd2f400640 1 -- 172.21.15.97:0/943680620 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd280ad420 con 0x560079f02e60 2024-09-10T21:30:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd2f400640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd18070ff0 0x7efd180734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd2f400640 1 -- 172.21.15.97:0/943680620 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7efd2812b6d0 con 0x560079f02e60 2024-09-10T21:30:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd35e00640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd18070ff0 0x7efd180734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560079f35060 0x56007a0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x560079fd9100 con 0x560079f35060 2024-09-10T21:30:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd37200640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560079f35060 0x56007a0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd35e00640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd18070ff0 0x7efd180734b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efd24001fd0 tx=0x7efd24003040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd37200640 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560079f35060 0x56007a0741e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd37200640 1 -- 172.21.15.97:0/943680620 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7efd30008070 con 0x560079f35060 2024-09-10T21:30:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560079f35060 msgr2=0x56007a0741e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560079f35060 0x56007a0741e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd18070ff0 msgr2=0x7efd180734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd18070ff0 0x7efd180734b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efd24001fd0 tx=0x7efd24003040 comp rx=0 tx=0).stop 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 msgr2=0x560079f36400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 --2- 172.21.15.97:0/943680620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560079f02e60 0x560079f36400 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efd28004040 tx=0x7efd28009770 comp rx=0 tx=0).stop 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd37200640 1 -- 172.21.15.97:0/943680620 reap_dead start 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 shutdown_connections 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 >> 172.21.15.97:0/943680620 conn(0x560079f31520 msgr2=0x560079fb8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 shutdown_connections 2024-09-10T21:30:52.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.584+0000 7efd395fc080 1 -- 172.21.15.97:0/943680620 wait complete. 2024-09-10T21:30:52.594 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2024-09-10T21:30:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 Processor -- start 2024-09-10T21:30:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 -- start start 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890db73940 0x55890dc16a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfdb00 0x55890dc021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890dbfe2a0 0x55890db76df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55890da92a00 con 0x55890dbfdb00 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55890daf0f40 con 0x55890dbfe2a0 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84eb911080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55890dc00080 con 0x55890db73940 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfdb00 0x55890dc021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfdb00 0x55890dc021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60706/0 (socket says 172.21.15.97:60706) 2024-09-10T21:30:52.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 -- 172.21.15.97:0/2388609110 learned_addr learned my addr 172.21.15.97:0/2388609110 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e8c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890db73940 0x55890dc16a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e9600640 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890dbfe2a0 0x55890db76df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 -- 172.21.15.97:0/2388609110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890db73940 msgr2=0x55890dc16a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890db73940 0x55890dc16a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 -- 172.21.15.97:0/2388609110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890dbfe2a0 msgr2=0x55890db76df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890dbfe2a0 0x55890db76df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 -- 172.21.15.97:0/2388609110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890da6bc10 con 0x55890dbfdb00 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e3e00640 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfdb00 0x55890dc021f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f84d4001e70 tx=0x7f84d400b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b46c9a31bdfc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e2a00640 1 -- 172.21.15.97:0/2388609110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d400e840 con 0x55890dbfdb00 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e2a00640 1 -- 172.21.15.97:0/2388609110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84d400ea00 con 0x55890dbfdb00 2024-09-10T21:30:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.684+0000 7f84e2a00640 1 -- 172.21.15.97:0/2388609110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d400ee50 con 0x55890dbfdb00 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- 172.21.15.97:0/2388609110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfdb00 msgr2=0x55890dc021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfdb00 0x55890dc021f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f84d4001e70 tx=0x7f84d400b740 comp rx=0 tx=0).stop 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- 172.21.15.97:0/2388609110 shutdown_connections 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890dbfe2a0 0x55890db76df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfdb00 0x55890dc021f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 --2- 172.21.15.97:0/2388609110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890db73940 0x55890dc16a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- 172.21.15.97:0/2388609110 >> 172.21.15.97:0/2388609110 conn(0x55890db72520 msgr2=0x55890db43da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- 172.21.15.97:0/2388609110 shutdown_connections 2024-09-10T21:30:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- 172.21.15.97:0/2388609110 wait complete. 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 Processor -- start 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- start start 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890db73940 0x55890dc14d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890dbfe2a0 0x55890dbfa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfac80 0x55890db76d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55890dc1bf10 con 0x55890dbfac80 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55890dc1bc70 con 0x55890db73940 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55890dc1c1b0 con 0x55890dbfe2a0 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfac80 0x55890db76d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfac80 0x55890db76d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60710/0 (socket says 172.21.15.97:60710) 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 -- 172.21.15.97:0/3968298576 learned_addr learned my addr 172.21.15.97:0/3968298576 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e8c00640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890db73940 0x55890dc14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e3e00640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890dbfe2a0 0x55890dbfa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 -- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890dbfe2a0 msgr2=0x55890dbfa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890dbfe2a0 0x55890dbfa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 -- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890db73940 msgr2=0x55890dc14d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55890db73940 0x55890dc14d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 -- 172.21.15.97:0/3968298576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890dc1a080 con 0x55890dbfac80 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e3e00640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55890dbfe2a0 0x55890dbfa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e9600640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfac80 0x55890db76d80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f84e4010a90 tx=0x7f84e4010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e1600640 1 -- 172.21.15.97:0/3968298576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e400edf0 con 0x55890dbfac80 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e1600640 1 -- 172.21.15.97:0/3968298576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84e4019440 con 0x55890dbfac80 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84e1600640 1 -- 172.21.15.97:0/3968298576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4019890 con 0x55890dbfac80 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55890dc1a320 con 0x55890dbfac80 2024-09-10T21:30:52.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.688+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890dc1a860 con 0x55890dbfac80 2024-09-10T21:30:52.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84e1600640 1 -- 172.21.15.97:0/3968298576 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f84e4017070 con 0x55890dbfac80 2024-09-10T21:30:52.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84e1600640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c8070fc0 0x7f84c8073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84e1600640 1 -- 172.21.15.97:0/3968298576 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f84e40932f0 con 0x55890dbfac80 2024-09-10T21:30:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84e8c00640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c8070fc0 0x7f84c8073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84eb911080 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55890dc16dd0 0x55890dc171b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55890dc198d0 con 0x55890dc16dd0 2024-09-10T21:30:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84e3e00640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55890dc16dd0 0x55890dc171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.692+0000 7f84e8c00640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c8070fc0 0x7f84c8073480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0ab000 tx=0x7f84dc019920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84e3e00640 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55890dc16dd0 0x55890dc171b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84e3e00640 1 -- 172.21.15.97:0/3968298576 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f84d4014070 con 0x55890dc16dd0 2024-09-10T21:30:52.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55890dc16dd0 msgr2=0x55890dc171b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55890dc16dd0 0x55890dc171b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c8070fc0 msgr2=0x7f84c8073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84c8070fc0 0x7f84c8073480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0ab000 tx=0x7f84dc019920 comp rx=0 tx=0).stop 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfac80 msgr2=0x55890db76d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 --2- 172.21.15.97:0/3968298576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55890dbfac80 0x55890db76d80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f84e4010a90 tx=0x7f84e4010f50 comp rx=0 tx=0).stop 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84e9600640 1 -- 172.21.15.97:0/3968298576 reap_dead start 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 shutdown_connections 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 >> 172.21.15.97:0/3968298576 conn(0x55890db72520 msgr2=0x55890db43da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 shutdown_connections 2024-09-10T21:30:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.696+0000 7f84eb911080 1 -- 172.21.15.97:0/3968298576 wait complete. 2024-09-10T21:30:52.703 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2024-09-10T21:30:52.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 Processor -- start 2024-09-10T21:30:52.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- start start 2024-09-10T21:30:52.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 0x55ac52fa4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f8ca00 0x55ac52f1c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f901d0 0x55ac52f18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac52e7ef40 con 0x55ac52f8ca00 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ac52e8bd00 con 0x55ac52f901d0 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ac52e20a00 con 0x55ac52f8acc0 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f8ca00 0x55ac52f1c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f8ca00 0x55ac52f1c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60728/0 (socket says 172.21.15.97:60728) 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 -- 172.21.15.97:0/2307676654 learned_addr learned my addr 172.21.15.97:0/2307676654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39aa00640 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 0x55ac52fa4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39b400640 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f901d0 0x55ac52f18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 -- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 msgr2=0x55ac52fa4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 0x55ac52fa4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 -- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f901d0 msgr2=0x55ac52f18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f901d0 0x55ac52f18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 -- 172.21.15.97:0/2307676654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac52f03480 con 0x55ac52f8ca00 2024-09-10T21:30:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39a000640 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f8ca00 0x55ac52f1c540 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b2d0 tx=0x7fb38800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e595f0261761bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb398c00640 1 -- 172.21.15.97:0/2307676654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388013030 con 0x55ac52f8ca00 2024-09-10T21:30:52.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb398c00640 1 -- 172.21.15.97:0/2307676654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb38800f620 con 0x55ac52f8ca00 2024-09-10T21:30:52.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb398c00640 1 -- 172.21.15.97:0/2307676654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38800a040 con 0x55ac52f8ca00 2024-09-10T21:30:52.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- 172.21.15.97:0/2307676654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f8ca00 msgr2=0x55ac52f1c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f8ca00 0x55ac52f1c540 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b2d0 tx=0x7fb38800b790 comp rx=0 tx=0).stop 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- 172.21.15.97:0/2307676654 shutdown_connections 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f901d0 0x55ac52f18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f8ca00 0x55ac52f1c540 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 --2- 172.21.15.97:0/2307676654 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 0x55ac52fa4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- 172.21.15.97:0/2307676654 >> 172.21.15.97:0/2307676654 conn(0x55ac52f1a0d0 msgr2=0x55ac52f8a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- 172.21.15.97:0/2307676654 shutdown_connections 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.744+0000 7fb39d764080 1 -- 172.21.15.97:0/2307676654 wait complete. 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 Processor -- start 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 -- start start 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 0x55ac53043ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f8ca00 0x55ac52fa6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f901d0 0x55ac52fb1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac52f18100 con 0x55ac52f901d0 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ac52f17e60 con 0x55ac52f8ca00 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ac52f183a0 con 0x55ac52f8acc0 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f901d0 0x55ac52fb1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f901d0 0x55ac52fb1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60736/0 (socket says 172.21.15.97:60736) 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 -- 172.21.15.97:0/2960067753 learned_addr learned my addr 172.21.15.97:0/2960067753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39aa00640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 0x55ac53043ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39a000640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f8ca00 0x55ac52fa6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 -- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 msgr2=0x55ac53043ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ac52f8acc0 0x55ac53043ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 -- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f8ca00 msgr2=0x55ac52fa6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ac52f8ca00 0x55ac52fa6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 -- 172.21.15.97:0/2960067753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac53044740 con 0x55ac52f901d0 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39b400640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f901d0 0x55ac52fb1220 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb39400d390 tx=0x7fb39400d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb393400640 1 -- 172.21.15.97:0/2960067753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394006020 con 0x55ac52f901d0 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb393400640 1 -- 172.21.15.97:0/2960067753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3940062b0 con 0x55ac52f901d0 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac530449e0 con 0x55ac52f901d0 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb393400640 1 -- 172.21.15.97:0/2960067753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940125f0 con 0x55ac52f901d0 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.748+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac530451c0 con 0x55ac52f901d0 2024-09-10T21:30:52.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb393400640 1 -- 172.21.15.97:0/2960067753 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb394012790 con 0x55ac52f901d0 2024-09-10T21:30:52.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb393400640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb37c070ff0 0x7fb37c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb393400640 1 -- 172.21.15.97:0/2960067753 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fb394091cf0 con 0x55ac52f901d0 2024-09-10T21:30:52.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ac52fa6640 0x55ac52ed18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55ac53042ff0 con 0x55ac52fa6640 2024-09-10T21:30:52.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39aa00640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb37c070ff0 0x7fb37c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39a000640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ac52fa6640 0x55ac52ed18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39aa00640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb37c070ff0 0x7fb37c0734b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0ab000 tx=0x7fb38c019b00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39a000640 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ac52fa6640 0x55ac52ed18b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39a000640 1 -- 172.21.15.97:0/2960067753 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fb38800d040 con 0x55ac52fa6640 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ac52fa6640 msgr2=0x55ac52ed18b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ac52fa6640 0x55ac52ed18b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb37c070ff0 msgr2=0x7fb37c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb37c070ff0 0x7fb37c0734b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0ab000 tx=0x7fb38c019b00 comp rx=0 tx=0).stop 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f901d0 msgr2=0x55ac52fb1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 --2- 172.21.15.97:0/2960067753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac52f901d0 0x55ac52fb1220 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb39400d390 tx=0x7fb39400d850 comp rx=0 tx=0).stop 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39b400640 1 -- 172.21.15.97:0/2960067753 reap_dead start 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 shutdown_connections 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 >> 172.21.15.97:0/2960067753 conn(0x55ac52f1a0d0 msgr2=0x55ac52fa4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 shutdown_connections 2024-09-10T21:30:52.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.752+0000 7fb39d764080 1 -- 172.21.15.97:0/2960067753 wait complete. 2024-09-10T21:30:52.761 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2024-09-10T21:30:52.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 Processor -- start 2024-09-10T21:30:52.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- start start 2024-09-10T21:30:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34e9e360 0x562e34f59fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34eccec0 0x562e34f56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34ecd2a0 0x562e34f57020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562e34deaa00 con 0x562e34eccec0 2024-09-10T21:30:52.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562e34e48f40 con 0x562e34e9e360 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x562e34f5b560 con 0x562e34ecd2a0 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34eccec0 0x562e34f56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34eccec0 0x562e34f56ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60748/0 (socket says 172.21.15.97:60748) 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 -- 172.21.15.97:0/3597119982 learned_addr learned my addr 172.21.15.97:0/3597119982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9341800640 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34ecd2a0 0x562e34f57020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9340e00640 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34e9e360 0x562e34f59fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 -- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34ecd2a0 msgr2=0x562e34f57020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34ecd2a0 0x562e34f57020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 -- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34e9e360 msgr2=0x562e34f59fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34e9e360 0x562e34f59fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 -- 172.21.15.97:0/3597119982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e34dc3c10 con 0x562e34eccec0 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9340e00640 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34e9e360 0x562e34f59fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933be00640 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34eccec0 0x562e34f56ae0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b670 tx=0x7f932c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9f077b7ad03383 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933aa00640 1 -- 172.21.15.97:0/3597119982 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c00a980 con 0x562e34eccec0 2024-09-10T21:30:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933aa00640 1 -- 172.21.15.97:0/3597119982 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f932c00ab20 con 0x562e34eccec0 2024-09-10T21:30:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f933aa00640 1 -- 172.21.15.97:0/3597119982 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c008460 con 0x562e34eccec0 2024-09-10T21:30:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- 172.21.15.97:0/3597119982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34eccec0 msgr2=0x562e34f56ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34eccec0 0x562e34f56ae0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b670 tx=0x7f932c004f90 comp rx=0 tx=0).stop 2024-09-10T21:30:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- 172.21.15.97:0/3597119982 shutdown_connections 2024-09-10T21:30:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34ecd2a0 0x562e34f57020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34eccec0 0x562e34f56ae0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 --2- 172.21.15.97:0/3597119982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34e9e360 0x562e34f59fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- 172.21.15.97:0/3597119982 >> 172.21.15.97:0/3597119982 conn(0x562e34ee40d0 msgr2=0x562e34f596b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- 172.21.15.97:0/3597119982 shutdown_connections 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.848+0000 7f9343bad080 1 -- 172.21.15.97:0/3597119982 wait complete. 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 Processor -- start 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 -- start start 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34e9e360 0x562e34f50a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34eccec0 0x562e34ee2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34ecd2a0 0x562e34f6b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562e34e9c370 con 0x562e34e9e360 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562e34e9c0d0 con 0x562e34ecd2a0 2024-09-10T21:30:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x562e34e9c610 con 0x562e34eccec0 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34e9e360 0x562e34f50a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34e9e360 0x562e34f50a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60752/0 (socket says 172.21.15.97:60752) 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 -- 172.21.15.97:0/3664651121 learned_addr learned my addr 172.21.15.97:0/3664651121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f933be00640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34eccec0 0x562e34ee2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9341800640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34ecd2a0 0x562e34f6b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 -- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34eccec0 msgr2=0x562e34ee2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562e34eccec0 0x562e34ee2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 -- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34ecd2a0 msgr2=0x562e34f6b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562e34ecd2a0 0x562e34f6b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 -- 172.21.15.97:0/3664651121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e34f6c1a0 con 0x562e34e9e360 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9340e00640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34e9e360 0x562e34f50a00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9334019990 tx=0x7f93340acc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9339600640 1 -- 172.21.15.97:0/3664651121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340ab5e0 con 0x562e34e9e360 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9339600640 1 -- 172.21.15.97:0/3664651121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93340bf070 con 0x562e34e9e360 2024-09-10T21:30:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9339600640 1 -- 172.21.15.97:0/3664651121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340b88e0 con 0x562e34e9e360 2024-09-10T21:30:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562e34f6c440 con 0x562e34e9e360 2024-09-10T21:30:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.852+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e34f6c980 con 0x562e34e9e360 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9339600640 1 -- 172.21.15.97:0/3664651121 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f93340b6040 con 0x562e34e9e360 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9339600640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9320070ff0 0x7f93200734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9339600640 1 -- 172.21.15.97:0/3664651121 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f933412d7d0 con 0x562e34e9e360 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x562e34f6b880 0x562e3500ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f933be00640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9320070ff0 0x7f93200734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x562e3500bf50 con 0x562e34f6b880 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9341800640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x562e34f6b880 0x562e3500ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9341800640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x562e34f6b880 0x562e3500ba10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f933be00640 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9320070ff0 0x7f93200734b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f932c000c10 tx=0x7f932c019040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9341800640 1 -- 172.21.15.97:0/3664651121 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv4 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f933c002740 con 0x562e34f6b880 2024-09-10T21:30:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x562e34f6b880 msgr2=0x562e3500ba10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x562e34f6b880 0x562e3500ba10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9320070ff0 msgr2=0x7f93200734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9320070ff0 0x7f93200734b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f932c000c10 tx=0x7f932c019040 comp rx=0 tx=0).stop 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34e9e360 msgr2=0x562e34f50a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 --2- 172.21.15.97:0/3664651121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562e34e9e360 0x562e34f50a00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9334019990 tx=0x7f93340acc40 comp rx=0 tx=0).stop 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9341800640 1 -- 172.21.15.97:0/3664651121 reap_dead start 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 shutdown_connections 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 >> 172.21.15.97:0/3664651121 conn(0x562e34ee40d0 msgr2=0x562e34f57aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 shutdown_connections 2024-09-10T21:30:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.856+0000 7f9343bad080 1 -- 172.21.15.97:0/3664651121 wait complete. 2024-09-10T21:30:52.865 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2024-09-10T21:30:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 Processor -- start 2024-09-10T21:30:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 -- start start 2024-09-10T21:30:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd08003b80 0x55f714b41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 0x55f714b41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f714b24820 0x55f714bd3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f714a17f40 con 0x55f714b24820 2024-09-10T21:30:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f714a24d00 con 0x7fcd08003b80 2024-09-10T21:30:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd2644e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f7149b9a00 con 0x55f714a8cf60 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f714b24820 0x55f714bd3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f714b24820 0x55f714bd3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60764/0 (socket says 172.21.15.97:60764) 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 -- 172.21.15.97:0/278884518 learned_addr learned my addr 172.21.15.97:0/278884518 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd22000640 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 0x55f714b41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd22a00640 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd08003b80 0x55f714b41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 -- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 msgr2=0x55f714b41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 0x55f714b41c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 -- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd08003b80 msgr2=0x55f714b41710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd08003b80 0x55f714b41710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 -- 172.21.15.97:0/278884518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f714a97810 con 0x55f714b24820 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd23400640 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f714b24820 0x55f714bd3fc0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009040 tx=0x7fcd08004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c247da68ed9469e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd20c00640 1 -- 172.21.15.97:0/278884518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c009a40 con 0x55f714b24820 2024-09-10T21:30:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd20c00640 1 -- 172.21.15.97:0/278884518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd1c0090d0 con 0x55f714b24820 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.908+0000 7fcd20c00640 1 -- 172.21.15.97:0/278884518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c001880 con 0x55f714b24820 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- 172.21.15.97:0/278884518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f714b24820 msgr2=0x55f714bd3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f714b24820 0x55f714bd3fc0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009040 tx=0x7fcd08004130 comp rx=0 tx=0).stop 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- 172.21.15.97:0/278884518 shutdown_connections 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f714b24820 0x55f714bd3fc0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 0x55f714b41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 --2- 172.21.15.97:0/278884518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd08003b80 0x55f714b41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- 172.21.15.97:0/278884518 >> 172.21.15.97:0/278884518 conn(0x55f714a6a140 msgr2=0x55f714a6a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- 172.21.15.97:0/278884518 shutdown_connections 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- 172.21.15.97:0/278884518 wait complete. 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 Processor -- start 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- start start 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 0x55f714a964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f714b24820 0x55f714b3f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08003b80 0x55f714b0d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f714a97a80 con 0x7fcd08003b80 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f714b42270 con 0x55f714b24820 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f714b25250 con 0x55f714a8cf60 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08003b80 0x55f714b0d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08003b80 0x55f714b0d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60776/0 (socket says 172.21.15.97:60776) 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 -- 172.21.15.97:0/549170323 learned_addr learned my addr 172.21.15.97:0/549170323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd22000640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f714b24820 0x55f714b3f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd22a00640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 0x55f714a964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 -- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 msgr2=0x55f714a964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f714a8cf60 0x55f714a964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 -- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f714b24820 msgr2=0x55f714b3f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f714b24820 0x55f714b3f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 -- 172.21.15.97:0/549170323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f714a96bd0 con 0x7fcd08003b80 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd23400640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08003b80 0x55f714b0d3c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c090 tx=0x7fcd1c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd1b400640 1 -- 172.21.15.97:0/549170323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c001880 con 0x7fcd08003b80 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd1b400640 1 -- 172.21.15.97:0/549170323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd1c0bc020 con 0x7fcd08003b80 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f714a96e70 con 0x7fcd08003b80 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd1b400640 1 -- 172.21.15.97:0/549170323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c001b90 con 0x7fcd08003b80 2024-09-10T21:30:52.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.912+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f714b10640 con 0x7fcd08003b80 2024-09-10T21:30:52.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd1b400640 1 -- 172.21.15.97:0/549170323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd1c007590 con 0x7fcd08003b80 2024-09-10T21:30:52.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd1b400640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd04071190 0x7fcd04073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd1b400640 1 -- 172.21.15.97:0/549170323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fcd1c12c550 con 0x7fcd08003b80 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd22a00640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd04071190 0x7fcd04073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55f714bcb5f0 0x55f714b10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f714b11020 con 0x55f714bcb5f0 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd22000640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55f714bcb5f0 0x55f714b10ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd22a00640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd04071190 0x7fcd04073650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcd14004fd0 tx=0x7fcd14002040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd22000640 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55f714bcb5f0 0x55f714b10ae0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd22000640 1 -- 172.21.15.97:0/549170323 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fcd1000b040 con 0x55f714bcb5f0 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55f714bcb5f0 msgr2=0x55f714b10ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55f714bcb5f0 0x55f714b10ae0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd04071190 msgr2=0x7fcd04073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd04071190 0x7fcd04073650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcd14004fd0 tx=0x7fcd14002040 comp rx=0 tx=0).stop 2024-09-10T21:30:52.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08003b80 msgr2=0x55f714b0d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 --2- 172.21.15.97:0/549170323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd08003b80 0x55f714b0d3c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00c090 tx=0x7fcd1c009ad0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd23400640 1 -- 172.21.15.97:0/549170323 reap_dead start 2024-09-10T21:30:52.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 shutdown_connections 2024-09-10T21:30:52.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 >> 172.21.15.97:0/549170323 conn(0x55f714a6a140 msgr2=0x55f714a99f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 shutdown_connections 2024-09-10T21:30:52.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.916+0000 7fcd2644e080 1 -- 172.21.15.97:0/549170323 wait complete. 2024-09-10T21:30:52.926 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2024-09-10T21:30:52.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 Processor -- start 2024-09-10T21:30:52.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- start start 2024-09-10T21:30:52.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df41b940 0x55f9df41e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df41f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df4a62a0 0x55f9df41cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9df398f40 con 0x55f9df4a62a0 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f9df3a5d00 con 0x55f9df4a5b00 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f9df33aa00 con 0x55f9df41b940 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df4a62a0 0x55f9df41cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df4a62a0 0x55f9df41cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60790/0 (socket says 172.21.15.97:60790) 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 -- 172.21.15.97:0/482900875 learned_addr learned my addr 172.21.15.97:0/482900875 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1be000640 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df41b940 0x55f9df41e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1b7e00640 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df41f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 -- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df41b940 msgr2=0x55f9df41e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df41b940 0x55f9df41e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 -- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 msgr2=0x55f9df41f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df41f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 -- 172.21.15.97:0/482900875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9df4c2a00 con 0x55f9df4a62a0 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1be000640 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df41b940 0x55f9df41e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1b7e00640 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df41f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bea00640 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df4a62a0 0x55f9df41cbf0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8001630 tx=0x7fe1b8001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ba257fc103f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bcc00640 1 -- 172.21.15.97:0/482900875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8016070 con 0x55f9df4a62a0 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bcc00640 1 -- 172.21.15.97:0/482900875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b800f820 con 0x55f9df4a62a0 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1bcc00640 1 -- 172.21.15.97:0/482900875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800fc70 con 0x55f9df4a62a0 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- 172.21.15.97:0/482900875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df4a62a0 msgr2=0x55f9df41cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df4a62a0 0x55f9df41cbf0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8001630 tx=0x7fe1b8001af0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- 172.21.15.97:0/482900875 shutdown_connections 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df4a62a0 0x55f9df41cbf0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df41f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/482900875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df41b940 0x55f9df41e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- 172.21.15.97:0/482900875 >> 172.21.15.97:0/482900875 conn(0x55f9df41a520 msgr2=0x55f9df4c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- 172.21.15.97:0/482900875 shutdown_connections 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.968+0000 7fe1c0e33080 1 -- 172.21.15.97:0/482900875 wait complete. 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 Processor -- start 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 -- start start 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df41b940 0x55f9df42e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df4312c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df4a62a0 0x55f9df4c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9df4c1d80 con 0x55f9df41b940 2024-09-10T21:30:52.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f9df41f860 con 0x55f9df4a5b00 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f9df4c2020 con 0x55f9df4a62a0 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df41b940 0x55f9df42e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df41b940 0x55f9df42e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60804/0 (socket says 172.21.15.97:60804) 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 -- 172.21.15.97:0/1977973809 learned_addr learned my addr 172.21.15.97:0/1977973809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1b7e00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df4312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1bea00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df4a62a0 0x55f9df4c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 -- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df4a62a0 msgr2=0x55f9df4c0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df4a62a0 0x55f9df4c0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 -- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 msgr2=0x55f9df4312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df4312c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 -- 172.21.15.97:0/1977973809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9df4be2c0 con 0x55f9df41b940 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1bea00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9df4a62a0 0x55f9df4c0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1b7e00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9df4a5b00 0x55f9df4312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:52.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1be000640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df41b940 0x55f9df42e810 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000cdc0 tx=0x7fe1b000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1b6a00640 1 -- 172.21.15.97:0/1977973809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0014070 con 0x55f9df41b940 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1b6a00640 1 -- 172.21.15.97:0/1977973809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b0010770 con 0x55f9df41b940 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1b6a00640 1 -- 172.21.15.97:0/1977973809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0010b80 con 0x55f9df41b940 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f9df4be560 con 0x55f9df41b940 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.972+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9df4beaa0 con 0x55f9df41b940 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1b6a00640 1 -- 172.21.15.97:0/1977973809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe1b000f3e0 con 0x55f9df41b940 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1b6a00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe19c070fc0 0x7fe19c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1b6a00640 1 -- 172.21.15.97:0/1977973809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7fe1b00922e0 con 0x55f9df41b940 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1b7e00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe19c070fc0 0x7fe19c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55f9df4bf450 0x55f9df4a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55f9df4a49e0 con 0x55f9df4bf450 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1bea00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55f9df4bf450 0x55f9df4a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:52.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1b7e00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe19c070fc0 0x7fe19c073480 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80ab000 tx=0x7fe1a8019a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1bea00640 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55f9df4bf450 0x55f9df4a44a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1bea00640 1 -- 172.21.15.97:0/1977973809 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7fe1b8004040 con 0x55f9df4bf450 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55f9df4bf450 msgr2=0x55f9df4a44a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55f9df4bf450 0x55f9df4a44a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe19c070fc0 msgr2=0x7fe19c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe19c070fc0 0x7fe19c073480 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80ab000 tx=0x7fe1a8019a00 comp rx=0 tx=0).stop 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df41b940 msgr2=0x55f9df42e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 --2- 172.21.15.97:0/1977973809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9df41b940 0x55f9df42e810 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000cdc0 tx=0x7fe1b000a5b0 comp rx=0 tx=0).stop 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1bea00640 1 -- 172.21.15.97:0/1977973809 reap_dead start 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 shutdown_connections 2024-09-10T21:30:52.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 >> 172.21.15.97:0/1977973809 conn(0x55f9df41a520 msgr2=0x55f9df4c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:52.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 shutdown_connections 2024-09-10T21:30:52.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:52.976+0000 7fe1c0e33080 1 -- 172.21.15.97:0/1977973809 wait complete. 2024-09-10T21:30:52.990 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2024-09-10T21:30:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.072+0000 7f7c6b080080 1 Processor -- start 2024-09-10T21:30:53.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- start start 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c0320940 0x5556c03b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03aab00 0x5556c03afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c03ab2a0 0x5556c045d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5556c023fa00 con 0x5556c03aab00 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556c029df40 con 0x5556c0320940 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5556c03ad080 con 0x5556c03ab2a0 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03aab00 0x5556c03afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03aab00 0x5556c03afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60814/0 (socket says 172.21.15.97:60814) 2024-09-10T21:30:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 -- 172.21.15.97:0/549170151 learned_addr learned my addr 172.21.15.97:0/549170151 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c68200640 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c0320940 0x5556c03b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c68c00640 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c03ab2a0 0x5556c045d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 -- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c03ab2a0 msgr2=0x5556c045d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c03ab2a0 0x5556c045d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 -- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c0320940 msgr2=0x5556c03b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c0320940 0x5556c03b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 -- 172.21.15.97:0/549170151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c0218c10 con 0x5556c03aab00 2024-09-10T21:30:53.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c5fe00640 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03aab00 0x5556c03afb60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7c50004ea0 tx=0x7f7c50017e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c5e08ab71d23ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c66e00640 1 -- 172.21.15.97:0/549170151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c500ab630 con 0x5556c03aab00 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c66e00640 1 -- 172.21.15.97:0/549170151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c500b9070 con 0x5556c03aab00 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c66e00640 1 -- 172.21.15.97:0/549170151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c500abd10 con 0x5556c03aab00 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- 172.21.15.97:0/549170151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03aab00 msgr2=0x5556c03afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03aab00 0x5556c03afb60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7c50004ea0 tx=0x7f7c50017e80 comp rx=0 tx=0).stop 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- 172.21.15.97:0/549170151 shutdown_connections 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c03ab2a0 0x5556c045d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03aab00 0x5556c03afb60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 --2- 172.21.15.97:0/549170151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c0320940 0x5556c03b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- 172.21.15.97:0/549170151 >> 172.21.15.97:0/549170151 conn(0x5556c031f520 msgr2=0x5556c03c7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- 172.21.15.97:0/549170151 shutdown_connections 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- 172.21.15.97:0/549170151 wait complete. 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 Processor -- start 2024-09-10T21:30:53.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.076+0000 7f7c6b080080 1 -- start start 2024-09-10T21:30:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c0320940 0x5556c03a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c03aab00 0x5556c03a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03ab2a0 0x5556c03c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5556c03b0530 con 0x5556c03ab2a0 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556c03b07d0 con 0x5556c03aab00 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5556c03b0a70 con 0x5556c0320940 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03ab2a0 0x5556c03c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03ab2a0 0x5556c03c4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60824/0 (socket says 172.21.15.97:60824) 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 -- 172.21.15.97:0/280557416 learned_addr learned my addr 172.21.15.97:0/280557416 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68200640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c0320940 0x5556c03a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c5fe00640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c03aab00 0x5556c03a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 -- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c0320940 msgr2=0x5556c03a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5556c0320940 0x5556c03a77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 -- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c03aab00 msgr2=0x5556c03a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c03aab00 0x5556c03a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 -- 172.21.15.97:0/280557416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c0323b90 con 0x5556c03ab2a0 2024-09-10T21:30:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c68c00640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03ab2a0 0x5556c03c4000 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e490 tx=0x7f7c6000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c65a00640 1 -- 172.21.15.97:0/280557416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60013680 con 0x5556c03ab2a0 2024-09-10T21:30:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c65a00640 1 -- 172.21.15.97:0/280557416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c60018070 con 0x5556c03ab2a0 2024-09-10T21:30:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c65a00640 1 -- 172.21.15.97:0/280557416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60013bd0 con 0x5556c03ab2a0 2024-09-10T21:30:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5556c0323e30 con 0x5556c03ab2a0 2024-09-10T21:30:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c0324370 con 0x5556c03ab2a0 2024-09-10T21:30:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.080+0000 7f7c65a00640 1 -- 172.21.15.97:0/280557416 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7c6001d460 con 0x5556c03ab2a0 2024-09-10T21:30:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c65a00640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c44070fc0 0x7f7c44073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c65a00640 1 -- 172.21.15.97:0/280557416 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f7c600920d0 con 0x5556c03ab2a0 2024-09-10T21:30:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c68200640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c44070fc0 0x7f7c44073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5556c03236f0 0x5556c04617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5556c03c78a0 con 0x5556c03236f0 2024-09-10T21:30:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c5fe00640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5556c03236f0 0x5556c04617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c68200640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c44070fc0 0x7f7c44073480 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7c580044c0 tx=0x7f7c5800f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c5fe00640 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5556c03236f0 0x5556c04617f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c5fe00640 1 -- 172.21.15.97:0/280557416 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f7c500abab0 con 0x5556c03236f0 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5556c03236f0 msgr2=0x5556c04617f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5556c03236f0 0x5556c04617f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c44070fc0 msgr2=0x7f7c44073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c44070fc0 0x7f7c44073480 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7c580044c0 tx=0x7f7c5800f040 comp rx=0 tx=0).stop 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03ab2a0 msgr2=0x5556c03c4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 --2- 172.21.15.97:0/280557416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556c03ab2a0 0x5556c03c4000 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000e490 tx=0x7f7c6000e950 comp rx=0 tx=0).stop 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c68c00640 1 -- 172.21.15.97:0/280557416 reap_dead start 2024-09-10T21:30:53.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 shutdown_connections 2024-09-10T21:30:53.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 >> 172.21.15.97:0/280557416 conn(0x5556c031f520 msgr2=0x5556c03c8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 shutdown_connections 2024-09-10T21:30:53.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.084+0000 7f7c6b080080 1 -- 172.21.15.97:0/280557416 wait complete. 2024-09-10T21:30:53.093 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2024-09-10T21:30:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 Processor -- start 2024-09-10T21:30:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 -- start start 2024-09-10T21:30:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524efd940 0x55c524fa0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524fa7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524f882a0 0x55c524ecbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c524e1ca00 con 0x55c524f882a0 2024-09-10T21:30:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c524e7af40 con 0x55c524efd940 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff48d9c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c524f8a080 con 0x55c524f87b00 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524f882a0 0x55c524ecbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524f882a0 0x55c524ecbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60850/0 (socket says 172.21.15.97:60850) 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 -- 172.21.15.97:0/7615604 learned_addr learned my addr 172.21.15.97:0/7615604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46000640 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524efd940 0x55c524fa0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff45600640 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524fa7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 -- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 msgr2=0x55c524fa7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524fa7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 -- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524efd940 msgr2=0x55c524fa0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524efd940 0x55c524fa0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 -- 172.21.15.97:0/7615604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c524df5c10 con 0x55c524f882a0 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff45600640 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524fa7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:53.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.180+0000 7eff46a00640 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524f882a0 0x55c524ecbd90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7eff40006f80 tx=0x7eff40001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e75a0b7ff7abe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff3fe00640 1 -- 172.21.15.97:0/7615604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40017070 con 0x55c524f882a0 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff3fe00640 1 -- 172.21.15.97:0/7615604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff400027b0 con 0x55c524f882a0 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff3fe00640 1 -- 172.21.15.97:0/7615604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40002c40 con 0x55c524f882a0 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- 172.21.15.97:0/7615604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524f882a0 msgr2=0x55c524ecbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524f882a0 0x55c524ecbd90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7eff40006f80 tx=0x7eff40001b10 comp rx=0 tx=0).stop 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- 172.21.15.97:0/7615604 shutdown_connections 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524f882a0 0x55c524ecbd90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524fa7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 --2- 172.21.15.97:0/7615604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524efd940 0x55c524fa0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- 172.21.15.97:0/7615604 >> 172.21.15.97:0/7615604 conn(0x55c524efc520 msgr2=0x55c524fa4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- 172.21.15.97:0/7615604 shutdown_connections 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- 172.21.15.97:0/7615604 wait complete. 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 Processor -- start 2024-09-10T21:30:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- start start 2024-09-10T21:30:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524efd940 0x55c524fa4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524f9e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524f882a0 0x55c524faa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c524f00ad0 con 0x55c524efd940 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c524eff580 con 0x55c524f882a0 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c524f00d70 con 0x55c524f87b00 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524efd940 0x55c524fa4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524efd940 0x55c524fa4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60866/0 (socket says 172.21.15.97:60866) 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 -- 172.21.15.97:0/512755950 learned_addr learned my addr 172.21.15.97:0/512755950 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46a00640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524f882a0 0x55c524faa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff45600640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524f9e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 -- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 msgr2=0x55c524f9e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c524f87b00 0x55c524f9e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 -- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524f882a0 msgr2=0x55c524faa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c524f882a0 0x55c524faa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 -- 172.21.15.97:0/512755950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c524fa35e0 con 0x55c524efd940 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff46000640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524efd940 0x55c524fa4260 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7eff38004f60 tx=0x7eff380ab680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff3ea00640 1 -- 172.21.15.97:0/512755950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff380bb020 con 0x55c524efd940 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff3ea00640 1 -- 172.21.15.97:0/512755950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff380b1040 con 0x55c524efd940 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff3ea00640 1 -- 172.21.15.97:0/512755950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff380ace80 con 0x55c524efd940 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.184+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c524fa3880 con 0x55c524efd940 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c524fa8eb0 con 0x55c524efd940 2024-09-10T21:30:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff3ea00640 1 -- 172.21.15.97:0/512755950 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff380b6040 con 0x55c524efd940 2024-09-10T21:30:53.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff3ea00640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff28070ff0 0x7eff280734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff3ea00640 1 -- 172.21.15.97:0/512755950 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7eff3812d320 con 0x55c524efd940 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff48d9c080 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c524f9ebb0 0x55c524fa24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55c524fa7980 con 0x55c524f9ebb0 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff45600640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff28070ff0 0x7eff280734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff46a00640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c524f9ebb0 0x55c524fa24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff45600640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff28070ff0 0x7eff280734b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c570 tx=0x7eff34004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.188+0000 7eff46a00640 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c524f9ebb0 0x55c524fa24c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stdout:stray0_head 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stdout:stray1_head 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stdout:stray2_head 2024-09-10T21:30:53.195 INFO:teuthology.orchestra.run.smithi097.stdout:stray3_head 2024-09-10T21:30:53.196 INFO:teuthology.orchestra.run.smithi097.stdout:stray4_head 2024-09-10T21:30:53.196 INFO:teuthology.orchestra.run.smithi097.stdout:stray5_head 2024-09-10T21:30:53.196 INFO:teuthology.orchestra.run.smithi097.stdout:stray6_head 2024-09-10T21:30:53.196 INFO:teuthology.orchestra.run.smithi097.stdout:stray7_head 2024-09-10T21:30:53.196 INFO:teuthology.orchestra.run.smithi097.stdout:stray8_head 2024-09-10T21:30:53.196 INFO:teuthology.orchestra.run.smithi097.stdout:stray9_head 2024-09-10T21:30:53.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff46a00640 1 -- 172.21.15.97:0/512755950 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) ==== 156+0+155 (crc 0 0 0) 0x7eff4000f070 con 0x55c524f9ebb0 2024-09-10T21:30:53.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c524f9ebb0 msgr2=0x55c524fa24c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c524f9ebb0 0x55c524fa24c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff28070ff0 msgr2=0x7eff280734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff28070ff0 0x7eff280734b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c570 tx=0x7eff34004890 comp rx=0 tx=0).stop 2024-09-10T21:30:53.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524efd940 msgr2=0x55c524fa4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 --2- 172.21.15.97:0/512755950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c524efd940 0x55c524fa4260 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7eff38004f60 tx=0x7eff380ab680 comp rx=0 tx=0).stop 2024-09-10T21:30:53.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff46a00640 1 -- 172.21.15.97:0/512755950 reap_dead start 2024-09-10T21:30:53.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 shutdown_connections 2024-09-10T21:30:53.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 >> 172.21.15.97:0/512755950 conn(0x55c524efc520 msgr2=0x55c524f18820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 shutdown_connections 2024-09-10T21:30:53.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.192+0000 7eff48d9c080 1 -- 172.21.15.97:0/512755950 wait complete. 2024-09-10T21:30:53.199 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2024-09-10T21:30:53.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 Processor -- start 2024-09-10T21:30:53.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 -- start start 2024-09-10T21:30:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 0x55e5626fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e56266b920 0x55e562652200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e562652740 0x55e56266c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e5625d2f40 con 0x55e56266b920 2024-09-10T21:30:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e5625dfd00 con 0x55e562652740 2024-09-10T21:30:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f452022b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55e562574a00 con 0x7f4500003100 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e56266b920 0x55e562652200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e56266b920 0x55e562652200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60882/0 (socket says 172.21.15.97:60882) 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 -- 172.21.15.97:0/3211084849 learned_addr learned my addr 172.21.15.97:0/3211084849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f451c600640 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 0x55e5626fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f451d000640 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e562652740 0x55e56266c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 -- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 msgr2=0x55e5626fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 0x55e5626fa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 -- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e562652740 msgr2=0x55e56266c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e562652740 0x55e56266c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 -- 172.21.15.97:0/3211084849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e562652b50 con 0x55e56266b920 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f4513e00640 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e56266b920 0x55e562652200 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f450401bdb0 tx=0x7f4500004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d343146e64f5c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f451b200640 1 -- 172.21.15.97:0/3211084849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040a4550 con 0x55e56266b920 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f451b200640 1 -- 172.21.15.97:0/3211084849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4504001440 con 0x55e56266b920 2024-09-10T21:30:53.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.284+0000 7f451b200640 1 -- 172.21.15.97:0/3211084849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040a4900 con 0x55e56266b920 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- 172.21.15.97:0/3211084849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e56266b920 msgr2=0x55e562652200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e56266b920 0x55e562652200 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f450401bdb0 tx=0x7f4500004130 comp rx=0 tx=0).stop 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- 172.21.15.97:0/3211084849 shutdown_connections 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e562652740 0x55e56266c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e56266b920 0x55e562652200 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 --2- 172.21.15.97:0/3211084849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 0x55e5626fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- 172.21.15.97:0/3211084849 >> 172.21.15.97:0/3211084849 conn(0x55e562625140 msgr2=0x55e5626f9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- 172.21.15.97:0/3211084849 shutdown_connections 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- 172.21.15.97:0/3211084849 wait complete. 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 Processor -- start 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- start start 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e562652740 0x55e562648c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e56266b920 0x55e5626f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 0x55e56266c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e56266af40 con 0x55e562652740 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e56266aca0 con 0x55e56266b920 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55e56266b1e0 con 0x7f4500003100 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e562652740 0x55e562648c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e562652740 0x55e562648c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60888/0 (socket says 172.21.15.97:60888) 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 -- 172.21.15.97:0/1794576183 learned_addr learned my addr 172.21.15.97:0/1794576183 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f4513e00640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e56266b920 0x55e5626f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451d000640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 0x55e56266c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 -- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 msgr2=0x55e56266c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4500003100 0x55e56266c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 -- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e56266b920 msgr2=0x55e5626f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e56266b920 0x55e5626f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 -- 172.21.15.97:0/1794576183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e562792330 con 0x55e562652740 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f451c600640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e562652740 0x55e562648c30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f450c00a7e0 tx=0x7f450c00aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f4519e00640 1 -- 172.21.15.97:0/1794576183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c009dc0 con 0x55e562652740 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f4519e00640 1 -- 172.21.15.97:0/1794576183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f450c008430 con 0x55e562652740 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5627925d0 con 0x55e562652740 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f4519e00640 1 -- 172.21.15.97:0/1794576183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c008840 con 0x55e562652740 2024-09-10T21:30:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.288+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e562792b10 con 0x55e562652740 2024-09-10T21:30:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f4519e00640 1 -- 172.21.15.97:0/1794576183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f450c0089e0 con 0x55e562652740 2024-09-10T21:30:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f4519e00640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44f8070ff0 0x7f44f80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f4519e00640 1 -- 172.21.15.97:0/1794576183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f450c092ed0 con 0x55e562652740 2024-09-10T21:30:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f452022b080 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e5627974a0 0x55e562797880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55e562797dc0 con 0x55e5627974a0 2024-09-10T21:30:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f4513e00640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44f8070ff0 0x7f44f80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f451d000640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e5627974a0 0x55e562797880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f4513e00640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44f8070ff0 0x7f44f80734b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45040b7000 tx=0x7f4504009230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.292+0000 7f451d000640 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e5627974a0 0x55e562797880 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f451d000640 1 -- 172.21.15.97:0/1794576183 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7f4514008040 con 0x55e5627974a0 2024-09-10T21:30:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e5627974a0 msgr2=0x55e562797880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e5627974a0 0x55e562797880 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44f8070ff0 msgr2=0x7f44f80734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44f8070ff0 0x7f44f80734b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45040b7000 tx=0x7f4504009230 comp rx=0 tx=0).stop 2024-09-10T21:30:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e562652740 msgr2=0x55e562648c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 --2- 172.21.15.97:0/1794576183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e562652740 0x55e562648c30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f450c00a7e0 tx=0x7f450c00aca0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f451d000640 1 -- 172.21.15.97:0/1794576183 reap_dead start 2024-09-10T21:30:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 shutdown_connections 2024-09-10T21:30:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 >> 172.21.15.97:0/1794576183 conn(0x55e562625140 msgr2=0x55e5626fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 shutdown_connections 2024-09-10T21:30:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.296+0000 7f452022b080 1 -- 172.21.15.97:0/1794576183 wait complete. 2024-09-10T21:30:53.302 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2024-09-10T21:30:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 Processor -- start 2024-09-10T21:30:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- start start 2024-09-10T21:30:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8058003ac0 0x56118766c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56118766b9a0 0x56118766aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 0x56118763d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5611875e7f40 con 0x56118766b9a0 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5611875f4d00 con 0x56118766b000 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561187589a00 con 0x7f8058003ac0 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56118766b9a0 0x56118766aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56118766b9a0 0x56118766aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60906/0 (socket says 172.21.15.97:60906) 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 -- 172.21.15.97:0/4291035087 learned_addr learned my addr 172.21.15.97:0/4291035087 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8072000640 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8058003ac0 0x56118766c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8072a00640 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 0x56118763d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 -- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8058003ac0 msgr2=0x56118766c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8058003ac0 0x56118766c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 -- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 msgr2=0x56118763d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 0x56118763d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 -- 172.21.15.97:0/4291035087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118766b700 con 0x56118766b9a0 2024-09-10T21:30:53.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8071600640 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56118766b9a0 0x56118766aac0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f806000b900 tx=0x7f806000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0154ceb8c882481 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f806be00640 1 -- 172.21.15.97:0/4291035087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060014030 con 0x56118766b9a0 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f806be00640 1 -- 172.21.15.97:0/4291035087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f806000d930 con 0x56118766b9a0 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f806be00640 1 -- 172.21.15.97:0/4291035087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806000dda0 con 0x56118766b9a0 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- 172.21.15.97:0/4291035087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56118766b9a0 msgr2=0x56118766aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56118766b9a0 0x56118766aac0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f806000b900 tx=0x7f806000bd00 comp rx=0 tx=0).stop 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- 172.21.15.97:0/4291035087 shutdown_connections 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 0x56118763d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56118766b9a0 0x56118766aac0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 --2- 172.21.15.97:0/4291035087 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8058003ac0 0x56118766c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- 172.21.15.97:0/4291035087 >> 172.21.15.97:0/4291035087 conn(0x5611876830d0 msgr2=0x5611876f06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- 172.21.15.97:0/4291035087 shutdown_connections 2024-09-10T21:30:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 -- 172.21.15.97:0/4291035087 wait complete. 2024-09-10T21:30:53.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.392+0000 7f8074de8080 1 Processor -- start 2024-09-10T21:30:53.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 -- start start 2024-09-10T21:30:53.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 0x5611877ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56118766b9a0 0x56118763a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8058003ac0 0x56118763a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5611876f11f0 con 0x7f8058003ac0 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56118766b410 con 0x56118766b000 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561187669820 con 0x56118766b9a0 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8058003ac0 0x56118763a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8058003ac0 0x56118763a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60918/0 (socket says 172.21.15.97:60918) 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 -- 172.21.15.97:0/2996949945 learned_addr learned my addr 172.21.15.97:0/2996949945 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8071600640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56118766b9a0 0x56118763a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072000640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 0x5611877ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 -- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56118766b9a0 msgr2=0x56118763a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56118766b9a0 0x56118763a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 -- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 msgr2=0x5611877ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56118766b000 0x5611877ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 -- 172.21.15.97:0/2996949945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561187711cd0 con 0x7f8058003ac0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8072a00640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8058003ac0 0x56118763a680 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f806c009d30 tx=0x7f806c001530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f806aa00640 1 -- 172.21.15.97:0/2996949945 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c001a80 con 0x7f8058003ac0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f806aa00640 1 -- 172.21.15.97:0/2996949945 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f806c0090d0 con 0x7f8058003ac0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f806aa00640 1 -- 172.21.15.97:0/2996949945 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c0a79a0 con 0x7f8058003ac0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561187711f70 con 0x7f8058003ac0 2024-09-10T21:30:53.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.396+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611877124b0 con 0x7f8058003ac0 2024-09-10T21:30:53.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f806aa00640 1 -- 172.21.15.97:0/2996949945 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f806c0c6060 con 0x7f8058003ac0 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f806aa00640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8054070ff0 0x7f80540734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f806aa00640 1 -- 172.21.15.97:0/2996949945 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f806c12e9e0 con 0x7f8058003ac0 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56118763c530 0x5611877a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5611877a9410 con 0x56118763c530 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8072000640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8054070ff0 0x7f80540734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8071600640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56118763c530 0x5611877a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8072000640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8054070ff0 0x7f80540734b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56118763fd50 tx=0x7f8064002040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8071600640 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56118763c530 0x5611877a8ed0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8071600640 1 -- 172.21.15.97:0/2996949945 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f806000b040 con 0x56118763c530 2024-09-10T21:30:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56118763c530 msgr2=0x5611877a8ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56118763c530 0x5611877a8ed0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8054070ff0 msgr2=0x7f80540734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8054070ff0 0x7f80540734b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56118763fd50 tx=0x7f8064002040 comp rx=0 tx=0).stop 2024-09-10T21:30:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8058003ac0 msgr2=0x56118763a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 --2- 172.21.15.97:0/2996949945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8058003ac0 0x56118763a680 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f806c009d30 tx=0x7f806c001530 comp rx=0 tx=0).stop 2024-09-10T21:30:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8072a00640 1 -- 172.21.15.97:0/2996949945 reap_dead start 2024-09-10T21:30:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 shutdown_connections 2024-09-10T21:30:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 >> 172.21.15.97:0/2996949945 conn(0x5611876830d0 msgr2=0x56118766a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 shutdown_connections 2024-09-10T21:30:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.400+0000 7f8074de8080 1 -- 172.21.15.97:0/2996949945 wait complete. 2024-09-10T21:30:53.409 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2024-09-10T21:30:53.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 Processor -- start 2024-09-10T21:30:53.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- start start 2024-09-10T21:30:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 0x561b06b6a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b6da00 0x561b06b8a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b711d0 0x561b06afd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b06a01a00 con 0x561b06b711d0 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561b06a5ff40 con 0x561b06b6bcc0 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561b06b6f710 con 0x561b06b6da00 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b711d0 0x561b06afd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b711d0 0x561b06afd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60926/0 (socket says 172.21.15.97:60926) 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 -- 172.21.15.97:0/2138664169 learned_addr learned my addr 172.21.15.97:0/2138664169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044d400640 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 0x561b06b6a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044ca00640 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b6da00 0x561b06b8a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 -- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b6da00 msgr2=0x561b06b8a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b6da00 0x561b06b8a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 -- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 msgr2=0x561b06b6a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 0x561b06b6a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 -- 172.21.15.97:0/2138664169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b069dac10 con 0x561b06b711d0 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f044de00640 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b711d0 0x561b06afd2d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0448019c10 tx=0x7f0448018300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb42cc7deff987f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0447400640 1 -- 172.21.15.97:0/2138664169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480a8af0 con 0x561b06b711d0 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0447400640 1 -- 172.21.15.97:0/2138664169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04480a8d80 con 0x561b06b711d0 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0447400640 1 -- 172.21.15.97:0/2138664169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480bc680 con 0x561b06b711d0 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- 172.21.15.97:0/2138664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b711d0 msgr2=0x561b06afd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b711d0 0x561b06afd2d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0448019c10 tx=0x7f0448018300 comp rx=0 tx=0).stop 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- 172.21.15.97:0/2138664169 shutdown_connections 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b711d0 0x561b06afd2d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b6da00 0x561b06b8a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 --2- 172.21.15.97:0/2138664169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 0x561b06b6a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- 172.21.15.97:0/2138664169 >> 172.21.15.97:0/2138664169 conn(0x561b06afb0d0 msgr2=0x561b06ae3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- 172.21.15.97:0/2138664169 shutdown_connections 2024-09-10T21:30:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.496+0000 7f0450161080 1 -- 172.21.15.97:0/2138664169 wait complete. 2024-09-10T21:30:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 Processor -- start 2024-09-10T21:30:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 -- start start 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 0x561b06ab2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b6da00 0x561b06afa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b711d0 0x561b06c22e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b06ae3230 con 0x561b06b6da00 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561b06ae34d0 con 0x561b06b6bcc0 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561b06ae3770 con 0x561b06b711d0 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b6da00 0x561b06afa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b6da00 0x561b06afa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60934/0 (socket says 172.21.15.97:60934) 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 -- 172.21.15.97:0/823889136 learned_addr learned my addr 172.21.15.97:0/823889136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044de00640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b711d0 0x561b06c22e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044d400640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 0x561b06ab2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 -- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b711d0 msgr2=0x561b06c22e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561b06b711d0 0x561b06c22e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 -- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 msgr2=0x561b06ab2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561b06b6bcc0 0x561b06ab2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 -- 172.21.15.97:0/823889136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b06ab27e0 con 0x561b06b6da00 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f044ca00640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b6da00 0x561b06afa590 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f043c00df80 tx=0x7f043c00b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0446000640 1 -- 172.21.15.97:0/823889136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c014020 con 0x561b06b6da00 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0446000640 1 -- 172.21.15.97:0/823889136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f043c001600 con 0x561b06b6da00 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0446000640 1 -- 172.21.15.97:0/823889136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c00a040 con 0x561b06b6da00 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561b06ab2a80 con 0x561b06b6da00 2024-09-10T21:30:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.500+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b06ab2fc0 con 0x561b06b6da00 2024-09-10T21:30:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0446000640 1 -- 172.21.15.97:0/823889136 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f043c001a50 con 0x561b06b6da00 2024-09-10T21:30:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0446000640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0430070ff0 0x7f04300734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0446000640 1 -- 172.21.15.97:0/823889136 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f043c092270 con 0x561b06b6da00 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f044d400640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0430070ff0 0x7f04300734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561b06b8bb50 0x561b06b871d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x561b06ab3700 con 0x561b06b8bb50 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f044de00640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561b06b8bb50 0x561b06b871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f044d400640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0430070ff0 0x7f04300734b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f04400044c0 tx=0x7f0440004940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f044de00640 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561b06b8bb50 0x561b06b871d0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f044de00640 1 -- 172.21.15.97:0/823889136 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f0448002050 con 0x561b06b8bb50 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561b06b8bb50 msgr2=0x561b06b871d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561b06b8bb50 0x561b06b871d0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0430070ff0 msgr2=0x7f04300734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0430070ff0 0x7f04300734b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f04400044c0 tx=0x7f0440004940 comp rx=0 tx=0).stop 2024-09-10T21:30:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b6da00 msgr2=0x561b06afa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 --2- 172.21.15.97:0/823889136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b06b6da00 0x561b06afa590 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f043c00df80 tx=0x7f043c00b680 comp rx=0 tx=0).stop 2024-09-10T21:30:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f044de00640 1 -- 172.21.15.97:0/823889136 reap_dead start 2024-09-10T21:30:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 shutdown_connections 2024-09-10T21:30:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 >> 172.21.15.97:0/823889136 conn(0x561b06afb0d0 msgr2=0x561b06b8a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 shutdown_connections 2024-09-10T21:30:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.504+0000 7f0450161080 1 -- 172.21.15.97:0/823889136 wait complete. 2024-09-10T21:30:53.513 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-09-10T21:30:53.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.596+0000 7f043796a080 1 Processor -- start 2024-09-10T21:30:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.596+0000 7f043796a080 1 -- start start 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d3d2940 0x55ca9d475f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d477a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d45d2a0 0x55ca9d50f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ca9d2f1a00 con 0x55ca9d45cb00 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca9d34ff40 con 0x55ca9d3d2940 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ca9d45f080 con 0x55ca9d45d2a0 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d477a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d477a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60946/0 (socket says 172.21.15.97:60946) 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 -- 172.21.15.97:0/3934051448 learned_addr learned my addr 172.21.15.97:0/3934051448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f0434c00640 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d3d2940 0x55ca9d475f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f0435600640 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d45d2a0 0x55ca9d50f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 -- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d45d2a0 msgr2=0x55ca9d50f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d45d2a0 0x55ca9d50f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 -- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d3d2940 msgr2=0x55ca9d475f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d3d2940 0x55ca9d475f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 -- 172.21.15.97:0/3934051448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9d2cac10 con 0x55ca9d45cb00 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042fe00640 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d477a50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f042000c590 tx=0x7f042000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1a0444d9b6431a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042ea00640 1 -- 172.21.15.97:0/3934051448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042000aa70 con 0x55ca9d45cb00 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042ea00640 1 -- 172.21.15.97:0/3934051448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f042000ac10 con 0x55ca9d45cb00 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f042ea00640 1 -- 172.21.15.97:0/3934051448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420008430 con 0x55ca9d45cb00 2024-09-10T21:30:53.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- 172.21.15.97:0/3934051448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 msgr2=0x55ca9d477a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d477a50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f042000c590 tx=0x7f042000ca50 comp rx=0 tx=0).stop 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- 172.21.15.97:0/3934051448 shutdown_connections 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d45d2a0 0x55ca9d50f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d477a50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 --2- 172.21.15.97:0/3934051448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d3d2940 0x55ca9d475f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- 172.21.15.97:0/3934051448 >> 172.21.15.97:0/3934051448 conn(0x55ca9d3d1520 msgr2=0x55ca9d479300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- 172.21.15.97:0/3934051448 shutdown_connections 2024-09-10T21:30:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- 172.21.15.97:0/3934051448 wait complete. 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 Processor -- start 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.600+0000 7f043796a080 1 -- start start 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d3d2940 0x55ca9d45ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d4600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d45d2a0 0x55ca9d4605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ca9d479030 con 0x55ca9d45cb00 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ca9d3d4240 con 0x55ca9d45d2a0 2024-09-10T21:30:53.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ca9d462780 con 0x55ca9d3d2940 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d4600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d4600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60956/0 (socket says 172.21.15.97:60956) 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 -- 172.21.15.97:0/1735578789 learned_addr learned my addr 172.21.15.97:0/1735578789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f0434c00640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d3d2940 0x55ca9d45ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f0435600640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d45d2a0 0x55ca9d4605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 -- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d3d2940 msgr2=0x55ca9d45ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ca9d3d2940 0x55ca9d45ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 -- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d45d2a0 msgr2=0x55ca9d4605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca9d45d2a0 0x55ca9d4605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 -- 172.21.15.97:0/1735578789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9d462a20 con 0x55ca9d45cb00 2024-09-10T21:30:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042fe00640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d4600a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0420004860 tx=0x7f04200146a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042d600640 1 -- 172.21.15.97:0/1735578789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042001a070 con 0x55ca9d45cb00 2024-09-10T21:30:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042d600640 1 -- 172.21.15.97:0/1735578789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0420014de0 con 0x55ca9d45cb00 2024-09-10T21:30:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ca9d462cc0 con 0x55ca9d45cb00 2024-09-10T21:30:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042d600640 1 -- 172.21.15.97:0/1735578789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042001e6b0 con 0x55ca9d45cb00 2024-09-10T21:30:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9d45b370 con 0x55ca9d45cb00 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.604+0000 7f042d600640 1 -- 172.21.15.97:0/1735578789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0420013070 con 0x55ca9d45cb00 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f042d600640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0414070ff0 0x7f04140734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f042d600640 1 -- 172.21.15.97:0/1735578789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6187+0+0 (secure 0 0 0) 0x7f0420093050 con 0x55ca9d45cb00 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f0434c00640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0414070ff0 0x7f04140734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ca9d50b760 0x55ca9d50bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x55ca9d50c080 con 0x55ca9d50b760 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f0435600640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ca9d50b760 0x55ca9d50bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f0434c00640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0414070ff0 0x7f04140734b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0428003f20 tx=0x7f04280038a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f0435600640 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ca9d50b760 0x55ca9d50bb40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f0435600640 1 -- 172.21.15.97:0/1735578789 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv6 ondisk = 0) ==== 154+0+20 (crc 0 0 0) 0x7f0430002050 con 0x55ca9d50b760 2024-09-10T21:30:53.612 INFO:teuthology.orchestra.run.smithi097.stdout:subdir_head 2024-09-10T21:30:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ca9d50b760 msgr2=0x55ca9d50bb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ca9d50b760 0x55ca9d50bb40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0414070ff0 msgr2=0x7f04140734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0414070ff0 0x7f04140734b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0428003f20 tx=0x7f04280038a0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 msgr2=0x55ca9d4600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 --2- 172.21.15.97:0/1735578789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca9d45cb00 0x55ca9d4600a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0420004860 tx=0x7f04200146a0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f0435600640 1 -- 172.21.15.97:0/1735578789 reap_dead start 2024-09-10T21:30:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 shutdown_connections 2024-09-10T21:30:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 >> 172.21.15.97:0/1735578789 conn(0x55ca9d3d1520 msgr2=0x55ca9d3d5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 shutdown_connections 2024-09-10T21:30:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.608+0000 7f043796a080 1 -- 172.21.15.97:0/1735578789 wait complete. 2024-09-10T21:30:53.617 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-09-10T21:30:53.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.700+0000 7f7eab048080 1 Processor -- start 2024-09-10T21:30:53.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.700+0000 7f7eab048080 1 -- start start 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x560466517520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664feb00 0x56046651e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664ff2a0 0x560466442d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560466393a00 con 0x5604664feb00 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5604663f1f40 con 0x560466474940 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560466501080 con 0x5604664ff2a0 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664feb00 0x56046651e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664feb00 0x56046651e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60970/0 (socket says 172.21.15.97:60970) 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 -- 172.21.15.97:0/23216384 learned_addr learned my addr 172.21.15.97:0/23216384 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7ea8000640 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664ff2a0 0x560466442d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7ea7600640 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x560466517520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 -- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664ff2a0 msgr2=0x560466442d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664ff2a0 0x560466442d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 -- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 msgr2=0x560466517520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x560466517520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 -- 172.21.15.97:0/23216384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046636cc10 con 0x5604664feb00 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7e9fe00640 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664feb00 0x56046651e160 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000b230 tx=0x7f7e9000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a195ebec6d1df1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7ea6200640 1 -- 172.21.15.97:0/23216384 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90016070 con 0x5604664feb00 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7ea6200640 1 -- 172.21.15.97:0/23216384 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e9000d4c0 con 0x5604664feb00 2024-09-10T21:30:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7ea6200640 1 -- 172.21.15.97:0/23216384 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000d710 con 0x5604664feb00 2024-09-10T21:30:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- 172.21.15.97:0/23216384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664feb00 msgr2=0x56046651e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664feb00 0x56046651e160 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000b230 tx=0x7f7e9000b6f0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- 172.21.15.97:0/23216384 shutdown_connections 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664ff2a0 0x560466442d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664feb00 0x56046651e160 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 --2- 172.21.15.97:0/23216384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x560466517520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- 172.21.15.97:0/23216384 >> 172.21.15.97:0/23216384 conn(0x560466473520 msgr2=0x56046651b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- 172.21.15.97:0/23216384 shutdown_connections 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- 172.21.15.97:0/23216384 wait complete. 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 Processor -- start 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.704+0000 7f7eab048080 1 -- start start 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x5604664fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664feb00 0x560466524ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664ff2a0 0x56046651a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5604664779f0 con 0x5604664ff2a0 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560466477c90 con 0x560466474940 2024-09-10T21:30:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560466476190 con 0x5604664feb00 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea7600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x5604664fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea8000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664ff2a0 0x56046651a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7e9fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664feb00 0x560466524ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea7600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x5604664fdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:59098/0 (socket says 172.21.15.97:59098) 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea7600640 1 -- 172.21.15.97:0/2649163431 learned_addr learned my addr 172.21.15.97:0/2649163431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7e9fe00640 1 -- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 msgr2=0x5604664fdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7e9fe00640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x5604664fdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7e9fe00640 1 -- 172.21.15.97:0/2649163431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664ff2a0 msgr2=0x56046651a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7e9fe00640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604664ff2a0 0x56046651a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7e9fe00640 1 -- 172.21.15.97:0/2649163431 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046651f2a0 con 0x5604664feb00 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea7600640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560466474940 0x5604664fdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7e9fe00640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664feb00 0x560466524ae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000b200 tx=0x7f7e9000dfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea4e00640 1 -- 172.21.15.97:0/2649163431 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90016040 con 0x5604664feb00 2024-09-10T21:30:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea4e00640 1 -- 172.21.15.97:0/2649163431 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e900012e0 con 0x5604664feb00 2024-09-10T21:30:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea4e00640 1 -- 172.21.15.97:0/2649163431 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e900106b0 con 0x5604664feb00 2024-09-10T21:30:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x56046651f540 con 0x5604664feb00 2024-09-10T21:30:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5604664fe250 con 0x5604664feb00 2024-09-10T21:30:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.708+0000 7f7ea4e00640 1 -- 172.21.15.97:0/2649163431 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7e90010850 con 0x5604664feb00 2024-09-10T21:30:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea4e00640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7e84070fc0 0x7f7e84073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea4e00640 1 -- 172.21.15.97:0/2649163431 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7f7e90092230 con 0x5604664feb00 2024-09-10T21:30:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560466525020 0x560466520190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x5604665206d0 con 0x560466525020 2024-09-10T21:30:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea7600640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7e84070fc0 0x7f7e84073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea8000640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560466525020 0x560466520190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea7600640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7e84070fc0 0x7f7e84073480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c10 tx=0x7f7e98019900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea8000640 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560466525020 0x560466520190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea8000640 1 -- 172.21.15.97:0/2649163431 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x7f7ea0002040 con 0x560466525020 2024-09-10T21:30:53.716 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/10000000000.00000000 mtime 2024-09-10T21:30:50.000000+0000, size 0 2024-09-10T21:30:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560466525020 msgr2=0x560466520190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560466525020 0x560466520190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7e84070fc0 msgr2=0x7f7e84073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7e84070fc0 0x7f7e84073480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c10 tx=0x7f7e98019900 comp rx=0 tx=0).stop 2024-09-10T21:30:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664feb00 msgr2=0x560466524ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 --2- 172.21.15.97:0/2649163431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5604664feb00 0x560466524ae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000b200 tx=0x7f7e9000dfd0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7ea8000640 1 -- 172.21.15.97:0/2649163431 reap_dead start 2024-09-10T21:30:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 shutdown_connections 2024-09-10T21:30:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 >> 172.21.15.97:0/2649163431 conn(0x560466473520 msgr2=0x560466442d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 shutdown_connections 2024-09-10T21:30:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.712+0000 7f7eab048080 1 -- 172.21.15.97:0/2649163431 wait complete. 2024-09-10T21:30:53.721 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2024-09-10T21:30:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 Processor -- start 2024-09-10T21:30:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- start start 2024-09-10T21:30:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d5f9c940 0x5597d5f6cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d6026b00 0x5597d5f9fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d5fa02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5597d5ebba00 con 0x5597d60272a0 2024-09-10T21:30:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5597d5f19f40 con 0x5597d5f9c940 2024-09-10T21:30:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5597d6029080 con 0x5597d6026b00 2024-09-10T21:30:53.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d5fa02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d5fa02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60994/0 (socket says 172.21.15.97:60994) 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 -- 172.21.15.97:0/1698759221 learned_addr learned my addr 172.21.15.97:0/1698759221 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b2fe00640 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d6026b00 0x5597d5f9fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b34c00640 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d5f9c940 0x5597d5f6cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 -- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d6026b00 msgr2=0x5597d5f9fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d6026b00 0x5597d5f9fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 -- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d5f9c940 msgr2=0x5597d5f6cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d5f9c940 0x5597d5f6cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 -- 172.21.15.97:0/1698759221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d5e94c10 con 0x5597d60272a0 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b35600640 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d5fa02a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4b300ad040 tx=0x7f4b300181c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd9b6b15d6e75ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b2ea00640 1 -- 172.21.15.97:0/1698759221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300af070 con 0x5597d60272a0 2024-09-10T21:30:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b2ea00640 1 -- 172.21.15.97:0/1698759221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b300ba070 con 0x5597d60272a0 2024-09-10T21:30:53.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b2ea00640 1 -- 172.21.15.97:0/1698759221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300b5420 con 0x5597d60272a0 2024-09-10T21:30:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- 172.21.15.97:0/1698759221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 msgr2=0x5597d5fa02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d5fa02a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4b300ad040 tx=0x7f4b300181c0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- 172.21.15.97:0/1698759221 shutdown_connections 2024-09-10T21:30:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d5fa02a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d6026b00 0x5597d5f9fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/1698759221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d5f9c940 0x5597d5f6cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- 172.21.15.97:0/1698759221 >> 172.21.15.97:0/1698759221 conn(0x5597d5f9b520 msgr2=0x5597d60433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- 172.21.15.97:0/1698759221 shutdown_connections 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.804+0000 7f4b37ad1080 1 -- 172.21.15.97:0/1698759221 wait complete. 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 Processor -- start 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 -- start start 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d5f9c940 0x5597d6042ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d6026b00 0x5597d602a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d6045d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5597d5f9de50 con 0x5597d60272a0 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5597d5f9dbb0 con 0x5597d6026b00 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5597d5f9e0f0 con 0x5597d5f9c940 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d6045d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d6045d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60996/0 (socket says 172.21.15.97:60996) 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 -- 172.21.15.97:0/534020541 learned_addr learned my addr 172.21.15.97:0/534020541 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b2fe00640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d6026b00 0x5597d602a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b34c00640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d5f9c940 0x5597d6042ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 -- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d5f9c940 msgr2=0x5597d6042ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5597d5f9c940 0x5597d6042ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 -- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d6026b00 msgr2=0x5597d602a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597d6026b00 0x5597d602a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 -- 172.21.15.97:0/534020541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d6043560 con 0x5597d60272a0 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b35600640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d6045d80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4b3001b250 tx=0x7f4b300aabb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b2d600640 1 -- 172.21.15.97:0/534020541 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300af070 con 0x5597d60272a0 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b2d600640 1 -- 172.21.15.97:0/534020541 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b300c7070 con 0x5597d60272a0 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b2d600640 1 -- 172.21.15.97:0/534020541 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b300aae60 con 0x5597d60272a0 2024-09-10T21:30:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5597d60212a0 con 0x5597d60272a0 2024-09-10T21:30:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.808+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d60217e0 con 0x5597d60272a0 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b2d600640 1 -- 172.21.15.97:0/534020541 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4b300b5990 con 0x5597d60272a0 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b2d600640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b14070ff0 0x7f4b140734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b2d600640 1 -- 172.21.15.97:0/534020541 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7f4b3012e2b0 con 0x5597d60272a0 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5597d60462c0 0x5597d60243d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b34c00640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b14070ff0 0x7f4b140734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x5597d60e07c0 con 0x5597d60462c0 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b2fe00640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5597d60462c0 0x5597d60243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b34c00640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b14070ff0 0x7f4b140734b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800b680 tx=0x7f4b2800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b2fe00640 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5597d60462c0 0x5597d60243d0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b2fe00640 1 -- 172.21.15.97:0/534020541 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f4b2000b910 con 0x5597d60462c0 2024-09-10T21:30:53.817 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/600.00000000 mtime 2024-09-10T21:30:37.000000+0000, size 0 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5597d60462c0 msgr2=0x5597d60243d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5597d60462c0 0x5597d60243d0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b14070ff0 msgr2=0x7f4b140734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4b14070ff0 0x7f4b140734b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800b680 tx=0x7f4b2800d040 comp rx=0 tx=0).stop 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 msgr2=0x5597d6045d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 --2- 172.21.15.97:0/534020541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597d60272a0 0x5597d6045d80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4b3001b250 tx=0x7f4b300aabb0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b35600640 1 -- 172.21.15.97:0/534020541 reap_dead start 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 shutdown_connections 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 >> 172.21.15.97:0/534020541 conn(0x5597d5f9b520 msgr2=0x5597d602af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 shutdown_connections 2024-09-10T21:30:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.812+0000 7f4b37ad1080 1 -- 172.21.15.97:0/534020541 wait complete. 2024-09-10T21:30:53.821 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2024-09-10T21:30:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.860+0000 7fd21462d080 1 Processor -- start 2024-09-10T21:30:53.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.860+0000 7fd21462d080 1 -- start start 2024-09-10T21:30:53.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 0x55bb9aeeb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bb9afa7e70 con 0x55bb9afa5b00 2024-09-10T21:30:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb9ae3aa00 con 0x55bb9afa62a0 2024-09-10T21:30:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55bb9afa7ff0 con 0x55bb9af1b940 2024-09-10T21:30:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32784/0 (socket says 172.21.15.97:32784) 2024-09-10T21:30:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 -- 172.21.15.97:0/1800074576 learned_addr learned my addr 172.21.15.97:0/1800074576 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd211800640 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 0x55bb9aeeb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd212200640 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 -- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 msgr2=0x55bb9aeeb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 0x55bb9aeeb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 -- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa62a0 msgr2=0x55bb9af9fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 -- 172.21.15.97:0/1800074576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9ae39a20 con 0x55bb9afa5b00 2024-09-10T21:30:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd210e00640 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f6d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c670 tx=0x7fd20000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b67683a3674d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd20b400640 1 -- 172.21.15.97:0/1800074576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20000add0 con 0x55bb9afa5b00 2024-09-10T21:30:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd20b400640 1 -- 172.21.15.97:0/1800074576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd200008540 con 0x55bb9afa5b00 2024-09-10T21:30:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd20b400640 1 -- 172.21.15.97:0/1800074576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200008970 con 0x55bb9afa5b00 2024-09-10T21:30:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- 172.21.15.97:0/1800074576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa5b00 msgr2=0x55bb9af9f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f6d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c670 tx=0x7fd20000cb30 comp rx=0 tx=0).stop 2024-09-10T21:30:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- 172.21.15.97:0/1800074576 shutdown_connections 2024-09-10T21:30:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f6d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 --2- 172.21.15.97:0/1800074576 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 0x55bb9aeeb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- 172.21.15.97:0/1800074576 >> 172.21.15.97:0/1800074576 conn(0x55bb9af1a520 msgr2=0x55bb9afc24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- 172.21.15.97:0/1800074576 shutdown_connections 2024-09-10T21:30:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- 172.21.15.97:0/1800074576 wait complete. 2024-09-10T21:30:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 Processor -- start 2024-09-10T21:30:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.864+0000 7fd21462d080 1 -- start start 2024-09-10T21:30:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 0x55bb9afa2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bb9afc3fc0 con 0x55bb9afa62a0 2024-09-10T21:30:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bb9aed84d0 con 0x55bb9afa5b00 2024-09-10T21:30:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55bb9afa4740 con 0x55bb9af1b940 2024-09-10T21:30:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd211800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 0x55bb9afa2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32792/0 (socket says 172.21.15.97:32792) 2024-09-10T21:30:53.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 -- 172.21.15.97:0/323622872 learned_addr learned my addr 172.21.15.97:0/323622872 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd210e00640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 -- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 msgr2=0x55bb9afa2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55bb9af1b940 0x55bb9afa2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 -- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa5b00 msgr2=0x55bb9af9f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 -- 172.21.15.97:0/323622872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9ae13c10 con 0x55bb9afa62a0 2024-09-10T21:30:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd210e00640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb9afa5b00 0x55bb9af9f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd212200640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fe70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0ad040 tx=0x7fd20c0aa580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd20a000640 1 -- 172.21.15.97:0/323622872 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c0bb030 con 0x55bb9afa62a0 2024-09-10T21:30:53.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd20a000640 1 -- 172.21.15.97:0/323622872 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd20c0b0040 con 0x55bb9afa62a0 2024-09-10T21:30:53.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bb9afa49e0 con 0x55bb9afa62a0 2024-09-10T21:30:53.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd20a000640 1 -- 172.21.15.97:0/323622872 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c0af070 con 0x55bb9afa62a0 2024-09-10T21:30:53.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9afa4f20 con 0x55bb9afa62a0 2024-09-10T21:30:53.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.868+0000 7fd20a000640 1 -- 172.21.15.97:0/323622872 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd20c0b5040 con 0x55bb9afa62a0 2024-09-10T21:30:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd20a000640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1f4070fc0 0x7fd1f4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd20a000640 1 -- 172.21.15.97:0/323622872 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7fd20c12d140 con 0x55bb9afa62a0 2024-09-10T21:30:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd211800640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1f4070fc0 0x7fd1f4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55bb9afbe200 0x55bb9af1d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x55bb9b05c230 con 0x55bb9afbe200 2024-09-10T21:30:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd210e00640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55bb9afbe200 0x55bb9af1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd211800640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1f4070fc0 0x7fd1f4073480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd204004fd0 tx=0x7fd20400f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd210e00640 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55bb9afbe200 0x55bb9af1d940 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd210e00640 1 -- 172.21.15.97:0/323622872 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv15 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7fd20000abd0 con 0x55bb9afbe200 2024-09-10T21:30:53.887 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/1.00000000.inode mtime 2024-09-10T21:30:50.000000+0000, size 582 2024-09-10T21:30:53.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55bb9afbe200 msgr2=0x55bb9af1d940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55bb9afbe200 0x55bb9af1d940 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1f4070fc0 msgr2=0x7fd1f4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1f4070fc0 0x7fd1f4073480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd204004fd0 tx=0x7fd20400f040 comp rx=0 tx=0).stop 2024-09-10T21:30:53.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa62a0 msgr2=0x55bb9af9fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 --2- 172.21.15.97:0/323622872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb9afa62a0 0x55bb9af9fe70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0ad040 tx=0x7fd20c0aa580 comp rx=0 tx=0).stop 2024-09-10T21:30:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd212200640 1 -- 172.21.15.97:0/323622872 reap_dead start 2024-09-10T21:30:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 shutdown_connections 2024-09-10T21:30:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 >> 172.21.15.97:0/323622872 conn(0x55bb9af1a520 msgr2=0x55bb9afa9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 shutdown_connections 2024-09-10T21:30:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.872+0000 7fd21462d080 1 -- 172.21.15.97:0/323622872 wait complete. 2024-09-10T21:30:53.892 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2024-09-10T21:30:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.976+0000 7f38a3e69080 1 Processor -- start 2024-09-10T21:30:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.976+0000 7f38a3e69080 1 -- start start 2024-09-10T21:30:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab34b4940 0x556ab355a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab353eb00 0x556ab355ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3539710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ab33d3a00 con 0x556ab353f2a0 2024-09-10T21:30:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ab3431f40 con 0x556ab353eb00 2024-09-10T21:30:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556ab3541080 con 0x556ab34b4940 2024-09-10T21:30:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3539710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3539710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32810/0 (socket says 172.21.15.97:32810) 2024-09-10T21:30:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 -- 172.21.15.97:0/4024668097 learned_addr learned my addr 172.21.15.97:0/4024668097 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1000640 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab34b4940 0x556ab355a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f389be00640 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab353eb00 0x556ab355ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 -- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab34b4940 msgr2=0x556ab355a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab34b4940 0x556ab355a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 -- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab353eb00 msgr2=0x556ab355ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab353eb00 0x556ab355ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 -- 172.21.15.97:0/4024668097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab33acc10 con 0x556ab353f2a0 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a1a00640 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3539710 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f389c0016c0 tx=0x7f389c001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce2c25602fee033 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f389aa00640 1 -- 172.21.15.97:0/4024668097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c016070 con 0x556ab353f2a0 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f389aa00640 1 -- 172.21.15.97:0/4024668097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f389c010880 con 0x556ab353f2a0 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f389aa00640 1 -- 172.21.15.97:0/4024668097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c010cd0 con 0x556ab353f2a0 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- 172.21.15.97:0/4024668097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 msgr2=0x556ab3539710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3539710 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f389c0016c0 tx=0x7f389c001b80 comp rx=0 tx=0).stop 2024-09-10T21:30:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- 172.21.15.97:0/4024668097 shutdown_connections 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3539710 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab353eb00 0x556ab355ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 --2- 172.21.15.97:0/4024668097 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab34b4940 0x556ab355a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- 172.21.15.97:0/4024668097 >> 172.21.15.97:0/4024668097 conn(0x556ab34b3520 msgr2=0x556ab355b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- 172.21.15.97:0/4024668097 shutdown_connections 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- 172.21.15.97:0/4024668097 wait complete. 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 Processor -- start 2024-09-10T21:30:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.980+0000 7f38a3e69080 1 -- start start 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab34b4940 0x556ab353bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab353eb00 0x556ab353aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3537890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ab355b170 con 0x556ab353f2a0 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ab355e900 con 0x556ab34b4940 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556ab355eba0 con 0x556ab353eb00 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3537890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3537890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32818/0 (socket says 172.21.15.97:32818) 2024-09-10T21:30:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 -- 172.21.15.97:0/1458447832 learned_addr learned my addr 172.21.15.97:0/1458447832 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f389be00640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab353eb00 0x556ab353aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1000640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab34b4940 0x556ab353bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 -- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab353eb00 msgr2=0x556ab353aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ab353eb00 0x556ab353aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 -- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab34b4940 msgr2=0x556ab353bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ab34b4940 0x556ab353bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 -- 172.21.15.97:0/1458447832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab353db30 con 0x556ab353f2a0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a1a00640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3537890 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f389c0012f0 tx=0x7f389c0105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f3899600640 1 -- 172.21.15.97:0/1458447832 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c021070 con 0x556ab353f2a0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f3899600640 1 -- 172.21.15.97:0/1458447832 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f389c010d80 con 0x556ab353f2a0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f3899600640 1 -- 172.21.15.97:0/1458447832 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c016040 con 0x556ab353f2a0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ab353ddd0 con 0x556ab353f2a0 2024-09-10T21:30:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab353e310 con 0x556ab353f2a0 2024-09-10T21:30:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.984+0000 7f3899600640 1 -- 172.21.15.97:0/1458447832 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f389c01c6a0 con 0x556ab353f2a0 2024-09-10T21:30:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f3899600640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3884070fc0 0x7f3884073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f3899600640 1 -- 172.21.15.97:0/1458447832 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7f389c093330 con 0x556ab353f2a0 2024-09-10T21:30:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ab35f8b60 0x556ab35f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a1000640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3884070fc0 0x7f3884073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x556ab35f9480 con 0x556ab35f8b60 2024-09-10T21:30:53.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f389be00640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ab35f8b60 0x556ab35f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:53.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a1000640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3884070fc0 0x7f3884073480 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f38940ab000 tx=0x7f38940199e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f389be00640 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ab35f8b60 0x556ab35f8f40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f389be00640 1 -- 172.21.15.97:0/1458447832 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f389000b9a0 con 0x556ab35f8b60 2024-09-10T21:30:53.992 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/200.00000000 mtime 2024-09-10T21:30:50.000000+0000, size 90 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ab35f8b60 msgr2=0x556ab35f8f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ab35f8b60 0x556ab35f8f40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3884070fc0 msgr2=0x7f3884073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3884070fc0 0x7f3884073480 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f38940ab000 tx=0x7f38940199e0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 msgr2=0x556ab3537890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 --2- 172.21.15.97:0/1458447832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ab353f2a0 0x556ab3537890 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f389c0012f0 tx=0x7f389c0105b0 comp rx=0 tx=0).stop 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a1a00640 1 -- 172.21.15.97:0/1458447832 reap_dead start 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 shutdown_connections 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 >> 172.21.15.97:0/1458447832 conn(0x556ab34b3520 msgr2=0x556ab355b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 shutdown_connections 2024-09-10T21:30:53.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:53.988+0000 7f38a3e69080 1 -- 172.21.15.97:0/1458447832 wait complete. 2024-09-10T21:30:53.997 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2024-09-10T21:30:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 Processor -- start 2024-09-10T21:30:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- start start 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503edfcaa0 0x56503ee27820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503eeb5ca0 0x56503eed2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee40210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56503ed4aa00 con 0x56503eeb5ca0 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56503eda8f40 con 0x56503edfcaa0 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56503ee2bda0 con 0x56503eeb6080 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503eeb5ca0 0x56503eed2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503eeb5ca0 0x56503eed2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32834/0 (socket says 172.21.15.97:32834) 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 -- 172.21.15.97:0/758984236 learned_addr learned my addr 172.21.15.97:0/758984236 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe888600640 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee40210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe887c00640 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503edfcaa0 0x56503ee27820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 -- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 msgr2=0x56503ee40210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee40210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 -- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503edfcaa0 msgr2=0x56503ee27820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503edfcaa0 0x56503ee27820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 -- 172.21.15.97:0/758984236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503ed23c10 con 0x56503eeb5ca0 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe888600640 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee40210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe87fe00640 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503eeb5ca0 0x56503eed2140 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe870019990 tx=0x7fe870018080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23d1c192b0ea46d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe886800640 1 -- 172.21.15.97:0/758984236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8700ba070 con 0x56503eeb5ca0 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe886800640 1 -- 172.21.15.97:0/758984236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8700ab980 con 0x56503eeb5ca0 2024-09-10T21:30:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe886800640 1 -- 172.21.15.97:0/758984236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8700abdd0 con 0x56503eeb5ca0 2024-09-10T21:30:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- 172.21.15.97:0/758984236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503eeb5ca0 msgr2=0x56503eed2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503eeb5ca0 0x56503eed2140 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe870019990 tx=0x7fe870018080 comp rx=0 tx=0).stop 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- 172.21.15.97:0/758984236 shutdown_connections 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee40210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503eeb5ca0 0x56503eed2140 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/758984236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503edfcaa0 0x56503ee27820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- 172.21.15.97:0/758984236 >> 172.21.15.97:0/758984236 conn(0x56503edfb140 msgr2=0x56503edfa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- 172.21.15.97:0/758984236 shutdown_connections 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- 172.21.15.97:0/758984236 wait complete. 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 Processor -- start 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.084+0000 7fe88aa2d080 1 -- start start 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503edfcaa0 0x56503ee1e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503eeb5ca0 0x56503eed4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee25740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56503eed9270 con 0x56503edfcaa0 2024-09-10T21:30:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56503eed9510 con 0x56503eeb5ca0 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56503ee27380 con 0x56503eeb6080 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503edfcaa0 0x56503ee1e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503edfcaa0 0x56503ee1e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32838/0 (socket says 172.21.15.97:32838) 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 -- 172.21.15.97:0/3990455709 learned_addr learned my addr 172.21.15.97:0/3990455709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe87fe00640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503eeb5ca0 0x56503eed4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe888600640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee25740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 -- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 msgr2=0x56503ee25740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56503eeb6080 0x56503ee25740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 -- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503eeb5ca0 msgr2=0x56503eed4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503eeb5ca0 0x56503eed4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 -- 172.21.15.97:0/3990455709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503eed51a0 con 0x56503edfcaa0 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe887c00640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503edfcaa0 0x56503ee1e8a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe87800df70 tx=0x7fe87800b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe885400640 1 -- 172.21.15.97:0/3990455709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780030d0 con 0x56503edfcaa0 2024-09-10T21:30:54.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe885400640 1 -- 172.21.15.97:0/3990455709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe878016070 con 0x56503edfcaa0 2024-09-10T21:30:54.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56503eed5440 con 0x56503edfcaa0 2024-09-10T21:30:54.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe885400640 1 -- 172.21.15.97:0/3990455709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780014e0 con 0x56503edfcaa0 2024-09-10T21:30:54.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.088+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503eed5980 con 0x56503edfcaa0 2024-09-10T21:30:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe885400640 1 -- 172.21.15.97:0/3990455709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe878001680 con 0x56503edfcaa0 2024-09-10T21:30:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe885400640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe864070fc0 0x7fe864073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe885400640 1 -- 172.21.15.97:0/3990455709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7fe8780920b0 con 0x56503edfcaa0 2024-09-10T21:30:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe87fe00640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe864070fc0 0x7fe864073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56503ee45b20 0x56503eec3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x56503eec3850 con 0x56503ee45b20 2024-09-10T21:30:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe888600640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56503ee45b20 0x56503eec3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe87fe00640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe864070fc0 0x7fe864073480 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8700b7000 tx=0x7fe870004950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe888600640 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56503ee45b20 0x56503eec3310 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe888600640 1 -- 172.21.15.97:0/3990455709 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fe88000b040 con 0x56503ee45b20 2024-09-10T21:30:54.096 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/200.00000001 mtime 2024-09-10T21:30:50.000000+0000, size 16801 2024-09-10T21:30:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56503ee45b20 msgr2=0x56503eec3310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56503ee45b20 0x56503eec3310 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe864070fc0 msgr2=0x7fe864073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe864070fc0 0x7fe864073480 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8700b7000 tx=0x7fe870004950 comp rx=0 tx=0).stop 2024-09-10T21:30:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503edfcaa0 msgr2=0x56503ee1e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 --2- 172.21.15.97:0/3990455709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56503edfcaa0 0x56503ee1e8a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe87800df70 tx=0x7fe87800b660 comp rx=0 tx=0).stop 2024-09-10T21:30:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe888600640 1 -- 172.21.15.97:0/3990455709 reap_dead start 2024-09-10T21:30:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 shutdown_connections 2024-09-10T21:30:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 >> 172.21.15.97:0/3990455709 conn(0x56503edfb140 msgr2=0x56503eed6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 shutdown_connections 2024-09-10T21:30:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.092+0000 7fe88aa2d080 1 -- 172.21.15.97:0/3990455709 wait complete. 2024-09-10T21:30:54.101 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2024-09-10T21:30:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.184+0000 7f5407970080 1 Processor -- start 2024-09-10T21:30:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.184+0000 7f5407970080 1 -- start start 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe32c80 0x55edfbd77fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe33060 0x55edfbe55eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 0x55edfbe563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55edfbcc6a00 con 0x55edfbe33060 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55edfbd24f40 con 0x55edfbe32c80 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55edfbd78f30 con 0x55edfbe36430 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe33060 0x55edfbe55eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe33060 0x55edfbe55eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32852/0 (socket says 172.21.15.97:32852) 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 -- 172.21.15.97:0/1911842764 learned_addr learned my addr 172.21.15.97:0/1911842764 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5404c00640 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe32c80 0x55edfbd77fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5405600640 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 0x55edfbe563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 -- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 msgr2=0x55edfbe563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 0x55edfbe563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 -- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe32c80 msgr2=0x55edfbd77fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe32c80 0x55edfbd77fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 -- 172.21.15.97:0/1911842764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfbc9fc10 con 0x55edfbe33060 2024-09-10T21:30:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53ffe00640 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe33060 0x55edfbe55eb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f53f000b2d0 tx=0x7f53f000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc30062b7949e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53fea00640 1 -- 172.21.15.97:0/1911842764 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f000a040 con 0x55edfbe33060 2024-09-10T21:30:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53fea00640 1 -- 172.21.15.97:0/1911842764 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f000f620 con 0x55edfbe33060 2024-09-10T21:30:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f53fea00640 1 -- 172.21.15.97:0/1911842764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f000fa70 con 0x55edfbe33060 2024-09-10T21:30:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- 172.21.15.97:0/1911842764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe33060 msgr2=0x55edfbe55eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe33060 0x55edfbe55eb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f53f000b2d0 tx=0x7f53f000b790 comp rx=0 tx=0).stop 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- 172.21.15.97:0/1911842764 shutdown_connections 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 0x55edfbe563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe33060 0x55edfbe55eb0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 --2- 172.21.15.97:0/1911842764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe32c80 0x55edfbd77fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- 172.21.15.97:0/1911842764 >> 172.21.15.97:0/1911842764 conn(0x55edfbd77140 msgr2=0x55edfbe31be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- 172.21.15.97:0/1911842764 shutdown_connections 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- 172.21.15.97:0/1911842764 wait complete. 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 Processor -- start 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.188+0000 7f5407970080 1 -- start start 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe32c80 0x55edfbd9a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe33060 0x55edfbe4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 0x55edfbe4d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55edfbe50f70 con 0x55edfbe32c80 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55edfbda6760 con 0x55edfbe33060 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55edfbda5b60 con 0x55edfbe36430 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe32c80 0x55edfbd9a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe32c80 0x55edfbd9a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32868/0 (socket says 172.21.15.97:32868) 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 -- 172.21.15.97:0/1613226771 learned_addr learned my addr 172.21.15.97:0/1613226771 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5405600640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 0x55edfbe4d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f53ffe00640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe33060 0x55edfbe4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 -- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 msgr2=0x55edfbe4d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55edfbe36430 0x55edfbe4d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 -- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe33060 msgr2=0x55edfbe4bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe33060 0x55edfbe4bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 -- 172.21.15.97:0/1613226771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfbda5e20 con 0x55edfbe32c80 2024-09-10T21:30:54.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f53ffe00640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55edfbe33060 0x55edfbe4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5404c00640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe32c80 0x55edfbd9a970 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f53f801b730 tx=0x7f53f80acaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f53fd600640 1 -- 172.21.15.97:0/1613226771 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f80acd50 con 0x55edfbe32c80 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f53fd600640 1 -- 172.21.15.97:0/1613226771 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f80b8440 con 0x55edfbe32c80 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55edfbda60c0 con 0x55edfbe32c80 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f53fd600640 1 -- 172.21.15.97:0/1613226771 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f80b8690 con 0x55edfbe32c80 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfbe4c080 con 0x55edfbe32c80 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.192+0000 7f53fd600640 1 -- 172.21.15.97:0/1613226771 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f53f80b8830 con 0x55edfbe32c80 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f53fd600640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53e4070fc0 0x7f53e4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f53fd600640 1 -- 172.21.15.97:0/1613226771 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7f53f812d610 con 0x55edfbe32c80 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f53ffe00640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53e4070fc0 0x7f53e4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55edfbe30610 0x55edfbe30a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x55edfbe30f60 con 0x55edfbe30610 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5405600640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55edfbe30610 0x55edfbe30a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f53ffe00640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53e4070fc0 0x7f53e4073480 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f53f0003040 tx=0x7f53f0000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5405600640 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55edfbe30610 0x55edfbe30a20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5405600640 1 -- 172.21.15.97:0/1613226771 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv4 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f540000b040 con 0x55edfbe30610 2024-09-10T21:30:54.201 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2024-09-10T21:30:50.000000+0000, size 0 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55edfbe30610 msgr2=0x55edfbe30a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55edfbe30610 0x55edfbe30a20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53e4070fc0 msgr2=0x7f53e4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53e4070fc0 0x7f53e4073480 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f53f0003040 tx=0x7f53f0000ee0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe32c80 msgr2=0x55edfbd9a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 --2- 172.21.15.97:0/1613226771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edfbe32c80 0x55edfbd9a970 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f53f801b730 tx=0x7f53f80acaa0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5405600640 1 -- 172.21.15.97:0/1613226771 reap_dead start 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 shutdown_connections 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 >> 172.21.15.97:0/1613226771 conn(0x55edfbd77140 msgr2=0x55edfbe510f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 shutdown_connections 2024-09-10T21:30:54.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.196+0000 7f5407970080 1 -- 172.21.15.97:0/1613226771 wait complete. 2024-09-10T21:30:54.204 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2024-09-10T21:30:54.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 Processor -- start 2024-09-10T21:30:54.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- start start 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 0x563007f0b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007e672a0 0x563007f0bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007ef32a0 0x563007e37b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563007d84a00 con 0x563007e672a0 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563007de2f40 con 0x563007ef32a0 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563007ef5560 con 0x563007e66ec0 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007e672a0 0x563007f0bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007e672a0 0x563007f0bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32888/0 (socket says 172.21.15.97:32888) 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 -- 172.21.15.97:0/277814478 learned_addr learned my addr 172.21.15.97:0/277814478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3c200640 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 0x563007f0b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3cc00640 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007ef32a0 0x563007e37b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 -- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 msgr2=0x563007f0b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 0x563007f0b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 -- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007ef32a0 msgr2=0x563007e37b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007ef32a0 0x563007e37b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 -- 172.21.15.97:0/277814478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563007d5dc10 con 0x563007e672a0 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3cc00640 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007ef32a0 0x563007e37b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3b800640 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007e672a0 0x563007f0bc80 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b640 tx=0x7f4e28004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdbaeeb80046b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3a400640 1 -- 172.21.15.97:0/277814478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800a950 con 0x563007e672a0 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3a400640 1 -- 172.21.15.97:0/277814478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e2800aaf0 con 0x563007e672a0 2024-09-10T21:30:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3a400640 1 -- 172.21.15.97:0/277814478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28008460 con 0x563007e672a0 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- 172.21.15.97:0/277814478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007e672a0 msgr2=0x563007f0bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007e672a0 0x563007f0bc80 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b640 tx=0x7f4e28004f90 comp rx=0 tx=0).stop 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- 172.21.15.97:0/277814478 shutdown_connections 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007ef32a0 0x563007e37b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007e672a0 0x563007f0bc80 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 --2- 172.21.15.97:0/277814478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 0x563007f0b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- 172.21.15.97:0/277814478 >> 172.21.15.97:0/277814478 conn(0x563007e7e0d0 msgr2=0x563007ef03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- 172.21.15.97:0/277814478 shutdown_connections 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 -- 172.21.15.97:0/277814478 wait complete. 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.292+0000 7f4e3f080080 1 Processor -- start 2024-09-10T21:30:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 -- start start 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 0x563007ee8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007e672a0 0x563007ef1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007ef32a0 0x563007ef18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563007ee9760 con 0x563007ef32a0 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563007f0cfd0 con 0x563007e672a0 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563007f0c550 con 0x563007e66ec0 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007ef32a0 0x563007ef18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007ef32a0 0x563007ef18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32902/0 (socket says 172.21.15.97:32902) 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 -- 172.21.15.97:0/2870682662 learned_addr learned my addr 172.21.15.97:0/2870682662 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3c200640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 0x563007ee8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3b800640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007e672a0 0x563007ef1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 -- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 msgr2=0x563007ee8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563007e66ec0 0x563007ee8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 -- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007e672a0 msgr2=0x563007ef1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563007e672a0 0x563007ef1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 -- 172.21.15.97:0/2870682662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563007eef7d0 con 0x563007ef32a0 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3cc00640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007ef32a0 0x563007ef18b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4e34019c10 tx=0x7f4e340a9a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e39000640 1 -- 172.21.15.97:0/2870682662 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e340c2070 con 0x563007ef32a0 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e39000640 1 -- 172.21.15.97:0/2870682662 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e340bd410 con 0x563007ef32a0 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e39000640 1 -- 172.21.15.97:0/2870682662 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e340bd800 con 0x563007ef32a0 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563007ee9a50 con 0x563007ef32a0 2024-09-10T21:30:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.296+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563007ee9f90 con 0x563007ef32a0 2024-09-10T21:30:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e39000640 1 -- 172.21.15.97:0/2870682662 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4e340b3400 con 0x563007ef32a0 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e39000640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e1c070ff0 0x7f4e1c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e39000640 1 -- 172.21.15.97:0/2870682662 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7f4e3412ce20 con 0x563007ef32a0 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3c200640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e1c070ff0 0x7f4e1c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563007eea940 0x563007eead20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x563007eeb260 con 0x563007eea940 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3b800640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563007eea940 0x563007eead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3c200640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e1c070ff0 0x7f4e1c0734b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c10 tx=0x7f4e2c001820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3b800640 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563007eea940 0x563007eead20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3b800640 1 -- 172.21.15.97:0/2870682662 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f4e2800acc0 con 0x563007eea940 2024-09-10T21:30:54.305 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/500.00000000 mtime 2024-09-10T21:30:37.000000+0000, size 90 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563007eea940 msgr2=0x563007eead20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563007eea940 0x563007eead20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e1c070ff0 msgr2=0x7f4e1c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e1c070ff0 0x7f4e1c0734b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c000c10 tx=0x7f4e2c001820 comp rx=0 tx=0).stop 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007ef32a0 msgr2=0x563007ef18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 --2- 172.21.15.97:0/2870682662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563007ef32a0 0x563007ef18b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4e34019c10 tx=0x7f4e340a9a50 comp rx=0 tx=0).stop 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3cc00640 1 -- 172.21.15.97:0/2870682662 reap_dead start 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 shutdown_connections 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 >> 172.21.15.97:0/2870682662 conn(0x563007e7e0d0 msgr2=0x563007f0ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 shutdown_connections 2024-09-10T21:30:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.300+0000 7f4e3f080080 1 -- 172.21.15.97:0/2870682662 wait complete. 2024-09-10T21:30:54.309 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2024-09-10T21:30:54.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.396+0000 7f1d72ec1080 1 Processor -- start 2024-09-10T21:30:54.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.396+0000 7f1d72ec1080 1 -- start start 2024-09-10T21:30:54.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7ad9cc0 0x557cf7a6b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7adba00 0x557cf7a6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adf1d0 0x557cf7a51070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557cf796fa00 con 0x557cf7ad9cc0 2024-09-10T21:30:54.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557cf79cdf40 con 0x557cf7adf1d0 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557cf7add710 con 0x557cf7adba00 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7ad9cc0 0x557cf7a6b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7ad9cc0 0x557cf7a6b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32920/0 (socket says 172.21.15.97:32920) 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 -- 172.21.15.97:0/2982261085 learned_addr learned my addr 172.21.15.97:0/2982261085 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 -- 172.21.15.97:0/2982261085 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7adba00 msgr2=0x557cf7a6b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 --2- 172.21.15.97:0/2982261085 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7adba00 0x557cf7a6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 -- 172.21.15.97:0/2982261085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adf1d0 msgr2=0x557cf7a51070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 --2- 172.21.15.97:0/2982261085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adf1d0 0x557cf7a51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 -- 172.21.15.97:0/2982261085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf7948c10 con 0x557cf7ad9cc0 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d70000640 1 --2- 172.21.15.97:0/2982261085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7ad9cc0 0x557cf7a6b2d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b2b0 tx=0x7f1d60001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b1d80caef9e66d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d6e200640 1 -- 172.21.15.97:0/2982261085 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60016070 con 0x557cf7ad9cc0 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d6e200640 1 -- 172.21.15.97:0/2982261085 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d6000d4c0 con 0x557cf7ad9cc0 2024-09-10T21:30:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d6e200640 1 -- 172.21.15.97:0/2982261085 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000d6f0 con 0x557cf7ad9cc0 2024-09-10T21:30:54.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- 172.21.15.97:0/2982261085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7ad9cc0 msgr2=0x557cf7a6b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/2982261085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7ad9cc0 0x557cf7a6b2d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b2b0 tx=0x7f1d60001e80 comp rx=0 tx=0).stop 2024-09-10T21:30:54.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- 172.21.15.97:0/2982261085 shutdown_connections 2024-09-10T21:30:54.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/2982261085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adf1d0 0x557cf7a51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/2982261085 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7adba00 0x557cf7a6b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/2982261085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7ad9cc0 0x557cf7a6b2d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- 172.21.15.97:0/2982261085 >> 172.21.15.97:0/2982261085 conn(0x557cf7a690d0 msgr2=0x557cf7a24360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- 172.21.15.97:0/2982261085 shutdown_connections 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- 172.21.15.97:0/2982261085 wait complete. 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 Processor -- start 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.400+0000 7f1d72ec1080 1 -- start start 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7ad9cc0 0x557cf7affcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adba00 0x557cf7a680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7adf1d0 0x557cf7b927b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557cf7af86a0 con 0x557cf7adf1d0 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557cf7af8900 con 0x557cf7adba00 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557cf7af8ba0 con 0x557cf7ad9cc0 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7adf1d0 0x557cf7b927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7adf1d0 0x557cf7b927b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32934/0 (socket says 172.21.15.97:32934) 2024-09-10T21:30:54.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 -- 172.21.15.97:0/3579972372 learned_addr learned my addr 172.21.15.97:0/3579972372 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70000640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7ad9cc0 0x557cf7affcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d6f600640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adba00 0x557cf7a680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 -- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7ad9cc0 msgr2=0x557cf7affcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557cf7ad9cc0 0x557cf7affcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 -- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adba00 msgr2=0x557cf7a680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adba00 0x557cf7a680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 -- 172.21.15.97:0/3579972372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf7af53b0 con 0x557cf7adf1d0 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d6f600640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cf7adba00 0x557cf7a680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d70a00640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7adf1d0 0x557cf7b927b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800a600 tx=0x7f1d680aad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d6ce00640 1 -- 172.21.15.97:0/3579972372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d680ba030 con 0x557cf7adf1d0 2024-09-10T21:30:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d6ce00640 1 -- 172.21.15.97:0/3579972372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d68004020 con 0x557cf7adf1d0 2024-09-10T21:30:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557cf7a63be0 con 0x557cf7adf1d0 2024-09-10T21:30:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d6ce00640 1 -- 172.21.15.97:0/3579972372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d680af030 con 0x557cf7adf1d0 2024-09-10T21:30:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cf7a68600 con 0x557cf7adf1d0 2024-09-10T21:30:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.404+0000 7f1d6ce00640 1 -- 172.21.15.97:0/3579972372 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d680c7060 con 0x557cf7adf1d0 2024-09-10T21:30:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d6ce00640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d50070ff0 0x7f1d500734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d6ce00640 1 -- 172.21.15.97:0/3579972372 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7f1d6812cb40 con 0x557cf7adf1d0 2024-09-10T21:30:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d70000640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d50070ff0 0x7f1d500734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557cf7b93200 0x557cf7a62a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x557cf7a62f90 con 0x557cf7b93200 2024-09-10T21:30:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d6f600640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557cf7b93200 0x557cf7a62a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d70000640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d50070ff0 0x7f1d500734b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1d60014040 tx=0x7f1d60001090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d6f600640 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557cf7b93200 0x557cf7a62a50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d6f600640 1 -- 172.21.15.97:0/3579972372 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f1d5c00c700 con 0x557cf7b93200 2024-09-10T21:30:54.412 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/mds_snaptable mtime 2024-09-10T21:30:37.000000+0000, size 70 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557cf7b93200 msgr2=0x557cf7a62a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557cf7b93200 0x557cf7a62a50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d50070ff0 msgr2=0x7f1d500734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d50070ff0 0x7f1d500734b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1d60014040 tx=0x7f1d60001090 comp rx=0 tx=0).stop 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7adf1d0 msgr2=0x557cf7b927b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 --2- 172.21.15.97:0/3579972372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557cf7adf1d0 0x557cf7b927b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800a600 tx=0x7f1d680aad70 comp rx=0 tx=0).stop 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d70a00640 1 -- 172.21.15.97:0/3579972372 reap_dead start 2024-09-10T21:30:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 shutdown_connections 2024-09-10T21:30:54.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 >> 172.21.15.97:0/3579972372 conn(0x557cf7a690d0 msgr2=0x557cf7af3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 shutdown_connections 2024-09-10T21:30:54.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.408+0000 7f1d72ec1080 1 -- 172.21.15.97:0/3579972372 wait complete. 2024-09-10T21:30:54.417 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2024-09-10T21:30:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.496+0000 7fc75b4c8080 1 Processor -- start 2024-09-10T21:30:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- start start 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7bfcc0 0x55cd1e7c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e8735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7c51d0 0x55cd1e873b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd1e655a00 con 0x55cd1e7bfcc0 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cd1e6b3f40 con 0x55cd1e7c1830 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cd1e7e8440 con 0x55cd1e7c51d0 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7bfcc0 0x55cd1e7c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7bfcc0 0x55cd1e7c20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32954/0 (socket says 172.21.15.97:32954) 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 -- 172.21.15.97:0/940953785 learned_addr learned my addr 172.21.15.97:0/940953785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc759000640 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7c51d0 0x55cd1e873b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc757c00640 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e8735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 -- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7c51d0 msgr2=0x55cd1e873b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7c51d0 0x55cd1e873b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 -- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 msgr2=0x55cd1e8735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e8735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 -- 172.21.15.97:0/940953785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1e62ec10 con 0x55cd1e7bfcc0 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc757c00640 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e8735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc758600640 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7bfcc0 0x55cd1e7c20c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c610 tx=0x7fc74800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd62dd10f5b6fd14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc756800640 1 -- 172.21.15.97:0/940953785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74800aa70 con 0x55cd1e7bfcc0 2024-09-10T21:30:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc756800640 1 -- 172.21.15.97:0/940953785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc74800ac10 con 0x55cd1e7bfcc0 2024-09-10T21:30:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc756800640 1 -- 172.21.15.97:0/940953785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748015440 con 0x55cd1e7bfcc0 2024-09-10T21:30:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- 172.21.15.97:0/940953785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7bfcc0 msgr2=0x55cd1e7c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7bfcc0 0x55cd1e7c20c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c610 tx=0x7fc74800cad0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- 172.21.15.97:0/940953785 shutdown_connections 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7c51d0 0x55cd1e873b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e8735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/940953785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7bfcc0 0x55cd1e7c20c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- 172.21.15.97:0/940953785 >> 172.21.15.97:0/940953785 conn(0x55cd1e74f0d0 msgr2=0x55cd1e737f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- 172.21.15.97:0/940953785 shutdown_connections 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- 172.21.15.97:0/940953785 wait complete. 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 Processor -- start 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.500+0000 7fc75b4c8080 1 -- start start 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7bfcc0 0x55cd1e877260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e879fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7c51d0 0x55cd1e7db460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd1e737200 con 0x55cd1e7c51d0 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cd1e7bf4c0 con 0x55cd1e7c1830 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cd1e7374a0 con 0x55cd1e7bfcc0 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7c51d0 0x55cd1e7db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7c51d0 0x55cd1e7db460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32970/0 (socket says 172.21.15.97:32970) 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 -- 172.21.15.97:0/2378412053 learned_addr learned my addr 172.21.15.97:0/2378412053 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc758600640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7bfcc0 0x55cd1e877260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc757c00640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e879fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 -- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7bfcc0 msgr2=0x55cd1e877260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7bfcc0 0x55cd1e877260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 -- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 msgr2=0x55cd1e879fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cd1e7c1830 0x55cd1e879fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 -- 172.21.15.97:0/2378412053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1e87ac10 con 0x55cd1e7c51d0 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc758600640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cd1e7bfcc0 0x55cd1e877260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc759000640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7c51d0 0x55cd1e7db460 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc75000d390 tx=0x7fc75000d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc755400640 1 -- 172.21.15.97:0/2378412053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750011590 con 0x55cd1e7c51d0 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc755400640 1 -- 172.21.15.97:0/2378412053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc750017070 con 0x55cd1e7c51d0 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cd1e87aeb0 con 0x55cd1e7c51d0 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc755400640 1 -- 172.21.15.97:0/2378412053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750011a60 con 0x55cd1e7c51d0 2024-09-10T21:30:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1e87b3f0 con 0x55cd1e7c51d0 2024-09-10T21:30:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.504+0000 7fc755400640 1 -- 172.21.15.97:0/2378412053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc750011c00 con 0x55cd1e7c51d0 2024-09-10T21:30:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc755400640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc738070ff0 0x7fc7380734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc755400640 1 -- 172.21.15.97:0/2378412053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7fc750091800 con 0x55cd1e7c51d0 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc758600640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc738070ff0 0x7fc7380734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cd1e7db9a0 0x55cd1e7df380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x55cd1e7df8c0 con 0x55cd1e7db9a0 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc757c00640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cd1e7db9a0 0x55cd1e7df380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc758600640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc738070ff0 0x7fc7380734b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc748000c10 tx=0x7fc748017040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc757c00640 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cd1e7db9a0 0x55cd1e7df380 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc757c00640 1 -- 172.21.15.97:0/2378412053 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv3 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7fc7440b3040 con 0x55cd1e7db9a0 2024-09-10T21:30:54.512 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/mds0_inotable mtime 2024-09-10T21:30:50.000000+0000, size 34 2024-09-10T21:30:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cd1e7db9a0 msgr2=0x55cd1e7df380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cd1e7db9a0 0x55cd1e7df380 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc738070ff0 msgr2=0x7fc7380734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc738070ff0 0x7fc7380734b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc748000c10 tx=0x7fc748017040 comp rx=0 tx=0).stop 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7c51d0 msgr2=0x55cd1e7db460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 --2- 172.21.15.97:0/2378412053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd1e7c51d0 0x55cd1e7db460 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc75000d390 tx=0x7fc75000d850 comp rx=0 tx=0).stop 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc759000640 1 -- 172.21.15.97:0/2378412053 reap_dead start 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 shutdown_connections 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 >> 172.21.15.97:0/2378412053 conn(0x55cd1e74f0d0 msgr2=0x55cd1e737f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 shutdown_connections 2024-09-10T21:30:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.508+0000 7fc75b4c8080 1 -- 172.21.15.97:0/2378412053 wait complete. 2024-09-10T21:30:54.517 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2024-09-10T21:30:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 Processor -- start 2024-09-10T21:30:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- start start 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe30c003100 0x55dabfd835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfdb05f0 0x55dabfee7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dabfe38810 0x55dabfee82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dabfd2df40 con 0x55dabfdb05f0 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dabfd3ad00 con 0x7fe30c003100 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55dabfccfa00 con 0x55dabfe38810 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfdb05f0 0x55dabfee7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfdb05f0 0x55dabfee7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32998/0 (socket says 172.21.15.97:32998) 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 -- 172.21.15.97:0/2379387737 learned_addr learned my addr 172.21.15.97:0/2379387737 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe326400640 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe30c003100 0x55dabfd835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe326e00640 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dabfe38810 0x55dabfee82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 -- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dabfe38810 msgr2=0x55dabfee82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dabfe38810 0x55dabfee82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 -- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe30c003100 msgr2=0x55dabfd835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe30c003100 0x55dabfd835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 -- 172.21.15.97:0/2379387737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabfdaf820 con 0x55dabfdb05f0 2024-09-10T21:30:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe325a00640 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfdb05f0 0x55dabfee7da0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe314097010 tx=0x7fe30c004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becddcf3ff958b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe31fe00640 1 -- 172.21.15.97:0/2379387737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3140a8930 con 0x55dabfdb05f0 2024-09-10T21:30:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe31fe00640 1 -- 172.21.15.97:0/2379387737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3140bb070 con 0x55dabfdb05f0 2024-09-10T21:30:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe31fe00640 1 -- 172.21.15.97:0/2379387737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3140a8e40 con 0x55dabfdb05f0 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- 172.21.15.97:0/2379387737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfdb05f0 msgr2=0x55dabfee7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfdb05f0 0x55dabfee7da0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe314097010 tx=0x7fe30c004130 comp rx=0 tx=0).stop 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- 172.21.15.97:0/2379387737 shutdown_connections 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dabfe38810 0x55dabfee82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfdb05f0 0x55dabfee7da0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 --2- 172.21.15.97:0/2379387737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe30c003100 0x55dabfd835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- 172.21.15.97:0/2379387737 >> 172.21.15.97:0/2379387737 conn(0x55dabfdc90d0 msgr2=0x55dabfdb1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- 172.21.15.97:0/2379387737 shutdown_connections 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.560+0000 7fe329277080 1 -- 172.21.15.97:0/2379387737 wait complete. 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 Processor -- start 2024-09-10T21:30:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 -- start start 2024-09-10T21:30:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dabfdb05f0 0x55dabfee2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfe38810 0x55dabfe54320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe30c003100 0x55dabfe55040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dabfee7660 con 0x55dabfe38810 2024-09-10T21:30:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dabfdafac0 con 0x55dabfdb05f0 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55dabfe396c0 con 0x7fe30c003100 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfe38810 0x55dabfe54320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfe38810 0x55dabfe54320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33004/0 (socket says 172.21.15.97:33004) 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 -- 172.21.15.97:0/234428017 learned_addr learned my addr 172.21.15.97:0/234428017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe326400640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dabfdb05f0 0x55dabfee2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe326e00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe30c003100 0x55dabfe55040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 -- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe30c003100 msgr2=0x55dabfe55040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe30c003100 0x55dabfe55040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 -- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dabfdb05f0 msgr2=0x55dabfee2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dabfdb05f0 0x55dabfee2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 -- 172.21.15.97:0/234428017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabfee34e0 con 0x55dabfe38810 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe326e00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe30c003100 0x55dabfe55040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe326400640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dabfdb05f0 0x55dabfee2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe325a00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfe38810 0x55dabfe54320 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe31401b540 tx=0x7fe3140a8960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe31ea00640 1 -- 172.21.15.97:0/234428017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3140b3830 con 0x55dabfe38810 2024-09-10T21:30:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe31ea00640 1 -- 172.21.15.97:0/234428017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3140c4070 con 0x55dabfe38810 2024-09-10T21:30:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dabfee3780 con 0x55dabfe38810 2024-09-10T21:30:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe31ea00640 1 -- 172.21.15.97:0/234428017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3140b3d40 con 0x55dabfe38810 2024-09-10T21:30:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.564+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dabfd81310 con 0x55dabfe38810 2024-09-10T21:30:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe31ea00640 1 -- 172.21.15.97:0/234428017 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe3140c9460 con 0x55dabfe38810 2024-09-10T21:30:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe31ea00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe308070fc0 0x7fe308073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe31ea00640 1 -- 172.21.15.97:0/234428017 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6361+0+0 (secure 0 0 0) 0x7fe31412e9a0 con 0x55dabfe38810 2024-09-10T21:30:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe326400640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe308070fc0 0x7fe308073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dabfd81cc0 0x55dabfd820a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x55dabfd825e0 con 0x55dabfd81cc0 2024-09-10T21:30:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe326e00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dabfd81cc0 0x55dabfd820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe326400640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe308070fc0 0x7fe308073480 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe318001cd0 tx=0x7fe318005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe326e00640 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dabfd81cc0 0x55dabfd820a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe326e00640 1 -- 172.21.15.97:0/234428017 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fe3200096b0 con 0x55dabfd81cc0 2024-09-10T21:30:54.574 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/100.00000000 mtime 2024-09-10T21:30:37.000000+0000, size 0 2024-09-10T21:30:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dabfd81cc0 msgr2=0x55dabfd820a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dabfd81cc0 0x55dabfd820a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe308070fc0 msgr2=0x7fe308073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe308070fc0 0x7fe308073480 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe318001cd0 tx=0x7fe318005040 comp rx=0 tx=0).stop 2024-09-10T21:30:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfe38810 msgr2=0x55dabfe54320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 --2- 172.21.15.97:0/234428017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabfe38810 0x55dabfe54320 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe31401b540 tx=0x7fe3140a8960 comp rx=0 tx=0).stop 2024-09-10T21:30:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe326e00640 1 -- 172.21.15.97:0/234428017 reap_dead start 2024-09-10T21:30:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 shutdown_connections 2024-09-10T21:30:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 >> 172.21.15.97:0/234428017 conn(0x55dabfdc90d0 msgr2=0x55dabfee6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 shutdown_connections 2024-09-10T21:30:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.568+0000 7fe329277080 1 -- 172.21.15.97:0/234428017 wait complete. 2024-09-10T21:30:54.578 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2024-09-10T21:30:54.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 Processor -- start 2024-09-10T21:30:54.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- start start 2024-09-10T21:30:54.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabe10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecbca0 0x557eabee9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecc080 0x557eabee6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557eabd60a00 con 0x557eabecbca0 2024-09-10T21:30:54.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557eabdbef40 con 0x557eabecc080 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557eabef22d0 con 0x557eabe12aa0 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecbca0 0x557eabee9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecbca0 0x557eabee9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33028/0 (socket says 172.21.15.97:33028) 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 -- 172.21.15.97:0/472829841 learned_addr learned my addr 172.21.15.97:0/472829841 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb4a00640 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecc080 0x557eabee6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb4000640 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabe10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 -- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 msgr2=0x557eabe10030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabe10030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 -- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecc080 msgr2=0x557eabee6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecc080 0x557eabee6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 -- 172.21.15.97:0/472829841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eabd39c10 con 0x557eabecbca0 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb3600640 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecbca0 0x557eabee9540 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000b670 tx=0x7f6ba0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531c242f6ca82014 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb2200640 1 -- 172.21.15.97:0/472829841 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000a920 con 0x557eabecbca0 2024-09-10T21:30:54.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb2200640 1 -- 172.21.15.97:0/472829841 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ba000aac0 con 0x557eabecbca0 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb2200640 1 -- 172.21.15.97:0/472829841 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0008460 con 0x557eabecbca0 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/472829841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecbca0 msgr2=0x557eabee9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecbca0 0x557eabee9540 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000b670 tx=0x7f6ba0004f90 comp rx=0 tx=0).stop 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/472829841 shutdown_connections 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecc080 0x557eabee6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecbca0 0x557eabee9540 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/472829841 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabe10030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/472829841 >> 172.21.15.97:0/472829841 conn(0x557eabe11140 msgr2=0x557eabee52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/472829841 shutdown_connections 2024-09-10T21:30:54.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.676+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/472829841 wait complete. 2024-09-10T21:30:54.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 Processor -- start 2024-09-10T21:30:54.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 -- start start 2024-09-10T21:30:54.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabecaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecbca0 0x557eabee4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecc080 0x557eabee4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557eabeeac20 con 0x557eabecc080 2024-09-10T21:30:54.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557eabeeaec0 con 0x557eabecbca0 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557eabee7e40 con 0x557eabe12aa0 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabecaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb3600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecbca0 0x557eabee4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecc080 0x557eabee4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabecaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:51934/0 (socket says 172.21.15.97:51934) 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb3600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecbca0 0x557eabee4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:59342/0 (socket says 172.21.15.97:59342) 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4000640 1 -- 172.21.15.97:0/38980060 learned_addr learned my addr 172.21.15.97:0/38980060 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4a00640 1 -- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 msgr2=0x557eabecaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4a00640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557eabe12aa0 0x557eabecaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4a00640 1 -- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecbca0 msgr2=0x557eabee4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4a00640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557eabecbca0 0x557eabee4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4a00640 1 -- 172.21.15.97:0/38980060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eabf85bf0 con 0x557eabecc080 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb4a00640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecc080 0x557eabee4b50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x557eabe58f20 tx=0x7f6bac0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb0e00640 1 -- 172.21.15.97:0/38980060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac004020 con 0x557eabecc080 2024-09-10T21:30:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557eabf85e90 con 0x557eabecc080 2024-09-10T21:30:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb0e00640 1 -- 172.21.15.97:0/38980060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bac0041c0 con 0x557eabecc080 2024-09-10T21:30:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb0e00640 1 -- 172.21.15.97:0/38980060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac003680 con 0x557eabecc080 2024-09-10T21:30:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.680+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eabecb450 con 0x557eabecc080 2024-09-10T21:30:54.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb0e00640 1 -- 172.21.15.97:0/38980060 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6bac003820 con 0x557eabecc080 2024-09-10T21:30:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb0e00640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb0e00640 1 -- 172.21.15.97:0/38980060 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f6bac12d1e0 con 0x557eabecc080 2024-09-10T21:30:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557eabee6e00 0x557eabee71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x557eabee7720 con 0x557eabee6e00 2024-09-10T21:30:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb4000640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb3600640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557eabee6e00 0x557eabee71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb3600640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557eabee6e00 0x557eabee71e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.684+0000 7f6bb4000640 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4011630 tx=0x7f6ba40017e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb3600640 1 -- 172.21.15.97:0/38980060 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv3 ondisk = 0) ==== 159+0+16 (crc 0 0 0) 0x7f6ba000ac90 con 0x557eabee6e00 2024-09-10T21:30:54.690 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/mds0_sessionmap mtime 2024-09-10T21:30:50.000000+0000, size 0 2024-09-10T21:30:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557eabee6e00 msgr2=0x557eabee71e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557eabee6e00 0x557eabee71e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 msgr2=0x7f6b940734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4011630 tx=0x7f6ba40017e0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecc080 msgr2=0x557eabee4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 --2- 172.21.15.97:0/38980060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557eabecc080 0x557eabee4b50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x557eabe58f20 tx=0x7f6bac0049d0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb4a00640 1 -- 172.21.15.97:0/38980060 reap_dead start 2024-09-10T21:30:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 shutdown_connections 2024-09-10T21:30:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 >> 172.21.15.97:0/38980060 conn(0x557eabe11140 msgr2=0x557eabee9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 shutdown_connections 2024-09-10T21:30:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.688+0000 7f6bb7c2b080 1 -- 172.21.15.97:0/38980060 wait complete. 2024-09-10T21:30:54.696 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2024-09-10T21:30:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 Processor -- start 2024-09-10T21:30:54.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- start start 2024-09-10T21:30:54.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd90c1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a4a00 0x55fdd901a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a81d0 0x55fdd901a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fdd8f38a00 con 0x55fdd90a4a00 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fdd8f96f40 con 0x55fdd90a81d0 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55fdd90a6710 con 0x55fdd90a2cc0 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a4a00 0x55fdd901a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a4a00 0x55fdd901a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33034/0 (socket says 172.21.15.97:33034) 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 -- 172.21.15.97:0/806844769 learned_addr learned my addr 172.21.15.97:0/806844769 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fed200640 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd90c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 -- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 msgr2=0x55fdd90c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fedc00640 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a81d0 0x55fdd901a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd90c1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 -- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a81d0 msgr2=0x55fdd901a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a81d0 0x55fdd901a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 -- 172.21.15.97:0/806844769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd8f11c10 con 0x55fdd90a4a00 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fed200640 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd90c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fedc00640 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a81d0 0x55fdd901a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe7e00640 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a4a00 0x55fdd901a200 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800b2e0 tx=0x7f7fd800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02639bcda9bde0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe6a00640 1 -- 172.21.15.97:0/806844769 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8012070 con 0x55fdd90a4a00 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe6a00640 1 -- 172.21.15.97:0/806844769 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fd800e930 con 0x55fdd90a4a00 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7fe6a00640 1 -- 172.21.15.97:0/806844769 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800ed80 con 0x55fdd90a4a00 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- 172.21.15.97:0/806844769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a4a00 msgr2=0x55fdd901a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a4a00 0x55fdd901a200 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800b2e0 tx=0x7f7fd800b7a0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- 172.21.15.97:0/806844769 shutdown_connections 2024-09-10T21:30:54.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a81d0 0x55fdd901a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a4a00 0x55fdd901a200 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 --2- 172.21.15.97:0/806844769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd90c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- 172.21.15.97:0/806844769 >> 172.21.15.97:0/806844769 conn(0x55fdd90320d0 msgr2=0x55fdd8fed360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- 172.21.15.97:0/806844769 shutdown_connections 2024-09-10T21:30:54.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 -- 172.21.15.97:0/806844769 wait complete. 2024-09-10T21:30:54.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.740+0000 7f7feff9e080 1 Processor -- start 2024-09-10T21:30:54.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 -- start start 2024-09-10T21:30:54.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd9159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a4a00 0x55fdd8fea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a81d0 0x55fdd8feaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9034450 con 0x55fdd90a81d0 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fdd90a1990 con 0x55fdd90a4a00 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55fdd90346f0 con 0x55fdd90a2cc0 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a81d0 0x55fdd8feaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a81d0 0x55fdd8feaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33044/0 (socket says 172.21.15.97:33044) 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 -- 172.21.15.97:0/2331241538 learned_addr learned my addr 172.21.15.97:0/2331241538 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fe7e00640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a4a00 0x55fdd8fea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fed200640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd9159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 -- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 msgr2=0x55fdd9159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fdd90a2cc0 0x55fdd9159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 -- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a4a00 msgr2=0x55fdd8fea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fdd90a4a00 0x55fdd8fea5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 -- 172.21.15.97:0/2331241538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd8fe9bf0 con 0x55fdd90a81d0 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fedc00640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a81d0 0x55fdd8feaaf0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8096f90 tx=0x7f7fe80a9440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fe5600640 1 -- 172.21.15.97:0/2331241538 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80c1070 con 0x55fdd90a81d0 2024-09-10T21:30:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fe5600640 1 -- 172.21.15.97:0/2331241538 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fe80a98d0 con 0x55fdd90a81d0 2024-09-10T21:30:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7fe5600640 1 -- 172.21.15.97:0/2331241538 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80a9ca0 con 0x55fdd90a81d0 2024-09-10T21:30:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fdd8fe9e90 con 0x55fdd90a81d0 2024-09-10T21:30:54.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.744+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd90c8f60 con 0x55fdd90a81d0 2024-09-10T21:30:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fe5600640 1 -- 172.21.15.97:0/2331241538 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7fe80b23f0 con 0x55fdd90a81d0 2024-09-10T21:30:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fe5600640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fcc070fc0 0x7f7fcc073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fe5600640 1 -- 172.21.15.97:0/2331241538 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f7fe812cb00 con 0x55fdd90a81d0 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fed200640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fcc070fc0 0x7f7fcc073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fdd915d320 0x55fdd915d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55fdd915dc40 con 0x55fdd915d320 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fe7e00640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fdd915d320 0x55fdd915d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fed200640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fcc070fc0 0x7f7fcc073480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c830 tx=0x7f7fe0004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fe7e00640 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fdd915d320 0x55fdd915d700 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fe7e00640 1 -- 172.21.15.97:0/2331241538 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f7fd8014030 con 0x55fdd915d320 2024-09-10T21:30:54.753 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/400.00000000 mtime 2024-09-10T21:30:37.000000+0000, size 22 2024-09-10T21:30:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fdd915d320 msgr2=0x55fdd915d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fdd915d320 0x55fdd915d700 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fcc070fc0 msgr2=0x7f7fcc073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fcc070fc0 0x7f7fcc073480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c830 tx=0x7f7fe0004890 comp rx=0 tx=0).stop 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a81d0 msgr2=0x55fdd8feaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 --2- 172.21.15.97:0/2331241538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd90a81d0 0x55fdd8feaaf0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8096f90 tx=0x7f7fe80a9440 comp rx=0 tx=0).stop 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7fedc00640 1 -- 172.21.15.97:0/2331241538 reap_dead start 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 shutdown_connections 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 >> 172.21.15.97:0/2331241538 conn(0x55fdd90320d0 msgr2=0x55fdd90c1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 shutdown_connections 2024-09-10T21:30:54.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.748+0000 7f7feff9e080 1 -- 172.21.15.97:0/2331241538 wait complete. 2024-09-10T21:30:54.757 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2024-09-10T21:30:54.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.836+0000 7fa590326080 1 Processor -- start 2024-09-10T21:30:54.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- start start 2024-09-10T21:30:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de6cc0 0x564d52de8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52de8830 0x564d52d5d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52dec1d0 0x564d52e03f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d52c7ca00 con 0x564d52de6cc0 2024-09-10T21:30:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564d52cdaf40 con 0x564d52dec1d0 2024-09-10T21:30:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564d52e0f440 con 0x564d52de8830 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de6cc0 0x564d52de8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de6cc0 0x564d52de8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33074/0 (socket says 172.21.15.97:33074) 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 -- 172.21.15.97:0/1549294955 learned_addr learned my addr 172.21.15.97:0/1549294955 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58cc00640 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52de8830 0x564d52d5d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58e000640 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52dec1d0 0x564d52e03f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 -- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52de8830 msgr2=0x564d52d5d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52de8830 0x564d52d5d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 -- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52dec1d0 msgr2=0x564d52e03f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52dec1d0 0x564d52e03f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 -- 172.21.15.97:0/1549294955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d52c55c10 con 0x564d52de6cc0 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58e000640 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52dec1d0 0x564d52e03f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa58d600640 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de6cc0 0x564d52de8e50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa58000b5b0 tx=0x7fa58000ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1c78fda62815af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa587400640 1 -- 172.21.15.97:0/1549294955 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580009a80 con 0x564d52de6cc0 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa587400640 1 -- 172.21.15.97:0/1549294955 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa580009c20 con 0x564d52de6cc0 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa587400640 1 -- 172.21.15.97:0/1549294955 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580014440 con 0x564d52de6cc0 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- 172.21.15.97:0/1549294955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de6cc0 msgr2=0x564d52de8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de6cc0 0x564d52de8e50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa58000b5b0 tx=0x7fa58000ba70 comp rx=0 tx=0).stop 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- 172.21.15.97:0/1549294955 shutdown_connections 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52dec1d0 0x564d52e03f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52de8830 0x564d52d5d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 --2- 172.21.15.97:0/1549294955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de6cc0 0x564d52de8e50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- 172.21.15.97:0/1549294955 >> 172.21.15.97:0/1549294955 conn(0x564d52d760d0 msgr2=0x564d52d5f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- 172.21.15.97:0/1549294955 shutdown_connections 2024-09-10T21:30:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- 172.21.15.97:0/1549294955 wait complete. 2024-09-10T21:30:54.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 Processor -- start 2024-09-10T21:30:54.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.840+0000 7fa590326080 1 -- start start 2024-09-10T21:30:54.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52de6cc0 0x564d52d712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de8830 0x564d52d786c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52dec1d0 0x564d52de3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d52d5cdd0 con 0x564d52de8830 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564d52d5cb30 con 0x564d52de6cc0 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564d52dfeec0 con 0x564d52dec1d0 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de8830 0x564d52d786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de8830 0x564d52d786c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33084/0 (socket says 172.21.15.97:33084) 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 -- 172.21.15.97:0/293569314 learned_addr learned my addr 172.21.15.97:0/293569314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58d600640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52de6cc0 0x564d52d712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58e000640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52dec1d0 0x564d52de3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 -- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52dec1d0 msgr2=0x564d52de3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564d52dec1d0 0x564d52de3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 -- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52de6cc0 msgr2=0x564d52d712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564d52de6cc0 0x564d52d712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 -- 172.21.15.97:0/293569314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d52d70620 con 0x564d52de8830 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa58cc00640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de8830 0x564d52d786c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa57c096d50 tx=0x7fa57c0acaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa586000640 1 -- 172.21.15.97:0/293569314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0bf020 con 0x564d52de8830 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa586000640 1 -- 172.21.15.97:0/293569314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa57c0acd50 con 0x564d52de8830 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa586000640 1 -- 172.21.15.97:0/293569314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0b86a0 con 0x564d52de8830 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564d52d708c0 con 0x564d52de8830 2024-09-10T21:30:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d52d70e00 con 0x564d52de8830 2024-09-10T21:30:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.844+0000 7fa586000640 1 -- 172.21.15.97:0/293569314 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa57c0b8840 con 0x564d52de8830 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa586000640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa570070fc0 0x7fa570073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa586000640 1 -- 172.21.15.97:0/293569314 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fa57c12d230 con 0x564d52de8830 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa58d600640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa570070fc0 0x7fa570073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564d52d78c00 0x564d52d730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x564d52dea230 con 0x564d52d78c00 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa58e000640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564d52d78c00 0x564d52d730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa58d600640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa570070fc0 0x7fa570073480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa580000c10 tx=0x7fa580016040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa58e000640 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564d52d78c00 0x564d52d730f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa58e000640 1 -- 172.21.15.97:0/293569314 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x7fa5880106d0 con 0x564d52d78c00 2024-09-10T21:30:54.852 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/100.00000000.inode mtime 2024-09-10T21:30:37.000000+0000, size 582 2024-09-10T21:30:54.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564d52d78c00 msgr2=0x564d52d730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564d52d78c00 0x564d52d730f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa570070fc0 msgr2=0x7fa570073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa570070fc0 0x7fa570073480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa580000c10 tx=0x7fa580016040 comp rx=0 tx=0).stop 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de8830 msgr2=0x564d52d786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 --2- 172.21.15.97:0/293569314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d52de8830 0x564d52d786c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa57c096d50 tx=0x7fa57c0acaa0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa58e000640 1 -- 172.21.15.97:0/293569314 reap_dead start 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 shutdown_connections 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 >> 172.21.15.97:0/293569314 conn(0x564d52d760d0 msgr2=0x564d52de49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 shutdown_connections 2024-09-10T21:30:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.848+0000 7fa590326080 1 -- 172.21.15.97:0/293569314 wait complete. 2024-09-10T21:30:54.856 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2024-09-10T21:30:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.936+0000 7f248a853080 1 Processor -- start 2024-09-10T21:30:54.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.936+0000 7f248a853080 1 -- start start 2024-09-10T21:30:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ee85520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee57600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55642eda3a00 con 0x55642ef0fa00 2024-09-10T21:30:54.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55642ee01f40 con 0x55642ef131d0 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55642ef11710 con 0x55642ef0dcc0 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ee85520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ee85520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33100/0 (socket says 172.21.15.97:33100) 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 -- 172.21.15.97:0/2651770211 learned_addr learned my addr 172.21.15.97:0/2651770211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487a00640 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2488400640 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee57600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 -- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 msgr2=0x55642ef29080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef29080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 -- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 msgr2=0x55642ee57600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee57600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 -- 172.21.15.97:0/2651770211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55642ed7cc10 con 0x55642ef0fa00 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487a00640 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef29080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2487000640 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ee85520 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f247400b7d0 tx=0x7f247400bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b748f0f34f1ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2485c00640 1 -- 172.21.15.97:0/2651770211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247400abb0 con 0x55642ef0fa00 2024-09-10T21:30:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2485c00640 1 -- 172.21.15.97:0/2651770211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247400ad50 con 0x55642ef0fa00 2024-09-10T21:30:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f2485c00640 1 -- 172.21.15.97:0/2651770211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24740086c0 con 0x55642ef0fa00 2024-09-10T21:30:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- 172.21.15.97:0/2651770211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 msgr2=0x55642ee85520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ee85520 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f247400b7d0 tx=0x7f247400bc90 comp rx=0 tx=0).stop 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- 172.21.15.97:0/2651770211 shutdown_connections 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee57600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ee85520 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 --2- 172.21.15.97:0/2651770211 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef29080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- 172.21.15.97:0/2651770211 >> 172.21.15.97:0/2651770211 conn(0x55642ee9d0d0 msgr2=0x55642ee85e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- 172.21.15.97:0/2651770211 shutdown_connections 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- 172.21.15.97:0/2651770211 wait complete. 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 Processor -- start 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.940+0000 7f248a853080 1 -- start start 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef28e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ef2d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee9f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55642ef0c980 con 0x55642ef0fa00 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55642ef29750 con 0x55642ef131d0 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55642ee9bec0 con 0x55642ef0dcc0 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ef2d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ef2d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33102/0 (socket says 172.21.15.97:33102) 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 -- 172.21.15.97:0/865500940 learned_addr learned my addr 172.21.15.97:0/865500940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487a00640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef28e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2488400640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee9f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 -- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 msgr2=0x55642ef28e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55642ef0dcc0 0x55642ef28e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 -- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 msgr2=0x55642ee9f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee9f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 -- 172.21.15.97:0/865500940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55642ef293d0 con 0x55642ef0fa00 2024-09-10T21:30:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2488400640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55642ef131d0 0x55642ee9f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f2487000640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ef2d0e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2474017ca0 tx=0x7f2474017cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f247fe00640 1 -- 172.21.15.97:0/865500940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474020440 con 0x55642ef0fa00 2024-09-10T21:30:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f247fe00640 1 -- 172.21.15.97:0/865500940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24740205e0 con 0x55642ef0fa00 2024-09-10T21:30:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f247fe00640 1 -- 172.21.15.97:0/865500940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474020a10 con 0x55642ef0fa00 2024-09-10T21:30:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55642ef2daf0 con 0x55642ef0fa00 2024-09-10T21:30:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55642ef2e030 con 0x55642ef0fa00 2024-09-10T21:30:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.944+0000 7f247fe00640 1 -- 172.21.15.97:0/865500940 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f247401e070 con 0x55642ef0fa00 2024-09-10T21:30:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f247fe00640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2468070ff0 0x7f24680734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f247fe00640 1 -- 172.21.15.97:0/865500940 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f2474093cd0 con 0x55642ef0fa00 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55642efc8980 0x55642efc8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55642ef2d620 con 0x55642efc8980 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f2487a00640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2468070ff0 0x7f24680734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f2488400640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55642efc8980 0x55642efc8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f2488400640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55642efc8980 0x55642efc8d60 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f2487a00640 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2468070ff0 0x7f24680734b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2478011490 tx=0x7f2478001740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f2488400640 1 -- 172.21.15.97:0/865500940 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 154+0+16 (crc 0 0 0) 0x7f2480002050 con 0x55642efc8980 2024-09-10T21:30:54.953 INFO:teuthology.orchestra.run.smithi097.stdout:cephfs_metadata/1.00000000 mtime 2024-09-10T21:30:50.000000+0000, size 0 2024-09-10T21:30:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55642efc8980 msgr2=0x55642efc8d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55642efc8980 0x55642efc8d60 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2468070ff0 msgr2=0x7f24680734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2468070ff0 0x7f24680734b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2478011490 tx=0x7f2478001740 comp rx=0 tx=0).stop 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 msgr2=0x55642ef2d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 --2- 172.21.15.97:0/865500940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55642ef0fa00 0x55642ef2d0e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2474017ca0 tx=0x7f2474017cd0 comp rx=0 tx=0).stop 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f2488400640 1 -- 172.21.15.97:0/865500940 reap_dead start 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 shutdown_connections 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 >> 172.21.15.97:0/865500940 conn(0x55642ee9d0d0 msgr2=0x55642ee9d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 shutdown_connections 2024-09-10T21:30:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:54.948+0000 7f248a853080 1 -- 172.21.15.97:0/865500940 wait complete. 2024-09-10T21:30:54.956 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2024-09-10T21:30:54.956 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2024-09-10T21:30:54.957 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2024-09-10T21:30:54.958 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2024-09-10T21:30:54.958 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2024-09-10T21:30:54.958 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2024-09-10T21:30:54.958 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2024-09-10T21:30:54.958 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-10T21:30:54.958 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2024-09-10T21:30:54.958 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2024-09-10T21:30:54.958 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:30:54.958 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:30:55.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:30:55.006 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:30:55.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:30:55.006 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:30:55.056 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:30:55.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:30:55.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:30:55.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 -- 172.21.15.97:0/353660104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 msgr2=0x7f1aac0b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/353660104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0b37b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009930 tx=0x7f1a9c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 -- 172.21.15.97:0/353660104 shutdown_connections 2024-09-10T21:30:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/353660104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1aac0ceee0 0x7f1aac0cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/353660104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0b37b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 -- 172.21.15.97:0/353660104 >> 172.21.15.97:0/353660104 conn(0x7f1aac0b1780 msgr2=0x7f1aac0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 -- 172.21.15.97:0/353660104 shutdown_connections 2024-09-10T21:30:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 -- 172.21.15.97:0/353660104 wait complete. 2024-09-10T21:30:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 Processor -- start 2024-09-10T21:30:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.244+0000 7f1ab1e00640 1 -- start start 2024-09-10T21:30:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aac0b3cf0 0x7f1aac1517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1aac0ceee0 0x7f1aac1f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac06e740 con 0x7f1aac1511f0 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac06e5c0 con 0x7f1aac0b3cf0 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1aac06e8c0 con 0x7f1aac0ceee0 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33120/0 (socket says 172.21.15.97:33120) 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 -- 172.21.15.97:0/112576149 learned_addr learned my addr 172.21.15.97:0/112576149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aaaa00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aac0b3cf0 0x7f1aac1517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aa2000640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1aac0ceee0 0x7f1aac1f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 -- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1aac0ceee0 msgr2=0x7f1aac1f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1aac0ceee0 0x7f1aac1f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 -- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aac0b3cf0 msgr2=0x7f1aac1517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aac0b3cf0 0x7f1aac1517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 -- 172.21.15.97:0/112576149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac14f0c0 con 0x7f1aac1511f0 2024-09-10T21:30:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aa2000640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1aac0ceee0 0x7f1aac1f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:55.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aab400640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0cdaf0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400b430 tx=0x7f1aa400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:55.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aa3e00640 1 -- 172.21.15.97:0/112576149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4015020 con 0x7f1aac1511f0 2024-09-10T21:30:55.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aa3e00640 1 -- 172.21.15.97:0/112576149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aa4011920 con 0x7f1aac1511f0 2024-09-10T21:30:55.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac150690 con 0x7f1aac1511f0 2024-09-10T21:30:55.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1aa3e00640 1 -- 172.21.15.97:0/112576149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4011d70 con 0x7f1aac1511f0 2024-09-10T21:30:55.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac150c50 con 0x7f1aac1511f0 2024-09-10T21:30:55.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.248+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aac14f590 con 0x7f1aac1511f0 2024-09-10T21:30:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.252+0000 7f1aa3e00640 1 -- 172.21.15.97:0/112576149 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1aa401a400 con 0x7f1aac1511f0 2024-09-10T21:30:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.252+0000 7f1aa3e00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a880725e0 0x7f1a88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.252+0000 7f1aa3e00640 1 -- 172.21.15.97:0/112576149 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f1aa40936b0 con 0x7f1aac1511f0 2024-09-10T21:30:55.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.252+0000 7f1aaaa00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a880725e0 0x7f1a88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.252+0000 7f1aaaa00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a880725e0 0x7f1a88074aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0b2000 tx=0x7f1a9c01d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:55.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.256+0000 7f1aa3e00640 1 -- 172.21.15.97:0/112576149 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1aa401f020 con 0x7f1aac1511f0 2024-09-10T21:30:55.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.452+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1aac14dfc0 con 0x7f1aac1511f0 2024-09-10T21:30:55.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.648+0000 7f1aa3e00640 1 -- 172.21.15.97:0/112576149 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7f1aa405eb70 con 0x7f1aac1511f0 2024-09-10T21:30:55.651 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:30:55.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.648+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a880725e0 msgr2=0x7f1a88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.648+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a880725e0 0x7f1a88074aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0b2000 tx=0x7f1a9c01d910 comp rx=0 tx=0).stop 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.648+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 msgr2=0x7f1aac0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.648+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0cdaf0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400b430 tx=0x7f1aa400b8f0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 shutdown_connections 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a880725e0 0x7f1a88074aa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1aac1511f0 0x7f1aac0cdaf0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1aac0ceee0 0x7f1aac1f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 --2- 172.21.15.97:0/112576149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aac0b3cf0 0x7f1aac1517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 >> 172.21.15.97:0/112576149 conn(0x7f1aac0b1780 msgr2=0x7f1aac15cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 shutdown_connections 2024-09-10T21:30:55.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.652+0000 7f1ab1e00640 1 -- 172.21.15.97:0/112576149 wait complete. 2024-09-10T21:30:55.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:30:55.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.812+0000 7fc2ad400640 1 -- 172.21.15.97:0/1833667758 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 msgr2=0x7fc2a80b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.812+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1833667758 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 0x7fc2a80b3070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009bf0 tx=0x7fc2a001ca70 comp rx=0 tx=0).stop 2024-09-10T21:30:55.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.812+0000 7fc2ad400640 1 -- 172.21.15.97:0/1833667758 shutdown_connections 2024-09-10T21:30:55.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.812+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1833667758 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 0x7fc2a80b3070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.812+0000 7fc2ad400640 1 -- 172.21.15.97:0/1833667758 >> 172.21.15.97:0/1833667758 conn(0x7fc2a80d1c60 msgr2=0x7fc2a80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:55.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.812+0000 7fc2ad400640 1 -- 172.21.15.97:0/1833667758 shutdown_connections 2024-09-10T21:30:55.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.812+0000 7fc2ad400640 1 -- 172.21.15.97:0/1833667758 wait complete. 2024-09-10T21:30:55.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 Processor -- start 2024-09-10T21:30:55.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 -- start start 2024-09-10T21:30:55.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a80d0150 0x7fc2a814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 0x7fc2a80b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a815f350 0x7fc2a80b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8169dc0 con 0x7fc2a80d0150 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8169c40 con 0x7fc2a815f350 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a8169f40 con 0x7fc2a80d05e0 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a80d0150 0x7fc2a814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a80d0150 0x7fc2a814ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33128/0 (socket says 172.21.15.97:33128) 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 -- 172.21.15.97:0/1081815737 learned_addr learned my addr 172.21.15.97:0/1081815737 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc29de00640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 0x7fc2a80b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a7200640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a815f350 0x7fc2a80b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 -- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 msgr2=0x7fc2a80b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 0x7fc2a80b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 -- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a815f350 msgr2=0x7fc2a80b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a815f350 0x7fc2a80b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 -- 172.21.15.97:0/1081815737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a815cf30 con 0x7fc2a80d0150 2024-09-10T21:30:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2a6800640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a80d0150 0x7fc2a814ec70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c700 tx=0x7fc29800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:55.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc29f400640 1 -- 172.21.15.97:0/1081815737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298016020 con 0x7fc2a80d0150 2024-09-10T21:30:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc29f400640 1 -- 172.21.15.97:0/1081815737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc29800a930 con 0x7fc2a80d0150 2024-09-10T21:30:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a815d210 con 0x7fc2a80d0150 2024-09-10T21:30:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc29f400640 1 -- 172.21.15.97:0/1081815737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29800ad80 con 0x7fc2a80d0150 2024-09-10T21:30:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a8155600 con 0x7fc2a80d0150 2024-09-10T21:30:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.816+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a80b3070 con 0x7fc2a80d0150 2024-09-10T21:30:55.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.820+0000 7fc29f400640 1 -- 172.21.15.97:0/1081815737 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc298012420 con 0x7fc2a80d0150 2024-09-10T21:30:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.820+0000 7fc29f400640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2840725e0 0x7fc284074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.820+0000 7fc29f400640 1 -- 172.21.15.97:0/1081815737 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fc2980938f0 con 0x7fc2a80d0150 2024-09-10T21:30:55.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.820+0000 7fc29de00640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2840725e0 0x7fc284074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:55.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.820+0000 7fc29de00640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2840725e0 0x7fc284074aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2a81f9c90 tx=0x7fc290009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:55.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:55.824+0000 7fc29f400640 1 -- 172.21.15.97:0/1081815737 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc298020020 con 0x7fc2a80d0150 2024-09-10T21:30:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.056+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc2a80b2520 con 0x7fc2a80d0150 2024-09-10T21:30:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.660+0000 7fc29f400640 1 -- 172.21.15.97:0/1081815737 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v89) ==== 107+0+0 (secure 0 0 0) 0x7fc29805e5d0 con 0x7fc2a80d0150 2024-09-10T21:30:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:30:56.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2840725e0 msgr2=0x7fc284074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:56.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2840725e0 0x7fc284074aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2a81f9c90 tx=0x7fc290009040 comp rx=0 tx=0).stop 2024-09-10T21:30:56.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a80d0150 msgr2=0x7fc2a814ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:56.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a80d0150 0x7fc2a814ec70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c700 tx=0x7fc29800cbc0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 shutdown_connections 2024-09-10T21:30:56.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc2840725e0 0x7fc284074aa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a815f350 0x7fc2a80b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2a80d05e0 0x7fc2a80b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 --2- 172.21.15.97:0/1081815737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a80d0150 0x7fc2a814ec70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 >> 172.21.15.97:0/1081815737 conn(0x7fc2a80d1c60 msgr2=0x7fc2a815d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:56.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 shutdown_connections 2024-09-10T21:30:56.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.664+0000 7fc2ad400640 1 -- 172.21.15.97:0/1081815737 wait complete. 2024-09-10T21:30:56.689 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:30:56.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 Processor -- start 2024-09-10T21:30:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 -- start start 2024-09-10T21:30:56.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa3c4940 0x560ffa46af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 0x560ffa46b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa44f2a0 0x560ffa452f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ffa2e3a00 con 0x560ffa44f2a0 2024-09-10T21:30:56.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ffa341f40 con 0x560ffa3c4940 2024-09-10T21:30:56.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e49120080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560ffa450e70 con 0x560ffa44eb00 2024-09-10T21:30:56.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e46e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa44f2a0 0x560ffa452f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e46e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa44f2a0 0x560ffa452f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33146/0 (socket says 172.21.15.97:33146) 2024-09-10T21:30:56.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.732+0000 7f8e46e00640 1 -- 172.21.15.97:0/3371032023 learned_addr learned my addr 172.21.15.97:0/3371032023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:56.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46400640 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa3c4940 0x560ffa46af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e45a00640 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 0x560ffa46b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46e00640 1 -- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 msgr2=0x560ffa46b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:56.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46e00640 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 0x560ffa46b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46e00640 1 -- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa3c4940 msgr2=0x560ffa46af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:56.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46e00640 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa3c4940 0x560ffa46af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46e00640 1 -- 172.21.15.97:0/3371032023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ffa2bcc10 con 0x560ffa44f2a0 2024-09-10T21:30:56.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46e00640 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa44f2a0 0x560ffa452f80 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c7c0 tx=0x7f8e4000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea6ecc70793d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e3fe00640 1 -- 172.21.15.97:0/3371032023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40013070 con 0x560ffa44f2a0 2024-09-10T21:30:56.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e3fe00640 1 -- 172.21.15.97:0/3371032023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e4000ab30 con 0x560ffa44f2a0 2024-09-10T21:30:56.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e3fe00640 1 -- 172.21.15.97:0/3371032023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000f460 con 0x560ffa44f2a0 2024-09-10T21:30:56.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- 172.21.15.97:0/3371032023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa44f2a0 msgr2=0x560ffa452f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:56.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa44f2a0 0x560ffa452f80 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c7c0 tx=0x7f8e4000cc80 comp rx=0 tx=0).stop 2024-09-10T21:30:56.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- 172.21.15.97:0/3371032023 shutdown_connections 2024-09-10T21:30:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa44f2a0 0x560ffa452f80 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 0x560ffa46b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 --2- 172.21.15.97:0/3371032023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa3c4940 0x560ffa46af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- 172.21.15.97:0/3371032023 >> 172.21.15.97:0/3371032023 conn(0x560ffa3c3520 msgr2=0x560ffa394da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- 172.21.15.97:0/3371032023 shutdown_connections 2024-09-10T21:30:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- 172.21.15.97:0/3371032023 wait complete. 2024-09-10T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 Processor -- start 2024-09-10T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- start start 2024-09-10T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa3c4940 0x560ffa46a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 0x560ffa46af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa44f2a0 0x560ffa46b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ffa46cfb0 con 0x560ffa3c4940 2024-09-10T21:30:56.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ffa46cd10 con 0x560ffa44f2a0 2024-09-10T21:30:56.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e49120080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560ffa46d250 con 0x560ffa44eb00 2024-09-10T21:30:56.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa3c4940 0x560ffa46a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa3c4940 0x560ffa46a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33152/0 (socket says 172.21.15.97:33152) 2024-09-10T21:30:56.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.736+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 learned_addr learned my addr 172.21.15.97:0/2818865298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:56.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e45a00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 0x560ffa46af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46e00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa44f2a0 0x560ffa46b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 msgr2=0x560ffa46af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46400640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ffa44eb00 0x560ffa46af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa44f2a0 msgr2=0x560ffa46b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46400640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa44f2a0 0x560ffa46b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ffa3d8ba0 con 0x560ffa3c4940 2024-09-10T21:30:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46e00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ffa44f2a0 0x560ffa46b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:30:56.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e46400640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa3c4940 0x560ffa46a3d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8e3801b710 tx=0x7f8e380aca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e3ea00640 1 -- 172.21.15.97:0/2818865298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380bb070 con 0x560ffa3c4940 2024-09-10T21:30:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e3ea00640 1 -- 172.21.15.97:0/2818865298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e380b7460 con 0x560ffa3c4940 2024-09-10T21:30:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e3ea00640 1 -- 172.21.15.97:0/2818865298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380b0040 con 0x560ffa3c4940 2024-09-10T21:30:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ffa3d8e40 con 0x560ffa3c4940 2024-09-10T21:30:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ffa3d9380 con 0x560ffa3c4940 2024-09-10T21:30:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e3ea00640 1 -- 172.21.15.97:0/2818865298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8e380b78b0 con 0x560ffa3c4940 2024-09-10T21:30:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e3ea00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e28070ff0 0x7f8e280734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.740+0000 7f8e3ea00640 1 -- 172.21.15.97:0/2818865298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f8e3812da60 con 0x560ffa3c4940 2024-09-10T21:30:56.754 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:30:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e45a00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e28070ff0 0x7f8e280734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560ffa3daad0 0x560ffa3dda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa46a910 con 0x560ffa3daad0 2024-09-10T21:30:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e46e00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560ffa3daad0 0x560ffa3dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e45a00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e28070ff0 0x7f8e280734b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8e34000c10 tx=0x7f8e34001780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e46e00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560ffa3daad0 0x560ffa3dda60 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v97'11 uv5 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.744+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa3ddfa0 con 0x560ffa3daad0 2024-09-10T21:30:56.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.748+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v97'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.748+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa3de340 con 0x560ffa3daad0 2024-09-10T21:30:56.756 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:30:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.748+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v97'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.748+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa512990 con 0x560ffa3daad0 2024-09-10T21:30:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.752+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v97'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.752+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa512d30 con 0x560ffa3daad0 2024-09-10T21:30:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.756+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v97'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.756+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa511980 con 0x560ffa3daad0 2024-09-10T21:30:56.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.756+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v97'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.756+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa511d70 con 0x560ffa3daad0 2024-09-10T21:30:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.760+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v97'17 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.760+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa512110 con 0x560ffa3daad0 2024-09-10T21:30:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.764+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v97'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.764+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa512500 con 0x560ffa3daad0 2024-09-10T21:30:56.768 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:30:56.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.764+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v97'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.764+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa512500 con 0x560ffa3daad0 2024-09-10T21:30:56.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.768+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v97'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.768+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.772+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v97'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.772+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.772+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v97'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.772+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.776+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v97'23 uv10 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.776+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.780+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v97'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.780+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.785 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:30:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.780+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v97'25 uv25 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.780+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.784+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v97'26 uv26 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.784+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.788+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v97'27 uv27 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.788+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5124b0 con 0x560ffa3daad0 2024-09-10T21:30:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.788+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v97'28 uv28 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.788+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.792+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v97'29 uv29 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.792+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.792+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v97'30 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.792+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.796+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.796+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.802 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:30:56.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.800+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v97'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.800+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa514590 con 0x560ffa3daad0 2024-09-10T21:30:56.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.800+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v97'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.800+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa514590 con 0x560ffa3daad0 2024-09-10T21:30:56.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.804+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v97'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.804+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.804+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v97'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.804+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.808+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v97'36 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.808+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.808+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v97'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.812+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.815 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:30:56.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.812+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v97'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.812+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa514590 con 0x560ffa3daad0 2024-09-10T21:30:56.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.816+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v97'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.816+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa514590 con 0x560ffa3daad0 2024-09-10T21:30:56.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.816+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v97'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.816+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa513580 con 0x560ffa3daad0 2024-09-10T21:30:56.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.820+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v97'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa3daad0 2024-09-10T21:30:56.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.820+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560ffa513a90 0x560ffa515e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.820+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5163c0 con 0x560ffa513a90 2024-09-10T21:30:56.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.820+0000 7f8e46400640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560ffa513a90 0x560ffa515e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.820+0000 7f8e46400640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560ffa513a90 0x560ffa515e80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.820+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v97'20 uv15 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.820+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa516760 con 0x560ffa513a90 2024-09-10T21:30:56.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.824+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v97'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.824+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa516b00 con 0x560ffa513a90 2024-09-10T21:30:56.828 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:30:56.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.824+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v97'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.824+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa517350 con 0x560ffa513a90 2024-09-10T21:30:56.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.828+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v97'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.828+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5176f0 con 0x560ffa513a90 2024-09-10T21:30:56.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.832+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v97'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.832+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa517a90 con 0x560ffa513a90 2024-09-10T21:30:56.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.832+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v97'25 uv19 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.832+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa517e30 con 0x560ffa513a90 2024-09-10T21:30:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.836+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v97'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.836+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.842 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:30:56.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.840+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v97'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.840+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.840+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v97'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.840+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.844+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v97'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.844+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.844+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v97'30 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.844+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.848+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.848+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.853 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:30:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.848+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v97'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.848+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51c3a0 con 0x560ffa513a90 2024-09-10T21:30:56.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.852+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v97'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.852+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51c3a0 con 0x560ffa513a90 2024-09-10T21:30:56.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.856+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v97'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.856+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.856+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v97'35 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.856+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.860+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v97'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.860+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.864 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:30:56.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.860+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v97'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.860+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5191e0 con 0x560ffa513a90 2024-09-10T21:30:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.864+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v97'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.864+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5191e0 con 0x560ffa513a90 2024-09-10T21:30:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.864+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v97'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.864+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5181d0 con 0x560ffa513a90 2024-09-10T21:30:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.868+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v97'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa513a90 2024-09-10T21:30:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.868+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560ffa5186e0 0x560ffa51aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.868+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51b010 con 0x560ffa5186e0 2024-09-10T21:30:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.868+0000 7f8e45a00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560ffa5186e0 0x560ffa51aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.868+0000 7f8e45a00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560ffa5186e0 0x560ffa51aad0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.868+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v97'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.868+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa51b470 con 0x560ffa5186e0 2024-09-10T21:30:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.872+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v97'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.872+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa516f70 con 0x560ffa5186e0 2024-09-10T21:30:56.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.872+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v97'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.876 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:30:56.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.872+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51d280 con 0x560ffa5186e0 2024-09-10T21:30:56.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.872+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v97'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.872+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51d620 con 0x560ffa5186e0 2024-09-10T21:30:56.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v97'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51c270 con 0x560ffa5186e0 2024-09-10T21:30:56.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v97'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560ffa51cb70 0x560ffa51f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51ff10 con 0x560ffa51cb70 2024-09-10T21:30:56.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e46e00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560ffa51cb70 0x560ffa51f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e46e00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560ffa51cb70 0x560ffa51f9d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v97'5 uv4 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.876+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa51ff10 con 0x560ffa51cb70 2024-09-10T21:30:56.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.880+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v97'6 uv6 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.880+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51ff90 con 0x560ffa51cb70 2024-09-10T21:30:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.884+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v97'7 uv7 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.886 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:30:56.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.884+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5203b0 con 0x560ffa51cb70 2024-09-10T21:30:56.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.884+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v97'8 uv8 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.884+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5203b0 con 0x560ffa5186e0 2024-09-10T21:30:56.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.888+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v97'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.888+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.892+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v97'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.892+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.897 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:30:56.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.892+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v97'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.896+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa521760 con 0x560ffa5186e0 2024-09-10T21:30:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.896+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v97'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.896+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa521760 con 0x560ffa5186e0 2024-09-10T21:30:56.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.900+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v97'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.900+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.904+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v97'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.904+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.904+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v97'4 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.904+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.908+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v97'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.908+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.912 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:30:56.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.908+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v97'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.908+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa521760 con 0x560ffa51cb70 2024-09-10T21:30:56.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v97'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa521760 con 0x560ffa51cb70 2024-09-10T21:30:56.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v97'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v97'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v97'10 uv3 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.912+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.916+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v97'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.916+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.919 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:30:56.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.916+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v97'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.916+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.916+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v97'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.916+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa51cb70 2024-09-10T21:30:56.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.920+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v97'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.920+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.920+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v97'2 uv1 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.920+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.920+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v97'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.920+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.925 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:30:56.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v97'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v97'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa520750 con 0x560ffa5186e0 2024-09-10T21:30:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v97'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa5186e0 2024-09-10T21:30:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ffa520dd0 0x560ffa5231c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa5237e0 con 0x560ffa520dd0 2024-09-10T21:30:56.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e46400640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ffa520dd0 0x560ffa5231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.924+0000 7f8e46400640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ffa520dd0 0x560ffa5231c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.928+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v97'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.928+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa520dd0 2024-09-10T21:30:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.928+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v97'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.928+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa520dd0 2024-09-10T21:30:56.933 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:30:56.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.932+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v97'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.932+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa524c10 con 0x560ffa520dd0 2024-09-10T21:30:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.932+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v97'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.932+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa524c10 con 0x560ffa520dd0 2024-09-10T21:30:56.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.932+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v97'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.932+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa520dd0 2024-09-10T21:30:56.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.936+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v97'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.936+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa520dd0 2024-09-10T21:30:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.936+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v97'4 uv3 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.936+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa520dd0 2024-09-10T21:30:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.936+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v97'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.936+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa520dd0 2024-09-10T21:30:56.941 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:30:56.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.936+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v97'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.940+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51be60 con 0x560ffa520dd0 2024-09-10T21:30:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.940+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v97'7 uv7 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.940+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51be60 con 0x560ffa520dd0 2024-09-10T21:30:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.940+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v97'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.940+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51be60 con 0x560ffa520dd0 2024-09-10T21:30:56.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.940+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v97'9 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.940+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa51be60 con 0x560ffa520dd0 2024-09-10T21:30:56.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.944+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v97'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.944+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51be60 con 0x560ffa520dd0 2024-09-10T21:30:56.947 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:30:56.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.944+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v97'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.944+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa51be60 con 0x560ffa520dd0 2024-09-10T21:30:56.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.944+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v97'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.944+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526a90 con 0x560ffa520dd0 2024-09-10T21:30:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.948+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v97'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.948+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.948+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v97'4 uv3 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.948+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.948+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v97'5 uv5 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.948+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.952+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v97'6 uv6 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.954 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:30:56.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.952+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.952+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v97'7 uv7 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.952+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.952+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v97'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.952+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.956+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v97'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.956+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.960 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:30:56.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.956+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v97'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.956+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.960+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v97'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.960+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.964+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v97'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.964+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa523c00 con 0x560ffa51cb70 2024-09-10T21:30:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.964+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v97'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e40015030 con 0x560ffa51cb70 2024-09-10T21:30:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.964+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560ffa524110 0x560ffa527150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.964+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa524110 2024-09-10T21:30:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.964+0000 7f8e45a00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560ffa524110 0x560ffa527150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.968+0000 7f8e45a00640 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560ffa524110 0x560ffa527150 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:56.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.968+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v97'2 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa524110 2024-09-10T21:30:56.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.968+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa524110 2024-09-10T21:30:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.968+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v97'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa524110 2024-09-10T21:30:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.968+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa524110 2024-09-10T21:30:56.974 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:30:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.972+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v97'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa524110 2024-09-10T21:30:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.972+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa524110 2024-09-10T21:30:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.972+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v97'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa524110 2024-09-10T21:30:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.972+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa524110 2024-09-10T21:30:56.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.976+0000 7f8e45a00640 1 -- 172.21.15.97:0/2818865298 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v97'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8e3401b040 con 0x560ffa524110 2024-09-10T21:30:56.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.976+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.976+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v97'7 uv3 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.976+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.980+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v97'8 uv8 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.980+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.984 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:30:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.980+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v97'9 uv9 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.980+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.984+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v97'10 uv10 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.984+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.988+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v97'11 uv11 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.988+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.992+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v97'12 uv6 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.992+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.992+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v97'13 uv13 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.992+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:56.998 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:30:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.996+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v97'14 uv14 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.996+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.996+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v97'15 uv15 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:56.996+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.000+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v97'16 uv16 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.000+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa526500 con 0x560ffa520dd0 2024-09-10T21:30:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v97'17 uv17 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x560ffa527690 con 0x560ffa520dd0 2024-09-10T21:30:57.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e46400640 1 -- 172.21.15.97:0/2818865298 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v97'18 uv18 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8e380c7070 con 0x560ffa520dd0 2024-09-10T21:30:57.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560ffa5186e0 msgr2=0x560ffa51aad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560ffa5186e0 0x560ffa51aad0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ffa520dd0 msgr2=0x560ffa5231c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ffa520dd0 0x560ffa5231c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560ffa524110 msgr2=0x560ffa527150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560ffa524110 0x560ffa527150 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560ffa51cb70 msgr2=0x560ffa51f9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560ffa51cb70 0x560ffa51f9d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560ffa513a90 msgr2=0x560ffa515e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560ffa513a90 0x560ffa515e80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 reap_dead start 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e46e00640 1 -- 172.21.15.97:0/2818865298 reap_dead start 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560ffa3daad0 msgr2=0x560ffa3dda60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560ffa3daad0 0x560ffa3dda60 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e28070ff0 msgr2=0x7f8e280734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e28070ff0 0x7f8e280734b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8e34000c10 tx=0x7f8e34001780 comp rx=0 tx=0).stop 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa3c4940 msgr2=0x560ffa46a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.004+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa3c4940 0x560ffa46a3d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8e3801b710 tx=0x7f8e380aca10 comp rx=0 tx=0).stop 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.008+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 shutdown_connections 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.008+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560ffa3daad0 0x560ffa3dda60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.008+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e28070ff0 0x7f8e280734b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.008+0000 7f8e49120080 1 --2- 172.21.15.97:0/2818865298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ffa3c4940 0x560ffa46a3d0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.008+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 >> 172.21.15.97:0/2818865298 conn(0x560ffa3c3520 msgr2=0x560ffa3c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:57.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.008+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 shutdown_connections 2024-09-10T21:30:57.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.008+0000 7f8e49120080 1 -- 172.21.15.97:0/2818865298 wait complete. 2024-09-10T21:30:57.014 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-09-10T21:30:57.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.096+0000 7fd34685e080 1 Processor -- start 2024-09-10T21:30:57.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- start start 2024-09-10T21:30:57.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 0x55a5d1c6d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c54a00 0x55a5d1bca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c581d0 0x55a5d1c6b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1ae8a00 con 0x55a5d1c581d0 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1b46f40 con 0x55a5d1c52cc0 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55a5d1c56710 con 0x55a5d1c54a00 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c581d0 0x55a5d1c6b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c581d0 0x55a5d1c6b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33168/0 (socket says 172.21.15.97:33168) 2024-09-10T21:30:57.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 -- 172.21.15.97:0/2608726049 learned_addr learned my addr 172.21.15.97:0/2608726049 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd33b000640 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c54a00 0x55a5d1bca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd343a00640 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 0x55a5d1c6d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 -- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c54a00 msgr2=0x55a5d1bca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c54a00 0x55a5d1bca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 -- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 msgr2=0x55a5d1c6d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 0x55a5d1c6d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 -- 172.21.15.97:0/2608726049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d1ac1c10 con 0x55a5d1c581d0 2024-09-10T21:30:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd344400640 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c581d0 0x55a5d1c6b0e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0ae040 tx=0x7fd33c018160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627cfb766ccfbf60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd342600640 1 -- 172.21.15.97:0/2608726049 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c004020 con 0x55a5d1c581d0 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd342600640 1 -- 172.21.15.97:0/2608726049 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd33c0042b0 con 0x55a5d1c581d0 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd342600640 1 -- 172.21.15.97:0/2608726049 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0acb70 con 0x55a5d1c581d0 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- 172.21.15.97:0/2608726049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c581d0 msgr2=0x55a5d1c6b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c581d0 0x55a5d1c6b0e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0ae040 tx=0x7fd33c018160 comp rx=0 tx=0).stop 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- 172.21.15.97:0/2608726049 shutdown_connections 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c581d0 0x55a5d1c6b0e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c54a00 0x55a5d1bca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 --2- 172.21.15.97:0/2608726049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 0x55a5d1c6d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- 172.21.15.97:0/2608726049 >> 172.21.15.97:0/2608726049 conn(0x55a5d1be20d0 msgr2=0x55a5d1c70f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- 172.21.15.97:0/2608726049 shutdown_connections 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- 172.21.15.97:0/2608726049 wait complete. 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 Processor -- start 2024-09-10T21:30:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.100+0000 7fd34685e080 1 -- start start 2024-09-10T21:30:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 0x55a5d1b9c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c54a00 0x55a5d1c51070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c581d0 0x55a5d1b99dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1be0ff0 con 0x55a5d1c54a00 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1be1290 con 0x55a5d1c52cc0 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55a5d1c515b0 con 0x55a5d1c581d0 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c54a00 0x55a5d1c51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c54a00 0x55a5d1c51070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33182/0 (socket says 172.21.15.97:33182) 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd343a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 0x55a5d1b9c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 -- 172.21.15.97:0/3090532219 learned_addr learned my addr 172.21.15.97:0/3090532219 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd344400640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c581d0 0x55a5d1b99dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 -- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c581d0 msgr2=0x55a5d1b99dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55a5d1c581d0 0x55a5d1b99dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 -- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 msgr2=0x55a5d1b9c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a5d1c52cc0 0x55a5d1b9c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 -- 172.21.15.97:0/3090532219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d1b9ce50 con 0x55a5d1c54a00 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd33b000640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c54a00 0x55a5d1c51070 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00b5f0 tx=0x7fd32c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd341200640 1 -- 172.21.15.97:0/3090532219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c013070 con 0x55a5d1c54a00 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd341200640 1 -- 172.21.15.97:0/3090532219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd32c00f9f0 con 0x55a5d1c54a00 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd341200640 1 -- 172.21.15.97:0/3090532219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c00fe40 con 0x55a5d1c54a00 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a5d1b99140 con 0x55a5d1c54a00 2024-09-10T21:30:57.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.104+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d1b99680 con 0x55a5d1c54a00 2024-09-10T21:30:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd341200640 1 -- 172.21.15.97:0/3090532219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd32c0213a0 con 0x55a5d1c54a00 2024-09-10T21:30:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd341200640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd320070ff0 0x7fd3200734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd341200640 1 -- 172.21.15.97:0/3090532219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fd32c092c20 con 0x55a5d1c54a00 2024-09-10T21:30:57.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd343a00640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd320070ff0 0x7fd3200734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a5d1bc8b20 0x55a5d1d09890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x55a5d1d09dd0 con 0x55a5d1bc8b20 2024-09-10T21:30:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd344400640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a5d1bc8b20 0x55a5d1d09890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd343a00640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd320070ff0 0x7fd3200734b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c810 tx=0x7fd334002040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:57.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd344400640 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a5d1bc8b20 0x55a5d1d09890 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd344400640 1 -- 172.21.15.97:0/3090532219 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v97'42 uv42 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd33c002050 con 0x55a5d1bc8b20 2024-09-10T21:30:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a5d1bc8b20 msgr2=0x55a5d1d09890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a5d1bc8b20 0x55a5d1d09890 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd320070ff0 msgr2=0x7fd3200734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd320070ff0 0x7fd3200734b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c810 tx=0x7fd334002040 comp rx=0 tx=0).stop 2024-09-10T21:30:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c54a00 msgr2=0x55a5d1c51070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 --2- 172.21.15.97:0/3090532219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a5d1c54a00 0x55a5d1c51070 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00b5f0 tx=0x7fd32c00bab0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd344400640 1 -- 172.21.15.97:0/3090532219 reap_dead start 2024-09-10T21:30:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 shutdown_connections 2024-09-10T21:30:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.108+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 >> 172.21.15.97:0/3090532219 conn(0x55a5d1be20d0 msgr2=0x55a5d1b9d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.112+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 shutdown_connections 2024-09-10T21:30:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.112+0000 7fd34685e080 1 -- 172.21.15.97:0/3090532219 wait complete. 2024-09-10T21:30:57.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:30:57.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- 172.21.15.97:0/1812383048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854015efc0 msgr2=0x7f85400cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 --2- 172.21.15.97:0/1812383048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854015efc0 0x7f85400cf840 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f854006daf0 tx=0x7f853001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- 172.21.15.97:0/1812383048 shutdown_connections 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 --2- 172.21.15.97:0/1812383048 >> v2:172.21.15.194:3300/0 conn(0x7f85400d0490 0x7f85400ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 --2- 172.21.15.97:0/1812383048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854015efc0 0x7f85400cf840 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- 172.21.15.97:0/1812383048 >> 172.21.15.97:0/1812383048 conn(0x7f85400d1a60 msgr2=0x7f85400d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- 172.21.15.97:0/1812383048 shutdown_connections 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- 172.21.15.97:0/1812383048 wait complete. 2024-09-10T21:30:57.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 Processor -- start 2024-09-10T21:30:57.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- start start 2024-09-10T21:30:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85400cdcb0 0x7f85400b6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85400d0490 0x7f8540153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854015efc0 0x7f85400c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8540168930 con 0x7f85400d0490 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f85401687b0 con 0x7f854015efc0 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8544c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8540168ab0 con 0x7f85400cdcb0 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8536a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85400d0490 0x7f8540153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8536a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85400d0490 0x7f8540153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33190/0 (socket says 172.21.15.97:33190) 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f8536a00640 1 -- 172.21.15.97:0/791998740 learned_addr learned my addr 172.21.15.97:0/791998740 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f853f400640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85400cdcb0 0x7f85400b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.260+0000 7f853fe00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854015efc0 0x7f85400c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8536a00640 1 -- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85400cdcb0 msgr2=0x7f85400b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8536a00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85400cdcb0 0x7f85400b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8536a00640 1 -- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854015efc0 msgr2=0x7f85400c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8536a00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854015efc0 0x7f85400c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8536a00640 1 -- 172.21.15.97:0/791998740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85400b3170 con 0x7f85400d0490 2024-09-10T21:30:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8536a00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85400d0490 0x7f8540153f10 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f852800c4b0 tx=0x7f852800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8537e00640 1 -- 172.21.15.97:0/791998740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280052d0 con 0x7f85400d0490 2024-09-10T21:30:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8537e00640 1 -- 172.21.15.97:0/791998740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8528005470 con 0x7f85400d0490 2024-09-10T21:30:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8537e00640 1 -- 172.21.15.97:0/791998740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528012870 con 0x7f85400d0490 2024-09-10T21:30:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85400b2270 con 0x7f85400d0490 2024-09-10T21:30:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85400b2830 con 0x7f85400d0490 2024-09-10T21:30:57.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85400ae930 con 0x7f85400d0490 2024-09-10T21:30:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8537e00640 1 -- 172.21.15.97:0/791998740 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8528003600 con 0x7f85400d0490 2024-09-10T21:30:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8537e00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f851c0725e0 0x7f851c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.264+0000 7f8537e00640 1 -- 172.21.15.97:0/791998740 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f8528092f20 con 0x7f85400d0490 2024-09-10T21:30:57.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.268+0000 7f853f400640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f851c0725e0 0x7f851c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:57.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.268+0000 7f853f400640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f851c0725e0 0x7f851c074aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f85300af000 tx=0x7f85300a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:57.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.268+0000 7f8537e00640 1 -- 172.21.15.97:0/791998740 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f852805dd10 con 0x7f85400d0490 2024-09-10T21:30:57.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.464+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8540071a00 con 0x7f85400d0490 2024-09-10T21:30:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8537e00640 1 -- 172.21.15.97:0/791998740 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v90) ==== 162+0+0 (secure 0 0 0) 0x7f8528062740 con 0x7f85400d0490 2024-09-10T21:30:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:30:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f851c0725e0 msgr2=0x7f851c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f851c0725e0 0x7f851c074aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f85300af000 tx=0x7f85300a7040 comp rx=0 tx=0).stop 2024-09-10T21:30:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85400d0490 msgr2=0x7f8540153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85400d0490 0x7f8540153f10 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f852800c4b0 tx=0x7f852800c970 comp rx=0 tx=0).stop 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 shutdown_connections 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f851c0725e0 0x7f851c074aa0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f854015efc0 0x7f85400c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85400d0490 0x7f8540153f10 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 --2- 172.21.15.97:0/791998740 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85400cdcb0 0x7f85400b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 >> 172.21.15.97:0/791998740 conn(0x7f85400d1a60 msgr2=0x7f85400cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 shutdown_connections 2024-09-10T21:30:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:57.664+0000 7f8544c00640 1 -- 172.21.15.97:0/791998740 wait complete. 2024-09-10T21:30:58.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:30:58.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 -- 172.21.15.97:0/3557539029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 msgr2=0x7f5c2c0b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 --2- 172.21.15.97:0/3557539029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c0b0490 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0b1950 tx=0x7f5c2401cd60 comp rx=0 tx=0).stop 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 -- 172.21.15.97:0/3557539029 shutdown_connections 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 --2- 172.21.15.97:0/3557539029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c0b0490 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 --2- 172.21.15.97:0/3557539029 >> v2:172.21.15.194:3300/0 conn(0x7f5c2c0d27f0 0x7f5c2c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 -- 172.21.15.97:0/3557539029 >> 172.21.15.97:0/3557539029 conn(0x7f5c2c0d1cd0 msgr2=0x7f5c2c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 -- 172.21.15.97:0/3557539029 shutdown_connections 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.232+0000 7f5c32a00640 1 -- 172.21.15.97:0/3557539029 wait complete. 2024-09-10T21:30:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 Processor -- start 2024-09-10T21:30:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 -- start start 2024-09-10T21:30:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c0d27f0 0x7f5c2c14e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:58.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c2c15eed0 0x7f5c2c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c168a90 con 0x7f5c2c0cca90 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c168910 con 0x7f5c2c0d27f0 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5c2c168c10 con 0x7f5c2c15eed0 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53810/0 (socket says 172.21.15.97:53810) 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 -- 172.21.15.97:0/4069204121 learned_addr learned my addr 172.21.15.97:0/4069204121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2b400640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c0d27f0 0x7f5c2c14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c30a00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c2c15eed0 0x7f5c2c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 -- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c2c15eed0 msgr2=0x7f5c2c0b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c2c15eed0 0x7f5c2c0b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 -- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c0d27f0 msgr2=0x7f5c2c14e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:58.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c0d27f0 0x7f5c2c14e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 -- 172.21.15.97:0/4069204121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c15cba0 con 0x7f5c2c0cca90 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c2be00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c15c660 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00b440 tx=0x7f5c1c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c23e00640 1 -- 172.21.15.97:0/4069204121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c015020 con 0x7f5c2c0cca90 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c23e00640 1 -- 172.21.15.97:0/4069204121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c1c011980 con 0x7f5c2c0cca90 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c23e00640 1 -- 172.21.15.97:0/4069204121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c011dd0 con 0x7f5c2c0cca90 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c14e890 con 0x7f5c2c0cca90 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.236+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1f7df0 con 0x7f5c2c0cca90 2024-09-10T21:30:58.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.240+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c15e700 con 0x7f5c2c0cca90 2024-09-10T21:30:58.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.240+0000 7f5c23e00640 1 -- 172.21.15.97:0/4069204121 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5c1c01a400 con 0x7f5c2c0cca90 2024-09-10T21:30:58.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.240+0000 7f5c23e00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c0c0725e0 0x7f5c0c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:58.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.240+0000 7f5c23e00640 1 -- 172.21.15.97:0/4069204121 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f5c1c093720 con 0x7f5c2c0cca90 2024-09-10T21:30:58.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.240+0000 7f5c2b400640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c0c0725e0 0x7f5c0c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:58.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.244+0000 7f5c2b400640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c0c0725e0 0x7f5c0c074aa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005fd0 tx=0x7f5c18005db0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:58.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.244+0000 7f5c23e00640 1 -- 172.21.15.97:0/4069204121 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c1c01f030 con 0x7f5c2c0cca90 2024-09-10T21:30:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.444+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c2c0cfd60 con 0x7f5c2c0cca90 2024-09-10T21:30:58.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.444+0000 7f5c23e00640 1 -- 172.21.15.97:0/4069204121 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5222 (secure 0 0 0) 0x7f5c1c00d040 con 0x7f5c2c0cca90 2024-09-10T21:30:58.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:30:58.449 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":91,"btime":"2024-09-10T21:30:57:670411+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:30:57.670388+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/3971758099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3971758099},{"type":"v1","addr":"172.21.15.97:6835","nonce":3971758099}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:30:58.449 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 91 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c0c0725e0 msgr2=0x7f5c0c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c0c0725e0 0x7f5c0c074aa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005fd0 tx=0x7f5c18005db0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 msgr2=0x7f5c2c15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c15c660 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00b440 tx=0x7f5c1c00b900 comp rx=0 tx=0).stop 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 shutdown_connections 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c0c0725e0 0x7f5c0c074aa0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c2c15eed0 0x7f5c2c0b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c0d27f0 0x7f5c2c14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 --2- 172.21.15.97:0/4069204121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c2c0cca90 0x7f5c2c15c660 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:58.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 >> 172.21.15.97:0/4069204121 conn(0x7f5c2c0d1cd0 msgr2=0x7f5c2c0cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:58.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 shutdown_connections 2024-09-10T21:30:58.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:30:58.448+0000 7f5c32a00640 1 -- 172.21.15.97:0/4069204121 wait complete. 2024-09-10T21:30:58.469 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:30:57.670388+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 91, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/3971758099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 3971758099}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3971758099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:30:58.469 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7167:up:replay 2024-09-10T21:30:58.470 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:31:01.635 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:01.632+0000 7f6ab5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:03.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 -- 172.21.15.97:0/2623855140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24241511f0 msgr2=0x7f24240b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 --2- 172.21.15.97:0/2623855140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24241511f0 0x7f24240b37b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f24240cc590 tx=0x7f241401f0b0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 -- 172.21.15.97:0/2623855140 shutdown_connections 2024-09-10T21:31:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 --2- 172.21.15.97:0/2623855140 >> v2:172.21.15.194:3300/0 conn(0x7f242406f180 0x7f24240ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 --2- 172.21.15.97:0/2623855140 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24240b3cf0 0x7f242406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 --2- 172.21.15.97:0/2623855140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24241511f0 0x7f24240b37b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 -- 172.21.15.97:0/2623855140 >> 172.21.15.97:0/2623855140 conn(0x7f24240b1780 msgr2=0x7f24240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 -- 172.21.15.97:0/2623855140 shutdown_connections 2024-09-10T21:31:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 -- 172.21.15.97:0/2623855140 wait complete. 2024-09-10T21:31:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 Processor -- start 2024-09-10T21:31:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.024+0000 7f242a400640 1 -- start start 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f242406f180 0x7f242414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24240b3cf0 0x7f242414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24241511f0 0x7f2424150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f242415a8b0 con 0x7f24240b3cf0 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f242415a730 con 0x7f242406f180 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f242415aa30 con 0x7f24241511f0 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24240b3cf0 0x7f242414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24240b3cf0 0x7f242414f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53834/0 (socket says 172.21.15.97:53834) 2024-09-10T21:31:04.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 -- 172.21.15.97:0/1973885555 learned_addr learned my addr 172.21.15.97:0/1973885555 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f2423400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f242406f180 0x7f242414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f2423e00640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24241511f0 0x7f2424150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 -- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24241511f0 msgr2=0x7f2424150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24241511f0 0x7f2424150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 -- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f242406f180 msgr2=0x7f242414f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f242406f180 0x7f242414f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 -- 172.21.15.97:0/1973885555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24241f47f0 con 0x7f24240b3cf0 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f2423e00640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24241511f0 0x7f2424150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241aa00640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24240b3cf0 0x7f242414f930 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c970 tx=0x7f240c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241be00640 1 -- 172.21.15.97:0/1973885555 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c007c70 con 0x7f24240b3cf0 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241be00640 1 -- 172.21.15.97:0/1973885555 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f240c00f450 con 0x7f24240b3cf0 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241be00640 1 -- 172.21.15.97:0/1973885555 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c00f8a0 con 0x7f24240b3cf0 2024-09-10T21:31:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24241f4ad0 con 0x7f24240b3cf0 2024-09-10T21:31:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241f5090 con 0x7f24240b3cf0 2024-09-10T21:31:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242406ec40 con 0x7f24240b3cf0 2024-09-10T21:31:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.028+0000 7f241be00640 1 -- 172.21.15.97:0/1973885555 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f240c00fa40 con 0x7f24240b3cf0 2024-09-10T21:31:04.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.032+0000 7f241be00640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24000725e0 0x7f2400074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.032+0000 7f241be00640 1 -- 172.21.15.97:0/1973885555 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7f240c093d50 con 0x7f24240b3cf0 2024-09-10T21:31:04.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.032+0000 7f2423400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24000725e0 0x7f2400074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.032+0000 7f2423400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24000725e0 0x7f2400074aa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24140b2000 tx=0x7f241401d8f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.036+0000 7f241be00640 1 -- 172.21.15.97:0/1973885555 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f240c014030 con 0x7f24240b3cf0 2024-09-10T21:31:04.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.232+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2424154880 con 0x7f24240b3cf0 2024-09-10T21:31:04.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.232+0000 7f241be00640 1 -- 172.21.15.97:0/1973885555 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5229 (secure 0 0 0) 0x7f240c05f100 con 0x7f24240b3cf0 2024-09-10T21:31:04.237 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:04.237 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":94,"btime":"2024-09-10T21:31:00:690981+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:00.690973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6835/3971758099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3971758099},{"type":"v1","addr":"172.21.15.97:6835","nonce":3971758099}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":6}]} 2024-09-10T21:31:04.238 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 94 2024-09-10T21:31:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24000725e0 msgr2=0x7f2400074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24000725e0 0x7f2400074aa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24140b2000 tx=0x7f241401d8f0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24240b3cf0 msgr2=0x7f242414f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24240b3cf0 0x7f242414f930 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c970 tx=0x7f240c00ce30 comp rx=0 tx=0).stop 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 shutdown_connections 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24000725e0 0x7f2400074aa0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24241511f0 0x7f2424150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24240b3cf0 0x7f242414f930 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 --2- 172.21.15.97:0/1973885555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f242406f180 0x7f242414f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 >> 172.21.15.97:0/1973885555 conn(0x7f24240b1780 msgr2=0x7f242415bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 shutdown_connections 2024-09-10T21:31:04.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.236+0000 7f242a400640 1 -- 172.21.15.97:0/1973885555 wait complete. 2024-09-10T21:31:04.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:31:00.690973+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.97:6835/3971758099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 3971758099}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3971758099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7167, 'qdb_cluster': [7167]} 2024-09-10T21:31:04.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:31:04.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:31:04.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 -- 172.21.15.97:0/1287117051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb000cc2b0 msgr2=0x7fcb00070fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1287117051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb000cc2b0 0x7fcb00070fc0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcb0006a1e0 tx=0x7fcae801c770 comp rx=0 tx=0).stop 2024-09-10T21:31:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 -- 172.21.15.97:0/1287117051 shutdown_connections 2024-09-10T21:31:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1287117051 >> v2:172.21.15.194:3300/0 conn(0x7fcb000b40d0 0x7fcb000b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1287117051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb000cc2b0 0x7fcb00070fc0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 -- 172.21.15.97:0/1287117051 >> 172.21.15.97:0/1287117051 conn(0x7fcb000cfe50 msgr2=0x7fcb000d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 -- 172.21.15.97:0/1287117051 shutdown_connections 2024-09-10T21:31:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.396+0000 7fcb05a00640 1 -- 172.21.15.97:0/1287117051 wait complete. 2024-09-10T21:31:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 Processor -- start 2024-09-10T21:31:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 -- start start 2024-09-10T21:31:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcb000b40d0 0x7fcb00158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb000cc2b0 0x7fcb0014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb001586a0 0x7fcb0015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00167810 con 0x7fcb001586a0 2024-09-10T21:31:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00073780 con 0x7fcb000cc2b0 2024-09-10T21:31:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcb00167990 con 0x7fcb000b40d0 2024-09-10T21:31:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb001586a0 0x7fcb0015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb001586a0 0x7fcb0015a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53860/0 (socket says 172.21.15.97:53860) 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 -- 172.21.15.97:0/1003997564 learned_addr learned my addr 172.21.15.97:0/1003997564 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcafea00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcb000b40d0 0x7fcb00158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaf6000640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb000cc2b0 0x7fcb0014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 -- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcb000b40d0 msgr2=0x7fcb00158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcb000b40d0 0x7fcb00158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 -- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb000cc2b0 msgr2=0x7fcb0014aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb000cc2b0 0x7fcb0014aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 -- 172.21.15.97:0/1003997564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb00148fc0 con 0x7fcb001586a0 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcafea00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcb000b40d0 0x7fcb00158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaff400640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb001586a0 0x7fcb0015a450 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800ea20 tx=0x7fcaf800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaf7400640 1 -- 172.21.15.97:0/1003997564 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800cde0 con 0x7fcb001586a0 2024-09-10T21:31:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaf7400640 1 -- 172.21.15.97:0/1003997564 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf8004c40 con 0x7fcb001586a0 2024-09-10T21:31:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcaf7400640 1 -- 172.21.15.97:0/1003997564 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8005280 con 0x7fcb001586a0 2024-09-10T21:31:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0014b030 con 0x7fcb001586a0 2024-09-10T21:31:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.400+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb00159690 con 0x7fcb001586a0 2024-09-10T21:31:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.404+0000 7fcaf7400640 1 -- 172.21.15.97:0/1003997564 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcaf800f040 con 0x7fcb001586a0 2024-09-10T21:31:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.404+0000 7fcaf7400640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcadc0725e0 0x7fcadc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.404+0000 7fcaf7400640 1 -- 172.21.15.97:0/1003997564 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fcaf8093c50 con 0x7fcb001586a0 2024-09-10T21:31:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.404+0000 7fcb08d7d000 1 Processor -- start 2024-09-10T21:31:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.404+0000 7fcafea00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcadc0725e0 0x7fcadc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.404+0000 7fcb08d7d000 1 -- start start 2024-09-10T21:31:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcafea00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcadc0725e0 0x7fcadc074aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0000f70 tx=0x7fcaf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfb9d0 0x55eaf6bfbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6baf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bb01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eaf6ad03f0 con 0x55eaf6bafe10 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eaf6a0db20 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55eaf69fcf50 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bb01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bb01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53868/0 (socket says 172.21.15.97:53868) 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcafea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfb9d0 0x55eaf6bfbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 -- 172.21.15.97:0/2420040888 learned_addr learned my addr 172.21.15.97:0/2420040888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaf6000640 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6baf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 -- 172.21.15.97:0/2420040888 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfb9d0 msgr2=0x55eaf6bfbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfb9d0 0x55eaf6bfbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 -- 172.21.15.97:0/2420040888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfc2f0 msgr2=0x55eaf6baf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6baf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 -- 172.21.15.97:0/2420040888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf69c4590 con 0x55eaf6bafe10 2024-09-10T21:31:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaff400640 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bb01f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e9f0 tx=0x7fcaf805e250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bf27f642b71a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaef400640 1 -- 172.21.15.97:0/2420040888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf801f040 con 0x55eaf6bafe10 2024-09-10T21:31:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaef400640 1 -- 172.21.15.97:0/2420040888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf8063070 con 0x55eaf6bafe10 2024-09-10T21:31:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcaef400640 1 -- 172.21.15.97:0/2420040888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf805de30 con 0x55eaf6bafe10 2024-09-10T21:31:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- 172.21.15.97:0/2420040888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 msgr2=0x55eaf6bb01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bb01f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e9f0 tx=0x7fcaf805e250 comp rx=0 tx=0).stop 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- 172.21.15.97:0/2420040888 shutdown_connections 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bb01f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6baf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/2420040888 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfb9d0 0x55eaf6bfbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- 172.21.15.97:0/2420040888 >> 172.21.15.97:0/2420040888 conn(0x55eaf6a99130 msgr2=0x55eaf6bdee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- 172.21.15.97:0/2420040888 shutdown_connections 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- 172.21.15.97:0/2420040888 wait complete. 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 -- start start 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.408+0000 7fcb08d7d000 1 Processor -- start 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bdc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6bdce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6bdd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eaf69975d0 con 0x55eaf6bafe10 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eaf6a00d60 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55eaf69a3500 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 10 client.0 _ll_register_callbacks cb 0x55eaf69a7810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bdc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bdc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53878/0 (socket says 172.21.15.97:53878) 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 -- 172.21.15.97:0/605232684 learned_addr learned my addr 172.21.15.97:0/605232684 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcaff400640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6bdd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcaf6000640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6bdce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 -- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 msgr2=0x55eaf6bdd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6bdd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 -- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 msgr2=0x55eaf6bdce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6bdce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 -- 172.21.15.97:0/605232684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf699b170 con 0x55eaf6bafe10 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcaf6000640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6bdce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcafea00640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bdc6a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000ea00 tx=0x7fcaf0001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7acbea07d86ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcae3e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcae3e00640 1 -- 172.21.15.97:0/605232684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0028060 con 0x55eaf6bafe10 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcae3e00640 1 -- 172.21.15.97:0/605232684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fcaf0010040 con 0x55eaf6bafe10 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eaf69950f0 con 0x55eaf6bafe10 2024-09-10T21:31:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcae3e00640 1 -- 172.21.15.97:0/605232684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fcaf0003d90 con 0x55eaf6bafe10 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 10 client.7647 fetch_fsmap learned FSMap version 94 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55eaf6a81ef0 con 0x55eaf6bafe10 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcae3e00640 1 -- 172.21.15.97:0/605232684 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55eaf69950f0 con 0x55eaf6bafe10 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcae3e00640 1 -- 172.21.15.97:0/605232684 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 94) ==== 3502+0+0 (secure 0 0 0) 0x7fcaf0013420 con 0x55eaf6bafe10 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 10 client.7647 fetch_fsmap finished waiting for FSMap version 94 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 10 client.7647 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 20 client.7647 populate_metadata read hostname 'smithi097' 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6bde0f0 0x55eaf6c0dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 4 client.7647 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 --> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaf6a0fa70 con 0x55eaf6bde0f0 2024-09-10T21:31:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcaf6000640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6bde0f0 0x55eaf6c0dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcaf6000640 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6bde0f0 0x55eaf6c0dc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc33f4703d3064c server_cookie=449d648f5ff58b50 in_seq=0 out_seq=0 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.412+0000 7fcae3e00640 10 client.7647 ms_handle_connect on v2:172.21.15.97:6833/3971758099 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcae3e00640 1 -- 172.21.15.97:0/605232684 <== mds.0 v2:172.21.15.97:6833/3971758099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eaf6a0fa70 con 0x55eaf6bde0f0 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcae3e00640 10 client.7647 handle_command_reply: tid=0 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 client.7647 shutdown 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 msgr2=0x55eaf6bdc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bdc6a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000ea00 tx=0x7fcaf0001090 comp rx=0 tx=0).stop 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 shutdown_connections 2024-09-10T21:31:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6bde0f0 0x55eaf6c0dc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6bdd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6bdce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/605232684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bafe10 0x55eaf6bdc6a0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcae3e00640 0 client.7647 ms_handle_reset on v2:172.21.15.97:6833/3971758099 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 >> 172.21.15.97:0/605232684 conn(0x55eaf6a99130 msgr2=0x55eaf6baefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 shutdown_connections 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 1 -- 172.21.15.97:0/605232684 wait complete. 2024-09-10T21:31:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.416+0000 7fcb08d7d000 20 client.7647 trim_cache size 0 max 16384 2024-09-10T21:31:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 Processor -- start 2024-09-10T21:31:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 -- start start 2024-09-10T21:31:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bafe10 0x55eaf6c6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6c6ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00204650 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb00203630 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.432+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcb00149d50 con 0x55eaf6bafe10 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6c6ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6c6ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53890/0 (socket says 172.21.15.97:53890) 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 -- 172.21.15.97:0/3475294746 learned_addr learned my addr 172.21.15.97:0/3475294746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaf6000640 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcafea00640 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bafe10 0x55eaf6c6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 -- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bafe10 msgr2=0x55eaf6c6dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bafe10 0x55eaf6c6dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 -- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 msgr2=0x55eaf6c6e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c6e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 -- 172.21.15.97:0/3475294746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaf6a58d90 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaf6000640 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c6e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaff400640 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6c6ea60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e9f0 tx=0x7fcaf8091d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1ce4ee0983bb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaeea00640 1 -- 172.21.15.97:0/3475294746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8078070 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaeea00640 1 -- 172.21.15.97:0/3475294746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf8063040 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaeea00640 1 -- 172.21.15.97:0/3475294746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf801f040 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- 172.21.15.97:0/3475294746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfc2f0 msgr2=0x55eaf6c6ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6c6ea60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e9f0 tx=0x7fcaf8091d20 comp rx=0 tx=0).stop 2024-09-10T21:31:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- 172.21.15.97:0/3475294746 shutdown_connections 2024-09-10T21:31:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfc2f0 0x55eaf6c6ea60 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c6e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/3475294746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bafe10 0x55eaf6c6dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- 172.21.15.97:0/3475294746 >> 172.21.15.97:0/3475294746 conn(0x55eaf6a99130 msgr2=0x55eaf6bb7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- 172.21.15.97:0/3475294746 shutdown_connections 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- 172.21.15.97:0/3475294746 wait complete. 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- start start 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 Processor -- start 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bafe10 0x55eaf6c72230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6c72cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eaf69975d0 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eaf6c12ce0 con 0x55eaf6bafe10 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55eaf6a81a60 con 0x55eaf6bfc2f0 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcb08d7d000 10 client.0 _ll_register_callbacks cb 0x7fcaf0001da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaf6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaf6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c72770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53892/0 (socket says 172.21.15.97:53892) 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.436+0000 7fcaf6000640 1 -- 172.21.15.97:0/4214173855 learned_addr learned my addr 172.21.15.97:0/4214173855 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaff400640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6c72cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcafea00640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bafe10 0x55eaf6c72230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaf6000640 1 -- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 msgr2=0x55eaf6c72cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaf6000640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6c72cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaf6000640 1 -- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bafe10 msgr2=0x55eaf6c72230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaf6000640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bafe10 0x55eaf6c72230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaf6000640 1 -- 172.21.15.97:0/4214173855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaf69c3a90 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaff400640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6c72cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcafea00640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bafe10 0x55eaf6c72230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcaf6000640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c72770 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcae8000c10 tx=0x7fcae8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ab56457eaee087 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 1 -- 172.21.15.97:0/4214173855 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80b2040 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 1 -- 172.21.15.97:0/4214173855 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fcae80c0d80 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf8063040 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 1 -- 172.21.15.97:0/4214173855 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fcae80053b0 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 10 client.7653 fetch_fsmap learned FSMap version 94 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55eaf69a2050 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 1 -- 172.21.15.97:0/4214173855 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf8063040 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 1 -- 172.21.15.97:0/4214173855 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 94) ==== 3502+0+0 (secure 0 0 0) 0x7fcae80046d0 con 0x55eaf6bfb9d0 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 10 client.7653 fetch_fsmap finished waiting for FSMap version 94 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 10 client.7653 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:31:04.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 20 client.7653 populate_metadata read hostname 'smithi097' 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6c10c70 0x55eaf6c0fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 4 client.7653 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 --> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] -- command(tid 0: {"prefix": "status"}) -- 0x55eaf6a0b520 con 0x55eaf6c10c70 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcafea00640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6c10c70 0x55eaf6c0fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcafea00640 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6c10c70 0x55eaf6c0fd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcbe0f2650b6f74 server_cookie=e07b2bfceb20a4e2 in_seq=0 out_seq=0 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 10 client.7653 ms_handle_connect on v2:172.21.15.97:6833/3971758099 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 1 -- 172.21.15.97:0/4214173855 <== mds.0 v2:172.21.15.97:6833/3971758099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55eaf6a0b520 con 0x55eaf6c10c70 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 10 client.7653 handle_command_reply: tid=0 2024-09-10T21:31:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 client.7653 shutdown 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfb9d0 msgr2=0x55eaf6c72770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c72770 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcae8000c10 tx=0x7fcae8005040 comp rx=0 tx=0).stop 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 shutdown_connections 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:6833/3971758099,v1:172.21.15.97:6835/3971758099] conn(0x55eaf6c10c70 0x55eaf6c0fd30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eaf6bfc2f0 0x55eaf6c72cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eaf6bfb9d0 0x55eaf6c72770 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcae3e00640 0 client.7653 ms_handle_reset on v2:172.21.15.97:6833/3971758099 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 --2- 172.21.15.97:0/4214173855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eaf6bafe10 0x55eaf6c72230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 >> 172.21.15.97:0/4214173855 conn(0x55eaf6a99130 msgr2=0x55eaf6bb7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 shutdown_connections 2024-09-10T21:31:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 1 -- 172.21.15.97:0/4214173855 wait complete. 2024-09-10T21:31:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb08d7d000 20 client.7653 trim_cache size 0 max 16384 2024-09-10T21:31:04.452 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:31:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:31:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:31:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7167, 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7715553179999999, 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 94, 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 97, 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 97, 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 13.752260291000001 2024-09-10T21:31:04.453 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:31:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcadc0725e0 msgr2=0x7fcadc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcadc0725e0 0x7fcadc074aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0000f70 tx=0x7fcaf0009040 comp rx=0 tx=0).stop 2024-09-10T21:31:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb001586a0 msgr2=0x7fcb0015a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb001586a0 0x7fcb0015a450 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800ea20 tx=0x7fcaf800eee0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 shutdown_connections 2024-09-10T21:31:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcadc0725e0 0x7fcadc074aa0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb001586a0 0x7fcb0015a450 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb000cc2b0 0x7fcb0014aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 --2- 172.21.15.97:0/1003997564 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcb000b40d0 0x7fcb00158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 >> 172.21.15.97:0/1003997564 conn(0x7fcb000cfe50 msgr2=0x7fcb00152220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 shutdown_connections 2024-09-10T21:31:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.440+0000 7fcb05a00640 1 -- 172.21.15.97:0/1003997564 wait complete. 2024-09-10T21:31:04.464 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:31:04.464 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2024-09-10T21:31:04.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:31:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 -- 172.21.15.97:0/1964403142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf006ec40 msgr2=0x7fbaf006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 --2- 172.21.15.97:0/1964403142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf006ec40 0x7fbaf006f090 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009930 tx=0x7fbae001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 -- 172.21.15.97:0/1964403142 shutdown_connections 2024-09-10T21:31:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 --2- 172.21.15.97:0/1964403142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf006ec40 0x7fbaf006f090 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 -- 172.21.15.97:0/1964403142 >> 172.21.15.97:0/1964403142 conn(0x7fbaf00b1780 msgr2=0x7fbaf00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 -- 172.21.15.97:0/1964403142 shutdown_connections 2024-09-10T21:31:04.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 -- 172.21.15.97:0/1964403142 wait complete. 2024-09-10T21:31:04.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 Processor -- start 2024-09-10T21:31:04.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 -- start start 2024-09-10T21:31:04.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaf006ec40 0x7fbaf00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaf00ceee0 0x7fbaf014a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf01511f0 0x7fbaf015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.604+0000 7fbaf8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf015b280 con 0x7fbaf01511f0 2024-09-10T21:31:04.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf8800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf015b100 con 0x7fbaf00ceee0 2024-09-10T21:31:04.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf8800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf00cf350 con 0x7fbaf006ec40 2024-09-10T21:31:04.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf01511f0 0x7fbaf015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf01511f0 0x7fbaf015c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53912/0 (socket says 172.21.15.97:53912) 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 -- 172.21.15.97:0/3742222952 learned_addr learned my addr 172.21.15.97:0/3742222952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf5400640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaf00ceee0 0x7fbaf014a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf5e00640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaf006ec40 0x7fbaf00aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 -- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaf006ec40 msgr2=0x7fbaf00aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaf006ec40 0x7fbaf00aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 -- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaf00ceee0 msgr2=0x7fbaf014a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaf00ceee0 0x7fbaf014a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 -- 172.21.15.97:0/3742222952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf00a9640 con 0x7fbaf01511f0 2024-09-10T21:31:04.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf6800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf01511f0 0x7fbaf015c9e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c7b0 tx=0x7fbaec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaea000640 1 -- 172.21.15.97:0/3742222952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec017070 con 0x7fbaf01511f0 2024-09-10T21:31:04.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaea000640 1 -- 172.21.15.97:0/3742222952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaec005070 con 0x7fbaf01511f0 2024-09-10T21:31:04.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaea000640 1 -- 172.21.15.97:0/3742222952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0054c0 con 0x7fbaf01511f0 2024-09-10T21:31:04.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf00a98e0 con 0x7fbaf01511f0 2024-09-10T21:31:04.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf00a9ea0 con 0x7fbaf01511f0 2024-09-10T21:31:04.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf006a010 con 0x7fbaf01511f0 2024-09-10T21:31:04.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.608+0000 7fbaea000640 1 -- 172.21.15.97:0/3742222952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbaec005660 con 0x7fbaf01511f0 2024-09-10T21:31:04.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.612+0000 7fbaea000640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbad40725e0 0x7fbad4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:04.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.612+0000 7fbaea000640 1 -- 172.21.15.97:0/3742222952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fbaec092b40 con 0x7fbaf01511f0 2024-09-10T21:31:04.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.612+0000 7fbaf5e00640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbad40725e0 0x7fbad4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:04.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.612+0000 7fbaf5e00640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbad40725e0 0x7fbad4074aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbae4007fd0 tx=0x7fbae4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.616+0000 7fbaea000640 1 -- 172.21.15.97:0/3742222952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbaec05d820 con 0x7fbaf01511f0 2024-09-10T21:31:04.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.820+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbaf0153c00 con 0x7fbaf01511f0 2024-09-10T21:31:04.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.820+0000 7fbaea000640 1 -- 172.21.15.97:0/3742222952 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) ==== 53+0+73 (secure 0 0 0) 0x7fbaec062250 con 0x7fbaf01511f0 2024-09-10T21:31:04.824 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:31:04.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbad40725e0 msgr2=0x7fbad4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbad40725e0 0x7fbad4074aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbae4007fd0 tx=0x7fbae4009040 comp rx=0 tx=0).stop 2024-09-10T21:31:04.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf01511f0 msgr2=0x7fbaf015c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:04.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf01511f0 0x7fbaf015c9e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c7b0 tx=0x7fbaec00cc70 comp rx=0 tx=0).stop 2024-09-10T21:31:04.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 shutdown_connections 2024-09-10T21:31:04.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbad40725e0 0x7fbad4074aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf01511f0 0x7fbaf015c9e0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbaf00ceee0 0x7fbaf014a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 --2- 172.21.15.97:0/3742222952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbaf006ec40 0x7fbaf00aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:04.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 >> 172.21.15.97:0/3742222952 conn(0x7fbaf00b1780 msgr2=0x7fbaf00b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:04.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 shutdown_connections 2024-09-10T21:31:04.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:04.824+0000 7fbaf8800640 1 -- 172.21.15.97:0/3742222952 wait complete. 2024-09-10T21:31:04.844 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:31:04.844 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:31:04.844 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:31:04.845 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:31:04.845 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:04.845 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:31:04.845 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:31:04.845 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:31:04.845 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:04.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:04.845 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:04.855 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42029sec preferred_lft 42029sec 2024-09-10T21:31:04.856 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.857 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:04.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:04.858 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:31:04.914 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:31:04.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:04.916 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:31:04.969 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:31:04.969 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:31:04.969 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:31:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:31:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:31:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:31:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:31:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:04.971 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:04.971 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:31:04.971 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:31:04.971 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:31:04.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:31:04.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:31:04.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:31:04.971 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:31:05.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:31:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:31:05.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:05.111 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:31:05.120 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:31:05.120 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:31:05.121 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:05.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:05.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:31:05.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:31:05.185 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:05.237 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:05.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:31:05.237 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:05.288 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:31:05.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:05.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:05.376 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:05.377 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:05.377 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:05.392 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:05.392 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:05.392 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:05.392 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:31:05.392 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:31:05.401 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:05.401 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:05.401 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:05.401 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:31:05.401 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:31:05.402 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:31:05.402 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:31:05.402 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:31:05.405 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:31:05.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:05.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:05.424 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:31:05.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:31:05.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:31:05.521 INFO:teuthology.orchestra.run.smithi097.stdout:client6569 2024-09-10T21:31:05.522 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:05.522 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:05.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:05.531 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:31:05.575 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:31:05.582 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:05.580+0000 7f6ab0e00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-10T21:31:05.583 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:31:05.583 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:31:05.583 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:05.583 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:31:05.584 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2024-09-10T21:31:05.584 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2024-09-10T21:31:05.584 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:05.584 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:05.633 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:05.633 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:05.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:31:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 -- 172.21.15.97:0/398462943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0ceee0 msgr2=0x7ff58c0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 --2- 172.21.15.97:0/398462943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0ceee0 0x7ff58c0cf350 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7ff58400ad90 tx=0x7ff584020450 comp rx=0 tx=0).stop 2024-09-10T21:31:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 -- 172.21.15.97:0/398462943 shutdown_connections 2024-09-10T21:31:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 --2- 172.21.15.97:0/398462943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c0ceee0 0x7ff58c0cf350 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 --2- 172.21.15.97:0/398462943 >> v2:172.21.15.194:3300/0 conn(0x7ff58c06ec40 0x7ff58c06f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 -- 172.21.15.97:0/398462943 >> 172.21.15.97:0/398462943 conn(0x7ff58c0b1780 msgr2=0x7ff58c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 -- 172.21.15.97:0/398462943 shutdown_connections 2024-09-10T21:31:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 -- 172.21.15.97:0/398462943 wait complete. 2024-09-10T21:31:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 Processor -- start 2024-09-10T21:31:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.868+0000 7ff591800640 1 -- start start 2024-09-10T21:31:05.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff58c06ec40 0x7ff58c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff58c0ceee0 0x7ff58c1517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c1511f0 0x7ff58c1e3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c06e740 con 0x7ff58c1511f0 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c06e5c0 con 0x7ff58c0ceee0 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c06e8c0 con 0x7ff58c06ec40 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c1511f0 0x7ff58c1e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c1511f0 0x7ff58c1e3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53930/0 (socket says 172.21.15.97:53930) 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 -- 172.21.15.97:0/1444592575 learned_addr learned my addr 172.21.15.97:0/1444592575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58aa00640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff58c06ec40 0x7ff58c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58a000640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff58c0ceee0 0x7ff58c1517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 -- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff58c06ec40 msgr2=0x7ff58c1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:05.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff58c06ec40 0x7ff58c1593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 -- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff58c0ceee0 msgr2=0x7ff58c1517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff58c0ceee0 0x7ff58c1517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 -- 172.21.15.97:0/1444592575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c1527b0 con 0x7ff58c1511f0 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff58b400640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c1511f0 0x7ff58c1e3120 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff58400ace0 tx=0x7ff58400a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff582a00640 1 -- 172.21.15.97:0/1444592575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840c0070 con 0x7ff58c1511f0 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff582a00640 1 -- 172.21.15.97:0/1444592575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5840b6040 con 0x7ff58c1511f0 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff582a00640 1 -- 172.21.15.97:0/1444592575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840b5070 con 0x7ff58c1511f0 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c1521f0 con 0x7ff58c1511f0 2024-09-10T21:31:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1e5a90 con 0x7ff58c1511f0 2024-09-10T21:31:05.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.872+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c06f090 con 0x7ff58c1511f0 2024-09-10T21:31:05.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.876+0000 7ff582a00640 1 -- 172.21.15.97:0/1444592575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff5840b0b00 con 0x7ff58c1511f0 2024-09-10T21:31:05.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.876+0000 7ff582a00640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff56c0725e0 0x7ff56c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:05.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.876+0000 7ff582a00640 1 -- 172.21.15.97:0/1444592575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7ff584130b70 con 0x7ff58c1511f0 2024-09-10T21:31:05.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.876+0000 7ff58aa00640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff56c0725e0 0x7ff56c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:05.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.876+0000 7ff58aa00640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff56c0725e0 0x7ff56c074aa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0098e0 tx=0x7ff57c009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:05.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:05.880+0000 7ff582a00640 1 -- 172.21.15.97:0/1444592575 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff584019fb0 con 0x7ff58c1511f0 2024-09-10T21:31:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff58c15c9a0 con 0x7ff58c1511f0 2024-09-10T21:31:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff582a00640 1 -- 172.21.15.97:0/1444592575 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v97) ==== 80+0+120 (secure 0 0 0) 0x7ff5840b0e20 con 0x7ff58c1511f0 2024-09-10T21:31:06.059 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:31:06.059 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:31:06.059 INFO:teuthology.orchestra.run.smithi097.stderr:listed 2 entries 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff56c0725e0 msgr2=0x7ff56c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff56c0725e0 0x7ff56c074aa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0098e0 tx=0x7ff57c009060 comp rx=0 tx=0).stop 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c1511f0 msgr2=0x7ff58c1e3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c1511f0 0x7ff58c1e3120 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff58400ace0 tx=0x7ff58400a4a0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 shutdown_connections 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff56c0725e0 0x7ff56c074aa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff58c1511f0 0x7ff58c1e3120 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff58c0ceee0 0x7ff58c1517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 --2- 172.21.15.97:0/1444592575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff58c06ec40 0x7ff58c1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 >> 172.21.15.97:0/1444592575 conn(0x7ff58c0b1780 msgr2=0x7ff58c150810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 shutdown_connections 2024-09-10T21:31:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.056+0000 7ff591800640 1 -- 172.21.15.97:0/1444592575 wait complete. 2024-09-10T21:31:06.081 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:31:06.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:06.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:31:06.104 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:06.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:06.104 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:06.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:06.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- 172.21.15.97:0/1684887248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab015bdd0 msgr2=0x7feab0074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 --2- 172.21.15.97:0/1684887248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab015bdd0 0x7feab0074380 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7feab00b07d0 tx=0x7feaa401c890 comp rx=0 tx=0).stop 2024-09-10T21:31:06.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- 172.21.15.97:0/1684887248 shutdown_connections 2024-09-10T21:31:06.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 --2- 172.21.15.97:0/1684887248 >> v2:172.21.15.194:3300/0 conn(0x7feab00cfad0 0x7feab00cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 --2- 172.21.15.97:0/1684887248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab015bdd0 0x7feab0074380 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- 172.21.15.97:0/1684887248 >> 172.21.15.97:0/1684887248 conn(0x7feab00b28c0 msgr2=0x7feab00b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- 172.21.15.97:0/1684887248 shutdown_connections 2024-09-10T21:31:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- 172.21.15.97:0/1684887248 wait complete. 2024-09-10T21:31:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 Processor -- start 2024-09-10T21:31:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- start start 2024-09-10T21:31:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feab006cb40 0x7feab0149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab00cfad0 0x7feab00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feab015bdd0 0x7feab0166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feab0169340 con 0x7feab00cfad0 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feab01691c0 con 0x7feab006cb40 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab8600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7feab01694c0 con 0x7feab015bdd0 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab00cfad0 0x7feab00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab00cfad0 0x7feab00b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53936/0 (socket says 172.21.15.97:53936) 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab5200640 1 -- 172.21.15.97:0/2810773374 learned_addr learned my addr 172.21.15.97:0/2810773374 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab5c00640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feab006cb40 0x7feab0149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.340+0000 7feab6600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feab015bdd0 0x7feab0166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab5200640 1 -- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feab015bdd0 msgr2=0x7feab0166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab5200640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feab015bdd0 0x7feab0166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab5200640 1 -- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feab006cb40 msgr2=0x7feab0149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab5200640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feab006cb40 0x7feab0149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab5200640 1 -- 172.21.15.97:0/2810773374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab00b4200 con 0x7feab00cfad0 2024-09-10T21:31:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab6600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feab015bdd0 0x7feab0166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab5200640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab00cfad0 0x7feab00b3cc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d980 tx=0x7fea9c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feaad600640 1 -- 172.21.15.97:0/2810773374 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c017070 con 0x7feab00cfad0 2024-09-10T21:31:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feaad600640 1 -- 172.21.15.97:0/2810773374 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea9c009d60 con 0x7feab00cfad0 2024-09-10T21:31:06.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab0166790 con 0x7feab00cfad0 2024-09-10T21:31:06.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feaad600640 1 -- 172.21.15.97:0/2810773374 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c005280 con 0x7feab00cfad0 2024-09-10T21:31:06.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab006fe30 con 0x7feab00cfad0 2024-09-10T21:31:06.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab00cff40 con 0x7feab00cfad0 2024-09-10T21:31:06.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.344+0000 7feaad600640 1 -- 172.21.15.97:0/2810773374 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fea9c005420 con 0x7feab00cfad0 2024-09-10T21:31:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.348+0000 7feaad600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea90072610 0x7fea90074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.348+0000 7feaad600640 1 -- 172.21.15.97:0/2810773374 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6361+0+0 (secure 0 0 0) 0x7fea9c093660 con 0x7feab00cfad0 2024-09-10T21:31:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.348+0000 7feab5c00640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea90072610 0x7fea90074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.348+0000 7feab5c00640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea90072610 0x7fea90074ad0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feaa40af000 tx=0x7feaa40a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:06.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.352+0000 7feaad600640 1 -- 172.21.15.97:0/2810773374 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea9c05e460 con 0x7feab00cfad0 2024-09-10T21:31:06.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.548+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feab0153570 con 0x7feab00cfad0 2024-09-10T21:31:06.552 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:06.548+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.552+0000 7feaad600640 1 -- 172.21.15.97:0/2810773374 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) ==== 161+0+0 (secure 0 0 0) 0x7fea9c0671d0 con 0x7feab00cfad0 2024-09-10T21:31:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:06.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.552+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea90072610 msgr2=0x7fea90074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.552+0000 7feab8600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea90072610 0x7fea90074ad0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feaa40af000 tx=0x7feaa40a7040 comp rx=0 tx=0).stop 2024-09-10T21:31:06.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.552+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab00cfad0 msgr2=0x7feab00b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.552+0000 7feab8600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab00cfad0 0x7feab00b3cc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d980 tx=0x7fea9c00de40 comp rx=0 tx=0).stop 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 shutdown_connections 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea90072610 0x7fea90074ad0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feab015bdd0 0x7feab0166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab00cfad0 0x7feab00b3cc0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 --2- 172.21.15.97:0/2810773374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feab006cb40 0x7feab0149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 >> 172.21.15.97:0/2810773374 conn(0x7feab00b28c0 msgr2=0x7feab00af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 shutdown_connections 2024-09-10T21:31:06.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.556+0000 7feab8600640 1 -- 172.21.15.97:0/2810773374 wait complete. 2024-09-10T21:31:06.562 INFO:tasks.ceph.mds.c.smithi097.stderr: -669> 2024-09-10T21:31:01.632+0000 7f6ab5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:06.564 INFO:tasks.ceph.mds.c.smithi097.stderr: -150> 2024-09-10T21:31:05.580+0000 7f6ab0e00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-10T21:31:06.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:06.605 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:31:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 -- 172.21.15.97:0/1082471288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 msgr2=0x7f17480cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 --2- 172.21.15.97:0/1082471288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 0x7f17480cea20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f173c009840 tx=0x7f173c01c820 comp rx=0 tx=0).stop 2024-09-10T21:31:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 -- 172.21.15.97:0/1082471288 shutdown_connections 2024-09-10T21:31:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 --2- 172.21.15.97:0/1082471288 >> v2:172.21.15.194:3300/0 conn(0x7f17480cd6e0 0x7f17480af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 --2- 172.21.15.97:0/1082471288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 0x7f17480cea20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 -- 172.21.15.97:0/1082471288 >> 172.21.15.97:0/1082471288 conn(0x7f17480d1c30 msgr2=0x7f17480d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 -- 172.21.15.97:0/1082471288 shutdown_connections 2024-09-10T21:31:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 -- 172.21.15.97:0/1082471288 wait complete. 2024-09-10T21:31:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 Processor -- start 2024-09-10T21:31:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.720+0000 7f174fc00640 1 -- start start 2024-09-10T21:31:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17480b0d50 0x7f1748156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17480cd6e0 0x7f17480ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 0x7f174815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1748168d80 con 0x7f17480cd6e0 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1748168c00 con 0x7f17480b0d50 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1748168f00 con 0x7f17480ce640 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17480cd6e0 0x7f17480ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17480cd6e0 0x7f17480ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53974/0 (socket says 172.21.15.97:53974) 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 -- 172.21.15.97:0/1631611954 learned_addr learned my addr 172.21.15.97:0/1631611954 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174dc00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 0x7f174815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174d200640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17480b0d50 0x7f1748156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 -- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 msgr2=0x7f174815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 0x7f174815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 -- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17480b0d50 msgr2=0x7f1748156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17480b0d50 0x7f1748156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:06.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 -- 172.21.15.97:0/1631611954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17480ca960 con 0x7f17480cd6e0 2024-09-10T21:31:06.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1747e00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17480cd6e0 0x7f17480ca0d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f173400c460 tx=0x7f173400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1744c00640 1 -- 172.21.15.97:0/1631611954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734016020 con 0x7f17480cd6e0 2024-09-10T21:31:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1744c00640 1 -- 172.21.15.97:0/1631611954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1734012920 con 0x7f17480cd6e0 2024-09-10T21:31:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17480cac40 con 0x7f17480cd6e0 2024-09-10T21:31:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f1744c00640 1 -- 172.21.15.97:0/1631611954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734012d50 con 0x7f17480cd6e0 2024-09-10T21:31:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17480ca610 con 0x7f17480cd6e0 2024-09-10T21:31:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.724+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1748156550 con 0x7f17480cd6e0 2024-09-10T21:31:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.728+0000 7f1744c00640 1 -- 172.21.15.97:0/1631611954 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f173401b400 con 0x7f17480cd6e0 2024-09-10T21:31:06.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.728+0000 7f1744c00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f17280725e0 0x7f1728074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:06.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.728+0000 7f1744c00640 1 -- 172.21.15.97:0/1631611954 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6447+0+0 (secure 0 0 0) 0x7f1734093680 con 0x7f17480cd6e0 2024-09-10T21:31:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.728+0000 7f174d200640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f17280725e0 0x7f1728074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.728+0000 7f174d200640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f17280725e0 0x7f1728074aa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f173c0af000 tx=0x7f173c0195c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:06.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.732+0000 7f1744c00640 1 -- 172.21.15.97:0/1631611954 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1734020020 con 0x7f17480cd6e0 2024-09-10T21:31:06.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:06.956+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f17480cea20 con 0x7f17480cd6e0 2024-09-10T21:31:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f1744c00640 1 -- 172.21.15.97:0/1631611954 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v96) ==== 107+0+0 (secure 0 0 0) 0x7f173405e430 con 0x7f17480cd6e0 2024-09-10T21:31:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:31:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f17280725e0 msgr2=0x7f1728074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f17280725e0 0x7f1728074aa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f173c0af000 tx=0x7f173c0195c0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17480cd6e0 msgr2=0x7f17480ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17480cd6e0 0x7f17480ca0d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f173400c460 tx=0x7f173400c920 comp rx=0 tx=0).stop 2024-09-10T21:31:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 shutdown_connections 2024-09-10T21:31:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f17280725e0 0x7f1728074aa0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17480ce640 0x7f174815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17480cd6e0 0x7f17480ca0d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 --2- 172.21.15.97:0/1631611954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17480b0d50 0x7f1748156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 >> 172.21.15.97:0/1631611954 conn(0x7f17480d1c30 msgr2=0x7f17480aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:07.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 shutdown_connections 2024-09-10T21:31:07.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.556+0000 7f174fc00640 1 -- 172.21.15.97:0/1631611954 wait complete. 2024-09-10T21:31:07.582 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:07.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 Processor -- start 2024-09-10T21:31:07.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 -- start start 2024-09-10T21:31:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee012d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee010a890 0x557ee01c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee01c3b80 0x557ee0087ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557ee0003f40 con 0x557ee010a890 2024-09-10T21:31:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557ee0010d00 con 0x7f9dbc003c90 2024-09-10T21:31:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd82d3080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557edffa5a00 con 0x557ee01c3b80 2024-09-10T21:31:07.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee010a890 0x557ee01c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee010a890 0x557ee01c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38570/0 (socket says 172.21.15.97:38570) 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 -- 172.21.15.97:0/1667187160 learned_addr learned my addr 172.21.15.97:0/1667187160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd5400640 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee012d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd5e00640 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee01c3b80 0x557ee0087ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 -- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee01c3b80 msgr2=0x557ee0087ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee01c3b80 0x557ee0087ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 -- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 msgr2=0x557ee012d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee012d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 -- 172.21.15.97:0/1667187160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee0056eb0 con 0x557ee010a890 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dd5400640 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee012d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.672+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee010a890 0x557ee01c3640 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009520 tx=0x7f9dc00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3365c664eff1c037 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcf400640 1 -- 172.21.15.97:0/1667187160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc000d760 con 0x557ee010a890 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcf400640 1 -- 172.21.15.97:0/1667187160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dc0014020 con 0x557ee010a890 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcf400640 1 -- 172.21.15.97:0/1667187160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc000db60 con 0x557ee010a890 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- 172.21.15.97:0/1667187160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee010a890 msgr2=0x557ee01c3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee010a890 0x557ee01c3640 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009520 tx=0x7f9dc00099e0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- 172.21.15.97:0/1667187160 shutdown_connections 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee01c3b80 0x557ee0087ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee010a890 0x557ee01c3640 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/1667187160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee012d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- 172.21.15.97:0/1667187160 >> 172.21.15.97:0/1667187160 conn(0x557ee0085520 msgr2=0x557ee010ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- 172.21.15.97:0/1667187160 shutdown_connections 2024-09-10T21:31:07.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- 172.21.15.97:0/1667187160 wait complete. 2024-09-10T21:31:07.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 Processor -- start 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- start start 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee010a890 0x557ee0110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee01c3b80 0x557ee0128810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee0128d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557ee01305e0 con 0x557ee01c3b80 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557ee0128480 con 0x7f9dbc003c90 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557ee0128600 con 0x557ee010a890 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee01c3b80 0x557ee0128810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee01c3b80 0x557ee0128810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38586/0 (socket says 172.21.15.97:38586) 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 learned_addr learned my addr 172.21.15.97:0/3511499104 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd5e00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee0128d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee010a890 0x557ee0110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee010a890 msgr2=0x557ee0110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee010a890 0x557ee0110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 msgr2=0x557ee0128d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9dbc003c90 0x557ee0128d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee0116390 con 0x557ee01c3b80 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557ee010a890 0x557ee0110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee01c3b80 0x557ee0128810 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00106b0 tx=0x7f9dc00106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dce000640 1 -- 172.21.15.97:0/3511499104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0013040 con 0x557ee01c3b80 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dce000640 1 -- 172.21.15.97:0/3511499104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dc0010d10 con 0x557ee01c3b80 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ee0116630 con 0x557ee01c3b80 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dce000640 1 -- 172.21.15.97:0/3511499104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00256e0 con 0x557ee01c3b80 2024-09-10T21:31:07.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.676+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee01c0ea0 con 0x557ee01c3b80 2024-09-10T21:31:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.680+0000 7f9dce000640 1 -- 172.21.15.97:0/3511499104 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9dc002d030 con 0x557ee01c3b80 2024-09-10T21:31:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.680+0000 7f9dce000640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9db4070fc0 0x7f9db4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.680+0000 7f9dce000640 1 -- 172.21.15.97:0/3511499104 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6447+0+0 (secure 0 0 0) 0x7f9dc0093e10 con 0x557ee01c3b80 2024-09-10T21:31:07.686 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:07.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.680+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557ee012bd50 0x557ee01c1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.680+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01c1d90 con 0x557ee012bd50 2024-09-10T21:31:07.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.680+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9db4070fc0 0x7f9db4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.680+0000 7f9dd5e00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557ee012bd50 0x557ee01c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.684+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9db4070fc0 0x7f9db4073480 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000c10 tx=0x7f9dc8003040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.684+0000 7f9dd5e00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557ee012bd50 0x557ee01c1850 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.684+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v98'43 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.684+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01c2160 con 0x557ee012bd50 2024-09-10T21:31:07.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.684+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v98'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.684+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4230 con 0x557ee012bd50 2024-09-10T21:31:07.690 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:07.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.688+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v98'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.688+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d55e0 con 0x557ee012bd50 2024-09-10T21:31:07.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.688+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v98'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.688+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d5980 con 0x557ee012bd50 2024-09-10T21:31:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.692+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v98'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.692+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d45d0 con 0x557ee012bd50 2024-09-10T21:31:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.696+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v98'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.696+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4970 con 0x557ee012bd50 2024-09-10T21:31:07.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.696+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v98'49 uv22 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.696+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.700+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v98'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.700+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.704+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v98'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.707 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:07.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.704+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.708+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.708+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.708+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.708+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.712+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v98'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.712+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.712+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v98'55 uv55 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.717 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:07.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.712+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.716+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v98'56 uv56 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.716+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.716+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v98'57 uv57 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.716+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.720+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v98'58 uv58 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.720+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.720+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v98'59 uv59 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.720+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.724+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 600.00000000 [create] v98'60 uv35 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.724+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.728+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 600.00000000 [delete] v98'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.728+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.732 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:07.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.728+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v98'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.728+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.732+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v98'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.732+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.732+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v98'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.732+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.736+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v98'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.736+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.736+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 603.00000000 [create] v98'66 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.740+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.740+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 603.00000000 [delete] v98'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.740+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.745 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:07.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.744+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v98'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.744+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.744+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v98'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.744+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.744+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v98'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.744+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d4d60 con 0x557ee012bd50 2024-09-10T21:31:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.748+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v98'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee012bd50 2024-09-10T21:31:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.748+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557ee01d5140 0x557ee01d8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.748+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d8780 con 0x557ee01d5140 2024-09-10T21:31:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.748+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557ee01d5140 0x557ee01d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.748+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557ee01d5140 0x557ee01d8240 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.752+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(30 1.00000000.inode [create] v98'45 uv24 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9dc800b040 con 0x557ee01d5140 2024-09-10T21:31:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.752+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01d8b20 con 0x557ee01d5140 2024-09-10T21:31:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.752+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(31 1.00000000.inode [delete] v98'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dc800b040 con 0x557ee01d5140 2024-09-10T21:31:07.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.752+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d8ec0 con 0x557ee01d5140 2024-09-10T21:31:07.757 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:07.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(32 1.00000000.inode [create] v98'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dc800b040 con 0x557ee01d5140 2024-09-10T21:31:07.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d9710 con 0x557ee01d5140 2024-09-10T21:31:07.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v98'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dc800b040 con 0x557ee01d5140 2024-09-10T21:31:07.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d9ab0 con 0x557ee01d5140 2024-09-10T21:31:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v98'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dc800b040 con 0x557ee01d5140 2024-09-10T21:31:07.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01d9e50 con 0x557ee01d5140 2024-09-10T21:31:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(35 200.00000000 [create] v98'50 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc800b040 con 0x557ee01d5140 2024-09-10T21:31:07.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.756+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01da1f0 con 0x557ee01d5140 2024-09-10T21:31:07.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.760+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(36 200.00000000 [delete] v98'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc800b040 con 0x557ee01d5140 2024-09-10T21:31:07.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.760+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.763 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:07.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.760+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(37 200.00000000 [create] v98'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.760+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.760+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v98'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.760+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.764+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v98'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.764+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.764+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(40 200.00000001 [create] v98'55 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.764+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.764+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(41 200.00000001 [delete] v98'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.764+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.770 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:07.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.768+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(42 200.00000001 [create] v98'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.768+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de760 con 0x557ee01d5140 2024-09-10T21:31:07.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.768+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(43 200.00000001 [write 0~16801] v98'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.768+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de760 con 0x557ee01d5140 2024-09-10T21:31:07.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.772+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v98'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.772+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.772+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(45 606.00000000 [create] v98'60 uv40 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.772+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.776+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(46 606.00000000 [delete] v98'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.776+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.780 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:07.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.776+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(47 606.00000000 [create] v98'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.776+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01db5a0 con 0x557ee01d5140 2024-09-10T21:31:07.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.780+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v98'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.780+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01db5a0 con 0x557ee01d5140 2024-09-10T21:31:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v98'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01da590 con 0x557ee01d5140 2024-09-10T21:31:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v98'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01d5140 2024-09-10T21:31:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557ee01daaa0 0x557ee01dce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01dd410 con 0x557ee01daaa0 2024-09-10T21:31:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557ee01daaa0 0x557ee01dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557ee01daaa0 0x557ee01dce90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(51 607.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.784+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01dd8f0 con 0x557ee01daaa0 2024-09-10T21:31:07.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.788+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(52 607.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.788+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01dddd0 con 0x557ee01daaa0 2024-09-10T21:31:07.793 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.788+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(53 607.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.788+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01df680 con 0x557ee01daaa0 2024-09-10T21:31:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.792+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.792+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01dfa20 con 0x557ee01daaa0 2024-09-10T21:31:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.796+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.796+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de670 con 0x557ee01daaa0 2024-09-10T21:31:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.796+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557ee01def70 0x557ee01e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2310 con 0x557ee01def70 2024-09-10T21:31:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd5e00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557ee01def70 0x557ee01e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd5e00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557ee01def70 0x557ee01e1dd0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v98'11 uv10 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e2310 con 0x557ee01def70 2024-09-10T21:31:07.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v98'12 uv12 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.800+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2390 con 0x557ee01def70 2024-09-10T21:31:07.807 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:07.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.804+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v98'13 uv13 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.804+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e27b0 con 0x557ee01def70 2024-09-10T21:31:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.808+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.808+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e27b0 con 0x557ee01daaa0 2024-09-10T21:31:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.808+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(61 608.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.808+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.812+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(62 608.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.812+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.816+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(63 608.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.819 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.816+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e3b60 con 0x557ee01daaa0 2024-09-10T21:31:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.820+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.820+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e3b60 con 0x557ee01daaa0 2024-09-10T21:31:07.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.820+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.820+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.824+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.824+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.824+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(67 604.00000000 [create] v98'15 uv9 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.824+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.828+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(68 604.00000000 [delete] v98'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.828+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.832 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.828+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(69 604.00000000 [create] v98'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.828+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e3b60 con 0x557ee01def70 2024-09-10T21:31:07.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.832+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v98'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.832+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e3b60 con 0x557ee01def70 2024-09-10T21:31:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.836+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v98'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.836+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.836+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v98'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.836+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.840+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(73 500.00000000 [create] v98'21 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.840+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.844+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(74 500.00000000 [delete] v98'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.844+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.849 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:07.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.848+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(75 500.00000000 [create] v98'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.848+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.848+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v98'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.848+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01def70 2024-09-10T21:31:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.852+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v98'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.852+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.852+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(78 mds_snaptable [create] v98'7 uv6 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.852+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.856+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(79 mds_snaptable [delete] v98'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.856+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.856+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(80 mds_snaptable [create] v98'9 uv9 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.860 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:07.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.856+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.856+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v98'10 uv10 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.860+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e2b50 con 0x557ee01daaa0 2024-09-10T21:31:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.860+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v98'11 uv11 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01daaa0 2024-09-10T21:31:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.860+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557ee01e3060 0x557ee01e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.860+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5ab0 con 0x557ee01e3060 2024-09-10T21:31:07.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.860+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557ee01e3060 0x557ee01e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.860+0000 7f9dd5400640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557ee01e3060 0x557ee01e5450 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.860+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(83 605.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.864+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e5e80 con 0x557ee01e3060 2024-09-10T21:31:07.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.864+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(84 605.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.864+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5e80 con 0x557ee01e3060 2024-09-10T21:31:07.872 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:07.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.868+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(85 605.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.868+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e6e90 con 0x557ee01e3060 2024-09-10T21:31:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.868+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.868+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e6e90 con 0x557ee01e3060 2024-09-10T21:31:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.872+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.872+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5e80 con 0x557ee01e3060 2024-09-10T21:31:07.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.872+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.872+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5e80 con 0x557ee01e3060 2024-09-10T21:31:07.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.876+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(89 mds0_inotable [create] v98'14 uv8 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.876+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e5e80 con 0x557ee01e3060 2024-09-10T21:31:07.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.876+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(90 mds0_inotable [delete] v98'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.876+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5e80 con 0x557ee01e3060 2024-09-10T21:31:07.881 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:07.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.876+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(91 mds0_inotable [create] v98'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.876+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e6270 con 0x557ee01e3060 2024-09-10T21:31:07.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.880+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v98'17 uv17 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.880+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e6270 con 0x557ee01e3060 2024-09-10T21:31:07.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.880+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v98'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.880+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5f20 con 0x557ee01e3060 2024-09-10T21:31:07.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.880+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(94 100.00000000 [create] v98'19 uv13 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.880+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e5f20 con 0x557ee01e3060 2024-09-10T21:31:07.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.884+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(95 100.00000000 [delete] v98'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.884+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5f20 con 0x557ee01e3060 2024-09-10T21:31:07.888 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.884+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(96 100.00000000 [create] v98'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.884+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e5f20 con 0x557ee01e3060 2024-09-10T21:31:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.884+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v98'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.884+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de2b0 con 0x557ee01e3060 2024-09-10T21:31:07.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.888+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v98'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.888+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.888+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v98'9 uv7 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.888+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.888+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v98'10 uv10 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.888+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.893 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:07.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.892+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v98'11 uv11 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.892+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.892+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v98'12 uv12 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.892+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.892+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v98'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.892+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.896+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.896+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.898 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:07.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.896+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.896+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de270 con 0x557ee01def70 2024-09-10T21:31:07.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.896+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v98'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.896+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de250 con 0x557ee01def70 2024-09-10T21:31:07.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v98'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01de250 con 0x557ee01def70 2024-09-10T21:31:07.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v98'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dd00a9070 con 0x557ee01def70 2024-09-10T21:31:07.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557ee01e6cf0 0x557ee01e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:07.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e6cf0 2024-09-10T21:31:07.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557ee01e6cf0 0x557ee01e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:07.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dcfe00640 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557ee01e6cf0 0x557ee01e94c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:07.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v98'7 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01e6cf0 2024-09-10T21:31:07.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.900+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e6cf0 2024-09-10T21:31:07.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.904+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v98'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01e6cf0 2024-09-10T21:31:07.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.904+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e6cf0 2024-09-10T21:31:07.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.904+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v98'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01e6cf0 2024-09-10T21:31:07.907 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:07.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.904+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e6cf0 2024-09-10T21:31:07.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.904+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v98'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01e6cf0 2024-09-10T21:31:07.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.904+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e6cf0 2024-09-10T21:31:07.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.908+0000 7f9dcfe00640 1 -- 172.21.15.97:0/3511499104 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v98'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9dc005ae80 con 0x557ee01e6cf0 2024-09-10T21:31:07.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.908+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.908+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(114 100.00000000.inode [create] v98'20 uv11 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.908+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.908+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(115 100.00000000.inode [delete] v98'21 uv21 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.908+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.914 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:07.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.912+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(116 100.00000000.inode [create] v98'22 uv22 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.912+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.912+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v98'23 uv23 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.912+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.912+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v98'24 uv24 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.912+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.916+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(119 1.00000000 [create] v98'25 uv18 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.916+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.916+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(120 1.00000000 [delete] v98'26 uv26 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.916+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.922 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:07.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.920+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(121 1.00000000 [create] v98'27 uv27 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.920+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01eaa10 con 0x557ee01e3060 2024-09-10T21:31:07.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.920+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v98'28 uv28 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.920+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01eaa10 con 0x557ee01e3060 2024-09-10T21:31:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.924+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v98'29 uv29 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.924+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.924+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v98'30 uv30 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.924+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x557ee01e9a00 con 0x557ee01e3060 2024-09-10T21:31:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd5400640 1 -- 172.21.15.97:0/3511499104 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v98'31 uv31 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9dc8017040 con 0x557ee01e3060 2024-09-10T21:31:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557ee01daaa0 msgr2=0x557ee01dce90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557ee01daaa0 0x557ee01dce90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557ee01e3060 msgr2=0x557ee01e5450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557ee01e3060 0x557ee01e5450 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557ee01e6cf0 msgr2=0x557ee01e94c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557ee01e6cf0 0x557ee01e94c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557ee01def70 msgr2=0x557ee01e1dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557ee01def70 0x557ee01e1dd0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557ee01d5140 msgr2=0x557ee01d8240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557ee01d5140 0x557ee01d8240 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557ee012bd50 msgr2=0x557ee01c1850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557ee012bd50 0x557ee01c1850 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 reap_dead start 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 reap_dead start 2024-09-10T21:31:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd5e00640 1 -- 172.21.15.97:0/3511499104 reap_dead start 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9db4070fc0 msgr2=0x7f9db4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9db4070fc0 0x7f9db4073480 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000c10 tx=0x7f9dc8003040 comp rx=0 tx=0).stop 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee01c3b80 msgr2=0x557ee0128810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee01c3b80 0x557ee0128810 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00106b0 tx=0x7f9dc00106e0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 shutdown_connections 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9db4070fc0 0x7f9db4073480 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 --2- 172.21.15.97:0/3511499104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ee01c3b80 0x557ee0128810 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 >> 172.21.15.97:0/3511499104 conn(0x557ee0085520 msgr2=0x557ee0114f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 shutdown_connections 2024-09-10T21:31:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:07.928+0000 7f9dd82d3080 1 -- 172.21.15.97:0/3511499104 wait complete. 2024-09-10T21:31:07.935 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2024-09-10T21:31:08.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 Processor -- start 2024-09-10T21:31:08.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- start start 2024-09-10T21:31:08.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251fa3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 0x55d252047920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 0x55d252030c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d25202f7b0 con 0x55d25202ccc0 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d251ec2a00 con 0x55d25202e830 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d25202f930 con 0x55d2520321d0 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251fa3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251fa3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38610/0 (socket says 172.21.15.97:38610) 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 -- 172.21.15.97:0/3690319649 learned_addr learned my addr 172.21.15.97:0/3690319649 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7637e00640 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 0x55d252047920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7639200640 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 0x55d252030c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 -- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 msgr2=0x55d252030c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 0x55d252030c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 -- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 msgr2=0x55d252047920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 0x55d252047920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 -- 172.21.15.97:0/3690319649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d251ec1a20 con 0x55d25202ccc0 2024-09-10T21:31:08.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7638800640 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251fa3030 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f762c00b640 tx=0x7f762c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917289f0a7b451c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7636a00640 1 -- 172.21.15.97:0/3690319649 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c00a950 con 0x55d25202ccc0 2024-09-10T21:31:08.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7636a00640 1 -- 172.21.15.97:0/3690319649 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f762c00aaf0 con 0x55d25202ccc0 2024-09-10T21:31:08.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f7636a00640 1 -- 172.21.15.97:0/3690319649 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c008460 con 0x55d25202ccc0 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- 172.21.15.97:0/3690319649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 msgr2=0x55d251fa3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251fa3030 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f762c00b640 tx=0x7f762c004f90 comp rx=0 tx=0).stop 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- 172.21.15.97:0/3690319649 shutdown_connections 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 0x55d252030c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 0x55d252047920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 --2- 172.21.15.97:0/3690319649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251fa3030 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- 172.21.15.97:0/3690319649 >> 172.21.15.97:0/3690319649 conn(0x55d251fbc0d0 msgr2=0x55d252048620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- 172.21.15.97:0/3690319649 shutdown_connections 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 -- 172.21.15.97:0/3690319649 wait complete. 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.020+0000 7f763b5c0080 1 Processor -- start 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 -- start start 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251f74be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 0x55d25202b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 0x55d251f73140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d25202c6f0 con 0x55d25202ccc0 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d25202bb90 con 0x55d25202e830 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d25202c990 con 0x55d2520321d0 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251f74be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251f74be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38618/0 (socket says 172.21.15.97:38618) 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 -- 172.21.15.97:0/1877107487 learned_addr learned my addr 172.21.15.97:0/1877107487 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7637e00640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 0x55d25202b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7639200640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 0x55d251f73140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 -- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 msgr2=0x55d251f73140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d2520321d0 0x55d251f73140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 -- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 msgr2=0x55d25202b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d25202e830 0x55d25202b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 -- 172.21.15.97:0/1877107487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d251e9bc10 con 0x55d25202ccc0 2024-09-10T21:31:08.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7638800640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251f74be0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f762c017e00 tx=0x7f762c017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7635600640 1 -- 172.21.15.97:0/1877107487 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c004550 con 0x55d25202ccc0 2024-09-10T21:31:08.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7635600640 1 -- 172.21.15.97:0/1877107487 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f762c00acc0 con 0x55d25202ccc0 2024-09-10T21:31:08.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f7635600640 1 -- 172.21.15.97:0/1877107487 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c0206a0 con 0x55d25202ccc0 2024-09-10T21:31:08.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2520461f0 con 0x55d25202ccc0 2024-09-10T21:31:08.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.024+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d251f73f60 con 0x55d25202ccc0 2024-09-10T21:31:08.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7635600640 1 -- 172.21.15.97:0/1877107487 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f762c01e070 con 0x55d25202ccc0 2024-09-10T21:31:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7635600640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7618070ff0 0x7f76180734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7635600640 1 -- 172.21.15.97:0/1877107487 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6447+0+0 (secure 0 0 0) 0x7f762c093cf0 con 0x55d25202ccc0 2024-09-10T21:31:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7637e00640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7618070ff0 0x7f76180734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d252031180 0x55d2520e4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55d2520e48a0 con 0x55d252031180 2024-09-10T21:31:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7639200640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d252031180 0x55d2520e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7637e00640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7618070ff0 0x7f76180734b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f762400c650 tx=0x7f7624004a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7639200640 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d252031180 0x55d2520e4360 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f7639200640 1 -- 172.21.15.97:0/1877107487 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 600.00000000 [delete] v98'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7630002050 con 0x55d252031180 2024-09-10T21:31:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d252031180 msgr2=0x55d2520e4360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d252031180 0x55d2520e4360 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7618070ff0 msgr2=0x7f76180734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7618070ff0 0x7f76180734b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f762400c650 tx=0x7f7624004a50 comp rx=0 tx=0).stop 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 msgr2=0x55d251f74be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.028+0000 7f763b5c0080 1 --2- 172.21.15.97:0/1877107487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d25202ccc0 0x55d251f74be0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f762c017e00 tx=0x7f762c017e30 comp rx=0 tx=0).stop 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.032+0000 7f7639200640 1 -- 172.21.15.97:0/1877107487 reap_dead start 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.032+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 shutdown_connections 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.032+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 >> 172.21.15.97:0/1877107487 conn(0x55d251fbc0d0 msgr2=0x55d252047ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.032+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 shutdown_connections 2024-09-10T21:31:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.032+0000 7f763b5c0080 1 -- 172.21.15.97:0/1877107487 wait complete. 2024-09-10T21:31:08.039 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 -- 172.21.15.97:0/2480728208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c144690 msgr2=0x7f4f4c0b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 --2- 172.21.15.97:0/2480728208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c144690 0x7f4f4c0b4120 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c15f790 tx=0x7f4f3401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 -- 172.21.15.97:0/2480728208 shutdown_connections 2024-09-10T21:31:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 --2- 172.21.15.97:0/2480728208 >> v2:172.21.15.194:3300/0 conn(0x7f4f4c0b4660 0x7f4f4c06fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 --2- 172.21.15.97:0/2480728208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c144690 0x7f4f4c0b4120 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 --2- 172.21.15.97:0/2480728208 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f4c143f90 0x7f4f4c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 -- 172.21.15.97:0/2480728208 >> 172.21.15.97:0/2480728208 conn(0x7f4f4c0b1780 msgr2=0x7f4f4c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 -- 172.21.15.97:0/2480728208 shutdown_connections 2024-09-10T21:31:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 -- 172.21.15.97:0/2480728208 wait complete. 2024-09-10T21:31:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.180+0000 7f4f53a00640 1 Processor -- start 2024-09-10T21:31:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 -- start start 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f4c0b4660 0x7f4f4c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c143f90 0x7f4f4c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f4c144690 0x7f4f4c1fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c15edc0 con 0x7f4f4c143f90 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c15ec40 con 0x7f4f4c0b4660 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c15ef40 con 0x7f4f4c144690 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c143f90 0x7f4f4c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c143f90 0x7f4f4c0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38632/0 (socket says 172.21.15.97:38632) 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 -- 172.21.15.97:0/1277559829 learned_addr learned my addr 172.21.15.97:0/1277559829 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f51000640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f4c0b4660 0x7f4f4c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f51a00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f4c144690 0x7f4f4c1fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 -- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f4c144690 msgr2=0x7f4f4c1fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f4c144690 0x7f4f4c1fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 -- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f4c0b4660 msgr2=0x7f4f4c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f4c0b4660 0x7f4f4c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 -- 172.21.15.97:0/1277559829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0b1510 con 0x7f4f4c143f90 2024-09-10T21:31:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f43e00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c143f90 0x7f4f4c0b0fd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4f340b0040 tx=0x7f4f34005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f49600640 1 -- 172.21.15.97:0/1277559829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340b2040 con 0x7f4f4c143f90 2024-09-10T21:31:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f49600640 1 -- 172.21.15.97:0/1277559829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f34004260 con 0x7f4f4c143f90 2024-09-10T21:31:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c138830 con 0x7f4f4c143f90 2024-09-10T21:31:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f49600640 1 -- 172.21.15.97:0/1277559829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34005070 con 0x7f4f4c143f90 2024-09-10T21:31:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c13aeb0 con 0x7f4f4c143f90 2024-09-10T21:31:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.184+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c13a9c0 con 0x7f4f4c143f90 2024-09-10T21:31:08.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.188+0000 7f4f49600640 1 -- 172.21.15.97:0/1277559829 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f34005210 con 0x7f4f4c143f90 2024-09-10T21:31:08.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.188+0000 7f4f49600640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f28072600 0x7f4f28074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.188+0000 7f4f49600640 1 -- 172.21.15.97:0/1277559829 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6447+0+0 (secure 0 0 0) 0x7f4f3412cd20 con 0x7f4f4c143f90 2024-09-10T21:31:08.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.188+0000 7f4f51000640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f28072600 0x7f4f28074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.188+0000 7f4f51000640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f28072600 0x7f4f28074ac0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009be0 tx=0x7f4f3c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.192+0000 7f4f49600640 1 -- 172.21.15.97:0/1277559829 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4f340164b0 con 0x7f4f4c143f90 2024-09-10T21:31:08.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.388+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4f4c15f4d0 con 0x7f4f4c143f90 2024-09-10T21:31:08.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.716+0000 7f4f49600640 1 -- 172.21.15.97:0/1277559829 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v97) ==== 162+0+0 (secure 0 0 0) 0x7f4f340f8110 con 0x7f4f4c143f90 2024-09-10T21:31:08.721 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:08.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f28072600 msgr2=0x7f4f28074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f28072600 0x7f4f28074ac0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009be0 tx=0x7f4f3c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c143f90 msgr2=0x7f4f4c0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c143f90 0x7f4f4c0b0fd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4f340b0040 tx=0x7f4f34005d60 comp rx=0 tx=0).stop 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 shutdown_connections 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f28072600 0x7f4f28074ac0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4f4c144690 0x7f4f4c1fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f4c143f90 0x7f4f4c0b0fd0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 --2- 172.21.15.97:0/1277559829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f4c0b4660 0x7f4f4c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 >> 172.21.15.97:0/1277559829 conn(0x7f4f4c0b1780 msgr2=0x7f4f4c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 shutdown_connections 2024-09-10T21:31:08.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.720+0000 7f4f53a00640 1 -- 172.21.15.97:0/1277559829 wait complete. 2024-09-10T21:31:08.748 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.884+0000 7f41cb400640 1 -- 172.21.15.97:0/2500548175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ce9d0 msgr2=0x7f41c415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.884+0000 7f41cb400640 1 --2- 172.21.15.97:0/2500548175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ce9d0 0x7f41c415f6d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009900 tx=0x7f41b001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- 172.21.15.97:0/2500548175 shutdown_connections 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 --2- 172.21.15.97:0/2500548175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ce9d0 0x7f41c415f6d0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 --2- 172.21.15.97:0/2500548175 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce5f0 0x7f41c4141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- 172.21.15.97:0/2500548175 >> 172.21.15.97:0/2500548175 conn(0x7f41c40b1780 msgr2=0x7f41c40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- 172.21.15.97:0/2500548175 shutdown_connections 2024-09-10T21:31:08.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- 172.21.15.97:0/2500548175 wait complete. 2024-09-10T21:31:08.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 Processor -- start 2024-09-10T21:31:08.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- start start 2024-09-10T21:31:08.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41c40ce5f0 0x7f41c4158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce9d0 0x7f41c41549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c415fc10 0x7f41c413f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41690c0 con 0x7f41c415fc10 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4168f40 con 0x7f41c40ce5f0 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41cb400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f41c4169240 con 0x7f41c40ce9d0 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c415fc10 0x7f41c413f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce9d0 0x7f41c41549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c8a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41c40ce5f0 0x7f41c4158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce9d0 0x7f41c41549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:38660/0 (socket says 172.21.15.97:38660) 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 -- 172.21.15.97:0/313189190 learned_addr learned my addr 172.21.15.97:0/313189190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:08.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c415fc10 0x7f41c413f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38648/0 (socket says 172.21.15.97:38648) 2024-09-10T21:31:08.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 -- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41c40ce5f0 msgr2=0x7f41c4158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41c40ce5f0 0x7f41c4158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 -- 172.21.15.97:0/313189190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c415fc10 msgr2=0x7f41c413f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c415fc10 0x7f41c413f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c3e00640 1 -- 172.21.15.97:0/313189190 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c4158cd0 con 0x7f41c40ce9d0 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c8a00640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41c40ce5f0 0x7f41c4158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.888+0000 7f41c9400640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c415fc10 0x7f41c413f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c3e00640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce9d0 0x7f41c41549b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009850 tx=0x7f41b00a56b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c0c00640 1 -- 172.21.15.97:0/313189190 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00b2040 con 0x7f41c40ce9d0 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c0c00640 1 -- 172.21.15.97:0/313189190 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41b0005070 con 0x7f41c40ce9d0 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c0c00640 1 -- 172.21.15.97:0/313189190 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00044f0 con 0x7f41c40ce9d0 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c41579d0 con 0x7f41c40ce9d0 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c4157f90 con 0x7f41c40ce9d0 2024-09-10T21:31:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c4141d20 con 0x7f41c40ce9d0 2024-09-10T21:31:08.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c0c00640 1 -- 172.21.15.97:0/313189190 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f41b00054a0 con 0x7f41c40ce9d0 2024-09-10T21:31:08.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c0c00640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41a40725e0 0x7f41a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:08.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c0c00640 1 -- 172.21.15.97:0/313189190 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(98..98 src has 1..98) ==== 6447+0+0 (secure 0 0 0) 0x7f41b012c750 con 0x7f41c40ce9d0 2024-09-10T21:31:08.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.892+0000 7f41c8a00640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41a40725e0 0x7f41a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:08.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.896+0000 7f41c8a00640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41a40725e0 0x7f41a4074aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41b80103d0 tx=0x7f41b800b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:08.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:08.900+0000 7f41c0c00640 1 -- 172.21.15.97:0/313189190 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f41b00164d0 con 0x7f41c40ce9d0 2024-09-10T21:31:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.096+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41c4141360 con 0x7f41c40ce9d0 2024-09-10T21:31:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.096+0000 7f41c0c00640 1 -- 172.21.15.97:0/313189190 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5222 (secure 0 0 0) 0x7f41b00f7460 con 0x7f41c40ce9d0 2024-09-10T21:31:09.100 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:09.100 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":98,"btime":"2024-09-10T21:31:08:725792+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:08.725766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7680},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7680":{"gid":7680,"name":"c","rank":0,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/2192988988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2192988988},{"type":"v1","addr":"172.21.15.97:6835","nonce":2192988988}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:09.100 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 98 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41a40725e0 msgr2=0x7f41a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41a40725e0 0x7f41a4074aa0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41b80103d0 tx=0x7f41b800b390 comp rx=0 tx=0).stop 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce9d0 msgr2=0x7f41c41549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce9d0 0x7f41c41549b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009850 tx=0x7f41b00a56b0 comp rx=0 tx=0).stop 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 shutdown_connections 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41a40725e0 0x7f41a4074aa0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c415fc10 0x7f41c413f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41c40ce9d0 0x7f41c41549b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 --2- 172.21.15.97:0/313189190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41c40ce5f0 0x7f41c4158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 >> 172.21.15.97:0/313189190 conn(0x7f41c40b1780 msgr2=0x7f41c415f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 shutdown_connections 2024-09-10T21:31:09.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:09.100+0000 7f41cb400640 1 -- 172.21.15.97:0/313189190 wait complete. 2024-09-10T21:31:09.121 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:31:09.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:09.360 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:11.750 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:11.748+0000 7fd5ac200640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-10T21:31:11.750 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:11.748+0000 7fd5ac200640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T21:31:11.751 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:11.748+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:11.751 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:11.748+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:31:12.566 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:12.564+0000 7fd5b1200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:14.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:14.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 -- 172.21.15.97:0/486020192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac80b3cf0 msgr2=0x7feac806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 --2- 172.21.15.97:0/486020192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac80b3cf0 0x7feac806ec40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feab4009840 tx=0x7feab401f010 comp rx=0 tx=0).stop 2024-09-10T21:31:14.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 -- 172.21.15.97:0/486020192 shutdown_connections 2024-09-10T21:31:14.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 --2- 172.21.15.97:0/486020192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac80b3cf0 0x7feac806ec40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 --2- 172.21.15.97:0/486020192 >> v2:172.21.15.194:3300/0 conn(0x7feac81511f0 0x7feac80b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 -- 172.21.15.97:0/486020192 >> 172.21.15.97:0/486020192 conn(0x7feac80b1780 msgr2=0x7feac80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:14.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 -- 172.21.15.97:0/486020192 shutdown_connections 2024-09-10T21:31:14.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 -- 172.21.15.97:0/486020192 wait complete. 2024-09-10T21:31:14.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 Processor -- start 2024-09-10T21:31:14.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.272+0000 7feace000640 1 -- start start 2024-09-10T21:31:14.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac806f180 0x7feac814b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac80b3cf0 0x7feac81e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac81511f0 0x7feac8153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feac806e740 con 0x7feac806f180 2024-09-10T21:31:14.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feac806e5c0 con 0x7feac80b3cf0 2024-09-10T21:31:14.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7feac806e8c0 con 0x7feac81511f0 2024-09-10T21:31:14.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac806f180 0x7feac814b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac806f180 0x7feac814b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38650/0 (socket says 172.21.15.97:38650) 2024-09-10T21:31:14.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 -- 172.21.15.97:0/2080920281 learned_addr learned my addr 172.21.15.97:0/2080920281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:14.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac6a00640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac80b3cf0 0x7feac81e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7e00640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac81511f0 0x7feac8153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 -- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac81511f0 msgr2=0x7feac8153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac81511f0 0x7feac8153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 -- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac80b3cf0 msgr2=0x7feac81e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac80b3cf0 0x7feac81e33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 -- 172.21.15.97:0/2080920281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac81f5c50 con 0x7feac806f180 2024-09-10T21:31:14.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feac7400640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac806f180 0x7feac814b650 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7feab800eb20 tx=0x7feab800c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:14.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feabf400640 1 -- 172.21.15.97:0/2080920281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab800ce10 con 0x7feac806f180 2024-09-10T21:31:14.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feabf400640 1 -- 172.21.15.97:0/2080920281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feab8004110 con 0x7feac806f180 2024-09-10T21:31:14.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac81f4e90 con 0x7feac806f180 2024-09-10T21:31:14.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feabf400640 1 -- 172.21.15.97:0/2080920281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8003600 con 0x7feac806f180 2024-09-10T21:31:14.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81f5450 con 0x7feac806f180 2024-09-10T21:31:14.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac806ec40 con 0x7feac806f180 2024-09-10T21:31:14.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.276+0000 7feabf400640 1 -- 172.21.15.97:0/2080920281 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feab800b040 con 0x7feac806f180 2024-09-10T21:31:14.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.280+0000 7feabf400640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feaa80725e0 0x7feaa8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.280+0000 7feabf400640 1 -- 172.21.15.97:0/2080920281 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7feab8093750 con 0x7feac806f180 2024-09-10T21:31:14.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.280+0000 7feac6a00640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feaa80725e0 0x7feaa8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.280+0000 7feac6a00640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feaa80725e0 0x7feaa8074aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feab40b2000 tx=0x7feab401d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:14.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.284+0000 7feabf400640 1 -- 172.21.15.97:0/2080920281 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feab805eae0 con 0x7feac806f180 2024-09-10T21:31:14.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.480+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feac8150480 con 0x7feac806f180 2024-09-10T21:31:14.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.480+0000 7feabf400640 1 -- 172.21.15.97:0/2080920281 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+4390 (secure 0 0 0) 0x7feab8067030 con 0x7feac806f180 2024-09-10T21:31:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:14.486 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":102,"btime":"2024-09-10T21:31:11:753203+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:11.753200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:14.486 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 102 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feaa80725e0 msgr2=0x7feaa8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feaa80725e0 0x7feaa8074aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feab40b2000 tx=0x7feab401d850 comp rx=0 tx=0).stop 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac806f180 msgr2=0x7feac814b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac806f180 0x7feac814b650 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7feab800eb20 tx=0x7feab800c3d0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 shutdown_connections 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feaa80725e0 0x7feaa8074aa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7feac81511f0 0x7feac8153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac80b3cf0 0x7feac81e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 --2- 172.21.15.97:0/2080920281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac806f180 0x7feac814b650 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 >> 172.21.15.97:0/2080920281 conn(0x7feac80b1780 msgr2=0x7feac815a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:14.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 shutdown_connections 2024-09-10T21:31:14.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.484+0000 7feace000640 1 -- 172.21.15.97:0/2080920281 wait complete. 2024-09-10T21:31:14.506 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:31:14.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:14.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 -- 172.21.15.97:0/4034557723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 msgr2=0x7f56500b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 --2- 172.21.15.97:0/4034557723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 0x7f56500b2a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5640009bc0 tx=0x7f564001cac0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 -- 172.21.15.97:0/4034557723 shutdown_connections 2024-09-10T21:31:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 --2- 172.21.15.97:0/4034557723 >> v2:172.21.15.194:3300/0 conn(0x7f5650144690 0x7f56500b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 --2- 172.21.15.97:0/4034557723 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 0x7f56500b2a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 -- 172.21.15.97:0/4034557723 >> 172.21.15.97:0/4034557723 conn(0x7f56500b1780 msgr2=0x7f56500b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 -- 172.21.15.97:0/4034557723 shutdown_connections 2024-09-10T21:31:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 -- 172.21.15.97:0/4034557723 wait complete. 2024-09-10T21:31:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 Processor -- start 2024-09-10T21:31:14.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.648+0000 7f5654c00640 1 -- start start 2024-09-10T21:31:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56500b4660 0x7f565015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 0x7f5650141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5650144690 0x7f5650141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5650167a70 con 0x7f5650144690 2024-09-10T21:31:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56500706b0 con 0x7f56500b4660 2024-09-10T21:31:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5650167bf0 con 0x7f5650143f90 2024-09-10T21:31:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5650144690 0x7f5650141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5650144690 0x7f5650141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38654/0 (socket says 172.21.15.97:38654) 2024-09-10T21:31:14.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 -- 172.21.15.97:0/2436558052 learned_addr learned my addr 172.21.15.97:0/2436558052 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564de00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56500b4660 0x7f565015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564d400640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 0x7f5650141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 -- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 msgr2=0x7f5650141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 0x7f5650141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 -- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56500b4660 msgr2=0x7f565015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56500b4660 0x7f565015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 -- 172.21.15.97:0/2436558052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5650134410 con 0x7f5650144690 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f564e800640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5650144690 0x7f5650141db0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f564800c700 tx=0x7f564800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5646000640 1 -- 172.21.15.97:0/2436558052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648016020 con 0x7f5650144690 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5646000640 1 -- 172.21.15.97:0/2436558052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f564800a950 con 0x7f5650144690 2024-09-10T21:31:14.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5650136880 con 0x7f5650144690 2024-09-10T21:31:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5646000640 1 -- 172.21.15.97:0/2436558052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564800ada0 con 0x7f5650144690 2024-09-10T21:31:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56501346f0 con 0x7f5650144690 2024-09-10T21:31:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56500b2a80 con 0x7f5650144690 2024-09-10T21:31:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.652+0000 7f5646000640 1 -- 172.21.15.97:0/2436558052 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5648012420 con 0x7f5650144690 2024-09-10T21:31:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.656+0000 7f5646000640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56300725e0 0x7f5630074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.656+0000 7f5646000640 1 -- 172.21.15.97:0/2436558052 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7f56480938d0 con 0x7f5650144690 2024-09-10T21:31:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.656+0000 7f564de00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56300725e0 0x7f5630074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.656+0000 7f564de00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56300725e0 0x7f5630074aa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56400af000 tx=0x7f564001b300 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:14.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.660+0000 7f5646000640 1 -- 172.21.15.97:0/2436558052 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5648020020 con 0x7f5650144690 2024-09-10T21:31:14.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f565015e820 con 0x7f5650144690 2024-09-10T21:31:14.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5646000640 1 -- 172.21.15.97:0/2436558052 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+4390 (secure 0 0 0) 0x7f564805e580 con 0x7f5650144690 2024-09-10T21:31:14.863 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:14.863 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":102,"btime":"2024-09-10T21:31:11:753203+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:11.753200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:14.863 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 102 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56300725e0 msgr2=0x7f5630074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56300725e0 0x7f5630074aa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56400af000 tx=0x7f564001b300 comp rx=0 tx=0).stop 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5650144690 msgr2=0x7f5650141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5650144690 0x7f5650141db0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f564800c700 tx=0x7f564800cbc0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 shutdown_connections 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56300725e0 0x7f5630074aa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5650144690 0x7f5650141db0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5650143f90 0x7f5650141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 --2- 172.21.15.97:0/2436558052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56500b4660 0x7f565015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 >> 172.21.15.97:0/2436558052 conn(0x7f56500b1780 msgr2=0x7f5650140f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 shutdown_connections 2024-09-10T21:31:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:14.860+0000 7f5654c00640 1 -- 172.21.15.97:0/2436558052 wait complete. 2024-09-10T21:31:14.884 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:14.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- 172.21.15.97:0/622742606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec15ac30 msgr2=0x7ff9ec0b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/622742606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec15ac30 0x7ff9ec0b5d20 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0cef60 tx=0x7ff9dc01f5a0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- 172.21.15.97:0/622742606 shutdown_connections 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/622742606 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9ec0691b0 0x7ff9ec0cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/622742606 >> v2:172.21.15.194:3300/0 conn(0x7ff9ec074b20 0x7ff9ec0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/622742606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec15ac30 0x7ff9ec0b5d20 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- 172.21.15.97:0/622742606 >> 172.21.15.97:0/622742606 conn(0x7ff9ec0b3c70 msgr2=0x7ff9ec0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- 172.21.15.97:0/622742606 shutdown_connections 2024-09-10T21:31:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- 172.21.15.97:0/622742606 wait complete. 2024-09-10T21:31:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 Processor -- start 2024-09-10T21:31:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- start start 2024-09-10T21:31:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec0691b0 0x7ff9ec150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9ec074b20 0x7ff9ec15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9ec15ac30 0x7ff9ec1e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec06dd10 con 0x7ff9ec0691b0 2024-09-10T21:31:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec06db90 con 0x7ff9ec074b20 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f3a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ec06de90 con 0x7ff9ec15ac30 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec0691b0 0x7ff9ec150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec0691b0 0x7ff9ec150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38678/0 (socket says 172.21.15.97:38678) 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.024+0000 7ff9f1000640 1 -- 172.21.15.97:0/1349340092 learned_addr learned my addr 172.21.15.97:0/1349340092 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f1000640 1 -- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9ec15ac30 msgr2=0x7ff9ec1e8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f1000640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9ec15ac30 0x7ff9ec1e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f1000640 1 -- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9ec074b20 msgr2=0x7ff9ec15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f1000640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9ec074b20 0x7ff9ec15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f1000640 1 -- 172.21.15.97:0/1349340092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec1f8890 con 0x7ff9ec0691b0 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f1000640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec0691b0 0x7ff9ec150280 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000f90 tx=0x7ff9dc0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9e8c00640 1 -- 172.21.15.97:0/1349340092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0b5070 con 0x7ff9ec0691b0 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9e8c00640 1 -- 172.21.15.97:0/1349340092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9dc0b6040 con 0x7ff9ec0691b0 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec1f8b30 con 0x7ff9ec0691b0 2024-09-10T21:31:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9e8c00640 1 -- 172.21.15.97:0/1349340092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0ac040 con 0x7ff9ec0691b0 2024-09-10T21:31:15.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec1f90f0 con 0x7ff9ec0691b0 2024-09-10T21:31:15.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ec06e3d0 con 0x7ff9ec0691b0 2024-09-10T21:31:15.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.028+0000 7ff9e8c00640 1 -- 172.21.15.97:0/1349340092 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff9dc0b05b0 con 0x7ff9ec0691b0 2024-09-10T21:31:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.032+0000 7ff9e8c00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9cc072610 0x7ff9cc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.032+0000 7ff9e8c00640 1 -- 172.21.15.97:0/1349340092 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7ff9dc131510 con 0x7ff9ec0691b0 2024-09-10T21:31:15.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.032+0000 7ff9ebe00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9cc072610 0x7ff9cc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.032+0000 7ff9ebe00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9cc072610 0x7ff9cc074ad0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec06f840 tx=0x7ff9d8006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:15.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.036+0000 7ff9e8c00640 1 -- 172.21.15.97:0/1349340092 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff9dc019100 con 0x7ff9ec0691b0 2024-09-10T21:31:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.232+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9ec0cde20 con 0x7ff9ec0691b0 2024-09-10T21:31:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.232+0000 7ff9e8c00640 1 -- 172.21.15.97:0/1349340092 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+4390 (secure 0 0 0) 0x7ff9dc0fc8a0 con 0x7ff9ec0691b0 2024-09-10T21:31:15.238 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:15.238 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":102,"btime":"2024-09-10T21:31:11:753203+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:11.753200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 102 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9cc072610 msgr2=0x7ff9cc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9cc072610 0x7ff9cc074ad0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec06f840 tx=0x7ff9d8006ca0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec0691b0 msgr2=0x7ff9ec150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec0691b0 0x7ff9ec150280 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000f90 tx=0x7ff9dc0093f0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 shutdown_connections 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9cc072610 0x7ff9cc074ad0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9ec15ac30 0x7ff9ec1e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9ec074b20 0x7ff9ec15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 --2- 172.21.15.97:0/1349340092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ec0691b0 0x7ff9ec150280 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 >> 172.21.15.97:0/1349340092 conn(0x7ff9ec0b3c70 msgr2=0x7ff9ec06fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 shutdown_connections 2024-09-10T21:31:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.236+0000 7ff9f3a00640 1 -- 172.21.15.97:0/1349340092 wait complete. 2024-09-10T21:31:15.257 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2024-09-10T21:31:15.258 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2024-09-10T21:31:15.258 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:15.258 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:15.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:31:15.270 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:15.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:15.270 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:31:15.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:31:15.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 -- 172.21.15.97:0/3465356635 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17700cfcc0 msgr2=0x7f177015b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 --2- 172.21.15.97:0/3465356635 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17700cfcc0 0x7f177015b010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1768009c80 tx=0x7f176801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 -- 172.21.15.97:0/3465356635 shutdown_connections 2024-09-10T21:31:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 --2- 172.21.15.97:0/3465356635 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f17700cfcc0 0x7f177015b010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 --2- 172.21.15.97:0/3465356635 >> v2:172.21.15.194:3300/0 conn(0x7f177015ac30 0x7f17700b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 -- 172.21.15.97:0/3465356635 >> 172.21.15.97:0/3465356635 conn(0x7f17700b3c70 msgr2=0x7f17700b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 -- 172.21.15.97:0/3465356635 shutdown_connections 2024-09-10T21:31:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 -- 172.21.15.97:0/3465356635 wait complete. 2024-09-10T21:31:15.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 Processor -- start 2024-09-10T21:31:15.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.516+0000 7f1776c00640 1 -- start start 2024-09-10T21:31:15.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1770074b20 0x7f17700998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17700cfcc0 0x7f17701592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f177015ac30 0x7f1770159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f177015b8e0 con 0x7f177015ac30 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f177015b760 con 0x7f17700cfcc0 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f177015ba60 con 0x7f1770074b20 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1774c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f177015ac30 0x7f1770159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1770074b20 0x7f17700998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1774c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f177015ac30 0x7f1770159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38690/0 (socket says 172.21.15.97:38690) 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1774c00640 1 -- 172.21.15.97:0/3050788089 learned_addr learned my addr 172.21.15.97:0/3050788089 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:15.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1767400640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17700cfcc0 0x7f17701592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176fe00640 1 -- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17700cfcc0 msgr2=0x7f17701592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176fe00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17700cfcc0 0x7f17701592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176fe00640 1 -- 172.21.15.97:0/3050788089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f177015ac30 msgr2=0x7f1770159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176fe00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f177015ac30 0x7f1770159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176fe00640 1 -- 172.21.15.97:0/3050788089 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17701f57b0 con 0x7f1770074b20 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1774c00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f177015ac30 0x7f1770159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176fe00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1770074b20 0x7f17700998f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f176000e6c0 tx=0x7f176000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176cc00640 1 -- 172.21.15.97:0/3050788089 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176000cab0 con 0x7f1770074b20 2024-09-10T21:31:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176cc00640 1 -- 172.21.15.97:0/3050788089 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f176000cd40 con 0x7f1770074b20 2024-09-10T21:31:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f176cc00640 1 -- 172.21.15.97:0/3050788089 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760010690 con 0x7f1770074b20 2024-09-10T21:31:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17701f5af0 con 0x7f1770074b20 2024-09-10T21:31:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1770099e30 con 0x7f1770074b20 2024-09-10T21:31:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.520+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177015bec0 con 0x7f1770074b20 2024-09-10T21:31:15.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.524+0000 7f176cc00640 1 -- 172.21.15.97:0/3050788089 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1760010830 con 0x7f1770074b20 2024-09-10T21:31:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.524+0000 7f176cc00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f174c072640 0x7f174c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.524+0000 7f176cc00640 1 -- 172.21.15.97:0/3050788089 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7f1760093ec0 con 0x7f1770074b20 2024-09-10T21:31:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.524+0000 7f1767400640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f174c072640 0x7f174c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.524+0000 7f1767400640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f174c072640 0x7f174c074b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f177015b3b0 tx=0x7f1758009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:15.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.528+0000 7f176cc00640 1 -- 172.21.15.97:0/3050788089 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f17701f5af0 con 0x7f1770074b20 2024-09-10T21:31:15.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.728+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1770069820 con 0x7f1770074b20 2024-09-10T21:31:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f176cc00640 1 -- 172.21.15.97:0/3050788089 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) ==== 161+0+0 (secure 0 0 0) 0x7f176005ec00 con 0x7f1770074b20 2024-09-10T21:31:15.778 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f174c072640 msgr2=0x7f174c074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f174c072640 0x7f174c074b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f177015b3b0 tx=0x7f1758009040 comp rx=0 tx=0).stop 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1770074b20 msgr2=0x7f17700998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1770074b20 0x7f17700998f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f176000e6c0 tx=0x7f176000eb80 comp rx=0 tx=0).stop 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 shutdown_connections 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f174c072640 0x7f174c074b00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f177015ac30 0x7f1770159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17700cfcc0 0x7f17701592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 --2- 172.21.15.97:0/3050788089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1770074b20 0x7f17700998f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 >> 172.21.15.97:0/3050788089 conn(0x7f17700b3c70 msgr2=0x7f1770070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:15.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 shutdown_connections 2024-09-10T21:31:15.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.776+0000 7f1776c00640 1 -- 172.21.15.97:0/3050788089 wait complete. 2024-09-10T21:31:15.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.940+0000 7ff690600640 1 -- 172.21.15.97:0/2759726184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 msgr2=0x7ff670004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.940+0000 7ff690600640 1 --2- 172.21.15.97:0/2759726184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 0x7ff670004810 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff6880d2c40 tx=0x7ff67401f6e0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.940+0000 7ff68e600640 1 -- 172.21.15.97:0/2759726184 reap_dead start 2024-09-10T21:31:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- 172.21.15.97:0/2759726184 shutdown_connections 2024-09-10T21:31:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- 172.21.15.97:0/2759726184 >> 172.21.15.97:0/2759726184 conn(0x7ff6880d1c70 msgr2=0x7ff6880d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- 172.21.15.97:0/2759726184 shutdown_connections 2024-09-10T21:31:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- 172.21.15.97:0/2759726184 wait complete. 2024-09-10T21:31:15.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 Processor -- start 2024-09-10T21:31:15.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- start start 2024-09-10T21:31:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 0x7ff688071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6881f8490 0x7ff6880caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6880cb470 0x7ff6880c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6880ccb10 con 0x7ff670004030 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff6880cc990 con 0x7ff6880cb470 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff6880b21a0 con 0x7ff6881f8490 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 0x7ff688071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 0x7ff688071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38704/0 (socket says 172.21.15.97:38704) 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 -- 172.21.15.97:0/1580624043 learned_addr learned my addr 172.21.15.97:0/1580624043 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68d200640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6881f8490 0x7ff6880caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68e600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6880cb470 0x7ff6880c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 -- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6881f8490 msgr2=0x7ff6880caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6881f8490 0x7ff6880caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 -- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6880cb470 msgr2=0x7ff6880c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6880cb470 0x7ff6880c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 -- 172.21.15.97:0/1580624043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6880cba80 con 0x7ff670004030 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68d200640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6881f8490 0x7ff6880caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff68dc00640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 0x7ff688071920 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff67c011ae0 tx=0x7ff67c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff685600640 1 -- 172.21.15.97:0/1580624043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c01e070 con 0x7ff670004030 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff685600640 1 -- 172.21.15.97:0/1580624043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff67c00fe40 con 0x7ff670004030 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff685600640 1 -- 172.21.15.97:0/1580624043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0042b0 con 0x7ff670004030 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.944+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6881f79b0 con 0x7ff670004030 2024-09-10T21:31:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.948+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff688151630 con 0x7ff670004030 2024-09-10T21:31:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.948+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6880b1c60 con 0x7ff670004030 2024-09-10T21:31:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.948+0000 7ff685600640 1 -- 172.21.15.97:0/1580624043 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff67c003600 con 0x7ff670004030 2024-09-10T21:31:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.948+0000 7ff685600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6680725e0 0x7ff668074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.948+0000 7ff685600640 1 -- 172.21.15.97:0/1580624043 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7ff67c094280 con 0x7ff670004030 2024-09-10T21:31:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.948+0000 7ff68d200640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6680725e0 0x7ff668074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:15.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.952+0000 7ff68d200640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6680725e0 0x7ff668074aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b0000 tx=0x7ff67401c520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:15.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:15.952+0000 7ff685600640 1 -- 172.21.15.97:0/1580624043 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff67c05f080 con 0x7ff670004030 2024-09-10T21:31:16.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.180+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff6880afe00 con 0x7ff670004030 2024-09-10T21:31:16.506 INFO:tasks.ceph.mds.c.smithi097.stderr: -50> 2024-09-10T21:31:11.748+0000 7fd5ac200640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-10T21:31:16.506 INFO:tasks.ceph.mds.c.smithi097.stderr: -48> 2024-09-10T21:31:11.748+0000 7fd5ac200640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T21:31:16.506 INFO:tasks.ceph.mds.c.smithi097.stderr: -34> 2024-09-10T21:31:12.564+0000 7fd5b1200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:16.544 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:31:16.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.776+0000 7ff685600640 1 -- 172.21.15.97:0/1580624043 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v104) ==== 101+0+0 (secure 0 0 0) 0x7ff67c063ab0 con 0x7ff670004030 2024-09-10T21:31:16.781 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:31:16.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6680725e0 msgr2=0x7ff668074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:16.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6680725e0 0x7ff668074aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b0000 tx=0x7ff67401c520 comp rx=0 tx=0).stop 2024-09-10T21:31:16.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 msgr2=0x7ff688071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:16.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 0x7ff688071920 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff67c011ae0 tx=0x7ff67c009ea0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 shutdown_connections 2024-09-10T21:31:16.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6680725e0 0x7ff668074aa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6880cb470 0x7ff6880c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff6881f8490 0x7ff6880caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 --2- 172.21.15.97:0/1580624043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff670004030 0x7ff688071920 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 >> 172.21.15.97:0/1580624043 conn(0x7ff6880d1c70 msgr2=0x7ff688070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:16.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 shutdown_connections 2024-09-10T21:31:16.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.780+0000 7ff690600640 1 -- 172.21.15.97:0/1580624043 wait complete. 2024-09-10T21:31:16.804 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 Processor -- start 2024-09-10T21:31:16.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- start start 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f70aec0 0x55b09f7b9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70b2a0 0x55b09f7b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f7972a0 0x55b09f794b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b09f628a00 con 0x55b09f70b2a0 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b09f686f40 con 0x55b09f7972a0 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b09f799560 con 0x55b09f70aec0 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70b2a0 0x55b09f7b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70b2a0 0x55b09f7b9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38742/0 (socket says 172.21.15.97:38742) 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 -- 172.21.15.97:0/1361989151 learned_addr learned my addr 172.21.15.97:0/1361989151 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1119800640 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f70aec0 0x55b09f7b9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111a200640 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f7972a0 0x55b09f794b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 -- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f70aec0 msgr2=0x55b09f7b9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f70aec0 0x55b09f7b9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 -- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f7972a0 msgr2=0x55b09f794b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f7972a0 0x55b09f794b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 -- 172.21.15.97:0/1361989151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09f601c10 con 0x55b09f70b2a0 2024-09-10T21:31:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111a200640 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f7972a0 0x55b09f794b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1118e00640 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70b2a0 0x55b09f7b9a90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1108017df0 tx=0x7f1108017f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe45a158a1b505af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1113400640 1 -- 172.21.15.97:0/1361989151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080b9020 con 0x55b09f70b2a0 2024-09-10T21:31:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1113400640 1 -- 172.21.15.97:0/1361989151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11080abc70 con 0x55b09f70b2a0 2024-09-10T21:31:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f1113400640 1 -- 172.21.15.97:0/1361989151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080b0040 con 0x55b09f70b2a0 2024-09-10T21:31:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- 172.21.15.97:0/1361989151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70b2a0 msgr2=0x55b09f7b9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70b2a0 0x55b09f7b9a90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1108017df0 tx=0x7f1108017f80 comp rx=0 tx=0).stop 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- 172.21.15.97:0/1361989151 shutdown_connections 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f7972a0 0x55b09f794b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70b2a0 0x55b09f7b9a90 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 --2- 172.21.15.97:0/1361989151 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f70aec0 0x55b09f7b9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- 172.21.15.97:0/1361989151 >> 172.21.15.97:0/1361989151 conn(0x55b09f7220d0 msgr2=0x55b09f6db350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- 172.21.15.97:0/1361989151 shutdown_connections 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 -- 172.21.15.97:0/1361989151 wait complete. 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.848+0000 7f111c65e080 1 Processor -- start 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 -- start start 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70aec0 0x55b09f794b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f70b2a0 0x55b09f7950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f7972a0 0x55b09f843690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b09f724180 con 0x55b09f70aec0 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b09f723ee0 con 0x55b09f70b2a0 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b09f724eb0 con 0x55b09f7972a0 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70aec0 0x55b09f794b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70aec0 0x55b09f794b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38750/0 (socket says 172.21.15.97:38750) 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 learned_addr learned my addr 172.21.15.97:0/1479659728 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f70b2a0 0x55b09f7950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111a200640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f7972a0 0x55b09f843690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f7972a0 msgr2=0x55b09f843690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b09f7972a0 0x55b09f843690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f70b2a0 msgr2=0x55b09f7950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f70b2a0 0x55b09f7950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09f797b90 con 0x55b09f70aec0 2024-09-10T21:31:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b09f70b2a0 0x55b09f7950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1119800640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70aec0 0x55b09f794b60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f110c00df70 tx=0x7f110c00b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1112000640 1 -- 172.21.15.97:0/1479659728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0030d0 con 0x55b09f70aec0 2024-09-10T21:31:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1112000640 1 -- 172.21.15.97:0/1479659728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f110c016070 con 0x55b09f70aec0 2024-09-10T21:31:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f1112000640 1 -- 172.21.15.97:0/1479659728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0014e0 con 0x55b09f70aec0 2024-09-10T21:31:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b09f842770 con 0x55b09f70aec0 2024-09-10T21:31:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.852+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09f842cb0 con 0x55b09f70aec0 2024-09-10T21:31:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f1112000640 1 -- 172.21.15.97:0/1479659728 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f110c001680 con 0x55b09f70aec0 2024-09-10T21:31:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f1112000640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10fc070ff0 0x7f10fc0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f1112000640 1 -- 172.21.15.97:0/1479659728 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7f110c0922f0 con 0x55b09f70aec0 2024-09-10T21:31:16.859 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b09f844590 0x55b09f854da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10fc070ff0 0x7f10fc0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8552e0 con 0x55b09f844590 2024-09-10T21:31:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f111a200640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b09f844590 0x55b09f854da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10fc070ff0 0x7f10fc0734b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55b09f6dabf0 tx=0x7f1108004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.856+0000 7f111a200640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b09f844590 0x55b09f854da0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.860+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v99'73 uv48 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.860+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f855680 con 0x55b09f844590 2024-09-10T21:31:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.860+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v99'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.860+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f855a20 con 0x55b09f844590 2024-09-10T21:31:16.864 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.860+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v99'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.860+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856dd0 con 0x55b09f844590 2024-09-10T21:31:16.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.864+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v99'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.864+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f857170 con 0x55b09f844590 2024-09-10T21:31:16.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.864+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v99'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.864+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f855dc0 con 0x55b09f844590 2024-09-10T21:31:16.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.868+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v99'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.868+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8561e0 con 0x55b09f844590 2024-09-10T21:31:16.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.868+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v99'79 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.868+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f8565d0 con 0x55b09f844590 2024-09-10T21:31:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.868+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v99'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.868+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8565d0 con 0x55b09f844590 2024-09-10T21:31:16.874 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.872+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v99'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.872+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.872+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v99'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.872+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.872+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v99'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.872+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.876+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v99'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.876+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.876+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v99'85 uv59 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.876+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.876+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v99'86 uv86 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.876+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.882 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:16.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.880+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v99'87 uv87 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.880+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.880+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v99'88 uv88 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.880+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.884+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v99'89 uv89 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.884+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f856580 con 0x55b09f844590 2024-09-10T21:31:16.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.884+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v99'90 uv90 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.884+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f857510 con 0x55b09f844590 2024-09-10T21:31:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.884+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v99'91 uv91 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.884+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f857510 con 0x55b09f844590 2024-09-10T21:31:16.889 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:16.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.888+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v99'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.888+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f858520 con 0x55b09f844590 2024-09-10T21:31:16.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.888+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v99'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.888+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f858520 con 0x55b09f844590 2024-09-10T21:31:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.888+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v99'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.888+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f857510 con 0x55b09f844590 2024-09-10T21:31:16.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.892+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v99'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.892+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f857510 con 0x55b09f844590 2024-09-10T21:31:16.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.892+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 603.00000000 [create] v99'96 uv71 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.892+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f857510 con 0x55b09f844590 2024-09-10T21:31:16.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.892+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 603.00000000 [delete] v99'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.892+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f857510 con 0x55b09f844590 2024-09-10T21:31:16.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.896+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v99'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.899 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:16.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.896+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f858520 con 0x55b09f844590 2024-09-10T21:31:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.896+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v99'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.896+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f858520 con 0x55b09f844590 2024-09-10T21:31:16.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.900+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v99'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.900+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f857510 con 0x55b09f844590 2024-09-10T21:31:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.904+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v99'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f844590 2024-09-10T21:31:16.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.904+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b09f857a20 0x55b09f859e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.904+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85a350 con 0x55b09f857a20 2024-09-10T21:31:16.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.904+0000 7f1119800640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b09f857a20 0x55b09f859e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.904+0000 7f1119800640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b09f857a20 0x55b09f859e10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.904+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(30 1.00000000.inode [create] v99'67 uv49 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.904+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f85a6f0 con 0x55b09f857a20 2024-09-10T21:31:16.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.908+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(31 1.00000000.inode [delete] v99'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.908+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85aa90 con 0x55b09f857a20 2024-09-10T21:31:16.911 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:16.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.908+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(32 1.00000000.inode [create] v99'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.908+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85b2e0 con 0x55b09f857a20 2024-09-10T21:31:16.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.908+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v99'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.908+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85b680 con 0x55b09f857a20 2024-09-10T21:31:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.912+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v99'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.912+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85ba20 con 0x55b09f857a20 2024-09-10T21:31:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.912+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(35 200.00000000 [create] v99'72 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.912+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f85bdc0 con 0x55b09f857a20 2024-09-10T21:31:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.912+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(36 200.00000000 [delete] v99'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.912+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.918 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:16.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.916+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(37 200.00000000 [create] v99'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.916+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.916+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v99'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.916+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.920+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v99'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.920+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.920+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(40 200.00000001 [create] v99'77 uv59 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.920+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.920+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(41 200.00000001 [delete] v99'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.920+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.926 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.924+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(42 200.00000001 [create] v99'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.924+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f860330 con 0x55b09f857a20 2024-09-10T21:31:16.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.924+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(43 200.00000001 [write 0~16801] v99'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.924+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f860330 con 0x55b09f857a20 2024-09-10T21:31:16.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.928+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v99'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.928+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.928+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(45 606.00000000 [create] v99'82 uv65 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.928+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.928+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(46 606.00000000 [delete] v99'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.928+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.934 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.932+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(47 606.00000000 [create] v99'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.932+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85d170 con 0x55b09f857a20 2024-09-10T21:31:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.932+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v99'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.932+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85d170 con 0x55b09f857a20 2024-09-10T21:31:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.932+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v99'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.932+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85c160 con 0x55b09f857a20 2024-09-10T21:31:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.936+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v99'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f857a20 2024-09-10T21:31:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.936+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b09f85c7e0 0x55b09f85ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.936+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85f1a0 con 0x55b09f85c7e0 2024-09-10T21:31:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.936+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b09f85c7e0 0x55b09f85ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.936+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b09f85c7e0 0x55b09f85ebd0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.936+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(51 607.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.936+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f85f650 con 0x55b09f85c7e0 2024-09-10T21:31:16.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.940+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(52 607.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.940+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85fb30 con 0x55b09f85c7e0 2024-09-10T21:31:16.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.940+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(53 607.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.943 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:16.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.940+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8613e0 con 0x55b09f85c7e0 2024-09-10T21:31:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.940+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.940+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f861780 con 0x55b09f85c7e0 2024-09-10T21:31:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8603d0 con 0x55b09f85c7e0 2024-09-10T21:31:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b09f860cd0 0x55b09f863b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8603d0 con 0x55b09f860cd0 2024-09-10T21:31:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f111a200640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b09f860cd0 0x55b09f863b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f111a200640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b09f860cd0 0x55b09f863b30 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.944+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f8603d0 con 0x55b09f860cd0 2024-09-10T21:31:16.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v99'16 uv16 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.950 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:16.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v99'17 uv17 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v99'18 uv18 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(61 608.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.948+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.952+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(62 608.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.952+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.952+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(63 608.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.955 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:16.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.952+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f865080 con 0x55b09f85c7e0 2024-09-10T21:31:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.952+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.952+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f865080 con 0x55b09f85c7e0 2024-09-10T21:31:16.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.956+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.956+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.960+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.960+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.960+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(67 604.00000000 [create] v99'26 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.960+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.960+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(68 604.00000000 [delete] v99'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.960+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.965 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:16.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(69 604.00000000 [create] v99'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f865080 con 0x55b09f860cd0 2024-09-10T21:31:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v99'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f865080 con 0x55b09f860cd0 2024-09-10T21:31:16.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v99'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v99'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.964+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.968+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(73 500.00000000 [create] v99'32 uv25 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.968+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.968+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(74 500.00000000 [delete] v99'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.968+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.972 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.968+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(75 500.00000000 [create] v99'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.968+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v99'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f860cd0 2024-09-10T21:31:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v99'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(78 mds_snaptable [create] v99'12 uv11 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(79 mds_snaptable [delete] v99'13 uv13 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.972+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(80 mds_snaptable [create] v99'14 uv14 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.978 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v99'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f864070 con 0x55b09f85c7e0 2024-09-10T21:31:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v99'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f85c7e0 2024-09-10T21:31:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b09f8646f0 0x55b09f866ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867120 con 0x55b09f8646f0 2024-09-10T21:31:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f1119800640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b09f8646f0 0x55b09f866ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:16.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.976+0000 7f1119800640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b09f8646f0 0x55b09f866ae0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:16.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.980+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(83 605.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.980+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f8674f0 con 0x55b09f8646f0 2024-09-10T21:31:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.980+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(84 605.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.980+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8674f0 con 0x55b09f8646f0 2024-09-10T21:31:16.984 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:16.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.980+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(85 605.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.980+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868500 con 0x55b09f8646f0 2024-09-10T21:31:16.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868500 con 0x55b09f8646f0 2024-09-10T21:31:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8674f0 con 0x55b09f8646f0 2024-09-10T21:31:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8674f0 con 0x55b09f8646f0 2024-09-10T21:31:16.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(89 mds0_inotable [create] v99'24 uv18 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.984+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f8674f0 con 0x55b09f8646f0 2024-09-10T21:31:16.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.988+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(90 mds0_inotable [delete] v99'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.988+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8674f0 con 0x55b09f8646f0 2024-09-10T21:31:16.991 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:16.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.988+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(91 mds0_inotable [create] v99'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.988+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8678e0 con 0x55b09f8646f0 2024-09-10T21:31:16.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.988+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v99'27 uv27 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.988+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867590 con 0x55b09f8646f0 2024-09-10T21:31:16.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.992+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v99'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.992+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867590 con 0x55b09f8646f0 2024-09-10T21:31:16.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.992+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(94 100.00000000 [create] v99'29 uv23 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.992+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f867590 con 0x55b09f8646f0 2024-09-10T21:31:16.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.992+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(95 100.00000000 [delete] v99'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.992+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867590 con 0x55b09f8646f0 2024-09-10T21:31:16.997 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:16.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.996+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(96 100.00000000 [create] v99'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.996+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867590 con 0x55b09f8646f0 2024-09-10T21:31:16.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.996+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v99'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:16.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.996+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85fff0 con 0x55b09f8646f0 2024-09-10T21:31:17.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.996+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v99'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:16.996+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f85ffb0 con 0x55b09f860cd0 2024-09-10T21:31:17.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.000+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v99'14 uv12 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.000+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f867980 con 0x55b09f860cd0 2024-09-10T21:31:17.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.000+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v99'15 uv15 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.000+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867980 con 0x55b09f860cd0 2024-09-10T21:31:17.004 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:17.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.000+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v99'16 uv16 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.000+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867980 con 0x55b09f860cd0 2024-09-10T21:31:17.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v99'17 uv17 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867980 con 0x55b09f860cd0 2024-09-10T21:31:17.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v99'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f867980 con 0x55b09f860cd0 2024-09-10T21:31:17.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867980 con 0x55b09f860cd0 2024-09-10T21:31:17.009 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:17.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v99'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.004+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868990 con 0x55b09f860cd0 2024-09-10T21:31:17.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.008+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v99'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.008+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868990 con 0x55b09f860cd0 2024-09-10T21:31:17.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.008+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v99'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.008+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f867980 con 0x55b09f860cd0 2024-09-10T21:31:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.012+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v99'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1114002040 con 0x55b09f860cd0 2024-09-10T21:31:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.012+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b09f868000 0x55b09f868400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.012+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868940 con 0x55b09f868000 2024-09-10T21:31:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.012+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b09f868000 0x55b09f868400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.012+0000 7f1118e00640 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b09f868000 0x55b09f868400 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.016+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v99'12 uv11 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f868000 2024-09-10T21:31:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.016+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f868940 con 0x55b09f868000 2024-09-10T21:31:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.020+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v99'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f868000 2024-09-10T21:31:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.020+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868940 con 0x55b09f868000 2024-09-10T21:31:17.024 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.020+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v99'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f868000 2024-09-10T21:31:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.020+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f868000 2024-09-10T21:31:17.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.024+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v99'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f868000 2024-09-10T21:31:17.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.024+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f868000 2024-09-10T21:31:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.024+0000 7f1118e00640 1 -- 172.21.15.97:0/1479659728 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v99'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f11080bb040 con 0x55b09f868000 2024-09-10T21:31:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.024+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.028+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(114 100.00000000.inode [create] v99'33 uv24 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.028+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.028+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(115 100.00000000.inode [delete] v99'34 uv34 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.028+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.033 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.032+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(116 100.00000000.inode [create] v99'35 uv35 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.032+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.032+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v99'36 uv36 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.032+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.036+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v99'37 uv37 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.036+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.036+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(119 1.00000000 [create] v99'38 uv31 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.036+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.036+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(120 1.00000000 [delete] v99'39 uv39 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.036+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.041 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:17.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.040+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(121 1.00000000 [create] v99'40 uv40 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.040+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.040+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v99'41 uv41 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.040+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.040+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.040+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f868d90 con 0x55b09f8646f0 2024-09-10T21:31:17.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.044+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v99'43 uv43 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.044+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55b09f8691a0 con 0x55b09f8646f0 2024-09-10T21:31:17.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.044+0000 7f1119800640 1 -- 172.21.15.97:0/1479659728 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v99'44 uv44 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c059170 con 0x55b09f8646f0 2024-09-10T21:31:17.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b09f85c7e0 msgr2=0x55b09f85ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b09f85c7e0 0x55b09f85ebd0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b09f8646f0 msgr2=0x55b09f866ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b09f8646f0 0x55b09f866ae0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b09f868000 msgr2=0x55b09f868400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b09f868000 0x55b09f868400 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b09f860cd0 msgr2=0x55b09f863b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b09f860cd0 0x55b09f863b30 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b09f857a20 msgr2=0x55b09f859e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 reap_dead start 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b09f857a20 0x55b09f859e10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111a200640 1 -- 172.21.15.97:0/1479659728 reap_dead start 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b09f844590 msgr2=0x55b09f854da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b09f844590 0x55b09f854da0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10fc070ff0 msgr2=0x7f10fc0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10fc070ff0 0x7f10fc0734b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55b09f6dabf0 tx=0x7f1108004890 comp rx=0 tx=0).stop 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70aec0 msgr2=0x55b09f794b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70aec0 0x55b09f794b60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f110c00df70 tx=0x7f110c00b660 comp rx=0 tx=0).stop 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 shutdown_connections 2024-09-10T21:31:17.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b09f857a20 0x55b09f859e10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b09f844590 0x55b09f854da0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f10fc070ff0 0x7f10fc0734b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 --2- 172.21.15.97:0/1479659728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b09f70aec0 0x55b09f794b60 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 >> 172.21.15.97:0/1479659728 conn(0x55b09f7220d0 msgr2=0x55b09f6dba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 shutdown_connections 2024-09-10T21:31:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.048+0000 7f111c65e080 1 -- 172.21.15.97:0/1479659728 wait complete. 2024-09-10T21:31:17.054 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2024-09-10T21:31:17.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 Processor -- start 2024-09-10T21:31:17.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 -- start start 2024-09-10T21:31:17.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f283940 0x56029f286600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 0x56029f255330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f30e2a0 0x56029f255870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56029f1a2a00 con 0x56029f30e2a0 2024-09-10T21:31:17.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56029f200f40 con 0x56029f283940 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56029f310080 con 0x56029f30db00 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f30e2a0 0x56029f255870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f283940 0x56029f286600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f30e2a0 0x56029f255870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38764/0 (socket says 172.21.15.97:38764) 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 -- 172.21.15.97:0/894192974 learned_addr learned my addr 172.21.15.97:0/894192974 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405ca00640 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 0x56029f255330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 -- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 msgr2=0x56029f255330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 0x56029f255330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 -- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f283940 msgr2=0x56029f286600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f283940 0x56029f286600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 -- 172.21.15.97:0/894192974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029f17bc10 con 0x56029f30e2a0 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405d400640 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f283940 0x56029f286600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f405de00640 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f30e2a0 0x56029f255870 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4058019a30 tx=0x7f4058018120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e8b70dfef37b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f4057400640 1 -- 172.21.15.97:0/894192974 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580ba030 con 0x56029f30e2a0 2024-09-10T21:31:17.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f4057400640 1 -- 172.21.15.97:0/894192974 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4058004110 con 0x56029f30e2a0 2024-09-10T21:31:17.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f4057400640 1 -- 172.21.15.97:0/894192974 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580af030 con 0x56029f30e2a0 2024-09-10T21:31:17.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 -- 172.21.15.97:0/894192974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f30e2a0 msgr2=0x56029f255870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.096+0000 7f40601e8080 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f30e2a0 0x56029f255870 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4058019a30 tx=0x7f4058018120 comp rx=0 tx=0).stop 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- 172.21.15.97:0/894192974 shutdown_connections 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f30e2a0 0x56029f255870 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 0x56029f255330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 --2- 172.21.15.97:0/894192974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f283940 0x56029f286600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- 172.21.15.97:0/894192974 >> 172.21.15.97:0/894192974 conn(0x56029f282520 msgr2=0x56029f253da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- 172.21.15.97:0/894192974 shutdown_connections 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- 172.21.15.97:0/894192974 wait complete. 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 Processor -- start 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- start start 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f283940 0x56029f32a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 0x56029f256850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f30e2a0 0x56029f285670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56029f287200 con 0x56029f283940 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56029f2874a0 con 0x56029f30e2a0 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56029f287740 con 0x56029f30db00 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f283940 0x56029f32a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f283940 0x56029f32a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38780/0 (socket says 172.21.15.97:38780) 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 -- 172.21.15.97:0/4058153831 learned_addr learned my addr 172.21.15.97:0/4058153831 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405ca00640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 0x56029f256850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405de00640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f30e2a0 0x56029f285670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 -- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 msgr2=0x56029f256850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56029f30db00 0x56029f256850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 -- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f30e2a0 msgr2=0x56029f285670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56029f30e2a0 0x56029f285670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 -- 172.21.15.97:0/4058153831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029f2571d0 con 0x56029f283940 2024-09-10T21:31:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f405d400640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f283940 0x56029f32a2b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f405000a780 tx=0x7f405000ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f4056000640 1 -- 172.21.15.97:0/4058153831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050009d60 con 0x56029f283940 2024-09-10T21:31:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f4056000640 1 -- 172.21.15.97:0/4058153831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4050008430 con 0x56029f283940 2024-09-10T21:31:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56029f257470 con 0x56029f283940 2024-09-10T21:31:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f4056000640 1 -- 172.21.15.97:0/4058153831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050008880 con 0x56029f283940 2024-09-10T21:31:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.100+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029f2579b0 con 0x56029f283940 2024-09-10T21:31:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f4056000640 1 -- 172.21.15.97:0/4058153831 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4050008a20 con 0x56029f283940 2024-09-10T21:31:17.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f4056000640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4040070ff0 0x7f40400734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f4056000640 1 -- 172.21.15.97:0/4058153831 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7f4050093330 con 0x56029f283940 2024-09-10T21:31:17.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f405ca00640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4040070ff0 0x7f40400734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f40601e8080 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56029f286800 0x56029f2588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x56029f256d90 con 0x56029f286800 2024-09-10T21:31:17.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f405de00640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56029f286800 0x56029f2588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.104+0000 7f405ca00640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4040070ff0 0x7f40400734b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f404c000f70 tx=0x7f404c002b40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f405de00640 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56029f286800 0x56029f2588e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f405de00640 1 -- 172.21.15.97:0/4058153831 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 1.00000000.inode [delete] v99'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4058002050 con 0x56029f286800 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56029f286800 msgr2=0x56029f2588e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56029f286800 0x56029f2588e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4040070ff0 msgr2=0x7f40400734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4040070ff0 0x7f40400734b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f404c000f70 tx=0x7f404c002b40 comp rx=0 tx=0).stop 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f283940 msgr2=0x56029f32a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 --2- 172.21.15.97:0/4058153831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56029f283940 0x56029f32a2b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f405000a780 tx=0x7f405000ac40 comp rx=0 tx=0).stop 2024-09-10T21:31:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f405de00640 1 -- 172.21.15.97:0/4058153831 reap_dead start 2024-09-10T21:31:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 shutdown_connections 2024-09-10T21:31:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 >> 172.21.15.97:0/4058153831 conn(0x56029f282520 msgr2=0x56029f2553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 shutdown_connections 2024-09-10T21:31:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.108+0000 7f40601e8080 1 -- 172.21.15.97:0/4058153831 wait complete. 2024-09-10T21:31:17.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.300+0000 7f26b0a00640 1 -- 172.21.15.97:0/4036676286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac15f350 msgr2=0x7f26ac0b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.300+0000 7f26b0a00640 1 --2- 172.21.15.97:0/4036676286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac15f350 0x7f26ac0b3070 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f269c009840 tx=0x7f269c01c790 comp rx=0 tx=0).stop 2024-09-10T21:31:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.300+0000 7f26b0a00640 1 -- 172.21.15.97:0/4036676286 shutdown_connections 2024-09-10T21:31:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.300+0000 7f26b0a00640 1 --2- 172.21.15.97:0/4036676286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac15f350 0x7f26ac0b3070 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.300+0000 7f26b0a00640 1 -- 172.21.15.97:0/4036676286 >> 172.21.15.97:0/4036676286 conn(0x7f26ac0d1c60 msgr2=0x7f26ac0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.300+0000 7f26b0a00640 1 -- 172.21.15.97:0/4036676286 shutdown_connections 2024-09-10T21:31:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.300+0000 7f26b0a00640 1 -- 172.21.15.97:0/4036676286 wait complete. 2024-09-10T21:31:17.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 Processor -- start 2024-09-10T21:31:17.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 -- start start 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0d0150 0x7f26ac0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d05e0 0x7f26ac0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15f350 0x7f26ac0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac169dc0 con 0x7f26ac0d05e0 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac169c40 con 0x7f26ac15f350 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f26ac169f40 con 0x7f26ac0d0150 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d05e0 0x7f26ac0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d05e0 0x7f26ac0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33836/0 (socket says 172.21.15.97:33836) 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 -- 172.21.15.97:0/791179269 learned_addr learned my addr 172.21.15.97:0/791179269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0d0150 0x7f26ac0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26aa800640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15f350 0x7f26ac0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 -- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0d0150 msgr2=0x7f26ac0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0d0150 0x7f26ac0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 -- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15f350 msgr2=0x7f26ac0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15f350 0x7f26ac0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 -- 172.21.15.97:0/791179269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac06e800 con 0x7f26ac0d05e0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a9400640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d05e0 0x7f26ac0c9740 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f269800c450 tx=0x7f269800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a2000640 1 -- 172.21.15.97:0/791179269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698017070 con 0x7f26ac0d05e0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a2000640 1 -- 172.21.15.97:0/791179269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2698003600 con 0x7f26ac0d05e0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26a2000640 1 -- 172.21.15.97:0/791179269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698003a50 con 0x7f26ac0d05e0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac06eaa0 con 0x7f26ac0d05e0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.304+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac06f060 con 0x7f26ac0d05e0 2024-09-10T21:31:17.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.308+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ac0b3070 con 0x7f26ac0d05e0 2024-09-10T21:31:17.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.308+0000 7f26a2000640 1 -- 172.21.15.97:0/791179269 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2698003bf0 con 0x7f26ac0d05e0 2024-09-10T21:31:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.308+0000 7f26a2000640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f268c072610 0x7f268c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.308+0000 7f26a2000640 1 -- 172.21.15.97:0/791179269 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6533+0+0 (secure 0 0 0) 0x7f2698092b10 con 0x7f26ac0d05e0 2024-09-10T21:31:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.308+0000 7f26a9e00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f268c072610 0x7f268c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.312+0000 7f26a9e00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f268c072610 0x7f268c074ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f269c0af000 tx=0x7f269c01afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.312+0000 7f26a2000640 1 -- 172.21.15.97:0/791179269 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f269805d7c0 con 0x7f26ac0d05e0 2024-09-10T21:31:17.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.512+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f26ac0b13a0 con 0x7f26ac0d05e0 2024-09-10T21:31:17.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.784+0000 7f26a2000640 1 -- 172.21.15.97:0/791179269 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v105) ==== 162+0+0 (secure 0 0 0) 0x7f26980621f0 con 0x7f26ac0d05e0 2024-09-10T21:31:17.788 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f268c072610 msgr2=0x7f268c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f268c072610 0x7f268c074ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f269c0af000 tx=0x7f269c01afd0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d05e0 msgr2=0x7f26ac0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d05e0 0x7f26ac0c9740 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f269800c450 tx=0x7f269800c910 comp rx=0 tx=0).stop 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 shutdown_connections 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f268c072610 0x7f268c074ad0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15f350 0x7f26ac0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d05e0 0x7f26ac0c9740 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 --2- 172.21.15.97:0/791179269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0d0150 0x7f26ac0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 >> 172.21.15.97:0/791179269 conn(0x7f26ac0d1c60 msgr2=0x7f26ac0b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:17.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 shutdown_connections 2024-09-10T21:31:17.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.788+0000 7f26b0a00640 1 -- 172.21.15.97:0/791179269 wait complete. 2024-09-10T21:31:17.799 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:17.796+0000 7fe18c800640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T21:31:17.799 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:17.796+0000 7fe18c800640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:31:17.800 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:17.796+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:17.800 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:17.796+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:31:17.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 -- 172.21.15.97:0/3017309615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 msgr2=0x7ff7000aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 --2- 172.21.15.97:0/3017309615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff7000aedc0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009840 tx=0x7ff6ec01c820 comp rx=0 tx=0).stop 2024-09-10T21:31:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 -- 172.21.15.97:0/3017309615 shutdown_connections 2024-09-10T21:31:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 --2- 172.21.15.97:0/3017309615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff7000aedc0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 --2- 172.21.15.97:0/3017309615 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7000ce640 0x7ff7000cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 -- 172.21.15.97:0/3017309615 >> 172.21.15.97:0/3017309615 conn(0x7ff7000d1c30 msgr2=0x7ff7000d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 -- 172.21.15.97:0/3017309615 shutdown_connections 2024-09-10T21:31:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.952+0000 7ff705200640 1 -- 172.21.15.97:0/3017309615 wait complete. 2024-09-10T21:31:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 Processor -- start 2024-09-10T21:31:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 -- start start 2024-09-10T21:31:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7000af300 0x7ff7000b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff700157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7000ce640 0x7ff700071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff700169270 con 0x7ff7000b0d50 2024-09-10T21:31:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7001690f0 con 0x7ff7000af300 2024-09-10T21:31:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff7001693f0 con 0x7ff7000ce640 2024-09-10T21:31:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff700157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff700157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33846/0 (socket says 172.21.15.97:33846) 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 -- 172.21.15.97:0/3253457940 learned_addr learned my addr 172.21.15.97:0/3253457940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fe800640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7000ce640 0x7ff700071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fde00640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7000af300 0x7ff7000b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 -- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7000ce640 msgr2=0x7ff700071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7000ce640 0x7ff700071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 -- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7000af300 msgr2=0x7ff7000b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7000af300 0x7ff7000b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 -- 172.21.15.97:0/3253457940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff700157870 con 0x7ff7000b0d50 2024-09-10T21:31:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6fd400640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff700157330 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec000c10 tx=0x7ff6ec005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6f6000640 1 -- 172.21.15.97:0/3253457940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec0b2040 con 0x7ff7000b0d50 2024-09-10T21:31:17.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6f6000640 1 -- 172.21.15.97:0/3253457940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6ec004240 con 0x7ff7000b0d50 2024-09-10T21:31:17.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7000b4be0 con 0x7ff7000b0d50 2024-09-10T21:31:17.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff6f6000640 1 -- 172.21.15.97:0/3253457940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec005020 con 0x7ff7000b0d50 2024-09-10T21:31:17.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.956+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7000b5180 con 0x7ff7000b0d50 2024-09-10T21:31:17.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.960+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7000aedc0 con 0x7ff7000b0d50 2024-09-10T21:31:17.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.960+0000 7ff6f6000640 1 -- 172.21.15.97:0/3253457940 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff6ec0b5070 con 0x7ff7000b0d50 2024-09-10T21:31:17.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.960+0000 7ff6f6000640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e00725e0 0x7ff6e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:17.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.960+0000 7ff6f6000640 1 -- 172.21.15.97:0/3253457940 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7ff6ec12ccb0 con 0x7ff7000b0d50 2024-09-10T21:31:17.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.960+0000 7ff6fde00640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e00725e0 0x7ff6e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:17.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.960+0000 7ff6fde00640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e00725e0 0x7ff6e0074aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009ad0 tx=0x7ff6f00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:17.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:17.964+0000 7ff6f6000640 1 -- 172.21.15.97:0/3253457940 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff6ec016410 con 0x7ff7000b0d50 2024-09-10T21:31:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.164+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7000cf1e0 con 0x7ff7000b0d50 2024-09-10T21:31:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.164+0000 7ff6f6000640 1 -- 172.21.15.97:0/3253457940 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4391 (secure 0 0 0) 0x7ff6ec0f7ff0 con 0x7ff7000b0d50 2024-09-10T21:31:18.168 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:18.168 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":107,"btime":"2024-09-10T21:31:17:802745+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:17.802683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 107 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e00725e0 msgr2=0x7ff6e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e00725e0 0x7ff6e0074aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009ad0 tx=0x7ff6f00091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 msgr2=0x7ff700157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff700157330 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec000c10 tx=0x7ff6ec005db0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 shutdown_connections 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6e00725e0 0x7ff6e0074aa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7000ce640 0x7ff700071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7000b0d50 0x7ff700157330 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 --2- 172.21.15.97:0/3253457940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7000af300 0x7ff7000b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 >> 172.21.15.97:0/3253457940 conn(0x7ff7000d1c30 msgr2=0x7ff70006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 shutdown_connections 2024-09-10T21:31:18.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.168+0000 7ff705200640 1 -- 172.21.15.97:0/3253457940 wait complete. 2024-09-10T21:31:18.191 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:18.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:18.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 -- 172.21.15.97:0/4154962832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780ce640 msgr2=0x7fb7780cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 --2- 172.21.15.97:0/4154962832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780ce640 0x7fb7780cea20 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb7780b36f0 tx=0x7fb76801cb80 comp rx=0 tx=0).stop 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 -- 172.21.15.97:0/4154962832 shutdown_connections 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 --2- 172.21.15.97:0/4154962832 >> v2:172.21.15.194:3300/0 conn(0x7fb7780b0d50 0x7fb7780aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 --2- 172.21.15.97:0/4154962832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780ce640 0x7fb7780cea20 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 -- 172.21.15.97:0/4154962832 >> 172.21.15.97:0/4154962832 conn(0x7fb7780d1c30 msgr2=0x7fb7780d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 -- 172.21.15.97:0/4154962832 shutdown_connections 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.328+0000 7fb77da00640 1 -- 172.21.15.97:0/4154962832 wait complete. 2024-09-10T21:31:18.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 Processor -- start 2024-09-10T21:31:18.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 -- start start 2024-09-10T21:31:18.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7780af300 0x7fb7781f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780b0d50 0x7fb7780b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7780ce640 0x7fb77815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb778169270 con 0x7fb7780b0d50 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781690f0 con 0x7fb7780af300 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb7781693f0 con 0x7fb7780ce640 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780b0d50 0x7fb7780b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780b0d50 0x7fb7780b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33874/0 (socket says 172.21.15.97:33874) 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 -- 172.21.15.97:0/2919131284 learned_addr learned my addr 172.21.15.97:0/2919131284 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb776a00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7780af300 0x7fb7781f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb777400640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7780ce640 0x7fb77815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 -- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7780ce640 msgr2=0x7fb77815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7780ce640 0x7fb77815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 -- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7780af300 msgr2=0x7fb7781f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7780af300 0x7fb7781f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 -- 172.21.15.97:0/2919131284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb778068fb0 con 0x7fb7780b0d50 2024-09-10T21:31:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76e000640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780b0d50 0x7fb7780b2890 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fb76000b580 tx=0x7fb76000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76f400640 1 -- 172.21.15.97:0/2919131284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760014070 con 0x7fb7780b0d50 2024-09-10T21:31:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76f400640 1 -- 172.21.15.97:0/2919131284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb760010980 con 0x7fb7780b0d50 2024-09-10T21:31:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb76f400640 1 -- 172.21.15.97:0/2919131284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760010dd0 con 0x7fb7780b0d50 2024-09-10T21:31:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7781fa0d0 con 0x7fb7780b0d50 2024-09-10T21:31:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb778157240 con 0x7fb7780b0d50 2024-09-10T21:31:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.332+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7780aedc0 con 0x7fb7780b0d50 2024-09-10T21:31:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.336+0000 7fb76f400640 1 -- 172.21.15.97:0/2919131284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb76000f3c0 con 0x7fb7780b0d50 2024-09-10T21:31:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.336+0000 7fb76f400640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7540725e0 0x7fb754074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.336+0000 7fb76f400640 1 -- 172.21.15.97:0/2919131284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7fb7600936d0 con 0x7fb7780b0d50 2024-09-10T21:31:18.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.336+0000 7fb776a00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7540725e0 0x7fb754074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.336+0000 7fb776a00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7540725e0 0x7fb754074aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7680af000 tx=0x7fb76801b3c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:18.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.340+0000 7fb76f400640 1 -- 172.21.15.97:0/2919131284 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb7781fa0d0 con 0x7fb7780b0d50 2024-09-10T21:31:18.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.540+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7780b7ac0 con 0x7fb7780b0d50 2024-09-10T21:31:18.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.540+0000 7fb76f400640 1 -- 172.21.15.97:0/2919131284 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4391 (secure 0 0 0) 0x7fb76005e340 con 0x7fb7780b0d50 2024-09-10T21:31:18.543 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:18.543 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":107,"btime":"2024-09-10T21:31:17:802745+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:17.802683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:18.544 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 107 2024-09-10T21:31:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.540+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7540725e0 msgr2=0x7fb754074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.540+0000 7fb77da00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7540725e0 0x7fb754074aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7680af000 tx=0x7fb76801b3c0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.540+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780b0d50 msgr2=0x7fb7780b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.540+0000 7fb77da00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780b0d50 0x7fb7780b2890 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fb76000b580 tx=0x7fb76000ba40 comp rx=0 tx=0).stop 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 shutdown_connections 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb7540725e0 0x7fb754074aa0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb7780ce640 0x7fb77815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7780b0d50 0x7fb7780b2890 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 --2- 172.21.15.97:0/2919131284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7780af300 0x7fb7781f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 >> 172.21.15.97:0/2919131284 conn(0x7fb7780d1c30 msgr2=0x7fb7780d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 shutdown_connections 2024-09-10T21:31:18.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.544+0000 7fb77da00640 1 -- 172.21.15.97:0/2919131284 wait complete. 2024-09-10T21:31:18.564 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:18.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 -- 172.21.15.97:0/3381395619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0cde30 msgr2=0x7f2afc0cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 --2- 172.21.15.97:0/3381395619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0cde30 0x7f2afc0cef00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f2afc073250 tx=0x7f2af401c800 comp rx=0 tx=0).stop 2024-09-10T21:31:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 -- 172.21.15.97:0/3381395619 shutdown_connections 2024-09-10T21:31:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 --2- 172.21.15.97:0/3381395619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0cde30 0x7f2afc0cef00 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 --2- 172.21.15.97:0/3381395619 >> v2:172.21.15.194:3300/0 conn(0x7f2afc0d0710 0x7f2afc0ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 -- 172.21.15.97:0/3381395619 >> 172.21.15.97:0/3381395619 conn(0x7f2afc0d1ca0 msgr2=0x7f2afc0d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:18.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 -- 172.21.15.97:0/3381395619 shutdown_connections 2024-09-10T21:31:18.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.704+0000 7f2b03800640 1 -- 172.21.15.97:0/3381395619 wait complete. 2024-09-10T21:31:18.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 Processor -- start 2024-09-10T21:31:18.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 -- start start 2024-09-10T21:31:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0cde30 0x7f2afc09c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0ce5e0 0x7f2afc0afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2afc0d0710 0x7f2afc09efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0cd350 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0cd1d0 con 0x7f2afc0cde30 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc169060 con 0x7f2afc0d0710 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0ce5e0 0x7f2afc0afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0ce5e0 0x7f2afc0afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33892/0 (socket says 172.21.15.97:33892) 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 -- 172.21.15.97:0/1780884545 learned_addr learned my addr 172.21.15.97:0/1780884545 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b00e00640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0cde30 0x7f2afc09c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b01800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2afc0d0710 0x7f2afc09efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 -- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2afc0d0710 msgr2=0x7f2afc09efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2afc0d0710 0x7f2afc09efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 -- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0cde30 msgr2=0x7f2afc09c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0cde30 0x7f2afc09c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 -- 172.21.15.97:0/1780884545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc15dc50 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b01800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2afc0d0710 0x7f2afc09efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:18.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2afbe00640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0ce5e0 0x7f2afc0afb00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400d950 tx=0x7f2ae400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2af8c00640 1 -- 172.21.15.97:0/1780884545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4017070 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2af8c00640 1 -- 172.21.15.97:0/1780884545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ae4009cc0 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2af8c00640 1 -- 172.21.15.97:0/1780884545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40106c0 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc15b4a0 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc15ba60 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.708+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc0b2010 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.712+0000 7f2af8c00640 1 -- 172.21.15.97:0/1780884545 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2ae4010860 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.712+0000 7f2af8c00640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ad80725e0 0x7f2ad8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.712+0000 7f2af8c00640 1 -- 172.21.15.97:0/1780884545 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7f2ae4093a50 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.712+0000 7f2b00e00640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ad80725e0 0x7f2ad8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.712+0000 7f2b00e00640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ad80725e0 0x7f2ad8074aa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0021a0 tx=0x7f2aec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:18.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.716+0000 7f2af8c00640 1 -- 172.21.15.97:0/1780884545 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ae405ed90 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.920+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2afc0b2390 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.920+0000 7f2af8c00640 1 -- 172.21.15.97:0/1780884545 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4391 (secure 0 0 0) 0x7f2ae405e690 con 0x7f2afc0ce5e0 2024-09-10T21:31:18.925 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:18.925 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":107,"btime":"2024-09-10T21:31:17:802745+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6951,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/769007942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":769007942},{"type":"v1","addr":"172.21.15.97:6834","nonce":769007942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:17.802683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:18.926 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 107 2024-09-10T21:31:18.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ad80725e0 msgr2=0x7f2ad8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ad80725e0 0x7f2ad8074aa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0021a0 tx=0x7f2aec009040 comp rx=0 tx=0).stop 2024-09-10T21:31:18.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0ce5e0 msgr2=0x7f2afc0afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0ce5e0 0x7f2afc0afb00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400d950 tx=0x7f2ae400de10 comp rx=0 tx=0).stop 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 shutdown_connections 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ad80725e0 0x7f2ad8074aa0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2afc0d0710 0x7f2afc09efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2afc0ce5e0 0x7f2afc0afb00 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 --2- 172.21.15.97:0/1780884545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0cde30 0x7f2afc09c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 >> 172.21.15.97:0/1780884545 conn(0x7f2afc0d1ca0 msgr2=0x7f2afc0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:18.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 shutdown_connections 2024-09-10T21:31:18.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:18.924+0000 7f2b03800640 1 -- 172.21.15.97:0/1780884545 wait complete. 2024-09-10T21:31:18.947 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2024-09-10T21:31:18.947 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2024-09-10T21:31:18.947 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:18.948 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:18.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:31:18.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:18.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:18.957 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:31:19.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:31:19.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 -- 172.21.15.97:0/3817185620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3230141d20 msgr2=0x7f323015f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 --2- 172.21.15.97:0/3817185620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3230141d20 0x7f323015f6d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f321c009840 tx=0x7f321c01c740 comp rx=0 tx=0).stop 2024-09-10T21:31:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 -- 172.21.15.97:0/3817185620 shutdown_connections 2024-09-10T21:31:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 --2- 172.21.15.97:0/3817185620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3230141d20 0x7f323015f6d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 -- 172.21.15.97:0/3817185620 >> 172.21.15.97:0/3817185620 conn(0x7f32300b1780 msgr2=0x7f32300b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 -- 172.21.15.97:0/3817185620 shutdown_connections 2024-09-10T21:31:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 -- 172.21.15.97:0/3817185620 wait complete. 2024-09-10T21:31:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 Processor -- start 2024-09-10T21:31:19.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.192+0000 7f3235000640 1 -- start start 2024-09-10T21:31:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32300ce5f0 0x7f32300b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230141d20 0x7f323013b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f323015fc10 0x7f3230166c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3230169cb0 con 0x7f3230141d20 2024-09-10T21:31:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3230169b30 con 0x7f323015fc10 2024-09-10T21:31:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3230169e30 con 0x7f32300ce5f0 2024-09-10T21:31:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230141d20 0x7f323013b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230141d20 0x7f323013b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33908/0 (socket says 172.21.15.97:33908) 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 -- 172.21.15.97:0/453299030 learned_addr learned my addr 172.21.15.97:0/453299030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322e800640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f323015fc10 0x7f3230166c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322de00640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32300ce5f0 0x7f32300b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 -- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32300ce5f0 msgr2=0x7f32300b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32300ce5f0 0x7f32300b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 -- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f323015fc10 msgr2=0x7f3230166c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f323015fc10 0x7f3230166c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 -- 172.21.15.97:0/453299030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323013b5d0 con 0x7f3230141d20 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f322d400640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230141d20 0x7f323013b090 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c10 tx=0x7f321c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3226000640 1 -- 172.21.15.97:0/453299030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0047d0 con 0x7f3230141d20 2024-09-10T21:31:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3226000640 1 -- 172.21.15.97:0/453299030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f321c004970 con 0x7f3230141d20 2024-09-10T21:31:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3226000640 1 -- 172.21.15.97:0/453299030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c004d60 con 0x7f3230141d20 2024-09-10T21:31:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32300b3350 con 0x7f3230141d20 2024-09-10T21:31:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32300b3910 con 0x7f3230141d20 2024-09-10T21:31:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.196+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323015f6d0 con 0x7f3230141d20 2024-09-10T21:31:19.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.200+0000 7f3226000640 1 -- 172.21.15.97:0/453299030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f321c0bb420 con 0x7f3230141d20 2024-09-10T21:31:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.200+0000 7f3226000640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32100725e0 0x7f3210074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.200+0000 7f3226000640 1 -- 172.21.15.97:0/453299030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7f321c12d5d0 con 0x7f3230141d20 2024-09-10T21:31:19.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.200+0000 7f322de00640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32100725e0 0x7f3210074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.200+0000 7f322de00640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32100725e0 0x7f3210074aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f32200012f0 tx=0x7f3220009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.204+0000 7f3226000640 1 -- 172.21.15.97:0/453299030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f321c0c2030 con 0x7f3230141d20 2024-09-10T21:31:19.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.400+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3230141360 con 0x7f3230141d20 2024-09-10T21:31:19.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3226000640 1 -- 172.21.15.97:0/453299030 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) ==== 161+0+0 (secure 0 0 0) 0x7f321c0ac030 con 0x7f3230141d20 2024-09-10T21:31:19.806 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32100725e0 msgr2=0x7f3210074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32100725e0 0x7f3210074aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f32200012f0 tx=0x7f3220009040 comp rx=0 tx=0).stop 2024-09-10T21:31:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230141d20 msgr2=0x7f323013b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230141d20 0x7f323013b090 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c10 tx=0x7f321c005dc0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 shutdown_connections 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32100725e0 0x7f3210074aa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f323015fc10 0x7f3230166c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230141d20 0x7f323013b090 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 --2- 172.21.15.97:0/453299030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32300ce5f0 0x7f32300b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 >> 172.21.15.97:0/453299030 conn(0x7f32300b1780 msgr2=0x7f323013e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 shutdown_connections 2024-09-10T21:31:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.804+0000 7f3235000640 1 -- 172.21.15.97:0/453299030 wait complete. 2024-09-10T21:31:19.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:19.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- 172.21.15.97:0/3274755937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa68806ec40 msgr2=0x7fa68806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 --2- 172.21.15.97:0/3274755937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa68806ec40 0x7fa68806f090 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa67000a980 tx=0x7fa670020340 comp rx=0 tx=0).stop 2024-09-10T21:31:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- 172.21.15.97:0/3274755937 shutdown_connections 2024-09-10T21:31:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 --2- 172.21.15.97:0/3274755937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa68806ec40 0x7fa68806f090 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- 172.21.15.97:0/3274755937 >> 172.21.15.97:0/3274755937 conn(0x7fa6880b1780 msgr2=0x7fa6880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- 172.21.15.97:0/3274755937 shutdown_connections 2024-09-10T21:31:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- 172.21.15.97:0/3274755937 wait complete. 2024-09-10T21:31:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 Processor -- start 2024-09-10T21:31:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- start start 2024-09-10T21:31:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa68806ec40 0x7fa6881e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6880ceee0 0x7fa6880b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6881511f0 0x7fa688142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa68815b280 con 0x7fa6880ceee0 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa68815b100 con 0x7fa68806ec40 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa68e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880cf350 con 0x7fa6881511f0 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa67ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6880ceee0 0x7fa6880b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa67ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6880ceee0 0x7fa6880b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33930/0 (socket says 172.21.15.97:33930) 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.972+0000 7fa67ea00640 1 -- 172.21.15.97:0/758815498 learned_addr learned my addr 172.21.15.97:0/758815498 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa687400640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa68806ec40 0x7fa6881e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa687e00640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6881511f0 0x7fa688142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67ea00640 1 -- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6881511f0 msgr2=0x7fa688142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67ea00640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6881511f0 0x7fa688142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67ea00640 1 -- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa68806ec40 msgr2=0x7fa6881e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67ea00640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa68806ec40 0x7fa6881e4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67ea00640 1 -- 172.21.15.97:0/758815498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6881f29e0 con 0x7fa6880ceee0 2024-09-10T21:31:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67ea00640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6880ceee0 0x7fa6880b0210 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c10 tx=0x7fa6700b0560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67fe00640 1 -- 172.21.15.97:0/758815498 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700b5070 con 0x7fa6880ceee0 2024-09-10T21:31:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67fe00640 1 -- 172.21.15.97:0/758815498 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6700b6040 con 0x7fa6880ceee0 2024-09-10T21:31:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67fe00640 1 -- 172.21.15.97:0/758815498 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670005290 con 0x7fa6880ceee0 2024-09-10T21:31:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6881f2cc0 con 0x7fa6880ceee0 2024-09-10T21:31:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881e5190 con 0x7fa6880ceee0 2024-09-10T21:31:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6880cf7d0 con 0x7fa6880ceee0 2024-09-10T21:31:19.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.976+0000 7fa67fe00640 1 -- 172.21.15.97:0/758815498 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa67000b040 con 0x7fa6880ceee0 2024-09-10T21:31:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.980+0000 7fa67fe00640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6640725e0 0x7fa664074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:19.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.980+0000 7fa67fe00640 1 -- 172.21.15.97:0/758815498 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7fa6701310e0 con 0x7fa6880ceee0 2024-09-10T21:31:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.980+0000 7fa687400640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6640725e0 0x7fa664074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:19.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.980+0000 7fa687400640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6640725e0 0x7fa664074aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa6780012f0 tx=0x7fa678009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:19.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:19.984+0000 7fa67fe00640 1 -- 172.21.15.97:0/758815498 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa6700c8020 con 0x7fa6880ceee0 2024-09-10T21:31:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.208+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa6881f27b0 con 0x7fa6880ceee0 2024-09-10T21:31:20.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.816+0000 7fa67fe00640 1 -- 172.21.15.97:0/758815498 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v109) ==== 101+0+0 (secure 0 0 0) 0x7fa670019ea0 con 0x7fa6880ceee0 2024-09-10T21:31:20.819 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:31:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.816+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6640725e0 msgr2=0x7fa664074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.816+0000 7fa68e200640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6640725e0 0x7fa664074aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa6780012f0 tx=0x7fa678009040 comp rx=0 tx=0).stop 2024-09-10T21:31:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.816+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6880ceee0 msgr2=0x7fa6880b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.816+0000 7fa68e200640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6880ceee0 0x7fa6880b0210 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c10 tx=0x7fa6700b0560 comp rx=0 tx=0).stop 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 shutdown_connections 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6640725e0 0x7fa664074aa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6881511f0 0x7fa688142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6880ceee0 0x7fa6880b0210 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 --2- 172.21.15.97:0/758815498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa68806ec40 0x7fa6881e4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 >> 172.21.15.97:0/758815498 conn(0x7fa6880b1780 msgr2=0x7fa6881537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 shutdown_connections 2024-09-10T21:31:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.820+0000 7fa68e200640 1 -- 172.21.15.97:0/758815498 wait complete. 2024-09-10T21:31:20.844 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 Processor -- start 2024-09-10T21:31:20.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- start start 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb466b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb4722a00 0x55cfb46983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb47261d0 0x55cfb46b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cfb45b6a00 con 0x55cfb4720cc0 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4614f40 con 0x55cfb4722a00 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cfb4724710 con 0x55cfb47261d0 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb466b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb466b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33952/0 (socket says 172.21.15.97:33952) 2024-09-10T21:31:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 -- 172.21.15.97:0/452394155 learned_addr learned my addr 172.21.15.97:0/452394155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f1043e00640 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb4722a00 0x55cfb46983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104b200640 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb47261d0 0x55cfb46b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 -- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb47261d0 msgr2=0x55cfb46b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb47261d0 0x55cfb46b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 -- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb4722a00 msgr2=0x55cfb46983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb4722a00 0x55cfb46983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 -- 172.21.15.97:0/452394155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb458fc10 con 0x55cfb4720cc0 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f1043e00640 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb4722a00 0x55cfb46983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104a800640 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb466b360 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f103c018df0 tx=0x7f103c017f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6364328680d31c93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f1049400640 1 -- 172.21.15.97:0/452394155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0b9020 con 0x55cfb4720cc0 2024-09-10T21:31:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f1049400640 1 -- 172.21.15.97:0/452394155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f103c0aba70 con 0x55cfb4720cc0 2024-09-10T21:31:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f1049400640 1 -- 172.21.15.97:0/452394155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0b0040 con 0x55cfb4720cc0 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- 172.21.15.97:0/452394155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 msgr2=0x55cfb466b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb466b360 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f103c018df0 tx=0x7f103c017f60 comp rx=0 tx=0).stop 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- 172.21.15.97:0/452394155 shutdown_connections 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb47261d0 0x55cfb46b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb4722a00 0x55cfb46983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 --2- 172.21.15.97:0/452394155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb466b360 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- 172.21.15.97:0/452394155 >> 172.21.15.97:0/452394155 conn(0x55cfb46b00d0 msgr2=0x55cfb4698d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.888+0000 7f104d6da080 1 -- 172.21.15.97:0/452394155 shutdown_connections 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 -- 172.21.15.97:0/452394155 wait complete. 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 Processor -- start 2024-09-10T21:31:20.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 -- start start 2024-09-10T21:31:20.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb471e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb4722a00 0x55cfb46aab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb47261d0 0x55cfb46b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cfb471f770 con 0x55cfb4720cc0 2024-09-10T21:31:20.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cfb471fa10 con 0x55cfb47261d0 2024-09-10T21:31:20.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55cfb473db30 con 0x55cfb4722a00 2024-09-10T21:31:20.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb471e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb471e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33962/0 (socket says 172.21.15.97:33962) 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 learned_addr learned my addr 172.21.15.97:0/1533932266 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb4722a00 msgr2=0x55cfb46aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55cfb4722a00 0x55cfb46aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb47261d0 msgr2=0x55cfb46b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cfb47261d0 0x55cfb46b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb46ab690 con 0x55cfb4720cc0 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104a800640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb471e5f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f103c018df0 tx=0x7f103c0ada20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f1043400640 1 -- 172.21.15.97:0/1533932266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0c5070 con 0x55cfb4720cc0 2024-09-10T21:31:20.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f1043400640 1 -- 172.21.15.97:0/1533932266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f103c0ade80 con 0x55cfb4720cc0 2024-09-10T21:31:20.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfb46ab930 con 0x55cfb4720cc0 2024-09-10T21:31:20.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f1043400640 1 -- 172.21.15.97:0/1533932266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c0b9040 con 0x55cfb4720cc0 2024-09-10T21:31:20.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.892+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb4727f10 con 0x55cfb4720cc0 2024-09-10T21:31:20.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f1043400640 1 -- 172.21.15.97:0/1533932266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f103c0abbc0 con 0x55cfb4720cc0 2024-09-10T21:31:20.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f1043400640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1028070fc0 0x7f1028073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f1043400640 1 -- 172.21.15.97:0/1533932266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7f103c12dd60 con 0x55cfb4720cc0 2024-09-10T21:31:20.900 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:20.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55cfb47283b0 0x55cfb4728790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb4728cd0 con 0x55cfb47283b0 2024-09-10T21:31:20.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f1043e00640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1028070fc0 0x7f1028073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f104b200640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55cfb47283b0 0x55cfb4728790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f1043e00640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1028070fc0 0x7f1028073480 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1034000f70 tx=0x7f1034003520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:20.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.896+0000 7f104b200640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55cfb47283b0 0x55cfb4728790 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:20.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.900+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v100'102 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.900+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47290a0 con 0x55cfb47283b0 2024-09-10T21:31:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.900+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v100'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.900+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb4729440 con 0x55cfb47283b0 2024-09-10T21:31:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.904+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v100'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.907 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:20.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.904+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e32d0 con 0x55cfb47283b0 2024-09-10T21:31:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.908+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.908+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e3670 con 0x55cfb47283b0 2024-09-10T21:31:20.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.908+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.908+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e22c0 con 0x55cfb47283b0 2024-09-10T21:31:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.912+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.912+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2690 con 0x55cfb47283b0 2024-09-10T21:31:20.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.912+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v100'108 uv84 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.912+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47e2a80 con 0x55cfb47283b0 2024-09-10T21:31:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.912+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v100'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.912+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a80 con 0x55cfb47283b0 2024-09-10T21:31:20.918 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.916+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v100'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.916+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.916+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.916+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.916+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.920+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.920+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.920+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.920+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v100'114 uv91 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.920+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.924+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'115 uv115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.924+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.924+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v100'116 uv116 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.924+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.924+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'117 uv117 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.924+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.928+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'118 uv118 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.928+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2a30 con 0x55cfb47283b0 2024-09-10T21:31:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.928+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'119 uv119 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.928+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb46ab0c0 con 0x55cfb47283b0 2024-09-10T21:31:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.932+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'120 uv120 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.932+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.936+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v100'121 uv95 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.936+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.936+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v100'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.936+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.941 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:20.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.940+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v100'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.940+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.940+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.940+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.944+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.944+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.944+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.944+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.948+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v100'127 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.948+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.952+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v100'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.952+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.957 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:20.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.952+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v100'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.952+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.956+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.956+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.960+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.960+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e2cb0 con 0x55cfb47283b0 2024-09-10T21:31:20.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.960+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f104400b040 con 0x55cfb47283b0 2024-09-10T21:31:20.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.960+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55cfb47e3a10 0x55cfb47e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.964+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e6340 con 0x55cfb47e3a10 2024-09-10T21:31:20.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.964+0000 7f104a800640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55cfb47e3a10 0x55cfb47e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.964+0000 7f104a800640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55cfb47e3a10 0x55cfb47e5e00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:20.970 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:20.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.964+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.964+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e6b90 con 0x55cfb47e3a10 2024-09-10T21:31:20.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.968+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v100'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.968+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e6f30 con 0x55cfb47e3a10 2024-09-10T21:31:20.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.968+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v100'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.968+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e72d0 con 0x55cfb47e3a10 2024-09-10T21:31:20.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.968+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 200.00000000 [create] v100'92 uv76 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.968+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47e7670 con 0x55cfb47e3a10 2024-09-10T21:31:20.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.972+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 200.00000000 [delete] v100'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.972+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7a10 con 0x55cfb47e3a10 2024-09-10T21:31:20.975 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:20.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.972+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v100'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.972+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.972+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v100'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.972+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.976+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v100'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.976+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.976+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000001 [create] v100'97 uv81 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.976+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.976+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000001 [delete] v100'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.976+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.981 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:20.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.980+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v100'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.980+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47ebf80 con 0x55cfb47e3a10 2024-09-10T21:31:20.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.980+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [write 0~16801] v100'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.980+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47ebf80 con 0x55cfb47e3a10 2024-09-10T21:31:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.980+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v100'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.980+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.984+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 606.00000000 [create] v100'102 uv87 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.984+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.984+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 606.00000000 [delete] v100'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.984+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.988 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:20.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.984+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v100'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.984+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e8dc0 con 0x55cfb47e3a10 2024-09-10T21:31:20.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v100'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e8dc0 con 0x55cfb47e3a10 2024-09-10T21:31:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v100'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e7db0 con 0x55cfb47e3a10 2024-09-10T21:31:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v100'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47e3a10 2024-09-10T21:31:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55cfb47e8430 0x55cfb47ea820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47ead60 con 0x55cfb47e8430 2024-09-10T21:31:20.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f1043e00640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55cfb47e8430 0x55cfb47ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:20.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.988+0000 7f1043e00640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55cfb47e8430 0x55cfb47ea820 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:20.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.992+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(51 607.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:20.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.992+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47eb1e0 con 0x55cfb47e8430 2024-09-10T21:31:20.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.992+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(52 607.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:20.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.992+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47e67d0 con 0x55cfb47e8430 2024-09-10T21:31:20.997 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.996+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(53 607.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.996+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47ecff0 con 0x55cfb47e8430 2024-09-10T21:31:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.996+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.996+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47ed390 con 0x55cfb47e8430 2024-09-10T21:31:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.996+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:20.996+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47ebfe0 con 0x55cfb47e8430 2024-09-10T21:31:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cfb47ec8e0 0x55cfb47ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47efc80 con 0x55cfb47ec8e0 2024-09-10T21:31:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104b200640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cfb47ec8e0 0x55cfb47ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104b200640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cfb47ec8e0 0x55cfb47ef740 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v100'19 uv18 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47efc80 con 0x55cfb47ec8e0 2024-09-10T21:31:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v100'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.000+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0070 con 0x55cfb47ec8e0 2024-09-10T21:31:21.005 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:21.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0070 con 0x55cfb47ec8e0 2024-09-10T21:31:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v100'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0070 con 0x55cfb47e8430 2024-09-10T21:31:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(61 608.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(62 608.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.004+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.010 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.008+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(63 608.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.008+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f1420 con 0x55cfb47e8430 2024-09-10T21:31:21.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.008+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.008+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f1420 con 0x55cfb47e8430 2024-09-10T21:31:21.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.008+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.008+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.012+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.012+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.012+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(67 604.00000000 [create] v100'37 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.012+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.012+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(68 604.00000000 [delete] v100'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.012+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.017 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:21.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.016+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(69 604.00000000 [create] v100'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.016+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f1420 con 0x55cfb47ec8e0 2024-09-10T21:31:21.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.016+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v100'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.016+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f1420 con 0x55cfb47ec8e0 2024-09-10T21:31:21.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.016+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v100'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.016+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v100'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(73 500.00000000 [create] v100'43 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(74 500.00000000 [delete] v100'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(75 500.00000000 [create] v100'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.025 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:21.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.020+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.024+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v100'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.024+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47ec8e0 2024-09-10T21:31:21.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.024+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v100'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.024+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.024+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(78 mds_snaptable [create] v100'17 uv16 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.024+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.028+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(79 mds_snaptable [delete] v100'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.028+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.033 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.032+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(80 mds_snaptable [create] v100'19 uv19 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.032+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.032+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v100'20 uv20 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.032+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f0410 con 0x55cfb47e8430 2024-09-10T21:31:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.036+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v100'21 uv21 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47e8430 2024-09-10T21:31:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.036+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cfb47f0d10 0x55cfb47f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.036+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3640 con 0x55cfb47f0d10 2024-09-10T21:31:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.036+0000 7f104a800640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cfb47f0d10 0x55cfb47f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.036+0000 7f104a800640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cfb47f0d10 0x55cfb47f3100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.036+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(83 605.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.036+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f3680 con 0x55cfb47f0d10 2024-09-10T21:31:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(84 605.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.043 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(85 605.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f4ab0 con 0x55cfb47f0d10 2024-09-10T21:31:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f4ab0 con 0x55cfb47f0d10 2024-09-10T21:31:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.040+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.044+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.044+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.044+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(89 mds0_inotable [create] v100'34 uv28 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.044+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.044+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(90 mds0_inotable [delete] v100'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.044+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.049 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.048+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(91 mds0_inotable [create] v100'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.048+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.048+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v100'37 uv37 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.048+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.048+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v100'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.048+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(94 100.00000000 [create] v100'39 uv33 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(95 100.00000000 [delete] v100'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.055 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:21.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(96 100.00000000 [create] v100'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3aa0 con 0x55cfb47f0d10 2024-09-10T21:31:21.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v100'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.052+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f68b0 con 0x55cfb47f0d10 2024-09-10T21:31:21.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.056+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v100'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.056+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.056+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v100'18 uv17 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.056+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.056+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v100'19 uv19 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.056+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.060+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v100'20 uv20 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.061 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:21.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.060+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.060+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v100'21 uv21 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.060+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.060+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v100'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.060+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.064+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.064+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.068 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.064+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v100'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.064+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f4f60 con 0x55cfb47ec8e0 2024-09-10T21:31:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.068+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v100'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.068+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f4f60 con 0x55cfb47ec8e0 2024-09-10T21:31:21.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.068+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v100'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.068+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f3f50 con 0x55cfb47ec8e0 2024-09-10T21:31:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.072+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1044002040 con 0x55cfb47ec8e0 2024-09-10T21:31:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.072+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55cfb47f45d0 0x55cfb47f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.072+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f4f30 con 0x55cfb47f45d0 2024-09-10T21:31:21.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.072+0000 7f1043e00640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55cfb47f45d0 0x55cfb47f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.072+0000 7f1043e00640 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55cfb47f45d0 0x55cfb47f49f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.076+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v100'17 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47f45d0 2024-09-10T21:31:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.076+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f4f30 con 0x55cfb47f45d0 2024-09-10T21:31:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.076+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v100'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47f45d0 2024-09-10T21:31:21.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.076+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f4f30 con 0x55cfb47f45d0 2024-09-10T21:31:21.080 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:21.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.076+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v100'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47f45d0 2024-09-10T21:31:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.076+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5370 con 0x55cfb47f45d0 2024-09-10T21:31:21.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.080+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v100'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47f45d0 2024-09-10T21:31:21.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.080+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5370 con 0x55cfb47f45d0 2024-09-10T21:31:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.080+0000 7f1043e00640 1 -- 172.21.15.97:0/1533932266 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v100'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f10340014c0 con 0x55cfb47f45d0 2024-09-10T21:31:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.080+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.084+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(114 100.00000000.inode [create] v100'45 uv37 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.084+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.088+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(115 100.00000000.inode [delete] v100'46 uv46 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.088+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.090 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:21.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.088+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(116 100.00000000.inode [create] v100'47 uv47 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.088+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.092+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v100'48 uv48 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.092+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.092+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v100'49 uv49 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.092+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.092+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(119 1.00000000 [create] v100'50 uv44 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.092+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.096+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(120 1.00000000 [delete] v100'51 uv51 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.096+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.096+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(121 1.00000000 [create] v100'52 uv52 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.100 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:21.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.096+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.096+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v100'53 uv53 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.096+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.100+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v100'54 uv54 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.100+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5320 con 0x55cfb47f0d10 2024-09-10T21:31:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.100+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v100'55 uv55 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.100+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55cfb47f5730 con 0x55cfb47f0d10 2024-09-10T21:31:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104a800640 1 -- 172.21.15.97:0/1533932266 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v100'56 uv56 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f103c0c7030 con 0x55cfb47f0d10 2024-09-10T21:31:21.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55cfb47e8430 msgr2=0x55cfb47ea820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55cfb47e8430 0x55cfb47ea820 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cfb47f0d10 msgr2=0x55cfb47f3100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55cfb47f0d10 0x55cfb47f3100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55cfb47f45d0 msgr2=0x55cfb47f49f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55cfb47f45d0 0x55cfb47f49f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cfb47ec8e0 msgr2=0x55cfb47ef740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55cfb47ec8e0 0x55cfb47ef740 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55cfb47e3a10 msgr2=0x55cfb47e5e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55cfb47e3a10 0x55cfb47e5e00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 reap_dead start 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104b200640 1 -- 172.21.15.97:0/1533932266 reap_dead start 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55cfb47283b0 msgr2=0x55cfb4728790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55cfb47283b0 0x55cfb4728790 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1028070fc0 msgr2=0x7f1028073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1028070fc0 0x7f1028073480 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1034000f70 tx=0x7f1034003520 comp rx=0 tx=0).stop 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 msgr2=0x55cfb471e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb471e5f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f103c018df0 tx=0x7f103c0ada20 comp rx=0 tx=0).stop 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 shutdown_connections 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55cfb47283b0 0x55cfb4728790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1028070fc0 0x7f1028073480 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 --2- 172.21.15.97:0/1533932266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfb4720cc0 0x55cfb471e5f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 >> 172.21.15.97:0/1533932266 conn(0x55cfb46b00d0 msgr2=0x55cfb473cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 shutdown_connections 2024-09-10T21:31:21.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.104+0000 7f104d6da080 1 -- 172.21.15.97:0/1533932266 wait complete. 2024-09-10T21:31:21.111 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2024-09-10T21:31:21.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 Processor -- start 2024-09-10T21:31:21.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- start start 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31c970aa0 0x55c31ca430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca29ca0 0x55c31ca49260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31ca2a080 0x55c31ca29030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8bea00 con 0x55c31c970aa0 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c31c91cf40 con 0x55c31ca29ca0 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c31ca502d0 con 0x55c31ca2a080 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31c970aa0 0x55c31ca430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31c970aa0 0x55c31ca430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33970/0 (socket says 172.21.15.97:33970) 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 -- 172.21.15.97:0/3621174493 learned_addr learned my addr 172.21.15.97:0/3621174493 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce0be00640 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca29ca0 0x55c31ca49260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce12200640 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31ca2a080 0x55c31ca29030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 -- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31ca2a080 msgr2=0x55c31ca29030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31ca2a080 0x55c31ca29030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 -- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca29ca0 msgr2=0x55c31ca49260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca29ca0 0x55c31ca49260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 -- 172.21.15.97:0/3621174493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31c897c10 con 0x55c31c970aa0 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce12200640 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31ca2a080 0x55c31ca29030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce11800640 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31c970aa0 0x55c31ca430a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fce04004ea0 tx=0x7fce04017f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c699795bf18f8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce0b400640 1 -- 172.21.15.97:0/3621174493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040ab630 con 0x55c31c970aa0 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce0b400640 1 -- 172.21.15.97:0/3621174493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce040bb070 con 0x55c31c970aa0 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce0b400640 1 -- 172.21.15.97:0/3621174493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040abd30 con 0x55c31c970aa0 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- 172.21.15.97:0/3621174493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31c970aa0 msgr2=0x55c31ca430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31c970aa0 0x55c31ca430a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fce04004ea0 tx=0x7fce04017f10 comp rx=0 tx=0).stop 2024-09-10T21:31:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- 172.21.15.97:0/3621174493 shutdown_connections 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31ca2a080 0x55c31ca29030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca29ca0 0x55c31ca49260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 --2- 172.21.15.97:0/3621174493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31c970aa0 0x55c31ca430a0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- 172.21.15.97:0/3621174493 >> 172.21.15.97:0/3621174493 conn(0x55c31c96f140 msgr2=0x55c31c96e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- 172.21.15.97:0/3621174493 shutdown_connections 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 -- 172.21.15.97:0/3621174493 wait complete. 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.196+0000 7fce14667080 1 Processor -- start 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 -- start start 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31c970aa0 0x55c31ca4aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31ca29ca0 0x55c31ca27220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca2a080 0x55c31ca27760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c31ca4d6b0 con 0x55c31ca29ca0 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c31ca27ca0 con 0x55c31ca2a080 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c31c99a1f0 con 0x55c31c970aa0 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31ca29ca0 0x55c31ca27220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31ca29ca0 0x55c31ca27220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33976/0 (socket says 172.21.15.97:33976) 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 -- 172.21.15.97:0/4142982549 learned_addr learned my addr 172.21.15.97:0/4142982549 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce11800640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31c970aa0 0x55c31ca4aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce12200640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca2a080 0x55c31ca27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 -- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31c970aa0 msgr2=0x55c31ca4aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c31c970aa0 0x55c31ca4aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 -- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca2a080 msgr2=0x55c31ca27760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c31ca2a080 0x55c31ca27760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 -- 172.21.15.97:0/4142982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31ca27ef0 con 0x55c31ca29ca0 2024-09-10T21:31:21.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0be00640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31ca29ca0 0x55c31ca27220 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00a760 tx=0x7fcdfc00ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0a000640 1 -- 172.21.15.97:0/4142982549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc01c070 con 0x55c31ca29ca0 2024-09-10T21:31:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0a000640 1 -- 172.21.15.97:0/4142982549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdfc008db0 con 0x55c31ca29ca0 2024-09-10T21:31:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce0a000640 1 -- 172.21.15.97:0/4142982549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0176e0 con 0x55c31ca29ca0 2024-09-10T21:31:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c31ca28190 con 0x55c31ca29ca0 2024-09-10T21:31:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.200+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31ca286d0 con 0x55c31ca29ca0 2024-09-10T21:31:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce0a000640 1 -- 172.21.15.97:0/4142982549 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcdfc017880 con 0x55c31ca29ca0 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce0a000640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcdf0070ff0 0x7fcdf00734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce0a000640 1 -- 172.21.15.97:0/4142982549 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7fcdfc092910 con 0x55c31ca29ca0 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce11800640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcdf0070ff0 0x7fcdf00734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c31ca26af0 0x55c31ca3df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x55c31ca3e460 con 0x55c31ca26af0 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce12200640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c31ca26af0 0x55c31ca3df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce11800640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcdf0070ff0 0x7fcdf00734b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fce040ba000 tx=0x7fce04004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce12200640 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c31ca26af0 0x55c31ca3df20 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce12200640 1 -- 172.21.15.97:0/4142982549 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000000 [delete] v100'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fce0c002930 con 0x55c31ca26af0 2024-09-10T21:31:21.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c31ca26af0 msgr2=0x55c31ca3df20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c31ca26af0 0x55c31ca3df20 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcdf0070ff0 msgr2=0x7fcdf00734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcdf0070ff0 0x7fcdf00734b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fce040ba000 tx=0x7fce04004890 comp rx=0 tx=0).stop 2024-09-10T21:31:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31ca29ca0 msgr2=0x55c31ca27220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.204+0000 7fce14667080 1 --2- 172.21.15.97:0/4142982549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c31ca29ca0 0x55c31ca27220 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00a760 tx=0x7fcdfc00ac20 comp rx=0 tx=0).stop 2024-09-10T21:31:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.208+0000 7fce12200640 1 -- 172.21.15.97:0/4142982549 reap_dead start 2024-09-10T21:31:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.208+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 shutdown_connections 2024-09-10T21:31:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.208+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 >> 172.21.15.97:0/4142982549 conn(0x55c31c96f140 msgr2=0x55c31ca46fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.208+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 shutdown_connections 2024-09-10T21:31:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.208+0000 7fce14667080 1 -- 172.21.15.97:0/4142982549 wait complete. 2024-09-10T21:31:21.214 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 -- 172.21.15.97:0/613939677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb95015efc0 msgr2=0x7fb9500cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 --2- 172.21.15.97:0/613939677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb95015efc0 0x7fb9500cf840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb944009870 tx=0x7fb94401c860 comp rx=0 tx=0).stop 2024-09-10T21:31:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 -- 172.21.15.97:0/613939677 shutdown_connections 2024-09-10T21:31:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 --2- 172.21.15.97:0/613939677 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb95015efc0 0x7fb9500cf840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 -- 172.21.15.97:0/613939677 >> 172.21.15.97:0/613939677 conn(0x7fb9500d1a60 msgr2=0x7fb9500d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 -- 172.21.15.97:0/613939677 shutdown_connections 2024-09-10T21:31:21.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 -- 172.21.15.97:0/613939677 wait complete. 2024-09-10T21:31:21.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 Processor -- start 2024-09-10T21:31:21.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.356+0000 7fb956800640 1 -- start start 2024-09-10T21:31:21.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9500cdcb0 0x7fb9501fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb9500d0490 0x7fb950163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95015efc0 0x7fb950164040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb950169c00 con 0x7fb95015efc0 2024-09-10T21:31:21.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb950169a80 con 0x7fb9500cdcb0 2024-09-10T21:31:21.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb950169d80 con 0x7fb9500d0490 2024-09-10T21:31:21.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95015efc0 0x7fb950164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9500cdcb0 0x7fb9501fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95015efc0 0x7fb950164040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33990/0 (socket says 172.21.15.97:33990) 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 -- 172.21.15.97:0/1867076229 learned_addr learned my addr 172.21.15.97:0/1867076229 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb954a00640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb9500d0490 0x7fb950163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 -- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb9500d0490 msgr2=0x7fb950163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb9500d0490 0x7fb950163b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 -- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9500cdcb0 msgr2=0x7fb9501fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9500cdcb0 0x7fb9501fa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 -- 172.21.15.97:0/1867076229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9501668b0 con 0x7fb95015efc0 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955400640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9500cdcb0 0x7fb9501fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:21.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb955e00640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95015efc0 0x7fb950164040 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb94800cb70 tx=0x7fb94800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb94d600640 1 -- 172.21.15.97:0/1867076229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948016020 con 0x7fb95015efc0 2024-09-10T21:31:21.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb94d600640 1 -- 172.21.15.97:0/1867076229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb94800ab40 con 0x7fb95015efc0 2024-09-10T21:31:21.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb950166b90 con 0x7fb95015efc0 2024-09-10T21:31:21.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb94d600640 1 -- 172.21.15.97:0/1867076229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948012420 con 0x7fb95015efc0 2024-09-10T21:31:21.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9500b1200 con 0x7fb95015efc0 2024-09-10T21:31:21.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9500ae930 con 0x7fb95015efc0 2024-09-10T21:31:21.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.360+0000 7fb94d600640 1 -- 172.21.15.97:0/1867076229 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb9480125c0 con 0x7fb95015efc0 2024-09-10T21:31:21.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.364+0000 7fb94d600640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb9300725e0 0x7fb930074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.364+0000 7fb94d600640 1 -- 172.21.15.97:0/1867076229 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6619+0+0 (secure 0 0 0) 0x7fb948093ae0 con 0x7fb95015efc0 2024-09-10T21:31:21.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.364+0000 7fb955400640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb9300725e0 0x7fb930074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.364+0000 7fb955400640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb9300725e0 0x7fb930074aa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb9440af000 tx=0x7fb94401b0a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.368+0000 7fb94d600640 1 -- 172.21.15.97:0/1867076229 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb948020020 con 0x7fb95015efc0 2024-09-10T21:31:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.564+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb95015e710 con 0x7fb95015efc0 2024-09-10T21:31:21.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.608+0000 7fb94d600640 1 -- 172.21.15.97:0/1867076229 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v110) ==== 162+0+0 (secure 0 0 0) 0x7fb94805e740 con 0x7fb95015efc0 2024-09-10T21:31:21.612 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:21.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb9300725e0 msgr2=0x7fb930074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb9300725e0 0x7fb930074aa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb9440af000 tx=0x7fb94401b0a0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95015efc0 msgr2=0x7fb950164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95015efc0 0x7fb950164040 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb94800cb70 tx=0x7fb94800cc60 comp rx=0 tx=0).stop 2024-09-10T21:31:21.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 shutdown_connections 2024-09-10T21:31:21.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb9300725e0 0x7fb930074aa0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb95015efc0 0x7fb950164040 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb9500d0490 0x7fb950163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 --2- 172.21.15.97:0/1867076229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9500cdcb0 0x7fb9501fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 >> 172.21.15.97:0/1867076229 conn(0x7fb9500d1a60 msgr2=0x7fb9500cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:21.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 shutdown_connections 2024-09-10T21:31:21.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.612+0000 7fb956800640 1 -- 172.21.15.97:0/1867076229 wait complete. 2024-09-10T21:31:21.618 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:31:21.616+0000 7f9331800640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-10T21:31:21.618 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:21.616+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:21.618 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:21.616+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:31:21.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:21.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- 172.21.15.97:0/3129709517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac070460 msgr2=0x7f82ac0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 --2- 172.21.15.97:0/3129709517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac070460 0x7f82ac0b2a80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009bc0 tx=0x7f82a401cba0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- 172.21.15.97:0/3129709517 shutdown_connections 2024-09-10T21:31:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 --2- 172.21.15.97:0/3129709517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac070460 0x7f82ac0b2a80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- 172.21.15.97:0/3129709517 >> 172.21.15.97:0/3129709517 conn(0x7f82ac0b1780 msgr2=0x7f82ac0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- 172.21.15.97:0/3129709517 shutdown_connections 2024-09-10T21:31:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- 172.21.15.97:0/3129709517 wait complete. 2024-09-10T21:31:21.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 Processor -- start 2024-09-10T21:31:21.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- start start 2024-09-10T21:31:21.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82ac070460 0x7f82ac143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac143f90 0x7f82ac1420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82ac144690 0x7f82ac13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac168e10 con 0x7f82ac143f90 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac168c90 con 0x7f82ac144690 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82b1600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac168f90 con 0x7f82ac070460 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82aa000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac143f90 0x7f82ac1420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82aa000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac143f90 0x7f82ac1420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34008/0 (socket says 172.21.15.97:34008) 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.776+0000 7f82aa000640 1 -- 172.21.15.97:0/1889703293 learned_addr learned my addr 172.21.15.97:0/1889703293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aaa00640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82ac070460 0x7f82ac143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82ab400640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82ac144690 0x7f82ac13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aa000640 1 -- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82ac070460 msgr2=0x7f82ac143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aa000640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82ac070460 0x7f82ac143140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aa000640 1 -- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82ac144690 msgr2=0x7f82ac13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aa000640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82ac144690 0x7f82ac13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aa000640 1 -- 172.21.15.97:0/1889703293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac134260 con 0x7f82ac143f90 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aaa00640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82ac070460 0x7f82ac143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82aa000640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac143f90 0x7f82ac1420e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f829800c430 tx=0x7f829800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82a2a00640 1 -- 172.21.15.97:0/1889703293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298017070 con 0x7f82ac143f90 2024-09-10T21:31:21.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82a2a00640 1 -- 172.21.15.97:0/1889703293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82980053c0 con 0x7f82ac143f90 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac142620 con 0x7f82ac143f90 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82a2a00640 1 -- 172.21.15.97:0/1889703293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298003860 con 0x7f82ac143f90 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1347e0 con 0x7f82ac143f90 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac0b2a80 con 0x7f82ac143f90 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82a2a00640 1 -- 172.21.15.97:0/1889703293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8298003a00 con 0x7f82ac143f90 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.780+0000 7f82a2a00640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f828c0725e0 0x7f828c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.784+0000 7f82a2a00640 1 -- 172.21.15.97:0/1889703293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7f8298092c70 con 0x7f82ac143f90 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.784+0000 7f82aaa00640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f828c0725e0 0x7f828c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:21.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.784+0000 7f82aaa00640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f828c0725e0 0x7f828c074aa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f829c000c10 tx=0x7f829c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:21.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.788+0000 7f82a2a00640 1 -- 172.21.15.97:0/1889703293 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f829805d980 con 0x7f82ac143f90 2024-09-10T21:31:21.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.988+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82ac06fec0 con 0x7f82ac143f90 2024-09-10T21:31:21.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.988+0000 7f82a2a00640 1 -- 172.21.15.97:0/1889703293 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+3569 (secure 0 0 0) 0x7f8298092020 con 0x7f82ac143f90 2024-09-10T21:31:21.992 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:21.992 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"btime":"2024-09-10T21:31:21:621186+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:21.621131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:21.993 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 111 2024-09-10T21:31:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f828c0725e0 msgr2=0x7f828c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f828c0725e0 0x7f828c074aa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f829c000c10 tx=0x7f829c009040 comp rx=0 tx=0).stop 2024-09-10T21:31:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac143f90 msgr2=0x7f82ac1420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac143f90 0x7f82ac1420e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f829800c430 tx=0x7f829800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 shutdown_connections 2024-09-10T21:31:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f828c0725e0 0x7f828c074aa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82ac144690 0x7f82ac13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82ac143f90 0x7f82ac1420e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 --2- 172.21.15.97:0/1889703293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f82ac070460 0x7f82ac143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 >> 172.21.15.97:0/1889703293 conn(0x7f82ac0b1780 msgr2=0x7f82ac13d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 shutdown_connections 2024-09-10T21:31:21.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:21.992+0000 7f82b1600640 1 -- 172.21.15.97:0/1889703293 wait complete. 2024-09-10T21:31:22.013 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:22.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:22.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.152+0000 7f7023e00640 1 -- 172.21.15.97:0/499997897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004003fe0 msgr2=0x7f7004004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.152+0000 7f7023e00640 1 --2- 172.21.15.97:0/499997897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004003fe0 0x7f7004004810 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7008009b90 tx=0x7f700801d270 comp rx=0 tx=0).stop 2024-09-10T21:31:22.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.152+0000 7f7021e00640 1 -- 172.21.15.97:0/499997897 reap_dead start 2024-09-10T21:31:22.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.152+0000 7f7023e00640 1 -- 172.21.15.97:0/499997897 shutdown_connections 2024-09-10T21:31:22.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.152+0000 7f7023e00640 1 -- 172.21.15.97:0/499997897 >> 172.21.15.97:0/499997897 conn(0x7f701c0b2790 msgr2=0x7f701c0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:22.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.152+0000 7f7023e00640 1 -- 172.21.15.97:0/499997897 shutdown_connections 2024-09-10T21:31:22.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.152+0000 7f7023e00640 1 -- 172.21.15.97:0/499997897 wait complete. 2024-09-10T21:31:22.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 Processor -- start 2024-09-10T21:31:22.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 -- start start 2024-09-10T21:31:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7004003fe0 0x7f701c150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f701c1f7870 0x7f701c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c1557f0 0x7f701c155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f701c161820 con 0x7f701c1557f0 2024-09-10T21:31:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f701c15f1e0 con 0x7f7004003fe0 2024-09-10T21:31:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f701c1619a0 con 0x7f701c1f7870 2024-09-10T21:31:22.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c1557f0 0x7f701c155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c1557f0 0x7f701c155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34030/0 (socket says 172.21.15.97:34030) 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 -- 172.21.15.97:0/2643659453 learned_addr learned my addr 172.21.15.97:0/2643659453 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021400640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7004003fe0 0x7f701c150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7020a00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f701c1f7870 0x7f701c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 -- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f701c1f7870 msgr2=0x7f701c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f701c1f7870 0x7f701c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 -- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7004003fe0 msgr2=0x7f701c150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7004003fe0 0x7f701c150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 -- 172.21.15.97:0/2643659453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c1f7d70 con 0x7f701c1557f0 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021400640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7004003fe0 0x7f701c150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7020a00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f701c1f7870 0x7f701c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7021e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c1557f0 0x7f701c155ca0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f701c070cb0 tx=0x7f7014006410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7019600640 1 -- 172.21.15.97:0/2643659453 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014003b80 con 0x7f701c1557f0 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7019600640 1 -- 172.21.15.97:0/2643659453 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7014003d20 con 0x7f701c1557f0 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7019600640 1 -- 172.21.15.97:0/2643659453 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140052d0 con 0x7f701c1557f0 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c1f8050 con 0x7f701c1557f0 2024-09-10T21:31:22.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.156+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1f8610 con 0x7f701c1557f0 2024-09-10T21:31:22.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.160+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c070130 con 0x7f701c1557f0 2024-09-10T21:31:22.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.160+0000 7f7019600640 1 -- 172.21.15.97:0/2643659453 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7014010040 con 0x7f701c1557f0 2024-09-10T21:31:22.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.160+0000 7f7019600640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ffc072610 0x7f6ffc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.160+0000 7f7019600640 1 -- 172.21.15.97:0/2643659453 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7f70140968b0 con 0x7f701c1557f0 2024-09-10T21:31:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.160+0000 7f7021400640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ffc072610 0x7f6ffc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.164+0000 7f7021400640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ffc072610 0x7f6ffc074ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7010009ce0 tx=0x7f70100091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:22.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.164+0000 7f7019600640 1 -- 172.21.15.97:0/2643659453 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f701c15e230 con 0x7f701c1557f0 2024-09-10T21:31:22.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.364+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f701c0b4480 con 0x7f701c1557f0 2024-09-10T21:31:22.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.364+0000 7f7019600640 1 -- 172.21.15.97:0/2643659453 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+3569 (secure 0 0 0) 0x7f7014061b00 con 0x7f701c1557f0 2024-09-10T21:31:22.369 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:22.369 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"btime":"2024-09-10T21:31:21:621186+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:21.621131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:22.369 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 111 2024-09-10T21:31:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ffc072610 msgr2=0x7f6ffc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ffc072610 0x7f6ffc074ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7010009ce0 tx=0x7f70100091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c1557f0 msgr2=0x7f701c155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c1557f0 0x7f701c155ca0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f701c070cb0 tx=0x7f7014006410 comp rx=0 tx=0).stop 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 shutdown_connections 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ffc072610 0x7f6ffc074ad0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c1557f0 0x7f701c155ca0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f701c1f7870 0x7f701c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 --2- 172.21.15.97:0/2643659453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7004003fe0 0x7f701c150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 >> 172.21.15.97:0/2643659453 conn(0x7f701c0b2790 msgr2=0x7f701c0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 shutdown_connections 2024-09-10T21:31:22.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.368+0000 7f7023e00640 1 -- 172.21.15.97:0/2643659453 wait complete. 2024-09-10T21:31:22.390 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:22.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:22.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- 172.21.15.97:0/3338784417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0cec10 msgr2=0x7f201c0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 --2- 172.21.15.97:0/3338784417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0cec10 0x7f201c0cfd40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f201c0735a0 tx=0x7f200801f0b0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- 172.21.15.97:0/3338784417 shutdown_connections 2024-09-10T21:31:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 --2- 172.21.15.97:0/3338784417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0cec10 0x7f201c0cfd40 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 --2- 172.21.15.97:0/3338784417 >> v2:172.21.15.194:3300/0 conn(0x7f201c0ce830 0x7f201c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- 172.21.15.97:0/3338784417 >> 172.21.15.97:0/3338784417 conn(0x7f201c0b37c0 msgr2=0x7f201c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- 172.21.15.97:0/3338784417 shutdown_connections 2024-09-10T21:31:22.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- 172.21.15.97:0/3338784417 wait complete. 2024-09-10T21:31:22.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 Processor -- start 2024-09-10T21:31:22.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- start start 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f201c0ce830 0x7f201c0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f201c0cec10 0x7f201c157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0d0280 0x7f201c1f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f201c0b4450 con 0x7f201c0d0280 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f201c0cde60 con 0x7f201c0ce830 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2023e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f201c0ce1a0 con 0x7f201c0cec10 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2021e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0d0280 0x7f201c1f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2021e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0d0280 0x7f201c1f7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34048/0 (socket says 172.21.15.97:34048) 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2021e00640 1 -- 172.21.15.97:0/2270654774 learned_addr learned my addr 172.21.15.97:0/2270654774 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2021400640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f201c0ce830 0x7f201c0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.532+0000 7f2020a00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f201c0cec10 0x7f201c157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2021e00640 1 -- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f201c0cec10 msgr2=0x7f201c157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2021e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f201c0cec10 0x7f201c157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2021e00640 1 -- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f201c0ce830 msgr2=0x7f201c0c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2021e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f201c0ce830 0x7f201c0c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2021e00640 1 -- 172.21.15.97:0/2270654774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c0cb190 con 0x7f201c0d0280 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2021e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0d0280 0x7f201c1f7e80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f201400c730 tx=0x7f201400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2019600640 1 -- 172.21.15.97:0/2270654774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014003ba0 con 0x7f201c0d0280 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2019600640 1 -- 172.21.15.97:0/2270654774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2014003d60 con 0x7f201c0d0280 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2019600640 1 -- 172.21.15.97:0/2270654774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201400aab0 con 0x7f201c0d0280 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c0cb470 con 0x7f201c0d0280 2024-09-10T21:31:22.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c0cba30 con 0x7f201c0d0280 2024-09-10T21:31:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201c0b2650 con 0x7f201c0d0280 2024-09-10T21:31:22.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2019600640 1 -- 172.21.15.97:0/2270654774 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f201400ac50 con 0x7f201c0d0280 2024-09-10T21:31:22.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2019600640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ffc0725e0 0x7f1ffc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:22.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.536+0000 7f2019600640 1 -- 172.21.15.97:0/2270654774 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7f2014093360 con 0x7f201c0d0280 2024-09-10T21:31:22.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.540+0000 7f2021400640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ffc0725e0 0x7f1ffc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:22.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.540+0000 7f2021400640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ffc0725e0 0x7f1ffc074aa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f201c0728f0 tx=0x7f20100091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:22.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.540+0000 7f2019600640 1 -- 172.21.15.97:0/2270654774 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f201405df60 con 0x7f201c0d0280 2024-09-10T21:31:22.552 INFO:tasks.ceph.mds.c.smithi097.stderr: -40> 2024-09-10T21:31:17.796+0000 7fe18c800640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T21:31:22.552 INFO:tasks.ceph.mds.c.smithi097.stderr: -38> 2024-09-10T21:31:17.796+0000 7fe18c800640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:31:22.590 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:31:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.744+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f201c071570 con 0x7f201c0d0280 2024-09-10T21:31:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.744+0000 7f2019600640 1 -- 172.21.15.97:0/2270654774 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+3569 (secure 0 0 0) 0x7f2014062990 con 0x7f201c0d0280 2024-09-10T21:31:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:22.748 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"btime":"2024-09-10T21:31:21:621186+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:21.621131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:22.748 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 111 2024-09-10T21:31:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ffc0725e0 msgr2=0x7f1ffc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ffc0725e0 0x7f1ffc074aa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f201c0728f0 tx=0x7f20100091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0d0280 msgr2=0x7f201c1f7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:22.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0d0280 0x7f201c1f7e80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f201400c730 tx=0x7f201400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 shutdown_connections 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ffc0725e0 0x7f1ffc074aa0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0d0280 0x7f201c1f7e80 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f201c0cec10 0x7f201c157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 --2- 172.21.15.97:0/2270654774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f201c0ce830 0x7f201c0c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 >> 172.21.15.97:0/2270654774 conn(0x7f201c0b37c0 msgr2=0x7f201c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 shutdown_connections 2024-09-10T21:31:22.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:22.748+0000 7f2023e00640 1 -- 172.21.15.97:0/2270654774 wait complete. 2024-09-10T21:31:22.770 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2024-09-10T21:31:22.770 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2024-09-10T21:31:22.770 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:22.770 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:22.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:31:22.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:22.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:22.779 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:22.830 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:31:22.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:31:22.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.012+0000 7fbd73600640 1 -- 172.21.15.97:0/4224036798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50003a10 msgr2=0x7fbd50003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.012+0000 7fbd73600640 1 --2- 172.21.15.97:0/4224036798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50003a10 0x7fbd50003df0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009ee0 tx=0x7fbd5c01d5f0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.012+0000 7fbd73600640 1 -- 172.21.15.97:0/4224036798 shutdown_connections 2024-09-10T21:31:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.012+0000 7fbd73600640 1 --2- 172.21.15.97:0/4224036798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd50003a10 0x7fbd50003df0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.012+0000 7fbd73600640 1 -- 172.21.15.97:0/4224036798 >> 172.21.15.97:0/4224036798 conn(0x7fbd6c0b1780 msgr2=0x7fbd6c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.012+0000 7fbd73600640 1 -- 172.21.15.97:0/4224036798 shutdown_connections 2024-09-10T21:31:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.012+0000 7fbd73600640 1 -- 172.21.15.97:0/4224036798 wait complete. 2024-09-10T21:31:23.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 Processor -- start 2024-09-10T21:31:23.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 -- start start 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd50003a10 0x7fbd6c0ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd6c13e160 0x7fbd6c0aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd6c13e5e0 0x7fbd6c0aa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c0b2e10 con 0x7fbd6c13e160 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c0b2c90 con 0x7fbd6c13e5e0 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6c070160 con 0x7fbd50003a10 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd6c13e160 0x7fbd6c0aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd6c13e160 0x7fbd6c0aa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34082/0 (socket says 172.21.15.97:34082) 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 -- 172.21.15.97:0/1394152008 learned_addr learned my addr 172.21.15.97:0/1394152008 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd71600640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd6c13e5e0 0x7fbd6c0aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd70c00640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd50003a10 0x7fbd6c0ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 -- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd50003a10 msgr2=0x7fbd6c0ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd50003a10 0x7fbd6c0ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 -- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd6c13e5e0 msgr2=0x7fbd6c0aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd6c13e5e0 0x7fbd6c0aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 -- 172.21.15.97:0/1394152008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c1f3130 con 0x7fbd6c13e160 2024-09-10T21:31:23.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd6be00640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd6c13e160 0x7fbd6c0aa100 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800e390 tx=0x7fbd5800e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:23.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd68c00640 1 -- 172.21.15.97:0/1394152008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58016070 con 0x7fbd6c13e160 2024-09-10T21:31:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd68c00640 1 -- 172.21.15.97:0/1394152008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd58003600 con 0x7fbd6c13e160 2024-09-10T21:31:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c0a9620 con 0x7fbd6c13e160 2024-09-10T21:31:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd68c00640 1 -- 172.21.15.97:0/1394152008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58003a50 con 0x7fbd6c13e160 2024-09-10T21:31:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c0a9be0 con 0x7fbd6c13e160 2024-09-10T21:31:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.016+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6c070630 con 0x7fbd6c13e160 2024-09-10T21:31:23.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.020+0000 7fbd68c00640 1 -- 172.21.15.97:0/1394152008 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbd58003bf0 con 0x7fbd6c13e160 2024-09-10T21:31:23.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.020+0000 7fbd68c00640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd4c0725e0 0x7fbd4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:23.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.020+0000 7fbd68c00640 1 -- 172.21.15.97:0/1394152008 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7fbd58097810 con 0x7fbd6c13e160 2024-09-10T21:31:23.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.020+0000 7fbd70c00640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd4c0725e0 0x7fbd4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:23.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.020+0000 7fbd70c00640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd4c0725e0 0x7fbd4c074aa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0af000 tx=0x7fbd5c01a390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:23.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.024+0000 7fbd68c00640 1 -- 172.21.15.97:0/1394152008 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd58062410 con 0x7fbd6c13e160 2024-09-10T21:31:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.224+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbd6c0a9db0 con 0x7fbd6c13e160 2024-09-10T21:31:23.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.832+0000 7fbd68c00640 1 -- 172.21.15.97:0/1394152008 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7fbd58066e40 con 0x7fbd6c13e160 2024-09-10T21:31:23.835 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:23.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.832+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd4c0725e0 msgr2=0x7fbd4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:23.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.832+0000 7fbd73600640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd4c0725e0 0x7fbd4c074aa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0af000 tx=0x7fbd5c01a390 comp rx=0 tx=0).stop 2024-09-10T21:31:23.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.832+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd6c13e160 msgr2=0x7fbd6c0aa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.832+0000 7fbd73600640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd6c13e160 0x7fbd6c0aa100 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800e390 tx=0x7fbd5800e850 comp rx=0 tx=0).stop 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 shutdown_connections 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd4c0725e0 0x7fbd4c074aa0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd6c13e5e0 0x7fbd6c0aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd6c13e160 0x7fbd6c0aa100 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 --2- 172.21.15.97:0/1394152008 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd50003a10 0x7fbd6c0ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 >> 172.21.15.97:0/1394152008 conn(0x7fbd6c0b1780 msgr2=0x7fbd6c15b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 shutdown_connections 2024-09-10T21:31:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:23.836+0000 7fbd73600640 1 -- 172.21.15.97:0/1394152008 wait complete. 2024-09-10T21:31:23.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:24.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 -- 172.21.15.97:0/3989114050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c94003a40 msgr2=0x7f4c94003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 --2- 172.21.15.97:0/3989114050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c94003a40 0x7f4c94003e60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0001660 tx=0x7f4ca001c760 comp rx=0 tx=0).stop 2024-09-10T21:31:24.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cafe00640 1 -- 172.21.15.97:0/3989114050 reap_dead start 2024-09-10T21:31:24.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 -- 172.21.15.97:0/3989114050 shutdown_connections 2024-09-10T21:31:24.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 -- 172.21.15.97:0/3989114050 >> 172.21.15.97:0/3989114050 conn(0x7f4cb00b37c0 msgr2=0x7f4cb00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:24.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 -- 172.21.15.97:0/3989114050 shutdown_connections 2024-09-10T21:31:24.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 -- 172.21.15.97:0/3989114050 wait complete. 2024-09-10T21:31:24.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 Processor -- start 2024-09-10T21:31:24.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.000+0000 7f4cb6200640 1 -- start start 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c94003a40 0x7f4cb015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb01470a0 0x7f4cb00709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0070f00 0x7f4cb01f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb00cddd0 con 0x7f4cb01470a0 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb00cdc50 con 0x7f4cb0070f00 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb00ce1a0 con 0x7f4c94003a40 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb01470a0 0x7f4cb00709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb01470a0 0x7f4cb00709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34088/0 (socket says 172.21.15.97:34088) 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 -- 172.21.15.97:0/20403104 learned_addr learned my addr 172.21.15.97:0/20403104 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4caf400640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c94003a40 0x7f4cb015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cafe00640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0070f00 0x7f4cb01f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 -- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c94003a40 msgr2=0x7f4cb015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c94003a40 0x7f4cb015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 -- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0070f00 msgr2=0x7f4cb01f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0070f00 0x7f4cb01f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 -- 172.21.15.97:0/20403104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb0147520 con 0x7f4cb01470a0 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca6a00640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb01470a0 0x7f4cb00709c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800f400 tx=0x7f4c9800f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca7e00640 1 -- 172.21.15.97:0/20403104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9801a070 con 0x7f4cb01470a0 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca7e00640 1 -- 172.21.15.97:0/20403104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c98008520 con 0x7f4cb01470a0 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca7e00640 1 -- 172.21.15.97:0/20403104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98006980 con 0x7f4cb01470a0 2024-09-10T21:31:24.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb01f5a80 con 0x7f4cb01470a0 2024-09-10T21:31:24.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb015d1e0 con 0x7f4cb01470a0 2024-09-10T21:31:24.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb0071310 con 0x7f4cb01470a0 2024-09-10T21:31:24.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.004+0000 7f4ca7e00640 1 -- 172.21.15.97:0/20403104 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4c98006b20 con 0x7f4cb01470a0 2024-09-10T21:31:24.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.008+0000 7f4ca7e00640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4c8c0725e0 0x7f4c8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.008+0000 7f4ca7e00640 1 -- 172.21.15.97:0/20403104 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7f4c98095ab0 con 0x7f4cb01470a0 2024-09-10T21:31:24.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.008+0000 7f4caf400640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4c8c0725e0 0x7f4c8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.008+0000 7f4caf400640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4c8c0725e0 0x7f4c8c074aa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4c94004790 tx=0x7f4ca00b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.012+0000 7f4ca7e00640 1 -- 172.21.15.97:0/20403104 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4cb0071310 con 0x7f4cb01470a0 2024-09-10T21:31:24.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.240+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f4cb00cf800 con 0x7f4cb01470a0 2024-09-10T21:31:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.836+0000 7f4ca7e00640 1 -- 172.21.15.97:0/20403104 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v114) ==== 101+0+0 (secure 0 0 0) 0x7f4c98060750 con 0x7f4cb01470a0 2024-09-10T21:31:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4c8c0725e0 msgr2=0x7f4c8c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4c8c0725e0 0x7f4c8c074aa0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4c94004790 tx=0x7f4ca00b1040 comp rx=0 tx=0).stop 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb01470a0 msgr2=0x7f4cb00709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb01470a0 0x7f4cb00709c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800f400 tx=0x7f4c9800f8c0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 shutdown_connections 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4c8c0725e0 0x7f4c8c074aa0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cb0070f00 0x7f4cb01f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb01470a0 0x7f4cb00709c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 --2- 172.21.15.97:0/20403104 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c94003a40 0x7f4cb015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 >> 172.21.15.97:0/20403104 conn(0x7f4cb00b37c0 msgr2=0x7f4cb0159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 shutdown_connections 2024-09-10T21:31:24.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.840+0000 7f4cb6200640 1 -- 172.21.15.97:0/20403104 wait complete. 2024-09-10T21:31:24.863 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:24.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 Processor -- start 2024-09-10T21:31:24.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- start start 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 0x559db85f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db8623820 0x559db8626010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db86ac490 0x559db86c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559db8543a00 con 0x559db8623820 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559db85a1f40 con 0x559db86ac490 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559db86ae0a0 con 0x7f04ec002880 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db8623820 0x559db8626010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db8623820 0x559db8626010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34108/0 (socket says 172.21.15.97:34108) 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 -- 172.21.15.97:0/2671609341 learned_addr learned my addr 172.21.15.97:0/2671609341 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0508000640 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 0x559db85f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0508a00640 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db86ac490 0x559db86c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 -- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 msgr2=0x559db85f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 0x559db85f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 -- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db86ac490 msgr2=0x559db86c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db86ac490 0x559db86c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 -- 172.21.15.97:0/2671609341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db851cc10 con 0x559db8623820 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0507600640 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db8623820 0x559db8626010 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f04f401bca0 tx=0x7f04ec004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4502dc326d1aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0506200640 1 -- 172.21.15.97:0/2671609341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40a4530 con 0x559db8623820 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0506200640 1 -- 172.21.15.97:0/2671609341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f40a48e0 con 0x559db8623820 2024-09-10T21:31:24.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f0506200640 1 -- 172.21.15.97:0/2671609341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40a4b10 con 0x559db8623820 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- 172.21.15.97:0/2671609341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db8623820 msgr2=0x559db8626010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db8623820 0x559db8626010 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f04f401bca0 tx=0x7f04ec004130 comp rx=0 tx=0).stop 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- 172.21.15.97:0/2671609341 shutdown_connections 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db86ac490 0x559db86c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db8623820 0x559db8626010 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 --2- 172.21.15.97:0/2671609341 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 0x559db85f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- 172.21.15.97:0/2671609341 >> 172.21.15.97:0/2671609341 conn(0x559db863d0d0 msgr2=0x559db86abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- 172.21.15.97:0/2671609341 shutdown_connections 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.908+0000 7f050add4080 1 -- 172.21.15.97:0/2671609341 wait complete. 2024-09-10T21:31:24.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 Processor -- start 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 -- start start 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db8623820 0x559db86cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db86ac490 0x559db86cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 0x559db86ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559db85f77d0 con 0x559db86ac490 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559db86248c0 con 0x559db8623820 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559db86b0e40 con 0x7f04ec002880 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db86ac490 0x559db86cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db86ac490 0x559db86cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34112/0 (socket says 172.21.15.97:34112) 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 learned_addr learned my addr 172.21.15.97:0/2951404364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0508a00640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 0x559db86ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db8623820 0x559db86cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 msgr2=0x559db86ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f04ec002880 0x559db86ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db8623820 msgr2=0x559db86cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db8623820 0x559db86cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db8637b70 con 0x559db86ac490 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559db8623820 0x559db86cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0507600640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db86ac490 0x559db86cc810 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f04f4097250 tx=0x7f04f40a4530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0504e00640 1 -- 172.21.15.97:0/2951404364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40a48e0 con 0x559db86ac490 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0504e00640 1 -- 172.21.15.97:0/2951404364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f40bf650 con 0x559db86ac490 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f0504e00640 1 -- 172.21.15.97:0/2951404364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40bf8a0 con 0x559db86ac490 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559db8637e10 con 0x559db86ac490 2024-09-10T21:31:24.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.912+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db8638350 con 0x559db86ac490 2024-09-10T21:31:24.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f0504e00640 1 -- 172.21.15.97:0/2951404364 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04f40b5070 con 0x559db86ac490 2024-09-10T21:31:24.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f0504e00640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04e8071190 0x7f04e8073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f0504e00640 1 -- 172.21.15.97:0/2951404364 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7f04f412d560 con 0x559db86ac490 2024-09-10T21:31:24.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04e8071190 0x7f04e8073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.920 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:24.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559db863a240 0x559db863a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db863ab60 con 0x559db863a240 2024-09-10T21:31:24.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f0508a00640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559db863a240 0x559db863a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04e8071190 0x7f04e8073650 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c970 tx=0x7f04f800f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.916+0000 7f0508a00640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559db863a240 0x559db863a620 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.920+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v101'133 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.920+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db876ec20 con 0x559db863a240 2024-09-10T21:31:24.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.920+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v101'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.920+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876efc0 con 0x559db863a240 2024-09-10T21:31:24.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.920+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v101'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.925 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:24.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.920+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8770370 con 0x559db863a240 2024-09-10T21:31:24.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.924+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.924+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8770710 con 0x559db863a240 2024-09-10T21:31:24.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.924+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.924+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876f360 con 0x559db863a240 2024-09-10T21:31:24.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.928+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.928+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876f700 con 0x559db863a240 2024-09-10T21:31:24.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.928+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v101'139 uv113 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.928+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.928+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v101'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.928+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.932+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v101'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.934 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:24.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.932+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.932+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.932+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.932+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.932+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.936+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.936+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.936+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v101'145 uv120 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.936+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.940+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v101'146 uv146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.940+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.943 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.940+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v101'147 uv147 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.940+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.940+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v101'148 uv148 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.940+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.944+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v101'149 uv149 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.944+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876faa0 con 0x559db863a240 2024-09-10T21:31:24.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.944+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v101'150 uv150 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.944+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fe70 con 0x559db863a240 2024-09-10T21:31:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.944+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v101'151 uv151 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.944+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.948+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v101'152 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.948+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.948+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v101'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.948+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.953 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:24.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.948+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v101'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.948+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.952+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v101'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.952+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.952+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v101'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.952+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.952+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v101'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.952+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.956+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v101'158 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.956+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.956+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v101'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.956+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.961 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:24.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.956+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v101'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.956+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.960+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v101'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.960+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.960+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v101'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.960+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db863a240 2024-09-10T21:31:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.960+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v101'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db863a240 2024-09-10T21:31:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.964+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559db8770fc0 0x559db87733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.964+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db876fd20 con 0x559db8770fc0 2024-09-10T21:31:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.964+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559db8770fc0 0x559db87733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.964+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559db8770fc0 0x559db87733b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.964+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v101'109 uv91 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.964+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db87738f0 con 0x559db8770fc0 2024-09-10T21:31:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.968+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v101'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.968+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8773c90 con 0x559db8770fc0 2024-09-10T21:31:24.971 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.968+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v101'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.968+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87744e0 con 0x559db8770fc0 2024-09-10T21:31:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.968+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v101'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.968+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8774880 con 0x559db8770fc0 2024-09-10T21:31:24.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.972+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v101'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.972+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8774c20 con 0x559db8770fc0 2024-09-10T21:31:24.975 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.972+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v101'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.972+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8774fc0 con 0x559db8770fc0 2024-09-10T21:31:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.972+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v101'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.972+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v101'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000001 [create] v101'117 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000001 [delete] v101'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v101'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.981 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:24.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.976+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8779530 con 0x559db8770fc0 2024-09-10T21:31:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.980+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [write 0~16801] v101'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.980+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8779530 con 0x559db8770fc0 2024-09-10T21:31:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.980+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v101'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.980+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.980+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 606.00000000 [create] v101'122 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.980+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.984+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 606.00000000 [delete] v101'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.984+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.987 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.984+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v101'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.984+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8776370 con 0x559db8770fc0 2024-09-10T21:31:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.984+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v101'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.984+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8776370 con 0x559db8770fc0 2024-09-10T21:31:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v101'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8775360 con 0x559db8770fc0 2024-09-10T21:31:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v101'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db8770fc0 2024-09-10T21:31:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559db8775870 0x559db8777c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87781a0 con 0x559db8775870 2024-09-10T21:31:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f0507600640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559db8775870 0x559db8777c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:24.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f0507600640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559db8775870 0x559db8777c60 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:24.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(51 607.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:24.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.988+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8778540 con 0x559db8775870 2024-09-10T21:31:24.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.992+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(52 607.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:24.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.992+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87789f0 con 0x559db8775870 2024-09-10T21:31:24.995 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:24.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.992+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(53 607.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:24.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.992+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877a2a0 con 0x559db8775870 2024-09-10T21:31:24.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.992+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:24.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.992+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877a640 con 0x559db8775870 2024-09-10T21:31:24.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.996+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:24.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.996+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8779290 con 0x559db8775870 2024-09-10T21:31:24.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.996+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.996+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559db8779b90 0x559db877c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.996+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877cf30 con 0x559db8779b90 2024-09-10T21:31:25.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.996+0000 7f0508a00640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559db8779b90 0x559db877c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:24.996+0000 7f0508a00640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559db8779b90 0x559db877c9f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v101'23 uv22 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db877cf30 con 0x559db8779b90 2024-09-10T21:31:25.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v101'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877cfb0 con 0x559db8779b90 2024-09-10T21:31:25.003 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:25.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'25 uv25 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d3a0 con 0x559db8779b90 2024-09-10T21:31:25.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v101'26 uv26 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.000+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d3a0 con 0x559db8775870 2024-09-10T21:31:25.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.004+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(61 608.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.004+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.004+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(62 608.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.004+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.008 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:25.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.004+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(63 608.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.004+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877e750 con 0x559db8775870 2024-09-10T21:31:25.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877e750 con 0x559db8775870 2024-09-10T21:31:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(67 604.00000000 [create] v101'48 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.008+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.012+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(68 604.00000000 [delete] v101'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.012+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.015 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:25.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.012+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(69 604.00000000 [create] v101'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.012+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877e750 con 0x559db8779b90 2024-09-10T21:31:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.012+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v101'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.012+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877e750 con 0x559db8779b90 2024-09-10T21:31:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(73 500.00000000 [create] v101'54 uv47 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(74 500.00000000 [delete] v101'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.016+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.021 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:25.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.020+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(75 500.00000000 [create] v101'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.020+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.020+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v101'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.020+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8779b90 2024-09-10T21:31:25.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.020+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v101'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.020+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(78 mds_snaptable [create] v101'22 uv21 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(79 mds_snaptable [delete] v101'23 uv23 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(80 mds_snaptable [create] v101'24 uv24 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.028 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:25.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v101'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.024+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db877d740 con 0x559db8775870 2024-09-10T21:31:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.028+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v101'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db8775870 2024-09-10T21:31:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.028+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559db877dc50 0x559db87800a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.028+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780720 con 0x559db877dc50 2024-09-10T21:31:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.028+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559db877dc50 0x559db87800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.028+0000 7f0508000640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559db877dc50 0x559db87800a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.028+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(83 605.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.028+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db87807e0 con 0x559db877dc50 2024-09-10T21:31:25.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(84 605.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.034 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:25.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(85 605.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8781c10 con 0x559db877dc50 2024-09-10T21:31:25.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8781c10 con 0x559db877dc50 2024-09-10T21:31:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.032+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.036+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.036+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.036+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(89 mds0_inotable [create] v101'44 uv38 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.036+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.036+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(90 mds0_inotable [delete] v101'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.036+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(91 mds0_inotable [create] v101'46 uv46 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.041 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v101'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v101'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(94 100.00000000 [create] v101'49 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.040+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.044+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(95 100.00000000 [delete] v101'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.044+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.048 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.044+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(96 100.00000000 [create] v101'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.044+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8780c00 con 0x559db877dc50 2024-09-10T21:31:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.044+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.044+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8783a10 con 0x559db877dc50 2024-09-10T21:31:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v101'22 uv21 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v101'23 uv23 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.052 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:25.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v101'24 uv24 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.048+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.052+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v101'25 uv25 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.052+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.052+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v101'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.052+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.052+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v101'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.052+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.058 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.056+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v101'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.056+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.060+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v101'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.060+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87820c0 con 0x559db8779b90 2024-09-10T21:31:25.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.060+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v101'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.060+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db87810b0 con 0x559db8779b90 2024-09-10T21:31:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.064+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v101'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0500002b60 con 0x559db8779b90 2024-09-10T21:31:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.064+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559db87819b0 0x559db87840d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.064+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db87819b0 2024-09-10T21:31:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.064+0000 7f0507600640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559db87819b0 0x559db87840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.064+0000 7f0507600640 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559db87819b0 0x559db87840d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.068+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v101'22 uv21 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db87819b0 2024-09-10T21:31:25.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.068+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8784610 con 0x559db87819b0 2024-09-10T21:31:25.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.068+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v101'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db87819b0 2024-09-10T21:31:25.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.068+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db87819b0 2024-09-10T21:31:25.072 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:25.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.068+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v101'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db87819b0 2024-09-10T21:31:25.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.068+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db87819b0 2024-09-10T21:31:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v101'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db87819b0 2024-09-10T21:31:25.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db87819b0 2024-09-10T21:31:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f0507600640 1 -- 172.21.15.97:0/2951404364 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v101'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04f40ca020 con 0x559db87819b0 2024-09-10T21:31:25.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(114 100.00000000.inode [create] v101'57 uv49 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(115 100.00000000.inode [delete] v101'58 uv58 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.072+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.078 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:25.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.076+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(116 100.00000000.inode [create] v101'59 uv59 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.076+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.076+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v101'60 uv60 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.076+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.080+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v101'61 uv61 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.080+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.084+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(119 1.00000000 [create] v101'62 uv56 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.084+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.084+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(120 1.00000000 [delete] v101'63 uv63 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.084+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.088+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(121 1.00000000 [create] v101'64 uv64 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.091 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:25.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.088+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8785620 con 0x559db877dc50 2024-09-10T21:31:25.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.092+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v101'65 uv65 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.092+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8785620 con 0x559db877dc50 2024-09-10T21:31:25.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.092+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v101'66 uv66 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.092+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.092+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v101'67 uv67 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.092+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x559db8784610 con 0x559db877dc50 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f0508000640 1 -- 172.21.15.97:0/2951404364 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v101'68 uv68 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04f801a480 con 0x559db877dc50 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559db8775870 msgr2=0x559db8777c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559db8775870 0x559db8777c60 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559db877dc50 msgr2=0x559db87800a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559db877dc50 0x559db87800a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559db87819b0 msgr2=0x559db87840d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559db87819b0 0x559db87840d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559db8779b90 msgr2=0x559db877c9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559db8779b90 0x559db877c9f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 reap_dead start 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559db8770fc0 msgr2=0x559db87733b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559db8770fc0 0x559db87733b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f0508a00640 1 -- 172.21.15.97:0/2951404364 reap_dead start 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559db863a240 msgr2=0x559db863a620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559db863a240 0x559db863a620 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04e8071190 msgr2=0x7f04e8073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04e8071190 0x7f04e8073650 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c970 tx=0x7f04f800f040 comp rx=0 tx=0).stop 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db86ac490 msgr2=0x559db86cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db86ac490 0x559db86cc810 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f04f4097250 tx=0x7f04f40a4530 comp rx=0 tx=0).stop 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 shutdown_connections 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559db863a240 0x559db863a620 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04e8071190 0x7f04e8073650 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 --2- 172.21.15.97:0/2951404364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db86ac490 0x559db86cc810 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 >> 172.21.15.97:0/2951404364 conn(0x559db863d0d0 msgr2=0x559db86abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 shutdown_connections 2024-09-10T21:31:25.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.096+0000 7f050add4080 1 -- 172.21.15.97:0/2951404364 wait complete. 2024-09-10T21:31:25.103 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2024-09-10T21:31:25.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 Processor -- start 2024-09-10T21:31:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 -- start start 2024-09-10T21:31:25.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3acc002330 0x55f9db516510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4f3bc0 0x55f9db4f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9db4fa2e0 0x55f9db4802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9db3e6f40 con 0x55f9db4f3bc0 2024-09-10T21:31:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f9db3f3d00 con 0x7f3acc002330 2024-09-10T21:31:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae7c03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f9db388a00 con 0x55f9db4fa2e0 2024-09-10T21:31:25.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4f3bc0 0x55f9db4f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3acc002330 0x55f9db516510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4f3bc0 0x55f9db4f9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34132/0 (socket says 172.21.15.97:34132) 2024-09-10T21:31:25.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3adfe00640 1 -- 172.21.15.97:0/2019439701 learned_addr learned my addr 172.21.15.97:0/2019439701 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:25.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.192+0000 7f3ae5800640 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9db4fa2e0 0x55f9db4802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adfe00640 1 -- 172.21.15.97:0/2019439701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9db4fa2e0 msgr2=0x55f9db4802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:25.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adfe00640 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9db4fa2e0 0x55f9db4802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adfe00640 1 -- 172.21.15.97:0/2019439701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3acc002330 msgr2=0x55f9db516510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:25.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adfe00640 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3acc002330 0x55f9db516510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adfe00640 1 -- 172.21.15.97:0/2019439701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9db4fa6f0 con 0x55f9db4f3bc0 2024-09-10T21:31:25.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adfe00640 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4f3bc0 0x55f9db4f9da0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0004860 tx=0x7f3ad0004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450d7997db1ef9b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adea00640 1 -- 172.21.15.97:0/2019439701 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000ac20 con 0x55f9db4f3bc0 2024-09-10T21:31:25.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adea00640 1 -- 172.21.15.97:0/2019439701 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad000adc0 con 0x55f9db4f3bc0 2024-09-10T21:31:25.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adea00640 1 -- 172.21.15.97:0/2019439701 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00086a0 con 0x55f9db4f3bc0 2024-09-10T21:31:25.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2019439701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4f3bc0 msgr2=0x55f9db4f9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4f3bc0 0x55f9db4f9da0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0004860 tx=0x7f3ad0004890 comp rx=0 tx=0).stop 2024-09-10T21:31:25.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2019439701 shutdown_connections 2024-09-10T21:31:25.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f9db4fa2e0 0x55f9db4802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4f3bc0 0x55f9db4f9da0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2019439701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3acc002330 0x55f9db516510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2019439701 >> 172.21.15.97:0/2019439701 conn(0x55f9db4820d0 msgr2=0x55f9db484b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:25.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2019439701 shutdown_connections 2024-09-10T21:31:25.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2019439701 wait complete. 2024-09-10T21:31:25.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 Processor -- start 2024-09-10T21:31:25.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- start start 2024-09-10T21:31:25.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9db4f3bc0 0x55f9db5a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4fa2e0 0x55f9db43b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3acc002330 0x55f9db43bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9db517470 con 0x55f9db4fa2e0 2024-09-10T21:31:25.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f9db439140 con 0x55f9db4f3bc0 2024-09-10T21:31:25.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae7c03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f9db47eba0 con 0x7f3acc002330 2024-09-10T21:31:25.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4fa2e0 0x55f9db43b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9db4f3bc0 0x55f9db5a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae5800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3acc002330 0x55f9db43bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9db4f3bc0 0x55f9db5a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:39290/0 (socket says 172.21.15.97:39290) 2024-09-10T21:31:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 -- 172.21.15.97:0/2897110184 learned_addr learned my addr 172.21.15.97:0/2897110184 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 -- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3acc002330 msgr2=0x55f9db43bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3acc002330 0x55f9db43bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 -- 172.21.15.97:0/2897110184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4fa2e0 msgr2=0x55f9db43b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9db4fa2e0 0x55f9db43b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 -- 172.21.15.97:0/2897110184 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9db5156c0 con 0x55f9db4f3bc0 2024-09-10T21:31:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.196+0000 7f3ae4e00640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9db4f3bc0 0x55f9db5a9b10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80971d0 tx=0x7f3ad8009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3add600640 1 -- 172.21.15.97:0/2897110184 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80a5d20 con 0x55f9db4f3bc0 2024-09-10T21:31:25.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f9db515960 con 0x55f9db4f3bc0 2024-09-10T21:31:25.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3add600640 1 -- 172.21.15.97:0/2897110184 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad80bb420 con 0x55f9db4f3bc0 2024-09-10T21:31:25.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3add600640 1 -- 172.21.15.97:0/2897110184 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80bb870 con 0x55f9db4f3bc0 2024-09-10T21:31:25.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9db515ea0 con 0x55f9db4f3bc0 2024-09-10T21:31:25.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3add600640 1 -- 172.21.15.97:0/2897110184 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ad80ad400 con 0x55f9db4f3bc0 2024-09-10T21:31:25.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3add600640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac4070fc0 0x7f3ac4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3add600640 1 -- 172.21.15.97:0/2897110184 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7f3ad812d2d0 con 0x55f9db4f3bc0 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3adfe00640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac4070fc0 0x7f3ac4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f9db5ab0d0 0x55f9db5ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.200+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x55f9db513ac0 con 0x55f9db5ab0d0 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae5800640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f9db5ab0d0 0x55f9db5ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3adfe00640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac4070fc0 0x7f3ac4073480 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0004830 tx=0x7f3ad0003030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae5800640 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f9db5ab0d0 0x55f9db5ab4b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae5800640 1 -- 172.21.15.97:0/2897110184 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000001 [delete] v101'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3ae0009b10 con 0x55f9db5ab0d0 2024-09-10T21:31:25.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f9db5ab0d0 msgr2=0x55f9db5ab4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55f9db5ab0d0 0x55f9db5ab4b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac4070fc0 msgr2=0x7f3ac4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ac4070fc0 0x7f3ac4073480 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0004830 tx=0x7f3ad0003030 comp rx=0 tx=0).stop 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9db4f3bc0 msgr2=0x55f9db5a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 --2- 172.21.15.97:0/2897110184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f9db4f3bc0 0x55f9db5a9b10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80971d0 tx=0x7f3ad8009730 comp rx=0 tx=0).stop 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae5800640 1 -- 172.21.15.97:0/2897110184 reap_dead start 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 shutdown_connections 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 >> 172.21.15.97:0/2897110184 conn(0x55f9db4820d0 msgr2=0x55f9db513120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 shutdown_connections 2024-09-10T21:31:25.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.204+0000 7f3ae7c03080 1 -- 172.21.15.97:0/2897110184 wait complete. 2024-09-10T21:31:25.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:25.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.360+0000 7fb727200640 1 -- 172.21.15.97:0/4142730839 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 msgr2=0x7fb7200b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.360+0000 7fb727200640 1 --2- 172.21.15.97:0/4142730839 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 0x7fb7200b2a80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb718009bf0 tx=0x7fb71801cbe0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.360+0000 7fb727200640 1 -- 172.21.15.97:0/4142730839 shutdown_connections 2024-09-10T21:31:25.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.360+0000 7fb727200640 1 --2- 172.21.15.97:0/4142730839 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 0x7fb7200b2a80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.360+0000 7fb727200640 1 -- 172.21.15.97:0/4142730839 >> 172.21.15.97:0/4142730839 conn(0x7fb7200b1780 msgr2=0x7fb7200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:25.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 -- 172.21.15.97:0/4142730839 shutdown_connections 2024-09-10T21:31:25.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 -- 172.21.15.97:0/4142730839 wait complete. 2024-09-10T21:31:25.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 Processor -- start 2024-09-10T21:31:25.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 -- start start 2024-09-10T21:31:25.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 0x7fb72013df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb720143f90 0x7fb720138c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb720144690 0x7fb7200cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb720168f80 con 0x7fb720143f90 2024-09-10T21:31:25.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb720168e00 con 0x7fb720144690 2024-09-10T21:31:25.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb727200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb720169100 con 0x7fb720070460 2024-09-10T21:31:25.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb720143f90 0x7fb720138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb720143f90 0x7fb720138c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34146/0 (socket says 172.21.15.97:34146) 2024-09-10T21:31:25.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 -- 172.21.15.97:0/125062540 learned_addr learned my addr 172.21.15.97:0/125062540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb725200640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb720144690 0x7fb7200cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb71fe00640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 0x7fb72013df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 -- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 msgr2=0x7fb72013df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 0x7fb72013df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 -- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb720144690 msgr2=0x7fb7200cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb720144690 0x7fb7200cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb717400640 1 -- 172.21.15.97:0/125062540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72015e0d0 con 0x7fb720143f90 2024-09-10T21:31:25.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.364+0000 7fb71fe00640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 0x7fb72013df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:25.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb717400640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb720143f90 0x7fb720138c70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c430 tx=0x7fb70800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb71cc00640 1 -- 172.21.15.97:0/125062540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708011790 con 0x7fb720143f90 2024-09-10T21:31:25.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb71cc00640 1 -- 172.21.15.97:0/125062540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb708017070 con 0x7fb720143f90 2024-09-10T21:31:25.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72015c5d0 con 0x7fb720143f90 2024-09-10T21:31:25.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb71cc00640 1 -- 172.21.15.97:0/125062540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708011cc0 con 0x7fb720143f90 2024-09-10T21:31:25.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72015c010 con 0x7fb720143f90 2024-09-10T21:31:25.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7200b2a80 con 0x7fb720143f90 2024-09-10T21:31:25.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb71cc00640 1 -- 172.21.15.97:0/125062540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb70801c460 con 0x7fb720143f90 2024-09-10T21:31:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.368+0000 7fb71cc00640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6fc0725e0 0x7fb6fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.372+0000 7fb71cc00640 1 -- 172.21.15.97:0/125062540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6705+0+0 (secure 0 0 0) 0x7fb7080933e0 con 0x7fb720143f90 2024-09-10T21:31:25.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.372+0000 7fb71fe00640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6fc0725e0 0x7fb6fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:25.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.372+0000 7fb71fe00640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6fc0725e0 0x7fb6fc074aa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb710007fd0 tx=0x7fb710009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:25.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.376+0000 7fb71cc00640 1 -- 172.21.15.97:0/125062540 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb72015e0d0 con 0x7fb720143f90 2024-09-10T21:31:25.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.576+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb7200cc4e0 con 0x7fb720143f90 2024-09-10T21:31:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb71cc00640 1 -- 172.21.15.97:0/125062540 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v115) ==== 162+0+0 (secure 0 0 0) 0x7fb70805e6c0 con 0x7fb720143f90 2024-09-10T21:31:25.842 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6fc0725e0 msgr2=0x7fb6fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6fc0725e0 0x7fb6fc074aa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb710007fd0 tx=0x7fb710009040 comp rx=0 tx=0).stop 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb720143f90 msgr2=0x7fb720138c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb720143f90 0x7fb720138c70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c430 tx=0x7fb70800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 shutdown_connections 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb6fc0725e0 0x7fb6fc074aa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb720144690 0x7fb7200cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb720143f90 0x7fb720138c70 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 --2- 172.21.15.97:0/125062540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb720070460 0x7fb72013df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 >> 172.21.15.97:0/125062540 conn(0x7fb7200b1780 msgr2=0x7fb72006ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 shutdown_connections 2024-09-10T21:31:25.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:25.840+0000 7fb727200640 1 -- 172.21.15.97:0/125062540 wait complete. 2024-09-10T21:31:25.856 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:25.852+0000 7fba89a00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T21:31:25.857 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:25.856+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:25.857 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:25.856+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:31:25.869 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:26.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- 172.21.15.97:0/2336342775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 msgr2=0x7fdcf813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 --2- 172.21.15.97:0/2336342775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf813f9b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8074ae0 tx=0x7fdcf001cd80 comp rx=0 tx=0).stop 2024-09-10T21:31:26.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- 172.21.15.97:0/2336342775 shutdown_connections 2024-09-10T21:31:26.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 --2- 172.21.15.97:0/2336342775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf813f9b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 --2- 172.21.15.97:0/2336342775 >> v2:172.21.15.194:3300/0 conn(0x7fdcf80ce5f0 0x7fdcf8141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- 172.21.15.97:0/2336342775 >> 172.21.15.97:0/2336342775 conn(0x7fdcf80b1780 msgr2=0x7fdcf80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:26.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- 172.21.15.97:0/2336342775 shutdown_connections 2024-09-10T21:31:26.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- 172.21.15.97:0/2336342775 wait complete. 2024-09-10T21:31:26.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 Processor -- start 2024-09-10T21:31:26.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- start start 2024-09-10T21:31:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcf80ce5f0 0x7fdcf81376f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcf80ce9d0 0x7fdcf813aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf8153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf8168bf0 con 0x7fdcf815fde0 2024-09-10T21:31:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf8168a70 con 0x7fdcf80ce5f0 2024-09-10T21:31:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcff000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf8168d70 con 0x7fdcf80ce9d0 2024-09-10T21:31:26.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf8153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf8153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34172/0 (socket says 172.21.15.97:34172) 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 -- 172.21.15.97:0/2040655953 learned_addr learned my addr 172.21.15.97:0/2040655953 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcf7e00640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcf80ce5f0 0x7fdcf81376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcf7400640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcf80ce9d0 0x7fdcf813aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 -- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcf80ce9d0 msgr2=0x7fdcf813aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcf80ce9d0 0x7fdcf813aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 -- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcf80ce5f0 msgr2=0x7fdcf81376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcf80ce5f0 0x7fdcf81376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcfd000640 1 -- 172.21.15.97:0/2040655953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf8139d30 con 0x7fdcf815fde0 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.008+0000 7fdcf7e00640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcf80ce5f0 0x7fdcf81376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:26.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcfd000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf8153150 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdcf001b430 tx=0x7fdcf0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:26.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcefe00640 1 -- 172.21.15.97:0/2040655953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00bb070 con 0x7fdcf815fde0 2024-09-10T21:31:26.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcefe00640 1 -- 172.21.15.97:0/2040655953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcf0004d40 con 0x7fdcf815fde0 2024-09-10T21:31:26.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcefe00640 1 -- 172.21.15.97:0/2040655953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00b2040 con 0x7fdcf815fde0 2024-09-10T21:31:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf813a010 con 0x7fdcf815fde0 2024-09-10T21:31:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf81385d0 con 0x7fdcf815fde0 2024-09-10T21:31:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf8141d20 con 0x7fdcf815fde0 2024-09-10T21:31:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcefe00640 1 -- 172.21.15.97:0/2040655953 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdcf00b5030 con 0x7fdcf815fde0 2024-09-10T21:31:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcefe00640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdcd80725e0 0x7fdcd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcefe00640 1 -- 172.21.15.97:0/2040655953 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7fdcf012d210 con 0x7fdcf815fde0 2024-09-10T21:31:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.012+0000 7fdcf7e00640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdcd80725e0 0x7fdcd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.016+0000 7fdcf7e00640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdcd80725e0 0x7fdcd8074aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdce80058d0 tx=0x7fdce8005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:26.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.016+0000 7fdcefe00640 1 -- 172.21.15.97:0/2040655953 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdcf0016970 con 0x7fdcf815fde0 2024-09-10T21:31:26.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.216+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcf8156380 con 0x7fdcf815fde0 2024-09-10T21:31:26.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.216+0000 7fdcefe00640 1 -- 172.21.15.97:0/2040655953 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+3569 (secure 0 0 0) 0x7fdcf00f7e50 con 0x7fdcf815fde0 2024-09-10T21:31:26.221 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:26.222 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":117,"btime":"2024-09-10T21:31:25:859772+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:25.859716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:26.222 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 117 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdcd80725e0 msgr2=0x7fdcd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdcd80725e0 0x7fdcd8074aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdce80058d0 tx=0x7fdce8005860 comp rx=0 tx=0).stop 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 msgr2=0x7fdcf8153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf8153150 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdcf001b430 tx=0x7fdcf0005b00 comp rx=0 tx=0).stop 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 shutdown_connections 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdcd80725e0 0x7fdcd8074aa0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf815fde0 0x7fdcf8153150 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcf80ce9d0 0x7fdcf813aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 --2- 172.21.15.97:0/2040655953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcf80ce5f0 0x7fdcf81376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 >> 172.21.15.97:0/2040655953 conn(0x7fdcf80b1780 msgr2=0x7fdcf8138a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:26.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 shutdown_connections 2024-09-10T21:31:26.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.220+0000 7fdcff000640 1 -- 172.21.15.97:0/2040655953 wait complete. 2024-09-10T21:31:26.242 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:26.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:26.373 INFO:tasks.ceph.mds.a.smithi097.stderr: -38> 2024-09-10T21:31:21.616+0000 7f9331800640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-10T21:31:26.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.380+0000 7f8e14e00640 1 -- 172.21.15.97:0/3487221144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100b0d50 msgr2=0x7f8e100ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.380+0000 7f8e14e00640 1 --2- 172.21.15.97:0/3487221144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100b0d50 0x7f8e100ccc10 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009c80 tx=0x7f8e0801cd60 comp rx=0 tx=0).stop 2024-09-10T21:31:26.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.380+0000 7f8e14e00640 1 -- 172.21.15.97:0/3487221144 shutdown_connections 2024-09-10T21:31:26.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.380+0000 7f8e14e00640 1 --2- 172.21.15.97:0/3487221144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100b0d50 0x7f8e100ccc10 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.380+0000 7f8e14e00640 1 --2- 172.21.15.97:0/3487221144 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e100afdd0 0x7f8e100cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.380+0000 7f8e14e00640 1 -- 172.21.15.97:0/3487221144 >> 172.21.15.97:0/3487221144 conn(0x7f8e100d1c30 msgr2=0x7f8e100d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:26.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 -- 172.21.15.97:0/3487221144 shutdown_connections 2024-09-10T21:31:26.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 -- 172.21.15.97:0/3487221144 wait complete. 2024-09-10T21:31:26.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 Processor -- start 2024-09-10T21:31:26.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 -- start start 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e100afdd0 0x7f8e100b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e100b0d50 0x7f8e100b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100ce640 0x7f8e100b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10168f40 con 0x7f8e100ce640 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10168dc0 con 0x7f8e100b0d50 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8e101690c0 con 0x7f8e100afdd0 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100ce640 0x7f8e100b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100ce640 0x7f8e100b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34176/0 (socket says 172.21.15.97:34176) 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 -- 172.21.15.97:0/21879632 learned_addr learned my addr 172.21.15.97:0/21879632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0de00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e100afdd0 0x7f8e100b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e07e00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e100b0d50 0x7f8e100b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 -- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e100afdd0 msgr2=0x7f8e100b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e100afdd0 0x7f8e100b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 -- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e100b0d50 msgr2=0x7f8e100b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e100b0d50 0x7f8e100b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 -- 172.21.15.97:0/21879632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e100b2330 con 0x7f8e100ce640 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e0e800640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100ce640 0x7f8e100b16c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8e0801cb90 tx=0x7f8e080093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e06000640 1 -- 172.21.15.97:0/21879632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e080bb070 con 0x7f8e100ce640 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e06000640 1 -- 172.21.15.97:0/21879632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e08004d60 con 0x7f8e100ce640 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e100b25d0 con 0x7f8e100ce640 2024-09-10T21:31:26.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.384+0000 7f8e06000640 1 -- 172.21.15.97:0/21879632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e080b2040 con 0x7f8e100ce640 2024-09-10T21:31:26.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.388+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e100b2b90 con 0x7f8e100ce640 2024-09-10T21:31:26.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.388+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e100cea20 con 0x7f8e100ce640 2024-09-10T21:31:26.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.388+0000 7f8e06000640 1 -- 172.21.15.97:0/21879632 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8e080b5030 con 0x7f8e100ce640 2024-09-10T21:31:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.388+0000 7f8e06000640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dec0725e0 0x7f8dec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.388+0000 7f8e06000640 1 -- 172.21.15.97:0/21879632 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f8e0812d4b0 con 0x7f8e100ce640 2024-09-10T21:31:26.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.392+0000 7f8e0de00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dec0725e0 0x7f8dec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.392+0000 7f8e0de00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dec0725e0 0x7f8dec074aa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000b810 tx=0x7f8e00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:26.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.392+0000 7f8e06000640 1 -- 172.21.15.97:0/21879632 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e080f1de0 con 0x7f8e100ce640 2024-09-10T21:31:26.412 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:31:26.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e100b4a60 con 0x7f8e100ce640 2024-09-10T21:31:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e06000640 1 -- 172.21.15.97:0/21879632 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+3569 (secure 0 0 0) 0x7f8e08016950 con 0x7f8e100ce640 2024-09-10T21:31:26.599 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:26.599 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":117,"btime":"2024-09-10T21:31:25:859772+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:25.859716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:26.599 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 117 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dec0725e0 msgr2=0x7f8dec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dec0725e0 0x7f8dec074aa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000b810 tx=0x7f8e00008040 comp rx=0 tx=0).stop 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100ce640 msgr2=0x7f8e100b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100ce640 0x7f8e100b16c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8e0801cb90 tx=0x7f8e080093c0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 shutdown_connections 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8dec0725e0 0x7f8dec074aa0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e100ce640 0x7f8e100b16c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e100b0d50 0x7f8e100b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 --2- 172.21.15.97:0/21879632 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e100afdd0 0x7f8e100b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 >> 172.21.15.97:0/21879632 conn(0x7f8e100d1c30 msgr2=0x7f8e1006eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:26.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 shutdown_connections 2024-09-10T21:31:26.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.596+0000 7f8e14e00640 1 -- 172.21.15.97:0/21879632 wait complete. 2024-09-10T21:31:26.620 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:26.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 -- 172.21.15.97:0/1986202066 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 msgr2=0x7eff180aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 --2- 172.21.15.97:0/1986202066 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 0x7eff180aedc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff04009840 tx=0x7eff0401c820 comp rx=0 tx=0).stop 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 -- 172.21.15.97:0/1986202066 shutdown_connections 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 --2- 172.21.15.97:0/1986202066 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 0x7eff180aedc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 --2- 172.21.15.97:0/1986202066 >> v2:172.21.15.194:3300/0 conn(0x7eff180ce640 0x7eff180cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 -- 172.21.15.97:0/1986202066 >> 172.21.15.97:0/1986202066 conn(0x7eff180d1c30 msgr2=0x7eff180d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 -- 172.21.15.97:0/1986202066 shutdown_connections 2024-09-10T21:31:26.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.760+0000 7eff1ec00640 1 -- 172.21.15.97:0/1986202066 wait complete. 2024-09-10T21:31:26.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 Processor -- start 2024-09-10T21:31:26.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 -- start start 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff180af300 0x7eff1815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 0x7eff18158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff180ce640 0x7eff18164d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff18169270 con 0x7eff180ce640 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff181690f0 con 0x7eff180af300 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7eff181693f0 con 0x7eff180b0d50 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff180ce640 0x7eff18164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff180ce640 0x7eff18164d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34212/0 (socket says 172.21.15.97:34212) 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 -- 172.21.15.97:0/3774319953 learned_addr learned my addr 172.21.15.97:0/3774319953 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff17e00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff180af300 0x7eff1815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff17400640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 0x7eff18158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 -- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 msgr2=0x7eff18158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 0x7eff18158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 -- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff180af300 msgr2=0x7eff1815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff180af300 0x7eff1815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 -- 172.21.15.97:0/3774319953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff18159600 con 0x7eff180ce640 2024-09-10T21:31:26.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1cc00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff180ce640 0x7eff18164d60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7eff1000c730 tx=0x7eff1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff0fe00640 1 -- 172.21.15.97:0/3774319953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10003b80 con 0x7eff180ce640 2024-09-10T21:31:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff0fe00640 1 -- 172.21.15.97:0/3774319953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff10003d40 con 0x7eff180ce640 2024-09-10T21:31:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff0fe00640 1 -- 172.21.15.97:0/3774319953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff100052d0 con 0x7eff180ce640 2024-09-10T21:31:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff180c5d50 con 0x7eff180ce640 2024-09-10T21:31:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff180c6310 con 0x7eff180ce640 2024-09-10T21:31:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.764+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff180aedc0 con 0x7eff180ce640 2024-09-10T21:31:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.768+0000 7eff0fe00640 1 -- 172.21.15.97:0/3774319953 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff10005680 con 0x7eff180ce640 2024-09-10T21:31:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.768+0000 7eff0fe00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efef80725e0 0x7efef8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.768+0000 7eff0fe00640 1 -- 172.21.15.97:0/3774319953 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7eff10093410 con 0x7eff180ce640 2024-09-10T21:31:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.768+0000 7eff17e00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efef80725e0 0x7efef8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.768+0000 7eff17e00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efef80725e0 0x7efef8074aa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7eff08009600 tx=0x7eff080091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:26.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.772+0000 7eff0fe00640 1 -- 172.21.15.97:0/3774319953 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff18159600 con 0x7eff180ce640 2024-09-10T21:31:26.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff1806e460 con 0x7eff180ce640 2024-09-10T21:31:26.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff0fe00640 1 -- 172.21.15.97:0/3774319953 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+4395 (secure 0 0 0) 0x7eff1005e0f0 con 0x7eff180ce640 2024-09-10T21:31:26.987 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:26.987 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":118,"btime":"2024-09-10T21:31:26:861957+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7917,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2013408839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2013408839},{"type":"v1","addr":"172.21.15.97:6834","nonce":2013408839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:25.859716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:26.987 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 118 2024-09-10T21:31:26.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efef80725e0 msgr2=0x7efef8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efef80725e0 0x7efef8074aa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7eff08009600 tx=0x7eff080091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff180ce640 msgr2=0x7eff18164d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:26.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff180ce640 0x7eff18164d60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7eff1000c730 tx=0x7eff1000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 shutdown_connections 2024-09-10T21:31:26.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efef80725e0 0x7efef8074aa0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff180ce640 0x7eff18164d60 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff180b0d50 0x7eff18158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 --2- 172.21.15.97:0/3774319953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff180af300 0x7eff1815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:26.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 >> 172.21.15.97:0/3774319953 conn(0x7eff180d1c30 msgr2=0x7eff180d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:26.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 shutdown_connections 2024-09-10T21:31:26.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:26.984+0000 7eff1ec00640 1 -- 172.21.15.97:0/3774319953 wait complete. 2024-09-10T21:31:27.009 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2024-09-10T21:31:27.009 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2024-09-10T21:31:27.010 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:27.010 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:27.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:31:27.018 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:27.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:27.018 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:27.068 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:31:27.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:31:27.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- 172.21.15.97:0/1825713975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968144690 msgr2=0x7f09680b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 --2- 172.21.15.97:0/1825713975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968144690 0x7f09680b2a80 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f09680725c0 tx=0x7f095401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- 172.21.15.97:0/1825713975 shutdown_connections 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 --2- 172.21.15.97:0/1825713975 >> v2:172.21.15.194:3300/0 conn(0x7f096806fd50 0x7f09680701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 --2- 172.21.15.97:0/1825713975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968144690 0x7f09680b2a80 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- 172.21.15.97:0/1825713975 >> 172.21.15.97:0/1825713975 conn(0x7f09680b1780 msgr2=0x7f09680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- 172.21.15.97:0/1825713975 shutdown_connections 2024-09-10T21:31:27.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- 172.21.15.97:0/1825713975 wait complete. 2024-09-10T21:31:27.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 Processor -- start 2024-09-10T21:31:27.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- start start 2024-09-10T21:31:27.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096806fd50 0x7f09681faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0968143f90 0x7f096815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0968144690 0x7f096815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0968167c40 con 0x7f096806fd50 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0968167ac0 con 0x7f0968144690 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f096e800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0968167dc0 con 0x7f0968143f90 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096806fd50 0x7f09681faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096806fd50 0x7f09681faa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34232/0 (socket says 172.21.15.97:34232) 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 -- 172.21.15.97:0/3730985111 learned_addr learned my addr 172.21.15.97:0/3730985111 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967e00640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0968144690 0x7f096815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0966a00640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0968143f90 0x7f096815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:27.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 -- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0968143f90 msgr2=0x7f096815f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0968143f90 0x7f096815f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 -- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0968144690 msgr2=0x7f096815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0968144690 0x7f096815e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 -- 172.21.15.97:0/3730985111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09681fbb20 con 0x7f096806fd50 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967e00640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0968144690 0x7f096815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.252+0000 7f0967400640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096806fd50 0x7f09681faa80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f095800c870 tx=0x7f095800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f095f400640 1 -- 172.21.15.97:0/3730985111 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958003b80 con 0x7f096806fd50 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f095f400640 1 -- 172.21.15.97:0/3730985111 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0958003d20 con 0x7f096806fd50 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f095f400640 1 -- 172.21.15.97:0/3730985111 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095800aab0 con 0x7f096806fd50 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681fbe00 con 0x7f096806fd50 2024-09-10T21:31:27.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096815dba0 con 0x7f096806fd50 2024-09-10T21:31:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096815d6b0 con 0x7f096806fd50 2024-09-10T21:31:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f095f400640 1 -- 172.21.15.97:0/3730985111 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f095800ac50 con 0x7f096806fd50 2024-09-10T21:31:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f095f400640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09480725e0 0x7f0948074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:27.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f095f400640 1 -- 172.21.15.97:0/3730985111 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f0958093ff0 con 0x7f096806fd50 2024-09-10T21:31:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.256+0000 7f0966a00640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09480725e0 0x7f0948074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:27.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.260+0000 7f0966a00640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09480725e0 0x7f0948074aa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f09540af000 tx=0x7f095401b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:27.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.260+0000 7f095f400640 1 -- 172.21.15.97:0/3730985111 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f095805f2b0 con 0x7f096806fd50 2024-09-10T21:31:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.456+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f09680701a0 con 0x7f096806fd50 2024-09-10T21:31:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f095f400640 1 -- 172.21.15.97:0/3730985111 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) ==== 161+0+0 (secure 0 0 0) 0x7f095805ebb0 con 0x7f096806fd50 2024-09-10T21:31:27.867 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09480725e0 msgr2=0x7f0948074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09480725e0 0x7f0948074aa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f09540af000 tx=0x7f095401b100 comp rx=0 tx=0).stop 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096806fd50 msgr2=0x7f09681faa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096806fd50 0x7f09681faa80 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f095800c870 tx=0x7f095800cd30 comp rx=0 tx=0).stop 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 shutdown_connections 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09480725e0 0x7f0948074aa0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0968144690 0x7f096815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0968143f90 0x7f096815f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 --2- 172.21.15.97:0/3730985111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096806fd50 0x7f09681faa80 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 >> 172.21.15.97:0/3730985111 conn(0x7f09680b1780 msgr2=0x7f0968141090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:27.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 shutdown_connections 2024-09-10T21:31:27.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:27.864+0000 7f096e800640 1 -- 172.21.15.97:0/3730985111 wait complete. 2024-09-10T21:31:27.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- 172.21.15.97:0/4236902173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 msgr2=0x7fbf00003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 --2- 172.21.15.97:0/4236902173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 0x7fbf00003e30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009ee0 tx=0x7fbf0c01d5d0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf20e00640 1 -- 172.21.15.97:0/4236902173 reap_dead start 2024-09-10T21:31:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- 172.21.15.97:0/4236902173 shutdown_connections 2024-09-10T21:31:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- 172.21.15.97:0/4236902173 >> 172.21.15.97:0/4236902173 conn(0x7fbf1c0b1780 msgr2=0x7fbf1c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- 172.21.15.97:0/4236902173 shutdown_connections 2024-09-10T21:31:28.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- 172.21.15.97:0/4236902173 wait complete. 2024-09-10T21:31:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 Processor -- start 2024-09-10T21:31:28.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- start start 2024-09-10T21:31:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 0x7fbf1c1fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf1c1567c0 0x7fbf1c0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf1c13a440 0x7fbf1c072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c140b20 con 0x7fbf00003a10 2024-09-10T21:31:28.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c1409a0 con 0x7fbf1c13a440 2024-09-10T21:31:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf22e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1c140ca0 con 0x7fbf1c1567c0 2024-09-10T21:31:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 0x7fbf1c1fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 0x7fbf1c1fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58210/0 (socket says 172.21.15.97:58210) 2024-09-10T21:31:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 -- 172.21.15.97:0/1939860864 learned_addr learned my addr 172.21.15.97:0/1939860864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf20e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf1c13a440 0x7fbf1c072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf13400640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf1c1567c0 0x7fbf1c0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 -- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf1c1567c0 msgr2=0x7fbf1c0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf1c1567c0 0x7fbf1c0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 -- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf1c13a440 msgr2=0x7fbf1c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf1c13a440 0x7fbf1c072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 -- 172.21.15.97:0/1939860864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c0b1370 con 0x7fbf00003a10 2024-09-10T21:31:28.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf20e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf1c13a440 0x7fbf1c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.032+0000 7fbf1be00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 0x7fbf1c1fb970 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c01de70 tx=0x7fbf0c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf18c00640 1 -- 172.21.15.97:0/1939860864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c005240 con 0x7fbf00003a10 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf18c00640 1 -- 172.21.15.97:0/1939860864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf0c0054d0 con 0x7fbf00003a10 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf18c00640 1 -- 172.21.15.97:0/1939860864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c005880 con 0x7fbf00003a10 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c157370 con 0x7fbf00003a10 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c1fae90 con 0x7fbf00003a10 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c141d20 con 0x7fbf00003a10 2024-09-10T21:31:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf18c00640 1 -- 172.21.15.97:0/1939860864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf0c0c9060 con 0x7fbf00003a10 2024-09-10T21:31:28.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf18c00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbef80725e0 0x7fbef8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf18c00640 1 -- 172.21.15.97:0/1939860864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7fbf0c12e870 con 0x7fbf00003a10 2024-09-10T21:31:28.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.036+0000 7fbf13400640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbef80725e0 0x7fbef8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.040+0000 7fbf13400640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbef80725e0 0x7fbef8074aa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c144860 tx=0x7fbf04006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.040+0000 7fbf18c00640 1 -- 172.21.15.97:0/1939860864 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf1c0b1370 con 0x7fbf00003a10 2024-09-10T21:31:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.268+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fbf1c141360 con 0x7fbf00003a10 2024-09-10T21:31:28.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf18c00640 1 -- 172.21.15.97:0/1939860864 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v120) ==== 101+0+0 (secure 0 0 0) 0x7fbf0c017340 con 0x7fbf00003a10 2024-09-10T21:31:28.877 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbef80725e0 msgr2=0x7fbef8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbef80725e0 0x7fbef8074aa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c144860 tx=0x7fbf04006040 comp rx=0 tx=0).stop 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 msgr2=0x7fbf1c1fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 0x7fbf1c1fb970 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c01de70 tx=0x7fbf0c009ba0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 shutdown_connections 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbef80725e0 0x7fbef8074aa0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf1c13a440 0x7fbf1c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbf1c1567c0 0x7fbf1c0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 --2- 172.21.15.97:0/1939860864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf00003a10 0x7fbf1c1fb970 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 >> 172.21.15.97:0/1939860864 conn(0x7fbf1c0b1780 msgr2=0x7fbf1c06fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 shutdown_connections 2024-09-10T21:31:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.876+0000 7fbf22e00640 1 -- 172.21.15.97:0/1939860864 wait complete. 2024-09-10T21:31:28.902 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:28.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 Processor -- start 2024-09-10T21:31:28.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 -- start start 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ff1147a00 con 0x564ff1228940 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564ff11a5f40 con 0x564ff12b32a0 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff30a159080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564ff12b5080 con 0x564ff12b2b00 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12d2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58218/0 (socket says 172.21.15.97:58218) 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 -- 172.21.15.97:0/3089457141 learned_addr learned my addr 172.21.15.97:0/3089457141 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:28.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff306a00640 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307e00640 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 -- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 msgr2=0x564ff12cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 -- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 msgr2=0x564ff12b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 -- 172.21.15.97:0/3089457141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff1120c10 con 0x564ff1228940 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307e00640 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff306a00640 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.944+0000 7ff307400640 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12d2700 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800b810 tx=0x7ff2f800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8b74019cb0014b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff305600640 1 -- 172.21.15.97:0/3089457141 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800abd0 con 0x564ff1228940 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff305600640 1 -- 172.21.15.97:0/3089457141 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2f800ad70 con 0x564ff1228940 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff305600640 1 -- 172.21.15.97:0/3089457141 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8008690 con 0x564ff1228940 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- 172.21.15.97:0/3089457141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 msgr2=0x564ff12d2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12d2700 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800b810 tx=0x7ff2f800bcd0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- 172.21.15.97:0/3089457141 shutdown_connections 2024-09-10T21:31:28.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 --2- 172.21.15.97:0/3089457141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12d2700 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- 172.21.15.97:0/3089457141 >> 172.21.15.97:0/3089457141 conn(0x564ff1227520 msgr2=0x564ff11f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:28.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- 172.21.15.97:0/3089457141 shutdown_connections 2024-09-10T21:31:28.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- 172.21.15.97:0/3089457141 wait complete. 2024-09-10T21:31:28.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 Processor -- start 2024-09-10T21:31:28.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- start start 2024-09-10T21:31:28.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ff12b25f0 con 0x564ff1228940 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564ff12b6b70 con 0x564ff12b32a0 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564ff123b8b0 con 0x564ff12b2b00 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58226/0 (socket says 172.21.15.97:58226) 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 learned_addr learned my addr 172.21.15.97:0/2360023043 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307e00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff306a00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 msgr2=0x564ff12d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564ff12b2b00 0x564ff12d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 msgr2=0x564ff12d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff123bb50 con 0x564ff1228940 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307e00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564ff12b32a0 0x564ff12d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff307400640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12ce090 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8004fd0 tx=0x7ff2f80156d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff2ffe00640 1 -- 172.21.15.97:0/2360023043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8015cd0 con 0x564ff1228940 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff2ffe00640 1 -- 172.21.15.97:0/2360023043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2f801f400 con 0x564ff1228940 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ff123bdf0 con 0x564ff1228940 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff2ffe00640 1 -- 172.21.15.97:0/2360023043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f801f850 con 0x564ff1228940 2024-09-10T21:31:28.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.948+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff12425f0 con 0x564ff1228940 2024-09-10T21:31:28.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff2ffe00640 1 -- 172.21.15.97:0/2360023043 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff2f8028440 con 0x564ff1228940 2024-09-10T21:31:28.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff2ffe00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2e8070ff0 0x7ff2e80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff2ffe00640 1 -- 172.21.15.97:0/2360023043 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7ff2f8093b90 con 0x564ff1228940 2024-09-10T21:31:28.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff306a00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2e8070ff0 0x7ff2e80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.957 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x564ff123c530 0x564ff123c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff123ce50 con 0x564ff123c530 2024-09-10T21:31:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff307e00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x564ff123c530 0x564ff123c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.952+0000 7ff306a00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2e8070ff0 0x7ff2e80734b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4001230 tx=0x7ff2f4001770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.956+0000 7ff307e00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x564ff123c530 0x564ff123c910 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.956+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v102'164 uv138 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.956+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff123d220 con 0x564ff123c530 2024-09-10T21:31:28.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.956+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v102'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.956+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff123d5f0 con 0x564ff123c530 2024-09-10T21:31:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.960+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v102'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.963 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.960+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13765a0 con 0x564ff123c530 2024-09-10T21:31:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.960+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.960+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376940 con 0x564ff123c530 2024-09-10T21:31:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.964+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.964+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375590 con 0x564ff123c530 2024-09-10T21:31:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.964+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.964+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375930 con 0x564ff123c530 2024-09-10T21:31:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.968+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v102'170 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.968+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1375d20 con 0x564ff123c530 2024-09-10T21:31:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.968+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v102'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.968+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375d20 con 0x564ff123c530 2024-09-10T21:31:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:28.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.972+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v102'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.972+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.972+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.976+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.976+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.976+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.980+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.980+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.984+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v102'176 uv151 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.984+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.984+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'177 uv177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.984+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.984+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v102'178 uv178 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.989 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:28.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.984+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.988+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'179 uv179 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.988+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.988+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'180 uv180 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.988+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1375cd0 con 0x564ff123c530 2024-09-10T21:31:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.992+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'181 uv181 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.992+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.992+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'182 uv182 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.992+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.992+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v102'183 uv157 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.992+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.996+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v102'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.996+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:28.999 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.996+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v102'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.996+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1377cf0 con 0x564ff123c530 2024-09-10T21:31:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.996+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v102'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:28.996+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1377cf0 con 0x564ff123c530 2024-09-10T21:31:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.000+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v102'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.000+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:29.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.000+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v102'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.000+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.004+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v102'189 uv163 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.004+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.004+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v102'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.004+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:29.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.004+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v102'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.009 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:29.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.004+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1377cf0 con 0x564ff123c530 2024-09-10T21:31:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.008+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v102'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.008+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1377cf0 con 0x564ff123c530 2024-09-10T21:31:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.008+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v102'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.008+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1376ce0 con 0x564ff123c530 2024-09-10T21:31:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.012+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v102'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff123c530 2024-09-10T21:31:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.012+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x564ff13771f0 0x564ff13795e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.012+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1379b20 con 0x564ff13771f0 2024-09-10T21:31:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.012+0000 7ff307400640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x564ff13771f0 0x564ff13795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.012+0000 7ff307400640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x564ff13771f0 0x564ff13795e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.012+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v102'129 uv113 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.012+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1379ec0 con 0x564ff13771f0 2024-09-10T21:31:29.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.016+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v102'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.016+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137a260 con 0x564ff13771f0 2024-09-10T21:31:29.019 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:29.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.016+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v102'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.016+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137aab0 con 0x564ff13771f0 2024-09-10T21:31:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.016+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v102'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.016+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137ae50 con 0x564ff13771f0 2024-09-10T21:31:29.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.020+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v102'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.020+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b1f0 con 0x564ff13771f0 2024-09-10T21:31:29.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.020+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v102'134 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.020+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff137b590 con 0x564ff13771f0 2024-09-10T21:31:29.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.020+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v102'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.020+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.025 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:29.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.024+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v102'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.024+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.024+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v102'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.024+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.024+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v102'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.024+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.029 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:29.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.028+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v102'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.028+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137fb00 con 0x564ff13771f0 2024-09-10T21:31:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.028+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [write 0~16801] v102'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.028+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137fb00 con 0x564ff13771f0 2024-09-10T21:31:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.028+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v102'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.028+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.032+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 606.00000000 [create] v102'142 uv127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.032+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.032+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 606.00000000 [delete] v102'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.032+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.037 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:29.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v102'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137c940 con 0x564ff13771f0 2024-09-10T21:31:29.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v102'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137c940 con 0x564ff13771f0 2024-09-10T21:31:29.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v102'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137b930 con 0x564ff13771f0 2024-09-10T21:31:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v102'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff13771f0 2024-09-10T21:31:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x564ff137be40 0x564ff137e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.036+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137e770 con 0x564ff137be40 2024-09-10T21:31:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.040+0000 7ff306a00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x564ff137be40 0x564ff137e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.040+0000 7ff306a00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x564ff137be40 0x564ff137e230 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.040+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(51 607.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.040+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff137eb90 con 0x564ff137be40 2024-09-10T21:31:29.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.040+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(52 607.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.040+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137a6f0 con 0x564ff137be40 2024-09-10T21:31:29.047 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:29.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.044+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(53 607.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.044+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13809a0 con 0x564ff137be40 2024-09-10T21:31:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.048+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.048+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1380d40 con 0x564ff137be40 2024-09-10T21:31:29.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.048+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.048+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff137f990 con 0x564ff137be40 2024-09-10T21:31:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.052+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.052+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x564ff1380290 0x564ff13830f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.052+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383630 con 0x564ff1380290 2024-09-10T21:31:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.052+0000 7ff307e00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x564ff1380290 0x564ff13830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.052+0000 7ff307e00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x564ff1380290 0x564ff13830f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v102'27 uv26 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1383630 con 0x564ff1380290 2024-09-10T21:31:29.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v102'28 uv28 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383a20 con 0x564ff1380290 2024-09-10T21:31:29.059 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'29 uv29 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383a20 con 0x564ff1380290 2024-09-10T21:31:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v102'30 uv30 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.056+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383a20 con 0x564ff137be40 2024-09-10T21:31:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.060+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(61 608.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.060+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.060+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(62 608.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.060+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.060+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(63 608.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.064 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:29.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.060+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1384dd0 con 0x564ff137be40 2024-09-10T21:31:29.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1384dd0 con 0x564ff137be40 2024-09-10T21:31:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(67 604.00000000 [create] v102'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.064+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(68 604.00000000 [delete] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.070 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:29.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(69 604.00000000 [create] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1384dd0 con 0x564ff1380290 2024-09-10T21:31:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1384dd0 con 0x564ff1380290 2024-09-10T21:31:29.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v102'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.068+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.072+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v102'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.072+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.072+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(73 500.00000000 [create] v102'65 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.072+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.072+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(74 500.00000000 [delete] v102'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.072+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(75 500.00000000 [create] v102'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.077 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:29.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v102'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff1380290 2024-09-10T21:31:29.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v102'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(78 mds_snaptable [create] v102'27 uv26 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.076+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.080+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(79 mds_snaptable [delete] v102'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.080+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.083 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:29.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.080+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(80 mds_snaptable [create] v102'29 uv29 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.080+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.080+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v102'30 uv30 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.080+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1383dc0 con 0x564ff137be40 2024-09-10T21:31:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v102'31 uv31 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff137be40 2024-09-10T21:31:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564ff1384440 0x564ff1386830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1386e00 con 0x564ff1384440 2024-09-10T21:31:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff307400640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564ff1384440 0x564ff1386830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff307400640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564ff1384440 0x564ff1386830 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(83 605.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1386ec0 con 0x564ff1384440 2024-09-10T21:31:29.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(84 605.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.084+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.089 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:29.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(85 605.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13882f0 con 0x564ff1384440 2024-09-10T21:31:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13882f0 con 0x564ff1384440 2024-09-10T21:31:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.088+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.092+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(89 mds0_inotable [create] v102'54 uv48 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.092+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.092+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(90 mds0_inotable [delete] v102'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.092+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.096 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.092+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(91 mds0_inotable [create] v102'56 uv56 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.092+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v102'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v102'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(94 100.00000000 [create] v102'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(95 100.00000000 [delete] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.096+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.101 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:29.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(96 100.00000000 [create] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13872e0 con 0x564ff1384440 2024-09-10T21:31:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff138a0f0 con 0x564ff1384440 2024-09-10T21:31:29.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v102'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v102'26 uv25 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.100+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.104+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v102'27 uv27 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.104+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.104+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v102'28 uv28 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.107 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.104+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.104+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v102'29 uv29 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.104+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.108+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v102'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.108+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.108+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.108+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.112 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.108+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.108+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13887a0 con 0x564ff1380290 2024-09-10T21:31:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff13887a0 con 0x564ff1380290 2024-09-10T21:31:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v102'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1387790 con 0x564ff1380290 2024-09-10T21:31:29.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v102'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff300002050 con 0x564ff1380290 2024-09-10T21:31:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x564ff1387e10 0x564ff13881f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388730 con 0x564ff1387e10 2024-09-10T21:31:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff306a00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x564ff1387e10 0x564ff13881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.112+0000 7ff306a00640 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x564ff1387e10 0x564ff13881f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.116+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v102'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff1387e10 2024-09-10T21:31:29.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.116+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1388730 con 0x564ff1387e10 2024-09-10T21:31:29.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.116+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v102'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff1387e10 2024-09-10T21:31:29.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.116+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388730 con 0x564ff1387e10 2024-09-10T21:31:29.120 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.116+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v102'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff1387e10 2024-09-10T21:31:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.116+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388730 con 0x564ff1387e10 2024-09-10T21:31:29.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v102'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff1387e10 2024-09-10T21:31:29.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388730 con 0x564ff1387e10 2024-09-10T21:31:29.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff306a00640 1 -- 172.21.15.97:0/2360023043 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v102'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff2f40020d0 con 0x564ff1387e10 2024-09-10T21:31:29.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388730 con 0x564ff1384440 2024-09-10T21:31:29.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(114 100.00000000.inode [create] v102'69 uv61 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1388730 con 0x564ff1384440 2024-09-10T21:31:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(115 100.00000000.inode [delete] v102'70 uv70 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.120+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388c70 con 0x564ff1384440 2024-09-10T21:31:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.124+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(116 100.00000000.inode [create] v102'71 uv71 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.126 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:29.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.124+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388c70 con 0x564ff1384440 2024-09-10T21:31:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.124+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v102'72 uv72 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.124+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388bd0 con 0x564ff1384440 2024-09-10T21:31:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.124+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v102'73 uv73 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.124+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388bd0 con 0x564ff1384440 2024-09-10T21:31:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.128+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(119 1.00000000 [create] v102'74 uv68 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.128+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x564ff1388750 con 0x564ff1384440 2024-09-10T21:31:29.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.132+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(120 1.00000000 [delete] v102'75 uv75 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.132+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1388bd0 con 0x564ff1384440 2024-09-10T21:31:29.137 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:29.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(121 1.00000000 [create] v102'76 uv76 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1389c40 con 0x564ff1384440 2024-09-10T21:31:29.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v102'77 uv77 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1389c00 con 0x564ff1384440 2024-09-10T21:31:29.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v102'78 uv78 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff1389c00 con 0x564ff1384440 2024-09-10T21:31:29.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v102'79 uv79 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.136+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x564ff138a040 con 0x564ff1384440 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff307400640 1 -- 172.21.15.97:0/2360023043 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v102'80 uv80 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff2f8025020 con 0x564ff1384440 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x564ff137be40 msgr2=0x564ff137e230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x564ff137be40 0x564ff137e230 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564ff1384440 msgr2=0x564ff1386830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x564ff1384440 0x564ff1386830 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x564ff1387e10 msgr2=0x564ff13881f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x564ff1387e10 0x564ff13881f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x564ff1380290 msgr2=0x564ff13830f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x564ff1380290 0x564ff13830f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x564ff13771f0 msgr2=0x564ff13795e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x564ff13771f0 0x564ff13795e0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 reap_dead start 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x564ff123c530 msgr2=0x564ff123c910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x564ff123c530 0x564ff123c910 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 reap_dead start 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff307e00640 1 -- 172.21.15.97:0/2360023043 reap_dead start 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2e8070ff0 msgr2=0x7ff2e80734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2e8070ff0 0x7ff2e80734b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4001230 tx=0x7ff2f4001770 comp rx=0 tx=0).stop 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 msgr2=0x564ff12ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12ce090 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8004fd0 tx=0x7ff2f80156d0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 shutdown_connections 2024-09-10T21:31:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2e8070ff0 0x7ff2e80734b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 --2- 172.21.15.97:0/2360023043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ff1228940 0x564ff12ce090 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 >> 172.21.15.97:0/2360023043 conn(0x564ff1227520 msgr2=0x564ff11f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 shutdown_connections 2024-09-10T21:31:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.140+0000 7ff30a159080 1 -- 172.21.15.97:0/2360023043 wait complete. 2024-09-10T21:31:29.148 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2024-09-10T21:31:29.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 Processor -- start 2024-09-10T21:31:29.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 -- start start 2024-09-10T21:31:29.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b98003c90 0x556b50ee9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50f03dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b50e5c8a0 0x556b50e5cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556b50dd8f40 con 0x556b50efcfd0 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556b50de5d00 con 0x556b50e5c8a0 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb5847080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556b50d7aa00 con 0x7f6b98003c90 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50f03dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b98003c90 0x556b50ee9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50f03dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58252/0 (socket says 172.21.15.97:58252) 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 -- 172.21.15.97:0/1398980675 learned_addr learned my addr 172.21.15.97:0/1398980675 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb2800640 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b50e5c8a0 0x556b50e5cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 -- 172.21.15.97:0/1398980675 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b98003c90 msgr2=0x556b50ee9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b98003c90 0x556b50ee9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 -- 172.21.15.97:0/1398980675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b50e5c8a0 msgr2=0x556b50e5cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b50e5c8a0 0x556b50e5cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 -- 172.21.15.97:0/1398980675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b50f01090 con 0x556b50efcfd0 2024-09-10T21:31:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6bb1400640 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50f03dc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000d820 tx=0x7f6ba000dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cf857b6b4d6d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6babe00640 1 -- 172.21.15.97:0/1398980675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000b870 con 0x556b50efcfd0 2024-09-10T21:31:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6babe00640 1 -- 172.21.15.97:0/1398980675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ba0014020 con 0x556b50efcfd0 2024-09-10T21:31:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.232+0000 7f6babe00640 1 -- 172.21.15.97:0/1398980675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000ba70 con 0x556b50efcfd0 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- 172.21.15.97:0/1398980675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 msgr2=0x556b50f03dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50f03dc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000d820 tx=0x7f6ba000dce0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- 172.21.15.97:0/1398980675 shutdown_connections 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b50e5c8a0 0x556b50e5cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50f03dc0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 --2- 172.21.15.97:0/1398980675 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b98003c90 0x556b50ee9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- 172.21.15.97:0/1398980675 >> 172.21.15.97:0/1398980675 conn(0x556b50e5a520 msgr2=0x556b50edfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- 172.21.15.97:0/1398980675 shutdown_connections 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- 172.21.15.97:0/1398980675 wait complete. 2024-09-10T21:31:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 Processor -- start 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- start start 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556b50e5c8a0 0x556b50effe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50e5e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b98003c90 0x556b50e5e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556b50f01330 con 0x556b50efcfd0 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556b50e2c1e0 con 0x7f6b98003c90 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556b50efe7f0 con 0x556b50e5c8a0 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50e5e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50e5e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58260/0 (socket says 172.21.15.97:58260) 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 -- 172.21.15.97:0/2982345432 learned_addr learned my addr 172.21.15.97:0/2982345432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1e00640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556b50e5c8a0 0x556b50effe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb2800640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b98003c90 0x556b50e5e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 -- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556b50e5c8a0 msgr2=0x556b50effe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556b50e5c8a0 0x556b50effe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 -- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b98003c90 msgr2=0x556b50e5e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b98003c90 0x556b50e5e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 -- 172.21.15.97:0/2982345432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b50efea90 con 0x556b50efcfd0 2024-09-10T21:31:29.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb1400640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50e5e1e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000e010 tx=0x7f6ba0001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6baaa00640 1 -- 172.21.15.97:0/2982345432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0013040 con 0x556b50efcfd0 2024-09-10T21:31:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6baaa00640 1 -- 172.21.15.97:0/2982345432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ba001d020 con 0x556b50efcfd0 2024-09-10T21:31:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6baaa00640 1 -- 172.21.15.97:0/2982345432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000f9f0 con 0x556b50efcfd0 2024-09-10T21:31:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b50efed30 con 0x556b50efcfd0 2024-09-10T21:31:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.236+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b50efe3c0 con 0x556b50efcfd0 2024-09-10T21:31:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6baaa00640 1 -- 172.21.15.97:0/2982345432 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6ba000fb90 con 0x556b50efcfd0 2024-09-10T21:31:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6baaa00640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6baaa00640 1 -- 172.21.15.97:0/2982345432 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f6ba0093a30 con 0x556b50efcfd0 2024-09-10T21:31:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6bb1e00640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6bb5847080 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556b50e5f630 0x556b50fa04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x556b50fa0a30 con 0x556b50e5f630 2024-09-10T21:31:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6bb2800640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556b50e5f630 0x556b50fa04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6bb1e00640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4001250 tx=0x7f6ba4003040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.240+0000 7f6bb2800640 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556b50e5f630 0x556b50fa04f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb2800640 1 -- 172.21.15.97:0/2982345432 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v102'31 uv31 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6bac002030 con 0x556b50e5f630 2024-09-10T21:31:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556b50e5f630 msgr2=0x556b50fa04f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556b50e5f630 0x556b50fa04f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 msgr2=0x7f6b940734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b94070ff0 0x7f6b940734b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4001250 tx=0x7f6ba4003040 comp rx=0 tx=0).stop 2024-09-10T21:31:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 msgr2=0x556b50e5e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 --2- 172.21.15.97:0/2982345432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556b50efcfd0 0x556b50e5e1e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000e010 tx=0x7f6ba0001b50 comp rx=0 tx=0).stop 2024-09-10T21:31:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb2800640 1 -- 172.21.15.97:0/2982345432 reap_dead start 2024-09-10T21:31:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 shutdown_connections 2024-09-10T21:31:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 >> 172.21.15.97:0/2982345432 conn(0x556b50e5a520 msgr2=0x556b50f043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 shutdown_connections 2024-09-10T21:31:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.244+0000 7f6bb5847080 1 -- 172.21.15.97:0/2982345432 wait complete. 2024-09-10T21:31:29.250 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 -- 172.21.15.97:0/2208284380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40af8c0 msgr2=0x7fbea40cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 --2- 172.21.15.97:0/2208284380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40af8c0 0x7fbea40cd1a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009900 tx=0x7fbe9401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 -- 172.21.15.97:0/2208284380 shutdown_connections 2024-09-10T21:31:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 --2- 172.21.15.97:0/2208284380 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbea40cdb40 0x7fbea406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 --2- 172.21.15.97:0/2208284380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40af8c0 0x7fbea40cd1a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 -- 172.21.15.97:0/2208284380 >> 172.21.15.97:0/2208284380 conn(0x7fbea40b3c70 msgr2=0x7fbea40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 -- 172.21.15.97:0/2208284380 shutdown_connections 2024-09-10T21:31:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 -- 172.21.15.97:0/2208284380 wait complete. 2024-09-10T21:31:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 Processor -- start 2024-09-10T21:31:29.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.436+0000 7fbeab200640 1 -- start start 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbea40af8c0 0x7fbea4150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40cdb40 0x7fbea41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbea4162570 0x7fbea4156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbea416b260 con 0x7fbea40cdb40 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbea416b0e0 con 0x7fbea40af8c0 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbea416b3e0 con 0x7fbea4162570 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40cdb40 0x7fbea41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbea40af8c0 0x7fbea4150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40cdb40 0x7fbea41569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58266/0 (socket says 172.21.15.97:58266) 2024-09-10T21:31:29.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbea40af8c0 0x7fbea4150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:58570/0 (socket says 172.21.15.97:58570) 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 -- 172.21.15.97:0/4015117288 learned_addr learned my addr 172.21.15.97:0/4015117288 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea9200640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbea4162570 0x7fbea4156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 -- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbea4162570 msgr2=0x7fbea4156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbea4162570 0x7fbea4156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 -- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbea40af8c0 msgr2=0x7fbea4150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbea40af8c0 0x7fbea4150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 -- 172.21.15.97:0/4015117288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea4150c70 con 0x7fbea40cdb40 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea3e00640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbea40af8c0 0x7fbea4150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbe9b400640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40cdb40 0x7fbea41569f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c3c0 tx=0x7fbe8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea0c00640 1 -- 172.21.15.97:0/4015117288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0052d0 con 0x7fbea40cdb40 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea0c00640 1 -- 172.21.15.97:0/4015117288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe8c007020 con 0x7fbea40cdb40 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea4150f50 con 0x7fbea40cdb40 2024-09-10T21:31:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea0c00640 1 -- 172.21.15.97:0/4015117288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c007470 con 0x7fbea40cdb40 2024-09-10T21:31:29.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea4151510 con 0x7fbea40cdb40 2024-09-10T21:31:29.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea406cb40 con 0x7fbea40cdb40 2024-09-10T21:31:29.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.440+0000 7fbea0c00640 1 -- 172.21.15.97:0/4015117288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbe8c0067e0 con 0x7fbea40cdb40 2024-09-10T21:31:29.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.444+0000 7fbea0c00640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe800725e0 0x7fbe80074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.444+0000 7fbea0c00640 1 -- 172.21.15.97:0/4015117288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7fbe8c092a30 con 0x7fbea40cdb40 2024-09-10T21:31:29.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.444+0000 7fbea3e00640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe800725e0 0x7fbe80074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.444+0000 7fbea3e00640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe800725e0 0x7fbe80074aa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbe940af000 tx=0x7fbe9401b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.448+0000 7fbea0c00640 1 -- 172.21.15.97:0/4015117288 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbe8c001090 con 0x7fbea40cdb40 2024-09-10T21:31:29.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.644+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbea40d1080 con 0x7fbea40cdb40 2024-09-10T21:31:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.884+0000 7fbea0c00640 1 -- 172.21.15.97:0/4015117288 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v121) ==== 162+0+0 (secure 0 0 0) 0x7fbe8c05d670 con 0x7fbea40cdb40 2024-09-10T21:31:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:29.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe800725e0 msgr2=0x7fbe80074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe800725e0 0x7fbe80074aa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbe940af000 tx=0x7fbe9401b120 comp rx=0 tx=0).stop 2024-09-10T21:31:29.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40cdb40 msgr2=0x7fbea41569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40cdb40 0x7fbea41569f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c3c0 tx=0x7fbe8c00c880 comp rx=0 tx=0).stop 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 shutdown_connections 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbe800725e0 0x7fbe80074aa0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbea4162570 0x7fbea4156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbea40cdb40 0x7fbea41569f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 --2- 172.21.15.97:0/4015117288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbea40af8c0 0x7fbea4150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 >> 172.21.15.97:0/4015117288 conn(0x7fbea40b3c70 msgr2=0x7fbea415d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 shutdown_connections 2024-09-10T21:31:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:29.888+0000 7fbeab200640 1 -- 172.21.15.97:0/4015117288 wait complete. 2024-09-10T21:31:29.895 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:31:29.892+0000 7f1bb7c00640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-10T21:31:30.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:30.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- 172.21.15.97:0/2032456025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac15ac30 msgr2=0x7f81ac0b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 --2- 172.21.15.97:0/2032456025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac15ac30 0x7f81ac0b5d20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0cef60 tx=0x7f819c01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- 172.21.15.97:0/2032456025 shutdown_connections 2024-09-10T21:31:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 --2- 172.21.15.97:0/2032456025 >> v2:172.21.15.194:3300/0 conn(0x7f81ac074b20 0x7f81ac0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 --2- 172.21.15.97:0/2032456025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac15ac30 0x7f81ac0b5d20 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- 172.21.15.97:0/2032456025 >> 172.21.15.97:0/2032456025 conn(0x7f81ac0b3c70 msgr2=0x7f81ac0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- 172.21.15.97:0/2032456025 shutdown_connections 2024-09-10T21:31:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- 172.21.15.97:0/2032456025 wait complete. 2024-09-10T21:31:30.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 Processor -- start 2024-09-10T21:31:30.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- start start 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f81ac0691b0 0x7f81ac1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac074b20 0x7f81ac09da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81ac15ac30 0x7f81ac1f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac06dd60 con 0x7f81ac074b20 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac0cff50 con 0x7f81ac15ac30 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b2a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f81ac06dee0 con 0x7f81ac0691b0 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac074b20 0x7f81ac09da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac074b20 0x7f81ac09da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58282/0 (socket says 172.21.15.97:58282) 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 -- 172.21.15.97:0/3297319984 learned_addr learned my addr 172.21.15.97:0/3297319984 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81abe00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f81ac0691b0 0x7f81ac1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81b0a00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81ac15ac30 0x7f81ac1f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 -- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f81ac0691b0 msgr2=0x7f81ac1f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f81ac0691b0 0x7f81ac1f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 -- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81ac15ac30 msgr2=0x7f81ac1f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81ac15ac30 0x7f81ac1f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 -- 172.21.15.97:0/3297319984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac09b260 con 0x7f81ac074b20 2024-09-10T21:31:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.460+0000 7f81a3400640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac074b20 0x7f81ac09da10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f819400d9b0 tx=0x7f819400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81a8c00640 1 -- 172.21.15.97:0/3297319984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194017070 con 0x7f81ac074b20 2024-09-10T21:31:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81a8c00640 1 -- 172.21.15.97:0/3297319984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8194009cc0 con 0x7f81ac074b20 2024-09-10T21:31:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81a8c00640 1 -- 172.21.15.97:0/3297319984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194005280 con 0x7f81ac074b20 2024-09-10T21:31:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac09b540 con 0x7f81ac074b20 2024-09-10T21:31:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac09bb00 con 0x7f81ac074b20 2024-09-10T21:31:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac06e360 con 0x7f81ac074b20 2024-09-10T21:31:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81a8c00640 1 -- 172.21.15.97:0/3297319984 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8194005420 con 0x7f81ac074b20 2024-09-10T21:31:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81a8c00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81880725e0 0x7f8188074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81a8c00640 1 -- 172.21.15.97:0/3297319984 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f8194093930 con 0x7f81ac074b20 2024-09-10T21:31:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.464+0000 7f81abe00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81880725e0 0x7f8188074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.468+0000 7f81abe00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81880725e0 0x7f8188074aa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f819c0b2000 tx=0x7f819c01d8f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.468+0000 7f81a8c00640 1 -- 172.21.15.97:0/3297319984 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f819405e4e0 con 0x7f81ac074b20 2024-09-10T21:31:30.609 INFO:tasks.ceph.mds.c.smithi097.stderr: -41> 2024-09-10T21:31:25.852+0000 7fba89a00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T21:31:30.647 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:31:30.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.672+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81ac074f40 con 0x7f81ac074b20 2024-09-10T21:31:30.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.672+0000 7f81a8c00640 1 -- 172.21.15.97:0/3297319984 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+4404 (secure 0 0 0) 0x7f8194067250 con 0x7f81ac074b20 2024-09-10T21:31:30.676 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:30.676 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":122,"btime":"2024-09-10T21:31:29:891350+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:29.891341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7917":{"gid":7917,"name":"a","rank":0,"incarnation":122,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/2013408839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2013408839},{"type":"v1","addr":"172.21.15.97:6834","nonce":2013408839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:30.677 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 122 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81880725e0 msgr2=0x7f8188074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81880725e0 0x7f8188074aa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f819c0b2000 tx=0x7f819c01d8f0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac074b20 msgr2=0x7f81ac09da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac074b20 0x7f81ac09da10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f819400d9b0 tx=0x7f819400de70 comp rx=0 tx=0).stop 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 shutdown_connections 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81880725e0 0x7f8188074aa0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81ac15ac30 0x7f81ac1f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81ac074b20 0x7f81ac09da10 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 --2- 172.21.15.97:0/3297319984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f81ac0691b0 0x7f81ac1f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 >> 172.21.15.97:0/3297319984 conn(0x7f81ac0b3c70 msgr2=0x7f81ac0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 shutdown_connections 2024-09-10T21:31:30.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:30.676+0000 7f81b2a00640 1 -- 172.21.15.97:0/3297319984 wait complete. 2024-09-10T21:31:30.698 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 122, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:31:29.891341+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7917}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7917': {'gid': 7917, 'name': 'a', 'rank': 0, 'incarnation': 122, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6834/2013408839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 2013408839}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 2013408839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:31:30.698 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7917:up:replay 2024-09-10T21:31:30.698 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:31:36.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:36.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 -- 172.21.15.97:0/3341246169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 msgr2=0x7f41240d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 --2- 172.21.15.97:0/3341246169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41240d1f80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4118009930 tx=0x7f411801c850 comp rx=0 tx=0).stop 2024-09-10T21:31:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 -- 172.21.15.97:0/3341246169 shutdown_connections 2024-09-10T21:31:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 --2- 172.21.15.97:0/3341246169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f412415ee70 0x7f41240d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 --2- 172.21.15.97:0/3341246169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41240d1f80 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 -- 172.21.15.97:0/3341246169 >> 172.21.15.97:0/3341246169 conn(0x7f41240d3e70 msgr2=0x7f41240d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 -- 172.21.15.97:0/3341246169 shutdown_connections 2024-09-10T21:31:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.252+0000 7f412c400640 1 -- 172.21.15.97:0/3341246169 wait complete. 2024-09-10T21:31:36.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 Processor -- start 2024-09-10T21:31:36.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 -- start start 2024-09-10T21:31:36.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41240b3280 0x7f41241523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41241505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f412415ee70 0x7f412415d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f412416b160 con 0x7f412415b9c0 2024-09-10T21:31:36.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f412416afe0 con 0x7f412415ee70 2024-09-10T21:31:36.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f412416b2e0 con 0x7f41240b3280 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41241505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41241505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58304/0 (socket says 172.21.15.97:58304) 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 -- 172.21.15.97:0/1034423406 learned_addr learned my addr 172.21.15.97:0/1034423406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129a00640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41240b3280 0x7f41241523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412a400640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f412415ee70 0x7f412415d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 -- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41240b3280 msgr2=0x7f41241523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41240b3280 0x7f41241523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 -- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f412415ee70 msgr2=0x7f412415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f412415ee70 0x7f412415d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 -- 172.21.15.97:0/1034423406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412415c1e0 con 0x7f412415b9c0 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129a00640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41240b3280 0x7f41241523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4129000640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41241505a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f411000c490 tx=0x7f411000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4121600640 1 -- 172.21.15.97:0/1034423406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110017070 con 0x7f412415b9c0 2024-09-10T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4121600640 1 -- 172.21.15.97:0/1034423406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41100053e0 con 0x7f412415b9c0 2024-09-10T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412415c4c0 con 0x7f412415b9c0 2024-09-10T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f4121600640 1 -- 172.21.15.97:0/1034423406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110003860 con 0x7f412415b9c0 2024-09-10T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.256+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412415cfa0 con 0x7f412415b9c0 2024-09-10T21:31:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.260+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4124161a30 con 0x7f412415b9c0 2024-09-10T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.260+0000 7f4121600640 1 -- 172.21.15.97:0/1034423406 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4110003a00 con 0x7f412415b9c0 2024-09-10T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.260+0000 7f4121600640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41040725e0 0x7f4104074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.260+0000 7f4121600640 1 -- 172.21.15.97:0/1034423406 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f4110092dc0 con 0x7f412415b9c0 2024-09-10T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.260+0000 7f4129a00640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41040725e0 0x7f4104074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.260+0000 7f4129a00640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41040725e0 0x7f4104074aa0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f41180af000 tx=0x7f411801b090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.264+0000 7f4121600640 1 -- 172.21.15.97:0/1034423406 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f411005da30 con 0x7f412415b9c0 2024-09-10T21:31:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.464+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f412408db10 con 0x7f412415b9c0 2024-09-10T21:31:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.464+0000 7f4121600640 1 -- 172.21.15.97:0/1034423406 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5237 (secure 0 0 0) 0x7f4110062460 con 0x7f412415b9c0 2024-09-10T21:31:36.469 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:36.469 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":125,"btime":"2024-09-10T21:31:32:904403+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2841033666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2841033666},{"type":"v1","addr":"172.21.15.97:6835","nonce":2841033666}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:32.904399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7917":{"gid":7917,"name":"a","rank":0,"incarnation":122,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6834/2013408839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2013408839},{"type":"v1","addr":"172.21.15.97:6834","nonce":2013408839}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7917,"qdb_cluster":[7917]},"id":6}]} 2024-09-10T21:31:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 125 2024-09-10T21:31:36.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41040725e0 msgr2=0x7f4104074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41040725e0 0x7f4104074aa0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f41180af000 tx=0x7f411801b090 comp rx=0 tx=0).stop 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 msgr2=0x7f41241505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41241505a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f411000c490 tx=0x7f411000c950 comp rx=0 tx=0).stop 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 shutdown_connections 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f41040725e0 0x7f4104074aa0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f412415ee70 0x7f412415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412415b9c0 0x7f41241505a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 --2- 172.21.15.97:0/1034423406 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f41240b3280 0x7f41241523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 >> 172.21.15.97:0/1034423406 conn(0x7f41240d3e70 msgr2=0x7f4124153b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 shutdown_connections 2024-09-10T21:31:36.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.468+0000 7f412c400640 1 -- 172.21.15.97:0/1034423406 wait complete. 2024-09-10T21:31:36.491 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:31:32.904399+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7917}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7917': {'gid': 7917, 'name': 'a', 'rank': 0, 'incarnation': 122, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6834/2013408839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 2013408839}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 2013408839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7917, 'qdb_cluster': [7917]} 2024-09-10T21:31:36.491 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:31:36.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:31:36.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.628+0000 7f34b6800640 1 -- 172.21.15.97:0/3351914728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00cec10 msgr2=0x7f34b00cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.628+0000 7f34b6800640 1 --2- 172.21.15.97:0/3351914728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00cec10 0x7f34b00cfd40 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f34b00735a0 tx=0x7f349801eff0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- 172.21.15.97:0/3351914728 shutdown_connections 2024-09-10T21:31:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 --2- 172.21.15.97:0/3351914728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00cec10 0x7f34b00cfd40 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 --2- 172.21.15.97:0/3351914728 >> v2:172.21.15.194:3300/0 conn(0x7f34b00ce830 0x7f34b00cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- 172.21.15.97:0/3351914728 >> 172.21.15.97:0/3351914728 conn(0x7f34b00b37c0 msgr2=0x7f34b00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- 172.21.15.97:0/3351914728 shutdown_connections 2024-09-10T21:31:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- 172.21.15.97:0/3351914728 wait complete. 2024-09-10T21:31:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 Processor -- start 2024-09-10T21:31:36.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- start start 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00ce830 0x7f34b0154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b00cec10 0x7f34b00b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00d0280 0x7f34b00b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00b4450 con 0x7f34b00d0280 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00cde60 con 0x7f34b00cec10 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34b6800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f34b00ce1a0 con 0x7f34b00ce830 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00d0280 0x7f34b00b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00d0280 0x7f34b00b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58322/0 (socket says 172.21.15.97:58322) 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 -- 172.21.15.97:0/4195009021 learned_addr learned my addr 172.21.15.97:0/4195009021 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34af400640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00ce830 0x7f34b0154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34a6a00640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b00cec10 0x7f34b00b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 -- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00ce830 msgr2=0x7f34b0154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00ce830 0x7f34b0154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 -- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b00cec10 msgr2=0x7f34b00b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b00cec10 0x7f34b00b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 -- 172.21.15.97:0/4195009021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b006ed10 con 0x7f34b00d0280 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34a6a00640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b00cec10 0x7f34b00b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.632+0000 7f34afe00640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00d0280 0x7f34b00b4df0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f34a800ca20 tx=0x7f34a800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34a7e00640 1 -- 172.21.15.97:0/4195009021 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8016020 con 0x7f34b00d0280 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34a7e00640 1 -- 172.21.15.97:0/4195009021 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34a800a950 con 0x7f34b00d0280 2024-09-10T21:31:36.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34a7e00640 1 -- 172.21.15.97:0/4195009021 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800ada0 con 0x7f34b00d0280 2024-09-10T21:31:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b006eff0 con 0x7f34b00d0280 2024-09-10T21:31:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b00c6950 con 0x7f34b00d0280 2024-09-10T21:31:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34a7e00640 1 -- 172.21.15.97:0/4195009021 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f34a8012420 con 0x7f34b00d0280 2024-09-10T21:31:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34a7e00640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f348c072610 0x7f348c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34a7e00640 1 -- 172.21.15.97:0/4195009021 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f34a8093a90 con 0x7f34b00d0280 2024-09-10T21:31:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34b9b99000 1 Processor -- start 2024-09-10T21:31:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34b9b99000 1 -- start start 2024-09-10T21:31:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.636+0000 7f34af400640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f348c072610 0x7f348c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34af400640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f348c072610 0x7f348c074ad0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f34a000bc00 tx=0x7f34a000b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913db1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555913c863f0 con 0x555913db22f0 2024-09-10T21:31:36.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555913bc3b20 con 0x555913db19d0 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555913bb2f50 con 0x555913d65e10 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58336/0 (socket says 172.21.15.97:58336) 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 -- 172.21.15.97:0/764044938 learned_addr learned my addr 172.21.15.97:0/764044938 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34af400640 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913db1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34afe00640 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 -- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 msgr2=0x555913d661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 -- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 msgr2=0x555913db1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913db1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 -- 172.21.15.97:0/764044938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555913b7a590 con 0x555913db22f0 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34a6a00640 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d658d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f349801cc20 tx=0x7f349801bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d572738d4454a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f349fe00640 1 -- 172.21.15.97:0/764044938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980b5040 con 0x555913db22f0 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f349fe00640 1 -- 172.21.15.97:0/764044938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3498004240 con 0x555913db22f0 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f349fe00640 1 -- 172.21.15.97:0/764044938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498005040 con 0x555913db22f0 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- 172.21.15.97:0/764044938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 msgr2=0x555913d658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d658d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f349801cc20 tx=0x7f349801bd90 comp rx=0 tx=0).stop 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- 172.21.15.97:0/764044938 shutdown_connections 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d658d0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 --2- 172.21.15.97:0/764044938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913db1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- 172.21.15.97:0/764044938 >> 172.21.15.97:0/764044938 conn(0x555913c4f130 msgr2=0x555913d94e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- 172.21.15.97:0/764044938 shutdown_connections 2024-09-10T21:31:36.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- 172.21.15.97:0/764044938 wait complete. 2024-09-10T21:31:36.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 -- start start 2024-09-10T21:31:36.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.640+0000 7f34b9b99000 1 Processor -- start 2024-09-10T21:31:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555913b4d5d0 con 0x555913db22f0 2024-09-10T21:31:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555913bb6d60 con 0x555913db19d0 2024-09-10T21:31:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555913b59500 con 0x555913d65e10 2024-09-10T21:31:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 10 client.0 _ll_register_callbacks cb 0x555913b5d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34af400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d933b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58344/0 (socket says 172.21.15.97:58344) 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 -- 172.21.15.97:0/739260254 learned_addr learned my addr 172.21.15.97:0/739260254 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34a6a00640 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 -- 172.21.15.97:0/739260254 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 msgr2=0x555913d926a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d926a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 -- 172.21.15.97:0/739260254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 msgr2=0x555913d92e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d92e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 -- 172.21.15.97:0/739260254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555913b51170 con 0x555913db22f0 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34afe00640 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d933b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f34a800a430 tx=0x7f34a805de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64323950aa05037 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:31:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 1 -- 172.21.15.97:0/739260254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a805e620 con 0x555913db22f0 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555913b4b0f0 con 0x555913db22f0 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 1 -- 172.21.15.97:0/739260254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f34a8065310 con 0x555913db22f0 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 1 -- 172.21.15.97:0/739260254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) ==== 24+0+0 (secure 0 0 0) 0x7f34a8063100 con 0x555913db22f0 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 10 client.7995 fetch_fsmap learned FSMap version 125 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) -- 0x555913c37ef0 con 0x555913db22f0 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 1 -- 172.21.15.97:0/739260254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555913b4b0f0 con 0x555913db22f0 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 1 -- 172.21.15.97:0/739260254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 125) ==== 3502+0+0 (secure 0 0 0) 0x7f34a8078080 con 0x555913db22f0 2024-09-10T21:31:36.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 10 client.7995 fetch_fsmap finished waiting for FSMap version 125 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 10 client.7995 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 20 client.7995 populate_metadata read hostname 'smithi097' 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913d940f0 0x555913dc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 4 client.7995 mds_command: new command op to 7917 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 --> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555913bc5a70 con 0x555913d940f0 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34af400640 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913d940f0 0x555913dc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34af400640 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913d940f0 0x555913dc3c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc185da616fab7e2 server_cookie=b04a91671df87ecb in_seq=0 out_seq=0 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 10 client.7995 ms_handle_connect on v2:172.21.15.97:6832/2013408839 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 1 -- 172.21.15.97:0/739260254 <== mds.0 v2:172.21.15.97:6832/2013408839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555913bc5a70 con 0x555913d940f0 2024-09-10T21:31:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f349cc00640 10 client.7995 handle_command_reply: tid=0 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.644+0000 7f34b9b99000 1 client.7995 shutdown 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 msgr2=0x555913d933b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d933b0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f34a800a430 tx=0x7f34a805de00 comp rx=0 tx=0).stop 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 shutdown_connections 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913d940f0 0x555913dc3c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db22f0 0x555913d933b0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d92e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f349cc00640 0 client.7995 ms_handle_reset on v2:172.21.15.97:6832/2013408839 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 --2- 172.21.15.97:0/739260254 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913d926a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 >> 172.21.15.97:0/739260254 conn(0x555913c4f130 msgr2=0x555913d64fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 shutdown_connections 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 1 -- 172.21.15.97:0/739260254 wait complete. 2024-09-10T21:31:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.648+0000 7f34b9b99000 20 client.7995 trim_cache size 0 max 16384 2024-09-10T21:31:36.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 Processor -- start 2024-09-10T21:31:36.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 -- start start 2024-09-10T21:31:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913d65e10 0x555913d652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d65800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913db22f0 0x555913dc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0201c80 con 0x555913d65e10 2024-09-10T21:31:36.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0200ca0 con 0x555913db19d0 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f34b0200080 con 0x555913db22f0 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913d65e10 0x555913d652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913d65e10 0x555913d652c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58356/0 (socket says 172.21.15.97:58356) 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.664+0000 7f34af400640 1 -- 172.21.15.97:0/3851420013 learned_addr learned my addr 172.21.15.97:0/3851420013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34afe00640 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913db22f0 0x555913dc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d65800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34af400640 1 -- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913db22f0 msgr2=0x555913dc3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34af400640 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913db22f0 0x555913dc3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34af400640 1 -- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 msgr2=0x555913d65800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34af400640 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d65800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34af400640 1 -- 172.21.15.97:0/3851420013 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555913c0ed90 con 0x555913d65e10 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34afe00640 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913db22f0 0x555913dc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34af400640 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913d65e10 0x555913d652c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f34a0011550 tx=0x7f34a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970356cadc635e20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f349f400640 1 -- 172.21.15.97:0/3851420013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00201d0 con 0x555913d65e10 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f349f400640 1 -- 172.21.15.97:0/3851420013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34a0020500 con 0x555913d65e10 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f349f400640 1 -- 172.21.15.97:0/3851420013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0020970 con 0x555913d65e10 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- 172.21.15.97:0/3851420013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913d65e10 msgr2=0x555913d652c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913d65e10 0x555913d652c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f34a0011550 tx=0x7f34a0013000 comp rx=0 tx=0).stop 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- 172.21.15.97:0/3851420013 shutdown_connections 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913db22f0 0x555913dc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db19d0 0x555913d65800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3851420013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913d65e10 0x555913d652c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- 172.21.15.97:0/3851420013 >> 172.21.15.97:0/3851420013 conn(0x555913c4f130 msgr2=0x555913db05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- 172.21.15.97:0/3851420013 shutdown_connections 2024-09-10T21:31:36.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- 172.21.15.97:0/3851420013 wait complete. 2024-09-10T21:31:36.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- start start 2024-09-10T21:31:36.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 Processor -- start 2024-09-10T21:31:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913e275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db19d0 0x555913e27b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db22f0 0x555913e28050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555913bbb390 con 0x555913db19d0 2024-09-10T21:31:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555913bc28f0 con 0x555913db22f0 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555913c37a60 con 0x555913d65e10 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34b9b99000 10 client.0 _ll_register_callbacks cb 0x7f34a805e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db19d0 0x555913e27b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db19d0 0x555913e27b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58362/0 (socket says 172.21.15.97:58362) 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 -- 172.21.15.97:0/3391974604 learned_addr learned my addr 172.21.15.97:0/3391974604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34af400640 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913e275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34afe00640 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db22f0 0x555913e28050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 -- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 msgr2=0x555913e275d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913e275d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 -- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db22f0 msgr2=0x555913e28050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db22f0 0x555913e28050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.668+0000 7f34a6a00640 1 -- 172.21.15.97:0/3391974604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555913b79a90 con 0x555913db19d0 2024-09-10T21:31:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34a6a00640 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db19d0 0x555913e27b10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3498009900 tx=0x7f3498004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da2612828a935da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 1 -- 172.21.15.97:0/3391974604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980b5070 con 0x555913db19d0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 1 -- 172.21.15.97:0/3391974604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f3498005230 con 0x555913db19d0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a0020500 con 0x555913db19d0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 1 -- 172.21.15.97:0/3391974604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) ==== 24+0+0 (secure 0 0 0) 0x7f3498009460 con 0x555913db19d0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 10 client.8001 fetch_fsmap learned FSMap version 125 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) -- 0x555913b58050 con 0x555913db19d0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 1 -- 172.21.15.97:0/3391974604 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34a0020500 con 0x555913db19d0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 1 -- 172.21.15.97:0/3391974604 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 125) ==== 3502+0+0 (secure 0 0 0) 0x7f3498005430 con 0x555913db19d0 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 10 client.8001 fetch_fsmap finished waiting for FSMap version 125 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 10 client.8001 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 20 client.8001 populate_metadata read hostname 'smithi097' 2024-09-10T21:31:36.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913dc6960 0x555913dc6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 4 client.8001 mds_command: new command op to 7917 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 --> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] -- command(tid 0: {"prefix": "status"}) -- 0x555913bc1520 con 0x555913dc6960 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34af400640 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913dc6960 0x555913dc6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34af400640 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913dc6960 0x555913dc6d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9a59769e672a10 server_cookie=e19b0d680eb1d19d in_seq=0 out_seq=0 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 10 client.8001 ms_handle_connect on v2:172.21.15.97:6832/2013408839 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 1 -- 172.21.15.97:0/3391974604 <== mds.0 v2:172.21.15.97:6832/2013408839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x555913bc1520 con 0x555913dc6960 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 10 client.8001 handle_command_reply: tid=0 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 client.8001 shutdown 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db19d0 msgr2=0x555913e27b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db19d0 0x555913e27b10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3498009900 tx=0x7f3498004510 comp rx=0 tx=0).stop 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 shutdown_connections 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:6832/2013408839,v1:172.21.15.97:6834/2013408839] conn(0x555913dc6960 0x555913dc6d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555913db22f0 0x555913e28050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555913db19d0 0x555913e27b10 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 --2- 172.21.15.97:0/3391974604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555913d65e10 0x555913e275d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f349cc00640 0 client.8001 ms_handle_reset on v2:172.21.15.97:6832/2013408839 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 >> 172.21.15.97:0/3391974604 conn(0x555913c4f130 msgr2=0x555913db0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 shutdown_connections 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 1 -- 172.21.15.97:0/3391974604 wait complete. 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b9b99000 20 client.8001 trim_cache size 0 max 16384 2024-09-10T21:31:36.681 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 7917, 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7845446660000004, 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 125, 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 102, 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 102, 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.260347143000001 2024-09-10T21:31:36.682 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:31:36.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f348c072610 msgr2=0x7f348c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f348c072610 0x7f348c074ad0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f34a000bc00 tx=0x7f34a000b390 comp rx=0 tx=0).stop 2024-09-10T21:31:36.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00d0280 msgr2=0x7f34b00b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00d0280 0x7f34b00b4df0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f34a800ca20 tx=0x7f34a800cb10 comp rx=0 tx=0).stop 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 shutdown_connections 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f348c072610 0x7f348c074ad0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34b00d0280 0x7f34b00b4df0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34b00cec10 0x7f34b00b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 --2- 172.21.15.97:0/4195009021 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34b00ce830 0x7f34b0154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 >> 172.21.15.97:0/4195009021 conn(0x7f34b00b37c0 msgr2=0x7f34b00b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 shutdown_connections 2024-09-10T21:31:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.672+0000 7f34b6800640 1 -- 172.21.15.97:0/4195009021 wait complete. 2024-09-10T21:31:36.695 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:31:36.695 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2024-09-10T21:31:36.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:31:36.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 -- 172.21.15.97:0/1018509963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d00af520 msgr2=0x7f94d0160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 --2- 172.21.15.97:0/1018509963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d00af520 0x7f94d0160220 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009900 tx=0x7f94bc01c850 comp rx=0 tx=0).stop 2024-09-10T21:31:36.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 -- 172.21.15.97:0/1018509963 shutdown_connections 2024-09-10T21:31:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 --2- 172.21.15.97:0/1018509963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94d006cb40 0x7f94d00d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 --2- 172.21.15.97:0/1018509963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d00af520 0x7f94d0160220 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 -- 172.21.15.97:0/1018509963 >> 172.21.15.97:0/1018509963 conn(0x7f94d00b3c70 msgr2=0x7f94d00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 -- 172.21.15.97:0/1018509963 shutdown_connections 2024-09-10T21:31:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.832+0000 7f94d8400640 1 -- 172.21.15.97:0/1018509963 wait complete. 2024-09-10T21:31:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 Processor -- start 2024-09-10T21:31:36.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 -- start start 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d006cb40 0x7f94d014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94d00af520 0x7f94d014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94d015f120 0x7f94d015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94d01694e0 con 0x7f94d006cb40 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94d0169360 con 0x7f94d00af520 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f94d0169660 con 0x7f94d015f120 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d006cb40 0x7f94d014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d006cb40 0x7f94d014b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58390/0 (socket says 172.21.15.97:58390) 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 -- 172.21.15.97:0/515945249 learned_addr learned my addr 172.21.15.97:0/515945249 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:36.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5000640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94d00af520 0x7f94d014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d6400640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94d015f120 0x7f94d015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 -- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94d015f120 msgr2=0x7f94d015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94d015f120 0x7f94d015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 -- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94d00af520 msgr2=0x7f94d014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94d00af520 0x7f94d014f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 -- 172.21.15.97:0/515945249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d015c7c0 con 0x7f94d006cb40 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d5a00640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d006cb40 0x7f94d014b3c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c3c0 tx=0x7f94c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94cd600640 1 -- 172.21.15.97:0/515945249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4017070 con 0x7f94d006cb40 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94cd600640 1 -- 172.21.15.97:0/515945249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94c40053c0 con 0x7f94d006cb40 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94cd600640 1 -- 172.21.15.97:0/515945249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4003860 con 0x7f94d006cb40 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d015caa0 con 0x7f94d006cb40 2024-09-10T21:31:36.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.836+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d015d060 con 0x7f94d006cb40 2024-09-10T21:31:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.840+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d00d0170 con 0x7f94d006cb40 2024-09-10T21:31:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.840+0000 7f94cd600640 1 -- 172.21.15.97:0/515945249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f94c4003a00 con 0x7f94d006cb40 2024-09-10T21:31:36.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.840+0000 7f94cd600640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94b0072610 0x7f94b0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:36.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.840+0000 7f94cd600640 1 -- 172.21.15.97:0/515945249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f94c4092ba0 con 0x7f94d006cb40 2024-09-10T21:31:36.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.840+0000 7f94d5000640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94b0072610 0x7f94b0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:36.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.840+0000 7f94d5000640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94b0072610 0x7f94b0074ad0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0af000 tx=0x7f94bc01b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:36.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:36.844+0000 7f94cd600640 1 -- 172.21.15.97:0/515945249 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f94d00cea30 con 0x7f94d006cb40 2024-09-10T21:31:36.871 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:31:36.868+0000 7f1bbcc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.044+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f94d00b5fe0 con 0x7f94d006cb40 2024-09-10T21:31:37.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.044+0000 7f94cd600640 1 -- 172.21.15.97:0/515945249 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v125) ==== 53+0+73 (secure 0 0 0) 0x7f94c405d7e0 con 0x7f94d006cb40 2024-09-10T21:31:37.049 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:31:37.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94b0072610 msgr2=0x7f94b0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:37.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94b0072610 0x7f94b0074ad0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0af000 tx=0x7f94bc01b0f0 comp rx=0 tx=0).stop 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d006cb40 msgr2=0x7f94d014b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d006cb40 0x7f94d014b3c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c3c0 tx=0x7f94c400c880 comp rx=0 tx=0).stop 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 shutdown_connections 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f94b0072610 0x7f94b0074ad0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f94d015f120 0x7f94d015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94d00af520 0x7f94d014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 --2- 172.21.15.97:0/515945249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94d006cb40 0x7f94d014b3c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 >> 172.21.15.97:0/515945249 conn(0x7f94d00b3c70 msgr2=0x7f94d0070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 shutdown_connections 2024-09-10T21:31:37.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37.048+0000 7f94d8400640 1 -- 172.21.15.97:0/515945249 wait complete. 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:31:37.070 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:37.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.071 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:31:37.080 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:31:37.080 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:31:37.080 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:37.081 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41996sec preferred_lft 41996sec 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:31:37.082 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:31:37.083 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:31:37.084 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:37.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.084 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:31:37.138 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:31:37.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.140 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:31:37.191 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:31:37.191 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:37.192 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:31:37.193 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:31:37.193 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:31:37.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:31:37.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:31:37.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:31:37.193 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:31:37.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:31:37.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:31:37.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.330 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:31:37.385 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:31:37.385 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:31:37.386 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:37.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.386 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:31:37.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:31:37.453 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:37.505 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:37.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:31:37.505 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:37.560 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:31:37.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:37.648 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:37.648 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:37.648 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:37.663 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:31:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:31:37.672 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:31:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:31:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:31:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:31:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:31:37.676 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:31:37.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.677 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:37.696 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:31:37.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:31:37.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:31:37.793 INFO:teuthology.orchestra.run.smithi097.stdout:client6818 2024-09-10T21:31:37.793 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:37.793 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:37.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:37.802 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:31:37.848 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:31:37.855 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:31:37.855 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:31:37.855 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:37.855 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:31:37.856 INFO:teuthology.orchestra.run.smithi097.stdout:sixmegs 2024-09-10T21:31:37.856 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2024-09-10T21:31:37.856 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2024-09-10T21:31:37.857 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:37.857 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:37.909 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:37.909 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:37.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:31:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 -- 172.21.15.97:0/1119072235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce40ceee0 msgr2=0x7f8ce40cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 --2- 172.21.15.97:0/1119072235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce40ceee0 0x7f8ce40cf390 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009c80 tx=0x7f8cdc021470 comp rx=0 tx=0).stop 2024-09-10T21:31:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 -- 172.21.15.97:0/1119072235 shutdown_connections 2024-09-10T21:31:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 --2- 172.21.15.97:0/1119072235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce40ceee0 0x7f8ce40cf390 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 --2- 172.21.15.97:0/1119072235 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ce41511f0 0x7f8ce40b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 -- 172.21.15.97:0/1119072235 >> 172.21.15.97:0/1119072235 conn(0x7f8ce40b1780 msgr2=0x7f8ce40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 -- 172.21.15.97:0/1119072235 shutdown_connections 2024-09-10T21:31:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 -- 172.21.15.97:0/1119072235 wait complete. 2024-09-10T21:31:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.144+0000 7f8ceba00640 1 Processor -- start 2024-09-10T21:31:38.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 -- start start 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ce40b3cf0 0x7f8ce4147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ce40ceee0 0x7f8ce414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce41511f0 0x7f8ce414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce406e740 con 0x7f8ce41511f0 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce406e5c0 con 0x7f8ce40ceee0 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce406e8c0 con 0x7f8ce40b3cf0 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce41511f0 0x7f8ce414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce41511f0 0x7f8ce414e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56892/0 (socket says 172.21.15.97:56892) 2024-09-10T21:31:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 -- 172.21.15.97:0/4098825448 learned_addr learned my addr 172.21.15.97:0/4098825448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9000640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ce40b3cf0 0x7f8ce4147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8cdbe00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ce40ceee0 0x7f8ce414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 -- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ce40b3cf0 msgr2=0x7f8ce4147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ce40b3cf0 0x7f8ce4147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 -- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ce40ceee0 msgr2=0x7f8ce414de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ce40ceee0 0x7f8ce414de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 -- 172.21.15.97:0/4098825448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce414cd70 con 0x7f8ce41511f0 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce9a00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce41511f0 0x7f8ce414e350 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc01fb20 tx=0x7f8cdc0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce1600640 1 -- 172.21.15.97:0/4098825448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0bf070 con 0x7f8ce41511f0 2024-09-10T21:31:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce1600640 1 -- 172.21.15.97:0/4098825448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cdc003c70 con 0x7f8ce41511f0 2024-09-10T21:31:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ce1600640 1 -- 172.21.15.97:0/4098825448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0b6040 con 0x7f8ce41511f0 2024-09-10T21:31:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce414d050 con 0x7f8ce41511f0 2024-09-10T21:31:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce414d610 con 0x7f8ce41511f0 2024-09-10T21:31:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.148+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce40cf390 con 0x7f8ce41511f0 2024-09-10T21:31:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.152+0000 7f8ce1600640 1 -- 172.21.15.97:0/4098825448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8cdc0b9030 con 0x7f8ce41511f0 2024-09-10T21:31:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.152+0000 7f8ce1600640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8cc00725e0 0x7f8cc0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.152+0000 7f8ce1600640 1 -- 172.21.15.97:0/4098825448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f8cdc130d50 con 0x7f8ce41511f0 2024-09-10T21:31:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.152+0000 7f8ce9000640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8cc00725e0 0x7f8cc0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.152+0000 7f8ce9000640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8cc00725e0 0x7f8cc0074aa0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4159980 tx=0x7f8cd40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:38.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.156+0000 7f8ce1600640 1 -- 172.21.15.97:0/4098825448 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8cdc01afd0 con 0x7f8ce41511f0 2024-09-10T21:31:38.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.332+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8ce415c050 con 0x7f8ce41511f0 2024-09-10T21:31:38.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.332+0000 7f8ce1600640 1 -- 172.21.15.97:0/4098825448 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v102) ==== 81+0+726 (secure 0 0 0) 0x7f8cdc0fb900 con 0x7f8ce41511f0 2024-09-10T21:31:38.337 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:31:38.337 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:31:38.337 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:31:38.337 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:31:38.337 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:31:38.338 INFO:teuthology.orchestra.run.smithi097.stderr:listed 12 entries 2024-09-10T21:31:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8cc00725e0 msgr2=0x7f8cc0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8cc00725e0 0x7f8cc0074aa0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4159980 tx=0x7f8cd40091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce41511f0 msgr2=0x7f8ce414e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce41511f0 0x7f8ce414e350 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc01fb20 tx=0x7f8cdc0092c0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 shutdown_connections 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8cc00725e0 0x7f8cc0074aa0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ce41511f0 0x7f8ce414e350 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ce40ceee0 0x7f8ce414de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 --2- 172.21.15.97:0/4098825448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ce40b3cf0 0x7f8ce4147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 >> 172.21.15.97:0/4098825448 conn(0x7f8ce40b1780 msgr2=0x7f8ce406ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 shutdown_connections 2024-09-10T21:31:38.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.336+0000 7f8ceba00640 1 -- 172.21.15.97:0/4098825448 wait complete. 2024-09-10T21:31:38.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:31:38.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:38.358 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:31:38.381 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:38.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:38.381 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:38.433 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- 172.21.15.97:0/3699056620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce9d0 msgr2=0x7f9f2815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 --2- 172.21.15.97:0/3699056620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce9d0 0x7f9f2815f6d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9f280725c0 tx=0x7f9f1401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- 172.21.15.97:0/3699056620 shutdown_connections 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 --2- 172.21.15.97:0/3699056620 >> v2:172.21.15.194:3300/0 conn(0x7f9f2815fc10 0x7f9f2815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 --2- 172.21.15.97:0/3699056620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce9d0 0x7f9f2815f6d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 --2- 172.21.15.97:0/3699056620 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f280ce5f0 0x7f9f28141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- 172.21.15.97:0/3699056620 >> 172.21.15.97:0/3699056620 conn(0x7f9f280b1780 msgr2=0x7f9f280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:38.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- 172.21.15.97:0/3699056620 shutdown_connections 2024-09-10T21:31:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- 172.21.15.97:0/3699056620 wait complete. 2024-09-10T21:31:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 Processor -- start 2024-09-10T21:31:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- start start 2024-09-10T21:31:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce5f0 0x7f9f28154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f280ce9d0 0x7f9f281581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f2815fc10 0x7f9f28165030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28140290 con 0x7f9f280ce5f0 2024-09-10T21:31:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28140110 con 0x7f9f2815fc10 2024-09-10T21:31:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f2d000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9f28140410 con 0x7f9f280ce9d0 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce5f0 0x7f9f28154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f25e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce5f0 0x7f9f28154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56898/0 (socket says 172.21.15.97:56898) 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f25e00640 1 -- 172.21.15.97:0/274723316 learned_addr learned my addr 172.21.15.97:0/274723316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f26800640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f2815fc10 0x7f9f28165030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.616+0000 7f9f25400640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f280ce9d0 0x7f9f281581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f25e00640 1 -- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f280ce9d0 msgr2=0x7f9f281581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f25e00640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f280ce9d0 0x7f9f281581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f25e00640 1 -- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f2815fc10 msgr2=0x7f9f28165030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f25e00640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f2815fc10 0x7f9f28165030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f25e00640 1 -- 172.21.15.97:0/274723316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f28158c30 con 0x7f9f280ce5f0 2024-09-10T21:31:38.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f26800640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f2815fc10 0x7f9f28165030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:38.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f25e00640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce5f0 0x7f9f28154f20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d780 tx=0x7f9f1800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:38.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f1e000640 1 -- 172.21.15.97:0/274723316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18017070 con 0x7f9f280ce5f0 2024-09-10T21:31:38.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f1e000640 1 -- 172.21.15.97:0/274723316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f1800de90 con 0x7f9f280ce5f0 2024-09-10T21:31:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f1e000640 1 -- 172.21.15.97:0/274723316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18005020 con 0x7f9f280ce5f0 2024-09-10T21:31:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2815d3b0 con 0x7f9f280ce5f0 2024-09-10T21:31:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f28155460 con 0x7f9f280ce5f0 2024-09-10T21:31:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2815f6d0 con 0x7f9f280ce5f0 2024-09-10T21:31:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f1e000640 1 -- 172.21.15.97:0/274723316 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9f180051c0 con 0x7f9f280ce5f0 2024-09-10T21:31:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.620+0000 7f9f1e000640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f080725e0 0x7f9f08074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:38.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.624+0000 7f9f1e000640 1 -- 172.21.15.97:0/274723316 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6791+0+0 (secure 0 0 0) 0x7f9f18093800 con 0x7f9f280ce5f0 2024-09-10T21:31:38.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.624+0000 7f9f25400640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f080725e0 0x7f9f08074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:38.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.624+0000 7f9f25400640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f080725e0 0x7f9f08074aa0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9f140af000 tx=0x7f9f1401b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:38.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.624+0000 7f9f1e000640 1 -- 172.21.15.97:0/274723316 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9f1805e3b0 con 0x7f9f280ce5f0 2024-09-10T21:31:38.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.824+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9f28141360 con 0x7f9f280ce5f0 2024-09-10T21:31:38.826 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:38.824+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:38.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.824+0000 7f9f1e000640 1 -- 172.21.15.97:0/274723316 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v126) ==== 161+0+0 (secure 0 0 0) 0x7f9f18067120 con 0x7f9f280ce5f0 2024-09-10T21:31:38.828 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f080725e0 msgr2=0x7f9f08074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f080725e0 0x7f9f08074aa0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9f140af000 tx=0x7f9f1401b100 comp rx=0 tx=0).stop 2024-09-10T21:31:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce5f0 msgr2=0x7f9f28154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce5f0 0x7f9f28154f20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d780 tx=0x7f9f1800dc40 comp rx=0 tx=0).stop 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 shutdown_connections 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f080725e0 0x7f9f08074aa0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f2815fc10 0x7f9f28165030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9f280ce9d0 0x7f9f281581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 --2- 172.21.15.97:0/274723316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f280ce5f0 0x7f9f28154f20 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 >> 172.21.15.97:0/274723316 conn(0x7f9f280b1780 msgr2=0x7f9f2806e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 shutdown_connections 2024-09-10T21:31:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.828+0000 7f9f2d000640 1 -- 172.21.15.97:0/274723316 wait complete. 2024-09-10T21:31:38.834 INFO:tasks.ceph.mds.a.smithi097.stderr: -1877> 2024-09-10T21:31:29.892+0000 7f1bb7c00640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-10T21:31:38.836 INFO:tasks.ceph.mds.a.smithi097.stderr: -546> 2024-09-10T21:31:36.868+0000 7f1bbcc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:38.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:38.878 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:31:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 -- 172.21.15.97:0/217824898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 msgr2=0x7f9b380b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/217824898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b380b37b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800aa10 tx=0x7f9b280203e0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 -- 172.21.15.97:0/217824898 shutdown_connections 2024-09-10T21:31:38.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/217824898 >> v2:172.21.15.194:3300/0 conn(0x7f9b380ceee0 0x7f9b380cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/217824898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b380b37b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:38.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 -- 172.21.15.97:0/217824898 >> 172.21.15.97:0/217824898 conn(0x7f9b380b1780 msgr2=0x7f9b380b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:38.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 -- 172.21.15.97:0/217824898 shutdown_connections 2024-09-10T21:31:38.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.992+0000 7f9b3ea00640 1 -- 172.21.15.97:0/217824898 wait complete. 2024-09-10T21:31:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 Processor -- start 2024-09-10T21:31:38.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 -- start start 2024-09-10T21:31:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b380b3cf0 0x7f9b380a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b380ceee0 0x7f9b381f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b381f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3806e740 con 0x7f9b381511f0 2024-09-10T21:31:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3806e5c0 con 0x7f9b380ceee0 2024-09-10T21:31:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3806e8c0 con 0x7f9b380b3cf0 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b381f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b381f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56924/0 (socket says 172.21.15.97:56924) 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 -- 172.21.15.97:0/3105480169 learned_addr learned my addr 172.21.15.97:0/3105480169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b37e00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b380b3cf0 0x7f9b380a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b2f400640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b380ceee0 0x7f9b381f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 -- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b380b3cf0 msgr2=0x7f9b380a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b380b3cf0 0x7f9b380a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 -- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b380ceee0 msgr2=0x7f9b381f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b380ceee0 0x7f9b381f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 -- 172.21.15.97:0/3105480169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b381f6e30 con 0x7f9b381511f0 2024-09-10T21:31:39.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b2f400640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b380ceee0 0x7f9b381f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:39.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b3ca00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b381f64d0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000ea20 tx=0x7f9b3000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b34c00640 1 -- 172.21.15.97:0/3105480169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000cde0 con 0x7f9b381511f0 2024-09-10T21:31:39.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b34c00640 1 -- 172.21.15.97:0/3105480169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b30004c40 con 0x7f9b381511f0 2024-09-10T21:31:39.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:38.996+0000 7f9b34c00640 1 -- 172.21.15.97:0/3105480169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30005280 con 0x7f9b381511f0 2024-09-10T21:31:39.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.000+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381f6b50 con 0x7f9b381511f0 2024-09-10T21:31:39.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.000+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381f2e20 con 0x7f9b381511f0 2024-09-10T21:31:39.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.000+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b381f2950 con 0x7f9b381511f0 2024-09-10T21:31:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.000+0000 7f9b34c00640 1 -- 172.21.15.97:0/3105480169 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9b3000f040 con 0x7f9b381511f0 2024-09-10T21:31:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.000+0000 7f9b34c00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b140725e0 0x7f9b14074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.000+0000 7f9b34c00640 1 -- 172.21.15.97:0/3105480169 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7f9b30093cc0 con 0x7f9b381511f0 2024-09-10T21:31:39.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.000+0000 7f9b37e00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b140725e0 0x7f9b14074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.004+0000 7f9b37e00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b140725e0 0x7f9b14074aa0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9b280b2000 tx=0x7f9b2801ec20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.004+0000 7f9b34c00640 1 -- 172.21.15.97:0/3105480169 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9b3009a080 con 0x7f9b381511f0 2024-09-10T21:31:39.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.228+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f9b380cf350 con 0x7f9b381511f0 2024-09-10T21:31:39.406 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:39.410 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:39.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b34c00640 1 -- 172.21.15.97:0/3105480169 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v127) ==== 107+0+0 (secure 0 0 0) 0x7f9b30016030 con 0x7f9b381511f0 2024-09-10T21:31:39.831 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:31:39.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b140725e0 msgr2=0x7f9b14074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:39.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b140725e0 0x7f9b14074aa0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9b280b2000 tx=0x7f9b2801ec20 comp rx=0 tx=0).stop 2024-09-10T21:31:39.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 msgr2=0x7f9b381f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:39.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b381f64d0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000ea20 tx=0x7f9b3000eee0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 shutdown_connections 2024-09-10T21:31:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b140725e0 0x7f9b14074aa0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b381511f0 0x7f9b381f64d0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b380ceee0 0x7f9b381f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 --2- 172.21.15.97:0/3105480169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b380b3cf0 0x7f9b380a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.828+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 >> 172.21.15.97:0/3105480169 conn(0x7f9b380b1780 msgr2=0x7f9b3815b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.832+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 shutdown_connections 2024-09-10T21:31:39.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.832+0000 7f9b3ea00640 1 -- 172.21.15.97:0/3105480169 wait complete. 2024-09-10T21:31:39.853 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:39.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 Processor -- start 2024-09-10T21:31:39.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 -- start start 2024-09-10T21:31:39.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e8389880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8390160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 0x5577e8423260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5577e8205a00 con 0x5577e82e6940 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5577e8263f40 con 0x5577e83712a0 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b3fb4080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5577e8373080 con 0x5577e8370b00 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e8389880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e8389880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56954/0 (socket says 172.21.15.97:56954) 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 -- 172.21.15.97:0/3033395289 learned_addr learned my addr 172.21.15.97:0/3033395289 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8390160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1c00640 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 0x5577e8423260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 -- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 msgr2=0x5577e8390160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8390160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 -- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 msgr2=0x5577e8423260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 0x5577e8423260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 -- 172.21.15.97:0/3033395289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e81dec10 con 0x5577e82e6940 2024-09-10T21:31:39.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8390160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:39.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e8389880 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b2e0 tx=0x7fe6a400b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a681f980a60092 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6aaa00640 1 -- 172.21.15.97:0/3033395289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4012070 con 0x5577e82e6940 2024-09-10T21:31:39.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6aaa00640 1 -- 172.21.15.97:0/3033395289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6a400e930 con 0x5577e82e6940 2024-09-10T21:31:39.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.896+0000 7fe6aaa00640 1 -- 172.21.15.97:0/3033395289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400ed80 con 0x5577e82e6940 2024-09-10T21:31:39.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3033395289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 msgr2=0x5577e8389880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:39.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e8389880 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b2e0 tx=0x7fe6a400b7a0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3033395289 shutdown_connections 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 0x5577e8423260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8390160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3033395289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e8389880 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3033395289 >> 172.21.15.97:0/3033395289 conn(0x5577e82e5520 msgr2=0x5577e82b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3033395289 shutdown_connections 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3033395289 wait complete. 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 Processor -- start 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- start start 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e838d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8397b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 0x5577e838c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5577e838b030 con 0x5577e82e6940 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5577e82b61d0 con 0x5577e83712a0 2024-09-10T21:31:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5577e82b6470 con 0x5577e8370b00 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e838d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e838d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56956/0 (socket says 172.21.15.97:56956) 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 learned_addr learned my addr 172.21.15.97:0/3007558755 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1c00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 0x5577e838c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8397b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 msgr2=0x5577e8397b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5577e8370b00 0x5577e8397b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 msgr2=0x5577e838c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5577e83712a0 0x5577e838c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e8388e40 con 0x5577e82e6940 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e838d450 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4010dd0 tx=0x7fe6a4010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6a9600640 1 -- 172.21.15.97:0/3007558755 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4025070 con 0x5577e82e6940 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6a9600640 1 -- 172.21.15.97:0/3007558755 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6a400edf0 con 0x5577e82e6940 2024-09-10T21:31:39.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6a9600640 1 -- 172.21.15.97:0/3007558755 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4012040 con 0x5577e82e6940 2024-09-10T21:31:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577e8397140 con 0x5577e82e6940 2024-09-10T21:31:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.900+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e838d990 con 0x5577e82e6940 2024-09-10T21:31:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6a9600640 1 -- 172.21.15.97:0/3007558755 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe6a40102d0 con 0x5577e82e6940 2024-09-10T21:31:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6a9600640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe690070fc0 0x7fe690073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6a9600640 1 -- 172.21.15.97:0/3007558755 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7fe6a4093bf0 con 0x5577e82e6940 2024-09-10T21:31:39.908 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:39.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe690070fc0 0x7fe690073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5577e8391630 0x5577e8388150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8398060 con 0x5577e8391630 2024-09-10T21:31:39.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6b1c00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5577e8391630 0x5577e8388150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe690070fc0 0x7fe690073480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00b640 tx=0x7fe69c004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.904+0000 7fe6b1c00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5577e8391630 0x5577e8388150 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.908+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v103'195 uv169 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.908+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8388690 con 0x5577e8391630 2024-09-10T21:31:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.908+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v103'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.908+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84337f0 con 0x5577e8391630 2024-09-10T21:31:39.913 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.912+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v103'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.912+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434ba0 con 0x5577e8391630 2024-09-10T21:31:39.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.912+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.912+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434f40 con 0x5577e8391630 2024-09-10T21:31:39.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.912+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.912+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433b90 con 0x5577e8391630 2024-09-10T21:31:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.916+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.916+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f80 con 0x5577e8391630 2024-09-10T21:31:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.916+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v103'201 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.916+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8433f80 con 0x5577e8391630 2024-09-10T21:31:39.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.916+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v103'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.916+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.922 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:39.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.920+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v103'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.920+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.920+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.920+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.924+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.924+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.924+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.924+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.924+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v103'207 uv182 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.924+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.928+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'208 uv208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.928+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.931 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:39.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.928+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v103'209 uv209 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.928+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.928+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'210 uv210 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.928+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.932+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'211 uv211 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.932+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8433f30 con 0x5577e8391630 2024-09-10T21:31:39.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.932+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'212 uv212 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.932+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434350 con 0x5577e8391630 2024-09-10T21:31:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.936+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'213 uv213 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.936+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84347c0 con 0x5577e8391630 2024-09-10T21:31:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.936+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v103'214 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.936+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e84347c0 con 0x5577e8391630 2024-09-10T21:31:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.936+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v103'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.936+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.941 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.940+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v103'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.940+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.940+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.940+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.940+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.940+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.944+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.944+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.944+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v103'220 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.944+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.948+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v103'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.948+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.950 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.948+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v103'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.948+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.948+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.948+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8434300 con 0x5577e8391630 2024-09-10T21:31:39.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e8391630 2024-09-10T21:31:39.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5577e84352e0 0x5577e84376d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8437c10 con 0x5577e84352e0 2024-09-10T21:31:39.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5577e84352e0 0x5577e84376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5577e84352e0 0x5577e84376d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'152 uv133 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.952+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8437fb0 con 0x5577e84352e0 2024-09-10T21:31:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.956+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v103'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.956+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8438350 con 0x5577e84352e0 2024-09-10T21:31:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.956+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v103'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.959 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:39.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.956+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8438ba0 con 0x5577e84352e0 2024-09-10T21:31:39.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.956+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v103'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.956+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8438f40 con 0x5577e84352e0 2024-09-10T21:31:39.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.960+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v103'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.960+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84392e0 con 0x5577e84352e0 2024-09-10T21:31:39.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.960+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v103'157 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.960+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.960+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v103'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.960+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.964+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v103'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:39.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.964+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.964+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v103'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.964+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.964+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v103'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.964+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.968+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v103'162 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.968+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.968+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v103'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.968+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.968+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v103'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.972 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:39.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.968+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843d850 con 0x5577e84352e0 2024-09-10T21:31:39.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.972+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v103'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.972+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843d850 con 0x5577e84352e0 2024-09-10T21:31:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.972+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v103'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.972+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v103'167 uv147 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v103'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v103'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.980 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:39.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843a690 con 0x5577e84352e0 2024-09-10T21:31:39.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v103'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.976+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843a690 con 0x5577e84352e0 2024-09-10T21:31:39.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.980+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v103'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.980+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8439680 con 0x5577e84352e0 2024-09-10T21:31:39.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.980+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v103'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e84352e0 2024-09-10T21:31:39.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.980+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5577e8439b90 0x5577e843bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.980+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843c4c0 con 0x5577e8439b90 2024-09-10T21:31:39.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.980+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5577e8439b90 0x5577e843bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:39.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.980+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5577e8439b90 0x5577e843bf80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.984+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.984+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e843c8a0 con 0x5577e8439b90 2024-09-10T21:31:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.984+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:39.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.984+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84387c0 con 0x5577e8439b90 2024-09-10T21:31:39.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.988+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:39.991 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:39.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.988+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843e6b0 con 0x5577e8439b90 2024-09-10T21:31:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.992+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.992+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843ea50 con 0x5577e8439b90 2024-09-10T21:31:39.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.992+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:39.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.992+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e843d6a0 con 0x5577e8439b90 2024-09-10T21:31:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.996+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.996+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5577e843dfa0 0x5577e8440e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.996+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441340 con 0x5577e843dfa0 2024-09-10T21:31:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.996+0000 7fe6b1c00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5577e843dfa0 0x5577e8440e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:39.996+0000 7fe6b1c00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5577e843dfa0 0x5577e8440e00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'34 uv33 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8441340 con 0x5577e843dfa0 2024-09-10T21:31:40.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v103'35 uv35 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84413c0 con 0x5577e843dfa0 2024-09-10T21:31:40.003 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:40.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v103'36 uv36 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84417e0 con 0x5577e843dfa0 2024-09-10T21:31:40.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v103'37 uv37 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.000+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84417e0 con 0x5577e8439b90 2024-09-10T21:31:40.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.004+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.004+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.004+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.004+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.008 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.004+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.004+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8442b90 con 0x5577e8439b90 2024-09-10T21:31:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.008+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.008+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8442b90 con 0x5577e8439b90 2024-09-10T21:31:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.008+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.008+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.008+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.008+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.012+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v103'70 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.012+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.012+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v103'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.012+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.016 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.012+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v103'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.012+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8442b90 con 0x5577e843dfa0 2024-09-10T21:31:40.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.016+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v103'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.016+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8442b90 con 0x5577e843dfa0 2024-09-10T21:31:40.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.016+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v103'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.016+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.016+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v103'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.016+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.020+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v103'76 uv69 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.020+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.020+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v103'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.020+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.023 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.020+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v103'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.020+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.020+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v103'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.024+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e843dfa0 2024-09-10T21:31:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.024+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v103'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.024+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.024+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v103'32 uv31 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.024+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.024+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v103'33 uv33 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.024+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v103'34 uv34 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.031 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v103'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8441b80 con 0x5577e8439b90 2024-09-10T21:31:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v103'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8439b90 2024-09-10T21:31:40.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5577e8442200 0x5577e84445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8444c50 con 0x5577e8442200 2024-09-10T21:31:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.028+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5577e8442200 0x5577e84445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.032+0000 7fe6b1200640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5577e8442200 0x5577e84445f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.032+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.032+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8444cd0 con 0x5577e8442200 2024-09-10T21:31:40.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.032+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.032+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.037 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:40.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8446100 con 0x5577e8442200 2024-09-10T21:31:40.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8446100 con 0x5577e8442200 2024-09-10T21:31:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.036+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.040+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v103'64 uv58 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.040+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.040+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v103'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.040+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.044 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.040+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v103'66 uv66 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.040+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.044+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v103'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.044+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.044+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v103'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.044+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.044+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v103'69 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.044+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.048+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v103'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.048+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.048+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v103'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.051 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.048+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84450f0 con 0x5577e8442200 2024-09-10T21:31:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.048+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v103'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.048+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8447e40 con 0x5577e8442200 2024-09-10T21:31:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.052+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v103'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.052+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.052+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v103'31 uv29 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.052+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.052+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v103'32 uv32 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.052+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.052+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v103'33 uv33 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.057 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:40.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.056+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.056+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v103'34 uv34 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.056+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.056+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v103'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.056+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.056+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.056+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.061 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e84454e0 con 0x5577e843dfa0 2024-09-10T21:31:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6ac0a9070 con 0x5577e843dfa0 2024-09-10T21:31:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5577e8445b60 0x5577e8448500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.060+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8445b60 2024-09-10T21:31:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.064+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5577e8445b60 0x5577e8448500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.064+0000 7fe6abe00640 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5577e8445b60 0x5577e8448500 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.064+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v103'32 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8445b60 2024-09-10T21:31:40.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.064+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8445b60 2024-09-10T21:31:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.064+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v103'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8445b60 2024-09-10T21:31:40.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.064+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8445b60 2024-09-10T21:31:40.071 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.068+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v103'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8445b60 2024-09-10T21:31:40.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.068+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8445b60 2024-09-10T21:31:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.072+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v103'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8445b60 2024-09-10T21:31:40.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.072+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8445b60 2024-09-10T21:31:40.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.072+0000 7fe6abe00640 1 -- 172.21.15.97:0/3007558755 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v103'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe69c00ab20 con 0x5577e8445b60 2024-09-10T21:31:40.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.072+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8442200 2024-09-10T21:31:40.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.076+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v103'82 uv73 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.076+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8442200 2024-09-10T21:31:40.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.076+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v103'83 uv83 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.076+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8447f90 con 0x5577e8442200 2024-09-10T21:31:40.080 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:40.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.076+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v103'84 uv84 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.076+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8448b30 con 0x5577e8442200 2024-09-10T21:31:40.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.080+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v103'85 uv85 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.080+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8448ab0 con 0x5577e8442200 2024-09-10T21:31:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.080+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v103'86 uv86 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.080+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8448ab0 con 0x5577e8442200 2024-09-10T21:31:40.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.084+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v103'87 uv80 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.084+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5577e8448ab0 con 0x5577e8442200 2024-09-10T21:31:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.084+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v103'88 uv88 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.084+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8448a90 con 0x5577e8442200 2024-09-10T21:31:40.089 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.088+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v103'89 uv89 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.088+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8449aa0 con 0x5577e8442200 2024-09-10T21:31:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.088+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v103'90 uv90 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.088+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8449aa0 con 0x5577e8442200 2024-09-10T21:31:40.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.088+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v103'91 uv91 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.088+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8448a90 con 0x5577e8442200 2024-09-10T21:31:40.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v103'92 uv92 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5577e8448a90 con 0x5577e8442200 2024-09-10T21:31:40.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b1200640 1 -- 172.21.15.97:0/3007558755 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v103'93 uv93 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe6a4027030 con 0x5577e8442200 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5577e8439b90 msgr2=0x5577e843bf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5577e8439b90 0x5577e843bf80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5577e8442200 msgr2=0x5577e84445f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5577e8442200 0x5577e84445f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5577e8445b60 msgr2=0x5577e8448500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5577e8445b60 0x5577e8448500 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5577e843dfa0 msgr2=0x5577e8440e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5577e843dfa0 0x5577e8440e00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5577e84352e0 msgr2=0x5577e84376d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 reap_dead start 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5577e84352e0 0x5577e84376d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b1c00640 1 -- 172.21.15.97:0/3007558755 reap_dead start 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5577e8391630 msgr2=0x5577e8388150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5577e8391630 0x5577e8388150 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe690070fc0 msgr2=0x7fe690073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe690070fc0 0x7fe690073480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00b640 tx=0x7fe69c004890 comp rx=0 tx=0).stop 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 msgr2=0x5577e838d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e838d450 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4010dd0 tx=0x7fe6a4010e00 comp rx=0 tx=0).stop 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 shutdown_connections 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5577e84352e0 0x5577e84376d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5577e8391630 0x5577e8388150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe690070fc0 0x7fe690073480 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 --2- 172.21.15.97:0/3007558755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5577e82e6940 0x5577e838d450 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 >> 172.21.15.97:0/3007558755 conn(0x5577e82e5520 msgr2=0x5577e836fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 shutdown_connections 2024-09-10T21:31:40.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.092+0000 7fe6b3fb4080 1 -- 172.21.15.97:0/3007558755 wait complete. 2024-09-10T21:31:40.100 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2024-09-10T21:31:40.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 Processor -- start 2024-09-10T21:31:40.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 -- start start 2024-09-10T21:31:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003100 0x55701b387c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55701b414390 0x55701b4315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b432010 0x55701b4323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55701b307f40 con 0x55701b432010 2024-09-10T21:31:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55701b314d00 con 0x55701b414390 2024-09-10T21:31:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb54b309080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55701b2a9a00 con 0x7fb52c003100 2024-09-10T21:31:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb549000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b432010 0x55701b4323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb547c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55701b414390 0x55701b4315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb548600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003100 0x55701b387c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb549000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b432010 0x55701b4323f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56982/0 (socket says 172.21.15.97:56982) 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.188+0000 7fb549000640 1 -- 172.21.15.97:0/3655205895 learned_addr learned my addr 172.21.15.97:0/3655205895 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb549000640 1 -- 172.21.15.97:0/3655205895 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003100 msgr2=0x55701b387c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb549000640 1 --2- 172.21.15.97:0/3655205895 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003100 0x55701b387c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb549000640 1 -- 172.21.15.97:0/3655205895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55701b414390 msgr2=0x55701b4315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb549000640 1 --2- 172.21.15.97:0/3655205895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55701b414390 0x55701b4315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb549000640 1 -- 172.21.15.97:0/3655205895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55701b38b130 con 0x55701b432010 2024-09-10T21:31:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb549000640 1 --2- 172.21.15.97:0/3655205895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b432010 0x55701b4323f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x55701b35c050 tx=0x7fb54000ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ac1f48dfc6c575 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb546800640 1 -- 172.21.15.97:0/3655205895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540004220 con 0x55701b432010 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb546800640 1 -- 172.21.15.97:0/3655205895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb54000ae20 con 0x55701b432010 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb546800640 1 -- 172.21.15.97:0/3655205895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540010b60 con 0x55701b432010 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- 172.21.15.97:0/3655205895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b432010 msgr2=0x55701b4323f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 --2- 172.21.15.97:0/3655205895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b432010 0x55701b4323f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x55701b35c050 tx=0x7fb54000ab10 comp rx=0 tx=0).stop 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- 172.21.15.97:0/3655205895 shutdown_connections 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 --2- 172.21.15.97:0/3655205895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b432010 0x55701b4323f0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 --2- 172.21.15.97:0/3655205895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55701b414390 0x55701b4315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 --2- 172.21.15.97:0/3655205895 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb52c003100 0x55701b387c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- 172.21.15.97:0/3655205895 >> 172.21.15.97:0/3655205895 conn(0x55701b35a140 msgr2=0x55701b414900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- 172.21.15.97:0/3655205895 shutdown_connections 2024-09-10T21:31:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- 172.21.15.97:0/3655205895 wait complete. 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 Processor -- start 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- start start 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b414390 0x55701b42e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55701b432010 0x55701b438ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb52c003100 0x55701b3891a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55701b388830 con 0x55701b414390 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55701b388ad0 con 0x7fb52c003100 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb54b309080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55701b388d70 con 0x55701b432010 2024-09-10T21:31:40.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b414390 0x55701b42e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b414390 0x55701b42e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56986/0 (socket says 172.21.15.97:56986) 2024-09-10T21:31:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 -- 172.21.15.97:0/1484818593 learned_addr learned my addr 172.21.15.97:0/1484818593 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb547c00640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55701b432010 0x55701b438ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb549000640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb52c003100 0x55701b3891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 -- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55701b432010 msgr2=0x55701b438ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55701b432010 0x55701b438ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 -- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb52c003100 msgr2=0x55701b3891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb52c003100 0x55701b3891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 -- 172.21.15.97:0/1484818593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55701b437770 con 0x55701b414390 2024-09-10T21:31:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb548600640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b414390 0x55701b42e890 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb5380107c0 tx=0x7fb538010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.192+0000 7fb545400640 1 -- 172.21.15.97:0/1484818593 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53800bb10 con 0x55701b414390 2024-09-10T21:31:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55701b4354c0 con 0x55701b414390 2024-09-10T21:31:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb545400640 1 -- 172.21.15.97:0/1484818593 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb53800bcb0 con 0x55701b414390 2024-09-10T21:31:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb545400640 1 -- 172.21.15.97:0/1484818593 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53800d640 con 0x55701b414390 2024-09-10T21:31:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701b435a00 con 0x55701b414390 2024-09-10T21:31:40.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb545400640 1 -- 172.21.15.97:0/1484818593 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb53800d7e0 con 0x55701b414390 2024-09-10T21:31:40.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb545400640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528070fc0 0x7fb528073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb545400640 1 -- 172.21.15.97:0/1484818593 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7fb538092fd0 con 0x55701b414390 2024-09-10T21:31:40.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb54b309080 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55701b4cede0 0x55701b4cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x55701b4cf700 con 0x55701b4cede0 2024-09-10T21:31:40.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb549000640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55701b4cede0 0x55701b4cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb547c00640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528070fc0 0x7fb528073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.196+0000 7fb549000640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55701b4cede0 0x55701b4cf1c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb547c00640 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528070fc0 0x7fb528073480 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb5340ac000 tx=0x7fb5340b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb549000640 1 -- 172.21.15.97:0/1484818593 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 500.00000000 [delete] v103'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540020040 con 0x55701b4cede0 2024-09-10T21:31:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55701b4cede0 msgr2=0x55701b4cf1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55701b4cede0 0x55701b4cf1c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528070fc0 msgr2=0x7fb528073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb528070fc0 0x7fb528073480 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb5340ac000 tx=0x7fb5340b1040 comp rx=0 tx=0).stop 2024-09-10T21:31:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b414390 msgr2=0x55701b42e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 --2- 172.21.15.97:0/1484818593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55701b414390 0x55701b42e890 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb5380107c0 tx=0x7fb538010c80 comp rx=0 tx=0).stop 2024-09-10T21:31:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb549000640 1 -- 172.21.15.97:0/1484818593 reap_dead start 2024-09-10T21:31:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 shutdown_connections 2024-09-10T21:31:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 >> 172.21.15.97:0/1484818593 conn(0x55701b35a140 msgr2=0x55701b3593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 shutdown_connections 2024-09-10T21:31:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.200+0000 7fb54b309080 1 -- 172.21.15.97:0/1484818593 wait complete. 2024-09-10T21:31:40.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:40.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 -- 172.21.15.97:0/1294281002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00b0d50 msgr2=0x7fc9e00ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 --2- 172.21.15.97:0/1294281002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00b0d50 0x7fc9e00ccc10 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00b36f0 tx=0x7fc9d801cd80 comp rx=0 tx=0).stop 2024-09-10T21:31:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 -- 172.21.15.97:0/1294281002 shutdown_connections 2024-09-10T21:31:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 --2- 172.21.15.97:0/1294281002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00b0d50 0x7fc9e00ccc10 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 --2- 172.21.15.97:0/1294281002 >> v2:172.21.15.194:3300/0 conn(0x7fc9e00afdd0 0x7fc9e00cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 -- 172.21.15.97:0/1294281002 >> 172.21.15.97:0/1294281002 conn(0x7fc9e00d1c30 msgr2=0x7fc9e00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 -- 172.21.15.97:0/1294281002 shutdown_connections 2024-09-10T21:31:40.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 -- 172.21.15.97:0/1294281002 wait complete. 2024-09-10T21:31:40.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 Processor -- start 2024-09-10T21:31:40.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.348+0000 7fc9e6600640 1 -- start start 2024-09-10T21:31:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9e00afdd0 0x7fc9e0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9e00b0d50 0x7fc9e00b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00ce640 0x7fc9e015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0168ee0 con 0x7fc9e00ce640 2024-09-10T21:31:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0168d60 con 0x7fc9e00afdd0 2024-09-10T21:31:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e0169060 con 0x7fc9e00b0d50 2024-09-10T21:31:40.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00ce640 0x7fc9e015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00ce640 0x7fc9e015c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56998/0 (socket says 172.21.15.97:56998) 2024-09-10T21:31:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 -- 172.21.15.97:0/4184685281 learned_addr learned my addr 172.21.15.97:0/4184685281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9df400640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9e00afdd0 0x7fc9e0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dea00640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9e00b0d50 0x7fc9e00b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 -- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9e00b0d50 msgr2=0x7fc9e00b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9e00b0d50 0x7fc9e00b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 -- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9e00afdd0 msgr2=0x7fc9e0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9e00afdd0 0x7fc9e0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 -- 172.21.15.97:0/4184685281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e0157350 con 0x7fc9e00ce640 2024-09-10T21:31:40.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9dfe00640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00ce640 0x7fc9e015c390 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8019990 tx=0x7fc9d8009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9d7400640 1 -- 172.21.15.97:0/4184685281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80bb070 con 0x7fc9e00ce640 2024-09-10T21:31:40.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9d7400640 1 -- 172.21.15.97:0/4184685281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9d8004d40 con 0x7fc9e00ce640 2024-09-10T21:31:40.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e00b7220 con 0x7fc9e00ce640 2024-09-10T21:31:40.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9d7400640 1 -- 172.21.15.97:0/4184685281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80b2040 con 0x7fc9e00ce640 2024-09-10T21:31:40.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e00b2690 con 0x7fc9e00ce640 2024-09-10T21:31:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.352+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e00cea20 con 0x7fc9e00ce640 2024-09-10T21:31:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.356+0000 7fc9d7400640 1 -- 172.21.15.97:0/4184685281 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc9d80b5030 con 0x7fc9e00ce640 2024-09-10T21:31:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.356+0000 7fc9d7400640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9c0072610 0x7fc9c0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.356+0000 7fc9d7400640 1 -- 172.21.15.97:0/4184685281 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7fc9d812d280 con 0x7fc9e00ce640 2024-09-10T21:31:40.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.356+0000 7fc9df400640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9c0072610 0x7fc9c0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.356+0000 7fc9df400640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9c0072610 0x7fc9c0074ad0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007fd0 tx=0x7fc9d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.360+0000 7fc9d7400640 1 -- 172.21.15.97:0/4184685281 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc9d80f1de0 con 0x7fc9e00ce640 2024-09-10T21:31:40.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.556+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc9e0075f90 con 0x7fc9e00ce640 2024-09-10T21:31:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.832+0000 7fc9d7400640 1 -- 172.21.15.97:0/4184685281 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v128) ==== 162+0+0 (secure 0 0 0) 0x7fc9d8016970 con 0x7fc9e00ce640 2024-09-10T21:31:40.836 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9c0072610 msgr2=0x7fc9c0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9c0072610 0x7fc9c0074ad0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007fd0 tx=0x7fc9d0009040 comp rx=0 tx=0).stop 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00ce640 msgr2=0x7fc9e015c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00ce640 0x7fc9e015c390 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8019990 tx=0x7fc9d8009300 comp rx=0 tx=0).stop 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 shutdown_connections 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc9c0072610 0x7fc9c0074ad0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9e00ce640 0x7fc9e015c390 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc9e00b0d50 0x7fc9e00b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 --2- 172.21.15.97:0/4184685281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9e00afdd0 0x7fc9e0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 >> 172.21.15.97:0/4184685281 conn(0x7fc9e00d1c30 msgr2=0x7fc9e006f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 shutdown_connections 2024-09-10T21:31:40.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:40.836+0000 7fc9e6600640 1 -- 172.21.15.97:0/4184685281 wait complete. 2024-09-10T21:31:41.267 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 -- 172.21.15.97:0/1048895583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0ce640 msgr2=0x7f603c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 --2- 172.21.15.97:0/1048895583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0ce640 0x7f603c0cea20 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f603c0b36f0 tx=0x7f602c01c800 comp rx=0 tx=0).stop 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 -- 172.21.15.97:0/1048895583 shutdown_connections 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 --2- 172.21.15.97:0/1048895583 >> v2:172.21.15.194:3300/0 conn(0x7f603c0b0d50 0x7f603c0aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 --2- 172.21.15.97:0/1048895583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0ce640 0x7f603c0cea20 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 -- 172.21.15.97:0/1048895583 >> 172.21.15.97:0/1048895583 conn(0x7f603c0d1c30 msgr2=0x7f603c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 -- 172.21.15.97:0/1048895583 shutdown_connections 2024-09-10T21:31:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 -- 172.21.15.97:0/1048895583 wait complete. 2024-09-10T21:31:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 Processor -- start 2024-09-10T21:31:41.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.412+0000 7f6043c00640 1 -- start start 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0af300 0x7f603c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f603c0b0d50 0x7f603c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f603c0ce640 0x7f603c0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f603c169270 con 0x7f603c0af300 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f603c1690f0 con 0x7f603c0ce640 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f603c1693f0 con 0x7f603c0b0d50 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0af300 0x7f603c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0af300 0x7f603c15e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57024/0 (socket says 172.21.15.97:57024) 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 -- 172.21.15.97:0/3500388306 learned_addr learned my addr 172.21.15.97:0/3500388306 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6033e00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f603c0b0d50 0x7f603c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f603c0ce640 0x7f603c0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 -- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f603c0b0d50 msgr2=0x7f603c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f603c0b0d50 0x7f603c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 -- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f603c0ce640 msgr2=0x7f603c0b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f603c0ce640 0x7f603c0b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 -- 172.21.15.97:0/3500388306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c15d7e0 con 0x7f603c0af300 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f603c0ce640 0x7f603c0b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6041200640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0af300 0x7f603c15e880 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c10 tx=0x7f602c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6039600640 1 -- 172.21.15.97:0/3500388306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0bb070 con 0x7f603c0af300 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6039600640 1 -- 172.21.15.97:0/3500388306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f602c005030 con 0x7f603c0af300 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c15dac0 con 0x7f603c0af300 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6039600640 1 -- 172.21.15.97:0/3500388306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0b2040 con 0x7f603c0af300 2024-09-10T21:31:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c15e080 con 0x7f603c0af300 2024-09-10T21:31:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c0aedc0 con 0x7f603c0af300 2024-09-10T21:31:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.416+0000 7f6039600640 1 -- 172.21.15.97:0/3500388306 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f602c0b5030 con 0x7f603c0af300 2024-09-10T21:31:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.420+0000 7f6039600640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60180725e0 0x7f6018074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.420+0000 7f6039600640 1 -- 172.21.15.97:0/3500388306 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7f602c12ce80 con 0x7f603c0af300 2024-09-10T21:31:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.420+0000 7f6033e00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60180725e0 0x7f6018074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.420+0000 7f6033e00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60180725e0 0x7f6018074aa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f603c15f040 tx=0x7f602400a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:41.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.424+0000 7f6039600640 1 -- 172.21.15.97:0/3500388306 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f602c0163f0 con 0x7f603c0af300 2024-09-10T21:31:41.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f603c0d2320 con 0x7f603c0af300 2024-09-10T21:31:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6039600640 1 -- 172.21.15.97:0/3500388306 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) ==== 95+0+5230 (secure 0 0 0) 0x7f602c0f79e0 con 0x7f603c0af300 2024-09-10T21:31:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:41.627 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":129,"btime":"2024-09-10T21:31:40:842559+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2841033666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2841033666},{"type":"v1","addr":"172.21.15.97:6835","nonce":2841033666}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:40.842544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":129,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/2383941694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2383941694},{"type":"v1","addr":"172.21.15.97:6834","nonce":2383941694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 129 2024-09-10T21:31:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60180725e0 msgr2=0x7f6018074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60180725e0 0x7f6018074aa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f603c15f040 tx=0x7f602400a2d0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0af300 msgr2=0x7f603c15e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0af300 0x7f603c15e880 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c10 tx=0x7f602c0040c0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 shutdown_connections 2024-09-10T21:31:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f60180725e0 0x7f6018074aa0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f603c0ce640 0x7f603c0b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f603c0b0d50 0x7f603c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 --2- 172.21.15.97:0/3500388306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c0af300 0x7f603c15e880 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.624+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 >> 172.21.15.97:0/3500388306 conn(0x7f603c0d1c30 msgr2=0x7f603c068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.628+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 shutdown_connections 2024-09-10T21:31:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:41.628+0000 7f6043c00640 1 -- 172.21.15.97:0/3500388306 wait complete. 2024-09-10T21:31:41.649 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 129, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:31:40.842544+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 103, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8025': {'gid': 8025, 'name': 'a', 'rank': 0, 'incarnation': 129, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6834/2383941694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 2383941694}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 2383941694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:31:41.649 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8025:up:replay 2024-09-10T21:31:41.649 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:31:44.838 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:31:44.840+0000 7f2154400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:47.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:47.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.200+0000 7fbe14c00640 1 -- 172.21.15.97:0/1059384062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 msgr2=0x7fbe100cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.200+0000 7fbe14c00640 1 --2- 172.21.15.97:0/1059384062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100cb030 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009900 tx=0x7fbe0001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- 172.21.15.97:0/1059384062 shutdown_connections 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 --2- 172.21.15.97:0/1059384062 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe10073a00 0x7fbe1015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 --2- 172.21.15.97:0/1059384062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100cb030 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- 172.21.15.97:0/1059384062 >> 172.21.15.97:0/1059384062 conn(0x7fbe100cd3d0 msgr2=0x7fbe100cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- 172.21.15.97:0/1059384062 shutdown_connections 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- 172.21.15.97:0/1059384062 wait complete. 2024-09-10T21:31:47.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 Processor -- start 2024-09-10T21:31:47.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- start start 2024-09-10T21:31:47.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe10073a00 0x7fbe100b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe10074010 0x7fbe100c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10167cb0 con 0x7fbe10073620 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10167b30 con 0x7fbe10074010 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe14c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbe10167e30 con 0x7fbe10073a00 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57054/0 (socket says 172.21.15.97:57054) 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 -- 172.21.15.97:0/2066225288 learned_addr learned my addr 172.21.15.97:0/2066225288 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0ea00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe10073a00 0x7fbe100b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0fe00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe10074010 0x7fbe100c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 -- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe10073a00 msgr2=0x7fbe100b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe10073a00 0x7fbe100b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 -- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe10074010 msgr2=0x7fbe100c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe10074010 0x7fbe100c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 -- 172.21.15.97:0/2066225288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe100c8970 con 0x7fbe10073620 2024-09-10T21:31:47.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.204+0000 7fbe0f400640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100c8430 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbe0001a510 tx=0x7fbe000a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe07400640 1 -- 172.21.15.97:0/2066225288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000ac070 con 0x7fbe10073620 2024-09-10T21:31:47.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe07400640 1 -- 172.21.15.97:0/2066225288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe00004320 con 0x7fbe10073620 2024-09-10T21:31:47.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe07400640 1 -- 172.21.15.97:0/2066225288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00005320 con 0x7fbe10073620 2024-09-10T21:31:47.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe100b6170 con 0x7fbe10073620 2024-09-10T21:31:47.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe100c8ef0 con 0x7fbe10073620 2024-09-10T21:31:47.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe100cb030 con 0x7fbe10073620 2024-09-10T21:31:47.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe07400640 1 -- 172.21.15.97:0/2066225288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbe000b8020 con 0x7fbe10073620 2024-09-10T21:31:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe07400640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbdf00725e0 0x7fbdf0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe07400640 1 -- 172.21.15.97:0/2066225288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7fbe0012d590 con 0x7fbe10073620 2024-09-10T21:31:47.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.208+0000 7fbe0ea00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbdf00725e0 0x7fbdf0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.212+0000 7fbe0ea00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbdf00725e0 0x7fbdf0074aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbe101595f0 tx=0x7fbdfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.212+0000 7fbe07400640 1 -- 172.21.15.97:0/2066225288 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbe000bf070 con 0x7fbe10073620 2024-09-10T21:31:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe100ce980 con 0x7fbe10073620 2024-09-10T21:31:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe07400640 1 -- 172.21.15.97:0/2066225288 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) ==== 95+0+5237 (secure 0 0 0) 0x7fbe000ffe20 con 0x7fbe10073620 2024-09-10T21:31:47.414 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:47.414 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":132,"btime":"2024-09-10T21:31:43:854989+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2841033666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2841033666},{"type":"v1","addr":"172.21.15.97:6835","nonce":2841033666}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:43.854982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"a","rank":0,"incarnation":129,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6834/2383941694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2383941694},{"type":"v1","addr":"172.21.15.97:6834","nonce":2383941694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8025,"qdb_cluster":[8025]},"id":6}]} 2024-09-10T21:31:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 132 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbdf00725e0 msgr2=0x7fbdf0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbdf00725e0 0x7fbdf0074aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbe101595f0 tx=0x7fbdfc009040 comp rx=0 tx=0).stop 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 msgr2=0x7fbe100c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100c8430 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbe0001a510 tx=0x7fbe000a6520 comp rx=0 tx=0).stop 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 shutdown_connections 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbdf00725e0 0x7fbdf0074aa0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe10074010 0x7fbe100c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbe10073a00 0x7fbe100b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 --2- 172.21.15.97:0/2066225288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe10073620 0x7fbe100c8430 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 >> 172.21.15.97:0/2066225288 conn(0x7fbe100cd3d0 msgr2=0x7fbe1015ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 shutdown_connections 2024-09-10T21:31:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.412+0000 7fbe14c00640 1 -- 172.21.15.97:0/2066225288 wait complete. 2024-09-10T21:31:47.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:31:43.854982+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 103, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8025': {'gid': 8025, 'name': 'a', 'rank': 0, 'incarnation': 129, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6834/2383941694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 2383941694}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 2383941694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8025, 'qdb_cluster': [8025]} 2024-09-10T21:31:47.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:31:47.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:31:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- 172.21.15.97:0/3759566930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 msgr2=0x7ffb740d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 --2- 172.21.15.97:0/3759566930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740d11f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009900 tx=0x7ffb6401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- 172.21.15.97:0/3759566930 shutdown_connections 2024-09-10T21:31:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 --2- 172.21.15.97:0/3759566930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740d11f0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- 172.21.15.97:0/3759566930 >> 172.21.15.97:0/3759566930 conn(0x7ffb740d3ce0 msgr2=0x7ffb740d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- 172.21.15.97:0/3759566930 shutdown_connections 2024-09-10T21:31:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- 172.21.15.97:0/3759566930 wait complete. 2024-09-10T21:31:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 Processor -- start 2024-09-10T21:31:47.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- start start 2024-09-10T21:31:47.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb740d29d0 0x7ffb740c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb741614f0 0x7ffb74153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7416c620 con 0x7ffb74161110 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7416c4a0 con 0x7ffb740d29d0 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7416c7a0 con 0x7ffb741614f0 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56390/0 (socket says 172.21.15.97:56390) 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 -- 172.21.15.97:0/1211070129 learned_addr learned my addr 172.21.15.97:0/1211070129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb73e00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb740d29d0 0x7ffb740c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb79000640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb741614f0 0x7ffb74153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 -- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb741614f0 msgr2=0x7ffb74153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb741614f0 0x7ffb74153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 -- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb740d29d0 msgr2=0x7ffb740c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb740d29d0 0x7ffb740c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.576+0000 7ffb6b400640 1 -- 172.21.15.97:0/1211070129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb74153e60 con 0x7ffb74161110 2024-09-10T21:31:47.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.580+0000 7ffb6b400640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740b7c40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c001ec0 tx=0x7ffb5c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.580+0000 7ffb70c00640 1 -- 172.21.15.97:0/1211070129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c017070 con 0x7ffb74161110 2024-09-10T21:31:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.580+0000 7ffb70c00640 1 -- 172.21.15.97:0/1211070129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb5c003600 con 0x7ffb74161110 2024-09-10T21:31:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.580+0000 7ffb70c00640 1 -- 172.21.15.97:0/1211070129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c003a50 con 0x7ffb74161110 2024-09-10T21:31:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.580+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb74154140 con 0x7ffb74161110 2024-09-10T21:31:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.580+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb74154700 con 0x7ffb74161110 2024-09-10T21:31:47.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.580+0000 7ffb70c00640 1 -- 172.21.15.97:0/1211070129 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffb5c003bf0 con 0x7ffb74161110 2024-09-10T21:31:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb70c00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb500725e0 0x7ffb50074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb70c00640 1 -- 172.21.15.97:0/1211070129 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7ffb5c092c00 con 0x7ffb74161110 2024-09-10T21:31:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 Processor -- start 2024-09-10T21:31:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb500725e0 0x7ffb50074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- start start 2024-09-10T21:31:47.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb500725e0 0x7ffb50074aa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffb640af000 tx=0x7ffb6401b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbb3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fb681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d4fa883f0 con 0x558d4fbb39d0 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558d4f9c5b20 con 0x558d4fb67e10 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558d4f9b4f50 con 0x558d4fbb42f0 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbb3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb6b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbb3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56396/0 (socket says 172.21.15.97:56396) 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 -- 172.21.15.97:0/3921170033 learned_addr learned my addr 172.21.15.97:0/3921170033 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:47.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb79000640 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fb681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 -- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 msgr2=0x558d4fb678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 -- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 msgr2=0x558d4fb681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fb681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 -- 172.21.15.97:0/3921170033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4f97c590 con 0x558d4fbb39d0 2024-09-10T21:31:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb79000640 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fb681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb73e00640 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbb3db0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ffb6401ae70 tx=0x7ffb640b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f4b1bea5b129e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb68c00640 1 -- 172.21.15.97:0/3921170033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640b2040 con 0x558d4fbb39d0 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb68c00640 1 -- 172.21.15.97:0/3921170033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb64005180 con 0x558d4fbb39d0 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb68c00640 1 -- 172.21.15.97:0/3921170033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64005530 con 0x558d4fbb39d0 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/3921170033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 msgr2=0x558d4fbb3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbb3db0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ffb6401ae70 tx=0x7ffb640b4000 comp rx=0 tx=0).stop 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/3921170033 shutdown_connections 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fb681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/3921170033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbb3db0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/3921170033 >> 172.21.15.97:0/3921170033 conn(0x558d4fa51130 msgr2=0x558d4fb96e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/3921170033 shutdown_connections 2024-09-10T21:31:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.584+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/3921170033 wait complete. 2024-09-10T21:31:47.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 -- start start 2024-09-10T21:31:47.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 Processor -- start 2024-09-10T21:31:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fb69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fb69780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d4f94f5d0 con 0x558d4fb67e10 2024-09-10T21:31:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558d4f9b8d60 con 0x558d4fbb39d0 2024-09-10T21:31:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558d4f95b500 con 0x558d4fbb42f0 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 10 client.0 _ll_register_callbacks cb 0x558d4f95f810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fb69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fb69240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56412/0 (socket says 172.21.15.97:56412) 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 -- 172.21.15.97:0/2312386592 learned_addr learned my addr 172.21.15.97:0/2312386592 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb79000640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb6b400640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fb69780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 -- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 msgr2=0x558d4fb69cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb69cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 -- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 msgr2=0x558d4fb69780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fb69780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 -- 172.21.15.97:0/2312386592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4f953170 con 0x558d4fb67e10 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb79000640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb69cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:47.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fb69240 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ffb640ba040 tx=0x7ffb64005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00a6c0d9851dc59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb61600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb61600640 1 -- 172.21.15.97:0/2312386592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640b2070 con 0x558d4fb67e10 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb61600640 1 -- 172.21.15.97:0/2312386592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7ffb640a56a0 con 0x558d4fb67e10 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d4f94d0f0 con 0x558d4fb67e10 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb61600640 1 -- 172.21.15.97:0/2312386592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) ==== 24+0+0 (secure 0 0 0) 0x7ffb640d2050 con 0x558d4fb67e10 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 10 client.8091 fetch_fsmap learned FSMap version 132 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) -- 0x558d4fa39ef0 con 0x558d4fb67e10 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb61600640 1 -- 172.21.15.97:0/2312386592 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558d4f94d0f0 con 0x558d4fb67e10 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb61600640 1 -- 172.21.15.97:0/2312386592 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 132) ==== 3502+0+0 (secure 0 0 0) 0x7ffb640d5020 con 0x558d4fb67e10 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 10 client.8091 fetch_fsmap finished waiting for FSMap version 132 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 10 client.8091 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 20 client.8091 populate_metadata read hostname 'smithi097' 2024-09-10T21:31:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fbc2c70 0x558d4fbc5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 4 client.8091 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 --> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d4f9c7a70 con 0x558d4fbc2c70 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb6b400640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fbc2c70 0x558d4fbc5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb6b400640 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fbc2c70 0x558d4fbc5060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d6115882c2afd4 server_cookie=c129ddc4c298feed in_seq=0 out_seq=0 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.588+0000 7ffb61600640 10 client.8091 ms_handle_connect on v2:172.21.15.97:6832/2383941694 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb61600640 1 -- 172.21.15.97:0/2312386592 <== mds.0 v2:172.21.15.97:6832/2383941694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d4f9c7a70 con 0x558d4fbc2c70 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb61600640 10 client.8091 handle_command_reply: tid=0 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 client.8091 shutdown 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 msgr2=0x558d4fb69240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fb69240 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ffb640ba040 tx=0x7ffb64005580 comp rx=0 tx=0).stop 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 shutdown_connections 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fbc2c70 0x558d4fbc5060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fb69cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fb69780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2312386592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fb69240 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb61600640 0 client.8091 ms_handle_reset on v2:172.21.15.97:6832/2383941694 2024-09-10T21:31:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 >> 172.21.15.97:0/2312386592 conn(0x558d4fa51130 msgr2=0x558d4fb66fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 shutdown_connections 2024-09-10T21:31:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2312386592 wait complete. 2024-09-10T21:31:47.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.592+0000 7ffb7e3f3000 20 client.8091 trim_cache size 0 max 16384 2024-09-10T21:31:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.608+0000 7ffb7e3f3000 1 Processor -- start 2024-09-10T21:31:47.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.608+0000 7ffb7e3f3000 1 -- start start 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fc259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fc25f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fc26470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb742051e0 con 0x558d4fb67e10 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb742041c0 con 0x558d4fbb39d0 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffb742034c0 con 0x558d4fbb42f0 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fc259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fc259f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56424/0 (socket says 172.21.15.97:56424) 2024-09-10T21:31:47.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 -- 172.21.15.97:0/2955848462 learned_addr learned my addr 172.21.15.97:0/2955848462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb6b400640 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fc25f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 -- 172.21.15.97:0/2955848462 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 msgr2=0x558d4fc26470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fc26470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 -- 172.21.15.97:0/2955848462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 msgr2=0x558d4fc25f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fc25f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 -- 172.21.15.97:0/2955848462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4fa10d90 con 0x558d4fb67e10 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fc259f0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009230 tx=0x7ffb640a54c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ace3e1fc9a5377 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb63e00640 1 -- 172.21.15.97:0/2955848462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640b2070 con 0x558d4fb67e10 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb63e00640 1 -- 172.21.15.97:0/2955848462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb640bd070 con 0x558d4fb67e10 2024-09-10T21:31:47.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb63e00640 1 -- 172.21.15.97:0/2955848462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb640046d0 con 0x558d4fb67e10 2024-09-10T21:31:47.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2955848462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 msgr2=0x558d4fc259f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fc259f0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009230 tx=0x7ffb640a54c0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2955848462 shutdown_connections 2024-09-10T21:31:47.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fc26470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fbb39d0 0x558d4fc25f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2955848462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fb67e10 0x558d4fc259f0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2955848462 >> 172.21.15.97:0/2955848462 conn(0x558d4fa51130 msgr2=0x558d4fbc5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2955848462 shutdown_connections 2024-09-10T21:31:47.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2955848462 wait complete. 2024-09-10T21:31:47.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- start start 2024-09-10T21:31:47.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 Processor -- start 2024-09-10T21:31:47.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fbca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbcaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fbe5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d4f9bd390 con 0x558d4fbb39d0 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74203b10 con 0x558d4fb67e10 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558d4f9c48f0 con 0x558d4fbb42f0 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.612+0000 7ffb7e3f3000 10 client.0 _ll_register_callbacks cb 0x7ffb6c004d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbcaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbcaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56430/0 (socket says 172.21.15.97:56430) 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 -- 172.21.15.97:0/2625057088 learned_addr learned my addr 172.21.15.97:0/2625057088 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fbca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79000640 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fbe5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 -- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 msgr2=0x558d4fbe5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fbe5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 -- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 msgr2=0x558d4fbca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fbca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 -- 172.21.15.97:0/2625057088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4f97ba90 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb6b400640 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbcaec0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c05aa60 tx=0x7ffb5c05d080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcf430e993d814c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 1 -- 172.21.15.97:0/2625057088 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c05cc50 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 1 -- 172.21.15.97:0/2625057088 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7ffb5c064420 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb640bd070 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 1 -- 172.21.15.97:0/2625057088 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) ==== 24+0+0 (secure 0 0 0) 0x7ffb5c062220 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 10 client.8097 fetch_fsmap learned FSMap version 132 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) -- 0x558d4f95a050 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 1 -- 172.21.15.97:0/2625057088 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb640bd070 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 1 -- 172.21.15.97:0/2625057088 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 132) ==== 3502+0+0 (secure 0 0 0) 0x7ffb5c082b00 con 0x558d4fbb39d0 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 10 client.8097 fetch_fsmap finished waiting for FSMap version 132 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 10 client.8097 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:31:47.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 20 client.8097 populate_metadata read hostname 'smithi097' 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fc29a30 0x558d4fc29e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 4 client.8097 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 --> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] -- command(tid 0: {"prefix": "status"}) -- 0x558d4f9c3520 con 0x558d4fc29a30 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fc29a30 0x558d4fc29e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb73e00640 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fc29a30 0x558d4fc29e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6a86ca17c7051a server_cookie=ca7f76d242a92ee7 in_seq=0 out_seq=0 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 10 client.8097 ms_handle_connect on v2:172.21.15.97:6832/2383941694 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 1 -- 172.21.15.97:0/2625057088 <== mds.0 v2:172.21.15.97:6832/2383941694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x558d4f9c3520 con 0x558d4fc29a30 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 10 client.8097 handle_command_reply: tid=0 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 client.8097 shutdown 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 msgr2=0x558d4fbcaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbcaec0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c05aa60 tx=0x7ffb5c05d080 comp rx=0 tx=0).stop 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 shutdown_connections 2024-09-10T21:31:47.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:6832/2383941694,v1:172.21.15.97:6834/2383941694] conn(0x558d4fc29a30 0x558d4fc29e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d4fbb42f0 0x558d4fbe5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d4fbb39d0 0x558d4fbcaec0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb61600640 0 client.8097 ms_handle_reset on v2:172.21.15.97:6832/2383941694 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 --2- 172.21.15.97:0/2625057088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d4fb67e10 0x558d4fbca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 >> 172.21.15.97:0/2625057088 conn(0x558d4fa51130 msgr2=0x558d4fbc5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 shutdown_connections 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 1 -- 172.21.15.97:0/2625057088 wait complete. 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb7e3f3000 20 client.8097 trim_cache size 0 max 16384 2024-09-10T21:31:47.624 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8025, 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7756891420000001, 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 132, 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 103, 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 103, 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 8.7388732069999993 2024-09-10T21:31:47.625 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:31:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb500725e0 msgr2=0x7ffb50074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb500725e0 0x7ffb50074aa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ffb640af000 tx=0x7ffb6401b000 comp rx=0 tx=0).stop 2024-09-10T21:31:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 msgr2=0x7ffb740b7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740b7c40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c001ec0 tx=0x7ffb5c00c890 comp rx=0 tx=0).stop 2024-09-10T21:31:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 shutdown_connections 2024-09-10T21:31:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb500725e0 0x7ffb50074aa0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb741614f0 0x7ffb74153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb74161110 0x7ffb740b7c40 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 --2- 172.21.15.97:0/1211070129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb740d29d0 0x7ffb740c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.616+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 >> 172.21.15.97:0/1211070129 conn(0x7ffb740d3ce0 msgr2=0x7ffb74159840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.620+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 shutdown_connections 2024-09-10T21:31:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.620+0000 7ffb79a00640 1 -- 172.21.15.97:0/1211070129 wait complete. 2024-09-10T21:31:47.640 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:31:47.640 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2024-09-10T21:31:47.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- 172.21.15.97:0/2526797136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab815fde0 msgr2=0x7f8ab813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 --2- 172.21.15.97:0/2526797136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab815fde0 0x7f8ab813f9b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8074ae0 tx=0x7f8ab001cd80 comp rx=0 tx=0).stop 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- 172.21.15.97:0/2526797136 shutdown_connections 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 --2- 172.21.15.97:0/2526797136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab815fde0 0x7f8ab813f9b0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 --2- 172.21.15.97:0/2526797136 >> v2:172.21.15.194:3300/0 conn(0x7f8ab80ce5f0 0x7f8ab8141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- 172.21.15.97:0/2526797136 >> 172.21.15.97:0/2526797136 conn(0x7f8ab80b1780 msgr2=0x7f8ab80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- 172.21.15.97:0/2526797136 shutdown_connections 2024-09-10T21:31:47.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- 172.21.15.97:0/2526797136 wait complete. 2024-09-10T21:31:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 Processor -- start 2024-09-10T21:31:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- start start 2024-09-10T21:31:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ab80ce5f0 0x7f8ab8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab80ce9d0 0x7f8ab8139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ab815fde0 0x7f8ab815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8168a70 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8140cd0 con 0x7f8ab80ce5f0 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8abd400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab8168bf0 con 0x7f8ab815fde0 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8aade00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab80ce9d0 0x7f8ab8139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8aade00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab80ce9d0 0x7f8ab8139bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56448/0 (socket says 172.21.15.97:56448) 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8aade00640 1 -- 172.21.15.97:0/3773362102 learned_addr learned my addr 172.21.15.97:0/3773362102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8ab6800640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ab80ce5f0 0x7f8ab8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8ab7200640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ab815fde0 0x7f8ab815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.780+0000 7f8aade00640 1 -- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ab815fde0 msgr2=0x7f8ab815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aade00640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ab815fde0 0x7f8ab815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aade00640 1 -- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ab80ce5f0 msgr2=0x7f8ab8154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aade00640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ab80ce5f0 0x7f8ab8154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aade00640 1 -- 172.21.15.97:0/3773362102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab81f9610 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8ab7200640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ab815fde0 0x7f8ab815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8ab6800640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ab80ce5f0 0x7f8ab8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aade00640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab80ce9d0 0x7f8ab8139bf0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c500 tx=0x7f8aa000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aaf400640 1 -- 172.21.15.97:0/3773362102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0017070 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aaf400640 1 -- 172.21.15.97:0/3773362102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aa00053c0 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aaf400640 1 -- 172.21.15.97:0/3773362102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0012890 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab8154510 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab8154ad0 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab8154cd0 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aaf400640 1 -- 172.21.15.97:0/3773362102 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8aa0003600 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aaf400640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a940725e0 0x7f8a94074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.784+0000 7f8aaf400640 1 -- 172.21.15.97:0/3773362102 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7f8aa0092cb0 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.788+0000 7f8ab6800640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a940725e0 0x7f8a94074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:47.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.788+0000 7f8ab6800640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a940725e0 0x7f8a94074aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009fd0 tx=0x7f8aa80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.788+0000 7f8aaf400640 1 -- 172.21.15.97:0/3773362102 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ab8154cd0 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.992+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ab8141d20 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.992+0000 7f8aaf400640 1 -- 172.21.15.97:0/3773362102 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) ==== 53+0+73 (secure 0 0 0) 0x7f8aa005d820 con 0x7f8ab80ce9d0 2024-09-10T21:31:47.996 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:31:47.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a940725e0 msgr2=0x7f8a94074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a940725e0 0x7f8a94074aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009fd0 tx=0x7f8aa80091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab80ce9d0 msgr2=0x7f8ab8139bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:47.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab80ce9d0 0x7f8ab8139bf0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c500 tx=0x7f8aa000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 shutdown_connections 2024-09-10T21:31:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a940725e0 0x7f8a94074aa0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8ab815fde0 0x7f8ab815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ab80ce9d0 0x7f8ab8139bf0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 --2- 172.21.15.97:0/3773362102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ab80ce5f0 0x7f8ab8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 >> 172.21.15.97:0/3773362102 conn(0x7f8ab80b1780 msgr2=0x7f8ab80b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 shutdown_connections 2024-09-10T21:31:47.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:47.996+0000 7f8abd400640 1 -- 172.21.15.97:0/3773362102 wait complete. 2024-09-10T21:31:48.017 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:31:48.017 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:31:48.017 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:31:48.017 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:31:48.017 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:48.017 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:31:48.018 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:31:48.018 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:31:48.018 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:48.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.018 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:31:48.027 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:31:48.027 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:31:48.027 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:48.028 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41985sec preferred_lft 41985sec 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.029 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:31:48.030 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:31:48.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.031 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:31:48.094 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:31:48.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.096 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:31:48.148 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:31:48.148 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:31:48.148 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:31:48.148 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:31:48.148 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:31:48.148 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:31:48.148 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:31:48.149 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:48.149 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:48.149 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:31:48.149 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:31:48.149 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:31:48.149 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:31:48.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:31:48.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:31:48.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:31:48.150 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:31:48.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:31:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:31:48.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.282 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:31:48.292 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:31:48.292 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:31:48.293 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:48.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:31:48.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:31:48.365 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:48.417 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:31:48.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:31:48.417 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:48.468 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:31:48.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:48.557 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:48.557 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:48.557 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:48.572 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:48.572 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:48.572 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:48.572 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:31:48.572 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:31:48.582 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:31:48.582 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:31:48.582 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:31:48.582 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:31:48.582 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:31:48.582 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:31:48.582 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:31:48.583 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:31:48.586 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:31:48.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:31:48.606 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:31:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:31:48.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:31:48.704 INFO:teuthology.orchestra.run.smithi097.stdout:client7525 2024-09-10T21:31:48.705 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:48.705 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:48.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:48.714 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:31:48.759 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:31:48.766 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:31:48.767 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:31:48.767 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:48.767 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:31:48.767 INFO:teuthology.orchestra.run.smithi097.stdout:sixmegs 2024-09-10T21:31:48.768 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2024-09-10T21:31:48.768 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2024-09-10T21:31:48.768 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:48.768 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:48.821 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:48.821 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:48.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 -- 172.21.15.97:0/2323220421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43415bdd0 msgr2=0x7fb434074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 --2- 172.21.15.97:0/2323220421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43415bdd0 0x7fb434074380 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb424009900 tx=0x7fb42401c8b0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 -- 172.21.15.97:0/2323220421 shutdown_connections 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 --2- 172.21.15.97:0/2323220421 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4340b0110 0x7fb4340cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 --2- 172.21.15.97:0/2323220421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb43415bdd0 0x7fb434074380 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 -- 172.21.15.97:0/2323220421 >> 172.21.15.97:0/2323220421 conn(0x7fb4340b28c0 msgr2=0x7fb4340b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 -- 172.21.15.97:0/2323220421 shutdown_connections 2024-09-10T21:31:49.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 -- 172.21.15.97:0/2323220421 wait complete. 2024-09-10T21:31:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 Processor -- start 2024-09-10T21:31:49.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.056+0000 7fb439e00640 1 -- start start 2024-09-10T21:31:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4340b0110 0x7fb4341f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4340cfad0 0x7fb4340b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb43415bdd0 0x7fb43415eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb434168bc0 con 0x7fb4340cfad0 2024-09-10T21:31:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb434074a60 con 0x7fb43415bdd0 2024-09-10T21:31:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb434168d40 con 0x7fb4340b0110 2024-09-10T21:31:49.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4340cfad0 0x7fb4340b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4340cfad0 0x7fb4340b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56464/0 (socket says 172.21.15.97:56464) 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 -- 172.21.15.97:0/3115029160 learned_addr learned my addr 172.21.15.97:0/3115029160 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb433400640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb43415bdd0 0x7fb43415eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb432a00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4340b0110 0x7fb4341f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 -- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4340b0110 msgr2=0x7fb4341f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4340b0110 0x7fb4341f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 -- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb43415bdd0 msgr2=0x7fb43415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb43415bdd0 0x7fb43415eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 -- 172.21.15.97:0/3115029160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43415d1e0 con 0x7fb4340cfad0 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb432a00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4340b0110 0x7fb4341f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb433400640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb43415bdd0 0x7fb43415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:49.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42be00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4340cfad0 0x7fb4340b50e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c500 tx=0x7fb41c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:49.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42b400640 1 -- 172.21.15.97:0/3115029160 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c017070 con 0x7fb4340cfad0 2024-09-10T21:31:49.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42b400640 1 -- 172.21.15.97:0/3115029160 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb41c003600 con 0x7fb4340cfad0 2024-09-10T21:31:49.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43415d4c0 con 0x7fb4340cfad0 2024-09-10T21:31:49.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42b400640 1 -- 172.21.15.97:0/3115029160 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c003a90 con 0x7fb4340cfad0 2024-09-10T21:31:49.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4340b4bc0 con 0x7fb4340cfad0 2024-09-10T21:31:49.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4340cff40 con 0x7fb4340cfad0 2024-09-10T21:31:49.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.060+0000 7fb42b400640 1 -- 172.21.15.97:0/3115029160 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb41c003c30 con 0x7fb4340cfad0 2024-09-10T21:31:49.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.064+0000 7fb42b400640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb410072610 0x7fb410074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.064+0000 7fb42b400640 1 -- 172.21.15.97:0/3115029160 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7fb41c092e60 con 0x7fb4340cfad0 2024-09-10T21:31:49.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.064+0000 7fb432a00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb410072610 0x7fb410074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.064+0000 7fb432a00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb410072610 0x7fb410074ad0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb4240af000 tx=0x7fb42401b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:49.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.068+0000 7fb42b400640 1 -- 172.21.15.97:0/3115029160 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb41c05e160 con 0x7fb4340cfad0 2024-09-10T21:31:49.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.248+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb43415d690 con 0x7fb4340cfad0 2024-09-10T21:31:49.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.248+0000 7fb42b400640 1 -- 172.21.15.97:0/3115029160 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v103) ==== 81+0+848 (secure 0 0 0) 0x7fb41c05da60 con 0x7fb4340cfad0 2024-09-10T21:31:49.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:31:49.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:31:49.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:31:49.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:31:49.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:31:49.253 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:listed 14 entries 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb410072610 msgr2=0x7fb410074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb410072610 0x7fb410074ad0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb4240af000 tx=0x7fb42401b0f0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4340cfad0 msgr2=0x7fb4340b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4340cfad0 0x7fb4340b50e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00c500 tx=0x7fb41c00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 shutdown_connections 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb410072610 0x7fb410074ad0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb43415bdd0 0x7fb43415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4340cfad0 0x7fb4340b50e0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 --2- 172.21.15.97:0/3115029160 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb4340b0110 0x7fb4341f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 >> 172.21.15.97:0/3115029160 conn(0x7fb4340b28c0 msgr2=0x7fb4341528e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 shutdown_connections 2024-09-10T21:31:49.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.252+0000 7fb439e00640 1 -- 172.21.15.97:0/3115029160 wait complete. 2024-09-10T21:31:49.274 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:31:49.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:49.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:31:49.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:49.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:49.297 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:49.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 -- 172.21.15.97:0/1834169066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 msgr2=0x7f9754003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 --2- 172.21.15.97:0/1834169066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 0x7f9754003e80 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9760009ee0 tx=0x7f976001d5f0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f976f200640 1 -- 172.21.15.97:0/1834169066 reap_dead start 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 -- 172.21.15.97:0/1834169066 shutdown_connections 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 -- 172.21.15.97:0/1834169066 >> 172.21.15.97:0/1834169066 conn(0x7f97700b1780 msgr2=0x7f97700b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 -- 172.21.15.97:0/1834169066 shutdown_connections 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 -- 172.21.15.97:0/1834169066 wait complete. 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 Processor -- start 2024-09-10T21:31:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.536+0000 7f9775400640 1 -- start start 2024-09-10T21:31:49.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 0x7f9770159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97701eb7e0 0x7f97701ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f977015bcd0 0x7f977015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9770165be0 con 0x7f9754003a60 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9770165a60 con 0x7f977015bcd0 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9770165d60 con 0x7f97701eb7e0 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 0x7f9770159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 0x7f9770159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56482/0 (socket says 172.21.15.97:56482) 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 -- 172.21.15.97:0/1114629789 learned_addr learned my addr 172.21.15.97:0/1114629789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 -- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97701eb7e0 msgr2=0x7f97701ebc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9765e00640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97701eb7e0 0x7f97701ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976f200640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f977015bcd0 0x7f977015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97701eb7e0 0x7f97701ebc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 -- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f977015bcd0 msgr2=0x7f977015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f977015bcd0 0x7f977015c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 -- 172.21.15.97:0/1114629789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977015a960 con 0x7f9754003a60 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9765e00640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97701eb7e0 0x7f97701ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976f200640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f977015bcd0 0x7f977015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f976e800640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 0x7f9770159f70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f97600ae040 tx=0x7f9760009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:49.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9767400640 1 -- 172.21.15.97:0/1114629789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760003980 con 0x7f9754003a60 2024-09-10T21:31:49.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9767400640 1 -- 172.21.15.97:0/1114629789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9760003b20 con 0x7f9754003a60 2024-09-10T21:31:49.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977015c8b0 con 0x7f9754003a60 2024-09-10T21:31:49.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9767400640 1 -- 172.21.15.97:0/1114629789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760004030 con 0x7f9754003a60 2024-09-10T21:31:49.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97700c9180 con 0x7f9754003a60 2024-09-10T21:31:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.540+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97700b2620 con 0x7f9754003a60 2024-09-10T21:31:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.544+0000 7f9767400640 1 -- 172.21.15.97:0/1114629789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f97600c7030 con 0x7f9754003a60 2024-09-10T21:31:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.544+0000 7f9767400640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f974c072610 0x7f974c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.544+0000 7f9767400640 1 -- 172.21.15.97:0/1114629789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6877+0+0 (secure 0 0 0) 0x7f976012edf0 con 0x7f9754003a60 2024-09-10T21:31:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.544+0000 7f9765e00640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f974c072610 0x7f974c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.544+0000 7f9765e00640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f974c072610 0x7f974c074ad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f97701ecbb0 tx=0x7f9758000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.544+0000 7f9767400640 1 -- 172.21.15.97:0/1114629789 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f977015a960 con 0x7f9754003a60 2024-09-10T21:31:49.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.744+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9770142b70 con 0x7f9754003a60 2024-09-10T21:31:49.747 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:49.744+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9767400640 1 -- 172.21.15.97:0/1114629789 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v133) ==== 161+0+0 (secure 0 0 0) 0x7f9760017300 con 0x7f9754003a60 2024-09-10T21:31:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:49.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f974c072610 msgr2=0x7f974c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f974c072610 0x7f974c074ad0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f97701ecbb0 tx=0x7f9758000ee0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 msgr2=0x7f9770159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 0x7f9770159f70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f97600ae040 tx=0x7f9760009500 comp rx=0 tx=0).stop 2024-09-10T21:31:49.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 shutdown_connections 2024-09-10T21:31:49.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f974c072610 0x7f974c074ad0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f977015bcd0 0x7f977015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97701eb7e0 0x7f97701ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 --2- 172.21.15.97:0/1114629789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9754003a60 0x7f9770159f70 secure :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f97600ae040 tx=0x7f9760009500 comp rx=0 tx=0).stop 2024-09-10T21:31:49.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 >> 172.21.15.97:0/1114629789 conn(0x7f97700b1780 msgr2=0x7f97700b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:49.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 shutdown_connections 2024-09-10T21:31:49.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.748+0000 7f9775400640 1 -- 172.21.15.97:0/1114629789 wait complete. 2024-09-10T21:31:49.757 INFO:tasks.ceph.mds.a.smithi097.stderr: -686> 2024-09-10T21:31:44.840+0000 7f2154400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:31:49.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:49.799 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:31:49.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.920+0000 7f1ce0600640 1 -- 172.21.15.97:0/3638214227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0004d50 msgr2=0x7f1cc00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.920+0000 7f1ce0600640 1 --2- 172.21.15.97:0/3638214227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc0004d50 0x7f1cc00051d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009ee0 tx=0x7f1cd001d5f0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.920+0000 7f1cde600640 1 -- 172.21.15.97:0/3638214227 reap_dead start 2024-09-10T21:31:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.920+0000 7f1ce0600640 1 -- 172.21.15.97:0/3638214227 shutdown_connections 2024-09-10T21:31:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.920+0000 7f1ce0600640 1 -- 172.21.15.97:0/3638214227 >> 172.21.15.97:0/3638214227 conn(0x7f1cd80b1780 msgr2=0x7f1cd80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.920+0000 7f1ce0600640 1 -- 172.21.15.97:0/3638214227 shutdown_connections 2024-09-10T21:31:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.920+0000 7f1ce0600640 1 -- 172.21.15.97:0/3638214227 wait complete. 2024-09-10T21:31:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 Processor -- start 2024-09-10T21:31:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 -- start start 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc0004d50 0x7f1cd815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd80cd5f0 0x7f1cd8073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cd80cda70 0x7f1cd8159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8070360 con 0x7f1cd80cda70 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd80701e0 con 0x7f1cd80cd5f0 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd80704e0 con 0x7f1cc0004d50 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cd80cda70 0x7f1cd8159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cd80cda70 0x7f1cd8159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56524/0 (socket says 172.21.15.97:56524) 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 -- 172.21.15.97:0/2650293342 learned_addr learned my addr 172.21.15.97:0/2650293342 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cddc00640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc0004d50 0x7f1cd815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cdd200640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd80cd5f0 0x7f1cd8073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 -- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc0004d50 msgr2=0x7f1cd815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc0004d50 0x7f1cd815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 -- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd80cd5f0 msgr2=0x7f1cd8073500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:49.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd80cd5f0 0x7f1cd8073500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:49.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 -- 172.21.15.97:0/2650293342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd81f7ef0 con 0x7f1cd80cda70 2024-09-10T21:31:49.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cde600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cd80cda70 0x7f1cd8159cf0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009ee0 tx=0x7f1cd0004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:49.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cd5600640 1 -- 172.21.15.97:0/2650293342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00ad070 con 0x7f1cd80cda70 2024-09-10T21:31:49.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cd5600640 1 -- 172.21.15.97:0/2650293342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cd00a7a90 con 0x7f1cd80cda70 2024-09-10T21:31:49.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd81f81d0 con 0x7f1cd80cda70 2024-09-10T21:31:49.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1cd5600640 1 -- 172.21.15.97:0/2650293342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00a7e40 con 0x7f1cd80cda70 2024-09-10T21:31:49.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.924+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81f8790 con 0x7f1cd80cda70 2024-09-10T21:31:49.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.928+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd80b2fe0 con 0x7f1cd80cda70 2024-09-10T21:31:49.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.928+0000 7f1cd5600640 1 -- 172.21.15.97:0/2650293342 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1cd0003980 con 0x7f1cd80cda70 2024-09-10T21:31:49.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.928+0000 7f1cd5600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cb8072640 0x7f1cb8074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:49.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.928+0000 7f1cd5600640 1 -- 172.21.15.97:0/2650293342 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6963+0+0 (secure 0 0 0) 0x7f1cd012ea20 con 0x7f1cd80cda70 2024-09-10T21:31:49.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.928+0000 7f1cddc00640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cb8072640 0x7f1cb8074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:49.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.928+0000 7f1cddc00640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cb8072640 0x7f1cb8074b00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc001cd0 tx=0x7f1ccc006730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:49.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:49.932+0000 7f1cd5600640 1 -- 172.21.15.97:0/2650293342 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1cd0017300 con 0x7f1cd80cda70 2024-09-10T21:31:50.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.156+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f1cd8072b80 con 0x7f1cd80cda70 2024-09-10T21:31:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.748+0000 7f1cd5600640 1 -- 172.21.15.97:0/2650293342 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v134) ==== 107+0+0 (secure 0 0 0) 0x7f1cd0102290 con 0x7f1cd80cda70 2024-09-10T21:31:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:31:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cb8072640 msgr2=0x7f1cb8074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cb8072640 0x7f1cb8074b00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc001cd0 tx=0x7f1ccc006730 comp rx=0 tx=0).stop 2024-09-10T21:31:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cd80cda70 msgr2=0x7f1cd8159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cd80cda70 0x7f1cd8159cf0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009ee0 tx=0x7f1cd0004f50 comp rx=0 tx=0).stop 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 shutdown_connections 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cb8072640 0x7f1cb8074b00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cd80cda70 0x7f1cd8159cf0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd80cd5f0 0x7f1cd8073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 --2- 172.21.15.97:0/2650293342 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1cc0004d50 0x7f1cd815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 >> 172.21.15.97:0/2650293342 conn(0x7f1cd80b1780 msgr2=0x7f1cd80b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 shutdown_connections 2024-09-10T21:31:50.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.752+0000 7f1ce0600640 1 -- 172.21.15.97:0/2650293342 wait complete. 2024-09-10T21:31:50.774 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 Processor -- start 2024-09-10T21:31:50.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- start start 2024-09-10T21:31:50.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a518940 0x56138a51a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a2b00 0x56138a4e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a5a32a0 0x56138a4e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56138a437a00 con 0x56138a5a2b00 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56138a495f40 con 0x56138a518940 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56138a5a5080 con 0x56138a5a32a0 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a2b00 0x56138a4e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a2b00 0x56138a4e6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56546/0 (socket says 172.21.15.97:56546) 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 -- 172.21.15.97:0/4169809892 learned_addr learned my addr 172.21.15.97:0/4169809892 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 -- 172.21.15.97:0/4169809892 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a5a32a0 msgr2=0x56138a4e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 --2- 172.21.15.97:0/4169809892 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a5a32a0 0x56138a4e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 -- 172.21.15.97:0/4169809892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a518940 msgr2=0x56138a51a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:50.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 --2- 172.21.15.97:0/4169809892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a518940 0x56138a51a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 -- 172.21.15.97:0/4169809892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138a410c10 con 0x56138a5a2b00 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559f600640 1 --2- 172.21.15.97:0/4169809892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a2b00 0x56138a4e6d90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f558c004f60 tx=0x7f558c017f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ef41a453c28743 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559e200640 1 -- 172.21.15.97:0/4169809892 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0b0040 con 0x56138a5a2b00 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559e200640 1 -- 172.21.15.97:0/4169809892 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f558c0acb20 con 0x56138a5a2b00 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f559e200640 1 -- 172.21.15.97:0/4169809892 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0b5410 con 0x56138a5a2b00 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- 172.21.15.97:0/4169809892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a2b00 msgr2=0x56138a4e6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 --2- 172.21.15.97:0/4169809892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a2b00 0x56138a4e6d90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f558c004f60 tx=0x7f558c017f20 comp rx=0 tx=0).stop 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- 172.21.15.97:0/4169809892 shutdown_connections 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 --2- 172.21.15.97:0/4169809892 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a5a32a0 0x56138a4e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 --2- 172.21.15.97:0/4169809892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a2b00 0x56138a4e6d90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 --2- 172.21.15.97:0/4169809892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a518940 0x56138a51a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- 172.21.15.97:0/4169809892 >> 172.21.15.97:0/4169809892 conn(0x56138a517520 msgr2=0x56138a5bf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- 172.21.15.97:0/4169809892 shutdown_connections 2024-09-10T21:31:50.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.860+0000 7f55a2d19080 1 -- 172.21.15.97:0/4169809892 wait complete. 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 Processor -- start 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 -- start start 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a518940 0x56138a5a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a5a2b00 0x56138a51b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a32a0 0x56138a51b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56138a5bc2f0 con 0x56138a5a32a0 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56138a5c2000 con 0x56138a5a2b00 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56138a5c22a0 con 0x56138a518940 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a32a0 0x56138a51b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a32a0 0x56138a51b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56552/0 (socket says 172.21.15.97:56552) 2024-09-10T21:31:50.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 learned_addr learned my addr 172.21.15.97:0/3939206759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0000640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a518940 0x56138a5a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f559f600640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a5a2b00 0x56138a51b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a518940 msgr2=0x56138a5a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56138a518940 0x56138a5a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a5a2b00 msgr2=0x56138a51b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56138a5a2b00 0x56138a51b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138a5c4320 con 0x56138a5a32a0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a0a00640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a32a0 0x56138a51b7a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5598004f30 tx=0x7f559800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f559ce00640 1 -- 172.21.15.97:0/3939206759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559800f870 con 0x56138a5a32a0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f559ce00640 1 -- 172.21.15.97:0/3939206759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f559800ae50 con 0x56138a5a32a0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56138a5c45c0 con 0x56138a5a32a0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f559ce00640 1 -- 172.21.15.97:0/3939206759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559800fdc0 con 0x56138a5a32a0 2024-09-10T21:31:50.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.864+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138a5a2310 con 0x56138a5a32a0 2024-09-10T21:31:50.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f559ce00640 1 -- 172.21.15.97:0/3939206759 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f55980203f0 con 0x56138a5a32a0 2024-09-10T21:31:50.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f559ce00640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5580070ff0 0x7f55800734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f559ce00640 1 -- 172.21.15.97:0/3939206759 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6963+0+0 (secure 0 0 0) 0x7f5598092b30 con 0x56138a5a32a0 2024-09-10T21:31:50.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f55a0000640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5580070ff0 0x7f55800734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.872 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:50.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56138a5bab20 0x56138a5c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a5c4fa0 con 0x56138a5bab20 2024-09-10T21:31:50.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f559f600640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56138a5bab20 0x56138a5c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f55a0000640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5580070ff0 0x7f55800734b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f559000e4d0 tx=0x7f5590002730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.868+0000 7f559f600640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56138a5bab20 0x56138a5c3630 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.872+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v104'226 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.872+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a5c3b70 con 0x56138a5bab20 2024-09-10T21:31:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.872+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v104'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.872+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a665a90 con 0x56138a5bab20 2024-09-10T21:31:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.876+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v104'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.877 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:50.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.876+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666e40 con 0x56138a5bab20 2024-09-10T21:31:50.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.876+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.876+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6671e0 con 0x56138a5bab20 2024-09-10T21:31:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.876+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.876+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a665e30 con 0x56138a5bab20 2024-09-10T21:31:50.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.880+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.880+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6661d0 con 0x56138a5bab20 2024-09-10T21:31:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.880+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v104'232 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.880+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.880+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v104'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.880+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.884+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v104'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.886 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:50.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.884+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.884+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.884+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.888+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.888+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.888+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.888+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.888+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v104'238 uv213 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.888+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.892+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'239 uv239 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.892+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.892+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v104'240 uv240 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.895 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.892+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.892+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'241 uv241 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.892+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.896+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'242 uv242 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.896+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666570 con 0x56138a5bab20 2024-09-10T21:31:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.896+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'243 uv243 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.896+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a666940 con 0x56138a5bab20 2024-09-10T21:31:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.896+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'244 uv244 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.896+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.900+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v104'245 uv219 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.900+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.900+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v104'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.900+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.900+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v104'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.904 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.900+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.904+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.904+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.904+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.904+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.904+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.908+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.908+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v104'251 uv225 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.908+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.908+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v104'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.908+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.913 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:50.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.912+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v104'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.912+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.912+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.912+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.912+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.912+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6667f0 con 0x56138a5bab20 2024-09-10T21:31:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.916+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a5bab20 2024-09-10T21:31:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.916+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56138a666960 0x56138a669aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.916+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a669fe0 con 0x56138a666960 2024-09-10T21:31:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.916+0000 7f55a0000640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56138a666960 0x56138a669aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.916+0000 7f55a0000640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56138a666960 0x56138a669aa0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.916+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'177 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.916+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a66a380 con 0x56138a666960 2024-09-10T21:31:50.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.920+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.920+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66a720 con 0x56138a666960 2024-09-10T21:31:50.923 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.920+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.920+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66af70 con 0x56138a666960 2024-09-10T21:31:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.920+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v104'180 uv180 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.920+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66b310 con 0x56138a666960 2024-09-10T21:31:50.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.924+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'181 uv181 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.924+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66b6b0 con 0x56138a666960 2024-09-10T21:31:50.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.924+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v104'182 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.924+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a66ba50 con 0x56138a666960 2024-09-10T21:31:50.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.924+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v104'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.924+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.929 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.928+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v104'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.928+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.928+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v104'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.928+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.928+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v104'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.928+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.932+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v104'187 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.932+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.932+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v104'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.932+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.935 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:50.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.932+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v104'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.932+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66ffc0 con 0x56138a666960 2024-09-10T21:31:50.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.936+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v104'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.936+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66ffc0 con 0x56138a666960 2024-09-10T21:31:50.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.936+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v104'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.936+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.936+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v104'192 uv172 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.936+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v104'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.942 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v104'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66ce00 con 0x56138a666960 2024-09-10T21:31:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v104'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66ce00 con 0x56138a666960 2024-09-10T21:31:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v104'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.940+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66bdf0 con 0x56138a666960 2024-09-10T21:31:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.944+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v104'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a666960 2024-09-10T21:31:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.944+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56138a66c350 0x56138a66e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.944+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66ed60 con 0x56138a66c350 2024-09-10T21:31:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.944+0000 7f559f600640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56138a66c350 0x56138a66e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.944+0000 7f559f600640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56138a66c350 0x56138a66e740 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.944+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.944+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a66ee20 con 0x56138a66c350 2024-09-10T21:31:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.948+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.948+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66f240 con 0x56138a66c350 2024-09-10T21:31:50.950 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.948+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.948+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a670250 con 0x56138a66c350 2024-09-10T21:31:50.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.948+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.948+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a670250 con 0x56138a66c350 2024-09-10T21:31:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a66f240 con 0x56138a66c350 2024-09-10T21:31:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56138a66f8c0 0x56138a671cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a672280 con 0x56138a66f8c0 2024-09-10T21:31:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f55a0a00640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56138a66f8c0 0x56138a671cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f55a0a00640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56138a66f8c0 0x56138a671cb0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'40 uv39 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.952+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a672790 con 0x56138a66f8c0 2024-09-10T21:31:50.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.956+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v104'41 uv41 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.956+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a672c90 con 0x56138a66f8c0 2024-09-10T21:31:50.959 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:50.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.956+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v104'42 uv42 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.956+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a673550 con 0x56138a66f8c0 2024-09-10T21:31:50.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.956+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v104'43 uv43 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.956+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6738f0 con 0x56138a66c350 2024-09-10T21:31:50.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a6738f0 con 0x56138a66c350 2024-09-10T21:31:50.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6738f0 con 0x56138a66c350 2024-09-10T21:31:50.963 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:50.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a674900 con 0x56138a66c350 2024-09-10T21:31:50.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.960+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a674900 con 0x56138a66c350 2024-09-10T21:31:50.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.964+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.964+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6738f0 con 0x56138a66c350 2024-09-10T21:31:50.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.964+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.964+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6738f0 con 0x56138a66f8c0 2024-09-10T21:31:50.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.964+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v104'83 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.964+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a673c90 con 0x56138a66f8c0 2024-09-10T21:31:50.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.968+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v104'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.968+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a674030 con 0x56138a66f8c0 2024-09-10T21:31:50.971 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:50.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.968+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v104'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.968+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6753e0 con 0x56138a66f8c0 2024-09-10T21:31:50.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.968+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v104'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.968+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6753e0 con 0x56138a66f8c0 2024-09-10T21:31:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v104'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66f8c0 2024-09-10T21:31:50.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v104'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66f8c0 2024-09-10T21:31:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v104'89 uv82 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66f8c0 2024-09-10T21:31:50.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v104'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.972+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66f8c0 2024-09-10T21:31:50.977 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.976+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v104'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.976+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66f8c0 2024-09-10T21:31:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.976+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v104'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.976+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66f8c0 2024-09-10T21:31:50.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.976+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v104'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:50.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.976+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66c350 2024-09-10T21:31:50.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v104'37 uv36 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66c350 2024-09-10T21:31:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v104'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66c350 2024-09-10T21:31:50.983 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v104'39 uv39 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66c350 2024-09-10T21:31:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v104'40 uv40 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.980+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a6743d0 con 0x56138a66c350 2024-09-10T21:31:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.984+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v104'41 uv41 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a66c350 2024-09-10T21:31:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.984+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56138a674a50 0x56138a676e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.984+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677460 con 0x56138a674a50 2024-09-10T21:31:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.984+0000 7f55a0000640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56138a674a50 0x56138a676e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:50.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.984+0000 7f55a0000640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56138a674a50 0x56138a676e40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.984+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.984+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:50.990 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a678840 con 0x56138a674a50 2024-09-10T21:31:50.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a678840 con 0x56138a674a50 2024-09-10T21:31:50.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.988+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.992+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.992+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.992+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v104'74 uv68 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.992+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.992+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v104'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.992+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.996+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v104'76 uv76 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.997 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.996+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677890 con 0x56138a674a50 2024-09-10T21:31:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.996+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v104'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.996+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677850 con 0x56138a674a50 2024-09-10T21:31:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.996+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v104'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:50.996+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677850 con 0x56138a674a50 2024-09-10T21:31:51.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v104'79 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a677850 con 0x56138a674a50 2024-09-10T21:31:51.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v104'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:51.003 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v104'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677830 con 0x56138a674a50 2024-09-10T21:31:51.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v104'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.000+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67a6f0 con 0x56138a674a50 2024-09-10T21:31:51.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.004+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v104'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.004+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67a6b0 con 0x56138a66f8c0 2024-09-10T21:31:51.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.004+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v104'36 uv34 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.004+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a67a6b0 con 0x56138a66f8c0 2024-09-10T21:31:51.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.004+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v104'37 uv37 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.004+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67a6b0 con 0x56138a66f8c0 2024-09-10T21:31:51.009 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.004+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v104'38 uv38 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.008+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67a6b0 con 0x56138a66f8c0 2024-09-10T21:31:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.008+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v104'39 uv39 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.008+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677d10 con 0x56138a66f8c0 2024-09-10T21:31:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.008+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v104'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.008+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a677d10 con 0x56138a66f8c0 2024-09-10T21:31:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.008+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v104'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.008+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677d10 con 0x56138a66f8c0 2024-09-10T21:31:51.013 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v104'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a678d20 con 0x56138a66f8c0 2024-09-10T21:31:51.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v104'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a678d20 con 0x56138a66f8c0 2024-09-10T21:31:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a677d10 con 0x56138a66f8c0 2024-09-10T21:31:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5598002030 con 0x56138a66f8c0 2024-09-10T21:31:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56138a678390 0x56138a67a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.012+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67adb0 con 0x56138a678390 2024-09-10T21:31:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.016+0000 7f559f600640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56138a678390 0x56138a67a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.016+0000 7f559f600640 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56138a678390 0x56138a67a780 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.016+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v104'37 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a678390 2024-09-10T21:31:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.016+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a67adb0 con 0x56138a678390 2024-09-10T21:31:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.016+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v104'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a678390 2024-09-10T21:31:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.016+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67adb0 con 0x56138a678390 2024-09-10T21:31:51.021 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v104'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a678390 2024-09-10T21:31:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67adb0 con 0x56138a678390 2024-09-10T21:31:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v104'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a678390 2024-09-10T21:31:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67adb0 con 0x56138a678390 2024-09-10T21:31:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f559f600640 1 -- 172.21.15.97:0/3939206759 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v104'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f558c0b3040 con 0x56138a678390 2024-09-10T21:31:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67adb0 con 0x56138a674a50 2024-09-10T21:31:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v104'95 uv86 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.020+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a67adb0 con 0x56138a674a50 2024-09-10T21:31:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.024+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v104'96 uv96 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.024+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67ad80 con 0x56138a674a50 2024-09-10T21:31:51.027 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.024+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v104'97 uv97 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.024+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67ada0 con 0x56138a674a50 2024-09-10T21:31:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.024+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v104'98 uv98 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.024+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67b370 con 0x56138a674a50 2024-09-10T21:31:51.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.028+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v104'99 uv99 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.028+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67b370 con 0x56138a674a50 2024-09-10T21:31:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.028+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v104'100 uv93 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.028+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x56138a67b370 con 0x56138a674a50 2024-09-10T21:31:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.028+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v104'101 uv101 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.028+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67b370 con 0x56138a674a50 2024-09-10T21:31:51.033 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.032+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v104'102 uv102 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.032+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67b320 con 0x56138a674a50 2024-09-10T21:31:51.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.032+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v104'103 uv103 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.032+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67c330 con 0x56138a674a50 2024-09-10T21:31:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.036+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v104'104 uv104 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.036+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67b320 con 0x56138a674a50 2024-09-10T21:31:51.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.036+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v104'105 uv105 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.036+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x56138a67b710 con 0x56138a674a50 2024-09-10T21:31:51.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a0000640 1 -- 172.21.15.97:0/3939206759 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v104'106 uv106 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f559000d040 con 0x56138a674a50 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56138a66c350 msgr2=0x56138a66e740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56138a66c350 0x56138a66e740 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56138a674a50 msgr2=0x56138a676e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56138a674a50 0x56138a676e40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56138a678390 msgr2=0x56138a67a780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56138a678390 0x56138a67a780 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56138a66f8c0 msgr2=0x56138a671cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56138a66f8c0 0x56138a671cb0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56138a666960 msgr2=0x56138a669aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56138a666960 0x56138a669aa0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 reap_dead start 2024-09-10T21:31:51.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a0a00640 1 -- 172.21.15.97:0/3939206759 reap_dead start 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56138a5bab20 msgr2=0x56138a5c3630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56138a5bab20 0x56138a5c3630 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5580070ff0 msgr2=0x7f55800734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5580070ff0 0x7f55800734b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f559000e4d0 tx=0x7f5590002730 comp rx=0 tx=0).stop 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a32a0 msgr2=0x56138a51b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a32a0 0x56138a51b7a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5598004f30 tx=0x7f559800cf90 comp rx=0 tx=0).stop 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 shutdown_connections 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56138a5bab20 0x56138a5c3630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5580070ff0 0x7f55800734b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 --2- 172.21.15.97:0/3939206759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56138a5a32a0 0x56138a51b7a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 >> 172.21.15.97:0/3939206759 conn(0x56138a517520 msgr2=0x56138a4e8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 shutdown_connections 2024-09-10T21:31:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.040+0000 7f55a2d19080 1 -- 172.21.15.97:0/3939206759 wait complete. 2024-09-10T21:31:51.048 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2024-09-10T21:31:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 Processor -- start 2024-09-10T21:31:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- start start 2024-09-10T21:31:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 0x557f0edb1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 0x557f0edba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f0ecbea00 con 0x557f0ee2e1d0 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557f0ed1cf40 con 0x557f0ee2aa00 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557f0ee2c710 con 0x557f0ee28cc0 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56568/0 (socket says 172.21.15.97:56568) 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f87e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 0x557f0edb1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 -- 172.21.15.97:0/266078206 learned_addr learned my addr 172.21.15.97:0/266078206 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f87400640 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 0x557f0edba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 -- 172.21.15.97:0/266078206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 msgr2=0x557f0edb1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 0x557f0edb1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 -- 172.21.15.97:0/266078206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 msgr2=0x557f0edba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 0x557f0edba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 -- 172.21.15.97:0/266078206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0ec97c10 con 0x557f0ee2e1d0 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f88800640 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f000 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000c790 tx=0x7f9f8000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9b0ae59b684b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f86000640 1 -- 172.21.15.97:0/266078206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80013070 con 0x557f0ee2e1d0 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f86000640 1 -- 172.21.15.97:0/266078206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f8000ab30 con 0x557f0ee2e1d0 2024-09-10T21:31:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f86000640 1 -- 172.21.15.97:0/266078206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000f460 con 0x557f0ee2e1d0 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- 172.21.15.97:0/266078206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 msgr2=0x557f0ed9f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f000 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000c790 tx=0x7f9f8000cc50 comp rx=0 tx=0).stop 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- 172.21.15.97:0/266078206 shutdown_connections 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f000 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 0x557f0edba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/266078206 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 0x557f0edb1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- 172.21.15.97:0/266078206 >> 172.21.15.97:0/266078206 conn(0x557f0edb80d0 msgr2=0x557f0eda0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- 172.21.15.97:0/266078206 shutdown_connections 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 -- 172.21.15.97:0/266078206 wait complete. 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.132+0000 7f9f8ac03080 1 Processor -- start 2024-09-10T21:31:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 -- start start 2024-09-10T21:31:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 0x557f0edb3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 0x557f0ed9ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f0eda0740 con 0x557f0ee2e1d0 2024-09-10T21:31:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557f0ed73890 con 0x557f0ee2aa00 2024-09-10T21:31:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557f0eda09e0 con 0x557f0ee28cc0 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56578/0 (socket says 172.21.15.97:56578) 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 -- 172.21.15.97:0/1612410689 learned_addr learned my addr 172.21.15.97:0/1612410689 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f87e00640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 0x557f0edb3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f87400640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 0x557f0ed9ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 -- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 msgr2=0x557f0edb3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557f0ee28cc0 0x557f0edb3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 -- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 msgr2=0x557f0ed9ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557f0ee2aa00 0x557f0ed9ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 -- 172.21.15.97:0/1612410689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0ee3f1b0 con 0x557f0ee2e1d0 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f88800640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f4d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9f800116c0 tx=0x7f9f800116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f84c00640 1 -- 172.21.15.97:0/1612410689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80013040 con 0x557f0ee2e1d0 2024-09-10T21:31:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f84c00640 1 -- 172.21.15.97:0/1612410689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f80011890 con 0x557f0ee2e1d0 2024-09-10T21:31:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557f0ee3f450 con 0x557f0ee2e1d0 2024-09-10T21:31:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f84c00640 1 -- 172.21.15.97:0/1612410689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800112e0 con 0x557f0ee2e1d0 2024-09-10T21:31:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.136+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0ee3f990 con 0x557f0ee2e1d0 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f84c00640 1 -- 172.21.15.97:0/1612410689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9f80011ca0 con 0x557f0ee2e1d0 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f84c00640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f68070ff0 0x7f9f680734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f84c00640 1 -- 172.21.15.97:0/1612410689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6963+0+0 (secure 0 0 0) 0x7f9f800943c0 con 0x557f0ee2e1d0 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f87e00640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f68070ff0 0x7f9f680734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557f0ed71310 0x557f0ed716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x557f0ed71c30 con 0x557f0ed71310 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f87400640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557f0ed71310 0x557f0ed716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f87e00640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f68070ff0 0x7f9f680734b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9f780ab000 tx=0x7f9f78019ae0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f87400640 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557f0ed71310 0x557f0ed716f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f87400640 1 -- 172.21.15.97:0/1612410689 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(1 mds_snaptable [delete] v104'42 uv42 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9f7400d040 con 0x557f0ed71310 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557f0ed71310 msgr2=0x557f0ed716f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557f0ed71310 0x557f0ed716f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f68070ff0 msgr2=0x7f9f680734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9f68070ff0 0x7f9f680734b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9f780ab000 tx=0x7f9f78019ae0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 msgr2=0x557f0ed9f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 --2- 172.21.15.97:0/1612410689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f0ee2e1d0 0x557f0ed9f4d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9f800116c0 tx=0x7f9f800116f0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f88800640 1 -- 172.21.15.97:0/1612410689 reap_dead start 2024-09-10T21:31:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 shutdown_connections 2024-09-10T21:31:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 >> 172.21.15.97:0/1612410689 conn(0x557f0edb80d0 msgr2=0x557f0edbae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.140+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 shutdown_connections 2024-09-10T21:31:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.144+0000 7f9f8ac03080 1 -- 172.21.15.97:0/1612410689 wait complete. 2024-09-10T21:31:51.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:51.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- 172.21.15.97:0/3672295092 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb006dc50 msgr2=0x7f1bb00cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 --2- 172.21.15.97:0/3672295092 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb006dc50 0x7f1bb00cfe30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1b980097f0 tx=0x7f1b9801c5d0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- 172.21.15.97:0/3672295092 shutdown_connections 2024-09-10T21:31:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 --2- 172.21.15.97:0/3672295092 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb006dc50 0x7f1bb00cfe30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- 172.21.15.97:0/3672295092 >> 172.21.15.97:0/3672295092 conn(0x7f1bb00d1c30 msgr2=0x7f1bb00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- 172.21.15.97:0/3672295092 shutdown_connections 2024-09-10T21:31:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- 172.21.15.97:0/3672295092 wait complete. 2024-09-10T21:31:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 Processor -- start 2024-09-10T21:31:51.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- start start 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb00b1590 0x7f1bb00c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb0161600 0x7f1bb00c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb015dec0 0x7f1bb015e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb016c360 con 0x7f1bb00b1590 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb016c1e0 con 0x7f1bb015dec0 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1bb5800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb016c4e0 con 0x7f1bb0161600 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb00b1590 0x7f1bb00c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb00b1590 0x7f1bb00c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56582/0 (socket says 172.21.15.97:56582) 2024-09-10T21:31:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 -- 172.21.15.97:0/2438680826 learned_addr learned my addr 172.21.15.97:0/2438680826 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1ba6000640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb0161600 0x7f1bb00c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 -- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb0161600 msgr2=0x7f1bb00c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb0161600 0x7f1bb00c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 -- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb015dec0 msgr2=0x7f1bb015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baf400640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb015dec0 0x7f1bb015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb015dec0 0x7f1bb015e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 -- 172.21.15.97:0/2438680826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb015ef80 con 0x7f1bb00b1590 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baf400640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb015dec0 0x7f1bb015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.336+0000 7f1baea00640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb00b1590 0x7f1bb00c5920 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c3c0 tx=0x7f1ba000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1ba7400640 1 -- 172.21.15.97:0/2438680826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0017070 con 0x7f1bb00b1590 2024-09-10T21:31:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1ba7400640 1 -- 172.21.15.97:0/2438680826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ba0003600 con 0x7f1bb00b1590 2024-09-10T21:31:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb015f260 con 0x7f1bb00b1590 2024-09-10T21:31:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1ba7400640 1 -- 172.21.15.97:0/2438680826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0003a50 con 0x7f1bb00b1590 2024-09-10T21:31:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb014ec20 con 0x7f1bb00b1590 2024-09-10T21:31:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb00cfe30 con 0x7f1bb00b1590 2024-09-10T21:31:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1ba7400640 1 -- 172.21.15.97:0/2438680826 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ba0003bf0 con 0x7f1bb00b1590 2024-09-10T21:31:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1ba7400640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b8c072610 0x7f1b8c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1ba7400640 1 -- 172.21.15.97:0/2438680826 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6963+0+0 (secure 0 0 0) 0x7f1ba0092bf0 con 0x7f1bb00b1590 2024-09-10T21:31:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.340+0000 7f1ba6000640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b8c072610 0x7f1b8c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.344+0000 7f1ba6000640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b8c072610 0x7f1b8c074ad0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1b980ae000 tx=0x7f1b9801ae10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.344+0000 7f1ba7400640 1 -- 172.21.15.97:0/2438680826 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ba005de10 con 0x7f1bb00b1590 2024-09-10T21:31:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.556+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1bb00d2390 con 0x7f1bb00b1590 2024-09-10T21:31:51.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1ba7400640 1 -- 172.21.15.97:0/2438680826 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v135) ==== 162+0+0 (secure 0 0 0) 0x7f1ba005d710 con 0x7f1bb00b1590 2024-09-10T21:31:51.617 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b8c072610 msgr2=0x7f1b8c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b8c072610 0x7f1b8c074ad0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1b980ae000 tx=0x7f1b9801ae10 comp rx=0 tx=0).stop 2024-09-10T21:31:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb00b1590 msgr2=0x7f1bb00c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb00b1590 0x7f1bb00c5920 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c3c0 tx=0x7f1ba000c880 comp rx=0 tx=0).stop 2024-09-10T21:31:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 shutdown_connections 2024-09-10T21:31:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b8c072610 0x7f1b8c074ad0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb015dec0 0x7f1bb015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb0161600 0x7f1bb00c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 --2- 172.21.15.97:0/2438680826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb00b1590 0x7f1bb00c5920 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 >> 172.21.15.97:0/2438680826 conn(0x7f1bb00d1c30 msgr2=0x7f1bb00af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:51.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 shutdown_connections 2024-09-10T21:31:51.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.616+0000 7f1bb5800640 1 -- 172.21.15.97:0/2438680826 wait complete. 2024-09-10T21:31:51.622 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:31:51.620+0000 7f9191800640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-10T21:31:51.622 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:31:51.620+0000 7f9191800640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-10T21:31:51.625 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:51.624+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:51.625 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:51.624+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:31:51.642 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:51.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 -- 172.21.15.97:0/2740170936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c004d50 msgr2=0x7f2b8c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 --2- 172.21.15.97:0/2740170936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c004d50 0x7f2b8c0051d0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009ee0 tx=0x7f2ba001d6c0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baca00640 1 -- 172.21.15.97:0/2740170936 reap_dead start 2024-09-10T21:31:51.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 -- 172.21.15.97:0/2740170936 shutdown_connections 2024-09-10T21:31:51.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 -- 172.21.15.97:0/2740170936 >> 172.21.15.97:0/2740170936 conn(0x7f2ba80b37c0 msgr2=0x7f2ba80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:51.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 -- 172.21.15.97:0/2740170936 shutdown_connections 2024-09-10T21:31:51.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 -- 172.21.15.97:0/2740170936 wait complete. 2024-09-10T21:31:51.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 Processor -- start 2024-09-10T21:31:51.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.784+0000 7f2baea00640 1 -- start start 2024-09-10T21:31:51.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2ba8153060 0x7f2ba80b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba81f4780 0x7f2ba80c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba81f4c00 0x7f2ba80c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba80ce1a0 con 0x7f2ba81f4c00 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba80cde60 con 0x7f2ba81f4780 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba80ce320 con 0x7f2ba8153060 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba81f4c00 0x7f2ba80c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba81f4c00 0x7f2ba80c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56586/0 (socket says 172.21.15.97:56586) 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 -- 172.21.15.97:0/3027291711 learned_addr learned my addr 172.21.15.97:0/3027291711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2ba7e00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2ba8153060 0x7f2ba80b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 -- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2ba8153060 msgr2=0x7f2ba80b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2ba8153060 0x7f2ba80b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2b9f400640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba81f4780 0x7f2ba80c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 -- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba81f4780 msgr2=0x7f2ba80c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba81f4780 0x7f2ba80c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 -- 172.21.15.97:0/3027291711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba81f51b0 con 0x7f2ba81f4c00 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2b9f400640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba81f4780 0x7f2ba80c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baca00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba81f4c00 0x7f2ba80c4cf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00b1040 tx=0x7f2ba0005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2ba4c00640 1 -- 172.21.15.97:0/3027291711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0009ab0 con 0x7f2ba81f4c00 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2ba4c00640 1 -- 172.21.15.97:0/3027291711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ba00a7c50 con 0x7f2ba81f4c00 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2ba4c00640 1 -- 172.21.15.97:0/3027291711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0009200 con 0x7f2ba81f4c00 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba81f5490 con 0x7f2ba81f4c00 2024-09-10T21:31:51.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba80b5f20 con 0x7f2ba81f4c00 2024-09-10T21:31:51.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba80d0140 con 0x7f2ba81f4c00 2024-09-10T21:31:51.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.788+0000 7f2ba4c00640 1 -- 172.21.15.97:0/3027291711 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2ba0005040 con 0x7f2ba81f4c00 2024-09-10T21:31:51.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.792+0000 7f2ba4c00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b840725e0 0x7f2b84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:51.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.792+0000 7f2ba4c00640 1 -- 172.21.15.97:0/3027291711 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7f2ba012e210 con 0x7f2ba81f4c00 2024-09-10T21:31:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.792+0000 7f2ba7e00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b840725e0 0x7f2b84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.792+0000 7f2ba7e00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b840725e0 0x7f2b84074aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2b98000fc0 tx=0x7f2b98003600 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.796+0000 7f2ba4c00640 1 -- 172.21.15.97:0/3027291711 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ba0017340 con 0x7f2ba81f4c00 2024-09-10T21:31:51.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.992+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ba8071be0 con 0x7f2ba81f4c00 2024-09-10T21:31:51.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2ba4c00640 1 -- 172.21.15.97:0/3027291711 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4395 (secure 0 0 0) 0x7f2ba00f93b0 con 0x7f2ba81f4c00 2024-09-10T21:31:51.998 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:51.998 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":136,"btime":"2024-09-10T21:31:51:627846+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2841033666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2841033666},{"type":"v1","addr":"172.21.15.97:6835","nonce":2841033666}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:51.627779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 136 2024-09-10T21:31:51.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b840725e0 msgr2=0x7f2b84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:51.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b840725e0 0x7f2b84074aa0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2b98000fc0 tx=0x7f2b98003600 comp rx=0 tx=0).stop 2024-09-10T21:31:51.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba81f4c00 msgr2=0x7f2ba80c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba81f4c00 0x7f2ba80c4cf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00b1040 tx=0x7f2ba0005b60 comp rx=0 tx=0).stop 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 shutdown_connections 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b840725e0 0x7f2b84074aa0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba81f4c00 0x7f2ba80c4cf0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba81f4780 0x7f2ba80c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 --2- 172.21.15.97:0/3027291711 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2ba8153060 0x7f2ba80b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 >> 172.21.15.97:0/3027291711 conn(0x7f2ba80b37c0 msgr2=0x7f2ba80b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 shutdown_connections 2024-09-10T21:31:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:51.996+0000 7f2baea00640 1 -- 172.21.15.97:0/3027291711 wait complete. 2024-09-10T21:31:52.019 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:52.019 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 -- 172.21.15.97:0/1454412319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700b0d50 msgr2=0x7f36700ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 --2- 172.21.15.97:0/1454412319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700b0d50 0x7f36700ccc10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3668009840 tx=0x7f366801c820 comp rx=0 tx=0).stop 2024-09-10T21:31:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 -- 172.21.15.97:0/1454412319 shutdown_connections 2024-09-10T21:31:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 --2- 172.21.15.97:0/1454412319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700b0d50 0x7f36700ccc10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 --2- 172.21.15.97:0/1454412319 >> v2:172.21.15.194:3300/0 conn(0x7f36700afdd0 0x7f36700cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 -- 172.21.15.97:0/1454412319 >> 172.21.15.97:0/1454412319 conn(0x7f36700d1c30 msgr2=0x7f36700d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:52.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 -- 172.21.15.97:0/1454412319 shutdown_connections 2024-09-10T21:31:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.156+0000 7f3676c00640 1 -- 172.21.15.97:0/1454412319 wait complete. 2024-09-10T21:31:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 Processor -- start 2024-09-10T21:31:52.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 -- start start 2024-09-10T21:31:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36700afdd0 0x7f36700cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36700b0d50 0x7f3670163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700ce640 0x7f3670164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3670168e90 con 0x7f36700afdd0 2024-09-10T21:31:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3670168d10 con 0x7f36700b0d50 2024-09-10T21:31:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3670169010 con 0x7f36700ce640 2024-09-10T21:31:52.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36700afdd0 0x7f36700cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36700afdd0 0x7f36700cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56598/0 (socket says 172.21.15.97:56598) 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 -- 172.21.15.97:0/3367985823 learned_addr learned my addr 172.21.15.97:0/3367985823 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3674c00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700ce640 0x7f3670164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3667400640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36700b0d50 0x7f3670163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 -- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700ce640 msgr2=0x7f3670164170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700ce640 0x7f3670164170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 -- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36700b0d50 msgr2=0x7f3670163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36700b0d50 0x7f3670163c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 -- 172.21.15.97:0/3367985823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36701fafb0 con 0x7f36700afdd0 2024-09-10T21:31:52.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366fe00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36700afdd0 0x7f36700cb340 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f366000c430 tx=0x7f366000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:52.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366cc00640 1 -- 172.21.15.97:0/3367985823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660011790 con 0x7f36700afdd0 2024-09-10T21:31:52.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366cc00640 1 -- 172.21.15.97:0/3367985823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3660017070 con 0x7f36700afdd0 2024-09-10T21:31:52.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36701fb250 con 0x7f36700afdd0 2024-09-10T21:31:52.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f366cc00640 1 -- 172.21.15.97:0/3367985823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660011ce0 con 0x7f36700afdd0 2024-09-10T21:31:52.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701fb810 con 0x7f36700afdd0 2024-09-10T21:31:52.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.160+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36700cea20 con 0x7f36700afdd0 2024-09-10T21:31:52.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.164+0000 7f366cc00640 1 -- 172.21.15.97:0/3367985823 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f366001c460 con 0x7f36700afdd0 2024-09-10T21:31:52.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.164+0000 7f366cc00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f364c0725e0 0x7f364c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.164+0000 7f366cc00640 1 -- 172.21.15.97:0/3367985823 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7f3660093320 con 0x7f36700afdd0 2024-09-10T21:31:52.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.164+0000 7f3667400640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f364c0725e0 0x7f364c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.164+0000 7f3667400640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f364c0725e0 0x7f364c074aa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36700b7390 tx=0x7f3658009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:52.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.168+0000 7f366cc00640 1 -- 172.21.15.97:0/3367985823 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f366005dfc0 con 0x7f36700afdd0 2024-09-10T21:31:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36700d0460 con 0x7f36700afdd0 2024-09-10T21:31:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f366cc00640 1 -- 172.21.15.97:0/3367985823 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4395 (secure 0 0 0) 0x7f36600629f0 con 0x7f36700afdd0 2024-09-10T21:31:52.371 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:52.371 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":136,"btime":"2024-09-10T21:31:51:627846+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2841033666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2841033666},{"type":"v1","addr":"172.21.15.97:6835","nonce":2841033666}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:51.627779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 136 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f364c0725e0 msgr2=0x7f364c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f364c0725e0 0x7f364c074aa0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36700b7390 tx=0x7f3658009500 comp rx=0 tx=0).stop 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36700afdd0 msgr2=0x7f36700cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36700afdd0 0x7f36700cb340 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f366000c430 tx=0x7f366000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 shutdown_connections 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f364c0725e0 0x7f364c074aa0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36700ce640 0x7f3670164170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36700b0d50 0x7f3670163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 --2- 172.21.15.97:0/3367985823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36700afdd0 0x7f36700cb340 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 >> 172.21.15.97:0/3367985823 conn(0x7f36700d1c30 msgr2=0x7f36700d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 shutdown_connections 2024-09-10T21:31:52.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.368+0000 7f3676c00640 1 -- 172.21.15.97:0/3367985823 wait complete. 2024-09-10T21:31:52.391 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:52.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:52.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 -- 172.21.15.97:0/1631575460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400ce640 msgr2=0x7f8a400cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 --2- 172.21.15.97:0/1631575460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400ce640 0x7f8a400cea20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009c80 tx=0x7f8a3001cda0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 -- 172.21.15.97:0/1631575460 shutdown_connections 2024-09-10T21:31:52.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 --2- 172.21.15.97:0/1631575460 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a400cd6e0 0x7f8a400af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 --2- 172.21.15.97:0/1631575460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400ce640 0x7f8a400cea20 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 -- 172.21.15.97:0/1631575460 >> 172.21.15.97:0/1631575460 conn(0x7f8a400d1c30 msgr2=0x7f8a400d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:52.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 -- 172.21.15.97:0/1631575460 shutdown_connections 2024-09-10T21:31:52.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.532+0000 7f8a45200640 1 -- 172.21.15.97:0/1631575460 wait complete. 2024-09-10T21:31:52.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 Processor -- start 2024-09-10T21:31:52.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 -- start start 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400b0d50 0x7f8a40071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a400cd6e0 0x7f8a40152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a400ce640 0x7f8a40165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40168cc0 con 0x7f8a400b0d50 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40168b40 con 0x7f8a400ce640 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8a40168e40 con 0x7f8a400cd6e0 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400b0d50 0x7f8a40071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400b0d50 0x7f8a40071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56616/0 (socket says 172.21.15.97:56616) 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 -- 172.21.15.97:0/3350793978 learned_addr learned my addr 172.21.15.97:0/3350793978 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3d400640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a400cd6e0 0x7f8a40152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3e800640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a400ce640 0x7f8a40165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 -- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a400cd6e0 msgr2=0x7f8a40152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a400cd6e0 0x7f8a40152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 -- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a400ce640 msgr2=0x7f8a40165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a400ce640 0x7f8a40165dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 -- 172.21.15.97:0/3350793978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4015ef30 con 0x7f8a400b0d50 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a3de00640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400b0d50 0x7f8a40071b90 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000f90 tx=0x7f8a30009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a36000640 1 -- 172.21.15.97:0/3350793978 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300b2040 con 0x7f8a400b0d50 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a36000640 1 -- 172.21.15.97:0/3350793978 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a300ad8e0 con 0x7f8a400b0d50 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a36000640 1 -- 172.21.15.97:0/3350793978 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300adcd0 con 0x7f8a400b0d50 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4006e7c0 con 0x7f8a400b0d50 2024-09-10T21:31:52.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a401652f0 con 0x7f8a400b0d50 2024-09-10T21:31:52.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.536+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a400cea20 con 0x7f8a400b0d50 2024-09-10T21:31:52.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.540+0000 7f8a36000640 1 -- 172.21.15.97:0/3350793978 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8a300b5070 con 0x7f8a400b0d50 2024-09-10T21:31:52.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.540+0000 7f8a36000640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a20072610 0x7f8a20074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:52.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.540+0000 7f8a36000640 1 -- 172.21.15.97:0/3350793978 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7f8a3012d420 con 0x7f8a400b0d50 2024-09-10T21:31:52.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.540+0000 7f8a3d400640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a20072610 0x7f8a20074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:52.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.540+0000 7f8a3d400640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a20072610 0x7f8a20074ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8a400ce2d0 tx=0x7f8a2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:52.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.544+0000 7f8a36000640 1 -- 172.21.15.97:0/3350793978 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8a30016990 con 0x7f8a400b0d50 2024-09-10T21:31:52.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a40156130 con 0x7f8a400b0d50 2024-09-10T21:31:52.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a36000640 1 -- 172.21.15.97:0/3350793978 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4395 (secure 0 0 0) 0x7f8a300f7f70 con 0x7f8a400b0d50 2024-09-10T21:31:52.746 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:52.746 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":136,"btime":"2024-09-10T21:31:51:627846+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2841033666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2841033666},{"type":"v1","addr":"172.21.15.97:6835","nonce":2841033666}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:51.627779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:52.746 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 136 2024-09-10T21:31:52.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a20072610 msgr2=0x7f8a20074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a20072610 0x7f8a20074ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8a400ce2d0 tx=0x7f8a2c009040 comp rx=0 tx=0).stop 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400b0d50 msgr2=0x7f8a40071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400b0d50 0x7f8a40071b90 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000f90 tx=0x7f8a30009340 comp rx=0 tx=0).stop 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 shutdown_connections 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a20072610 0x7f8a20074ad0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a400ce640 0x7f8a40165dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8a400cd6e0 0x7f8a40152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 --2- 172.21.15.97:0/3350793978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a400b0d50 0x7f8a40071b90 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 >> 172.21.15.97:0/3350793978 conn(0x7f8a400d1c30 msgr2=0x7f8a40155440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 shutdown_connections 2024-09-10T21:31:52.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:52.744+0000 7f8a45200640 1 -- 172.21.15.97:0/3350793978 wait complete. 2024-09-10T21:31:52.768 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2024-09-10T21:31:52.768 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2024-09-10T21:31:52.768 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:52.768 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:52.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:31:52.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:52.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:52.777 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:52.828 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:31:52.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:31:52.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 -- 172.21.15.97:0/522701782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb240ceee0 msgr2=0x7fdb240cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 --2- 172.21.15.97:0/522701782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb240ceee0 0x7fdb240cf350 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00ad90 tx=0x7fdb1c020450 comp rx=0 tx=0).stop 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 -- 172.21.15.97:0/522701782 shutdown_connections 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 --2- 172.21.15.97:0/522701782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb240ceee0 0x7fdb240cf350 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 --2- 172.21.15.97:0/522701782 >> v2:172.21.15.194:3300/0 conn(0x7fdb2406ec40 0x7fdb2406f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 -- 172.21.15.97:0/522701782 >> 172.21.15.97:0/522701782 conn(0x7fdb240b1780 msgr2=0x7fdb240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 -- 172.21.15.97:0/522701782 shutdown_connections 2024-09-10T21:31:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.012+0000 7fdb2c200640 1 -- 172.21.15.97:0/522701782 wait complete. 2024-09-10T21:31:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 Processor -- start 2024-09-10T21:31:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 -- start start 2024-09-10T21:31:53.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2406ec40 0x7fdb2414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:53.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb240ceee0 0x7fdb241f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:53.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb241511f0 0x7fdb24140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:53.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2406e740 con 0x7fdb2406ec40 2024-09-10T21:31:53.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2406e5c0 con 0x7fdb240ceee0 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2406e8c0 con 0x7fdb241511f0 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2406ec40 0x7fdb2414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2406ec40 0x7fdb2414ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56624/0 (socket says 172.21.15.97:56624) 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 -- 172.21.15.97:0/3483221507 learned_addr learned my addr 172.21.15.97:0/3483221507 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb28e00640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb240ceee0 0x7fdb241f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2a200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb241511f0 0x7fdb24140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 -- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb241511f0 msgr2=0x7fdb24140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb241511f0 0x7fdb24140630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 -- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb240ceee0 msgr2=0x7fdb241f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb240ceee0 0x7fdb241f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 -- 172.21.15.97:0/3483221507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb241f53d0 con 0x7fdb2406ec40 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2a200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb241511f0 0x7fdb24140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:53.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb29800640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2406ec40 0x7fdb2414ff10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800b410 tx=0x7fdb1800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb21600640 1 -- 172.21.15.97:0/3483221507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18015030 con 0x7fdb2406ec40 2024-09-10T21:31:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb21600640 1 -- 172.21.15.97:0/3483221507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb18004020 con 0x7fdb2406ec40 2024-09-10T21:31:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb21600640 1 -- 172.21.15.97:0/3483221507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180044b0 con 0x7fdb2406ec40 2024-09-10T21:31:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb241f56b0 con 0x7fdb2406ec40 2024-09-10T21:31:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.016+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241f5060 con 0x7fdb2406ec40 2024-09-10T21:31:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.020+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2406f090 con 0x7fdb2406ec40 2024-09-10T21:31:53.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.020+0000 7fdb21600640 1 -- 172.21.15.97:0/3483221507 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb18022060 con 0x7fdb2406ec40 2024-09-10T21:31:53.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.020+0000 7fdb21600640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb040725e0 0x7fdb04074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.020+0000 7fdb21600640 1 -- 172.21.15.97:0/3483221507 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7fdb18093330 con 0x7fdb2406ec40 2024-09-10T21:31:53.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.020+0000 7fdb28e00640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb040725e0 0x7fdb04074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:53.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.024+0000 7fdb28e00640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb040725e0 0x7fdb04074aa0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdb100058d0 tx=0x7fdb10005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.024+0000 7fdb21600640 1 -- 172.21.15.97:0/3483221507 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb1805e4c0 con 0x7fdb2406ec40 2024-09-10T21:31:53.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.224+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdb24159a60 con 0x7fdb2406ec40 2024-09-10T21:31:53.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb21600640 1 -- 172.21.15.97:0/3483221507 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v137) ==== 161+0+0 (secure 0 0 0) 0x7fdb1805ddc0 con 0x7fdb2406ec40 2024-09-10T21:31:53.894 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb040725e0 msgr2=0x7fdb04074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb040725e0 0x7fdb04074aa0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdb100058d0 tx=0x7fdb10005860 comp rx=0 tx=0).stop 2024-09-10T21:31:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2406ec40 msgr2=0x7fdb2414ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2406ec40 0x7fdb2414ff10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800b410 tx=0x7fdb1800b8d0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 shutdown_connections 2024-09-10T21:31:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb040725e0 0x7fdb04074aa0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdb241511f0 0x7fdb24140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb240ceee0 0x7fdb241f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 --2- 172.21.15.97:0/3483221507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2406ec40 0x7fdb2414ff10 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.892+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 >> 172.21.15.97:0/3483221507 conn(0x7fdb240b1780 msgr2=0x7fdb241505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.896+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 shutdown_connections 2024-09-10T21:31:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:53.896+0000 7fdb2c200640 1 -- 172.21.15.97:0/3483221507 wait complete. 2024-09-10T21:31:53.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:54.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 -- 172.21.15.97:0/3624190022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 msgr2=0x7f3d0c0ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 --2- 172.21.15.97:0/3624190022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c0ceee0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c074f10 tx=0x7f3d0401f580 comp rx=0 tx=0).stop 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 -- 172.21.15.97:0/3624190022 shutdown_connections 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 --2- 172.21.15.97:0/3624190022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c0ceee0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 --2- 172.21.15.97:0/3624190022 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d0c0b3cf0 0x7f3d0c06ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 --2- 172.21.15.97:0/3624190022 >> v2:172.21.15.194:3300/0 conn(0x7f3d0c1511f0 0x7f3d0c0b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 -- 172.21.15.97:0/3624190022 >> 172.21.15.97:0/3624190022 conn(0x7f3d0c0b1780 msgr2=0x7f3d0c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 -- 172.21.15.97:0/3624190022 shutdown_connections 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 -- 172.21.15.97:0/3624190022 wait complete. 2024-09-10T21:31:54.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 Processor -- start 2024-09-10T21:31:54.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.060+0000 7f3d14400640 1 -- start start 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d0c0b3cf0 0x7f3d0c13a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d0c1511f0 0x7f3d0c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c06e530 con 0x7f3d0c06f180 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c06e3b0 con 0x7f3d0c0b3cf0 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c06e6b0 con 0x7f3d0c1511f0 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56638/0 (socket says 172.21.15.97:56638) 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 -- 172.21.15.97:0/3891571766 learned_addr learned my addr 172.21.15.97:0/3891571766 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11000640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d0c0b3cf0 0x7f3d0c13a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d12400640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d0c1511f0 0x7f3d0c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 -- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d0c1511f0 msgr2=0x7f3d0c159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d0c1511f0 0x7f3d0c159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 -- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d0c0b3cf0 msgr2=0x7f3d0c13a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d0c0b3cf0 0x7f3d0c13a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 -- 172.21.15.97:0/3891571766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c166130 con 0x7f3d0c06f180 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d11a00640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c159e60 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000ca20 tx=0x7f3d0000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d09600640 1 -- 172.21.15.97:0/3891571766 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00007c70 con 0x7f3d0c06f180 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d09600640 1 -- 172.21.15.97:0/3891571766 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d0000f450 con 0x7f3d0c06f180 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d09600640 1 -- 172.21.15.97:0/3891571766 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000f8a0 con 0x7f3d0c06f180 2024-09-10T21:31:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c166410 con 0x7f3d0c06f180 2024-09-10T21:31:54.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1596b0 con 0x7f3d0c06f180 2024-09-10T21:31:54.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.064+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c0ceee0 con 0x7f3d0c06f180 2024-09-10T21:31:54.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.068+0000 7f3d09600640 1 -- 172.21.15.97:0/3891571766 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3d0000fa40 con 0x7f3d0c06f180 2024-09-10T21:31:54.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.068+0000 7f3d09600640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3cec0725e0 0x7f3cec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.068+0000 7f3d09600640 1 -- 172.21.15.97:0/3891571766 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7f3d00093eb0 con 0x7f3d0c06f180 2024-09-10T21:31:54.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.068+0000 7f3d11000640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3cec0725e0 0x7f3cec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.068+0000 7f3d11000640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3cec0725e0 0x7f3cec074aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c149d30 tx=0x7f3cf800a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:54.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.072+0000 7f3d09600640 1 -- 172.21.15.97:0/3891571766 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d00014030 con 0x7f3d0c06f180 2024-09-10T21:31:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.296+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3d0c14c270 con 0x7f3d0c06f180 2024-09-10T21:31:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.896+0000 7f3d09600640 1 -- 172.21.15.97:0/3891571766 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v138) ==== 101+0+0 (secure 0 0 0) 0x7f3d0005f040 con 0x7f3d0c06f180 2024-09-10T21:31:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:31:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3cec0725e0 msgr2=0x7f3cec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3cec0725e0 0x7f3cec074aa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c149d30 tx=0x7f3cf800a2d0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 msgr2=0x7f3d0c159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c159e60 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000ca20 tx=0x7f3d0000cee0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 shutdown_connections 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3cec0725e0 0x7f3cec074aa0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3d0c1511f0 0x7f3d0c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d0c0b3cf0 0x7f3d0c13a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 --2- 172.21.15.97:0/3891571766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d0c06f180 0x7f3d0c159e60 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 >> 172.21.15.97:0/3891571766 conn(0x7f3d0c0b1780 msgr2=0x7f3d0c14ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 shutdown_connections 2024-09-10T21:31:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.900+0000 7f3d14400640 1 -- 172.21.15.97:0/3891571766 wait complete. 2024-09-10T21:31:54.924 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:54.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 Processor -- start 2024-09-10T21:31:54.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 -- start start 2024-09-10T21:31:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94002300 0x5619c65d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 0x5619c64fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5619c6546a00 0x5619c65d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5619c644ea00 con 0x7f2e94002300 2024-09-10T21:31:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5619c64acf40 con 0x5619c652c800 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eb1331080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5619c65de2c0 con 0x5619c6546a00 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94002300 0x5619c65d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94002300 0x5619c65d6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56660/0 (socket says 172.21.15.97:56660) 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 -- 172.21.15.97:0/3337537599 learned_addr learned my addr 172.21.15.97:0/3337537599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 0x5619c64fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eaf000640 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5619c6546a00 0x5619c65d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 -- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5619c6546a00 msgr2=0x5619c65d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5619c6546a00 0x5619c65d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 -- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 msgr2=0x5619c64fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 0x5619c64fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 -- 172.21.15.97:0/3337537599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c6427c10 con 0x7f2e94002300 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eaf000640 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5619c6546a00 0x5619c65d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2eae600640 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94002300 0x5619c65d6640 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2ea001bc30 tx=0x7f2e94004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87da09b1d838ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2ead200640 1 -- 172.21.15.97:0/3337537599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00a5a00 con 0x7f2e94002300 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2ead200640 1 -- 172.21.15.97:0/3337537599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ea00b6070 con 0x7f2e94002300 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.968+0000 7f2ead200640 1 -- 172.21.15.97:0/3337537599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00b1450 con 0x7f2e94002300 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- 172.21.15.97:0/3337537599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94002300 msgr2=0x5619c65d6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94002300 0x5619c65d6640 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2ea001bc30 tx=0x7f2e94004130 comp rx=0 tx=0).stop 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- 172.21.15.97:0/3337537599 shutdown_connections 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5619c6546a00 0x5619c65d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 0x5619c64fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3337537599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94002300 0x5619c65d6640 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- 172.21.15.97:0/3337537599 >> 172.21.15.97:0/3337537599 conn(0x5619c64ff140 msgr2=0x5619c65477f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- 172.21.15.97:0/3337537599 shutdown_connections 2024-09-10T21:31:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- 172.21.15.97:0/3337537599 wait complete. 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 Processor -- start 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- start start 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 0x5619c65b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5619c6546a00 0x5619c65c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e94002300 0x5619c65dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5619c65d3860 con 0x5619c6546a00 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5619c65d35c0 con 0x5619c652c800 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5619c65d5f00 con 0x7f2e94002300 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5619c6546a00 0x5619c65c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5619c6546a00 0x5619c65c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56670/0 (socket says 172.21.15.97:56670) 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 learned_addr learned my addr 172.21.15.97:0/3981860825 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eaf000640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e94002300 0x5619c65dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e94002300 msgr2=0x5619c65dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2e94002300 0x5619c65dc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 msgr2=0x5619c65b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5619c652c800 0x5619c65b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c65c9700 con 0x5619c6546a00 2024-09-10T21:31:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5619c6546a00 0x5619c65c8e60 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800a740 tx=0x7f2e9800ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea6a00640 1 -- 172.21.15.97:0/3981860825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9801c070 con 0x5619c6546a00 2024-09-10T21:31:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea6a00640 1 -- 172.21.15.97:0/3981860825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e9800ae50 con 0x5619c6546a00 2024-09-10T21:31:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5619c65c99a0 con 0x5619c6546a00 2024-09-10T21:31:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2ea6a00640 1 -- 172.21.15.97:0/3981860825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98017450 con 0x5619c6546a00 2024-09-10T21:31:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.972+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c65c9ee0 con 0x5619c6546a00 2024-09-10T21:31:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2ea6a00640 1 -- 172.21.15.97:0/3981860825 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2e980175f0 con 0x5619c6546a00 2024-09-10T21:31:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2ea6a00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e8c070fc0 0x7f2e8c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2ea6a00640 1 -- 172.21.15.97:0/3981860825 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7f2e980933a0 con 0x5619c6546a00 2024-09-10T21:31:54.980 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2eae600640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e8c070fc0 0x7f2e8c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5619c65402b0 0x5619c6546010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:54.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65b6fe0 con 0x5619c65402b0 2024-09-10T21:31:54.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2eaf000640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5619c65402b0 0x5619c6546010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:54.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2eae600640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e8c070fc0 0x7f2e8c073480 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00b7000 tx=0x7f2ea001b750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:54.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.976+0000 7f2eaf000640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5619c65402b0 0x5619c6546010 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:54.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.980+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v105'257 uv231 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.980+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c65b7380 con 0x5619c65402b0 2024-09-10T21:31:54.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.980+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v105'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.980+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65b7720 con 0x5619c65402b0 2024-09-10T21:31:54.985 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:54.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.984+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v105'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.984+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65b7ac0 con 0x5619c65402b0 2024-09-10T21:31:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.984+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.984+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c667eb80 con 0x5619c65402b0 2024-09-10T21:31:54.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.984+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.984+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca620 con 0x5619c65402b0 2024-09-10T21:31:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.988+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.988+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.988+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v105'263 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.988+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:54.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.992+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v105'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.992+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:54.995 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:54.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.992+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v105'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.992+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:54.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.992+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'266 uv266 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.992+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:54.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.996+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.996+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.996+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.996+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:55.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.996+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v105'269 uv244 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:54.996+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.000+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'270 uv270 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.000+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:55.003 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.000+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v105'271 uv271 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.000+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.000+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'272 uv272 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.000+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.004+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'273 uv273 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.004+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65ca9c0 con 0x5619c65402b0 2024-09-10T21:31:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.004+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'274 uv274 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.004+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.008+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'275 uv275 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.008+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.008+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v105'276 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.008+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.008+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v105'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.008+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.013 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.012+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v105'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.012+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.012+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.012+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.012+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.012+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.016+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.016+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.016+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v105'282 uv256 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.016+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.016+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v105'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.016+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.022 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.020+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v105'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.020+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.020+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.020+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.024+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.024+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65402b0 2024-09-10T21:31:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.024+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c65402b0 2024-09-10T21:31:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.024+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5619c65cb230 0x5619c6681440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.024+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c65cae70 con 0x5619c65cb230 2024-09-10T21:31:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.024+0000 7f2eae600640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5619c65cb230 0x5619c6681440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.024+0000 7f2eae600640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5619c65cb230 0x5619c6681440 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.028+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'198 uv181 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.028+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c6681980 con 0x5619c65cb230 2024-09-10T21:31:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.028+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'199 uv199 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.028+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6681d20 con 0x5619c65cb230 2024-09-10T21:31:55.032 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:55.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.028+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'200 uv200 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.028+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6682570 con 0x5619c65cb230 2024-09-10T21:31:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.032+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v105'201 uv201 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.032+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6682910 con 0x5619c65cb230 2024-09-10T21:31:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.032+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'202 uv202 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.032+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6682cb0 con 0x5619c65cb230 2024-09-10T21:31:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.032+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v105'203 uv186 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.032+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v105'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.038 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v105'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.036+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.040+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v105'208 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.040+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.040+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v105'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.040+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.040+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v105'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.044 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.040+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6687220 con 0x5619c65cb230 2024-09-10T21:31:55.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.044+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v105'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.044+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6687220 con 0x5619c65cb230 2024-09-10T21:31:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.044+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v105'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.044+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.044+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v105'213 uv197 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.044+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v105'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v105'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.051 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:55.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6684060 con 0x5619c65cb230 2024-09-10T21:31:55.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v105'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6684060 con 0x5619c65cb230 2024-09-10T21:31:55.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v105'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.048+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6683050 con 0x5619c65cb230 2024-09-10T21:31:55.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.052+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v105'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c65cb230 2024-09-10T21:31:55.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.052+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5619c6683920 0x5619c6685d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.052+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6686250 con 0x5619c6683920 2024-09-10T21:31:55.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.052+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5619c6683920 0x5619c6685d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.052+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5619c6683920 0x5619c6685d10 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.052+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.052+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c66865f0 con 0x5619c6683920 2024-09-10T21:31:55.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6686aa0 con 0x5619c6683920 2024-09-10T21:31:55.058 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:55.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6688350 con 0x5619c6683920 2024-09-10T21:31:55.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c66886f0 con 0x5619c6683920 2024-09-10T21:31:55.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.056+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6687340 con 0x5619c6683920 2024-09-10T21:31:55.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5619c6687c40 0x5619c668aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668afe0 con 0x5619c6687c40 2024-09-10T21:31:55.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eaf000640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5619c6687c40 0x5619c668aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eaf000640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5619c6687c40 0x5619c668aaa0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'44 uv43 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668afe0 con 0x5619c6687c40 2024-09-10T21:31:55.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v105'45 uv45 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.060+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b3d0 con 0x5619c6687c40 2024-09-10T21:31:55.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.064+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v105'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.066 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:55.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.064+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b3d0 con 0x5619c6687c40 2024-09-10T21:31:55.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.064+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v105'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.064+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b3d0 con 0x5619c6683920 2024-09-10T21:31:55.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.064+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.064+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6683920 2024-09-10T21:31:55.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.068+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.068+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6683920 2024-09-10T21:31:55.070 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.068+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.068+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668c780 con 0x5619c6683920 2024-09-10T21:31:55.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.068+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.068+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668c780 con 0x5619c6683920 2024-09-10T21:31:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6683920 2024-09-10T21:31:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v105'94 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v105'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.072+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.078 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.076+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v105'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.076+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668c780 con 0x5619c6687c40 2024-09-10T21:31:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.076+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v105'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.076+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668c780 con 0x5619c6687c40 2024-09-10T21:31:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.076+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v105'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.076+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.080+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v105'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.080+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.080+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v105'100 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.080+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.080+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v105'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.080+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.084+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v105'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.085 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:55.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.084+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.084+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v105'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.084+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6687c40 2024-09-10T21:31:55.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.084+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v105'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.084+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6683920 2024-09-10T21:31:55.089 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:55.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v105'43 uv43 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6683920 2024-09-10T21:31:55.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [write 0~70] v105'44 uv44 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668b770 con 0x5619c6683920 2024-09-10T21:31:55.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v105'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c6683920 2024-09-10T21:31:55.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5619c668bdf0 0x5619c668e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668e840 con 0x5619c668bdf0 2024-09-10T21:31:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2eae600640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5619c668bdf0 0x5619c668e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.088+0000 7f2eae600640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5619c668bdf0 0x5619c668e1e0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.092+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(83 605.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.092+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668e900 con 0x5619c668bdf0 2024-09-10T21:31:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.092+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(84 605.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.092+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.092+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(85 605.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.096 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.092+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668fd30 con 0x5619c668bdf0 2024-09-10T21:31:55.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668fd30 con 0x5619c668bdf0 2024-09-10T21:31:55.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(89 mds0_inotable [create] v105'84 uv78 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.096+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(90 mds0_inotable [delete] v105'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(91 mds0_inotable [create] v105'86 uv86 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.102 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:55.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v105'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v105'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.100+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.104+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(94 100.00000000 [create] v105'89 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.104+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.104+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(95 100.00000000 [delete] v105'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.104+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.104+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(96 100.00000000 [create] v105'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.108 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.104+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668ed20 con 0x5619c668bdf0 2024-09-10T21:31:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.108+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v105'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.108+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6691b30 con 0x5619c668bdf0 2024-09-10T21:31:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.108+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v105'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.108+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v105'40 uv39 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v105'41 uv41 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v105'42 uv42 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.115 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:55.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v105'43 uv43 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.112+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.116+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v105'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.116+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.116+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v105'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.116+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.116+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v105'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.120 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.116+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c66901e0 con 0x5619c6687c40 2024-09-10T21:31:55.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v105'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c66901e0 con 0x5619c6687c40 2024-09-10T21:31:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v105'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c668f1d0 con 0x5619c6687c40 2024-09-10T21:31:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v105'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2ea80099c0 con 0x5619c6687c40 2024-09-10T21:31:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5619c668f850 0x5619c668fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668f850 2024-09-10T21:31:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5619c668f850 0x5619c668fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.120+0000 7f2ea7e00640 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5619c668f850 0x5619c668fc30 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.124+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v105'42 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c668f850 2024-09-10T21:31:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.124+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668f850 2024-09-10T21:31:55.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.124+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v105'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c668f850 2024-09-10T21:31:55.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.124+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668f850 2024-09-10T21:31:55.129 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:55.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v105'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c668f850 2024-09-10T21:31:55.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668f850 2024-09-10T21:31:55.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v105'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c668f850 2024-09-10T21:31:55.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668f850 2024-09-10T21:31:55.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2ea7e00640 1 -- 172.21.15.97:0/3981860825 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v105'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2e9801e030 con 0x5619c668f850 2024-09-10T21:31:55.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(114 100.00000000.inode [create] v105'107 uv99 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.128+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.132+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(115 100.00000000.inode [delete] v105'108 uv108 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.132+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.132+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(116 100.00000000.inode [create] v105'109 uv109 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.135 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:55.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.132+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.132+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v105'110 uv110 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.132+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.136+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v105'111 uv111 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.136+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.136+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(119 1.00000000 [create] v105'112 uv106 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.136+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.136+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(120 1.00000000 [delete] v105'113 uv113 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.136+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.141 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:55.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(121 1.00000000 [create] v105'114 uv114 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v105'115 uv115 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v105'116 uv116 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6690170 con 0x5619c668bdf0 2024-09-10T21:31:55.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v105'117 uv117 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.140+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5619c6691660 con 0x5619c668bdf0 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eae600640 1 -- 172.21.15.97:0/3981860825 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v105'118 uv118 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2ea00ad020 con 0x5619c668bdf0 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5619c6683920 msgr2=0x5619c6685d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5619c6683920 0x5619c6685d10 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5619c668bdf0 msgr2=0x5619c668e1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5619c668bdf0 0x5619c668e1e0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5619c668f850 msgr2=0x5619c668fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5619c668f850 0x5619c668fc30 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5619c6687c40 msgr2=0x5619c668aaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5619c6687c40 0x5619c668aaa0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5619c65cb230 msgr2=0x5619c6681440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5619c65cb230 0x5619c6681440 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 reap_dead start 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5619c65402b0 msgr2=0x5619c6546010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5619c65402b0 0x5619c6546010 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 reap_dead start 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eaf000640 1 -- 172.21.15.97:0/3981860825 reap_dead start 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e8c070fc0 msgr2=0x7f2e8c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e8c070fc0 0x7f2e8c073480 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00b7000 tx=0x7f2ea001b750 comp rx=0 tx=0).stop 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5619c6546a00 msgr2=0x5619c65c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5619c6546a00 0x5619c65c8e60 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800a740 tx=0x7f2e9800ac00 comp rx=0 tx=0).stop 2024-09-10T21:31:55.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 shutdown_connections 2024-09-10T21:31:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2e8c070fc0 0x7f2e8c073480 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 --2- 172.21.15.97:0/3981860825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5619c6546a00 0x5619c65c8e60 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 >> 172.21.15.97:0/3981860825 conn(0x5619c64ff140 msgr2=0x5619c6546fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 shutdown_connections 2024-09-10T21:31:55.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.144+0000 7f2eb1331080 1 -- 172.21.15.97:0/3981860825 wait complete. 2024-09-10T21:31:55.151 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2024-09-10T21:31:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 Processor -- start 2024-09-10T21:31:55.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- start start 2024-09-10T21:31:55.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 0x5561a55232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a54aeba0 0x5561a5469b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a546a090 0x5561a546a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5561a5416f40 con 0x5561a546a090 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561a5423d00 con 0x5561a54afac0 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5561a53b8a00 con 0x5561a54aeba0 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a546a090 0x5561a546a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a546a090 0x5561a546a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56686/0 (socket says 172.21.15.97:56686) 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 -- 172.21.15.97:0/4005659012 learned_addr learned my addr 172.21.15.97:0/4005659012 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb46fe00640 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a54aeba0 0x5561a5469b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb477a00640 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 0x5561a55232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 -- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a54aeba0 msgr2=0x5561a5469b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:55.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a54aeba0 0x5561a5469b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 -- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 msgr2=0x5561a55232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 0x5561a55232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 -- 172.21.15.97:0/4005659012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a5523800 con 0x5561a546a090 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb478400640 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a546a090 0x5561a546a4a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb470009880 tx=0x7fb470001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c735680669aec235 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb476600640 1 -- 172.21.15.97:0/4005659012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47000e640 con 0x5561a546a090 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb476600640 1 -- 172.21.15.97:0/4005659012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb47000e7e0 con 0x5561a546a090 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb476600640 1 -- 172.21.15.97:0/4005659012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47000ebf0 con 0x5561a546a090 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- 172.21.15.97:0/4005659012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a546a090 msgr2=0x5561a546a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a546a090 0x5561a546a4a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb470009880 tx=0x7fb470001c70 comp rx=0 tx=0).stop 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- 172.21.15.97:0/4005659012 shutdown_connections 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a546a090 0x5561a546a4a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a54aeba0 0x5561a5469b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 --2- 172.21.15.97:0/4005659012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 0x5561a55232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- 172.21.15.97:0/4005659012 >> 172.21.15.97:0/4005659012 conn(0x5561a54b20d0 msgr2=0x5561a54b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:55.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- 172.21.15.97:0/4005659012 shutdown_connections 2024-09-10T21:31:55.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.236+0000 7fb47a73f080 1 -- 172.21.15.97:0/4005659012 wait complete. 2024-09-10T21:31:55.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 Processor -- start 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 -- start start 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a546a090 0x5561a5540670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a54aeba0 0x5561a5469140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 0x5561a5469680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5561a54b1710 con 0x5561a54aeba0 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561a54b19b0 con 0x5561a54afac0 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5561a5523b60 con 0x5561a546a090 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a54aeba0 0x5561a5469140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb477a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a546a090 0x5561a5540670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a54aeba0 0x5561a5469140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56692/0 (socket says 172.21.15.97:56692) 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 -- 172.21.15.97:0/212348569 learned_addr learned my addr 172.21.15.97:0/212348569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:55.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb478400640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 0x5561a5469680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 -- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a546a090 msgr2=0x5561a5540670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5561a546a090 0x5561a5540670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 -- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 msgr2=0x5561a5469680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561a54afac0 0x5561a5469680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 -- 172.21.15.97:0/212348569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a55cbd40 con 0x5561a54aeba0 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb46fe00640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a54aeba0 0x5561a5469140 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb4600017e0 tx=0x7fb46000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb475200640 1 -- 172.21.15.97:0/212348569 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000d7b0 con 0x5561a54aeba0 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb475200640 1 -- 172.21.15.97:0/212348569 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb46000d970 con 0x5561a54aeba0 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb475200640 1 -- 172.21.15.97:0/212348569 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000ddc0 con 0x5561a54aeba0 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5561a55cbfe0 con 0x5561a54aeba0 2024-09-10T21:31:55.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.240+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a55cc520 con 0x5561a54aeba0 2024-09-10T21:31:55.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb475200640 1 -- 172.21.15.97:0/212348569 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4600193a0 con 0x5561a54aeba0 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb475200640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb454070ff0 0x7fb4540734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb475200640 1 -- 172.21.15.97:0/212348569 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7fb4600952c0 con 0x5561a54aeba0 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb477a00640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb454070ff0 0x7fb4540734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb47a73f080 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5561a5533cb0 0x5561a5534090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561a55345d0 con 0x5561a5533cb0 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb478400640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5561a5533cb0 0x5561a5534090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb477a00640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb454070ff0 0x7fb4540734b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4680ac000 tx=0x7fb4680ae040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb478400640 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5561a5533cb0 0x5561a5534090 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb478400640 1 -- 172.21.15.97:0/212348569 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 mds0_inotable [delete] v105'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb47000e740 con 0x5561a5533cb0 2024-09-10T21:31:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5561a5533cb0 msgr2=0x5561a5534090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.244+0000 7fb47a73f080 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5561a5533cb0 0x5561a5534090 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb454070ff0 msgr2=0x7fb4540734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb454070ff0 0x7fb4540734b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4680ac000 tx=0x7fb4680ae040 comp rx=0 tx=0).stop 2024-09-10T21:31:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a54aeba0 msgr2=0x5561a5469140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 --2- 172.21.15.97:0/212348569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5561a54aeba0 0x5561a5469140 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb4600017e0 tx=0x7fb46000f840 comp rx=0 tx=0).stop 2024-09-10T21:31:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb478400640 1 -- 172.21.15.97:0/212348569 reap_dead start 2024-09-10T21:31:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 shutdown_connections 2024-09-10T21:31:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 >> 172.21.15.97:0/212348569 conn(0x5561a54b20d0 msgr2=0x5561a5522890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 shutdown_connections 2024-09-10T21:31:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.248+0000 7fb47a73f080 1 -- 172.21.15.97:0/212348569 wait complete. 2024-09-10T21:31:55.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 -- 172.21.15.97:0/111761184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 msgr2=0x7f36a8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 --2- 172.21.15.97:0/111761184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 0x7f36a8005220 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f36c40cc060 tx=0x7f36bc01cee0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c2800640 1 -- 172.21.15.97:0/111761184 reap_dead start 2024-09-10T21:31:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 -- 172.21.15.97:0/111761184 shutdown_connections 2024-09-10T21:31:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 -- 172.21.15.97:0/111761184 >> 172.21.15.97:0/111761184 conn(0x7f36c40d1a60 msgr2=0x7f36c40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 -- 172.21.15.97:0/111761184 shutdown_connections 2024-09-10T21:31:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 -- 172.21.15.97:0/111761184 wait complete. 2024-09-10T21:31:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 Processor -- start 2024-09-10T21:31:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.396+0000 7f36c3200640 1 -- start start 2024-09-10T21:31:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 0x7f36c415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c40b72d0 0x7f36c40cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c40b7750 0x7f36c40cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36c40cfbd0 con 0x7f36a8004da0 2024-09-10T21:31:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f36c40cfa50 con 0x7f36c40b7750 2024-09-10T21:31:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f36c40af040 con 0x7f36c40b72d0 2024-09-10T21:31:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 0x7f36c415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 0x7f36c415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56706/0 (socket says 172.21.15.97:56706) 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 -- 172.21.15.97:0/3483968393 learned_addr learned my addr 172.21.15.97:0/3483968393 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36b9400640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c40b72d0 0x7f36c40cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 -- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c40b72d0 msgr2=0x7f36c40cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c40b72d0 0x7f36c40cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 -- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c40b7750 msgr2=0x7f36c40cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c40b7750 0x7f36c40cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 -- 172.21.15.97:0/3483968393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c409f000 con 0x7f36a8004da0 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36b9400640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c40b72d0 0x7f36c40cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c1e00640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 0x7f36c415cac0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f36b4011d00 tx=0x7f36b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36baa00640 1 -- 172.21.15.97:0/3483968393 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b401b670 con 0x7f36a8004da0 2024-09-10T21:31:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36baa00640 1 -- 172.21.15.97:0/3483968393 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36b4016070 con 0x7f36a8004da0 2024-09-10T21:31:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36baa00640 1 -- 172.21.15.97:0/3483968393 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b401ba90 con 0x7f36a8004da0 2024-09-10T21:31:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c41f32e0 con 0x7f36a8004da0 2024-09-10T21:31:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c409f580 con 0x7f36a8004da0 2024-09-10T21:31:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c409f7c0 con 0x7f36a8004da0 2024-09-10T21:31:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.400+0000 7f36baa00640 1 -- 172.21.15.97:0/3483968393 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f36b4023030 con 0x7f36a8004da0 2024-09-10T21:31:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.404+0000 7f36baa00640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36a00725e0 0x7f36a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.404+0000 7f36baa00640 1 -- 172.21.15.97:0/3483968393 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7049+0+0 (secure 0 0 0) 0x7f36b4094700 con 0x7f36a8004da0 2024-09-10T21:31:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.404+0000 7f36b9400640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36a00725e0 0x7f36a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:55.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.404+0000 7f36b9400640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36a00725e0 0x7f36a0074aa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f36ac000c10 tx=0x7f36ac006b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.408+0000 7f36baa00640 1 -- 172.21.15.97:0/3483968393 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f36c409f7c0 con 0x7f36a8004da0 2024-09-10T21:31:55.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.604+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f36c40ce780 con 0x7f36a8004da0 2024-09-10T21:31:55.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36baa00640 1 -- 172.21.15.97:0/3483968393 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v139) ==== 162+0+0 (secure 0 0 0) 0x7f36b405f8a0 con 0x7f36a8004da0 2024-09-10T21:31:55.905 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:55.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36a00725e0 msgr2=0x7f36a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36a00725e0 0x7f36a0074aa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f36ac000c10 tx=0x7f36ac006b80 comp rx=0 tx=0).stop 2024-09-10T21:31:55.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 msgr2=0x7f36c415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 0x7f36c415cac0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f36b4011d00 tx=0x7f36b400f730 comp rx=0 tx=0).stop 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 shutdown_connections 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f36a00725e0 0x7f36a0074aa0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36c40b7750 0x7f36c40cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f36c40b72d0 0x7f36c40cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 --2- 172.21.15.97:0/3483968393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a8004da0 0x7f36c415cac0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 >> 172.21.15.97:0/3483968393 conn(0x7f36c40d1a60 msgr2=0x7f36c40cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 shutdown_connections 2024-09-10T21:31:55.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:55.904+0000 7f36c3200640 1 -- 172.21.15.97:0/3483968393 wait complete. 2024-09-10T21:31:55.913 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:55.912+0000 7effb7e00640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-10T21:31:55.913 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:31:55.912+0000 7effb7e00640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-10T21:31:55.914 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:55.912+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:31:55.914 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:31:55.912+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:31:55.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 -- 172.21.15.97:0/2703469675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 msgr2=0x7f24d80ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 --2- 172.21.15.97:0/2703469675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d80ccc10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009bf0 tx=0x7f24d001cba0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 -- 172.21.15.97:0/2703469675 shutdown_connections 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 --2- 172.21.15.97:0/2703469675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d80ccc10 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 --2- 172.21.15.97:0/2703469675 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80afdd0 0x7f24d80cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 -- 172.21.15.97:0/2703469675 >> 172.21.15.97:0/2703469675 conn(0x7f24d80d1c30 msgr2=0x7f24d80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 -- 172.21.15.97:0/2703469675 shutdown_connections 2024-09-10T21:31:56.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.072+0000 7f24de000640 1 -- 172.21.15.97:0/2703469675 wait complete. 2024-09-10T21:31:56.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 Processor -- start 2024-09-10T21:31:56.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 -- start start 2024-09-10T21:31:56.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d80afdd0 0x7f24d8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d8156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce640 0x7f24d806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8168f40 con 0x7f24d80b0d50 2024-09-10T21:31:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8168dc0 con 0x7f24d80afdd0 2024-09-10T21:31:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f24d81690c0 con 0x7f24d80ce640 2024-09-10T21:31:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d8156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d8156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56718/0 (socket says 172.21.15.97:56718) 2024-09-10T21:31:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 -- 172.21.15.97:0/1097832776 learned_addr learned my addr 172.21.15.97:0/1097832776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d7400640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d80afdd0 0x7f24d8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d7e00640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce640 0x7f24d806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 -- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce640 msgr2=0x7f24d806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce640 0x7f24d806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 -- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d80afdd0 msgr2=0x7f24d8156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d80afdd0 0x7f24d8156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 -- 172.21.15.97:0/1097832776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d80ccf30 con 0x7f24d80b0d50 2024-09-10T21:31:56.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24d6a00640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d8156650 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d980 tx=0x7f24c400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24cf400640 1 -- 172.21.15.97:0/1097832776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017070 con 0x7f24d80b0d50 2024-09-10T21:31:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24cf400640 1 -- 172.21.15.97:0/1097832776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c4009d20 con 0x7f24d80b0d50 2024-09-10T21:31:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24cf400640 1 -- 172.21.15.97:0/1097832776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4004280 con 0x7f24d80b0d50 2024-09-10T21:31:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d80b4e20 con 0x7f24d80b0d50 2024-09-10T21:31:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d80b53e0 con 0x7f24d80b0d50 2024-09-10T21:31:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.076+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d80cea20 con 0x7f24d80b0d50 2024-09-10T21:31:56.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.080+0000 7f24cf400640 1 -- 172.21.15.97:0/1097832776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f24c4003600 con 0x7f24d80b0d50 2024-09-10T21:31:56.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.080+0000 7f24cf400640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b80725e0 0x7f24b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.080+0000 7f24cf400640 1 -- 172.21.15.97:0/1097832776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7f24c4093920 con 0x7f24d80b0d50 2024-09-10T21:31:56.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.080+0000 7f24d7400640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b80725e0 0x7f24b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.080+0000 7f24d7400640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b80725e0 0x7f24b8074aa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f24c80046c0 tx=0x7f24c8007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:56.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.084+0000 7f24cf400640 1 -- 172.21.15.97:0/1097832776 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24c405e380 con 0x7f24d80b0d50 2024-09-10T21:31:56.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.288+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24d8156b90 con 0x7f24d80b0d50 2024-09-10T21:31:56.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.288+0000 7f24cf400640 1 -- 172.21.15.97:0/1097832776 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) ==== 95+0+3569 (secure 0 0 0) 0x7f24c4062db0 con 0x7f24d80b0d50 2024-09-10T21:31:56.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:56.291 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"btime":"2024-09-10T21:31:55:917304+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:55.917232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:56.291 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 141 2024-09-10T21:31:56.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.288+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b80725e0 msgr2=0x7f24b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.288+0000 7f24de000640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b80725e0 0x7f24b8074aa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f24c80046c0 tx=0x7f24c8007ec0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.288+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 msgr2=0x7f24d8156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.288+0000 7f24de000640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d8156650 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d980 tx=0x7f24c400de40 comp rx=0 tx=0).stop 2024-09-10T21:31:56.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 shutdown_connections 2024-09-10T21:31:56.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b80725e0 0x7f24b8074aa0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce640 0x7f24d806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80b0d50 0x7f24d8156650 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 --2- 172.21.15.97:0/1097832776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d80afdd0 0x7f24d8156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 >> 172.21.15.97:0/1097832776 conn(0x7f24d80d1c30 msgr2=0x7f24d815a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:56.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 shutdown_connections 2024-09-10T21:31:56.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.292+0000 7f24de000640 1 -- 172.21.15.97:0/1097832776 wait complete. 2024-09-10T21:31:56.313 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:56.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:56.376 INFO:tasks.ceph.mds.a.smithi097.stderr: -49> 2024-09-10T21:31:51.620+0000 7f9191800640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-10T21:31:56.377 INFO:tasks.ceph.mds.a.smithi097.stderr: -46> 2024-09-10T21:31:51.620+0000 7f9191800640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-10T21:31:56.414 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:31:56.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- 172.21.15.97:0/3589045000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480ce640 msgr2=0x7fbd480cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3589045000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480ce640 0x7fbd480cea20 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fbd480b36f0 tx=0x7fbd3801c800 comp rx=0 tx=0).stop 2024-09-10T21:31:56.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- 172.21.15.97:0/3589045000 shutdown_connections 2024-09-10T21:31:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3589045000 >> v2:172.21.15.194:3300/0 conn(0x7fbd480b0d50 0x7fbd480aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3589045000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480ce640 0x7fbd480cea20 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- 172.21.15.97:0/3589045000 >> 172.21.15.97:0/3589045000 conn(0x7fbd480d1c30 msgr2=0x7fbd480d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- 172.21.15.97:0/3589045000 shutdown_connections 2024-09-10T21:31:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- 172.21.15.97:0/3589045000 wait complete. 2024-09-10T21:31:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 Processor -- start 2024-09-10T21:31:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- start start 2024-09-10T21:31:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480af300 0x7fbd480c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd480b0d50 0x7fbd481571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd480ce640 0x7fbd48071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48169270 con 0x7fbd480af300 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481690f0 con 0x7fbd480b0d50 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd4f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbd481693f0 con 0x7fbd480ce640 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480af300 0x7fbd480c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480af300 0x7fbd480c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56758/0 (socket says 172.21.15.97:56758) 2024-09-10T21:31:56.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.456+0000 7fbd47e00640 1 -- 172.21.15.97:0/3150122141 learned_addr learned my addr 172.21.15.97:0/3150122141 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd47e00640 1 -- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd480ce640 msgr2=0x7fbd48071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd47e00640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd480ce640 0x7fbd48071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd47e00640 1 -- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd480b0d50 msgr2=0x7fbd481571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd47e00640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd480b0d50 0x7fbd481571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd47e00640 1 -- 172.21.15.97:0/3150122141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd481579f0 con 0x7fbd480af300 2024-09-10T21:31:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd47e00640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480af300 0x7fbd480c7db0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c10 tx=0x7fbd380a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd44c00640 1 -- 172.21.15.97:0/3150122141 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380bb070 con 0x7fbd480af300 2024-09-10T21:31:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd44c00640 1 -- 172.21.15.97:0/3150122141 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd38004470 con 0x7fbd480af300 2024-09-10T21:31:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd48157cd0 con 0x7fbd480af300 2024-09-10T21:31:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd44c00640 1 -- 172.21.15.97:0/3150122141 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b2040 con 0x7fbd480af300 2024-09-10T21:31:56.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd48158290 con 0x7fbd480af300 2024-09-10T21:31:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd481584d0 con 0x7fbd480af300 2024-09-10T21:31:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd44c00640 1 -- 172.21.15.97:0/3150122141 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbd380b5030 con 0x7fbd480af300 2024-09-10T21:31:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd44c00640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd24072610 0x7fbd24074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.460+0000 7fbd44c00640 1 -- 172.21.15.97:0/3150122141 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7fbd3812cf10 con 0x7fbd480af300 2024-09-10T21:31:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.464+0000 7fbd3f400640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd24072610 0x7fbd24074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.464+0000 7fbd3f400640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd24072610 0x7fbd24074ad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbd300058d0 tx=0x7fbd30005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:56.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.464+0000 7fbd44c00640 1 -- 172.21.15.97:0/3150122141 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd380f2de0 con 0x7fbd480af300 2024-09-10T21:31:56.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.664+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd480aedc0 con 0x7fbd480af300 2024-09-10T21:31:56.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.664+0000 7fbd44c00640 1 -- 172.21.15.97:0/3150122141 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) ==== 95+0+3569 (secure 0 0 0) 0x7fbd481584d0 con 0x7fbd480af300 2024-09-10T21:31:56.668 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:56.668 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":141,"btime":"2024-09-10T21:31:55:917304+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:55.917232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:56.669 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 141 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd24072610 msgr2=0x7fbd24074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd24072610 0x7fbd24074ad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbd300058d0 tx=0x7fbd30005860 comp rx=0 tx=0).stop 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480af300 msgr2=0x7fbd480c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480af300 0x7fbd480c7db0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c10 tx=0x7fbd380a5650 comp rx=0 tx=0).stop 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 shutdown_connections 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbd24072610 0x7fbd24074ad0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbd480ce640 0x7fbd48071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd480b0d50 0x7fbd481571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 --2- 172.21.15.97:0/3150122141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd480af300 0x7fbd480c7db0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 >> 172.21.15.97:0/3150122141 conn(0x7fbd480d1c30 msgr2=0x7fbd480af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 shutdown_connections 2024-09-10T21:31:56.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.668+0000 7fbd4f200640 1 -- 172.21.15.97:0/3150122141 wait complete. 2024-09-10T21:31:56.689 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:31:56.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:31:56.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 -- 172.21.15.97:0/2031639621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49841511f0 msgr2=0x7f49840b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 --2- 172.21.15.97:0/2031639621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49841511f0 0x7f49840b37b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4974009900 tx=0x7f497401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 -- 172.21.15.97:0/2031639621 shutdown_connections 2024-09-10T21:31:56.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 --2- 172.21.15.97:0/2031639621 >> v2:172.21.15.194:3300/0 conn(0x7f49840cf5f0 0x7f498415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 --2- 172.21.15.97:0/2031639621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49841511f0 0x7f49840b37b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 -- 172.21.15.97:0/2031639621 >> 172.21.15.97:0/2031639621 conn(0x7f49840b1780 msgr2=0x7f49840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:56.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 -- 172.21.15.97:0/2031639621 shutdown_connections 2024-09-10T21:31:56.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.828+0000 7f498b200640 1 -- 172.21.15.97:0/2031639621 wait complete. 2024-09-10T21:31:56.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 Processor -- start 2024-09-10T21:31:56.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 -- start start 2024-09-10T21:31:56.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49840b3cf0 0x7f49841f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49840cf5f0 0x7f498414d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49841511f0 0x7f4984159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f498406e3b0 con 0x7f49841511f0 2024-09-10T21:31:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f498415b290 con 0x7f49840b3cf0 2024-09-10T21:31:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f498406e530 con 0x7f49840cf5f0 2024-09-10T21:31:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4989200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49841511f0 0x7f4984159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4989200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49841511f0 0x7f4984159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56764/0 (socket says 172.21.15.97:56764) 2024-09-10T21:31:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4989200640 1 -- 172.21.15.97:0/3981106068 learned_addr learned my addr 172.21.15.97:0/3981106068 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4983e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49840b3cf0 0x7f49841f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f497be00640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49840cf5f0 0x7f498414d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4983e00640 1 -- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49840cf5f0 msgr2=0x7f498414d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4983e00640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49840cf5f0 0x7f498414d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4983e00640 1 -- 172.21.15.97:0/3981106068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49841511f0 msgr2=0x7f4984159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4983e00640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49841511f0 0x7f4984159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4983e00640 1 -- 172.21.15.97:0/3981106068 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498414dab0 con 0x7f49840b3cf0 2024-09-10T21:31:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4989200640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49841511f0 0x7f4984159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4983e00640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49840b3cf0 0x7f49841f8b90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f49740af000 tx=0x7f4974005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4980c00640 1 -- 172.21.15.97:0/3981106068 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740b5040 con 0x7f49840b3cf0 2024-09-10T21:31:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4980c00640 1 -- 172.21.15.97:0/3981106068 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49740bf070 con 0x7f49840b3cf0 2024-09-10T21:31:56.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49841e5a60 con 0x7f49840b3cf0 2024-09-10T21:31:56.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f4980c00640 1 -- 172.21.15.97:0/3981106068 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974003990 con 0x7f49840b3cf0 2024-09-10T21:31:56.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49840ccde0 con 0x7f49840b3cf0 2024-09-10T21:31:56.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.832+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498415a730 con 0x7f49840b3cf0 2024-09-10T21:31:56.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.836+0000 7f4980c00640 1 -- 172.21.15.97:0/3981106068 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f49740b8050 con 0x7f49840b3cf0 2024-09-10T21:31:56.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.836+0000 7f4980c00640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49600725e0 0x7f4960074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:56.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.836+0000 7f4980c00640 1 -- 172.21.15.97:0/3981106068 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7f49741318a0 con 0x7f49840b3cf0 2024-09-10T21:31:56.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.836+0000 7f497be00640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49600725e0 0x7f4960074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:56.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.836+0000 7f497be00640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49600725e0 0x7f4960074aa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f49840cdf00 tx=0x7f496c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:56.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:56.840+0000 7f4980c00640 1 -- 172.21.15.97:0/3981106068 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4974018c30 con 0x7f49840b3cf0 2024-09-10T21:31:57.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.040+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f498406f7b0 con 0x7f49840b3cf0 2024-09-10T21:31:57.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.040+0000 7f4980c00640 1 -- 172.21.15.97:0/3981106068 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+4395 (secure 0 0 0) 0x7f49740fc380 con 0x7f49840b3cf0 2024-09-10T21:31:57.045 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:31:57.045 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":142,"btime":"2024-09-10T21:31:56:919601+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2364393791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2364393791},{"type":"v1","addr":"172.21.15.97:6834","nonce":2364393791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":142}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:55.917232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:31:57.045 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 142 2024-09-10T21:31:57.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49600725e0 msgr2=0x7f4960074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:57.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49600725e0 0x7f4960074aa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f49840cdf00 tx=0x7f496c009040 comp rx=0 tx=0).stop 2024-09-10T21:31:57.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49840b3cf0 msgr2=0x7f49841f8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:57.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49840b3cf0 0x7f49841f8b90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f49740af000 tx=0x7f4974005780 comp rx=0 tx=0).stop 2024-09-10T21:31:57.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 shutdown_connections 2024-09-10T21:31:57.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49600725e0 0x7f4960074aa0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49841511f0 0x7f4984159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49840cf5f0 0x7f498414d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 --2- 172.21.15.97:0/3981106068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49840b3cf0 0x7f49841f8b90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 >> 172.21.15.97:0/3981106068 conn(0x7f49840b1780 msgr2=0x7f4984150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:57.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 shutdown_connections 2024-09-10T21:31:57.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.044+0000 7f498b200640 1 -- 172.21.15.97:0/3981106068 wait complete. 2024-09-10T21:31:57.068 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2024-09-10T21:31:57.068 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2024-09-10T21:31:57.068 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:31:57.068 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:31:57.076 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:31:57.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:31:57.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:31:57.077 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:31:57.132 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:31:57.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:31:57.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:31:57.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.316+0000 7fd1eb600640 1 -- 172.21.15.97:0/1478304358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e40ce5f0 msgr2=0x7fd1e4141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:57.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.316+0000 7fd1eb600640 1 --2- 172.21.15.97:0/1478304358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e40ce5f0 0x7fd1e4141d20 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009900 tx=0x7fd1d401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.316+0000 7fd1eb600640 1 -- 172.21.15.97:0/1478304358 shutdown_connections 2024-09-10T21:31:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.316+0000 7fd1eb600640 1 --2- 172.21.15.97:0/1478304358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e40ce5f0 0x7fd1e4141d20 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.316+0000 7fd1eb600640 1 -- 172.21.15.97:0/1478304358 >> 172.21.15.97:0/1478304358 conn(0x7fd1e40b1780 msgr2=0x7fd1e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.316+0000 7fd1eb600640 1 -- 172.21.15.97:0/1478304358 shutdown_connections 2024-09-10T21:31:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.316+0000 7fd1eb600640 1 -- 172.21.15.97:0/1478304358 wait complete. 2024-09-10T21:31:57.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 Processor -- start 2024-09-10T21:31:57.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 -- start start 2024-09-10T21:31:57.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e40ce5f0 0x7fd1e4156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:57.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1e40ce9d0 0x7fd1e41f8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e415fde0 0x7fd1e4144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e4169dc0 con 0x7fd1e415fde0 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e4169c40 con 0x7fd1e40ce5f0 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e4169f40 con 0x7fd1e40ce9d0 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e415fde0 0x7fd1e4144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e415fde0 0x7fd1e4144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60894/0 (socket says 172.21.15.97:60894) 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 -- 172.21.15.97:0/2470545155 learned_addr learned my addr 172.21.15.97:0/2470545155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e8c00640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e40ce5f0 0x7fd1e4156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1dbe00640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1e40ce9d0 0x7fd1e41f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 -- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1e40ce9d0 msgr2=0x7fd1e41f8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1e40ce9d0 0x7fd1e41f8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 -- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e40ce5f0 msgr2=0x7fd1e4156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:57.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e40ce5f0 0x7fd1e4156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 -- 172.21.15.97:0/2470545155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e4144db0 con 0x7fd1e415fde0 2024-09-10T21:31:57.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e9600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e415fde0 0x7fd1e4144870 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c730 tx=0x7fd1dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:57.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e1600640 1 -- 172.21.15.97:0/2470545155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc003b80 con 0x7fd1e415fde0 2024-09-10T21:31:57.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e1600640 1 -- 172.21.15.97:0/2470545155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1dc003d40 con 0x7fd1e415fde0 2024-09-10T21:31:57.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1e1600640 1 -- 172.21.15.97:0/2470545155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0052d0 con 0x7fd1e415fde0 2024-09-10T21:31:57.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e4156da0 con 0x7fd1e415fde0 2024-09-10T21:31:57.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.320+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e4157360 con 0x7fd1e415fde0 2024-09-10T21:31:57.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.324+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e4141d20 con 0x7fd1e415fde0 2024-09-10T21:31:57.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.324+0000 7fd1e1600640 1 -- 172.21.15.97:0/2470545155 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1dc005680 con 0x7fd1e415fde0 2024-09-10T21:31:57.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.324+0000 7fd1e1600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1c00725e0 0x7fd1c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:57.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.324+0000 7fd1e1600640 1 -- 172.21.15.97:0/2470545155 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7fd1dc093790 con 0x7fd1e415fde0 2024-09-10T21:31:57.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.324+0000 7fd1e8c00640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1c00725e0 0x7fd1c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:57.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.324+0000 7fd1e8c00640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1c00725e0 0x7fd1c0074aa0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40af000 tx=0x7fd1d401b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:57.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.328+0000 7fd1e1600640 1 -- 172.21.15.97:0/2470545155 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd1dc05e160 con 0x7fd1e415fde0 2024-09-10T21:31:57.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.524+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd1e4070670 con 0x7fd1e415fde0 2024-09-10T21:31:57.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1e1600640 1 -- 172.21.15.97:0/2470545155 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v143) ==== 161+0+0 (secure 0 0 0) 0x7fd1dc062b90 con 0x7fd1e415fde0 2024-09-10T21:31:57.924 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:31:57.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1c00725e0 msgr2=0x7fd1c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:57.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1c00725e0 0x7fd1c0074aa0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40af000 tx=0x7fd1d401b120 comp rx=0 tx=0).stop 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e415fde0 msgr2=0x7fd1e4144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e415fde0 0x7fd1e4144870 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c730 tx=0x7fd1dc00cbf0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 shutdown_connections 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1c00725e0 0x7fd1c0074aa0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e415fde0 0x7fd1e4144870 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1e40ce9d0 0x7fd1e41f8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 --2- 172.21.15.97:0/2470545155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1e40ce5f0 0x7fd1e4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 >> 172.21.15.97:0/2470545155 conn(0x7fd1e40b1780 msgr2=0x7fd1e415f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 shutdown_connections 2024-09-10T21:31:57.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:57.924+0000 7fd1eb600640 1 -- 172.21.15.97:0/2470545155 wait complete. 2024-09-10T21:31:57.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 -- 172.21.15.97:0/4062288183 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa41806fd50 msgr2=0x7fa4180701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 --2- 172.21.15.97:0/4062288183 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa41806fd50 0x7fa4180701a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa410009900 tx=0x7fa41001c880 comp rx=0 tx=0).stop 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 -- 172.21.15.97:0/4062288183 shutdown_connections 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 --2- 172.21.15.97:0/4062288183 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa41806fd50 0x7fa4180701a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 --2- 172.21.15.97:0/4062288183 >> v2:172.21.15.194:3300/0 conn(0x7fa418143f90 0x7fa4180b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 -- 172.21.15.97:0/4062288183 >> 172.21.15.97:0/4062288183 conn(0x7fa4180b1780 msgr2=0x7fa4180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 -- 172.21.15.97:0/4062288183 shutdown_connections 2024-09-10T21:31:58.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.088+0000 7fa41e000640 1 -- 172.21.15.97:0/4062288183 wait complete. 2024-09-10T21:31:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 Processor -- start 2024-09-10T21:31:58.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 -- start start 2024-09-10T21:31:58.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa41806fd50 0x7fa41813c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:58.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa418143f90 0x7fa41815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:58.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa418144690 0x7fa418135d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:58.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa418167ca0 con 0x7fa41806fd50 2024-09-10T21:31:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa418167b20 con 0x7fa418144690 2024-09-10T21:31:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa418167e20 con 0x7fa418143f90 2024-09-10T21:31:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa41806fd50 0x7fa41813c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa41806fd50 0x7fa41813c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60904/0 (socket says 172.21.15.97:60904) 2024-09-10T21:31:58.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 -- 172.21.15.97:0/213974987 learned_addr learned my addr 172.21.15.97:0/213974987 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417e00640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa418144690 0x7fa418135d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa416a00640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa418143f90 0x7fa41815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:58.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 -- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa418143f90 msgr2=0x7fa41815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa418143f90 0x7fa41815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 -- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa418144690 msgr2=0x7fa418135d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa418144690 0x7fa418135d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 -- 172.21.15.97:0/213974987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa418137b40 con 0x7fa41806fd50 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417e00640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa418144690 0x7fa418135d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa416a00640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa418143f90 0x7fa41815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa417400640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa41806fd50 0x7fa41813c460 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa40800ec00 tx=0x7fa40800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa40f400640 1 -- 172.21.15.97:0/213974987 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408010630 con 0x7fa41806fd50 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa40f400640 1 -- 172.21.15.97:0/213974987 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4080108c0 con 0x7fa41806fd50 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa40f400640 1 -- 172.21.15.97:0/213974987 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408010cf0 con 0x7fa41806fd50 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa418137e20 con 0x7fa41806fd50 2024-09-10T21:31:58.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181383e0 con 0x7fa41806fd50 2024-09-10T21:31:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.092+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4180701a0 con 0x7fa41806fd50 2024-09-10T21:31:58.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.096+0000 7fa40f400640 1 -- 172.21.15.97:0/213974987 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa408018420 con 0x7fa41806fd50 2024-09-10T21:31:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.096+0000 7fa40f400640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3f8072610 0x7fa3f8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.096+0000 7fa40f400640 1 -- 172.21.15.97:0/213974987 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7fa4080945c0 con 0x7fa41806fd50 2024-09-10T21:31:58.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.096+0000 7fa416a00640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3f8072610 0x7fa3f8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.096+0000 7fa416a00640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3f8072610 0x7fa3f8074ad0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa418137630 tx=0x7fa404009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.100+0000 7fa40f400640 1 -- 172.21.15.97:0/213974987 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa40805f7a0 con 0x7fa41806fd50 2024-09-10T21:31:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.324+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa418138750 con 0x7fa41806fd50 2024-09-10T21:31:58.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.932+0000 7fa40f400640 1 -- 172.21.15.97:0/213974987 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v144) ==== 101+0+0 (secure 0 0 0) 0x7fa40805f0a0 con 0x7fa41806fd50 2024-09-10T21:31:58.936 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3f8072610 msgr2=0x7fa3f8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3f8072610 0x7fa3f8074ad0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa418137630 tx=0x7fa404009040 comp rx=0 tx=0).stop 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa41806fd50 msgr2=0x7fa41813c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa41806fd50 0x7fa41813c460 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa40800ec00 tx=0x7fa40800c6a0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 shutdown_connections 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3f8072610 0x7fa3f8074ad0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa418144690 0x7fa418135d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa418143f90 0x7fa41815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 --2- 172.21.15.97:0/213974987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa41806fd50 0x7fa41813c460 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:58.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 >> 172.21.15.97:0/213974987 conn(0x7fa4180b1780 msgr2=0x7fa4180cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:58.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 shutdown_connections 2024-09-10T21:31:58.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:58.936+0000 7fa41e000640 1 -- 172.21.15.97:0/213974987 wait complete. 2024-09-10T21:31:58.959 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:31:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 Processor -- start 2024-09-10T21:31:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 -- start start 2024-09-10T21:31:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559280778c80 0x55928079c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 0x559280794970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55928077c430 0x559280794eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5592806bc030 con 0x55928077c430 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55928060ca00 con 0x559280778c80 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.000+0000 7fb960359080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5592806bc1b0 con 0x559280779060 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55928077c430 0x559280794eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55928077c430 0x559280794eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60922/0 (socket says 172.21.15.97:60922) 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 -- 172.21.15.97:0/2313993291 learned_addr learned my addr 172.21.15.97:0/2313993291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95d600640 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559280778c80 0x55928079c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95cc00640 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 0x559280794970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 -- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 msgr2=0x559280794970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 0x559280794970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 -- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559280778c80 msgr2=0x55928079c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559280778c80 0x55928079c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 -- 172.21.15.97:0/2313993291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928060ba20 con 0x55928077c430 2024-09-10T21:31:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55928077c430 0x559280794eb0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c590 tx=0x7fb95800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf679cebf95eb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb957400640 1 -- 172.21.15.97:0/2313993291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95800aa70 con 0x55928077c430 2024-09-10T21:31:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb957400640 1 -- 172.21.15.97:0/2313993291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb95800ac10 con 0x55928077c430 2024-09-10T21:31:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb957400640 1 -- 172.21.15.97:0/2313993291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958015440 con 0x55928077c430 2024-09-10T21:31:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- 172.21.15.97:0/2313993291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55928077c430 msgr2=0x559280794eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55928077c430 0x559280794eb0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c590 tx=0x7fb95800ca50 comp rx=0 tx=0).stop 2024-09-10T21:31:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- 172.21.15.97:0/2313993291 shutdown_connections 2024-09-10T21:31:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55928077c430 0x559280794eb0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 0x559280794970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 --2- 172.21.15.97:0/2313993291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559280778c80 0x55928079c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- 172.21.15.97:0/2313993291 >> 172.21.15.97:0/2313993291 conn(0x5592806bd140 msgr2=0x5592807963a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- 172.21.15.97:0/2313993291 shutdown_connections 2024-09-10T21:31:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- 172.21.15.97:0/2313993291 wait complete. 2024-09-10T21:31:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 Processor -- start 2024-09-10T21:31:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- start start 2024-09-10T21:31:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559280778c80 0x5592806ed290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 0x559280779550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55928077c430 0x559280779a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559280792aa0 con 0x559280778c80 2024-09-10T21:31:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559280792800 con 0x55928077c430 2024-09-10T21:31:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb960359080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559280798820 con 0x559280779060 2024-09-10T21:31:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559280778c80 0x5592806ed290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559280778c80 0x5592806ed290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60926/0 (socket says 172.21.15.97:60926) 2024-09-10T21:31:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 learned_addr learned my addr 172.21.15.97:0/33764923 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95e000640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55928077c430 0x559280779a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.004+0000 7fb95cc00640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 0x559280779550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 msgr2=0x559280779550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb95d600640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559280779060 0x559280779550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55928077c430 msgr2=0x559280779a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb95d600640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55928077c430 0x559280779a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592805e5c10 con 0x559280778c80 2024-09-10T21:31:59.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb95d600640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559280778c80 0x5592806ed290 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb95001b970 tx=0x7fb9500ac6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb956000640 1 -- 172.21.15.97:0/33764923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950004220 con 0x559280778c80 2024-09-10T21:31:59.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb956000640 1 -- 172.21.15.97:0/33764923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9500ab610 con 0x559280778c80 2024-09-10T21:31:59.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb956000640 1 -- 172.21.15.97:0/33764923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500aba60 con 0x559280778c80 2024-09-10T21:31:59.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5592806e7130 con 0x559280778c80 2024-09-10T21:31:59.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592806e64b0 con 0x559280778c80 2024-09-10T21:31:59.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb956000640 1 -- 172.21.15.97:0/33764923 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb9500abc00 con 0x559280778c80 2024-09-10T21:31:59.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb956000640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb940070fc0 0x7fb940073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.008+0000 7fb956000640 1 -- 172.21.15.97:0/33764923 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7fb95012e660 con 0x559280778c80 2024-09-10T21:31:59.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb95cc00640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb940070fc0 0x7fb940073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.019 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:31:59.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55928079b860 0x5592806ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592806e5db0 con 0x55928079b860 2024-09-10T21:31:59.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb95e000640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55928079b860 0x5592806ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb95cc00640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb940070fc0 0x7fb940073480 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c710 tx=0x7fb94c00e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb95e000640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55928079b860 0x5592806ebd50 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v106'288 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.012+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5592806ec290 con 0x55928079b860 2024-09-10T21:31:59.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.016+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v106'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.016+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592806ec630 con 0x55928079b860 2024-09-10T21:31:59.022 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:31:59.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.016+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v106'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.016+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083bfc0 con 0x55928079b860 2024-09-10T21:31:59.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.020+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v106'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.020+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083c360 con 0x55928079b860 2024-09-10T21:31:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.020+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v106'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.020+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083afb0 con 0x55928079b860 2024-09-10T21:31:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.020+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v106'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.020+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b3a0 con 0x55928079b860 2024-09-10T21:31:59.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.024+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v106'294 uv268 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.024+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928083b790 con 0x55928079b860 2024-09-10T21:31:59.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.024+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v106'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.024+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b790 con 0x55928079b860 2024-09-10T21:31:59.028 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:31:59.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.024+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v106'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.024+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.028+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v106'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.028+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.028+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v106'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.028+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.032+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v106'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.032+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.036+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v106'300 uv275 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.036+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.040+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v106'301 uv301 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.040+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.043 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:31:59.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.040+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v106'302 uv302 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.040+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.044+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v106'303 uv303 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.044+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.044+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v106'304 uv304 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.044+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b740 con 0x55928079b860 2024-09-10T21:31:59.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.044+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v106'305 uv305 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.044+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083bb60 con 0x55928079b860 2024-09-10T21:31:59.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.048+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v106'306 uv306 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.048+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.048+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v106'307 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.048+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.048+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v106'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.052+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.054 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:31:59.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.052+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v106'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.052+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.052+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v106'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.052+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.056+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v106'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.056+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.056+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v106'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.056+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.056+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v106'313 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.056+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.060+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v106'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.060+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.063 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:31:59.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.060+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v106'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.060+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v106'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v106'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083b9c0 con 0x55928079b860 2024-09-10T21:31:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v106'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x55928079b860 2024-09-10T21:31:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55928083bb30 0x55928083ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083f160 con 0x55928083bb30 2024-09-10T21:31:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.064+0000 7fb95d600640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55928083bb30 0x55928083ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.068+0000 7fb95d600640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55928083bb30 0x55928083ec20 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.068+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v106'219 uv202 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.068+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928083f500 con 0x55928083bb30 2024-09-10T21:31:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.068+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v106'220 uv220 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.068+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928083f8a0 con 0x55928083bb30 2024-09-10T21:31:59.073 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:31:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.072+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v106'221 uv221 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.072+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592808400f0 con 0x55928083bb30 2024-09-10T21:31:59.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.072+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v106'222 uv222 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.072+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840490 con 0x55928083bb30 2024-09-10T21:31:59.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.072+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v106'223 uv223 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.072+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840830 con 0x55928083bb30 2024-09-10T21:31:59.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v106'224 uv207 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x559280840bd0 con 0x55928083bb30 2024-09-10T21:31:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v106'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.079 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:31:59.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v106'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v106'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.076+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.080+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v106'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.080+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.080+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v106'229 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.080+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.080+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v106'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.080+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.085 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:31:59.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.084+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v106'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.084+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280845140 con 0x55928083bb30 2024-09-10T21:31:59.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.084+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v106'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.084+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280845140 con 0x55928083bb30 2024-09-10T21:31:59.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.084+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v106'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.084+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.088+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v106'234 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.088+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.088+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v106'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.088+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.091 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:31:59.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.088+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v106'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.088+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280841f80 con 0x55928083bb30 2024-09-10T21:31:59.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v106'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280841f80 con 0x55928083bb30 2024-09-10T21:31:59.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v106'238 uv238 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280840f70 con 0x55928083bb30 2024-09-10T21:31:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v106'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x55928083bb30 2024-09-10T21:31:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559280841480 0x559280843870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280843ed0 con 0x559280841480 2024-09-10T21:31:59.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb95cc00640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559280841480 0x559280843870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.092+0000 7fb95cc00640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559280841480 0x559280843870 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.096+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.096+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x559280844720 con 0x559280841480 2024-09-10T21:31:59.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.096+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.096+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280844ac0 con 0x559280841480 2024-09-10T21:31:59.101 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:31:59.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.100+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.100+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280845eb0 con 0x559280841480 2024-09-10T21:31:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.100+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.100+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280846310 con 0x559280841480 2024-09-10T21:31:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.100+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.100+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280844ea0 con 0x559280841480 2024-09-10T21:31:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.104+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.104+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5592808459a0 0x559280848710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.104+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280848ce0 con 0x5592808459a0 2024-09-10T21:31:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.104+0000 7fb95e000640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5592808459a0 0x559280848710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.104+0000 7fb95e000640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5592808459a0 0x559280848710 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.104+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v106'48 uv47 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.104+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5592808490d0 con 0x5592808459a0 2024-09-10T21:31:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v106'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592808490d0 con 0x5592808459a0 2024-09-10T21:31:59.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v106'50 uv50 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.110 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:31:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592808490d0 con 0x5592808459a0 2024-09-10T21:31:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v106'51 uv51 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280848ff0 con 0x559280841480 2024-09-10T21:31:59.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.108+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5592808494c0 con 0x559280841480 2024-09-10T21:31:59.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.112+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.112+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592808494c0 con 0x559280841480 2024-09-10T21:31:59.115 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:31:59.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.112+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.112+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084a4d0 con 0x559280841480 2024-09-10T21:31:59.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.112+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.112+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084a4d0 con 0x559280841480 2024-09-10T21:31:59.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.116+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.116+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592808494c0 con 0x559280841480 2024-09-10T21:31:59.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.116+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.116+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5592808494c0 con 0x5592808459a0 2024-09-10T21:31:59.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.116+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v106'105 uv99 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.116+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x5592808494c0 con 0x5592808459a0 2024-09-10T21:31:59.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v106'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849470 con 0x5592808459a0 2024-09-10T21:31:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v106'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.122 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:31:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084a480 con 0x5592808459a0 2024-09-10T21:31:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v106'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084a460 con 0x5592808459a0 2024-09-10T21:31:59.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v106'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.120+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x5592808459a0 2024-09-10T21:31:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.124+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v106'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.124+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x5592808459a0 2024-09-10T21:31:59.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.124+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v106'111 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.124+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x559280849450 con 0x5592808459a0 2024-09-10T21:31:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.124+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v106'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.124+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x5592808459a0 2024-09-10T21:31:59.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v106'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.129 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:31:59.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x5592808459a0 2024-09-10T21:31:59.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v106'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x5592808459a0 2024-09-10T21:31:59.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v106'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x559280841480 2024-09-10T21:31:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v106'46 uv45 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.128+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x559280849450 con 0x559280841480 2024-09-10T21:31:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.132+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v106'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.132+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x559280841480 2024-09-10T21:31:59.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.132+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v106'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.135 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:31:59.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.132+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x559280841480 2024-09-10T21:31:59.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.132+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v106'49 uv49 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.132+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280849450 con 0x559280841480 2024-09-10T21:31:59.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v106'50 uv50 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x559280841480 2024-09-10T21:31:59.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559280849d70 0x55928084a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084c8c0 con 0x559280849d70 2024-09-10T21:31:59.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb95d600640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559280849d70 0x55928084a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb95d600640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559280849d70 0x55928084a1b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928084c8c0 con 0x559280849d70 2024-09-10T21:31:59.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.136+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084c8c0 con 0x559280849d70 2024-09-10T21:31:59.141 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:31:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.140+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.140+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084dcc0 con 0x559280849d70 2024-09-10T21:31:59.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.140+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.140+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084dcc0 con 0x559280849d70 2024-09-10T21:31:59.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.140+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.140+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.146 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:31:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v106'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [write 0~34] v106'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v106'97 uv97 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.144+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 100.00000000 [create] v106'98 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 100.00000000 [delete] v106'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.151 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:31:59.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v106'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084ccb0 con 0x559280849d70 2024-09-10T21:31:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v106'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.148+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fac0 con 0x559280849d70 2024-09-10T21:31:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.152+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v106'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.152+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.152+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v106'44 uv43 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.152+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.152+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v106'45 uv45 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.152+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.156+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v106'46 uv46 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.157 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:31:59.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.156+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.156+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v106'47 uv47 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.156+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.156+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v106'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.156+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.160+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v106'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.160+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.160+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v106'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.162 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:31:59.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.160+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084e170 con 0x5592808459a0 2024-09-10T21:31:59.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.160+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v106'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.160+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084e170 con 0x5592808459a0 2024-09-10T21:31:59.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.160+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v106'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084d160 con 0x5592808459a0 2024-09-10T21:31:59.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v106'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb95800a870 con 0x5592808459a0 2024-09-10T21:31:59.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55928084d670 0x55928084ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x55928084d670 2024-09-10T21:31:59.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb95cc00640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55928084d670 0x55928084ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb95cc00640 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55928084d670 0x55928084ffc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v106'47 uv46 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x55928084d670 2024-09-10T21:31:59.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.164+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928084fa80 con 0x55928084d670 2024-09-10T21:31:59.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v106'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x55928084d670 2024-09-10T21:31:59.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x55928084d670 2024-09-10T21:31:59.170 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:31:59.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v106'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x55928084d670 2024-09-10T21:31:59.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x55928084d670 2024-09-10T21:31:59.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v106'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x55928084d670 2024-09-10T21:31:59.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x55928084d670 2024-09-10T21:31:59.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb95cc00640 1 -- 172.21.15.97:0/33764923 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v106'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb94c008860 con 0x55928084d670 2024-09-10T21:31:59.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.168+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.172+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(114 100.00000000.inode [create] v106'119 uv111 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.172+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.172+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(115 100.00000000.inode [delete] v106'120 uv120 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.172+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.176 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:31:59.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.172+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v106'121 uv121 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.172+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v106'122 uv122 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v106'123 uv123 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 1.00000000 [create] v106'124 uv118 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 1.00000000 [delete] v106'125 uv125 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.176+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.181 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:31:59.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v106'126 uv126 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v106'127 uv127 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v106'128 uv128 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55928084fa80 con 0x559280849d70 2024-09-10T21:31:59.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v106'129 uv129 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.180+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x559280850560 con 0x559280849d70 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb95d600640 1 -- 172.21.15.97:0/33764923 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v106'130 uv130 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb950002050 con 0x559280849d70 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559280841480 msgr2=0x559280843870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559280841480 0x559280843870 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559280849d70 msgr2=0x55928084a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559280849d70 0x55928084a1b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55928084d670 msgr2=0x55928084ffc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55928084d670 0x55928084ffc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5592808459a0 msgr2=0x559280848710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5592808459a0 0x559280848710 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55928083bb30 msgr2=0x55928083ec20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 reap_dead start 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55928083bb30 0x55928083ec20 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb95e000640 1 -- 172.21.15.97:0/33764923 reap_dead start 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55928079b860 msgr2=0x5592806ebd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55928079b860 0x5592806ebd50 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb940070fc0 msgr2=0x7fb940073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb940070fc0 0x7fb940073480 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c710 tx=0x7fb94c00e040 comp rx=0 tx=0).stop 2024-09-10T21:31:59.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559280778c80 msgr2=0x5592806ed290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559280778c80 0x5592806ed290 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb95001b970 tx=0x7fb9500ac6e0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 shutdown_connections 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55928083bb30 0x55928083ec20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55928079b860 0x5592806ebd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb940070fc0 0x7fb940073480 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 --2- 172.21.15.97:0/33764923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559280778c80 0x5592806ed290 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 >> 172.21.15.97:0/33764923 conn(0x5592806bd140 msgr2=0x559280796640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 shutdown_connections 2024-09-10T21:31:59.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.184+0000 7fb960359080 1 -- 172.21.15.97:0/33764923 wait complete. 2024-09-10T21:31:59.191 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2024-09-10T21:31:59.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 Processor -- start 2024-09-10T21:31:59.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- start start 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0984940 0x560ae0a0e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0953da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0a0f2a0 0x560ae09542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ae0a10e70 con 0x560ae0a0eb00 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ae08a3a00 con 0x560ae0984940 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560ae0a10ff0 con 0x560ae0a0f2a0 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0953da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0953da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60950/0 (socket says 172.21.15.97:60950) 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 -- 172.21.15.97:0/1721474869 learned_addr learned my addr 172.21.15.97:0/1721474869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:59.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977c000640 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0984940 0x560ae0a0e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977ca00640 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0a0f2a0 0x560ae09542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 -- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0a0f2a0 msgr2=0x560ae09542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0a0f2a0 0x560ae09542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 -- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0984940 msgr2=0x560ae0a0e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0984940 0x560ae0a0e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 -- 172.21.15.97:0/1721474869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae08a2a20 con 0x560ae0a0eb00 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977ca00640 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0a0f2a0 0x560ae09542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977b600640 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0953da0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f976800b5f0 tx=0x7f976800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a16fe2b2ebe7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977a200640 1 -- 172.21.15.97:0/1721474869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976800a950 con 0x560ae0a0eb00 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977a200640 1 -- 172.21.15.97:0/1721474869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f976800aaf0 con 0x560ae0a0eb00 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977a200640 1 -- 172.21.15.97:0/1721474869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768015470 con 0x560ae0a0eb00 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- 172.21.15.97:0/1721474869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 msgr2=0x560ae0953da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0953da0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f976800b5f0 tx=0x7f976800bab0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- 172.21.15.97:0/1721474869 shutdown_connections 2024-09-10T21:31:59.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0a0f2a0 0x560ae09542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0953da0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 --2- 172.21.15.97:0/1721474869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0984940 0x560ae0a0e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- 172.21.15.97:0/1721474869 >> 172.21.15.97:0/1721474869 conn(0x560ae0983520 msgr2=0x560ae09549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:59.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- 172.21.15.97:0/1721474869 shutdown_connections 2024-09-10T21:31:59.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- 172.21.15.97:0/1721474869 wait complete. 2024-09-10T21:31:59.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 Processor -- start 2024-09-10T21:31:59.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.276+0000 7f977eee0080 1 -- start start 2024-09-10T21:31:59.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0984940 0x560ae0953da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0a07a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0a0f2a0 0x560ae0a2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ae0985a20 con 0x560ae0a0eb00 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ae0a2d2b0 con 0x560ae0a0f2a0 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560ae0985cc0 con 0x560ae0984940 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0a07a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0a07a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60952/0 (socket says 172.21.15.97:60952) 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 -- 172.21.15.97:0/3666437029 learned_addr learned my addr 172.21.15.97:0/3666437029 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977c000640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0984940 0x560ae0953da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977ca00640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0a0f2a0 0x560ae0a2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 -- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0984940 msgr2=0x560ae0953da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560ae0984940 0x560ae0953da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 -- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0a0f2a0 msgr2=0x560ae0a2afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ae0a0f2a0 0x560ae0a2afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 -- 172.21.15.97:0/3666437029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae087cc10 con 0x560ae0a0eb00 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977b600640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0a07a00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9768015700 tx=0x7f9768015730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f9778e00640 1 -- 172.21.15.97:0/3666437029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976800f030 con 0x560ae0a0eb00 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f9778e00640 1 -- 172.21.15.97:0/3666437029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9768004550 con 0x560ae0a0eb00 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f9778e00640 1 -- 172.21.15.97:0/3666437029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768015b60 con 0x560ae0a0eb00 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ae0a0b8b0 con 0x560ae0a0eb00 2024-09-10T21:31:59.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae0a0bdf0 con 0x560ae0a0eb00 2024-09-10T21:31:59.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.280+0000 7f9778e00640 1 -- 172.21.15.97:0/3666437029 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9768008460 con 0x560ae0a0eb00 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f9778e00640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f975c070fc0 0x7f975c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f9778e00640 1 -- 172.21.15.97:0/3666437029 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7f9768093c40 con 0x560ae0a0eb00 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977c000640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f975c070fc0 0x7f975c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ae0a07f40 0x560ae0a09f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x560ae0a0a4b0 con 0x560ae0a07f40 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977ca00640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ae0a07f40 0x560ae0a09f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977c000640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f975c070fc0 0x7f975c073480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f976c004fd0 tx=0x7f976c00f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977ca00640 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ae0a07f40 0x560ae0a09f70 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977ca00640 1 -- 172.21.15.97:0/3666437029 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000 [delete] v106'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9774002050 con 0x560ae0a07f40 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ae0a07f40 msgr2=0x560ae0a09f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560ae0a07f40 0x560ae0a09f70 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f975c070fc0 msgr2=0x7f975c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f975c070fc0 0x7f975c073480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f976c004fd0 tx=0x7f976c00f040 comp rx=0 tx=0).stop 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 msgr2=0x560ae0a07a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 --2- 172.21.15.97:0/3666437029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ae0a0eb00 0x560ae0a07a00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9768015700 tx=0x7f9768015730 comp rx=0 tx=0).stop 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977ca00640 1 -- 172.21.15.97:0/3666437029 reap_dead start 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 shutdown_connections 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 >> 172.21.15.97:0/3666437029 conn(0x560ae0983520 msgr2=0x560ae0a2ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 shutdown_connections 2024-09-10T21:31:59.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.284+0000 7f977eee0080 1 -- 172.21.15.97:0/3666437029 wait complete. 2024-09-10T21:31:59.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- 172.21.15.97:0/1979455078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 msgr2=0x7fdf7815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 --2- 172.21.15.97:0/1979455078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf7815f6d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fdf78141580 tx=0x7fdf6001c7e0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- 172.21.15.97:0/1979455078 shutdown_connections 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 --2- 172.21.15.97:0/1979455078 >> v2:172.21.15.194:3300/0 conn(0x7fdf7815fc10 0x7fdf7813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 --2- 172.21.15.97:0/1979455078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf7815f6d0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- 172.21.15.97:0/1979455078 >> 172.21.15.97:0/1979455078 conn(0x7fdf780b1780 msgr2=0x7fdf780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- 172.21.15.97:0/1979455078 shutdown_connections 2024-09-10T21:31:59.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- 172.21.15.97:0/1979455078 wait complete. 2024-09-10T21:31:59.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 Processor -- start 2024-09-10T21:31:59.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- start start 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf780ce5f0 0x7fdf78070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf78150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdf7815fc10 0x7fdf78152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf781689e0 con 0x7fdf78141d20 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdf78168860 con 0x7fdf780ce5f0 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf7e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdf78168b60 con 0x7fdf7815fc10 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf6ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf78150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf6ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf78150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60980/0 (socket says 172.21.15.97:60980) 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf6ea00640 1 -- 172.21.15.97:0/3015733023 learned_addr learned my addr 172.21.15.97:0/3015733023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf77e00640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdf7815fc10 0x7fdf78152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.436+0000 7fdf77400640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf780ce5f0 0x7fdf78070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6ea00640 1 -- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdf7815fc10 msgr2=0x7fdf78152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6ea00640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdf7815fc10 0x7fdf78152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6ea00640 1 -- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf780ce5f0 msgr2=0x7fdf78070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6ea00640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf780ce5f0 0x7fdf78070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6ea00640 1 -- 172.21.15.97:0/3015733023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7815f110 con 0x7fdf78141d20 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf77400640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf780ce5f0 0x7fdf78070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6ea00640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf78150db0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fdf6001ccc0 tx=0x7fdf60005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6fe00640 1 -- 172.21.15.97:0/3015733023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf600b2040 con 0x7fdf78141d20 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6fe00640 1 -- 172.21.15.97:0/3015733023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf600bb070 con 0x7fdf78141d20 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7815ee30 con 0x7fdf78141d20 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6fe00640 1 -- 172.21.15.97:0/3015733023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60003600 con 0x7fdf78141d20 2024-09-10T21:31:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf78152980 con 0x7fdf78141d20 2024-09-10T21:31:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7815f6d0 con 0x7fdf78141d20 2024-09-10T21:31:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6fe00640 1 -- 172.21.15.97:0/3015733023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdf600b5070 con 0x7fdf78141d20 2024-09-10T21:31:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6fe00640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf540725e0 0x7fdf54074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf6fe00640 1 -- 172.21.15.97:0/3015733023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7fdf6012d910 con 0x7fdf78141d20 2024-09-10T21:31:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.440+0000 7fdf77400640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf540725e0 0x7fdf54074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.444+0000 7fdf77400640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf540725e0 0x7fdf54074aa0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000f70 tx=0x7fdf68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.444+0000 7fdf6fe00640 1 -- 172.21.15.97:0/3015733023 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdf600163d0 con 0x7fdf78141d20 2024-09-10T21:31:59.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.644+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fdf78141360 con 0x7fdf78141d20 2024-09-10T21:31:59.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf6fe00640 1 -- 172.21.15.97:0/3015733023 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v145) ==== 162+0+0 (secure 0 0 0) 0x7fdf600f8a50 con 0x7fdf78141d20 2024-09-10T21:31:59.938 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf540725e0 msgr2=0x7fdf54074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf540725e0 0x7fdf54074aa0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000f70 tx=0x7fdf68009040 comp rx=0 tx=0).stop 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 msgr2=0x7fdf78150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf78150db0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fdf6001ccc0 tx=0x7fdf60005db0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 shutdown_connections 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdf540725e0 0x7fdf54074aa0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdf7815fc10 0x7fdf78152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf78141d20 0x7fdf78150db0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 --2- 172.21.15.97:0/3015733023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf780ce5f0 0x7fdf78070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:59.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 >> 172.21.15.97:0/3015733023 conn(0x7fdf780b1780 msgr2=0x7fdf7813d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:59.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 shutdown_connections 2024-09-10T21:31:59.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:31:59.936+0000 7fdf7e200640 1 -- 172.21.15.97:0/3015733023 wait complete. 2024-09-10T21:31:59.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:00.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- 172.21.15.97:0/3475276634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480b0d50 msgr2=0x7f49480aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 --2- 172.21.15.97:0/3475276634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480b0d50 0x7f49480aedc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f49480733b0 tx=0x7f493401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- 172.21.15.97:0/3475276634 shutdown_connections 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 --2- 172.21.15.97:0/3475276634 >> v2:172.21.15.194:3300/0 conn(0x7f49480af300 0x7f49480af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 --2- 172.21.15.97:0/3475276634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480b0d50 0x7f49480aedc0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 --2- 172.21.15.97:0/3475276634 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49480ce640 0x7f49480cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- 172.21.15.97:0/3475276634 >> 172.21.15.97:0/3475276634 conn(0x7f49480d1c30 msgr2=0x7f49480d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- 172.21.15.97:0/3475276634 shutdown_connections 2024-09-10T21:32:00.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- 172.21.15.97:0/3475276634 wait complete. 2024-09-10T21:32:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 Processor -- start 2024-09-10T21:32:00.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- start start 2024-09-10T21:32:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49480af300 0x7f4948155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49480b0d50 0x7f494806f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480ce640 0x7f494806f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:00.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49480cd270 con 0x7f49480ce640 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49480cd0f0 con 0x7f49480af300 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f4950600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f49480cd3f0 con 0x7f49480b0d50 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480ce640 0x7f494806f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480ce640 0x7f494806f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60996/0 (socket says 172.21.15.97:60996) 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 -- 172.21.15.97:0/3137910229 learned_addr learned my addr 172.21.15.97:0/3137910229 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494d200640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49480b0d50 0x7f494806f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494dc00640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49480af300 0x7f4948155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 -- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49480b0d50 msgr2=0x7f494806f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49480b0d50 0x7f494806f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 -- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49480af300 msgr2=0x7f4948155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49480af300 0x7f4948155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.104+0000 7f494e600640 1 -- 172.21.15.97:0/3137910229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494806e330 con 0x7f49480ce640 2024-09-10T21:32:00.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f494e600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480ce640 0x7f494806f8d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f494000ca80 tx=0x7f494000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4945600640 1 -- 172.21.15.97:0/3137910229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000ae20 con 0x7f49480ce640 2024-09-10T21:32:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4945600640 1 -- 172.21.15.97:0/3137910229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f494000f560 con 0x7f49480ce640 2024-09-10T21:32:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4945600640 1 -- 172.21.15.97:0/3137910229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000f9b0 con 0x7f49480ce640 2024-09-10T21:32:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494806e8b0 con 0x7f49480ce640 2024-09-10T21:32:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494806ee70 con 0x7f49480ce640 2024-09-10T21:32:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49480cd950 con 0x7f49480ce640 2024-09-10T21:32:00.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4945600640 1 -- 172.21.15.97:0/3137910229 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f494000fb50 con 0x7f49480ce640 2024-09-10T21:32:00.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4945600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49280725e0 0x7f4928074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:00.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f4945600640 1 -- 172.21.15.97:0/3137910229 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7135+0+0 (secure 0 0 0) 0x7f4940094240 con 0x7f49480ce640 2024-09-10T21:32:00.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.108+0000 7f494dc00640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49280725e0 0x7f4928074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:00.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.112+0000 7f494dc00640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49280725e0 0x7f4928074aa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f493c00ab40 tx=0x7f493c00a250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:00.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.112+0000 7f4945600640 1 -- 172.21.15.97:0/3137910229 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4940014030 con 0x7f49480ce640 2024-09-10T21:32:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.308+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49480aee60 con 0x7f49480ce640 2024-09-10T21:32:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4945600640 1 -- 172.21.15.97:0/3137910229 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+4404 (secure 0 0 0) 0x7f494005eca0 con 0x7f49480ce640 2024-09-10T21:32:00.313 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:00.313 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":146,"btime":"2024-09-10T21:31:59:942222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:31:59.942204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":146,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/2364393791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2364393791},{"type":"v1","addr":"172.21.15.97:6834","nonce":2364393791}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 146 2024-09-10T21:32:00.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49280725e0 msgr2=0x7f4928074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:00.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49280725e0 0x7f4928074aa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f493c00ab40 tx=0x7f493c00a250 comp rx=0 tx=0).stop 2024-09-10T21:32:00.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480ce640 msgr2=0x7f494806f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480ce640 0x7f494806f8d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f494000ca80 tx=0x7f494000cf40 comp rx=0 tx=0).stop 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 shutdown_connections 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49280725e0 0x7f4928074aa0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49480ce640 0x7f494806f8d0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49480b0d50 0x7f494806f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 --2- 172.21.15.97:0/3137910229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49480af300 0x7f4948155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 >> 172.21.15.97:0/3137910229 conn(0x7f49480d1c30 msgr2=0x7f4948071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 shutdown_connections 2024-09-10T21:32:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:00.312+0000 7f4950600640 1 -- 172.21.15.97:0/3137910229 wait complete. 2024-09-10T21:32:00.334 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:32:00.668 INFO:tasks.ceph.mds.c.smithi097.stderr: -43> 2024-09-10T21:31:55.912+0000 7effb7e00640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-10T21:32:00.668 INFO:tasks.ceph.mds.c.smithi097.stderr: -42> 2024-09-10T21:31:55.912+0000 7effb7e00640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-10T21:32:00.707 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:32:02.964 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:02.960+0000 7fa0f1c00640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-10T21:32:02.965 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:02.964+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:02.965 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:02.964+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:32:05.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- 172.21.15.97:0/226693838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 msgr2=0x7f63c4004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 --2- 172.21.15.97:0/226693838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 0x7f63c4004070 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f63d0006ec0 tx=0x7f63d001ce00 comp rx=0 tx=0).stop 2024-09-10T21:32:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63de800640 1 -- 172.21.15.97:0/226693838 reap_dead start 2024-09-10T21:32:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- 172.21.15.97:0/226693838 shutdown_connections 2024-09-10T21:32:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- 172.21.15.97:0/226693838 >> 172.21.15.97:0/226693838 conn(0x7f63e00b3580 msgr2=0x7f63e00b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- 172.21.15.97:0/226693838 shutdown_connections 2024-09-10T21:32:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- 172.21.15.97:0/226693838 wait complete. 2024-09-10T21:32:05.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 Processor -- start 2024-09-10T21:32:05.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- start start 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 0x7f63e006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e014f8c0 0x7f63e015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63e015bbe0 0x7f63e00d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00af6d0 con 0x7f63c4003c70 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00b2930 con 0x7f63e014f8c0 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63df200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f63e01682c0 con 0x7f63e015bbe0 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 0x7f63e006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 0x7f63e006f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32782/0 (socket says 172.21.15.97:32782) 2024-09-10T21:32:05.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 -- 172.21.15.97:0/2078735691 learned_addr learned my addr 172.21.15.97:0/2078735691 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63d5400640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e014f8c0 0x7f63e015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63de800640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63e015bbe0 0x7f63e00d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 -- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63e015bbe0 msgr2=0x7f63e00d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63e015bbe0 0x7f63e00d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 -- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e014f8c0 msgr2=0x7f63e015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e014f8c0 0x7f63e015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 -- 172.21.15.97:0/2078735691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e006e0c0 con 0x7f63c4003c70 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63d5400640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e014f8c0 0x7f63e015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63de800640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63e015bbe0 0x7f63e00d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.488+0000 7f63dde00640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 0x7f63e006f7b0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f63d001b500 tx=0x7f63d0006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63d6a00640 1 -- 172.21.15.97:0/2078735691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d00a5ba0 con 0x7f63c4003c70 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63d6a00640 1 -- 172.21.15.97:0/2078735691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63d0001460 con 0x7f63c4003c70 2024-09-10T21:32:05.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63d6a00640 1 -- 172.21.15.97:0/2078735691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d00060d0 con 0x7f63c4003c70 2024-09-10T21:32:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e01fb560 con 0x7f63c4003c70 2024-09-10T21:32:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e01fbb20 con 0x7f63c4003c70 2024-09-10T21:32:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e015bff0 con 0x7f63c4003c70 2024-09-10T21:32:05.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63d6a00640 1 -- 172.21.15.97:0/2078735691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f63d0006290 con 0x7f63c4003c70 2024-09-10T21:32:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63d6a00640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63bc072610 0x7f63bc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63d6a00640 1 -- 172.21.15.97:0/2078735691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7f63d012cdd0 con 0x7f63c4003c70 2024-09-10T21:32:05.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.492+0000 7f63d5400640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63bc072610 0x7f63bc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.496+0000 7f63d5400640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63bc072610 0x7f63bc074ad0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000c10 tx=0x7f63c8006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:05.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.496+0000 7f63d6a00640 1 -- 172.21.15.97:0/2078735691 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f63e015bff0 con 0x7f63c4003c70 2024-09-10T21:32:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.696+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e015f770 con 0x7f63c4003c70 2024-09-10T21:32:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.696+0000 7f63d6a00640 1 -- 172.21.15.97:0/2078735691 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) ==== 95+0+4395 (secure 0 0 0) 0x7f63d0016bb0 con 0x7f63c4003c70 2024-09-10T21:32:05.701 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:05.701 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":150,"btime":"2024-09-10T21:32:02:968027+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:02.968024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 150 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63bc072610 msgr2=0x7f63bc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63bc072610 0x7f63bc074ad0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000c10 tx=0x7f63c8006b20 comp rx=0 tx=0).stop 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 msgr2=0x7f63e006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 0x7f63e006f7b0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f63d001b500 tx=0x7f63d0006b10 comp rx=0 tx=0).stop 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 shutdown_connections 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63bc072610 0x7f63bc074ad0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63e015bbe0 0x7f63e00d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e014f8c0 0x7f63e015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 --2- 172.21.15.97:0/2078735691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63c4003c70 0x7f63e006f7b0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 >> 172.21.15.97:0/2078735691 conn(0x7f63e00b3580 msgr2=0x7f63e0071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 shutdown_connections 2024-09-10T21:32:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.700+0000 7f63df200640 1 -- 172.21.15.97:0/2078735691 wait complete. 2024-09-10T21:32:05.722 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:32:05.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 -- 172.21.15.97:0/743525516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9080003fe0 msgr2=0x7f9080004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 --2- 172.21.15.97:0/743525516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9080003fe0 0x7f9080004810 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f9084009b90 tx=0x7f908401d270 comp rx=0 tx=0).stop 2024-09-10T21:32:05.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f909be00640 1 -- 172.21.15.97:0/743525516 reap_dead start 2024-09-10T21:32:05.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 -- 172.21.15.97:0/743525516 shutdown_connections 2024-09-10T21:32:05.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 -- 172.21.15.97:0/743525516 >> 172.21.15.97:0/743525516 conn(0x7f909c0b1780 msgr2=0x7f909c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:05.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 -- 172.21.15.97:0/743525516 shutdown_connections 2024-09-10T21:32:05.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 -- 172.21.15.97:0/743525516 wait complete. 2024-09-10T21:32:05.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 Processor -- start 2024-09-10T21:32:05.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.864+0000 7f90a2000640 1 -- start start 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9080003fe0 0x7f909c159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c13e410 0x7f909c158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f909c1589b0 0x7f909c06f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f909c15f6d0 con 0x7f909c13e410 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f909c140bd0 con 0x7f909c1589b0 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f909c15f850 con 0x7f9080003fe0 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c13e410 0x7f909c158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c13e410 0x7f909c158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32796/0 (socket says 172.21.15.97:32796) 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 -- 172.21.15.97:0/3218935873 learned_addr learned my addr 172.21.15.97:0/3218935873 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f909be00640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f909c1589b0 0x7f909c06f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f909b400640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9080003fe0 0x7f909c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 -- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9080003fe0 msgr2=0x7f909c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9080003fe0 0x7f909c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 -- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f909c1589b0 msgr2=0x7f909c06f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f909c1589b0 0x7f909c06f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 -- 172.21.15.97:0/3218935873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c1f8090 con 0x7f909c13e410 2024-09-10T21:32:05.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093e00640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c13e410 0x7f909c158470 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f9084009b90 tx=0x7f9084005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093400640 1 -- 172.21.15.97:0/3218935873 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90840b1040 con 0x7f909c13e410 2024-09-10T21:32:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093400640 1 -- 172.21.15.97:0/3218935873 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9084005610 con 0x7f909c13e410 2024-09-10T21:32:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c1f8370 con 0x7f909c13e410 2024-09-10T21:32:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093400640 1 -- 172.21.15.97:0/3218935873 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90840046d0 con 0x7f909c13e410 2024-09-10T21:32:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c1f8930 con 0x7f909c13e410 2024-09-10T21:32:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909c141d20 con 0x7f909c13e410 2024-09-10T21:32:05.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.868+0000 7f9093400640 1 -- 172.21.15.97:0/3218935873 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9084004870 con 0x7f909c13e410 2024-09-10T21:32:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.872+0000 7f9093400640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f90780725e0 0x7f9078074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.872+0000 7f9093400640 1 -- 172.21.15.97:0/3218935873 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7f908412e590 con 0x7f909c13e410 2024-09-10T21:32:05.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.872+0000 7f909b400640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f90780725e0 0x7f9078074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:05.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.872+0000 7f909b400640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f90780725e0 0x7f9078074aa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f908c009de0 tx=0x7f908c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:05.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:05.876+0000 7f9093400640 1 -- 172.21.15.97:0/3218935873 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9084016fc0 con 0x7f909c13e410 2024-09-10T21:32:06.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.072+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f909c141360 con 0x7f909c13e410 2024-09-10T21:32:06.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f9093400640 1 -- 172.21.15.97:0/3218935873 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) ==== 95+0+4395 (secure 0 0 0) 0x7f90840f8f90 con 0x7f909c13e410 2024-09-10T21:32:06.077 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:06.078 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":150,"btime":"2024-09-10T21:32:02:968027+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:02.968024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:06.078 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 150 2024-09-10T21:32:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f90780725e0 msgr2=0x7f9078074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f90780725e0 0x7f9078074aa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f908c009de0 tx=0x7f908c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c13e410 msgr2=0x7f909c158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c13e410 0x7f909c158470 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f9084009b90 tx=0x7f9084005520 comp rx=0 tx=0).stop 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 shutdown_connections 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f90780725e0 0x7f9078074aa0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f909c1589b0 0x7f909c06f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c13e410 0x7f909c158470 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 --2- 172.21.15.97:0/3218935873 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9080003fe0 0x7f909c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 >> 172.21.15.97:0/3218935873 conn(0x7f909c0b1780 msgr2=0x7f909c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 shutdown_connections 2024-09-10T21:32:06.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.076+0000 7f90a2000640 1 -- 172.21.15.97:0/3218935873 wait complete. 2024-09-10T21:32:06.098 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:06.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 -- 172.21.15.97:0/2468534506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 msgr2=0x7ffa540b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 --2- 172.21.15.97:0/2468534506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa540b37b0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009930 tx=0x7ffa4401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 -- 172.21.15.97:0/2468534506 shutdown_connections 2024-09-10T21:32:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 --2- 172.21.15.97:0/2468534506 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa540cf5f0 0x7ffa5415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 --2- 172.21.15.97:0/2468534506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa540b37b0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 -- 172.21.15.97:0/2468534506 >> 172.21.15.97:0/2468534506 conn(0x7ffa540b1780 msgr2=0x7ffa540b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:06.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 -- 172.21.15.97:0/2468534506 shutdown_connections 2024-09-10T21:32:06.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.236+0000 7ffa5a000640 1 -- 172.21.15.97:0/2468534506 wait complete. 2024-09-10T21:32:06.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 Processor -- start 2024-09-10T21:32:06.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 -- start start 2024-09-10T21:32:06.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa540b3cf0 0x7ffa54159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa540cf5f0 0x7ffa5413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa54153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5406e530 con 0x7ffa541511f0 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5406e3b0 con 0x7ffa540b3cf0 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5406e6b0 con 0x7ffa540cf5f0 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa54153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa54153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32814/0 (socket says 172.21.15.97:32814) 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 -- 172.21.15.97:0/476860024 learned_addr learned my addr 172.21.15.97:0/476860024 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53400640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa540b3cf0 0x7ffa54159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 -- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa540cf5f0 msgr2=0x7ffa5413f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa4aa00640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa540cf5f0 0x7ffa5413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa540cf5f0 0x7ffa5413f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 -- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa540b3cf0 msgr2=0x7ffa54159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa540b3cf0 0x7ffa54159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 -- 172.21.15.97:0/476860024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa54152a60 con 0x7ffa541511f0 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53400640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa540b3cf0 0x7ffa54159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa4aa00640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa540cf5f0 0x7ffa5413f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa53e00640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa54153b00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9b0 tx=0x7ffa4c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa4be00640 1 -- 172.21.15.97:0/476860024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c004d30 con 0x7ffa541511f0 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa4be00640 1 -- 172.21.15.97:0/476860024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa4c00cb20 con 0x7ffa541511f0 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa4be00640 1 -- 172.21.15.97:0/476860024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c005020 con 0x7ffa541511f0 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa54152d40 con 0x7ffa541511f0 2024-09-10T21:32:06.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.240+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa54153300 con 0x7ffa541511f0 2024-09-10T21:32:06.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.244+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5415a730 con 0x7ffa541511f0 2024-09-10T21:32:06.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.244+0000 7ffa4be00640 1 -- 172.21.15.97:0/476860024 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffa4c022060 con 0x7ffa541511f0 2024-09-10T21:32:06.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.244+0000 7ffa4be00640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa30072610 0x7ffa30074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.244+0000 7ffa4be00640 1 -- 172.21.15.97:0/476860024 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7ffa4c093d40 con 0x7ffa541511f0 2024-09-10T21:32:06.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.244+0000 7ffa53400640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa30072610 0x7ffa30074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:06.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.244+0000 7ffa53400640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa30072610 0x7ffa30074ad0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b2000 tx=0x7ffa4401d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:06.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.248+0000 7ffa4be00640 1 -- 172.21.15.97:0/476860024 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffa4c016030 con 0x7ffa541511f0 2024-09-10T21:32:06.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.444+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa5415be20 con 0x7ffa541511f0 2024-09-10T21:32:06.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa4be00640 1 -- 172.21.15.97:0/476860024 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) ==== 95+0+4395 (secure 0 0 0) 0x7ffa4c05ee20 con 0x7ffa541511f0 2024-09-10T21:32:06.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:06.449 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":150,"btime":"2024-09-10T21:32:02:968027+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:02.968024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 150 2024-09-10T21:32:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa30072610 msgr2=0x7ffa30074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa30072610 0x7ffa30074ad0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b2000 tx=0x7ffa4401d910 comp rx=0 tx=0).stop 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 msgr2=0x7ffa54153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa54153b00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9b0 tx=0x7ffa4c00ee70 comp rx=0 tx=0).stop 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 shutdown_connections 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa30072610 0x7ffa30074ad0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa541511f0 0x7ffa54153b00 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa540cf5f0 0x7ffa5413f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 --2- 172.21.15.97:0/476860024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa540b3cf0 0x7ffa54159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 >> 172.21.15.97:0/476860024 conn(0x7ffa540b1780 msgr2=0x7ffa5415ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 shutdown_connections 2024-09-10T21:32:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.448+0000 7ffa5a000640 1 -- 172.21.15.97:0/476860024 wait complete. 2024-09-10T21:32:06.471 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2024-09-10T21:32:06.471 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2024-09-10T21:32:06.471 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:06.471 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:06.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:32:06.480 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:06.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:06.480 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:32:06.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:32:06.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:06.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 -- 172.21.15.97:0/3129913586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea815fde0 msgr2=0x7f1ea813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 --2- 172.21.15.97:0/3129913586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea815fde0 0x7f1ea813f9b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8074ae0 tx=0x7f1ea001cb80 comp rx=0 tx=0).stop 2024-09-10T21:32:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 -- 172.21.15.97:0/3129913586 shutdown_connections 2024-09-10T21:32:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 --2- 172.21.15.97:0/3129913586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea815fde0 0x7f1ea813f9b0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 --2- 172.21.15.97:0/3129913586 >> v2:172.21.15.194:3300/0 conn(0x7f1ea80ce5f0 0x7f1ea8141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 -- 172.21.15.97:0/3129913586 >> 172.21.15.97:0/3129913586 conn(0x7f1ea80b1780 msgr2=0x7f1ea80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 -- 172.21.15.97:0/3129913586 shutdown_connections 2024-09-10T21:32:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.712+0000 7f1eaea00640 1 -- 172.21.15.97:0/3129913586 wait complete. 2024-09-10T21:32:06.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 Processor -- start 2024-09-10T21:32:06.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 -- start start 2024-09-10T21:32:06.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea80ce5f0 0x7f1ea815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ea80ce9d0 0x7f1ea815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ea815fde0 0x7f1ea8164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8168a70 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8140cd0 con 0x7f1ea815fde0 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea8168bf0 con 0x7f1ea80ce9d0 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea80ce5f0 0x7f1ea815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea80ce5f0 0x7f1ea815e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32830/0 (socket says 172.21.15.97:32830) 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 -- 172.21.15.97:0/1223465572 learned_addr learned my addr 172.21.15.97:0/1223465572 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaca00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ea815fde0 0x7f1ea8164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 -- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ea80ce9d0 msgr2=0x7f1ea815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ea80ce9d0 0x7f1ea815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 -- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ea815fde0 msgr2=0x7f1ea8164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ea815fde0 0x7f1ea8164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 -- 172.21.15.97:0/1223465572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea80b4270 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaca00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ea815fde0 0x7f1ea8164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1ea7e00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea80ce5f0 0x7f1ea815e800 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800ea20 tx=0x7f1e9800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1e9fe00640 1 -- 172.21.15.97:0/1223465572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800cd80 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1e9fe00640 1 -- 172.21.15.97:0/1223465572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e98010430 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea813bce0 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1e9fe00640 1 -- 172.21.15.97:0/1223465572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98010860 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea80b4a00 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.716+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea8141d20 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.720+0000 7f1e9fe00640 1 -- 172.21.15.97:0/1223465572 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1e98010a00 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.720+0000 7f1e9fe00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e88072610 0x7f1e88074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.720+0000 7f1e9fe00640 1 -- 172.21.15.97:0/1223465572 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7f1e98094480 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.720+0000 7f1ea7400640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e88072610 0x7f1e88074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.720+0000 7f1ea7400640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e88072610 0x7f1e88074ad0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007fd0 tx=0x7f1e94007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:06.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.724+0000 7f1e9fe00640 1 -- 172.21.15.97:0/1223465572 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1e9805f4d0 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.920+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1ea80cca70 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.928 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:06.924+0000 7fa0f6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:32:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1e9fe00640 1 -- 172.21.15.97:0/1223465572 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v151) ==== 161+0+0 (secure 0 0 0) 0x7f1e9805edd0 con 0x7f1ea80ce5f0 2024-09-10T21:32:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e88072610 msgr2=0x7f1e88074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e88072610 0x7f1e88074ad0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1e94007fd0 tx=0x7f1e94007900 comp rx=0 tx=0).stop 2024-09-10T21:32:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea80ce5f0 msgr2=0x7f1ea815e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea80ce5f0 0x7f1ea815e800 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800ea20 tx=0x7f1e9800eee0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 shutdown_connections 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e88072610 0x7f1e88074ad0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ea815fde0 0x7f1ea8164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1ea80ce9d0 0x7f1ea815ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 --2- 172.21.15.97:0/1223465572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea80ce5f0 0x7f1ea815e800 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 >> 172.21.15.97:0/1223465572 conn(0x7f1ea80b1780 msgr2=0x7f1ea813df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 shutdown_connections 2024-09-10T21:32:06.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:06.972+0000 7f1eaea00640 1 -- 172.21.15.97:0/1223465572 wait complete. 2024-09-10T21:32:06.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 -- 172.21.15.97:0/465697815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 msgr2=0x7fbdc00b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/465697815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc00b37b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000adc0 tx=0x7fbdb0020470 comp rx=0 tx=0).stop 2024-09-10T21:32:07.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 -- 172.21.15.97:0/465697815 shutdown_connections 2024-09-10T21:32:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/465697815 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbdc00ceee0 0x7fbdc00cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/465697815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc00b37b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 -- 172.21.15.97:0/465697815 >> 172.21.15.97:0/465697815 conn(0x7fbdc00b1780 msgr2=0x7fbdc00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 -- 172.21.15.97:0/465697815 shutdown_connections 2024-09-10T21:32:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 -- 172.21.15.97:0/465697815 wait complete. 2024-09-10T21:32:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 Processor -- start 2024-09-10T21:32:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.136+0000 7fbdc6a00640 1 -- start start 2024-09-10T21:32:07.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdc00b3cf0 0x7fbdc01e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:07.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbdc00ceee0 0x7fbdc013c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:07.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc0150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:07.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc006e740 con 0x7fbdc01511f0 2024-09-10T21:32:07.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc006e5c0 con 0x7fbdc00b3cf0 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc006e8c0 con 0x7fbdc00ceee0 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc0150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32858/0 (socket says 172.21.15.97:32858) 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 -- 172.21.15.97:0/687141376 learned_addr learned my addr 172.21.15.97:0/687141376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdbfe00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdc00b3cf0 0x7fbdc01e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdbf400640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbdc00ceee0 0x7fbdc013c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 -- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbdc00ceee0 msgr2=0x7fbdc013c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbdc00ceee0 0x7fbdc013c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 -- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdc00b3cf0 msgr2=0x7fbdc01e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdc00b3cf0 0x7fbdc01e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 -- 172.21.15.97:0/687141376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc013c5e0 con 0x7fbdc01511f0 2024-09-10T21:32:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdbfe00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdc00b3cf0 0x7fbdc01e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:07.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdbf400640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbdc00ceee0 0x7fbdc013c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:07.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc4a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc0150180 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800b430 tx=0x7fbdb800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:07.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdb7e00640 1 -- 172.21.15.97:0/687141376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8015020 con 0x7fbdc01511f0 2024-09-10T21:32:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdb7e00640 1 -- 172.21.15.97:0/687141376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdb8011920 con 0x7fbdc01511f0 2024-09-10T21:32:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc01e40d0 con 0x7fbdc01511f0 2024-09-10T21:32:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdb7e00640 1 -- 172.21.15.97:0/687141376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8011d70 con 0x7fbdc01511f0 2024-09-10T21:32:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc01e4690 con 0x7fbdc01511f0 2024-09-10T21:32:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.140+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc00cf370 con 0x7fbdc01511f0 2024-09-10T21:32:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.144+0000 7fbdb7e00640 1 -- 172.21.15.97:0/687141376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbdb801a400 con 0x7fbdc01511f0 2024-09-10T21:32:07.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.144+0000 7fbdb7e00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbda00725e0 0x7fbda0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:07.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.144+0000 7fbdb7e00640 1 -- 172.21.15.97:0/687141376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7fbdb8093a50 con 0x7fbdc01511f0 2024-09-10T21:32:07.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.144+0000 7fbdbfe00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbda00725e0 0x7fbda0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:07.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.144+0000 7fbdbfe00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbda00725e0 0x7fbda0074aa0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00b2000 tx=0x7fbdb001ecb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:07.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.148+0000 7fbdb7e00640 1 -- 172.21.15.97:0/687141376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbdb801f020 con 0x7fbdc01511f0 2024-09-10T21:32:07.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.376+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fbdc01e5a90 con 0x7fbdc01511f0 2024-09-10T21:32:07.718 INFO:tasks.ceph.mds.a.smithi097.stderr: -58> 2024-09-10T21:32:02.960+0000 7fa0f1c00640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-10T21:32:07.718 INFO:tasks.ceph.mds.a.smithi097.stderr: -20> 2024-09-10T21:32:06.924+0000 7fa0f6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:32:07.756 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:32:07.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.975+0000 7fbdb7e00640 1 -- 172.21.15.97:0/687141376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v152) ==== 101+0+0 (secure 0 0 0) 0x7fbdb805eb30 con 0x7fbdc01511f0 2024-09-10T21:32:07.981 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:32:07.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbda00725e0 msgr2=0x7fbda0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:07.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbda00725e0 0x7fbda0074aa0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00b2000 tx=0x7fbdb001ecb0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 msgr2=0x7fbdc0150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc0150180 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800b430 tx=0x7fbdb800b8f0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 shutdown_connections 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbda00725e0 0x7fbda0074aa0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc01511f0 0x7fbdc0150180 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbdc00ceee0 0x7fbdc013c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 --2- 172.21.15.97:0/687141376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdc00b3cf0 0x7fbdc01e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 >> 172.21.15.97:0/687141376 conn(0x7fbdc00b1780 msgr2=0x7fbdc015ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 shutdown_connections 2024-09-10T21:32:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:07.979+0000 7fbdc6a00640 1 -- 172.21.15.97:0/687141376 wait complete. 2024-09-10T21:32:08.004 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:08.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 Processor -- start 2024-09-10T21:32:08.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 -- start start 2024-09-10T21:32:08.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe234003640 0x55d8c133ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8c13c67b0 0x55d8c1477cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c14781f0 0x55d8c1310380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d8c12baf40 con 0x55d8c14781f0 2024-09-10T21:32:08.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d8c12c7d00 con 0x55d8c13c67b0 2024-09-10T21:32:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe251621080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d8c125ca00 con 0x7fe234003640 2024-09-10T21:32:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c14781f0 0x55d8c1310380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c14781f0 0x55d8c1310380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53546/0 (socket says 172.21.15.97:53546) 2024-09-10T21:32:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 -- 172.21.15.97:0/4047298596 learned_addr learned my addr 172.21.15.97:0/4047298596 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24de00640 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8c13c67b0 0x55d8c1477cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24e800640 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe234003640 0x55d8c133ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 -- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe234003640 msgr2=0x55d8c133ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe234003640 0x55d8c133ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 -- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8c13c67b0 msgr2=0x55d8c1477cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8c13c67b0 0x55d8c1477cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 -- 172.21.15.97:0/4047298596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c133f4b0 con 0x55d8c14781f0 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24f200640 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c14781f0 0x55d8c1310380 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fe2480096b0 tx=0x7fe234004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676a73e5de15a329 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24ca00640 1 -- 172.21.15.97:0/4047298596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480018b0 con 0x55d8c14781f0 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24ca00640 1 -- 172.21.15.97:0/4047298596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2480011d0 con 0x55d8c14781f0 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.047+0000 7fe24ca00640 1 -- 172.21.15.97:0/4047298596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480076f0 con 0x55d8c14781f0 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- 172.21.15.97:0/4047298596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c14781f0 msgr2=0x55d8c1310380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c14781f0 0x55d8c1310380 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fe2480096b0 tx=0x7fe234004130 comp rx=0 tx=0).stop 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- 172.21.15.97:0/4047298596 shutdown_connections 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c14781f0 0x55d8c1310380 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8c13c67b0 0x55d8c1477cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 --2- 172.21.15.97:0/4047298596 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe234003640 0x55d8c133ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- 172.21.15.97:0/4047298596 >> 172.21.15.97:0/4047298596 conn(0x55d8c13560d0 msgr2=0x55d8c13cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- 172.21.15.97:0/4047298596 shutdown_connections 2024-09-10T21:32:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- 172.21.15.97:0/4047298596 wait complete. 2024-09-10T21:32:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 Processor -- start 2024-09-10T21:32:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- start start 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c13c67b0 0x55d8c130eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d8c14781f0 0x55d8c13100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe234003640 0x55d8c13e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d8c13c5b80 con 0x55d8c13c67b0 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d8c13399a0 con 0x7fe234003640 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d8c1339c00 con 0x55d8c14781f0 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c13c67b0 0x55d8c130eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c13c67b0 0x55d8c130eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53562/0 (socket says 172.21.15.97:53562) 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 learned_addr learned my addr 172.21.15.97:0/804811270 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24de00640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d8c14781f0 0x55d8c13100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24f200640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe234003640 0x55d8c13e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d8c14781f0 msgr2=0x55d8c13100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d8c14781f0 0x55d8c13100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe234003640 msgr2=0x55d8c13e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe234003640 0x55d8c13e23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c130f1d0 con 0x55d8c13c67b0 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24f200640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe234003640 0x55d8c13e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe24e800640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c13c67b0 0x55d8c130eba0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe24000f480 tx=0x7fe24000f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe247400640 1 -- 172.21.15.97:0/804811270 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24000e8f0 con 0x55d8c13c67b0 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe247400640 1 -- 172.21.15.97:0/804811270 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe24000eb80 con 0x55d8c13c67b0 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d8c130f470 con 0x55d8c13c67b0 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe247400640 1 -- 172.21.15.97:0/804811270 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24001a3f0 con 0x55d8c13c67b0 2024-09-10T21:32:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.051+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c130f9b0 con 0x55d8c13c67b0 2024-09-10T21:32:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe247400640 1 -- 172.21.15.97:0/804811270 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe24000c3e0 con 0x55d8c13c67b0 2024-09-10T21:32:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe247400640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe230070ff0 0x7fe2300734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe247400640 1 -- 172.21.15.97:0/804811270 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7fe2400957a0 con 0x55d8c13c67b0 2024-09-10T21:32:08.059 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe24de00640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe230070ff0 0x7fe2300734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d8c13cb610 0x55d8c147f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c147fbd0 con 0x55d8c13cb610 2024-09-10T21:32:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe24f200640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d8c13cb610 0x55d8c147f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe24de00640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe230070ff0 0x7fe2300734b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c950 tx=0x7fe23c004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.055+0000 7fe24f200640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d8c13cb610 0x55d8c147f690 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.059+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v107'319 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.059+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c147ff70 con 0x55d8c13cb610 2024-09-10T21:32:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.059+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v107'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.059+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148bc50 con 0x55d8c13cb610 2024-09-10T21:32:08.065 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.063+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v107'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.063+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148d000 con 0x55d8c13cb610 2024-09-10T21:32:08.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.063+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.063+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148d3a0 con 0x55d8c13cb610 2024-09-10T21:32:08.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.063+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.063+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148bff0 con 0x55d8c13cb610 2024-09-10T21:32:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.067+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.067+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c3e0 con 0x55d8c13cb610 2024-09-10T21:32:08.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.067+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v107'325 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.067+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c148c3e0 con 0x55d8c13cb610 2024-09-10T21:32:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.071+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v107'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.071+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.071+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v107'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.074 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:08.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.071+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.071+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.071+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.075+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.075+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.075+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.075+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.075+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v107'331 uv306 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.075+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.079+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'332 uv332 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.079+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.079+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v107'333 uv333 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.085 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.079+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.083+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'334 uv334 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.083+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.083+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'335 uv335 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.083+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148c390 con 0x55d8c13cb610 2024-09-10T21:32:08.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.083+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'336 uv336 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.083+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cc30 con 0x55d8c13cb610 2024-09-10T21:32:08.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.087+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'337 uv337 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.087+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cc30 con 0x55d8c13cb610 2024-09-10T21:32:08.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.087+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v107'338 uv312 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.087+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.087+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v107'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.087+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.091+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v107'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.094 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:08.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.091+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.091+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.091+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.095+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.095+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.095+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.095+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.095+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v107'344 uv318 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.095+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.099+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v107'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.099+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.099+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v107'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.103 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:08.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.099+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.099+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.099+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.103+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.103+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c13cb610 2024-09-10T21:32:08.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.103+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c13cb610 2024-09-10T21:32:08.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.103+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d8c148dc50 0x55d8c1490040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.103+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c148cac0 con 0x55d8c148dc50 2024-09-10T21:32:08.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.103+0000 7fe24e800640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d8c148dc50 0x55d8c1490040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.103+0000 7fe24e800640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d8c148dc50 0x55d8c1490040 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.107+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'241 uv223 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.107+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c1490580 con 0x55d8c148dc50 2024-09-10T21:32:08.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.107+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'242 uv242 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.107+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1490920 con 0x55d8c148dc50 2024-09-10T21:32:08.112 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:08.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.107+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'243 uv243 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.107+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491170 con 0x55d8c148dc50 2024-09-10T21:32:08.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.111+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v107'244 uv244 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.111+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491510 con 0x55d8c148dc50 2024-09-10T21:32:08.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.111+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'245 uv245 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.111+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14918b0 con 0x55d8c148dc50 2024-09-10T21:32:08.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.111+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v107'246 uv228 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.111+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c1491c50 con 0x55d8c148dc50 2024-09-10T21:32:08.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.115+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v107'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.115+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.115+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v107'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.119 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:08.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.115+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.115+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.115+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.119+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.119+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.119+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v107'251 uv233 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.119+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.119+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v107'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.119+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.123+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v107'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.126 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:08.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.123+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14961c0 con 0x55d8c148dc50 2024-09-10T21:32:08.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.123+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v107'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.123+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14961c0 con 0x55d8c148dc50 2024-09-10T21:32:08.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.123+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.123+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.127+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v107'256 uv239 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.127+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.127+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v107'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.127+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.127+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v107'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.133 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:08.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.127+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1493000 con 0x55d8c148dc50 2024-09-10T21:32:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.131+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.131+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1493000 con 0x55d8c148dc50 2024-09-10T21:32:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.131+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.131+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1491ff0 con 0x55d8c148dc50 2024-09-10T21:32:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c148dc50 2024-09-10T21:32:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d8c1492500 0x55d8c14948f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1494e30 con 0x55d8c1492500 2024-09-10T21:32:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe24de00640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d8c1492500 0x55d8c14948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe24de00640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d8c1492500 0x55d8c14948f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c1495230 con 0x55d8c1492500 2024-09-10T21:32:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.135+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1490db0 con 0x55d8c1492500 2024-09-10T21:32:08.144 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.139+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.139+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1497040 con 0x55d8c1492500 2024-09-10T21:32:08.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.143+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.143+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14973e0 con 0x55d8c1492500 2024-09-10T21:32:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.147+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.147+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1496030 con 0x55d8c1492500 2024-09-10T21:32:08.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.151+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.151+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d8c1496930 0x55d8c1499790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.151+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1499cd0 con 0x55d8c1496930 2024-09-10T21:32:08.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.151+0000 7fe24f200640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d8c1496930 0x55d8c1499790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.155+0000 7fe24f200640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d8c1496930 0x55d8c1499790 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.155+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'52 uv51 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.155+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c1499cd0 con 0x55d8c1496930 2024-09-10T21:32:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.155+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'53 uv53 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.155+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c1499d50 con 0x55d8c1496930 2024-09-10T21:32:08.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.155+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'54 uv54 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.155+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.159+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'55 uv55 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.159+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.159+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.159+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.159+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.159+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.165 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.163+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.163+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149b150 con 0x55d8c1492500 2024-09-10T21:32:08.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.163+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.163+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149b150 con 0x55d8c1492500 2024-09-10T21:32:08.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.163+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.163+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v107'116 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v107'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.172 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:08.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v107'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.167+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149b150 con 0x55d8c1496930 2024-09-10T21:32:08.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149b150 con 0x55d8c1496930 2024-09-10T21:32:08.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v107'122 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.171+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.175+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v107'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.175+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.179 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:08.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.175+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v107'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.175+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.175+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.175+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1496930 2024-09-10T21:32:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v107'51 uv50 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v107'52 uv52 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.184 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v107'53 uv53 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.179+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.183+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'54 uv54 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.183+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149a140 con 0x55d8c1492500 2024-09-10T21:32:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.183+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c1492500 2024-09-10T21:32:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.183+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d8c149a7c0 0x55d8c149cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.183+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d180 con 0x55d8c149a7c0 2024-09-10T21:32:08.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.183+0000 7fe24e800640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d8c149a7c0 0x55d8c149cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.183+0000 7fe24e800640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d8c149a7c0 0x55d8c149cbb0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149d240 con 0x55d8c149a7c0 2024-09-10T21:32:08.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.191 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:08.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149e670 con 0x55d8c149a7c0 2024-09-10T21:32:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.187+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149e670 con 0x55d8c149a7c0 2024-09-10T21:32:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v107'104 uv97 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v107'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.191+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.197 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.195+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v107'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.195+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.195+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'107 uv107 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.195+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.195+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.195+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.201 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:08.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.199+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v107'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.199+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149d660 con 0x55d8c149a7c0 2024-09-10T21:32:08.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.199+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v107'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.199+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0470 con 0x55d8c149a7c0 2024-09-10T21:32:08.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.199+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v107'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.199+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(99 mds0_sessionmap [create] v107'49 uv47 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v107'50 uv50 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.207 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:08.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(101 mds0_sessionmap [create] v107'51 uv51 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v107'52 uv52 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.203+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.207+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(103 609.00000000 [create] v107'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.207+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.207+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(104 609.00000000 [delete] v107'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.207+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.212 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.207+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(105 609.00000000 [create] v107'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.207+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v107'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v107'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c149db10 con 0x55d8c1496930 2024-09-10T21:32:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v107'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe2480074d0 con 0x55d8c1496930 2024-09-10T21:32:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d8c149e190 0x55d8c14a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149e190 2024-09-10T21:32:08.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe24de00640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d8c149e190 0x55d8c14a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.211+0000 7fe24de00640 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d8c149e190 0x55d8c14a0a90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.215+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v107'52 uv51 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c149e190 2024-09-10T21:32:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.215+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149e190 2024-09-10T21:32:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.215+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v107'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c149e190 2024-09-10T21:32:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.215+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149e190 2024-09-10T21:32:08.221 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.219+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v107'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c149e190 2024-09-10T21:32:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.219+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149e190 2024-09-10T21:32:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.219+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v107'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c149e190 2024-09-10T21:32:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.219+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149e190 2024-09-10T21:32:08.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.219+0000 7fe24de00640 1 -- 172.21.15.97:0/804811270 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v107'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe23c00ad80 con 0x55d8c149e190 2024-09-10T21:32:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.219+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(114 100.00000000.inode [create] v107'132 uv123 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(115 100.00000000.inode [delete] v107'133 uv133 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.227 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v107'134 uv134 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v107'135 uv135 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.223+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.227+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v107'136 uv136 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.227+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.227+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 1.00000000 [create] v107'137 uv130 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.227+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.227+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 1.00000000 [delete] v107'138 uv138 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.227+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.233 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:08.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.231+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v107'139 uv139 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.231+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.231+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v107'140 uv140 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.231+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.231+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v107'141 uv141 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.231+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0580 con 0x55d8c149a7c0 2024-09-10T21:32:08.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v107'142 uv142 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55d8c14a0fd0 con 0x55d8c149a7c0 2024-09-10T21:32:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe24e800640 1 -- 172.21.15.97:0/804811270 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v107'143 uv143 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe240065d70 con 0x55d8c149a7c0 2024-09-10T21:32:08.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d8c1492500 msgr2=0x55d8c14948f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d8c1492500 0x55d8c14948f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d8c149a7c0 msgr2=0x55d8c149cbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d8c149a7c0 0x55d8c149cbb0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d8c149e190 msgr2=0x55d8c14a0a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d8c149e190 0x55d8c14a0a90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d8c1496930 msgr2=0x55d8c1499790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d8c1496930 0x55d8c1499790 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d8c148dc50 msgr2=0x55d8c1490040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d8c148dc50 0x55d8c1490040 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d8c13cb610 msgr2=0x55d8c147f690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d8c13cb610 0x55d8c147f690 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 reap_dead start 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 reap_dead start 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 reap_dead start 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe24f200640 1 -- 172.21.15.97:0/804811270 reap_dead start 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe230070ff0 msgr2=0x7fe2300734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe230070ff0 0x7fe2300734b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c950 tx=0x7fe23c004890 comp rx=0 tx=0).stop 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c13c67b0 msgr2=0x55d8c130eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c13c67b0 0x55d8c130eba0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe24000f480 tx=0x7fe24000f940 comp rx=0 tx=0).stop 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 shutdown_connections 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe230070ff0 0x7fe2300734b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 --2- 172.21.15.97:0/804811270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8c13c67b0 0x55d8c130eba0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 >> 172.21.15.97:0/804811270 conn(0x55d8c13560d0 msgr2=0x55d8c133a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 shutdown_connections 2024-09-10T21:32:08.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.235+0000 7fe251621080 1 -- 172.21.15.97:0/804811270 wait complete. 2024-09-10T21:32:08.243 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2024-09-10T21:32:08.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 Processor -- start 2024-09-10T21:32:08.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 -- start start 2024-09-10T21:32:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552cbc80 0x55be552e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 0x55be552ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552e63c0 0x55be552ec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be551bdf40 con 0x55be552cbc80 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55be551cad00 con 0x55be552cc060 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a132a8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55be5515fa00 con 0x55be552e63c0 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552cbc80 0x55be552e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552cbc80 0x55be552e53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53578/0 (socket says 172.21.15.97:53578) 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 -- 172.21.15.97:0/327558849 learned_addr learned my addr 172.21.15.97:0/327558849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a0fa00640 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 0x55be552ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10e00640 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552e63c0 0x55be552ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 -- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552e63c0 msgr2=0x55be552ec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552e63c0 0x55be552ec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 -- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 msgr2=0x55be552ebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 0x55be552ebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 -- 172.21.15.97:0/327558849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be5523de90 con 0x55be552cbc80 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a10400640 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552cbc80 0x55be552e53d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b230 tx=0x7f2a0000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eab341379246fd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a0e600640 1 -- 172.21.15.97:0/327558849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00016070 con 0x55be552cbc80 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.363+0000 7f2a0e600640 1 -- 172.21.15.97:0/327558849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a0000d4c0 con 0x55be552cbc80 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a0e600640 1 -- 172.21.15.97:0/327558849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000d710 con 0x55be552cbc80 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- 172.21.15.97:0/327558849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552cbc80 msgr2=0x55be552e53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552cbc80 0x55be552e53d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b230 tx=0x7f2a0000b6f0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- 172.21.15.97:0/327558849 shutdown_connections 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552e63c0 0x55be552ec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 0x55be552ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 --2- 172.21.15.97:0/327558849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552cbc80 0x55be552e53d0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- 172.21.15.97:0/327558849 >> 172.21.15.97:0/327558849 conn(0x55be55210140 msgr2=0x55be55210d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- 172.21.15.97:0/327558849 shutdown_connections 2024-09-10T21:32:08.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- 172.21.15.97:0/327558849 wait complete. 2024-09-10T21:32:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 Processor -- start 2024-09-10T21:32:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- start start 2024-09-10T21:32:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552cbc80 0x55be55381a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 0x55be552ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552e63c0 0x55be552effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be55211a30 con 0x55be552e63c0 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55be552ea3d0 con 0x55be552cc060 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55be552ec750 con 0x55be552cbc80 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552e63c0 0x55be552effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552e63c0 0x55be552effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53590/0 (socket says 172.21.15.97:53590) 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552cbc80 0x55be55381a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 -- 172.21.15.97:0/2416495912 learned_addr learned my addr 172.21.15.97:0/2416495912 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a0fa00640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 0x55be552ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 -- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552cbc80 msgr2=0x55be55381a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55be552cbc80 0x55be55381a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 -- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 msgr2=0x55be552ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be552cc060 0x55be552ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 -- 172.21.15.97:0/2416495912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be55381fc0 con 0x55be552e63c0 2024-09-10T21:32:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a10e00640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552e63c0 0x55be552effb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f2a08006fb0 tx=0x7f2a0800d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a0d200640 1 -- 172.21.15.97:0/2416495912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800de20 con 0x55be552e63c0 2024-09-10T21:32:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a0d200640 1 -- 172.21.15.97:0/2416495912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a08018070 con 0x55be552e63c0 2024-09-10T21:32:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a0d200640 1 -- 172.21.15.97:0/2416495912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08012830 con 0x55be552e63c0 2024-09-10T21:32:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55be552eec00 con 0x55be552e63c0 2024-09-10T21:32:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.367+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be552ef140 con 0x55be552e63c0 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a0d200640 1 -- 172.21.15.97:0/2416495912 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a080129d0 con 0x55be552e63c0 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a0d200640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29f0070fc0 0x7f29f0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a0d200640 1 -- 172.21.15.97:0/2416495912 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7f2a08092550 con 0x55be552e63c0 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a10400640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29f0070fc0 0x7f29f0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a132a8080 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55be552cb200 0x55be552d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55be552d53f0 con 0x55be552cb200 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a0fa00640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55be552cb200 0x55be552d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a10400640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29f0070fc0 0x7f29f0073480 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b230 tx=0x7f2a00001090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.371+0000 7f2a0fa00640 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55be552cb200 0x55be552d4eb0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a0fa00640 1 -- 172.21.15.97:0/2416495912 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v107'53 uv53 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f29fc0aba80 con 0x55be552cb200 2024-09-10T21:32:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55be552cb200 msgr2=0x55be552d4eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55be552cb200 0x55be552d4eb0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29f0070fc0 msgr2=0x7f29f0073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29f0070fc0 0x7f29f0073480 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b230 tx=0x7f2a00001090 comp rx=0 tx=0).stop 2024-09-10T21:32:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552e63c0 msgr2=0x55be552effb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 --2- 172.21.15.97:0/2416495912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be552e63c0 0x55be552effb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f2a08006fb0 tx=0x7f2a0800d5f0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a10e00640 1 -- 172.21.15.97:0/2416495912 reap_dead start 2024-09-10T21:32:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 shutdown_connections 2024-09-10T21:32:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 >> 172.21.15.97:0/2416495912 conn(0x55be55210140 msgr2=0x55be552ebcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 shutdown_connections 2024-09-10T21:32:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.375+0000 7f2a132a8080 1 -- 172.21.15.97:0/2416495912 wait complete. 2024-09-10T21:32:08.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:08.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 -- 172.21.15.97:0/1239783451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce5f0 msgr2=0x7f8e6c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 --2- 172.21.15.97:0/1239783451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce5f0 0x7f8e6c141d20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0725c0 tx=0x7f8e5c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 -- 172.21.15.97:0/1239783451 shutdown_connections 2024-09-10T21:32:08.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 --2- 172.21.15.97:0/1239783451 >> v2:172.21.15.194:3300/0 conn(0x7f8e6c15fc10 0x7f8e6c15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 --2- 172.21.15.97:0/1239783451 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e6c0ce9d0 0x7f8e6c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 --2- 172.21.15.97:0/1239783451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce5f0 0x7f8e6c141d20 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 -- 172.21.15.97:0/1239783451 >> 172.21.15.97:0/1239783451 conn(0x7f8e6c0b1780 msgr2=0x7f8e6c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:08.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 -- 172.21.15.97:0/1239783451 shutdown_connections 2024-09-10T21:32:08.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 -- 172.21.15.97:0/1239783451 wait complete. 2024-09-10T21:32:08.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.523+0000 7f8e71a00640 1 Processor -- start 2024-09-10T21:32:08.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 -- start start 2024-09-10T21:32:08.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e6c0ce5f0 0x7f8e6c143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce9d0 0x7f8e6c1f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e6c15fc10 0x7f8e6c13e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c140150 con 0x7f8e6c0ce9d0 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c13ffd0 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c1402d0 con 0x7f8e6c15fc10 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce9d0 0x7f8e6c1f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce9d0 0x7f8e6c1f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53598/0 (socket says 172.21.15.97:53598) 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6a000640 1 -- 172.21.15.97:0/2934430489 learned_addr learned my addr 172.21.15.97:0/2934430489 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6aa00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e6c0ce5f0 0x7f8e6c143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6b400640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e6c15fc10 0x7f8e6c13e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6aa00640 1 -- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e6c15fc10 msgr2=0x7f8e6c13e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6aa00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e6c15fc10 0x7f8e6c13e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6aa00640 1 -- 172.21.15.97:0/2934430489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce9d0 msgr2=0x7f8e6c1f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6aa00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce9d0 0x7f8e6c1f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6aa00640 1 -- 172.21.15.97:0/2934430489 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c156710 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6a000640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce9d0 0x7f8e6c1f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:32:08.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e6aa00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e6c0ce5f0 0x7f8e6c143b60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0ac000 tx=0x7f8e5c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e62a00640 1 -- 172.21.15.97:0/2934430489 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0b2040 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e62a00640 1 -- 172.21.15.97:0/2934430489 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e5c003600 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1569b0 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e62a00640 1 -- 172.21.15.97:0/2934430489 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c003a50 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c156f70 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.527+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c15f6d0 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.531+0000 7f8e62a00640 1 -- 172.21.15.97:0/2934430489 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8e5c003bf0 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.531+0000 7f8e62a00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e4c0725e0 0x7f8e4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:08.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.531+0000 7f8e62a00640 1 -- 172.21.15.97:0/2934430489 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7221+0+0 (secure 0 0 0) 0x7f8e5c12cfe0 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.531+0000 7f8e6a000640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e4c0725e0 0x7f8e4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:08.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.531+0000 7f8e6a000640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e4c0725e0 0x7f8e4c074aa0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c157eb0 tx=0x7f8e5800a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.535+0000 7f8e62a00640 1 -- 172.21.15.97:0/2934430489 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e5c0164b0 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.735+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8e6c1598d0 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.987+0000 7f8e62a00640 1 -- 172.21.15.97:0/2934430489 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v153) ==== 162+0+0 (secure 0 0 0) 0x7f8e5c0f7c50 con 0x7f8e6c0ce5f0 2024-09-10T21:32:08.992 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e4c0725e0 msgr2=0x7f8e4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e4c0725e0 0x7f8e4c074aa0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c157eb0 tx=0x7f8e5800a2d0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e6c0ce5f0 msgr2=0x7f8e6c143b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e6c0ce5f0 0x7f8e6c143b60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0ac000 tx=0x7f8e5c0040c0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 shutdown_connections 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8e4c0725e0 0x7f8e4c074aa0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8e6c15fc10 0x7f8e6c13e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c0ce9d0 0x7f8e6c1f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 --2- 172.21.15.97:0/2934430489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e6c0ce5f0 0x7f8e6c143b60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 >> 172.21.15.97:0/2934430489 conn(0x7f8e6c0b1780 msgr2=0x7f8e6c1418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 shutdown_connections 2024-09-10T21:32:08.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:08.991+0000 7f8e71a00640 1 -- 172.21.15.97:0/2934430489 wait complete. 2024-09-10T21:32:08.999 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:08.995+0000 7fb6d7800640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-10T21:32:08.999 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:08.995+0000 7fb6d7800640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-10T21:32:08.999 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:08.995+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:08.999 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:08.995+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:32:09.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:09.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 -- 172.21.15.97:0/2078888667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2300b1590 msgr2=0x7ff2300d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 --2- 172.21.15.97:0/2078888667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2300b1590 0x7ff2300d0370 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7ff23006f650 tx=0x7ff22801d550 comp rx=0 tx=0).stop 2024-09-10T21:32:09.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 -- 172.21.15.97:0/2078888667 shutdown_connections 2024-09-10T21:32:09.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 --2- 172.21.15.97:0/2078888667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2300b1590 0x7ff2300d0370 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 --2- 172.21.15.97:0/2078888667 >> v2:172.21.15.194:3300/0 conn(0x7ff23006dc50 0x7ff2300cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 -- 172.21.15.97:0/2078888667 >> 172.21.15.97:0/2078888667 conn(0x7ff2300d1c30 msgr2=0x7ff2300d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:09.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 -- 172.21.15.97:0/2078888667 shutdown_connections 2024-09-10T21:32:09.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.155+0000 7ff236800640 1 -- 172.21.15.97:0/2078888667 wait complete. 2024-09-10T21:32:09.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 Processor -- start 2024-09-10T21:32:09.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 -- start start 2024-09-10T21:32:09.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2300b1590 0x7ff23014ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff230161600 0x7ff2300b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2301fac00 0x7ff23014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff23016ad20 con 0x7ff230161600 2024-09-10T21:32:09.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff2300b07c0 con 0x7ff2300b1590 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff23016aea0 con 0x7ff2301fac00 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff230161600 0x7ff2300b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff230161600 0x7ff2300b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53602/0 (socket says 172.21.15.97:53602) 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 -- 172.21.15.97:0/1030974521 learned_addr learned my addr 172.21.15.97:0/1030974521 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22f400640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2300b1590 0x7ff23014ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22fe00640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2301fac00 0x7ff23014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 -- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2301fac00 msgr2=0x7ff23014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2301fac00 0x7ff23014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 -- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2300b1590 msgr2=0x7ff23014ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2300b1590 0x7ff23014ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 -- 172.21.15.97:0/1030974521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff230150590 con 0x7ff230161600 2024-09-10T21:32:09.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff22ea00640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff230161600 0x7ff2300b2250 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d980 tx=0x7ff21c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:09.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff227400640 1 -- 172.21.15.97:0/1030974521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c017070 con 0x7ff230161600 2024-09-10T21:32:09.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff227400640 1 -- 172.21.15.97:0/1030974521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff21c009d20 con 0x7ff230161600 2024-09-10T21:32:09.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff227400640 1 -- 172.21.15.97:0/1030974521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0106c0 con 0x7ff230161600 2024-09-10T21:32:09.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff230150870 con 0x7ff230161600 2024-09-10T21:32:09.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff230150e30 con 0x7ff230161600 2024-09-10T21:32:09.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.159+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2300d0a60 con 0x7ff230161600 2024-09-10T21:32:09.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.163+0000 7ff227400640 1 -- 172.21.15.97:0/1030974521 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff21c010860 con 0x7ff230161600 2024-09-10T21:32:09.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.163+0000 7ff227400640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2100725e0 0x7ff210074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.163+0000 7ff227400640 1 -- 172.21.15.97:0/1030974521 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7ff21c093d40 con 0x7ff230161600 2024-09-10T21:32:09.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.163+0000 7ff22f400640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2100725e0 0x7ff210074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.163+0000 7ff22f400640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2100725e0 0x7ff210074aa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff220007fd0 tx=0x7ff220007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:09.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.167+0000 7ff227400640 1 -- 172.21.15.97:0/1030974521 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff21c05e6f0 con 0x7ff230161600 2024-09-10T21:32:09.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.363+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff230072560 con 0x7ff230161600 2024-09-10T21:32:09.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.363+0000 7ff227400640 1 -- 172.21.15.97:0/1030974521 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+4395 (secure 0 0 0) 0x7ff21c063120 con 0x7ff230161600 2024-09-10T21:32:09.368 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:09.368 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":155,"btime":"2024-09-10T21:32:09:002555+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:09.002477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:09.369 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 155 2024-09-10T21:32:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2100725e0 msgr2=0x7ff210074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2100725e0 0x7ff210074aa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff220007fd0 tx=0x7ff220007ea0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff230161600 msgr2=0x7ff2300b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff230161600 0x7ff2300b2250 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d980 tx=0x7ff21c00de40 comp rx=0 tx=0).stop 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 shutdown_connections 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2100725e0 0x7ff210074aa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2301fac00 0x7ff23014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff230161600 0x7ff2300b2250 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 --2- 172.21.15.97:0/1030974521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2300b1590 0x7ff23014ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 >> 172.21.15.97:0/1030974521 conn(0x7ff2300d1c30 msgr2=0x7ff2300b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 shutdown_connections 2024-09-10T21:32:09.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.367+0000 7ff236800640 1 -- 172.21.15.97:0/1030974521 wait complete. 2024-09-10T21:32:09.390 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:09.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:09.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:09.488 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- 172.21.15.97:0/977848774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e8003a60 msgr2=0x7ff9e8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 --2- 172.21.15.97:0/977848774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e8003a60 0x7ff9e8003e40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400abf0 tx=0x7ff9f401cd90 comp rx=0 tx=0).stop 2024-09-10T21:32:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- 172.21.15.97:0/977848774 shutdown_connections 2024-09-10T21:32:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 --2- 172.21.15.97:0/977848774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e8003a60 0x7ff9e8003e40 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- 172.21.15.97:0/977848774 >> 172.21.15.97:0/977848774 conn(0x7ffa040b1780 msgr2=0x7ffa040b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- 172.21.15.97:0/977848774 shutdown_connections 2024-09-10T21:32:09.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- 172.21.15.97:0/977848774 wait complete. 2024-09-10T21:32:09.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 Processor -- start 2024-09-10T21:32:09.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- start start 2024-09-10T21:32:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9e8003a60 0x7ffa0414e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04159170 0x7ffa041595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa04152810 0x7ffa04152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa040cf4d0 con 0x7ffa04159170 2024-09-10T21:32:09.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffa040cf350 con 0x7ffa04152810 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa0a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffa040cf650 con 0x7ff9e8003a60 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04159170 0x7ffa041595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04159170 0x7ffa041595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53614/0 (socket says 172.21.15.97:53614) 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa02a00640 1 -- 172.21.15.97:0/768399122 learned_addr learned my addr 172.21.15.97:0/768399122 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa03e00640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa04152810 0x7ffa04152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.531+0000 7ffa03400640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9e8003a60 0x7ffa0414e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa02a00640 1 -- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9e8003a60 msgr2=0x7ffa0414e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa02a00640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9e8003a60 0x7ffa0414e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa02a00640 1 -- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa04152810 msgr2=0x7ffa04152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa02a00640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa04152810 0x7ffa04152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa02a00640 1 -- 172.21.15.97:0/768399122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa04153500 con 0x7ffa04159170 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa03e00640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa04152810 0x7ffa04152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa02a00640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04159170 0x7ffa041595d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c700 tx=0x7ff9f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:09.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ff9fb400640 1 -- 172.21.15.97:0/768399122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0017070 con 0x7ffa04159170 2024-09-10T21:32:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ff9fb400640 1 -- 172.21.15.97:0/768399122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9f000ce10 con 0x7ffa04159170 2024-09-10T21:32:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ff9fb400640 1 -- 172.21.15.97:0/768399122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0003860 con 0x7ffa04159170 2024-09-10T21:32:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa041537e0 con 0x7ffa04159170 2024-09-10T21:32:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa04153da0 con 0x7ffa04159170 2024-09-10T21:32:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa04159b10 con 0x7ffa04159170 2024-09-10T21:32:09.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ff9fb400640 1 -- 172.21.15.97:0/768399122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff9f0003a00 con 0x7ffa04159170 2024-09-10T21:32:09.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ff9fb400640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9e4072610 0x7ff9e4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.535+0000 7ff9fb400640 1 -- 172.21.15.97:0/768399122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7ff9f0092fc0 con 0x7ffa04159170 2024-09-10T21:32:09.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.539+0000 7ffa03400640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9e4072610 0x7ff9e4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.539+0000 7ffa03400640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9e4072610 0x7ff9e4074ad0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40af000 tx=0x7ff9f4019b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:09.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.539+0000 7ff9fb400640 1 -- 172.21.15.97:0/768399122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff9f005da00 con 0x7ffa04159170 2024-09-10T21:32:09.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.739+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa040b0790 con 0x7ffa04159170 2024-09-10T21:32:09.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ff9fb400640 1 -- 172.21.15.97:0/768399122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+4395 (secure 0 0 0) 0x7ff9f0062430 con 0x7ffa04159170 2024-09-10T21:32:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:09.745 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":155,"btime":"2024-09-10T21:32:09:002555+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:09.002477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:09.746 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 155 2024-09-10T21:32:09.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9e4072610 msgr2=0x7ff9e4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9e4072610 0x7ff9e4074ad0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40af000 tx=0x7ff9f4019b30 comp rx=0 tx=0).stop 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04159170 msgr2=0x7ffa041595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04159170 0x7ffa041595d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c700 tx=0x7ff9f000cbc0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 shutdown_connections 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9e4072610 0x7ff9e4074ad0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa04152810 0x7ffa04152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa04159170 0x7ffa041595d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 --2- 172.21.15.97:0/768399122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9e8003a60 0x7ffa0414e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 >> 172.21.15.97:0/768399122 conn(0x7ffa040b1780 msgr2=0x7ffa040b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 shutdown_connections 2024-09-10T21:32:09.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.743+0000 7ffa0a000640 1 -- 172.21.15.97:0/768399122 wait complete. 2024-09-10T21:32:09.766 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:09.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:09.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.943+0000 7f9ce8800640 1 -- 172.21.15.97:0/3717810992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 msgr2=0x7f9ce00b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.943+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3717810992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce00b37b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009900 tx=0x7f9cd401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- 172.21.15.97:0/3717810992 shutdown_connections 2024-09-10T21:32:09.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3717810992 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ce00b3cf0 0x7f9ce00ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3717810992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce00b37b0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- 172.21.15.97:0/3717810992 >> 172.21.15.97:0/3717810992 conn(0x7f9ce00b1780 msgr2=0x7f9ce00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:09.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- 172.21.15.97:0/3717810992 shutdown_connections 2024-09-10T21:32:09.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- 172.21.15.97:0/3717810992 wait complete. 2024-09-10T21:32:09.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 Processor -- start 2024-09-10T21:32:09.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- start start 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce00b3cf0 0x7f9ce0159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ce00cf420 0x7f9ce01596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce01e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce006e530 con 0x7f9ce01511f0 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce006e3b0 con 0x7f9ce00b3cf0 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce8800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce006e6b0 con 0x7f9ce00cf420 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce01e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce01e6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53622/0 (socket says 172.21.15.97:53622) 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 -- 172.21.15.97:0/3109502230 learned_addr learned my addr 172.21.15.97:0/3109502230 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:09.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce5400640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ce00cf420 0x7f9ce01596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce5e00640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce00b3cf0 0x7f9ce0159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 -- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ce00cf420 msgr2=0x7f9ce01596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ce00cf420 0x7f9ce01596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 -- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce00b3cf0 msgr2=0x7f9ce0159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce00b3cf0 0x7f9ce0159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 -- 172.21.15.97:0/3109502230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce0154700 con 0x7f9ce01511f0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9ce6800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce01e6e00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00c780 tx=0x7f9cdc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.947+0000 7f9cda000640 1 -- 172.21.15.97:0/3109502230 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004b70 con 0x7f9ce01511f0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9cda000640 1 -- 172.21.15.97:0/3109502230 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cdc004e00 con 0x7f9ce01511f0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce01e9ab0 con 0x7f9ce01511f0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9cda000640 1 -- 172.21.15.97:0/3109502230 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00aa90 con 0x7f9ce01511f0 2024-09-10T21:32:09.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce01402b0 con 0x7f9ce01511f0 2024-09-10T21:32:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce013fe00 con 0x7f9ce01511f0 2024-09-10T21:32:09.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9cda000640 1 -- 172.21.15.97:0/3109502230 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9cdc00ac30 con 0x7f9ce01511f0 2024-09-10T21:32:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9cda000640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cc40725e0 0x7f9cc4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9cda000640 1 -- 172.21.15.97:0/3109502230 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f9cdc093c20 con 0x7f9ce01511f0 2024-09-10T21:32:09.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.951+0000 7f9ce5e00640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cc40725e0 0x7f9cc4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:09.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.955+0000 7f9ce5e00640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cc40725e0 0x7f9cc4074aa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40b2000 tx=0x7f9cd401d910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:09.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:09.955+0000 7f9cda000640 1 -- 172.21.15.97:0/3109502230 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9cdc020070 con 0x7f9ce01511f0 2024-09-10T21:32:10.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.155+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ce013faa0 con 0x7f9ce01511f0 2024-09-10T21:32:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9cda000640 1 -- 172.21.15.97:0/3109502230 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+4395 (secure 0 0 0) 0x7f9cdc05ec20 con 0x7f9ce01511f0 2024-09-10T21:32:10.161 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:10.161 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":155,"btime":"2024-09-10T21:32:09:002555+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:09.002477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:10.161 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 155 2024-09-10T21:32:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cc40725e0 msgr2=0x7f9cc4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cc40725e0 0x7f9cc4074aa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40b2000 tx=0x7f9cd401d910 comp rx=0 tx=0).stop 2024-09-10T21:32:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 msgr2=0x7f9ce01e6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:10.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce01e6e00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00c780 tx=0x7f9cdc00cc40 comp rx=0 tx=0).stop 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 shutdown_connections 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9cc40725e0 0x7f9cc4074aa0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ce01511f0 0x7f9ce01e6e00 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ce00cf420 0x7f9ce01596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 --2- 172.21.15.97:0/3109502230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce00b3cf0 0x7f9ce0159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 >> 172.21.15.97:0/3109502230 conn(0x7f9ce00b1780 msgr2=0x7f9ce015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 shutdown_connections 2024-09-10T21:32:10.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.159+0000 7f9ce8800640 1 -- 172.21.15.97:0/3109502230 wait complete. 2024-09-10T21:32:10.182 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2024-09-10T21:32:10.182 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2024-09-10T21:32:10.182 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:10.182 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:10.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:32:10.191 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:10.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:10.191 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:10.244 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:32:10.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:32:10.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- 172.21.15.97:0/2780243995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 msgr2=0x7fa65806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 --2- 172.21.15.97:0/2780243995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa65806cf70 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa6580d1c20 tx=0x7fa65001c890 comp rx=0 tx=0).stop 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- 172.21.15.97:0/2780243995 shutdown_connections 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 --2- 172.21.15.97:0/2780243995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa65806cf70 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 --2- 172.21.15.97:0/2780243995 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6580cfad0 0x7fa6580cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 --2- 172.21.15.97:0/2780243995 >> v2:172.21.15.194:3300/0 conn(0x7fa65815bdd0 0x7fa658074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- 172.21.15.97:0/2780243995 >> 172.21.15.97:0/2780243995 conn(0x7fa6580b28c0 msgr2=0x7fa6580b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- 172.21.15.97:0/2780243995 shutdown_connections 2024-09-10T21:32:10.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- 172.21.15.97:0/2780243995 wait complete. 2024-09-10T21:32:10.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 Processor -- start 2024-09-10T21:32:10.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- start start 2024-09-10T21:32:10.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa6581489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:10.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6580cfad0 0x7fa65815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:10.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa65815bdd0 0x7fa658152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:10.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6580af8f0 con 0x7fa65806cb40 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6580cd6f0 con 0x7fa6580cfad0 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa65e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa6580afa70 con 0x7fa65815bdd0 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa657400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa6581489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa657400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa6581489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53646/0 (socket says 172.21.15.97:53646) 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa657400640 1 -- 172.21.15.97:0/2911913494 learned_addr learned my addr 172.21.15.97:0/2911913494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa656a00640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6580cfad0 0x7fa65815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.427+0000 7fa657e00640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa65815bdd0 0x7fa658152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa657400640 1 -- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa65815bdd0 msgr2=0x7fa658152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa657400640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa65815bdd0 0x7fa658152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa657400640 1 -- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6580cfad0 msgr2=0x7fa65815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa657400640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6580cfad0 0x7fa65815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa657400640 1 -- 172.21.15.97:0/2911913494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa658068fe0 con 0x7fa65806cb40 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa657e00640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa65815bdd0 0x7fa658152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa657400640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa6581489e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa64800eaa0 tx=0x7fa64800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa64f400640 1 -- 172.21.15.97:0/2911913494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64800ce10 con 0x7fa65806cb40 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa64f400640 1 -- 172.21.15.97:0/2911913494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa648011470 con 0x7fa65806cb40 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65815f030 con 0x7fa65806cb40 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa64f400640 1 -- 172.21.15.97:0/2911913494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6480118a0 con 0x7fa65806cb40 2024-09-10T21:32:10.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65806fd60 con 0x7fa65806cb40 2024-09-10T21:32:10.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65815f550 con 0x7fa65806cb40 2024-09-10T21:32:10.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.431+0000 7fa64f400640 1 -- 172.21.15.97:0/2911913494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa648022030 con 0x7fa65806cb40 2024-09-10T21:32:10.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.435+0000 7fa64f400640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6380725e0 0x7fa638074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:10.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.435+0000 7fa64f400640 1 -- 172.21.15.97:0/2911913494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7fa648094170 con 0x7fa65806cb40 2024-09-10T21:32:10.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.435+0000 7fa656a00640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6380725e0 0x7fa638074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:10.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.435+0000 7fa656a00640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6380725e0 0x7fa638074aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6581f9000 tx=0x7fa64400b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:10.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.439+0000 7fa64f400640 1 -- 172.21.15.97:0/2911913494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa65815f550 con 0x7fa65806cb40 2024-09-10T21:32:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.635+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa6580cff10 con 0x7fa65806cb40 2024-09-10T21:32:10.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.991+0000 7fa64f400640 1 -- 172.21.15.97:0/2911913494 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v156) ==== 161+0+0 (secure 0 0 0) 0x7fa648016030 con 0x7fa65806cb40 2024-09-10T21:32:10.997 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6380725e0 msgr2=0x7fa638074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6380725e0 0x7fa638074aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6581f9000 tx=0x7fa64400b390 comp rx=0 tx=0).stop 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 msgr2=0x7fa6581489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa6581489e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa64800eaa0 tx=0x7fa64800ef60 comp rx=0 tx=0).stop 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 shutdown_connections 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6380725e0 0x7fa638074aa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa65815bdd0 0x7fa658152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6580cfad0 0x7fa65815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 --2- 172.21.15.97:0/2911913494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65806cb40 0x7fa6581489e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 >> 172.21.15.97:0/2911913494 conn(0x7fa6580b28c0 msgr2=0x7fa6580d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 shutdown_connections 2024-09-10T21:32:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:10.995+0000 7fa65e200640 1 -- 172.21.15.97:0/2911913494 wait complete. 2024-09-10T21:32:11.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- 172.21.15.97:0/3053125617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 msgr2=0x7eff580b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 --2- 172.21.15.97:0/3053125617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff580b2a80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7eff58073080 tx=0x7eff4801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- 172.21.15.97:0/3053125617 shutdown_connections 2024-09-10T21:32:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 --2- 172.21.15.97:0/3053125617 >> v2:172.21.15.194:3300/0 conn(0x7eff5806fd50 0x7eff58070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 --2- 172.21.15.97:0/3053125617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff580b2a80 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- 172.21.15.97:0/3053125617 >> 172.21.15.97:0/3053125617 conn(0x7eff580b1780 msgr2=0x7eff580b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- 172.21.15.97:0/3053125617 shutdown_connections 2024-09-10T21:32:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- 172.21.15.97:0/3053125617 wait complete. 2024-09-10T21:32:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 Processor -- start 2024-09-10T21:32:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- start start 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff5806fd50 0x7eff581fb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff5815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff58144690 0x7eff5806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff58167c40 con 0x7eff58143f90 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff58167ac0 con 0x7eff5806fd50 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5fa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7eff58167dc0 con 0x7eff58144690 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff5815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff5815cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53654/0 (socket says 172.21.15.97:53654) 2024-09-10T21:32:11.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 -- 172.21.15.97:0/1810571220 learned_addr learned my addr 172.21.15.97:0/1810571220 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5d000640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff5806fd50 0x7eff581fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5da00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff58144690 0x7eff5806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 -- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff58144690 msgr2=0x7eff5806ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff58144690 0x7eff5806ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 -- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff5806fd50 msgr2=0x7eff581fb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff5806fd50 0x7eff581fb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 -- 172.21.15.97:0/1810571220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5815e720 con 0x7eff58143f90 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff5d000640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff5806fd50 0x7eff581fb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.159+0000 7eff4fe00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff5815cc50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c3c0 tx=0x7eff4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff55600640 1 -- 172.21.15.97:0/1810571220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40017070 con 0x7eff58143f90 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff55600640 1 -- 172.21.15.97:0/1810571220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff400053c0 con 0x7eff58143f90 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff55600640 1 -- 172.21.15.97:0/1810571220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40003860 con 0x7eff58143f90 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5815ea00 con 0x7eff58143f90 2024-09-10T21:32:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5815efc0 con 0x7eff58143f90 2024-09-10T21:32:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff58070160 con 0x7eff58143f90 2024-09-10T21:32:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff55600640 1 -- 172.21.15.97:0/1810571220 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff40003a00 con 0x7eff58143f90 2024-09-10T21:32:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff55600640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff55600640 1 -- 172.21.15.97:0/1810571220 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7eff40092eb0 con 0x7eff58143f90 2024-09-10T21:32:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.163+0000 7eff5d000640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:11.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.167+0000 7eff5d000640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff480af000 tx=0x7eff4801b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:11.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.167+0000 7eff55600640 1 -- 172.21.15.97:0/1810571220 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff580706a0 con 0x7eff58143f90 2024-09-10T21:32:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.395+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7eff580cc4e0 con 0x7eff58143f90 2024-09-10T21:32:12.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:11.999+0000 7eff55600640 1 -- 172.21.15.97:0/1810571220 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v157) ==== 101+0+0 (secure 0 0 0) 0x7eff4005d860 con 0x7eff58143f90 2024-09-10T21:32:12.003 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:32:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 msgr2=0x7eff34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7eff480af000 tx=0x7eff4801b100 comp rx=0 tx=0).stop 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 msgr2=0x7eff5815cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff5815cc50 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c3c0 tx=0x7eff4000c880 comp rx=0 tx=0).stop 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 shutdown_connections 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff340725e0 0x7eff34074aa0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff58144690 0x7eff5806ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58143f90 0x7eff5815cc50 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 --2- 172.21.15.97:0/1810571220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff5806fd50 0x7eff581fb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 >> 172.21.15.97:0/1810571220 conn(0x7eff580b1780 msgr2=0x7eff5815da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:12.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 shutdown_connections 2024-09-10T21:32:12.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.003+0000 7eff5fa00640 1 -- 172.21.15.97:0/1810571220 wait complete. 2024-09-10T21:32:12.028 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.071+0000 7fdb58bce080 1 Processor -- start 2024-09-10T21:32:12.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.071+0000 7fdb58bce080 1 -- start start 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.071+0000 7fdb58bce080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453ca9ccc0 0x56453cab87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.071+0000 7fdb58bce080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9e830 0x56453ca29e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.071+0000 7fdb58bce080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453caa21d0 0x56453ca2a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56453c932a00 con 0x56453ca9ccc0 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56453c990f40 con 0x56453caa21d0 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56453cac5440 con 0x56453ca9e830 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453ca9ccc0 0x56453cab87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453ca9ccc0 0x56453cab87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53658/0 (socket says 172.21.15.97:53658) 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 -- 172.21.15.97:0/1161565052 learned_addr learned my addr 172.21.15.97:0/1161565052 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:12.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55400640 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9e830 0x56453ca29e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb56800640 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453caa21d0 0x56453ca2a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 -- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9e830 msgr2=0x56453ca29e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9e830 0x56453ca29e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 -- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453caa21d0 msgr2=0x56453ca2a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453caa21d0 0x56453ca2a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 -- 172.21.15.97:0/1161565052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453c90bc10 con 0x56453ca9ccc0 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55400640 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9e830 0x56453ca29e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:12.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb56800640 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453caa21d0 0x56453ca2a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb55e00640 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453ca9ccc0 0x56453cab87a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fdb48018e30 tx=0x7fdb48017fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc6bbb18d8020b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb4fe00640 1 -- 172.21.15.97:0/1161565052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480b0040 con 0x56453ca9ccc0 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb4fe00640 1 -- 172.21.15.97:0/1161565052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb480ac860 con 0x56453ca9ccc0 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb4fe00640 1 -- 172.21.15.97:0/1161565052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480accb0 con 0x56453ca9ccc0 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- 172.21.15.97:0/1161565052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453ca9ccc0 msgr2=0x56453cab87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453ca9ccc0 0x56453cab87a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fdb48018e30 tx=0x7fdb48017fa0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- 172.21.15.97:0/1161565052 shutdown_connections 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453caa21d0 0x56453ca2a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9e830 0x56453ca29e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 --2- 172.21.15.97:0/1161565052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453ca9ccc0 0x56453cab87a0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- 172.21.15.97:0/1161565052 >> 172.21.15.97:0/1161565052 conn(0x56453ca2c0d0 msgr2=0x56453ca14ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:12.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- 172.21.15.97:0/1161565052 shutdown_connections 2024-09-10T21:32:12.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- 172.21.15.97:0/1161565052 wait complete. 2024-09-10T21:32:12.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 Processor -- start 2024-09-10T21:32:12.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- start start 2024-09-10T21:32:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9ccc0 0x56453cb56c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453ca9e830 0x56453caba170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453caa21d0 0x56453ca9ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56453cab80a0 con 0x56453caa21d0 2024-09-10T21:32:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56453ca9b910 con 0x56453ca9e830 2024-09-10T21:32:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb58bce080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56453ca2a8e0 con 0x56453ca9ccc0 2024-09-10T21:32:12.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.075+0000 7fdb56800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453caa21d0 0x56453ca9ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453ca9e830 0x56453caba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9ccc0 0x56453cb56c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9ccc0 0x56453cb56c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:33950/0 (socket says 172.21.15.97:33950) 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 learned_addr learned my addr 172.21.15.97:0/3315889429 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453ca9e830 0x56453caba170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:40754/0 (socket says 172.21.15.97:40754) 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453ca9e830 msgr2=0x56453caba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453ca9e830 0x56453caba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453caa21d0 msgr2=0x56453ca9ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56453caa21d0 0x56453ca9ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56453cababd0 con 0x56453ca9ccc0 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55400640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56453ca9e830 0x56453caba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb55e00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9ccc0 0x56453cb56c70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb48096c90 tx=0x7fdb480acb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb4ea00640 1 -- 172.21.15.97:0/3315889429 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480c7070 con 0x56453ca9ccc0 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb4ea00640 1 -- 172.21.15.97:0/3315889429 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb480bf560 con 0x56453ca9ccc0 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb4ea00640 1 -- 172.21.15.97:0/3315889429 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480bf9b0 con 0x56453ca9ccc0 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x56453cabae70 con 0x56453ca9ccc0 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x56453cabb3b0 con 0x56453ca9ccc0 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.079+0000 7fdb4ea00640 1 -- 172.21.15.97:0/3315889429 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb480bfb50 con 0x56453ca9ccc0 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb4ea00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb38070ff0 0x7fdb380734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb4ea00640 1 -- 172.21.15.97:0/3315889429 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7fdb4812e3b0 con 0x56453ca9ccc0 2024-09-10T21:32:12.087 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:12.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56453caba6b0 0x56453cb548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb54de0 con 0x56453caba6b0 2024-09-10T21:32:12.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb55400640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb38070ff0 0x7fdb380734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb56800640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56453caba6b0 0x56453cb548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb55400640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb38070ff0 0x7fdb380734b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdb44004700 tx=0x7fdb44004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.083+0000 7fdb56800640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56453caba6b0 0x56453cb548a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.087+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v108'350 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.087+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb551b0 con 0x56453caba6b0 2024-09-10T21:32:12.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.087+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v108'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.087+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb60560 con 0x56453caba6b0 2024-09-10T21:32:12.092 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:12.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.087+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v108'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.087+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb61910 con 0x56453caba6b0 2024-09-10T21:32:12.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.091+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.091+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb61cb0 con 0x56453caba6b0 2024-09-10T21:32:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.091+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.091+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb60900 con 0x56453caba6b0 2024-09-10T21:32:12.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.095+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'355 uv355 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.095+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb60d20 con 0x56453caba6b0 2024-09-10T21:32:12.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.095+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v108'356 uv330 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.095+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb61110 con 0x56453caba6b0 2024-09-10T21:32:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.095+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v108'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.095+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb61110 con 0x56453caba6b0 2024-09-10T21:32:12.102 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.099+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v108'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.099+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.099+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.099+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.103+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.103+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.103+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.103+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.103+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v108'362 uv337 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.103+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.107+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'363 uv363 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.107+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.111 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:12.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.107+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v108'364 uv364 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.107+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.111+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'365 uv365 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.111+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.111+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'366 uv366 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.111+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.111+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'367 uv367 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.111+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.115+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'368 uv368 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.115+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.115+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v108'369 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.115+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.115+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v108'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.115+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.121 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.119+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v108'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.119+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.119+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.119+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.119+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.119+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.123+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.123+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.123+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v108'375 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.123+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.127+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v108'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.127+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.130 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:12.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.127+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v108'377 uv377 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.127+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.127+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.127+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.131+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.131+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb610c0 con 0x56453caba6b0 2024-09-10T21:32:12.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.131+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453caba6b0 2024-09-10T21:32:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.131+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56453cb62050 0x56453cb64440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.131+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb64980 con 0x56453cb62050 2024-09-10T21:32:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.131+0000 7fdb55e00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56453cb62050 0x56453cb64440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.131+0000 7fdb55e00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56453cb62050 0x56453cb64440 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.135+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'262 uv245 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.135+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb64d20 con 0x56453cb62050 2024-09-10T21:32:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.135+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'263 uv263 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.135+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb650c0 con 0x56453cb62050 2024-09-10T21:32:12.140 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.139+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'264 uv264 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.139+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65460 con 0x56453cb62050 2024-09-10T21:32:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.139+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v108'265 uv265 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.139+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65800 con 0x56453cb62050 2024-09-10T21:32:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.139+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'266 uv266 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.139+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65ba0 con 0x56453cb62050 2024-09-10T21:32:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.143+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v108'267 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.143+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.143+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v108'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.143+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.148 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.143+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v108'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.143+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.147+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.147+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.147+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.147+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.147+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v108'272 uv255 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.147+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.151+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v108'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.151+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.154 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:12.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.151+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v108'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.151+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.151+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v108'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.151+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.155+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.155+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.155+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v108'277 uv261 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.155+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.155+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v108'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.155+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.161 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v108'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb65f40 con 0x56453cb62050 2024-09-10T21:32:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb62050 2024-09-10T21:32:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56453cb66830 0x56453cb66c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.159+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb693c0 con 0x56453cb66830 2024-09-10T21:32:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.163+0000 7fdb55400640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56453cb66830 0x56453cb66c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.163+0000 7fdb55400640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56453cb66830 0x56453cb66c50 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.163+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.163+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb69760 con 0x56453cb66830 2024-09-10T21:32:12.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.163+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.163+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69b00 con 0x56453cb66830 2024-09-10T21:32:12.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.167+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.172 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.167+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6b3e0 con 0x56453cb66830 2024-09-10T21:32:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6b780 con 0x56453cb66830 2024-09-10T21:32:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6a3d0 con 0x56453cb66830 2024-09-10T21:32:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56453cb6acd0 0x56453cb6db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6e070 con 0x56453cb6acd0 2024-09-10T21:32:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.171+0000 7fdb56800640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56453cb6acd0 0x56453cb6db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.175+0000 7fdb56800640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56453cb6acd0 0x56453cb6db30 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.175+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'56 uv55 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.175+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb6e070 con 0x56453cb6acd0 2024-09-10T21:32:12.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.175+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'57 uv57 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.175+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6e490 con 0x56453cb6acd0 2024-09-10T21:32:12.180 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.175+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'58 uv58 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.175+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6e490 con 0x56453cb6acd0 2024-09-10T21:32:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'59 uv59 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6e490 con 0x56453cb66830 2024-09-10T21:32:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb6e490 con 0x56453cb66830 2024-09-10T21:32:12.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6e490 con 0x56453cb66830 2024-09-10T21:32:12.184 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.179+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6f4a0 con 0x56453cb66830 2024-09-10T21:32:12.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.183+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.183+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb6f4a0 con 0x56453cb66830 2024-09-10T21:32:12.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.183+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.183+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb66830 2024-09-10T21:32:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.183+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.183+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v108'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v108'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v108'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.192 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:12.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.187+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.191+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.191+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.191+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.191+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.195+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v108'133 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.195+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.195+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v108'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.195+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.199 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.195+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v108'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.195+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.195+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.199+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb6acd0 2024-09-10T21:32:12.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.199+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.199+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb66830 2024-09-10T21:32:12.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.199+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v108'56 uv55 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.199+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb69fa0 con 0x56453cb66830 2024-09-10T21:32:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.199+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v108'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.199+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69f20 con 0x56453cb66830 2024-09-10T21:32:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v108'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.205 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:12.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69f20 con 0x56453cb66830 2024-09-10T21:32:12.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'59 uv59 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb69f20 con 0x56453cb66830 2024-09-10T21:32:12.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'60 uv60 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb66830 2024-09-10T21:32:12.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56453cb6ee50 0x56453cb71240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71780 con 0x56453cb6ee50 2024-09-10T21:32:12.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb55e00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56453cb6ee50 0x56453cb71240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.203+0000 7fdb55e00640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56453cb6ee50 0x56453cb71240 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.207+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.207+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb71780 con 0x56453cb6ee50 2024-09-10T21:32:12.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.207+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.207+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.211 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.207+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.207+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72b80 con 0x56453cb6ee50 2024-09-10T21:32:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72b80 con 0x56453cb6ee50 2024-09-10T21:32:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v108'112 uv108 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.211+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.215+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v108'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.215+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.218 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:12.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.215+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v108'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.215+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.215+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.215+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v108'117 uv111 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v108'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.224 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:12.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v108'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.219+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb71b70 con 0x56453cb6ee50 2024-09-10T21:32:12.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.223+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.223+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb74980 con 0x56453cb6ee50 2024-09-10T21:32:12.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.223+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.223+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72020 con 0x56453cb6acd0 2024-09-10T21:32:12.228 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:12.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.223+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v108'54 uv54 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.223+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72020 con 0x56453cb6acd0 2024-09-10T21:32:12.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.227+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v108'55 uv55 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.227+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72020 con 0x56453cb6acd0 2024-09-10T21:32:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.227+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 609.00000000 [create] v108'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.227+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb72020 con 0x56453cb6acd0 2024-09-10T21:32:12.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.227+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 609.00000000 [delete] v108'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.231+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72020 con 0x56453cb6acd0 2024-09-10T21:32:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.231+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v108'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.231+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72020 con 0x56453cb6acd0 2024-09-10T21:32:12.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.231+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v108'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.231+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73030 con 0x56453cb6acd0 2024-09-10T21:32:12.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.231+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v108'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.231+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72020 con 0x56453cb6acd0 2024-09-10T21:32:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [omap-set-header] v108'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb50002930 con 0x56453cb6acd0 2024-09-10T21:32:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56453cb72530 0x56453cb72910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72e50 con 0x56453cb72530 2024-09-10T21:32:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb55400640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56453cb72530 0x56453cb72910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb55400640 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56453cb72530 0x56453cb72910 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(109 400.00000000 [create] v108'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb72530 2024-09-10T21:32:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb72e50 con 0x56453cb72530 2024-09-10T21:32:12.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(110 400.00000000 [delete] v108'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb72530 2024-09-10T21:32:12.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.235+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72e50 con 0x56453cb72530 2024-09-10T21:32:12.241 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:12.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.239+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(111 400.00000000 [create] v108'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb72530 2024-09-10T21:32:12.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.239+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72e50 con 0x56453cb72530 2024-09-10T21:32:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.239+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v108'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb72530 2024-09-10T21:32:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.239+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72e50 con 0x56453cb72530 2024-09-10T21:32:12.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.239+0000 7fdb55400640 1 -- 172.21.15.97:0/3315889429 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v108'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdb4400b800 con 0x56453cb72530 2024-09-10T21:32:12.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.239+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb72e50 con 0x56453cb6ee50 2024-09-10T21:32:12.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(114 100.00000000.inode [create] v108'144 uv136 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb72e50 con 0x56453cb6ee50 2024-09-10T21:32:12.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(115 100.00000000.inode [delete] v108'145 uv145 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb734c0 con 0x56453cb6ee50 2024-09-10T21:32:12.247 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(116 100.00000000.inode [create] v108'146 uv146 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb734c0 con 0x56453cb6ee50 2024-09-10T21:32:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v108'147 uv147 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.243+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.247+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v108'148 uv148 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.247+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.247+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(119 1.00000000 [create] v108'149 uv143 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.247+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.247+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(120 1.00000000 [delete] v108'150 uv150 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.247+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(121 1.00000000 [create] v108'151 uv151 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.253 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:12.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v108'152 uv152 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v108'153 uv153 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v108'154 uv154 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.251+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x56453cb73470 con 0x56453cb6ee50 2024-09-10T21:32:12.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb55e00640 1 -- 172.21.15.97:0/3315889429 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v108'155 uv155 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdb480c9030 con 0x56453cb6ee50 2024-09-10T21:32:12.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56453cb66830 msgr2=0x56453cb66c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56453cb66830 0x56453cb66c50 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56453cb6ee50 msgr2=0x56453cb71240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x56453cb6ee50 0x56453cb71240 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56453cb72530 msgr2=0x56453cb72910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x56453cb72530 0x56453cb72910 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56453cb6acd0 msgr2=0x56453cb6db30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x56453cb6acd0 0x56453cb6db30 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 reap_dead start 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56453cb62050 msgr2=0x56453cb64440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x56453cb62050 0x56453cb64440 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb56800640 1 -- 172.21.15.97:0/3315889429 reap_dead start 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56453caba6b0 msgr2=0x56453cb548a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56453caba6b0 0x56453cb548a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb38070ff0 msgr2=0x7fdb380734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb38070ff0 0x7fdb380734b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdb44004700 tx=0x7fdb44004890 comp rx=0 tx=0).stop 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9ccc0 msgr2=0x56453cb56c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9ccc0 0x56453cb56c70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb48096c90 tx=0x7fdb480acb60 comp rx=0 tx=0).stop 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 shutdown_connections 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x56453caba6b0 0x56453cb548a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb38070ff0 0x7fdb380734b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 --2- 172.21.15.97:0/3315889429 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56453ca9ccc0 0x56453cb56c70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 >> 172.21.15.97:0/3315889429 conn(0x56453ca2c0d0 msgr2=0x56453ca9bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 shutdown_connections 2024-09-10T21:32:12.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.255+0000 7fdb58bce080 1 -- 172.21.15.97:0/3315889429 wait complete. 2024-09-10T21:32:12.263 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2024-09-10T21:32:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 Processor -- start 2024-09-10T21:32:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- start start 2024-09-10T21:32:12.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4003cc0 0x5621c3f95540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4005a00 0x5621c401fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 0x5621c4020430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5621c3e99a00 con 0x5621c4003cc0 2024-09-10T21:32:12.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5621c3ef7f40 con 0x5621c4005a00 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5621c4007710 con 0x5621c40091d0 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4003cc0 0x5621c3f95540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4003cc0 0x5621c3f95540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53670/0 (socket says 172.21.15.97:53670) 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 -- 172.21.15.97:0/1089401047 learned_addr learned my addr 172.21.15.97:0/1089401047 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ca00640 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 0x5621c4020430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047b600640 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4005a00 0x5621c401fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 -- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 msgr2=0x5621c4020430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 0x5621c4020430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 -- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4005a00 msgr2=0x5621c401fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4005a00 0x5621c401fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 -- 172.21.15.97:0/1089401047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c3e72c10 con 0x5621c4003cc0 2024-09-10T21:32:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047c000640 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4003cc0 0x5621c3f95540 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f046c00b230 tx=0x7f046c00b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdc4b13d21ab81a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047a200640 1 -- 172.21.15.97:0/1089401047 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c016070 con 0x5621c4003cc0 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047a200640 1 -- 172.21.15.97:0/1089401047 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f046c00d4c0 con 0x5621c4003cc0 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047a200640 1 -- 172.21.15.97:0/1089401047 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c00be60 con 0x5621c4003cc0 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- 172.21.15.97:0/1089401047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4003cc0 msgr2=0x5621c3f95540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4003cc0 0x5621c3f95540 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f046c00b230 tx=0x7f046c00b6f0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- 172.21.15.97:0/1089401047 shutdown_connections 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 0x5621c4020430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4005a00 0x5621c401fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 --2- 172.21.15.97:0/1089401047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4003cc0 0x5621c3f95540 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- 172.21.15.97:0/1089401047 >> 172.21.15.97:0/1089401047 conn(0x5621c3f930d0 msgr2=0x5621c3f4e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- 172.21.15.97:0/1089401047 shutdown_connections 2024-09-10T21:32:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.347+0000 7f047ee03080 1 -- 172.21.15.97:0/1089401047 wait complete. 2024-09-10T21:32:12.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 Processor -- start 2024-09-10T21:32:12.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 -- start start 2024-09-10T21:32:12.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4003cc0 0x5621c4009b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4005a00 0x5621c40021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 0x5621c4002700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5621c3f7c520 con 0x5621c4005a00 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5621c3f7c280 con 0x5621c4003cc0 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5621c4022830 con 0x5621c40091d0 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4005a00 0x5621c40021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4005a00 0x5621c40021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53674/0 (socket says 172.21.15.97:53674) 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 -- 172.21.15.97:0/3534356533 learned_addr learned my addr 172.21.15.97:0/3534356533 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ca00640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 0x5621c4002700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047c000640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4003cc0 0x5621c4009b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 -- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 msgr2=0x5621c4002700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5621c40091d0 0x5621c4002700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 -- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4003cc0 msgr2=0x5621c4009b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4003cc0 0x5621c4009b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 -- 172.21.15.97:0/3534356533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c3f8fda0 con 0x5621c4005a00 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047c000640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621c4003cc0 0x5621c4009b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047b600640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4005a00 0x5621c40021c0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0468096d40 tx=0x7f04680ac990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f0478e00640 1 -- 172.21.15.97:0/3534356533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680acc40 con 0x5621c4005a00 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f0478e00640 1 -- 172.21.15.97:0/3534356533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04680b8440 con 0x5621c4005a00 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5621c3f90040 con 0x5621c4005a00 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f0478e00640 1 -- 172.21.15.97:0/3534356533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680b8650 con 0x5621c4005a00 2024-09-10T21:32:12.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.351+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c3f90580 con 0x5621c4005a00 2024-09-10T21:32:12.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f0478e00640 1 -- 172.21.15.97:0/3534356533 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04680b87f0 con 0x5621c4005a00 2024-09-10T21:32:12.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f0478e00640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f045c070ff0 0x7f045c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f0478e00640 1 -- 172.21.15.97:0/3534356533 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f046812da10 con 0x5621c4005a00 2024-09-10T21:32:12.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f047c000640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f045c070ff0 0x7f045c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f047ee03080 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5621c400a050 0x5621c3f8e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x5621c40230e0 con 0x5621c400a050 2024-09-10T21:32:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f047ca00640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5621c400a050 0x5621c3f8e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f047c000640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f045c070ff0 0x7f045c0734b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f046c014040 tx=0x7f046c001090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.355+0000 7f047ca00640 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5621c400a050 0x5621c3f8e020 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ca00640 1 -- 172.21.15.97:0/3534356533 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 400.00000000 [delete] v108'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0474002040 con 0x5621c400a050 2024-09-10T21:32:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5621c400a050 msgr2=0x5621c3f8e020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5621c400a050 0x5621c3f8e020 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f045c070ff0 msgr2=0x7f045c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f045c070ff0 0x7f045c0734b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f046c014040 tx=0x7f046c001090 comp rx=0 tx=0).stop 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4005a00 msgr2=0x5621c40021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 --2- 172.21.15.97:0/3534356533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621c4005a00 0x5621c40021c0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f0468096d40 tx=0x7f04680ac990 comp rx=0 tx=0).stop 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ca00640 1 -- 172.21.15.97:0/3534356533 reap_dead start 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 shutdown_connections 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 >> 172.21.15.97:0/3534356533 conn(0x5621c3f930d0 msgr2=0x5621c3f7b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 shutdown_connections 2024-09-10T21:32:12.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.359+0000 7f047ee03080 1 -- 172.21.15.97:0/3534356533 wait complete. 2024-09-10T21:32:12.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 -- 172.21.15.97:0/2877681173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815f150 msgr2=0x7fea380b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 --2- 172.21.15.97:0/2877681173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815f150 0x7fea380b4380 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fea380cd1a0 tx=0x7fea2801caf0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 -- 172.21.15.97:0/2877681173 shutdown_connections 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 --2- 172.21.15.97:0/2877681173 >> v2:172.21.15.194:3300/0 conn(0x7fea3815e750 0x7fea380cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 --2- 172.21.15.97:0/2877681173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815f150 0x7fea380b4380 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 -- 172.21.15.97:0/2877681173 >> 172.21.15.97:0/2877681173 conn(0x7fea380b28c0 msgr2=0x7fea380b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 -- 172.21.15.97:0/2877681173 shutdown_connections 2024-09-10T21:32:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.547+0000 7fea3ca00640 1 -- 172.21.15.97:0/2877681173 wait complete. 2024-09-10T21:32:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 Processor -- start 2024-09-10T21:32:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 -- start start 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815e750 0x7fea3814d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea3815f150 0x7fea38150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea381564d0 0x7fea38152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea381683a0 con 0x7fea3815e750 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea38168220 con 0x7fea3815f150 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fea38168520 con 0x7fea381564d0 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815e750 0x7fea3814d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815e750 0x7fea3814d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53686/0 (socket says 172.21.15.97:53686) 2024-09-10T21:32:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 -- 172.21.15.97:0/688851864 learned_addr learned my addr 172.21.15.97:0/688851864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 -- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea381564d0 msgr2=0x7fea38152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea381564d0 0x7fea38152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 -- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea3815f150 msgr2=0x7fea38150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea3815f150 0x7fea38150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 -- 172.21.15.97:0/688851864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea380b4380 con 0x7fea3815e750 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea35e00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815e750 0x7fea3814d250 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fea28000f90 tx=0x7fea28009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea2ea00640 1 -- 172.21.15.97:0/688851864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea280bb070 con 0x7fea3815e750 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea2ea00640 1 -- 172.21.15.97:0/688851864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea28004d40 con 0x7fea3815e750 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea2ea00640 1 -- 172.21.15.97:0/688851864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea280b2040 con 0x7fea3815e750 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3814fa70 con 0x7fea3815e750 2024-09-10T21:32:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.551+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3814fd10 con 0x7fea3815e750 2024-09-10T21:32:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.555+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea380cd580 con 0x7fea3815e750 2024-09-10T21:32:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.555+0000 7fea2ea00640 1 -- 172.21.15.97:0/688851864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fea280b5030 con 0x7fea3815e750 2024-09-10T21:32:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.555+0000 7fea2ea00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea140725e0 0x7fea14074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.555+0000 7fea2ea00640 1 -- 172.21.15.97:0/688851864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7fea2812d800 con 0x7fea3815e750 2024-09-10T21:32:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.555+0000 7fea2d400640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea140725e0 0x7fea14074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.555+0000 7fea2d400640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea140725e0 0x7fea14074aa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fea200046c0 tx=0x7fea20007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:12.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.559+0000 7fea2ea00640 1 -- 172.21.15.97:0/688851864 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea28016770 con 0x7fea3815e750 2024-09-10T21:32:12.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:12.763+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fea380ce7b0 con 0x7fea3815e750 2024-09-10T21:32:13.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.003+0000 7fea2ea00640 1 -- 172.21.15.97:0/688851864 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v158) ==== 162+0+0 (secure 0 0 0) 0x7fea280f8810 con 0x7fea3815e750 2024-09-10T21:32:13.007 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea140725e0 msgr2=0x7fea14074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea140725e0 0x7fea14074aa0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fea200046c0 tx=0x7fea20007910 comp rx=0 tx=0).stop 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815e750 msgr2=0x7fea3814d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815e750 0x7fea3814d250 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fea28000f90 tx=0x7fea28009220 comp rx=0 tx=0).stop 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 shutdown_connections 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea140725e0 0x7fea14074aa0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea381564d0 0x7fea38152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea3815f150 0x7fea38150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 --2- 172.21.15.97:0/688851864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea3815e750 0x7fea3814d250 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 >> 172.21.15.97:0/688851864 conn(0x7fea380b28c0 msgr2=0x7fea380b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 shutdown_connections 2024-09-10T21:32:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.007+0000 7fea3ca00640 1 -- 172.21.15.97:0/688851864 wait complete. 2024-09-10T21:32:13.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:13.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.575+0000 7f500f600640 1 -- 172.21.15.97:0/2238401263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 msgr2=0x7f4fec003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:13.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.575+0000 7f500f600640 1 --2- 172.21.15.97:0/2238401263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f4fec003e80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800c2e0 tx=0x7f4ff801cf60 comp rx=0 tx=0).stop 2024-09-10T21:32:13.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.575+0000 7f500f600640 1 -- 172.21.15.97:0/2238401263 shutdown_connections 2024-09-10T21:32:13.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.575+0000 7f500f600640 1 --2- 172.21.15.97:0/2238401263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f4fec003e80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.575+0000 7f500f600640 1 -- 172.21.15.97:0/2238401263 >> 172.21.15.97:0/2238401263 conn(0x7f50080d1c30 msgr2=0x7f50080d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:13.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.575+0000 7f500f600640 1 -- 172.21.15.97:0/2238401263 shutdown_connections 2024-09-10T21:32:13.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.575+0000 7f500f600640 1 -- 172.21.15.97:0/2238401263 wait complete. 2024-09-10T21:32:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 Processor -- start 2024-09-10T21:32:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 -- start start 2024-09-10T21:32:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f500815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5008152470 0x7f50080c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f500806edd0 0x7f500806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f50080ccf90 con 0x7f4fec003a60 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f50080cce10 con 0x7f5008152470 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f50080cd110 con 0x7f500806edd0 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f500815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f500815e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53704/0 (socket says 172.21.15.97:53704) 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 -- 172.21.15.97:0/2714566106 learned_addr learned my addr 172.21.15.97:0/2714566106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f4fffe00640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5008152470 0x7f50080c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500d600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f500806edd0 0x7f500806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 -- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f500806edd0 msgr2=0x7f500806f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f500806edd0 0x7f500806f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 -- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5008152470 msgr2=0x7f50080c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5008152470 0x7f50080c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 -- 172.21.15.97:0/2714566106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500815d440 con 0x7f4fec003a60 2024-09-10T21:32:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500cc00640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f500815e780 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f4ff801ca20 tx=0x7f4ff80aea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:13.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f5005600640 1 -- 172.21.15.97:0/2714566106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8006ca0 con 0x7f4fec003a60 2024-09-10T21:32:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f5005600640 1 -- 172.21.15.97:0/2714566106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ff80aec70 con 0x7f4fec003a60 2024-09-10T21:32:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500815d6e0 con 0x7f4fec003a60 2024-09-10T21:32:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f5005600640 1 -- 172.21.15.97:0/2714566106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8001d20 con 0x7f4fec003a60 2024-09-10T21:32:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500815dca0 con 0x7f4fec003a60 2024-09-10T21:32:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.579+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50080cd670 con 0x7f4fec003a60 2024-09-10T21:32:13.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.583+0000 7f5005600640 1 -- 172.21.15.97:0/2714566106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4ff8003600 con 0x7f4fec003a60 2024-09-10T21:32:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.583+0000 7f5005600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe4072610 0x7f4fe4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.583+0000 7f5005600640 1 -- 172.21.15.97:0/2714566106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f4ff812dd00 con 0x7f4fec003a60 2024-09-10T21:32:13.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.583+0000 7f4fffe00640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe4072610 0x7f4fe4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:13.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.583+0000 7f4fffe00640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe4072610 0x7f4fe4074ad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f50080b57c0 tx=0x7f4ff00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:13.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.587+0000 7f5005600640 1 -- 172.21.15.97:0/2714566106 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ff8016d10 con 0x7f4fec003a60 2024-09-10T21:32:13.751 INFO:tasks.ceph.mds.a.smithi097.stderr: -50> 2024-09-10T21:32:08.995+0000 7fb6d7800640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-10T21:32:13.751 INFO:tasks.ceph.mds.a.smithi097.stderr: -49> 2024-09-10T21:32:08.995+0000 7fb6d7800640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-10T21:32:13.788 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:32:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.787+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50080d2010 con 0x7f4fec003a60 2024-09-10T21:32:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.787+0000 7f5005600640 1 -- 172.21.15.97:0/2714566106 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 159 v159) ==== 95+0+4404 (secure 0 0 0) 0x7f4ff80aee60 con 0x7f4fec003a60 2024-09-10T21:32:13.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:13.792 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":159,"btime":"2024-09-10T21:32:13:012031+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:13.012022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8280":{"gid":8280,"name":"c","rank":0,"incarnation":159,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:13.792 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 159 2024-09-10T21:32:13.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe4072610 msgr2=0x7f4fe4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:13.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe4072610 0x7f4fe4074ad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f50080b57c0 tx=0x7f4ff00091e0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 msgr2=0x7f500815e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f500815e780 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f4ff801ca20 tx=0x7f4ff80aea80 comp rx=0 tx=0).stop 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 shutdown_connections 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fe4072610 0x7f4fe4074ad0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f500806edd0 0x7f500806f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5008152470 0x7f50080c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 --2- 172.21.15.97:0/2714566106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fec003a60 0x7f500815e780 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 >> 172.21.15.97:0/2714566106 conn(0x7f50080d1c30 msgr2=0x7f50080b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 shutdown_connections 2024-09-10T21:32:13.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:13.791+0000 7f500f600640 1 -- 172.21.15.97:0/2714566106 wait complete. 2024-09-10T21:32:13.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 159, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:32:13.012022+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8280}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8280': {'gid': 8280, 'name': 'c', 'rank': 0, 'incarnation': 159, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/2243090003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2243090003}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2243090003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:32:13.813 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8280:up:replay 2024-09-10T21:32:13.813 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:32:19.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:19.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 -- 172.21.15.97:0/1049032653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce5f0 msgr2=0x7f4df8141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 --2- 172.21.15.97:0/1049032653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce5f0 0x7f4df8141d20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009900 tx=0x7f4de801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 -- 172.21.15.97:0/1049032653 shutdown_connections 2024-09-10T21:32:19.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 --2- 172.21.15.97:0/1049032653 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df80ce9d0 0x7f4df815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 --2- 172.21.15.97:0/1049032653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce5f0 0x7f4df8141d20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 -- 172.21.15.97:0/1049032653 >> 172.21.15.97:0/1049032653 conn(0x7f4df80b1780 msgr2=0x7f4df80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 -- 172.21.15.97:0/1049032653 shutdown_connections 2024-09-10T21:32:19.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 -- 172.21.15.97:0/1049032653 wait complete. 2024-09-10T21:32:19.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 Processor -- start 2024-09-10T21:32:19.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.367+0000 7f4dfde00640 1 -- start start 2024-09-10T21:32:19.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df80ce5f0 0x7f4df813a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce9d0 0x7f4df813a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df815fc10 0x7f4df806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4df81690c0 con 0x7f4df80ce9d0 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8168f40 con 0x7f4df80ce5f0 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4df8169240 con 0x7f4df815fc10 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce9d0 0x7f4df813a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce9d0 0x7f4df813a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41180/0 (socket says 172.21.15.97:41180) 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 -- 172.21.15.97:0/444791688 learned_addr learned my addr 172.21.15.97:0/444791688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6a00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df80ce5f0 0x7f4df813a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df7400640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df815fc10 0x7f4df806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 -- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df815fc10 msgr2=0x7f4df806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df815fc10 0x7f4df806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 -- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df80ce5f0 msgr2=0x7f4df813a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df80ce5f0 0x7f4df813a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 -- 172.21.15.97:0/444791688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df80b4660 con 0x7f4df80ce9d0 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4df6000640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce9d0 0x7f4df813a1a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d950 tx=0x7f4de400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4def400640 1 -- 172.21.15.97:0/444791688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4017070 con 0x7f4df80ce9d0 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4def400640 1 -- 172.21.15.97:0/444791688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4de4009d20 con 0x7f4df80ce9d0 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df80b4900 con 0x7f4df80ce9d0 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4def400640 1 -- 172.21.15.97:0/444791688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40106c0 con 0x7f4df80ce9d0 2024-09-10T21:32:19.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df815e6e0 con 0x7f4df80ce9d0 2024-09-10T21:32:19.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.371+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df8141d20 con 0x7f4df80ce9d0 2024-09-10T21:32:19.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.375+0000 7f4def400640 1 -- 172.21.15.97:0/444791688 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4de4010860 con 0x7f4df80ce9d0 2024-09-10T21:32:19.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.375+0000 7f4def400640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4dd80725e0 0x7f4dd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.375+0000 7f4def400640 1 -- 172.21.15.97:0/444791688 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f4de4093bd0 con 0x7f4df80ce9d0 2024-09-10T21:32:19.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.375+0000 7f4df6a00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4dd80725e0 0x7f4dd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.375+0000 7f4df6a00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4dd80725e0 0x7f4dd8074aa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4de80af000 tx=0x7f4de801b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.379+0000 7f4def400640 1 -- 172.21.15.97:0/444791688 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4de405e610 con 0x7f4df80ce9d0 2024-09-10T21:32:19.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4df8141360 con 0x7f4df80ce9d0 2024-09-10T21:32:19.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4def400640 1 -- 172.21.15.97:0/444791688 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) ==== 95+0+5237 (secure 0 0 0) 0x7f4de4063040 con 0x7f4df80ce9d0 2024-09-10T21:32:19.582 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:19.582 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":162,"btime":"2024-09-10T21:32:16:023830+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2415223919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2415223919},{"type":"v1","addr":"172.21.15.97:6834","nonce":2415223919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:16.023822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8280":{"gid":8280,"name":"c","rank":0,"incarnation":159,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6835/2243090003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2243090003},{"type":"v1","addr":"172.21.15.97:6835","nonce":2243090003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8280,"qdb_cluster":[8280]},"id":6}]} 2024-09-10T21:32:19.582 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 162 2024-09-10T21:32:19.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4dd80725e0 msgr2=0x7f4dd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4dd80725e0 0x7f4dd8074aa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4de80af000 tx=0x7f4de801b120 comp rx=0 tx=0).stop 2024-09-10T21:32:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce9d0 msgr2=0x7f4df813a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce9d0 0x7f4df813a1a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4de400d950 tx=0x7f4de400de10 comp rx=0 tx=0).stop 2024-09-10T21:32:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 shutdown_connections 2024-09-10T21:32:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4dd80725e0 0x7f4dd8074aa0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df815fc10 0x7f4df806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df80ce9d0 0x7f4df813a1a0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 --2- 172.21.15.97:0/444791688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df80ce5f0 0x7f4df813a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 >> 172.21.15.97:0/444791688 conn(0x7f4df80b1780 msgr2=0x7f4df80b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 shutdown_connections 2024-09-10T21:32:19.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.579+0000 7f4dfde00640 1 -- 172.21.15.97:0/444791688 wait complete. 2024-09-10T21:32:19.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:32:16.023822+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8280}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8280': {'gid': 8280, 'name': 'c', 'rank': 0, 'incarnation': 159, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.97:6835/2243090003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2243090003}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2243090003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8280, 'qdb_cluster': [8280]} 2024-09-10T21:32:19.604 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:32:19.604 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 -- 172.21.15.97:0/2663558840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf0b0 msgr2=0x7f5d5806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 --2- 172.21.15.97:0/2663558840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf0b0 0x7f5d5806e3b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800b6c0 tx=0x7f5d4801d410 comp rx=0 tx=0).stop 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 -- 172.21.15.97:0/2663558840 shutdown_connections 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 --2- 172.21.15.97:0/2663558840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d5815b110 0x7f5d580b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 --2- 172.21.15.97:0/2663558840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf0b0 0x7f5d5806e3b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 -- 172.21.15.97:0/2663558840 >> 172.21.15.97:0/2663558840 conn(0x7f5d580b1780 msgr2=0x7f5d580b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 -- 172.21.15.97:0/2663558840 shutdown_connections 2024-09-10T21:32:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 -- 172.21.15.97:0/2663558840 wait complete. 2024-09-10T21:32:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 Processor -- start 2024-09-10T21:32:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.743+0000 7f5d5f400640 1 -- start start 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d580cf0b0 0x7f5d58149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf490 0x7f5d5815d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d5815b110 0x7f5d5815de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580b3b60 con 0x7f5d580cf490 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5d580b39e0 con 0x7f5d580cf0b0 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5d580709c0 con 0x7f5d5815b110 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf490 0x7f5d5815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf490 0x7f5d5815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41196/0 (socket says 172.21.15.97:41196) 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 -- 172.21.15.97:0/321154311 learned_addr learned my addr 172.21.15.97:0/321154311 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5d400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d5815b110 0x7f5d5815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d580cf0b0 0x7f5d58149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 -- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d5815b110 msgr2=0x7f5d5815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d5815b110 0x7f5d5815de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 -- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d580cf0b0 msgr2=0x7f5d58149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d580cf0b0 0x7f5d58149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 -- 172.21.15.97:0/321154311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d58154420 con 0x7f5d580cf490 2024-09-10T21:32:19.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d57e00640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf490 0x7f5d5815d950 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c450 tx=0x7f5d4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d54c00640 1 -- 172.21.15.97:0/321154311 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44005480 con 0x7f5d580cf490 2024-09-10T21:32:19.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d54c00640 1 -- 172.21.15.97:0/321154311 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d44003600 con 0x7f5d580cf490 2024-09-10T21:32:19.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d54c00640 1 -- 172.21.15.97:0/321154311 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003a30 con 0x7f5d580cf490 2024-09-10T21:32:19.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d581f8570 con 0x7f5d580cf490 2024-09-10T21:32:19.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.747+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d581f8b30 con 0x7f5d580cf490 2024-09-10T21:32:19.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d54c00640 1 -- 172.21.15.97:0/321154311 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5d44003bd0 con 0x7f5d580cf490 2024-09-10T21:32:19.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 Processor -- start 2024-09-10T21:32:19.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d54c00640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d380725e0 0x7f5d38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 -- start start 2024-09-10T21:32:19.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d54c00640 1 -- 172.21.15.97:0/321154311 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f5d44093490 con 0x7f5d580cf490 2024-09-10T21:32:19.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5d400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d380725e0 0x7f5d38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068a9d0 0x55eb8068adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 0x55eb8063e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8063ee10 0x55eb8063f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eb8055f3f0 con 0x55eb8068a9d0 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eb8049cb20 con 0x55eb8063ee10 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55eb8048bf50 con 0x55eb8068b2f0 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068a9d0 0x55eb8068adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068a9d0 0x55eb8068adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41198/0 (socket says 172.21.15.97:41198) 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 -- 172.21.15.97:0/2636309965 learned_addr learned my addr 172.21.15.97:0/2636309965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5d400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d380725e0 0x7f5d38074aa0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5d580b2e70 tx=0x7f5d50005d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5d400640 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8063ee10 0x55eb8063f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d57e00640 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 0x55eb8063e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 -- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 msgr2=0x55eb8063e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 0x55eb8063e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 -- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8063ee10 msgr2=0x55eb8063f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8063ee10 0x55eb8063f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 -- 172.21.15.97:0/2636309965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb80453590 con 0x55eb8068a9d0 2024-09-10T21:32:19.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068a9d0 0x55eb8068adb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f5d4801bac0 tx=0x7f5d4801a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c53294b5e2d49af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d4d600640 1 -- 172.21.15.97:0/2636309965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d480ba070 con 0x55eb8068a9d0 2024-09-10T21:32:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d4d600640 1 -- 172.21.15.97:0/2636309965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d480b0040 con 0x55eb8068a9d0 2024-09-10T21:32:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d4d600640 1 -- 172.21.15.97:0/2636309965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d480af030 con 0x55eb8068a9d0 2024-09-10T21:32:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 -- 172.21.15.97:0/2636309965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068a9d0 msgr2=0x55eb8068adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.751+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068a9d0 0x55eb8068adb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f5d4801bac0 tx=0x7f5d4801a1b0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- 172.21.15.97:0/2636309965 shutdown_connections 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8063ee10 0x55eb8063f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 0x55eb8063e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2636309965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068a9d0 0x55eb8068adb0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- 172.21.15.97:0/2636309965 >> 172.21.15.97:0/2636309965 conn(0x55eb80528130 msgr2=0x55eb8066de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- 172.21.15.97:0/2636309965 shutdown_connections 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- 172.21.15.97:0/2636309965 wait complete. 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- start start 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 Processor -- start 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8063ee10 0x55eb80670280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb806707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 0x55eb80670d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eb804265d0 con 0x55eb8063ee10 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eb8048fd60 con 0x55eb8068a9d0 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55eb80432500 con 0x55eb8068b2f0 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 10 client.0 _ll_register_callbacks cb 0x55eb80436810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8063ee10 0x55eb80670280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8063ee10 0x55eb80670280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41210/0 (socket says 172.21.15.97:41210) 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 -- 172.21.15.97:0/4147493711 learned_addr learned my addr 172.21.15.97:0/4147493711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 -- 172.21.15.97:0/4147493711 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 msgr2=0x55eb80670d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 0x55eb80670d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 -- 172.21.15.97:0/4147493711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 msgr2=0x55eb806707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb806707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 -- 172.21.15.97:0/4147493711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8042a170 con 0x55eb8063ee10 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8063ee10 0x55eb80670280 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f5d4801a020 tx=0x7f5d480095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b0af03b6a2b3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d42000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:32:19.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d42000640 1 -- 172.21.15.97:0/4147493711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d480c5060 con 0x55eb8063ee10 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d42000640 1 -- 172.21.15.97:0/4147493711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f5d480099d0 con 0x55eb8063ee10 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb804240f0 con 0x55eb8063ee10 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d42000640 1 -- 172.21.15.97:0/4147493711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=162) ==== 24+0+0 (secure 0 0 0) 0x7f5d480cb050 con 0x55eb8063ee10 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 10 client.8439 fetch_fsmap learned FSMap version 162 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=162,monmap=2+}) -- 0x55eb80510ef0 con 0x55eb8063ee10 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d42000640 1 -- 172.21.15.97:0/4147493711 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55eb804240f0 con 0x55eb8063ee10 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d42000640 1 -- 172.21.15.97:0/4147493711 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 162) ==== 3502+0+0 (secure 0 0 0) 0x7f5d480ce020 con 0x55eb8063ee10 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 10 client.8439 fetch_fsmap finished waiting for FSMap version 162 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 10 client.8439 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 20 client.8439 populate_metadata read hostname 'smithi097' 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80671710 0x55eb806a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 4 client.8439 mds_command: new command op to 8280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:32:19.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 --> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb8049ea70 con 0x55eb80671710 2024-09-10T21:32:19.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.755+0000 7f5d57e00640 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80671710 0x55eb806a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d42000640 10 client.8439 ms_handle_connect on v2:172.21.15.97:6833/2243090003 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d57e00640 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80671710 0x55eb806a0cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db6ee8112ef5132 server_cookie=f59117e63b2faa28 in_seq=0 out_seq=0 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d42000640 1 -- 172.21.15.97:0/4147493711 <== mds.0 v2:172.21.15.97:6833/2243090003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eb8049ea70 con 0x55eb80671710 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d42000640 10 client.8439 handle_command_reply: tid=0 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 client.8439 shutdown 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8063ee10 msgr2=0x55eb80670280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8063ee10 0x55eb80670280 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f5d4801a020 tx=0x7f5d480095e0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 shutdown_connections 2024-09-10T21:32:19.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8068b2f0 0x55eb80670d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb806707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80671710 0x55eb806a0cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 --2- 172.21.15.97:0/4147493711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8063ee10 0x55eb80670280 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 >> 172.21.15.97:0/4147493711 conn(0x55eb80528130 msgr2=0x55eb8063dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d42000640 0 client.8439 ms_handle_reset on v2:172.21.15.97:6833/2243090003 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 shutdown_connections 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 1 -- 172.21.15.97:0/4147493711 wait complete. 2024-09-10T21:32:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.759+0000 7f5d6287d000 20 client.8439 trim_cache size 0 max 16384 2024-09-10T21:32:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.775+0000 7f5d6287d000 1 Processor -- start 2024-09-10T21:32:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.775+0000 7f5d6287d000 1 -- start start 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb807019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb80701f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb80702470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58200a40 con 0x55eb8068b2f0 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5d581ffaa0 con 0x55eb8068a9d0 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5d58162690 con 0x55eb8063ee10 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb80702470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb80702470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41216/0 (socket says 172.21.15.97:41216) 2024-09-10T21:32:19.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 -- 172.21.15.97:0/1674194871 learned_addr learned my addr 172.21.15.97:0/1674194871 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb807019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d57e00640 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb80701f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 -- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 msgr2=0x55eb807019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb807019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 -- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 msgr2=0x55eb80701f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb80701f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 -- 172.21.15.97:0/1674194871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb804e7d90 con 0x55eb8068b2f0 2024-09-10T21:32:19.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d5d400640 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb80702470 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000ede0 tx=0x7f5d50004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b8d21fdcf52d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d4cc00640 1 -- 172.21.15.97:0/1674194871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50022070 con 0x55eb8068b2f0 2024-09-10T21:32:19.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d4cc00640 1 -- 172.21.15.97:0/1674194871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d50027770 con 0x55eb8068b2f0 2024-09-10T21:32:19.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d4cc00640 1 -- 172.21.15.97:0/1674194871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50027bc0 con 0x55eb8068b2f0 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- 172.21.15.97:0/1674194871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 msgr2=0x55eb80702470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb80702470 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000ede0 tx=0x7f5d50004740 comp rx=0 tx=0).stop 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- 172.21.15.97:0/1674194871 shutdown_connections 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb80702470 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb80701f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 --2- 172.21.15.97:0/1674194871 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb807019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- 172.21.15.97:0/1674194871 >> 172.21.15.97:0/1674194871 conn(0x55eb80528130 msgr2=0x55eb8069c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- 172.21.15.97:0/1674194871 shutdown_connections 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- 172.21.15.97:0/1674194871 wait complete. 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 -- start start 2024-09-10T21:32:19.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.779+0000 7f5d6287d000 1 Processor -- start 2024-09-10T21:32:19.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb8066ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb8066b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb806a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eb8048f070 con 0x55eb8068b2f0 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55eb80494390 con 0x55eb8068a9d0 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55eb80510a60 con 0x55eb8063ee10 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 10 client.0 _ll_register_callbacks cb 0x7f5d50014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb806a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb806a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41224/0 (socket says 172.21.15.97:41224) 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 -- 172.21.15.97:0/2975542953 learned_addr learned my addr 172.21.15.97:0/2975542953 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d57e00640 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb8066b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb8066ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 -- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 msgr2=0x55eb8066ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb8066ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 -- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 msgr2=0x55eb8066b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb8066b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 -- 172.21.15.97:0/2975542953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb80452a90 con 0x55eb8068b2f0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5d400640 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb806a1d20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000ede0 tx=0x7f5d50027bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c76767838e7f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 1 -- 172.21.15.97:0/2975542953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50034060 con 0x55eb8068b2f0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 1 -- 172.21.15.97:0/2975542953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f5d5001e840 con 0x55eb8068b2f0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d50027770 con 0x55eb8068b2f0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 1 -- 172.21.15.97:0/2975542953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=162) ==== 24+0+0 (secure 0 0 0) 0x7f5d5003a050 con 0x55eb8068b2f0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 10 client.8445 fetch_fsmap learned FSMap version 162 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=162,monmap=2+}) -- 0x55eb80431050 con 0x55eb8068b2f0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 1 -- 172.21.15.97:0/2975542953 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d50027770 con 0x55eb8068b2f0 2024-09-10T21:32:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 1 -- 172.21.15.97:0/2975542953 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 162) ==== 3502+0+0 (secure 0 0 0) 0x7f5d5003d020 con 0x55eb8068b2f0 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 10 client.8445 fetch_fsmap finished waiting for FSMap version 162 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 10 client.8445 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 20 client.8445 populate_metadata read hostname 'smithi097' 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80705470 0x55eb80705850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 4 client.8445 mds_command: new command op to 8280 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 --> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] -- command(tid 0: {"prefix": "status"}) -- 0x55eb8049a520 con 0x55eb80705470 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80705470 0x55eb80705850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d5ca00640 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80705470 0x55eb80705850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6a7072324f2b7d8 server_cookie=f7510a02758ffc0b in_seq=0 out_seq=0 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 10 client.8445 ms_handle_connect on v2:172.21.15.97:6833/2243090003 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 1 -- 172.21.15.97:0/2975542953 <== mds.0 v2:172.21.15.97:6833/2243090003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55eb8049a520 con 0x55eb80705470 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 10 client.8445 handle_command_reply: tid=0 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 client.8445 shutdown 2024-09-10T21:32:19.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 msgr2=0x55eb806a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb806a1d20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000ede0 tx=0x7f5d50027bf0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 shutdown_connections 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:6833/2243090003,v1:172.21.15.97:6835/2243090003] conn(0x55eb80705470 0x55eb80705850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb8068b2f0 0x55eb806a1d20 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55eb8068a9d0 0x55eb8066b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d42000640 0 client.8445 ms_handle_reset on v2:172.21.15.97:6833/2243090003 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 --2- 172.21.15.97:0/2975542953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55eb8063ee10 0x55eb8066ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 >> 172.21.15.97:0/2975542953 conn(0x55eb80528130 msgr2=0x55eb806462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 shutdown_connections 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 1 -- 172.21.15.97:0/2975542953 wait complete. 2024-09-10T21:32:19.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.783+0000 7f5d6287d000 20 client.8445 trim_cache size 0 max 16384 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8280, 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7767182190000002, 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 162, 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 108, 2024-09-10T21:32:19.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 108, 2024-09-10T21:32:19.794 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 19.078199049999998 2024-09-10T21:32:19.794 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:32:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d380725e0 msgr2=0x7f5d38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d380725e0 0x7f5d38074aa0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5d580b2e70 tx=0x7f5d50005d10 comp rx=0 tx=0).stop 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf490 msgr2=0x7f5d5815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf490 0x7f5d5815d950 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c450 tx=0x7f5d4400c910 comp rx=0 tx=0).stop 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 shutdown_connections 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5d380725e0 0x7f5d38074aa0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5d5815b110 0x7f5d5815de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d580cf490 0x7f5d5815d950 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 --2- 172.21.15.97:0/321154311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5d580cf0b0 0x7f5d58149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 >> 172.21.15.97:0/321154311 conn(0x7f5d580b1780 msgr2=0x7f5d5814d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 shutdown_connections 2024-09-10T21:32:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.787+0000 7f5d5f400640 1 -- 172.21.15.97:0/321154311 wait complete. 2024-09-10T21:32:19.808 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:32:19.808 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2024-09-10T21:32:19.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 -- 172.21.15.97:0/1737327803 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 msgr2=0x7fe0440af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 --2- 172.21.15.97:0/1737327803 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 0x7fe0440af4d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009c80 tx=0x7fe03c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 -- 172.21.15.97:0/1737327803 shutdown_connections 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 --2- 172.21.15.97:0/1737327803 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 0x7fe0440af4d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 --2- 172.21.15.97:0/1737327803 >> v2:172.21.15.194:3300/0 conn(0x7fe0440ce640 0x7fe0440cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 -- 172.21.15.97:0/1737327803 >> 172.21.15.97:0/1737327803 conn(0x7fe0440d1c30 msgr2=0x7fe0440d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 -- 172.21.15.97:0/1737327803 shutdown_connections 2024-09-10T21:32:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 -- 172.21.15.97:0/1737327803 wait complete. 2024-09-10T21:32:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 Processor -- start 2024-09-10T21:32:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.947+0000 7fe04c400640 1 -- start start 2024-09-10T21:32:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0440b0d50 0x7fe044157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 0x7fe0440b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0440ce640 0x7fe0440b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe044168c70 con 0x7fe0440b0d50 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe044168af0 con 0x7fe0440ce640 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe044168df0 con 0x7fe0440cd6e0 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0440b0d50 0x7fe044157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0440b0d50 0x7fe044157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41234/0 (socket says 172.21.15.97:41234) 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 -- 172.21.15.97:0/2265524298 learned_addr learned my addr 172.21.15.97:0/2265524298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049000640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 0x7fe0440b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04a400640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0440ce640 0x7fe0440b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 -- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 msgr2=0x7fe0440b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 0x7fe0440b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 -- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0440ce640 msgr2=0x7fe0440b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0440ce640 0x7fe0440b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 -- 172.21.15.97:0/2265524298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0440b1820 con 0x7fe0440b0d50 2024-09-10T21:32:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe049a00640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0440b0d50 0x7fe044157a80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe03800e7a0 tx=0x7fe03800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe041600640 1 -- 172.21.15.97:0/2265524298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03800ccc0 con 0x7fe0440b0d50 2024-09-10T21:32:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe041600640 1 -- 172.21.15.97:0/2265524298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe038010430 con 0x7fe0440b0d50 2024-09-10T21:32:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe041600640 1 -- 172.21.15.97:0/2265524298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038010860 con 0x7fe0440b0d50 2024-09-10T21:32:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0440b1b00 con 0x7fe0440b0d50 2024-09-10T21:32:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0440b6ee0 con 0x7fe0440b0d50 2024-09-10T21:32:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.951+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0440cea20 con 0x7fe0440b0d50 2024-09-10T21:32:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.955+0000 7fe041600640 1 -- 172.21.15.97:0/2265524298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe038010a00 con 0x7fe0440b0d50 2024-09-10T21:32:19.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.955+0000 7fe041600640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0240725e0 0x7fe024074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.955+0000 7fe041600640 1 -- 172.21.15.97:0/2265524298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7fe038094250 con 0x7fe0440b0d50 2024-09-10T21:32:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.955+0000 7fe049000640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0240725e0 0x7fe024074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.955+0000 7fe049000640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0240725e0 0x7fe024074aa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe044071900 tx=0x7fe030009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:19.959+0000 7fe041600640 1 -- 172.21.15.97:0/2265524298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe03805ec90 con 0x7fe0440b0d50 2024-09-10T21:32:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.159+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe0440b42f0 con 0x7fe0440b0d50 2024-09-10T21:32:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.159+0000 7fe041600640 1 -- 172.21.15.97:0/2265524298 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v162) ==== 53+0+73 (secure 0 0 0) 0x7fe0380636c0 con 0x7fe0440b0d50 2024-09-10T21:32:20.164 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0240725e0 msgr2=0x7fe024074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0240725e0 0x7fe024074aa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe044071900 tx=0x7fe030009040 comp rx=0 tx=0).stop 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0440b0d50 msgr2=0x7fe044157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0440b0d50 0x7fe044157a80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe03800e7a0 tx=0x7fe03800ec60 comp rx=0 tx=0).stop 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 shutdown_connections 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0240725e0 0x7fe024074aa0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0440ce640 0x7fe0440b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0440cd6e0 0x7fe0440b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 --2- 172.21.15.97:0/2265524298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0440b0d50 0x7fe044157a80 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 >> 172.21.15.97:0/2265524298 conn(0x7fe0440d1c30 msgr2=0x7fe0440d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 shutdown_connections 2024-09-10T21:32:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20.163+0000 7fe04c400640 1 -- 172.21.15.97:0/2265524298 wait complete. 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:32:20.187 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:32:20.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.187 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:32:20.197 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41953sec preferred_lft 41953sec 2024-09-10T21:32:20.198 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:32:20.199 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.200 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:32:20.200 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:32:20.200 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:32:20.200 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:32:20.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.200 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:32:20.258 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:32:20.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.261 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:32:20.320 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:32:20.320 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:32:20.320 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:32:20.320 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:32:20.320 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:32:20.320 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:32:20.321 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:32:20.321 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:32:20.321 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:32:20.321 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:32:20.321 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:32:20.321 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:32:20.321 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:32:20.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:32:20.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:32:20.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:32:20.322 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:32:20.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:32:20.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:32:20.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.456 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:32:20.465 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:32:20.465 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:32:20.466 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:32:20.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:32:20.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:32:20.532 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:32:20.580 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:32:20.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:32:20.581 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:32:20.632 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:32:20.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:32:20.720 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:32:20.721 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:32:20.721 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:32:20.736 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:32:20.736 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:32:20.736 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:32:20.736 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:32:20.736 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:32:20.745 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:32:20.749 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:32:20.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:32:20.768 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:32:20.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:32:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:32:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:client7807 2024-09-10T21:32:20.868 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:20.868 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:20.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:20.877 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:32:20.923 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:32:20.931 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:32:21.417 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:32:21.417 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:21.417 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:32:21.417 INFO:teuthology.orchestra.run.smithi097.stdout:sixmegs 2024-09-10T21:32:21.419 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:32:20.951+0000 7f2f1d400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:32:27.418 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2024-09-10T21:32:27.419 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2024-09-10T21:32:27.419 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:27.419 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:27.429 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:27.429 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:27.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:32:27.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b79000640 1 -- 172.21.15.97:0/4146763305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 msgr2=0x7f0b58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:27.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b79000640 1 --2- 172.21.15.97:0/4146763305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 0x7f0b58004810 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c640 tx=0x7f0b5c01fa00 comp rx=0 tx=0).stop 2024-09-10T21:32:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b72800640 1 -- 172.21.15.97:0/4146763305 reap_dead start 2024-09-10T21:32:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b79000640 1 -- 172.21.15.97:0/4146763305 shutdown_connections 2024-09-10T21:32:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b79000640 1 -- 172.21.15.97:0/4146763305 >> 172.21.15.97:0/4146763305 conn(0x7f0b740b1780 msgr2=0x7f0b740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b79000640 1 -- 172.21.15.97:0/4146763305 shutdown_connections 2024-09-10T21:32:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b79000640 1 -- 172.21.15.97:0/4146763305 wait complete. 2024-09-10T21:32:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.679+0000 7f0b79000640 1 Processor -- start 2024-09-10T21:32:27.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 -- start start 2024-09-10T21:32:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 0x7f0b74151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b7413fb20 0x7f0b740c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b74159b30 0x7f0b740c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b740b49b0 con 0x7f0b58004080 2024-09-10T21:32:27.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0b740b4830 con 0x7f0b7413fb20 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0b740701a0 con 0x7f0b74159b30 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 0x7f0b74151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 0x7f0b74151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38962/0 (socket says 172.21.15.97:38962) 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 -- 172.21.15.97:0/1747621416 learned_addr learned my addr 172.21.15.97:0/1747621416 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b69400640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b7413fb20 0x7f0b740c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b72800640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b74159b30 0x7f0b740c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 -- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b74159b30 msgr2=0x7f0b740c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b74159b30 0x7f0b740c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 -- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b7413fb20 msgr2=0x7f0b740c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b7413fb20 0x7f0b740c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 -- 172.21.15.97:0/1747621416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b74140020 con 0x7f0b58004080 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b69400640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b7413fb20 0x7f0b740c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:27.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b71e00640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 0x7f0b74151300 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0b64011b40 tx=0x7f0b64011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:27.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b6aa00640 1 -- 172.21.15.97:0/1747621416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6401e070 con 0x7f0b58004080 2024-09-10T21:32:27.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b6aa00640 1 -- 172.21.15.97:0/1747621416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b64019430 con 0x7f0b58004080 2024-09-10T21:32:27.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b740c66e0 con 0x7f0b58004080 2024-09-10T21:32:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b6aa00640 1 -- 172.21.15.97:0/1747621416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64019840 con 0x7f0b58004080 2024-09-10T21:32:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b74151840 con 0x7f0b58004080 2024-09-10T21:32:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.683+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b740c6510 con 0x7f0b58004080 2024-09-10T21:32:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.687+0000 7f0b6aa00640 1 -- 172.21.15.97:0/1747621416 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0b640199e0 con 0x7f0b58004080 2024-09-10T21:32:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.687+0000 7f0b6aa00640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b500725e0 0x7f0b50074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:27.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.687+0000 7f0b6aa00640 1 -- 172.21.15.97:0/1747621416 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7f0b64094710 con 0x7f0b58004080 2024-09-10T21:32:27.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.687+0000 7f0b69400640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b500725e0 0x7f0b50074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:27.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.687+0000 7f0b69400640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b500725e0 0x7f0b50074aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0af000 tx=0x7f0b5c01c840 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:27.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.691+0000 7f0b6aa00640 1 -- 172.21.15.97:0/1747621416 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b6405f7a0 con 0x7f0b58004080 2024-09-10T21:32:27.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.867+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0b740b2a80 con 0x7f0b58004080 2024-09-10T21:32:27.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.867+0000 7f0b6aa00640 1 -- 172.21.15.97:0/1747621416 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 24 entries v108) ==== 81+0+1456 (secure 0 0 0) 0x7f0b6405f0a0 con 0x7f0b58004080 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:32:27.873 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:32:27.874 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:32:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:listed 24 entries 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b500725e0 msgr2=0x7f0b50074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b500725e0 0x7f0b50074aa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0af000 tx=0x7f0b5c01c840 comp rx=0 tx=0).stop 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 msgr2=0x7f0b74151300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 0x7f0b74151300 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0b64011b40 tx=0x7f0b64011f40 comp rx=0 tx=0).stop 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 shutdown_connections 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b500725e0 0x7f0b50074aa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b74159b30 0x7f0b740c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b7413fb20 0x7f0b740c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 --2- 172.21.15.97:0/1747621416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b58004080 0x7f0b74151300 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 >> 172.21.15.97:0/1747621416 conn(0x7f0b740b1780 msgr2=0x7f0b740c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 shutdown_connections 2024-09-10T21:32:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:27.871+0000 7f0b79000640 1 -- 172.21.15.97:0/1747621416 wait complete. 2024-09-10T21:32:27.894 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:32:27.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:27.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:32:27.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:27.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:27.917 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:27.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 -- 172.21.15.97:0/2766212194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc074370 msgr2=0x7fefcc06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 --2- 172.21.15.97:0/2766212194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc074370 0x7fefcc06cb40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0749f0 tx=0x7fefb801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 -- 172.21.15.97:0/2766212194 shutdown_connections 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 --2- 172.21.15.97:0/2766212194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc074370 0x7fefcc06cb40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 --2- 172.21.15.97:0/2766212194 >> v2:172.21.15.194:3300/0 conn(0x7fefcc159470 0x7fefcc070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 -- 172.21.15.97:0/2766212194 >> 172.21.15.97:0/2766212194 conn(0x7fefcc0b26b0 msgr2=0x7fefcc0b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 -- 172.21.15.97:0/2766212194 shutdown_connections 2024-09-10T21:32:28.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.159+0000 7fefd0a00640 1 -- 172.21.15.97:0/2766212194 wait complete. 2024-09-10T21:32:28.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 Processor -- start 2024-09-10T21:32:28.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 -- start start 2024-09-10T21:32:28.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefcc074370 0x7fefcc0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefcc159470 0x7fefcc14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc15cae0 0x7fefcc1fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc168fa0 con 0x7fefcc15cae0 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc168e20 con 0x7fefcc074370 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc169120 con 0x7fefcc159470 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc15cae0 0x7fefcc1fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc15cae0 0x7fefcc1fb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38986/0 (socket says 172.21.15.97:38986) 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 -- 172.21.15.97:0/3654430791 learned_addr learned my addr 172.21.15.97:0/3654430791 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcb400640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefcc074370 0x7fefcc0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcaa00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefcc159470 0x7fefcc14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 -- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefcc159470 msgr2=0x7fefcc14e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefcc159470 0x7fefcc14e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 -- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefcc074370 msgr2=0x7fefcc0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefcc074370 0x7fefcc0b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 -- 172.21.15.97:0/3654430791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc1536a0 con 0x7fefcc15cae0 2024-09-10T21:32:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefcbe00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc15cae0 0x7fefcc1fb4d0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fefc400c730 tx=0x7fefc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefc3400640 1 -- 172.21.15.97:0/3654430791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4003ba0 con 0x7fefcc15cae0 2024-09-10T21:32:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefc3400640 1 -- 172.21.15.97:0/3654430791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefc4003d60 con 0x7fefcc15cae0 2024-09-10T21:32:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefcc152e20 con 0x7fefcc15cae0 2024-09-10T21:32:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefc3400640 1 -- 172.21.15.97:0/3654430791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc400aab0 con 0x7fefcc15cae0 2024-09-10T21:32:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefcc1533c0 con 0x7fefcc15cae0 2024-09-10T21:32:28.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.163+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefcc06cb40 con 0x7fefcc15cae0 2024-09-10T21:32:28.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.167+0000 7fefc3400640 1 -- 172.21.15.97:0/3654430791 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fefc400ac50 con 0x7fefcc15cae0 2024-09-10T21:32:28.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.167+0000 7fefc3400640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefac0725e0 0x7fefac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.167+0000 7fefc3400640 1 -- 172.21.15.97:0/3654430791 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7307+0+0 (secure 0 0 0) 0x7fefc40936d0 con 0x7fefcc15cae0 2024-09-10T21:32:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.167+0000 7fefcb400640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefac0725e0 0x7fefac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.167+0000 7fefcb400640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefac0725e0 0x7fefac074aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005d90 tx=0x7fefbc00b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:28.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.171+0000 7fefc3400640 1 -- 172.21.15.97:0/3654430791 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fefc405dff0 con 0x7fefcc15cae0 2024-09-10T21:32:28.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.371+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fefcc071190 con 0x7fefcc15cae0 2024-09-10T21:32:28.375 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:28.371+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:28.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefc3400640 1 -- 172.21.15.97:0/3654430791 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v163) ==== 161+0+0 (secure 0 0 0) 0x7fefc4062a20 con 0x7fefcc15cae0 2024-09-10T21:32:28.377 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:28.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefac0725e0 msgr2=0x7fefac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefac0725e0 0x7fefac074aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005d90 tx=0x7fefbc00b390 comp rx=0 tx=0).stop 2024-09-10T21:32:28.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc15cae0 msgr2=0x7fefcc1fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc15cae0 0x7fefcc1fb4d0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fefc400c730 tx=0x7fefc400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 shutdown_connections 2024-09-10T21:32:28.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fefac0725e0 0x7fefac074aa0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefcc15cae0 0x7fefcc1fb4d0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefcc159470 0x7fefcc14e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 --2- 172.21.15.97:0/3654430791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefcc074370 0x7fefcc0b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 >> 172.21.15.97:0/3654430791 conn(0x7fefcc0b26b0 msgr2=0x7fefcc15c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:28.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 shutdown_connections 2024-09-10T21:32:28.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.375+0000 7fefd0a00640 1 -- 172.21.15.97:0/3654430791 wait complete. 2024-09-10T21:32:28.391 INFO:tasks.ceph.mds.c.smithi097.stderr: -379> 2024-09-10T21:32:20.951+0000 7f2f1d400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:32:28.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:28.434 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:32:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.547+0000 7f8d31400640 1 -- 172.21.15.97:0/2450494039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c15efc0 msgr2=0x7f8d2c0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.547+0000 7f8d31400640 1 --2- 172.21.15.97:0/2450494039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c15efc0 0x7f8d2c0cf840 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009840 tx=0x7f8d1c01c820 comp rx=0 tx=0).stop 2024-09-10T21:32:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.547+0000 7f8d31400640 1 -- 172.21.15.97:0/2450494039 shutdown_connections 2024-09-10T21:32:28.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.547+0000 7f8d31400640 1 --2- 172.21.15.97:0/2450494039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c15efc0 0x7f8d2c0cf840 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.547+0000 7f8d31400640 1 -- 172.21.15.97:0/2450494039 >> 172.21.15.97:0/2450494039 conn(0x7f8d2c0d1a60 msgr2=0x7f8d2c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.547+0000 7f8d31400640 1 -- 172.21.15.97:0/2450494039 shutdown_connections 2024-09-10T21:32:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.547+0000 7f8d31400640 1 -- 172.21.15.97:0/2450494039 wait complete. 2024-09-10T21:32:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 Processor -- start 2024-09-10T21:32:28.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 -- start start 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d2c0cdcb0 0x7f8d2c155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c0d0490 0x7f8d2c1fb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d2c15efc0 0x7f8d2c159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c169b50 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1699d0 con 0x7f8d2c0d0490 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8d2c169cd0 con 0x7f8d2c15efc0 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d2c0cdcb0 0x7f8d2c155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d2c0cdcb0 0x7f8d2c155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39004/0 (socket says 172.21.15.97:39004) 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 -- 172.21.15.97:0/3979678115 learned_addr learned my addr 172.21.15.97:0/3979678115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2b400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c0d0490 0x7f8d2c1fb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 -- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d2c15efc0 msgr2=0x7f8d2c159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d30a00640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d2c15efc0 0x7f8d2c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d2c15efc0 0x7f8d2c159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 -- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c0d0490 msgr2=0x7f8d2c1fb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c0d0490 0x7f8d2c1fb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 -- 172.21.15.97:0/3979678115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c15a160 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2b400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c0d0490 0x7f8d2c1fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:28.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d30a00640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d2c15efc0 0x7f8d2c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d2be00640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d2c0cdcb0 0x7f8d2c155800 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c01cd00 tx=0x7f8d1c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d23e00640 1 -- 172.21.15.97:0/3979678115 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0bb070 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d23e00640 1 -- 172.21.15.97:0/3979678115 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d1c004330 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c0b2420 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d23e00640 1 -- 172.21.15.97:0/3979678115 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0b2040 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.551+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c15a6e0 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.555+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c0ae930 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.555+0000 7f8d23e00640 1 -- 172.21.15.97:0/3979678115 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8d1c0b5070 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.555+0000 7f8d23e00640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d0c0725e0 0x7f8d0c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.555+0000 7f8d23e00640 1 -- 172.21.15.97:0/3979678115 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7393+0+0 (secure 0 0 0) 0x7f8d1c12d320 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.555+0000 7f8d2b400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d0c0725e0 0x7f8d0c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:28.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.555+0000 7f8d2b400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d0c0725e0 0x7f8d0c074aa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000f70 tx=0x7f8d18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:28.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.559+0000 7f8d23e00640 1 -- 172.21.15.97:0/3979678115 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8d1c016410 con 0x7f8d2c0cdcb0 2024-09-10T21:32:28.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:28.795+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8d2c15d130 con 0x7f8d2c0cdcb0 2024-09-10T21:32:29.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.375+0000 7f8d23e00640 1 -- 172.21.15.97:0/3979678115 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v164) ==== 107+0+0 (secure 0 0 0) 0x7f8d1c0f82c0 con 0x7f8d2c0cdcb0 2024-09-10T21:32:29.379 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d0c0725e0 msgr2=0x7f8d0c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d0c0725e0 0x7f8d0c074aa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000f70 tx=0x7f8d18009040 comp rx=0 tx=0).stop 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d2c0cdcb0 msgr2=0x7f8d2c155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d2c0cdcb0 0x7f8d2c155800 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c01cd00 tx=0x7f8d1c005d30 comp rx=0 tx=0).stop 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 shutdown_connections 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8d0c0725e0 0x7f8d0c074aa0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8d2c15efc0 0x7f8d2c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d2c0d0490 0x7f8d2c1fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 --2- 172.21.15.97:0/3979678115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d2c0cdcb0 0x7f8d2c155800 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 >> 172.21.15.97:0/3979678115 conn(0x7f8d2c0d1a60 msgr2=0x7f8d2c06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:29.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 shutdown_connections 2024-09-10T21:32:29.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.379+0000 7f8d31400640 1 -- 172.21.15.97:0/3979678115 wait complete. 2024-09-10T21:32:29.406 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:29.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.495+0000 7fb3a4890080 1 Processor -- start 2024-09-10T21:32:29.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.495+0000 7fb3a4890080 1 -- start start 2024-09-10T21:32:29.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb388002350 0x561a1342a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a1340a880 0x561a134270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 0x561a13427fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a132fef40 con 0x561a1340a880 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561a1330bd00 con 0x7fb388002350 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561a132a0a00 con 0x561a13427630 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a1340a880 0x561a134270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a1340a880 0x561a134270f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39024/0 (socket says 172.21.15.97:39024) 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 -- 172.21.15.97:0/266513193 learned_addr learned my addr 172.21.15.97:0/266513193 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a2400640 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 0x561a13427fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a1a00640 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb388002350 0x561a1342a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 -- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 msgr2=0x561a13427fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 0x561a13427fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 -- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb388002350 msgr2=0x561a1342a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb388002350 0x561a1342a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 -- 172.21.15.97:0/266513193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a13429310 con 0x561a1340a880 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39be00640 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a1340a880 0x561a134270f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0016e0 tx=0x7fb38c009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3041c1e09938a131 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39b400640 1 -- 172.21.15.97:0/266513193 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0097a0 con 0x561a1340a880 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39b400640 1 -- 172.21.15.97:0/266513193 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb38c014020 con 0x561a1340a880 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb39b400640 1 -- 172.21.15.97:0/266513193 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00dd80 con 0x561a1340a880 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- 172.21.15.97:0/266513193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a1340a880 msgr2=0x561a134270f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a1340a880 0x561a134270f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0016e0 tx=0x7fb38c009d10 comp rx=0 tx=0).stop 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- 172.21.15.97:0/266513193 shutdown_connections 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 0x561a13427fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a1340a880 0x561a134270f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 --2- 172.21.15.97:0/266513193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb388002350 0x561a1342a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- 172.21.15.97:0/266513193 >> 172.21.15.97:0/266513193 conn(0x561a13380520 msgr2=0x561a1340e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:29.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- 172.21.15.97:0/266513193 shutdown_connections 2024-09-10T21:32:29.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- 172.21.15.97:0/266513193 wait complete. 2024-09-10T21:32:29.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 Processor -- start 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.499+0000 7fb3a4890080 1 -- start start 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a1340a880 0x561a13423fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 0x561a13384700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb388002350 0x561a13382670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a134295b0 con 0x7fb388002350 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561a1342a770 con 0x561a1340a880 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561a13429850 con 0x561a13427630 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb388002350 0x561a13382670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a1a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a1340a880 0x561a13423fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb388002350 0x561a13382670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39032/0 (socket says 172.21.15.97:39032) 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 learned_addr learned my addr 172.21.15.97:0/1095085356 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb39be00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 0x561a13384700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 msgr2=0x561a13384700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561a13427630 0x561a13384700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a1340a880 msgr2=0x561a13423fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561a1340a880 0x561a13423fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a133510f0 con 0x7fb388002350 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a2400640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb388002350 0x561a13382670 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x561a13359540 tx=0x7fb39c010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb39a000640 1 -- 172.21.15.97:0/1095085356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c01b070 con 0x7fb388002350 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb39a000640 1 -- 172.21.15.97:0/1095085356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb39c00e810 con 0x7fb388002350 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561a13351390 con 0x7fb388002350 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb39a000640 1 -- 172.21.15.97:0/1095085356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00ec60 con 0x7fb388002350 2024-09-10T21:32:29.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a133518d0 con 0x7fb388002350 2024-09-10T21:32:29.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.503+0000 7fb39a000640 1 -- 172.21.15.97:0/1095085356 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb39c0074d0 con 0x7fb388002350 2024-09-10T21:32:29.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb39a000640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb380070ff0 0x7fb3800734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb39a000640 1 -- 172.21.15.97:0/1095085356 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7393+0+0 (secure 0 0 0) 0x7fb39c0922e0 con 0x7fb388002350 2024-09-10T21:32:29.509 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:29.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb3a1a00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb380070ff0 0x7fb3800734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561a13352010 0x561a134c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a13384c40 con 0x561a13352010 2024-09-10T21:32:29.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb39be00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561a13352010 0x561a134c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb3a1a00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb380070ff0 0x7fb3800734b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb3940ab000 tx=0x7fb3940184b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb39be00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561a13352010 0x561a134c32f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v109'381 uv355 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.507+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134c3830 con 0x561a13352010 2024-09-10T21:32:29.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.511+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v109'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.511+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134cf4d0 con 0x561a13352010 2024-09-10T21:32:29.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.511+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v109'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.516 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:29.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.511+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0880 con 0x561a13352010 2024-09-10T21:32:29.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.519+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.519+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0c20 con 0x561a13352010 2024-09-10T21:32:29.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.519+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.519+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134cf870 con 0x561a13352010 2024-09-10T21:32:29.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.523+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.523+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134cfc60 con 0x561a13352010 2024-09-10T21:32:29.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.523+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v109'387 uv361 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.523+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.523+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v109'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.523+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.526 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:29.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.527+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v109'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.527+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.527+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.527+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.531+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.531+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.531+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.531+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.531+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v109'393 uv368 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'394 uv394 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.536 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:29.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v109'395 uv395 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'396 uv396 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'397 uv397 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.535+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0000 con 0x561a13352010 2024-09-10T21:32:29.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'398 uv398 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'399 uv399 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v109'400 uv374 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v109'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.548 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:29.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v109'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.543+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.547+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.547+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.547+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.547+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.555+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.555+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.555+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v109'406 uv380 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.555+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.555+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v109'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.555+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.559+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v109'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.558 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:29.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.559+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.559+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.559+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d0280 con 0x561a13352010 2024-09-10T21:32:29.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a13352010 2024-09-10T21:32:29.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561a134d0fc0 0x561a134d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d38f0 con 0x561a134d0fc0 2024-09-10T21:32:29.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a1a00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561a134d0fc0 0x561a134d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a1a00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561a134d0fc0 0x561a134d33b0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'287 uv266 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d3c90 con 0x561a134d0fc0 2024-09-10T21:32:29.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'288 uv288 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d4030 con 0x561a134d0fc0 2024-09-10T21:32:29.568 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:29.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'289 uv289 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.563+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d4880 con 0x561a134d0fc0 2024-09-10T21:32:29.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.567+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v109'290 uv290 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.567+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d4c20 con 0x561a134d0fc0 2024-09-10T21:32:29.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.567+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'291 uv291 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.567+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d4fc0 con 0x561a134d0fc0 2024-09-10T21:32:29.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.571+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v109'292 uv285 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.571+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d5360 con 0x561a134d0fc0 2024-09-10T21:32:29.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.571+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v109'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.571+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.575 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:29.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.571+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v109'294 uv294 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.571+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.575+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.575+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.575+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.575+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.575+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v109'297 uv286 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.575+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.579+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v109'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.579+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.582 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:29.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.579+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v109'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.579+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d98d0 con 0x561a134d0fc0 2024-09-10T21:32:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.579+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v109'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.579+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d98d0 con 0x561a134d0fc0 2024-09-10T21:32:29.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.583+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.583+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.583+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v109'302 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.583+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.583+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v109'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.583+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.588 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:29.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.583+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v109'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.587+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d6710 con 0x561a134d0fc0 2024-09-10T21:32:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.587+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'305 uv305 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.587+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d6710 con 0x561a134d0fc0 2024-09-10T21:32:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.587+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.587+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d5700 con 0x561a134d0fc0 2024-09-10T21:32:29.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.587+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134d0fc0 2024-09-10T21:32:29.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561a134d5c10 0x561a134d8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d8580 con 0x561a134d5c10 2024-09-10T21:32:29.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb39be00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561a134d5c10 0x561a134d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb39be00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561a134d5c10 0x561a134d8000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134d8640 con 0x561a134d5c10 2024-09-10T21:32:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.591+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d8a60 con 0x561a134d5c10 2024-09-10T21:32:29.597 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.595+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.595+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d9a70 con 0x561a134d5c10 2024-09-10T21:32:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.595+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.595+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d9a70 con 0x561a134d5c10 2024-09-10T21:32:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.595+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.595+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134d8a60 con 0x561a134d5c10 2024-09-10T21:32:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561a134d90e0 0x561a134db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dbac0 con 0x561a134d90e0 2024-09-10T21:32:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a2400640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561a134d90e0 0x561a134db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a2400640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561a134d90e0 0x561a134db4d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'63 uv62 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134dbfd0 con 0x561a134d90e0 2024-09-10T21:32:29.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'64 uv64 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.599+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dc490 con 0x561a134d90e0 2024-09-10T21:32:29.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.603+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'65 uv65 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.606 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:29.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.603+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dcd80 con 0x561a134d90e0 2024-09-10T21:32:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.603+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'66 uv66 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.603+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dd120 con 0x561a134d5c10 2024-09-10T21:32:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.603+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.603+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134dd120 con 0x561a134d5c10 2024-09-10T21:32:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dd120 con 0x561a134d5c10 2024-09-10T21:32:29.610 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134de130 con 0x561a134d5c10 2024-09-10T21:32:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134de130 con 0x561a134d5c10 2024-09-10T21:32:29.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.607+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dd120 con 0x561a134d5c10 2024-09-10T21:32:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.611+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.611+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dd120 con 0x561a134d90e0 2024-09-10T21:32:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.611+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v109'138 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.611+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134dd4c0 con 0x561a134d90e0 2024-09-10T21:32:29.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.611+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v109'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.611+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dd860 con 0x561a134d90e0 2024-09-10T21:32:29.617 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.615+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v109'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.615+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dec10 con 0x561a134d90e0 2024-09-10T21:32:29.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.615+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.615+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dec10 con 0x561a134d90e0 2024-09-10T21:32:29.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.615+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.615+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d90e0 2024-09-10T21:32:29.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.619+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.619+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d90e0 2024-09-10T21:32:29.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.619+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v109'144 uv137 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.619+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d90e0 2024-09-10T21:32:29.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.619+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v109'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.619+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d90e0 2024-09-10T21:32:29.625 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:29.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.623+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v109'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.623+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d90e0 2024-09-10T21:32:29.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.623+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.623+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d90e0 2024-09-10T21:32:29.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.623+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.623+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d5c10 2024-09-10T21:32:29.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.627+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v109'61 uv60 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.627+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d5c10 2024-09-10T21:32:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.627+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v109'62 uv62 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.627+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d5c10 2024-09-10T21:32:29.631 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:29.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.627+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v109'63 uv63 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.627+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d5c10 2024-09-10T21:32:29.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v109'64 uv64 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134ddc00 con 0x561a134d5c10 2024-09-10T21:32:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v109'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134d5c10 2024-09-10T21:32:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561a134de280 0x561a134e0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e0bf0 con 0x561a134de280 2024-09-10T21:32:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb3a1a00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561a134de280 0x561a134e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb3a1a00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561a134de280 0x561a134e0670 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.631+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134e0fc0 con 0x561a134de280 2024-09-10T21:32:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.635+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.635+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e0fc0 con 0x561a134de280 2024-09-10T21:32:29.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.635+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.639 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:29.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.635+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1fd0 con 0x561a134de280 2024-09-10T21:32:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.635+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.635+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1fd0 con 0x561a134de280 2024-09-10T21:32:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.639+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.639+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e0fc0 con 0x561a134de280 2024-09-10T21:32:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.639+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.639+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e0fc0 con 0x561a134de280 2024-09-10T21:32:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.639+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v109'122 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.639+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134e0fc0 con 0x561a134de280 2024-09-10T21:32:29.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v109'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e0fc0 con 0x561a134de280 2024-09-10T21:32:29.646 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:29.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v109'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dc970 con 0x561a134de280 2024-09-10T21:32:29.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v109'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dc970 con 0x561a134de280 2024-09-10T21:32:29.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v109'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.643+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dc970 con 0x561a134de280 2024-09-10T21:32:29.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v109'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134dc970 con 0x561a134de280 2024-09-10T21:32:29.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v109'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dc970 con 0x561a134de280 2024-09-10T21:32:29.651 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:29.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v109'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134dc970 con 0x561a134de280 2024-09-10T21:32:29.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v109'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.647+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e3e10 con 0x561a134de280 2024-09-10T21:32:29.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.651+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v109'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.651+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.651+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v109'57 uv55 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.651+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.651+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v109'58 uv58 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.651+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.657 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:29.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.655+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v109'59 uv59 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.655+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.655+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v109'60 uv60 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.655+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.655+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v109'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.655+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v109'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.661 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:29.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v109'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e2480 con 0x561a134d90e0 2024-09-10T21:32:29.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v109'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e2480 con 0x561a134d90e0 2024-09-10T21:32:29.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v109'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.659+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e1470 con 0x561a134d90e0 2024-09-10T21:32:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v109'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb39c01d030 con 0x561a134d90e0 2024-09-10T21:32:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561a134e1af0 0x561a134e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e44d0 con 0x561a134e1af0 2024-09-10T21:32:29.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb39be00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561a134e1af0 0x561a134e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb39be00640 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561a134e1af0 0x561a134e3ee0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v109'64 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134e1af0 2024-09-10T21:32:29.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134e44d0 con 0x561a134e1af0 2024-09-10T21:32:29.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v109'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134e1af0 2024-09-10T21:32:29.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.663+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e44d0 con 0x561a134e1af0 2024-09-10T21:32:29.669 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:29.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.667+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v109'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134e1af0 2024-09-10T21:32:29.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.667+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e48c0 con 0x561a134e1af0 2024-09-10T21:32:29.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.667+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v109'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134e1af0 2024-09-10T21:32:29.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.667+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e48c0 con 0x561a134e1af0 2024-09-10T21:32:29.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.667+0000 7fb39be00640 1 -- 172.21.15.97:0/1095085356 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v109'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb38c00db20 con 0x561a134e1af0 2024-09-10T21:32:29.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.667+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4630 con 0x561a134de280 2024-09-10T21:32:29.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v109'157 uv148 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134e4630 con 0x561a134de280 2024-09-10T21:32:29.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v109'158 uv158 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4630 con 0x561a134de280 2024-09-10T21:32:29.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v109'159 uv159 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.675 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:29.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4630 con 0x561a134de280 2024-09-10T21:32:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v109'160 uv160 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.671+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4610 con 0x561a134de280 2024-09-10T21:32:29.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.675+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v109'161 uv161 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.675+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4610 con 0x561a134de280 2024-09-10T21:32:29.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.675+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v109'162 uv155 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.675+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x561a134e4610 con 0x561a134de280 2024-09-10T21:32:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.675+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v109'163 uv163 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.675+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4610 con 0x561a134de280 2024-09-10T21:32:29.681 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v109'164 uv164 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4610 con 0x561a134de280 2024-09-10T21:32:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v109'165 uv165 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4610 con 0x561a134de280 2024-09-10T21:32:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v109'166 uv166 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4610 con 0x561a134de280 2024-09-10T21:32:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v109'167 uv167 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.679+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x561a134e4b30 con 0x561a134de280 2024-09-10T21:32:29.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a1a00640 1 -- 172.21.15.97:0/1095085356 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v109'168 uv168 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb3940a97f0 con 0x561a134de280 2024-09-10T21:32:29.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561a134d5c10 msgr2=0x561a134d8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561a134d5c10 0x561a134d8000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561a134de280 msgr2=0x561a134e0670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561a134de280 0x561a134e0670 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561a134e1af0 msgr2=0x561a134e3ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561a134e1af0 0x561a134e3ee0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561a134d90e0 msgr2=0x561a134db4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561a134d90e0 0x561a134db4d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561a134d0fc0 msgr2=0x561a134d33b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561a134d0fc0 0x561a134d33b0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561a13352010 msgr2=0x561a134c32f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561a13352010 0x561a134c32f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 reap_dead start 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 reap_dead start 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 reap_dead start 2024-09-10T21:32:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a2400640 1 -- 172.21.15.97:0/1095085356 reap_dead start 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb380070ff0 msgr2=0x7fb3800734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb380070ff0 0x7fb3800734b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb3940ab000 tx=0x7fb3940184b0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb388002350 msgr2=0x561a13382670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb388002350 0x561a13382670 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x561a13359540 tx=0x7fb39c010bf0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 shutdown_connections 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb380070ff0 0x7fb3800734b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 --2- 172.21.15.97:0/1095085356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb388002350 0x561a13382670 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 >> 172.21.15.97:0/1095085356 conn(0x561a13380520 msgr2=0x561a13427fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 shutdown_connections 2024-09-10T21:32:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.683+0000 7fb3a4890080 1 -- 172.21.15.97:0/1095085356 wait complete. 2024-09-10T21:32:29.691 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2024-09-10T21:32:29.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 Processor -- start 2024-09-10T21:32:29.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- start start 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eabfbe60 0x5568eabfa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eac6ed10 0x5568eabe0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 0x5568eabe06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568eab00a00 con 0x5568eac6ed10 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5568eab5ef40 con 0x5568eabfbe60 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5568eac95790 con 0x5568eac6f0f0 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eac6ed10 0x5568eabe0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eac6ed10 0x5568eabe0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39046/0 (socket says 172.21.15.97:39046) 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 -- 172.21.15.97:0/1046319695 learned_addr learned my addr 172.21.15.97:0/1046319695 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6986a00640 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 0x5568eabe06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6986000640 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eabfbe60 0x5568eabfa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 -- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 msgr2=0x5568eabe06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 0x5568eabe06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 -- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eabfbe60 msgr2=0x5568eabfa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eabfbe60 0x5568eabfa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 -- 172.21.15.97:0/1046319695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568eaad9c10 con 0x5568eac6ed10 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6985600640 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eac6ed10 0x5568eabe0170 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f697400c640 tx=0x7f697400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ad270aab0b01bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f697fe00640 1 -- 172.21.15.97:0/1046319695 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400ab20 con 0x5568eac6ed10 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f697fe00640 1 -- 172.21.15.97:0/1046319695 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f697400acc0 con 0x5568eac6ed10 2024-09-10T21:32:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f697fe00640 1 -- 172.21.15.97:0/1046319695 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974008690 con 0x5568eac6ed10 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- 172.21.15.97:0/1046319695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eac6ed10 msgr2=0x5568eabe0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eac6ed10 0x5568eabe0170 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f697400c640 tx=0x7f697400cb00 comp rx=0 tx=0).stop 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- 172.21.15.97:0/1046319695 shutdown_connections 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 0x5568eabe06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eac6ed10 0x5568eabe0170 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 --2- 172.21.15.97:0/1046319695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eabfbe60 0x5568eabfa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- 172.21.15.97:0/1046319695 >> 172.21.15.97:0/1046319695 conn(0x5568eabb1140 msgr2=0x5568eabde370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- 172.21.15.97:0/1046319695 shutdown_connections 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.779+0000 7f6989c2b080 1 -- 172.21.15.97:0/1046319695 wait complete. 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 Processor -- start 2024-09-10T21:32:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 -- start start 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eabfbe60 0x5568eac88030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eac6ed10 0x5568eac8f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 0x5568eac8fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568eac8b210 con 0x5568eabfbe60 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5568eac6bf00 con 0x5568eac6ed10 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5568eac84de0 con 0x5568eac6f0f0 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eabfbe60 0x5568eac88030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eabfbe60 0x5568eac88030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39048/0 (socket says 172.21.15.97:39048) 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 -- 172.21.15.97:0/3133255951 learned_addr learned my addr 172.21.15.97:0/3133255951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6985600640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eac6ed10 0x5568eac8f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986a00640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 0x5568eac8fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 -- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 msgr2=0x5568eac8fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5568eac6f0f0 0x5568eac8fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 -- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eac6ed10 msgr2=0x5568eac8f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568eac6ed10 0x5568eac8f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 -- 172.21.15.97:0/3133255951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568eac8e4d0 con 0x5568eabfbe60 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6986000640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eabfbe60 0x5568eac88030 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f697801b710 tx=0x7f69780acad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f697ea00640 1 -- 172.21.15.97:0/3133255951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69780b8440 con 0x5568eabfbe60 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f697ea00640 1 -- 172.21.15.97:0/3133255951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69780b86d0 con 0x5568eabfbe60 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568eac87110 con 0x5568eabfbe60 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f697ea00640 1 -- 172.21.15.97:0/3133255951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69780b8920 con 0x5568eabfbe60 2024-09-10T21:32:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.783+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568eac87650 con 0x5568eabfbe60 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f697ea00640 1 -- 172.21.15.97:0/3133255951 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f69780b8ac0 con 0x5568eabfbe60 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f697ea00640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6968070ff0 0x7f69680734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f697ea00640 1 -- 172.21.15.97:0/3133255951 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7393+0+0 (secure 0 0 0) 0x7f697812dd40 con 0x5568eabfbe60 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f6989c2b080 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5568eac6a910 0x5568ead2d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x5568eac88570 con 0x5568eac6a910 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f6985600640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6968070ff0 0x7f69680734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f6986a00640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5568eac6a910 0x5568ead2d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f6985600640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6968070ff0 0x7f69680734b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f697400c5e0 tx=0x7f6974016040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.787+0000 7f6986a00640 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5568eac6a910 0x5568ead2d570 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6986a00640 1 -- 172.21.15.97:0/3133255951 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000.inode [delete] v109'169 uv169 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6980002620 con 0x5568eac6a910 2024-09-10T21:32:29.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5568eac6a910 msgr2=0x5568ead2d570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5568eac6a910 0x5568ead2d570 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6968070ff0 msgr2=0x7f69680734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6968070ff0 0x7f69680734b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f697400c5e0 tx=0x7f6974016040 comp rx=0 tx=0).stop 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eabfbe60 msgr2=0x5568eac88030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 --2- 172.21.15.97:0/3133255951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568eabfbe60 0x5568eac88030 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f697801b710 tx=0x7f69780acad0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6986a00640 1 -- 172.21.15.97:0/3133255951 reap_dead start 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 shutdown_connections 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 >> 172.21.15.97:0/3133255951 conn(0x5568eabb1140 msgr2=0x5568eabfa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 shutdown_connections 2024-09-10T21:32:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.791+0000 7f6989c2b080 1 -- 172.21.15.97:0/3133255951 wait complete. 2024-09-10T21:32:29.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.987+0000 7fd37ec00640 1 -- 172.21.15.97:0/2257039721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce5f0 msgr2=0x7fd378141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.987+0000 7fd37ec00640 1 --2- 172.21.15.97:0/2257039721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce5f0 0x7fd378141d20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd378141580 tx=0x7fd36801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.987+0000 7fd37ec00640 1 -- 172.21.15.97:0/2257039721 shutdown_connections 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.987+0000 7fd37ec00640 1 --2- 172.21.15.97:0/2257039721 >> v2:172.21.15.194:3300/0 conn(0x7fd37815fde0 0x7fd37813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.987+0000 7fd37ec00640 1 --2- 172.21.15.97:0/2257039721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce5f0 0x7fd378141d20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.987+0000 7fd37ec00640 1 -- 172.21.15.97:0/2257039721 >> 172.21.15.97:0/2257039721 conn(0x7fd3780b1780 msgr2=0x7fd3780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.987+0000 7fd37ec00640 1 -- 172.21.15.97:0/2257039721 shutdown_connections 2024-09-10T21:32:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 -- 172.21.15.97:0/2257039721 wait complete. 2024-09-10T21:32:29.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 Processor -- start 2024-09-10T21:32:29.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 -- start start 2024-09-10T21:32:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3780ce5f0 0x7fd37813b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce9d0 0x7fd37815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd37815fde0 0x7fd37815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd378168a70 con 0x7fd3780ce9d0 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd378140cd0 con 0x7fd3780ce5f0 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd378168bf0 con 0x7fd37815fde0 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce9d0 0x7fd37815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce9d0 0x7fd37815e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39058/0 (socket says 172.21.15.97:39058) 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 -- 172.21.15.97:0/700889139 learned_addr learned my addr 172.21.15.97:0/700889139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd377e00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3780ce5f0 0x7fd37813b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37cc00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd37815fde0 0x7fd37815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 -- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd37815fde0 msgr2=0x7fd37815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd37815fde0 0x7fd37815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 -- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3780ce5f0 msgr2=0x7fd37813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3780ce5f0 0x7fd37813b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 -- 172.21.15.97:0/700889139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37813c700 con 0x7fd3780ce9d0 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd36f400640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce9d0 0x7fd37815e7f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd36000c4e0 tx=0x7fd36000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd374c00640 1 -- 172.21.15.97:0/700889139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3600052d0 con 0x7fd3780ce9d0 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd374c00640 1 -- 172.21.15.97:0/700889139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd360005470 con 0x7fd3780ce9d0 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd374c00640 1 -- 172.21.15.97:0/700889139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360012890 con 0x7fd3780ce9d0 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.991+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37813c9e0 con 0x7fd3780ce9d0 2024-09-10T21:32:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.995+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37813b660 con 0x7fd3780ce9d0 2024-09-10T21:32:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.995+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd37806e3b0 con 0x7fd3780ce9d0 2024-09-10T21:32:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.995+0000 7fd374c00640 1 -- 172.21.15.97:0/700889139 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd360003600 con 0x7fd3780ce9d0 2024-09-10T21:32:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.995+0000 7fd374c00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd354072610 0x7fd354074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.995+0000 7fd374c00640 1 -- 172.21.15.97:0/700889139 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7393+0+0 (secure 0 0 0) 0x7fd360093310 con 0x7fd3780ce9d0 2024-09-10T21:32:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.995+0000 7fd377e00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd354072610 0x7fd354074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:29.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.995+0000 7fd377e00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd354072610 0x7fd354074ad0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd3680af000 tx=0x7fd36801b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:29.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:29.999+0000 7fd374c00640 1 -- 172.21.15.97:0/700889139 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd36005dcf0 con 0x7fd3780ce9d0 2024-09-10T21:32:30.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.199+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd378141d20 con 0x7fd3780ce9d0 2024-09-10T21:32:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.643+0000 7fd374c00640 1 -- 172.21.15.97:0/700889139 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v165) ==== 162+0+0 (secure 0 0 0) 0x7fd360062720 con 0x7fd3780ce9d0 2024-09-10T21:32:30.643 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd354072610 msgr2=0x7fd354074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd354072610 0x7fd354074ad0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd3680af000 tx=0x7fd36801b100 comp rx=0 tx=0).stop 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce9d0 msgr2=0x7fd37815e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce9d0 0x7fd37815e7f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd36000c4e0 tx=0x7fd36000c9a0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 shutdown_connections 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd354072610 0x7fd354074ad0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd37815fde0 0x7fd37815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3780ce9d0 0x7fd37815e7f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 --2- 172.21.15.97:0/700889139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3780ce5f0 0x7fd37813b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 >> 172.21.15.97:0/700889139 conn(0x7fd3780b1780 msgr2=0x7fd37815bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 shutdown_connections 2024-09-10T21:32:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.647+0000 7fd37ec00640 1 -- 172.21.15.97:0/700889139 wait complete. 2024-09-10T21:32:30.655 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:32:30.655+0000 7fcb7ea00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T21:32:30.655 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:32:30.655+0000 7fcb7ea00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:32:30.656 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:30.655+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:30.656 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:30.655+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:32:30.672 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:30.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 -- 172.21.15.97:0/4051968402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280b0d50 msgr2=0x7ff7280ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:30.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 --2- 172.21.15.97:0/4051968402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280b0d50 0x7ff7280ccc10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff7280b36f0 tx=0x7ff72001cd80 comp rx=0 tx=0).stop 2024-09-10T21:32:30.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 -- 172.21.15.97:0/4051968402 shutdown_connections 2024-09-10T21:32:30.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 --2- 172.21.15.97:0/4051968402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280b0d50 0x7ff7280ccc10 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 --2- 172.21.15.97:0/4051968402 >> v2:172.21.15.194:3300/0 conn(0x7ff7280afdd0 0x7ff7280cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 -- 172.21.15.97:0/4051968402 >> 172.21.15.97:0/4051968402 conn(0x7ff7280d1c30 msgr2=0x7ff7280d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:30.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 -- 172.21.15.97:0/4051968402 shutdown_connections 2024-09-10T21:32:30.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 -- 172.21.15.97:0/4051968402 wait complete. 2024-09-10T21:32:30.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 Processor -- start 2024-09-10T21:32:30.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.815+0000 7ff72da00640 1 -- start start 2024-09-10T21:32:30.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7280afdd0 0x7ff728156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:30.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7280b0d50 0x7ff72815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:30.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280ce640 0x7ff7280c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:30.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff728168ee0 con 0x7ff7280ce640 2024-09-10T21:32:30.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff728168d60 con 0x7ff7280b0d50 2024-09-10T21:32:30.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff728169060 con 0x7ff7280afdd0 2024-09-10T21:32:30.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280ce640 0x7ff7280c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280ce640 0x7ff7280c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39072/0 (socket says 172.21.15.97:39072) 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 -- 172.21.15.97:0/1777297770 learned_addr learned my addr 172.21.15.97:0/1777297770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff71fe00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7280b0d50 0x7ff72815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff726a00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7280afdd0 0x7ff728156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 -- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7280afdd0 msgr2=0x7ff728156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7280afdd0 0x7ff728156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 -- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7280b0d50 msgr2=0x7ff72815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7280b0d50 0x7ff72815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 -- 172.21.15.97:0/1777297770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7280c7700 con 0x7ff7280ce640 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff726a00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7280afdd0 0x7ff728156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff727400640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280ce640 0x7ff7280c5760 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff720009bd0 tx=0x7ff720009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff71ea00640 1 -- 172.21.15.97:0/1777297770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7200bb070 con 0x7ff7280ce640 2024-09-10T21:32:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff71ea00640 1 -- 172.21.15.97:0/1777297770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff720004d40 con 0x7ff7280ce640 2024-09-10T21:32:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff71ea00640 1 -- 172.21.15.97:0/1777297770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7200b2040 con 0x7ff7280ce640 2024-09-10T21:32:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7280c79e0 con 0x7ff7280ce640 2024-09-10T21:32:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7280c7fa0 con 0x7ff7280ce640 2024-09-10T21:32:30.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.819+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7280cea20 con 0x7ff7280ce640 2024-09-10T21:32:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.823+0000 7ff71ea00640 1 -- 172.21.15.97:0/1777297770 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7200b5030 con 0x7ff7280ce640 2024-09-10T21:32:30.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.823+0000 7ff71ea00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff704072610 0x7ff704074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:30.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.823+0000 7ff71ea00640 1 -- 172.21.15.97:0/1777297770 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7ff72012d3c0 con 0x7ff7280ce640 2024-09-10T21:32:30.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.823+0000 7ff726a00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff704072610 0x7ff704074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:30.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.823+0000 7ff726a00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff704072610 0x7ff704074ad0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff7180098b0 tx=0x7ff718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:30.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:30.827+0000 7ff71ea00640 1 -- 172.21.15.97:0/1777297770 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7200f1de0 con 0x7ff7280ce640 2024-09-10T21:32:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.027+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7280726e0 con 0x7ff7280ce640 2024-09-10T21:32:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.027+0000 7ff71ea00640 1 -- 172.21.15.97:0/1777297770 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) ==== 95+0+4395 (secure 0 0 0) 0x7ff720016970 con 0x7ff7280ce640 2024-09-10T21:32:31.028 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:31.028 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":167,"btime":"2024-09-10T21:32:30:659458+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2415223919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2415223919},{"type":"v1","addr":"172.21.15.97:6834","nonce":2415223919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:30.659391+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 167 2024-09-10T21:32:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff704072610 msgr2=0x7ff704074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff704072610 0x7ff704074ad0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff7180098b0 tx=0x7ff718008040 comp rx=0 tx=0).stop 2024-09-10T21:32:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280ce640 msgr2=0x7ff7280c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280ce640 0x7ff7280c5760 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ff720009bd0 tx=0x7ff720009300 comp rx=0 tx=0).stop 2024-09-10T21:32:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 shutdown_connections 2024-09-10T21:32:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff704072610 0x7ff704074ad0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7280ce640 0x7ff7280c5760 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7280b0d50 0x7ff72815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 --2- 172.21.15.97:0/1777297770 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7280afdd0 0x7ff728156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 >> 172.21.15.97:0/1777297770 conn(0x7ff7280d1c30 msgr2=0x7ff72806f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 shutdown_connections 2024-09-10T21:32:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.031+0000 7ff72da00640 1 -- 172.21.15.97:0/1777297770 wait complete. 2024-09-10T21:32:31.049 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:31.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 -- 172.21.15.97:0/1259554779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c1614f0 msgr2=0x7f4a9c0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 --2- 172.21.15.97:0/1259554779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c1614f0 0x7f4a9c0cf850 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0cf540 tx=0x7f4a8801c800 comp rx=0 tx=0).stop 2024-09-10T21:32:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 -- 172.21.15.97:0/1259554779 shutdown_connections 2024-09-10T21:32:31.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 --2- 172.21.15.97:0/1259554779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c1614f0 0x7f4a9c0cf850 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 --2- 172.21.15.97:0/1259554779 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a9c161110 0x7f4a9c0d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 -- 172.21.15.97:0/1259554779 >> 172.21.15.97:0/1259554779 conn(0x7f4a9c0d3ce0 msgr2=0x7f4a9c0d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:31.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 -- 172.21.15.97:0/1259554779 shutdown_connections 2024-09-10T21:32:31.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 -- 172.21.15.97:0/1259554779 wait complete. 2024-09-10T21:32:31.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.191+0000 7f4aa1200640 1 Processor -- start 2024-09-10T21:32:31.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 -- start start 2024-09-10T21:32:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c0d18b0 0x7f4a9c1f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c161110 0x7f4a9c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a9c1614f0 0x7f4a9c152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c16ac80 con 0x7f4a9c161110 2024-09-10T21:32:31.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b77d0 con 0x7f4a9c0d18b0 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c16ae00 con 0x7f4a9c1614f0 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c161110 0x7f4a9c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c161110 0x7f4a9c1531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39082/0 (socket says 172.21.15.97:39082) 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 -- 172.21.15.97:0/2356601812 learned_addr learned my addr 172.21.15.97:0/2356601812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9b400640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c0d18b0 0x7f4a9c1f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9be00640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a9c1614f0 0x7f4a9c152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 -- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a9c1614f0 msgr2=0x7f4a9c152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a9c1614f0 0x7f4a9c152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 -- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c0d18b0 msgr2=0x7f4a9c1f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c0d18b0 0x7f4a9c1f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 -- 172.21.15.97:0/2356601812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c153890 con 0x7f4a9c161110 2024-09-10T21:32:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9b400640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c0d18b0 0x7f4a9c1f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a9aa00640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c161110 0x7f4a9c1531b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4a880b0040 tx=0x7f4a880a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a93400640 1 -- 172.21.15.97:0/2356601812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880b2040 con 0x7f4a9c161110 2024-09-10T21:32:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a93400640 1 -- 172.21.15.97:0/2356601812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a880042c0 con 0x7f4a9c161110 2024-09-10T21:32:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4a93400640 1 -- 172.21.15.97:0/2356601812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880052d0 con 0x7f4a9c161110 2024-09-10T21:32:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c153b70 con 0x7f4a9c161110 2024-09-10T21:32:31.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c154130 con 0x7f4a9c161110 2024-09-10T21:32:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.195+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c06aea0 con 0x7f4a9c161110 2024-09-10T21:32:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.199+0000 7f4a93400640 1 -- 172.21.15.97:0/2356601812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a880046d0 con 0x7f4a9c161110 2024-09-10T21:32:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.199+0000 7f4a93400640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a7c0725e0 0x7f4a7c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.199+0000 7f4a93400640 1 -- 172.21.15.97:0/2356601812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7f4a8812cff0 con 0x7f4a9c161110 2024-09-10T21:32:31.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.199+0000 7f4a9b400640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a7c0725e0 0x7f4a7c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.199+0000 7f4a9b400640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a7c0725e0 0x7f4a7c074aa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009c30 tx=0x7f4a8c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:31.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.203+0000 7f4a93400640 1 -- 172.21.15.97:0/2356601812 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a880164b0 con 0x7f4a9c161110 2024-09-10T21:32:31.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.403+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a9c0d11f0 con 0x7f4a9c161110 2024-09-10T21:32:31.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.403+0000 7f4a93400640 1 -- 172.21.15.97:0/2356601812 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) ==== 95+0+4395 (secure 0 0 0) 0x7f4a880044b0 con 0x7f4a9c161110 2024-09-10T21:32:31.403 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:31.403 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":167,"btime":"2024-09-10T21:32:30:659458+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2415223919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2415223919},{"type":"v1","addr":"172.21.15.97:6834","nonce":2415223919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:30.659391+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:31.403 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 167 2024-09-10T21:32:31.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a7c0725e0 msgr2=0x7f4a7c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a7c0725e0 0x7f4a7c074aa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009c30 tx=0x7f4a8c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c161110 msgr2=0x7f4a9c1531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c161110 0x7f4a9c1531b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4a880b0040 tx=0x7f4a880a56d0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 shutdown_connections 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a7c0725e0 0x7f4a7c074aa0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a9c1614f0 0x7f4a9c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a9c161110 0x7f4a9c1531b0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 --2- 172.21.15.97:0/2356601812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c0d18b0 0x7f4a9c1f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 >> 172.21.15.97:0/2356601812 conn(0x7f4a9c0d3ce0 msgr2=0x7f4a9c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 shutdown_connections 2024-09-10T21:32:31.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.407+0000 7f4aa1200640 1 -- 172.21.15.97:0/2356601812 wait complete. 2024-09-10T21:32:31.425 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:31.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:31.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- 172.21.15.97:0/3565003030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ec003a10 msgr2=0x7fd6ec003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 --2- 172.21.15.97:0/3565003030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ec003a10 0x7fd6ec003e30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c290 tx=0x7fd6f801f700 comp rx=0 tx=0).stop 2024-09-10T21:32:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd707e00640 1 -- 172.21.15.97:0/3565003030 reap_dead start 2024-09-10T21:32:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- 172.21.15.97:0/3565003030 shutdown_connections 2024-09-10T21:32:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- 172.21.15.97:0/3565003030 >> 172.21.15.97:0/3565003030 conn(0x7fd7080b1780 msgr2=0x7fd7080b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- 172.21.15.97:0/3565003030 shutdown_connections 2024-09-10T21:32:31.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- 172.21.15.97:0/3565003030 wait complete. 2024-09-10T21:32:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 Processor -- start 2024-09-10T21:32:31.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- start start 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6ec003a10 0x7fd7081e9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7081f5350 0x7fd708149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd708152e80 0x7fd708153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd70806f3f0 con 0x7fd7081f5350 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd70806f270 con 0x7fd708152e80 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd70e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd70806e3b0 con 0x7fd6ec003a10 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7081f5350 0x7fd708149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd707400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6ec003a10 0x7fd7081e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7081f5350 0x7fd708149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39104/0 (socket says 172.21.15.97:39104) 2024-09-10T21:32:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 -- 172.21.15.97:0/138678333 learned_addr learned my addr 172.21.15.97:0/138678333 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:31.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd707e00640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd708152e80 0x7fd708153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 -- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6ec003a10 msgr2=0x7fd7081e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6ec003a10 0x7fd7081e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 -- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd708152e80 msgr2=0x7fd708153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd708152e80 0x7fd708153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.575+0000 7fd6fea00640 1 -- 172.21.15.97:0/138678333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7081e91e0 con 0x7fd7081f5350 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd6fea00640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7081f5350 0x7fd708149170 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0011830 tx=0x7fd6f0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd6ffe00640 1 -- 172.21.15.97:0/138678333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0004460 con 0x7fd7081f5350 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd6ffe00640 1 -- 172.21.15.97:0/138678333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6f0016070 con 0x7fd7081f5350 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd708149c00 con 0x7fd7081f5350 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd6ffe00640 1 -- 172.21.15.97:0/138678333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000fae0 con 0x7fd7081f5350 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7081f5860 con 0x7fd7081f5350 2024-09-10T21:32:31.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70806e940 con 0x7fd7081f5350 2024-09-10T21:32:31.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd6ffe00640 1 -- 172.21.15.97:0/138678333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd6f0023030 con 0x7fd7081f5350 2024-09-10T21:32:31.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd6ffe00640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6e4072610 0x7fd6e4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:31.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd6ffe00640 1 -- 172.21.15.97:0/138678333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7fd6f0094090 con 0x7fd7081f5350 2024-09-10T21:32:31.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.579+0000 7fd707400640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6e4072610 0x7fd6e4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:31.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.583+0000 7fd707400640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6e4072610 0x7fd6e4074ad0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80b0000 tx=0x7fd6f801c540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:31.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.583+0000 7fd6ffe00640 1 -- 172.21.15.97:0/138678333 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6f005ea20 con 0x7fd7081f5350 2024-09-10T21:32:31.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.783+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd70815d4d0 con 0x7fd7081f5350 2024-09-10T21:32:31.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.783+0000 7fd6ffe00640 1 -- 172.21.15.97:0/138678333 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) ==== 95+0+4395 (secure 0 0 0) 0x7fd6f0063450 con 0x7fd7081f5350 2024-09-10T21:32:31.785 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:31.786 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":167,"btime":"2024-09-10T21:32:30:659458+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2415223919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2415223919},{"type":"v1","addr":"172.21.15.97:6834","nonce":2415223919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:30.659391+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:31.787 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 167 2024-09-10T21:32:31.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6e4072610 msgr2=0x7fd6e4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6e4072610 0x7fd6e4074ad0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80b0000 tx=0x7fd6f801c540 comp rx=0 tx=0).stop 2024-09-10T21:32:31.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7081f5350 msgr2=0x7fd708149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:31.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7081f5350 0x7fd708149170 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0011830 tx=0x7fd6f0011cf0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 shutdown_connections 2024-09-10T21:32:31.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6e4072610 0x7fd6e4074ad0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd708152e80 0x7fd708153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7081f5350 0x7fd708149170 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 --2- 172.21.15.97:0/138678333 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6ec003a10 0x7fd7081e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:31.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 >> 172.21.15.97:0/138678333 conn(0x7fd7080b1780 msgr2=0x7fd7080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:31.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 shutdown_connections 2024-09-10T21:32:31.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:31.787+0000 7fd70e200640 1 -- 172.21.15.97:0/138678333 wait complete. 2024-09-10T21:32:31.805 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2024-09-10T21:32:31.806 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2024-09-10T21:32:31.806 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:31.806 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:31.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:32:31.815 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:31.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:31.815 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:31.866 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:32:31.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:32:31.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- 172.21.15.97:0/2705784202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c0003fe0 msgr2=0x7f74c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 --2- 172.21.15.97:0/2705784202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c0003fe0 0x7f74c0004810 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009b60 tx=0x7f74c401d300 comp rx=0 tx=0).stop 2024-09-10T21:32:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74db400640 1 -- 172.21.15.97:0/2705784202 reap_dead start 2024-09-10T21:32:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- 172.21.15.97:0/2705784202 shutdown_connections 2024-09-10T21:32:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- 172.21.15.97:0/2705784202 >> 172.21.15.97:0/2705784202 conn(0x7f74dc0b37c0 msgr2=0x7f74dc0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- 172.21.15.97:0/2705784202 shutdown_connections 2024-09-10T21:32:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- 172.21.15.97:0/2705784202 wait complete. 2024-09-10T21:32:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 Processor -- start 2024-09-10T21:32:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- start start 2024-09-10T21:32:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74c0003fe0 0x7f74dc1f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74dc0c67b0 0x7f74dc0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc0c7400 0x7f74dc1f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc0b4420 con 0x7f74dc0c7400 2024-09-10T21:32:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc0cde60 con 0x7f74c0003fe0 2024-09-10T21:32:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.051+0000 7f74e1600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f74dc0ce1a0 con 0x7f74dc0c67b0 2024-09-10T21:32:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc0c7400 0x7f74dc1f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc0c7400 0x7f74dc1f4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39124/0 (socket says 172.21.15.97:39124) 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 -- 172.21.15.97:0/352115074 learned_addr learned my addr 172.21.15.97:0/352115074 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74daa00640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74c0003fe0 0x7f74dc1f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74d2000640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74dc0c67b0 0x7f74dc0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 -- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74dc0c67b0 msgr2=0x7f74dc0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74dc0c67b0 0x7f74dc0c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 -- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74c0003fe0 msgr2=0x7f74dc1f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74c0003fe0 0x7f74dc1f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 -- 172.21.15.97:0/352115074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc072810 con 0x7f74dc0c7400 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74d2000640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74dc0c67b0 0x7f74dc0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:32.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74db400640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc0c7400 0x7f74dc1f4df0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0d0660 tx=0x7f74d400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74d3400640 1 -- 172.21.15.97:0/352115074 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d400c9a0 con 0x7f74dc0c7400 2024-09-10T21:32:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74d3400640 1 -- 172.21.15.97:0/352115074 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74d4018070 con 0x7f74dc0c7400 2024-09-10T21:32:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74d3400640 1 -- 172.21.15.97:0/352115074 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4011460 con 0x7f74dc0c7400 2024-09-10T21:32:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc1f8440 con 0x7f74dc0c7400 2024-09-10T21:32:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc1f8a00 con 0x7f74dc0c7400 2024-09-10T21:32:32.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.055+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74dc0c7ce0 con 0x7f74dc0c7400 2024-09-10T21:32:32.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.059+0000 7f74d3400640 1 -- 172.21.15.97:0/352115074 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f74d4011600 con 0x7f74dc0c7400 2024-09-10T21:32:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.059+0000 7f74d3400640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74b80725e0 0x7f74b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.059+0000 7f74d3400640 1 -- 172.21.15.97:0/352115074 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7f74d4094780 con 0x7f74dc0c7400 2024-09-10T21:32:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.059+0000 7f74daa00640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74b80725e0 0x7f74b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.059+0000 7f74daa00640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74b80725e0 0x7f74b8074aa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0cc1c0 tx=0x7f74cc0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:32.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.063+0000 7f74d3400640 1 -- 172.21.15.97:0/352115074 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f74d405f850 con 0x7f74dc0c7400 2024-09-10T21:32:32.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.259+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f74dc0b0000 con 0x7f74dc0c7400 2024-09-10T21:32:32.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.651+0000 7f74d3400640 1 -- 172.21.15.97:0/352115074 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v168) ==== 161+0+0 (secure 0 0 0) 0x7f74d405f150 con 0x7f74dc0c7400 2024-09-10T21:32:32.653 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74b80725e0 msgr2=0x7f74b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74b80725e0 0x7f74b8074aa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0cc1c0 tx=0x7f74cc0091e0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc0c7400 msgr2=0x7f74dc1f4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc0c7400 0x7f74dc1f4df0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0d0660 tx=0x7f74d400ed10 comp rx=0 tx=0).stop 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 shutdown_connections 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f74b80725e0 0x7f74b8074aa0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74dc0c7400 0x7f74dc1f4df0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74dc0c67b0 0x7f74dc0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 --2- 172.21.15.97:0/352115074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74c0003fe0 0x7f74dc1f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 >> 172.21.15.97:0/352115074 conn(0x7f74dc0b37c0 msgr2=0x7f74dc0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 shutdown_connections 2024-09-10T21:32:32.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.655+0000 7f74e1600640 1 -- 172.21.15.97:0/352115074 wait complete. 2024-09-10T21:32:32.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:32.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 -- 172.21.15.97:0/1868879400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d0015bdd0 msgr2=0x7f1d00074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 --2- 172.21.15.97:0/1868879400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d0015bdd0 0x7f1d00074380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009900 tx=0x7f1cf001c8b0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 -- 172.21.15.97:0/1868879400 shutdown_connections 2024-09-10T21:32:32.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 --2- 172.21.15.97:0/1868879400 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d000b0110 0x7f1d000cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 --2- 172.21.15.97:0/1868879400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d0015bdd0 0x7f1d00074380 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 -- 172.21.15.97:0/1868879400 >> 172.21.15.97:0/1868879400 conn(0x7f1d000b28c0 msgr2=0x7f1d000b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:32.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 -- 172.21.15.97:0/1868879400 shutdown_connections 2024-09-10T21:32:32.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 -- 172.21.15.97:0/1868879400 wait complete. 2024-09-10T21:32:32.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.819+0000 7f1d05400640 1 Processor -- start 2024-09-10T21:32:32.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 -- start start 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d000b0110 0x7f1d0014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d000cfad0 0x7f1d0015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d0015bdd0 0x7f1d0015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d00168b30 con 0x7f1d000cfad0 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001689b0 con 0x7f1d000b0110 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1d00168cb0 con 0x7f1d0015bdd0 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d000cfad0 0x7f1d0015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d000cfad0 0x7f1d0015a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39128/0 (socket says 172.21.15.97:39128) 2024-09-10T21:32:32.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 -- 172.21.15.97:0/1713496417 learned_addr learned my addr 172.21.15.97:0/1713496417 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cfe800640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d000b0110 0x7f1d0014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cff200640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d0015bdd0 0x7f1d0015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 -- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d0015bdd0 msgr2=0x7f1d0015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d0015bdd0 0x7f1d0015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 -- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d000b0110 msgr2=0x7f1d0014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d000b0110 0x7f1d0014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 -- 172.21.15.97:0/1713496417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0015b450 con 0x7f1d000cfad0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf5e00640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d000cfad0 0x7f1d0015a110 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800c8a0 tx=0x7f1ce800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf7400640 1 -- 172.21.15.97:0/1713496417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8003b80 con 0x7f1d000cfad0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf7400640 1 -- 172.21.15.97:0/1713496417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ce8003d40 con 0x7f1d000cfad0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d00159350 con 0x7f1d000cfad0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1cf7400640 1 -- 172.21.15.97:0/1713496417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800aab0 con 0x7f1d000cfad0 2024-09-10T21:32:32.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d00159910 con 0x7f1d000cfad0 2024-09-10T21:32:32.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.823+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d000cff40 con 0x7f1d000cfad0 2024-09-10T21:32:32.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.827+0000 7f1cf7400640 1 -- 172.21.15.97:0/1713496417 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ce800ac50 con 0x7f1d000cfad0 2024-09-10T21:32:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.827+0000 7f1cf7400640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cdc0725e0 0x7f1cdc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.827+0000 7f1cf7400640 1 -- 172.21.15.97:0/1713496417 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7f1ce8093790 con 0x7f1d000cfad0 2024-09-10T21:32:32.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.827+0000 7f1cfe800640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cdc0725e0 0x7f1cdc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:32.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.827+0000 7f1cfe800640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cdc0725e0 0x7f1cdc074aa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00af000 tx=0x7f1cf001b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:32.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:32.831+0000 7f1cf7400640 1 -- 172.21.15.97:0/1713496417 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ce805e090 con 0x7f1d000cfad0 2024-09-10T21:32:33.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.059+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f1d000b3f10 con 0x7f1d000cfad0 2024-09-10T21:32:33.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1cf7400640 1 -- 172.21.15.97:0/1713496417 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v169) ==== 101+0+0 (secure 0 0 0) 0x7f1ce8062ac0 con 0x7f1d000cfad0 2024-09-10T21:32:33.662 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cdc0725e0 msgr2=0x7f1cdc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cdc0725e0 0x7f1cdc074aa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00af000 tx=0x7f1cf001b0f0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d000cfad0 msgr2=0x7f1d0015a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d000cfad0 0x7f1d0015a110 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800c8a0 tx=0x7f1ce800cd60 comp rx=0 tx=0).stop 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 shutdown_connections 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1cdc0725e0 0x7f1cdc074aa0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d0015bdd0 0x7f1d0015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d000cfad0 0x7f1d0015a110 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 --2- 172.21.15.97:0/1713496417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d000b0110 0x7f1d0014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 >> 172.21.15.97:0/1713496417 conn(0x7f1d000b28c0 msgr2=0x7f1d00071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:33.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 shutdown_connections 2024-09-10T21:32:33.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.663+0000 7f1d05400640 1 -- 172.21.15.97:0/1713496417 wait complete. 2024-09-10T21:32:33.684 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:33.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.727+0000 7fec9f228080 1 Processor -- start 2024-09-10T21:32:33.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- start start 2024-09-10T21:32:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec103940 0x560fec1a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec18db00 0x560fec1ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18e2a0 0x560fec1ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fec022a00 con 0x560fec103940 2024-09-10T21:32:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560fec080f40 con 0x560fec18db00 2024-09-10T21:32:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560fec190080 con 0x560fec18e2a0 2024-09-10T21:32:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec103940 0x560fec1a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec103940 0x560fec1a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39142/0 (socket says 172.21.15.97:39142) 2024-09-10T21:32:33.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 -- 172.21.15.97:0/2740043180 learned_addr learned my addr 172.21.15.97:0/2740043180 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:33.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec93a00640 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec18db00 0x560fec1ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9ce00640 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18e2a0 0x560fec1ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 -- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18e2a0 msgr2=0x560fec1ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:33.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18e2a0 0x560fec1ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 -- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec18db00 msgr2=0x560fec1ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:33.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec18db00 0x560fec1ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 -- 172.21.15.97:0/2740043180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560febffbc10 con 0x560fec103940 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9c400640 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec103940 0x560fec1a70a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b2a0 tx=0x7fec8c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8baa331c7783b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9b000640 1 -- 172.21.15.97:0/2740043180 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00a040 con 0x560fec103940 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9b000640 1 -- 172.21.15.97:0/2740043180 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec8c00f620 con 0x560fec103940 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9b000640 1 -- 172.21.15.97:0/2740043180 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00fa70 con 0x560fec103940 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- 172.21.15.97:0/2740043180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec103940 msgr2=0x560fec1a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec103940 0x560fec1a70a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00b2a0 tx=0x7fec8c00b760 comp rx=0 tx=0).stop 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- 172.21.15.97:0/2740043180 shutdown_connections 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18e2a0 0x560fec1ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec18db00 0x560fec1ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 --2- 172.21.15.97:0/2740043180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec103940 0x560fec1a70a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- 172.21.15.97:0/2740043180 >> 172.21.15.97:0/2740043180 conn(0x560fec102520 msgr2=0x560fec1aa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- 172.21.15.97:0/2740043180 shutdown_connections 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- 172.21.15.97:0/2740043180 wait complete. 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 Processor -- start 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.731+0000 7fec9f228080 1 -- start start 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec103940 0x560fec1a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18db00 0x560fec18cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec18e2a0 0x560fec18d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fec106c90 con 0x560fec18e2a0 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560fec105660 con 0x560fec103940 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560fec106ef0 con 0x560fec18db00 2024-09-10T21:32:33.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec18e2a0 0x560fec18d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec18e2a0 0x560fec18d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39156/0 (socket says 172.21.15.97:39156) 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 learned_addr learned my addr 172.21.15.97:0/3404593641 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9c400640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec103940 0x560fec1a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec93a00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18db00 0x560fec18cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18db00 msgr2=0x560fec18cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560fec18db00 0x560fec18cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec103940 msgr2=0x560fec1a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560fec103940 0x560fec1a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fec23db70 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9ce00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec18e2a0 0x560fec18d2c0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fec940ad040 tx=0x7fec940049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec99c00640 1 -- 172.21.15.97:0/3404593641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec940ba030 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec99c00640 1 -- 172.21.15.97:0/3404593641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec940b0040 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec99c00640 1 -- 172.21.15.97:0/3404593641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec940af040 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560fec23de10 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.735+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fec23e350 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec99c00640 1 -- 172.21.15.97:0/3404593641 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fec940b5040 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec99c00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fec78070fc0 0x7fec78073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec99c00640 1 -- 172.21.15.97:0/3404593641 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7fec9412d1b0 con 0x560fec18e2a0 2024-09-10T21:32:33.744 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:33.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec9c400640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fec78070fc0 0x7fec78073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560fec23ea90 0x560fec23ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec23f3b0 con 0x560fec23ea90 2024-09-10T21:32:33.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec93a00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560fec23ea90 0x560fec23ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec9c400640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fec78070fc0 0x7fec78073480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00c010 tx=0x7fec8c000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.739+0000 7fec93a00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560fec23ea90 0x560fec23ee70 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.743+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v110'412 uv386 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.743+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec23f780 con 0x560fec23ea90 2024-09-10T21:32:33.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.743+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v110'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.743+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec23fb20 con 0x560fec23ea90 2024-09-10T21:32:33.749 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.743+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v110'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.743+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2518d0 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.747+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.747+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec251c70 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.747+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.747+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2508c0 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.751+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.751+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.751+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v110'418 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.751+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.751+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v110'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.751+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.753 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.755+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v110'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.755+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.755+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'421 uv421 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.755+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.759+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.759+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.759+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.759+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.759+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v110'424 uv399 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.759+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.763+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'425 uv425 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.763+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.763 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.763+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v110'426 uv426 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.763+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.763+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'427 uv427 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.763+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.767+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'428 uv428 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.767+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250cb0 con 0x560fec23ea90 2024-09-10T21:32:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.767+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'429 uv429 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.767+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2510d0 con 0x560fec23ea90 2024-09-10T21:32:33.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.767+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'430 uv430 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.771+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.771+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v110'431 uv405 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.771+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.771+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v110'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.771+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.775+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v110'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.773 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:33.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.775+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.775+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.775+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.775+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.775+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.779+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.779+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.779+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v110'437 uv411 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.779+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.783+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v110'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.783+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.782 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.783+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v110'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.783+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.783+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.783+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.787+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.787+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec250f30 con 0x560fec23ea90 2024-09-10T21:32:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.787+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'442 uv442 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec23ea90 2024-09-10T21:32:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.787+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560fec252010 0x560fec254400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.787+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec254940 con 0x560fec252010 2024-09-10T21:32:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.787+0000 7fec9c400640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560fec252010 0x560fec254400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.787+0000 7fec9c400640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560fec252010 0x560fec254400 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.791+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'308 uv291 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.791+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec254ce0 con 0x560fec252010 2024-09-10T21:32:33.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.791+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'309 uv309 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.791+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec255080 con 0x560fec252010 2024-09-10T21:32:33.792 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.791+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'310 uv310 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.795+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2558d0 con 0x560fec252010 2024-09-10T21:32:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.795+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v110'311 uv311 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.795+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec255c70 con 0x560fec252010 2024-09-10T21:32:33.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.795+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'312 uv312 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.795+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec256010 con 0x560fec252010 2024-09-10T21:32:33.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.799+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v110'313 uv296 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.799+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.799+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v110'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.799+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.799+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v110'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.801 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:33.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.799+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.803+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.803+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.803+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.803+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.803+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v110'318 uv301 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.803+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.807+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v110'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.807+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.807+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v110'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.807 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:33.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.807+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25a580 con 0x560fec252010 2024-09-10T21:32:33.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v110'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25a580 con 0x560fec252010 2024-09-10T21:32:33.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v110'323 uv307 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v110'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.811+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.815+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v110'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.813 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:33.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.815+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2573c0 con 0x560fec252010 2024-09-10T21:32:33.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.815+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.815+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2573c0 con 0x560fec252010 2024-09-10T21:32:33.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.815+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.815+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2563b0 con 0x560fec252010 2024-09-10T21:32:33.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.819+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec252010 2024-09-10T21:32:33.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.819+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560fec256e20 0x560fec257220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.819+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec259810 con 0x560fec256e20 2024-09-10T21:32:33.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.819+0000 7fec93a00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560fec256e20 0x560fec257220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.819+0000 7fec93a00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560fec256e20 0x560fec257220 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.819+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.819+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec259810 con 0x560fec256e20 2024-09-10T21:32:33.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.823+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.823+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec259810 con 0x560fec256e20 2024-09-10T21:32:33.822 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:33.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.823+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.823+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25a820 con 0x560fec256e20 2024-09-10T21:32:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.823+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.823+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25a820 con 0x560fec256e20 2024-09-10T21:32:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec259810 con 0x560fec256e20 2024-09-10T21:32:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560fec259ee0 0x560fec25c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25c9e0 con 0x560fec259ee0 2024-09-10T21:32:33.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec9ce00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560fec259ee0 0x560fec25c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec9ce00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560fec259ee0 0x560fec25c310 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'67 uv66 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.827+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec25cef0 con 0x560fec259ee0 2024-09-10T21:32:33.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.831+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.831+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25d3f0 con 0x560fec259ee0 2024-09-10T21:32:33.830 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:33.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.831+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.831+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25dcb0 con 0x560fec259ee0 2024-09-10T21:32:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.831+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.831+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25e050 con 0x560fec256e20 2024-09-10T21:32:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec25e050 con 0x560fec256e20 2024-09-10T21:32:33.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25e050 con 0x560fec256e20 2024-09-10T21:32:33.835 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25f060 con 0x560fec256e20 2024-09-10T21:32:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.835+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25f060 con 0x560fec256e20 2024-09-10T21:32:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.839+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.839+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25e050 con 0x560fec256e20 2024-09-10T21:32:33.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.839+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.839+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25e050 con 0x560fec259ee0 2024-09-10T21:32:33.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.839+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v110'149 uv143 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.839+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec25e3f0 con 0x560fec259ee0 2024-09-10T21:32:33.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.843+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v110'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.843+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25e790 con 0x560fec259ee0 2024-09-10T21:32:33.842 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:33.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.843+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v110'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.843+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25fb40 con 0x560fec259ee0 2024-09-10T21:32:33.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.843+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.843+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25fb40 con 0x560fec259ee0 2024-09-10T21:32:33.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.847+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.847+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec259ee0 2024-09-10T21:32:33.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.847+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.847+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec259ee0 2024-09-10T21:32:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.847+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v110'155 uv148 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.847+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec259ee0 2024-09-10T21:32:33.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v110'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec259ee0 2024-09-10T21:32:33.850 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:33.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v110'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec259ee0 2024-09-10T21:32:33.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec259ee0 2024-09-10T21:32:33.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.851+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec256e20 2024-09-10T21:32:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.855+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v110'66 uv65 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.855+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec256e20 2024-09-10T21:32:33.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.855+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v110'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.855+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec256e20 2024-09-10T21:32:33.855 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:33.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.855+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v110'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.855+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec256e20 2024-09-10T21:32:33.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'69 uv69 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec25eb30 con 0x560fec256e20 2024-09-10T21:32:33.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'70 uv70 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec256e20 2024-09-10T21:32:33.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560fec25f1b0 0x560fec2615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec261b70 con 0x560fec25f1b0 2024-09-10T21:32:33.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec9c400640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560fec25f1b0 0x560fec2615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec9c400640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560fec25f1b0 0x560fec2615a0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.859+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec261c30 con 0x560fec25f1b0 2024-09-10T21:32:33.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.863+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.863+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.862 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:33.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.863+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.863+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec263060 con 0x560fec25f1b0 2024-09-10T21:32:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.863+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.863+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec263060 con 0x560fec25f1b0 2024-09-10T21:32:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.867+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.867+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.867+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.867+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.867+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v110'132 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.867+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v110'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.869 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v110'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v110'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v110'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.871+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.875+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v110'137 uv131 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.875+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.875+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v110'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.875+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.875 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:33.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.875+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v110'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.875+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262050 con 0x560fec25f1b0 2024-09-10T21:32:33.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v110'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec264e60 con 0x560fec25f1b0 2024-09-10T21:32:33.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v110'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v110'61 uv60 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v110'62 uv62 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.879+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.881 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:33.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.883+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v110'63 uv63 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.883+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.883+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v110'64 uv64 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.883+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.883+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v110'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.883+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v110'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.886 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v110'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec263510 con 0x560fec259ee0 2024-09-10T21:32:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v110'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec263510 con 0x560fec259ee0 2024-09-10T21:32:33.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v110'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.887+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec262500 con 0x560fec259ee0 2024-09-10T21:32:33.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v110'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec940c6070 con 0x560fec259ee0 2024-09-10T21:32:33.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560fec262b80 0x560fec262f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:33.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec262b80 2024-09-10T21:32:33.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec93a00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560fec262b80 0x560fec262f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:33.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec93a00640 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560fec262b80 0x560fec262f60 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:33.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v110'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec262b80 2024-09-10T21:32:33.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec262b80 2024-09-10T21:32:33.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v110'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec262b80 2024-09-10T21:32:33.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.891+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec262b80 2024-09-10T21:32:33.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.895+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v110'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec262b80 2024-09-10T21:32:33.893 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:33.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.895+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec262b80 2024-09-10T21:32:33.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.895+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v110'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec262b80 2024-09-10T21:32:33.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.895+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec262b80 2024-09-10T21:32:33.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.895+0000 7fec93a00640 1 -- 172.21.15.97:0/3404593641 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v110'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec8400c730 con 0x560fec262b80 2024-09-10T21:32:33.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.895+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.896 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:33.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v110'170 uv170 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v110'171 uv171 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v110'172 uv172 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 1.00000000 [create] v110'173 uv168 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.899+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.903+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 1.00000000 [delete] v110'174 uv174 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.903+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.903 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:33.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.903+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v110'175 uv175 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.903+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.903+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v110'176 uv176 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.903+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v110'177 uv177 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec2634a0 con 0x560fec25f1b0 2024-09-10T21:32:33.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v110'178 uv178 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x560fec264a30 con 0x560fec25f1b0 2024-09-10T21:32:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9c400640 1 -- 172.21.15.97:0/3404593641 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v110'179 uv179 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fec8c00d040 con 0x560fec25f1b0 2024-09-10T21:32:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560fec256e20 msgr2=0x560fec257220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560fec256e20 0x560fec257220 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560fec25f1b0 msgr2=0x560fec2615a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560fec25f1b0 0x560fec2615a0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560fec262b80 msgr2=0x560fec262f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560fec262b80 0x560fec262f60 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560fec259ee0 msgr2=0x560fec25c310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560fec259ee0 0x560fec25c310 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560fec252010 msgr2=0x560fec254400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 reap_dead start 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560fec252010 0x560fec254400 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9ce00640 1 -- 172.21.15.97:0/3404593641 reap_dead start 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560fec23ea90 msgr2=0x560fec23ee70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560fec23ea90 0x560fec23ee70 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fec78070fc0 msgr2=0x7fec78073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fec78070fc0 0x7fec78073480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00c010 tx=0x7fec8c000ee0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec18e2a0 msgr2=0x560fec18d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec18e2a0 0x560fec18d2c0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fec940ad040 tx=0x7fec940049d0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 shutdown_connections 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560fec23ea90 0x560fec23ee70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fec78070fc0 0x7fec78073480 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 --2- 172.21.15.97:0/3404593641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fec18e2a0 0x560fec18d2c0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 >> 172.21.15.97:0/3404593641 conn(0x560fec102520 msgr2=0x560fec120c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 shutdown_connections 2024-09-10T21:32:33.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:33.907+0000 7fec9f228080 1 -- 172.21.15.97:0/3404593641 wait complete. 2024-09-10T21:32:33.917 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2024-09-10T21:32:34.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 Processor -- start 2024-09-10T21:32:34.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- start start 2024-09-10T21:32:34.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cbcc0 0x556bdd65d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cda00 0x556bdd6e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 0x556bdd6e8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556bdd561a00 con 0x556bdd6cda00 2024-09-10T21:32:34.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556bdd5bff40 con 0x556bdd6cbcc0 2024-09-10T21:32:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556bdd6cf710 con 0x556bdd6d11d0 2024-09-10T21:32:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cda00 0x556bdd6e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cda00 0x556bdd6e7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39164/0 (socket says 172.21.15.97:39164) 2024-09-10T21:32:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 0x556bdd6e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 -- 172.21.15.97:0/582571408 learned_addr learned my addr 172.21.15.97:0/582571408 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749e000640 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cbcc0 0x556bdd65d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 -- 172.21.15.97:0/582571408 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 msgr2=0x556bdd6e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 0x556bdd6e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 -- 172.21.15.97:0/582571408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cbcc0 msgr2=0x556bdd65d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cbcc0 0x556bdd65d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 -- 172.21.15.97:0/582571408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bdd53ac10 con 0x556bdd6cda00 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f749d600640 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cda00 0x556bdd6e7ef0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f748c00b2b0 tx=0x7f748c00b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25c2085f3496100 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f7497e00640 1 -- 172.21.15.97:0/582571408 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c016070 con 0x556bdd6cda00 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f7497e00640 1 -- 172.21.15.97:0/582571408 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f748c00d4c0 con 0x556bdd6cda00 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f7497e00640 1 -- 172.21.15.97:0/582571408 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c00d6f0 con 0x556bdd6cda00 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- 172.21.15.97:0/582571408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cda00 msgr2=0x556bdd6e7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cda00 0x556bdd6e7ef0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f748c00b2b0 tx=0x7f748c00b770 comp rx=0 tx=0).stop 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- 172.21.15.97:0/582571408 shutdown_connections 2024-09-10T21:32:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 0x556bdd6e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cda00 0x556bdd6e7ef0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 --2- 172.21.15.97:0/582571408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cbcc0 0x556bdd65d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- 172.21.15.97:0/582571408 >> 172.21.15.97:0/582571408 conn(0x556bdd65b0d0 msgr2=0x556bdd616360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- 172.21.15.97:0/582571408 shutdown_connections 2024-09-10T21:32:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.007+0000 7f74a0da0080 1 -- 172.21.15.97:0/582571408 wait complete. 2024-09-10T21:32:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 Processor -- start 2024-09-10T21:32:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 -- start start 2024-09-10T21:32:34.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cbcc0 0x556bdd7858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cda00 0x556bdd659f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 0x556bdd65a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556bdd644520 con 0x556bdd6cbcc0 2024-09-10T21:32:34.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556bdd644280 con 0x556bdd6cda00 2024-09-10T21:32:34.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556bdd6ea830 con 0x556bdd6d11d0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cbcc0 0x556bdd7858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cbcc0 0x556bdd7858d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39178/0 (socket says 172.21.15.97:39178) 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 -- 172.21.15.97:0/4005013298 learned_addr learned my addr 172.21.15.97:0/4005013298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749ea00640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 0x556bdd65a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749d600640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cda00 0x556bdd659f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 -- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 msgr2=0x556bdd65a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556bdd6d11d0 0x556bdd65a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 -- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cda00 msgr2=0x556bdd659f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bdd6cda00 0x556bdd659f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 -- 172.21.15.97:0/4005013298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bdd6ecb10 con 0x556bdd6cbcc0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f749e000640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cbcc0 0x556bdd7858d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7490004fd0 tx=0x7f749000abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f7496a00640 1 -- 172.21.15.97:0/4005013298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490009d90 con 0x556bdd6cbcc0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f7496a00640 1 -- 172.21.15.97:0/4005013298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f749000ae10 con 0x556bdd6cbcc0 2024-09-10T21:32:34.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f7496a00640 1 -- 172.21.15.97:0/4005013298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74900086a0 con 0x556bdd6cbcc0 2024-09-10T21:32:34.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556bdd6ecdb0 con 0x556bdd6cbcc0 2024-09-10T21:32:34.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.011+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bdd6ed2f0 con 0x556bdd6cbcc0 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f7496a00640 1 -- 172.21.15.97:0/4005013298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7490008840 con 0x556bdd6cbcc0 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f7496a00640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480070ff0 0x7f74800734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f7496a00640 1 -- 172.21.15.97:0/4005013298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7f7490093150 con 0x556bdd6cbcc0 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f749d600640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480070ff0 0x7f74800734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f74a0da0080 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556bdd612140 0x556bdd612520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x556bdd612a60 con 0x556bdd612140 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f749ea00640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556bdd612140 0x556bdd612520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f749d600640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480070ff0 0x7f74800734b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f748c000c10 tx=0x7f748c001090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f749ea00640 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556bdd612140 0x556bdd612520 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f749ea00640 1 -- 172.21.15.97:0/4005013298 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 1.00000000 [delete] v110'180 uv180 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7498002050 con 0x556bdd612140 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556bdd612140 msgr2=0x556bdd612520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f74a0da0080 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556bdd612140 0x556bdd612520 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480070ff0 msgr2=0x7f74800734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.015+0000 7f74a0da0080 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480070ff0 0x7f74800734b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f748c000c10 tx=0x7f748c001090 comp rx=0 tx=0).stop 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.019+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cbcc0 msgr2=0x556bdd7858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.019+0000 7f74a0da0080 1 --2- 172.21.15.97:0/4005013298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bdd6cbcc0 0x556bdd7858d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7490004fd0 tx=0x7f749000abc0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.019+0000 7f749ea00640 1 -- 172.21.15.97:0/4005013298 reap_dead start 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.019+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 shutdown_connections 2024-09-10T21:32:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.019+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 >> 172.21.15.97:0/4005013298 conn(0x556bdd65b0d0 msgr2=0x556bdd616360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.019+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 shutdown_connections 2024-09-10T21:32:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.019+0000 7f74a0da0080 1 -- 172.21.15.97:0/4005013298 wait complete. 2024-09-10T21:32:34.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 -- 172.21.15.97:0/2432523681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c15fde0 msgr2=0x7f433c13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 --2- 172.21.15.97:0/2432523681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c15fde0 0x7f433c13f9b0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4334009c80 tx=0x7f433401cda0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 -- 172.21.15.97:0/2432523681 shutdown_connections 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 --2- 172.21.15.97:0/2432523681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c15fde0 0x7f433c13f9b0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 --2- 172.21.15.97:0/2432523681 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f433c0ce5f0 0x7f433c141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 -- 172.21.15.97:0/2432523681 >> 172.21.15.97:0/2432523681 conn(0x7f433c0b1780 msgr2=0x7f433c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 -- 172.21.15.97:0/2432523681 shutdown_connections 2024-09-10T21:32:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.207+0000 7f4341800640 1 -- 172.21.15.97:0/2432523681 wait complete. 2024-09-10T21:32:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 Processor -- start 2024-09-10T21:32:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 -- start start 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433c0ce5f0 0x7f433c15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0ce9d0 0x7f433c156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f433c15fde0 0x7f433c153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f433c168bf0 con 0x7f433c0ce9d0 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f433c168a70 con 0x7f433c0ce5f0 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f433c168d70 con 0x7f433c15fde0 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0ce9d0 0x7f433c156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0ce9d0 0x7f433c156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39198/0 (socket says 172.21.15.97:39198) 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 -- 172.21.15.97:0/2615297044 learned_addr learned my addr 172.21.15.97:0/2615297044 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f433aa00640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433c0ce5f0 0x7f433c15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f433b400640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f433c15fde0 0x7f433c153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 -- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f433c15fde0 msgr2=0x7f433c153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f433c15fde0 0x7f433c153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 -- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433c0ce5f0 msgr2=0x7f433c15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433c0ce5f0 0x7f433c15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 -- 172.21.15.97:0/2615297044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c1f8c20 con 0x7f433c0ce9d0 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f433aa00640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433c0ce5f0 0x7f433c15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f433b400640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f433c15fde0 0x7f433c153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4332000640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0ce9d0 0x7f433c156f80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f432400c480 tx=0x7f432400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4333400640 1 -- 172.21.15.97:0/2615297044 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240052d0 con 0x7f433c0ce9d0 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4333400640 1 -- 172.21.15.97:0/2615297044 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4324005470 con 0x7f433c0ce9d0 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c074630 con 0x7f433c0ce9d0 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4333400640 1 -- 172.21.15.97:0/2615297044 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324012870 con 0x7f433c0ce9d0 2024-09-10T21:32:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.211+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c1f9450 con 0x7f433c0ce9d0 2024-09-10T21:32:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.215+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c1f9620 con 0x7f433c0ce9d0 2024-09-10T21:32:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.215+0000 7f4333400640 1 -- 172.21.15.97:0/2615297044 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4324003600 con 0x7f433c0ce9d0 2024-09-10T21:32:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.215+0000 7f4333400640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43180725e0 0x7f4318074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.215+0000 7f4333400640 1 -- 172.21.15.97:0/2615297044 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7f43240935a0 con 0x7f433c0ce9d0 2024-09-10T21:32:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.215+0000 7f433aa00640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43180725e0 0x7f4318074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.215+0000 7f433aa00640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43180725e0 0x7f4318074aa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f432c0058d0 tx=0x7f432c005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.219+0000 7f4333400640 1 -- 172.21.15.97:0/2615297044 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f432405e580 con 0x7f433c0ce9d0 2024-09-10T21:32:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.419+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f433c141360 con 0x7f433c0ce9d0 2024-09-10T21:32:34.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.667+0000 7f4333400640 1 -- 172.21.15.97:0/2615297044 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v170) ==== 162+0+0 (secure 0 0 0) 0x7f432405de80 con 0x7f433c0ce9d0 2024-09-10T21:32:34.668 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43180725e0 msgr2=0x7f4318074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43180725e0 0x7f4318074aa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f432c0058d0 tx=0x7f432c005860 comp rx=0 tx=0).stop 2024-09-10T21:32:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0ce9d0 msgr2=0x7f433c156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0ce9d0 0x7f433c156f80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f432400c480 tx=0x7f432400c940 comp rx=0 tx=0).stop 2024-09-10T21:32:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 shutdown_connections 2024-09-10T21:32:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43180725e0 0x7f4318074aa0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f433c15fde0 0x7f433c153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f433c0ce9d0 0x7f433c156f80 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 --2- 172.21.15.97:0/2615297044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f433c0ce5f0 0x7f433c15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 >> 172.21.15.97:0/2615297044 conn(0x7f433c0b1780 msgr2=0x7f433c1555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 shutdown_connections 2024-09-10T21:32:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.671+0000 7f4341800640 1 -- 172.21.15.97:0/2615297044 wait complete. 2024-09-10T21:32:34.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:34.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 -- 172.21.15.97:0/1105814739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 msgr2=0x7f732c1622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 --2- 172.21.15.97:0/1105814739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1622f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f732c0cf190 tx=0x7f732401caf0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 -- 172.21.15.97:0/1105814739 shutdown_connections 2024-09-10T21:32:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 --2- 172.21.15.97:0/1105814739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1622f0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 --2- 172.21.15.97:0/1105814739 >> v2:172.21.15.194:3300/0 conn(0x7f732c0b1e00 0x7f732c161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 -- 172.21.15.97:0/1105814739 >> 172.21.15.97:0/1105814739 conn(0x7f732c0b37c0 msgr2=0x7f732c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 -- 172.21.15.97:0/1105814739 shutdown_connections 2024-09-10T21:32:34.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.835+0000 7f7332a00640 1 -- 172.21.15.97:0/1105814739 wait complete. 2024-09-10T21:32:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 Processor -- start 2024-09-10T21:32:34.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 -- start start 2024-09-10T21:32:34.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f732c0b2bb0 0x7f732c151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f732c15b7f0 0x7f732c15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f732c16b080 con 0x7f732c0b42e0 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f732c0d15e0 con 0x7f732c15b7f0 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f732c16b200 con 0x7f732c0b2bb0 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39216/0 (socket says 172.21.15.97:39216) 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 -- 172.21.15.97:0/837702271 learned_addr learned my addr 172.21.15.97:0/837702271 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7330a00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f732c15b7f0 0x7f732c15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732be00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f732c0b2bb0 0x7f732c151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 -- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f732c0b2bb0 msgr2=0x7f732c151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f732c0b2bb0 0x7f732c151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 -- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f732c15b7f0 msgr2=0x7f732c15bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:34.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f732c15b7f0 0x7f732c15bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:34.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 -- 172.21.15.97:0/837702271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c143c00 con 0x7f732c0b42e0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f732b400640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1436c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7318005bd0 tx=0x7f731800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7323e00640 1 -- 172.21.15.97:0/837702271 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318009c30 con 0x7f732c0b42e0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7323e00640 1 -- 172.21.15.97:0/837702271 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7318004030 con 0x7f732c0b42e0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7323e00640 1 -- 172.21.15.97:0/837702271 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318004460 con 0x7f732c0b42e0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.839+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c15afe0 con 0x7f732c0b42e0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.843+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c142070 con 0x7f732c0b42e0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.843+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c142270 con 0x7f732c0b42e0 2024-09-10T21:32:34.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.843+0000 7f7323e00640 1 -- 172.21.15.97:0/837702271 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7318003600 con 0x7f732c0b42e0 2024-09-10T21:32:34.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.843+0000 7f7323e00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f730c0725e0 0x7f730c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:34.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.843+0000 7f7323e00640 1 -- 172.21.15.97:0/837702271 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7479+0+0 (secure 0 0 0) 0x7f7318093b00 con 0x7f732c0b42e0 2024-09-10T21:32:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.843+0000 7f732be00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f730c0725e0 0x7f730c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:34.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.843+0000 7f732be00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f730c0725e0 0x7f730c074aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f731c000f70 tx=0x7f731c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:34.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:34.847+0000 7f7323e00640 1 -- 172.21.15.97:0/837702271 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f732c143c00 con 0x7f732c0b42e0 2024-09-10T21:32:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.047+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f732c141b80 con 0x7f732c0b42e0 2024-09-10T21:32:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.047+0000 7f7323e00640 1 -- 172.21.15.97:0/837702271 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) ==== 95+0+4404 (secure 0 0 0) 0x7f7318067030 con 0x7f732c0b42e0 2024-09-10T21:32:35.047 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:35.047 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":171,"btime":"2024-09-10T21:32:34:672612+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:34.672600+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7780":{"gid":7780,"name":"a","rank":0,"incarnation":171,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/2415223919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2415223919},{"type":"v1","addr":"172.21.15.97:6834","nonce":2415223919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 171 2024-09-10T21:32:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f730c0725e0 msgr2=0x7f730c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f730c0725e0 0x7f730c074aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f731c000f70 tx=0x7f731c009040 comp rx=0 tx=0).stop 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 msgr2=0x7f732c1436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1436c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7318005bd0 tx=0x7f731800dde0 comp rx=0 tx=0).stop 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 shutdown_connections 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f730c0725e0 0x7f730c074aa0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f732c15b7f0 0x7f732c15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732c0b42e0 0x7f732c1436c0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 --2- 172.21.15.97:0/837702271 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f732c0b2bb0 0x7f732c151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 >> 172.21.15.97:0/837702271 conn(0x7f732c0b37c0 msgr2=0x7f732c161440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 shutdown_connections 2024-09-10T21:32:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:35.051+0000 7f7332a00640 1 -- 172.21.15.97:0/837702271 wait complete. 2024-09-10T21:32:35.069 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:32:35.408 INFO:tasks.ceph.mds.c.smithi097.stderr: -48> 2024-09-10T21:32:30.655+0000 7fcb7ea00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T21:32:35.408 INFO:tasks.ceph.mds.c.smithi097.stderr: -38> 2024-09-10T21:32:30.655+0000 7fcb7ea00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:32:35.446 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:32:37.697 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:37.699+0000 7f5117800640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-10T21:32:37.697 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:37.699+0000 7f5117800640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T21:32:37.698 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:37.699+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:37.698 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:37.699+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:32:39.021 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:39.023+0000 7f511c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:32:39.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:39.551 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:40.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.219+0000 7f20d6000640 1 -- 172.21.15.97:0/1427061903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d015e750 msgr2=0x7f20d00cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.219+0000 7f20d6000640 1 --2- 172.21.15.97:0/1427061903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d015e750 0x7f20d00cd580 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009c80 tx=0x7f20c801cd10 comp rx=0 tx=0).stop 2024-09-10T21:32:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.219+0000 7f20d6000640 1 -- 172.21.15.97:0/1427061903 shutdown_connections 2024-09-10T21:32:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.219+0000 7f20d6000640 1 --2- 172.21.15.97:0/1427061903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d015e750 0x7f20d00cd580 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.219+0000 7f20d6000640 1 --2- 172.21.15.97:0/1427061903 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20d015f150 0x7f20d00b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.219+0000 7f20d6000640 1 -- 172.21.15.97:0/1427061903 >> 172.21.15.97:0/1427061903 conn(0x7f20d00b28c0 msgr2=0x7f20d00b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:40.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 -- 172.21.15.97:0/1427061903 shutdown_connections 2024-09-10T21:32:40.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 -- 172.21.15.97:0/1427061903 wait complete. 2024-09-10T21:32:40.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 Processor -- start 2024-09-10T21:32:40.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 -- start start 2024-09-10T21:32:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20d015e750 0x7f20d0154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20d015f150 0x7f20d00b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d01fb790 0x7f20d01fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0168490 con 0x7f20d01fb790 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0168310 con 0x7f20d015e750 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f20d0168610 con 0x7f20d015f150 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d01fb790 0x7f20d01fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d01fb790 0x7f20d01fbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34896/0 (socket says 172.21.15.97:34896) 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 -- 172.21.15.97:0/169997583 learned_addr learned my addr 172.21.15.97:0/169997583 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cf400640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20d015e750 0x7f20d0154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cea00640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20d015f150 0x7f20d00b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 -- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20d015f150 msgr2=0x7f20d00b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20d015f150 0x7f20d00b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 -- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20d015e750 msgr2=0x7f20d0154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20d015e750 0x7f20d0154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 -- 172.21.15.97:0/169997583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d0155380 con 0x7f20d01fb790 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cf400640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20d015e750 0x7f20d0154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cea00640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20d015f150 0x7f20d00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20cfe00640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d01fb790 0x7f20d01fbc10 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009bd0 tx=0x7f20c8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20c7400640 1 -- 172.21.15.97:0/169997583 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80bb070 con 0x7f20d01fb790 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20c7400640 1 -- 172.21.15.97:0/169997583 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20c8004e00 con 0x7f20d01fb790 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d00af000 con 0x7f20d01fb790 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.223+0000 7f20c7400640 1 -- 172.21.15.97:0/169997583 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80b2040 con 0x7f20d01fb790 2024-09-10T21:32:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.227+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d00af5c0 con 0x7f20d01fb790 2024-09-10T21:32:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.227+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d00cd580 con 0x7f20d01fb790 2024-09-10T21:32:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.227+0000 7f20c7400640 1 -- 172.21.15.97:0/169997583 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f20c80b5030 con 0x7f20d01fb790 2024-09-10T21:32:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.227+0000 7f20c7400640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20b00725e0 0x7f20b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.227+0000 7f20c7400640 1 -- 172.21.15.97:0/169997583 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7f20c812d6b0 con 0x7f20d01fb790 2024-09-10T21:32:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.231+0000 7f20cf400640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20b00725e0 0x7f20b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.231+0000 7f20cf400640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20b00725e0 0x7f20b0074aa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005fd0 tx=0x7f20c0005dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:40.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.231+0000 7f20c7400640 1 -- 172.21.15.97:0/169997583 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f20c8016990 con 0x7f20d01fb790 2024-09-10T21:32:40.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20d00ce7b0 con 0x7f20d01fb790 2024-09-10T21:32:40.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20c7400640 1 -- 172.21.15.97:0/169997583 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) ==== 95+0+4395 (secure 0 0 0) 0x7f20c80f85b0 con 0x7f20d01fb790 2024-09-10T21:32:40.429 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:40.429 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":175,"btime":"2024-09-10T21:32:37:700763+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1863388792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":1863388792},{"type":"v1","addr":"172.21.15.97:6835","nonce":1863388792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:37.700758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:40.430 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 175 2024-09-10T21:32:40.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20b00725e0 msgr2=0x7f20b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20b00725e0 0x7f20b0074aa0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005fd0 tx=0x7f20c0005dc0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d01fb790 msgr2=0x7f20d01fbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d01fb790 0x7f20d01fbc10 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009bd0 tx=0x7f20c8009390 comp rx=0 tx=0).stop 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 shutdown_connections 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f20b00725e0 0x7f20b0074aa0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d01fb790 0x7f20d01fbc10 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20d015f150 0x7f20d00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 --2- 172.21.15.97:0/169997583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20d015e750 0x7f20d0154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 >> 172.21.15.97:0/169997583 conn(0x7f20d00b28c0 msgr2=0x7f20d0071180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 shutdown_connections 2024-09-10T21:32:40.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.431+0000 7f20d6000640 1 -- 172.21.15.97:0/169997583 wait complete. 2024-09-10T21:32:40.451 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:32:40.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 -- 172.21.15.97:0/64283448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 msgr2=0x7ff22c0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 --2- 172.21.15.97:0/64283448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0d0370 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff22c073590 tx=0x7ff22401cd10 comp rx=0 tx=0).stop 2024-09-10T21:32:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 -- 172.21.15.97:0/64283448 shutdown_connections 2024-09-10T21:32:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 --2- 172.21.15.97:0/64283448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0d0370 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 --2- 172.21.15.97:0/64283448 >> v2:172.21.15.194:3300/0 conn(0x7ff22c06dc50 0x7ff22c0cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 --2- 172.21.15.97:0/64283448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff22c161600 0x7ff22c0d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 -- 172.21.15.97:0/64283448 >> 172.21.15.97:0/64283448 conn(0x7ff22c0d1c30 msgr2=0x7ff22c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 -- 172.21.15.97:0/64283448 shutdown_connections 2024-09-10T21:32:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 -- 172.21.15.97:0/64283448 wait complete. 2024-09-10T21:32:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.591+0000 7ff232800640 1 Processor -- start 2024-09-10T21:32:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 -- start start 2024-09-10T21:32:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff22c161600 0x7ff22c14f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff22c15c5f0 0x7ff22c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c0afd10 con 0x7ff22c0b1590 2024-09-10T21:32:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c0afb90 con 0x7ff22c15c5f0 2024-09-10T21:32:40.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff22c0afe90 con 0x7ff22c161600 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34910/0 (socket says 172.21.15.97:34910) 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 -- 172.21.15.97:0/3913986410 learned_addr learned my addr 172.21.15.97:0/3913986410 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22be00640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff22c15c5f0 0x7ff22c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22aa00640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff22c161600 0x7ff22c14f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 -- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff22c161600 msgr2=0x7ff22c14f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff22c161600 0x7ff22c14f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 -- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff22c15c5f0 msgr2=0x7ff22c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff22c15c5f0 0x7ff22c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 -- 172.21.15.97:0/3913986410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c0b5a80 con 0x7ff22c0b1590 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22aa00640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff22c161600 0x7ff22c14f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:40.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff22b400640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0b5540 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00b440 tx=0x7ff21c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:40.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff223400640 1 -- 172.21.15.97:0/3913986410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c015020 con 0x7ff22c0b1590 2024-09-10T21:32:40.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff223400640 1 -- 172.21.15.97:0/3913986410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff21c0119a0 con 0x7ff22c0b1590 2024-09-10T21:32:40.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c15d040 con 0x7ff22c0b1590 2024-09-10T21:32:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff223400640 1 -- 172.21.15.97:0/3913986410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c011df0 con 0x7ff22c0b1590 2024-09-10T21:32:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c0b4a60 con 0x7ff22c0b1590 2024-09-10T21:32:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.595+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22c0d0a60 con 0x7ff22c0b1590 2024-09-10T21:32:40.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.599+0000 7ff223400640 1 -- 172.21.15.97:0/3913986410 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff21c01a400 con 0x7ff22c0b1590 2024-09-10T21:32:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.599+0000 7ff223400640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff20c072640 0x7ff20c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.599+0000 7ff223400640 1 -- 172.21.15.97:0/3913986410 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7ff21c093e70 con 0x7ff22c0b1590 2024-09-10T21:32:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.599+0000 7ff22aa00640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff20c072640 0x7ff20c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.599+0000 7ff22aa00640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff20c072640 0x7ff20c074b00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff22c15da40 tx=0x7ff218008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:40.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.603+0000 7ff223400640 1 -- 172.21.15.97:0/3913986410 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff21c01f020 con 0x7ff22c0b1590 2024-09-10T21:32:40.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff22c0715a0 con 0x7ff22c0b1590 2024-09-10T21:32:40.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff223400640 1 -- 172.21.15.97:0/3913986410 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) ==== 95+0+4395 (secure 0 0 0) 0x7ff21c00d040 con 0x7ff22c0b1590 2024-09-10T21:32:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:40.802 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":175,"btime":"2024-09-10T21:32:37:700763+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1863388792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":1863388792},{"type":"v1","addr":"172.21.15.97:6835","nonce":1863388792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:37.700758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:40.802 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 175 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff20c072640 msgr2=0x7ff20c074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff20c072640 0x7ff20c074b00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff22c15da40 tx=0x7ff218008040 comp rx=0 tx=0).stop 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 msgr2=0x7ff22c0b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0b5540 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00b440 tx=0x7ff21c00b900 comp rx=0 tx=0).stop 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 shutdown_connections 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff20c072640 0x7ff20c074b00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff22c15c5f0 0x7ff22c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff22c161600 0x7ff22c14f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 --2- 172.21.15.97:0/3913986410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff22c0b1590 0x7ff22c0b5540 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 >> 172.21.15.97:0/3913986410 conn(0x7ff22c0d1c30 msgr2=0x7ff22c15f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 shutdown_connections 2024-09-10T21:32:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.803+0000 7ff232800640 1 -- 172.21.15.97:0/3913986410 wait complete. 2024-09-10T21:32:40.823 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:40.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- 172.21.15.97:0/325033565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 msgr2=0x7f12cc0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 --2- 172.21.15.97:0/325033565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc0cfa40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009900 tx=0x7f12b801c850 comp rx=0 tx=0).stop 2024-09-10T21:32:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- 172.21.15.97:0/325033565 shutdown_connections 2024-09-10T21:32:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 --2- 172.21.15.97:0/325033565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc0cfa40 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 --2- 172.21.15.97:0/325033565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f12cc15f350 0x7f12cc0b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- 172.21.15.97:0/325033565 >> 172.21.15.97:0/325033565 conn(0x7f12cc0d1c60 msgr2=0x7f12cc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:40.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- 172.21.15.97:0/325033565 shutdown_connections 2024-09-10T21:32:40.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- 172.21.15.97:0/325033565 wait complete. 2024-09-10T21:32:40.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 Processor -- start 2024-09-10T21:32:40.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- start start 2024-09-10T21:32:40.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12cc0cff80 0x7f12cc06f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f12cc15f350 0x7f12cc151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1691d0 con 0x7f12cc0d05e0 2024-09-10T21:32:40.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc169050 con 0x7f12cc0cff80 2024-09-10T21:32:40.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d2e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc169350 con 0x7f12cc15f350 2024-09-10T21:32:40.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc15c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34936/0 (socket says 172.21.15.97:34936) 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 -- 172.21.15.97:0/3829298547 learned_addr learned my addr 172.21.15.97:0/3829298547 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cbe00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12cc0cff80 0x7f12cc06f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12d0e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f12cc15f350 0x7f12cc151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 -- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f12cc15f350 msgr2=0x7f12cc151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f12cc15f350 0x7f12cc151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 -- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12cc0cff80 msgr2=0x7f12cc06f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12cc0cff80 0x7f12cc06f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:40.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 -- 172.21.15.97:0/3829298547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc0cf460 con 0x7f12cc0d05e0 2024-09-10T21:32:40.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.967+0000 7f12cb400640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc15c320 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f12b801cd30 tx=0x7f12b8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:40.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12c3e00640 1 -- 172.21.15.97:0/3829298547 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b80b2040 con 0x7f12cc0d05e0 2024-09-10T21:32:40.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12c3e00640 1 -- 172.21.15.97:0/3829298547 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b80a58c0 con 0x7f12cc0d05e0 2024-09-10T21:32:40.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc0b6e10 con 0x7f12cc0d05e0 2024-09-10T21:32:40.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12c3e00640 1 -- 172.21.15.97:0/3829298547 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b80a5c90 con 0x7f12cc0d05e0 2024-09-10T21:32:40.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc0b73d0 con 0x7f12cc0d05e0 2024-09-10T21:32:40.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc0b75a0 con 0x7f12cc0d05e0 2024-09-10T21:32:40.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12c3e00640 1 -- 172.21.15.97:0/3829298547 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f12b8003600 con 0x7f12cc0d05e0 2024-09-10T21:32:40.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12c3e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12ac0725e0 0x7f12ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:40.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12c3e00640 1 -- 172.21.15.97:0/3829298547 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7f12b812dc90 con 0x7f12cc0d05e0 2024-09-10T21:32:40.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.971+0000 7f12cbe00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12ac0725e0 0x7f12ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:40.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.975+0000 7f12cbe00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12ac0725e0 0x7f12ac074aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009b40 tx=0x7f12bc0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:40.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:40.975+0000 7f12c3e00640 1 -- 172.21.15.97:0/3829298547 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f12b80164d0 con 0x7f12cc0d05e0 2024-09-10T21:32:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.171+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12cc0cfa40 con 0x7f12cc0d05e0 2024-09-10T21:32:41.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.171+0000 7f12c3e00640 1 -- 172.21.15.97:0/3829298547 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) ==== 95+0+4395 (secure 0 0 0) 0x7f12b80f8ba0 con 0x7f12cc0d05e0 2024-09-10T21:32:41.172 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:41.172 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":175,"btime":"2024-09-10T21:32:37:700763+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1863388792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":1863388792},{"type":"v1","addr":"172.21.15.97:6835","nonce":1863388792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:37.700758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:41.173 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 175 2024-09-10T21:32:41.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12ac0725e0 msgr2=0x7f12ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12ac0725e0 0x7f12ac074aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009b40 tx=0x7f12bc0091e0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 msgr2=0x7f12cc15c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc15c320 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f12b801cd30 tx=0x7f12b8005780 comp rx=0 tx=0).stop 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 shutdown_connections 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f12ac0725e0 0x7f12ac074aa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f12cc15f350 0x7f12cc151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12cc0d05e0 0x7f12cc15c320 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 --2- 172.21.15.97:0/3829298547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12cc0cff80 0x7f12cc06f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 >> 172.21.15.97:0/3829298547 conn(0x7f12cc0d1c60 msgr2=0x7f12cc0b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 shutdown_connections 2024-09-10T21:32:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.175+0000 7f12d2e00640 1 -- 172.21.15.97:0/3829298547 wait complete. 2024-09-10T21:32:41.195 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2024-09-10T21:32:41.195 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2024-09-10T21:32:41.195 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:41.196 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:41.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:32:41.204 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:41.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:41.205 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:41.255 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:32:41.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:32:41.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- 172.21.15.97:0/3139875623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e40691b0 msgr2=0x7fd2e40cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 --2- 172.21.15.97:0/3139875623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e40691b0 0x7fd2e40cfaf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40b63b0 tx=0x7fd2dc01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- 172.21.15.97:0/3139875623 shutdown_connections 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 --2- 172.21.15.97:0/3139875623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e40691b0 0x7fd2e40cfaf0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 --2- 172.21.15.97:0/3139875623 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2e4074b20 0x7fd2e40cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 --2- 172.21.15.97:0/3139875623 >> v2:172.21.15.194:3300/0 conn(0x7fd2e415ac30 0x7fd2e40b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- 172.21.15.97:0/3139875623 >> 172.21.15.97:0/3139875623 conn(0x7fd2e40b3c70 msgr2=0x7fd2e40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- 172.21.15.97:0/3139875623 shutdown_connections 2024-09-10T21:32:41.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- 172.21.15.97:0/3139875623 wait complete. 2024-09-10T21:32:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 Processor -- start 2024-09-10T21:32:41.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- start start 2024-09-10T21:32:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e40691b0 0x7fd2e4159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4074b20 0x7fd2e4153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2e415ac30 0x7fd2e40af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e406dcb0 con 0x7fd2e4074b20 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e406db30 con 0x7fd2e40691b0 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e9800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e406de30 con 0x7fd2e415ac30 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4074b20 0x7fd2e4153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4074b20 0x7fd2e4153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34954/0 (socket says 172.21.15.97:34954) 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e2000640 1 -- 172.21.15.97:0/4048483490 learned_addr learned my addr 172.21.15.97:0/4048483490 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.443+0000 7fd2e2a00640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e40691b0 0x7fd2e4159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e3400640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2e415ac30 0x7fd2e40af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e2000640 1 -- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2e415ac30 msgr2=0x7fd2e40af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e2000640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2e415ac30 0x7fd2e40af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e2000640 1 -- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e40691b0 msgr2=0x7fd2e4159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e2000640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e40691b0 0x7fd2e4159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e2000640 1 -- 172.21.15.97:0/4048483490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e41f4f30 con 0x7fd2e4074b20 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e2a00640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e40691b0 0x7fd2e4159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e3400640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2e415ac30 0x7fd2e40af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e2000640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4074b20 0x7fd2e4153880 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000d720 tx=0x7fd2d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2daa00640 1 -- 172.21.15.97:0/4048483490 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0017070 con 0x7fd2e4074b20 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2daa00640 1 -- 172.21.15.97:0/4048483490 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2d0009d40 con 0x7fd2e4074b20 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e4159d30 con 0x7fd2e4074b20 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2daa00640 1 -- 172.21.15.97:0/4048483490 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0005280 con 0x7fd2e4074b20 2024-09-10T21:32:41.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41f51f0 con 0x7fd2e4074b20 2024-09-10T21:32:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e406e2e0 con 0x7fd2e4074b20 2024-09-10T21:32:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.447+0000 7fd2daa00640 1 -- 172.21.15.97:0/4048483490 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd2d0005420 con 0x7fd2e4074b20 2024-09-10T21:32:41.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.451+0000 7fd2daa00640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2c4072610 0x7fd2c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.451+0000 7fd2daa00640 1 -- 172.21.15.97:0/4048483490 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7fd2d0093b50 con 0x7fd2e4074b20 2024-09-10T21:32:41.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.451+0000 7fd2e2a00640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2c4072610 0x7fd2c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.451+0000 7fd2e2a00640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2c4072610 0x7fd2c4074ad0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40092f0 tx=0x7fd2d4009280 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:41.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.455+0000 7fd2daa00640 1 -- 172.21.15.97:0/4048483490 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd2d005ea60 con 0x7fd2e4074b20 2024-09-10T21:32:41.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.651+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd2e4069820 con 0x7fd2e4074b20 2024-09-10T21:32:41.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.723+0000 7fd2daa00640 1 -- 172.21.15.97:0/4048483490 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v176) ==== 161+0+0 (secure 0 0 0) 0x7fd2d0067180 con 0x7fd2e4074b20 2024-09-10T21:32:41.723 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:41.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2c4072610 msgr2=0x7fd2c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2c4072610 0x7fd2c4074ad0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40092f0 tx=0x7fd2d4009280 comp rx=0 tx=0).stop 2024-09-10T21:32:41.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4074b20 msgr2=0x7fd2e4153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4074b20 0x7fd2e4153880 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000d720 tx=0x7fd2d000dbe0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 shutdown_connections 2024-09-10T21:32:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd2c4072610 0x7fd2c4074ad0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd2e415ac30 0x7fd2e40af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2e4074b20 0x7fd2e4153880 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 --2- 172.21.15.97:0/4048483490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e40691b0 0x7fd2e4159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 >> 172.21.15.97:0/4048483490 conn(0x7fd2e40b3c70 msgr2=0x7fd2e415eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 shutdown_connections 2024-09-10T21:32:41.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.727+0000 7fd2e9800640 1 -- 172.21.15.97:0/4048483490 wait complete. 2024-09-10T21:32:41.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:41.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 -- 172.21.15.97:0/3066461900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7140141d20 msgr2=0x7f714015f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 --2- 172.21.15.97:0/3066461900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7140141d20 0x7f714015f6d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f712c009870 tx=0x7f712c01c860 comp rx=0 tx=0).stop 2024-09-10T21:32:41.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 -- 172.21.15.97:0/3066461900 shutdown_connections 2024-09-10T21:32:41.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 --2- 172.21.15.97:0/3066461900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7140141d20 0x7f714015f6d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 -- 172.21.15.97:0/3066461900 >> 172.21.15.97:0/3066461900 conn(0x7f71400b1780 msgr2=0x7f71400b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:41.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 -- 172.21.15.97:0/3066461900 shutdown_connections 2024-09-10T21:32:41.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 -- 172.21.15.97:0/3066461900 wait complete. 2024-09-10T21:32:41.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 Processor -- start 2024-09-10T21:32:41.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.895+0000 7f7146600640 1 -- start start 2024-09-10T21:32:41.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71400ce5f0 0x7f7140156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7140141d20 0x7f7140072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714015fc10 0x7f7140158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7140169c90 con 0x7f714015fc10 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7140160080 con 0x7f7140141d20 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7140169e10 con 0x7f71400ce5f0 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714015fc10 0x7f7140158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714015fc10 0x7f7140158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34966/0 (socket says 172.21.15.97:34966) 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 -- 172.21.15.97:0/842330144 learned_addr learned my addr 172.21.15.97:0/842330144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7140141d20 0x7f7140072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713f400640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71400ce5f0 0x7f7140156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 -- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71400ce5f0 msgr2=0x7f7140156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71400ce5f0 0x7f7140156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 -- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7140141d20 msgr2=0x7f7140072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:41.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7140141d20 0x7f7140072630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:41.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 -- 172.21.15.97:0/842330144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7140072b70 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f713fe00640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714015fc10 0x7f7140158f50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f713800cb40 tx=0x7f713800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7137400640 1 -- 172.21.15.97:0/842330144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138016020 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7137400640 1 -- 172.21.15.97:0/842330144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f713800ab60 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7137400640 1 -- 172.21.15.97:0/842330144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138012420 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7140072e10 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71400733d0 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7140155b70 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.899+0000 7f7137400640 1 -- 172.21.15.97:0/842330144 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71380125c0 con 0x7f714015fc10 2024-09-10T21:32:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.903+0000 7f7137400640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71200725e0 0x7f7120074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.903+0000 7f7137400640 1 -- 172.21.15.97:0/842330144 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7f7138093d10 con 0x7f714015fc10 2024-09-10T21:32:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.903+0000 7f713f400640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71200725e0 0x7f7120074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.903+0000 7f713f400640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71200725e0 0x7f7120074aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7130000c10 tx=0x7f7130009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:41.907+0000 7f7137400640 1 -- 172.21.15.97:0/842330144 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7140155b70 con 0x7f714015fc10 2024-09-10T21:32:42.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.131+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f714015f6d0 con 0x7f714015fc10 2024-09-10T21:32:42.453 INFO:tasks.ceph.mds.a.smithi097.stderr: -53> 2024-09-10T21:32:37.699+0000 7f5117800640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-10T21:32:42.453 INFO:tasks.ceph.mds.a.smithi097.stderr: -51> 2024-09-10T21:32:37.699+0000 7f5117800640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T21:32:42.453 INFO:tasks.ceph.mds.a.smithi097.stderr: -31> 2024-09-10T21:32:39.023+0000 7f511c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:32:42.491 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:32:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7137400640 1 -- 172.21.15.97:0/842330144 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v177) ==== 101+0+0 (secure 0 0 0) 0x7f7138099020 con 0x7f714015fc10 2024-09-10T21:32:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:32:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71200725e0 msgr2=0x7f7120074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71200725e0 0x7f7120074aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7130000c10 tx=0x7f7130009040 comp rx=0 tx=0).stop 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714015fc10 msgr2=0x7f7140158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714015fc10 0x7f7140158f50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f713800cb40 tx=0x7f713800cc30 comp rx=0 tx=0).stop 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 shutdown_connections 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71200725e0 0x7f7120074aa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714015fc10 0x7f7140158f50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7140141d20 0x7f7140072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 --2- 172.21.15.97:0/842330144 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f71400ce5f0 0x7f7140156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 >> 172.21.15.97:0/842330144 conn(0x7f71400b1780 msgr2=0x7f7140159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 shutdown_connections 2024-09-10T21:32:42.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.731+0000 7f7146600640 1 -- 172.21.15.97:0/842330144 wait complete. 2024-09-10T21:32:42.753 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:42.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 Processor -- start 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 -- start start 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f51daa0 0x557c3f5f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f5d6ca0 0x557c3f5d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c3f54cda0 con 0x557c3f5d7080 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557c3f46ba00 con 0x557c3f51daa0 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cd19c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557c3f563fd0 con 0x557c3f5d6ca0 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35024/0 (socket says 172.21.15.97:35024) 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 -- 172.21.15.97:0/1231448759 learned_addr learned my addr 172.21.15.97:0/1231448759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67ca400640 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f51daa0 0x557c3f5f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67c3e00640 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f5d6ca0 0x557c3f5d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 -- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f5d6ca0 msgr2=0x557c3f5d5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:42.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f5d6ca0 0x557c3f5d5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 -- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f51daa0 msgr2=0x557c3f5f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:42.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f51daa0 0x557c3f5f0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 -- 172.21.15.97:0/1231448759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3f46aa20 con 0x557c3f5d7080 2024-09-10T21:32:42.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67ca400640 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f51daa0 0x557c3f5f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67cae00640 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5f10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f67c400a4b0 tx=0x7f67c400a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87779b87dd2ba22a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67c9000640 1 -- 172.21.15.97:0/1231448759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400f7b0 con 0x557c3f5d7080 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67c9000640 1 -- 172.21.15.97:0/1231448759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67c4015070 con 0x557c3f5d7080 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.799+0000 7f67c9000640 1 -- 172.21.15.97:0/1231448759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400fce0 con 0x557c3f5d7080 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- 172.21.15.97:0/1231448759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 msgr2=0x557c3f5f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5f10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f67c400a4b0 tx=0x7f67c400a970 comp rx=0 tx=0).stop 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- 172.21.15.97:0/1231448759 shutdown_connections 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5f10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f5d6ca0 0x557c3f5d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 --2- 172.21.15.97:0/1231448759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f51daa0 0x557c3f5f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- 172.21.15.97:0/1231448759 >> 172.21.15.97:0/1231448759 conn(0x557c3f51c140 msgr2=0x557c3f5f49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:42.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- 172.21.15.97:0/1231448759 shutdown_connections 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- 172.21.15.97:0/1231448759 wait complete. 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 Processor -- start 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- start start 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f51daa0 0x557c3f68b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f5d6ca0 0x557c3f68dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c3f5f3110 con 0x557c3f5d7080 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557c3f5f4450 con 0x557c3f5d6ca0 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x557c3f5f33b0 con 0x557c3f51daa0 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35026/0 (socket says 172.21.15.97:35026) 2024-09-10T21:32:42.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 learned_addr learned my addr 172.21.15.97:0/3327077046 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67ca400640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f51daa0 0x557c3f68b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f5d6ca0 0x557c3f68dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f51daa0 msgr2=0x557c3f68b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x557c3f51daa0 0x557c3f68b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f5d6ca0 msgr2=0x557c3f68dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f5d6ca0 0x557c3f68dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3f444c10 con 0x557c3f5d7080 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c3f5d6ca0 0x557c3f68dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:42.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cae00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5c70 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f67c400ae70 tx=0x7f67c4010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67c2a00640 1 -- 172.21.15.97:0/3327077046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4010860 con 0x557c3f5d7080 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67c2a00640 1 -- 172.21.15.97:0/3327077046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67c4015070 con 0x557c3f5d7080 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67c2a00640 1 -- 172.21.15.97:0/3327077046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4010ce0 con 0x557c3f5d7080 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c3f5f7fe0 con 0x557c3f5d7080 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.803+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3f5f8520 con 0x557c3f5d7080 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67c2a00640 1 -- 172.21.15.97:0/3327077046 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f67c402c440 con 0x557c3f5d7080 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67c2a00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f67a8070ff0 0x7f67a80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67c2a00640 1 -- 172.21.15.97:0/3327077046 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7f67c4094160 con 0x557c3f5d7080 2024-09-10T21:32:42.808 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:42.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557c3f5f7ad0 0x557c3f5f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f5f9440 con 0x557c3f5f7ad0 2024-09-10T21:32:42.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67ca400640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f67a8070ff0 0x7f67a80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557c3f5f7ad0 0x557c3f5f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67ca400640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f67a8070ff0 0x7f67a80734b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0ab000 tx=0x7f67bc019a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.807+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557c3f5f7ad0 0x557c3f5f8f00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.811+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v111'443 uv417 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.811+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f5f9810 con 0x557c3f5f7ad0 2024-09-10T21:32:42.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.811+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v111'444 uv444 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.811+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697730 con 0x557c3f5f7ad0 2024-09-10T21:32:42.813 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:42.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.815+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v111'445 uv445 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.815+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f698ae0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.815+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'446 uv446 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.815+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f698e80 con 0x557c3f5f7ad0 2024-09-10T21:32:42.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.819+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.819+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ad0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.819+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.819+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.819+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v111'449 uv423 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.819+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.823+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v111'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.823+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.823 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:42.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.823+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v111'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.823+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.827+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.827+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.827+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.827+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.827+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.831+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.831+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v111'455 uv430 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.831+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.831+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'456 uv456 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.831+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.833 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.835+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v111'457 uv457 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.835+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.835+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'458 uv458 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.835+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.839+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'459 uv459 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.839+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.839+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'460 uv460 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.839+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.839+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'461 uv461 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.839+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.843+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v111'462 uv436 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.843+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.843+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v111'463 uv463 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.843+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.845 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:42.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.847+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v111'464 uv464 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.847+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.847+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'465 uv465 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.847+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.847+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'466 uv466 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.847+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.851+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'467 uv467 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.851+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.851+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v111'468 uv442 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.851+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.855+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v111'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.855+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.854 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.855+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v111'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.855+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.855+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.855+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.859+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.859+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f697ec0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.859+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f5f7ad0 2024-09-10T21:32:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.859+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557c3f699220 0x557c3f69b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.859+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69bb50 con 0x557c3f699220 2024-09-10T21:32:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.859+0000 7f67ca400640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557c3f699220 0x557c3f69b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.859+0000 7f67ca400640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557c3f699220 0x557c3f69b610 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.863+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'330 uv312 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.863+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f69bef0 con 0x557c3f699220 2024-09-10T21:32:42.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.863+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'331 uv331 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.863+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69c290 con 0x557c3f699220 2024-09-10T21:32:42.865 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:42.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.867+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'332 uv332 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.867+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69c630 con 0x557c3f699220 2024-09-10T21:32:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.867+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v111'333 uv333 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.867+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69c9d0 con 0x557c3f699220 2024-09-10T21:32:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.871+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'334 uv334 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.871+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69cd70 con 0x557c3f699220 2024-09-10T21:32:42.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.871+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v111'335 uv317 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.871+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.871+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v111'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.871+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.875+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v111'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.873 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:42.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.875+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.875+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'338 uv338 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.875+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.875+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.879+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.879+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v111'340 uv322 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.879+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.879+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v111'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.879+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.883+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v111'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.881 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:42.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.883+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a12e0 con 0x557c3f699220 2024-09-10T21:32:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.883+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v111'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.883+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a12e0 con 0x557c3f699220 2024-09-10T21:32:42.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.883+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.883+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.887+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v111'345 uv328 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.887+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.887+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v111'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.887+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.888 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.887+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v111'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.891+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69e120 con 0x557c3f699220 2024-09-10T21:32:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.891+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.891+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69e120 con 0x557c3f699220 2024-09-10T21:32:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.891+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.891+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f69d110 con 0x557c3f699220 2024-09-10T21:32:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.895+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f699220 2024-09-10T21:32:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.895+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557c3f69d790 0x557c3f69fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.895+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a01e0 con 0x557c3f69d790 2024-09-10T21:32:42.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.895+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557c3f69d790 0x557c3f69fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.895+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557c3f69d790 0x557c3f69fb80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.895+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.895+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a05b0 con 0x557c3f69d790 2024-09-10T21:32:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a05b0 con 0x557c3f69d790 2024-09-10T21:32:42.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.899 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a15c0 con 0x557c3f69d790 2024-09-10T21:32:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a15c0 con 0x557c3f69d790 2024-09-10T21:32:42.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.899+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a05b0 con 0x557c3f69d790 2024-09-10T21:32:42.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.903+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.903+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557c3f6a0c30 0x557c3f6a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.903+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a36f0 con 0x557c3f6a0c30 2024-09-10T21:32:42.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.903+0000 7f67cae00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557c3f6a0c30 0x557c3f6a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.903+0000 7f67cae00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557c3f6a0c30 0x557c3f6a3020 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.903+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'71 uv70 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.903+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a3c00 con 0x557c3f6a0c30 2024-09-10T21:32:42.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'72 uv72 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a4100 con 0x557c3f6a0c30 2024-09-10T21:32:42.905 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'73 uv73 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a49c0 con 0x557c3f6a0c30 2024-09-10T21:32:42.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'74 uv74 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a4d60 con 0x557c3f69d790 2024-09-10T21:32:42.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.907+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a4d60 con 0x557c3f69d790 2024-09-10T21:32:42.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.911+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.911+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a4d60 con 0x557c3f69d790 2024-09-10T21:32:42.910 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:42.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.911+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.911+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5d70 con 0x557c3f69d790 2024-09-10T21:32:42.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.911+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.911+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5d70 con 0x557c3f69d790 2024-09-10T21:32:42.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.915+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.915+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a4d60 con 0x557c3f69d790 2024-09-10T21:32:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.915+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.915+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a4d60 con 0x557c3f6a0c30 2024-09-10T21:32:42.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.915+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v111'160 uv154 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.915+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a5100 con 0x557c3f6a0c30 2024-09-10T21:32:42.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v111'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a54a0 con 0x557c3f6a0c30 2024-09-10T21:32:42.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v111'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.918 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a6850 con 0x557c3f6a0c30 2024-09-10T21:32:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a6850 con 0x557c3f6a0c30 2024-09-10T21:32:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.919+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f6a0c30 2024-09-10T21:32:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.923+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.923+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f6a0c30 2024-09-10T21:32:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.923+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v111'166 uv159 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.923+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f6a0c30 2024-09-10T21:32:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.923+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v111'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.923+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f6a0c30 2024-09-10T21:32:42.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v111'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.925 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:42.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f6a0c30 2024-09-10T21:32:42.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f6a0c30 2024-09-10T21:32:42.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f69d790 2024-09-10T21:32:42.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v111'71 uv70 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.927+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f69d790 2024-09-10T21:32:42.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.931+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v111'72 uv72 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.931+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f69d790 2024-09-10T21:32:42.930 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:42.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.931+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v111'73 uv73 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.931+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f69d790 2024-09-10T21:32:42.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.931+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'74 uv74 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.931+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a5840 con 0x557c3f69d790 2024-09-10T21:32:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f69d790 2024-09-10T21:32:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557c3f6a5ec0 0x557c3f6a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8880 con 0x557c3f6a5ec0 2024-09-10T21:32:42.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67ca400640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557c3f6a5ec0 0x557c3f6a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67ca400640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557c3f6a5ec0 0x557c3f6a82b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.935+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.939+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.939+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.938 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:42.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.939+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a9c60 con 0x557c3f6a5ec0 2024-09-10T21:32:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.939+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.939+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a9c60 con 0x557c3f6a5ec0 2024-09-10T21:32:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.939+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.939+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.943+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.943+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.943+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v111'142 uv136 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.943+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.943+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v111'143 uv143 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.943+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.947+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v111'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.945 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:42.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.947+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.947+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.947+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.947+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.947+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v111'147 uv141 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v111'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.951 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v111'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a8c50 con 0x557c3f6a5ec0 2024-09-10T21:32:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v111'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.951+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a4600 con 0x557c3f6a5ec0 2024-09-10T21:32:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.955+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v111'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.955+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45c0 con 0x557c3f6a0c30 2024-09-10T21:32:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.955+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v111'66 uv64 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.955+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a45c0 con 0x557c3f6a0c30 2024-09-10T21:32:42.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.955+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v111'67 uv67 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.955+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45c0 con 0x557c3f6a0c30 2024-09-10T21:32:42.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.959+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v111'68 uv68 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.957 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:42.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.959+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45c0 con 0x557c3f6a0c30 2024-09-10T21:32:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.959+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v111'69 uv69 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.959+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45a0 con 0x557c3f6a0c30 2024-09-10T21:32:42.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.959+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v111'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.959+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6a45a0 con 0x557c3f6a0c30 2024-09-10T21:32:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.963+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v111'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.963+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45a0 con 0x557c3f6a0c30 2024-09-10T21:32:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.963+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v111'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.962 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.963+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45a0 con 0x557c3f6a0c30 2024-09-10T21:32:42.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.963+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v111'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.963+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45a0 con 0x557c3f6a0c30 2024-09-10T21:32:42.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.963+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v111'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6a45a0 con 0x557c3f6a0c30 2024-09-10T21:32:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v111'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67c4029020 con 0x557c3f6a0c30 2024-09-10T21:32:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557c3f6a9a50 0x557c3f6ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a9a50 2024-09-10T21:32:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557c3f6a9a50 0x557c3f6ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67c3e00640 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557c3f6a9a50 0x557c3f6ac1a0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:42.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v111'74 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f6a9a50 2024-09-10T21:32:42.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.967+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a9a50 2024-09-10T21:32:42.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v111'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f6a9a50 2024-09-10T21:32:42.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a9a50 2024-09-10T21:32:42.970 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:42.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v111'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f6a9a50 2024-09-10T21:32:42.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a9a50 2024-09-10T21:32:42.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v111'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f6a9a50 2024-09-10T21:32:42.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a9a50 2024-09-10T21:32:42.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67c3e00640 1 -- 172.21.15.97:0/3327077046 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v111'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f67b400c6a0 con 0x557c3f6a9a50 2024-09-10T21:32:42.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.971+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.975+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v111'182 uv172 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.975+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.975+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v111'183 uv183 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.975+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.976 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:42.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.975+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v111'184 uv184 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.975+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.979+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v111'185 uv185 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.979+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.979+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v111'186 uv186 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.979+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.979+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v111'187 uv187 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.980 INFO:teuthology.orchestra.run.smithi097.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:42.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.979+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ad6f0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v111'188 uv188 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ad6f0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v111'189 uv189 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v111'190 uv190 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x557c3f6ac6e0 con 0x557c3f6a5ec0 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67ca400640 1 -- 172.21.15.97:0/3327077046 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v111'191 uv191 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f67bc0b3040 con 0x557c3f6a5ec0 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557c3f69d790 msgr2=0x557c3f69fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x557c3f69d790 0x557c3f69fb80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557c3f6a5ec0 msgr2=0x557c3f6a82b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x557c3f6a5ec0 0x557c3f6a82b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557c3f6a9a50 msgr2=0x557c3f6ac1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x557c3f6a9a50 0x557c3f6ac1a0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557c3f6a0c30 msgr2=0x557c3f6a3020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x557c3f6a0c30 0x557c3f6a3020 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557c3f699220 msgr2=0x557c3f69b610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x557c3f699220 0x557c3f69b610 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 reap_dead start 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557c3f5f7ad0 msgr2=0x557c3f5f8f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x557c3f5f7ad0 0x557c3f5f8f00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.983+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 reap_dead start 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cae00640 1 -- 172.21.15.97:0/3327077046 reap_dead start 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f67a8070ff0 msgr2=0x7f67a80734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f67a8070ff0 0x7f67a80734b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0ab000 tx=0x7f67bc019a50 comp rx=0 tx=0).stop 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 msgr2=0x557c3f5f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5c70 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f67c400ae70 tx=0x7f67c4010440 comp rx=0 tx=0).stop 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 shutdown_connections 2024-09-10T21:32:42.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f67a8070ff0 0x7f67a80734b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 --2- 172.21.15.97:0/3327077046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c3f5d7080 0x557c3f5f5c70 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 >> 172.21.15.97:0/3327077046 conn(0x557c3f51c140 msgr2=0x557c3f5644e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 shutdown_connections 2024-09-10T21:32:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:42.987+0000 7f67cd19c080 1 -- 172.21.15.97:0/3327077046 wait complete. 2024-09-10T21:32:42.990 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2024-09-10T21:32:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 Processor -- start 2024-09-10T21:32:43.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- start start 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f59073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58eda00 0x5629f587d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58f11d0 0x5629f5909f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5629f5781a00 con 0x5629f58ebcc0 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5629f57dff40 con 0x5629f58f11d0 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5629f58ef710 con 0x5629f58eda00 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f59073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f59073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35036/0 (socket says 172.21.15.97:35036) 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 -- 172.21.15.97:0/3613866433 learned_addr learned my addr 172.21.15.97:0/3613866433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989e00640 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58f11d0 0x5629f5909f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd983e00640 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58eda00 0x5629f587d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 -- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58eda00 msgr2=0x5629f587d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:43.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58eda00 0x5629f587d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 -- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58f11d0 msgr2=0x5629f5909f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:43.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58f11d0 0x5629f5909f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 -- 172.21.15.97:0/3613866433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f575ac10 con 0x5629f58ebcc0 2024-09-10T21:32:43.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd989400640 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f59073f0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd97c01bb40 tx=0x7fd97c018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7449132f81f299c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd983400640 1 -- 172.21.15.97:0/3613866433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b0040 con 0x5629f58ebcc0 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd983400640 1 -- 172.21.15.97:0/3613866433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd97c0ba070 con 0x5629f58ebcc0 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd983400640 1 -- 172.21.15.97:0/3613866433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0abdc0 con 0x5629f58ebcc0 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- 172.21.15.97:0/3613866433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 msgr2=0x5629f59073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f59073f0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd97c01bb40 tx=0x7fd97c018040 comp rx=0 tx=0).stop 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- 172.21.15.97:0/3613866433 shutdown_connections 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58f11d0 0x5629f5909f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58eda00 0x5629f587d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 --2- 172.21.15.97:0/3613866433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f59073f0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- 172.21.15.97:0/3613866433 >> 172.21.15.97:0/3613866433 conn(0x5629f587b0d0 msgr2=0x5629f5863ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- 172.21.15.97:0/3613866433 shutdown_connections 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.083+0000 7fd98ce48080 1 -- 172.21.15.97:0/3613866433 wait complete. 2024-09-10T21:32:43.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 Processor -- start 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 -- start start 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f58358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58eda00 0x5629f5906f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58f11d0 0x5629f5907440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5629f587a6a0 con 0x5629f58ebcc0 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5629f587a400 con 0x5629f58eda00 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5629f587a940 con 0x5629f58f11d0 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f58358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f58358c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35040/0 (socket says 172.21.15.97:35040) 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 -- 172.21.15.97:0/879430197 learned_addr learned my addr 172.21.15.97:0/879430197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989e00640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58f11d0 0x5629f5907440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd983e00640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58eda00 0x5629f5906f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 -- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58f11d0 msgr2=0x5629f5907440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5629f58f11d0 0x5629f5907440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 -- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58eda00 msgr2=0x5629f5906f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629f58eda00 0x5629f5906f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 -- 172.21.15.97:0/879430197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f58344d0 con 0x5629f58ebcc0 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd989400640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f58358c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0ab7a0 tx=0x7fd97c0acaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd982000640 1 -- 172.21.15.97:0/879430197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0c3070 con 0x5629f58ebcc0 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd982000640 1 -- 172.21.15.97:0/879430197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd97c0ba070 con 0x5629f58ebcc0 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd982000640 1 -- 172.21.15.97:0/879430197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b5980 con 0x5629f58ebcc0 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5629f5834770 con 0x5629f58ebcc0 2024-09-10T21:32:43.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.087+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f5834cb0 con 0x5629f58ebcc0 2024-09-10T21:32:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd982000640 1 -- 172.21.15.97:0/879430197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd97c0acce0 con 0x5629f58ebcc0 2024-09-10T21:32:43.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd982000640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd968070ff0 0x7fd9680734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd982000640 1 -- 172.21.15.97:0/879430197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7fd97c12ddd0 con 0x5629f58ebcc0 2024-09-10T21:32:43.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd983e00640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd968070ff0 0x7fd9680734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd98ce48080 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5629f5835e00 0x5629f5832140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5629f5832680 con 0x5629f5835e00 2024-09-10T21:32:43.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd989e00640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5629f5835e00 0x5629f5832140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd983e00640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd968070ff0 0x7fd9680734b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd974003f20 tx=0x7fd9740027e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.091+0000 7fd989e00640 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5629f5835e00 0x5629f5832140 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd989e00640 1 -- 172.21.15.97:0/879430197 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v111'351 uv351 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd98400b040 con 0x5629f5835e00 2024-09-10T21:32:43.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5629f5835e00 msgr2=0x5629f5832140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5629f5835e00 0x5629f5832140 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd968070ff0 msgr2=0x7fd9680734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd968070ff0 0x7fd9680734b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd974003f20 tx=0x7fd9740027e0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 msgr2=0x5629f58358c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 --2- 172.21.15.97:0/879430197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629f58ebcc0 0x5629f58358c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0ab7a0 tx=0x7fd97c0acaf0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd989e00640 1 -- 172.21.15.97:0/879430197 reap_dead start 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 shutdown_connections 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 >> 172.21.15.97:0/879430197 conn(0x5629f587b0d0 msgr2=0x5629f58ede40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 shutdown_connections 2024-09-10T21:32:43.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.095+0000 7fd98ce48080 1 -- 172.21.15.97:0/879430197 wait complete. 2024-09-10T21:32:43.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- 172.21.15.97:0/3618585702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df015bdd0 msgr2=0x7f5df0074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 --2- 172.21.15.97:0/3618585702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df015bdd0 0x7f5df0074380 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5df00ce960 tx=0x7f5de401c890 comp rx=0 tx=0).stop 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- 172.21.15.97:0/3618585702 shutdown_connections 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 --2- 172.21.15.97:0/3618585702 >> v2:172.21.15.194:3300/0 conn(0x7f5df006cb40 0x7f5df006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 --2- 172.21.15.97:0/3618585702 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df00cfad0 0x7f5df00cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 --2- 172.21.15.97:0/3618585702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df015bdd0 0x7f5df0074380 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- 172.21.15.97:0/3618585702 >> 172.21.15.97:0/3618585702 conn(0x7f5df00b28c0 msgr2=0x7f5df00b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- 172.21.15.97:0/3618585702 shutdown_connections 2024-09-10T21:32:43.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- 172.21.15.97:0/3618585702 wait complete. 2024-09-10T21:32:43.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 Processor -- start 2024-09-10T21:32:43.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- start start 2024-09-10T21:32:43.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df006cb40 0x7f5df0149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df00cfad0 0x7f5df014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df015bdd0 0x7f5df0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5df00af910 con 0x7f5df006cb40 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5df00cd6f0 con 0x7f5df00cfad0 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df8000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5df00afa90 con 0x7f5df015bdd0 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df006cb40 0x7f5df0149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df006cb40 0x7f5df0149880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35058/0 (socket says 172.21.15.97:35058) 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df5600640 1 -- 172.21.15.97:0/4162665024 learned_addr learned my addr 172.21.15.97:0/4162665024 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df4c00640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df00cfad0 0x7f5df014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.287+0000 7f5df6000640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df015bdd0 0x7f5df0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df5600640 1 -- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df015bdd0 msgr2=0x7f5df0159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df5600640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df015bdd0 0x7f5df0159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df5600640 1 -- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df00cfad0 msgr2=0x7f5df014d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df5600640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df00cfad0 0x7f5df014d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df5600640 1 -- 172.21.15.97:0/4162665024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df01f9470 con 0x7f5df006cb40 2024-09-10T21:32:43.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df5600640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df006cb40 0x7f5df0149880 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5de40b0040 tx=0x7f5de4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5ded600640 1 -- 172.21.15.97:0/4162665024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40b2040 con 0x7f5df006cb40 2024-09-10T21:32:43.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5ded600640 1 -- 172.21.15.97:0/4162665024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de4004430 con 0x7f5df006cb40 2024-09-10T21:32:43.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5ded600640 1 -- 172.21.15.97:0/4162665024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4005290 con 0x7f5df006cb40 2024-09-10T21:32:43.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df01f9750 con 0x7f5df006cb40 2024-09-10T21:32:43.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01f9d10 con 0x7f5df006cb40 2024-09-10T21:32:43.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df00cff10 con 0x7f5df006cb40 2024-09-10T21:32:43.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5ded600640 1 -- 172.21.15.97:0/4162665024 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5de4005430 con 0x7f5df006cb40 2024-09-10T21:32:43.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5ded600640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5dd00725e0 0x7f5dd0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.291+0000 7f5ded600640 1 -- 172.21.15.97:0/4162665024 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7565+0+0 (secure 0 0 0) 0x7f5de412d3b0 con 0x7f5df006cb40 2024-09-10T21:32:43.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.295+0000 7f5df4c00640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5dd00725e0 0x7f5dd0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.295+0000 7f5df4c00640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5dd00725e0 0x7f5dd0074aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5df00b1a20 tx=0x7f5ddc00b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.295+0000 7f5ded600640 1 -- 172.21.15.97:0/4162665024 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5de40164b0 con 0x7f5df006cb40 2024-09-10T21:32:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.495+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f5df009f0f0 con 0x7f5df006cb40 2024-09-10T21:32:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.731+0000 7f5ded600640 1 -- 172.21.15.97:0/4162665024 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v178) ==== 162+0+0 (secure 0 0 0) 0x7f5de40f7d90 con 0x7f5df006cb40 2024-09-10T21:32:43.731 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5dd00725e0 msgr2=0x7f5dd0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5dd00725e0 0x7f5dd0074aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5df00b1a20 tx=0x7f5ddc00b390 comp rx=0 tx=0).stop 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df006cb40 msgr2=0x7f5df0149880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df006cb40 0x7f5df0149880 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5de40b0040 tx=0x7f5de4004040 comp rx=0 tx=0).stop 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 shutdown_connections 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5dd00725e0 0x7f5dd0074aa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5df015bdd0 0x7f5df0159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df00cfad0 0x7f5df014d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 --2- 172.21.15.97:0/4162665024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5df006cb40 0x7f5df0149880 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 >> 172.21.15.97:0/4162665024 conn(0x7f5df00b28c0 msgr2=0x7f5df01582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 shutdown_connections 2024-09-10T21:32:43.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.735+0000 7f5df8000640 1 -- 172.21.15.97:0/4162665024 wait complete. 2024-09-10T21:32:43.745 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:43.747+0000 7fb4ed600640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-10T21:32:43.745 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:43.747+0000 7fb4ed600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:32:43.746 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:43.747+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:43.746 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:43.747+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:32:43.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.899+0000 7f2889e00640 1 -- 172.21.15.97:0/1400169757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840b0d50 msgr2=0x7f28840ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.899+0000 7f2889e00640 1 --2- 172.21.15.97:0/1400169757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840b0d50 0x7f28840ccde0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f287c009c80 tx=0x7f287c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.899+0000 7f2889e00640 1 -- 172.21.15.97:0/1400169757 shutdown_connections 2024-09-10T21:32:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.899+0000 7f2889e00640 1 --2- 172.21.15.97:0/1400169757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840b0d50 0x7f28840ccde0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.899+0000 7f2889e00640 1 -- 172.21.15.97:0/1400169757 >> 172.21.15.97:0/1400169757 conn(0x7f28840d1c30 msgr2=0x7f28840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.899+0000 7f2889e00640 1 -- 172.21.15.97:0/1400169757 shutdown_connections 2024-09-10T21:32:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.899+0000 7f2889e00640 1 -- 172.21.15.97:0/1400169757 wait complete. 2024-09-10T21:32:43.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 Processor -- start 2024-09-10T21:32:43.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 -- start start 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840afdd0 0x7f2884151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28840b0d50 0x7f288414b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28840ce640 0x7f288414b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2884169ff0 con 0x7f28840afdd0 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2884169e70 con 0x7f28840ce640 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f288416a170 con 0x7f28840b0d50 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840afdd0 0x7f2884151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840afdd0 0x7f2884151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35080/0 (socket says 172.21.15.97:35080) 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 -- 172.21.15.97:0/2263593451 learned_addr learned my addr 172.21.15.97:0/2263593451 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882000640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28840b0d50 0x7f288414b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2883400640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28840ce640 0x7f288414b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 -- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28840b0d50 msgr2=0x7f288414b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28840b0d50 0x7f288414b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 -- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28840ce640 msgr2=0x7f288414b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28840ce640 0x7f288414b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 -- 172.21.15.97:0/2263593451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288414ff30 con 0x7f28840afdd0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882000640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28840b0d50 0x7f288414b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2883400640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28840ce640 0x7f288414b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2882a00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840afdd0 0x7f2884151700 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f287400b410 tx=0x7f287400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f287b400640 1 -- 172.21.15.97:0/2263593451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874015030 con 0x7f28840afdd0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f287b400640 1 -- 172.21.15.97:0/2263593451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2874004020 con 0x7f28840afdd0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f287b400640 1 -- 172.21.15.97:0/2263593451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874004470 con 0x7f28840afdd0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28841501f0 con 0x7f28840afdd0 2024-09-10T21:32:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28841507b0 con 0x7f28840afdd0 2024-09-10T21:32:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.903+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28840cea20 con 0x7f28840afdd0 2024-09-10T21:32:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.907+0000 7f287b400640 1 -- 172.21.15.97:0/2263593451 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2874022060 con 0x7f28840afdd0 2024-09-10T21:32:43.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.907+0000 7f287b400640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2864072610 0x7f2864074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:43.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.907+0000 7f287b400640 1 -- 172.21.15.97:0/2263593451 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7f2874093590 con 0x7f28840afdd0 2024-09-10T21:32:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.907+0000 7f2882000640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2864072610 0x7f2864074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.907+0000 7f2882000640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2864072610 0x7f2864074ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f288415b980 tx=0x7f2870009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:43.911+0000 7f287b400640 1 -- 172.21.15.97:0/2263593451 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f287405e430 con 0x7f28840afdd0 2024-09-10T21:32:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.107+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28841580b0 con 0x7f28840afdd0 2024-09-10T21:32:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.107+0000 7f287b400640 1 -- 172.21.15.97:0/2263593451 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 180 v180) ==== 95+0+4395 (secure 0 0 0) 0x7f287405dd30 con 0x7f28840afdd0 2024-09-10T21:32:44.108 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:44.109 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":180,"btime":"2024-09-10T21:32:43:749504+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1863388792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":1863388792},{"type":"v1","addr":"172.21.15.97:6835","nonce":1863388792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":180,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:43.749443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:44.109 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 180 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2864072610 msgr2=0x7f2864074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2864072610 0x7f2864074ad0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f288415b980 tx=0x7f2870009040 comp rx=0 tx=0).stop 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840afdd0 msgr2=0x7f2884151700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840afdd0 0x7f2884151700 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f287400b410 tx=0x7f287400b8d0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 shutdown_connections 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2864072610 0x7f2864074ad0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28840ce640 0x7f288414b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f28840b0d50 0x7f288414b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 --2- 172.21.15.97:0/2263593451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28840afdd0 0x7f2884151700 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 >> 172.21.15.97:0/2263593451 conn(0x7f28840d1c30 msgr2=0x7f28840cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 shutdown_connections 2024-09-10T21:32:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.111+0000 7f2889e00640 1 -- 172.21.15.97:0/2263593451 wait complete. 2024-09-10T21:32:44.129 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:44.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:44.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 -- 172.21.15.97:0/2576169970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80ce640 msgr2=0x7f33c80cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 --2- 172.21.15.97:0/2576169970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80ce640 0x7f33c80cea20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f33c80c8c90 tx=0x7f33b801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 -- 172.21.15.97:0/2576169970 shutdown_connections 2024-09-10T21:32:44.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 --2- 172.21.15.97:0/2576169970 >> v2:172.21.15.194:3300/0 conn(0x7f33c80cd6e0 0x7f33c80af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 --2- 172.21.15.97:0/2576169970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80ce640 0x7f33c80cea20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 -- 172.21.15.97:0/2576169970 >> 172.21.15.97:0/2576169970 conn(0x7f33c80d1c30 msgr2=0x7f33c80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:44.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 -- 172.21.15.97:0/2576169970 shutdown_connections 2024-09-10T21:32:44.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.271+0000 7f33cee00640 1 -- 172.21.15.97:0/2576169970 wait complete. 2024-09-10T21:32:44.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 Processor -- start 2024-09-10T21:32:44.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 -- start start 2024-09-10T21:32:44.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33c80b0d50 0x7f33c8165000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80cd6e0 0x7f33c80b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33c80ce640 0x7f33c80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8168d80 con 0x7f33c80cd6e0 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8168c00 con 0x7f33c80ce640 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f33c8168f00 con 0x7f33c80b0d50 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80cd6e0 0x7f33c80b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80cd6e0 0x7f33c80b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35094/0 (socket says 172.21.15.97:35094) 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 -- 172.21.15.97:0/4078845713 learned_addr learned my addr 172.21.15.97:0/4078845713 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33c7e00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33c80b0d50 0x7f33c8165000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cce00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33c80ce640 0x7f33c80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 -- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33c80b0d50 msgr2=0x7f33c8165000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33c80b0d50 0x7f33c8165000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 -- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33c80ce640 msgr2=0x7f33c80b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33c80ce640 0x7f33c80b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 -- 172.21.15.97:0/4078845713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c815c4c0 con 0x7f33c80cd6e0 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33bf400640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80cd6e0 0x7f33c80b6fc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c450 tx=0x7f33b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33c4c00640 1 -- 172.21.15.97:0/4078845713 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0012830 con 0x7f33c80cd6e0 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33c4c00640 1 -- 172.21.15.97:0/4078845713 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b0017070 con 0x7f33c80cd6e0 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c8165540 con 0x7f33c80cd6e0 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33c4c00640 1 -- 172.21.15.97:0/4078845713 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0012d80 con 0x7f33c80cd6e0 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.275+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c8164800 con 0x7f33c80cd6e0 2024-09-10T21:32:44.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.279+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8164620 con 0x7f33c80cd6e0 2024-09-10T21:32:44.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.279+0000 7f33c4c00640 1 -- 172.21.15.97:0/4078845713 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f33b001c460 con 0x7f33c80cd6e0 2024-09-10T21:32:44.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.279+0000 7f33c4c00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33a40725e0 0x7f33a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.279+0000 7f33c4c00640 1 -- 172.21.15.97:0/4078845713 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7f33b0093720 con 0x7f33c80cd6e0 2024-09-10T21:32:44.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.279+0000 7f33c7e00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33a40725e0 0x7f33a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.279+0000 7f33c7e00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33a40725e0 0x7f33a4074aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f33b80af000 tx=0x7f33b8019660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:44.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.283+0000 7f33c4c00640 1 -- 172.21.15.97:0/4078845713 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f33c8164620 con 0x7f33c80cd6e0 2024-09-10T21:32:44.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c80cea20 con 0x7f33c80cd6e0 2024-09-10T21:32:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33c4c00640 1 -- 172.21.15.97:0/4078845713 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 180 v180) ==== 95+0+4395 (secure 0 0 0) 0x7f33b005e000 con 0x7f33c80cd6e0 2024-09-10T21:32:44.482 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:44.482 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":180,"btime":"2024-09-10T21:32:43:749504+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1863388792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":1863388792},{"type":"v1","addr":"172.21.15.97:6835","nonce":1863388792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":180,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:43.749443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:44.482 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 180 2024-09-10T21:32:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33a40725e0 msgr2=0x7f33a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33a40725e0 0x7f33a4074aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f33b80af000 tx=0x7f33b8019660 comp rx=0 tx=0).stop 2024-09-10T21:32:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80cd6e0 msgr2=0x7f33c80b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80cd6e0 0x7f33c80b6fc0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c450 tx=0x7f33b000c910 comp rx=0 tx=0).stop 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 shutdown_connections 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33a40725e0 0x7f33a4074aa0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33c80ce640 0x7f33c80b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c80cd6e0 0x7f33c80b6fc0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 --2- 172.21.15.97:0/4078845713 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33c80b0d50 0x7f33c8165000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 >> 172.21.15.97:0/4078845713 conn(0x7f33c80d1c30 msgr2=0x7f33c80aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 shutdown_connections 2024-09-10T21:32:44.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.483+0000 7f33cee00640 1 -- 172.21.15.97:0/4078845713 wait complete. 2024-09-10T21:32:44.504 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:44.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:44.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 -- 172.21.15.97:0/1824979385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce9d0 msgr2=0x7f44b015f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 --2- 172.21.15.97:0/1824979385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce9d0 0x7f44b015f6d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f44b0074ae0 tx=0x7f449c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 -- 172.21.15.97:0/1824979385 shutdown_connections 2024-09-10T21:32:44.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 --2- 172.21.15.97:0/1824979385 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b015fc10 0x7f44b015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 --2- 172.21.15.97:0/1824979385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce9d0 0x7f44b015f6d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 --2- 172.21.15.97:0/1824979385 >> v2:172.21.15.194:3300/0 conn(0x7f44b00ce5f0 0x7f44b0141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 -- 172.21.15.97:0/1824979385 >> 172.21.15.97:0/1824979385 conn(0x7f44b00b1780 msgr2=0x7f44b00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:44.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 -- 172.21.15.97:0/1824979385 shutdown_connections 2024-09-10T21:32:44.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.647+0000 7f44b7400640 1 -- 172.21.15.97:0/1824979385 wait complete. 2024-09-10T21:32:44.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 Processor -- start 2024-09-10T21:32:44.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 -- start start 2024-09-10T21:32:44.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce5f0 0x7f44b013c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b00ce9d0 0x7f44b00b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b015fc10 0x7f44b0153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0140200 con 0x7f44b00ce5f0 2024-09-10T21:32:44.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0140080 con 0x7f44b015fc10 2024-09-10T21:32:44.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f44b0140380 con 0x7f44b00ce9d0 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce5f0 0x7f44b013c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce5f0 0x7f44b013c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35116/0 (socket says 172.21.15.97:35116) 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 -- 172.21.15.97:0/3583262569 learned_addr learned my addr 172.21.15.97:0/3583262569 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44afe00640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b00ce9d0 0x7f44b00b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b5400640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b015fc10 0x7f44b0153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 -- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b00ce9d0 msgr2=0x7f44b00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b00ce9d0 0x7f44b00b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 -- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b015fc10 msgr2=0x7f44b0153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b015fc10 0x7f44b0153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 -- 172.21.15.97:0/3583262569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b013c5e0 con 0x7f44b00ce5f0 2024-09-10T21:32:44.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b4a00640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce5f0 0x7f44b013c0a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f44a000ead0 tx=0x7f44a000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:44.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44acc00640 1 -- 172.21.15.97:0/3583262569 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a000ce40 con 0x7f44b00ce5f0 2024-09-10T21:32:44.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44acc00640 1 -- 172.21.15.97:0/3583262569 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44a0011560 con 0x7f44b00ce5f0 2024-09-10T21:32:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b0154060 con 0x7f44b00ce5f0 2024-09-10T21:32:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44acc00640 1 -- 172.21.15.97:0/3583262569 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0011990 con 0x7f44b00ce5f0 2024-09-10T21:32:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.651+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b0154620 con 0x7f44b00ce5f0 2024-09-10T21:32:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.655+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b015f6d0 con 0x7f44b00ce5f0 2024-09-10T21:32:44.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.655+0000 7f44acc00640 1 -- 172.21.15.97:0/3583262569 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f44a0022030 con 0x7f44b00ce5f0 2024-09-10T21:32:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.655+0000 7f44acc00640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44900725e0 0x7f4490074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.655+0000 7f44acc00640 1 -- 172.21.15.97:0/3583262569 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7f44a0094300 con 0x7f44b00ce5f0 2024-09-10T21:32:44.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.655+0000 7f44afe00640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44900725e0 0x7f4490074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:44.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.655+0000 7f44afe00640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44900725e0 0x7f4490074aa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f449c0af000 tx=0x7f449c01b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:44.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.659+0000 7f44acc00640 1 -- 172.21.15.97:0/3583262569 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44a0016030 con 0x7f44b00ce5f0 2024-09-10T21:32:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.859+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44b0141360 con 0x7f44b00ce5f0 2024-09-10T21:32:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.859+0000 7f44acc00640 1 -- 172.21.15.97:0/3583262569 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 180 v180) ==== 95+0+4395 (secure 0 0 0) 0x7f44a005f230 con 0x7f44b00ce5f0 2024-09-10T21:32:44.859 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:44.859 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":180,"btime":"2024-09-10T21:32:43:749504+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1863388792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":1863388792},{"type":"v1","addr":"172.21.15.97:6835","nonce":1863388792}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":180,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:43.749443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:44.860 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 180 2024-09-10T21:32:44.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44900725e0 msgr2=0x7f4490074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44900725e0 0x7f4490074aa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f449c0af000 tx=0x7f449c01b100 comp rx=0 tx=0).stop 2024-09-10T21:32:44.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce5f0 msgr2=0x7f44b013c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce5f0 0x7f44b013c0a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f44a000ead0 tx=0x7f44a000ef90 comp rx=0 tx=0).stop 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 shutdown_connections 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44900725e0 0x7f4490074aa0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b015fc10 0x7f44b0153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b00ce9d0 0x7f44b00b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 --2- 172.21.15.97:0/3583262569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b00ce5f0 0x7f44b013c0a0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 >> 172.21.15.97:0/3583262569 conn(0x7f44b00b1780 msgr2=0x7f44b015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 shutdown_connections 2024-09-10T21:32:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:44.863+0000 7f44b7400640 1 -- 172.21.15.97:0/3583262569 wait complete. 2024-09-10T21:32:44.880 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2024-09-10T21:32:44.881 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2024-09-10T21:32:44.881 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:44.881 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:44.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:32:44.890 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:44.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:44.890 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:44.939 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:32:44.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:32:44.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:45.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- 172.21.15.97:0/3434302362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc15fc10 msgr2=0x7f2abc13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3434302362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc15fc10 0x7f2abc13f9b0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f2abc074800 tx=0x7f2ab401cb60 comp rx=0 tx=0).stop 2024-09-10T21:32:45.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- 172.21.15.97:0/3434302362 shutdown_connections 2024-09-10T21:32:45.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3434302362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc15fc10 0x7f2abc13f9b0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3434302362 >> v2:172.21.15.194:3300/0 conn(0x7f2abc141d20 0x7f2abc15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- 172.21.15.97:0/3434302362 >> 172.21.15.97:0/3434302362 conn(0x7f2abc0b1780 msgr2=0x7f2abc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:45.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- 172.21.15.97:0/3434302362 shutdown_connections 2024-09-10T21:32:45.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- 172.21.15.97:0/3434302362 wait complete. 2024-09-10T21:32:45.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 Processor -- start 2024-09-10T21:32:45.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- start start 2024-09-10T21:32:45.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2abc0ce5f0 0x7f2abc0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc141d20 0x7f2abc0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2abc15fc10 0x7f2abc13b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc168a70 con 0x7f2abc141d20 2024-09-10T21:32:45.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc140cd0 con 0x7f2abc15fc10 2024-09-10T21:32:45.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac4200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2abc168bf0 con 0x7f2abc0ce5f0 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc141d20 0x7f2abc0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc141d20 0x7f2abc0b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35130/0 (socket says 172.21.15.97:35130) 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 -- 172.21.15.97:0/3049628224 learned_addr learned my addr 172.21.15.97:0/3049628224 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac1800640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2abc0ce5f0 0x7f2abc0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac2200640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2abc15fc10 0x7f2abc13b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 -- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2abc0ce5f0 msgr2=0x7f2abc0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2abc0ce5f0 0x7f2abc0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 -- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2abc15fc10 msgr2=0x7f2abc13b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2abc15fc10 0x7f2abc13b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac0e00640 1 -- 172.21.15.97:0/3049628224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc1fb920 con 0x7f2abc141d20 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.131+0000 7f2ac1800640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2abc0ce5f0 0x7f2abc0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:45.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ac0e00640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc141d20 0x7f2abc0b2b80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d950 tx=0x7f2aa800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:45.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ab9600640 1 -- 172.21.15.97:0/3049628224 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8017070 con 0x7f2abc141d20 2024-09-10T21:32:45.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ab9600640 1 -- 172.21.15.97:0/3049628224 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aa8009d20 con 0x7f2abc141d20 2024-09-10T21:32:45.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ab9600640 1 -- 172.21.15.97:0/3049628224 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80106c0 con 0x7f2abc141d20 2024-09-10T21:32:45.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc164cb0 con 0x7f2abc141d20 2024-09-10T21:32:45.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc13fa20 con 0x7f2abc141d20 2024-09-10T21:32:45.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc15f6d0 con 0x7f2abc141d20 2024-09-10T21:32:45.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ab9600640 1 -- 172.21.15.97:0/3049628224 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2aa8010860 con 0x7f2abc141d20 2024-09-10T21:32:45.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ab9600640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a9c0725e0 0x7f2a9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ab9600640 1 -- 172.21.15.97:0/3049628224 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7f2aa8093db0 con 0x7f2abc141d20 2024-09-10T21:32:45.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.135+0000 7f2ac1800640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a9c0725e0 0x7f2a9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.139+0000 7f2ac1800640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a9c0725e0 0x7f2a9c074aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0007fd0 tx=0x7f2ab0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:45.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.139+0000 7f2ab9600640 1 -- 172.21.15.97:0/3049628224 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2aa8098070 con 0x7f2abc141d20 2024-09-10T21:32:45.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.339+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2abc0cca70 con 0x7f2abc141d20 2024-09-10T21:32:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ab9600640 1 -- 172.21.15.97:0/3049628224 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v181) ==== 161+0+0 (secure 0 0 0) 0x7f2aa805e690 con 0x7f2abc141d20 2024-09-10T21:32:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a9c0725e0 msgr2=0x7f2a9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a9c0725e0 0x7f2a9c074aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0007fd0 tx=0x7f2ab0009040 comp rx=0 tx=0).stop 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc141d20 msgr2=0x7f2abc0b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc141d20 0x7f2abc0b2b80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d950 tx=0x7f2aa800de10 comp rx=0 tx=0).stop 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 shutdown_connections 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a9c0725e0 0x7f2a9c074aa0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2abc15fc10 0x7f2abc13b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc141d20 0x7f2abc0b2b80 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 --2- 172.21.15.97:0/3049628224 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2abc0ce5f0 0x7f2abc0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.763+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 >> 172.21.15.97:0/3049628224 conn(0x7f2abc0b1780 msgr2=0x7f2abc13a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.767+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 shutdown_connections 2024-09-10T21:32:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.767+0000 7f2ac4200640 1 -- 172.21.15.97:0/3049628224 wait complete. 2024-09-10T21:32:45.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 -- 172.21.15.97:0/712689130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15281511f0 msgr2=0x7f15280b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 --2- 172.21.15.97:0/712689130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15281511f0 0x7f15280b37b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f151800aa40 tx=0x7f1518020400 comp rx=0 tx=0).stop 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 -- 172.21.15.97:0/712689130 shutdown_connections 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 --2- 172.21.15.97:0/712689130 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f15280ceee0 0x7f15280cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 --2- 172.21.15.97:0/712689130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15281511f0 0x7f15280b37b0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 -- 172.21.15.97:0/712689130 >> 172.21.15.97:0/712689130 conn(0x7f15280b1780 msgr2=0x7f15280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 -- 172.21.15.97:0/712689130 shutdown_connections 2024-09-10T21:32:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.927+0000 7f152da00640 1 -- 172.21.15.97:0/712689130 wait complete. 2024-09-10T21:32:45.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 Processor -- start 2024-09-10T21:32:45.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 -- start start 2024-09-10T21:32:45.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15280b3cf0 0x7f15281f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15280ceee0 0x7f15281f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f15281511f0 0x7f15281f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f152806e740 con 0x7f15280b3cf0 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f152806e5c0 con 0x7f15280ceee0 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f152806e8c0 con 0x7f15281511f0 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15280b3cf0 0x7f15281f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15280b3cf0 0x7f15281f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35158/0 (socket says 172.21.15.97:35158) 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 -- 172.21.15.97:0/2405339334 learned_addr learned my addr 172.21.15.97:0/2405339334 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1527400640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f15281511f0 0x7f15281f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526000640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15280ceee0 0x7f15281f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 -- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f15281511f0 msgr2=0x7f15281f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f15281511f0 0x7f15281f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 -- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15280ceee0 msgr2=0x7f15281f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15280ceee0 0x7f15281f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 -- 172.21.15.97:0/2405339334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15281f4c70 con 0x7f15280b3cf0 2024-09-10T21:32:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1527400640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f15281511f0 0x7f15281f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f1526a00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15280b3cf0 0x7f15281f2390 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f151800a960 tx=0x7f1518005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f151ea00640 1 -- 172.21.15.97:0/2405339334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15180b5040 con 0x7f15280b3cf0 2024-09-10T21:32:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f151ea00640 1 -- 172.21.15.97:0/2405339334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1518005310 con 0x7f15280b3cf0 2024-09-10T21:32:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f151ea00640 1 -- 172.21.15.97:0/2405339334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15180b08a0 con 0x7f15280b3cf0 2024-09-10T21:32:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.931+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15281f5c40 con 0x7f15280b3cf0 2024-09-10T21:32:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.935+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15281f8fa0 con 0x7f15280b3cf0 2024-09-10T21:32:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.935+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15280cf370 con 0x7f15280b3cf0 2024-09-10T21:32:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.935+0000 7f151ea00640 1 -- 172.21.15.97:0/2405339334 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f15180b8070 con 0x7f15280b3cf0 2024-09-10T21:32:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.935+0000 7f151ea00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f15080725e0 0x7f1508074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.935+0000 7f151ea00640 1 -- 172.21.15.97:0/2405339334 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7f1518130dd0 con 0x7f15280b3cf0 2024-09-10T21:32:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.935+0000 7f1526000640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f15080725e0 0x7f1508074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.935+0000 7f1526000640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f15080725e0 0x7f1508074aa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f15280cd210 tx=0x7f1514009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:45.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:45.939+0000 7f151ea00640 1 -- 172.21.15.97:0/2405339334 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1518019f60 con 0x7f15280b3cf0 2024-09-10T21:32:46.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.163+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f15281518a0 con 0x7f15280b3cf0 2024-09-10T21:32:46.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f151ea00640 1 -- 172.21.15.97:0/2405339334 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v182) ==== 101+0+0 (secure 0 0 0) 0x7f15180fbd00 con 0x7f15280b3cf0 2024-09-10T21:32:46.769 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f15080725e0 msgr2=0x7f1508074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f15080725e0 0x7f1508074aa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f15280cd210 tx=0x7f1514009040 comp rx=0 tx=0).stop 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15280b3cf0 msgr2=0x7f15281f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15280b3cf0 0x7f15281f2390 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f151800a960 tx=0x7f1518005760 comp rx=0 tx=0).stop 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 shutdown_connections 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f15080725e0 0x7f1508074aa0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f15281511f0 0x7f15281f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15280ceee0 0x7f15281f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 --2- 172.21.15.97:0/2405339334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15280b3cf0 0x7f15281f2390 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 >> 172.21.15.97:0/2405339334 conn(0x7f15280b1780 msgr2=0x7f152815c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:46.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 shutdown_connections 2024-09-10T21:32:46.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.771+0000 7f152da00640 1 -- 172.21.15.97:0/2405339334 wait complete. 2024-09-10T21:32:46.793 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 Processor -- start 2024-09-10T21:32:46.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 -- start start 2024-09-10T21:32:46.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f31400023a0 0x5557079205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55570787c550 0x555707896690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555707896bd0 0x55570784b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55570779aa00 con 0x555707896bd0 2024-09-10T21:32:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5557077f8f40 con 0x55570787c550 2024-09-10T21:32:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315dd85080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555707905ea0 con 0x7f31400023a0 2024-09-10T21:32:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555707896bd0 0x55570784b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555707896bd0 0x55570784b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35180/0 (socket says 172.21.15.97:35180) 2024-09-10T21:32:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.839+0000 7f315ba00640 1 -- 172.21.15.97:0/2952947849 learned_addr learned my addr 172.21.15.97:0/2952947849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f3153e00640 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55570787c550 0x555707896690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315b000640 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f31400023a0 0x5557079205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 -- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f31400023a0 msgr2=0x5557079205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f31400023a0 0x5557079205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 -- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55570787c550 msgr2=0x555707896690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55570787c550 0x555707896690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 -- 172.21.15.97:0/2952947849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555707773c10 con 0x555707896bd0 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555707896bd0 0x55570784b140 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x55570788d2c0 tx=0x7f3154009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1ebbf40f12f053 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f3159c00640 1 -- 172.21.15.97:0/2952947849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154007590 con 0x555707896bd0 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f3159c00640 1 -- 172.21.15.97:0/2952947849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3154005050 con 0x555707896bd0 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f3159c00640 1 -- 172.21.15.97:0/2952947849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154007ae0 con 0x555707896bd0 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- 172.21.15.97:0/2952947849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555707896bd0 msgr2=0x55570784b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:46.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555707896bd0 0x55570784b140 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x55570788d2c0 tx=0x7f3154009940 comp rx=0 tx=0).stop 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- 172.21.15.97:0/2952947849 shutdown_connections 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555707896bd0 0x55570784b140 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55570787c550 0x555707896690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 --2- 172.21.15.97:0/2952947849 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f31400023a0 0x5557079205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- 172.21.15.97:0/2952947849 >> 172.21.15.97:0/2952947849 conn(0x5557078940d0 msgr2=0x55570784e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- 172.21.15.97:0/2952947849 shutdown_connections 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- 172.21.15.97:0/2952947849 wait complete. 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 Processor -- start 2024-09-10T21:32:46.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- start start 2024-09-10T21:32:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55570787c550 0x555707922430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555707896bd0 0x55570784b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31400023a0 0x55570784cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555707895770 con 0x7f31400023a0 2024-09-10T21:32:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5557078954d0 con 0x555707896bd0 2024-09-10T21:32:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315dd85080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555707895a10 con 0x55570787c550 2024-09-10T21:32:46.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31400023a0 0x55570784cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31400023a0 0x55570784cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35182/0 (socket says 172.21.15.97:35182) 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 learned_addr learned my addr 172.21.15.97:0/1244548541 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55570787c550 0x555707922430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.843+0000 7f3153e00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555707896bd0 0x55570784b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55570787c550 msgr2=0x555707922430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315ba00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55570787c550 0x555707922430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555707896bd0 msgr2=0x55570784b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315ba00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555707896bd0 0x55570784b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570784bba0 con 0x7f31400023a0 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55570787c550 0x555707922430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315ba00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31400023a0 0x55570784cac0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3154009580 tx=0x7f3154008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f3153400640 1 -- 172.21.15.97:0/1244548541 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154015820 con 0x7f31400023a0 2024-09-10T21:32:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f3153400640 1 -- 172.21.15.97:0/1244548541 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3154005050 con 0x7f31400023a0 2024-09-10T21:32:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f3153400640 1 -- 172.21.15.97:0/1244548541 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154015d30 con 0x7f31400023a0 2024-09-10T21:32:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55570784be40 con 0x7f31400023a0 2024-09-10T21:32:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570784c380 con 0x7f31400023a0 2024-09-10T21:32:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f3153400640 1 -- 172.21.15.97:0/1244548541 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f31540133c0 con 0x7f31400023a0 2024-09-10T21:32:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f3153400640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3138070ff0 0x7f31380734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.847+0000 7f3153400640 1 -- 172.21.15.97:0/1244548541 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7f3154093400 con 0x7f31400023a0 2024-09-10T21:32:46.850 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:46.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55570784b680 0x5557079bd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3138070ff0 0x7f31380734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x555707922e30 con 0x55570784b680 2024-09-10T21:32:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f3153e00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55570784b680 0x5557079bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3138070ff0 0x7f31380734b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f314c0ab000 tx=0x7f314c018530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f3153e00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55570784b680 0x5557079bd580 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v112'474 uv448 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.851+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x555707923200 con 0x55570784b680 2024-09-10T21:32:46.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.855+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v112'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.855+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079235a0 con 0x55570784b680 2024-09-10T21:32:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:46.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.855+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v112'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.855+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557078f94f0 con 0x55570784b680 2024-09-10T21:32:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.859+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.859+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557078f9900 con 0x55570784b680 2024-09-10T21:32:46.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.859+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.859+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557078f9cd0 con 0x55570784b680 2024-09-10T21:32:46.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.859+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.859+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079ca690 con 0x55570784b680 2024-09-10T21:32:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.863+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v112'480 uv454 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.863+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.863+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v112'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.863+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.867+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v112'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.866 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:46.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.867+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.867+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.867+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.867+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'484 uv484 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.867+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.871+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'485 uv485 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.871+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.871+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v112'486 uv461 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.871+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.875+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'487 uv487 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.875+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.875+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v112'488 uv488 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.875 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.875+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.879+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'489 uv489 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.879+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.879+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'490 uv490 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.879+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.879+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'491 uv491 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.879+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.883+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'492 uv492 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.883+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.883+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v112'493 uv467 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.883+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.887+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v112'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.887+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.887 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:46.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.887+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v112'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.887+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.891+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.891+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.891+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.891+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.891+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.891+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.895+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v112'499 uv473 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.895+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.895+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v112'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.895+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.896 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:46.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.899+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v112'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.899+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.899+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.899+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cba40 con 0x55570784b680 2024-09-10T21:32:46.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.899+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.899+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079caa30 con 0x55570784b680 2024-09-10T21:32:46.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.903+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x55570784b680 2024-09-10T21:32:46.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.903+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5557079cb0b0 0x5557079cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.903+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cd9e0 con 0x5557079cb0b0 2024-09-10T21:32:46.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.903+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5557079cb0b0 0x5557079cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.903+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5557079cb0b0 0x5557079cd4a0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.907+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'352 uv351 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.907+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079cdd80 con 0x5557079cb0b0 2024-09-10T21:32:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.907+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'353 uv353 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.907+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079ce120 con 0x5557079cb0b0 2024-09-10T21:32:46.907 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.907+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'354 uv354 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.907+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079ce970 con 0x5557079cb0b0 2024-09-10T21:32:46.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v112'355 uv355 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079ced10 con 0x5557079cb0b0 2024-09-10T21:32:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'356 uv356 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf0b0 con 0x5557079cb0b0 2024-09-10T21:32:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v112'357 uv339 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079cf450 con 0x5557079cb0b0 2024-09-10T21:32:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v112'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.911+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.914 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:46.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.915+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v112'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.915+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.915+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.915+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.915+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.915+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.919+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v112'362 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.919+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.919+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v112'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.919+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.919 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.919+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v112'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.919+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v112'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'366 uv366 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v112'367 uv350 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v112'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.923+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.925 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.927+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v112'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.927+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.927+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.927+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.927+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.927+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079cf7f0 con 0x5557079cb0b0 2024-09-10T21:32:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079cb0b0 2024-09-10T21:32:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5557079d00c0 0x5557079d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d29f0 con 0x5557079d00c0 2024-09-10T21:32:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f3153e00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5557079d00c0 0x5557079d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f3153e00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5557079d00c0 0x5557079d24b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079d2a30 con 0x5557079d00c0 2024-09-10T21:32:46.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.931+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d2e50 con 0x5557079d00c0 2024-09-10T21:32:46.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.935+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.933 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:46.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.935+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d3e60 con 0x5557079d00c0 2024-09-10T21:32:46.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.935+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.935+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d3e60 con 0x5557079d00c0 2024-09-10T21:32:46.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.935+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.935+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d2e50 con 0x5557079d00c0 2024-09-10T21:32:46.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5557079d34d0 0x5557079d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d5f50 con 0x5557079d34d0 2024-09-10T21:32:46.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315ba00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5557079d34d0 0x5557079d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315ba00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5557079d34d0 0x5557079d58c0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'75 uv74 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079d6460 con 0x5557079d34d0 2024-09-10T21:32:46.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'76 uv76 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.939+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d6960 con 0x5557079d34d0 2024-09-10T21:32:46.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.943+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'77 uv77 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.941 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:46.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.943+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d7220 con 0x5557079d34d0 2024-09-10T21:32:46.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.943+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'78 uv78 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.943+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d75c0 con 0x5557079d00c0 2024-09-10T21:32:46.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.943+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.943+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079d75c0 con 0x5557079d00c0 2024-09-10T21:32:46.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d75c0 con 0x5557079d00c0 2024-09-10T21:32:46.945 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d85d0 con 0x5557079d00c0 2024-09-10T21:32:46.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d85d0 con 0x5557079d00c0 2024-09-10T21:32:46.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.947+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d75c0 con 0x5557079d00c0 2024-09-10T21:32:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.951+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.951+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d75c0 con 0x5557079d34d0 2024-09-10T21:32:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.951+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v112'171 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.951+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079d7960 con 0x5557079d34d0 2024-09-10T21:32:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.951+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v112'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.951+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d7d00 con 0x5557079d34d0 2024-09-10T21:32:46.953 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.955+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v112'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.955+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d90b0 con 0x5557079d34d0 2024-09-10T21:32:46.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.955+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.955+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d90b0 con 0x5557079d34d0 2024-09-10T21:32:46.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.955+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.955+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d34d0 2024-09-10T21:32:46.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d34d0 2024-09-10T21:32:46.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v112'177 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d34d0 2024-09-10T21:32:46.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v112'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d34d0 2024-09-10T21:32:46.960 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:46.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v112'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.959+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d34d0 2024-09-10T21:32:46.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.963+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.963+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d34d0 2024-09-10T21:32:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.963+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.963+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d00c0 2024-09-10T21:32:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.963+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v112'76 uv75 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.963+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d00c0 2024-09-10T21:32:46.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v112'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d00c0 2024-09-10T21:32:46.966 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v112'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d00c0 2024-09-10T21:32:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'79 uv79 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d80a0 con 0x5557079d00c0 2024-09-10T21:32:46.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'80 uv80 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079d00c0 2024-09-10T21:32:46.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5557079d8720 0x5557079dab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:46.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.967+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db0e0 con 0x5557079d8720 2024-09-10T21:32:46.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.971+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5557079d8720 0x5557079dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:46.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.971+0000 7f315b000640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5557079d8720 0x5557079dab10 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.971+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.971+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d8720 2024-09-10T21:32:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.971+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.971+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d8720 2024-09-10T21:32:46.972 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:46.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dc4c0 con 0x5557079d8720 2024-09-10T21:32:46.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dc4c0 con 0x5557079d8720 2024-09-10T21:32:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d8720 2024-09-10T21:32:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.975+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d8720 2024-09-10T21:32:46.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.979+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v112'152 uv146 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.979+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d8720 2024-09-10T21:32:46.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.979+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v112'153 uv153 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.979+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d8720 2024-09-10T21:32:46.979 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.979+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v112'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.979+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d6e10 con 0x5557079d8720 2024-09-10T21:32:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.983+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.983+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d6e10 con 0x5557079d8720 2024-09-10T21:32:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.983+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.983+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d6e10 con 0x5557079d8720 2024-09-10T21:32:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.983+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v112'157 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.983+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079d6e10 con 0x5557079d8720 2024-09-10T21:32:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.983+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v112'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.987+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d6e10 con 0x5557079d8720 2024-09-10T21:32:46.985 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:46.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.987+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v112'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.987+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079d6e10 con 0x5557079d8720 2024-09-10T21:32:46.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.987+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.987+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079de390 con 0x5557079d8720 2024-09-10T21:32:46.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.987+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:46.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.987+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079de350 con 0x5557079d34d0 2024-09-10T21:32:46.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.991+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v112'70 uv69 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.991+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079de350 con 0x5557079d34d0 2024-09-10T21:32:46.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.991+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v112'71 uv71 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.991+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079de350 con 0x5557079d34d0 2024-09-10T21:32:46.992 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:46.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.991+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v112'72 uv72 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.991+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d34d0 2024-09-10T21:32:46.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.995+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v112'73 uv73 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.995+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d34d0 2024-09-10T21:32:46.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.995+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v112'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.995+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d34d0 2024-09-10T21:32:46.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.995+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v112'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.995+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d34d0 2024-09-10T21:32:46.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v112'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.997 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:46.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d34d0 2024-09-10T21:32:46.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v112'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d34d0 2024-09-10T21:32:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v112'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079db4b0 con 0x5557079d34d0 2024-09-10T21:32:47.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v112'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3154027070 con 0x5557079d34d0 2024-09-10T21:32:47.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5557079db9c0 0x5557079dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dea50 con 0x5557079db9c0 2024-09-10T21:32:47.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:46.999+0000 7f3153e00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5557079db9c0 0x5557079dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.003+0000 7f3153e00640 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5557079db9c0 0x5557079dddb0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.003+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v112'79 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079db9c0 2024-09-10T21:32:47.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.003+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079dea50 con 0x5557079db9c0 2024-09-10T21:32:47.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.003+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v112'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079db9c0 2024-09-10T21:32:47.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.003+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dea90 con 0x5557079db9c0 2024-09-10T21:32:47.004 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.003+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v112'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079db9c0 2024-09-10T21:32:47.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.003+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dee80 con 0x5557079db9c0 2024-09-10T21:32:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.007+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v112'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079db9c0 2024-09-10T21:32:47.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.007+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dee80 con 0x5557079db9c0 2024-09-10T21:32:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.007+0000 7f3153e00640 1 -- 172.21.15.97:0/1244548541 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v112'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f314400c610 con 0x5557079db9c0 2024-09-10T21:32:47.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.007+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079decc0 con 0x5557079d8720 2024-09-10T21:32:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.007+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v112'192 uv186 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.007+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079dec80 con 0x5557079d8720 2024-09-10T21:32:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v112'193 uv193 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079df130 con 0x5557079d8720 2024-09-10T21:32:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v112'194 uv194 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.010 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:47.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079df130 con 0x5557079d8720 2024-09-10T21:32:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v112'195 uv195 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079df090 con 0x5557079d8720 2024-09-10T21:32:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v112'196 uv196 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.011+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079ded80 con 0x5557079d8720 2024-09-10T21:32:47.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.015+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v112'197 uv191 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.015+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x5557079ded80 con 0x5557079d8720 2024-09-10T21:32:47.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.015+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v112'198 uv198 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.015+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079ded30 con 0x5557079d8720 2024-09-10T21:32:47.015 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.015+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v112'199 uv199 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.015+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dfd40 con 0x5557079d8720 2024-09-10T21:32:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v112'200 uv200 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dfd40 con 0x5557079d8720 2024-09-10T21:32:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v112'201 uv201 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079dec80 con 0x5557079d8720 2024-09-10T21:32:47.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v112'202 uv202 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x5557079df090 con 0x5557079d8720 2024-09-10T21:32:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315b000640 1 -- 172.21.15.97:0/1244548541 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v112'203 uv203 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f314c0bc070 con 0x5557079d8720 2024-09-10T21:32:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5557079d00c0 msgr2=0x5557079d24b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5557079d00c0 0x5557079d24b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5557079d8720 msgr2=0x5557079dab10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5557079d8720 0x5557079dab10 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5557079db9c0 msgr2=0x5557079dddb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5557079db9c0 0x5557079dddb0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5557079d34d0 msgr2=0x5557079d58c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5557079d34d0 0x5557079d58c0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5557079cb0b0 msgr2=0x5557079cd4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 reap_dead start 2024-09-10T21:32:47.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5557079cb0b0 0x5557079cd4a0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315ba00640 1 -- 172.21.15.97:0/1244548541 reap_dead start 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55570784b680 msgr2=0x5557079bd580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.019+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55570784b680 0x5557079bd580 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3138070ff0 msgr2=0x7f31380734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3138070ff0 0x7f31380734b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f314c0ab000 tx=0x7f314c018530 comp rx=0 tx=0).stop 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31400023a0 msgr2=0x55570784cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31400023a0 0x55570784cac0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f3154009580 tx=0x7f3154008ff0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 shutdown_connections 2024-09-10T21:32:47.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5557079cb0b0 0x5557079cd4a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55570784b680 0x5557079bd580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3138070ff0 0x7f31380734b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 --2- 172.21.15.97:0/1244548541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31400023a0 0x55570784cac0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 >> 172.21.15.97:0/1244548541 conn(0x5557078940d0 msgr2=0x555707929420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 shutdown_connections 2024-09-10T21:32:47.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.023+0000 7f315dd85080 1 -- 172.21.15.97:0/1244548541 wait complete. 2024-09-10T21:32:47.026 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2024-09-10T21:32:47.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 Processor -- start 2024-09-10T21:32:47.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 -- start start 2024-09-10T21:32:47.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128532ba70 0x56128536f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561285359ec0 0x5612853e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56128535a2a0 0x5612853e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561285277a00 con 0x56128535a2a0 2024-09-10T21:32:47.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612852d5f40 con 0x561285359ec0 2024-09-10T21:32:47.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb56482d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5612853e8560 con 0x56128532ba70 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56128535a2a0 0x5612853e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56128535a2a0 0x5612853e41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35206/0 (socket says 172.21.15.97:35206) 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 -- 172.21.15.97:0/795001071 learned_addr learned my addr 172.21.15.97:0/795001071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb561a00640 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128532ba70 0x56128536f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb561000640 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561285359ec0 0x5612853e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 -- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128532ba70 msgr2=0x56128536f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128532ba70 0x56128536f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 -- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561285359ec0 msgr2=0x5612853e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561285359ec0 0x5612853e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 -- 172.21.15.97:0/795001071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561285250c10 con 0x56128535a2a0 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb562400640 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56128535a2a0 0x5612853e41f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0ae040 tx=0x7fb55c018240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a883bb961f29395 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb55b400640 1 -- 172.21.15.97:0/795001071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0b0030 con 0x56128535a2a0 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb55b400640 1 -- 172.21.15.97:0/795001071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb55c004130 con 0x56128535a2a0 2024-09-10T21:32:47.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.115+0000 7fb55b400640 1 -- 172.21.15.97:0/795001071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0aba50 con 0x56128535a2a0 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- 172.21.15.97:0/795001071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56128535a2a0 msgr2=0x5612853e41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56128535a2a0 0x5612853e41f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0ae040 tx=0x7fb55c018240 comp rx=0 tx=0).stop 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- 172.21.15.97:0/795001071 shutdown_connections 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56128535a2a0 0x5612853e41f0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561285359ec0 0x5612853e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 --2- 172.21.15.97:0/795001071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128532ba70 0x56128536f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- 172.21.15.97:0/795001071 >> 172.21.15.97:0/795001071 conn(0x5612853710d0 msgr2=0x56128532a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:47.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- 172.21.15.97:0/795001071 shutdown_connections 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- 172.21.15.97:0/795001071 wait complete. 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 Processor -- start 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- start start 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56128532ba70 0x5612853dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561285359ec0 0x5612853f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128535a2a0 0x5612853f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561285370120 con 0x561285359ec0 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612853703c0 con 0x56128532ba70 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561285370660 con 0x56128535a2a0 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561285359ec0 0x5612853f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561285359ec0 0x5612853f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35212/0 (socket says 172.21.15.97:35212) 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 -- 172.21.15.97:0/1905120465 learned_addr learned my addr 172.21.15.97:0/1905120465 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561a00640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56128532ba70 0x5612853dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb562400640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128535a2a0 0x5612853f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 -- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128535a2a0 msgr2=0x5612853f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56128535a2a0 0x5612853f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 -- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56128532ba70 msgr2=0x5612853dceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56128532ba70 0x5612853dceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 -- 172.21.15.97:0/1905120465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128549b290 con 0x561285359ec0 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb561000640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561285359ec0 0x5612853f8270 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb55000c400 tx=0x7fb55000c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb55a000640 1 -- 172.21.15.97:0/1905120465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550018620 con 0x561285359ec0 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb55a000640 1 -- 172.21.15.97:0/1905120465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb550013030 con 0x561285359ec0 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb55a000640 1 -- 172.21.15.97:0/1905120465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550018b30 con 0x561285359ec0 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56128549b530 con 0x561285359ec0 2024-09-10T21:32:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.119+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128549ba70 con 0x561285359ec0 2024-09-10T21:32:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb55a000640 1 -- 172.21.15.97:0/1905120465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb55000f410 con 0x561285359ec0 2024-09-10T21:32:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb55a000640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb544070fc0 0x7fb544073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb55a000640 1 -- 172.21.15.97:0/1905120465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7fb550092b30 con 0x561285359ec0 2024-09-10T21:32:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb561a00640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb544070fc0 0x7fb544073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb56482d080 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561285498960 0x561285498d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x561285499280 con 0x561285498960 2024-09-10T21:32:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb562400640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561285498960 0x561285498d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb561a00640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb544070fc0 0x7fb544073480 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb55400c970 tx=0x7fb55400f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.123+0000 7fb562400640 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561285498960 0x561285498d40 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb562400640 1 -- 172.21.15.97:0/1905120465 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v112'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb55c002050 con 0x561285498960 2024-09-10T21:32:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561285498960 msgr2=0x561285498d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561285498960 0x561285498d40 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb544070fc0 msgr2=0x7fb544073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb544070fc0 0x7fb544073480 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb55400c970 tx=0x7fb55400f040 comp rx=0 tx=0).stop 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561285359ec0 msgr2=0x5612853f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 --2- 172.21.15.97:0/1905120465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561285359ec0 0x5612853f8270 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb55000c400 tx=0x7fb55000c4f0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb562400640 1 -- 172.21.15.97:0/1905120465 reap_dead start 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 shutdown_connections 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 >> 172.21.15.97:0/1905120465 conn(0x5612853710d0 msgr2=0x56128532a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 shutdown_connections 2024-09-10T21:32:47.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.127+0000 7fb56482d080 1 -- 172.21.15.97:0/1905120465 wait complete. 2024-09-10T21:32:47.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- 172.21.15.97:0/642987173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f481415eed0 msgr2=0x7f48140d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 --2- 172.21.15.97:0/642987173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f481415eed0 0x7f48140d27f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4804009840 tx=0x7f480401c7f0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- 172.21.15.97:0/642987173 shutdown_connections 2024-09-10T21:32:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 --2- 172.21.15.97:0/642987173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48140ccc60 0x7f48140b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 --2- 172.21.15.97:0/642987173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f481415eed0 0x7f48140d27f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- 172.21.15.97:0/642987173 >> 172.21.15.97:0/642987173 conn(0x7f48140d1cd0 msgr2=0x7f48140d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- 172.21.15.97:0/642987173 shutdown_connections 2024-09-10T21:32:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- 172.21.15.97:0/642987173 wait complete. 2024-09-10T21:32:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 Processor -- start 2024-09-10T21:32:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- start start 2024-09-10T21:32:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48140ccc60 0x7f48140b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48140cdd40 0x7f48140cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481415eed0 0x7f48140cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4814168880 con 0x7f481415eed0 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4814168700 con 0x7f48140cdd40 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f481b600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4814168a00 con 0x7f48140ccc60 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f4819600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481415eed0 0x7f48140cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f4818c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48140ccc60 0x7f48140b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f4819600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481415eed0 0x7f48140cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33268/0 (socket says 172.21.15.97:33268) 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f4819600640 1 -- 172.21.15.97:0/3244263520 learned_addr learned my addr 172.21.15.97:0/3244263520 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.323+0000 7f480be00640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48140cdd40 0x7f48140cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4818c00640 1 -- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48140cdd40 msgr2=0x7f48140cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4818c00640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48140cdd40 0x7f48140cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4818c00640 1 -- 172.21.15.97:0/3244263520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481415eed0 msgr2=0x7f48140cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4818c00640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481415eed0 0x7f48140cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4818c00640 1 -- 172.21.15.97:0/3244263520 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48141f78c0 con 0x7f48140ccc60 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4819600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481415eed0 0x7f48140cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4818c00640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48140ccc60 0x7f48140b5130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48140708e0 tx=0x7f48040a5730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4811600640 1 -- 172.21.15.97:0/3244263520 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040b2040 con 0x7f48140ccc60 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4811600640 1 -- 172.21.15.97:0/3244263520 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4804004470 con 0x7f48140ccc60 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4811600640 1 -- 172.21.15.97:0/3244263520 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804005280 con 0x7f48140ccc60 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48141fb9f0 con 0x7f48140ccc60 2024-09-10T21:32:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48141f7e60 con 0x7f48140ccc60 2024-09-10T21:32:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48141f76d0 con 0x7f48140ccc60 2024-09-10T21:32:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4811600640 1 -- 172.21.15.97:0/3244263520 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f48040b5070 con 0x7f48140ccc60 2024-09-10T21:32:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4811600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.327+0000 7f4811600640 1 -- 172.21.15.97:0/3244263520 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(112..112 src has 1..112) ==== 7651+0+0 (secure 0 0 0) 0x7f480412d200 con 0x7f48140ccc60 2024-09-10T21:32:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.331+0000 7f480be00640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.331+0000 7f480be00640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4814153630 tx=0x7f47fc001440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.331+0000 7f4811600640 1 -- 172.21.15.97:0/3244263520 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4804016410 con 0x7f48140ccc60 2024-09-10T21:32:47.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.535+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f48140b0490 con 0x7f48140ccc60 2024-09-10T21:32:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.775+0000 7f4811600640 1 -- 172.21.15.97:0/3244263520 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v183) ==== 162+0+0 (secure 0 0 0) 0x7f48040f7ae0 con 0x7f48140ccc60 2024-09-10T21:32:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 msgr2=0x7f47f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4814153630 tx=0x7f47fc001440 comp rx=0 tx=0).stop 2024-09-10T21:32:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48140ccc60 msgr2=0x7f48140b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48140ccc60 0x7f48140b5130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48140708e0 tx=0x7f48040a5730 comp rx=0 tx=0).stop 2024-09-10T21:32:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 shutdown_connections 2024-09-10T21:32:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f47f00725e0 0x7f47f0074aa0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f481415eed0 0x7f48140cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48140cdd40 0x7f48140cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 --2- 172.21.15.97:0/3244263520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f48140ccc60 0x7f48140b5130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 >> 172.21.15.97:0/3244263520 conn(0x7f48140d1cd0 msgr2=0x7f481414c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 shutdown_connections 2024-09-10T21:32:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.779+0000 7f481b600640 1 -- 172.21.15.97:0/3244263520 wait complete. 2024-09-10T21:32:47.782 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:32:47.783+0000 7f86e1c00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-10T21:32:47.783 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:47.783+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:47.783 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:47.783+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:32:47.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- 172.21.15.97:0/2014784496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d681511f0 msgr2=0x7f2d680b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 --2- 172.21.15.97:0/2014784496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d681511f0 0x7f2d680b37b0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009cb0 tx=0x7f2d5c01f5a0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- 172.21.15.97:0/2014784496 shutdown_connections 2024-09-10T21:32:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 --2- 172.21.15.97:0/2014784496 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d680ceee0 0x7f2d680cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 --2- 172.21.15.97:0/2014784496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d681511f0 0x7f2d680b37b0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- 172.21.15.97:0/2014784496 >> 172.21.15.97:0/2014784496 conn(0x7f2d680b1780 msgr2=0x7f2d680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- 172.21.15.97:0/2014784496 shutdown_connections 2024-09-10T21:32:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- 172.21.15.97:0/2014784496 wait complete. 2024-09-10T21:32:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 Processor -- start 2024-09-10T21:32:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- start start 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d680b3cf0 0x7f2d681f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d680ceee0 0x7f2d681e5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d681511f0 0x7f2d6815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6806e740 con 0x7f2d680b3cf0 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6806e5c0 con 0x7f2d681511f0 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d70200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6806e8c0 con 0x7f2d680ceee0 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d680b3cf0 0x7f2d681f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d680b3cf0 0x7f2d681f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33300/0 (socket says 172.21.15.97:33300) 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 -- 172.21.15.97:0/4266667045 learned_addr learned my addr 172.21.15.97:0/4266667045 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6e200640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d681511f0 0x7f2d6815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6ce00640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d680ceee0 0x7f2d681e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 -- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d680ceee0 msgr2=0x7f2d681e5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d680ceee0 0x7f2d681e5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 -- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d681511f0 msgr2=0x7f2d6815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d681511f0 0x7f2d6815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.947+0000 7f2d6d800640 1 -- 172.21.15.97:0/4266667045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d681f8a50 con 0x7f2d680b3cf0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d6d800640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d680b3cf0 0x7f2d681f83c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009bd0 tx=0x7f2d5c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d65600640 1 -- 172.21.15.97:0/4266667045 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c0af8a0 con 0x7f2d680b3cf0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d65600640 1 -- 172.21.15.97:0/4266667045 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d5c0afb30 con 0x7f2d680b3cf0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d65600640 1 -- 172.21.15.97:0/4266667045 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c004040 con 0x7f2d680b3cf0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681f8d30 con 0x7f2d680b3cf0 2024-09-10T21:32:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681e2d30 con 0x7f2d680b3cf0 2024-09-10T21:32:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d680cf370 con 0x7f2d680b3cf0 2024-09-10T21:32:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d65600640 1 -- 172.21.15.97:0/4266667045 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2d5c0041e0 con 0x7f2d680b3cf0 2024-09-10T21:32:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d65600640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2d480725e0 0x7f2d48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d65600640 1 -- 172.21.15.97:0/4266667045 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7f2d5c132830 con 0x7f2d680b3cf0 2024-09-10T21:32:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.951+0000 7f2d6ce00640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2d480725e0 0x7f2d48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.955+0000 7f2d6ce00640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2d480725e0 0x7f2d48074aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2d6806ab80 tx=0x7f2d54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:47.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:47.955+0000 7f2d65600640 1 -- 172.21.15.97:0/4266667045 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2d5c0c9060 con 0x7f2d680b3cf0 2024-09-10T21:32:48.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.155+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d680b4800 con 0x7f2d680b3cf0 2024-09-10T21:32:48.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.155+0000 7f2d65600640 1 -- 172.21.15.97:0/4266667045 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) ==== 95+0+3569 (secure 0 0 0) 0x7f2d5c0ac050 con 0x7f2d680b3cf0 2024-09-10T21:32:48.156 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:48.156 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":185,"btime":"2024-09-10T21:32:47:786713+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:47.786625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:48.156 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 185 2024-09-10T21:32:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2d480725e0 msgr2=0x7f2d48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2d480725e0 0x7f2d48074aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2d6806ab80 tx=0x7f2d54009040 comp rx=0 tx=0).stop 2024-09-10T21:32:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d680b3cf0 msgr2=0x7f2d681f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d680b3cf0 0x7f2d681f83c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009bd0 tx=0x7f2d5c0093d0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 shutdown_connections 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2d480725e0 0x7f2d48074aa0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d681511f0 0x7f2d6815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2d680ceee0 0x7f2d681e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 --2- 172.21.15.97:0/4266667045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d680b3cf0 0x7f2d681f83c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 >> 172.21.15.97:0/4266667045 conn(0x7f2d680b1780 msgr2=0x7f2d6815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 shutdown_connections 2024-09-10T21:32:48.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.159+0000 7f2d70200640 1 -- 172.21.15.97:0/4266667045 wait complete. 2024-09-10T21:32:48.178 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:48.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 -- 172.21.15.97:0/2343770392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940ce640 msgr2=0x7f79940cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 --2- 172.21.15.97:0/2343770392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940ce640 0x7f79940cea20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7984009840 tx=0x7f798401c820 comp rx=0 tx=0).stop 2024-09-10T21:32:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 -- 172.21.15.97:0/2343770392 shutdown_connections 2024-09-10T21:32:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 --2- 172.21.15.97:0/2343770392 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f79940cd6e0 0x7f79940af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 --2- 172.21.15.97:0/2343770392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940ce640 0x7f79940cea20 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 -- 172.21.15.97:0/2343770392 >> 172.21.15.97:0/2343770392 conn(0x7f79940d1c30 msgr2=0x7f79940d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:48.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 -- 172.21.15.97:0/2343770392 shutdown_connections 2024-09-10T21:32:48.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.319+0000 7f7999e00640 1 -- 172.21.15.97:0/2343770392 wait complete. 2024-09-10T21:32:48.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 Processor -- start 2024-09-10T21:32:48.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 -- start start 2024-09-10T21:32:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79940b0d50 0x7f79941f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940cd6e0 0x7f79940b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f79940ce640 0x7f79940c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7994168c70 con 0x7f79940cd6e0 2024-09-10T21:32:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7994168af0 con 0x7f79940b0d50 2024-09-10T21:32:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7994168df0 con 0x7f79940ce640 2024-09-10T21:32:48.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940cd6e0 0x7f79940b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940cd6e0 0x7f79940b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33314/0 (socket says 172.21.15.97:33314) 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 -- 172.21.15.97:0/2000431190 learned_addr learned my addr 172.21.15.97:0/2000431190 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992a00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79940b0d50 0x7f79941f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7993400640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f79940ce640 0x7f79940c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 -- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f79940ce640 msgr2=0x7f79940c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f79940ce640 0x7f79940c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 -- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79940b0d50 msgr2=0x7f79941f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79940b0d50 0x7f79941f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 -- 172.21.15.97:0/2000431190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79940b34f0 con 0x7f79940cd6e0 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7993400640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f79940ce640 0x7f79940c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:48.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7992000640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940cd6e0 0x7f79940b6100 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f798000c490 tx=0x7f798000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:48.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f798b400640 1 -- 172.21.15.97:0/2000431190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980016020 con 0x7f79940cd6e0 2024-09-10T21:32:48.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f798b400640 1 -- 172.21.15.97:0/2000431190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7980012940 con 0x7f79940cd6e0 2024-09-10T21:32:48.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79941f6ec0 con 0x7f79940cd6e0 2024-09-10T21:32:48.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f798b400640 1 -- 172.21.15.97:0/2000431190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980012d90 con 0x7f79940cd6e0 2024-09-10T21:32:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.323+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941f7480 con 0x7f79940cd6e0 2024-09-10T21:32:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.327+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79940cea20 con 0x7f79940cd6e0 2024-09-10T21:32:48.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.327+0000 7f798b400640 1 -- 172.21.15.97:0/2000431190 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f798001b400 con 0x7f79940cd6e0 2024-09-10T21:32:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.327+0000 7f798b400640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7974072610 0x7f7974074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.327+0000 7f798b400640 1 -- 172.21.15.97:0/2000431190 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7f79800939d0 con 0x7f79940cd6e0 2024-09-10T21:32:48.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.327+0000 7f7992a00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7974072610 0x7f7974074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.327+0000 7f7992a00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7974072610 0x7f7974074ad0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f79840af000 tx=0x7f79840195c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:48.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.331+0000 7f798b400640 1 -- 172.21.15.97:0/2000431190 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7980020030 con 0x7f79940cd6e0 2024-09-10T21:32:48.500 INFO:tasks.ceph.mds.a.smithi097.stderr: -40> 2024-09-10T21:32:43.747+0000 7fb4ed600640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-10T21:32:48.500 INFO:tasks.ceph.mds.a.smithi097.stderr: -38> 2024-09-10T21:32:43.747+0000 7fb4ed600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:32:48.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.531+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7994156040 con 0x7f79940cd6e0 2024-09-10T21:32:48.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f798b400640 1 -- 172.21.15.97:0/2000431190 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) ==== 95+0+3569 (secure 0 0 0) 0x7f798005e970 con 0x7f79940cd6e0 2024-09-10T21:32:48.533 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:48.533 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":185,"btime":"2024-09-10T21:32:47:786713+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:47.786625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:48.533 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 185 2024-09-10T21:32:48.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7974072610 msgr2=0x7f7974074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7974072610 0x7f7974074ad0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f79840af000 tx=0x7f79840195c0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940cd6e0 msgr2=0x7f79940b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940cd6e0 0x7f79940b6100 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f798000c490 tx=0x7f798000c950 comp rx=0 tx=0).stop 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 shutdown_connections 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7974072610 0x7f7974074ad0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f79940ce640 0x7f79940c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79940cd6e0 0x7f79940b6100 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 --2- 172.21.15.97:0/2000431190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79940b0d50 0x7f79941f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 >> 172.21.15.97:0/2000431190 conn(0x7f79940d1c30 msgr2=0x7f79940b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 shutdown_connections 2024-09-10T21:32:48.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.535+0000 7f7999e00640 1 -- 172.21.15.97:0/2000431190 wait complete. 2024-09-10T21:32:48.538 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:32:48.557 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:48.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:48.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- 172.21.15.97:0/939423883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 msgr2=0x7ffb44141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 --2- 172.21.15.97:0/939423883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb44141d20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009840 tx=0x7ffb3401c820 comp rx=0 tx=0).stop 2024-09-10T21:32:48.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- 172.21.15.97:0/939423883 shutdown_connections 2024-09-10T21:32:48.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 --2- 172.21.15.97:0/939423883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb44141d20 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- 172.21.15.97:0/939423883 >> 172.21.15.97:0/939423883 conn(0x7ffb440b1780 msgr2=0x7ffb440b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:48.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- 172.21.15.97:0/939423883 shutdown_connections 2024-09-10T21:32:48.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- 172.21.15.97:0/939423883 wait complete. 2024-09-10T21:32:48.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 Processor -- start 2024-09-10T21:32:48.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- start start 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb440b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb440ce9d0 0x7ffb440b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4415fde0 0x7ffb4413b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44169d60 con 0x7ffb440ce5f0 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44169be0 con 0x7ffb440ce9d0 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb48e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffb44169ee0 con 0x7ffb4415fde0 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb440b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb440b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33346/0 (socket says 172.21.15.97:33346) 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 -- 172.21.15.97:0/554678427 learned_addr learned my addr 172.21.15.97:0/554678427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41400640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb440ce9d0 0x7ffb440b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb42800640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4415fde0 0x7ffb4413b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 -- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4415fde0 msgr2=0x7ffb4413b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4415fde0 0x7ffb4413b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 -- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb440ce9d0 msgr2=0x7ffb440b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb440ce9d0 0x7ffb440b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb41e00640 1 -- 172.21.15.97:0/554678427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4415eb00 con 0x7ffb440ce5f0 2024-09-10T21:32:48.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.703+0000 7ffb42800640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4415fde0 0x7ffb4413b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb41e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb440b4440 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009790 tx=0x7ffb340a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb3a000640 1 -- 172.21.15.97:0/554678427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb340bf020 con 0x7ffb440ce5f0 2024-09-10T21:32:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb3a000640 1 -- 172.21.15.97:0/554678427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb34004430 con 0x7ffb440ce5f0 2024-09-10T21:32:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb44164b80 con 0x7ffb440ce5f0 2024-09-10T21:32:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb3a000640 1 -- 172.21.15.97:0/554678427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34005280 con 0x7ffb440ce5f0 2024-09-10T21:32:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb441fb7c0 con 0x7ffb440ce5f0 2024-09-10T21:32:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb44141d20 con 0x7ffb440ce5f0 2024-09-10T21:32:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb3a000640 1 -- 172.21.15.97:0/554678427 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffb340046d0 con 0x7ffb440ce5f0 2024-09-10T21:32:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb3a000640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb240725e0 0x7ffb24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb3a000640 1 -- 172.21.15.97:0/554678427 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7ffb3412e460 con 0x7ffb440ce5f0 2024-09-10T21:32:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.707+0000 7ffb41400640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb240725e0 0x7ffb24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.711+0000 7ffb41400640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb240725e0 0x7ffb24074aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffb44156ac0 tx=0x7ffb30007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:48.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.711+0000 7ffb3a000640 1 -- 172.21.15.97:0/554678427 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffb34016410 con 0x7ffb440ce5f0 2024-09-10T21:32:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb440cca70 con 0x7ffb440ce5f0 2024-09-10T21:32:48.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb3a000640 1 -- 172.21.15.97:0/554678427 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) ==== 95+0+4395 (secure 0 0 0) 0x7ffb340f8c60 con 0x7ffb440ce5f0 2024-09-10T21:32:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:48.918 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":186,"btime":"2024-09-10T21:32:48:788251+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2194082021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2194082021},{"type":"v1","addr":"172.21.15.97:6834","nonce":2194082021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:47.786625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:48.918 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 186 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb240725e0 msgr2=0x7ffb24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb240725e0 0x7ffb24074aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffb44156ac0 tx=0x7ffb30007cd0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 msgr2=0x7ffb440b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb440b4440 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009790 tx=0x7ffb340a6760 comp rx=0 tx=0).stop 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 shutdown_connections 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb240725e0 0x7ffb24074aa0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4415fde0 0x7ffb4413b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb440ce9d0 0x7ffb440b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 --2- 172.21.15.97:0/554678427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb440ce5f0 0x7ffb440b4440 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 >> 172.21.15.97:0/554678427 conn(0x7ffb440b1780 msgr2=0x7ffb440729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 shutdown_connections 2024-09-10T21:32:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:48.919+0000 7ffb48e00640 1 -- 172.21.15.97:0/554678427 wait complete. 2024-09-10T21:32:48.938 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2024-09-10T21:32:48.938 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2024-09-10T21:32:48.939 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:48.939 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:48.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:32:48.947 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:48.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:48.948 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:48.995 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:32:48.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:32:48.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 -- 172.21.15.97:0/4293029105 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec406fd50 msgr2=0x7faec40701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 --2- 172.21.15.97:0/4293029105 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec406fd50 0x7faec40701a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faebc009c80 tx=0x7faebc01cd40 comp rx=0 tx=0).stop 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 -- 172.21.15.97:0/4293029105 shutdown_connections 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 --2- 172.21.15.97:0/4293029105 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec406fd50 0x7faec40701a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 --2- 172.21.15.97:0/4293029105 >> v2:172.21.15.194:3300/0 conn(0x7faec4143f90 0x7faec40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 -- 172.21.15.97:0/4293029105 >> 172.21.15.97:0/4293029105 conn(0x7faec40b1780 msgr2=0x7faec40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 -- 172.21.15.97:0/4293029105 shutdown_connections 2024-09-10T21:32:49.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 -- 172.21.15.97:0/4293029105 wait complete. 2024-09-10T21:32:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 Processor -- start 2024-09-10T21:32:49.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.183+0000 7faecae00640 1 -- start start 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec406fd50 0x7faec4137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faec4143f90 0x7faec40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec4144690 0x7faec40730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faec4167cb0 con 0x7faec406fd50 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faec4167b30 con 0x7faec4143f90 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7faec4167e30 con 0x7faec4144690 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec406fd50 0x7faec4137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec406fd50 0x7faec4137ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33348/0 (socket says 172.21.15.97:33348) 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 -- 172.21.15.97:0/1882508967 learned_addr learned my addr 172.21.15.97:0/1882508967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec8e00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec4144690 0x7faec40730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faebb400640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faec4143f90 0x7faec40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 -- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec4144690 msgr2=0x7faec40730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec4144690 0x7faec40730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 -- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faec4143f90 msgr2=0x7faec40cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faec4143f90 0x7faec40cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 -- 172.21.15.97:0/1882508967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec415ec30 con 0x7faec406fd50 2024-09-10T21:32:49.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec3e00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec406fd50 0x7faec4137ea0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ea10 tx=0x7faeb400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:49.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec0c00640 1 -- 172.21.15.97:0/1882508967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb400cde0 con 0x7faec406fd50 2024-09-10T21:32:49.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec0c00640 1 -- 172.21.15.97:0/1882508967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faeb4004020 con 0x7faec406fd50 2024-09-10T21:32:49.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec415eea0 con 0x7faec406fd50 2024-09-10T21:32:49.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faec0c00640 1 -- 172.21.15.97:0/1882508967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4004470 con 0x7faec406fd50 2024-09-10T21:32:49.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec4136960 con 0x7faec406fd50 2024-09-10T21:32:49.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.187+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec40701a0 con 0x7faec406fd50 2024-09-10T21:32:49.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.191+0000 7faec0c00640 1 -- 172.21.15.97:0/1882508967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faeb4003600 con 0x7faec406fd50 2024-09-10T21:32:49.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.191+0000 7faec0c00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea00725e0 0x7faea0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.191+0000 7faec0c00640 1 -- 172.21.15.97:0/1882508967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7faeb40943a0 con 0x7faec406fd50 2024-09-10T21:32:49.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.191+0000 7faebb400640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea00725e0 0x7faea0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.191+0000 7faebb400640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea00725e0 0x7faea0074aa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faec4141ba0 tx=0x7faeac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:49.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.195+0000 7faec0c00640 1 -- 172.21.15.97:0/1882508967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faeb405eba0 con 0x7faec406fd50 2024-09-10T21:32:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.395+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7faec415ab90 con 0x7faec406fd50 2024-09-10T21:32:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faec0c00640 1 -- 172.21.15.97:0/1882508967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v187) ==== 161+0+0 (secure 0 0 0) 0x7faeb40635d0 con 0x7faec406fd50 2024-09-10T21:32:49.794 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea00725e0 msgr2=0x7faea0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea00725e0 0x7faea0074aa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faec4141ba0 tx=0x7faeac009040 comp rx=0 tx=0).stop 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec406fd50 msgr2=0x7faec4137ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec406fd50 0x7faec4137ea0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ea10 tx=0x7faeb400eed0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 shutdown_connections 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faea00725e0 0x7faea0074aa0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faec4144690 0x7faec40730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faec4143f90 0x7faec40cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 --2- 172.21.15.97:0/1882508967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec406fd50 0x7faec4137ea0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 >> 172.21.15.97:0/1882508967 conn(0x7faec40b1780 msgr2=0x7faec4142980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 shutdown_connections 2024-09-10T21:32:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.795+0000 7faecae00640 1 -- 172.21.15.97:0/1882508967 wait complete. 2024-09-10T21:32:49.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:49.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 -- 172.21.15.97:0/2133699667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 msgr2=0x7f3b7c0b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2133699667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c0b3be0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c073500 tx=0x7f3b6c01c770 comp rx=0 tx=0).stop 2024-09-10T21:32:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 -- 172.21.15.97:0/2133699667 shutdown_connections 2024-09-10T21:32:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2133699667 >> v2:172.21.15.194:3300/0 conn(0x7f3b7c15bbe0 0x7f3b7c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2133699667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c0b3be0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 -- 172.21.15.97:0/2133699667 >> 172.21.15.97:0/2133699667 conn(0x7f3b7c0b2ee0 msgr2=0x7f3b7c0d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 -- 172.21.15.97:0/2133699667 shutdown_connections 2024-09-10T21:32:49.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.959+0000 7f3b7be00640 1 -- 172.21.15.97:0/2133699667 wait complete. 2024-09-10T21:32:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 Processor -- start 2024-09-10T21:32:49.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 -- start start 2024-09-10T21:32:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b7c15bbe0 0x7f3b7c14c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b7c15c5d0 0x7f3b7c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c0afa50 con 0x7f3b7c155090 2024-09-10T21:32:49.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c0af8d0 con 0x7f3b7c15c5d0 2024-09-10T21:32:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7c0afbd0 con 0x7f3b7c15bbe0 2024-09-10T21:32:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33370/0 (socket says 172.21.15.97:33370) 2024-09-10T21:32:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 -- 172.21.15.97:0/2280053940 learned_addr learned my addr 172.21.15.97:0/2280053940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7b400640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b7c15c5d0 0x7f3b7c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7a000640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b7c15bbe0 0x7f3b7c14c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 -- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b7c15bbe0 msgr2=0x7f3b7c14c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b7c15bbe0 0x7f3b7c14c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 -- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b7c15c5d0 msgr2=0x7f3b7c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b7c15c5d0 0x7f3b7c0ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 -- 172.21.15.97:0/2280053940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7c06f650 con 0x7f3b7c155090 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7a000640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b7c15bbe0 0x7f3b7c14c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7aa00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c158210 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c000c10 tx=0x7f3b6c0a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:49.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b73400640 1 -- 172.21.15.97:0/2280053940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0bc070 con 0x7f3b7c155090 2024-09-10T21:32:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b73400640 1 -- 172.21.15.97:0/2280053940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b6c004430 con 0x7f3b7c155090 2024-09-10T21:32:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b73400640 1 -- 172.21.15.97:0/2280053940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0b2040 con 0x7f3b7c155090 2024-09-10T21:32:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7c06f8f0 con 0x7f3b7c155090 2024-09-10T21:32:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.963+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b7c071560 con 0x7f3b7c155090 2024-09-10T21:32:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.967+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b7c0b00f0 con 0x7f3b7c155090 2024-09-10T21:32:49.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.967+0000 7f3b73400640 1 -- 172.21.15.97:0/2280053940 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3b6c0b5070 con 0x7f3b7c155090 2024-09-10T21:32:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.967+0000 7f3b73400640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3b5c0725e0 0x7f3b5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.967+0000 7f3b73400640 1 -- 172.21.15.97:0/2280053940 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7f3b6c12cfd0 con 0x7f3b7c155090 2024-09-10T21:32:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.967+0000 7f3b7a000640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3b5c0725e0 0x7f3b5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.971+0000 7f3b7a000640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3b5c0725e0 0x7f3b5c074aa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c1503e0 tx=0x7f3b6800b670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:49.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:49.971+0000 7f3b73400640 1 -- 172.21.15.97:0/2280053940 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3b6c0f2de0 con 0x7f3b7c155090 2024-09-10T21:32:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.199+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3b7c0b53c0 con 0x7f3b7c155090 2024-09-10T21:32:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.799+0000 7f3b73400640 1 -- 172.21.15.97:0/2280053940 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v188) ==== 101+0+0 (secure 0 0 0) 0x7f3b6c0163f0 con 0x7f3b7c155090 2024-09-10T21:32:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3b5c0725e0 msgr2=0x7f3b5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3b5c0725e0 0x7f3b5c074aa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c1503e0 tx=0x7f3b6800b670 comp rx=0 tx=0).stop 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 msgr2=0x7f3b7c158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c158210 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c000c10 tx=0x7f3b6c0a5670 comp rx=0 tx=0).stop 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 shutdown_connections 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3b5c0725e0 0x7f3b5c074aa0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b7c15c5d0 0x7f3b7c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b7c15bbe0 0x7f3b7c14c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 --2- 172.21.15.97:0/2280053940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7c155090 0x7f3b7c158210 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 >> 172.21.15.97:0/2280053940 conn(0x7f3b7c0b2ee0 msgr2=0x7f3b7c154690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 shutdown_connections 2024-09-10T21:32:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.803+0000 7f3b7be00640 1 -- 172.21.15.97:0/2280053940 wait complete. 2024-09-10T21:32:50.825 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:50.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.871+0000 7fecf99d2080 1 Processor -- start 2024-09-10T21:32:50.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.871+0000 7fecf99d2080 1 -- start start 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d58681cc0 0x559d585c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58683a00 0x559d586134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d586871d0 0x559d5869cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559d58575f40 con 0x559d58683a00 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d58582d00 con 0x559d586871d0 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559d58517a00 con 0x559d58681cc0 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58683a00 0x559d586134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58683a00 0x559d586134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33388/0 (socket says 172.21.15.97:33388) 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 -- 172.21.15.97:0/1897167890 learned_addr learned my addr 172.21.15.97:0/1897167890 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d58681cc0 0x559d585c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf7600640 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d586871d0 0x559d5869cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 -- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d58681cc0 msgr2=0x559d585c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d58681cc0 0x559d585c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 -- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d586871d0 msgr2=0x559d5869cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d586871d0 0x559d5869cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 -- 172.21.15.97:0/1897167890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d58610640 con 0x559d58683a00 2024-09-10T21:32:50.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf6200640 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58683a00 0x559d586134b0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fece400b310 tx=0x7fece400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89b4fe4b6cac648 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf4e00640 1 -- 172.21.15.97:0/1897167890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4012070 con 0x559d58683a00 2024-09-10T21:32:50.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf4e00640 1 -- 172.21.15.97:0/1897167890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fece400e950 con 0x559d58683a00 2024-09-10T21:32:50.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf4e00640 1 -- 172.21.15.97:0/1897167890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece400eda0 con 0x559d58683a00 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- 172.21.15.97:0/1897167890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58683a00 msgr2=0x559d586134b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58683a00 0x559d586134b0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fece400b310 tx=0x7fece400b7d0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- 172.21.15.97:0/1897167890 shutdown_connections 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d586871d0 0x559d5869cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58683a00 0x559d586134b0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1897167890 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d58681cc0 0x559d585c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- 172.21.15.97:0/1897167890 >> 172.21.15.97:0/1897167890 conn(0x559d586110d0 msgr2=0x559d58681540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- 172.21.15.97:0/1897167890 shutdown_connections 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- 172.21.15.97:0/1897167890 wait complete. 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 Processor -- start 2024-09-10T21:32:50.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.875+0000 7fecf99d2080 1 -- start start 2024-09-10T21:32:50.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58681cc0 0x559d5869c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d58683a00 0x559d586a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d586871d0 0x559d586a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559d586108e0 con 0x559d58681cc0 2024-09-10T21:32:50.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d58610b80 con 0x559d58683a00 2024-09-10T21:32:50.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559d5869caa0 con 0x559d586871d0 2024-09-10T21:32:50.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58681cc0 0x559d5869c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58681cc0 0x559d5869c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33394/0 (socket says 172.21.15.97:33394) 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 learned_addr learned my addr 172.21.15.97:0/1440672870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d58683a00 0x559d586a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf7600640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d586871d0 0x559d586a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d586871d0 msgr2=0x559d586a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d586871d0 0x559d586a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d58683a00 msgr2=0x559d586a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d58683a00 0x559d586a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d5869f9b0 con 0x559d58681cc0 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf7600640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559d586871d0 0x559d586a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d58683a00 0x559d586a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:50.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58681cc0 0x559d5869c030 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fece8004f80 tx=0x7fece80ab590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecef400640 1 -- 172.21.15.97:0/1440672870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece80b0040 con 0x559d58681cc0 2024-09-10T21:32:50.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecef400640 1 -- 172.21.15.97:0/1440672870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fece80aba70 con 0x559d58681cc0 2024-09-10T21:32:50.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecef400640 1 -- 172.21.15.97:0/1440672870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece80b63f0 con 0x559d58681cc0 2024-09-10T21:32:50.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559d5869fc50 con 0x559d58681cc0 2024-09-10T21:32:50.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.879+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d586a0190 con 0x559d58681cc0 2024-09-10T21:32:50.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecef400640 1 -- 172.21.15.97:0/1440672870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fece80b6590 con 0x559d58681cc0 2024-09-10T21:32:50.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecef400640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fecd8070ff0 0x7fecd80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecef400640 1 -- 172.21.15.97:0/1440672870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7fece812d260 con 0x559d58681cc0 2024-09-10T21:32:50.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fecd8070ff0 0x7fecd80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.882 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559d586a13d0 0x559d58739800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58739d40 con 0x559d586a13d0 2024-09-10T21:32:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecf7600640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559d586a13d0 0x559d58739800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fecd8070ff0 0x7fecd80734b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fece400b310 tx=0x7fece400e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.883+0000 7fecf7600640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559d586a13d0 0x559d58739800 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.887+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v113'505 uv479 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.887+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5873a110 con 0x559d586a13d0 2024-09-10T21:32:50.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.887+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v113'506 uv506 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.887+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58745750 con 0x559d586a13d0 2024-09-10T21:32:50.887 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:50.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.887+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v113'507 uv507 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.887+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746b00 con 0x559d586a13d0 2024-09-10T21:32:50.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.891+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'508 uv508 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.891+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746ea0 con 0x559d586a13d0 2024-09-10T21:32:50.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.891+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'509 uv509 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.891+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58745af0 con 0x559d586a13d0 2024-09-10T21:32:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.891+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.891+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58745ee0 con 0x559d586a13d0 2024-09-10T21:32:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.895+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v113'511 uv485 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.895+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d587462d0 con 0x559d586a13d0 2024-09-10T21:32:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.895+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v113'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.895+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d587462d0 con 0x559d586a13d0 2024-09-10T21:32:50.895 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.895+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v113'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.895+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.899+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.899+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.899+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.899+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.899+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.899+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.903+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v113'517 uv492 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.903+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.903+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'518 uv518 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.903+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.904 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:50.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.903+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v113'519 uv519 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.903+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.907+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'520 uv520 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.907+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.907+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'521 uv521 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.907+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746280 con 0x559d586a13d0 2024-09-10T21:32:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.907+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'522 uv522 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.911+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746670 con 0x559d586a13d0 2024-09-10T21:32:50.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.911+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'523 uv523 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.911+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.911+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v113'524 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.911+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.911+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v113'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.915+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.914 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.915+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v113'526 uv526 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.915+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.915+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'527 uv527 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.915+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'528 uv528 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'529 uv529 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v113'530 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v113'531 uv531 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.919+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.921 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:50.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.923+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v113'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.923+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.923+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.923+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.923+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.923+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58746500 con 0x559d586a13d0 2024-09-10T21:32:50.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.927+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d586a13d0 2024-09-10T21:32:50.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.927+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559d58747240 0x559d58749630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.927+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58749b70 con 0x559d58747240 2024-09-10T21:32:50.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.927+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559d58747240 0x559d58749630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.927+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559d58747240 0x559d58749630 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.931+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'374 uv356 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.931+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58749f10 con 0x559d58747240 2024-09-10T21:32:50.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.931+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'375 uv375 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.931+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874a2b0 con 0x559d58747240 2024-09-10T21:32:50.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.931+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'376 uv376 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.931 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:50.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.931+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874ab00 con 0x559d58747240 2024-09-10T21:32:50.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.931+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v113'377 uv377 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.935+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874aea0 con 0x559d58747240 2024-09-10T21:32:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.935+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'378 uv378 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.935+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b240 con 0x559d58747240 2024-09-10T21:32:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.935+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v113'379 uv373 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.935+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.935+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v113'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.935+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.939+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v113'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.937 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:50.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.939+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.939+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.939+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.939+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.939+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.943+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v113'384 uv366 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.943+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.943+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v113'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.943+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.943 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.943+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v113'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.943+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874f7b0 con 0x559d58747240 2024-09-10T21:32:50.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.947+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v113'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.947+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874f7b0 con 0x559d58747240 2024-09-10T21:32:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.947+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.947+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.947+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v113'389 uv372 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.947+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v113'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874b5e0 con 0x559d58747240 2024-09-10T21:32:50.949 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v113'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874c5f0 con 0x559d58747240 2024-09-10T21:32:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874c5f0 con 0x559d58747240 2024-09-10T21:32:50.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.951+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874ba90 con 0x559d58747240 2024-09-10T21:32:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.955+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d58747240 2024-09-10T21:32:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.955+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559d5874bfa0 0x559d5874e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.955+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874e8d0 con 0x559d5874bfa0 2024-09-10T21:32:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.955+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559d5874bfa0 0x559d5874e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.955+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559d5874bfa0 0x559d5874e390 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.955+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.955+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5874ecb0 con 0x559d5874bfa0 2024-09-10T21:32:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874f160 con 0x559d5874bfa0 2024-09-10T21:32:50.957 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58750a40 con 0x559d5874bfa0 2024-09-10T21:32:50.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58750de0 con 0x559d5874bfa0 2024-09-10T21:32:50.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.959+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5874fa30 con 0x559d5874bfa0 2024-09-10T21:32:50.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559d58750330 0x559d58753190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d587536d0 con 0x559d58750330 2024-09-10T21:32:50.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf7600640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559d58750330 0x559d58753190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf7600640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559d58750330 0x559d58753190 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'79 uv78 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d587536d0 con 0x559d58750330 2024-09-10T21:32:50.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'80 uv80 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.963+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753750 con 0x559d58750330 2024-09-10T21:32:50.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.967+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'81 uv81 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.965 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:50.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.967+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753b40 con 0x559d58750330 2024-09-10T21:32:50.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.967+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'82 uv82 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.967+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753b40 con 0x559d5874bfa0 2024-09-10T21:32:50.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.967+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.967+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.970 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:50.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58754ef0 con 0x559d5874bfa0 2024-09-10T21:32:50.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58754ef0 con 0x559d5874bfa0 2024-09-10T21:32:50.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.971+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.975+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.975+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.975+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v113'182 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.975+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.975+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v113'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.975+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.977 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.979+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v113'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.979+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58754ef0 con 0x559d58750330 2024-09-10T21:32:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.979+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.979+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58754ef0 con 0x559d58750330 2024-09-10T21:32:50.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.979+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.979+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.983+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.983+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.983+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v113'188 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.983+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.983+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v113'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.983+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.984 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v113'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d58750330 2024-09-10T21:32:50.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:50.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v113'81 uv80 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.987+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.991+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v113'82 uv82 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.991+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.990 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.991+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v113'83 uv83 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.991+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.991+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'84 uv84 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.991+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58753ee0 con 0x559d5874bfa0 2024-09-10T21:32:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d5874bfa0 2024-09-10T21:32:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559d587543f0 0x559d58756860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58756e90 con 0x559d587543f0 2024-09-10T21:32:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559d587543f0 0x559d58756860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:50.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf6c00640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559d587543f0 0x559d58756860 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:50.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:50.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58756f50 con 0x559d587543f0 2024-09-10T21:32:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:50.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.995+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.999+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:50.997 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.999+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58758380 con 0x559d587543f0 2024-09-10T21:32:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.999+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:50.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.999+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58758380 con 0x559d587543f0 2024-09-10T21:32:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.999+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:50.999+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v113'162 uv156 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v113'163 uv163 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.004 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:51.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v113'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.003+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.007+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.007+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.007+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.007+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.007+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v113'167 uv161 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.007+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.011+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v113'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.011+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.010 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.011+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v113'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.011+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757370 con 0x559d587543f0 2024-09-10T21:32:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.011+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.011+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875a180 con 0x559d587543f0 2024-09-10T21:32:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.015+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.015+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.015+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v113'74 uv73 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.015+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.019+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'75 uv75 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.019+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.018 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.019+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v113'76 uv76 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.019+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.019+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'77 uv77 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.019+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v113'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v113'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.023 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v113'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.023+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.027+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.027+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d58757820 con 0x559d58750330 2024-09-10T21:32:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.027+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecf0001680 con 0x559d58750330 2024-09-10T21:32:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.027+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559d58758120 0x559d5875a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.027+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d58758120 2024-09-10T21:32:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.027+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559d58758120 0x559d5875a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.027+0000 7fecf6200640 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559d58758120 0x559d5875a800 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v113'84 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d58758120 2024-09-10T21:32:51.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d58758120 2024-09-10T21:32:51.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v113'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d58758120 2024-09-10T21:32:51.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d58758120 2024-09-10T21:32:51.031 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v113'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d58758120 2024-09-10T21:32:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d58758120 2024-09-10T21:32:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d58758120 2024-09-10T21:32:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.031+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d58758120 2024-09-10T21:32:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.035+0000 7fecf6200640 1 -- 172.21.15.97:0/1440672870 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fece400e6c0 con 0x559d58758120 2024-09-10T21:32:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.035+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.035+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v113'204 uv196 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.035+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.035+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v113'205 uv205 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.035+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.038 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:51.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.039+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v113'206 uv206 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.039+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.039+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v113'207 uv207 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.039+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'208 uv208 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v113'209 uv203 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v113'210 uv210 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.044 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v113'211 uv211 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.043+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875bd50 con 0x559d587543f0 2024-09-10T21:32:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.047+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'212 uv212 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.047+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875bd50 con 0x559d587543f0 2024-09-10T21:32:51.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.047+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'213 uv213 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.047+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.047+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'214 uv214 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.047+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559d5875ad40 con 0x559d587543f0 2024-09-10T21:32:51.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf6c00640 1 -- 172.21.15.97:0/1440672870 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'215 uv215 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fece80b3070 con 0x559d587543f0 2024-09-10T21:32:51.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559d5874bfa0 msgr2=0x559d5874e390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559d5874bfa0 0x559d5874e390 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559d587543f0 msgr2=0x559d58756860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559d587543f0 0x559d58756860 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559d58758120 msgr2=0x559d5875a800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559d58758120 0x559d5875a800 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559d58750330 msgr2=0x559d58753190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559d58750330 0x559d58753190 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559d58747240 msgr2=0x559d58749630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 reap_dead start 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559d58747240 0x559d58749630 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf7600640 1 -- 172.21.15.97:0/1440672870 reap_dead start 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559d586a13d0 msgr2=0x559d58739800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559d586a13d0 0x559d58739800 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fecd8070ff0 msgr2=0x7fecd80734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fecd8070ff0 0x7fecd80734b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fece400b310 tx=0x7fece400e000 comp rx=0 tx=0).stop 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58681cc0 msgr2=0x559d5869c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58681cc0 0x559d5869c030 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fece8004f80 tx=0x7fece80ab590 comp rx=0 tx=0).stop 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 shutdown_connections 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559d58747240 0x559d58749630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559d586a13d0 0x559d58739800 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fecd8070ff0 0x7fecd80734b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 --2- 172.21.15.97:0/1440672870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559d58681cc0 0x559d5869c030 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 >> 172.21.15.97:0/1440672870 conn(0x559d586110d0 msgr2=0x559d58681540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 shutdown_connections 2024-09-10T21:32:51.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.051+0000 7fecf99d2080 1 -- 172.21.15.97:0/1440672870 wait complete. 2024-09-10T21:32:51.054 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2024-09-10T21:32:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 Processor -- start 2024-09-10T21:32:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- start start 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 0x5647c66e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c6757a00 0x5647c66cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c675b1d0 0x5647c66cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5647c6649f40 con 0x5647c6757a00 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5647c6656d00 con 0x5647c675b1d0 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5647c65eba00 con 0x5647c6755cc0 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c6757a00 0x5647c66cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c6757a00 0x5647c66cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33418/0 (socket says 172.21.15.97:33418) 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 -- 172.21.15.97:0/1910220367 learned_addr learned my addr 172.21.15.97:0/1910220367 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e11200640 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c675b1d0 0x5647c66cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e10800640 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 0x5647c66e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 -- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 msgr2=0x5647c66e7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 0x5647c66e7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 -- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c675b1d0 msgr2=0x5647c66cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c675b1d0 0x5647c66cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 -- 172.21.15.97:0/1910220367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c6773d90 con 0x5647c6757a00 2024-09-10T21:32:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e11200640 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c675b1d0 0x5647c66cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0fe00640 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c6757a00 0x5647c66cd410 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b310 tx=0x7f0dfc00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7066289cc69cce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0ea00640 1 -- 172.21.15.97:0/1910220367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc012070 con 0x5647c6757a00 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0ea00640 1 -- 172.21.15.97:0/1910220367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dfc00e930 con 0x5647c6757a00 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e0ea00640 1 -- 172.21.15.97:0/1910220367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc00ed80 con 0x5647c6757a00 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- 172.21.15.97:0/1910220367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c6757a00 msgr2=0x5647c66cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c6757a00 0x5647c66cd410 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b310 tx=0x7f0dfc00b7d0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- 172.21.15.97:0/1910220367 shutdown_connections 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c675b1d0 0x5647c66cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c6757a00 0x5647c66cd410 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 --2- 172.21.15.97:0/1910220367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 0x5647c66e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- 172.21.15.97:0/1910220367 >> 172.21.15.97:0/1910220367 conn(0x5647c66e50d0 msgr2=0x5647c6755540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- 172.21.15.97:0/1910220367 shutdown_connections 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.143+0000 7f0e13591080 1 -- 172.21.15.97:0/1910220367 wait complete. 2024-09-10T21:32:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 Processor -- start 2024-09-10T21:32:51.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 -- start start 2024-09-10T21:32:51.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 0x5647c6771110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c6757a00 0x5647c6771650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c675b1d0 0x5647c6753390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5647c6774030 con 0x5647c675b1d0 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5647c67742d0 con 0x5647c6757a00 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5647c66e45a0 con 0x5647c6755cc0 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c675b1d0 0x5647c6753390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c675b1d0 0x5647c6753390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33430/0 (socket says 172.21.15.97:33430) 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 -- 172.21.15.97:0/2527047747 learned_addr learned my addr 172.21.15.97:0/2527047747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e10800640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 0x5647c6771110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e0fe00640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c6757a00 0x5647c6771650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 -- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 msgr2=0x5647c6771110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5647c6755cc0 0x5647c6771110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 -- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c6757a00 msgr2=0x5647c6771650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647c6757a00 0x5647c6771650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 -- 172.21.15.97:0/2527047747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c66e4840 con 0x5647c675b1d0 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e11200640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c675b1d0 0x5647c6753390 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d440 tx=0x7f0e0800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e0d600640 1 -- 172.21.15.97:0/2527047747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08018070 con 0x5647c675b1d0 2024-09-10T21:32:51.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e0d600640 1 -- 172.21.15.97:0/2527047747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e080032e0 con 0x5647c675b1d0 2024-09-10T21:32:51.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5647c66e4ae0 con 0x5647c675b1d0 2024-09-10T21:32:51.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e0d600640 1 -- 172.21.15.97:0/2527047747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08001810 con 0x5647c675b1d0 2024-09-10T21:32:51.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.147+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c676ed90 con 0x5647c675b1d0 2024-09-10T21:32:51.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e0d600640 1 -- 172.21.15.97:0/2527047747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0e080019b0 con 0x5647c675b1d0 2024-09-10T21:32:51.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e0d600640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0df0070ff0 0x7f0df00734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e0d600640 1 -- 172.21.15.97:0/2527047747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7f0e08092160 con 0x5647c675b1d0 2024-09-10T21:32:51.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e10800640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0df0070ff0 0x7f0df00734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e13591080 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5647c676d880 0x5647c676dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5647c66dedd0 con 0x5647c676d880 2024-09-10T21:32:51.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e0fe00640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5647c676d880 0x5647c676dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e10800640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0df0070ff0 0x7f0df00734b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0e040ab000 tx=0x7f0e04019a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.151+0000 7f0e0fe00640 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5647c676d880 0x5647c676dc60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e0fe00640 1 -- 172.21.15.97:0/2527047747 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v113'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0dfc014030 con 0x5647c676d880 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5647c676d880 msgr2=0x5647c676dc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5647c676d880 0x5647c676dc60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0df0070ff0 msgr2=0x7f0df00734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0df0070ff0 0x7f0df00734b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0e040ab000 tx=0x7f0e04019a40 comp rx=0 tx=0).stop 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c675b1d0 msgr2=0x5647c6753390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 --2- 172.21.15.97:0/2527047747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5647c675b1d0 0x5647c6753390 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d440 tx=0x7f0e0800d900 comp rx=0 tx=0).stop 2024-09-10T21:32:51.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e11200640 1 -- 172.21.15.97:0/2527047747 reap_dead start 2024-09-10T21:32:51.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 shutdown_connections 2024-09-10T21:32:51.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 >> 172.21.15.97:0/2527047747 conn(0x5647c66e50d0 msgr2=0x5647c676f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 shutdown_connections 2024-09-10T21:32:51.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.155+0000 7f0e13591080 1 -- 172.21.15.97:0/2527047747 wait complete. 2024-09-10T21:32:51.158 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.343+0000 7ff765400640 1 -- 172.21.15.97:0/2438938925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 msgr2=0x7ff7600701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 --2- 172.21.15.97:0/2438938925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 0x7ff7600701e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff758009bc0 tx=0x7ff75801cba0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- 172.21.15.97:0/2438938925 shutdown_connections 2024-09-10T21:32:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 --2- 172.21.15.97:0/2438938925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 0x7ff7600701e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 --2- 172.21.15.97:0/2438938925 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff760144690 0x7ff7600b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- 172.21.15.97:0/2438938925 >> 172.21.15.97:0/2438938925 conn(0x7ff7600b1780 msgr2=0x7ff7600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- 172.21.15.97:0/2438938925 shutdown_connections 2024-09-10T21:32:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- 172.21.15.97:0/2438938925 wait complete. 2024-09-10T21:32:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 Processor -- start 2024-09-10T21:32:51.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- start start 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 0x7ff76013d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff760143f90 0x7ff760159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff760144690 0x7ff76013aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff760167ca0 con 0x7ff760144690 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff760167b20 con 0x7ff76006fd50 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff765400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff760167e20 con 0x7ff760143f90 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff760144690 0x7ff76013aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff760144690 0x7ff76013aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33442/0 (socket says 172.21.15.97:33442) 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 -- 172.21.15.97:0/1417679530 learned_addr learned my addr 172.21.15.97:0/1417679530 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75e800640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 0x7ff76013d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff755e00640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff760143f90 0x7ff760159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 -- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff760143f90 msgr2=0x7ff760159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff760143f90 0x7ff760159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 -- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 msgr2=0x7ff76013d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 0x7ff76013d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 -- 172.21.15.97:0/1417679530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7601f98d0 con 0x7ff760144690 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff755e00640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff760143f90 0x7ff760159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:51.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.347+0000 7ff75f200640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff760144690 0x7ff76013aa50 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7ff7580b0040 tx=0x7ff758005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff757400640 1 -- 172.21.15.97:0/1417679530 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580b2040 con 0x7ff760144690 2024-09-10T21:32:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff757400640 1 -- 172.21.15.97:0/1417679530 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff758004040 con 0x7ff760144690 2024-09-10T21:32:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff760137600 con 0x7ff760144690 2024-09-10T21:32:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff757400640 1 -- 172.21.15.97:0/1417679530 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758004470 con 0x7ff760144690 2024-09-10T21:32:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7601f95f0 con 0x7ff760144690 2024-09-10T21:32:51.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7600701e0 con 0x7ff760144690 2024-09-10T21:32:51.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff757400640 1 -- 172.21.15.97:0/1417679530 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7580b5070 con 0x7ff760144690 2024-09-10T21:32:51.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff757400640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff73c072610 0x7ff73c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff757400640 1 -- 172.21.15.97:0/1417679530 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7737+0+0 (secure 0 0 0) 0x7ff75812d850 con 0x7ff760144690 2024-09-10T21:32:51.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.351+0000 7ff75e800640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff73c072610 0x7ff73c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.355+0000 7ff75e800640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff73c072610 0x7ff73c074ad0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff750001910 tx=0x7ff750001420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.355+0000 7ff757400640 1 -- 172.21.15.97:0/1417679530 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff758016790 con 0x7ff760144690 2024-09-10T21:32:51.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.555+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff7600cf790 con 0x7ff760144690 2024-09-10T21:32:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff757400640 1 -- 172.21.15.97:0/1417679530 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v189) ==== 162+0+0 (secure 0 0 0) 0x7ff7580f86b0 con 0x7ff760144690 2024-09-10T21:32:51.630 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:51.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff73c072610 msgr2=0x7ff73c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff73c072610 0x7ff73c074ad0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff750001910 tx=0x7ff750001420 comp rx=0 tx=0).stop 2024-09-10T21:32:51.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff760144690 msgr2=0x7ff76013aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff760144690 0x7ff76013aa50 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7ff7580b0040 tx=0x7ff758005a80 comp rx=0 tx=0).stop 2024-09-10T21:32:51.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 shutdown_connections 2024-09-10T21:32:51.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff73c072610 0x7ff73c074ad0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff760144690 0x7ff76013aa50 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff760143f90 0x7ff760159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 --2- 172.21.15.97:0/1417679530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff76006fd50 0x7ff76013d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 >> 172.21.15.97:0/1417679530 conn(0x7ff7600b1780 msgr2=0x7ff7600730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 shutdown_connections 2024-09-10T21:32:51.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.631+0000 7ff765400640 1 -- 172.21.15.97:0/1417679530 wait complete. 2024-09-10T21:32:51.638 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:32:51.639+0000 7ff78a000640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T21:32:51.639 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:51.639+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:32:51.639 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:32:51.639+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:32:51.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:51.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 -- 172.21.15.97:0/2768608365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 msgr2=0x7efea80cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 --2- 172.21.15.97:0/2768608365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea80cff50 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7efea806daf0 tx=0x7efe9401c830 comp rx=0 tx=0).stop 2024-09-10T21:32:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 -- 172.21.15.97:0/2768608365 shutdown_connections 2024-09-10T21:32:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 --2- 172.21.15.97:0/2768608365 >> v2:172.21.15.194:3300/0 conn(0x7efea80d0490 0x7efea80ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 --2- 172.21.15.97:0/2768608365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea80cff50 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 -- 172.21.15.97:0/2768608365 >> 172.21.15.97:0/2768608365 conn(0x7efea80d1a60 msgr2=0x7efea80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 -- 172.21.15.97:0/2768608365 shutdown_connections 2024-09-10T21:32:51.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.795+0000 7efeaf000640 1 -- 172.21.15.97:0/2768608365 wait complete. 2024-09-10T21:32:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 Processor -- start 2024-09-10T21:32:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 -- start start 2024-09-10T21:32:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efea80d0490 0x7efea8165a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efea815efc0 0x7efea80c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efea8168930 con 0x7efea80cdcb0 2024-09-10T21:32:51.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efea81687b0 con 0x7efea80d0490 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efea8168ab0 con 0x7efea815efc0 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33464/0 (socket says 172.21.15.97:33464) 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 -- 172.21.15.97:0/2960202917 learned_addr learned my addr 172.21.15.97:0/2960202917 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efead200640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efea80d0490 0x7efea8165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeae600640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efea815efc0 0x7efea80c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 -- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efea815efc0 msgr2=0x7efea80c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efea815efc0 0x7efea80c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 -- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efea80d0490 msgr2=0x7efea8165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efea80d0490 0x7efea8165a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 -- 172.21.15.97:0/2960202917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea8164c40 con 0x7efea80cdcb0 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efead200640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efea80d0490 0x7efea8165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeae600640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efea815efc0 0x7efea80c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeadc00640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea814cea0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00b410 tx=0x7efe9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efea5600640 1 -- 172.21.15.97:0/2960202917 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c015030 con 0x7efea80cdcb0 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efea5600640 1 -- 172.21.15.97:0/2960202917 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe9c004020 con 0x7efea80cdcb0 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea8164f20 con 0x7efea80cdcb0 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.799+0000 7efea5600640 1 -- 172.21.15.97:0/2960202917 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0044b0 con 0x7efea80cdcb0 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.803+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea814d3e0 con 0x7efea80cdcb0 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.803+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea8165640 con 0x7efea80cdcb0 2024-09-10T21:32:51.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.803+0000 7efea5600640 1 -- 172.21.15.97:0/2960202917 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe9c022060 con 0x7efea80cdcb0 2024-09-10T21:32:51.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.803+0000 7efea5600640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe880725e0 0x7efe88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.803+0000 7efea5600640 1 -- 172.21.15.97:0/2960202917 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7efe9c0936c0 con 0x7efea80cdcb0 2024-09-10T21:32:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.803+0000 7efead200640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe880725e0 0x7efe88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.807+0000 7efead200640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe880725e0 0x7efe88074aa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efe940af000 tx=0x7efe9401b070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:51.811+0000 7efea5600640 1 -- 172.21.15.97:0/2960202917 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe9c05e4c0 con 0x7efea80cdcb0 2024-09-10T21:32:52.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efea80ae930 con 0x7efea80cdcb0 2024-09-10T21:32:52.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efea5600640 1 -- 172.21.15.97:0/2960202917 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+3569 (secure 0 0 0) 0x7efe9c05ddc0 con 0x7efea80cdcb0 2024-09-10T21:32:52.010 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:52.010 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":190,"btime":"2024-09-10T21:32:51:642642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:51.642562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:52.011 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 190 2024-09-10T21:32:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe880725e0 msgr2=0x7efe88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe880725e0 0x7efe88074aa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efe940af000 tx=0x7efe9401b070 comp rx=0 tx=0).stop 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 msgr2=0x7efea814cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea814cea0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00b410 tx=0x7efe9c00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 shutdown_connections 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe880725e0 0x7efe88074aa0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efea815efc0 0x7efea80c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efea80d0490 0x7efea8165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 --2- 172.21.15.97:0/2960202917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea80cdcb0 0x7efea814cea0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.011+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 >> 172.21.15.97:0/2960202917 conn(0x7efea80d1a60 msgr2=0x7efea815db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.015+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 shutdown_connections 2024-09-10T21:32:52.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.015+0000 7efeaf000640 1 -- 172.21.15.97:0/2960202917 wait complete. 2024-09-10T21:32:52.033 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:52.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.175+0000 7fcf1ca00640 1 -- 172.21.15.97:0/2201486727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 msgr2=0x7fcf18072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.175+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/2201486727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf18072630 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fcf180733e0 tx=0x7fcf0001db00 comp rx=0 tx=0).stop 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- 172.21.15.97:0/2201486727 shutdown_connections 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/2201486727 >> v2:172.21.15.194:3300/0 conn(0x7fcf1815b110 0x7fcf18072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/2201486727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf18072630 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- 172.21.15.97:0/2201486727 >> 172.21.15.97:0/2201486727 conn(0x7fcf180b1780 msgr2=0x7fcf180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- 172.21.15.97:0/2201486727 shutdown_connections 2024-09-10T21:32:52.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- 172.21.15.97:0/2201486727 wait complete. 2024-09-10T21:32:52.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 Processor -- start 2024-09-10T21:32:52.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- start start 2024-09-10T21:32:52.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf180cf0b0 0x7fcf18142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf1814cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcf1815b110 0x7fcf1814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18166ee0 con 0x7fcf180cf490 2024-09-10T21:32:52.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18166d60 con 0x7fcf180cf0b0 2024-09-10T21:32:52.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcf18167060 con 0x7fcf1815b110 2024-09-10T21:32:52.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf1814cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf1814cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33476/0 (socket says 172.21.15.97:33476) 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 -- 172.21.15.97:0/4043515765 learned_addr learned my addr 172.21.15.97:0/4043515765 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf16800640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcf1815b110 0x7fcf1814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf15e00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf180cf0b0 0x7fcf18142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 -- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcf1815b110 msgr2=0x7fcf1814d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcf1815b110 0x7fcf1814d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 -- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf180cf0b0 msgr2=0x7fcf18142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf180cf0b0 0x7fcf18142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 -- 172.21.15.97:0/4043515765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf18151aa0 con 0x7fcf180cf490 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf15e00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf180cf0b0 0x7fcf18142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0d400640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf1814cdc0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000c10 tx=0x7fcf000a6a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0ea00640 1 -- 172.21.15.97:0/4043515765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000be070 con 0x7fcf180cf490 2024-09-10T21:32:52.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0ea00640 1 -- 172.21.15.97:0/4043515765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf000039f0 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf0ea00640 1 -- 172.21.15.97:0/4043515765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000b3040 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.179+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf18151d80 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.183+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf18152340 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.183+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1806e3b0 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.183+0000 7fcf0ea00640 1 -- 172.21.15.97:0/4043515765 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcf0000b040 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.183+0000 7fcf0ea00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcef40725e0 0x7fcef4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.183+0000 7fcf0ea00640 1 -- 172.21.15.97:0/4043515765 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7fcf0012deb0 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.183+0000 7fcf15e00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcef40725e0 0x7fcef4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.187+0000 7fcf15e00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcef40725e0 0x7fcef4074aa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcf080021a0 tx=0x7fcf08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.187+0000 7fcf0ea00640 1 -- 172.21.15.97:0/4043515765 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcf00017720 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf180af6c0 con 0x7fcf180cf490 2024-09-10T21:32:52.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf0ea00640 1 -- 172.21.15.97:0/4043515765 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+3569 (secure 0 0 0) 0x7fcf000f8d40 con 0x7fcf180cf490 2024-09-10T21:32:52.474 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:52.474 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":190,"btime":"2024-09-10T21:32:51:642642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:51.642562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:52.475 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 190 2024-09-10T21:32:52.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcef40725e0 msgr2=0x7fcef4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcef40725e0 0x7fcef4074aa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcf080021a0 tx=0x7fcf08009040 comp rx=0 tx=0).stop 2024-09-10T21:32:52.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 msgr2=0x7fcf1814cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf1814cdc0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000c10 tx=0x7fcf000a6a60 comp rx=0 tx=0).stop 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 shutdown_connections 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcef40725e0 0x7fcef4074aa0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcf1815b110 0x7fcf1814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf180cf490 0x7fcf1814cdc0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 --2- 172.21.15.97:0/4043515765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf180cf0b0 0x7fcf18142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 >> 172.21.15.97:0/4043515765 conn(0x7fcf180b1780 msgr2=0x7fcf181463b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 shutdown_connections 2024-09-10T21:32:52.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.391+0000 7fcf1ca00640 1 -- 172.21.15.97:0/4043515765 wait complete. 2024-09-10T21:32:52.476 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:32:52.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:52.535 INFO:tasks.ceph.mds.c.smithi097.stderr: -40> 2024-09-10T21:32:47.783+0000 7f86e1c00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-10T21:32:52.573 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:32:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.619+0000 7f5cdc400640 1 -- 172.21.15.97:0/1040642407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cbc003a40 msgr2=0x7f5cbc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.619+0000 7f5cdc400640 1 --2- 172.21.15.97:0/1040642407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cbc003a40 0x7f5cbc003e60 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009b60 tx=0x7f5cc801d1b0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.619+0000 7f5cda400640 1 -- 172.21.15.97:0/1040642407 reap_dead start 2024-09-10T21:32:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- 172.21.15.97:0/1040642407 shutdown_connections 2024-09-10T21:32:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- 172.21.15.97:0/1040642407 >> 172.21.15.97:0/1040642407 conn(0x7f5cd40d1c30 msgr2=0x7f5cd40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- 172.21.15.97:0/1040642407 shutdown_connections 2024-09-10T21:32:52.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- 172.21.15.97:0/1040642407 wait complete. 2024-09-10T21:32:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 Processor -- start 2024-09-10T21:32:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- start start 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cbc003a40 0x7f5cd41f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cd406f7b0 0x7f5cd41fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cd41fbbe0 0x7f5cd40b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4168270 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41680f0 con 0x7f5cd406f7b0 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd41683f0 con 0x7f5cbc003a40 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cd41fbbe0 0x7f5cd40b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cd41fbbe0 0x7f5cd40b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33514/0 (socket says 172.21.15.97:33514) 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 -- 172.21.15.97:0/1974377914 learned_addr learned my addr 172.21.15.97:0/1974377914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cd9a00640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cbc003a40 0x7f5cd41f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cd9000640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cd406f7b0 0x7f5cd41fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 -- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cbc003a40 msgr2=0x7f5cd41f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cbc003a40 0x7f5cd41f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 -- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cd406f7b0 msgr2=0x7f5cd41fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cd406f7b0 0x7f5cd41fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 -- 172.21.15.97:0/1974377914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd406fc30 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cda400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cd41fbbe0 0x7f5cd40b3720 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40d0f20 tx=0x7f5ccc006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cd1600640 1 -- 172.21.15.97:0/1974377914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc015020 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cd1600640 1 -- 172.21.15.97:0/1974377914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ccc011900 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cd1600640 1 -- 172.21.15.97:0/1974377914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc011d50 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41fc120 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.623+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41f8400 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.627+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd40aedc0 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.627+0000 7f5cd1600640 1 -- 172.21.15.97:0/1974377914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5ccc0103c0 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.627+0000 7f5cd1600640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cb40725e0 0x7f5cb4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.627+0000 7f5cd1600640 1 -- 172.21.15.97:0/1974377914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f5ccc094480 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.627+0000 7f5cd9a00640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cb40725e0 0x7f5cb4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.631+0000 7f5cd9a00640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cb40725e0 0x7f5cb4074aa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80ae000 tx=0x7f5cc8019f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.631+0000 7f5cd1600640 1 -- 172.21.15.97:0/1974377914 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5ccc01f030 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.831+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cd40b11f0 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cd1600640 1 -- 172.21.15.97:0/1974377914 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) ==== 95+0+4395 (secure 0 0 0) 0x7f5ccc05ec30 con 0x7f5cd41fbbe0 2024-09-10T21:32:52.833 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:52.833 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":191,"btime":"2024-09-10T21:32:52:806113+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3111622295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3111622295},{"type":"v1","addr":"172.21.15.97:6835","nonce":3111622295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:51.642562+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:52.833 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 191 2024-09-10T21:32:52.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cb40725e0 msgr2=0x7f5cb4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cb40725e0 0x7f5cb4074aa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80ae000 tx=0x7f5cc8019f50 comp rx=0 tx=0).stop 2024-09-10T21:32:52.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cd41fbbe0 msgr2=0x7f5cd40b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cd41fbbe0 0x7f5cd40b3720 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40d0f20 tx=0x7f5ccc006430 comp rx=0 tx=0).stop 2024-09-10T21:32:52.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 shutdown_connections 2024-09-10T21:32:52.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cb40725e0 0x7f5cb4074aa0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cd41fbbe0 0x7f5cd40b3720 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cd406f7b0 0x7f5cd41fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 --2- 172.21.15.97:0/1974377914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cbc003a40 0x7f5cd41f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 >> 172.21.15.97:0/1974377914 conn(0x7f5cd40d1c30 msgr2=0x7f5cd415b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 shutdown_connections 2024-09-10T21:32:52.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:52.835+0000 7f5cdc400640 1 -- 172.21.15.97:0/1974377914 wait complete. 2024-09-10T21:32:52.854 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2024-09-10T21:32:52.854 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2024-09-10T21:32:52.854 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:32:52.854 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:32:52.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:32:52.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:32:52.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:32:52.864 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:32:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:32:52.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:32:52.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:32:53.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- 172.21.15.97:0/452232758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40cec10 msgr2=0x7f87e40cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 --2- 172.21.15.97:0/452232758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40cec10 0x7f87e40cfd40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f87e40744b0 tx=0x7f87d001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- 172.21.15.97:0/452232758 shutdown_connections 2024-09-10T21:32:53.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 --2- 172.21.15.97:0/452232758 >> v2:172.21.15.194:3300/0 conn(0x7f87e40d0280 0x7f87e415fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 --2- 172.21.15.97:0/452232758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40cec10 0x7f87e40cfd40 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 --2- 172.21.15.97:0/452232758 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e40ce830 0x7f87e40cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- 172.21.15.97:0/452232758 >> 172.21.15.97:0/452232758 conn(0x7f87e40b37c0 msgr2=0x7f87e40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:53.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- 172.21.15.97:0/452232758 shutdown_connections 2024-09-10T21:32:53.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- 172.21.15.97:0/452232758 wait complete. 2024-09-10T21:32:53.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 Processor -- start 2024-09-10T21:32:53.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- start start 2024-09-10T21:32:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40ce830 0x7f87e4150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40cec10 0x7f87e40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e40d0280 0x7f87e41f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87e40b4420 con 0x7f87e40ce830 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87e40cde60 con 0x7f87e40cec10 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.103+0000 7f87eaa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f87e40ce1a0 con 0x7f87e40d0280 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40ce830 0x7f87e4150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40ce830 0x7f87e4150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33528/0 (socket says 172.21.15.97:33528) 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 -- 172.21.15.97:0/2903485805 learned_addr learned my addr 172.21.15.97:0/2903485805 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e8a00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e40d0280 0x7f87e41f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3400640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40cec10 0x7f87e40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 -- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e40d0280 msgr2=0x7f87e41f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e40d0280 0x7f87e41f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 -- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40cec10 msgr2=0x7f87e40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40cec10 0x7f87e40cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 -- 172.21.15.97:0/2903485805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e415e5d0 con 0x7f87e40ce830 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87e3e00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40ce830 0x7f87e4150110 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f87d400ead0 tx=0x7f87d400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87dbe00640 1 -- 172.21.15.97:0/2903485805 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d400ce40 con 0x7f87e40ce830 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87dbe00640 1 -- 172.21.15.97:0/2903485805 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87d4004c20 con 0x7f87e40ce830 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e415e8b0 con 0x7f87e40ce830 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87dbe00640 1 -- 172.21.15.97:0/2903485805 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4005280 con 0x7f87e40ce830 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e40c7670 con 0x7f87e40ce830 2024-09-10T21:32:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e415fd20 con 0x7f87e40ce830 2024-09-10T21:32:53.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.107+0000 7f87dbe00640 1 -- 172.21.15.97:0/2903485805 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f87d400f040 con 0x7f87e40ce830 2024-09-10T21:32:53.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.111+0000 7f87dbe00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c40725e0 0x7f87c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.111+0000 7f87dbe00640 1 -- 172.21.15.97:0/2903485805 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f87d4094240 con 0x7f87e40ce830 2024-09-10T21:32:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.111+0000 7f87e3400640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c40725e0 0x7f87c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.111+0000 7f87e3400640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c40725e0 0x7f87c4074aa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f87d00b2000 tx=0x7f87d001d8f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:53.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.115+0000 7f87dbe00640 1 -- 172.21.15.97:0/2903485805 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f87d4016030 con 0x7f87e40ce830 2024-09-10T21:32:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.315+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f87e40b7af0 con 0x7f87e40ce830 2024-09-10T21:32:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.811+0000 7f87dbe00640 1 -- 172.21.15.97:0/2903485805 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v192) ==== 161+0+0 (secure 0 0 0) 0x7f87d405e960 con 0x7f87e40ce830 2024-09-10T21:32:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:32:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c40725e0 msgr2=0x7f87c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c40725e0 0x7f87c4074aa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f87d00b2000 tx=0x7f87d001d8f0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40ce830 msgr2=0x7f87e4150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40ce830 0x7f87e4150110 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f87d400ead0 tx=0x7f87d400ef90 comp rx=0 tx=0).stop 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 shutdown_connections 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c40725e0 0x7f87c4074aa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e40d0280 0x7f87e41f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40cec10 0x7f87e40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 --2- 172.21.15.97:0/2903485805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e40ce830 0x7f87e4150110 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 >> 172.21.15.97:0/2903485805 conn(0x7f87e40b37c0 msgr2=0x7f87e40af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 shutdown_connections 2024-09-10T21:32:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.815+0000 7f87eaa00640 1 -- 172.21.15.97:0/2903485805 wait complete. 2024-09-10T21:32:53.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:32:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- 172.21.15.97:0/3426049378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd050003a60 msgr2=0x7fd050003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 --2- 172.21.15.97:0/3426049378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd050003a60 0x7fd050003e80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009b60 tx=0x7fd05c01d270 comp rx=0 tx=0).stop 2024-09-10T21:32:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- 172.21.15.97:0/3426049378 shutdown_connections 2024-09-10T21:32:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 --2- 172.21.15.97:0/3426049378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd050003a60 0x7fd050003e80 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- 172.21.15.97:0/3426049378 >> 172.21.15.97:0/3426049378 conn(0x7fd06c0d1ca0 msgr2=0x7fd06c0d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- 172.21.15.97:0/3426049378 shutdown_connections 2024-09-10T21:32:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- 172.21.15.97:0/3426049378 wait complete. 2024-09-10T21:32:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 Processor -- start 2024-09-10T21:32:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- start start 2024-09-10T21:32:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd050003a60 0x7fd06c1fb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06c0b7630 0x7fd06c0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd06c15ce30 0x7fd06c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c0cf5f0 con 0x7fd06c0b7630 2024-09-10T21:32:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c0cf470 con 0x7fd050003a60 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd072e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c15fd90 con 0x7fd06c15ce30 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.983+0000 7fd063400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06c0b7630 0x7fd06c0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06c0b7630 0x7fd06c0ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33544/0 (socket says 172.21.15.97:33544) 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 -- 172.21.15.97:0/160933733 learned_addr learned my addr 172.21.15.97:0/160933733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd06be00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd050003a60 0x7fd06c1fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd070e00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd06c15ce30 0x7fd06c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 -- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd06c15ce30 msgr2=0x7fd06c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd06c15ce30 0x7fd06c15d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 -- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd050003a60 msgr2=0x7fd06c1fb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd050003a60 0x7fd06c1fb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 -- 172.21.15.97:0/160933733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c1fba80 con 0x7fd06c0b7630 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd06be00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd050003a60 0x7fd06c1fb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd063400640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06c0b7630 0x7fd06c0ca860 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fd054004d70 tx=0x7fd054001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd068c00640 1 -- 172.21.15.97:0/160933733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054001d50 con 0x7fd06c0b7630 2024-09-10T21:32:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd068c00640 1 -- 172.21.15.97:0/160933733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd054016070 con 0x7fd06c0b7630 2024-09-10T21:32:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd068c00640 1 -- 172.21.15.97:0/160933733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054010b40 con 0x7fd06c0b7630 2024-09-10T21:32:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c1fbd60 con 0x7fd06c0b7630 2024-09-10T21:32:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c0b4af0 con 0x7fd06c0b7630 2024-09-10T21:32:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06c1602f0 con 0x7fd06c0b7630 2024-09-10T21:32:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.987+0000 7fd068c00640 1 -- 172.21.15.97:0/160933733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd054010ce0 con 0x7fd06c0b7630 2024-09-10T21:32:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.991+0000 7fd068c00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd0480725e0 0x7fd048074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.991+0000 7fd068c00640 1 -- 172.21.15.97:0/160933733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7fd054094310 con 0x7fd06c0b7630 2024-09-10T21:32:53.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.991+0000 7fd06be00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd0480725e0 0x7fd048074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.991+0000 7fd06be00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd0480725e0 0x7fd048074aa0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0ae000 tx=0x7fd05c01a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:53.991+0000 7fd068c00640 1 -- 172.21.15.97:0/160933733 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd05405f110 con 0x7fd06c0b7630 2024-09-10T21:32:54.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.215+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd06c0d0060 con 0x7fd06c0b7630 2024-09-10T21:32:54.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.823+0000 7fd068c00640 1 -- 172.21.15.97:0/160933733 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v193) ==== 101+0+0 (secure 0 0 0) 0x7fd05405ea10 con 0x7fd06c0b7630 2024-09-10T21:32:54.823 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:32:54.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd0480725e0 msgr2=0x7fd048074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:54.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd0480725e0 0x7fd048074aa0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0ae000 tx=0x7fd05c01a010 comp rx=0 tx=0).stop 2024-09-10T21:32:54.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06c0b7630 msgr2=0x7fd06c0ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:54.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06c0b7630 0x7fd06c0ca860 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fd054004d70 tx=0x7fd054001640 comp rx=0 tx=0).stop 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 shutdown_connections 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd0480725e0 0x7fd048074aa0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd06c15ce30 0x7fd06c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06c0b7630 0x7fd06c0ca860 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 --2- 172.21.15.97:0/160933733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd050003a60 0x7fd06c1fb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 >> 172.21.15.97:0/160933733 conn(0x7fd06c0d1ca0 msgr2=0x7fd06c15f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 shutdown_connections 2024-09-10T21:32:54.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.827+0000 7fd072e00640 1 -- 172.21.15.97:0/160933733 wait complete. 2024-09-10T21:32:54.847 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:32:54.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 Processor -- start 2024-09-10T21:32:54.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- start start 2024-09-10T21:32:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf1204aa0 0x55daf12da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12bdca0 0x55daf12d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf12be080 0x55daf12d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55daf1152a00 con 0x55daf1204aa0 2024-09-10T21:32:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55daf11b0f40 con 0x55daf12bdca0 2024-09-10T21:32:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55daf12e42d0 con 0x55daf12be080 2024-09-10T21:32:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf1204aa0 0x55daf12da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf1204aa0 0x55daf12da0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33564/0 (socket says 172.21.15.97:33564) 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 -- 172.21.15.97:0/1286144689 learned_addr learned my addr 172.21.15.97:0/1286144689 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb18e00640 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12bdca0 0x55daf12d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1a200640 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf12be080 0x55daf12d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 -- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf12be080 msgr2=0x55daf12d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf12be080 0x55daf12d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 -- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12bdca0 msgr2=0x55daf12d7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12bdca0 0x55daf12d7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 -- 172.21.15.97:0/1286144689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf112bc10 con 0x55daf1204aa0 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1a200640 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf12be080 0x55daf12d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb19800640 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf1204aa0 0x55daf12da0b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7feb0c019900 tx=0x7feb0c017ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d07e3feae1a1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb13400640 1 -- 172.21.15.97:0/1286144689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0ab630 con 0x55daf1204aa0 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb13400640 1 -- 172.21.15.97:0/1286144689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb0c0bb070 con 0x55daf1204aa0 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb13400640 1 -- 172.21.15.97:0/1286144689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0abd30 con 0x55daf1204aa0 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- 172.21.15.97:0/1286144689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf1204aa0 msgr2=0x55daf12da0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf1204aa0 0x55daf12da0b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7feb0c019900 tx=0x7feb0c017ff0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- 172.21.15.97:0/1286144689 shutdown_connections 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf12be080 0x55daf12d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12bdca0 0x55daf12d7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 --2- 172.21.15.97:0/1286144689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf1204aa0 0x55daf12da0b0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- 172.21.15.97:0/1286144689 >> 172.21.15.97:0/1286144689 conn(0x55daf1203140 msgr2=0x55daf1202030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- 172.21.15.97:0/1286144689 shutdown_connections 2024-09-10T21:32:54.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- 172.21.15.97:0/1286144689 wait complete. 2024-09-10T21:32:54.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 Processor -- start 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.895+0000 7feb1c517080 1 -- start start 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf1204aa0 0x55daf12bc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf12bdca0 0x55daf1245c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12be080 0x55daf12e1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55daf12dbb70 con 0x55daf12bdca0 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55daf12da720 con 0x55daf12be080 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55daf12da980 con 0x55daf1204aa0 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf12bdca0 0x55daf1245c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf12bdca0 0x55daf1245c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33568/0 (socket says 172.21.15.97:33568) 2024-09-10T21:32:54.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 learned_addr learned my addr 172.21.15.97:0/3409439272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb19800640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf1204aa0 0x55daf12bc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1a200640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12be080 0x55daf12e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf1204aa0 msgr2=0x55daf12bc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55daf1204aa0 0x55daf12bc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12be080 msgr2=0x55daf12e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55daf12be080 0x55daf12e1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf1244d70 con 0x55daf12bdca0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb18e00640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf12bdca0 0x55daf1245c90 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7feb08008580 tx=0x7feb08008a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb12000640 1 -- 172.21.15.97:0/3409439272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0800aa70 con 0x55daf12bdca0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb12000640 1 -- 172.21.15.97:0/3409439272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb0800ac10 con 0x55daf12bdca0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55daf1245010 con 0x55daf12bdca0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb12000640 1 -- 172.21.15.97:0/3409439272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08015420 con 0x55daf12bdca0 2024-09-10T21:32:54.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf1245550 con 0x55daf12bdca0 2024-09-10T21:32:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.899+0000 7feb12000640 1 -- 172.21.15.97:0/3409439272 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feb080155c0 con 0x55daf12bdca0 2024-09-10T21:32:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb12000640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feafc070ff0 0x7feafc0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb12000640 1 -- 172.21.15.97:0/3409439272 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7feb08092bf0 con 0x55daf12bdca0 2024-09-10T21:32:54.903 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:32:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb19800640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feafc070ff0 0x7feafc0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55daf12461d0 0x55daf12bf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1244050 con 0x55daf12461d0 2024-09-10T21:32:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb1a200640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55daf12461d0 0x55daf12bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb19800640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feafc070ff0 0x7feafc0734b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0ba000 tx=0x7feb0c004950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:54.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.903+0000 7feb1a200640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55daf12461d0 0x55daf12bf300 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:54.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.907+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v114'536 uv510 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.907+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1244420 con 0x55daf12461d0 2024-09-10T21:32:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.907+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v114'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.907+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf12447c0 con 0x55daf12461d0 2024-09-10T21:32:54.907 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:32:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.907+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v114'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.907+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137f3a0 con 0x55daf12461d0 2024-09-10T21:32:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.911+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.911+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137f740 con 0x55daf12461d0 2024-09-10T21:32:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.911+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.911+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137e390 con 0x55daf12461d0 2024-09-10T21:32:54.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.915+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.915+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137e780 con 0x55daf12461d0 2024-09-10T21:32:54.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.915+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v114'542 uv516 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.915+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf137eb70 con 0x55daf12461d0 2024-09-10T21:32:54.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.915+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v114'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.915+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb70 con 0x55daf12461d0 2024-09-10T21:32:54.917 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:32:54.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.919+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v114'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.919+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.919+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.919+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.923+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.923+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.923+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.923+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.923+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v114'548 uv523 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.923+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.927+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'549 uv549 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.927+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.927 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:32:54.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.927+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v114'550 uv550 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.927+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.927+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'551 uv551 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.931+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.931+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'552 uv552 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.931+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137eb20 con 0x55daf12461d0 2024-09-10T21:32:54.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.931+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'553 uv553 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.931+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.935+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'554 uv554 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.935+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.935+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v114'555 uv529 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.935+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.935+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v114'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.935+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.937 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:32:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.939+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v114'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.939+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1380af0 con 0x55daf12461d0 2024-09-10T21:32:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.939+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.939+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1380af0 con 0x55daf12461d0 2024-09-10T21:32:54.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.943+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.943+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.943+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.943+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.943+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v114'561 uv535 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.943+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.947+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v114'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.947+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.946 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:32:54.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.947+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v114'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.947+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1380af0 con 0x55daf12461d0 2024-09-10T21:32:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.947+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.947+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1380af0 con 0x55daf12461d0 2024-09-10T21:32:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.951+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.951+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf137fae0 con 0x55daf12461d0 2024-09-10T21:32:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.951+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf12461d0 2024-09-10T21:32:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.951+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55daf137fff0 0x55daf13823e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.951+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1382920 con 0x55daf137fff0 2024-09-10T21:32:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.951+0000 7feb19800640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55daf137fff0 0x55daf13823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.951+0000 7feb19800640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55daf137fff0 0x55daf13823e0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.955+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'396 uv378 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.955+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1382cc0 con 0x55daf137fff0 2024-09-10T21:32:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.955+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'397 uv397 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.955+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1383060 con 0x55daf137fff0 2024-09-10T21:32:54.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.959+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'398 uv398 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.957 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:32:54.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.959+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf13838b0 con 0x55daf137fff0 2024-09-10T21:32:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.959+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v114'399 uv399 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.959+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1383c50 con 0x55daf137fff0 2024-09-10T21:32:54.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.959+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'400 uv400 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.959+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1383ff0 con 0x55daf137fff0 2024-09-10T21:32:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.963+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v114'401 uv383 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.963+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1384390 con 0x55daf137fff0 2024-09-10T21:32:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.963+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v114'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.963+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.963+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v114'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.964 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:32:54.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.963+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.967+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.967+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.967+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.967+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.967+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v114'406 uv395 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.967+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.971+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v114'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.971+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.971+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v114'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.970 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:32:54.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.971+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1388900 con 0x55daf137fff0 2024-09-10T21:32:54.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.971+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v114'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.971+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1388900 con 0x55daf137fff0 2024-09-10T21:32:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v114'411 uv394 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v114'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.976 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:32:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v114'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.975+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1384730 con 0x55daf137fff0 2024-09-10T21:32:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf137fff0 2024-09-10T21:32:54.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55daf1385030 0x55daf1387420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1387960 con 0x55daf1385030 2024-09-10T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb18e00640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55daf1385030 0x55daf1387420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.979+0000 7feb18e00640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55daf1385030 0x55daf1387420 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:54.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.983+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.983+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1387d40 con 0x55daf1385030 2024-09-10T21:32:54.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.983+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.983+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf13881b0 con 0x55daf1385030 2024-09-10T21:32:54.983 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:32:54.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.983+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.983+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1389a60 con 0x55daf1385030 2024-09-10T21:32:54.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1389e00 con 0x55daf1385030 2024-09-10T21:32:54.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1388a50 con 0x55daf1385030 2024-09-10T21:32:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55daf1389370 0x55daf138c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:54.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138c6f0 con 0x55daf1389370 2024-09-10T21:32:54.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb1a200640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55daf1389370 0x55daf138c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:54.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.987+0000 7feb1a200640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55daf1389370 0x55daf138c1b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:54.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.991+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'83 uv82 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:54.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.991+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf138c6f0 con 0x55daf1389370 2024-09-10T21:32:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.991+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'84 uv84 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:54.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.991+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138c6f0 con 0x55daf1389370 2024-09-10T21:32:54.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.991+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'85 uv85 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:54.991 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:32:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.991+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138cb10 con 0x55daf1389370 2024-09-10T21:32:54.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'86 uv86 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:54.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138cb10 con 0x55daf1385030 2024-09-10T21:32:54.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:54.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:54.996 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:32:54.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.995+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138dec0 con 0x55daf1385030 2024-09-10T21:32:54.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.999+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.999+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138dec0 con 0x55daf1385030 2024-09-10T21:32:54.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.999+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.999+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.999+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:54.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:54.999+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.003+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v114'193 uv187 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.003+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.003+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v114'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.003+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.003 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:32:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.003+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v114'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.003+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138dec0 con 0x55daf1389370 2024-09-10T21:32:55.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138dec0 con 0x55daf1389370 2024-09-10T21:32:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v114'199 uv192 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.007+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.011+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v114'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.011+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.010 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:32:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.011+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v114'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.011+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.011+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.011+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1389370 2024-09-10T21:32:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.015+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.015+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.015+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v114'86 uv85 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.015+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.015+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v114'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.015+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:55.016 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:32:55.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v114'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'89 uv89 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ceb0 con 0x55daf1385030 2024-09-10T21:32:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'90 uv90 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1385030 2024-09-10T21:32:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55daf138d530 0x55daf138f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf138ff70 con 0x55daf138d530 2024-09-10T21:32:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb19800640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55daf138d530 0x55daf138f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.019+0000 7feb19800640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55daf138d530 0x55daf138f940 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.023+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.023+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1390030 con 0x55daf138d530 2024-09-10T21:32:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.023+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.023+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.023 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:32:55.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.023+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.023+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1391460 con 0x55daf138d530 2024-09-10T21:32:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1391460 con 0x55daf138d530 2024-09-10T21:32:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v114'172 uv166 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.027+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.031+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v114'173 uv173 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.031+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.030 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:32:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.031+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v114'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.031+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.031+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.031+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.035+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.035+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.035+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v114'177 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.035+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.035+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v114'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.035+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.037 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:32:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.039+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v114'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.039+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390450 con 0x55daf138d530 2024-09-10T21:32:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.039+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.039+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393260 con 0x55daf138d530 2024-09-10T21:32:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.039+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.039+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.043+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v114'78 uv77 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.043+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.043+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'79 uv79 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.043+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.043 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:32:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.043+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v114'80 uv80 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.043+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'81 uv81 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v114'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v114'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.048 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:32:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v114'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.047+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1390900 con 0x55daf1389370 2024-09-10T21:32:55.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb14003950 con 0x55daf1389370 2024-09-10T21:32:55.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55daf1391200 0x55daf1393880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf1391200 2024-09-10T21:32:55.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb18e00640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55daf1391200 0x55daf1393880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.051+0000 7feb18e00640 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55daf1391200 0x55daf1393880 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.055+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v114'89 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1391200 2024-09-10T21:32:55.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.055+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf1391200 2024-09-10T21:32:55.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.055+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v114'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1391200 2024-09-10T21:32:55.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.055+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf1391200 2024-09-10T21:32:55.055 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:32:55.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.055+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v114'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1391200 2024-09-10T21:32:55.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.055+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf1391200 2024-09-10T21:32:55.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.059+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1391200 2024-09-10T21:32:55.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.059+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf1391200 2024-09-10T21:32:55.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.059+0000 7feb18e00640 1 -- 172.21.15.97:0/3409439272 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feb08059620 con 0x55daf1391200 2024-09-10T21:32:55.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.059+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.059+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v114'216 uv208 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.059+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.063+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v114'217 uv217 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.063+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.062 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:32:55.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.063+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v114'218 uv218 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.063+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.063+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v114'219 uv219 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.063+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.063+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v114'220 uv220 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.067+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.067+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v114'221 uv215 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.067+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.067+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v114'222 uv222 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.067+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.067 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:32:55.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.067+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v114'223 uv223 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.067+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1394dd0 con 0x55daf138d530 2024-09-10T21:32:55.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.071+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v114'224 uv224 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.071+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1394dd0 con 0x55daf138d530 2024-09-10T21:32:55.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.071+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v114'225 uv225 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.071+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.071+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v114'226 uv226 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.071+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55daf1393dc0 con 0x55daf138d530 2024-09-10T21:32:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb19800640 1 -- 172.21.15.97:0/3409439272 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v114'227 uv227 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feb0c0b3070 con 0x55daf138d530 2024-09-10T21:32:55.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55daf1385030 msgr2=0x55daf1387420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55daf1385030 0x55daf1387420 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55daf138d530 msgr2=0x55daf138f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55daf138d530 0x55daf138f940 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55daf1391200 msgr2=0x55daf1393880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55daf1391200 0x55daf1393880 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55daf1389370 msgr2=0x55daf138c1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55daf1389370 0x55daf138c1b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 reap_dead start 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55daf137fff0 msgr2=0x55daf13823e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55daf137fff0 0x55daf13823e0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1a200640 1 -- 172.21.15.97:0/3409439272 reap_dead start 2024-09-10T21:32:55.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55daf12461d0 msgr2=0x55daf12bf300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55daf12461d0 0x55daf12bf300 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feafc070ff0 msgr2=0x7feafc0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feafc070ff0 0x7feafc0734b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0ba000 tx=0x7feb0c004950 comp rx=0 tx=0).stop 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf12bdca0 msgr2=0x55daf1245c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf12bdca0 0x55daf1245c90 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7feb08008580 tx=0x7feb08008a40 comp rx=0 tx=0).stop 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 shutdown_connections 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55daf12461d0 0x55daf12bf300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7feafc070ff0 0x7feafc0734b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 --2- 172.21.15.97:0/3409439272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf12bdca0 0x55daf1245c90 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 >> 172.21.15.97:0/3409439272 conn(0x55daf1203140 msgr2=0x55daf12d7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 shutdown_connections 2024-09-10T21:32:55.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.075+0000 7feb1c517080 1 -- 172.21.15.97:0/3409439272 wait complete. 2024-09-10T21:32:55.078 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2024-09-10T21:32:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 Processor -- start 2024-09-10T21:32:55.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 -- start start 2024-09-10T21:32:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2267e6940 0x55e2268701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e226870b00 0x55e22686c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e226705a00 con 0x55e2268712a0 2024-09-10T21:32:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e226872e70 con 0x55e2267e6940 2024-09-10T21:32:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4fad1080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55e226870710 con 0x55e226870b00 2024-09-10T21:32:55.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33590/0 (socket says 172.21.15.97:33590) 2024-09-10T21:32:55.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 -- 172.21.15.97:0/1973132158 learned_addr learned my addr 172.21.15.97:0/1973132158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:55.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b47e00640 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e226870b00 0x55e22686c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4cc00640 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2267e6940 0x55e2268701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 -- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e226870b00 msgr2=0x55e22686c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:55.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e226870b00 0x55e22686c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 -- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2267e6940 msgr2=0x55e2268701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2267e6940 0x55e2268701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 -- 172.21.15.97:0/1973132158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e226704a20 con 0x55e2268712a0 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b4d600640 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5da0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800b950 tx=0x7f6b4800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754e5c2701d058d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b46a00640 1 -- 172.21.15.97:0/1973132158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800dbf0 con 0x55e2268712a0 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b46a00640 1 -- 172.21.15.97:0/1973132158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b4800de80 con 0x55e2268712a0 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.167+0000 7f6b46a00640 1 -- 172.21.15.97:0/1973132158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800f690 con 0x55e2268712a0 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1973132158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 msgr2=0x55e2267b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5da0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800b950 tx=0x7f6b4800be10 comp rx=0 tx=0).stop 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1973132158 shutdown_connections 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5da0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e226870b00 0x55e22686c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1973132158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2267e6940 0x55e2268701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1973132158 >> 172.21.15.97:0/1973132158 conn(0x55e2267e5520 msgr2=0x55e22688d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:55.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1973132158 shutdown_connections 2024-09-10T21:32:55.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1973132158 wait complete. 2024-09-10T21:32:55.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 Processor -- start 2024-09-10T21:32:55.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- start start 2024-09-10T21:32:55.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e2267e6940 0x55e226889470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e226870b00 0x55e22686d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e22686ba40 con 0x55e2268712a0 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e22686dbc0 con 0x55e226870b00 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55e22686de60 con 0x55e2267e6940 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33604/0 (socket says 172.21.15.97:33604) 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 -- 172.21.15.97:0/1439175611 learned_addr learned my addr 172.21.15.97:0/1439175611 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b47e00640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e226870b00 0x55e22686d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4cc00640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e2267e6940 0x55e226889470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 -- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e2267e6940 msgr2=0x55e226889470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55e2267e6940 0x55e226889470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 -- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e226870b00 msgr2=0x55e22686d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e226870b00 0x55e22686d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 -- 172.21.15.97:0/1439175611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2266dec10 con 0x55e2268712a0 2024-09-10T21:32:55.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4d600640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5ea0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800d720 tx=0x7f6b4800dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b45600640 1 -- 172.21.15.97:0/1439175611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48027070 con 0x55e2268712a0 2024-09-10T21:32:55.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b45600640 1 -- 172.21.15.97:0/1439175611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b4800f640 con 0x55e2268712a0 2024-09-10T21:32:55.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e22686e100 con 0x55e2268712a0 2024-09-10T21:32:55.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b45600640 1 -- 172.21.15.97:0/1439175611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800fa70 con 0x55e2268712a0 2024-09-10T21:32:55.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.171+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e22686e640 con 0x55e2268712a0 2024-09-10T21:32:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b45600640 1 -- 172.21.15.97:0/1439175611 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6b4800fc10 con 0x55e2268712a0 2024-09-10T21:32:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b45600640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b2c070ff0 0x7f6b2c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b45600640 1 -- 172.21.15.97:0/1439175611 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f6b48094120 con 0x55e2268712a0 2024-09-10T21:32:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b4cc00640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b2c070ff0 0x7f6b2c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e2267e9860 0x55e2267e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55e2267ea180 con 0x55e2267e9860 2024-09-10T21:32:55.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b47e00640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e2267e9860 0x55e2267e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b4cc00640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b2c070ff0 0x7f6b2c0734b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000c970 tx=0x7f6b4000f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.175+0000 7f6b47e00640 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e2267e9860 0x55e2267e9c40 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b47e00640 1 -- 172.21.15.97:0/1439175611 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v114'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6b380ac990 con 0x55e2267e9860 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e2267e9860 msgr2=0x55e2267e9c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55e2267e9860 0x55e2267e9c40 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b2c070ff0 msgr2=0x7f6b2c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6b2c070ff0 0x7f6b2c0734b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000c970 tx=0x7f6b4000f040 comp rx=0 tx=0).stop 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 msgr2=0x55e2267b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 --2- 172.21.15.97:0/1439175611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2268712a0 0x55e2267b5ea0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800d720 tx=0x7f6b4800dd60 comp rx=0 tx=0).stop 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4d600640 1 -- 172.21.15.97:0/1439175611 reap_dead start 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 shutdown_connections 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 >> 172.21.15.97:0/1439175611 conn(0x55e2267e5520 msgr2=0x55e22686c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 shutdown_connections 2024-09-10T21:32:55.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.179+0000 7f6b4fad1080 1 -- 172.21.15.97:0/1439175611 wait complete. 2024-09-10T21:32:55.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:32:55.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.367+0000 7f2b66c00640 1 -- 172.21.15.97:0/703526424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b600cfad0 msgr2=0x7f2b600cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.367+0000 7f2b66c00640 1 --2- 172.21.15.97:0/703526424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b600cfad0 0x7f2b600cff40 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009840 tx=0x7f2b4801c820 comp rx=0 tx=0).stop 2024-09-10T21:32:55.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- 172.21.15.97:0/703526424 shutdown_connections 2024-09-10T21:32:55.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 --2- 172.21.15.97:0/703526424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b600cfad0 0x7f2b600cff40 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- 172.21.15.97:0/703526424 >> 172.21.15.97:0/703526424 conn(0x7f2b600b28c0 msgr2=0x7f2b600b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:55.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- 172.21.15.97:0/703526424 shutdown_connections 2024-09-10T21:32:55.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- 172.21.15.97:0/703526424 wait complete. 2024-09-10T21:32:55.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 Processor -- start 2024-09-10T21:32:55.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- start start 2024-09-10T21:32:55.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b600af100 0x7f2b601f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b600cfad0 0x7f2b6014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015bdd0 0x7f2b600707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60169fd0 con 0x7f2b6015bdd0 2024-09-10T21:32:55.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60169e50 con 0x7f2b600af100 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b66c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6016a150 con 0x7f2b600cfad0 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015bdd0 0x7f2b600707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015bdd0 0x7f2b600707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33632/0 (socket says 172.21.15.97:33632) 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 -- 172.21.15.97:0/2077864409 learned_addr learned my addr 172.21.15.97:0/2077864409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b5fe00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b600af100 0x7f2b601f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b57400640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b600cfad0 0x7f2b6014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 -- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b600cfad0 msgr2=0x7f2b6014c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b600cfad0 0x7f2b6014c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 -- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b600af100 msgr2=0x7f2b601f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b600af100 0x7f2b601f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 -- 172.21.15.97:0/2077864409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6015ad40 con 0x7f2b6015bdd0 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b57400640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b600cfad0 0x7f2b6014c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.371+0000 7f2b64c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015bdd0 0x7f2b600707f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800c730 tx=0x7f2b5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b5cc00640 1 -- 172.21.15.97:0/2077864409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58003db0 con 0x7f2b6015bdd0 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b5cc00640 1 -- 172.21.15.97:0/2077864409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b5800ce40 con 0x7f2b6015bdd0 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b5cc00640 1 -- 172.21.15.97:0/2077864409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b580052d0 con 0x7f2b6015bdd0 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6015b020 con 0x7f2b6015bdd0 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b601f7e60 con 0x7f2b6015bdd0 2024-09-10T21:32:55.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b600cff40 con 0x7f2b6015bdd0 2024-09-10T21:32:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b5cc00640 1 -- 172.21.15.97:0/2077864409 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b58005470 con 0x7f2b6015bdd0 2024-09-10T21:32:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b5cc00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b3c0725e0 0x7f2b3c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b5cc00640 1 -- 172.21.15.97:0/2077864409 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f2b58093940 con 0x7f2b6015bdd0 2024-09-10T21:32:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.375+0000 7f2b5fe00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b3c0725e0 0x7f2b3c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.379+0000 7f2b5fe00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b3c0725e0 0x7f2b3c074aa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2b50007fd0 tx=0x7f2b50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.379+0000 7f2b5cc00640 1 -- 172.21.15.97:0/2077864409 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b5805e740 con 0x7f2b6015bdd0 2024-09-10T21:32:55.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.583+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2b60157850 con 0x7f2b6015bdd0 2024-09-10T21:32:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b5cc00640 1 -- 172.21.15.97:0/2077864409 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v194) ==== 162+0+0 (secure 0 0 0) 0x7f2b5805e040 con 0x7f2b6015bdd0 2024-09-10T21:32:55.825 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:32:55.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b3c0725e0 msgr2=0x7f2b3c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b3c0725e0 0x7f2b3c074aa0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2b50007fd0 tx=0x7f2b50009040 comp rx=0 tx=0).stop 2024-09-10T21:32:55.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015bdd0 msgr2=0x7f2b600707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015bdd0 0x7f2b600707f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800c730 tx=0x7f2b5800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 shutdown_connections 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b3c0725e0 0x7f2b3c074aa0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6015bdd0 0x7f2b600707f0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b600cfad0 0x7f2b6014c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 --2- 172.21.15.97:0/2077864409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b600af100 0x7f2b601f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 >> 172.21.15.97:0/2077864409 conn(0x7f2b600b28c0 msgr2=0x7f2b600b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 shutdown_connections 2024-09-10T21:32:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.827+0000 7f2b66c00640 1 -- 172.21.15.97:0/2077864409 wait complete. 2024-09-10T21:32:55.834 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:32:55.835+0000 7f0b36c00640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-10T21:32:55.834 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:32:55.835+0000 7f0b36c00640 -1 mds.0.195 unhandled write error (22) Invalid argument, force readonly... 2024-09-10T21:32:55.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:55.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 -- 172.21.15.97:0/1477928230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb354004d50 msgr2=0x7fb3540051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 --2- 172.21.15.97:0/1477928230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb354004d50 0x7fb3540051d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb368009ee0 tx=0x7fb36801d5f0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb36e800640 1 -- 172.21.15.97:0/1477928230 reap_dead start 2024-09-10T21:32:55.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 -- 172.21.15.97:0/1477928230 shutdown_connections 2024-09-10T21:32:55.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 -- 172.21.15.97:0/1477928230 >> 172.21.15.97:0/1477928230 conn(0x7fb3700b37c0 msgr2=0x7fb3700b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:55.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 -- 172.21.15.97:0/1477928230 shutdown_connections 2024-09-10T21:32:55.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 -- 172.21.15.97:0/1477928230 wait complete. 2024-09-10T21:32:55.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 Processor -- start 2024-09-10T21:32:55.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 -- start start 2024-09-10T21:32:55.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.995+0000 7fb374e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb354004d50 0x7fb37015d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700c4720 0x7fb3700c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3700c7470 0x7fb3700c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700ce1a0 con 0x7fb3700c4720 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700cde60 con 0x7fb3700c7470 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb3700ce320 con 0x7fb354004d50 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700c4720 0x7fb3700c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700c4720 0x7fb3700c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33648/0 (socket says 172.21.15.97:33648) 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 -- 172.21.15.97:0/990390213 learned_addr learned my addr 172.21.15.97:0/990390213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb36de00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb354004d50 0x7fb37015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb36e800640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3700c7470 0x7fb3700c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 -- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb354004d50 msgr2=0x7fb37015d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb354004d50 0x7fb37015d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 -- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3700c7470 msgr2=0x7fb3700c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3700c7470 0x7fb3700c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 -- 172.21.15.97:0/990390213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3701f8600 con 0x7fb3700c4720 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb365400640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700c4720 0x7fb3700c9ef0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb358011ae0 tx=0x7fb358009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb366a00640 1 -- 172.21.15.97:0/990390213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35800fcd0 con 0x7fb3700c4720 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb366a00640 1 -- 172.21.15.97:0/990390213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb35800fe70 con 0x7fb3700c4720 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb366a00640 1 -- 172.21.15.97:0/990390213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358004260 con 0x7fb3700c4720 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3701f88e0 con 0x7fb3700c4720 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3700b5f20 con 0x7fb3700c4720 2024-09-10T21:32:55.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3700cf800 con 0x7fb3700c4720 2024-09-10T21:32:56.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:55.999+0000 7fb366a00640 1 -- 172.21.15.97:0/990390213 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb358003600 con 0x7fb3700c4720 2024-09-10T21:32:56.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.003+0000 7fb366a00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb34c0725e0 0x7fb34c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:56.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.003+0000 7fb366a00640 1 -- 172.21.15.97:0/990390213 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7fb358094540 con 0x7fb3700c4720 2024-09-10T21:32:56.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.003+0000 7fb36de00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb34c0725e0 0x7fb34c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:56.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.003+0000 7fb36de00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb34c0725e0 0x7fb34c074aa0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb3700c5170 tx=0x7fb360001640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:56.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.007+0000 7fb366a00640 1 -- 172.21.15.97:0/990390213 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb35805ed80 con 0x7fb3700c4720 2024-09-10T21:32:56.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.203+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb370071f40 con 0x7fb3700c4720 2024-09-10T21:32:56.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.203+0000 7fb366a00640 1 -- 172.21.15.97:0/990390213 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) ==== 95+0+4404 (secure 0 0 0) 0x7fb3580637b0 con 0x7fb3700c4720 2024-09-10T21:32:56.205 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:56.205 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":195,"btime":"2024-09-10T21:32:55:830688+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:55.830637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8089":{"gid":8089,"name":"c","rank":0,"incarnation":195,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/3111622295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3111622295},{"type":"v1","addr":"172.21.15.97:6835","nonce":3111622295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:56.205 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 195 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb34c0725e0 msgr2=0x7fb34c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb34c0725e0 0x7fb34c074aa0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb3700c5170 tx=0x7fb360001640 comp rx=0 tx=0).stop 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700c4720 msgr2=0x7fb3700c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700c4720 0x7fb3700c9ef0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb358011ae0 tx=0x7fb358009f20 comp rx=0 tx=0).stop 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 shutdown_connections 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb34c0725e0 0x7fb34c074aa0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3700c7470 0x7fb3700c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3700c4720 0x7fb3700c9ef0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 --2- 172.21.15.97:0/990390213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb354004d50 0x7fb37015d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 >> 172.21.15.97:0/990390213 conn(0x7fb3700b37c0 msgr2=0x7fb37015f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:56.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 shutdown_connections 2024-09-10T21:32:56.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.207+0000 7fb374e00640 1 -- 172.21.15.97:0/990390213 wait complete. 2024-09-10T21:32:56.391 INFO:tasks.ceph.mds.a.smithi097.stderr: -40> 2024-09-10T21:32:51.639+0000 7ff78a000640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T21:32:56.428 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:32:56.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 -- 172.21.15.97:0/225364808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8460155090 msgr2=0x7f84600b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 --2- 172.21.15.97:0/225364808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8460155090 0x7f84600b3be0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8460073500 tx=0x7f845001c770 comp rx=0 tx=0).stop 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 -- 172.21.15.97:0/225364808 shutdown_connections 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 --2- 172.21.15.97:0/225364808 >> v2:172.21.15.194:3300/0 conn(0x7f846015bbe0 0x7f846015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 --2- 172.21.15.97:0/225364808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8460155090 0x7f84600b3be0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 -- 172.21.15.97:0/225364808 >> 172.21.15.97:0/225364808 conn(0x7f84600b2ee0 msgr2=0x7f84600d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 -- 172.21.15.97:0/225364808 shutdown_connections 2024-09-10T21:32:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.771+0000 7f8466000640 1 -- 172.21.15.97:0/225364808 wait complete. 2024-09-10T21:32:56.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 Processor -- start 2024-09-10T21:32:56.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 -- start start 2024-09-10T21:32:56.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8460155090 0x7f8460148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:56.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f846015bbe0 0x7f84601f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:56.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f846015c5d0 0x7f84601f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:56.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84600af9b0 con 0x7f846015bbe0 2024-09-10T21:32:56.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84600af830 con 0x7f8460155090 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f84600afb30 con 0x7f846015c5d0 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f846015bbe0 0x7f84601f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f846015bbe0 0x7f84601f8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33682/0 (socket says 172.21.15.97:33682) 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 -- 172.21.15.97:0/1823323478 learned_addr learned my addr 172.21.15.97:0/1823323478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8464c00640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8460155090 0x7f8460148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8465600640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f846015c5d0 0x7f84601f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 -- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f846015c5d0 msgr2=0x7f84601f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f846015c5d0 0x7f84601f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 -- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8460155090 msgr2=0x7f8460148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8460155090 0x7f8460148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 -- 172.21.15.97:0/1823323478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846014efc0 con 0x7f846015bbe0 2024-09-10T21:32:56.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8464c00640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8460155090 0x7f8460148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8465600640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f846015c5d0 0x7f84601f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8457e00640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f846015bbe0 0x7f84601f8cc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f844800d980 tx=0x7f844800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f845d600640 1 -- 172.21.15.97:0/1823323478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448017070 con 0x7f846015bbe0 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f845d600640 1 -- 172.21.15.97:0/1823323478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8448009d20 con 0x7f846015bbe0 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f845d600640 1 -- 172.21.15.97:0/1823323478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448004280 con 0x7f846015bbe0 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8460149030 con 0x7f846015bbe0 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846014e720 con 0x7f846015bbe0 2024-09-10T21:32:56.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.775+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84600b0050 con 0x7f846015bbe0 2024-09-10T21:32:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.779+0000 7f845d600640 1 -- 172.21.15.97:0/1823323478 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8448003600 con 0x7f846015bbe0 2024-09-10T21:32:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.779+0000 7f845d600640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f843c0725e0 0x7f843c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.779+0000 7f845d600640 1 -- 172.21.15.97:0/1823323478 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f8448093d10 con 0x7f846015bbe0 2024-09-10T21:32:56.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.779+0000 7f8464c00640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f843c0725e0 0x7f843c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:56.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.779+0000 7f8464c00640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f843c0725e0 0x7f843c074aa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f84500af000 tx=0x7f845001afb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:56.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.783+0000 7f845d600640 1 -- 172.21.15.97:0/1823323478 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8448099020 con 0x7f846015bbe0 2024-09-10T21:32:56.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84600b1620 con 0x7f846015bbe0 2024-09-10T21:32:56.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f845d600640 1 -- 172.21.15.97:0/1823323478 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) ==== 95+0+5233 (secure 0 0 0) 0x7f844805eb10 con 0x7f846015bbe0 2024-09-10T21:32:56.982 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:32:56.982 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":196,"btime":"2024-09-10T21:32:56:832813+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8841,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2402514654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2402514654},{"type":"v1","addr":"172.21.15.97:6834","nonce":2402514654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:55.842918+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8089":{"gid":8089,"name":"c","rank":0,"incarnation":195,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.97:6835/3111622295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3111622295},{"type":"v1","addr":"172.21.15.97:6835","nonce":3111622295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:32:56.983 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 196 2024-09-10T21:32:56.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f843c0725e0 msgr2=0x7f843c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:56.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f843c0725e0 0x7f843c074aa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f84500af000 tx=0x7f845001afb0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f846015bbe0 msgr2=0x7f84601f8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:56.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f846015bbe0 0x7f84601f8cc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f844800d980 tx=0x7f844800de40 comp rx=0 tx=0).stop 2024-09-10T21:32:56.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 shutdown_connections 2024-09-10T21:32:56.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f843c0725e0 0x7f843c074aa0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f846015c5d0 0x7f84601f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f846015bbe0 0x7f84601f8cc0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 --2- 172.21.15.97:0/1823323478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8460155090 0x7f8460148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 >> 172.21.15.97:0/1823323478 conn(0x7f84600b2ee0 msgr2=0x7f84600cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 shutdown_connections 2024-09-10T21:32:56.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:32:56.983+0000 7f8466000640 1 -- 172.21.15.97:0/1823323478 wait complete. 2024-09-10T21:32:57.004 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 196, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:32:55.842918+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8089}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8089': {'gid': 8089, 'name': 'c', 'rank': 0, 'incarnation': 195, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.97:6835/3111622295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 3111622295}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3111622295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:32:57.004 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8089:up:reconnect 2024-09-10T21:32:57.004 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:33:02.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:02.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- 172.21.15.97:0/340296972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc0cf5f0 msgr2=0x7fc0fc06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 --2- 172.21.15.97:0/340296972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc0cf5f0 0x7fc0fc06f090 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400adc0 tx=0x7fc0f4020470 comp rx=0 tx=0).stop 2024-09-10T21:33:02.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- 172.21.15.97:0/340296972 shutdown_connections 2024-09-10T21:33:02.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 --2- 172.21.15.97:0/340296972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc0cf5f0 0x7fc0fc06f090 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- 172.21.15.97:0/340296972 >> 172.21.15.97:0/340296972 conn(0x7fc0fc0b1780 msgr2=0x7fc0fc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:02.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- 172.21.15.97:0/340296972 shutdown_connections 2024-09-10T21:33:02.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- 172.21.15.97:0/340296972 wait complete. 2024-09-10T21:33:02.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 Processor -- start 2024-09-10T21:33:02.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- start start 2024-09-10T21:33:02.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0fc06ec40 0x7fc0fc13c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0fc0cf5f0 0x7fc0fc15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc1511f0 0x7fc0fc15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc06e850 con 0x7fc0fc1511f0 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc06e6d0 con 0x7fc0fc0cf5f0 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc102e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc0fc06e9d0 con 0x7fc0fc06ec40 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc1511f0 0x7fc0fc15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc1511f0 0x7fc0fc15cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58932/0 (socket says 172.21.15.97:58932) 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 -- 172.21.15.97:0/793723488 learned_addr learned my addr 172.21.15.97:0/793723488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc0f3400640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0fc0cf5f0 0x7fc0fc15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc0fbe00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0fc06ec40 0x7fc0fc13c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 -- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0fc06ec40 msgr2=0x7fc0fc13c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0fc06ec40 0x7fc0fc13c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 -- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0fc0cf5f0 msgr2=0x7fc0fc15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0fc0cf5f0 0x7fc0fc15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 -- 172.21.15.97:0/793723488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc13f960 con 0x7fc0fc1511f0 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.159+0000 7fc100e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc1511f0 0x7fc0fc15cae0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ace0 tx=0x7fc0f400a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc0f8c00640 1 -- 172.21.15.97:0/793723488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40c0070 con 0x7fc0fc1511f0 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc0f8c00640 1 -- 172.21.15.97:0/793723488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0f40b6040 con 0x7fc0fc1511f0 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc0f8c00640 1 -- 172.21.15.97:0/793723488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40b5070 con 0x7fc0fc1511f0 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc13ca80 con 0x7fc0fc1511f0 2024-09-10T21:33:02.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc13d040 con 0x7fc0fc1511f0 2024-09-10T21:33:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc06f090 con 0x7fc0fc1511f0 2024-09-10T21:33:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc0f8c00640 1 -- 172.21.15.97:0/793723488 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc0f40b0580 con 0x7fc0fc1511f0 2024-09-10T21:33:02.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc0f8c00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0d80725e0 0x7fc0d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.163+0000 7fc0f8c00640 1 -- 172.21.15.97:0/793723488 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7fc0f41312d0 con 0x7fc0fc1511f0 2024-09-10T21:33:02.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.167+0000 7fc0fbe00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0d80725e0 0x7fc0d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.167+0000 7fc0fbe00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0d80725e0 0x7fc0d8074aa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec001650 tx=0x7fc0ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:02.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.171+0000 7fc0f8c00640 1 -- 172.21.15.97:0/793723488 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc0f4019fd0 con 0x7fc0fc1511f0 2024-09-10T21:33:02.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.367+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0fc15c100 con 0x7fc0fc1511f0 2024-09-10T21:33:02.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.367+0000 7fc0f8c00640 1 -- 172.21.15.97:0/793723488 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+5237 (secure 0 0 0) 0x7fc0f40fc150 con 0x7fc0fc1511f0 2024-09-10T21:33:02.368 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:02.368 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":198,"btime":"2024-09-10T21:32:58:849452+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8841,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2402514654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2402514654},{"type":"v1","addr":"172.21.15.97:6834","nonce":2402514654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:58.849445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8089":{"gid":8089,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/3111622295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3111622295},{"type":"v1","addr":"172.21.15.97:6835","nonce":3111622295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8089,"qdb_cluster":[8089]},"id":6}]} 2024-09-10T21:33:02.368 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 198 2024-09-10T21:33:02.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0d80725e0 msgr2=0x7fc0d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0d80725e0 0x7fc0d8074aa0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec001650 tx=0x7fc0ec009040 comp rx=0 tx=0).stop 2024-09-10T21:33:02.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc1511f0 msgr2=0x7fc0fc15cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc1511f0 0x7fc0fc15cae0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ace0 tx=0x7fc0f400a4f0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 shutdown_connections 2024-09-10T21:33:02.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc0d80725e0 0x7fc0d8074aa0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0fc1511f0 0x7fc0fc15cae0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0fc0cf5f0 0x7fc0fc15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 --2- 172.21.15.97:0/793723488 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc0fc06ec40 0x7fc0fc13c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 >> 172.21.15.97:0/793723488 conn(0x7fc0fc0b1780 msgr2=0x7fc0fc0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:02.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 shutdown_connections 2024-09-10T21:33:02.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.371+0000 7fc102e00640 1 -- 172.21.15.97:0/793723488 wait complete. 2024-09-10T21:33:02.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 -- 172.21.15.97:0/3198945981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567806d180 msgr2=0x7f5678161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 --2- 172.21.15.97:0/3198945981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567806d180 0x7f5678161820 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5678070c80 tx=0x7f566001c830 comp rx=0 tx=0).stop 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 -- 172.21.15.97:0/3198945981 shutdown_connections 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 --2- 172.21.15.97:0/3198945981 >> v2:172.21.15.194:3300/0 conn(0x7f567815e9d0 0x7f56780b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 --2- 172.21.15.97:0/3198945981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567806d180 0x7f5678161820 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 --2- 172.21.15.97:0/3198945981 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f567806cda0 0x7f5678070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 -- 172.21.15.97:0/3198945981 >> 172.21.15.97:0/3198945981 conn(0x7f56780b2790 msgr2=0x7f56780b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 -- 172.21.15.97:0/3198945981 shutdown_connections 2024-09-10T21:33:02.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.939+0000 7f567ce00640 1 -- 172.21.15.97:0/3198945981 wait complete. 2024-09-10T21:33:02.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 Processor -- start 2024-09-10T21:33:02.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 -- start start 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f567806cda0 0x7f56781415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f567806d180 0x7f567814fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567815e9d0 0x7f5678151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5678073ea0 con 0x7f567815e9d0 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5678073d20 con 0x7f567806d180 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5678074020 con 0x7f567806cda0 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567815e9d0 0x7f5678151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567815e9d0 0x7f5678151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58940/0 (socket says 172.21.15.97:58940) 2024-09-10T21:33:02.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 -- 172.21.15.97:0/791936967 learned_addr learned my addr 172.21.15.97:0/791936967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f566d400640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f567806d180 0x7f567814fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5675e00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f567806cda0 0x7f56781415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 -- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f567806cda0 msgr2=0x7f56781415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f567806cda0 0x7f56781415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 -- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f567806d180 msgr2=0x7f567814fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f567806d180 0x7f567814fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 -- 172.21.15.97:0/791936967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780af890 con 0x7f567815e9d0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f5676800640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567815e9d0 0x7f5678151740 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f567000d660 tx=0x7f567000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f566ea00640 1 -- 172.21.15.97:0/791936967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670017070 con 0x7f567815e9d0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f566ea00640 1 -- 172.21.15.97:0/791936967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5670005310 con 0x7f567815e9d0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f566ea00640 1 -- 172.21.15.97:0/791936967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670009820 con 0x7f567815e9d0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56780afb70 con 0x7f567815e9d0 2024-09-10T21:33:02.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56780adc30 con 0x7f567815e9d0 2024-09-10T21:33:02.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.943+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678161820 con 0x7f567815e9d0 2024-09-10T21:33:02.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.947+0000 7f566ea00640 1 -- 172.21.15.97:0/791936967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f56700099c0 con 0x7f567815e9d0 2024-09-10T21:33:02.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.947+0000 7f566ea00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56540725e0 0x7f5654074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:02.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.947+0000 7f566ea00640 1 -- 172.21.15.97:0/791936967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f5670093a40 con 0x7f567815e9d0 2024-09-10T21:33:02.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.947+0000 7f5675e00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56540725e0 0x7f5654074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:02.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.947+0000 7f5675e00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56540725e0 0x7f5654074aa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5668009f00 tx=0x7f5668009650 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:02.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:02.951+0000 7f566ea00640 1 -- 172.21.15.97:0/791936967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f567005e280 con 0x7f567815e9d0 2024-09-10T21:33:03.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56780b4600 con 0x7f567815e9d0 2024-09-10T21:33:03.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f566ea00640 1 -- 172.21.15.97:0/791936967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+5237 (secure 0 0 0) 0x7f5670062cb0 con 0x7f567815e9d0 2024-09-10T21:33:03.150 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:03.150 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":198,"btime":"2024-09-10T21:32:58:849452+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8841,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2402514654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2402514654},{"type":"v1","addr":"172.21.15.97:6834","nonce":2402514654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:58.849445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8089":{"gid":8089,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/3111622295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3111622295},{"type":"v1","addr":"172.21.15.97:6835","nonce":3111622295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8089,"qdb_cluster":[8089]},"id":6}]} 2024-09-10T21:33:03.150 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 198 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56540725e0 msgr2=0x7f5654074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56540725e0 0x7f5654074aa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5668009f00 tx=0x7f5668009650 comp rx=0 tx=0).stop 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567815e9d0 msgr2=0x7f5678151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567815e9d0 0x7f5678151740 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f567000d660 tx=0x7f567000db20 comp rx=0 tx=0).stop 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 shutdown_connections 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56540725e0 0x7f5654074aa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f567815e9d0 0x7f5678151740 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f567806d180 0x7f567814fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 --2- 172.21.15.97:0/791936967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f567806cda0 0x7f56781415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 >> 172.21.15.97:0/791936967 conn(0x7f56780b2790 msgr2=0x7f56780b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 shutdown_connections 2024-09-10T21:33:03.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.151+0000 7f567ce00640 1 -- 172.21.15.97:0/791936967 wait complete. 2024-09-10T21:33:03.171 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 198, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:32:58.849445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8089}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8089': {'gid': 8089, 'name': 'c', 'rank': 0, 'incarnation': 195, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6835/3111622295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 3111622295}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3111622295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8089, 'qdb_cluster': [8089]} 2024-09-10T21:33:03.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:33:03.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 -- 172.21.15.97:0/694191632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 msgr2=0x7f0e6815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/694191632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e6815fd20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0e680735a0 tx=0x7f0e6001f580 comp rx=0 tx=0).stop 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 -- 172.21.15.97:0/694191632 shutdown_connections 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/694191632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e6815fd20 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/694191632 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e680cec10 0x7f0e680cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/694191632 >> v2:172.21.15.194:3300/0 conn(0x7f0e680ce830 0x7f0e680cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 -- 172.21.15.97:0/694191632 >> 172.21.15.97:0/694191632 conn(0x7f0e680b37c0 msgr2=0x7f0e680b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 -- 172.21.15.97:0/694191632 shutdown_connections 2024-09-10T21:33:03.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 -- 172.21.15.97:0/694191632 wait complete. 2024-09-10T21:33:03.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 Processor -- start 2024-09-10T21:33:03.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.315+0000 7f0e6dc00640 1 -- start start 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e680ce830 0x7f0e680cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e680cec10 0x7f0e680c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e680c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680b4480 con 0x7f0e680d0280 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680cde60 con 0x7f0e680ce830 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0e680ce1a0 con 0x7f0e680cec10 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e680c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e680c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58952/0 (socket says 172.21.15.97:58952) 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 -- 172.21.15.97:0/2162195247 learned_addr learned my addr 172.21.15.97:0/2162195247 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e66000640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e680cec10 0x7f0e680c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e66a00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e680ce830 0x7f0e680cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 -- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e680cec10 msgr2=0x7f0e680c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e680cec10 0x7f0e680c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 -- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e680ce830 msgr2=0x7f0e680cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e680ce830 0x7f0e680cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 -- 172.21.15.97:0/2162195247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e680c6db0 con 0x7f0e680d0280 2024-09-10T21:33:03.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e66000640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e680cec10 0x7f0e680c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:03.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e67400640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e680c63d0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0e600b3040 tx=0x7f0e60009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e5ea00640 1 -- 172.21.15.97:0/2162195247 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600bf070 con 0x7f0e680d0280 2024-09-10T21:33:03.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e5ea00640 1 -- 172.21.15.97:0/2162195247 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e60004c00 con 0x7f0e680d0280 2024-09-10T21:33:03.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e5ea00640 1 -- 172.21.15.97:0/2162195247 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600b5040 con 0x7f0e680d0280 2024-09-10T21:33:03.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e680c8ea0 con 0x7f0e680d0280 2024-09-10T21:33:03.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.319+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e680c9460 con 0x7f0e680d0280 2024-09-10T21:33:03.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e5ea00640 1 -- 172.21.15.97:0/2162195247 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0e600b8070 con 0x7f0e680d0280 2024-09-10T21:33:03.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e5ea00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e480725e0 0x7f0e48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 Processor -- start 2024-09-10T21:33:03.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e5ea00640 1 -- 172.21.15.97:0/2162195247 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f0e60130f40 con 0x7f0e680d0280 2024-09-10T21:33:03.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 -- start start 2024-09-10T21:33:03.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e66a00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e480725e0 0x7f0e48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121329d0 0x55b812132db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121332f0 0x55b8120e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e66a00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e480725e0 0x7f0e48074aa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0e580058d0 tx=0x7f0e58005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8120e6e10 0x55b8120e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b8120073f0 con 0x55b8120e6e10 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b811f44b20 con 0x55b8121329d0 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b811f33f50 con 0x55b8121332f0 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8120e6e10 0x55b8120e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e66a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121329d0 0x55b812132db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8120e6e10 0x55b8120e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58954/0 (socket says 172.21.15.97:58954) 2024-09-10T21:33:03.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 -- 172.21.15.97:0/3192364984 learned_addr learned my addr 172.21.15.97:0/3192364984 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e66000640 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121332f0 0x55b8120e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 -- 172.21.15.97:0/3192364984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121332f0 msgr2=0x55b8120e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121332f0 0x55b8120e68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 -- 172.21.15.97:0/3192364984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121329d0 msgr2=0x55b812132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121329d0 0x55b812132db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 -- 172.21.15.97:0/3192364984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811efb590 con 0x55b8120e6e10 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e67400640 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8120e6e10 0x55b8120e71f0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f0e6001fe20 tx=0x7f0e600c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179f639ee40d82d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e53400640 1 -- 172.21.15.97:0/3192364984 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600fb640 con 0x55b8120e6e10 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e53400640 1 -- 172.21.15.97:0/3192364984 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e60100120 con 0x55b8120e6e10 2024-09-10T21:33:03.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.323+0000 7f0e53400640 1 -- 172.21.15.97:0/3192364984 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60104b40 con 0x55b8120e6e10 2024-09-10T21:33:03.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- 172.21.15.97:0/3192364984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8120e6e10 msgr2=0x55b8120e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8120e6e10 0x55b8120e71f0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f0e6001fe20 tx=0x7f0e600c2000 comp rx=0 tx=0).stop 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- 172.21.15.97:0/3192364984 shutdown_connections 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8120e6e10 0x55b8120e71f0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121332f0 0x55b8120e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3192364984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121329d0 0x55b812132db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- 172.21.15.97:0/3192364984 >> 172.21.15.97:0/3192364984 conn(0x55b811fd0130 msgr2=0x55b812115e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- 172.21.15.97:0/3192364984 shutdown_connections 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- 172.21.15.97:0/3192364984 wait complete. 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- start start 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 Processor -- start 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b8121136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812113e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8121143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b811ece5d0 con 0x55b8121329d0 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b811f37d60 con 0x55b8121332f0 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b811eda500 con 0x55b8120e6e10 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 10 client.0 _ll_register_callbacks cb 0x55b811ede810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58964/0 (socket says 172.21.15.97:58964) 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 -- 172.21.15.97:0/2793158283 learned_addr learned my addr 172.21.15.97:0/2793158283 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e67400640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8121143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66a00640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b8121136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 -- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 msgr2=0x55b8121136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b8121136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 -- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 msgr2=0x55b8121143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8121143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 -- 172.21.15.97:0/2793158283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b811ed2170 con 0x55b8121329d0 2024-09-10T21:33:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e67400640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8121143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e66000640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812113e70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009e40 tx=0x7f0e540117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc2dfe27ee65f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e47e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e47e00640 1 -- 172.21.15.97:0/2793158283 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54012860 con 0x55b8121329d0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b811ecc0f0 con 0x55b8121329d0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e47e00640 1 -- 172.21.15.97:0/2793158283 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f0e54012a00 con 0x55b8121329d0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e47e00640 1 -- 172.21.15.97:0/2793158283 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) ==== 24+0+0 (secure 0 0 0) 0x7f0e54012c00 con 0x55b8121329d0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 10 client.8877 fetch_fsmap learned FSMap version 198 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=2+}) -- 0x55b811fb8ef0 con 0x55b8121329d0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.327+0000 7f0e47e00640 1 -- 172.21.15.97:0/2793158283 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b811ecc0f0 con 0x55b8121329d0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e47e00640 1 -- 172.21.15.97:0/2793158283 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 198) ==== 3502+0+0 (secure 0 0 0) 0x7f0e5400f040 con 0x55b8121329d0 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 10 client.8877 fetch_fsmap finished waiting for FSMap version 198 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 10 client.8877 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 20 client.8877 populate_metadata read hostname 'smithi097' 2024-09-10T21:33:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b8121150f0 0x55b812144c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 4 client.8877 mds_command: new command op to 8089 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:33:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 --> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b811f46a70 con 0x55b8121150f0 2024-09-10T21:33:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e66a00640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b8121150f0 0x55b812144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e66a00640 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b8121150f0 0x55b812144c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ceb5b220bac8635 server_cookie=5281a510b0f2837f in_seq=0 out_seq=0 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e47e00640 10 client.8877 ms_handle_connect on v2:172.21.15.97:6833/3111622295 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e47e00640 1 -- 172.21.15.97:0/2793158283 <== mds.0 v2:172.21.15.97:6833/3111622295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b811f46a70 con 0x55b8121150f0 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e47e00640 10 client.8877 handle_command_reply: tid=0 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 client.8877 shutdown 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 msgr2=0x55b812113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812113e70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009e40 tx=0x7f0e540117b0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 shutdown_connections 2024-09-10T21:33:03.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b8121150f0 0x55b812144c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8121143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812113e70 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e47e00640 0 client.8877 ms_handle_reset on v2:172.21.15.97:6833/3111622295 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 --2- 172.21.15.97:0/2793158283 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b8121136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 >> 172.21.15.97:0/2793158283 conn(0x55b811fd0130 msgr2=0x55b8120e5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 shutdown_connections 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 1 -- 172.21.15.97:0/2793158283 wait complete. 2024-09-10T21:33:03.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.331+0000 7f0e710d7000 20 client.8877 trim_cache size 0 max 16384 2024-09-10T21:33:03.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 Processor -- start 2024-09-10T21:33:03.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- start start 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b812131cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812132200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8120e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e682026b0 con 0x55b8121329d0 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0e68201690 con 0x55b8121332f0 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0e681f8610 con 0x55b8120e6e10 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812132200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812132200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58980/0 (socket says 172.21.15.97:58980) 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 -- 172.21.15.97:0/120865225 learned_addr learned my addr 172.21.15.97:0/120865225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66a00640 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b812131cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e67400640 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8120e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 -- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 msgr2=0x55b812131cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b812131cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 -- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 msgr2=0x55b8120e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8120e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 -- 172.21.15.97:0/120865225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b811f8fd90 con 0x55b8121329d0 2024-09-10T21:33:03.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e66000640 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812132200 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0e54000c10 tx=0x7f0e54012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515178fb3276c390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e52a00640 1 -- 172.21.15.97:0/120865225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54012660 con 0x55b8121329d0 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e52a00640 1 -- 172.21.15.97:0/120865225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e540128f0 con 0x55b8121329d0 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e52a00640 1 -- 172.21.15.97:0/120865225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e540286c0 con 0x55b8121329d0 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- 172.21.15.97:0/120865225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 msgr2=0x55b812132200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812132200 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0e54000c10 tx=0x7f0e54012ec0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- 172.21.15.97:0/120865225 shutdown_connections 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8121332f0 0x55b8120e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121329d0 0x55b812132200 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 --2- 172.21.15.97:0/120865225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8120e6e10 0x55b812131cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- 172.21.15.97:0/120865225 >> 172.21.15.97:0/120865225 conn(0x55b811fd0130 msgr2=0x55b8121315b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- 172.21.15.97:0/120865225 shutdown_connections 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- 172.21.15.97:0/120865225 wait complete. 2024-09-10T21:33:03.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.351+0000 7f0e710d7000 1 -- start start 2024-09-10T21:33:03.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 Processor -- start 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8120e6e10 0x55b8121b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121329d0 0x55b8121b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121332f0 0x55b8121b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b812149ce0 con 0x55b8121332f0 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b811f3c390 con 0x55b8120e6e10 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b811fb8a60 con 0x55b8121329d0 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 10 client.0 _ll_register_callbacks cb 0x7f0e54009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121332f0 0x55b8121b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121332f0 0x55b8121b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58982/0 (socket says 172.21.15.97:58982) 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 -- 172.21.15.97:0/3269962328 learned_addr learned my addr 172.21.15.97:0/3269962328 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:03.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e66000640 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121329d0 0x55b8121b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e66a00640 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8120e6e10 0x55b8121b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 -- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121329d0 msgr2=0x55b8121b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121329d0 0x55b8121b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 -- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8120e6e10 msgr2=0x55b8121b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8120e6e10 0x55b8121b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 -- 172.21.15.97:0/3269962328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b811efaa90 con 0x55b8121332f0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e67400640 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121332f0 0x55b8121b1e90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f0e600fb5d0 tx=0x7f0e600b0620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc44c9381edde8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 1 -- 172.21.15.97:0/3269962328 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600fbd40 con 0x55b8121332f0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e540128f0 con 0x55b8121332f0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 1 -- 172.21.15.97:0/3269962328 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f0e6011cb00 con 0x55b8121332f0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 1 -- 172.21.15.97:0/3269962328 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=198) ==== 24+0+0 (secure 0 0 0) 0x7f0e60104b40 con 0x55b8121332f0 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 10 client.8883 fetch_fsmap learned FSMap version 198 2024-09-10T21:33:03.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=198,monmap=2+}) -- 0x55b811ed9050 con 0x55b8121332f0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 1 -- 172.21.15.97:0/3269962328 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e60113030 con 0x55b8121332f0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 1 -- 172.21.15.97:0/3269962328 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 198) ==== 3502+0+0 (secure 0 0 0) 0x7f0e60115050 con 0x55b8121332f0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 10 client.8883 fetch_fsmap finished waiting for FSMap version 198 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 10 client.8883 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 20 client.8883 populate_metadata read hostname 'smithi097' 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b812146fb0 0x55b812147390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 4 client.8883 mds_command: new command op to 8089 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 --> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] -- command(tid 0: {"prefix": "status"}) -- 0x55b811f42520 con 0x55b812146fb0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e66a00640 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b812146fb0 0x55b812147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e66a00640 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b812146fb0 0x55b812147390 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3126af830139f27 server_cookie=c155787b1f5f1b5b in_seq=0 out_seq=0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 10 client.8883 ms_handle_connect on v2:172.21.15.97:6833/3111622295 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 1 -- 172.21.15.97:0/3269962328 <== mds.0 v2:172.21.15.97:6833/3111622295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55b811f42520 con 0x55b812146fb0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e47e00640 10 client.8883 handle_command_reply: tid=0 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 client.8883 shutdown 2024-09-10T21:33:03.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121332f0 msgr2=0x55b8121b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121332f0 0x55b8121b1e90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f0e600fb5d0 tx=0x7f0e600b0620 comp rx=0 tx=0).stop 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 shutdown_connections 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.355+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:6833/3111622295,v1:172.21.15.97:6835/3111622295] conn(0x55b812146fb0 0x55b812147390 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8121332f0 0x55b8121b1e90 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b8121329d0 0x55b8121b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e710d7000 1 --2- 172.21.15.97:0/3269962328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8120e6e10 0x55b8121b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e47e00640 0 client.8883 ms_handle_reset on v2:172.21.15.97:6833/3111622295 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 >> 172.21.15.97:0/3269962328 conn(0x55b811fd0130 msgr2=0x55b8120e50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 shutdown_connections 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e710d7000 1 -- 172.21.15.97:0/3269962328 wait complete. 2024-09-10T21:33:03.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e710d7000 20 client.8883 trim_cache size 0 max 16384 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 8089, 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 7.5258622490000002, 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 198, 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 114, 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 114, 2024-09-10T21:33:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.779738397999999 2024-09-10T21:33:03.362 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:33:03.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e480725e0 msgr2=0x7f0e48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e480725e0 0x7f0e48074aa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0e580058d0 tx=0x7f0e58005860 comp rx=0 tx=0).stop 2024-09-10T21:33:03.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 msgr2=0x7f0e680c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e680c63d0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0e600b3040 tx=0x7f0e60009350 comp rx=0 tx=0).stop 2024-09-10T21:33:03.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 shutdown_connections 2024-09-10T21:33:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0e480725e0 0x7f0e48074aa0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e680d0280 0x7f0e680c63d0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0e680cec10 0x7f0e680c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 --2- 172.21.15.97:0/2162195247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e680ce830 0x7f0e680cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 >> 172.21.15.97:0/2162195247 conn(0x7f0e680b37c0 msgr2=0x7f0e680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 shutdown_connections 2024-09-10T21:33:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.359+0000 7f0e6dc00640 1 -- 172.21.15.97:0/2162195247 wait complete. 2024-09-10T21:33:03.375 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:33:03.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 -- 172.21.15.97:0/380521105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb4c003a10 msgr2=0x7ffb4c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 --2- 172.21.15.97:0/380521105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb4c003a10 0x7ffb4c003e30 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c2e0 tx=0x7ffb5801cec0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb66800640 1 -- 172.21.15.97:0/380521105 reap_dead start 2024-09-10T21:33:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 -- 172.21.15.97:0/380521105 shutdown_connections 2024-09-10T21:33:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 -- 172.21.15.97:0/380521105 >> 172.21.15.97:0/380521105 conn(0x7ffb680b1780 msgr2=0x7ffb680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 -- 172.21.15.97:0/380521105 shutdown_connections 2024-09-10T21:33:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 -- 172.21.15.97:0/380521105 wait complete. 2024-09-10T21:33:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 Processor -- start 2024-09-10T21:33:03.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.519+0000 7ffb6ce00640 1 -- start start 2024-09-10T21:33:03.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4c003a10 0x7ffb68156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb6813e5a0 0x7ffb68143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68159d90 0x7ffb6815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681669d0 con 0x7ffb68159d90 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68166850 con 0x7ffb6813e5a0 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffb68166b50 con 0x7ffb4c003a10 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68159d90 0x7ffb6815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68159d90 0x7ffb6815a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58996/0 (socket says 172.21.15.97:58996) 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 -- 172.21.15.97:0/3069043291 learned_addr learned my addr 172.21.15.97:0/3069043291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb65e00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4c003a10 0x7ffb68156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb5d400640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb6813e5a0 0x7ffb68143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 -- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4c003a10 msgr2=0x7ffb68156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4c003a10 0x7ffb68156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 -- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb6813e5a0 msgr2=0x7ffb68143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb6813e5a0 0x7ffb68143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 -- 172.21.15.97:0/3069043291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb68143560 con 0x7ffb68159d90 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb66800640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68159d90 0x7ffb6815a210 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ffb6815fff0 tx=0x7ffb60010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb5ea00640 1 -- 172.21.15.97:0/3069043291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60004d30 con 0x7ffb68159d90 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb5ea00640 1 -- 172.21.15.97:0/3069043291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb6000ea30 con 0x7ffb68159d90 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6813ea20 con 0x7ffb68159d90 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb5ea00640 1 -- 172.21.15.97:0/3069043291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000ee60 con 0x7ffb68159d90 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6815d820 con 0x7ffb68159d90 2024-09-10T21:33:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.523+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6813f9b0 con 0x7ffb68159d90 2024-09-10T21:33:03.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.527+0000 7ffb5ea00640 1 -- 172.21.15.97:0/3069043291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffb6000f040 con 0x7ffb68159d90 2024-09-10T21:33:03.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.527+0000 7ffb5ea00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb440725e0 0x7ffb44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.527+0000 7ffb5ea00640 1 -- 172.21.15.97:0/3069043291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7ffb600942a0 con 0x7ffb68159d90 2024-09-10T21:33:03.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.527+0000 7ffb65e00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb440725e0 0x7ffb44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.527+0000 7ffb65e00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb440725e0 0x7ffb44074aa0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffb580ac000 tx=0x7ffb580b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.531+0000 7ffb5ea00640 1 -- 172.21.15.97:0/3069043291 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffb6005eaf0 con 0x7ffb68159d90 2024-09-10T21:33:03.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.727+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb680cede0 con 0x7ffb68159d90 2024-09-10T21:33:03.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb5ea00640 1 -- 172.21.15.97:0/3069043291 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+5237 (secure 0 0 0) 0x7ffb60063520 con 0x7ffb68159d90 2024-09-10T21:33:03.729 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:03.729 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":198,"btime":"2024-09-10T21:32:58:849452+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8841,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2402514654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2402514654},{"type":"v1","addr":"172.21.15.97:6834","nonce":2402514654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:32:58.849445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8089},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8089":{"gid":8089,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/3111622295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3111622295},{"type":"v1","addr":"172.21.15.97:6835","nonce":3111622295}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8089,"qdb_cluster":[8089]},"id":6}]} 2024-09-10T21:33:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 198 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb440725e0 msgr2=0x7ffb44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb440725e0 0x7ffb44074aa0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffb580ac000 tx=0x7ffb580b0040 comp rx=0 tx=0).stop 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68159d90 msgr2=0x7ffb6815a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68159d90 0x7ffb6815a210 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ffb6815fff0 tx=0x7ffb60010da0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 shutdown_connections 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb440725e0 0x7ffb44074aa0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68159d90 0x7ffb6815a210 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb6813e5a0 0x7ffb68143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 --2- 172.21.15.97:0/3069043291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffb4c003a10 0x7ffb68156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 >> 172.21.15.97:0/3069043291 conn(0x7ffb680b1780 msgr2=0x7ffb6815ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 shutdown_connections 2024-09-10T21:33:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.731+0000 7ffb6ce00640 1 -- 172.21.15.97:0/3069043291 wait complete. 2024-09-10T21:33:03.751 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2024-09-10T21:33:03.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:33:03.813 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:33:03.815+0000 7f0b3a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:33:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 -- 172.21.15.97:0/2149912166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240ce640 msgr2=0x7f3a240cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 --2- 172.21.15.97:0/2149912166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240ce640 0x7f3a240cea20 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009840 tx=0x7f3a1801c820 comp rx=0 tx=0).stop 2024-09-10T21:33:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 -- 172.21.15.97:0/2149912166 shutdown_connections 2024-09-10T21:33:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 --2- 172.21.15.97:0/2149912166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240ce640 0x7f3a240cea20 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 -- 172.21.15.97:0/2149912166 >> 172.21.15.97:0/2149912166 conn(0x7f3a240d1c30 msgr2=0x7f3a240d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 -- 172.21.15.97:0/2149912166 shutdown_connections 2024-09-10T21:33:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 -- 172.21.15.97:0/2149912166 wait complete. 2024-09-10T21:33:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 Processor -- start 2024-09-10T21:33:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.895+0000 7f3a2be00640 1 -- start start 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a240b0d50 0x7f3a24157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240cd6e0 0x7f3a240b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3a240ce640 0x7f3a240b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2416a8a0 con 0x7f3a240cd6e0 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2416a720 con 0x7f3a240b0d50 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2416aa20 con 0x7f3a240ce640 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240cd6e0 0x7f3a240b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240cd6e0 0x7f3a240b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59004/0 (socket says 172.21.15.97:59004) 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 -- 172.21.15.97:0/3419254618 learned_addr learned my addr 172.21.15.97:0/3419254618 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a29400640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a240b0d50 0x7f3a24157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a29e00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3a240ce640 0x7f3a240b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 -- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3a240ce640 msgr2=0x7f3a240b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3a240ce640 0x7f3a240b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 -- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a240b0d50 msgr2=0x7f3a24157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a240b0d50 0x7f3a24157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 -- 172.21.15.97:0/3419254618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a241552b0 con 0x7f3a240cd6e0 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a29400640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a240b0d50 0x7f3a24157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a29e00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3a240ce640 0x7f3a240b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a28a00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240cd6e0 0x7f3a240b52c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c530 tx=0x7f3a1000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a21600640 1 -- 172.21.15.97:0/3419254618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10017070 con 0x7f3a240cd6e0 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a21600640 1 -- 172.21.15.97:0/3419254618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a10003600 con 0x7f3a240cd6e0 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a21600640 1 -- 172.21.15.97:0/3419254618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10003a50 con 0x7f3a240cd6e0 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a240b4b00 con 0x7f3a240cd6e0 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a240b4da0 con 0x7f3a240cd6e0 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.899+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a240cea20 con 0x7f3a240cd6e0 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.903+0000 7f3a21600640 1 -- 172.21.15.97:0/3419254618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3a10003bf0 con 0x7f3a240cd6e0 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.903+0000 7f3a21600640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a040725e0 0x7f3a04074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.903+0000 7f3a21600640 1 -- 172.21.15.97:0/3419254618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f3a10093210 con 0x7f3a240cd6e0 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.903+0000 7f3a29400640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a040725e0 0x7f3a04074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.903+0000 7f3a29400640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a040725e0 0x7f3a04074aa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3a180af000 tx=0x7f3a1801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:03.907+0000 7f3a21600640 1 -- 172.21.15.97:0/3419254618 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3a1005e160 con 0x7f3a240cd6e0 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.103+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3a240d04b0 con 0x7f3a240cd6e0 2024-09-10T21:33:04.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.103+0000 7f3a21600640 1 -- 172.21.15.97:0/3419254618 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v198) ==== 53+0+73 (secure 0 0 0) 0x7f3a1005da60 con 0x7f3a240cd6e0 2024-09-10T21:33:04.104 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a040725e0 msgr2=0x7f3a04074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a040725e0 0x7f3a04074aa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3a180af000 tx=0x7f3a1801b060 comp rx=0 tx=0).stop 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240cd6e0 msgr2=0x7f3a240b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240cd6e0 0x7f3a240b52c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c530 tx=0x7f3a1000c9f0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 shutdown_connections 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a040725e0 0x7f3a04074aa0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3a240ce640 0x7f3a240b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a240cd6e0 0x7f3a240b52c0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 --2- 172.21.15.97:0/3419254618 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a240b0d50 0x7f3a24157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 >> 172.21.15.97:0/3419254618 conn(0x7f3a240d1c30 msgr2=0x7f3a24157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 shutdown_connections 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04.107+0000 7f3a2be00640 1 -- 172.21.15.97:0/3419254618 wait complete. 2024-09-10T21:33:04.124 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:33:04.125 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:33:04.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.126 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41909sec preferred_lft 41909sec 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:33:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.139 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:33:04.197 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:33:04.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.200 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:33:04.251 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:33:04.251 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:33:04.252 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:33:04.253 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:33:04.253 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:33:04.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:33:04.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:33:04.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:33:04.253 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:33:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:33:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:33:04.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.385 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:33:04.395 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:33:04.395 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:33:04.395 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:33:04.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:33:04.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:33:04.456 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:33:04.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:33:04.513 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:04.563 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:33:04.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:04.652 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:04.652 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:33:04.653 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:33:04.667 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:04.668 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:33:04.668 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:33:04.668 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:33:04.668 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:33:04.677 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:33:04.680 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:33:04.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:04.701 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:33:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:33:04.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:33:04.801 INFO:teuthology.orchestra.run.smithi097.stdout:client8901 2024-09-10T21:33:04.801 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:04.801 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:04.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.811 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:33:04.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:04.855 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2024-09-10T21:33:04.862 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:33:04.862 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:33:04.862 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:04.862 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:33:04.862 INFO:teuthology.orchestra.run.smithi097.stdout:sixmegs 2024-09-10T21:33:04.870 INFO:teuthology.orchestra.run.smithi097.stderr:mkdir: cannot create directory 'foo': Read-only file system 2024-09-10T21:33:04.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:04.871 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory 'foo': Read-only file system 2024-09-10T21:33:04.871 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2024-09-10T21:33:04.871 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:04.871 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:04.921 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:04.921 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:04.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:33:05.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- 172.21.15.97:0/2727825869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0ce830 msgr2=0x7f49dc0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 --2- 172.21.15.97:0/2727825869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0ce830 0x7f49dc0cf800 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009900 tx=0x7f49d001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- 172.21.15.97:0/2727825869 shutdown_connections 2024-09-10T21:33:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 --2- 172.21.15.97:0/2727825869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49dc0cec10 0x7f49dc0cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 --2- 172.21.15.97:0/2727825869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0ce830 0x7f49dc0cf800 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- 172.21.15.97:0/2727825869 >> 172.21.15.97:0/2727825869 conn(0x7f49dc0b37c0 msgr2=0x7f49dc0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- 172.21.15.97:0/2727825869 shutdown_connections 2024-09-10T21:33:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- 172.21.15.97:0/2727825869 wait complete. 2024-09-10T21:33:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 Processor -- start 2024-09-10T21:33:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- start start 2024-09-10T21:33:05.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49dc0ce830 0x7f49dc0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0cec10 0x7f49dc0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49dc0d0280 0x7f49dc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc0cde60 con 0x7f49dc0cec10 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc0b44a0 con 0x7f49dc0d0280 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e4600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f49dc0ce1a0 con 0x7f49dc0ce830 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0cec10 0x7f49dc0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0cec10 0x7f49dc0c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59036/0 (socket says 172.21.15.97:59036) 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e1200640 1 -- 172.21.15.97:0/2370949869 learned_addr learned my addr 172.21.15.97:0/2370949869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e1c00640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49dc0ce830 0x7f49dc0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.159+0000 7f49e2600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49dc0d0280 0x7f49dc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e1200640 1 -- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49dc0ce830 msgr2=0x7f49dc0b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e1200640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49dc0ce830 0x7f49dc0b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e1200640 1 -- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49dc0d0280 msgr2=0x7f49dc0b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e1200640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49dc0d0280 0x7f49dc0b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e1200640 1 -- 172.21.15.97:0/2370949869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc1f5500 con 0x7f49dc0cec10 2024-09-10T21:33:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e2600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49dc0d0280 0x7f49dc0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e1200640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0cec10 0x7f49dc0c4ab0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f49c800d980 tx=0x7f49c800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49d9600640 1 -- 172.21.15.97:0/2370949869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8017070 con 0x7f49dc0cec10 2024-09-10T21:33:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49d9600640 1 -- 172.21.15.97:0/2370949869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c8009d00 con 0x7f49dc0cec10 2024-09-10T21:33:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49d9600640 1 -- 172.21.15.97:0/2370949869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8005280 con 0x7f49dc0cec10 2024-09-10T21:33:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc0b5430 con 0x7f49dc0cec10 2024-09-10T21:33:05.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc0b6450 con 0x7f49dc0cec10 2024-09-10T21:33:05.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49dc0b2780 con 0x7f49dc0cec10 2024-09-10T21:33:05.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49d9600640 1 -- 172.21.15.97:0/2370949869 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f49c8005420 con 0x7f49dc0cec10 2024-09-10T21:33:05.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49d9600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49bc0725e0 0x7f49bc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.163+0000 7f49d9600640 1 -- 172.21.15.97:0/2370949869 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7f49c8093ba0 con 0x7f49dc0cec10 2024-09-10T21:33:05.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.167+0000 7f49e1c00640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49bc0725e0 0x7f49bc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.167+0000 7f49e1c00640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49bc0725e0 0x7f49bc074aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f49d00b2000 tx=0x7f49d00aa040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:05.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.167+0000 7f49d9600640 1 -- 172.21.15.97:0/2370949869 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f49c805e3e0 con 0x7f49dc0cec10 2024-09-10T21:33:05.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.343+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f49dc0b0a30 con 0x7f49dc0cec10 2024-09-10T21:33:05.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.343+0000 7f49d9600640 1 -- 172.21.15.97:0/2370949869 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 36 entries v114) ==== 81+0+2186 (secure 0 0 0) 0x7f49c8067150 con 0x7f49dc0cec10 2024-09-10T21:33:05.344 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:33:05.344 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:33:05.344 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:33:05.345 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:33:05.346 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:33:05.347 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:33:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:listed 36 entries 2024-09-10T21:33:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49bc0725e0 msgr2=0x7f49bc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49bc0725e0 0x7f49bc074aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f49d00b2000 tx=0x7f49d00aa040 comp rx=0 tx=0).stop 2024-09-10T21:33:05.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0cec10 msgr2=0x7f49dc0c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0cec10 0x7f49dc0c4ab0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f49c800d980 tx=0x7f49c800de40 comp rx=0 tx=0).stop 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 shutdown_connections 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f49bc0725e0 0x7f49bc074aa0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49dc0d0280 0x7f49dc0b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49dc0cec10 0x7f49dc0c4ab0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 --2- 172.21.15.97:0/2370949869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f49dc0ce830 0x7f49dc0b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 >> 172.21.15.97:0/2370949869 conn(0x7f49dc0b37c0 msgr2=0x7f49dc15f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 shutdown_connections 2024-09-10T21:33:05.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.347+0000 7f49e4600640 1 -- 172.21.15.97:0/2370949869 wait complete. 2024-09-10T21:33:05.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:33:05.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:05.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:33:05.389 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:05.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:05.390 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:05.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.631+0000 7fac74000640 1 -- 172.21.15.97:0/2701685457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac540039f0 msgr2=0x7fac54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.631+0000 7fac74000640 1 --2- 172.21.15.97:0/2701685457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac540039f0 0x7fac54003dd0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fac600017f0 tx=0x7fac6001c900 comp rx=0 tx=0).stop 2024-09-10T21:33:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.631+0000 7fac72000640 1 -- 172.21.15.97:0/2701685457 reap_dead start 2024-09-10T21:33:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.631+0000 7fac74000640 1 -- 172.21.15.97:0/2701685457 shutdown_connections 2024-09-10T21:33:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.631+0000 7fac74000640 1 -- 172.21.15.97:0/2701685457 >> 172.21.15.97:0/2701685457 conn(0x7fac6c0b37c0 msgr2=0x7fac6c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.631+0000 7fac74000640 1 -- 172.21.15.97:0/2701685457 shutdown_connections 2024-09-10T21:33:05.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.631+0000 7fac74000640 1 -- 172.21.15.97:0/2701685457 wait complete. 2024-09-10T21:33:05.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 Processor -- start 2024-09-10T21:33:05.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 -- start start 2024-09-10T21:33:05.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac540039f0 0x7fac6c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac6c153a90 0x7fac6c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6c1fafd0 0x7fac6c1fb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1698d0 con 0x7fac6c1fafd0 2024-09-10T21:33:05.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c169750 con 0x7fac6c153a90 2024-09-10T21:33:05.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c169a50 con 0x7fac540039f0 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6c1fafd0 0x7fac6c1fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6c1fafd0 0x7fac6c1fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59056/0 (socket says 172.21.15.97:59056) 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 -- 172.21.15.97:0/2191262249 learned_addr learned my addr 172.21.15.97:0/2191262249 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac70c00640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac6c153a90 0x7fac6c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac71600640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac540039f0 0x7fac6c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 -- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac540039f0 msgr2=0x7fac6c153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac540039f0 0x7fac6c153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 -- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac6c153a90 msgr2=0x7fac6c153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac6c153a90 0x7fac6c153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 -- 172.21.15.97:0/2191262249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c1fbdb0 con 0x7fac6c1fafd0 2024-09-10T21:33:05.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac72000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6c1fafd0 0x7fac6c1fb450 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fac64010ab0 tx=0x7fac64010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:05.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac69600640 1 -- 172.21.15.97:0/2191262249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64003980 con 0x7fac6c1fafd0 2024-09-10T21:33:05.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac69600640 1 -- 172.21.15.97:0/2191262249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac64015070 con 0x7fac6c1fafd0 2024-09-10T21:33:05.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac69600640 1 -- 172.21.15.97:0/2191262249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64004030 con 0x7fac6c1fafd0 2024-09-10T21:33:05.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c15b180 con 0x7fac6c1fafd0 2024-09-10T21:33:05.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c15b740 con 0x7fac6c1fafd0 2024-09-10T21:33:05.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.635+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c161db0 con 0x7fac6c1fafd0 2024-09-10T21:33:05.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.639+0000 7fac69600640 1 -- 172.21.15.97:0/2191262249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fac640041d0 con 0x7fac6c1fafd0 2024-09-10T21:33:05.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.639+0000 7fac69600640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac4c0725e0 0x7fac4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:05.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.639+0000 7fac69600640 1 -- 172.21.15.97:0/2191262249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7823+0+0 (secure 0 0 0) 0x7fac64094330 con 0x7fac6c1fafd0 2024-09-10T21:33:05.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.639+0000 7fac71600640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac4c0725e0 0x7fac4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:05.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.639+0000 7fac71600640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac4c0725e0 0x7fac4c074aa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fac600ac000 tx=0x7fac600b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:05.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.643+0000 7fac69600640 1 -- 172.21.15.97:0/2191262249 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fac6405f130 con 0x7fac6c1fafd0 2024-09-10T21:33:05.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.843+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fac6c06d7e0 con 0x7fac6c1fafd0 2024-09-10T21:33:05.841 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:05.843+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.843+0000 7fac69600640 1 -- 172.21.15.97:0/2191262249 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v199) ==== 161+0+0 (secure 0 0 0) 0x7fac6405ea30 con 0x7fac6c1fafd0 2024-09-10T21:33:05.843 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac4c0725e0 msgr2=0x7fac4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac4c0725e0 0x7fac4c074aa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fac600ac000 tx=0x7fac600b0040 comp rx=0 tx=0).stop 2024-09-10T21:33:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6c1fafd0 msgr2=0x7fac6c1fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6c1fafd0 0x7fac6c1fb450 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fac64010ab0 tx=0x7fac64010f70 comp rx=0 tx=0).stop 2024-09-10T21:33:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 shutdown_connections 2024-09-10T21:33:05.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fac4c0725e0 0x7fac4c074aa0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6c1fafd0 0x7fac6c1fb450 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac6c153a90 0x7fac6c153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 --2- 172.21.15.97:0/2191262249 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fac540039f0 0x7fac6c153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 >> 172.21.15.97:0/2191262249 conn(0x7fac6c0b37c0 msgr2=0x7fac6c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 shutdown_connections 2024-09-10T21:33:05.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:05.847+0000 7fac74000640 1 -- 172.21.15.97:0/2191262249 wait complete. 2024-09-10T21:33:05.849 INFO:tasks.ceph.mds.c.smithi097.stderr: -1888> 2024-09-10T21:32:55.835+0000 7f0b36c00640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-10T21:33:05.849 INFO:tasks.ceph.mds.c.smithi097.stderr: -1884> 2024-09-10T21:32:55.835+0000 7f0b36c00640 -1 mds.0.195 unhandled write error (22) Invalid argument, force readonly... 2024-09-10T21:33:05.852 INFO:tasks.ceph.mds.c.smithi097.stderr: -569> 2024-09-10T21:33:03.815+0000 7f0b3a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:33:05.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:05.892 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:33:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 -- 172.21.15.97:0/3090622106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415b570 msgr2=0x7fd8740cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 --2- 172.21.15.97:0/3090622106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415b570 0x7fd8740cd580 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009c80 tx=0x7fd86c01cd10 comp rx=0 tx=0).stop 2024-09-10T21:33:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 -- 172.21.15.97:0/3090622106 shutdown_connections 2024-09-10T21:33:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 --2- 172.21.15.97:0/3090622106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415b570 0x7fd8740cd580 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 --2- 172.21.15.97:0/3090622106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd87415e750 0x7fd87415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 -- 172.21.15.97:0/3090622106 >> 172.21.15.97:0/3090622106 conn(0x7fd8740b28c0 msgr2=0x7fd8740b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 -- 172.21.15.97:0/3090622106 shutdown_connections 2024-09-10T21:33:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 -- 172.21.15.97:0/3090622106 wait complete. 2024-09-10T21:33:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 Processor -- start 2024-09-10T21:33:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.015+0000 7fd87b000640 1 -- start start 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd87415b570 0x7fd8741f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415e750 0x7fd874157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd87415f150 0x7fd8741f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd874168440 con 0x7fd87415e750 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741682c0 con 0x7fd87415f150 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd8741685c0 con 0x7fd87415b570 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415e750 0x7fd874157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415e750 0x7fd874157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59090/0 (socket says 172.21.15.97:59090) 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 -- 172.21.15.97:0/1403574278 learned_addr learned my addr 172.21.15.97:0/1403574278 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd879000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd87415f150 0x7fd8741f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd873e00640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd87415b570 0x7fd8741f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 -- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd87415b570 msgr2=0x7fd8741f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd87415b570 0x7fd8741f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 -- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd87415f150 msgr2=0x7fd8741f9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd87415f150 0x7fd8741f9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 -- 172.21.15.97:0/1403574278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741f9800 con 0x7fd87415e750 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd879000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd87415f150 0x7fd8741f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd86b400640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415e750 0x7fd874157dc0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c9a0 tx=0x7fd85c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd870c00640 1 -- 172.21.15.97:0/1403574278 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c007c10 con 0x7fd87415e750 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd870c00640 1 -- 172.21.15.97:0/1403574278 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd85c00f450 con 0x7fd87415e750 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741f6da0 con 0x7fd87415e750 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd870c00640 1 -- 172.21.15.97:0/1403574278 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00f8a0 con 0x7fd87415e750 2024-09-10T21:33:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741f9a90 con 0x7fd87415e750 2024-09-10T21:33:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8740cd580 con 0x7fd87415e750 2024-09-10T21:33:06.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.019+0000 7fd870c00640 1 -- 172.21.15.97:0/1403574278 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd85c00fa40 con 0x7fd87415e750 2024-09-10T21:33:06.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.023+0000 7fd870c00640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd8500725e0 0x7fd850074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:06.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.023+0000 7fd870c00640 1 -- 172.21.15.97:0/1403574278 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7909+0+0 (secure 0 0 0) 0x7fd85c0943f0 con 0x7fd87415e750 2024-09-10T21:33:06.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.023+0000 7fd873e00640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd8500725e0 0x7fd850074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:06.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.023+0000 7fd873e00640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd8500725e0 0x7fd850074aa0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd86400b810 tx=0x7fd864008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:06.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.027+0000 7fd870c00640 1 -- 172.21.15.97:0/1403574278 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd85c014030 con 0x7fd87415e750 2024-09-10T21:33:06.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.251+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd8740ce7b0 con 0x7fd87415e750 2024-09-10T21:33:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.883+0000 7fd870c00640 1 -- 172.21.15.97:0/1403574278 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v200) ==== 107+0+0 (secure 0 0 0) 0x7fd85c05f190 con 0x7fd87415e750 2024-09-10T21:33:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:33:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd8500725e0 msgr2=0x7fd850074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd8500725e0 0x7fd850074aa0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd86400b810 tx=0x7fd864008040 comp rx=0 tx=0).stop 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415e750 msgr2=0x7fd874157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415e750 0x7fd874157dc0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c9a0 tx=0x7fd85c00ce60 comp rx=0 tx=0).stop 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 shutdown_connections 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd8500725e0 0x7fd850074aa0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd87415f150 0x7fd8741f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87415e750 0x7fd874157dc0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 --2- 172.21.15.97:0/1403574278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd87415b570 0x7fd8741f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 >> 172.21.15.97:0/1403574278 conn(0x7fd8740b28c0 msgr2=0x7fd874071320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 shutdown_connections 2024-09-10T21:33:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:06.887+0000 7fd87b000640 1 -- 172.21.15.97:0/1403574278 wait complete. 2024-09-10T21:33:06.909 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 Processor -- start 2024-09-10T21:33:07.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 -- start start 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54e86940 0x560d54e56da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f10b00 0x560d54e8a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54f112a0 0x560d54e886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d54da5a00 con 0x560d54f112a0 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560d54e03f40 con 0x560d54e86940 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2039bc6080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560d54f12e70 con 0x560d54f10b00 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54f112a0 0x560d54e886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54f112a0 0x560d54e886a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59108/0 (socket says 172.21.15.97:59108) 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 -- 172.21.15.97:0/430282821 learned_addr learned my addr 172.21.15.97:0/430282821 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2036e00640 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54e86940 0x560d54e56da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2036400640 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f10b00 0x560d54e8a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 -- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f10b00 msgr2=0x560d54e8a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f10b00 0x560d54e8a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 -- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54e86940 msgr2=0x560d54e56da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54e86940 0x560d54e56da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 -- 172.21.15.97:0/430282821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d54d7ec10 con 0x560d54f112a0 2024-09-10T21:33:07.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2036400640 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f10b00 0x560d54e8a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2036e00640 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54e86940 0x560d54e56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2037800640 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54f112a0 0x560d54e886a0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f203000c790 tx=0x7f203000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecd982b9c4ce1f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2035000640 1 -- 172.21.15.97:0/430282821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030013070 con 0x560d54f112a0 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2035000640 1 -- 172.21.15.97:0/430282821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f203000ab30 con 0x560d54f112a0 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.003+0000 7f2035000640 1 -- 172.21.15.97:0/430282821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203000f460 con 0x560d54f112a0 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- 172.21.15.97:0/430282821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54f112a0 msgr2=0x560d54e886a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54f112a0 0x560d54e886a0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f203000c790 tx=0x7f203000cc50 comp rx=0 tx=0).stop 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- 172.21.15.97:0/430282821 shutdown_connections 2024-09-10T21:33:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54f112a0 0x560d54e886a0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f10b00 0x560d54e8a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 --2- 172.21.15.97:0/430282821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54e86940 0x560d54e56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- 172.21.15.97:0/430282821 >> 172.21.15.97:0/430282821 conn(0x560d54e85520 msgr2=0x560d54f2d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- 172.21.15.97:0/430282821 shutdown_connections 2024-09-10T21:33:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- 172.21.15.97:0/430282821 wait complete. 2024-09-10T21:33:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 Processor -- start 2024-09-10T21:33:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- start start 2024-09-10T21:33:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54e86940 0x560d54f0ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54f10b00 0x560d54f0d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f112a0 0x560d54e88810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d54e87a20 con 0x560d54e86940 2024-09-10T21:33:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560d54e57590 con 0x560d54f10b00 2024-09-10T21:33:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560d54e87cc0 con 0x560d54f112a0 2024-09-10T21:33:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54e86940 0x560d54f0ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54e86940 0x560d54f0ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59112/0 (socket says 172.21.15.97:59112) 2024-09-10T21:33:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 learned_addr learned my addr 172.21.15.97:0/1758072279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2037800640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f112a0 0x560d54e88810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54f10b00 0x560d54f0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f112a0 msgr2=0x560d54e88810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560d54f112a0 0x560d54e88810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54f10b00 msgr2=0x560d54f0d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54f10b00 0x560d54f0d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d54f0db00 con 0x560d54e86940 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d54f10b00 0x560d54f0d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2036e00640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54e86940 0x560d54f0ea20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f2028096ac0 tx=0x7f20280ab620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f202f400640 1 -- 172.21.15.97:0/1758072279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280b0040 con 0x560d54e86940 2024-09-10T21:33:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f202f400640 1 -- 172.21.15.97:0/1758072279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20280abb30 con 0x560d54e86940 2024-09-10T21:33:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f202f400640 1 -- 172.21.15.97:0/1758072279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280b63f0 con 0x560d54e86940 2024-09-10T21:33:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560d54f0dda0 con 0x560d54e86940 2024-09-10T21:33:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.007+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d54f0e2e0 con 0x560d54e86940 2024-09-10T21:33:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f202f400640 1 -- 172.21.15.97:0/1758072279 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f20280b6590 con 0x560d54e86940 2024-09-10T21:33:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f202f400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2018070ff0 0x7f20180734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f202f400640 1 -- 172.21.15.97:0/1758072279 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7909+0+0 (secure 0 0 0) 0x7f202812d320 con 0x560d54e86940 2024-09-10T21:33:07.014 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:07.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2018070ff0 0x7f20180734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560d54e88d50 0x560d54e99810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54f0d690 con 0x560d54e88d50 2024-09-10T21:33:07.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f2037800640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560d54e88d50 0x560d54e99810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.011+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2018070ff0 0x7f20180734b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f20240014b0 tx=0x7f2024002670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.015+0000 7f2037800640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560d54e88d50 0x560d54e99810 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.015+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v115'567 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.015+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54e99d50 con 0x560d54e88d50 2024-09-10T21:33:07.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.019+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v115'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.019+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd37a0 con 0x560d54e88d50 2024-09-10T21:33:07.020 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:07.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.019+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v115'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.019+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd4b50 con 0x560d54e88d50 2024-09-10T21:33:07.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.023+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.023+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd4ef0 con 0x560d54e88d50 2024-09-10T21:33:07.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.023+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'571 uv571 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.023+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3b40 con 0x560d54e88d50 2024-09-10T21:33:07.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.023+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'572 uv572 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.023+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.027+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v115'573 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.027+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.027+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v115'574 uv574 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.027+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.028 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:07.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.031+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v115'575 uv575 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.031+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.031+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.031+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.031+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.035+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.035+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.035+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.035+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v115'579 uv554 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.035+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.039+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'580 uv580 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.039+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.039 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:07.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.039+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v115'581 uv581 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.039+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.043+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'582 uv582 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.043+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.043+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'583 uv583 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.043+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.043+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'584 uv584 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.043+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.047+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'585 uv585 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.047+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.047+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v115'586 uv560 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.047+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.051+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v115'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.051+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.050 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.051+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v115'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.051+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.055+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.055+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.055+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.055+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.055+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.055+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.059+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v115'592 uv566 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.059+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.059+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v115'593 uv593 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.059+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.061 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.063+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v115'594 uv594 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.063+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.063+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'595 uv595 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.063+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.063+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'596 uv596 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.063+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd3f30 con 0x560d54e88d50 2024-09-10T21:33:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.067+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'597 uv597 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54e88d50 2024-09-10T21:33:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.067+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560d54fd4440 0x560d54fd72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.067+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd77e0 con 0x560d54fd4440 2024-09-10T21:33:07.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.067+0000 7f2036e00640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560d54fd4440 0x560d54fd72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.067+0000 7f2036e00640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560d54fd4440 0x560d54fd72a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.067+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'421 uv400 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.067+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd7b80 con 0x560d54fd4440 2024-09-10T21:33:07.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.071+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'422 uv422 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.071+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd7f20 con 0x560d54fd4440 2024-09-10T21:33:07.071 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:07.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.071+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'423 uv423 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.071+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd8770 con 0x560d54fd4440 2024-09-10T21:33:07.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.075+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v115'424 uv424 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.075+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd8b10 con 0x560d54fd4440 2024-09-10T21:33:07.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.075+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'425 uv425 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.075+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd8eb0 con 0x560d54fd4440 2024-09-10T21:33:07.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.075+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v115'426 uv419 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.075+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.079+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v115'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.079+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.077 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:07.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.079+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v115'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.079+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.079+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.079+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.083+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.083+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.083+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v115'431 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.083+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.083+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v115'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.083+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.084 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:07.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.087+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v115'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.087+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fdd420 con 0x560d54fd4440 2024-09-10T21:33:07.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.087+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v115'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.087+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fdd420 con 0x560d54fd4440 2024-09-10T21:33:07.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.087+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.087+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.091+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v115'436 uv416 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.091+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.091+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v115'437 uv437 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.091+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.095+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v115'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.092 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:07.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.095+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fda260 con 0x560d54fd4440 2024-09-10T21:33:07.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.095+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.095+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fda260 con 0x560d54fd4440 2024-09-10T21:33:07.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.095+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.095+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fd9250 con 0x560d54fd4440 2024-09-10T21:33:07.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fd4440 2024-09-10T21:33:07.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560d54fd98d0 0x560d54fdbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fdc290 con 0x560d54fd98d0 2024-09-10T21:33:07.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560d54fd98d0 0x560d54fdbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560d54fd98d0 0x560d54fdbcc0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fdc740 con 0x560d54fd98d0 2024-09-10T21:33:07.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.099+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fdcbf0 con 0x560d54fd98d0 2024-09-10T21:33:07.100 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:07.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fde4a0 con 0x560d54fd98d0 2024-09-10T21:33:07.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fde840 con 0x560d54fd98d0 2024-09-10T21:33:07.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fdd490 con 0x560d54fd98d0 2024-09-10T21:33:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560d54fddde0 0x560d54fe0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.103+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe1130 con 0x560d54fddde0 2024-09-10T21:33:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.107+0000 7f2037800640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560d54fddde0 0x560d54fe0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.107+0000 7f2037800640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560d54fddde0 0x560d54fe0bf0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.107+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'87 uv86 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.107+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe1130 con 0x560d54fddde0 2024-09-10T21:33:07.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.107+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.107+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe1550 con 0x560d54fddde0 2024-09-10T21:33:07.108 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:07.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe1550 con 0x560d54fddde0 2024-09-10T21:33:07.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe1550 con 0x560d54fd98d0 2024-09-10T21:33:07.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.111+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.113 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:07.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.115+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.115+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe2900 con 0x560d54fd98d0 2024-09-10T21:33:07.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.115+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.115+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe2900 con 0x560d54fd98d0 2024-09-10T21:33:07.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.115+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.115+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.119+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.119+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.119+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v115'205 uv198 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.119+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.119+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v115'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.119+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.121 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:07.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.123+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v115'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.123+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe2900 con 0x560d54fddde0 2024-09-10T21:33:07.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.123+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.123+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe2900 con 0x560d54fddde0 2024-09-10T21:33:07.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.123+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.123+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v115'211 uv204 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v115'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.128 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v115'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.127+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.131+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.131+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fddde0 2024-09-10T21:33:07.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.131+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.131+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.131+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v115'91 uv90 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.131+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.135+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v115'92 uv92 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.135+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:07.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.135+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v115'93 uv93 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.135+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.135+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.135+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe18f0 con 0x560d54fd98d0 2024-09-10T21:33:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fd98d0 2024-09-10T21:33:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560d54fe1f70 0x560d54fe4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4970 con 0x560d54fe1f70 2024-09-10T21:33:07.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2036e00640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560d54fe1f70 0x560d54fe4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2036e00640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560d54fe1f70 0x560d54fe4360 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe4a30 con 0x560d54fe1f70 2024-09-10T21:33:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.139+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.140 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5e60 con 0x560d54fe1f70 2024-09-10T21:33:07.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5e60 con 0x560d54fe1f70 2024-09-10T21:33:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.143+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.147+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v115'182 uv176 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.147+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.147+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v115'183 uv183 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.147+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.147 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:07.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.147+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v115'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.147+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.151+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.151+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.151+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.151+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.151+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v115'187 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.151+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v115'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.153 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:07.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v115'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe4e50 con 0x560d54fe1f70 2024-09-10T21:33:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe7c60 con 0x560d54fe1f70 2024-09-10T21:33:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.155+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.159+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v115'83 uv81 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.159+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.159+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'84 uv84 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.159+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.159 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:07.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.159+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v115'85 uv85 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.159+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'86 uv86 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v115'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v115'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.164 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:07.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v115'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.163+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fddde0 2024-09-10T21:33:07.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2030015030 con 0x560d54fddde0 2024-09-10T21:33:07.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560d54fe5c00 0x560d54fe8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe5c00 2024-09-10T21:33:07.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560d54fe5c00 0x560d54fe8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.167+0000 7f2036400640 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560d54fe5c00 0x560d54fe8240 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.171+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v115'94 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fe5c00 2024-09-10T21:33:07.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.171+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe5c00 2024-09-10T21:33:07.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.171+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v115'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fe5c00 2024-09-10T21:33:07.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.171+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe5c00 2024-09-10T21:33:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.171+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v115'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fe5c00 2024-09-10T21:33:07.171 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.171+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe5c00 2024-09-10T21:33:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fe5c00 2024-09-10T21:33:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe5c00 2024-09-10T21:33:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2036400640 1 -- 172.21.15.97:0/1758072279 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f20240108e0 con 0x560d54fe5c00 2024-09-10T21:33:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v115'229 uv220 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v115'230 uv230 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.175+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.177 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:07.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.179+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v115'231 uv231 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.179+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.179+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v115'232 uv232 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.179+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.179+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'233 uv233 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.179+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.183+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v115'234 uv227 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.183+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.183+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v115'235 uv235 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.183+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.183 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.183+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v115'236 uv236 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.183+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.187+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v115'237 uv237 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.187+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.187+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v115'238 uv238 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.187+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe5300 con 0x560d54fe1f70 2024-09-10T21:33:07.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.187+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v115'239 uv239 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.187+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x560d54fe87f0 con 0x560d54fe1f70 2024-09-10T21:33:07.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2036e00640 1 -- 172.21.15.97:0/1758072279 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v115'240 uv240 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f20280b3070 con 0x560d54fe1f70 2024-09-10T21:33:07.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560d54fd98d0 msgr2=0x560d54fdbcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560d54fd98d0 0x560d54fdbcc0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560d54fe1f70 msgr2=0x560d54fe4360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560d54fe1f70 0x560d54fe4360 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560d54fe5c00 msgr2=0x560d54fe8240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560d54fe5c00 0x560d54fe8240 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560d54fddde0 msgr2=0x560d54fe0bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560d54fddde0 0x560d54fe0bf0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 reap_dead start 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560d54fd4440 msgr2=0x560d54fd72a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560d54fd4440 0x560d54fd72a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560d54e88d50 msgr2=0x560d54e99810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2037800640 1 -- 172.21.15.97:0/1758072279 reap_dead start 2024-09-10T21:33:07.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560d54e88d50 0x560d54e99810 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2018070ff0 msgr2=0x7f20180734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2018070ff0 0x7f20180734b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f20240014b0 tx=0x7f2024002670 comp rx=0 tx=0).stop 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54e86940 msgr2=0x560d54f0ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54e86940 0x560d54f0ea20 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f2028096ac0 tx=0x7f20280ab620 comp rx=0 tx=0).stop 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 shutdown_connections 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560d54e88d50 0x560d54e99810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2018070ff0 0x7f20180734b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 --2- 172.21.15.97:0/1758072279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d54e86940 0x560d54f0ea20 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 >> 172.21.15.97:0/1758072279 conn(0x560d54e85520 msgr2=0x560d54e56da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 shutdown_connections 2024-09-10T21:33:07.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.191+0000 7f2039bc6080 1 -- 172.21.15.97:0/1758072279 wait complete. 2024-09-10T21:33:07.193 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2024-09-10T21:33:07.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 Processor -- start 2024-09-10T21:33:07.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- start start 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01d9c2940 0x55b01da4c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01da4cb00 0x55b01d991da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4d2a0 0x55b01d9922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b01da4ee70 con 0x55b01d9c2940 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b01d8e1a00 con 0x55b01da4d2a0 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b01da4eff0 con 0x55b01da4cb00 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01d9c2940 0x55b01da4c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01d9c2940 0x55b01da4c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59120/0 (socket says 172.21.15.97:59120) 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 -- 172.21.15.97:0/391848409 learned_addr learned my addr 172.21.15.97:0/391848409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a3a00640 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01da4cb00 0x55b01d991da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4e00640 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4d2a0 0x55b01d9922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 -- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01da4cb00 msgr2=0x55b01d991da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01da4cb00 0x55b01d991da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 -- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4d2a0 msgr2=0x55b01d9922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4d2a0 0x55b01d9922e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 -- 172.21.15.97:0/391848409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01d8e0a20 con 0x55b01d9c2940 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a3a00640 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01da4cb00 0x55b01d991da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4e00640 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4d2a0 0x55b01d9922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:07.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a4400640 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01d9c2940 0x55b01da4c0c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f879400c6a0 tx=0x7f879400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdeacfb0ae64712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a2600640 1 -- 172.21.15.97:0/391848409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879400add0 con 0x55b01d9c2940 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a2600640 1 -- 172.21.15.97:0/391848409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8794008450 con 0x55b01d9c2940 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a2600640 1 -- 172.21.15.97:0/391848409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940088a0 con 0x55b01d9c2940 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- 172.21.15.97:0/391848409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01d9c2940 msgr2=0x55b01da4c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01d9c2940 0x55b01da4c0c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f879400c6a0 tx=0x7f879400cb60 comp rx=0 tx=0).stop 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- 172.21.15.97:0/391848409 shutdown_connections 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4d2a0 0x55b01d9922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01da4cb00 0x55b01d991da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 --2- 172.21.15.97:0/391848409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01d9c2940 0x55b01da4c0c0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- 172.21.15.97:0/391848409 >> 172.21.15.97:0/391848409 conn(0x55b01d9c1520 msgr2=0x55b01d9929d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- 172.21.15.97:0/391848409 shutdown_connections 2024-09-10T21:33:07.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.287+0000 7f87a713f080 1 -- 172.21.15.97:0/391848409 wait complete. 2024-09-10T21:33:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 Processor -- start 2024-09-10T21:33:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 -- start start 2024-09-10T21:33:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01d9c2940 0x55b01d9c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4cb00 0x55b01da6be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01da4d2a0 0x55b01da49270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b01d9c3a20 con 0x55b01da4d2a0 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b01da6b2b0 con 0x55b01da4cb00 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b01d9c3cc0 con 0x55b01d9c2940 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01da4d2a0 0x55b01da49270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01da4d2a0 0x55b01da49270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59132/0 (socket says 172.21.15.97:59132) 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4e00640 1 -- 172.21.15.97:0/3818232240 learned_addr learned my addr 172.21.15.97:0/3818232240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4400640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01d9c2940 0x55b01d9c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a3a00640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4cb00 0x55b01da6be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4400640 1 -- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4cb00 msgr2=0x55b01da6be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4400640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4cb00 0x55b01da6be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4400640 1 -- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01da4d2a0 msgr2=0x55b01da49270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4400640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01da4d2a0 0x55b01da49270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4400640 1 -- 172.21.15.97:0/3818232240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01d8bac10 con 0x55b01d9c2940 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4e00640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b01da4d2a0 0x55b01da49270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a3a00640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b01da4cb00 0x55b01da6be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a4400640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01d9c2940 0x55b01d9c64e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8794014000 tx=0x7f8794008bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a1200640 1 -- 172.21.15.97:0/3818232240 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794008450 con 0x55b01d9c2940 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a1200640 1 -- 172.21.15.97:0/3818232240 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8794015440 con 0x55b01d9c2940 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a1200640 1 -- 172.21.15.97:0/3818232240 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794015890 con 0x55b01d9c2940 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b01da485f0 con 0x55b01d9c2940 2024-09-10T21:33:07.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.291+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55b01da48b30 con 0x55b01d9c2940 2024-09-10T21:33:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a1200640 1 -- 172.21.15.97:0/3818232240 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8794013020 con 0x55b01d9c2940 2024-09-10T21:33:07.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a1200640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8784071160 0x7f8784073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a1200640 1 -- 172.21.15.97:0/3818232240 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(115..115 src has 1..115) ==== 7909+0+0 (secure 0 0 0) 0x7f8794093870 con 0x55b01d9c2940 2024-09-10T21:33:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a3a00640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8784071160 0x7f8784073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a713f080 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b01da696b0 0x55b01da49fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x55b01da6d120 con 0x55b01da696b0 2024-09-10T21:33:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a4e00640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b01da696b0 0x55b01da49fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a3a00640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8784071160 0x7f8784073620 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f879000ca00 tx=0x7f87900049d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.295+0000 7f87a4e00640 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b01da696b0 0x55b01da49fc0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a4e00640 1 -- 172.21.15.97:0/3818232240 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v115'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f879c002050 con 0x55b01da696b0 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b01da696b0 msgr2=0x55b01da49fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b01da696b0 0x55b01da49fc0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8784071160 msgr2=0x7f8784073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8784071160 0x7f8784073620 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f879000ca00 tx=0x7f87900049d0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01d9c2940 msgr2=0x55b01d9c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 --2- 172.21.15.97:0/3818232240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b01d9c2940 0x55b01d9c64e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8794014000 tx=0x7f8794008bc0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a4e00640 1 -- 172.21.15.97:0/3818232240 reap_dead start 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 shutdown_connections 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 >> 172.21.15.97:0/3818232240 conn(0x55b01d9c1520 msgr2=0x55b01da6aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 shutdown_connections 2024-09-10T21:33:07.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.299+0000 7f87a713f080 1 -- 172.21.15.97:0/3818232240 wait complete. 2024-09-10T21:33:07.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 -- 172.21.15.97:0/398628277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f516015fc10 msgr2=0x7f516013f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 --2- 172.21.15.97:0/398628277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f516015fc10 0x7f516013f9b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5160074800 tx=0x7f515801c7e0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 -- 172.21.15.97:0/398628277 shutdown_connections 2024-09-10T21:33:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 --2- 172.21.15.97:0/398628277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f516015fc10 0x7f516013f9b0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 --2- 172.21.15.97:0/398628277 >> v2:172.21.15.194:3300/0 conn(0x7f5160141d20 0x7f516015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 -- 172.21.15.97:0/398628277 >> 172.21.15.97:0/398628277 conn(0x7f51600b1780 msgr2=0x7f51600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 -- 172.21.15.97:0/398628277 shutdown_connections 2024-09-10T21:33:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.487+0000 7f5167c00640 1 -- 172.21.15.97:0/398628277 wait complete. 2024-09-10T21:33:07.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 Processor -- start 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 -- start start 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51600ce5f0 0x7f5160165fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5160141d20 0x7f5160138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f516015fc10 0x7f51600b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5160168a70 con 0x7f51600ce5f0 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5160140cd0 con 0x7f516015fc10 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5160168bf0 con 0x7f5160141d20 2024-09-10T21:33:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51600ce5f0 0x7f5160165fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51600ce5f0 0x7f5160165fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55156/0 (socket says 172.21.15.97:55156) 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 -- 172.21.15.97:0/3985228788 learned_addr learned my addr 172.21.15.97:0/3985228788 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f515fe00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5160141d20 0x7f5160138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165c00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f516015fc10 0x7f51600b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 -- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5160141d20 msgr2=0x7f5160138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5160141d20 0x7f5160138ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 -- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f516015fc10 msgr2=0x7f51600b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f516015fc10 0x7f51600b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 -- 172.21.15.97:0/3985228788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516015e500 con 0x7f51600ce5f0 2024-09-10T21:33:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5165200640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51600ce5f0 0x7f5160165fe0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f515000cba0 tx=0x7f515000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f515cc00640 1 -- 172.21.15.97:0/3985228788 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150016020 con 0x7f51600ce5f0 2024-09-10T21:33:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f515cc00640 1 -- 172.21.15.97:0/3985228788 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f515000ab40 con 0x7f51600ce5f0 2024-09-10T21:33:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516015ecb0 con 0x7f51600ce5f0 2024-09-10T21:33:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f515cc00640 1 -- 172.21.15.97:0/3985228788 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150012420 con 0x7f51600ce5f0 2024-09-10T21:33:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5160139020 con 0x7f51600ce5f0 2024-09-10T21:33:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.491+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5160166520 con 0x7f51600ce5f0 2024-09-10T21:33:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.495+0000 7f515cc00640 1 -- 172.21.15.97:0/3985228788 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f515001b400 con 0x7f51600ce5f0 2024-09-10T21:33:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.495+0000 7f515cc00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51400725e0 0x7f5140074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.495+0000 7f515cc00640 1 -- 172.21.15.97:0/3985228788 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7909+0+0 (secure 0 0 0) 0x7f5150093fe0 con 0x7f51600ce5f0 2024-09-10T21:33:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.495+0000 7f515fe00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51400725e0 0x7f5140074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.495+0000 7f515fe00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51400725e0 0x7f5140074aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f51600ceba0 tx=0x7f514c00b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.499+0000 7f515cc00640 1 -- 172.21.15.97:0/3985228788 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5150020020 con 0x7f51600ce5f0 2024-09-10T21:33:07.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.695+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f516015f6d0 con 0x7f51600ce5f0 2024-09-10T21:33:07.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f515cc00640 1 -- 172.21.15.97:0/3985228788 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v201) ==== 162+0+0 (secure 0 0 0) 0x7f515005e790 con 0x7f51600ce5f0 2024-09-10T21:33:07.886 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51400725e0 msgr2=0x7f5140074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51400725e0 0x7f5140074aa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f51600ceba0 tx=0x7f514c00b390 comp rx=0 tx=0).stop 2024-09-10T21:33:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51600ce5f0 msgr2=0x7f5160165fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51600ce5f0 0x7f5160165fe0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f515000cba0 tx=0x7f515000cc90 comp rx=0 tx=0).stop 2024-09-10T21:33:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 shutdown_connections 2024-09-10T21:33:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51400725e0 0x7f5140074aa0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f516015fc10 0x7f51600b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5160141d20 0x7f5160138ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 --2- 172.21.15.97:0/3985228788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51600ce5f0 0x7f5160165fe0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 >> 172.21.15.97:0/3985228788 conn(0x7f51600b1780 msgr2=0x7f516013b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 shutdown_connections 2024-09-10T21:33:07.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:07.887+0000 7f5167c00640 1 -- 172.21.15.97:0/3985228788 wait complete. 2024-09-10T21:33:07.900 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:33:07.899+0000 7fd041e00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:33:07.900 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:07.903+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:07.901 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:07.903+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:07.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- 172.21.15.97:0/3747978180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68cc004810 msgr2=0x7f68cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 --2- 172.21.15.97:0/3747978180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68cc004810 0x7f68cc004c90 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f68e80b24d0 tx=0x7f68d401d210 comp rx=0 tx=0).stop 2024-09-10T21:33:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 -- 172.21.15.97:0/3747978180 reap_dead start 2024-09-10T21:33:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- 172.21.15.97:0/3747978180 shutdown_connections 2024-09-10T21:33:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- 172.21.15.97:0/3747978180 >> 172.21.15.97:0/3747978180 conn(0x7f68e80b1780 msgr2=0x7f68e80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- 172.21.15.97:0/3747978180 shutdown_connections 2024-09-10T21:33:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- 172.21.15.97:0/3747978180 wait complete. 2024-09-10T21:33:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 Processor -- start 2024-09-10T21:33:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- start start 2024-09-10T21:33:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68cc004810 0x7f68e813fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68e8140ee0 0x7f68e81f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e81f3b40 0x7f68e80ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68e80cd770 con 0x7f68e81f3b40 2024-09-10T21:33:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f68e80cd5f0 con 0x7f68e8140ee0 2024-09-10T21:33:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68ee600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f68e80b2a80 con 0x7f68cc004810 2024-09-10T21:33:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e81f3b40 0x7f68e80ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e81f3b40 0x7f68e80ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55170/0 (socket says 172.21.15.97:55170) 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 -- 172.21.15.97:0/2077036943 learned_addr learned my addr 172.21.15.97:0/2077036943 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e6a00640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68e8140ee0 0x7f68e81f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7400640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68cc004810 0x7f68e813fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 -- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68cc004810 msgr2=0x7f68e813fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68cc004810 0x7f68e813fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 -- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68e8140ee0 msgr2=0x7f68e81f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68e8140ee0 0x7f68e81f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.059+0000 7f68e7e00640 1 -- 172.21.15.97:0/2077036943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e80ad7d0 con 0x7f68e81f3b40 2024-09-10T21:33:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68e7e00640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e81f3b40 0x7f68e80ae870 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f68e80cfae0 tx=0x7f68e00127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68df400640 1 -- 172.21.15.97:0/2077036943 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0013860 con 0x7f68e81f3b40 2024-09-10T21:33:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68df400640 1 -- 172.21.15.97:0/2077036943 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e0013a00 con 0x7f68e81f3b40 2024-09-10T21:33:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68df400640 1 -- 172.21.15.97:0/2077036943 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0013e50 con 0x7f68e81f3b40 2024-09-10T21:33:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e80adab0 con 0x7f68e81f3b40 2024-09-10T21:33:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e80ae070 con 0x7f68e81f3b40 2024-09-10T21:33:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e80701e0 con 0x7f68e81f3b40 2024-09-10T21:33:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.063+0000 7f68df400640 1 -- 172.21.15.97:0/2077036943 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f68e0010480 con 0x7f68e81f3b40 2024-09-10T21:33:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.067+0000 7f68df400640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68c80727b0 0x7f68c8074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.067+0000 7f68df400640 1 -- 172.21.15.97:0/2077036943 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7f68e00978f0 con 0x7f68e81f3b40 2024-09-10T21:33:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.067+0000 7f68e7400640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68c80727b0 0x7f68c8074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.067+0000 7f68e7400640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68c80727b0 0x7f68c8074c70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f68d8000f90 tx=0x7f68d8006ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:08.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.071+0000 7f68df400640 1 -- 172.21.15.97:0/2077036943 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68e0062080 con 0x7f68e81f3b40 2024-09-10T21:33:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.271+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68e80ceb60 con 0x7f68e81f3b40 2024-09-10T21:33:08.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.271+0000 7f68df400640 1 -- 172.21.15.97:0/2077036943 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) ==== 95+0+4395 (secure 0 0 0) 0x7f68e0066ab0 con 0x7f68e81f3b40 2024-09-10T21:33:08.272 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:08.272 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":203,"btime":"2024-09-10T21:33:07:904130+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8841,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2402514654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2402514654},{"type":"v1","addr":"172.21.15.97:6834","nonce":2402514654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:07.904058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:08.272 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 203 2024-09-10T21:33:08.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68c80727b0 msgr2=0x7f68c8074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68c80727b0 0x7f68c8074c70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f68d8000f90 tx=0x7f68d8006ea0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e81f3b40 msgr2=0x7f68e80ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e81f3b40 0x7f68e80ae870 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f68e80cfae0 tx=0x7f68e00127b0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 shutdown_connections 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f68c80727b0 0x7f68c8074c70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68e81f3b40 0x7f68e80ae870 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68e8140ee0 0x7f68e81f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 --2- 172.21.15.97:0/2077036943 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f68cc004810 0x7f68e813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 >> 172.21.15.97:0/2077036943 conn(0x7f68e80b1780 msgr2=0x7f68e80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 shutdown_connections 2024-09-10T21:33:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.275+0000 7f68ee600640 1 -- 172.21.15.97:0/2077036943 wait complete. 2024-09-10T21:33:08.294 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:08.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 -- 172.21.15.97:0/2398608888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f975015eed0 msgr2=0x7f97500d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 --2- 172.21.15.97:0/2398608888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f975015eed0 0x7f97500d27f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9744009840 tx=0x7f974401c7f0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 -- 172.21.15.97:0/2398608888 shutdown_connections 2024-09-10T21:33:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 --2- 172.21.15.97:0/2398608888 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97500ccc60 0x7f97500b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 --2- 172.21.15.97:0/2398608888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f975015eed0 0x7f97500d27f0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 -- 172.21.15.97:0/2398608888 >> 172.21.15.97:0/2398608888 conn(0x7f97500d1cd0 msgr2=0x7f97500d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:08.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 -- 172.21.15.97:0/2398608888 shutdown_connections 2024-09-10T21:33:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 -- 172.21.15.97:0/2398608888 wait complete. 2024-09-10T21:33:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 Processor -- start 2024-09-10T21:33:08.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.439+0000 7f9758800640 1 -- start start 2024-09-10T21:33:08.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97500ccc60 0x7f9750071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97500cdd40 0x7f975014a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f975015eed0 0x7f97500cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9750168880 con 0x7f97500cdd40 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9750168700 con 0x7f97500ccc60 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9750168a00 con 0x7f975015eed0 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97500cdd40 0x7f975014a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97500cdd40 0x7f975014a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55176/0 (socket says 172.21.15.97:55176) 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 -- 172.21.15.97:0/2369701908 learned_addr learned my addr 172.21.15.97:0/2369701908 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755e00640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97500ccc60 0x7f9750071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9756800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f975015eed0 0x7f97500cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 -- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f975015eed0 msgr2=0x7f97500cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f975015eed0 0x7f97500cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 -- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97500ccc60 msgr2=0x7f9750071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97500ccc60 0x7f9750071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 -- 172.21.15.97:0/2369701908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97501f7810 con 0x7f97500cdd40 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9756800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f975015eed0 0x7f97500cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9755400640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97500cdd40 0x7f975014a530 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f974000c4d0 tx=0x7f974000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f974a000640 1 -- 172.21.15.97:0/2369701908 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740016020 con 0x7f97500cdd40 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f974a000640 1 -- 172.21.15.97:0/2369701908 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9740012920 con 0x7f97500cdd40 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97501f7ad0 con 0x7f97500cdd40 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f974a000640 1 -- 172.21.15.97:0/2369701908 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740012d30 con 0x7f97500cdd40 2024-09-10T21:33:08.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97501f8090 con 0x7f97500cdd40 2024-09-10T21:33:08.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97500cfd60 con 0x7f97500cdd40 2024-09-10T21:33:08.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.443+0000 7f974a000640 1 -- 172.21.15.97:0/2369701908 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f974001b400 con 0x7f97500cdd40 2024-09-10T21:33:08.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.447+0000 7f974a000640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9734072610 0x7f9734074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.447+0000 7f974a000640 1 -- 172.21.15.97:0/2369701908 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7f9740093d80 con 0x7f97500cdd40 2024-09-10T21:33:08.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.447+0000 7f9755e00640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9734072610 0x7f9734074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.447+0000 7f9755e00640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9734072610 0x7f9734074ad0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f97440af000 tx=0x7f974401b030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:08.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.451+0000 7f974a000640 1 -- 172.21.15.97:0/2369701908 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9740020020 con 0x7f97500cdd40 2024-09-10T21:33:08.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.647+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97500b0470 con 0x7f97500cdd40 2024-09-10T21:33:08.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.647+0000 7f974a000640 1 -- 172.21.15.97:0/2369701908 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) ==== 95+0+4395 (secure 0 0 0) 0x7f974005eaf0 con 0x7f97500cdd40 2024-09-10T21:33:08.646 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:08.647 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":203,"btime":"2024-09-10T21:33:07:904130+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8841,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2402514654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2402514654},{"type":"v1","addr":"172.21.15.97:6834","nonce":2402514654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:07.904058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:08.647 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 203 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9734072610 msgr2=0x7f9734074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9734072610 0x7f9734074ad0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f97440af000 tx=0x7f974401b030 comp rx=0 tx=0).stop 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97500cdd40 msgr2=0x7f975014a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97500cdd40 0x7f975014a530 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f974000c4d0 tx=0x7f974000c990 comp rx=0 tx=0).stop 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 shutdown_connections 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9734072610 0x7f9734074ad0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f975015eed0 0x7f97500cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97500cdd40 0x7f975014a530 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 --2- 172.21.15.97:0/2369701908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97500ccc60 0x7f9750071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 >> 172.21.15.97:0/2369701908 conn(0x7f97500d1cd0 msgr2=0x7f97500cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 shutdown_connections 2024-09-10T21:33:08.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.651+0000 7f9758800640 1 -- 172.21.15.97:0/2369701908 wait complete. 2024-09-10T21:33:08.669 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:08.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 -- 172.21.15.97:0/36327482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb418159980 msgr2=0x7fb4180745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 --2- 172.21.15.97:0/36327482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb418159980 0x7fb4180745f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fb408009bc0 tx=0x7fb40801cb10 comp rx=0 tx=0).stop 2024-09-10T21:33:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 -- 172.21.15.97:0/36327482 shutdown_connections 2024-09-10T21:33:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 --2- 172.21.15.97:0/36327482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb41806cb40 0x7fb41806cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 --2- 172.21.15.97:0/36327482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb418159980 0x7fb4180745f0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 -- 172.21.15.97:0/36327482 >> 172.21.15.97:0/36327482 conn(0x7fb4180b28c0 msgr2=0x7fb4180b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 -- 172.21.15.97:0/36327482 shutdown_connections 2024-09-10T21:33:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.811+0000 7fb41e200640 1 -- 172.21.15.97:0/36327482 wait complete. 2024-09-10T21:33:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 Processor -- start 2024-09-10T21:33:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 -- start start 2024-09-10T21:33:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41806cb40 0x7fb4180b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4180cd650 0x7fb4180b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb418159980 0x7fb418145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb418168a90 con 0x7fb41806cb40 2024-09-10T21:33:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb418168910 con 0x7fb4180cd650 2024-09-10T21:33:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb418168c10 con 0x7fb418159980 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41806cb40 0x7fb4180b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41806cb40 0x7fb4180b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55196/0 (socket says 172.21.15.97:55196) 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 -- 172.21.15.97:0/2116428780 learned_addr learned my addr 172.21.15.97:0/2116428780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 -- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb418159980 msgr2=0x7fb418145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb418159980 0x7fb418145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 -- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4180cd650 msgr2=0x7fb4180b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4180cd650 0x7fb4180b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 -- 172.21.15.97:0/2116428780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4180af840 con 0x7fb41806cb40 2024-09-10T21:33:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb417400640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41806cb40 0x7fb4180b4e40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb408009b10 tx=0x7fb4080092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb40f400640 1 -- 172.21.15.97:0/2116428780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080bb070 con 0x7fb41806cb40 2024-09-10T21:33:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb40f400640 1 -- 172.21.15.97:0/2116428780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb408004d40 con 0x7fb41806cb40 2024-09-10T21:33:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4181fba10 con 0x7fb41806cb40 2024-09-10T21:33:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb40f400640 1 -- 172.21.15.97:0/2116428780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b2040 con 0x7fb41806cb40 2024-09-10T21:33:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.815+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb418142f80 con 0x7fb41806cb40 2024-09-10T21:33:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.819+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41806cfb0 con 0x7fb41806cb40 2024-09-10T21:33:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.819+0000 7fb40f400640 1 -- 172.21.15.97:0/2116428780 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4080b5030 con 0x7fb41806cb40 2024-09-10T21:33:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.819+0000 7fb40f400640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3f80725e0 0x7fb3f8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.819+0000 7fb40f400640 1 -- 172.21.15.97:0/2116428780 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7fb40812d870 con 0x7fb41806cb40 2024-09-10T21:33:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.819+0000 7fb416a00640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3f80725e0 0x7fb3f8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:08.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.819+0000 7fb416a00640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3f80725e0 0x7fb3f8074aa0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb4040058d0 tx=0x7fb404005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:08.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:08.823+0000 7fb40f400640 1 -- 172.21.15.97:0/2116428780 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb408016790 con 0x7fb41806cb40 2024-09-10T21:33:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4180cdc80 con 0x7fb41806cb40 2024-09-10T21:33:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb40f400640 1 -- 172.21.15.97:0/2116428780 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) ==== 95+0+4395 (secure 0 0 0) 0x7fb4080f85d0 con 0x7fb41806cb40 2024-09-10T21:33:09.025 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:09.025 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":203,"btime":"2024-09-10T21:33:07:904130+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8841,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2402514654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2402514654},{"type":"v1","addr":"172.21.15.97:6834","nonce":2402514654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:07.904058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:09.025 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 203 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3f80725e0 msgr2=0x7fb3f8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3f80725e0 0x7fb3f8074aa0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb4040058d0 tx=0x7fb404005860 comp rx=0 tx=0).stop 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41806cb40 msgr2=0x7fb4180b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41806cb40 0x7fb4180b4e40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fb408009b10 tx=0x7fb4080092a0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 shutdown_connections 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb3f80725e0 0x7fb3f8074aa0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb418159980 0x7fb418145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4180cd650 0x7fb4180b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 --2- 172.21.15.97:0/2116428780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb41806cb40 0x7fb4180b4e40 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 >> 172.21.15.97:0/2116428780 conn(0x7fb4180b28c0 msgr2=0x7fb418158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:09.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 shutdown_connections 2024-09-10T21:33:09.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.027+0000 7fb41e200640 1 -- 172.21.15.97:0/2116428780 wait complete. 2024-09-10T21:33:09.046 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2024-09-10T21:33:09.046 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2024-09-10T21:33:09.046 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:09.047 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:09.055 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:09.055 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:09.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:09.056 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:09.107 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:09.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:09.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:09.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 -- 172.21.15.97:0/1991551145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40cfa40 msgr2=0x7f51d40cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:09.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 --2- 172.21.15.97:0/1991551145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40cfa40 0x7f51d40cc650 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009bc0 tx=0x7f51cc01cb70 comp rx=0 tx=0).stop 2024-09-10T21:33:09.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 -- 172.21.15.97:0/1991551145 shutdown_connections 2024-09-10T21:33:09.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 --2- 172.21.15.97:0/1991551145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40cfa40 0x7f51d40cc650 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 --2- 172.21.15.97:0/1991551145 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51d415fdb0 0x7f51d40d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 -- 172.21.15.97:0/1991551145 >> 172.21.15.97:0/1991551145 conn(0x7f51d40b36d0 msgr2=0x7f51d40b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:09.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 -- 172.21.15.97:0/1991551145 shutdown_connections 2024-09-10T21:33:09.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.299+0000 7f51da400640 1 -- 172.21.15.97:0/1991551145 wait complete. 2024-09-10T21:33:09.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 Processor -- start 2024-09-10T21:33:09.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 -- start start 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40b1470 0x7f51d40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51d40cfa40 0x7f51d4156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51d415fdb0 0x7f51d415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40cd9d0 con 0x7f51d40cfa40 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40cd850 con 0x7f51d40b1470 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f51d41690a0 con 0x7f51d415fdb0 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51d40cfa40 0x7f51d4156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51d40cfa40 0x7f51d4156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55198/0 (socket says 172.21.15.97:55198) 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 -- 172.21.15.97:0/15328375 learned_addr learned my addr 172.21.15.97:0/15328375 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51d9000640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40b1470 0x7f51d40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51d9a00640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51d415fdb0 0x7f51d415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 -- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51d415fdb0 msgr2=0x7f51d415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51d415fdb0 0x7f51d415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 -- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40b1470 msgr2=0x7f51d40af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40b1470 0x7f51d40af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 -- 172.21.15.97:0/15328375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d415b740 con 0x7f51d40cfa40 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51d9000640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40b1470 0x7f51d40af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51cbe00640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51d40cfa40 0x7f51d4156bd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00c460 tx=0x7f51bc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51d1600640 1 -- 172.21.15.97:0/15328375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc016020 con 0x7f51d40cfa40 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51d1600640 1 -- 172.21.15.97:0/15328375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51bc012920 con 0x7f51d40cfa40 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51d1600640 1 -- 172.21.15.97:0/15328375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc012d50 con 0x7f51d40cfa40 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d40af980 con 0x7f51d40cfa40 2024-09-10T21:33:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.303+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d415cb20 con 0x7f51d40cfa40 2024-09-10T21:33:09.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.307+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d40cc650 con 0x7f51d40cfa40 2024-09-10T21:33:09.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.307+0000 7f51d1600640 1 -- 172.21.15.97:0/15328375 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f51bc01b400 con 0x7f51d40cfa40 2024-09-10T21:33:09.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.307+0000 7f51d1600640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51b00725e0 0x7f51b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:09.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.307+0000 7f51d1600640 1 -- 172.21.15.97:0/15328375 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7f51bc093a30 con 0x7f51d40cfa40 2024-09-10T21:33:09.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.307+0000 7f51d9000640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51b00725e0 0x7f51b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:09.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.307+0000 7f51d9000640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51b00725e0 0x7f51b0074aa0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009f30 tx=0x7f51c40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:09.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.311+0000 7f51d1600640 1 -- 172.21.15.97:0/15328375 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f51bc020030 con 0x7f51d40cfa40 2024-09-10T21:33:09.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.511+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f51d4069d80 con 0x7f51d40cfa40 2024-09-10T21:33:09.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:09.624 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:09.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51d1600640 1 -- 172.21.15.97:0/15328375 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v204) ==== 161+0+0 (secure 0 0 0) 0x7f51bc05e1c0 con 0x7f51d40cfa40 2024-09-10T21:33:09.914 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:09.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51b00725e0 msgr2=0x7f51b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:09.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51b00725e0 0x7f51b0074aa0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009f30 tx=0x7f51c40091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51d40cfa40 msgr2=0x7f51d4156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51d40cfa40 0x7f51d4156bd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00c460 tx=0x7f51bc00c920 comp rx=0 tx=0).stop 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 shutdown_connections 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51b00725e0 0x7f51b0074aa0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51d415fdb0 0x7f51d415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51d40cfa40 0x7f51d4156bd0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 --2- 172.21.15.97:0/15328375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51d40b1470 0x7f51d40af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.915+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 >> 172.21.15.97:0/15328375 conn(0x7f51d40b36d0 msgr2=0x7f51d40ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.919+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 shutdown_connections 2024-09-10T21:33:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:09.919+0000 7f51da400640 1 -- 172.21.15.97:0/15328375 wait complete. 2024-09-10T21:33:09.936 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:10.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 -- 172.21.15.97:0/1410180456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f40d0490 msgr2=0x7ff2f40ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:10.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/1410180456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f40d0490 0x7ff2f40ae930 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009840 tx=0x7ff2ec01c820 comp rx=0 tx=0).stop 2024-09-10T21:33:10.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 -- 172.21.15.97:0/1410180456 shutdown_connections 2024-09-10T21:33:10.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/1410180456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f40d0490 0x7ff2f40ae930 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/1410180456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f415efc0 0x7ff2f40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 -- 172.21.15.97:0/1410180456 >> 172.21.15.97:0/1410180456 conn(0x7ff2f40d1a60 msgr2=0x7ff2f40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:10.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 -- 172.21.15.97:0/1410180456 shutdown_connections 2024-09-10T21:33:10.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.079+0000 7ff2f3e00640 1 -- 172.21.15.97:0/1410180456 wait complete. 2024-09-10T21:33:10.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 Processor -- start 2024-09-10T21:33:10.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 -- start start 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f40cdcb0 0x7ff2f4153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f40d0490 0x7ff2f40b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f415efc0 0x7ff2f406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41689c0 con 0x7ff2f415efc0 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4168b40 con 0x7ff2f40cdcb0 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f4168cc0 con 0x7ff2f40d0490 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f415efc0 0x7ff2f406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f415efc0 0x7ff2f406dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55216/0 (socket says 172.21.15.97:55216) 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 -- 172.21.15.97:0/3311855545 learned_addr learned my addr 172.21.15.97:0/3311855545 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2ea000640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f40d0490 0x7ff2f40b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f2a00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f40cdcb0 0x7ff2f4153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 -- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f40d0490 msgr2=0x7ff2f40b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f40d0490 0x7ff2f40b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 -- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f40cdcb0 msgr2=0x7ff2f4153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f40cdcb0 0x7ff2f4153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 -- 172.21.15.97:0/3311855545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f40cc920 con 0x7ff2f415efc0 2024-09-10T21:33:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f2a00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f40cdcb0 0x7ff2f4153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3400640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f415efc0 0x7ff2f406dcc0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009790 tx=0x7ff2ec0a64f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2eb400640 1 -- 172.21.15.97:0/3311855545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0a66e0 con 0x7ff2f415efc0 2024-09-10T21:33:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2eb400640 1 -- 172.21.15.97:0/3311855545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2ec004130 con 0x7ff2f415efc0 2024-09-10T21:33:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f40b4d50 con 0x7ff2f415efc0 2024-09-10T21:33:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2eb400640 1 -- 172.21.15.97:0/3311855545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec003600 con 0x7ff2f415efc0 2024-09-10T21:33:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.083+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f40b1c10 con 0x7ff2f415efc0 2024-09-10T21:33:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.087+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f40ae930 con 0x7ff2f415efc0 2024-09-10T21:33:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.087+0000 7ff2eb400640 1 -- 172.21.15.97:0/3311855545 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff2ec0037a0 con 0x7ff2f415efc0 2024-09-10T21:33:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.087+0000 7ff2eb400640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d0072780 0x7ff2d0074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.087+0000 7ff2eb400640 1 -- 172.21.15.97:0/3311855545 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7ff2ec12e910 con 0x7ff2f415efc0 2024-09-10T21:33:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.087+0000 7ff2f2a00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d0072780 0x7ff2d0074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.087+0000 7ff2f2a00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d0072780 0x7ff2d0074c40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009d90 tx=0x7ff2e4009320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.091+0000 7ff2eb400640 1 -- 172.21.15.97:0/3311855545 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff2ec0c9020 con 0x7ff2f415efc0 2024-09-10T21:33:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.315+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff2f40c4ee0 con 0x7ff2f415efc0 2024-09-10T21:33:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2eb400640 1 -- 172.21.15.97:0/3311855545 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v205) ==== 101+0+0 (secure 0 0 0) 0x7ff2f40b4d50 con 0x7ff2f415efc0 2024-09-10T21:33:10.921 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d0072780 msgr2=0x7ff2d0074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d0072780 0x7ff2d0074c40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009d90 tx=0x7ff2e4009320 comp rx=0 tx=0).stop 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f415efc0 msgr2=0x7ff2f406dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f415efc0 0x7ff2f406dcc0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009790 tx=0x7ff2ec0a64f0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 shutdown_connections 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d0072780 0x7ff2d0074c40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f415efc0 0x7ff2f406dcc0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f40d0490 0x7ff2f40b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 --2- 172.21.15.97:0/3311855545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f40cdcb0 0x7ff2f4153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:10.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 >> 172.21.15.97:0/3311855545 conn(0x7ff2f40d1a60 msgr2=0x7ff2f415ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:10.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 shutdown_connections 2024-09-10T21:33:10.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.923+0000 7ff2f3e00640 1 -- 172.21.15.97:0/3311855545 wait complete. 2024-09-10T21:33:10.949 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:10.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.995+0000 7fba30e21080 1 Processor -- start 2024-09-10T21:33:10.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.995+0000 7fba30e21080 1 -- start start 2024-09-10T21:33:10.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a2760c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:10.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a276b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:10.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a27699c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:10.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a276987b0 con 0x559a27695cc0 2024-09-10T21:33:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a2752ba00 con 0x559a27697830 2024-09-10T21:33:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559a27698930 con 0x559a2769b1d0 2024-09-10T21:33:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a2760c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a2760c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55224/0 (socket says 172.21.15.97:55224) 2024-09-10T21:33:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 -- 172.21.15.97:0/4909122 learned_addr learned my addr 172.21.15.97:0/4909122 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2d600640 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a276b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:10.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2ea00640 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a27699c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 -- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 msgr2=0x559a27699c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:10.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a27699c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 -- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 msgr2=0x559a276b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a276b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 -- 172.21.15.97:0/4909122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a2752aa20 con 0x559a27695cc0 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba2e000640 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a2760c030 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c610 tx=0x7fba2000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6681591a51fdc0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba27e00640 1 -- 172.21.15.97:0/4909122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000aaf0 con 0x559a27695cc0 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba27e00640 1 -- 172.21.15.97:0/4909122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba2000ac90 con 0x559a27695cc0 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba27e00640 1 -- 172.21.15.97:0/4909122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba200156a0 con 0x559a27695cc0 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- 172.21.15.97:0/4909122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 msgr2=0x559a2760c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a2760c030 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c610 tx=0x7fba2000cad0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- 172.21.15.97:0/4909122 shutdown_connections 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a27699c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a276b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 --2- 172.21.15.97:0/4909122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a2760c030 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- 172.21.15.97:0/4909122 >> 172.21.15.97:0/4909122 conn(0x559a276250d0 msgr2=0x559a276b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- 172.21.15.97:0/4909122 shutdown_connections 2024-09-10T21:33:11.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 -- 172.21.15.97:0/4909122 wait complete. 2024-09-10T21:33:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:10.999+0000 7fba30e21080 1 Processor -- start 2024-09-10T21:33:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 -- start start 2024-09-10T21:33:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a276bb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a2760bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a2760c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a27695990 con 0x559a27695cc0 2024-09-10T21:33:11.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a276956f0 con 0x559a27697830 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559a276b4280 con 0x559a2769b1d0 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a276bb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a276bb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55236/0 (socket says 172.21.15.97:55236) 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a2760bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 learned_addr learned my addr 172.21.15.97:0/2366130705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2ea00640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a2760c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 msgr2=0x559a2760c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a2760c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 msgr2=0x559a2760bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a2760bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a27504c10 con 0x559a27695cc0 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2ea00640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559a2769b1d0 0x559a2760c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:11.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2d600640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a27697830 0x559a2760bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba2e000640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a276bb8f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fba200159e0 tx=0x7fba20015a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba26a00640 1 -- 172.21.15.97:0/2366130705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000ae80 con 0x559a27695cc0 2024-09-10T21:33:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba26a00640 1 -- 172.21.15.97:0/2366130705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba20015dc0 con 0x559a27695cc0 2024-09-10T21:33:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a275dd7f0 con 0x559a27695cc0 2024-09-10T21:33:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba26a00640 1 -- 172.21.15.97:0/2366130705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2001e650 con 0x559a27695cc0 2024-09-10T21:33:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.003+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a275ddd30 con 0x559a27695cc0 2024-09-10T21:33:11.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba26a00640 1 -- 172.21.15.97:0/2366130705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba2001d070 con 0x559a27695cc0 2024-09-10T21:33:11.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba26a00640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fba10070fc0 0x7fba10073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba26a00640 1 -- 172.21.15.97:0/2366130705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7fba200940b0 con 0x559a27695cc0 2024-09-10T21:33:11.006 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559a2774eac0 0x559a275dc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a275dcb90 con 0x559a2774eac0 2024-09-10T21:33:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba2d600640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fba10070fc0 0x7fba10073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba2ea00640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559a2774eac0 0x559a275dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba2d600640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fba10070fc0 0x7fba10073480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00db90 tx=0x7fba1c00e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.007+0000 7fba2ea00640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559a2774eac0 0x559a275dc650 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.011+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v116'598 uv572 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.011+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a275dcf30 con 0x559a2774eac0 2024-09-10T21:33:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.011+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v116'599 uv599 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.011+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a275dd300 con 0x559a2774eac0 2024-09-10T21:33:11.012 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:11.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.011+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v116'600 uv600 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.011+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775ac90 con 0x559a2774eac0 2024-09-10T21:33:11.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.015+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.015+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775b030 con 0x559a2774eac0 2024-09-10T21:33:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.015+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.015+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27759c80 con 0x559a2774eac0 2024-09-10T21:33:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.019+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.019+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a070 con 0x559a2774eac0 2024-09-10T21:33:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.019+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v116'604 uv578 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.019+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775a070 con 0x559a2774eac0 2024-09-10T21:33:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.019+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v116'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.019+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.021 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.023+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v116'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.023+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.023+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.023+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.027+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.027+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.027+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.027+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.027+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v116'610 uv585 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.031+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.031+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'611 uv611 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.031+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.031 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:11.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.031+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v116'612 uv612 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.031+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.035+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'613 uv613 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.035+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.035+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'614 uv614 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.035+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a020 con 0x559a2774eac0 2024-09-10T21:33:11.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.035+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'615 uv615 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.035+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a410 con 0x559a2774eac0 2024-09-10T21:33:11.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.039+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'616 uv616 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.039+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a410 con 0x559a2774eac0 2024-09-10T21:33:11.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.039+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v116'617 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.039+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775a800 con 0x559a2774eac0 2024-09-10T21:33:11.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.043+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v116'618 uv618 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.043+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a800 con 0x559a2774eac0 2024-09-10T21:33:11.042 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.043+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v116'619 uv619 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.043+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.043+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'620 uv620 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.043+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.047+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'621 uv621 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.047+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.047+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.047+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.047+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v116'623 uv597 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.047+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.051+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v116'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.051+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.051 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:11.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.051+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v116'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.051+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.055+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.055+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.055+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.055+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2774eac0 2024-09-10T21:33:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.055+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a2774eac0 2024-09-10T21:33:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.055+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559a2775b3d0 0x559a2775d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.055+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775a2a0 con 0x559a2775b3d0 2024-09-10T21:33:11.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.059+0000 7fba2e000640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559a2775b3d0 0x559a2775d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.059+0000 7fba2e000640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559a2775b3d0 0x559a2775d7c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.059+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'442 uv425 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.059+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775dd00 con 0x559a2775b3d0 2024-09-10T21:33:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.059+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'443 uv443 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.059+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775e0a0 con 0x559a2775b3d0 2024-09-10T21:33:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.063+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'444 uv444 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.062 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:11.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.063+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775e8f0 con 0x559a2775b3d0 2024-09-10T21:33:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.063+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v116'445 uv445 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.063+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775ec90 con 0x559a2775b3d0 2024-09-10T21:33:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.067+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'446 uv446 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.067+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f030 con 0x559a2775b3d0 2024-09-10T21:33:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.067+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v116'447 uv430 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.067+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775f3d0 con 0x559a2775b3d0 2024-09-10T21:33:11.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.067+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v116'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.067+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.069 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.071+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v116'449 uv449 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.071+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.071+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.071+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.071+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.071+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.075+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v116'452 uv435 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.075+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.075+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v116'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.075+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.079+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v116'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.076 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:11.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.079+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27763940 con 0x559a2775b3d0 2024-09-10T21:33:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.079+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v116'455 uv455 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.079+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27763940 con 0x559a2775b3d0 2024-09-10T21:33:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.079+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'456 uv456 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.079+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.083+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v116'457 uv441 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.083+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.083+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v116'458 uv458 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.083+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.083+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v116'459 uv459 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.084 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.087+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27760780 con 0x559a2775b3d0 2024-09-10T21:33:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.087+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'460 uv460 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.087+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27760780 con 0x559a2775b3d0 2024-09-10T21:33:11.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.087+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'461 uv461 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.087+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2775f770 con 0x559a2775b3d0 2024-09-10T21:33:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'462 uv462 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a2775b3d0 2024-09-10T21:33:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559a2775fc80 0x559a27762070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277625b0 con 0x559a2775fc80 2024-09-10T21:33:11.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba2d600640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559a2775fc80 0x559a27762070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba2d600640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559a2775fc80 0x559a27762070 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a27762e00 con 0x559a2775fc80 2024-09-10T21:33:11.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.091+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277631a0 con 0x559a2775fc80 2024-09-10T21:33:11.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.095+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.093 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:11.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.095+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27764550 con 0x559a2775fc80 2024-09-10T21:33:11.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.095+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.095+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277649d0 con 0x559a2775fc80 2024-09-10T21:33:11.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.095+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.095+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27763540 con 0x559a2775fc80 2024-09-10T21:33:11.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559a27764000 0x559a27766e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277673a0 con 0x559a27764000 2024-09-10T21:33:11.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba2ea00640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559a27764000 0x559a27766e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba2ea00640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559a27764000 0x559a27766e00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'91 uv90 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a27767790 con 0x559a27764000 2024-09-10T21:33:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'92 uv92 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.099+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767790 con 0x559a27764000 2024-09-10T21:33:11.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'93 uv93 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.101 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:11.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767790 con 0x559a27764000 2024-09-10T21:33:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'94 uv94 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767760 con 0x559a2775fc80 2024-09-10T21:33:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a277676e0 con 0x559a2775fc80 2024-09-10T21:33:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.103+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277676e0 con 0x559a2775fc80 2024-09-10T21:33:11.105 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:11.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.107+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.107+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277676a0 con 0x559a2775fc80 2024-09-10T21:33:11.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.107+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.107+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277676a0 con 0x559a2775fc80 2024-09-10T21:33:11.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.107+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.107+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277676a0 con 0x559a2775fc80 2024-09-10T21:33:11.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.111+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.111+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277676a0 con 0x559a27764000 2024-09-10T21:33:11.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.111+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v116'216 uv210 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.111+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a277676a0 con 0x559a27764000 2024-09-10T21:33:11.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.111+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v116'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.111+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a277676a0 con 0x559a27764000 2024-09-10T21:33:11.112 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:11.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v116'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27768b70 con 0x559a27764000 2024-09-10T21:33:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27768b70 con 0x559a27764000 2024-09-10T21:33:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a27764000 2024-09-10T21:33:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.115+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a27764000 2024-09-10T21:33:11.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.119+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v116'222 uv215 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.119+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a27764000 2024-09-10T21:33:11.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.119+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v116'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.119+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a27764000 2024-09-10T21:33:11.119 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:11.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.119+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v116'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.119+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a27764000 2024-09-10T21:33:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.119+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.123+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a27764000 2024-09-10T21:33:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.123+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.123+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a2775fc80 2024-09-10T21:33:11.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.123+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v116'97 uv96 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.123+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a2775fc80 2024-09-10T21:33:11.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.123+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v116'98 uv98 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.123+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a2775fc80 2024-09-10T21:33:11.125 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:11.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v116'99 uv99 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a2775fc80 2024-09-10T21:33:11.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'100 uv100 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a27767b60 con 0x559a2775fc80 2024-09-10T21:33:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'101 uv101 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2775fc80 2024-09-10T21:33:11.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559a27768070 0x559a2776a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776a9f0 con 0x559a27768070 2024-09-10T21:33:11.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.127+0000 7fba2e000640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559a27768070 0x559a2776a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.131+0000 7fba2e000640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559a27768070 0x559a2776a460 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.131+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.131+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.131+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.131+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.131 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:11.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.131+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.131+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776bdd0 con 0x559a27768070 2024-09-10T21:33:11.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776bdd0 con 0x559a27768070 2024-09-10T21:33:11.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v116'192 uv186 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.135+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.139+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v116'193 uv193 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.139+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.138 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:11.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.139+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v116'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.139+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.139+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.139+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v116'197 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v116'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v116'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.144 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:11.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.143+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776adc0 con 0x559a27768070 2024-09-10T21:33:11.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.147+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.147+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dc50 con 0x559a27768070 2024-09-10T21:33:11.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.147+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.147+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dc10 con 0x559a27764000 2024-09-10T21:33:11.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.147+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v116'87 uv86 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.147+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776dc10 con 0x559a27764000 2024-09-10T21:33:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'88 uv88 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dc10 con 0x559a27764000 2024-09-10T21:33:11.149 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:11.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v116'89 uv89 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dc10 con 0x559a27764000 2024-09-10T21:33:11.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'90 uv90 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dbf0 con 0x559a27764000 2024-09-10T21:33:11.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.151+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776dbf0 con 0x559a27764000 2024-09-10T21:33:11.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.155+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v116'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.155+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dbf0 con 0x559a27764000 2024-09-10T21:33:11.154 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:11.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.155+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v116'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.155+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dbf0 con 0x559a27764000 2024-09-10T21:33:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.155+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.155+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dbd0 con 0x559a27764000 2024-09-10T21:33:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776dbd0 con 0x559a27764000 2024-09-10T21:33:11.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba28002050 con 0x559a27764000 2024-09-10T21:33:11.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559a2776dbd0 0x559a2776e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a2776dbd0 2024-09-10T21:33:11.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba2d600640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559a2776dbd0 0x559a2776e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba2d600640 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559a2776dbd0 0x559a2776e310 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v116'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2776dbd0 2024-09-10T21:33:11.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.159+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a2776dbd0 2024-09-10T21:33:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.163+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v116'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2776dbd0 2024-09-10T21:33:11.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.163+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a2776dbd0 2024-09-10T21:33:11.162 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.163+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v116'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2776dbd0 2024-09-10T21:33:11.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.163+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a2776dbd0 2024-09-10T21:33:11.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.163+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2776dbd0 2024-09-10T21:33:11.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.163+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a2776dbd0 2024-09-10T21:33:11.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba2d600640 1 -- 172.21.15.97:0/2366130705 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fba1c00ca70 con 0x559a2776dbd0 2024-09-10T21:33:11.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v116'241 uv233 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v116'242 uv242 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.168 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v116'243 uv243 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.167+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.171+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v116'244 uv244 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.171+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.171+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'245 uv245 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.171+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.171+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v116'246 uv240 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.171+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v116'247 uv247 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.173 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:11.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v116'248 uv248 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'249 uv249 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'250 uv250 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.175+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776d770 con 0x559a27768070 2024-09-10T21:33:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.179+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'251 uv251 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.179+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x559a2776e910 con 0x559a27768070 2024-09-10T21:33:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba2e000640 1 -- 172.21.15.97:0/2366130705 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'252 uv252 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fba2005aa00 con 0x559a27768070 2024-09-10T21:33:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559a2775fc80 msgr2=0x559a27762070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x559a2775fc80 0x559a27762070 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559a27768070 msgr2=0x559a2776a460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x559a27768070 0x559a2776a460 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559a2776dbd0 msgr2=0x559a2776e310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x559a2776dbd0 0x559a2776e310 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559a27764000 msgr2=0x559a27766e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x559a27764000 0x559a27766e00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 reap_dead start 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559a2775b3d0 msgr2=0x559a2775d7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x559a2775b3d0 0x559a2775d7c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559a2774eac0 msgr2=0x559a275dc650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x559a2774eac0 0x559a275dc650 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fba10070fc0 msgr2=0x7fba10073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fba10070fc0 0x7fba10073480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00db90 tx=0x7fba1c00e040 comp rx=0 tx=0).stop 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 msgr2=0x559a276bb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 --2- 172.21.15.97:0/2366130705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a27695cc0 0x559a276bb8f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fba200159e0 tx=0x7fba20015a10 comp rx=0 tx=0).stop 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba2ea00640 1 -- 172.21.15.97:0/2366130705 reap_dead start 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 shutdown_connections 2024-09-10T21:33:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 >> 172.21.15.97:0/2366130705 conn(0x559a276250d0 msgr2=0x559a276242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 shutdown_connections 2024-09-10T21:33:11.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.183+0000 7fba30e21080 1 -- 172.21.15.97:0/2366130705 wait complete. 2024-09-10T21:33:11.186 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-09-10T21:33:11.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 Processor -- start 2024-09-10T21:33:11.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- start start 2024-09-10T21:33:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d17058200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 0x558d17058740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d16f76a00 con 0x558d170e0cc0 2024-09-10T21:33:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558d16fd4f40 con 0x558d170e2a00 2024-09-10T21:33:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558d170e4710 con 0x558d170e61d0 2024-09-10T21:33:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170ff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55242/0 (socket says 172.21.15.97:55242) 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 -- 172.21.15.97:0/2347808548 learned_addr learned my addr 172.21.15.97:0/2347808548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc03e00640 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d17058200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0b200640 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 0x558d17058740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 -- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 msgr2=0x558d17058740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 0x558d17058740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 -- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 msgr2=0x558d17058200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d17058200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 -- 172.21.15.97:0/2347808548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d16f4fc10 con 0x558d170e0cc0 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc03e00640 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d17058200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0a800640 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170ff630 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b7d0 tx=0x7efbfc004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4d821063f8c626 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc09400640 1 -- 172.21.15.97:0/2347808548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00ab70 con 0x558d170e0cc0 2024-09-10T21:33:11.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc09400640 1 -- 172.21.15.97:0/2347808548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbfc00ad10 con 0x558d170e0cc0 2024-09-10T21:33:11.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc09400640 1 -- 172.21.15.97:0/2347808548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc008690 con 0x558d170e0cc0 2024-09-10T21:33:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- 172.21.15.97:0/2347808548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 msgr2=0x558d170ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170ff630 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b7d0 tx=0x7efbfc004f90 comp rx=0 tx=0).stop 2024-09-10T21:33:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- 172.21.15.97:0/2347808548 shutdown_connections 2024-09-10T21:33:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 0x558d17058740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d17058200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 --2- 172.21.15.97:0/2347808548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170ff630 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- 172.21.15.97:0/2347808548 >> 172.21.15.97:0/2347808548 conn(0x558d170700d0 msgr2=0x558d1702b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- 172.21.15.97:0/2347808548 shutdown_connections 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.279+0000 7efc0d511080 1 -- 172.21.15.97:0/2347808548 wait complete. 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 Processor -- start 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 -- start start 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d1719b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 0x558d170287c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558d17072450 con 0x558d170e0cc0 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558d170df990 con 0x558d170e2a00 2024-09-10T21:33:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558d170726f0 con 0x558d170e61d0 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170fc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55248/0 (socket says 172.21.15.97:55248) 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 -- 172.21.15.97:0/2307591187 learned_addr learned my addr 172.21.15.97:0/2307591187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc03e00640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d1719b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0b200640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 0x558d170287c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 -- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 msgr2=0x558d170287c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558d170e61d0 0x558d170287c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 -- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 msgr2=0x558d1719b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d170e2a00 0x558d1719b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 -- 172.21.15.97:0/2307591187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1706eb80 con 0x558d170e0cc0 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0a800640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170fc4a0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00a7b0 tx=0x7efbfc0156d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc03400640 1 -- 172.21.15.97:0/2307591187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc015d00 con 0x558d170e0cc0 2024-09-10T21:33:11.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc03400640 1 -- 172.21.15.97:0/2307591187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbfc01e440 con 0x558d170e0cc0 2024-09-10T21:33:11.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc03400640 1 -- 172.21.15.97:0/2307591187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc01e890 con 0x558d170e0cc0 2024-09-10T21:33:11.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d1706ee20 con 0x558d170e0cc0 2024-09-10T21:33:11.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.283+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1706f360 con 0x558d170e0cc0 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc03400640 1 -- 172.21.15.97:0/2307591187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efbfc01c020 con 0x558d170e0cc0 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc03400640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efbe8070fc0 0x7efbe8073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc03400640 1 -- 172.21.15.97:0/2307591187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7efbfc093650 con 0x558d170e0cc0 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc03e00640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efbe8070fc0 0x7efbe8073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc0d511080 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558d17027140 0x558d17027520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x558d17027a60 con 0x558d17027140 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc0b200640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558d17027140 0x558d17027520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc03e00640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efbe8070fc0 0x7efbe8073480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efbf40105c0 tx=0x7efbf4002800 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.287+0000 7efc0b200640 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558d17027140 0x558d17027520 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0b200640 1 -- 172.21.15.97:0/2307591187 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v116'202 uv202 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc040a7070 con 0x558d17027140 2024-09-10T21:33:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558d17027140 msgr2=0x558d17027520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558d17027140 0x558d17027520 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efbe8070fc0 msgr2=0x7efbe8073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efbe8070fc0 0x7efbe8073480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efbf40105c0 tx=0x7efbf4002800 comp rx=0 tx=0).stop 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 msgr2=0x558d170fc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 --2- 172.21.15.97:0/2307591187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558d170e0cc0 0x558d170fc4a0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00a7b0 tx=0x7efbfc0156d0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0b200640 1 -- 172.21.15.97:0/2307591187 reap_dead start 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 shutdown_connections 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 >> 172.21.15.97:0/2307591187 conn(0x558d170700d0 msgr2=0x558d170ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 shutdown_connections 2024-09-10T21:33:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.291+0000 7efc0d511080 1 -- 172.21.15.97:0/2307591187 wait complete. 2024-09-10T21:33:11.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:11.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.483+0000 7ffa29a00640 1 -- 172.21.15.97:0/449994048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa241511f0 msgr2=0x7ffa240b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.483+0000 7ffa29a00640 1 --2- 172.21.15.97:0/449994048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa241511f0 0x7ffa240b37b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009c80 tx=0x7ffa1401f560 comp rx=0 tx=0).stop 2024-09-10T21:33:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.483+0000 7ffa29a00640 1 -- 172.21.15.97:0/449994048 shutdown_connections 2024-09-10T21:33:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.483+0000 7ffa29a00640 1 --2- 172.21.15.97:0/449994048 >> v2:172.21.15.194:3300/0 conn(0x7ffa240b3cf0 0x7ffa2406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.483+0000 7ffa29a00640 1 --2- 172.21.15.97:0/449994048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa241511f0 0x7ffa240b37b0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.483+0000 7ffa29a00640 1 -- 172.21.15.97:0/449994048 >> 172.21.15.97:0/449994048 conn(0x7ffa240b1780 msgr2=0x7ffa240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.483+0000 7ffa29a00640 1 -- 172.21.15.97:0/449994048 shutdown_connections 2024-09-10T21:33:11.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 -- 172.21.15.97:0/449994048 wait complete. 2024-09-10T21:33:11.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 Processor -- start 2024-09-10T21:33:11.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 -- start start 2024-09-10T21:33:11.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa2406f180 0x7ffa241f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa240b3cf0 0x7ffa2415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa241511f0 0x7ffa24154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2406e740 con 0x7ffa240b3cf0 2024-09-10T21:33:11.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2406e5c0 con 0x7ffa241511f0 2024-09-10T21:33:11.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2406e8c0 con 0x7ffa2406f180 2024-09-10T21:33:11.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa240b3cf0 0x7ffa2415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa240b3cf0 0x7ffa2415baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55276/0 (socket says 172.21.15.97:55276) 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 -- 172.21.15.97:0/3869804323 learned_addr learned my addr 172.21.15.97:0/3869804323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 -- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa2406f180 msgr2=0x7ffa241f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa2406f180 0x7ffa241f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa2406f180 0x7ffa241f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa23400640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa241511f0 0x7ffa24154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 -- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa241511f0 msgr2=0x7ffa24154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa241511f0 0x7ffa24154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 -- 172.21.15.97:0/3869804323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa24158850 con 0x7ffa240b3cf0 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa2406f180 0x7ffa241f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa23400640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa241511f0 0x7ffa24154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa22000640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa240b3cf0 0x7ffa2415baf0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000ea90 tx=0x7ffa1000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa1aa00640 1 -- 172.21.15.97:0/3869804323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1001b070 con 0x7ffa240b3cf0 2024-09-10T21:33:11.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa1aa00640 1 -- 172.21.15.97:0/3869804323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa1000ce30 con 0x7ffa240b3cf0 2024-09-10T21:33:11.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa241f6780 con 0x7ffa240b3cf0 2024-09-10T21:33:11.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa1aa00640 1 -- 172.21.15.97:0/3869804323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10016650 con 0x7ffa240b3cf0 2024-09-10T21:33:11.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.487+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa241562f0 con 0x7ffa240b3cf0 2024-09-10T21:33:11.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.491+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa240b37b0 con 0x7ffa240b3cf0 2024-09-10T21:33:11.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.491+0000 7ffa1aa00640 1 -- 172.21.15.97:0/3869804323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffa1000b040 con 0x7ffa240b3cf0 2024-09-10T21:33:11.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.491+0000 7ffa1aa00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa040725e0 0x7ffa04074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.491+0000 7ffa1aa00640 1 -- 172.21.15.97:0/3869804323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7995+0+0 (secure 0 0 0) 0x7ffa10093de0 con 0x7ffa240b3cf0 2024-09-10T21:33:11.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.491+0000 7ffa22a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa040725e0 0x7ffa04074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.491+0000 7ffa22a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa040725e0 0x7ffa04074aa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ffa140b2000 tx=0x7ffa1401dda0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.495+0000 7ffa1aa00640 1 -- 172.21.15.97:0/3869804323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffa24158850 con 0x7ffa240b3cf0 2024-09-10T21:33:11.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.691+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ffa24159fe0 con 0x7ffa240b3cf0 2024-09-10T21:33:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa1aa00640 1 -- 172.21.15.97:0/3869804323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v206) ==== 162+0+0 (secure 0 0 0) 0x7ffa1005e570 con 0x7ffa240b3cf0 2024-09-10T21:33:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa040725e0 msgr2=0x7ffa04074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa040725e0 0x7ffa04074aa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ffa140b2000 tx=0x7ffa1401dda0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa240b3cf0 msgr2=0x7ffa2415baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa240b3cf0 0x7ffa2415baf0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000ea90 tx=0x7ffa1000ef50 comp rx=0 tx=0).stop 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 shutdown_connections 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa040725e0 0x7ffa04074aa0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa241511f0 0x7ffa24154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa240b3cf0 0x7ffa2415baf0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 --2- 172.21.15.97:0/3869804323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa2406f180 0x7ffa241f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 >> 172.21.15.97:0/3869804323 conn(0x7ffa240b1780 msgr2=0x7ffa24150530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 shutdown_connections 2024-09-10T21:33:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:11.931+0000 7ffa29a00640 1 -- 172.21.15.97:0/3869804323 wait complete. 2024-09-10T21:33:11.937 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:33:11.939+0000 7f3908800640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:33:11.938 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:11.939+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:11.938 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:11.939+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:11.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:12.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 -- 172.21.15.97:0/3887060716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0af100 msgr2=0x7f753c0af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 --2- 172.21.15.97:0/3887060716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0af100 0x7f753c0af530 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f753c0b07d0 tx=0x7f753401c800 comp rx=0 tx=0).stop 2024-09-10T21:33:12.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 -- 172.21.15.97:0/3887060716 shutdown_connections 2024-09-10T21:33:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 --2- 172.21.15.97:0/3887060716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0af100 0x7f753c0af530 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 --2- 172.21.15.97:0/3887060716 >> v2:172.21.15.194:3300/0 conn(0x7f753c0cfad0 0x7f753c0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 -- 172.21.15.97:0/3887060716 >> 172.21.15.97:0/3887060716 conn(0x7f753c0b28c0 msgr2=0x7f753c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 -- 172.21.15.97:0/3887060716 shutdown_connections 2024-09-10T21:33:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.099+0000 7f7542800640 1 -- 172.21.15.97:0/3887060716 wait complete. 2024-09-10T21:33:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 Processor -- start 2024-09-10T21:33:12.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 -- start start 2024-09-10T21:33:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f753c0af100 0x7f753c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0cfad0 0x7f753c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f753c15bdd0 0x7f753c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f753c168e30 con 0x7f753c0cfad0 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f753c168cb0 con 0x7f753c0af100 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f753c168fb0 con 0x7f753c15bdd0 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0cfad0 0x7f753c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0cfad0 0x7f753c150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55304/0 (socket says 172.21.15.97:55304) 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 -- 172.21.15.97:0/1465475624 learned_addr learned my addr 172.21.15.97:0/1465475624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f753b400640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f753c0af100 0x7f753c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f753be00640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f753c15bdd0 0x7f753c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 -- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f753c15bdd0 msgr2=0x7f753c0707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f753c15bdd0 0x7f753c0707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 -- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f753c0af100 msgr2=0x7f753c0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f753c0af100 0x7f753c0b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 -- 172.21.15.97:0/1465475624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c151900 con 0x7f753c0cfad0 2024-09-10T21:33:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7532a00640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0cfad0 0x7f753c150fb0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f752400e9f0 tx=0x7f752400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7533e00640 1 -- 172.21.15.97:0/1465475624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400cd80 con 0x7f753c0cfad0 2024-09-10T21:33:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7533e00640 1 -- 172.21.15.97:0/1465475624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7524010430 con 0x7f753c0cfad0 2024-09-10T21:33:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7533e00640 1 -- 172.21.15.97:0/1465475624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524010860 con 0x7f753c0cfad0 2024-09-10T21:33:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c151be0 con 0x7f753c0cfad0 2024-09-10T21:33:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1521a0 con 0x7f753c0cfad0 2024-09-10T21:33:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.103+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c0af530 con 0x7f753c0cfad0 2024-09-10T21:33:12.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.107+0000 7f7533e00640 1 -- 172.21.15.97:0/1465475624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7524029080 con 0x7f753c0cfad0 2024-09-10T21:33:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.107+0000 7f7533e00640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75180725e0 0x7f7518074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.107+0000 7f7533e00640 1 -- 172.21.15.97:0/1465475624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7f7524094ac0 con 0x7f753c0cfad0 2024-09-10T21:33:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.107+0000 7f753b400640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75180725e0 0x7f7518074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.107+0000 7f753b400640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75180725e0 0x7f7518074aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f752c0017b0 tx=0x7f752c001740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:12.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.111+0000 7f7533e00640 1 -- 172.21.15.97:0/1465475624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f752405f430 con 0x7f753c0cfad0 2024-09-10T21:33:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f753c1569e0 con 0x7f753c0cfad0 2024-09-10T21:33:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7533e00640 1 -- 172.21.15.97:0/1465475624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) ==== 95+0+3569 (secure 0 0 0) 0x7f7524063e60 con 0x7f753c0cfad0 2024-09-10T21:33:12.309 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":208,"btime":"2024-09-10T21:33:11:941527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:11.941452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:12.310 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 208 2024-09-10T21:33:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75180725e0 msgr2=0x7f7518074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75180725e0 0x7f7518074aa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f752c0017b0 tx=0x7f752c001740 comp rx=0 tx=0).stop 2024-09-10T21:33:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0cfad0 msgr2=0x7f753c150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0cfad0 0x7f753c150fb0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f752400e9f0 tx=0x7f752400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 shutdown_connections 2024-09-10T21:33:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f75180725e0 0x7f7518074aa0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f753c15bdd0 0x7f753c0707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f753c0cfad0 0x7f753c150fb0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 --2- 172.21.15.97:0/1465475624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f753c0af100 0x7f753c0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 >> 172.21.15.97:0/1465475624 conn(0x7f753c0b28c0 msgr2=0x7f753c15a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 shutdown_connections 2024-09-10T21:33:12.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.311+0000 7f7542800640 1 -- 172.21.15.97:0/1465475624 wait complete. 2024-09-10T21:33:12.332 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:12.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:12.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 -- 172.21.15.97:0/1163506499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b1815efc0 msgr2=0x7f2b180cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 --2- 172.21.15.97:0/1163506499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b1815efc0 0x7f2b180cf840 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009840 tx=0x7f2b0c01c820 comp rx=0 tx=0).stop 2024-09-10T21:33:12.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 -- 172.21.15.97:0/1163506499 shutdown_connections 2024-09-10T21:33:12.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 --2- 172.21.15.97:0/1163506499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b180d0490 0x7f2b180ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 --2- 172.21.15.97:0/1163506499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b1815efc0 0x7f2b180cf840 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 -- 172.21.15.97:0/1163506499 >> 172.21.15.97:0/1163506499 conn(0x7f2b180d1a60 msgr2=0x7f2b180d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 -- 172.21.15.97:0/1163506499 shutdown_connections 2024-09-10T21:33:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 -- 172.21.15.97:0/1163506499 wait complete. 2024-09-10T21:33:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 Processor -- start 2024-09-10T21:33:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.475+0000 7f2b1de00640 1 -- start start 2024-09-10T21:33:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b180cdcb0 0x7f2b1815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b180d0490 0x7f2b181f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1815efc0 0x7f2b18154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18168930 con 0x7f2b1815efc0 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181687b0 con 0x7f2b180cdcb0 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b18168ab0 con 0x7f2b180d0490 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1815efc0 0x7f2b18154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b180cdcb0 0x7f2b1815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b180cdcb0 0x7f2b1815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:38980/0 (socket says 172.21.15.97:38980) 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 -- 172.21.15.97:0/3985004789 learned_addr learned my addr 172.21.15.97:0/3985004789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b17e00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b180d0490 0x7f2b181f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 -- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b180d0490 msgr2=0x7f2b181f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b180d0490 0x7f2b181f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 -- 172.21.15.97:0/3985004789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1815efc0 msgr2=0x7f2b18154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1815efc0 0x7f2b18154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 -- 172.21.15.97:0/3985004789 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b181f4e90 con 0x7f2b180cdcb0 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1d400640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1815efc0 0x7f2b18154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1ca00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b180cdcb0 0x7f2b1815c0f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009790 tx=0x7f2b0c005ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b14c00640 1 -- 172.21.15.97:0/3985004789 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0b2040 con 0x7f2b180cdcb0 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b14c00640 1 -- 172.21.15.97:0/3985004789 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b0c004330 con 0x7f2b180cdcb0 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b181f5130 con 0x7f2b180cdcb0 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b14c00640 1 -- 172.21.15.97:0/3985004789 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c003840 con 0x7f2b180cdcb0 2024-09-10T21:33:12.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181f56f0 con 0x7f2b180cdcb0 2024-09-10T21:33:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.479+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b181f4c10 con 0x7f2b180cdcb0 2024-09-10T21:33:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.483+0000 7f2b14c00640 1 -- 172.21.15.97:0/3985004789 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b0c0b5070 con 0x7f2b180cdcb0 2024-09-10T21:33:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.483+0000 7f2b14c00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2af80725e0 0x7f2af8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.483+0000 7f2b14c00640 1 -- 172.21.15.97:0/3985004789 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7f2b0c12d120 con 0x7f2b180cdcb0 2024-09-10T21:33:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.483+0000 7f2b17e00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2af80725e0 0x7f2af8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.483+0000 7f2b17e00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2af80725e0 0x7f2af8074aa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2b1815c630 tx=0x7f2b04001440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:12.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.487+0000 7f2b14c00640 1 -- 172.21.15.97:0/3985004789 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b181f4c10 con 0x7f2b180cdcb0 2024-09-10T21:33:12.652 INFO:tasks.ceph.mds.c.smithi097.stderr: -43> 2024-09-10T21:33:07.899+0000 7fd041e00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:33:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b1815b760 con 0x7f2b180cdcb0 2024-09-10T21:33:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b14c00640 1 -- 172.21.15.97:0/3985004789 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) ==== 95+0+3569 (secure 0 0 0) 0x7f2b0c016410 con 0x7f2b180cdcb0 2024-09-10T21:33:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:12.686 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":208,"btime":"2024-09-10T21:33:11:941527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:11.941452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 208 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2af80725e0 msgr2=0x7f2af8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2af80725e0 0x7f2af8074aa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2b1815c630 tx=0x7f2b04001440 comp rx=0 tx=0).stop 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b180cdcb0 msgr2=0x7f2b1815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b180cdcb0 0x7f2b1815c0f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009790 tx=0x7f2b0c005ca0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 shutdown_connections 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2af80725e0 0x7f2af8074aa0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1815efc0 0x7f2b18154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b180d0490 0x7f2b181f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 --2- 172.21.15.97:0/3985004789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b180cdcb0 0x7f2b1815c0f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 >> 172.21.15.97:0/3985004789 conn(0x7f2b180d1a60 msgr2=0x7f2b180cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 shutdown_connections 2024-09-10T21:33:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.687+0000 7f2b1de00640 1 -- 172.21.15.97:0/3985004789 wait complete. 2024-09-10T21:33:12.689 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:33:12.711 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:12.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:12.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 -- 172.21.15.97:0/2228798324 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 msgr2=0x7f7ee40d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2228798324 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 0x7f7ee40d27f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009840 tx=0x7f7ed401c820 comp rx=0 tx=0).stop 2024-09-10T21:33:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 -- 172.21.15.97:0/2228798324 shutdown_connections 2024-09-10T21:33:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2228798324 >> v2:172.21.15.97:3300/0 conn(0x7f7ee40cdd40 0x7f7ee40cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2228798324 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 0x7f7ee40d27f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 -- 172.21.15.97:0/2228798324 >> 172.21.15.97:0/2228798324 conn(0x7f7ee40d1cd0 msgr2=0x7f7ee40d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 -- 172.21.15.97:0/2228798324 shutdown_connections 2024-09-10T21:33:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.851+0000 7f7eeb000640 1 -- 172.21.15.97:0/2228798324 wait complete. 2024-09-10T21:33:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 Processor -- start 2024-09-10T21:33:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 -- start start 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ee40cca90 0x7f7ee41fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee40cdd40 0x7f7ee414c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 0x7f7ee406f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4168c80 con 0x7f7ee40cdd40 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4168b00 con 0x7f7ee40cca90 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee4168e00 con 0x7f7ee415eed0 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee40cdd40 0x7f7ee414c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee40cdd40 0x7f7ee414c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55336/0 (socket says 172.21.15.97:55336) 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 -- 172.21.15.97:0/2962862203 learned_addr learned my addr 172.21.15.97:0/2962862203 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7ee3e00640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ee40cca90 0x7f7ee41fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7ee9000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 0x7f7ee406f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 -- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 msgr2=0x7f7ee406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 0x7f7ee406f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 -- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ee40cca90 msgr2=0x7f7ee41fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ee40cca90 0x7f7ee41fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 -- 172.21.15.97:0/2962862203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee406eb00 con 0x7f7ee40cdd40 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7ee9000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 0x7f7ee406f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7edb400640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee40cdd40 0x7f7ee414c550 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00ea20 tx=0x7f7ecc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7ee0c00640 1 -- 172.21.15.97:0/2962862203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00cd80 con 0x7f7ee40cdd40 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7ee0c00640 1 -- 172.21.15.97:0/2962862203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ecc010430 con 0x7f7ee40cdd40 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7ee0c00640 1 -- 172.21.15.97:0/2962862203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc010860 con 0x7f7ee40cdd40 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee406ede0 con 0x7f7ee40cdd40 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.855+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee41fb470 con 0x7f7ee40cdd40 2024-09-10T21:33:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.859+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee40cc550 con 0x7f7ee40cdd40 2024-09-10T21:33:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.859+0000 7f7ee0c00640 1 -- 172.21.15.97:0/2962862203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ecc010a00 con 0x7f7ee40cdd40 2024-09-10T21:33:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.859+0000 7f7ee0c00640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ec00725e0 0x7f7ec0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.859+0000 7f7ee0c00640 1 -- 172.21.15.97:0/2962862203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7f7ecc094740 con 0x7f7ee40cdd40 2024-09-10T21:33:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.859+0000 7f7ee3e00640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ec00725e0 0x7f7ec0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:12.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.859+0000 7f7ee3e00640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ec00725e0 0x7f7ec0074aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40af000 tx=0x7f7ed401b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:12.863+0000 7f7ee0c00640 1 -- 172.21.15.97:0/2962862203 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ecc05ede0 con 0x7f7ee40cdd40 2024-09-10T21:33:13.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.063+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ee40cfd60 con 0x7f7ee40cdd40 2024-09-10T21:33:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.063+0000 7f7ee0c00640 1 -- 172.21.15.97:0/2962862203 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+4395 (secure 0 0 0) 0x7f7ecc063810 con 0x7f7ee40cdd40 2024-09-10T21:33:13.064 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:13.064 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":209,"btime":"2024-09-10T21:33:12:943450+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3203242917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3203242917},{"type":"v1","addr":"172.21.15.97:6835","nonce":3203242917}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:11.941452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 209 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ec00725e0 msgr2=0x7f7ec0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ec00725e0 0x7f7ec0074aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40af000 tx=0x7f7ed401b060 comp rx=0 tx=0).stop 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee40cdd40 msgr2=0x7f7ee414c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee40cdd40 0x7f7ee414c550 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00ea20 tx=0x7f7ecc00eee0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 shutdown_connections 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ec00725e0 0x7f7ec0074aa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7ee415eed0 0x7f7ee406f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee40cdd40 0x7f7ee414c550 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 --2- 172.21.15.97:0/2962862203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ee40cca90 0x7f7ee41fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 >> 172.21.15.97:0/2962862203 conn(0x7f7ee40d1cd0 msgr2=0x7f7ee4156810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 shutdown_connections 2024-09-10T21:33:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.067+0000 7f7eeb000640 1 -- 172.21.15.97:0/2962862203 wait complete. 2024-09-10T21:33:13.085 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2024-09-10T21:33:13.086 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2024-09-10T21:33:13.086 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:13.086 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:13.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:13.095 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:13.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:13.095 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:13.147 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:13.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:13.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- 172.21.15.97:0/863769100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff7015fde0 msgr2=0x7eff7013f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 --2- 172.21.15.97:0/863769100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff7015fde0 0x7eff7013f9b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7eff68009bc0 tx=0x7eff6801cba0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- 172.21.15.97:0/863769100 shutdown_connections 2024-09-10T21:33:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 --2- 172.21.15.97:0/863769100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff7015fde0 0x7eff7013f9b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 --2- 172.21.15.97:0/863769100 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff700ce5f0 0x7eff70141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- 172.21.15.97:0/863769100 >> 172.21.15.97:0/863769100 conn(0x7eff700b1780 msgr2=0x7eff700b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- 172.21.15.97:0/863769100 shutdown_connections 2024-09-10T21:33:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- 172.21.15.97:0/863769100 wait complete. 2024-09-10T21:33:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 Processor -- start 2024-09-10T21:33:13.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- start start 2024-09-10T21:33:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff700ce5f0 0x7eff7015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff700ce9d0 0x7eff70154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff7015fde0 0x7eff7015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff70168bf0 con 0x7eff700ce5f0 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff70168a70 con 0x7eff700ce9d0 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.335+0000 7eff75000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7eff70168d70 con 0x7eff7015fde0 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff700ce5f0 0x7eff7015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff700ce5f0 0x7eff7015ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55338/0 (socket says 172.21.15.97:55338) 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 -- 172.21.15.97:0/2758968213 learned_addr learned my addr 172.21.15.97:0/2758968213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff65400640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff700ce9d0 0x7eff70154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6e800640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff7015fde0 0x7eff7015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 -- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff7015fde0 msgr2=0x7eff7015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff7015fde0 0x7eff7015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 -- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff700ce9d0 msgr2=0x7eff70154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff700ce9d0 0x7eff70154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 -- 172.21.15.97:0/2758968213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff701fb5c0 con 0x7eff700ce5f0 2024-09-10T21:33:13.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff65400640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff700ce9d0 0x7eff70154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:13.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff6de00640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff700ce5f0 0x7eff7015ee20 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9f0 tx=0x7eff6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:13.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff66a00640 1 -- 172.21.15.97:0/2758968213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000cd20 con 0x7eff700ce5f0 2024-09-10T21:33:13.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff66a00640 1 -- 172.21.15.97:0/2758968213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff60010430 con 0x7eff700ce5f0 2024-09-10T21:33:13.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7015b0c0 con 0x7eff700ce5f0 2024-09-10T21:33:13.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff66a00640 1 -- 172.21.15.97:0/2758968213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60010860 con 0x7eff700ce5f0 2024-09-10T21:33:13.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7015e340 con 0x7eff700ce5f0 2024-09-10T21:33:13.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff70141d20 con 0x7eff700ce5f0 2024-09-10T21:33:13.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.339+0000 7eff66a00640 1 -- 172.21.15.97:0/2758968213 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff60010a00 con 0x7eff700ce5f0 2024-09-10T21:33:13.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.343+0000 7eff66a00640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff4c072610 0x7eff4c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:13.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.343+0000 7eff66a00640 1 -- 172.21.15.97:0/2758968213 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7eff60094680 con 0x7eff700ce5f0 2024-09-10T21:33:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.343+0000 7eff65400640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff4c072610 0x7eff4c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:13.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.343+0000 7eff65400640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff4c072610 0x7eff4c074ad0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7eff58007f30 tx=0x7eff58007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:13.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.347+0000 7eff66a00640 1 -- 172.21.15.97:0/2758968213 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff6005ee70 con 0x7eff700ce5f0 2024-09-10T21:33:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.543+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7eff70141360 con 0x7eff700ce5f0 2024-09-10T21:33:13.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.947+0000 7eff66a00640 1 -- 172.21.15.97:0/2758968213 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v210) ==== 161+0+0 (secure 0 0 0) 0x7eff600638a0 con 0x7eff700ce5f0 2024-09-10T21:33:13.947 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff4c072610 msgr2=0x7eff4c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff4c072610 0x7eff4c074ad0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7eff58007f30 tx=0x7eff58007ec0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff700ce5f0 msgr2=0x7eff7015ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff700ce5f0 0x7eff7015ee20 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9f0 tx=0x7eff6000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 shutdown_connections 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff4c072610 0x7eff4c074ad0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff7015fde0 0x7eff7015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff700ce9d0 0x7eff70154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 --2- 172.21.15.97:0/2758968213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff700ce5f0 0x7eff7015ee20 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 >> 172.21.15.97:0/2758968213 conn(0x7eff700b1780 msgr2=0x7eff7013ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 shutdown_connections 2024-09-10T21:33:13.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:13.951+0000 7eff75000640 1 -- 172.21.15.97:0/2758968213 wait complete. 2024-09-10T21:33:13.970 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.111+0000 7f42dae00640 1 -- 172.21.15.97:0/3398271778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 msgr2=0x7f42d40d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.111+0000 7f42dae00640 1 --2- 172.21.15.97:0/3398271778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 0x7f42d40d27f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009840 tx=0x7f42c401c820 comp rx=0 tx=0).stop 2024-09-10T21:33:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- 172.21.15.97:0/3398271778 shutdown_connections 2024-09-10T21:33:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 --2- 172.21.15.97:0/3398271778 >> v2:172.21.15.194:3300/0 conn(0x7f42d40cdd40 0x7f42d40cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 --2- 172.21.15.97:0/3398271778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 0x7f42d40d27f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- 172.21.15.97:0/3398271778 >> 172.21.15.97:0/3398271778 conn(0x7f42d40d1cd0 msgr2=0x7f42d40d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- 172.21.15.97:0/3398271778 shutdown_connections 2024-09-10T21:33:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- 172.21.15.97:0/3398271778 wait complete. 2024-09-10T21:33:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 Processor -- start 2024-09-10T21:33:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- start start 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42d40cca90 0x7f42d40b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42d40cdd40 0x7f42d414ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 0x7f42d41f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4168c80 con 0x7f42d40cca90 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4168b00 con 0x7f42d40cdd40 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42dae00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f42d4168e00 con 0x7f42d415eed0 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42d40cca90 0x7f42d40b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42d40cca90 0x7f42d40b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55346/0 (socket says 172.21.15.97:55346) 2024-09-10T21:33:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 -- 172.21.15.97:0/356321672 learned_addr learned my addr 172.21.15.97:0/356321672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3400640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42d40cdd40 0x7f42d414ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 -- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 msgr2=0x7f42d41f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d8e00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 0x7f42d41f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 0x7f42d41f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 -- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42d40cdd40 msgr2=0x7f42d414ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42d40cdd40 0x7f42d414ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 -- 172.21.15.97:0/356321672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d41f9ab0 con 0x7f42d40cca90 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3400640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42d40cdd40 0x7f42d414ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:14.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42d3e00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42d40cca90 0x7f42d40b29e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009790 tx=0x7f42c40a6500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42cbe00640 1 -- 172.21.15.97:0/356321672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c40a66f0 con 0x7f42d40cca90 2024-09-10T21:33:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42cbe00640 1 -- 172.21.15.97:0/356321672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42c4004130 con 0x7f42d40cca90 2024-09-10T21:33:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.115+0000 7f42cbe00640 1 -- 172.21.15.97:0/356321672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4003600 con 0x7f42d40cca90 2024-09-10T21:33:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.119+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d40b3080 con 0x7f42d40cca90 2024-09-10T21:33:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.119+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d40b3640 con 0x7f42d40cca90 2024-09-10T21:33:14.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.119+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d40cc550 con 0x7f42d40cca90 2024-09-10T21:33:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.119+0000 7f42cbe00640 1 -- 172.21.15.97:0/356321672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f42c40037a0 con 0x7f42d40cca90 2024-09-10T21:33:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.119+0000 7f42cbe00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f42b40727b0 0x7f42b4074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.119+0000 7f42cbe00640 1 -- 172.21.15.97:0/356321672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7f42c412e8e0 con 0x7f42d40cca90 2024-09-10T21:33:14.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.119+0000 7f42d3400640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f42b40727b0 0x7f42b4074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:14.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.123+0000 7f42d3400640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f42b40727b0 0x7f42b4074c70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f42d41500d0 tx=0x7f42c00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:14.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.123+0000 7f42cbe00640 1 -- 172.21.15.97:0/356321672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f42c40c8020 con 0x7f42d40cca90 2024-09-10T21:33:14.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.351+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f42d40cfd60 con 0x7f42d40cca90 2024-09-10T21:33:14.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42cbe00640 1 -- 172.21.15.97:0/356321672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v211) ==== 101+0+0 (secure 0 0 0) 0x7f42d41f9ab0 con 0x7f42d40cca90 2024-09-10T21:33:14.957 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f42b40727b0 msgr2=0x7f42b4074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f42b40727b0 0x7f42b4074c70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f42d41500d0 tx=0x7f42c00091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42d40cca90 msgr2=0x7f42d40b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42d40cca90 0x7f42d40b29e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009790 tx=0x7f42c40a6500 comp rx=0 tx=0).stop 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 shutdown_connections 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f42b40727b0 0x7f42b4074c70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f42d415eed0 0x7f42d41f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42d40cdd40 0x7f42d414ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 --2- 172.21.15.97:0/356321672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42d40cca90 0x7f42d40b29e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:14.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 >> 172.21.15.97:0/356321672 conn(0x7f42d40d1cd0 msgr2=0x7f42d415ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:14.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 shutdown_connections 2024-09-10T21:33:14.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:14.959+0000 7f42dae00640 1 -- 172.21.15.97:0/356321672 wait complete. 2024-09-10T21:33:14.979 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:15.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.023+0000 7f0315b70080 1 Processor -- start 2024-09-10T21:33:15.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- start start 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6cfe7940 0x561c6cfeab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6d071b00 0x561c6cfeb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d0722a0 0x561c6d124770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561c6cf06a00 con 0x561c6d071b00 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c6cf64f40 con 0x561c6cfe7940 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561c6d074080 con 0x561c6d0722a0 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6d071b00 0x561c6cfeb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6d071b00 0x561c6cfeb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55354/0 (socket says 172.21.15.97:55354) 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 -- 172.21.15.97:0/2262000968 learned_addr learned my addr 172.21.15.97:0/2262000968 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312e00640 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6cfe7940 0x561c6cfeab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0313800640 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d0722a0 0x561c6d124770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 -- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d0722a0 msgr2=0x561c6d124770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d0722a0 0x561c6d124770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 -- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6cfe7940 msgr2=0x561c6cfeab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6cfe7940 0x561c6cfeab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 -- 172.21.15.97:0/2262000968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6cedfc10 con 0x561c6d071b00 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0313800640 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d0722a0 0x561c6d124770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312e00640 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6cfe7940 0x561c6cfeab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:15.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0312400640 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6d071b00 0x561c6cfeb060 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f030000b5b0 tx=0x7f030000ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45797d81408c653 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0311000640 1 -- 172.21.15.97:0/2262000968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300009a80 con 0x561c6d071b00 2024-09-10T21:33:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0311000640 1 -- 172.21.15.97:0/2262000968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0300009c20 con 0x561c6d071b00 2024-09-10T21:33:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0311000640 1 -- 172.21.15.97:0/2262000968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300014440 con 0x561c6d071b00 2024-09-10T21:33:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- 172.21.15.97:0/2262000968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6d071b00 msgr2=0x561c6cfeb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6d071b00 0x561c6cfeb060 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f030000b5b0 tx=0x7f030000ba70 comp rx=0 tx=0).stop 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- 172.21.15.97:0/2262000968 shutdown_connections 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d0722a0 0x561c6d124770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6d071b00 0x561c6cfeb060 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 --2- 172.21.15.97:0/2262000968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6cfe7940 0x561c6cfeab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- 172.21.15.97:0/2262000968 >> 172.21.15.97:0/2262000968 conn(0x561c6cfe6520 msgr2=0x561c6d08e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- 172.21.15.97:0/2262000968 shutdown_connections 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- 172.21.15.97:0/2262000968 wait complete. 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 Processor -- start 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.027+0000 7f0315b70080 1 -- start start 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6cfe7940 0x561c6d070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d071b00 0x561c6d0772b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6d0722a0 0x561c6d1287b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561c6d08f0d0 con 0x561c6cfe7940 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c6d078020 con 0x561c6d0722a0 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x561c6d0782c0 con 0x561c6d071b00 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6cfe7940 0x561c6d070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6cfe7940 0x561c6d070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55368/0 (socket says 172.21.15.97:55368) 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 learned_addr learned my addr 172.21.15.97:0/1618137587 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d071b00 0x561c6d0772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0313800640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6d0722a0 0x561c6d1287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d071b00 msgr2=0x561c6d0772b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x561c6d071b00 0x561c6d0772b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6d0722a0 msgr2=0x561c6d1287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c6d0722a0 0x561c6d1287b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6d070f70 con 0x561c6cfe7940 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0312e00640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6cfe7940 0x561c6d070a30 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0304019790 tx=0x7f03040aca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f030b400640 1 -- 172.21.15.97:0/1618137587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040ba070 con 0x561c6cfe7940 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f030b400640 1 -- 172.21.15.97:0/1618137587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03040b63f0 con 0x561c6cfe7940 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f030b400640 1 -- 172.21.15.97:0/1618137587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040b0040 con 0x561c6cfe7940 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561c6d076630 con 0x561c6cfe7940 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.031+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6d076b70 con 0x561c6cfe7940 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f030b400640 1 -- 172.21.15.97:0/1618137587 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f03040b67c0 con 0x561c6cfe7940 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f030b400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02f4070ff0 0x7f02f40734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f030b400640 1 -- 172.21.15.97:0/1618137587 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7f030412dc30 con 0x561c6cfe7940 2024-09-10T21:33:15.034 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:15.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561c6cfeab20 0x561c6cfeaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d090150 con 0x561c6cfeab20 2024-09-10T21:33:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f0312400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02f4070ff0 0x7f02f40734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f0313800640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561c6cfeab20 0x561c6cfeaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f0312400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02f4070ff0 0x7f02f40734b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0300000c10 tx=0x7f0300016040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.035+0000 7f0313800640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561c6cfeab20 0x561c6cfeaf00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.039+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v117'629 uv603 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.039+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d090520 con 0x561c6cfeab20 2024-09-10T21:33:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.039+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v117'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.039+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d0908c0 con 0x561c6cfeab20 2024-09-10T21:33:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.039+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v117'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.040 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:15.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.039+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135ea0 con 0x561c6cfeab20 2024-09-10T21:33:15.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.043+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.043+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d136240 con 0x561c6cfeab20 2024-09-10T21:33:15.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.043+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.043+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d134e90 con 0x561c6cfeab20 2024-09-10T21:33:15.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.047+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.047+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1352b0 con 0x561c6cfeab20 2024-09-10T21:33:15.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.047+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v117'635 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.047+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d1356a0 con 0x561c6cfeab20 2024-09-10T21:33:15.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.051+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v117'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.051+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1356a0 con 0x561c6cfeab20 2024-09-10T21:33:15.051 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:15.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.051+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v117'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.051+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.055+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'638 uv638 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.055+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.055+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'639 uv639 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.055+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.055+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'640 uv640 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.055+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.059+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v117'641 uv616 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.059+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.059+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'642 uv642 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.059+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.061 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.063+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v117'643 uv643 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.063+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.063+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'644 uv644 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.063+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.067+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'645 uv645 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.067+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d135650 con 0x561c6cfeab20 2024-09-10T21:33:15.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.067+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'646 uv646 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.067+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.067+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'647 uv647 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.067+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.071+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v117'648 uv622 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.071+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.071+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v117'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.071+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.073 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:15.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.075+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v117'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.075+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1375f0 con 0x561c6cfeab20 2024-09-10T21:33:15.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.075+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.075+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1375f0 con 0x561c6cfeab20 2024-09-10T21:33:15.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.079+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.079+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.079+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.079+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.079+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v117'654 uv628 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.079+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.083+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v117'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.083+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.082 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:15.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.083+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v117'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.083+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1375f0 con 0x561c6cfeab20 2024-09-10T21:33:15.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.087+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.087+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1375f0 con 0x561c6cfeab20 2024-09-10T21:33:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.087+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.087+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1365e0 con 0x561c6cfeab20 2024-09-10T21:33:15.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.087+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'659 uv659 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6cfeab20 2024-09-10T21:33:15.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.087+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561c6d136af0 0x561c6d138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.087+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d139420 con 0x561c6d136af0 2024-09-10T21:33:15.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.091+0000 7f0312e00640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561c6d136af0 0x561c6d138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.091+0000 7f0312e00640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561c6d136af0 0x561c6d138ee0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.091+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'463 uv446 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.091+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d1397c0 con 0x561c6d136af0 2024-09-10T21:33:15.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.091+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'464 uv464 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.091+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d139b60 con 0x561c6d136af0 2024-09-10T21:33:15.093 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:15.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.095+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'465 uv465 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.095+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13a3b0 con 0x561c6d136af0 2024-09-10T21:33:15.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.095+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v117'466 uv466 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.095+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13a750 con 0x561c6d136af0 2024-09-10T21:33:15.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.099+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'467 uv467 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.099+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13aaf0 con 0x561c6d136af0 2024-09-10T21:33:15.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.099+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v117'468 uv451 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.099+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d13ae90 con 0x561c6d136af0 2024-09-10T21:33:15.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.099+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v117'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.099+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.101 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:15.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.103+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v117'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.103+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.103+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.103+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.107+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.107+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.107+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v117'473 uv456 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.107+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.107+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v117'474 uv474 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.107+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.108 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:15.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.111+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v117'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.111+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13f400 con 0x561c6d136af0 2024-09-10T21:33:15.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.111+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v117'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.111+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13f400 con 0x561c6d136af0 2024-09-10T21:33:15.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.111+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.111+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.115+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v117'478 uv462 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.115+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.115+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v117'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.115+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.116 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:15.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.115+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v117'480 uv480 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.115+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13c240 con 0x561c6d136af0 2024-09-10T21:33:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.119+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.119+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13c240 con 0x561c6d136af0 2024-09-10T21:33:15.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.119+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.119+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13b230 con 0x561c6d136af0 2024-09-10T21:33:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d136af0 2024-09-10T21:33:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561c6d13b8b0 0x561c6d13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13e2b0 con 0x561c6d13b8b0 2024-09-10T21:33:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0312400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561c6d13b8b0 0x561c6d13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0312400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561c6d13b8b0 0x561c6d13dca0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d13e770 con 0x561c6d13b8b0 2024-09-10T21:33:15.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.123+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13ec50 con 0x561c6d13b8b0 2024-09-10T21:33:15.125 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:15.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.127+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.127+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d140500 con 0x561c6d13b8b0 2024-09-10T21:33:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.127+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.127+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d1408a0 con 0x561c6d13b8b0 2024-09-10T21:33:15.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.127+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.127+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d13f4f0 con 0x561c6d13b8b0 2024-09-10T21:33:15.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.131+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.131+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561c6d13fdf0 0x561c6d142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.131+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143190 con 0x561c6d13fdf0 2024-09-10T21:33:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.131+0000 7f0313800640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561c6d13fdf0 0x561c6d142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.131+0000 7f0313800640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561c6d13fdf0 0x561c6d142c50 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.131+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'95 uv94 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.131+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d143190 con 0x561c6d13fdf0 2024-09-10T21:33:15.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'96 uv96 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.133 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:15.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'97 uv97 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'98 uv98 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13b8b0 2024-09-10T21:33:15.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.135+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13b8b0 2024-09-10T21:33:15.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.139+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.139+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13b8b0 2024-09-10T21:33:15.138 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:15.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.139+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.139+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d144590 con 0x561c6d13b8b0 2024-09-10T21:33:15.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.139+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.139+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d144590 con 0x561c6d13b8b0 2024-09-10T21:33:15.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.143+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.143+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13b8b0 2024-09-10T21:33:15.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.143+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.143+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.143+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v117'227 uv221 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.143+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.147+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v117'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.147+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.146 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:15.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.147+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v117'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.147+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d144590 con 0x561c6d13fdf0 2024-09-10T21:33:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.147+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.147+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d144590 con 0x561c6d13fdf0 2024-09-10T21:33:15.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.151+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.151+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.151+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.151+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.151+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v117'233 uv226 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.151+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.155+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v117'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.155+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.154 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.155+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v117'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.155+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.155+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.155+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13fdf0 2024-09-10T21:33:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.159+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.159+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13b8b0 2024-09-10T21:33:15.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.159+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v117'102 uv101 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.159+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13b8b0 2024-09-10T21:33:15.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.159+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v117'103 uv103 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.159+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143580 con 0x561c6d13b8b0 2024-09-10T21:33:15.160 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v117'104 uv104 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143970 con 0x561c6d13b8b0 2024-09-10T21:33:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d143970 con 0x561c6d13b8b0 2024-09-10T21:33:15.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d13b8b0 2024-09-10T21:33:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561c6d143cb0 0x561c6d146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146780 con 0x561c6d143cb0 2024-09-10T21:33:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0312e00640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561c6d143cb0 0x561c6d146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.163+0000 7f0312e00640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561c6d143cb0 0x561c6d146100 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d146840 con 0x561c6d143cb0 2024-09-10T21:33:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.167 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147c70 con 0x561c6d143cb0 2024-09-10T21:33:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.167+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147c70 con 0x561c6d143cb0 2024-09-10T21:33:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.171+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.171+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.171+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.171+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.171+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v117'203 uv202 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.171+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.175+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v117'204 uv204 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.175+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.174 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.175+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v117'205 uv205 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.175+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.175+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'206 uv206 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.175+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.179+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'207 uv207 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.179+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.179+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v117'208 uv201 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.179+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.179+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v117'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.179+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.180 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.183+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v117'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.183+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d146c60 con 0x561c6d143cb0 2024-09-10T21:33:15.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.183+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.183+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149a70 con 0x561c6d143cb0 2024-09-10T21:33:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.183+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.183+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.187+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v117'91 uv90 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.187+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.187+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'92 uv92 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.187+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.187 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:15.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.187+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v117'93 uv93 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.187+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'94 uv94 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v117'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v117'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.192 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:15.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v117'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.191+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d148120 con 0x561c6d13fdf0 2024-09-10T21:33:15.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d148120 con 0x561c6d13fdf0 2024-09-10T21:33:15.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d147110 con 0x561c6d13fdf0 2024-09-10T21:33:15.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f030c010630 con 0x561c6d13fdf0 2024-09-10T21:33:15.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561c6d147790 0x561c6d14a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d147790 2024-09-10T21:33:15.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.195+0000 7f0312400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561c6d147790 0x561c6d14a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.199+0000 7f0312400640 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561c6d147790 0x561c6d14a070 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.199+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v117'104 uv103 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d147790 2024-09-10T21:33:15.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.199+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d147790 2024-09-10T21:33:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.199+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v117'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d147790 2024-09-10T21:33:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.199+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d147790 2024-09-10T21:33:15.200 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.199+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v117'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d147790 2024-09-10T21:33:15.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.199+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d147790 2024-09-10T21:33:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.203+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d147790 2024-09-10T21:33:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.203+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d147790 2024-09-10T21:33:15.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.203+0000 7f0312400640 1 -- 172.21.15.97:0/1618137587 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0300009ae0 con 0x561c6d147790 2024-09-10T21:33:15.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.203+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d143cb0 2024-09-10T21:33:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.203+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v117'253 uv245 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.203+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d143cb0 2024-09-10T21:33:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.207+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v117'254 uv254 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.207+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d143cb0 2024-09-10T21:33:15.206 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.207+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v117'255 uv255 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.207+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d14a5b0 con 0x561c6d143cb0 2024-09-10T21:33:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.207+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v117'256 uv256 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.207+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d14a5b0 con 0x561c6d143cb0 2024-09-10T21:33:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.211+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'257 uv257 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.211+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d14a5b0 con 0x561c6d143cb0 2024-09-10T21:33:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.211+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v117'258 uv252 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.211+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d143cb0 2024-09-10T21:33:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.211+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v117'259 uv259 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.211+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d149b80 con 0x561c6d143cb0 2024-09-10T21:33:15.212 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v117'260 uv260 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d14b5c0 con 0x561c6d143cb0 2024-09-10T21:33:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'261 uv261 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d14b5c0 con 0x561c6d143cb0 2024-09-10T21:33:15.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'262 uv262 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d14a5b0 con 0x561c6d143cb0 2024-09-10T21:33:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'263 uv263 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.215+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x561c6d14a5b0 con 0x561c6d143cb0 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0312e00640 1 -- 172.21.15.97:0/1618137587 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'264 uv264 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f03040c6070 con 0x561c6d143cb0 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561c6d13b8b0 msgr2=0x561c6d13dca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x561c6d13b8b0 0x561c6d13dca0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561c6d143cb0 msgr2=0x561c6d146100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x561c6d143cb0 0x561c6d146100 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561c6d147790 msgr2=0x561c6d14a070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x561c6d147790 0x561c6d14a070 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561c6d13fdf0 msgr2=0x561c6d142c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x561c6d13fdf0 0x561c6d142c50 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561c6d136af0 msgr2=0x561c6d138ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 reap_dead start 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561c6d136af0 0x561c6d138ee0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0313800640 1 -- 172.21.15.97:0/1618137587 reap_dead start 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561c6cfeab20 msgr2=0x561c6cfeaf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561c6cfeab20 0x561c6cfeaf00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02f4070ff0 msgr2=0x7f02f40734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02f4070ff0 0x7f02f40734b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0300000c10 tx=0x7f0300016040 comp rx=0 tx=0).stop 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6cfe7940 msgr2=0x561c6d070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6cfe7940 0x561c6d070a30 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0304019790 tx=0x7f03040aca20 comp rx=0 tx=0).stop 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 shutdown_connections 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x561c6d136af0 0x561c6d138ee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x561c6cfeab20 0x561c6cfeaf00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f02f4070ff0 0x7f02f40734b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 --2- 172.21.15.97:0/1618137587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c6cfe7940 0x561c6d070a30 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 >> 172.21.15.97:0/1618137587 conn(0x561c6cfe6520 msgr2=0x561c6d08f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 shutdown_connections 2024-09-10T21:33:15.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.219+0000 7f0315b70080 1 -- 172.21.15.97:0/1618137587 wait complete. 2024-09-10T21:33:15.222 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2024-09-10T21:33:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 Processor -- start 2024-09-10T21:33:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 -- start start 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b689e7e60 0x560b689cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 0x560b68a571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b68a5b0f0 0x560b689e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560b688eca00 con 0x560b689e7e60 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560b6894af40 con 0x560b68a5b0f0 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b04248080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560b68a81790 con 0x560b68a5ad10 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b689e7e60 0x560b689cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b689e7e60 0x560b689cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55384/0 (socket says 172.21.15.97:55384) 2024-09-10T21:33:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 -- 172.21.15.97:0/3299843109 learned_addr learned my addr 172.21.15.97:0/3299843109 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b01200640 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b68a5b0f0 0x560b689e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2affe00640 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 0x560b68a571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 -- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 msgr2=0x560b68a571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 0x560b68a571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 -- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b68a5b0f0 msgr2=0x560b689e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b68a5b0f0 0x560b689e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.311+0000 7f2b00800640 1 -- 172.21.15.97:0/3299843109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b688c5c10 con 0x560b689e7e60 2024-09-10T21:33:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b00800640 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b689e7e60 0x560b689cbe60 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2af4004ea0 tx=0x7f2af4017ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e26128c928098df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2afea00640 1 -- 172.21.15.97:0/3299843109 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af40ac910 con 0x560b689e7e60 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2afea00640 1 -- 172.21.15.97:0/3299843109 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af40acba0 con 0x560b689e7e60 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2afea00640 1 -- 172.21.15.97:0/3299843109 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af40acdf0 con 0x560b689e7e60 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- 172.21.15.97:0/3299843109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b689e7e60 msgr2=0x560b689cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b689e7e60 0x560b689cbe60 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2af4004ea0 tx=0x7f2af4017ef0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- 172.21.15.97:0/3299843109 shutdown_connections 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b68a5b0f0 0x560b689e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 0x560b68a571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 --2- 172.21.15.97:0/3299843109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b689e7e60 0x560b689cbe60 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- 172.21.15.97:0/3299843109 >> 172.21.15.97:0/3299843109 conn(0x560b6899d140 msgr2=0x560b68a59cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- 172.21.15.97:0/3299843109 shutdown_connections 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- 172.21.15.97:0/3299843109 wait complete. 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 Processor -- start 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- start start 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b689e7e60 0x560b68a7bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 0x560b689e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b68a5b0f0 0x560b689e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560b68a74ff0 con 0x560b68a5b0f0 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560b689cb990 con 0x560b689e7e60 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560b68a75290 con 0x560b68a5ad10 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b68a5b0f0 0x560b689e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b68a5b0f0 0x560b689e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55400/0 (socket says 172.21.15.97:55400) 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 -- 172.21.15.97:0/2700131287 learned_addr learned my addr 172.21.15.97:0/2700131287 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2affe00640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 0x560b689e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b00800640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b689e7e60 0x560b68a7bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 -- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 msgr2=0x560b689e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560b68a5ad10 0x560b689e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 -- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b689e7e60 msgr2=0x560b68a7bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560b689e7e60 0x560b68a7bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 -- 172.21.15.97:0/2700131287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b68a7e900 con 0x560b68a5b0f0 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b01200640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b68a5b0f0 0x560b689e6a10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006520 tx=0x7f2af800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2afd600640 1 -- 172.21.15.97:0/2700131287 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8006e20 con 0x560b68a5b0f0 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2afd600640 1 -- 172.21.15.97:0/2700131287 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af8018070 con 0x560b68a5b0f0 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2afd600640 1 -- 172.21.15.97:0/2700131287 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8012830 con 0x560b68a5b0f0 2024-09-10T21:33:15.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.315+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560b68a7eba0 con 0x560b68a5b0f0 2024-09-10T21:33:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b68a7f0e0 con 0x560b68a5b0f0 2024-09-10T21:33:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2afd600640 1 -- 172.21.15.97:0/2700131287 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2af80129d0 con 0x560b68a5b0f0 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2afd600640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ae0070ff0 0x7f2ae00734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2afd600640 1 -- 172.21.15.97:0/2700131287 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7f2af8092110 con 0x560b68a5b0f0 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2b00800640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ae0070ff0 0x7f2ae00734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2b04248080 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560b68b11320 0x560b68b11700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x560b68b11c40 con 0x560b68b11320 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2affe00640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560b68b11320 0x560b68b11700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.319+0000 7f2b00800640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ae0070ff0 0x7f2ae00734b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2af40bb000 tx=0x7f2af4004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2affe00640 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560b68b11320 0x560b68b11700 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2affe00640 1 -- 172.21.15.97:0/2700131287 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v117'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2aec00c7b0 con 0x560b68b11320 2024-09-10T21:33:15.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560b68b11320 msgr2=0x560b68b11700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560b68b11320 0x560b68b11700 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ae0070ff0 msgr2=0x7f2ae00734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2ae0070ff0 0x7f2ae00734b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2af40bb000 tx=0x7f2af4004890 comp rx=0 tx=0).stop 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b68a5b0f0 msgr2=0x560b689e6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 --2- 172.21.15.97:0/2700131287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b68a5b0f0 0x560b689e6a10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006520 tx=0x7f2af800d820 comp rx=0 tx=0).stop 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b01200640 1 -- 172.21.15.97:0/2700131287 reap_dead start 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 shutdown_connections 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 >> 172.21.15.97:0/2700131287 conn(0x560b6899d140 msgr2=0x560b68a56770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 shutdown_connections 2024-09-10T21:33:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.323+0000 7f2b04248080 1 -- 172.21.15.97:0/2700131287 wait complete. 2024-09-10T21:33:15.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 -- 172.21.15.97:0/3924900943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c143f90 msgr2=0x7f6e7c0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 --2- 172.21.15.97:0/3924900943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c143f90 0x7f6e7c0b2a80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c15e650 tx=0x7f6e6c01c800 comp rx=0 tx=0).stop 2024-09-10T21:33:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 -- 172.21.15.97:0/3924900943 shutdown_connections 2024-09-10T21:33:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 --2- 172.21.15.97:0/3924900943 >> v2:172.21.15.194:3300/0 conn(0x7f6e7c144690 0x7f6e7c0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 --2- 172.21.15.97:0/3924900943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c143f90 0x7f6e7c0b2a80 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 -- 172.21.15.97:0/3924900943 >> 172.21.15.97:0/3924900943 conn(0x7f6e7c0b1780 msgr2=0x7f6e7c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 -- 172.21.15.97:0/3924900943 shutdown_connections 2024-09-10T21:33:15.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.511+0000 7f6e82e00640 1 -- 172.21.15.97:0/3924900943 wait complete. 2024-09-10T21:33:15.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 Processor -- start 2024-09-10T21:33:15.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 -- start start 2024-09-10T21:33:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e7c0b4660 0x7f6e7c15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6e7c143f90 0x7f6e7c13a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c144690 0x7f6e7c13ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1679e0 con 0x7f6e7c144690 2024-09-10T21:33:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c167860 con 0x7f6e7c0b4660 2024-09-10T21:33:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7c167b60 con 0x7f6e7c143f90 2024-09-10T21:33:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c144690 0x7f6e7c13ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e7be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e7c0b4660 0x7f6e7c15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e7b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6e7c143f90 0x7f6e7c13a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c144690 0x7f6e7c13ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55414/0 (socket says 172.21.15.97:55414) 2024-09-10T21:33:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e7b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6e7c143f90 0x7f6e7c13a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:50376/0 (socket says 172.21.15.97:50376) 2024-09-10T21:33:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 -- 172.21.15.97:0/87252773 learned_addr learned my addr 172.21.15.97:0/87252773 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e7be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e7c0b4660 0x7f6e7c15a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:39104/0 (socket says 172.21.15.97:39104) 2024-09-10T21:33:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 -- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6e7c143f90 msgr2=0x7f6e7c13a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6e7c143f90 0x7f6e7c13a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 -- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e7c0b4660 msgr2=0x7f6e7c15a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e7c0b4660 0x7f6e7c15a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 -- 172.21.15.97:0/87252773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c1f7800 con 0x7f6e7c144690 2024-09-10T21:33:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e80e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c144690 0x7f6e7c13ac70 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400cbc0 tx=0x7f6e7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e73e00640 1 -- 172.21.15.97:0/87252773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74016020 con 0x7f6e7c144690 2024-09-10T21:33:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e73e00640 1 -- 172.21.15.97:0/87252773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e7400ab00 con 0x7f6e7c144690 2024-09-10T21:33:15.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e73e00640 1 -- 172.21.15.97:0/87252773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74012420 con 0x7f6e7c144690 2024-09-10T21:33:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7c1f7ac0 con 0x7f6e7c144690 2024-09-10T21:33:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.515+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7c1f8080 con 0x7f6e7c144690 2024-09-10T21:33:15.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.519+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7c0b2a80 con 0x7f6e7c144690 2024-09-10T21:33:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.519+0000 7f6e73e00640 1 -- 172.21.15.97:0/87252773 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6e740125c0 con 0x7f6e7c144690 2024-09-10T21:33:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.519+0000 7f6e73e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6e5c0725e0 0x7f6e5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.519+0000 7f6e73e00640 1 -- 172.21.15.97:0/87252773 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8081+0+0 (secure 0 0 0) 0x7f6e74093fd0 con 0x7f6e7c144690 2024-09-10T21:33:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.519+0000 7f6e7be00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6e5c0725e0 0x7f6e5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:15.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.519+0000 7f6e7be00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6e5c0725e0 0x7f6e5c074aa0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0af000 tx=0x7f6e6c01b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:15.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.523+0000 7f6e73e00640 1 -- 172.21.15.97:0/87252773 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6e74020020 con 0x7f6e7c144690 2024-09-10T21:33:15.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.723+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f6e7c06fb80 con 0x7f6e7c144690 2024-09-10T21:33:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.963+0000 7f6e73e00640 1 -- 172.21.15.97:0/87252773 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v212) ==== 162+0+0 (secure 0 0 0) 0x7f6e7405e7a0 con 0x7f6e7c144690 2024-09-10T21:33:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.963+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6e5c0725e0 msgr2=0x7f6e5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.963+0000 7f6e82e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6e5c0725e0 0x7f6e5c074aa0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0af000 tx=0x7f6e6c01b040 comp rx=0 tx=0).stop 2024-09-10T21:33:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.963+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c144690 msgr2=0x7f6e7c13ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.963+0000 7f6e82e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c144690 0x7f6e7c13ac70 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400cbc0 tx=0x7f6e7400ccb0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 shutdown_connections 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6e5c0725e0 0x7f6e5c074aa0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c144690 0x7f6e7c13ac70 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6e7c143f90 0x7f6e7c13a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 --2- 172.21.15.97:0/87252773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e7c0b4660 0x7f6e7c15a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 >> 172.21.15.97:0/87252773 conn(0x7f6e7c0b1780 msgr2=0x7f6e7c140420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 shutdown_connections 2024-09-10T21:33:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:15.967+0000 7f6e82e00640 1 -- 172.21.15.97:0/87252773 wait complete. 2024-09-10T21:33:15.969 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:33:15.971+0000 7f14e6800640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T21:33:15.970 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:15.971+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:15.970 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:15.971+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:15.988 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:16.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 -- 172.21.15.97:0/4040109494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871815fc10 msgr2=0x7f871815c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 --2- 172.21.15.97:0/4040109494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871815fc10 0x7f871815c100 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f8718069e70 tx=0x7f871001cd80 comp rx=0 tx=0).stop 2024-09-10T21:33:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 -- 172.21.15.97:0/4040109494 shutdown_connections 2024-09-10T21:33:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 --2- 172.21.15.97:0/4040109494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871815fc10 0x7f871815c100 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 --2- 172.21.15.97:0/4040109494 >> v2:172.21.15.194:3300/0 conn(0x7f87180ce9d0 0x7f871815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 --2- 172.21.15.97:0/4040109494 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180ce5f0 0x7f8718141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 -- 172.21.15.97:0/4040109494 >> 172.21.15.97:0/4040109494 conn(0x7f87180b1780 msgr2=0x7f87180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.127+0000 7f871ee00640 1 -- 172.21.15.97:0/4040109494 shutdown_connections 2024-09-10T21:33:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 -- 172.21.15.97:0/4040109494 wait complete. 2024-09-10T21:33:16.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 Processor -- start 2024-09-10T21:33:16.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 -- start start 2024-09-10T21:33:16.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180ce5f0 0x7f8718154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180ce9d0 0x7f87181fae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f871815fc10 0x7f87181fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8718140290 con 0x7f87180ce5f0 2024-09-10T21:33:16.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8718140110 con 0x7f871815fc10 2024-09-10T21:33:16.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8718140410 con 0x7f87180ce9d0 2024-09-10T21:33:16.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180ce5f0 0x7f8718154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180ce5f0 0x7f8718154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55432/0 (socket says 172.21.15.97:55432) 2024-09-10T21:33:16.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 -- 172.21.15.97:0/4020604977 learned_addr learned my addr 172.21.15.97:0/4020604977 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:16.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f870f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180ce9d0 0x7f87181fae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ce00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f871815fc10 0x7f87181fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 -- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180ce9d0 msgr2=0x7f87181fae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180ce9d0 0x7f87181fae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 -- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f871815fc10 msgr2=0x7f87181fb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f871815fc10 0x7f87181fb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 -- 172.21.15.97:0/4020604977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87181fbc10 con 0x7f87180ce5f0 2024-09-10T21:33:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8717e00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180ce5f0 0x7f8718154780 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f870800d750 tx=0x7f870800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8714c00640 1 -- 172.21.15.97:0/4020604977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708017070 con 0x7f87180ce5f0 2024-09-10T21:33:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8714c00640 1 -- 172.21.15.97:0/4020604977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8708009d20 con 0x7f87180ce5f0 2024-09-10T21:33:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871815de30 con 0x7f87180ce5f0 2024-09-10T21:33:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f8714c00640 1 -- 172.21.15.97:0/4020604977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870800de60 con 0x7f87180ce5f0 2024-09-10T21:33:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.131+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181fb910 con 0x7f87180ce5f0 2024-09-10T21:33:16.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.135+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871815dc20 con 0x7f87180ce5f0 2024-09-10T21:33:16.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.135+0000 7f8714c00640 1 -- 172.21.15.97:0/4020604977 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8708005280 con 0x7f87180ce5f0 2024-09-10T21:33:16.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.135+0000 7f8714c00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.135+0000 7f8714c00640 1 -- 172.21.15.97:0/4020604977 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7f8708093da0 con 0x7f87180ce5f0 2024-09-10T21:33:16.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.135+0000 7f870f400640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.135+0000 7f870f400640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f871815b9c0 tx=0x7f870000b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.139+0000 7f8714c00640 1 -- 172.21.15.97:0/4020604977 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f870805e3f0 con 0x7f87180ce5f0 2024-09-10T21:33:16.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.339+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f871815f6d0 con 0x7f87180ce5f0 2024-09-10T21:33:16.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.339+0000 7f8714c00640 1 -- 172.21.15.97:0/4020604977 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+3569 (secure 0 0 0) 0x7f8708067160 con 0x7f87180ce5f0 2024-09-10T21:33:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:16.339 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":214,"btime":"2024-09-10T21:33:15:973660+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:15.973585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:16.339 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 214 2024-09-10T21:33:16.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 msgr2=0x7f86f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f871815b9c0 tx=0x7f870000b390 comp rx=0 tx=0).stop 2024-09-10T21:33:16.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180ce5f0 msgr2=0x7f8718154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180ce5f0 0x7f8718154780 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f870800d750 tx=0x7f870800dc10 comp rx=0 tx=0).stop 2024-09-10T21:33:16.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 shutdown_connections 2024-09-10T21:33:16.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f86f40725e0 0x7f86f4074aa0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f871815fc10 0x7f87181fb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87180ce9d0 0x7f87181fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 --2- 172.21.15.97:0/4020604977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87180ce5f0 0x7f8718154780 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 >> 172.21.15.97:0/4020604977 conn(0x7f87180b1780 msgr2=0x7f871806efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:16.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 shutdown_connections 2024-09-10T21:33:16.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.343+0000 7f871ee00640 1 -- 172.21.15.97:0/4020604977 wait complete. 2024-09-10T21:33:16.360 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:16.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.503+0000 7f9bf7600640 1 -- 172.21.15.97:0/520824290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf006dc50 msgr2=0x7f9bf00cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.503+0000 7f9bf7600640 1 --2- 172.21.15.97:0/520824290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf006dc50 0x7f9bf00cfe30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0097c0 tx=0x7f9bdc01c4b0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.503+0000 7f9bf7600640 1 -- 172.21.15.97:0/520824290 shutdown_connections 2024-09-10T21:33:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.503+0000 7f9bf7600640 1 --2- 172.21.15.97:0/520824290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf006dc50 0x7f9bf00cfe30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.503+0000 7f9bf7600640 1 -- 172.21.15.97:0/520824290 >> 172.21.15.97:0/520824290 conn(0x7f9bf00d1c30 msgr2=0x7f9bf00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.503+0000 7f9bf7600640 1 -- 172.21.15.97:0/520824290 shutdown_connections 2024-09-10T21:33:16.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.503+0000 7f9bf7600640 1 -- 172.21.15.97:0/520824290 wait complete. 2024-09-10T21:33:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 Processor -- start 2024-09-10T21:33:16.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 -- start start 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9bf00b1590 0x7f9bf00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf0161600 0x7f9bf014fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf01500a0 0x7f9bf014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf016c2c0 con 0x7f9bf01500a0 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf016c140 con 0x7f9bf0161600 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf016c440 con 0x7f9bf00b1590 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf01500a0 0x7f9bf014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf01500a0 0x7f9bf014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55448/0 (socket says 172.21.15.97:55448) 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 -- 172.21.15.97:0/4072479843 learned_addr learned my addr 172.21.15.97:0/4072479843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9befe00640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf0161600 0x7f9bf014fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf4c00640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9bf00b1590 0x7f9bf00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 -- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9bf00b1590 msgr2=0x7f9bf00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9bf00b1590 0x7f9bf00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 -- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf0161600 msgr2=0x7f9bf014fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf0161600 0x7f9bf014fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 -- 172.21.15.97:0/4072479843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf00b6fc0 con 0x7f9bf01500a0 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf5600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf01500a0 0x7f9bf014e9e0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007fd0 tx=0x7f9be800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9becc00640 1 -- 172.21.15.97:0/4072479843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8017070 con 0x7f9bf01500a0 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9becc00640 1 -- 172.21.15.97:0/4072479843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9be8003600 con 0x7f9bf01500a0 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9becc00640 1 -- 172.21.15.97:0/4072479843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8003a50 con 0x7f9bf01500a0 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf00b4c00 con 0x7f9bf01500a0 2024-09-10T21:33:16.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.507+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf00b4ea0 con 0x7f9bf01500a0 2024-09-10T21:33:16.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.511+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf00cfe30 con 0x7f9bf01500a0 2024-09-10T21:33:16.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.511+0000 7f9becc00640 1 -- 172.21.15.97:0/4072479843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9be8003bf0 con 0x7f9bf01500a0 2024-09-10T21:33:16.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.511+0000 7f9becc00640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9bd0072610 0x7f9bd0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.511+0000 7f9becc00640 1 -- 172.21.15.97:0/4072479843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7f9be8093270 con 0x7f9bf01500a0 2024-09-10T21:33:16.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.511+0000 7f9bf4c00640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9bd0072610 0x7f9bd0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.515+0000 7f9bf4c00640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9bd0072610 0x7f9bd0074ad0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9be00012f0 tx=0x7f9be0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:16.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.515+0000 7f9becc00640 1 -- 172.21.15.97:0/4072479843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9be805d8c0 con 0x7f9bf01500a0 2024-09-10T21:33:16.692 INFO:tasks.ceph.mds.a.smithi097.stderr: -45> 2024-09-10T21:33:11.939+0000 7f3908800640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:33:16.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.715+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bf015be90 con 0x7f9bf01500a0 2024-09-10T21:33:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.715+0000 7f9becc00640 1 -- 172.21.15.97:0/4072479843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+3569 (secure 0 0 0) 0x7f9be80622f0 con 0x7f9bf01500a0 2024-09-10T21:33:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:16.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":214,"btime":"2024-09-10T21:33:15:973660+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:15.973585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 214 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9bd0072610 msgr2=0x7f9bd0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9bd0072610 0x7f9bd0074ad0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9be00012f0 tx=0x7f9be0009040 comp rx=0 tx=0).stop 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf01500a0 msgr2=0x7f9bf014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf01500a0 0x7f9bf014e9e0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007fd0 tx=0x7f9be800c890 comp rx=0 tx=0).stop 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 shutdown_connections 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9bd0072610 0x7f9bd0074ad0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf01500a0 0x7f9bf014e9e0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf0161600 0x7f9bf014fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 --2- 172.21.15.97:0/4072479843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9bf00b1590 0x7f9bf00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 >> 172.21.15.97:0/4072479843 conn(0x7f9bf00d1c30 msgr2=0x7f9bf00af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 shutdown_connections 2024-09-10T21:33:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.719+0000 7f9bf7600640 1 -- 172.21.15.97:0/4072479843 wait complete. 2024-09-10T21:33:16.730 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:33:16.739 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:16.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:16.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.879+0000 7f9e2fe00640 1 -- 172.21.15.97:0/3376706184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 msgr2=0x7f9e300cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.879+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/3376706184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 0x7f9e300cfd80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009840 tx=0x7f9e1c01c820 comp rx=0 tx=0).stop 2024-09-10T21:33:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- 172.21.15.97:0/3376706184 shutdown_connections 2024-09-10T21:33:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/3376706184 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 0x7f9e300cfd80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/3376706184 >> v2:172.21.15.194:3300/0 conn(0x7f9e3015efc0 0x7f9e300cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- 172.21.15.97:0/3376706184 >> 172.21.15.97:0/3376706184 conn(0x7f9e300d1a60 msgr2=0x7f9e300d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- 172.21.15.97:0/3376706184 shutdown_connections 2024-09-10T21:33:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- 172.21.15.97:0/3376706184 wait complete. 2024-09-10T21:33:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 Processor -- start 2024-09-10T21:33:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- start start 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 0x7f9e30152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e300d02c0 0x7f9e3006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e3015efc0 0x7f9e3006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30168f70 con 0x7f9e300d02c0 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30168df0 con 0x7f9e3015efc0 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9e301690f0 con 0x7f9e300cdcb0 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e300d02c0 0x7f9e3006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e300d02c0 0x7f9e3006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55488/0 (socket says 172.21.15.97:55488) 2024-09-10T21:33:16.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 -- 172.21.15.97:0/63556935 learned_addr learned my addr 172.21.15.97:0/63556935 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2ea00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 0x7f9e30152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2f400640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e3015efc0 0x7f9e3006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 -- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 msgr2=0x7f9e30152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 0x7f9e30152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 -- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e3015efc0 msgr2=0x7f9e3006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e3015efc0 0x7f9e3006e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.883+0000 7f9e2e000640 1 -- 172.21.15.97:0/63556935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e300b10d0 con 0x7f9e300d02c0 2024-09-10T21:33:16.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e2e000640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e300d02c0 0x7f9e3006dce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c000c10 tx=0x7f9e1c0a84b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:16.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e27400640 1 -- 172.21.15.97:0/63556935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c005030 con 0x7f9e300d02c0 2024-09-10T21:33:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e27400640 1 -- 172.21.15.97:0/63556935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e1c0051d0 con 0x7f9e300d02c0 2024-09-10T21:33:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e300b13b0 con 0x7f9e300d02c0 2024-09-10T21:33:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e27400640 1 -- 172.21.15.97:0/63556935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0055c0 con 0x7f9e300d02c0 2024-09-10T21:33:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e300b1970 con 0x7f9e300d02c0 2024-09-10T21:33:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e300ce1e0 con 0x7f9e300d02c0 2024-09-10T21:33:16.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e27400640 1 -- 172.21.15.97:0/63556935 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e1c0c3030 con 0x7f9e300d02c0 2024-09-10T21:33:16.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e27400640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e100725e0 0x7f9e10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:16.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e27400640 1 -- 172.21.15.97:0/63556935 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7f9e1c12daa0 con 0x7f9e300d02c0 2024-09-10T21:33:16.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.887+0000 7f9e2ea00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e100725e0 0x7f9e10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:16.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.891+0000 7f9e2ea00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e100725e0 0x7f9e10074aa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009ec0 tx=0x7f9e20009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:16.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:16.891+0000 7f9e27400640 1 -- 172.21.15.97:0/63556935 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e1c0ac030 con 0x7f9e300d02c0 2024-09-10T21:33:17.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.095+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e3015e050 con 0x7f9e300d02c0 2024-09-10T21:33:17.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.095+0000 7f9e27400640 1 -- 172.21.15.97:0/63556935 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 215 v215) ==== 95+0+4395 (secure 0 0 0) 0x7f9e1c016410 con 0x7f9e300d02c0 2024-09-10T21:33:17.095 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:17.095 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":215,"btime":"2024-09-10T21:33:16:975666+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1929373906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1929373906},{"type":"v1","addr":"172.21.15.97:6834","nonce":1929373906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":215}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:15.973585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:17.095 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 215 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e100725e0 msgr2=0x7f9e10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e100725e0 0x7f9e10074aa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009ec0 tx=0x7f9e20009500 comp rx=0 tx=0).stop 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e300d02c0 msgr2=0x7f9e3006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e300d02c0 0x7f9e3006dce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c000c10 tx=0x7f9e1c0a84b0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 shutdown_connections 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9e100725e0 0x7f9e10074aa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e3015efc0 0x7f9e3006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e300d02c0 0x7f9e3006dce0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 --2- 172.21.15.97:0/63556935 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9e300cdcb0 0x7f9e30152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 >> 172.21.15.97:0/63556935 conn(0x7f9e300d1a60 msgr2=0x7f9e300719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:17.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 shutdown_connections 2024-09-10T21:33:17.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.099+0000 7f9e2fe00640 1 -- 172.21.15.97:0/63556935 wait complete. 2024-09-10T21:33:17.117 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2024-09-10T21:33:17.117 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2024-09-10T21:33:17.117 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:17.117 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:17.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:17.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:17.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:17.126 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:17.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:17.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 -- 172.21.15.97:0/4089395352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 msgr2=0x7f7ad8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 --2- 172.21.15.97:0/4089395352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 0x7f7ad8005220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009eb0 tx=0x7f7aec01d580 comp rx=0 tx=0).stop 2024-09-10T21:33:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7af9600640 1 -- 172.21.15.97:0/4089395352 reap_dead start 2024-09-10T21:33:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 -- 172.21.15.97:0/4089395352 shutdown_connections 2024-09-10T21:33:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 -- 172.21.15.97:0/4089395352 >> 172.21.15.97:0/4089395352 conn(0x7f7af40d1c60 msgr2=0x7f7af40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:17.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 -- 172.21.15.97:0/4089395352 shutdown_connections 2024-09-10T21:33:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 -- 172.21.15.97:0/4089395352 wait complete. 2024-09-10T21:33:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 Processor -- start 2024-09-10T21:33:17.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.367+0000 7f7afb600640 1 -- start start 2024-09-10T21:33:17.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 0x7f7af4155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af41f5810 0x7f7af4156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7af415c5b0 0x7f7af40b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4167a20 con 0x7f7af41f5810 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7af40b0e50 con 0x7f7ad8004da0 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7af4167ba0 con 0x7f7af415c5b0 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 0x7f7af4155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7aebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af41f5810 0x7f7af4156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 0x7f7af4155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:40422/0 (socket says 172.21.15.97:40422) 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7aebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af41f5810 0x7f7af4156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33490/0 (socket says 172.21.15.97:33490) 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 -- 172.21.15.97:0/3191136528 learned_addr learned my addr 172.21.15.97:0/3191136528 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:17.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af9600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7af415c5b0 0x7f7af40b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 -- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7af415c5b0 msgr2=0x7f7af40b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7af415c5b0 0x7f7af40b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 -- 172.21.15.97:0/3191136528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af41f5810 msgr2=0x7f7af4156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af41f5810 0x7f7af4156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 -- 172.21.15.97:0/3191136528 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af40d2320 con 0x7f7ad8004da0 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7aebe00640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af41f5810 0x7f7af4156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af8c00640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 0x7f7af4155fb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006400 tx=0x7f7ae4003ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af1600640 1 -- 172.21.15.97:0/3191136528 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4015070 con 0x7f7ad8004da0 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af1600640 1 -- 172.21.15.97:0/3191136528 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ae4011840 con 0x7f7ad8004da0 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af1600640 1 -- 172.21.15.97:0/3191136528 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4011c90 con 0x7f7ad8004da0 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af41f5d00 con 0x7f7ad8004da0 2024-09-10T21:33:17.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af41f62c0 con 0x7f7ad8004da0 2024-09-10T21:33:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af40aeb00 con 0x7f7ad8004da0 2024-09-10T21:33:17.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.371+0000 7f7af1600640 1 -- 172.21.15.97:0/3191136528 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ae40103c0 con 0x7f7ad8004da0 2024-09-10T21:33:17.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.375+0000 7f7af1600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ad00725e0 0x7f7ad0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:17.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.375+0000 7f7af1600640 1 -- 172.21.15.97:0/3191136528 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7f7ae40950e0 con 0x7f7ad8004da0 2024-09-10T21:33:17.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.375+0000 7f7aebe00640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ad00725e0 0x7f7ad0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:17.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.375+0000 7f7aebe00640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ad00725e0 0x7f7ad0074aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7af415c4e0 tx=0x7f7adc006b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:17.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.375+0000 7f7af1600640 1 -- 172.21.15.97:0/3191136528 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ae405f7b0 con 0x7f7ad8004da0 2024-09-10T21:33:17.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.575+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7af415acd0 con 0x7f7ad8004da0 2024-09-10T21:33:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7af1600640 1 -- 172.21.15.97:0/3191136528 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v216) ==== 161+0+0 (secure 0 0 0) 0x7f7ae40641e0 con 0x7f7ad8004da0 2024-09-10T21:33:17.989 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ad00725e0 msgr2=0x7f7ad0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ad00725e0 0x7f7ad0074aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7af415c4e0 tx=0x7f7adc006b20 comp rx=0 tx=0).stop 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 msgr2=0x7f7af4155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 0x7f7af4155fb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006400 tx=0x7f7ae4003ab0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 shutdown_connections 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ad00725e0 0x7f7ad0074aa0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7af415c5b0 0x7f7af40b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af41f5810 0x7f7af4156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 --2- 172.21.15.97:0/3191136528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ad8004da0 0x7f7af4155fb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 >> 172.21.15.97:0/3191136528 conn(0x7f7af40d1c60 msgr2=0x7f7af40cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 shutdown_connections 2024-09-10T21:33:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:17.991+0000 7f7afb600640 1 -- 172.21.15.97:0/3191136528 wait complete. 2024-09-10T21:33:18.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- 172.21.15.97:0/3497513251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 msgr2=0x7f638c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 --2- 172.21.15.97:0/3497513251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f638c00b9d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009ee0 tx=0x7f63a001d630 comp rx=0 tx=0).stop 2024-09-10T21:33:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- 172.21.15.97:0/3497513251 shutdown_connections 2024-09-10T21:33:18.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 --2- 172.21.15.97:0/3497513251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f638c00b9d0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- 172.21.15.97:0/3497513251 >> 172.21.15.97:0/3497513251 conn(0x7f63a80b1780 msgr2=0x7f63a80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- 172.21.15.97:0/3497513251 shutdown_connections 2024-09-10T21:33:18.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- 172.21.15.97:0/3497513251 wait complete. 2024-09-10T21:33:18.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 Processor -- start 2024-09-10T21:33:18.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- start start 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f63a81593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63a81e6b80 0x7f63a81f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63a81f8e30 0x7f63a806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80cf7d0 con 0x7f638c005220 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80cf650 con 0x7f63a81e6b80 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63ad200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f63a815a730 con 0x7f63a81f8e30 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f63a81593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f63a81593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33506/0 (socket says 172.21.15.97:33506) 2024-09-10T21:33:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63a5e00640 1 -- 172.21.15.97:0/4039055612 learned_addr learned my addr 172.21.15.97:0/4039055612 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f639d400640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63a81e6b80 0x7f63a81f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.159+0000 7f63a6800640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63a81f8e30 0x7f63a806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63a5e00640 1 -- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63a81f8e30 msgr2=0x7f63a806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63a5e00640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63a81f8e30 0x7f63a806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63a5e00640 1 -- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63a81e6b80 msgr2=0x7f63a81f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63a5e00640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63a81e6b80 0x7f63a81f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63a5e00640 1 -- 172.21.15.97:0/4039055612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a81f5400 con 0x7f638c005220 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63a6800640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63a81f8e30 0x7f63a806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63a5e00640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f63a81593a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6398004cd0 tx=0x7f6398001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f639ea00640 1 -- 172.21.15.97:0/4039055612 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398015020 con 0x7f638c005220 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f639ea00640 1 -- 172.21.15.97:0/4039055612 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63980117b0 con 0x7f638c005220 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f639ea00640 1 -- 172.21.15.97:0/4039055612 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398011c00 con 0x7f638c005220 2024-09-10T21:33:18.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a81f56e0 con 0x7f638c005220 2024-09-10T21:33:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81f80f0 con 0x7f638c005220 2024-09-10T21:33:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a806a010 con 0x7f638c005220 2024-09-10T21:33:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.163+0000 7f639ea00640 1 -- 172.21.15.97:0/4039055612 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f63980103c0 con 0x7f638c005220 2024-09-10T21:33:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.167+0000 7f639ea00640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63840725e0 0x7f6384074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.167+0000 7f639ea00640 1 -- 172.21.15.97:0/4039055612 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7f63980945a0 con 0x7f638c005220 2024-09-10T21:33:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.167+0000 7f639d400640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63840725e0 0x7f6384074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.167+0000 7f639d400640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63840725e0 0x7f6384074aa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6390000c10 tx=0x7f6390006730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.171+0000 7f639ea00640 1 -- 172.21.15.97:0/4039055612 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f639801f030 con 0x7f638c005220 2024-09-10T21:33:18.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.399+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f63a815ac20 con 0x7f638c005220 2024-09-10T21:33:18.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.995+0000 7f639ea00640 1 -- 172.21.15.97:0/4039055612 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v217) ==== 101+0+0 (secure 0 0 0) 0x7f639805f2e0 con 0x7f638c005220 2024-09-10T21:33:18.995 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:18.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63840725e0 msgr2=0x7f6384074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63840725e0 0x7f6384074aa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6390000c10 tx=0x7f6390006730 comp rx=0 tx=0).stop 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 msgr2=0x7f63a81593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f63a81593a0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6398004cd0 tx=0x7f6398001810 comp rx=0 tx=0).stop 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 shutdown_connections 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f63840725e0 0x7f6384074aa0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f63a81f8e30 0x7f63a806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63a81e6b80 0x7f63a81f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 --2- 172.21.15.97:0/4039055612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f638c005220 0x7f63a81593a0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 >> 172.21.15.97:0/4039055612 conn(0x7f63a80b1780 msgr2=0x7f63a815f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 shutdown_connections 2024-09-10T21:33:18.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:18.999+0000 7f63ad200640 1 -- 172.21.15.97:0/4039055612 wait complete. 2024-09-10T21:33:19.017 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:19.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 Processor -- start 2024-09-10T21:33:19.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 -- start start 2024-09-10T21:33:19.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4beec940 0x55da4bf91970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4bf76b00 0x55da4bf91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da4be0ba00 con 0x55da4bf772a0 2024-09-10T21:33:19.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55da4be69f40 con 0x55da4beec940 2024-09-10T21:33:19.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c722c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55da4bf79080 con 0x55da4bf76b00 2024-09-10T21:33:19.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf923f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33510/0 (socket says 172.21.15.97:33510) 2024-09-10T21:33:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 -- 172.21.15.97:0/467968787 learned_addr learned my addr 172.21.15.97:0/467968787 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c2a00640 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4bf76b00 0x55da4bf91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3400640 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4beec940 0x55da4bf91970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 -- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4bf76b00 msgr2=0x55da4bf91eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:19.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4bf76b00 0x55da4bf91eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 -- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4beec940 msgr2=0x55da4bf91970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4beec940 0x55da4bf91970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3e00640 1 -- 172.21.15.97:0/467968787 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4bde4c10 con 0x55da4bf772a0 2024-09-10T21:33:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.063+0000 7f98c3400640 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4beec940 0x55da4bf91970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf923f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f98bc01bca0 tx=0x7f98bc0181a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725dbc8786939339 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c1600640 1 -- 172.21.15.97:0/467968787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0bb020 con 0x55da4bf772a0 2024-09-10T21:33:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c1600640 1 -- 172.21.15.97:0/467968787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98bc0b1040 con 0x55da4bf772a0 2024-09-10T21:33:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c1600640 1 -- 172.21.15.97:0/467968787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0b0070 con 0x55da4bf772a0 2024-09-10T21:33:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- 172.21.15.97:0/467968787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 msgr2=0x55da4bf923f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf923f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f98bc01bca0 tx=0x7f98bc0181a0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- 172.21.15.97:0/467968787 shutdown_connections 2024-09-10T21:33:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf923f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4bf76b00 0x55da4bf91eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 --2- 172.21.15.97:0/467968787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4beec940 0x55da4bf91970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- 172.21.15.97:0/467968787 >> 172.21.15.97:0/467968787 conn(0x55da4beeb520 msgr2=0x55da4bf93300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- 172.21.15.97:0/467968787 shutdown_connections 2024-09-10T21:33:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- 172.21.15.97:0/467968787 wait complete. 2024-09-10T21:33:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 Processor -- start 2024-09-10T21:33:19.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- start start 2024-09-10T21:33:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4beec940 0x55da4bf96390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4bf76b00 0x55da4bf76070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf99650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da4bf96a10 con 0x55da4bf772a0 2024-09-10T21:33:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55da4beee630 con 0x55da4bf76b00 2024-09-10T21:33:19.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c722c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55da4bf765b0 con 0x55da4beec940 2024-09-10T21:33:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf99650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf99650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33520/0 (socket says 172.21.15.97:33520) 2024-09-10T21:33:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 learned_addr learned my addr 172.21.15.97:0/3317265129 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c2a00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4bf76b00 0x55da4bf76070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4beec940 0x55da4bf96390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4beec940 msgr2=0x55da4bf96390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4beec940 0x55da4bf96390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4bf76b00 msgr2=0x55da4bf76070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55da4bf76b00 0x55da4bf76070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4bf92710 con 0x55da4bf772a0 2024-09-10T21:33:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55da4beec940 0x55da4bf96390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98c3e00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf99650 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f98bc004b40 tx=0x7f98bc0ab9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98bbe00640 1 -- 172.21.15.97:0/3317265129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0c7070 con 0x55da4bf772a0 2024-09-10T21:33:19.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.067+0000 7f98bbe00640 1 -- 172.21.15.97:0/3317265129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98bc0abe10 con 0x55da4bf772a0 2024-09-10T21:33:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da4bf95470 con 0x55da4bf772a0 2024-09-10T21:33:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98bbe00640 1 -- 172.21.15.97:0/3317265129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0bb030 con 0x55da4bf772a0 2024-09-10T21:33:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4bf959b0 con 0x55da4bf772a0 2024-09-10T21:33:19.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98bbe00640 1 -- 172.21.15.97:0/3317265129 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f98bc0b76e0 con 0x55da4bf772a0 2024-09-10T21:33:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98bbe00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f98a4070fc0 0x7f98a4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98bbe00640 1 -- 172.21.15.97:0/3317265129 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7f98bc12dc50 con 0x55da4bf772a0 2024-09-10T21:33:19.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f98a4070fc0 0x7f98a4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.080 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:19.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55da4bf93e70 0x55da4bf94250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4bf94790 con 0x55da4bf93e70 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98c2a00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55da4bf93e70 0x55da4bf94250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.071+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f98a4070fc0 0x7f98a4073480 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f98b400f520 tx=0x7f98b4003720 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.075+0000 7f98c2a00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55da4bf93e70 0x55da4bf94250 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.075+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v118'660 uv634 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.075+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4bf94b30 con 0x55da4bf93e70 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.079+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v118'661 uv661 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.079+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4bf94ed0 con 0x55da4bf93e70 2024-09-10T21:33:19.082 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:19.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.079+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v118'662 uv662 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.079+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03aa50 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.079+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'663 uv663 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.079+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03adf0 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.083+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.083+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039a40 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.083+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.083+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.083+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v118'666 uv640 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.083+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.087+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v118'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.087+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.087+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v118'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.087 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:19.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.087+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.091+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.091+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.091+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.091+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.091+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.091+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.095+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v118'672 uv647 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.095+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.095+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'673 uv673 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.095+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.095+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v118'674 uv674 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.096 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:19.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.095+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.099+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'675 uv675 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.099+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.099+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'676 uv676 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.099+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c039e60 con 0x55da4bf93e70 2024-09-10T21:33:19.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.103+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'677 uv677 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.103+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03b190 con 0x55da4bf93e70 2024-09-10T21:33:19.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.103+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'678 uv678 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.103+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.103+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v118'679 uv653 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.103+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.107+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v118'680 uv680 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.107+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.107+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v118'681 uv681 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.106 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:19.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.107+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.107+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'682 uv682 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.107+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.111+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'683 uv683 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.111+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.111+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'684 uv684 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.111+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.115+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v118'685 uv659 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.115+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.115+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v118'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.115+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.115+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v118'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.115 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:19.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.115+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.119+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.119+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.119+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.119+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4bf93e70 2024-09-10T21:33:19.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.119+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4bf93e70 2024-09-10T21:33:19.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.119+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55da4c03b6a0 0x55da4c03da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.119+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03a590 con 0x55da4c03b6a0 2024-09-10T21:33:19.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.123+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55da4c03b6a0 0x55da4c03da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.123+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55da4c03b6a0 0x55da4c03da90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.123+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'484 uv467 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.123+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c03dfd0 con 0x55da4c03b6a0 2024-09-10T21:33:19.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.123+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'485 uv485 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.123+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03e370 con 0x55da4c03b6a0 2024-09-10T21:33:19.125 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:19.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.127+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'486 uv486 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.127+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03ebc0 con 0x55da4c03b6a0 2024-09-10T21:33:19.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.127+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v118'487 uv487 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.127+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03ef60 con 0x55da4c03b6a0 2024-09-10T21:33:19.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.127+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'488 uv488 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.127+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f300 con 0x55da4c03b6a0 2024-09-10T21:33:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.131+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v118'489 uv472 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.131+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.131+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v118'490 uv490 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.131+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.132 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.131+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v118'491 uv491 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.131+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.135+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'492 uv492 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.135+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.135+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'493 uv493 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.135+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.135+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v118'494 uv477 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.135+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.139+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v118'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.139+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.138 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.139+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v118'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.139+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c043870 con 0x55da4c03b6a0 2024-09-10T21:33:19.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.143+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v118'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.143+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c043870 con 0x55da4c03b6a0 2024-09-10T21:33:19.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.143+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.143+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.143+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v118'499 uv483 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.143+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.147+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v118'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.147+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.146 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.147+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v118'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.147+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c0406b0 con 0x55da4c03b6a0 2024-09-10T21:33:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.147+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.147+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c0406b0 con 0x55da4c03b6a0 2024-09-10T21:33:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c03f6a0 con 0x55da4c03b6a0 2024-09-10T21:33:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c03b6a0 2024-09-10T21:33:19.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55da4c03fd20 0x55da4c042110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c0426e0 con 0x55da4c03fd20 2024-09-10T21:33:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c2a00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55da4c03fd20 0x55da4c042110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c2a00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55da4c03fd20 0x55da4c042110 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.151+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c042ab0 con 0x55da4c03fd20 2024-09-10T21:33:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.155+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.155+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c042ab0 con 0x55da4c03fd20 2024-09-10T21:33:19.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.155+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.155 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:19.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.155+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c043ac0 con 0x55da4c03fd20 2024-09-10T21:33:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.155+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.155+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c043ac0 con 0x55da4c03fd20 2024-09-10T21:33:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c042ab0 con 0x55da4c03fd20 2024-09-10T21:33:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55da4c043180 0x55da4c045570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c045c20 con 0x55da4c043180 2024-09-10T21:33:19.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c3e00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55da4c043180 0x55da4c045570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c3e00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55da4c043180 0x55da4c045570 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'99 uv98 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.159+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c046130 con 0x55da4c043180 2024-09-10T21:33:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.163+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'100 uv100 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.163+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c046630 con 0x55da4c043180 2024-09-10T21:33:19.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.163+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'101 uv101 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.162 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.163+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c046ef0 con 0x55da4c043180 2024-09-10T21:33:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.163+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'102 uv102 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.163+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047290 con 0x55da4c03fd20 2024-09-10T21:33:19.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.167+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.167+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c047290 con 0x55da4c03fd20 2024-09-10T21:33:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.167+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.167+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047290 con 0x55da4c03fd20 2024-09-10T21:33:19.167 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.167+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.167+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c0482a0 con 0x55da4c03fd20 2024-09-10T21:33:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.171+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.171+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c0482a0 con 0x55da4c03fd20 2024-09-10T21:33:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.171+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.171+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047290 con 0x55da4c03fd20 2024-09-10T21:33:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.171+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.171+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047290 con 0x55da4c043180 2024-09-10T21:33:19.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.171+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v118'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.175+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c047630 con 0x55da4c043180 2024-09-10T21:33:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.175+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v118'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.175+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c0479d0 con 0x55da4c043180 2024-09-10T21:33:19.174 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.175+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v118'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.175+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c048d80 con 0x55da4c043180 2024-09-10T21:33:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.175+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.175+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c048d80 con 0x55da4c043180 2024-09-10T21:33:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.179+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.179+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c043180 2024-09-10T21:33:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.179+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'243 uv243 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.179+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c043180 2024-09-10T21:33:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.179+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v118'244 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.179+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c043180 2024-09-10T21:33:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.183+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v118'245 uv245 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.183+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c043180 2024-09-10T21:33:19.181 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.183+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v118'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.183+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c043180 2024-09-10T21:33:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.183+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.183+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c043180 2024-09-10T21:33:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c03fd20 2024-09-10T21:33:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v118'107 uv106 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c03fd20 2024-09-10T21:33:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v118'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c03fd20 2024-09-10T21:33:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v118'109 uv109 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.188 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.187+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c03fd20 2024-09-10T21:33:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.191+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'110 uv110 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.191+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c047d70 con 0x55da4c03fd20 2024-09-10T21:33:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.191+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'111 uv111 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b000c780 con 0x55da4c03fd20 2024-09-10T21:33:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.191+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55da4c0483f0 0x55da4c04a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.191+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04ae40 con 0x55da4c0483f0 2024-09-10T21:33:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.191+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55da4c0483f0 0x55da4c04a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.191+0000 7f98c3400640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55da4c0483f0 0x55da4c04a7e0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.195+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.195+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04b210 con 0x55da4c0483f0 2024-09-10T21:33:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.195+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.195+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b210 con 0x55da4c0483f0 2024-09-10T21:33:19.195 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.195+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.195+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04c220 con 0x55da4c0483f0 2024-09-10T21:33:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.199+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.199+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04c220 con 0x55da4c0483f0 2024-09-10T21:33:19.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.199+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.199+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b210 con 0x55da4c0483f0 2024-09-10T21:33:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.199+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.199+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b210 con 0x55da4c0483f0 2024-09-10T21:33:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.203+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v118'213 uv207 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.203+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04b210 con 0x55da4c0483f0 2024-09-10T21:33:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.203+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v118'214 uv214 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.203+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b210 con 0x55da4c0483f0 2024-09-10T21:33:19.203 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:19.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.203+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v118'215 uv215 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.203+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b2b0 con 0x55da4c0483f0 2024-09-10T21:33:19.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'216 uv216 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b2b0 con 0x55da4c0483f0 2024-09-10T21:33:19.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'217 uv217 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b2b0 con 0x55da4c0483f0 2024-09-10T21:33:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v118'218 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04b2b0 con 0x55da4c0483f0 2024-09-10T21:33:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v118'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.207+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b2b0 con 0x55da4c0483f0 2024-09-10T21:33:19.209 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:19.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.211+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v118'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.211+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b2b0 con 0x55da4c0483f0 2024-09-10T21:33:19.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.211+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.211+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e0c0 con 0x55da4c0483f0 2024-09-10T21:33:19.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.211+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.211+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v118'95 uv94 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'96 uv96 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.215 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:19.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v118'97 uv97 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'98 uv98 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.215+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.219+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v118'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.219+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.219+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v118'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.219+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.219 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:19.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.219+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.219+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04c770 con 0x55da4c043180 2024-09-10T21:33:19.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04b760 con 0x55da4c043180 2024-09-10T21:33:19.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98bc002050 con 0x55da4c043180 2024-09-10T21:33:19.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55da4c04bde0 0x55da4c04e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c04bde0 2024-09-10T21:33:19.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c2a00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55da4c04bde0 0x55da4c04e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.223+0000 7f98c2a00640 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55da4c04bde0 0x55da4c04e6e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v118'110 uv109 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98b0025040 con 0x55da4c04bde0 2024-09-10T21:33:19.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c04bde0 2024-09-10T21:33:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v118'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b0025040 con 0x55da4c04bde0 2024-09-10T21:33:19.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c04bde0 2024-09-10T21:33:19.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v118'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b0025040 con 0x55da4c04bde0 2024-09-10T21:33:19.227 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:19.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c04bde0 2024-09-10T21:33:19.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b0025040 con 0x55da4c04bde0 2024-09-10T21:33:19.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.227+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c04bde0 2024-09-10T21:33:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.231+0000 7f98c2a00640 1 -- 172.21.15.97:0/3317265129 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98b0025040 con 0x55da4c04bde0 2024-09-10T21:33:19.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.231+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.231+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v118'265 uv257 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.231+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.235+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v118'266 uv266 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.235+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.234 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.235+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v118'267 uv267 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.235+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.235+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v118'268 uv268 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.235+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.239+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'269 uv269 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.239+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.239+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v118'270 uv264 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.239+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.239+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v118'271 uv271 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.239+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v118'272 uv272 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.241 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:19.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'273 uv273 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'274 uv274 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04e1d0 con 0x55da4c0483f0 2024-09-10T21:33:19.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'275 uv275 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.243+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55da4c04ec20 con 0x55da4c0483f0 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c3400640 1 -- 172.21.15.97:0/3317265129 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'276 uv276 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98b4011870 con 0x55da4c0483f0 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55da4c03fd20 msgr2=0x55da4c042110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55da4c03fd20 0x55da4c042110 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55da4c0483f0 msgr2=0x55da4c04a7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55da4c0483f0 0x55da4c04a7e0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55da4c04bde0 msgr2=0x55da4c04e6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55da4c04bde0 0x55da4c04e6e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55da4c043180 msgr2=0x55da4c045570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55da4c043180 0x55da4c045570 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 reap_dead start 2024-09-10T21:33:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55da4c03b6a0 msgr2=0x55da4c03da90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55da4c03b6a0 0x55da4c03da90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55da4bf93e70 msgr2=0x55da4bf94250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55da4bf93e70 0x55da4bf94250 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f98a4070fc0 msgr2=0x7f98a4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f98a4070fc0 0x7f98a4073480 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f98b400f520 tx=0x7f98b4003720 comp rx=0 tx=0).stop 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 msgr2=0x55da4bf99650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 --2- 172.21.15.97:0/3317265129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da4bf772a0 0x55da4bf99650 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f98bc004b40 tx=0x7f98bc0ab9b0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c3e00640 1 -- 172.21.15.97:0/3317265129 reap_dead start 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 shutdown_connections 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 >> 172.21.15.97:0/3317265129 conn(0x55da4beeb520 msgr2=0x55da4bf09b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 shutdown_connections 2024-09-10T21:33:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.247+0000 7f98c722c080 1 -- 172.21.15.97:0/3317265129 wait complete. 2024-09-10T21:33:19.252 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2024-09-10T21:33:19.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 Processor -- start 2024-09-10T21:33:19.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 -- start start 2024-09-10T21:33:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf90f5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 0x55adf911ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf903d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55adf8f8ca00 con 0x7f4f44002350 2024-09-10T21:33:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55adf8feaf40 con 0x55adf90f5930 2024-09-10T21:33:19.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f60ce2080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55adf90fa1d0 con 0x55adf9112230 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf90f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf90f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33546/0 (socket says 172.21.15.97:33546) 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 -- 172.21.15.97:0/1111982714 learned_addr learned my addr 172.21.15.97:0/1111982714 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5d400640 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 0x55adf911ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5e800640 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf903d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 -- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 msgr2=0x55adf903d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf903d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 -- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 msgr2=0x55adf911ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 0x55adf911ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 -- 172.21.15.97:0/1111982714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf8f65c10 con 0x7f4f44002350 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.347+0000 7f4f5de00640 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf90f5380 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4f5001bdf0 tx=0x7f4f44004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaedc51f149c947b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f57e00640 1 -- 172.21.15.97:0/1111982714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f500013c0 con 0x7f4f44002350 2024-09-10T21:33:19.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f57e00640 1 -- 172.21.15.97:0/1111982714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f50001560 con 0x7f4f44002350 2024-09-10T21:33:19.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f57e00640 1 -- 172.21.15.97:0/1111982714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f500a4cd0 con 0x7f4f44002350 2024-09-10T21:33:19.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- 172.21.15.97:0/1111982714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 msgr2=0x55adf90f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf90f5380 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4f5001bdf0 tx=0x7f4f44004130 comp rx=0 tx=0).stop 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- 172.21.15.97:0/1111982714 shutdown_connections 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf903d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 0x55adf911ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/1111982714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf90f5380 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- 172.21.15.97:0/1111982714 >> 172.21.15.97:0/1111982714 conn(0x55adf90860d0 msgr2=0x55adf9114f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- 172.21.15.97:0/1111982714 shutdown_connections 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- 172.21.15.97:0/1111982714 wait complete. 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 Processor -- start 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- start start 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 0x55adf91b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf91110a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf906c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55adf911b2a0 con 0x7f4f44002350 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55adf9040980 con 0x55adf90f5930 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f60ce2080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55adf9040c20 con 0x55adf9112230 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf91110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf906c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 0x55adf91b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf91110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:40826/0 (socket says 172.21.15.97:40826) 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 -- 172.21.15.97:0/2858782352 learned_addr learned my addr 172.21.15.97:0/2858782352 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 -- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 msgr2=0x55adf91b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf90f5930 0x55adf91b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 -- 172.21.15.97:0/2858782352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 msgr2=0x55adf906c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f44002350 0x55adf906c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 -- 172.21.15.97:0/2858782352 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf9040ec0 con 0x55adf9112230 2024-09-10T21:33:19.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f5d400640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf91110a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c008700 tx=0x7f4f4c008bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f56a00640 1 -- 172.21.15.97:0/2858782352 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c01c070 con 0x55adf9112230 2024-09-10T21:33:19.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f56a00640 1 -- 172.21.15.97:0/2858782352 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f4c00ad50 con 0x55adf9112230 2024-09-10T21:33:19.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.351+0000 7f4f56a00640 1 -- 172.21.15.97:0/2858782352 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0176c0 con 0x55adf9112230 2024-09-10T21:33:19.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55adf9041160 con 0x55adf9112230 2024-09-10T21:33:19.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55adf9111b50 con 0x55adf9112230 2024-09-10T21:33:19.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f56a00640 1 -- 172.21.15.97:0/2858782352 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f4c017860 con 0x55adf9112230 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f56a00640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f40070fc0 0x7f4f40073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f56a00640 1 -- 172.21.15.97:0/2858782352 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7f4f4c093420 con 0x55adf9112230 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f5de00640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f40070fc0 0x7f4f40073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55adf907ff60 0x55adf9080340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55adf9080e70 con 0x55adf907ff60 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f5e800640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55adf907ff60 0x55adf9080340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.355+0000 7f4f5de00640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f40070fc0 0x7f4f40073480 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4f5001a010 tx=0x7f4f500ba040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f5e800640 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55adf907ff60 0x55adf9080340 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f5e800640 1 -- 172.21.15.97:0/2858782352 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v118'277 uv277 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f5800b040 con 0x55adf907ff60 2024-09-10T21:33:19.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55adf907ff60 msgr2=0x55adf9080340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55adf907ff60 0x55adf9080340 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f40070fc0 msgr2=0x7f4f40073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4f40070fc0 0x7f4f40073480 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4f5001a010 tx=0x7f4f500ba040 comp rx=0 tx=0).stop 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 msgr2=0x55adf91110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 --2- 172.21.15.97:0/2858782352 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adf9112230 0x55adf91110a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c008700 tx=0x7f4f4c008bc0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f5e800640 1 -- 172.21.15.97:0/2858782352 reap_dead start 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 shutdown_connections 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 >> 172.21.15.97:0/2858782352 conn(0x55adf90860d0 msgr2=0x55adf906d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 shutdown_connections 2024-09-10T21:33:19.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.359+0000 7f4f60ce2080 1 -- 172.21.15.97:0/2858782352 wait complete. 2024-09-10T21:33:19.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:19.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 -- 172.21.15.97:0/4070018725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0cec10 msgr2=0x7ffa5c0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 --2- 172.21.15.97:0/4070018725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0cec10 0x7ffa5c0cfd40 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0744b0 tx=0x7ffa4401f0b0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 -- 172.21.15.97:0/4070018725 shutdown_connections 2024-09-10T21:33:19.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 --2- 172.21.15.97:0/4070018725 >> v2:172.21.15.194:3300/0 conn(0x7ffa5c0d0280 0x7ffa5c15fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 --2- 172.21.15.97:0/4070018725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0cec10 0x7ffa5c0cfd40 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 --2- 172.21.15.97:0/4070018725 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa5c0ce830 0x7ffa5c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 -- 172.21.15.97:0/4070018725 >> 172.21.15.97:0/4070018725 conn(0x7ffa5c0b37c0 msgr2=0x7ffa5c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 -- 172.21.15.97:0/4070018725 shutdown_connections 2024-09-10T21:33:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.547+0000 7ffa62a00640 1 -- 172.21.15.97:0/4070018725 wait complete. 2024-09-10T21:33:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 Processor -- start 2024-09-10T21:33:19.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 -- start start 2024-09-10T21:33:19.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ce830 0x7ffa5c0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa5c0cec10 0x7ffa5c155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa5c0d0280 0x7ffa5c0c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c0b4420 con 0x7ffa5c0ce830 2024-09-10T21:33:19.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c0cde60 con 0x7ffa5c0d0280 2024-09-10T21:33:19.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5c0ce1a0 con 0x7ffa5c0cec10 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ce830 0x7ffa5c0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ce830 0x7ffa5c0c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33560/0 (socket says 172.21.15.97:33560) 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 -- 172.21.15.97:0/1667794582 learned_addr learned my addr 172.21.15.97:0/1667794582 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa53e00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa5c0cec10 0x7ffa5c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa60a00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa5c0d0280 0x7ffa5c0c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 -- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa5c0cec10 msgr2=0x7ffa5c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa5c0cec10 0x7ffa5c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 -- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa5c0d0280 msgr2=0x7ffa5c0c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa5c0d0280 0x7ffa5c0c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 -- 172.21.15.97:0/1667794582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c0b5650 con 0x7ffa5c0ce830 2024-09-10T21:33:19.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa5be00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ce830 0x7ffa5c0c5760 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e940 tx=0x7ffa4c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa58c00640 1 -- 172.21.15.97:0/1667794582 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00cd80 con 0x7ffa5c0ce830 2024-09-10T21:33:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa58c00640 1 -- 172.21.15.97:0/1667794582 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa4c005020 con 0x7ffa5c0ce830 2024-09-10T21:33:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa58c00640 1 -- 172.21.15.97:0/1667794582 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c005450 con 0x7ffa5c0ce830 2024-09-10T21:33:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c0b5930 con 0x7ffa5c0ce830 2024-09-10T21:33:19.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.551+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c0b5ef0 con 0x7ffa5c0ce830 2024-09-10T21:33:19.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.555+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c15fd20 con 0x7ffa5c0ce830 2024-09-10T21:33:19.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.555+0000 7ffa58c00640 1 -- 172.21.15.97:0/1667794582 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffa4c0047b0 con 0x7ffa5c0ce830 2024-09-10T21:33:19.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.555+0000 7ffa58c00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa380725e0 0x7ffa38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.555+0000 7ffa58c00640 1 -- 172.21.15.97:0/1667794582 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8167+0+0 (secure 0 0 0) 0x7ffa4c0946a0 con 0x7ffa5c0ce830 2024-09-10T21:33:19.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.555+0000 7ffa53e00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa380725e0 0x7ffa38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.555+0000 7ffa53e00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa380725e0 0x7ffa38074aa0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b2000 tx=0x7ffa4401d8f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.559+0000 7ffa58c00640 1 -- 172.21.15.97:0/1667794582 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffa4c05ecf0 con 0x7ffa5c0ce830 2024-09-10T21:33:19.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.759+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ffa5c0b2580 con 0x7ffa5c0ce830 2024-09-10T21:33:19.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa58c00640 1 -- 172.21.15.97:0/1667794582 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v218) ==== 162+0+0 (secure 0 0 0) 0x7ffa4c063720 con 0x7ffa5c0ce830 2024-09-10T21:33:19.996 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa380725e0 msgr2=0x7ffa38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa380725e0 0x7ffa38074aa0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b2000 tx=0x7ffa4401d8f0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ce830 msgr2=0x7ffa5c0c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ce830 0x7ffa5c0c5760 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e940 tx=0x7ffa4c00ee00 comp rx=0 tx=0).stop 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 shutdown_connections 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffa380725e0 0x7ffa38074aa0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa5c0d0280 0x7ffa5c0c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffa5c0cec10 0x7ffa5c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 --2- 172.21.15.97:0/1667794582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ce830 0x7ffa5c0c5760 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 >> 172.21.15.97:0/1667794582 conn(0x7ffa5c0b37c0 msgr2=0x7ffa5c0b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:20.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 shutdown_connections 2024-09-10T21:33:20.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:19.999+0000 7ffa62a00640 1 -- 172.21.15.97:0/1667794582 wait complete. 2024-09-10T21:33:20.010 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:33:20.011+0000 7f1660600640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-10T21:33:20.010 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:33:20.011+0000 7f1660600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:33:20.011 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:20.011+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:20.011 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:20.011+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:20.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 -- 172.21.15.97:0/2246568398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67806fd50 msgr2=0x7ff678070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 --2- 172.21.15.97:0/2246568398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67806fd50 0x7ff678070160 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7ff678074ee0 tx=0x7ff67001cb80 comp rx=0 tx=0).stop 2024-09-10T21:33:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 -- 172.21.15.97:0/2246568398 shutdown_connections 2024-09-10T21:33:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 --2- 172.21.15.97:0/2246568398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67806fd50 0x7ff678070160 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 --2- 172.21.15.97:0/2246568398 >> v2:172.21.15.194:3300/0 conn(0x7ff678143f90 0x7ff6780b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 -- 172.21.15.97:0/2246568398 >> 172.21.15.97:0/2246568398 conn(0x7ff6780b1780 msgr2=0x7ff6780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 -- 172.21.15.97:0/2246568398 shutdown_connections 2024-09-10T21:33:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 -- 172.21.15.97:0/2246568398 wait complete. 2024-09-10T21:33:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 Processor -- start 2024-09-10T21:33:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 -- start start 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.167+0000 7ff67fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff67806fd50 0x7ff67815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff678143f90 0x7ff6780cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678144690 0x7ff67813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff678167ca0 con 0x7ff678144690 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff678167b20 con 0x7ff67806fd50 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff678167e20 con 0x7ff678143f90 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678144690 0x7ff67813cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678144690 0x7ff67813cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33576/0 (socket says 172.21.15.97:33576) 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 -- 172.21.15.97:0/3490724225 learned_addr learned my addr 172.21.15.97:0/3490724225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:20.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67d000640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff67806fd50 0x7ff67815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff66fe00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff678143f90 0x7ff6780cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 -- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff678143f90 msgr2=0x7ff6780cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff678143f90 0x7ff6780cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 -- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff67806fd50 msgr2=0x7ff67815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff67806fd50 0x7ff67815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 -- 172.21.15.97:0/3490724225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff678073290 con 0x7ff678144690 2024-09-10T21:33:20.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67da00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678144690 0x7ff67813cc40 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7ff670009b10 tx=0x7ff670009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:20.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff675600640 1 -- 172.21.15.97:0/3490724225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6700bb070 con 0x7ff678144690 2024-09-10T21:33:20.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff675600640 1 -- 172.21.15.97:0/3490724225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff670004b40 con 0x7ff678144690 2024-09-10T21:33:20.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff678073570 con 0x7ff678144690 2024-09-10T21:33:20.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff675600640 1 -- 172.21.15.97:0/3490724225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6700b2040 con 0x7ff678144690 2024-09-10T21:33:20.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67815aea0 con 0x7ff678144690 2024-09-10T21:33:20.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff678070160 con 0x7ff678144690 2024-09-10T21:33:20.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.171+0000 7ff675600640 1 -- 172.21.15.97:0/3490724225 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff6700b5030 con 0x7ff678144690 2024-09-10T21:33:20.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.175+0000 7ff675600640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6540725e0 0x7ff654074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.175+0000 7ff675600640 1 -- 172.21.15.97:0/3490724225 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7ff67012d640 con 0x7ff678144690 2024-09-10T21:33:20.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.175+0000 7ff67d000640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6540725e0 0x7ff654074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.175+0000 7ff67d000640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6540725e0 0x7ff654074aa0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff668009be0 tx=0x7ff6680091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:20.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.179+0000 7ff675600640 1 -- 172.21.15.97:0/3490724225 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff670016770 con 0x7ff678144690 2024-09-10T21:33:20.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.375+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6780cf7a0 con 0x7ff678144690 2024-09-10T21:33:20.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.375+0000 7ff675600640 1 -- 172.21.15.97:0/3490724225 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+3569 (secure 0 0 0) 0x7ff6700f7c30 con 0x7ff678144690 2024-09-10T21:33:20.376 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:20.376 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":220,"btime":"2024-09-10T21:33:20:014701+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:20.014646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:20.376 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 220 2024-09-10T21:33:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6540725e0 msgr2=0x7ff654074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6540725e0 0x7ff654074aa0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff668009be0 tx=0x7ff6680091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678144690 msgr2=0x7ff67813cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678144690 0x7ff67813cc40 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7ff670009b10 tx=0x7ff670009300 comp rx=0 tx=0).stop 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 shutdown_connections 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff6540725e0 0x7ff654074aa0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff678144690 0x7ff67813cc40 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff678143f90 0x7ff6780cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 --2- 172.21.15.97:0/3490724225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff67806fd50 0x7ff67815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 >> 172.21.15.97:0/3490724225 conn(0x7ff6780b1780 msgr2=0x7ff678141570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 shutdown_connections 2024-09-10T21:33:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.379+0000 7ff67fa00640 1 -- 172.21.15.97:0/3490724225 wait complete. 2024-09-10T21:33:20.397 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:20.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 -- 172.21.15.97:0/4209997548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0ce640 msgr2=0x7fd28c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 --2- 172.21.15.97:0/4209997548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0ce640 0x7fd28c0cea20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009840 tx=0x7fd27c01c820 comp rx=0 tx=0).stop 2024-09-10T21:33:20.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 -- 172.21.15.97:0/4209997548 shutdown_connections 2024-09-10T21:33:20.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 --2- 172.21.15.97:0/4209997548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0b0d50 0x7fd28c0aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 --2- 172.21.15.97:0/4209997548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0ce640 0x7fd28c0cea20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 -- 172.21.15.97:0/4209997548 >> 172.21.15.97:0/4209997548 conn(0x7fd28c0d1c30 msgr2=0x7fd28c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:20.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 -- 172.21.15.97:0/4209997548 shutdown_connections 2024-09-10T21:33:20.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 -- 172.21.15.97:0/4209997548 wait complete. 2024-09-10T21:33:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 Processor -- start 2024-09-10T21:33:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.539+0000 7fd292000640 1 -- start start 2024-09-10T21:33:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0af300 0x7fd28c154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0b0d50 0x7fd28c165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ce640 0x7fd28c0708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1690f0 con 0x7fd28c0ce640 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c0b00d0 con 0x7fd28c0b0d50 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd28c169270 con 0x7fd28c0af300 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ce640 0x7fd28c0708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0af300 0x7fd28c154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0af300 0x7fd28c154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:40872/0 (socket says 172.21.15.97:40872) 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 -- 172.21.15.97:0/779021240 learned_addr learned my addr 172.21.15.97:0/779021240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28aa00640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0b0d50 0x7fd28c165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 -- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0b0d50 msgr2=0x7fd28c165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0b0d50 0x7fd28c165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 -- 172.21.15.97:0/779021240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ce640 msgr2=0x7fd28c0708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ce640 0x7fd28c0708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 -- 172.21.15.97:0/779021240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c0b6b20 con 0x7fd28c0af300 2024-09-10T21:33:20.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd28b400640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0af300 0x7fd28c154ca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd27c01aed0 tx=0x7fd27c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:20.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd283400640 1 -- 172.21.15.97:0/779021240 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0b2040 con 0x7fd28c0af300 2024-09-10T21:33:20.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd283400640 1 -- 172.21.15.97:0/779021240 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd27c0043e0 con 0x7fd28c0af300 2024-09-10T21:33:20.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd283400640 1 -- 172.21.15.97:0/779021240 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0052d0 con 0x7fd28c0af300 2024-09-10T21:33:20.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c165a70 con 0x7fd28c0af300 2024-09-10T21:33:20.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c0b7080 con 0x7fd28c0af300 2024-09-10T21:33:20.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.543+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c0aedc0 con 0x7fd28c0af300 2024-09-10T21:33:20.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.547+0000 7fd283400640 1 -- 172.21.15.97:0/779021240 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd27c0b5070 con 0x7fd28c0af300 2024-09-10T21:33:20.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.547+0000 7fd283400640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd26c0725e0 0x7fd26c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.547+0000 7fd283400640 1 -- 172.21.15.97:0/779021240 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7fd27c12d170 con 0x7fd28c0af300 2024-09-10T21:33:20.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.547+0000 7fd28aa00640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd26c0725e0 0x7fd26c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.547+0000 7fd28aa00640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd26c0725e0 0x7fd26c074aa0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0714f0 tx=0x7fd2780091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:20.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.551+0000 7fd283400640 1 -- 172.21.15.97:0/779021240 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd28c0b6b20 con 0x7fd28c0af300 2024-09-10T21:33:20.723 INFO:tasks.ceph.mds.c.smithi097.stderr: -56> 2024-09-10T21:33:15.971+0000 7f14e6800640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T21:33:20.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.751+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd28c0d0460 con 0x7fd28c0af300 2024-09-10T21:33:20.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.751+0000 7fd283400640 1 -- 172.21.15.97:0/779021240 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+3569 (secure 0 0 0) 0x7fd27c016410 con 0x7fd28c0af300 2024-09-10T21:33:20.751 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:20.751 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":220,"btime":"2024-09-10T21:33:20:014701+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:20.014646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:20.751 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 220 2024-09-10T21:33:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd26c0725e0 msgr2=0x7fd26c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd26c0725e0 0x7fd26c074aa0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0714f0 tx=0x7fd2780091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0af300 msgr2=0x7fd28c154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0af300 0x7fd28c154ca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd27c01aed0 tx=0x7fd27c005d90 comp rx=0 tx=0).stop 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 shutdown_connections 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd26c0725e0 0x7fd26c074aa0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd28c0ce640 0x7fd28c0708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd28c0b0d50 0x7fd28c165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 --2- 172.21.15.97:0/779021240 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd28c0af300 0x7fd28c154ca0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 >> 172.21.15.97:0/779021240 conn(0x7fd28c0d1c30 msgr2=0x7fd28c0b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 shutdown_connections 2024-09-10T21:33:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.755+0000 7fd292000640 1 -- 172.21.15.97:0/779021240 wait complete. 2024-09-10T21:33:20.761 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:33:20.775 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:20.775 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.915+0000 7f0c4d800640 1 -- 172.21.15.97:0/1040732277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480b2570 msgr2=0x7f0c48166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.915+0000 7f0c4d800640 1 --2- 172.21.15.97:0/1040732277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480b2570 0x7f0c48166fe0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009bc0 tx=0x7f0c4001f3b0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- 172.21.15.97:0/1040732277 shutdown_connections 2024-09-10T21:33:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 --2- 172.21.15.97:0/1040732277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480b2570 0x7f0c48166fe0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- 172.21.15.97:0/1040732277 >> 172.21.15.97:0/1040732277 conn(0x7f0c480b37c0 msgr2=0x7f0c480b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- 172.21.15.97:0/1040732277 shutdown_connections 2024-09-10T21:33:20.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- 172.21.15.97:0/1040732277 wait complete. 2024-09-10T21:33:20.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 Processor -- start 2024-09-10T21:33:20.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- start start 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c480b2570 0x7f0c48166f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c480ce830 0x7f0c480c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480cec10 0x7f0c4815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c480cddd0 con 0x7f0c480cec10 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0c480cdc50 con 0x7f0c480ce830 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c4d800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0c480ce1a0 con 0x7f0c480b2570 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480cec10 0x7f0c4815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480cec10 0x7f0c4815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33614/0 (socket says 172.21.15.97:33614) 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c46a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c480b2570 0x7f0c48166f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 -- 172.21.15.97:0/2227868231 learned_addr learned my addr 172.21.15.97:0/2227868231 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:20.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c46000640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c480ce830 0x7f0c480c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 -- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c480b2570 msgr2=0x7f0c48166f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c480b2570 0x7f0c48166f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 -- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c480ce830 msgr2=0x7f0c480c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c480ce830 0x7f0c480c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 -- 172.21.15.97:0/2227868231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c480c9e60 con 0x7f0c480cec10 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c46a00640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c480b2570 0x7f0c48166f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.919+0000 7f0c47400640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480cec10 0x7f0c4815a3d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f0c4001cfe0 tx=0x7f0c40003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c3ea00640 1 -- 172.21.15.97:0/2227868231 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c400b4040 con 0x7f0c480cec10 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c3ea00640 1 -- 172.21.15.97:0/2227868231 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c400be070 con 0x7f0c480cec10 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4814cd00 con 0x7f0c480cec10 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c3ea00640 1 -- 172.21.15.97:0/2227868231 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c400af8a0 con 0x7f0c480cec10 2024-09-10T21:33:20.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4814d2c0 con 0x7f0c480cec10 2024-09-10T21:33:20.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c480cf800 con 0x7f0c480cec10 2024-09-10T21:33:20.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c3ea00640 1 -- 172.21.15.97:0/2227868231 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0c400af4c0 con 0x7f0c480cec10 2024-09-10T21:33:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c3ea00640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0c28072610 0x7f0c28074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c3ea00640 1 -- 172.21.15.97:0/2227868231 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7f0c40131b90 con 0x7f0c480cec10 2024-09-10T21:33:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.923+0000 7f0c46a00640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0c28072610 0x7f0c28074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:20.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.927+0000 7f0c46a00640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0c28072610 0x7f0c28074ad0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007fd0 tx=0x7f0c38001850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:20.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:20.927+0000 7f0c3ea00640 1 -- 172.21.15.97:0/2227868231 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0c40018f10 con 0x7f0c480cec10 2024-09-10T21:33:21.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.127+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c480b7af0 con 0x7f0c480cec10 2024-09-10T21:33:21.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.127+0000 7f0c3ea00640 1 -- 172.21.15.97:0/2227868231 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) ==== 95+0+4392 (secure 0 0 0) 0x7f0c400fc210 con 0x7f0c480cec10 2024-09-10T21:33:21.128 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:21.129 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":221,"btime":"2024-09-10T21:33:21:018381+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/899393328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":899393328},{"type":"v1","addr":"172.21.15.97:6835","nonce":899393328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":221}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:20.014646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":119,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:21.129 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 221 2024-09-10T21:33:21.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0c28072610 msgr2=0x7f0c28074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:21.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0c28072610 0x7f0c28074ad0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007fd0 tx=0x7f0c38001850 comp rx=0 tx=0).stop 2024-09-10T21:33:21.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480cec10 msgr2=0x7f0c4815a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:21.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480cec10 0x7f0c4815a3d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f0c4001cfe0 tx=0x7f0c40003f80 comp rx=0 tx=0).stop 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 shutdown_connections 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0c28072610 0x7f0c28074ad0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c480cec10 0x7f0c4815a3d0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c480ce830 0x7f0c480c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 --2- 172.21.15.97:0/2227868231 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0c480b2570 0x7f0c48166f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 >> 172.21.15.97:0/2227868231 conn(0x7f0c480b37c0 msgr2=0x7f0c480cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 shutdown_connections 2024-09-10T21:33:21.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.131+0000 7f0c4d800640 1 -- 172.21.15.97:0/2227868231 wait complete. 2024-09-10T21:33:21.150 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2024-09-10T21:33:21.150 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2024-09-10T21:33:21.150 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:21.150 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:21.159 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:21.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:21.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:21.159 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:21.211 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:21.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:21.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 -- 172.21.15.97:0/152017895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef415eed0 msgr2=0x7f0ef40d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 --2- 172.21.15.97:0/152017895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef415eed0 0x7f0ef40d27f0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009bc0 tx=0x7f0ee401cb30 comp rx=0 tx=0).stop 2024-09-10T21:33:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 -- 172.21.15.97:0/152017895 shutdown_connections 2024-09-10T21:33:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 --2- 172.21.15.97:0/152017895 >> v2:172.21.15.194:3300/0 conn(0x7f0ef40ccc60 0x7f0ef40b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 --2- 172.21.15.97:0/152017895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef415eed0 0x7f0ef40d27f0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 -- 172.21.15.97:0/152017895 >> 172.21.15.97:0/152017895 conn(0x7f0ef40d1cd0 msgr2=0x7f0ef40d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 -- 172.21.15.97:0/152017895 shutdown_connections 2024-09-10T21:33:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.403+0000 7f0efbc00640 1 -- 172.21.15.97:0/152017895 wait complete. 2024-09-10T21:33:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 Processor -- start 2024-09-10T21:33:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 -- start start 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ef40ccc60 0x7f0ef40b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef40cdd40 0x7f0ef406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ef415eed0 0x7f0ef414c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4168880 con 0x7f0ef40cdd40 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4168700 con 0x7f0ef415eed0 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef4168a00 con 0x7f0ef40ccc60 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef40cdd40 0x7f0ef406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:21.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef40cdd40 0x7f0ef406dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33634/0 (socket says 172.21.15.97:33634) 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 -- 172.21.15.97:0/2093436881 learned_addr learned my addr 172.21.15.97:0/2093436881 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef9c00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ef415eed0 0x7f0ef414c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef9200640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ef40ccc60 0x7f0ef40b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 -- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ef40ccc60 msgr2=0x7f0ef40b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ef40ccc60 0x7f0ef40b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 -- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ef415eed0 msgr2=0x7f0ef414c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ef415eed0 0x7f0ef414c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 -- 172.21.15.97:0/2093436881 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef415dc60 con 0x7f0ef40cdd40 2024-09-10T21:33:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef3e00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef40cdd40 0x7f0ef406dc50 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000c4d0 tx=0x7f0ee000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef0c00640 1 -- 172.21.15.97:0/2093436881 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0016020 con 0x7f0ef40cdd40 2024-09-10T21:33:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef0c00640 1 -- 172.21.15.97:0/2093436881 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ee0012920 con 0x7f0ef40cdd40 2024-09-10T21:33:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0ef0c00640 1 -- 172.21.15.97:0/2093436881 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0012d30 con 0x7f0ef40cdd40 2024-09-10T21:33:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef415df40 con 0x7f0ef40cdd40 2024-09-10T21:33:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef4158780 con 0x7f0ef40cdd40 2024-09-10T21:33:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.407+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef40cfd60 con 0x7f0ef40cdd40 2024-09-10T21:33:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.411+0000 7f0ef0c00640 1 -- 172.21.15.97:0/2093436881 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0ee001b400 con 0x7f0ef40cdd40 2024-09-10T21:33:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.411+0000 7f0ef0c00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ed40725e0 0x7f0ed4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.411+0000 7f0ef0c00640 1 -- 172.21.15.97:0/2093436881 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7f0ee0093e70 con 0x7f0ef40cdd40 2024-09-10T21:33:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.411+0000 7f0ef9200640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ed40725e0 0x7f0ed4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:21.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.411+0000 7f0ef9200640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ed40725e0 0x7f0ed4074aa0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40af000 tx=0x7f0ee401b370 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:21.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.415+0000 7f0ef0c00640 1 -- 172.21.15.97:0/2093436881 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0ee0020020 con 0x7f0ef40cdd40 2024-09-10T21:33:21.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:21.615+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0ef40b51f0 con 0x7f0ef40cdd40 2024-09-10T21:33:22.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.027+0000 7f0ef0c00640 1 -- 172.21.15.97:0/2093436881 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v222) ==== 161+0+0 (secure 0 0 0) 0x7f0ee0091e20 con 0x7f0ef40cdd40 2024-09-10T21:33:22.027 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:22.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ed40725e0 msgr2=0x7f0ed4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:22.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ed40725e0 0x7f0ed4074aa0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40af000 tx=0x7f0ee401b370 comp rx=0 tx=0).stop 2024-09-10T21:33:22.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef40cdd40 msgr2=0x7f0ef406dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:22.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef40cdd40 0x7f0ef406dc50 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000c4d0 tx=0x7f0ee000c990 comp rx=0 tx=0).stop 2024-09-10T21:33:22.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 shutdown_connections 2024-09-10T21:33:22.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ed40725e0 0x7f0ed4074aa0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ef415eed0 0x7f0ef414c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef40cdd40 0x7f0ef406dc50 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 --2- 172.21.15.97:0/2093436881 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ef40ccc60 0x7f0ef40b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 >> 172.21.15.97:0/2093436881 conn(0x7f0ef40d1cd0 msgr2=0x7f0ef415c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:22.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 shutdown_connections 2024-09-10T21:33:22.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.031+0000 7f0efbc00640 1 -- 172.21.15.97:0/2093436881 wait complete. 2024-09-10T21:33:22.049 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:22.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 -- 172.21.15.97:0/1635225090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 msgr2=0x7face80af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:22.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 --2- 172.21.15.97:0/1635225090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face80af4d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7face0009bc0 tx=0x7face001cba0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 -- 172.21.15.97:0/1635225090 shutdown_connections 2024-09-10T21:33:22.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 --2- 172.21.15.97:0/1635225090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face80af4d0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 --2- 172.21.15.97:0/1635225090 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7face80ce640 0x7face80cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 -- 172.21.15.97:0/1635225090 >> 172.21.15.97:0/1635225090 conn(0x7face80d1c30 msgr2=0x7face80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:22.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 -- 172.21.15.97:0/1635225090 shutdown_connections 2024-09-10T21:33:22.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.191+0000 7faceee00640 1 -- 172.21.15.97:0/1635225090 wait complete. 2024-09-10T21:33:22.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 Processor -- start 2024-09-10T21:33:22.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 -- start start 2024-09-10T21:33:22.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face80b0d50 0x7face815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:22.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:22.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7face80ce640 0x7face8159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:22.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7face8168cc0 con 0x7face80cd6e0 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7face8168b40 con 0x7face80b0d50 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7face8168e40 con 0x7face80ce640 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face815edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33650/0 (socket says 172.21.15.97:33650) 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 -- 172.21.15.97:0/1030022425 learned_addr learned my addr 172.21.15.97:0/1030022425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facece00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7face80ce640 0x7face8159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7face7e00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face80b0d50 0x7face815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 -- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7face80ce640 msgr2=0x7face8159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7face80ce640 0x7face8159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 -- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face80b0d50 msgr2=0x7face815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face80b0d50 0x7face815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 -- 172.21.15.97:0/1030022425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face8159570 con 0x7face80cd6e0 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7face7e00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face80b0d50 0x7face815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7facdf400640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face815edf0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7facd000c430 tx=0x7facd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7face4c00640 1 -- 172.21.15.97:0/1030022425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0017070 con 0x7face80cd6e0 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7face4c00640 1 -- 172.21.15.97:0/1030022425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facd0003600 con 0x7face80cd6e0 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face815e030 con 0x7face80cd6e0 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7face4c00640 1 -- 172.21.15.97:0/1030022425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0003a50 con 0x7face80cd6e0 2024-09-10T21:33:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face815e5f0 con 0x7face80cd6e0 2024-09-10T21:33:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.195+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face80cea20 con 0x7face80cd6e0 2024-09-10T21:33:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.199+0000 7face4c00640 1 -- 172.21.15.97:0/1030022425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7facd0003bf0 con 0x7face80cd6e0 2024-09-10T21:33:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.199+0000 7face4c00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc4072610 0x7facc4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.199+0000 7face4c00640 1 -- 172.21.15.97:0/1030022425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7facd00932c0 con 0x7face80cd6e0 2024-09-10T21:33:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.199+0000 7face7e00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc4072610 0x7facc4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.199+0000 7face7e00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc4072610 0x7facc4074ad0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7facd8005fd0 tx=0x7facd8005dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.203+0000 7face4c00640 1 -- 172.21.15.97:0/1030022425 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7face8159570 con 0x7face80cd6e0 2024-09-10T21:33:22.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:22.427+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7face8075f90 con 0x7face80cd6e0 2024-09-10T21:33:23.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7face4c00640 1 -- 172.21.15.97:0/1030022425 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v223) ==== 101+0+0 (secure 0 0 0) 0x7facd005df00 con 0x7face80cd6e0 2024-09-10T21:33:23.037 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc4072610 msgr2=0x7facc4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc4072610 0x7facc4074ad0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7facd8005fd0 tx=0x7facd8005dc0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 msgr2=0x7face815edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face815edf0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7facd000c430 tx=0x7facd000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 shutdown_connections 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc4072610 0x7facc4074ad0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7face80ce640 0x7face8159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7face80cd6e0 0x7face815edf0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 --2- 172.21.15.97:0/1030022425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face80b0d50 0x7face815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 >> 172.21.15.97:0/1030022425 conn(0x7face80d1c30 msgr2=0x7face80b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 shutdown_connections 2024-09-10T21:33:23.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.039+0000 7faceee00640 1 -- 172.21.15.97:0/1030022425 wait complete. 2024-09-10T21:33:23.060 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 Processor -- start 2024-09-10T21:33:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 -- start start 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c18aa0 0x560562c18060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd4d10 0x560562cf1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562cd50f0 0x560562cf1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560562b66a00 con 0x560562cd4d10 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560562bc4f40 con 0x560562cd50f0 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560562c5ddc0 con 0x560562c18aa0 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd4d10 0x560562cf1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd37000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562cd50f0 0x560562cf1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd36600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c18aa0 0x560562c18060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd4d10 0x560562cf1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33660/0 (socket says 172.21.15.97:33660) 2024-09-10T21:33:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 -- 172.21.15.97:0/3329225140 learned_addr learned my addr 172.21.15.97:0/3329225140 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd37000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562cd50f0 0x560562cf1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:40634/0 (socket says 172.21.15.97:40634) 2024-09-10T21:33:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 -- 172.21.15.97:0/3329225140 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c18aa0 msgr2=0x560562c18060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 --2- 172.21.15.97:0/3329225140 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c18aa0 0x560562c18060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 -- 172.21.15.97:0/3329225140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562cd50f0 msgr2=0x560562cf1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 --2- 172.21.15.97:0/3329225140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562cd50f0 0x560562cf1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 -- 172.21.15.97:0/3329225140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560562b3fc10 con 0x560562cd4d10 2024-09-10T21:33:23.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd35c00640 1 --2- 172.21.15.97:0/3329225140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd4d10 0x560562cf1020 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c610 tx=0x7fdd2400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd527d71f03362f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd2fe00640 1 -- 172.21.15.97:0/3329225140 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400aa70 con 0x560562cd4d10 2024-09-10T21:33:23.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd2fe00640 1 -- 172.21.15.97:0/3329225140 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd2400ac10 con 0x560562cd4d10 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd2fe00640 1 -- 172.21.15.97:0/3329225140 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24015440 con 0x560562cd4d10 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 -- 172.21.15.97:0/3329225140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd4d10 msgr2=0x560562cf1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.107+0000 7fdd3944e080 1 --2- 172.21.15.97:0/3329225140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd4d10 0x560562cf1020 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c610 tx=0x7fdd2400cad0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- 172.21.15.97:0/3329225140 shutdown_connections 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 --2- 172.21.15.97:0/3329225140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562cd50f0 0x560562cf1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 --2- 172.21.15.97:0/3329225140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd4d10 0x560562cf1020 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 --2- 172.21.15.97:0/3329225140 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c18aa0 0x560562c18060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- 172.21.15.97:0/3329225140 >> 172.21.15.97:0/3329225140 conn(0x560562c17140 msgr2=0x560562cd1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- 172.21.15.97:0/3329225140 shutdown_connections 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- 172.21.15.97:0/3329225140 wait complete. 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 Processor -- start 2024-09-10T21:33:23.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- start start 2024-09-10T21:33:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562c18aa0 0x560562cf34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd50f0 0x560562ceb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c44710 0x560562c44af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560562cd0fd0 con 0x560562cd50f0 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560562cd1270 con 0x560562c18aa0 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560562c47a80 con 0x560562c44710 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd50f0 0x560562ceb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd50f0 0x560562ceb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33676/0 (socket says 172.21.15.97:33676) 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 learned_addr learned my addr 172.21.15.97:0/77035978 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd36600640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562c18aa0 0x560562cf34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd37000640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c44710 0x560562c44af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c44710 msgr2=0x560562c44af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560562c44710 0x560562c44af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562c18aa0 msgr2=0x560562cf34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560562c18aa0 0x560562cf34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560562cf25d0 con 0x560562cd50f0 2024-09-10T21:33:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd35c00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd50f0 0x560562ceb000 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x560562c463d0 tx=0x7fdd24004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd2ea00640 1 -- 172.21.15.97:0/77035978 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24015950 con 0x560562cd50f0 2024-09-10T21:33:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd2ea00640 1 -- 172.21.15.97:0/77035978 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd24023070 con 0x560562cd50f0 2024-09-10T21:33:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd2ea00640 1 -- 172.21.15.97:0/77035978 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24015d70 con 0x560562cd50f0 2024-09-10T21:33:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560562cf2870 con 0x560562cd50f0 2024-09-10T21:33:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.111+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560562cf2db0 con 0x560562cd50f0 2024-09-10T21:33:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd2ea00640 1 -- 172.21.15.97:0/77035978 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdd24013070 con 0x560562cd50f0 2024-09-10T21:33:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd2ea00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdd18070ff0 0x7fdd180734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd2ea00640 1 -- 172.21.15.97:0/77035978 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7fdd24094480 con 0x560562cd50f0 2024-09-10T21:33:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd36600640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdd18070ff0 0x7fdd180734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.115 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560562d8b880 0x560562d8bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562ceb540 con 0x560562d8b880 2024-09-10T21:33:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd37000640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560562d8b880 0x560562d8bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd36600640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdd18070ff0 0x7fdd180734b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdd28011570 tx=0x7fdd280017e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.115+0000 7fdd37000640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560562d8b880 0x560562d8bc60 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.119+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v119'691 uv665 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.119+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d8c1a0 con 0x560562d8b880 2024-09-10T21:33:23.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.119+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v119'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.119+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d94b50 con 0x560562d8b880 2024-09-10T21:33:23.121 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.123+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v119'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.123+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95f00 con 0x560562d8b880 2024-09-10T21:33:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.123+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.123+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d962a0 con 0x560562d8b880 2024-09-10T21:33:23.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.127+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.127+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d94ef0 con 0x560562d8b880 2024-09-10T21:33:23.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.127+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.127+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d952e0 con 0x560562d8b880 2024-09-10T21:33:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.131+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v119'697 uv671 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.131+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.131+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v119'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.131+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.131 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.131+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v119'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.131+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.135+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.135+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.135+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'701 uv701 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.135+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.139+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'702 uv702 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.139+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.139+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v119'703 uv678 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.139+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.139+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'704 uv704 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.139+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.141 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.143+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v119'705 uv705 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.143+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.143+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'706 uv706 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.143+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.147+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'707 uv707 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.147+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.147+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'708 uv708 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.147+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.147+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'709 uv709 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.147+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.151+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v119'710 uv684 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.151+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.151+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v119'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.151+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.528 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.155+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v119'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.155+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.155+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.155+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.155+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.155+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.159+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.159+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.159+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v119'716 uv690 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.159+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.163+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v119'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.163+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.531 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:23.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.163+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v119'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.163+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d95680 con 0x560562d8b880 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d8b880 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560562d96640 0x560562d98a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d98f70 con 0x560562d96640 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.167+0000 7fdd36600640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560562d96640 0x560562d98a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.171+0000 7fdd36600640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560562d96640 0x560562d98a30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.171+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'505 uv488 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.171+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d99310 con 0x560562d96640 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.171+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'506 uv506 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.171+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d996b0 con 0x560562d96640 2024-09-10T21:33:23.534 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.175+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'507 uv507 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.175+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d99a50 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.175+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v119'508 uv508 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.175+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d99df0 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.179+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'509 uv509 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.179+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a190 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.179+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v119'510 uv493 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.179+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d9a530 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.179+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v119'511 uv511 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.179+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a920 con 0x560562d96640 2024-09-10T21:33:23.536 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.183+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v119'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.183+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a920 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.183+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.183+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.187+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.187+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.187+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v119'515 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.187+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.191+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v119'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.191+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.538 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.191+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v119'517 uv517 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.191+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.191+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v119'518 uv518 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.191+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.195+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'519 uv519 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.195+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.195+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v119'520 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.195+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.199+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v119'521 uv521 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.199+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.540 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:23.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.199+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v119'522 uv522 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.199+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.199+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'523 uv523 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.199+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9b8e0 con 0x560562d96640 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.203+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'524 uv524 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.203+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9a8d0 con 0x560562d96640 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.203+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562d96640 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.203+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560562d9af50 0x560562d9b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.203+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9de00 con 0x560562d9af50 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.203+0000 7fdd35c00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560562d9af50 0x560562d9b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.203+0000 7fdd35c00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560562d9af50 0x560562d9b330 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562d9e1a0 con 0x560562d9af50 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9e540 con 0x560562d9af50 2024-09-10T21:33:23.543 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:23.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9f990 con 0x560562d9af50 2024-09-10T21:33:23.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.207+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9fe10 con 0x560562d9af50 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.211+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.211+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562d9e8e0 con 0x560562d9af50 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.211+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.211+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560562d9f350 0x560562da2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.211+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2790 con 0x560562d9f350 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.211+0000 7fdd37000640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560562d9f350 0x560562da2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.211+0000 7fdd37000640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560562d9f350 0x560562da2250 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'103 uv102 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da2b80 con 0x560562d9f350 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'104 uv104 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2b80 con 0x560562d9f350 2024-09-10T21:33:23.546 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'105 uv105 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2b80 con 0x560562d9f350 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'106 uv106 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.215+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2ac0 con 0x560562d9af50 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.219+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.219+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da2f70 con 0x560562d9af50 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.219+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.219+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2f70 con 0x560562d9af50 2024-09-10T21:33:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.219+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.548 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.219+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da3f80 con 0x560562d9af50 2024-09-10T21:33:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.223+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.223+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da3f80 con 0x560562d9af50 2024-09-10T21:33:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.223+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.223+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2f70 con 0x560562d9af50 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.223+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.223+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2f70 con 0x560562d9f350 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v119'249 uv243 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da2f70 con 0x560562d9f350 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v119'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2f20 con 0x560562d9f350 2024-09-10T21:33:23.550 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v119'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da3f30 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.227+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da3f30 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.231+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.231+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2f20 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.231+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.231+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2ed0 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.231+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v119'255 uv248 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.231+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da2ed0 con 0x560562d9f350 2024-09-10T21:33:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v119'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2ed0 con 0x560562d9f350 2024-09-10T21:33:23.553 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v119'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2ed0 con 0x560562d9f350 2024-09-10T21:33:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2ed0 con 0x560562d9f350 2024-09-10T21:33:23.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.235+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2ed0 con 0x560562d9af50 2024-09-10T21:33:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.239+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v119'112 uv111 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.239+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da2e60 con 0x560562d9af50 2024-09-10T21:33:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.239+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v119'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.239+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2e60 con 0x560562d9af50 2024-09-10T21:33:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.239+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v119'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.555 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.239+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2e60 con 0x560562d9af50 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da2e60 con 0x560562d9af50 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562d9af50 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560562da3840 0x560562da5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da6170 con 0x560562da3840 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd36600640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560562da3840 0x560562da5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd36600640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560562da3840 0x560562da5c30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.243+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da61f0 con 0x560562da3840 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.247+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.247+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.558 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:23.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.247+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.247+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da75f0 con 0x560562da3840 2024-09-10T21:33:23.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.247+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.247+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da75f0 con 0x560562da3840 2024-09-10T21:33:23.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v119'223 uv217 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v119'224 uv224 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.251+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.560 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:23.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.255+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v119'225 uv225 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.255+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.255+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'226 uv226 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.255+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.255+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'227 uv227 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.255+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v119'228 uv222 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v119'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.562 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:23.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v119'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da65e0 con 0x560562da3840 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.259+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da93f0 con 0x560562da3840 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.263+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.263+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.263+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v119'99 uv98 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.263+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.263+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'100 uv100 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.263+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.564 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:23.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.267+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v119'101 uv101 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.267+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.267+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'102 uv102 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.267+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.267+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.267+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.566 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7aa0 con 0x560562d9f350 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7aa0 con 0x560562d9f350 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.271+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da6a90 con 0x560562d9f350 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd30002050 con 0x560562d9f350 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560562da7110 0x560562da74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da7110 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd35c00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560562da7110 0x560562da74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd35c00640 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560562da7110 0x560562da74f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v119'115 uv114 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562da7110 2024-09-10T21:33:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da7110 2024-09-10T21:33:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v119'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562da7110 2024-09-10T21:33:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.275+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da7110 2024-09-10T21:33:23.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.279+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v119'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562da7110 2024-09-10T21:33:23.569 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:23.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.279+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da7110 2024-09-10T21:33:23.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.279+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562da7110 2024-09-10T21:33:23.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.279+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da7110 2024-09-10T21:33:23.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.279+0000 7fdd35c00640 1 -- 172.21.15.97:0/77035978 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdd2405ad70 con 0x560562da7110 2024-09-10T21:33:23.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.279+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.283+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v119'278 uv277 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.283+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.283+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v119'279 uv279 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.283+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.571 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:23.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.283+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v119'280 uv280 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.283+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v119'281 uv281 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'282 uv282 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v119'283 uv276 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v119'284 uv284 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.287+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.574 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.291+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v119'285 uv285 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.291+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.291+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'286 uv286 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.291+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.291+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'287 uv287 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.291+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da7a30 con 0x560562da3840 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'288 uv288 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x560562da9060 con 0x560562da3840 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd36600640 1 -- 172.21.15.97:0/77035978 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'289 uv289 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdd2800d040 con 0x560562da3840 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560562d9af50 msgr2=0x560562d9b330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x560562d9af50 0x560562d9b330 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560562da3840 msgr2=0x560562da5c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560562da3840 0x560562da5c30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560562da7110 msgr2=0x560562da74f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x560562da7110 0x560562da74f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560562d9f350 msgr2=0x560562da2250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x560562d9f350 0x560562da2250 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 reap_dead start 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560562d96640 msgr2=0x560562d98a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x560562d96640 0x560562d98a30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560562d8b880 msgr2=0x560562d8bc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x560562d8b880 0x560562d8bc60 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdd18070ff0 msgr2=0x7fdd180734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdd18070ff0 0x7fdd180734b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdd28011570 tx=0x7fdd280017e0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd50f0 msgr2=0x560562ceb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 --2- 172.21.15.97:0/77035978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560562cd50f0 0x560562ceb000 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x560562c463d0 tx=0x7fdd24004e00 comp rx=0 tx=0).stop 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd37000640 1 -- 172.21.15.97:0/77035978 reap_dead start 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 shutdown_connections 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 >> 172.21.15.97:0/77035978 conn(0x560562c17140 msgr2=0x560562cd16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 shutdown_connections 2024-09-10T21:33:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.295+0000 7fdd3944e080 1 -- 172.21.15.97:0/77035978 wait complete. 2024-09-10T21:33:23.578 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2024-09-10T21:33:23.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 Processor -- start 2024-09-10T21:33:23.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 -- start start 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fadd4003b10 0x556ec3a22350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39af940 0x556ec3a26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ec39afd50 0x556ec3a272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ec3915f40 con 0x556ec39afd50 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ec3922d00 con 0x556ec39af940 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf23f6080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556ec38b7a00 con 0x7fadd4003b10 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ec39afd50 0x556ec3a272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ec39afd50 0x556ec3a272c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33700/0 (socket says 172.21.15.97:33700) 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 -- 172.21.15.97:0/2793546875 learned_addr learned my addr 172.21.15.97:0/2793546875 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadeec00640 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39af940 0x556ec3a26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadef600640 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fadd4003b10 0x556ec3a22350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 -- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fadd4003b10 msgr2=0x556ec3a22350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fadd4003b10 0x556ec3a22350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 -- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39af940 msgr2=0x556ec3a26d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39af940 0x556ec3a26d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 -- 172.21.15.97:0/2793546875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec3969cd0 con 0x556ec39afd50 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadeec00640 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39af940 0x556ec3a26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.667+0000 7fadf0000640 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ec39afd50 0x556ec3a272c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fade801eac0 tx=0x7fadd4004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1042f0b77487cde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7faded800640 1 -- 172.21.15.97:0/2793546875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8001a80 con 0x556ec39afd50 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7faded800640 1 -- 172.21.15.97:0/2793546875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fade80090d0 con 0x556ec39afd50 2024-09-10T21:33:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7faded800640 1 -- 172.21.15.97:0/2793546875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80a7a30 con 0x556ec39afd50 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- 172.21.15.97:0/2793546875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ec39afd50 msgr2=0x556ec3a272c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ec39afd50 0x556ec3a272c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fade801eac0 tx=0x7fadd4004130 comp rx=0 tx=0).stop 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- 172.21.15.97:0/2793546875 shutdown_connections 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ec39afd50 0x556ec3a272c0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39af940 0x556ec3a26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2793546875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fadd4003b10 0x556ec3a22350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- 172.21.15.97:0/2793546875 >> 172.21.15.97:0/2793546875 conn(0x556ec39b10d0 msgr2=0x556ec39b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- 172.21.15.97:0/2793546875 shutdown_connections 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- 172.21.15.97:0/2793546875 wait complete. 2024-09-10T21:33:23.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 Processor -- start 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- start start 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ec39af940 0x556ec3968a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39afd50 0x556ec3a3ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd4003b10 0x556ec39b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ec3a22de0 con 0x7fadd4003b10 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ec3a22b40 con 0x556ec39afd50 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556ec3a23080 con 0x556ec39af940 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd4003b10 0x556ec39b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd4003b10 0x556ec39b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33716/0 (socket says 172.21.15.97:33716) 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 -- 172.21.15.97:0/2662116669 learned_addr learned my addr 172.21.15.97:0/2662116669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadeec00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39afd50 0x556ec3a3ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadef600640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ec39af940 0x556ec3968a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 -- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ec39af940 msgr2=0x556ec3968a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556ec39af940 0x556ec3968a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 -- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39afd50 msgr2=0x556ec3a3ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ec39afd50 0x556ec3a3ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 -- 172.21.15.97:0/2662116669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec3ac64a0 con 0x7fadd4003b10 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf0000640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd4003b10 0x556ec39b01d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fade80b3040 tx=0x7fade8009160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fade7e00640 1 -- 172.21.15.97:0/2662116669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80093a0 con 0x7fadd4003b10 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fade7e00640 1 -- 172.21.15.97:0/2662116669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fade80c0020 con 0x7fadd4003b10 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fade7e00640 1 -- 172.21.15.97:0/2662116669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8001a50 con 0x7fadd4003b10 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ec3ac6740 con 0x7fadd4003b10 2024-09-10T21:33:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.671+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec3ac6c80 con 0x7fadd4003b10 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fade7e00640 1 -- 172.21.15.97:0/2662116669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fade80a7a30 con 0x7fadd4003b10 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fade7e00640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fadd0070fc0 0x7fadd0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fade7e00640 1 -- 172.21.15.97:0/2662116669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7fade81306a0 con 0x7fadd4003b10 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fadef600640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fadd0070fc0 0x7fadd0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ec39b24d0 0x556ec39b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x556ec3a3e410 con 0x556ec39b24d0 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fadeec00640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ec39b24d0 0x556ec39b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fadef600640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fadd0070fc0 0x7fadd0073480 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fade000c950 tx=0x7fade000f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.675+0000 7fadeec00640 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ec39b24d0 0x556ec39b28b0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadeec00640 1 -- 172.21.15.97:0/2662116669 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v119'526 uv526 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7faddc00b040 con 0x556ec39b24d0 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ec39b24d0 msgr2=0x556ec39b28b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556ec39b24d0 0x556ec39b28b0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fadd0070fc0 msgr2=0x7fadd0073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fadd0070fc0 0x7fadd0073480 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fade000c950 tx=0x7fade000f040 comp rx=0 tx=0).stop 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd4003b10 msgr2=0x556ec39b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 --2- 172.21.15.97:0/2662116669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd4003b10 0x556ec39b01d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fade80b3040 tx=0x7fade8009160 comp rx=0 tx=0).stop 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf0000640 1 -- 172.21.15.97:0/2662116669 reap_dead start 2024-09-10T21:33:23.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 shutdown_connections 2024-09-10T21:33:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 >> 172.21.15.97:0/2662116669 conn(0x556ec39b10d0 msgr2=0x556ec39b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 shutdown_connections 2024-09-10T21:33:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.679+0000 7fadf23f6080 1 -- 172.21.15.97:0/2662116669 wait complete. 2024-09-10T21:33:23.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:23.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- 172.21.15.97:0/1965412572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f800815fde0 msgr2=0x7f8008166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 --2- 172.21.15.97:0/1965412572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f800815fde0 0x7f8008166850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8000009bf0 tx=0x7f800001cbe0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- 172.21.15.97:0/1965412572 shutdown_connections 2024-09-10T21:33:23.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 --2- 172.21.15.97:0/1965412572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f800815fde0 0x7f8008166850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- 172.21.15.97:0/1965412572 >> 172.21.15.97:0/1965412572 conn(0x7f80080b1780 msgr2=0x7f80080b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:23.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- 172.21.15.97:0/1965412572 shutdown_connections 2024-09-10T21:33:23.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- 172.21.15.97:0/1965412572 wait complete. 2024-09-10T21:33:23.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 Processor -- start 2024-09-10T21:33:23.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- start start 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80080ce5f0 0x7f8008139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8008141d20 0x7f8008144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f800815fde0 0x7f8008144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f800816a180 con 0x7f80080ce5f0 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f800816a000 con 0x7f800815fde0 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f800d400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f800816a300 con 0x7f8008141d20 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80080ce5f0 0x7f8008139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80080ce5f0 0x7f8008139bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33732/0 (socket says 172.21.15.97:33732) 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 -- 172.21.15.97:0/1353377425 learned_addr learned my addr 172.21.15.97:0/1353377425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f7ffde00640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8008141d20 0x7f8008144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8007200640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f800815fde0 0x7f8008144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 -- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8008141d20 msgr2=0x7f8008144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8008141d20 0x7f8008144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 -- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f800815fde0 msgr2=0x7f8008144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f800815fde0 0x7f8008144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.871+0000 7f8006800640 1 -- 172.21.15.97:0/1353377425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800813a130 con 0x7f80080ce5f0 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f8006800640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80080ce5f0 0x7f8008139bf0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c4c0 tx=0x7f7ff800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f7fff400640 1 -- 172.21.15.97:0/1353377425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80052d0 con 0x7f80080ce5f0 2024-09-10T21:33:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f7fff400640 1 -- 172.21.15.97:0/1353377425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff8005490 con 0x7f80080ce5f0 2024-09-10T21:33:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8008142610 con 0x7f80080ce5f0 2024-09-10T21:33:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f7fff400640 1 -- 172.21.15.97:0/1353377425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8012870 con 0x7f80080ce5f0 2024-09-10T21:33:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8008142f70 con 0x7f80080ce5f0 2024-09-10T21:33:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80080ce9d0 con 0x7f80080ce5f0 2024-09-10T21:33:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f7fff400640 1 -- 172.21.15.97:0/1353377425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ff8012a10 con 0x7f80080ce5f0 2024-09-10T21:33:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f7fff400640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fe40725e0 0x7f7fe4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f7fff400640 1 -- 172.21.15.97:0/1353377425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8253+0+0 (secure 0 0 0) 0x7f7ff8093690 con 0x7f80080ce5f0 2024-09-10T21:33:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.875+0000 7f7ffde00640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fe40725e0 0x7f7fe4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.879+0000 7f7ffde00640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fe40725e0 0x7f7fe4074aa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0007f30 tx=0x7f7ff0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:23.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:23.879+0000 7f7fff400640 1 -- 172.21.15.97:0/1353377425 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ff805dce0 con 0x7f80080ce5f0 2024-09-10T21:33:24.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:24.079+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f80080cca70 con 0x7f80080ce5f0 2024-09-10T21:33:24.763 INFO:tasks.ceph.mds.a.smithi097.stderr: -49> 2024-09-10T21:33:20.011+0000 7f1660600640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-10T21:33:24.763 INFO:tasks.ceph.mds.a.smithi097.stderr: -41> 2024-09-10T21:33:20.011+0000 7f1660600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:33:24.802 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:33:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.047+0000 7f7fff400640 1 -- 172.21.15.97:0/1353377425 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v224) ==== 162+0+0 (secure 0 0 0) 0x7f7ff8062710 con 0x7f80080ce5f0 2024-09-10T21:33:25.046 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fe40725e0 msgr2=0x7f7fe4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fe40725e0 0x7f7fe4074aa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0007f30 tx=0x7f7ff0007ec0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80080ce5f0 msgr2=0x7f8008139bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80080ce5f0 0x7f8008139bf0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c4c0 tx=0x7f7ff800c980 comp rx=0 tx=0).stop 2024-09-10T21:33:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 shutdown_connections 2024-09-10T21:33:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7fe40725e0 0x7f7fe4074aa0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f800815fde0 0x7f8008144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8008141d20 0x7f8008144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 --2- 172.21.15.97:0/1353377425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80080ce5f0 0x7f8008139bf0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 >> 172.21.15.97:0/1353377425 conn(0x7f80080b1780 msgr2=0x7f800813e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 shutdown_connections 2024-09-10T21:33:25.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.051+0000 7f800d400640 1 -- 172.21.15.97:0/1353377425 wait complete. 2024-09-10T21:33:25.062 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:33:25.063+0000 7fc57aa00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T21:33:25.062 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:33:25.063+0000 7fc57aa00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:33:25.062 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:25.063+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:25.062 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:25.063+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:25.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 -- 172.21.15.97:0/3460429531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f268806dc50 msgr2=0x7f26880cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 --2- 172.21.15.97:0/3460429531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f268806dc50 0x7f26880cfe30 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f26880d2d10 tx=0x7f267001c830 comp rx=0 tx=0).stop 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 -- 172.21.15.97:0/3460429531 shutdown_connections 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 --2- 172.21.15.97:0/3460429531 >> v2:172.21.15.194:3300/0 conn(0x7f26880b1590 0x7f26880d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 --2- 172.21.15.97:0/3460429531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f268806dc50 0x7f26880cfe30 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 --2- 172.21.15.97:0/3460429531 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2688161600 0x7f26880d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 -- 172.21.15.97:0/3460429531 >> 172.21.15.97:0/3460429531 conn(0x7f26880d1c30 msgr2=0x7f26880d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 -- 172.21.15.97:0/3460429531 shutdown_connections 2024-09-10T21:33:25.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.219+0000 7f268cc00640 1 -- 172.21.15.97:0/3460429531 wait complete. 2024-09-10T21:33:25.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 Processor -- start 2024-09-10T21:33:25.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 -- start start 2024-09-10T21:33:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26880b1590 0x7f26880c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2688161600 0x7f26881f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26881f7de0 0x7f2688158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26880afd10 con 0x7f26881f7de0 2024-09-10T21:33:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26880afb90 con 0x7f26880b1590 2024-09-10T21:33:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f26880afe90 con 0x7f2688161600 2024-09-10T21:33:25.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26881f7de0 0x7f2688158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2685e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26880b1590 0x7f26880c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26881f7de0 0x7f2688158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33764/0 (socket says 172.21.15.97:33764) 2024-09-10T21:33:25.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 -- 172.21.15.97:0/3564007122 learned_addr learned my addr 172.21.15.97:0/3564007122 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:25.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f267d400640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2688161600 0x7f26881f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 -- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2688161600 msgr2=0x7f26881f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2688161600 0x7f26881f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 -- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26880b1590 msgr2=0x7f26880c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26880b1590 0x7f26880c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 -- 172.21.15.97:0/3564007122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26880b70f0 con 0x7f26881f7de0 2024-09-10T21:33:25.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2685e00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26880b1590 0x7f26880c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:25.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f2686800640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26881f7de0 0x7f2688158d60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f268000e9d0 tx=0x7f268000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:25.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f267ea00640 1 -- 172.21.15.97:0/3564007122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268000cde0 con 0x7f26881f7de0 2024-09-10T21:33:25.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f267ea00640 1 -- 172.21.15.97:0/3564007122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2680004c20 con 0x7f26881f7de0 2024-09-10T21:33:25.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26880b73d0 con 0x7f26881f7de0 2024-09-10T21:33:25.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f267ea00640 1 -- 172.21.15.97:0/3564007122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680005020 con 0x7f26881f7de0 2024-09-10T21:33:25.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.223+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26880b6dc0 con 0x7f26881f7de0 2024-09-10T21:33:25.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.227+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26880d0a60 con 0x7f26881f7de0 2024-09-10T21:33:25.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.227+0000 7f267ea00640 1 -- 172.21.15.97:0/3564007122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2680020030 con 0x7f26881f7de0 2024-09-10T21:33:25.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.227+0000 7f267ea00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26640725e0 0x7f2664074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.227+0000 7f267ea00640 1 -- 172.21.15.97:0/3564007122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7f2680094670 con 0x7f26881f7de0 2024-09-10T21:33:25.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.227+0000 7f2685e00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26640725e0 0x7f2664074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.227+0000 7f2685e00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26640725e0 0x7f2664074aa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f26880cf2a0 tx=0x7f26780091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:25.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.231+0000 7f267ea00640 1 -- 172.21.15.97:0/3564007122 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f268005ec90 con 0x7f26881f7de0 2024-09-10T21:33:25.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.431+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f268815d600 con 0x7f26881f7de0 2024-09-10T21:33:25.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.431+0000 7f267ea00640 1 -- 172.21.15.97:0/3564007122 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) ==== 95+0+4392 (secure 0 0 0) 0x7f26800636c0 con 0x7f26881f7de0 2024-09-10T21:33:25.430 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:25.430 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":226,"btime":"2024-09-10T21:33:25:066071+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/899393328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":899393328},{"type":"v1","addr":"172.21.15.97:6835","nonce":899393328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":221}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:25.065994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:25.431 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 226 2024-09-10T21:33:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.431+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26640725e0 msgr2=0x7f2664074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.431+0000 7f268cc00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26640725e0 0x7f2664074aa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f26880cf2a0 tx=0x7f26780091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.431+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26881f7de0 msgr2=0x7f2688158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.431+0000 7f268cc00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26881f7de0 0x7f2688158d60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f268000e9d0 tx=0x7f268000ee90 comp rx=0 tx=0).stop 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 shutdown_connections 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26640725e0 0x7f2664074aa0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26881f7de0 0x7f2688158d60 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2688161600 0x7f26881f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 --2- 172.21.15.97:0/3564007122 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26880b1590 0x7f26880c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 >> 172.21.15.97:0/3564007122 conn(0x7f26880d1c30 msgr2=0x7f2688070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 shutdown_connections 2024-09-10T21:33:25.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.435+0000 7f268cc00640 1 -- 172.21.15.97:0/3564007122 wait complete. 2024-09-10T21:33:25.453 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:25.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:25.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 -- 172.21.15.97:0/3748323144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040cec10 msgr2=0x7fdd040cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 --2- 172.21.15.97:0/3748323144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040cec10 0x7fdd040cff10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009840 tx=0x7fdcec01f010 comp rx=0 tx=0).stop 2024-09-10T21:33:25.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 -- 172.21.15.97:0/3748323144 shutdown_connections 2024-09-10T21:33:25.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 --2- 172.21.15.97:0/3748323144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040cec10 0x7fdd040cff10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 -- 172.21.15.97:0/3748323144 >> 172.21.15.97:0/3748323144 conn(0x7fdd040b37c0 msgr2=0x7fdd040b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:25.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 -- 172.21.15.97:0/3748323144 shutdown_connections 2024-09-10T21:33:25.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 -- 172.21.15.97:0/3748323144 wait complete. 2024-09-10T21:33:25.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 Processor -- start 2024-09-10T21:33:25.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.595+0000 7fdd09800640 1 -- start start 2024-09-10T21:33:25.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040b1560 0x7fdd0415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd040ce830 0x7fdd041f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdd040cec10 0x7fdd0415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd040ce320 con 0x7fdd040b1560 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdd040ce1a0 con 0x7fdd040ce830 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdd040d0450 con 0x7fdd040cec10 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040b1560 0x7fdd0415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040b1560 0x7fdd0415c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33776/0 (socket says 172.21.15.97:33776) 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 -- 172.21.15.97:0/3391177536 learned_addr learned my addr 172.21.15.97:0/3391177536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdcfa000640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd040ce830 0x7fdd041f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd03400640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdd040cec10 0x7fdd0415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 -- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdd040cec10 msgr2=0x7fdd0415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdd040cec10 0x7fdd0415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 -- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd040ce830 msgr2=0x7fdd041f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd040ce830 0x7fdd041f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 -- 172.21.15.97:0/3391177536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd040c87e0 con 0x7fdd040b1560 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd03400640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdd040cec10 0x7fdd0415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd02a00640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040b1560 0x7fdd0415c0f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c4b0 tx=0x7fdcf400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdcfb400640 1 -- 172.21.15.97:0/3391177536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40052d0 con 0x7fdd040b1560 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdcfb400640 1 -- 172.21.15.97:0/3391177536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcf4005470 con 0x7fdd040b1560 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd041f29c0 con 0x7fdd040b1560 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdcfb400640 1 -- 172.21.15.97:0/3391177536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4012870 con 0x7fdd040b1560 2024-09-10T21:33:25.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd041f2f30 con 0x7fdd040b1560 2024-09-10T21:33:25.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd040b1970 con 0x7fdd040b1560 2024-09-10T21:33:25.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.599+0000 7fdcfb400640 1 -- 172.21.15.97:0/3391177536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdcf4003600 con 0x7fdd040b1560 2024-09-10T21:33:25.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.603+0000 7fdcfb400640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdce00725e0 0x7fdce0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.603+0000 7fdcfb400640 1 -- 172.21.15.97:0/3391177536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7fdcf40938e0 con 0x7fdd040b1560 2024-09-10T21:33:25.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.603+0000 7fdcfa000640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdce00725e0 0x7fdce0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.603+0000 7fdcfa000640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdce00725e0 0x7fdce0074aa0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0b2000 tx=0x7fdcec01d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:25.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.607+0000 7fdcfb400640 1 -- 172.21.15.97:0/3391177536 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd040c87e0 con 0x7fdd040b1560 2024-09-10T21:33:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.803+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd040b0c90 con 0x7fdd040b1560 2024-09-10T21:33:25.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.803+0000 7fdcfb400640 1 -- 172.21.15.97:0/3391177536 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) ==== 95+0+4392 (secure 0 0 0) 0x7fdcf405e520 con 0x7fdd040b1560 2024-09-10T21:33:25.804 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:25.804 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":226,"btime":"2024-09-10T21:33:25:066071+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/899393328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":899393328},{"type":"v1","addr":"172.21.15.97:6835","nonce":899393328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":221}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:25.065994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 226 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdce00725e0 msgr2=0x7fdce0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdce00725e0 0x7fdce0074aa0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0b2000 tx=0x7fdcec01d850 comp rx=0 tx=0).stop 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040b1560 msgr2=0x7fdd0415c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040b1560 0x7fdd0415c0f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c4b0 tx=0x7fdcf400c970 comp rx=0 tx=0).stop 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 shutdown_connections 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdce00725e0 0x7fdce0074aa0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdd040cec10 0x7fdd0415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd040ce830 0x7fdd041f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 --2- 172.21.15.97:0/3391177536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd040b1560 0x7fdd0415c0f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 >> 172.21.15.97:0/3391177536 conn(0x7fdd040b37c0 msgr2=0x7fdd040cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 shutdown_connections 2024-09-10T21:33:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.807+0000 7fdd09800640 1 -- 172.21.15.97:0/3391177536 wait complete. 2024-09-10T21:33:25.824 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:25.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:25.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.967+0000 7f890bc00640 1 -- 172.21.15.97:0/3302056092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89040b2bb0 msgr2=0x7f89040b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.967+0000 7f890bc00640 1 --2- 172.21.15.97:0/3302056092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89040b2bb0 0x7f89040b2f90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009840 tx=0x7f88f401c790 comp rx=0 tx=0).stop 2024-09-10T21:33:25.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.967+0000 7f890bc00640 1 -- 172.21.15.97:0/3302056092 shutdown_connections 2024-09-10T21:33:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.967+0000 7f890bc00640 1 --2- 172.21.15.97:0/3302056092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89040b2bb0 0x7f89040b2f90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.967+0000 7f890bc00640 1 -- 172.21.15.97:0/3302056092 >> 172.21.15.97:0/3302056092 conn(0x7f89040b37c0 msgr2=0x7f89040b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.967+0000 7f890bc00640 1 -- 172.21.15.97:0/3302056092 shutdown_connections 2024-09-10T21:33:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.967+0000 7f890bc00640 1 -- 172.21.15.97:0/3302056092 wait complete. 2024-09-10T21:33:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 Processor -- start 2024-09-10T21:33:25.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 -- start start 2024-09-10T21:33:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f89040b2bb0 0x7f89040afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89040b42e0 0x7f89040b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904072280 0x7f89040b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f890416bd60 con 0x7f89040b42e0 2024-09-10T21:33:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f890416bbe0 con 0x7f8904072280 2024-09-10T21:33:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f890416bee0 con 0x7f89040b2bb0 2024-09-10T21:33:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89040b42e0 0x7f89040b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89040b42e0 0x7f89040b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33784/0 (socket says 172.21.15.97:33784) 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 -- 172.21.15.97:0/4273650808 learned_addr learned my addr 172.21.15.97:0/4273650808 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f8909c00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904072280 0x7f89040b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f8909200640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f89040b2bb0 0x7f89040afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 -- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f89040b2bb0 msgr2=0x7f89040afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f89040b2bb0 0x7f89040afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 -- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904072280 msgr2=0x7f89040b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904072280 0x7f89040b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 -- 172.21.15.97:0/4273650808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89041f7990 con 0x7f89040b42e0 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f8909c00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904072280 0x7f89040b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f88fbe00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89040b42e0 0x7f89040b5e40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c460 tx=0x7f88ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f8901600640 1 -- 172.21.15.97:0/4273650808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec011850 con 0x7f89040b42e0 2024-09-10T21:33:25.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f8901600640 1 -- 172.21.15.97:0/4273650808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88ec017070 con 0x7f89040b42e0 2024-09-10T21:33:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f8901600640 1 -- 172.21.15.97:0/4273650808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec011de0 con 0x7f89040b42e0 2024-09-10T21:33:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89041f7c50 con 0x7f89040b42e0 2024-09-10T21:33:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041f8210 con 0x7f89040b42e0 2024-09-10T21:33:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.971+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89040b2f90 con 0x7f89040b42e0 2024-09-10T21:33:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.975+0000 7f8901600640 1 -- 172.21.15.97:0/4273650808 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f88ec01c460 con 0x7f89040b42e0 2024-09-10T21:33:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.975+0000 7f8901600640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88e00725e0 0x7f88e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.975+0000 7f8901600640 1 -- 172.21.15.97:0/4273650808 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7f88ec093ba0 con 0x7f89040b42e0 2024-09-10T21:33:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.975+0000 7f8909200640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88e00725e0 0x7f88e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:25.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.975+0000 7f8909200640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88e00725e0 0x7f88e0074aa0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f88f40af000 tx=0x7f88f401afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:25.979+0000 7f8901600640 1 -- 172.21.15.97:0/4273650808 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f88ec05e780 con 0x7f89040b42e0 2024-09-10T21:33:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f890406d7e0 con 0x7f89040b42e0 2024-09-10T21:33:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f8901600640 1 -- 172.21.15.97:0/4273650808 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) ==== 95+0+4392 (secure 0 0 0) 0x7f88ec05e080 con 0x7f89040b42e0 2024-09-10T21:33:26.177 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:26.177 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":226,"btime":"2024-09-10T21:33:25:066071+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/899393328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":899393328},{"type":"v1","addr":"172.21.15.97:6835","nonce":899393328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":221}],"filesystems":[{"mdsmap":{"epoch":226,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:25.065994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:26.178 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 226 2024-09-10T21:33:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88e00725e0 msgr2=0x7f88e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88e00725e0 0x7f88e0074aa0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f88f40af000 tx=0x7f88f401afd0 comp rx=0 tx=0).stop 2024-09-10T21:33:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89040b42e0 msgr2=0x7f89040b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89040b42e0 0x7f89040b5e40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c460 tx=0x7f88ec00c920 comp rx=0 tx=0).stop 2024-09-10T21:33:26.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 shutdown_connections 2024-09-10T21:33:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88e00725e0 0x7f88e0074aa0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8904072280 0x7f89040b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89040b42e0 0x7f89040b5e40 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 --2- 172.21.15.97:0/4273650808 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f89040b2bb0 0x7f89040afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 >> 172.21.15.97:0/4273650808 conn(0x7f89040b37c0 msgr2=0x7f8904142eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 shutdown_connections 2024-09-10T21:33:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.179+0000 7f890bc00640 1 -- 172.21.15.97:0/4273650808 wait complete. 2024-09-10T21:33:26.198 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2024-09-10T21:33:26.198 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2024-09-10T21:33:26.199 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:26.199 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:26.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:26.208 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:26.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:26.208 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:26.259 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:26.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:26.260 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:26.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.447+0000 7fd252800640 1 -- 172.21.15.97:0/1250908717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd230003a10 msgr2=0x7fd230003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:26.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.447+0000 7fd252800640 1 --2- 172.21.15.97:0/1250908717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd230003a10 0x7fd230003e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00a6b0 tx=0x7fd23c01c890 comp rx=0 tx=0).stop 2024-09-10T21:33:26.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.447+0000 7fd24be00640 1 -- 172.21.15.97:0/1250908717 reap_dead start 2024-09-10T21:33:26.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.447+0000 7fd252800640 1 -- 172.21.15.97:0/1250908717 shutdown_connections 2024-09-10T21:33:26.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.447+0000 7fd252800640 1 -- 172.21.15.97:0/1250908717 >> 172.21.15.97:0/1250908717 conn(0x7fd24c0b1780 msgr2=0x7fd24c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:26.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.447+0000 7fd252800640 1 -- 172.21.15.97:0/1250908717 shutdown_connections 2024-09-10T21:33:26.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.447+0000 7fd252800640 1 -- 172.21.15.97:0/1250908717 wait complete. 2024-09-10T21:33:26.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 Processor -- start 2024-09-10T21:33:26.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 -- start start 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd230003a10 0x7fd24c13a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd24c13a8e0 0x7fd24c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd24c14d170 0x7fd24c14d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c15e0d0 con 0x7fd230003a10 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c070930 con 0x7fd24c13a8e0 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd24c15e250 con 0x7fd24c14d170 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd230003a10 0x7fd24c13a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd230003a10 0x7fd24c13a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33800/0 (socket says 172.21.15.97:33800) 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 -- 172.21.15.97:0/1790361628 learned_addr learned my addr 172.21.15.97:0/1790361628 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:26.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd242a00640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd24c13a8e0 0x7fd24c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24be00640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd24c14d170 0x7fd24c14d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 -- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd24c14d170 msgr2=0x7fd24c14d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd24c14d170 0x7fd24c14d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 -- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd24c13a8e0 msgr2=0x7fd24c154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd24c13a8e0 0x7fd24c154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 -- 172.21.15.97:0/1790361628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c13cb10 con 0x7fd230003a10 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd24b400640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd230003a10 0x7fd24c13a080 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd23c000c10 tx=0x7fd23c0a6ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd243e00640 1 -- 172.21.15.97:0/1790361628 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00a310 con 0x7fd230003a10 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd243e00640 1 -- 172.21.15.97:0/1790361628 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd23c0a6550 con 0x7fd230003a10 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c13a5c0 con 0x7fd230003a10 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd243e00640 1 -- 172.21.15.97:0/1790361628 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c005470 con 0x7fd230003a10 2024-09-10T21:33:26.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.451+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c13d090 con 0x7fd230003a10 2024-09-10T21:33:26.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.455+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24c06e3b0 con 0x7fd230003a10 2024-09-10T21:33:26.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.455+0000 7fd243e00640 1 -- 172.21.15.97:0/1790361628 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd23c0046d0 con 0x7fd230003a10 2024-09-10T21:33:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.455+0000 7fd243e00640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd228072610 0x7fd228074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.455+0000 7fd243e00640 1 -- 172.21.15.97:0/1790361628 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7fd23c12de00 con 0x7fd230003a10 2024-09-10T21:33:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.455+0000 7fd242a00640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd228072610 0x7fd228074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:26.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.455+0000 7fd242a00640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd228072610 0x7fd228074ad0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd24c1e5a20 tx=0x7fd2340091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:26.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.459+0000 7fd243e00640 1 -- 172.21.15.97:0/1790361628 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd23c0bf060 con 0x7fd230003a10 2024-09-10T21:33:26.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:26.659+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd24c154670 con 0x7fd230003a10 2024-09-10T21:33:27.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.055+0000 7fd243e00640 1 -- 172.21.15.97:0/1790361628 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v227) ==== 161+0+0 (secure 0 0 0) 0x7fd23c0ab020 con 0x7fd230003a10 2024-09-10T21:33:27.056 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd228072610 msgr2=0x7fd228074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd228072610 0x7fd228074ad0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd24c1e5a20 tx=0x7fd2340091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd230003a10 msgr2=0x7fd24c13a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd230003a10 0x7fd24c13a080 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd23c000c10 tx=0x7fd23c0a6ec0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 shutdown_connections 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd228072610 0x7fd228074ad0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd24c14d170 0x7fd24c14d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd24c13a8e0 0x7fd24c154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 --2- 172.21.15.97:0/1790361628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd230003a10 0x7fd24c13a080 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 >> 172.21.15.97:0/1790361628 conn(0x7fd24c0b1780 msgr2=0x7fd24c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 shutdown_connections 2024-09-10T21:33:27.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.059+0000 7fd252800640 1 -- 172.21.15.97:0/1790361628 wait complete. 2024-09-10T21:33:27.078 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- 172.21.15.97:0/43591415 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe3800ceee0 msgr2=0x7fe3800cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 --2- 172.21.15.97:0/43591415 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe3800ceee0 0x7fe3800cf300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe368009c80 tx=0x7fe36801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- 172.21.15.97:0/43591415 shutdown_connections 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 --2- 172.21.15.97:0/43591415 >> v2:172.21.15.194:3300/0 conn(0x7fe38015a730 0x7fe38015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 --2- 172.21.15.97:0/43591415 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe3800ceee0 0x7fe3800cf300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- 172.21.15.97:0/43591415 >> 172.21.15.97:0/43591415 conn(0x7fe3800b1780 msgr2=0x7fe3800b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- 172.21.15.97:0/43591415 shutdown_connections 2024-09-10T21:33:27.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- 172.21.15.97:0/43591415 wait complete. 2024-09-10T21:33:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 Processor -- start 2024-09-10T21:33:27.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- start start 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3800ceee0 0x7fe38013d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3801511f0 0x7fe380139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38015a730 0x7fe38013a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe38006e530 con 0x7fe3801511f0 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe38006e3b0 con 0x7fe3800ceee0 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe385600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe38006e6b0 con 0x7fe38015a730 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe376000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3801511f0 0x7fe380139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe376000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3801511f0 0x7fe380139d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33814/0 (socket says 172.21.15.97:33814) 2024-09-10T21:33:27.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe376000640 1 -- 172.21.15.97:0/3430686518 learned_addr learned my addr 172.21.15.97:0/3430686518 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe37ea00640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3800ceee0 0x7fe38013d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.223+0000 7fe37f400640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38015a730 0x7fe38013a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe376000640 1 -- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38015a730 msgr2=0x7fe38013a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe376000640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38015a730 0x7fe38013a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe376000640 1 -- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3800ceee0 msgr2=0x7fe38013d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe376000640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3800ceee0 0x7fe38013d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe376000640 1 -- 172.21.15.97:0/3430686518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38014a070 con 0x7fe3801511f0 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe376000640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3801511f0 0x7fe380139d80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe3680262f0 tx=0x7fe368005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe377400640 1 -- 172.21.15.97:0/3430686518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680af610 con 0x7fe3801511f0 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe377400640 1 -- 172.21.15.97:0/3430686518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3680af990 con 0x7fe3801511f0 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe377400640 1 -- 172.21.15.97:0/3430686518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680afd80 con 0x7fe3801511f0 2024-09-10T21:33:27.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe380152ff0 con 0x7fe3801511f0 2024-09-10T21:33:27.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3801535b0 con 0x7fe3801511f0 2024-09-10T21:33:27.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38015abc0 con 0x7fe3801511f0 2024-09-10T21:33:27.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe377400640 1 -- 172.21.15.97:0/3430686518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe368004040 con 0x7fe3801511f0 2024-09-10T21:33:27.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe377400640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe35c0725e0 0x7fe35c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:27.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.227+0000 7fe377400640 1 -- 172.21.15.97:0/3430686518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7fe368132790 con 0x7fe3801511f0 2024-09-10T21:33:27.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.231+0000 7fe37ea00640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe35c0725e0 0x7fe35c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:27.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.231+0000 7fe37ea00640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe35c0725e0 0x7fe35c074aa0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe370000f70 tx=0x7fe370009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.231+0000 7fe377400640 1 -- 172.21.15.97:0/3430686518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe3680ac050 con 0x7fe3801511f0 2024-09-10T21:33:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:27.459+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe38015b860 con 0x7fe3801511f0 2024-09-10T21:33:28.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.063+0000 7fe377400640 1 -- 172.21.15.97:0/3430686518 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v228) ==== 101+0+0 (secure 0 0 0) 0x7fe368019100 con 0x7fe3801511f0 2024-09-10T21:33:28.064 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe35c0725e0 msgr2=0x7fe35c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe35c0725e0 0x7fe35c074aa0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe370000f70 tx=0x7fe370009040 comp rx=0 tx=0).stop 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3801511f0 msgr2=0x7fe380139d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3801511f0 0x7fe380139d80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe3680262f0 tx=0x7fe368005b00 comp rx=0 tx=0).stop 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 shutdown_connections 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe35c0725e0 0x7fe35c074aa0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe38015a730 0x7fe38013a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3801511f0 0x7fe380139d80 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 --2- 172.21.15.97:0/3430686518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3800ceee0 0x7fe38013d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 >> 172.21.15.97:0/3430686518 conn(0x7fe3800b1780 msgr2=0x7fe3800b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 shutdown_connections 2024-09-10T21:33:28.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.067+0000 7fe385600640 1 -- 172.21.15.97:0/3430686518 wait complete. 2024-09-10T21:33:28.086 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.131+0000 7f4ebda03080 1 Processor -- start 2024-09-10T21:33:28.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.131+0000 7f4ebda03080 1 -- start start 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc340aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 0x55fc33febda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 0x55fc340a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc33f98f40 con 0x7f4ea0003cc0 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fc33fa5d00 con 0x55fc340ac3e0 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55fc33f3aa00 con 0x55fc340a49d0 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc340aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc340aa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43574/0 (socket says 172.21.15.97:43574) 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 -- 172.21.15.97:0/2339011940 learned_addr learned my addr 172.21.15.97:0/2339011940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebb600640 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 0x55fc340a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 0x55fc33febda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 -- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 msgr2=0x55fc340a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 0x55fc340a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 -- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 msgr2=0x55fc33febda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 0x55fc33febda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 -- 172.21.15.97:0/2339011940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc340c4e00 con 0x7f4ea0003cc0 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebac00640 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc340aa900 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00b800 tx=0x7f4eac004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2feb321134c6f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4eb9800640 1 -- 172.21.15.97:0/2339011940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00abc0 con 0x7f4ea0003cc0 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4eb9800640 1 -- 172.21.15.97:0/2339011940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eac00ad60 con 0x7f4ea0003cc0 2024-09-10T21:33:28.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4eb9800640 1 -- 172.21.15.97:0/2339011940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0086a0 con 0x7f4ea0003cc0 2024-09-10T21:33:28.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- 172.21.15.97:0/2339011940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 msgr2=0x55fc340aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc340aa900 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00b800 tx=0x7f4eac004890 comp rx=0 tx=0).stop 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- 172.21.15.97:0/2339011940 shutdown_connections 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 0x55fc340a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 0x55fc33febda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 --2- 172.21.15.97:0/2339011940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc340aa900 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- 172.21.15.97:0/2339011940 >> 172.21.15.97:0/2339011940 conn(0x55fc3401a520 msgr2=0x55fc3409ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- 172.21.15.97:0/2339011940 shutdown_connections 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- 172.21.15.97:0/2339011940 wait complete. 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 Processor -- start 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.135+0000 7f4ebda03080 1 -- start start 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 0x55fc340c1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 0x55fc3401ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc3401f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc340c5340 con 0x7f4ea0003cc0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fc340c50a0 con 0x55fc340ac3e0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55fc340c55e0 con 0x55fc340a49d0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc3401f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc3401f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43586/0 (socket says 172.21.15.97:43586) 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 learned_addr learned my addr 172.21.15.97:0/65539232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebac00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 0x55fc340c1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 0x55fc3401ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 msgr2=0x55fc340c1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc340a49d0 0x55fc340c1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 msgr2=0x55fc3401ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc340ac3e0 0x55fc3401ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc340ae8c0 con 0x7f4ea0003cc0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebb600640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc3401f140 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4eb401a210 tx=0x7f4eb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4eb3400640 1 -- 172.21.15.97:0/65539232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40b9070 con 0x7f4ea0003cc0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4eb3400640 1 -- 172.21.15.97:0/65539232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eb40013b0 con 0x7f4ea0003cc0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fc340aeb60 con 0x7f4ea0003cc0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4eb3400640 1 -- 172.21.15.97:0/65539232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40ab040 con 0x7f4ea0003cc0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc340af0a0 con 0x7f4ea0003cc0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.139+0000 7f4eb3400640 1 -- 172.21.15.97:0/65539232 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4eb4007590 con 0x7f4ea0003cc0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4eb3400640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e98070ff0 0x7f4e980734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4eb3400640 1 -- 172.21.15.97:0/65539232 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7f4eb412c1e0 con 0x7f4ea0003cc0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4ebac00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e98070ff0 0x7f4e980734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.144 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:28.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc340b1e90 0x55fc340b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc340b27b0 con 0x55fc340b1e90 2024-09-10T21:33:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc340b1e90 0x55fc340b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4ebac00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e98070ff0 0x7f4e980734b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00bfd0 tx=0x7f4eac00e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.143+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc340b1e90 0x55fc340b2270 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.147+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v120'722 uv696 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.147+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc340afb40 con 0x55fc340b1e90 2024-09-10T21:33:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.147+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v120'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.147+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc340afee0 con 0x55fc340b1e90 2024-09-10T21:33:28.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.147+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v120'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.147 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:28.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.147+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc340b02d0 con 0x55fc340b1e90 2024-09-10T21:33:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.151+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.151+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc340b0720 con 0x55fc340b1e90 2024-09-10T21:33:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.151+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'726 uv726 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.151+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34169dd0 con 0x55fc340b1e90 2024-09-10T21:33:28.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.151+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'727 uv727 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.155+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a170 con 0x55fc340b1e90 2024-09-10T21:33:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.155+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v120'728 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.155+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.155+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v120'729 uv729 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.155+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.157 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:28.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.159+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v120'730 uv730 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.159+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.159+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'731 uv731 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.159+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.163+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.163+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.163+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.163+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.163+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v120'734 uv709 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.163+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.167+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'735 uv735 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.167+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.166 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:28.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.167+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v120'736 uv736 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.167+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.171+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'737 uv737 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.171+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.171+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'738 uv738 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.171+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.171+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'739 uv739 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.171+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.175+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'740 uv740 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.175+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.175+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v120'741 uv715 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.175+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.175+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v120'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.175+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.177 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:28.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.179+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v120'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.179+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.179+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.179+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.183+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.183+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.183+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.183+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.183+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v120'747 uv721 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.187+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.187+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v120'748 uv748 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.187+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.187 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:28.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.187+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v120'749 uv749 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.187+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'750 uv750 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416b520 con 0x55fc340b1e90 2024-09-10T21:33:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'751 uv751 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416a510 con 0x55fc340b1e90 2024-09-10T21:33:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'752 uv752 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc340b1e90 2024-09-10T21:33:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc3416aa20 0x55fc3416ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416d350 con 0x55fc3416aa20 2024-09-10T21:33:28.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.191+0000 7f4ebac00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc3416aa20 0x55fc3416ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.195+0000 7f4ebac00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc3416aa20 0x55fc3416ce10 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.195+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'527 uv526 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.195+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416d6f0 con 0x55fc3416aa20 2024-09-10T21:33:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.195+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'528 uv528 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.195+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416da90 con 0x55fc3416aa20 2024-09-10T21:33:28.196 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.199+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'529 uv529 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.199+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416e2e0 con 0x55fc3416aa20 2024-09-10T21:33:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.199+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v120'530 uv530 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.199+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416e680 con 0x55fc3416aa20 2024-09-10T21:33:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.199+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'531 uv531 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.199+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416ea20 con 0x55fc3416aa20 2024-09-10T21:33:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.203+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v120'532 uv514 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.203+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416edc0 con 0x55fc3416aa20 2024-09-10T21:33:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.203+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v120'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.203+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.203 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.203+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v120'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.203+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.207+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.207+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.207+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'536 uv536 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.207+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.207+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v120'537 uv519 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.207+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.211+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v120'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.211+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.209 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.211+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v120'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.211+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34173330 con 0x55fc3416aa20 2024-09-10T21:33:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.211+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v120'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.211+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34173330 con 0x55fc3416aa20 2024-09-10T21:33:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.215+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.215+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.215+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v120'542 uv525 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.215+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.215+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v120'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.215+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.217 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.219+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v120'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.219+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34170170 con 0x55fc3416aa20 2024-09-10T21:33:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.219+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.219+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34170170 con 0x55fc3416aa20 2024-09-10T21:33:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.219+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.219+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3416f160 con 0x55fc3416aa20 2024-09-10T21:33:28.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc3416aa20 2024-09-10T21:33:28.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc3416f7e0 0x55fc34171bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341721e0 con 0x55fc3416f7e0 2024-09-10T21:33:28.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc3416f7e0 0x55fc34171bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc3416f7e0 0x55fc34171bd0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc341725b0 con 0x55fc3416f7e0 2024-09-10T21:33:28.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.223+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.227+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341725b0 con 0x55fc3416f7e0 2024-09-10T21:33:28.226 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:28.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.227+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.227+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341735c0 con 0x55fc3416f7e0 2024-09-10T21:33:28.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.227+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.227+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341735c0 con 0x55fc3416f7e0 2024-09-10T21:33:28.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.227+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.227+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341725b0 con 0x55fc3416f7e0 2024-09-10T21:33:28.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc34172c30 0x55fc34175050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34175720 con 0x55fc34172c30 2024-09-10T21:33:28.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4ebb600640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc34172c30 0x55fc34175050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4ebb600640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc34172c30 0x55fc34175050 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'107 uv106 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc34175c20 con 0x55fc34172c30 2024-09-10T21:33:28.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.231+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'108 uv108 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.235+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34176120 con 0x55fc34172c30 2024-09-10T21:33:28.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.235+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'109 uv109 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.233 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:28.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.235+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341769e0 con 0x55fc34172c30 2024-09-10T21:33:28.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.235+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.235+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34176d80 con 0x55fc3416f7e0 2024-09-10T21:33:28.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.235+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.235+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc34176d80 con 0x55fc3416f7e0 2024-09-10T21:33:28.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.239+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.239+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34176d80 con 0x55fc3416f7e0 2024-09-10T21:33:28.238 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:28.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.239+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.239+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34177d90 con 0x55fc3416f7e0 2024-09-10T21:33:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.239+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.239+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34177d90 con 0x55fc3416f7e0 2024-09-10T21:33:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34176d80 con 0x55fc3416f7e0 2024-09-10T21:33:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc34176d80 con 0x55fc34172c30 2024-09-10T21:33:28.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v120'260 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc34177120 con 0x55fc34172c30 2024-09-10T21:33:28.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v120'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.243+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc34172c30 2024-09-10T21:33:28.245 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:28.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.247+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v120'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.247+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341784d0 con 0x55fc34172c30 2024-09-10T21:33:28.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.247+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'263 uv263 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.247+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341784d0 con 0x55fc34172c30 2024-09-10T21:33:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.247+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.247+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc34172c30 2024-09-10T21:33:28.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.251+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.251+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc34172c30 2024-09-10T21:33:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.251+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v120'266 uv259 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.251+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc34172c30 2024-09-10T21:33:28.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.251+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v120'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.251+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc34172c30 2024-09-10T21:33:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:28.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v120'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc34172c30 2024-09-10T21:33:28.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc34172c30 2024-09-10T21:33:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc3416f7e0 2024-09-10T21:33:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v120'117 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.255+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc3416f7e0 2024-09-10T21:33:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.259+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v120'118 uv118 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.259+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc3416f7e0 2024-09-10T21:33:28.258 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.259+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v120'119 uv119 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.259+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc3416f7e0 2024-09-10T21:33:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.259+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'120 uv120 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.259+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341774c0 con 0x55fc3416f7e0 2024-09-10T21:33:28.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'121 uv121 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3416f7e0 2024-09-10T21:33:28.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc341779d0 0x55fc34179de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417a460 con 0x55fc341779d0 2024-09-10T21:33:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebac00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc341779d0 0x55fc34179de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebac00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc341779d0 0x55fc34179de0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3417a830 con 0x55fc341779d0 2024-09-10T21:33:28.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.263+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417a830 con 0x55fc341779d0 2024-09-10T21:33:28.264 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:28.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417b840 con 0x55fc341779d0 2024-09-10T21:33:28.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417b840 con 0x55fc341779d0 2024-09-10T21:33:28.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417a830 con 0x55fc341779d0 2024-09-10T21:33:28.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.267+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417a830 con 0x55fc341779d0 2024-09-10T21:33:28.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.271+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v120'233 uv227 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.271+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3417a830 con 0x55fc341779d0 2024-09-10T21:33:28.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.271+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v120'234 uv234 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.271+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417a830 con 0x55fc341779d0 2024-09-10T21:33:28.271 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:28.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.271+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v120'235 uv235 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.271+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341765d0 con 0x55fc341779d0 2024-09-10T21:33:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'236 uv236 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341765d0 con 0x55fc341779d0 2024-09-10T21:33:28.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'237 uv237 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341765d0 con 0x55fc341779d0 2024-09-10T21:33:28.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v120'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc341765d0 con 0x55fc341779d0 2024-09-10T21:33:28.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v120'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.275+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc341765d0 con 0x55fc341779d0 2024-09-10T21:33:28.278 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:28.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.279+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v120'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.279+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417ace0 con 0x55fc341779d0 2024-09-10T21:33:28.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.279+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.279+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d750 con 0x55fc341779d0 2024-09-10T21:33:28.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.279+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.279+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.283+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v120'103 uv102 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.283+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.283+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'104 uv104 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.283+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.283 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:28.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.283+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v120'105 uv105 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.283+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'106 uv106 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v120'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.290 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.287+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc34172c30 2024-09-10T21:33:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4eb40bb030 con 0x55fc34172c30 2024-09-10T21:33:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc3417ad40 0x55fc3417de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc3417ad40 2024-09-10T21:33:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc3417ad40 0x55fc3417de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.291+0000 7f4eb3e00640 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc3417ad40 0x55fc3417de10 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.295+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v120'120 uv119 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3417ad40 2024-09-10T21:33:28.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.295+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc3417ad40 2024-09-10T21:33:28.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.295+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v120'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3417ad40 2024-09-10T21:33:28.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.295+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc3417ad40 2024-09-10T21:33:28.295 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:28.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.295+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v120'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3417ad40 2024-09-10T21:33:28.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.295+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc3417ad40 2024-09-10T21:33:28.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.299+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3417ad40 2024-09-10T21:33:28.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.299+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc3417ad40 2024-09-10T21:33:28.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.299+0000 7f4eb3e00640 1 -- 172.21.15.97:0/65539232 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ea400b040 con 0x55fc3417ad40 2024-09-10T21:33:28.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.299+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc341779d0 2024-09-10T21:33:28.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.299+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v120'290 uv282 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.299+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc341779d0 2024-09-10T21:33:28.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v120'291 uv291 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc341779d0 2024-09-10T21:33:28.302 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:28.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v120'292 uv292 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417e3a0 con 0x55fc341779d0 2024-09-10T21:33:28.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v120'293 uv293 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417e350 con 0x55fc341779d0 2024-09-10T21:33:28.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'294 uv294 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.303+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417e350 con 0x55fc341779d0 2024-09-10T21:33:28.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.307+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v120'295 uv289 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.307+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x55fc3417e350 con 0x55fc341779d0 2024-09-10T21:33:28.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.307+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v120'296 uv296 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.307+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417e350 con 0x55fc341779d0 2024-09-10T21:33:28.307 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:28.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.307+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v120'297 uv297 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.307+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc341779d0 2024-09-10T21:33:28.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'298 uv298 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc341779d0 2024-09-10T21:33:28.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'299 uv299 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417d710 con 0x55fc341779d0 2024-09-10T21:33:28.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'300 uv300 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55fc3417e350 con 0x55fc341779d0 2024-09-10T21:33:28.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebac00640 1 -- 172.21.15.97:0/65539232 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'301 uv301 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4eac023520 con 0x55fc341779d0 2024-09-10T21:33:28.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc3416f7e0 msgr2=0x55fc34171bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.311+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc3416f7e0 0x55fc34171bd0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc341779d0 msgr2=0x55fc34179de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc341779d0 0x55fc34179de0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc3417ad40 msgr2=0x55fc3417de10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc3417ad40 0x55fc3417de10 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc34172c30 msgr2=0x55fc34175050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc34172c30 0x55fc34175050 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc3416aa20 msgr2=0x55fc3416ce10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 reap_dead start 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc3416aa20 0x55fc3416ce10 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebb600640 1 -- 172.21.15.97:0/65539232 reap_dead start 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc340b1e90 msgr2=0x55fc340b2270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc340b1e90 0x55fc340b2270 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e98070ff0 msgr2=0x7f4e980734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e98070ff0 0x7f4e980734b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00bfd0 tx=0x7f4eac00e040 comp rx=0 tx=0).stop 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 msgr2=0x55fc3401f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc3401f140 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4eb401a210 tx=0x7f4eb4009bb0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 shutdown_connections 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc340b1e90 0x55fc340b2270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e98070ff0 0x7f4e980734b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 --2- 172.21.15.97:0/65539232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0003cc0 0x55fc3401f140 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 >> 172.21.15.97:0/65539232 conn(0x55fc3401a520 msgr2=0x55fc340369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 shutdown_connections 2024-09-10T21:33:28.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.315+0000 7f4ebda03080 1 -- 172.21.15.97:0/65539232 wait complete. 2024-09-10T21:33:28.319 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2024-09-10T21:33:28.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.407+0000 7fb192edc080 1 Processor -- start 2024-09-10T21:33:28.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.407+0000 7fb192edc080 1 -- start start 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1bde360 0x5634e1c963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1c0cec0 0x5634e1c968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1bdb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5634e1c9a450 con 0x5634e1c0cec0 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5634e1b2aa00 con 0x5634e1c0d2a0 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5634e1c9a5d0 con 0x5634e1bde360 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb187e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1c0cec0 0x5634e1c968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1bde360 0x5634e1c963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb187e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1c0cec0 0x5634e1c968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43606/0 (socket says 172.21.15.97:43606) 2024-09-10T21:33:28.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1bde360 0x5634e1c963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:34914/0 (socket says 172.21.15.97:34914) 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb187e00640 1 -- 172.21.15.97:0/1300460931 learned_addr learned my addr 172.21.15.97:0/1300460931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190a00640 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1bdb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 -- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 msgr2=0x5634e1bdb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1bdb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 -- 172.21.15.97:0/1300460931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1c0cec0 msgr2=0x5634e1c968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1c0cec0 0x5634e1c968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 -- 172.21.15.97:0/1300460931 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e1b29a20 con 0x5634e1bde360 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb187e00640 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1c0cec0 0x5634e1c968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb190000640 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1bde360 0x5634e1c963b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb18000c540 tx=0x7fb18000ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef6e4bd7a6b4fa86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb18ec00640 1 -- 172.21.15.97:0/1300460931 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18000aa70 con 0x5634e1bde360 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb18ec00640 1 -- 172.21.15.97:0/1300460931 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb18000ad00 con 0x5634e1bde360 2024-09-10T21:33:28.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb18ec00640 1 -- 172.21.15.97:0/1300460931 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180008650 con 0x5634e1bde360 2024-09-10T21:33:28.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1bde360 msgr2=0x5634e1c963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1bde360 0x5634e1c963b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb18000c540 tx=0x7fb18000ca00 comp rx=0 tx=0).stop 2024-09-10T21:33:28.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- 172.21.15.97:0/1300460931 shutdown_connections 2024-09-10T21:33:28.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1bdb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1c0cec0 0x5634e1c968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 --2- 172.21.15.97:0/1300460931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1bde360 0x5634e1c963b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- 172.21.15.97:0/1300460931 >> 172.21.15.97:0/1300460931 conn(0x5634e1c240d0 msgr2=0x5634e1cb22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- 172.21.15.97:0/1300460931 shutdown_connections 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- 172.21.15.97:0/1300460931 wait complete. 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 Processor -- start 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.411+0000 7fb192edc080 1 -- start start 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1bde360 0x5634e1d4c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1c0cec0 0x5634e1cad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1cb0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5634e1c26d50 con 0x5634e1bde360 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5634e1c25ee0 con 0x5634e1c0d2a0 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5634e1c97530 con 0x5634e1c0cec0 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1bde360 0x5634e1d4c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1bde360 0x5634e1d4c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43612/0 (socket says 172.21.15.97:43612) 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 -- 172.21.15.97:0/460108368 learned_addr learned my addr 172.21.15.97:0/460108368 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190a00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1cb0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb187e00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1c0cec0 0x5634e1cad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 -- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1c0cec0 msgr2=0x5634e1cad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1c0cec0 0x5634e1cad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 -- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 msgr2=0x5634e1cb0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1cb0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 -- 172.21.15.97:0/460108368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e1b03c10 con 0x5634e1bde360 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190a00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5634e1c0d2a0 0x5634e1cb0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb187e00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5634e1c0cec0 0x5634e1cad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb190000640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1bde360 0x5634e1d4c970 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fb180016650 tx=0x7fb180016680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb18d800640 1 -- 172.21.15.97:0/460108368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180004c60 con 0x5634e1bde360 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb18d800640 1 -- 172.21.15.97:0/460108368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1800168a0 con 0x5634e1bde360 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5634e1d4d120 con 0x5634e1bde360 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb18d800640 1 -- 172.21.15.97:0/460108368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180016cf0 con 0x5634e1bde360 2024-09-10T21:33:28.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e1d4d660 con 0x5634e1bde360 2024-09-10T21:33:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.415+0000 7fb18d800640 1 -- 172.21.15.97:0/460108368 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb180013070 con 0x5634e1bde360 2024-09-10T21:33:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb18d800640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb16c070fc0 0x7fb16c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb18d800640 1 -- 172.21.15.97:0/460108368 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7fb180093850 con 0x5634e1bde360 2024-09-10T21:33:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb187e00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb16c070fc0 0x7fb16c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5634e1d4f4d0 0x5634e1d4f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5634e1d4fdf0 con 0x5634e1d4f4d0 2024-09-10T21:33:28.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb187e00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb16c070fc0 0x7fb16c073480 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb17800b760 tx=0x7fb178004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb190a00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5634e1d4f4d0 0x5634e1d4f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb190a00640 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5634e1d4f4d0 0x5634e1d4f8b0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb190a00640 1 -- 172.21.15.97:0/460108368 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v120'548 uv548 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb188002050 con 0x5634e1d4f4d0 2024-09-10T21:33:28.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5634e1d4f4d0 msgr2=0x5634e1d4f8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5634e1d4f4d0 0x5634e1d4f8b0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb16c070fc0 msgr2=0x7fb16c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb16c070fc0 0x7fb16c073480 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb17800b760 tx=0x7fb178004890 comp rx=0 tx=0).stop 2024-09-10T21:33:28.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1bde360 msgr2=0x5634e1d4c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 --2- 172.21.15.97:0/460108368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634e1bde360 0x5634e1d4c970 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fb180016650 tx=0x7fb180016680 comp rx=0 tx=0).stop 2024-09-10T21:33:28.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb190a00640 1 -- 172.21.15.97:0/460108368 reap_dead start 2024-09-10T21:33:28.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 shutdown_connections 2024-09-10T21:33:28.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 >> 172.21.15.97:0/460108368 conn(0x5634e1c240d0 msgr2=0x5634e1c963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:28.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 shutdown_connections 2024-09-10T21:33:28.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.419+0000 7fb192edc080 1 -- 172.21.15.97:0/460108368 wait complete. 2024-09-10T21:33:28.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- 172.21.15.97:0/1536830863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400d0710 msgr2=0x7f4a400ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 --2- 172.21.15.97:0/1536830863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400d0710 0x7f4a400ce9c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4a40074160 tx=0x7f4a2801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- 172.21.15.97:0/1536830863 shutdown_connections 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 --2- 172.21.15.97:0/1536830863 >> v2:172.21.15.194:3300/0 conn(0x7f4a400cde30 0x7f4a400cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 --2- 172.21.15.97:0/1536830863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400d0710 0x7f4a400ce9c0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 --2- 172.21.15.97:0/1536830863 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a400ce5e0 0x7f4a400b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- 172.21.15.97:0/1536830863 >> 172.21.15.97:0/1536830863 conn(0x7f4a400d1ca0 msgr2=0x7f4a400d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- 172.21.15.97:0/1536830863 shutdown_connections 2024-09-10T21:33:28.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- 172.21.15.97:0/1536830863 wait complete. 2024-09-10T21:33:28.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 Processor -- start 2024-09-10T21:33:28.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- start start 2024-09-10T21:33:28.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a400cde30 0x7f4a401f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400ce5e0 0x7f4a400cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a400d0710 0x7f4a40145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40160930 con 0x7f4a400ce5e0 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a401607b0 con 0x7f4a400d0710 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a45000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4a40160ab0 con 0x7f4a400cde30 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400ce5e0 0x7f4a400cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400ce5e0 0x7f4a400cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43640/0 (socket says 172.21.15.97:43640) 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 -- 172.21.15.97:0/133801410 learned_addr learned my addr 172.21.15.97:0/133801410 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:28.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a3de00640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a400cde30 0x7f4a401f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a3e800640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a400d0710 0x7f4a40145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 -- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a400cde30 msgr2=0x7f4a401f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a400cde30 0x7f4a401f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 -- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a400d0710 msgr2=0x7f4a40145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:28.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a400d0710 0x7f4a40145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:28.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 -- 172.21.15.97:0/133801410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4015c240 con 0x7f4a400ce5e0 2024-09-10T21:33:28.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.615+0000 7f4a35400640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400ce5e0 0x7f4a400cbae0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4a280b0040 tx=0x7f4a280a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a36a00640 1 -- 172.21.15.97:0/133801410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a280b2040 con 0x7f4a400ce5e0 2024-09-10T21:33:28.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a36a00640 1 -- 172.21.15.97:0/133801410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a280044b0 con 0x7f4a400ce5e0 2024-09-10T21:33:28.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a40157f70 con 0x7f4a400ce5e0 2024-09-10T21:33:28.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a36a00640 1 -- 172.21.15.97:0/133801410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28003860 con 0x7f4a400ce5e0 2024-09-10T21:33:28.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4015c4e0 con 0x7f4a400ce5e0 2024-09-10T21:33:28.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a400b2010 con 0x7f4a400ce5e0 2024-09-10T21:33:28.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a36a00640 1 -- 172.21.15.97:0/133801410 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a28003a00 con 0x7f4a400ce5e0 2024-09-10T21:33:28.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a36a00640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a1c0725e0 0x7f4a1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:28.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a36a00640 1 -- 172.21.15.97:0/133801410 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8339+0+0 (secure 0 0 0) 0x7f4a2812d9e0 con 0x7f4a400ce5e0 2024-09-10T21:33:28.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.619+0000 7f4a3de00640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a1c0725e0 0x7f4a1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:28.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.623+0000 7f4a3de00640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a1c0725e0 0x7f4a1c074aa0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4a400b7920 tx=0x7f4a3000a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:28.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.623+0000 7f4a36a00640 1 -- 172.21.15.97:0/133801410 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a280bd060 con 0x7f4a400ce5e0 2024-09-10T21:33:28.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:28.827+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4a400d0060 con 0x7f4a400ce5e0 2024-09-10T21:33:29.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a36a00640 1 -- 172.21.15.97:0/133801410 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v229) ==== 162+0+0 (secure 0 0 0) 0x7f4a280164b0 con 0x7f4a400ce5e0 2024-09-10T21:33:29.069 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:29.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a1c0725e0 msgr2=0x7f4a1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a1c0725e0 0x7f4a1c074aa0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4a400b7920 tx=0x7f4a3000a2d0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400ce5e0 msgr2=0x7f4a400cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400ce5e0 0x7f4a400cbae0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4a280b0040 tx=0x7f4a280a57c0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 shutdown_connections 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a1c0725e0 0x7f4a1c074aa0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a400d0710 0x7f4a40145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a400ce5e0 0x7f4a400cbae0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 --2- 172.21.15.97:0/133801410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a400cde30 0x7f4a401f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 >> 172.21.15.97:0/133801410 conn(0x7f4a400d1ca0 msgr2=0x7f4a400b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 shutdown_connections 2024-09-10T21:33:29.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.071+0000 7f4a45000640 1 -- 172.21.15.97:0/133801410 wait complete. 2024-09-10T21:33:29.079 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:33:29.079+0000 7f5403a00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:33:29.080 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:29.079+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:29.080 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:29.079+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:29.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:29.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.235+0000 7f4a3aa00640 1 -- 172.21.15.97:0/1170353366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340cec10 msgr2=0x7f4a340cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.235+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/1170353366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340cec10 0x7f4a340cff10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009840 tx=0x7f4a1c01f010 comp rx=0 tx=0).stop 2024-09-10T21:33:29.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.235+0000 7f4a3aa00640 1 -- 172.21.15.97:0/1170353366 shutdown_connections 2024-09-10T21:33:29.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.235+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/1170353366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340cec10 0x7f4a340cff10 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.235+0000 7f4a3aa00640 1 -- 172.21.15.97:0/1170353366 >> 172.21.15.97:0/1170353366 conn(0x7f4a340b37c0 msgr2=0x7f4a340b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:29.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.235+0000 7f4a3aa00640 1 -- 172.21.15.97:0/1170353366 shutdown_connections 2024-09-10T21:33:29.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.235+0000 7f4a3aa00640 1 -- 172.21.15.97:0/1170353366 wait complete. 2024-09-10T21:33:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 Processor -- start 2024-09-10T21:33:29.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 -- start start 2024-09-10T21:33:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340b1560 0x7f4a3415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a340ce830 0x7f4a340c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a340cec10 0x7f4a341f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a340ce320 con 0x7f4a340b1560 2024-09-10T21:33:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a340ce1a0 con 0x7f4a340ce830 2024-09-10T21:33:29.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4a340d0450 con 0x7f4a340cec10 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340b1560 0x7f4a3415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340b1560 0x7f4a3415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43666/0 (socket says 172.21.15.97:43666) 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 -- 172.21.15.97:0/3527066709 learned_addr learned my addr 172.21.15.97:0/3527066709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a2b400640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a340ce830 0x7f4a340c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a38a00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a340cec10 0x7f4a341f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 -- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a340cec10 msgr2=0x7f4a341f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a340cec10 0x7f4a341f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 -- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a340ce830 msgr2=0x7f4a340c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a340ce830 0x7f4a340c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 -- 172.21.15.97:0/3527066709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a340c87b0 con 0x7f4a340b1560 2024-09-10T21:33:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a38a00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a340cec10 0x7f4a341f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a33e00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340b1560 0x7f4a3415e210 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400c530 tx=0x7f4a2400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a30c00640 1 -- 172.21.15.97:0/3527066709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24017070 con 0x7f4a340b1560 2024-09-10T21:33:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a30c00640 1 -- 172.21.15.97:0/3527066709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a240053c0 con 0x7f4a340b1560 2024-09-10T21:33:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a30c00640 1 -- 172.21.15.97:0/3527066709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24012890 con 0x7f4a340b1560 2024-09-10T21:33:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a341f7f10 con 0x7f4a340b1560 2024-09-10T21:33:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.239+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a341f8750 con 0x7f4a340b1560 2024-09-10T21:33:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.243+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a340b1970 con 0x7f4a340b1560 2024-09-10T21:33:29.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.243+0000 7f4a30c00640 1 -- 172.21.15.97:0/3527066709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a24003600 con 0x7f4a340b1560 2024-09-10T21:33:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.243+0000 7f4a30c00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a100725e0 0x7f4a10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.243+0000 7f4a30c00640 1 -- 172.21.15.97:0/3527066709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7f4a24093320 con 0x7f4a340b1560 2024-09-10T21:33:29.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.243+0000 7f4a2b400640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a100725e0 0x7f4a10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.247+0000 7f4a2b400640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a100725e0 0x7f4a10074aa0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0b2000 tx=0x7f4a1c01d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.247+0000 7f4a30c00640 1 -- 172.21.15.97:0/3527066709 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a341f7f10 con 0x7f4a340b1560 2024-09-10T21:33:29.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.443+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a341f8da0 con 0x7f4a340b1560 2024-09-10T21:33:29.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a30c00640 1 -- 172.21.15.97:0/3527066709 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 231 v231) ==== 95+0+3569 (secure 0 0 0) 0x7f4a340c87b0 con 0x7f4a340b1560 2024-09-10T21:33:29.445 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:29.445 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":231,"btime":"2024-09-10T21:33:29:083627+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":231,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:29.083571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:29.445 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 231 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a100725e0 msgr2=0x7f4a10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a100725e0 0x7f4a10074aa0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0b2000 tx=0x7f4a1c01d850 comp rx=0 tx=0).stop 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340b1560 msgr2=0x7f4a3415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340b1560 0x7f4a3415e210 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400c530 tx=0x7f4a2400c9f0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 shutdown_connections 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a100725e0 0x7f4a10074aa0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a340cec10 0x7f4a341f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a340ce830 0x7f4a340c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 --2- 172.21.15.97:0/3527066709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a340b1560 0x7f4a3415e210 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 >> 172.21.15.97:0/3527066709 conn(0x7f4a340b37c0 msgr2=0x7f4a3415f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 shutdown_connections 2024-09-10T21:33:29.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.447+0000 7f4a3aa00640 1 -- 172.21.15.97:0/3527066709 wait complete. 2024-09-10T21:33:29.467 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:29.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:29.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 -- 172.21.15.97:0/4078958686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 msgr2=0x7fc1980af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 --2- 172.21.15.97:0/4078958686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1980af810 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc190009bc0 tx=0x7fc19001cba0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 -- 172.21.15.97:0/4078958686 shutdown_connections 2024-09-10T21:33:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 --2- 172.21.15.97:0/4078958686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1980af810 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 -- 172.21.15.97:0/4078958686 >> 172.21.15.97:0/4078958686 conn(0x7fc1980b28c0 msgr2=0x7fc1980b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 -- 172.21.15.97:0/4078958686 shutdown_connections 2024-09-10T21:33:29.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 -- 172.21.15.97:0/4078958686 wait complete. 2024-09-10T21:33:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 Processor -- start 2024-09-10T21:33:29.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 -- start start 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1980cfad0 0x7fc1980b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.611+0000 7fc19da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1981473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc19da00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc19815bdd0 0x7fc19814f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc19da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc198169d50 con 0x7fc1980cff40 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc19da00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc198169bd0 con 0x7fc1980cfad0 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc19da00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc198169ed0 con 0x7fc19815bdd0 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1981473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc196a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1980cfad0 0x7fc1980b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1981473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43682/0 (socket says 172.21.15.97:43682) 2024-09-10T21:33:29.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc197400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc19815bdd0 0x7fc19814f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 -- 172.21.15.97:0/1811618985 learned_addr learned my addr 172.21.15.97:0/1811618985 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:29.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 -- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc19815bdd0 msgr2=0x7fc19814f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc19815bdd0 0x7fc19814f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 -- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1980cfad0 msgr2=0x7fc1980b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1980cfad0 0x7fc1980b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 -- 172.21.15.97:0/1811618985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc198147930 con 0x7fc1980cff40 2024-09-10T21:33:29.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18fe00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1981473f0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c490 tx=0x7fc18000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18ea00640 1 -- 172.21.15.97:0/1811618985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180016020 con 0x7fc1980cff40 2024-09-10T21:33:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18ea00640 1 -- 172.21.15.97:0/1811618985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc180012940 con 0x7fc1980cff40 2024-09-10T21:33:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1981fc000 con 0x7fc1980cff40 2024-09-10T21:33:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18ea00640 1 -- 172.21.15.97:0/1811618985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180012d90 con 0x7fc1980cff40 2024-09-10T21:33:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981fb760 con 0x7fc1980cff40 2024-09-10T21:33:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc198073fb0 con 0x7fc1980cff40 2024-09-10T21:33:29.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.615+0000 7fc18ea00640 1 -- 172.21.15.97:0/1811618985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc18001b400 con 0x7fc1980cff40 2024-09-10T21:33:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.619+0000 7fc18ea00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1740725e0 0x7fc174074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.619+0000 7fc18ea00640 1 -- 172.21.15.97:0/1811618985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7fc180093c60 con 0x7fc1980cff40 2024-09-10T21:33:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.619+0000 7fc196a00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1740725e0 0x7fc174074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.619+0000 7fc196a00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1740725e0 0x7fc174074aa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc188007fd0 tx=0x7fc188009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:29.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.623+0000 7fc18ea00640 1 -- 172.21.15.97:0/1811618985 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc180020030 con 0x7fc1980cff40 2024-09-10T21:33:29.816 INFO:tasks.ceph.mds.a.smithi097.stderr: -29> 2024-09-10T21:33:25.063+0000 7fc57aa00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T21:33:29.816 INFO:tasks.ceph.mds.a.smithi097.stderr: -27> 2024-09-10T21:33:25.063+0000 7fc57aa00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:33:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1980d1330 con 0x7fc1980cff40 2024-09-10T21:33:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc18ea00640 1 -- 172.21.15.97:0/1811618985 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 231 v231) ==== 95+0+3569 (secure 0 0 0) 0x7fc18005e2f0 con 0x7fc1980cff40 2024-09-10T21:33:29.821 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:29.821 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":231,"btime":"2024-09-10T21:33:29:083627+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":231,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:29.083571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:29.821 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 231 2024-09-10T21:33:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1740725e0 msgr2=0x7fc174074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1740725e0 0x7fc174074aa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc188007fd0 tx=0x7fc188009040 comp rx=0 tx=0).stop 2024-09-10T21:33:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 msgr2=0x7fc1981473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1981473f0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c490 tx=0x7fc18000c950 comp rx=0 tx=0).stop 2024-09-10T21:33:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 shutdown_connections 2024-09-10T21:33:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1740725e0 0x7fc174074aa0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc19815bdd0 0x7fc19814f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1980cff40 0x7fc1981473f0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 --2- 172.21.15.97:0/1811618985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1980cfad0 0x7fc1980b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 >> 172.21.15.97:0/1811618985 conn(0x7fc1980b28c0 msgr2=0x7fc198147e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 shutdown_connections 2024-09-10T21:33:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.823+0000 7fc19da00640 1 -- 172.21.15.97:0/1811618985 wait complete. 2024-09-10T21:33:29.844 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:29.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:29.855 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:33:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.987+0000 7f0b9b800640 1 -- 172.21.15.97:0/831574004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9406cb40 msgr2=0x7f0b9406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.987+0000 7f0b9b800640 1 --2- 172.21.15.97:0/831574004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9406cb40 0x7f0b9406cfc0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009bc0 tx=0x7f0b8c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- 172.21.15.97:0/831574004 shutdown_connections 2024-09-10T21:33:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 --2- 172.21.15.97:0/831574004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b9406cb40 0x7f0b9406cfc0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- 172.21.15.97:0/831574004 >> 172.21.15.97:0/831574004 conn(0x7f0b940b28c0 msgr2=0x7f0b940b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- 172.21.15.97:0/831574004 shutdown_connections 2024-09-10T21:33:29.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- 172.21.15.97:0/831574004 wait complete. 2024-09-10T21:33:29.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 Processor -- start 2024-09-10T21:33:29.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- start start 2024-09-10T21:33:29.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b9406cb40 0x7f0b9414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b94073a70 0x7f0b94158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b940d0060 0x7f0b94158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9416bfd0 con 0x7f0b940d0060 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9416be50 con 0x7f0b94073a70 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b9b800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9416c150 con 0x7f0b9406cb40 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b940d0060 0x7f0b94158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b940d0060 0x7f0b94158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43734/0 (socket says 172.21.15.97:43734) 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 -- 172.21.15.97:0/855911094 learned_addr learned my addr 172.21.15.97:0/855911094 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b98e00640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b9406cb40 0x7f0b9414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b8be00640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b94073a70 0x7f0b94158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 -- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b9406cb40 msgr2=0x7f0b9414e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b9406cb40 0x7f0b9414e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 -- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b94073a70 msgr2=0x7f0b94158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b94073a70 0x7f0b94158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 -- 172.21.15.97:0/855911094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b94157ab0 con 0x7f0b940d0060 2024-09-10T21:33:29.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.991+0000 7f0b99800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b940d0060 0x7f0b94158c70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c01c610 tx=0x7f0b8c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b91600640 1 -- 172.21.15.97:0/855911094 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0b2040 con 0x7f0b940d0060 2024-09-10T21:33:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b91600640 1 -- 172.21.15.97:0/855911094 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b8c0bb070 con 0x7f0b940d0060 2024-09-10T21:33:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b91600640 1 -- 172.21.15.97:0/855911094 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c004570 con 0x7f0b940d0060 2024-09-10T21:33:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b94157d50 con 0x7f0b940d0060 2024-09-10T21:33:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b94158310 con 0x7f0b940d0060 2024-09-10T21:33:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b94152310 con 0x7f0b940d0060 2024-09-10T21:33:29.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b91600640 1 -- 172.21.15.97:0/855911094 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0b8c0b5070 con 0x7f0b940d0060 2024-09-10T21:33:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b91600640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b70072610 0x7f0b70074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b91600640 1 -- 172.21.15.97:0/855911094 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7f0b8c12e160 con 0x7f0b940d0060 2024-09-10T21:33:29.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.995+0000 7f0b98e00640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b70072610 0x7f0b70074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:29.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.999+0000 7f0b98e00640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b70072610 0x7f0b70074ad0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0b840046c0 tx=0x7f0b84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:30.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:29.999+0000 7f0b91600640 1 -- 172.21.15.97:0/855911094 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b8c0a5e20 con 0x7f0b940d0060 2024-09-10T21:33:30.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.203+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b9406cfc0 con 0x7f0b940d0060 2024-09-10T21:33:30.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.203+0000 7f0b91600640 1 -- 172.21.15.97:0/855911094 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 232 v232) ==== 95+0+4392 (secure 0 0 0) 0x7f0b8c016790 con 0x7f0b940d0060 2024-09-10T21:33:30.204 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:30.204 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":232,"btime":"2024-09-10T21:33:30:085197+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/507601338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":507601338},{"type":"v1","addr":"172.21.15.97:6834","nonce":507601338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":232}],"filesystems":[{"mdsmap":{"epoch":231,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:29.083571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:30.205 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 232 2024-09-10T21:33:30.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b70072610 msgr2=0x7f0b70074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:30.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b70072610 0x7f0b70074ad0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0b840046c0 tx=0x7f0b84009040 comp rx=0 tx=0).stop 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b940d0060 msgr2=0x7f0b94158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b940d0060 0x7f0b94158c70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c01c610 tx=0x7f0b8c005ae0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 shutdown_connections 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b70072610 0x7f0b70074ad0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b940d0060 0x7f0b94158c70 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b94073a70 0x7f0b94158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 --2- 172.21.15.97:0/855911094 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b9406cb40 0x7f0b9414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 >> 172.21.15.97:0/855911094 conn(0x7f0b940b28c0 msgr2=0x7f0b94156640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 shutdown_connections 2024-09-10T21:33:30.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.207+0000 7f0b9b800640 1 -- 172.21.15.97:0/855911094 wait complete. 2024-09-10T21:33:30.225 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2024-09-10T21:33:30.225 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2024-09-10T21:33:30.226 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:30.226 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:30.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:30.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:30.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:30.280 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:30.335 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:30.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:30.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 -- 172.21.15.97:0/2171767194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40cff80 msgr2=0x7fa6a40b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2171767194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40cff80 0x7fa6a40b15c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40736b0 tx=0x7fa69c01ca30 comp rx=0 tx=0).stop 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 -- 172.21.15.97:0/2171767194 shutdown_connections 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2171767194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40cff80 0x7fa6a40b15c0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2171767194 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6a40d05e0 0x7fa6a40cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2171767194 >> v2:172.21.15.194:3300/0 conn(0x7fa6a415f350 0x7fa6a40b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 -- 172.21.15.97:0/2171767194 >> 172.21.15.97:0/2171767194 conn(0x7fa6a40d1c60 msgr2=0x7fa6a40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 -- 172.21.15.97:0/2171767194 shutdown_connections 2024-09-10T21:33:30.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 -- 172.21.15.97:0/2171767194 wait complete. 2024-09-10T21:33:30.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 Processor -- start 2024-09-10T21:33:30.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.523+0000 7fa6ab200640 1 -- start start 2024-09-10T21:33:30.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6a40cff80 0x7fa6a415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40d05e0 0x7fa6a4152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a415f350 0x7fa6a415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40b0430 con 0x7fa6a40d05e0 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40b02b0 con 0x7fa6a415f350 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a40b05b0 con 0x7fa6a40cff80 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40d05e0 0x7fa6a4152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40d05e0 0x7fa6a4152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43756/0 (socket says 172.21.15.97:43756) 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 -- 172.21.15.97:0/2164250323 learned_addr learned my addr 172.21.15.97:0/2164250323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6a3e00640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6a40cff80 0x7fa6a415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6a9200640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a415f350 0x7fa6a415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 -- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6a40cff80 msgr2=0x7fa6a415b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6a40cff80 0x7fa6a415b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 -- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a415f350 msgr2=0x7fa6a415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a415f350 0x7fa6a415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 -- 172.21.15.97:0/2164250323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a415a010 con 0x7fa6a40d05e0 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa69b400640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40d05e0 0x7fa6a4152040 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00d980 tx=0x7fa68c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6a0c00640 1 -- 172.21.15.97:0/2164250323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c017070 con 0x7fa6a40d05e0 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6a0c00640 1 -- 172.21.15.97:0/2164250323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa68c009d60 con 0x7fa6a40d05e0 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6a0c00640 1 -- 172.21.15.97:0/2164250323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c005280 con 0x7fa6a40d05e0 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a415a2b0 con 0x7fa6a40d05e0 2024-09-10T21:33:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a415a870 con 0x7fa6a40d05e0 2024-09-10T21:33:30.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a40cfa40 con 0x7fa6a40d05e0 2024-09-10T21:33:30.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.527+0000 7fa6a0c00640 1 -- 172.21.15.97:0/2164250323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa68c005420 con 0x7fa6a40d05e0 2024-09-10T21:33:30.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.531+0000 7fa6a0c00640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6800725e0 0x7fa680074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:30.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.531+0000 7fa6a0c00640 1 -- 172.21.15.97:0/2164250323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7fa68c093e30 con 0x7fa6a40d05e0 2024-09-10T21:33:30.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.531+0000 7fa6a3e00640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6800725e0 0x7fa680074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:30.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.531+0000 7fa6a3e00640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6800725e0 0x7fa680074aa0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa69400bad0 tx=0x7fa69400b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:30.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.535+0000 7fa6a0c00640 1 -- 172.21.15.97:0/2164250323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa68c05e400 con 0x7fa6a40d05e0 2024-09-10T21:33:30.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:30.735+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa6a40b13a0 con 0x7fa6a40d05e0 2024-09-10T21:33:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6a0c00640 1 -- 172.21.15.97:0/2164250323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v233) ==== 161+0+0 (secure 0 0 0) 0x7fa68c067170 con 0x7fa6a40d05e0 2024-09-10T21:33:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6800725e0 msgr2=0x7fa680074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6800725e0 0x7fa680074aa0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa69400bad0 tx=0x7fa69400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40d05e0 msgr2=0x7fa6a4152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40d05e0 0x7fa6a4152040 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00d980 tx=0x7fa68c00de40 comp rx=0 tx=0).stop 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 shutdown_connections 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6800725e0 0x7fa680074aa0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a415f350 0x7fa6a415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6a40d05e0 0x7fa6a4152040 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 --2- 172.21.15.97:0/2164250323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6a40cff80 0x7fa6a415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 >> 172.21.15.97:0/2164250323 conn(0x7fa6a40d1c60 msgr2=0x7fa6a415e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 shutdown_connections 2024-09-10T21:33:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.091+0000 7fa6ab200640 1 -- 172.21.15.97:0/2164250323 wait complete. 2024-09-10T21:33:31.111 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:31.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 -- 172.21.15.97:0/3307204529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e300d0150 msgr2=0x7f5e300afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:31.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 --2- 172.21.15.97:0/3307204529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e300d0150 0x7f5e300afc40 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009840 tx=0x7f5e2801c790 comp rx=0 tx=0).stop 2024-09-10T21:33:31.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 -- 172.21.15.97:0/3307204529 shutdown_connections 2024-09-10T21:33:31.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 --2- 172.21.15.97:0/3307204529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e300d0150 0x7f5e300afc40 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 --2- 172.21.15.97:0/3307204529 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e3015f350 0x7f5e300b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 -- 172.21.15.97:0/3307204529 >> 172.21.15.97:0/3307204529 conn(0x7f5e300d1c60 msgr2=0x7f5e300d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:31.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 -- 172.21.15.97:0/3307204529 shutdown_connections 2024-09-10T21:33:31.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.255+0000 7f5e38400640 1 -- 172.21.15.97:0/3307204529 wait complete. 2024-09-10T21:33:31.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 Processor -- start 2024-09-10T21:33:31.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 -- start start 2024-09-10T21:33:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e300d0150 0x7f5e30150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e300d05e0 0x7f5e301fae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3015f350 0x7f5e300ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30168c30 con 0x7f5e3015f350 2024-09-10T21:33:31.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5e300b0ee0 con 0x7f5e300d0150 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5e30168db0 con 0x7f5e300d05e0 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3015f350 0x7f5e300ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3015f350 0x7f5e300ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43770/0 (socket says 172.21.15.97:43770) 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 -- 172.21.15.97:0/833781083 learned_addr learned my addr 172.21.15.97:0/833781083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e35000640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e300d05e0 0x7f5e301fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e35a00640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e300d0150 0x7f5e30150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 -- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e300d05e0 msgr2=0x7f5e301fae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e300d05e0 0x7f5e301fae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 -- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e300d0150 msgr2=0x7f5e30150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e300d0150 0x7f5e30150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 -- 172.21.15.97:0/833781083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e301fb830 con 0x7f5e3015f350 2024-09-10T21:33:31.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e35a00640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e300d0150 0x7f5e30150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:31.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e35000640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e300d05e0 0x7f5e301fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:31.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e36400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3015f350 0x7f5e300ce230 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f5e2801a3c0 tx=0x7f5e280a55b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:31.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e2d600640 1 -- 172.21.15.97:0/833781083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280ba020 con 0x7f5e3015f350 2024-09-10T21:33:31.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e2d600640 1 -- 172.21.15.97:0/833781083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e280043b0 con 0x7f5e3015f350 2024-09-10T21:33:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e301fbb10 con 0x7f5e3015f350 2024-09-10T21:33:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e2d600640 1 -- 172.21.15.97:0/833781083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280b2040 con 0x7f5e3015f350 2024-09-10T21:33:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e301513f0 con 0x7f5e3015f350 2024-09-10T21:33:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.259+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e301fbd50 con 0x7f5e3015f350 2024-09-10T21:33:31.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.263+0000 7f5e2d600640 1 -- 172.21.15.97:0/833781083 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5e280b5070 con 0x7f5e3015f350 2024-09-10T21:33:31.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.263+0000 7f5e2d600640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e100725e0 0x7f5e10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:31.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.263+0000 7f5e2d600640 1 -- 172.21.15.97:0/833781083 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7f5e2812dab0 con 0x7f5e3015f350 2024-09-10T21:33:31.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.263+0000 7f5e35a00640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e100725e0 0x7f5e10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:31.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.263+0000 7f5e35a00640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e100725e0 0x7f5e10074aa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009e70 tx=0x7f5e24009850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:31.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.267+0000 7f5e2d600640 1 -- 172.21.15.97:0/833781083 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5e301fbd50 con 0x7f5e3015f350 2024-09-10T21:33:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:31.491+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5e300afc40 con 0x7f5e3015f350 2024-09-10T21:33:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e2d600640 1 -- 172.21.15.97:0/833781083 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v234) ==== 101+0+0 (secure 0 0 0) 0x7f5e280c4020 con 0x7f5e3015f350 2024-09-10T21:33:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e100725e0 msgr2=0x7f5e10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e100725e0 0x7f5e10074aa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009e70 tx=0x7f5e24009850 comp rx=0 tx=0).stop 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3015f350 msgr2=0x7f5e300ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3015f350 0x7f5e300ce230 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f5e2801a3c0 tx=0x7f5e280a55b0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 shutdown_connections 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e100725e0 0x7f5e10074aa0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3015f350 0x7f5e300ce230 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5e300d05e0 0x7f5e301fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 --2- 172.21.15.97:0/833781083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e300d0150 0x7f5e30150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 >> 172.21.15.97:0/833781083 conn(0x7f5e300d1c60 msgr2=0x7f5e3006dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 shutdown_connections 2024-09-10T21:33:32.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.099+0000 7f5e38400640 1 -- 172.21.15.97:0/833781083 wait complete. 2024-09-10T21:33:32.120 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:32.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 Processor -- start 2024-09-10T21:33:32.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 -- start start 2024-09-10T21:33:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4003100 0x55fc493032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc491e9f80 0x55fc49277be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4925a710 0x55fc491c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc4914df40 con 0x55fc4925a710 2024-09-10T21:33:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fc4915ad00 con 0x7f76b4003100 2024-09-10T21:33:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cfd59080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55fc490efa00 con 0x55fc491e9f80 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4925a710 0x55fc491c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4925a710 0x55fc491c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43800/0 (socket says 172.21.15.97:43800) 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 -- 172.21.15.97:0/2993489466 learned_addr learned my addr 172.21.15.97:0/2993489466 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cd000640 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4003100 0x55fc493032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76c7e00640 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc491e9f80 0x55fc49277be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 -- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc491e9f80 msgr2=0x55fc49277be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc491e9f80 0x55fc49277be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 -- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4003100 msgr2=0x55fc493032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4003100 0x55fc493032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 -- 172.21.15.97:0/2993489466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4925aeb0 con 0x55fc4925a710 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76cda00640 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4925a710 0x55fc491c3040 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x55fc491ce100 tx=0x7f76c8001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48a19aedb15508 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76c6a00640 1 -- 172.21.15.97:0/2993489466 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c80057f0 con 0x55fc4925a710 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76c6a00640 1 -- 172.21.15.97:0/2993489466 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c8005990 con 0x55fc4925a710 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.171+0000 7f76c6a00640 1 -- 172.21.15.97:0/2993489466 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8005da0 con 0x55fc4925a710 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- 172.21.15.97:0/2993489466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4925a710 msgr2=0x55fc491c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4925a710 0x55fc491c3040 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x55fc491ce100 tx=0x7f76c8001b50 comp rx=0 tx=0).stop 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- 172.21.15.97:0/2993489466 shutdown_connections 2024-09-10T21:33:32.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4925a710 0x55fc491c3040 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55fc491e9f80 0x55fc49277be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 --2- 172.21.15.97:0/2993489466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b4003100 0x55fc493032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- 172.21.15.97:0/2993489466 >> 172.21.15.97:0/2993489466 conn(0x55fc491a0140 msgr2=0x55fc4925c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- 172.21.15.97:0/2993489466 shutdown_connections 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- 172.21.15.97:0/2993489466 wait complete. 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 Processor -- start 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- start start 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc491e9f80 0x55fc49311c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4925a710 0x55fc491ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4003100 0x55fc49275170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc4925b150 con 0x55fc491e9f80 2024-09-10T21:33:32.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fc4925b3f0 con 0x55fc4925a710 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55fc49302b60 con 0x7f76b4003100 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc491e9f80 0x55fc49311c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc491e9f80 0x55fc49311c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43816/0 (socket says 172.21.15.97:43816) 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 learned_addr learned my addr 172.21.15.97:0/1915943501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76c7e00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4925a710 0x55fc491ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cda00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4003100 0x55fc49275170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4003100 msgr2=0x55fc49275170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f76b4003100 0x55fc49275170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4925a710 msgr2=0x55fc491ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4925a710 0x55fc491ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc491cec10 con 0x55fc491e9f80 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cd000640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc491e9f80 0x55fc49311c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f76c00106f0 tx=0x7f76c0010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76c5600640 1 -- 172.21.15.97:0/1915943501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c000ebb0 con 0x55fc491e9f80 2024-09-10T21:33:32.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76c5600640 1 -- 172.21.15.97:0/1915943501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c000ed50 con 0x55fc491e9f80 2024-09-10T21:33:32.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76c5600640 1 -- 172.21.15.97:0/1915943501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c00193f0 con 0x55fc491e9f80 2024-09-10T21:33:32.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fc491ceeb0 con 0x55fc491e9f80 2024-09-10T21:33:32.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.175+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc491cf3f0 con 0x55fc491e9f80 2024-09-10T21:33:32.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76c5600640 1 -- 172.21.15.97:0/1915943501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76c00183a0 con 0x55fc491e9f80 2024-09-10T21:33:32.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76c5600640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76ac070fc0 0x7f76ac073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76c5600640 1 -- 172.21.15.97:0/1915943501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7f76c0093020 con 0x55fc491e9f80 2024-09-10T21:33:32.179 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:32.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc492756b0 0x55fc491cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76c7e00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76ac070fc0 0x7f76ac073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc491d0070 con 0x55fc492756b0 2024-09-10T21:33:32.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76cda00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc492756b0 0x55fc491cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76c7e00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76ac070fc0 0x7f76ac073480 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f76b80ac000 tx=0x7f76b80a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.179+0000 7f76cda00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc492756b0 0x55fc491cfb30 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.183+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v121'753 uv727 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.183+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc491d0440 con 0x55fc492756b0 2024-09-10T21:33:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.183+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v121'754 uv754 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.183+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931e200 con 0x55fc492756b0 2024-09-10T21:33:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.187+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v121'755 uv755 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.185 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:32.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.187+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931f5b0 con 0x55fc492756b0 2024-09-10T21:33:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.187+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'756 uv756 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.187+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931f950 con 0x55fc492756b0 2024-09-10T21:33:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.187+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.191+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931e5a0 con 0x55fc492756b0 2024-09-10T21:33:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.191+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.191+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931e940 con 0x55fc492756b0 2024-09-10T21:33:32.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.191+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v121'759 uv733 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.191+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.195+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v121'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.195+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.195 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.195+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v121'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.195+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.199+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.199+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.199+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.199+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.199+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.203+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.203+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v121'765 uv740 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.203+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.203+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'766 uv766 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.203+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.207+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v121'767 uv767 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.204 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:32.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.207+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.207+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'768 uv768 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.207+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.207+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'769 uv769 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.207+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.211+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'770 uv770 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.211+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.211+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'771 uv771 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.211+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.211+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v121'772 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.211+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.215+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v121'773 uv773 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.215+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.215 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.215+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v121'774 uv774 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.215+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.219+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'775 uv775 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.219+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.219+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'776 uv776 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.219+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.219+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'777 uv777 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.219+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.223+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v121'778 uv752 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.223+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.223+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v121'779 uv779 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.223+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.225 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.227+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v121'780 uv780 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.227+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.227+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'781 uv781 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.227+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.231+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.231+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc492756b0 2024-09-10T21:33:32.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.231+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc492756b0 2024-09-10T21:33:32.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.231+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc4931f0d0 0x55fc49322210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.231+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4931ece0 con 0x55fc4931f0d0 2024-09-10T21:33:32.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.231+0000 7f76cd000640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc4931f0d0 0x55fc49322210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.231+0000 7f76cd000640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc4931f0d0 0x55fc49322210 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.235+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'549 uv531 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.235+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49322750 con 0x55fc4931f0d0 2024-09-10T21:33:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.235+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'550 uv550 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.235+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49322b70 con 0x55fc4931f0d0 2024-09-10T21:33:32.236 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:32.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.235+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'551 uv551 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.235+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323410 con 0x55fc4931f0d0 2024-09-10T21:33:32.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.239+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v121'552 uv552 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.239+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493237b0 con 0x55fc4931f0d0 2024-09-10T21:33:32.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.239+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'553 uv553 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.239+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323b50 con 0x55fc4931f0d0 2024-09-10T21:33:32.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v121'554 uv548 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v121'555 uv555 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.242 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:32.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v121'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.243+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.247+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.247+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.247+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v121'559 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.247+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.247+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v121'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.247+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.249 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:32.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.251+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v121'561 uv561 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.251+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.251+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v121'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.251+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.251+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.251+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.255+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v121'564 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.255+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.255+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v121'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.255+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.255 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:32.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.255+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v121'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.255+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'567 uv567 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323ef0 con 0x55fc4931f0d0 2024-09-10T21:33:32.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4931f0d0 2024-09-10T21:33:32.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc49324860 0x55fc49326c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49327190 con 0x55fc49324860 2024-09-10T21:33:32.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.259+0000 7f76c7e00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc49324860 0x55fc49326c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.263+0000 7f76c7e00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc49324860 0x55fc49326c50 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.263+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.263+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49327590 con 0x55fc49324860 2024-09-10T21:33:32.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.263+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.263+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49327a70 con 0x55fc49324860 2024-09-10T21:33:32.264 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49323040 con 0x55fc49324860 2024-09-10T21:33:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493293e0 con 0x55fc49324860 2024-09-10T21:33:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49328310 con 0x55fc49324860 2024-09-10T21:33:32.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc49328dd0 0x55fc4932b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932bd40 con 0x55fc49328dd0 2024-09-10T21:33:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.267+0000 7f76cda00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc49328dd0 0x55fc4932b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.271+0000 7f76cda00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc49328dd0 0x55fc4932b800 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.271+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'111 uv110 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.271+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932c110 con 0x55fc49328dd0 2024-09-10T21:33:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.271+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.271+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c110 con 0x55fc49328dd0 2024-09-10T21:33:32.272 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.271+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.271+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c110 con 0x55fc49328dd0 2024-09-10T21:33:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.275+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'114 uv114 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.275+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c070 con 0x55fc49324860 2024-09-10T21:33:32.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.275+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.275+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932c500 con 0x55fc49324860 2024-09-10T21:33:32.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.275+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.275+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c500 con 0x55fc49324860 2024-09-10T21:33:32.277 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.279+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.279+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932d510 con 0x55fc49324860 2024-09-10T21:33:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.279+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.279+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932d510 con 0x55fc49324860 2024-09-10T21:33:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.279+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.279+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c500 con 0x55fc49324860 2024-09-10T21:33:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.283+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.283+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c500 con 0x55fc49328dd0 2024-09-10T21:33:32.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.283+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v121'271 uv265 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.283+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932c500 con 0x55fc49328dd0 2024-09-10T21:33:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.283+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v121'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.283+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c4b0 con 0x55fc49328dd0 2024-09-10T21:33:32.284 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:32.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.287+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v121'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.287+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c4a0 con 0x55fc49328dd0 2024-09-10T21:33:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.287+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.287+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932d990 con 0x55fc49328dd0 2024-09-10T21:33:32.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.287+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.287+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932d990 con 0x55fc49328dd0 2024-09-10T21:33:32.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932d910 con 0x55fc49328dd0 2024-09-10T21:33:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v121'277 uv270 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932c8a0 con 0x55fc49328dd0 2024-09-10T21:33:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v121'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49328dd0 2024-09-10T21:33:32.291 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v121'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.291+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49328dd0 2024-09-10T21:33:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.295+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.295+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49328dd0 2024-09-10T21:33:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.295+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.295+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49324860 2024-09-10T21:33:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.295+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v121'122 uv121 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.295+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49324860 2024-09-10T21:33:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.299+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v121'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.299+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49324860 2024-09-10T21:33:32.297 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.299+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v121'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.299+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49324860 2024-09-10T21:33:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.299+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.299+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932c850 con 0x55fc49324860 2024-09-10T21:33:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49324860 2024-09-10T21:33:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc4932d220 0x55fc4932f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932fb50 con 0x55fc4932d220 2024-09-10T21:33:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cd000640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc4932d220 0x55fc4932f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cd000640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc4932d220 0x55fc4932f610 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932fbd0 con 0x55fc4932d220 2024-09-10T21:33:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.303+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.304 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.307+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.307+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330fd0 con 0x55fc4932d220 2024-09-10T21:33:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.307+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.307+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330fd0 con 0x55fc4932d220 2024-09-10T21:33:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.307+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.307+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.311+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.311+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.311+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v121'243 uv237 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.311+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.311+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v121'244 uv244 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.311+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.312 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:32.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.311+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v121'245 uv245 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.315+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.315+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'246 uv246 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.315+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.315+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'247 uv247 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.315+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.315+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v121'248 uv242 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.315+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.319+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v121'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.319+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.318 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.319+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v121'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.319+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc4932ffc0 con 0x55fc4932d220 2024-09-10T21:33:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.319+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.319+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49332dd0 con 0x55fc4932d220 2024-09-10T21:33:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.323+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.323+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.323+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v121'107 uv106 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.323+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.323+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'108 uv108 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.323+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.324 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v121'109 uv109 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'110 uv110 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v121'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.327+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.329 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.331+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.331+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49331480 con 0x55fc49328dd0 2024-09-10T21:33:32.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.331+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.331+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49331480 con 0x55fc49328dd0 2024-09-10T21:33:32.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.331+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.331+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49330470 con 0x55fc49328dd0 2024-09-10T21:33:32.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76c80091f0 con 0x55fc49328dd0 2024-09-10T21:33:32.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc49330af0 0x55fc49333410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49332f20 con 0x55fc49330af0 2024-09-10T21:33:32.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76c7e00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc49330af0 0x55fc49333410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76c7e00640 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc49330af0 0x55fc49333410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v121'125 uv124 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49330af0 2024-09-10T21:33:32.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc49332f20 con 0x55fc49330af0 2024-09-10T21:33:32.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v121'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49330af0 2024-09-10T21:33:32.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.335+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49332f20 con 0x55fc49330af0 2024-09-10T21:33:32.336 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:32.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.339+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v121'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49330af0 2024-09-10T21:33:32.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.339+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc49332f20 con 0x55fc49330af0 2024-09-10T21:33:32.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.339+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49330af0 2024-09-10T21:33:32.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.339+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc49330af0 2024-09-10T21:33:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.339+0000 7f76c7e00640 1 -- 172.21.15.97:0/1915943501 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f76b80ad070 con 0x55fc49330af0 2024-09-10T21:33:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.339+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.343+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v121'302 uv294 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.343+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.343+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v121'303 uv303 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.343+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.343+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v121'304 uv304 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.343 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.343+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v121'305 uv305 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'306 uv306 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v121'307 uv301 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v121'308 uv308 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.347+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.351+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v121'309 uv309 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.349 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.351+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493349b0 con 0x55fc4932d220 2024-09-10T21:33:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.351+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'310 uv310 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.351+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493349b0 con 0x55fc4932d220 2024-09-10T21:33:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.351+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'311 uv311 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.351+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493339a0 con 0x55fc4932d220 2024-09-10T21:33:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'312 uv312 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55fc493340c0 con 0x55fc4932d220 2024-09-10T21:33:32.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cd000640 1 -- 172.21.15.97:0/1915943501 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'313 uv313 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f76c0020070 con 0x55fc4932d220 2024-09-10T21:33:32.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc49324860 msgr2=0x55fc49326c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55fc49324860 0x55fc49326c50 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc4932d220 msgr2=0x55fc4932f610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55fc4932d220 0x55fc4932f610 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc49330af0 msgr2=0x55fc49333410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55fc49330af0 0x55fc49333410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc49328dd0 msgr2=0x55fc4932b800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 reap_dead start 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55fc49328dd0 0x55fc4932b800 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc4931f0d0 msgr2=0x55fc49322210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55fc4931f0d0 0x55fc49322210 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc492756b0 msgr2=0x55fc491cfb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55fc492756b0 0x55fc491cfb30 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76ac070fc0 msgr2=0x7f76ac073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f76ac070fc0 0x7f76ac073480 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f76b80ac000 tx=0x7f76b80a9040 comp rx=0 tx=0).stop 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc491e9f80 msgr2=0x55fc49311c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 --2- 172.21.15.97:0/1915943501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc491e9f80 0x55fc49311c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f76c00106f0 tx=0x7f76c0010bb0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cda00640 1 -- 172.21.15.97:0/1915943501 reap_dead start 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 shutdown_connections 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 >> 172.21.15.97:0/1915943501 conn(0x55fc491a0140 msgr2=0x55fc49303fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 shutdown_connections 2024-09-10T21:33:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.355+0000 7f76cfd59080 1 -- 172.21.15.97:0/1915943501 wait complete. 2024-09-10T21:33:32.359 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2024-09-10T21:33:32.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 Processor -- start 2024-09-10T21:33:32.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 -- start start 2024-09-10T21:33:32.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facc8002300 0x558f737923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 0x558f73793da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f737942e0 0x558f736ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558f7366af40 con 0x558f737942e0 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f73677d00 con 0x7facc8002300 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face58a8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558f7360ca00 con 0x558f73791bf0 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f737942e0 0x558f736ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f737942e0 0x558f736ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43844/0 (socket says 172.21.15.97:43844) 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 -- 172.21.15.97:0/2065245475 learned_addr learned my addr 172.21.15.97:0/2065245475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face2a00640 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facc8002300 0x558f737923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7facdbe00640 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 0x558f73793da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 -- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 msgr2=0x558f73793da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 0x558f73793da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 -- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facc8002300 msgr2=0x558f737923d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:32.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facc8002300 0x558f737923d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 -- 172.21.15.97:0/2065245475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f736ef580 con 0x558f737942e0 2024-09-10T21:33:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face3400640 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f737942e0 0x558f736ee880 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7facdc004fd0 tx=0x7facdc010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435a05c588fb0746 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face1600640 1 -- 172.21.15.97:0/2065245475 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc010230 con 0x558f737942e0 2024-09-10T21:33:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face1600640 1 -- 172.21.15.97:0/2065245475 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facdc010e50 con 0x558f737942e0 2024-09-10T21:33:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.407+0000 7face1600640 1 -- 172.21.15.97:0/2065245475 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc00bba0 con 0x558f737942e0 2024-09-10T21:33:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- 172.21.15.97:0/2065245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f737942e0 msgr2=0x558f736ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f737942e0 0x558f736ee880 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7facdc004fd0 tx=0x7facdc010c00 comp rx=0 tx=0).stop 2024-09-10T21:33:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- 172.21.15.97:0/2065245475 shutdown_connections 2024-09-10T21:33:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f737942e0 0x558f736ee880 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 0x558f73793da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 --2- 172.21.15.97:0/2065245475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7facc8002300 0x558f737923d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- 172.21.15.97:0/2065245475 >> 172.21.15.97:0/2065245475 conn(0x558f737060d0 msgr2=0x558f7377b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- 172.21.15.97:0/2065245475 shutdown_connections 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- 172.21.15.97:0/2065245475 wait complete. 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 Processor -- start 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- start start 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 0x558f736be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f737942e0 0x558f737910b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc8002300 0x558f737915f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558f736c0ae0 con 0x7facc8002300 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f737946f0 con 0x558f737942e0 2024-09-10T21:33:32.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558f736edea0 con 0x558f73791bf0 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc8002300 0x558f737915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc8002300 0x558f737915f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43854/0 (socket says 172.21.15.97:43854) 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 -- 172.21.15.97:0/2084637400 learned_addr learned my addr 172.21.15.97:0/2084637400 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7facdbe00640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f737942e0 0x558f737910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face2a00640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 0x558f736be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 -- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 msgr2=0x558f736be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558f73791bf0 0x558f736be4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 -- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f737942e0 msgr2=0x558f737910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f737942e0 0x558f737910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 -- 172.21.15.97:0/2084637400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f7382c720 con 0x7facc8002300 2024-09-10T21:33:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face3400640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc8002300 0x558f737915f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7facdc01a670 tx=0x7facdc01a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7facdb400640 1 -- 172.21.15.97:0/2084637400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc028070 con 0x7facc8002300 2024-09-10T21:33:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7facdb400640 1 -- 172.21.15.97:0/2084637400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facdc010e50 con 0x7facc8002300 2024-09-10T21:33:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7facdb400640 1 -- 172.21.15.97:0/2084637400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc01ad70 con 0x7facc8002300 2024-09-10T21:33:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f7382c9c0 con 0x7facc8002300 2024-09-10T21:33:32.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.411+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f737041a0 con 0x7facc8002300 2024-09-10T21:33:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7facdb400640 1 -- 172.21.15.97:0/2084637400 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7facdc02c3b0 con 0x7facc8002300 2024-09-10T21:33:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7facdb400640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc0070ff0 0x7facc00734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7facdb400640 1 -- 172.21.15.97:0/2084637400 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7facdc094e70 con 0x7facc8002300 2024-09-10T21:33:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7face2a00640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc0070ff0 0x7facc00734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7face58a8080 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558f736be9f0 0x558f736bd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x558f736c1100 con 0x558f736be9f0 2024-09-10T21:33:32.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7facdbe00640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558f736be9f0 0x558f736bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7face2a00640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc0070ff0 0x7facc00734b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7facd40af000 tx=0x7facd401a740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.415+0000 7facdbe00640 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558f736be9f0 0x558f736bd650 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7facdbe00640 1 -- 172.21.15.97:0/2084637400 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v121'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7faccc009660 con 0x558f736be9f0 2024-09-10T21:33:32.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558f736be9f0 msgr2=0x558f736bd650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558f736be9f0 0x558f736bd650 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc0070ff0 msgr2=0x7facc00734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7facc0070ff0 0x7facc00734b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7facd40af000 tx=0x7facd401a740 comp rx=0 tx=0).stop 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc8002300 msgr2=0x558f737915f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 --2- 172.21.15.97:0/2084637400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc8002300 0x558f737915f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7facdc01a670 tx=0x7facdc01a6a0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face3400640 1 -- 172.21.15.97:0/2084637400 reap_dead start 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 shutdown_connections 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 >> 172.21.15.97:0/2084637400 conn(0x558f737060d0 msgr2=0x558f7377b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 shutdown_connections 2024-09-10T21:33:32.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.419+0000 7face58a8080 1 -- 172.21.15.97:0/2084637400 wait complete. 2024-09-10T21:33:32.422 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 -- 172.21.15.97:0/1836280543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d4015fc10 msgr2=0x7f1d4015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 --2- 172.21.15.97:0/1836280543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d4015fc10 0x7f1d4015c100 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f1d40069e70 tx=0x7f1d3801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 -- 172.21.15.97:0/1836280543 shutdown_connections 2024-09-10T21:33:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 --2- 172.21.15.97:0/1836280543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d4015fc10 0x7f1d4015c100 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 --2- 172.21.15.97:0/1836280543 >> v2:172.21.15.194:3300/0 conn(0x7f1d400ce9d0 0x7f1d4015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 --2- 172.21.15.97:0/1836280543 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d400ce5f0 0x7f1d40141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 -- 172.21.15.97:0/1836280543 >> 172.21.15.97:0/1836280543 conn(0x7f1d400b1780 msgr2=0x7f1d400b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 -- 172.21.15.97:0/1836280543 shutdown_connections 2024-09-10T21:33:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.607+0000 7f1d46600640 1 -- 172.21.15.97:0/1836280543 wait complete. 2024-09-10T21:33:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 Processor -- start 2024-09-10T21:33:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 -- start start 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d400ce5f0 0x7f1d40154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d400ce9d0 0x7f1d4013acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d4015fc10 0x7f1d401385a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40140290 con 0x7f1d400ce5f0 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1d40140110 con 0x7f1d4015fc10 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1d40140410 con 0x7f1d400ce9d0 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d400ce5f0 0x7f1d40154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d400ce5f0 0x7f1d40154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43862/0 (socket says 172.21.15.97:43862) 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 -- 172.21.15.97:0/1622447639 learned_addr learned my addr 172.21.15.97:0/1622447639 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3ea00640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d400ce9d0 0x7f1d4013acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3fe00640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d4015fc10 0x7f1d401385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 -- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d400ce9d0 msgr2=0x7f1d4013acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d400ce9d0 0x7f1d4013acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 -- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d4015fc10 msgr2=0x7f1d401385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d4015fc10 0x7f1d401385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 -- 172.21.15.97:0/1622447639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4013b410 con 0x7f1d400ce5f0 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d3f400640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d400ce5f0 0x7f1d40154ac0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000eb30 tx=0x7f1d3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d37400640 1 -- 172.21.15.97:0/1622447639 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30004b30 con 0x7f1d400ce5f0 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d37400640 1 -- 172.21.15.97:0/1622447639 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d30004cd0 con 0x7f1d400ce5f0 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4013ba60 con 0x7f1d400ce5f0 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d37400640 1 -- 172.21.15.97:0/1622447639 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30005280 con 0x7f1d400ce5f0 2024-09-10T21:33:32.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.611+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d40138ae0 con 0x7f1d400ce5f0 2024-09-10T21:33:32.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.615+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4015f6d0 con 0x7f1d400ce5f0 2024-09-10T21:33:32.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.615+0000 7f1d37400640 1 -- 172.21.15.97:0/1622447639 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d30020030 con 0x7f1d400ce5f0 2024-09-10T21:33:32.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.615+0000 7f1d37400640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d200725e0 0x7f1d20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.615+0000 7f1d37400640 1 -- 172.21.15.97:0/1622447639 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8425+0+0 (secure 0 0 0) 0x7f1d300947d0 con 0x7f1d400ce5f0 2024-09-10T21:33:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.615+0000 7f1d3ea00640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d200725e0 0x7f1d20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:32.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.615+0000 7f1d3ea00640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d200725e0 0x7f1d20074aa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1d40156e30 tx=0x7f1d2c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:32.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.619+0000 7f1d37400640 1 -- 172.21.15.97:0/1622447639 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d3005eda0 con 0x7f1d400ce5f0 2024-09-10T21:33:32.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:32.819+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1d40141360 con 0x7f1d400ce5f0 2024-09-10T21:33:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.107+0000 7f1d37400640 1 -- 172.21.15.97:0/1622447639 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v235) ==== 162+0+0 (secure 0 0 0) 0x7f1d300637d0 con 0x7f1d400ce5f0 2024-09-10T21:33:33.106 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d200725e0 msgr2=0x7f1d20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d200725e0 0x7f1d20074aa0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1d40156e30 tx=0x7f1d2c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d400ce5f0 msgr2=0x7f1d40154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d400ce5f0 0x7f1d40154ac0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000eb30 tx=0x7f1d3000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 shutdown_connections 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d200725e0 0x7f1d20074aa0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d4015fc10 0x7f1d401385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d400ce9d0 0x7f1d4013acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 --2- 172.21.15.97:0/1622447639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d400ce5f0 0x7f1d40154ac0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 >> 172.21.15.97:0/1622447639 conn(0x7f1d400b1780 msgr2=0x7f1d4006fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:33.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 shutdown_connections 2024-09-10T21:33:33.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.111+0000 7f1d46600640 1 -- 172.21.15.97:0/1622447639 wait complete. 2024-09-10T21:33:33.120 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:33:33.123+0000 7f0ebd400640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:33:33.121 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:33.123+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:33.121 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:33.123+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:33.133 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 -- 172.21.15.97:0/40076718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400ce830 msgr2=0x7f9b400cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 --2- 172.21.15.97:0/40076718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400ce830 0x7f9b400cf800 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009900 tx=0x7f9b3001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 -- 172.21.15.97:0/40076718 shutdown_connections 2024-09-10T21:33:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 --2- 172.21.15.97:0/40076718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b400cec10 0x7f9b400cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 --2- 172.21.15.97:0/40076718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400ce830 0x7f9b400cf800 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 -- 172.21.15.97:0/40076718 >> 172.21.15.97:0/40076718 conn(0x7f9b400b37c0 msgr2=0x7f9b400b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 -- 172.21.15.97:0/40076718 shutdown_connections 2024-09-10T21:33:33.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.275+0000 7f9b47800640 1 -- 172.21.15.97:0/40076718 wait complete. 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 Processor -- start 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 -- start start 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b400ce830 0x7f9b4006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400cec10 0x7f9b400c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b400d0280 0x7f9b401f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b400cde60 con 0x7f9b400cec10 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9b400b44a0 con 0x7f9b400d0280 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9b400ce1a0 con 0x7f9b400ce830 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400cec10 0x7f9b400c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b44e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b400ce830 0x7f9b4006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b45800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b400d0280 0x7f9b401f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400cec10 0x7f9b400c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43878/0 (socket says 172.21.15.97:43878) 2024-09-10T21:33:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 -- 172.21.15.97:0/1896824106 learned_addr learned my addr 172.21.15.97:0/1896824106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 -- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b400ce830 msgr2=0x7f9b4006f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b400ce830 0x7f9b4006f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 -- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b400d0280 msgr2=0x7f9b401f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b400d0280 0x7f9b401f81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 -- 172.21.15.97:0/1896824106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b400cb180 con 0x7f9b400cec10 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b37e00640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400cec10 0x7f9b400c8060 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800d9b0 tx=0x7f9b2800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b3d600640 1 -- 172.21.15.97:0/1896824106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28017030 con 0x7f9b400cec10 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b3d600640 1 -- 172.21.15.97:0/1896824106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b28004110 con 0x7f9b400cec10 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b3d600640 1 -- 172.21.15.97:0/1896824106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b280099d0 con 0x7f9b400cec10 2024-09-10T21:33:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.279+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b400ca8e0 con 0x7f9b400cec10 2024-09-10T21:33:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.283+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b400caea0 con 0x7f9b400cec10 2024-09-10T21:33:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.283+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b400b2780 con 0x7f9b400cec10 2024-09-10T21:33:33.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.283+0000 7f9b3d600640 1 -- 172.21.15.97:0/1896824106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9b28024060 con 0x7f9b400cec10 2024-09-10T21:33:33.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.283+0000 7f9b3d600640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b1c072610 0x7f9b1c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.283+0000 7f9b3d600640 1 -- 172.21.15.97:0/1896824106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f9b28093ed0 con 0x7f9b400cec10 2024-09-10T21:33:33.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.283+0000 7f9b44e00640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b1c072610 0x7f9b1c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.287+0000 7f9b44e00640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b1c072610 0x7f9b1c074ad0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9b300b2000 tx=0x7f9b300aa040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:33.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.287+0000 7f9b3d600640 1 -- 172.21.15.97:0/1896824106 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9b2805e3c0 con 0x7f9b400cec10 2024-09-10T21:33:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.487+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b4015e250 con 0x7f9b400cec10 2024-09-10T21:33:33.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.487+0000 7f9b3d600640 1 -- 172.21.15.97:0/1896824106 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) ==== 95+0+3569 (secure 0 0 0) 0x7f9b28096030 con 0x7f9b400cec10 2024-09-10T21:33:33.487 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:33.488 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":237,"btime":"2024-09-10T21:33:33:125018+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":237,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:33.124958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:33.488 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 237 2024-09-10T21:33:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b1c072610 msgr2=0x7f9b1c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b1c072610 0x7f9b1c074ad0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9b300b2000 tx=0x7f9b300aa040 comp rx=0 tx=0).stop 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400cec10 msgr2=0x7f9b400c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400cec10 0x7f9b400c8060 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800d9b0 tx=0x7f9b2800de70 comp rx=0 tx=0).stop 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 shutdown_connections 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b1c072610 0x7f9b1c074ad0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b400d0280 0x7f9b401f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b400cec10 0x7f9b400c8060 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 --2- 172.21.15.97:0/1896824106 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b400ce830 0x7f9b4006f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 >> 172.21.15.97:0/1896824106 conn(0x7f9b400b37c0 msgr2=0x7f9b400b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 shutdown_connections 2024-09-10T21:33:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.491+0000 7f9b47800640 1 -- 172.21.15.97:0/1896824106 wait complete. 2024-09-10T21:33:33.508 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:33.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:33.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 -- 172.21.15.97:0/3963701503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141511f0 msgr2=0x7f19140b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 --2- 172.21.15.97:0/3963701503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141511f0 0x7f19140b37b0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f190400a980 tx=0x7f1904020340 comp rx=0 tx=0).stop 2024-09-10T21:33:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 -- 172.21.15.97:0/3963701503 shutdown_connections 2024-09-10T21:33:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 --2- 172.21.15.97:0/3963701503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19141511f0 0x7f19140b37b0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 -- 172.21.15.97:0/3963701503 >> 172.21.15.97:0/3963701503 conn(0x7f19140b1780 msgr2=0x7f19140b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 -- 172.21.15.97:0/3963701503 shutdown_connections 2024-09-10T21:33:33.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 -- 172.21.15.97:0/3963701503 wait complete. 2024-09-10T21:33:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 Processor -- start 2024-09-10T21:33:33.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.651+0000 7f191b600640 1 -- start start 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19140b3cf0 0x7f19141e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19140ceee0 0x7f19140cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19141511f0 0x7f19141400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f191415b290 con 0x7f19140ceee0 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f191415b110 con 0x7f19140b3cf0 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f19140cf350 con 0x7f19141511f0 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19140ceee0 0x7f19140cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19140ceee0 0x7f19140cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43886/0 (socket says 172.21.15.97:43886) 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 -- 172.21.15.97:0/2083091799 learned_addr learned my addr 172.21.15.97:0/2083091799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f1918c00640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19140b3cf0 0x7f19141e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f1919600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19141511f0 0x7f19141400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 -- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19141511f0 msgr2=0x7f19141400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19141511f0 0x7f19141400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 -- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19140b3cf0 msgr2=0x7f19141e7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19140b3cf0 0x7f19141e7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 -- 172.21.15.97:0/2083091799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19141e8570 con 0x7f19140ceee0 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f1918c00640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19140b3cf0 0x7f19141e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f190be00640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19140ceee0 0x7f19140cfc70 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c4b0 tx=0x7f18fc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f1911600640 1 -- 172.21.15.97:0/2083091799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0052d0 con 0x7f19140ceee0 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f1911600640 1 -- 172.21.15.97:0/2083091799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18fc005470 con 0x7f19140ceee0 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f1911600640 1 -- 172.21.15.97:0/2083091799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc012870 con 0x7f19140ceee0 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19141e8850 con 0x7f19140ceee0 2024-09-10T21:33:33.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19141518f0 con 0x7f19140ceee0 2024-09-10T21:33:33.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19140cf800 con 0x7f19140ceee0 2024-09-10T21:33:33.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.655+0000 7f1911600640 1 -- 172.21.15.97:0/2083091799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18fc003600 con 0x7f19140ceee0 2024-09-10T21:33:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.659+0000 7f1911600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18f00725e0 0x7f18f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.659+0000 7f1911600640 1 -- 172.21.15.97:0/2083091799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f18fc0938e0 con 0x7f19140ceee0 2024-09-10T21:33:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.659+0000 7f1918c00640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18f00725e0 0x7f18f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:33.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.659+0000 7f1918c00640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18f00725e0 0x7f18f0074aa0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f19040b2000 tx=0x7f190401eb80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:33.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.663+0000 7f1911600640 1 -- 172.21.15.97:0/2083091799 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f18fc05e4b0 con 0x7f19140ceee0 2024-09-10T21:33:33.833 INFO:tasks.ceph.mds.c.smithi097.stderr: -40> 2024-09-10T21:33:29.079+0000 7f5403a00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:33:33.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.863+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f191415d630 con 0x7f19140ceee0 2024-09-10T21:33:33.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.863+0000 7f1911600640 1 -- 172.21.15.97:0/2083091799 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) ==== 95+0+3569 (secure 0 0 0) 0x7f18fc05ddb0 con 0x7f19140ceee0 2024-09-10T21:33:33.862 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:33.862 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":237,"btime":"2024-09-10T21:33:33:125018+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":237,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:33.124958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:33.863 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 237 2024-09-10T21:33:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18f00725e0 msgr2=0x7f18f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18f00725e0 0x7f18f0074aa0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f19040b2000 tx=0x7f190401eb80 comp rx=0 tx=0).stop 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19140ceee0 msgr2=0x7f19140cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19140ceee0 0x7f19140cfc70 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c4b0 tx=0x7f18fc00c970 comp rx=0 tx=0).stop 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 shutdown_connections 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18f00725e0 0x7f18f0074aa0 secure :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f19040b2000 tx=0x7f190401eb80 comp rx=0 tx=0).stop 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19141511f0 0x7f19141400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19140ceee0 0x7f19140cfc70 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 --2- 172.21.15.97:0/2083091799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19140b3cf0 0x7f19141e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 >> 172.21.15.97:0/2083091799 conn(0x7f19140b1780 msgr2=0x7f19140ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 shutdown_connections 2024-09-10T21:33:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:33.867+0000 7f191b600640 1 -- 172.21.15.97:0/2083091799 wait complete. 2024-09-10T21:33:33.873 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:33:33.886 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:33.886 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:34.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.027+0000 7ff806600640 1 -- 172.21.15.97:0/2180805821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000cec10 msgr2=0x7ff8000cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.027+0000 7ff806600640 1 --2- 172.21.15.97:0/2180805821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000cec10 0x7ff8000cff10 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009900 tx=0x7ff7e801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- 172.21.15.97:0/2180805821 shutdown_connections 2024-09-10T21:33:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 --2- 172.21.15.97:0/2180805821 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8000b1560 0x7ff8000b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 --2- 172.21.15.97:0/2180805821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000cec10 0x7ff8000cff10 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- 172.21.15.97:0/2180805821 >> 172.21.15.97:0/2180805821 conn(0x7ff8000b37c0 msgr2=0x7ff8000b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- 172.21.15.97:0/2180805821 shutdown_connections 2024-09-10T21:33:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- 172.21.15.97:0/2180805821 wait complete. 2024-09-10T21:33:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 Processor -- start 2024-09-10T21:33:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- start start 2024-09-10T21:33:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8000b1560 0x7ff80015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000ce830 0x7ff8000b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8000cec10 0x7ff8000b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8000ce320 con 0x7ff8000ce830 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8000ce1a0 con 0x7ff8000b1560 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff806600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff8000d0450 con 0x7ff8000cec10 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000ce830 0x7ff8000b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000ce830 0x7ff8000b7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43906/0 (socket says 172.21.15.97:43906) 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 -- 172.21.15.97:0/1476242809 learned_addr learned my addr 172.21.15.97:0/1476242809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7ff400640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8000b1560 0x7ff80015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7ffe00640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8000cec10 0x7ff8000b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 -- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8000cec10 msgr2=0x7ff8000b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8000cec10 0x7ff8000b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 -- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8000b1560 msgr2=0x7ff80015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8000b1560 0x7ff80015c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 -- 172.21.15.97:0/1476242809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80015c820 con 0x7ff8000ce830 2024-09-10T21:33:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.031+0000 7ff7f6a00640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000ce830 0x7ff8000b7100 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009850 tx=0x7ff7e80a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff7f7e00640 1 -- 172.21.15.97:0/1476242809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80b5040 con 0x7ff8000ce830 2024-09-10T21:33:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff7f7e00640 1 -- 172.21.15.97:0/1476242809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7e8003970 con 0x7ff8000ce830 2024-09-10T21:33:34.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff7f7e00640 1 -- 172.21.15.97:0/1476242809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8003d40 con 0x7ff8000ce830 2024-09-10T21:33:34.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8000afd60 con 0x7ff8000ce830 2024-09-10T21:33:34.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8000b0320 con 0x7ff8000ce830 2024-09-10T21:33:34.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8000b1970 con 0x7ff8000ce830 2024-09-10T21:33:34.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff7f7e00640 1 -- 172.21.15.97:0/1476242809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7e80b8070 con 0x7ff8000ce830 2024-09-10T21:33:34.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7dc072610 0x7ff7dc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff7f7e00640 1 -- 172.21.15.97:0/1476242809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7ff7e8130a70 con 0x7ff8000ce830 2024-09-10T21:33:34.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.035+0000 7ff7ff400640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7dc072610 0x7ff7dc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.039+0000 7ff7ff400640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7dc072610 0x7ff7dc074ad0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0007fd0 tx=0x7ff7f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:34.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.039+0000 7ff7f7e00640 1 -- 172.21.15.97:0/1476242809 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff80015c820 con 0x7ff8000ce830 2024-09-10T21:33:34.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.243+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8000b2450 con 0x7ff8000ce830 2024-09-10T21:33:34.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.243+0000 7ff7f7e00640 1 -- 172.21.15.97:0/1476242809 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 238 v238) ==== 95+0+4395 (secure 0 0 0) 0x7ff7e8018c30 con 0x7ff8000ce830 2024-09-10T21:33:34.243 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:34.243 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":238,"btime":"2024-09-10T21:33:34:128302+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2264649568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2264649568},{"type":"v1","addr":"172.21.15.97:6835","nonce":2264649568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":238}],"filesystems":[{"mdsmap":{"epoch":237,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:33.124958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:34.243 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 238 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7dc072610 msgr2=0x7ff7dc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7dc072610 0x7ff7dc074ad0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0007fd0 tx=0x7ff7f0009040 comp rx=0 tx=0).stop 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000ce830 msgr2=0x7ff8000b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000ce830 0x7ff8000b7100 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009850 tx=0x7ff7e80a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 shutdown_connections 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7dc072610 0x7ff7dc074ad0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff8000cec10 0x7ff8000b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8000ce830 0x7ff8000b7100 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 --2- 172.21.15.97:0/1476242809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8000b1560 0x7ff80015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 >> 172.21.15.97:0/1476242809 conn(0x7ff8000b37c0 msgr2=0x7ff80015f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:34.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 shutdown_connections 2024-09-10T21:33:34.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.247+0000 7ff806600640 1 -- 172.21.15.97:0/1476242809 wait complete. 2024-09-10T21:33:34.265 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2024-09-10T21:33:34.265 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2024-09-10T21:33:34.265 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:34.265 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:34.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:34.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:34.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:34.274 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:34.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:34.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.515+0000 7fb38b800640 1 -- 172.21.15.97:0/3687357712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36c003a10 msgr2=0x7fb36c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.515+0000 7fb38b800640 1 --2- 172.21.15.97:0/3687357712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb36c003a10 0x7fb36c003e30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fb378005fd0 tx=0x7fb37801cda0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.515+0000 7fb389800640 1 -- 172.21.15.97:0/3687357712 reap_dead start 2024-09-10T21:33:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.515+0000 7fb38b800640 1 -- 172.21.15.97:0/3687357712 shutdown_connections 2024-09-10T21:33:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.515+0000 7fb38b800640 1 -- 172.21.15.97:0/3687357712 >> 172.21.15.97:0/3687357712 conn(0x7fb3840b1780 msgr2=0x7fb3840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.515+0000 7fb38b800640 1 -- 172.21.15.97:0/3687357712 shutdown_connections 2024-09-10T21:33:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.515+0000 7fb38b800640 1 -- 172.21.15.97:0/3687357712 wait complete. 2024-09-10T21:33:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 Processor -- start 2024-09-10T21:33:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 -- start start 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb36c003a10 0x7fb38406f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f75d0 0x7fb38406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb3841fae20 0x7fb3841fb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3840b4490 con 0x7fb3841f75d0 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb3840b4310 con 0x7fb36c003a10 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb384070680 con 0x7fb3841fae20 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f75d0 0x7fb38406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f75d0 0x7fb38406f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43918/0 (socket says 172.21.15.97:43918) 2024-09-10T21:33:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 -- 172.21.15.97:0/1578106886 learned_addr learned my addr 172.21.15.97:0/1578106886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb388e00640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb36c003a10 0x7fb38406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb389800640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb3841fae20 0x7fb3841fb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 -- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb3841fae20 msgr2=0x7fb3841fb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb3841fae20 0x7fb3841fb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 -- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb36c003a10 msgr2=0x7fb38406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb36c003a10 0x7fb38406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 -- 172.21.15.97:0/1578106886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3841f7b00 con 0x7fb3841f75d0 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb383e00640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f75d0 0x7fb38406f980 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb370003f20 tx=0x7fb37000ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb380c00640 1 -- 172.21.15.97:0/1578106886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370010790 con 0x7fb3841f75d0 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb380c00640 1 -- 172.21.15.97:0/1578106886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb370010930 con 0x7fb3841f75d0 2024-09-10T21:33:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb380c00640 1 -- 172.21.15.97:0/1578106886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370010d60 con 0x7fb3841f75d0 2024-09-10T21:33:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3841f7de0 con 0x7fb3841f75d0 2024-09-10T21:33:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3841f83a0 con 0x7fb3841f75d0 2024-09-10T21:33:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.519+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3840b2a80 con 0x7fb3841f75d0 2024-09-10T21:33:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.523+0000 7fb380c00640 1 -- 172.21.15.97:0/1578106886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb370023440 con 0x7fb3841f75d0 2024-09-10T21:33:34.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.523+0000 7fb380c00640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb364072610 0x7fb364074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:34.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.523+0000 7fb380c00640 1 -- 172.21.15.97:0/1578106886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7fb370094820 con 0x7fb3841f75d0 2024-09-10T21:33:34.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.523+0000 7fb388e00640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb364072610 0x7fb364074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.523+0000 7fb388e00640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb364072610 0x7fb364074ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb3780b0000 tx=0x7fb3780a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.527+0000 7fb380c00640 1 -- 172.21.15.97:0/1578106886 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb37005eda0 con 0x7fb3841f75d0 2024-09-10T21:33:34.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:34.727+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb38406ec40 con 0x7fb3841f75d0 2024-09-10T21:33:35.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb380c00640 1 -- 172.21.15.97:0/1578106886 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v239) ==== 161+0+0 (secure 0 0 0) 0x7fb3700637d0 con 0x7fb3841f75d0 2024-09-10T21:33:35.137 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:35.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb364072610 msgr2=0x7fb364074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:35.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb364072610 0x7fb364074ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb3780b0000 tx=0x7fb3780a7040 comp rx=0 tx=0).stop 2024-09-10T21:33:35.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f75d0 msgr2=0x7fb38406f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:35.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f75d0 0x7fb38406f980 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb370003f20 tx=0x7fb37000ba60 comp rx=0 tx=0).stop 2024-09-10T21:33:35.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 shutdown_connections 2024-09-10T21:33:35.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb364072610 0x7fb364074ad0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb3841fae20 0x7fb3841fb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3841f75d0 0x7fb38406f980 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 --2- 172.21.15.97:0/1578106886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb36c003a10 0x7fb38406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 >> 172.21.15.97:0/1578106886 conn(0x7fb3840b1780 msgr2=0x7fb3840cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 shutdown_connections 2024-09-10T21:33:35.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.139+0000 7fb38b800640 1 -- 172.21.15.97:0/1578106886 wait complete. 2024-09-10T21:33:35.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 -- 172.21.15.97:0/2842435684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e80051d0 msgr2=0x7f99e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 --2- 172.21.15.97:0/2842435684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e80051d0 0x7f99e800b9d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009ee0 tx=0x7f99fc01d630 comp rx=0 tx=0).stop 2024-09-10T21:33:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 -- 172.21.15.97:0/2842435684 shutdown_connections 2024-09-10T21:33:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 --2- 172.21.15.97:0/2842435684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e80051d0 0x7f99e800b9d0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 -- 172.21.15.97:0/2842435684 >> 172.21.15.97:0/2842435684 conn(0x7f9a040b37c0 msgr2=0x7f9a040b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 -- 172.21.15.97:0/2842435684 shutdown_connections 2024-09-10T21:33:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 -- 172.21.15.97:0/2842435684 wait complete. 2024-09-10T21:33:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 Processor -- start 2024-09-10T21:33:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.303+0000 7f9a08a00640 1 -- start start 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99e80051d0 0x7f9a040b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a040b5d60 0x7f9a040b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a040b53b0 0x7f9a040c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a040ce320 con 0x7f9a040b53b0 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a040ce1a0 con 0x7f9a040b5d60 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9a040b42e0 con 0x7f99e80051d0 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a040b53b0 0x7f9a040c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a040b53b0 0x7f9a040c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43926/0 (socket says 172.21.15.97:43926) 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 -- 172.21.15.97:0/1632550976 learned_addr learned my addr 172.21.15.97:0/1632550976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a01400640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a040b5d60 0x7f9a040b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a01e00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99e80051d0 0x7f9a040b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:35.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 -- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99e80051d0 msgr2=0x7f9a040b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99e80051d0 0x7f9a040b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 -- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a040b5d60 msgr2=0x7f9a040b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a040b5d60 0x7f9a040b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 -- 172.21.15.97:0/1632550976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0406fae0 con 0x7f9a040b53b0 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a01e00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99e80051d0 0x7f9a040b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a02800640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a040b53b0 0x7f9a040c46a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f99fc01d460 tx=0x7f99fc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f99fa000640 1 -- 172.21.15.97:0/1632550976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc009ab0 con 0x7f9a040b53b0 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f99fa000640 1 -- 172.21.15.97:0/1632550976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99fc004a50 con 0x7f9a040b53b0 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0406f560 con 0x7f9a040b53b0 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f99fa000640 1 -- 172.21.15.97:0/1632550976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc004870 con 0x7f9a040b53b0 2024-09-10T21:33:35.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a040b65b0 con 0x7f9a040b53b0 2024-09-10T21:33:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a040713e0 con 0x7f9a040b53b0 2024-09-10T21:33:35.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.307+0000 7f99fa000640 1 -- 172.21.15.97:0/1632550976 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f99fc005030 con 0x7f9a040b53b0 2024-09-10T21:33:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.311+0000 7f99fa000640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99e40725e0 0x7f99e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.311+0000 7f99fa000640 1 -- 172.21.15.97:0/1632550976 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f99fc12dcd0 con 0x7f9a040b53b0 2024-09-10T21:33:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.311+0000 7f9a01e00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99e40725e0 0x7f99e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:35.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.311+0000 7f9a01e00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99e40725e0 0x7f99e4074aa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f99f4000fc0 tx=0x7f99f4001610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:35.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.315+0000 7f99fa000640 1 -- 172.21.15.97:0/1632550976 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99fc0b1070 con 0x7f9a040b53b0 2024-09-10T21:33:35.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:35.539+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f9a04071100 con 0x7f9a040b53b0 2024-09-10T21:33:36.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.147+0000 7f99fa000640 1 -- 172.21.15.97:0/1632550976 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v240) ==== 101+0+0 (secure 0 0 0) 0x7f99fc017340 con 0x7f9a040b53b0 2024-09-10T21:33:36.147 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:36.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99e40725e0 msgr2=0x7f99e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99e40725e0 0x7f99e4074aa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f99f4000fc0 tx=0x7f99f4001610 comp rx=0 tx=0).stop 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a040b53b0 msgr2=0x7f9a040c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a040b53b0 0x7f9a040c46a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f99fc01d460 tx=0x7f99fc004680 comp rx=0 tx=0).stop 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 shutdown_connections 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99e40725e0 0x7f99e4074aa0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a040b53b0 0x7f9a040c46a0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a040b5d60 0x7f9a040b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 --2- 172.21.15.97:0/1632550976 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99e80051d0 0x7f9a040b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 >> 172.21.15.97:0/1632550976 conn(0x7f9a040b37c0 msgr2=0x7f9a04156090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 shutdown_connections 2024-09-10T21:33:36.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.151+0000 7f9a08a00640 1 -- 172.21.15.97:0/1632550976 wait complete. 2024-09-10T21:33:36.170 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:36.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.215+0000 7fad84018080 1 Processor -- start 2024-09-10T21:33:36.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.215+0000 7fad84018080 1 -- start start 2024-09-10T21:33:36.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68002870 0x55d1973e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973dbd50 0x55d1973b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d1973b9d00 0x55d1973c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d1972b2f40 con 0x7fad68002870 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d1972bfd00 con 0x55d1973dbd50 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d197254a00 con 0x55d1973b9d00 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68002870 0x55d1973e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68002870 0x55d1973e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43946/0 (socket says 172.21.15.97:43946) 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 -- 172.21.15.97:0/3020072099 learned_addr learned my addr 172.21.15.97:0/3020072099 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad7be00640 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973dbd50 0x55d1973b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81c00640 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d1973b9d00 0x55d1973c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 -- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d1973b9d00 msgr2=0x55d1973c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d1973b9d00 0x55d1973c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 -- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973dbd50 msgr2=0x55d1973b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973dbd50 0x55d1973b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 -- 172.21.15.97:0/3020072099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1973e5590 con 0x7fad68002870 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81c00640 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d1973b9d00 0x55d1973c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:36.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad81200640 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68002870 0x55d1973e7d50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c460 tx=0x7fad7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa73cdc687f297a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad7aa00640 1 -- 172.21.15.97:0/3020072099 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74013070 con 0x7fad68002870 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad7aa00640 1 -- 172.21.15.97:0/3020072099 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad7400e770 con 0x7fad68002870 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad7aa00640 1 -- 172.21.15.97:0/3020072099 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7400ebc0 con 0x7fad68002870 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- 172.21.15.97:0/3020072099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68002870 msgr2=0x55d1973e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68002870 0x55d1973e7d50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c460 tx=0x7fad7400c920 comp rx=0 tx=0).stop 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- 172.21.15.97:0/3020072099 shutdown_connections 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d1973b9d00 0x55d1973c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973dbd50 0x55d1973b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- 172.21.15.97:0/3020072099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68002870 0x55d1973e7d50 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- 172.21.15.97:0/3020072099 >> 172.21.15.97:0/3020072099 conn(0x55d197334520 msgr2=0x55d1973dc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- 172.21.15.97:0/3020072099 shutdown_connections 2024-09-10T21:33:36.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- 172.21.15.97:0/3020072099 wait complete. 2024-09-10T21:33:36.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 Processor -- start 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 -- start start 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.219+0000 7fad84018080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1973dbd50 0x55d1973bb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad84018080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fad68002870 0x55d1973d8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad84018080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973e7cb0 0x55d1973da910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad84018080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d1973bee60 con 0x55d1973dbd50 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad84018080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d1973e5830 con 0x55d1973e7cb0 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad84018080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d1973bf100 con 0x7fad68002870 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1973dbd50 0x55d1973bb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1973dbd50 0x55d1973bb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43962/0 (socket says 172.21.15.97:43962) 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 learned_addr learned my addr 172.21.15.97:0/799471038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:36.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad7be00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fad68002870 0x55d1973d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81c00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973e7cb0 0x55d1973da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fad68002870 msgr2=0x55d1973d8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fad68002870 0x55d1973d8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973e7cb0 msgr2=0x55d1973da910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1973e7cb0 0x55d1973da910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1973e68a0 con 0x55d1973dbd50 2024-09-10T21:33:36.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad81200640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1973dbd50 0x55d1973bb8f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fad7400ce20 tx=0x7fad7400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad79600640 1 -- 172.21.15.97:0/799471038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74013030 con 0x55d1973dbd50 2024-09-10T21:33:36.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad79600640 1 -- 172.21.15.97:0/799471038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad74002570 con 0x55d1973dbd50 2024-09-10T21:33:36.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1973e6b40 con 0x55d1973dbd50 2024-09-10T21:33:36.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad79600640 1 -- 172.21.15.97:0/799471038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740017f0 con 0x55d1973dbd50 2024-09-10T21:33:36.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1973e7080 con 0x55d1973dbd50 2024-09-10T21:33:36.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.223+0000 7fad79600640 1 -- 172.21.15.97:0/799471038 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fad74020070 con 0x55d1973dbd50 2024-09-10T21:33:36.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad79600640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fad60070fc0 0x7fad60073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad79600640 1 -- 172.21.15.97:0/799471038 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7fad74094380 con 0x55d1973dbd50 2024-09-10T21:33:36.226 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d1973de2d0 0x55d1973de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1973d86a0 con 0x55d1973de2d0 2024-09-10T21:33:36.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad7be00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fad60070fc0 0x7fad60073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad81c00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d1973de2d0 0x55d1973de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad7be00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fad60070fc0 0x7fad60073480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0ab000 tx=0x7fad6c0184b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad81c00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d1973de2d0 0x55d1973de6b0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v122'784 uv758 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.227+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d1973debf0 con 0x55d1973de2d0 2024-09-10T21:33:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.231+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v122'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.231+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197483980 con 0x55d1973de2d0 2024-09-10T21:33:36.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.231+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v122'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.232 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.231+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484d30 con 0x55d1973de2d0 2024-09-10T21:33:36.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.235+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.235+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974850d0 con 0x55d1973de2d0 2024-09-10T21:33:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.235+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.235+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197483d20 con 0x55d1973de2d0 2024-09-10T21:33:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.239+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.239+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484110 con 0x55d1973de2d0 2024-09-10T21:33:36.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.239+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v122'790 uv764 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.239+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197484110 con 0x55d1973de2d0 2024-09-10T21:33:36.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.243+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v122'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.243+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.242 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:36.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.243+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v122'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.243+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.247+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.247+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.247+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.247+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.247+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.247+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.251+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v122'796 uv771 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.251+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.251+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'797 uv797 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.251+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.252 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:36.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.255+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v122'798 uv798 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.255+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.255+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'799 uv799 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.255+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.255+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'800 uv800 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.255+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974840c0 con 0x55d1973de2d0 2024-09-10T21:33:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.259+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'801 uv801 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.259+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974844b0 con 0x55d1973de2d0 2024-09-10T21:33:36.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.259+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'802 uv802 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.259+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974844b0 con 0x55d1973de2d0 2024-09-10T21:33:36.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.263+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v122'803 uv777 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.263+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.263+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v122'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.263+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.264 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:36.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.267+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v122'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.267+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.267+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.267+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.267+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.267+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.271+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.271+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.271+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v122'809 uv783 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.271+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.275+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v122'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.275+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.274 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.275+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v122'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.275+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.275+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.275+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.279+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.279+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197484340 con 0x55d1973de2d0 2024-09-10T21:33:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.279+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d1973de2d0 2024-09-10T21:33:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.279+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d197484340 0x55d197487480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.279+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974879c0 con 0x55d197484340 2024-09-10T21:33:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.279+0000 7fad81200640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d197484340 0x55d197487480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.283+0000 7fad81200640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d197484340 0x55d197487480 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.283+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'571 uv553 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.283+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197487d60 con 0x55d197484340 2024-09-10T21:33:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.283+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'572 uv572 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.283+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197488100 con 0x55d197484340 2024-09-10T21:33:36.285 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.287+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'573 uv573 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.287+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197488950 con 0x55d197484340 2024-09-10T21:33:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.287+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v122'574 uv574 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.287+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197488cf0 con 0x55d197484340 2024-09-10T21:33:36.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.291+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'575 uv575 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.291+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489100 con 0x55d197484340 2024-09-10T21:33:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.291+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v122'576 uv558 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.291+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d1974894f0 con 0x55d197484340 2024-09-10T21:33:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.291+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v122'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.291+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.295+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v122'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.292 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:36.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.295+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.295+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'579 uv579 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.295+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.295+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'580 uv580 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.295+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.299+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v122'581 uv570 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.299+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.299+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v122'582 uv582 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.299+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.299+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v122'583 uv583 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.299 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.299+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748da60 con 0x55d197484340 2024-09-10T21:33:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.303+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v122'584 uv584 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.303+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748da60 con 0x55d197484340 2024-09-10T21:33:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.303+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'585 uv585 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.303+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.303+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v122'586 uv569 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.303+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.307+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v122'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.307+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.307+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v122'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.306 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.307+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748a8a0 con 0x55d197484340 2024-09-10T21:33:36.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.307+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.307+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748a8a0 con 0x55d197484340 2024-09-10T21:33:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.311+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.311+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197489890 con 0x55d197484340 2024-09-10T21:33:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.311+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74028020 con 0x55d197484340 2024-09-10T21:33:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.311+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d19748a160 0x55d19748a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.311+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748ccc0 con 0x55d19748a160 2024-09-10T21:33:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.311+0000 7fad7be00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d19748a160 0x55d19748a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.311+0000 7fad7be00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d19748a160 0x55d19748a580 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.315+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.315+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d19748d060 con 0x55d19748a160 2024-09-10T21:33:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.315+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.315+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748d400 con 0x55d19748a160 2024-09-10T21:33:36.315 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:36.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.315+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.315+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748e7b0 con 0x55d19748a160 2024-09-10T21:33:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748ec30 con 0x55d19748a160 2024-09-10T21:33:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d19748d7a0 con 0x55d19748a160 2024-09-10T21:33:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d19748e260 0x55d197491070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974915d0 con 0x55d19748e260 2024-09-10T21:33:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad81c00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d19748e260 0x55d197491070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.319+0000 7fad81c00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d19748e260 0x55d197491070 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.323+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'115 uv114 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.323+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d1974919c0 con 0x55d19748e260 2024-09-10T21:33:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.323+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'116 uv116 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.323+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974919c0 con 0x55d19748e260 2024-09-10T21:33:36.323 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.323+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'117 uv117 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.323+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974919c0 con 0x55d19748e260 2024-09-10T21:33:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.327+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'118 uv118 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.327+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491900 con 0x55d19748a160 2024-09-10T21:33:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.327+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.327+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197491900 con 0x55d19748a160 2024-09-10T21:33:36.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.327+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.327+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748a160 2024-09-10T21:33:36.328 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:36.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748a160 2024-09-10T21:33:36.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748a160 2024-09-10T21:33:36.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748a160 2024-09-10T21:33:36.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.331+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.335+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v122'282 uv276 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.335+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.335+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v122'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.335+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.336 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:36.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v122'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.339+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.343+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v122'288 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.343+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.343+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v122'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.343+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:36.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.343+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v122'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.343+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974918e0 con 0x55d19748e260 2024-09-10T21:33:36.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d19748e260 2024-09-10T21:33:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d19748a160 2024-09-10T21:33:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v122'127 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d19748a160 2024-09-10T21:33:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v122'128 uv128 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.347+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d19748a160 2024-09-10T21:33:36.348 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v122'129 uv129 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d19748a160 2024-09-10T21:33:36.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'130 uv130 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d19748a160 2024-09-10T21:33:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'131 uv131 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d19748a160 2024-09-10T21:33:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d1974926d0 0x55d197494e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad81200640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d1974926d0 0x55d197494e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.351+0000 7fad81200640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d1974926d0 0x55d197494e10 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.355+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.355+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.355+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.355+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.355 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.355+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.355+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.359+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.359+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.359+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.359+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.359+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.359+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.363+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v122'253 uv247 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.363+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.363+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v122'254 uv254 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.363+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197491dd0 con 0x55d1974926d0 2024-09-10T21:33:36.362 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:36.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.363+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v122'255 uv255 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.363+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974953a0 con 0x55d1974926d0 2024-09-10T21:33:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'256 uv256 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d1974953a0 con 0x55d1974926d0 2024-09-10T21:33:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'257 uv257 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495350 con 0x55d1974926d0 2024-09-10T21:33:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v122'258 uv252 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197495350 con 0x55d1974926d0 2024-09-10T21:33:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v122'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.367+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495350 con 0x55d1974926d0 2024-09-10T21:33:36.369 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.371+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v122'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.371+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495350 con 0x55d1974926d0 2024-09-10T21:33:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.371+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.371+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.371+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.371+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.375+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v122'111 uv110 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.375+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.375+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'112 uv112 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.375+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.379+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v122'113 uv113 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.377 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:36.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.379+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.379+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'114 uv114 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.379+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.379+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v122'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.379+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.381 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.383+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d19748e260 2024-09-10T21:33:36.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.387+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad7c005290 con 0x55d19748e260 2024-09-10T21:33:36.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.387+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d1974963d0 0x55d1974987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.387+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974963d0 2024-09-10T21:33:36.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.387+0000 7fad7be00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d1974963d0 0x55d1974987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.387+0000 7fad7be00640 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d1974963d0 0x55d1974987c0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.387+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v122'130 uv129 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d1974963d0 2024-09-10T21:33:36.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.387+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974963d0 2024-09-10T21:33:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v122'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d1974963d0 2024-09-10T21:33:36.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974963d0 2024-09-10T21:33:36.389 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:36.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v122'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d1974963d0 2024-09-10T21:33:36.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974963d0 2024-09-10T21:33:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d1974963d0 2024-09-10T21:33:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974963d0 2024-09-10T21:33:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad7be00640 1 -- 172.21.15.97:0/799471038 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fad6c0ad020 con 0x55d1974963d0 2024-09-10T21:33:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.391+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.395+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v122'314 uv306 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.395+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.395+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v122'315 uv315 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.395+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.395 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.395+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v122'316 uv316 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.395+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.399+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v122'317 uv317 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.399+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.399+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'318 uv318 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.399+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.399+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v122'319 uv313 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.399+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.403+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v122'320 uv320 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.403+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.401 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.403+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v122'321 uv321 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.403+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.403+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'322 uv322 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.403+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'323 uv323 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197495800 con 0x55d1974926d0 2024-09-10T21:33:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'324 uv324 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55d197498d00 con 0x55d1974926d0 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad81200640 1 -- 172.21.15.97:0/799471038 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'325 uv325 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fad74049040 con 0x55d1974926d0 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d19748a160 msgr2=0x55d19748a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55d19748a160 0x55d19748a580 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d1974926d0 msgr2=0x55d197494e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55d1974926d0 0x55d197494e10 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d1974963d0 msgr2=0x55d1974987c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55d1974963d0 0x55d1974987c0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d19748e260 msgr2=0x55d197491070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d19748e260 0x55d197491070 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d197484340 msgr2=0x55d197487480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55d197484340 0x55d197487480 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 reap_dead start 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d1973de2d0 msgr2=0x55d1973de6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55d1973de2d0 0x55d1973de6b0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 reap_dead start 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad81c00640 1 -- 172.21.15.97:0/799471038 reap_dead start 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fad60070fc0 msgr2=0x7fad60073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fad60070fc0 0x7fad60073480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0ab000 tx=0x7fad6c0184b0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1973dbd50 msgr2=0x55d1973bb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1973dbd50 0x55d1973bb8f0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fad7400ce20 tx=0x7fad7400ef80 comp rx=0 tx=0).stop 2024-09-10T21:33:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 shutdown_connections 2024-09-10T21:33:36.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fad60070fc0 0x7fad60073480 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 --2- 172.21.15.97:0/799471038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1973dbd50 0x55d1973bb8f0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 >> 172.21.15.97:0/799471038 conn(0x55d197334520 msgr2=0x55d1973c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:36.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 shutdown_connections 2024-09-10T21:33:36.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.407+0000 7fad84018080 1 -- 172.21.15.97:0/799471038 wait complete. 2024-09-10T21:33:36.412 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2024-09-10T21:33:36.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 Processor -- start 2024-09-10T21:33:36.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- start start 2024-09-10T21:33:36.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f91cc0 0x55adb6f23540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6eda350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f971d0 0x55adb6eda890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55adb6e27a00 con 0x55adb6f91cc0 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55adb6e85f40 con 0x55adb6f93a00 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55adb6f95710 con 0x55adb6f971d0 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f91cc0 0x55adb6f23540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f91cc0 0x55adb6f23540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43974/0 (socket says 172.21.15.97:43974) 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208c400640 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f971d0 0x55adb6eda890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 -- 172.21.15.97:0/962899699 learned_addr learned my addr 172.21.15.97:0/962899699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208b000640 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6eda350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 -- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f971d0 msgr2=0x55adb6eda890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:36.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f971d0 0x55adb6eda890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 -- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 msgr2=0x55adb6eda350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6eda350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 -- 172.21.15.97:0/962899699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb6e00c10 con 0x55adb6f91cc0 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208b000640 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6eda350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208ba00640 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f91cc0 0x55adb6f23540 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b670 tx=0x7f207c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f360a2d0cd65673 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f2089c00640 1 -- 172.21.15.97:0/962899699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00a980 con 0x55adb6f91cc0 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f2089c00640 1 -- 172.21.15.97:0/962899699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f207c00ab20 con 0x55adb6f91cc0 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f2089c00640 1 -- 172.21.15.97:0/962899699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c008460 con 0x55adb6f91cc0 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- 172.21.15.97:0/962899699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f91cc0 msgr2=0x55adb6f23540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f91cc0 0x55adb6f23540 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b670 tx=0x7f207c004f90 comp rx=0 tx=0).stop 2024-09-10T21:33:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- 172.21.15.97:0/962899699 shutdown_connections 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f971d0 0x55adb6eda890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6eda350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 --2- 172.21.15.97:0/962899699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f91cc0 0x55adb6f23540 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- 172.21.15.97:0/962899699 >> 172.21.15.97:0/962899699 conn(0x55adb6f210d0 msgr2=0x55adb6f09e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- 172.21.15.97:0/962899699 shutdown_connections 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.503+0000 7f208e770080 1 -- 172.21.15.97:0/962899699 wait complete. 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 Processor -- start 2024-09-10T21:33:36.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 -- start start 2024-09-10T21:33:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f91cc0 0x55adb6fad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6fb7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f971d0 0x55adb6fb7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55adb6faff00 con 0x55adb6f971d0 2024-09-10T21:33:36.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55adb6edca90 con 0x55adb6f93a00 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55adb6fb01a0 con 0x55adb6f91cc0 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f971d0 0x55adb6fb7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208ba00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f91cc0 0x55adb6fad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f971d0 0x55adb6fb7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43978/0 (socket says 172.21.15.97:43978) 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208c400640 1 -- 172.21.15.97:0/1178517964 learned_addr learned my addr 172.21.15.97:0/1178517964 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208b000640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6fb7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208ba00640 1 -- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 msgr2=0x55adb6fb7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208ba00640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adb6f93a00 0x55adb6fb7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208ba00640 1 -- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f971d0 msgr2=0x55adb6fb7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208ba00640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f971d0 0x55adb6fb7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208ba00640 1 -- 172.21.15.97:0/1178517964 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb6f0a030 con 0x55adb6f91cc0 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208c400640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55adb6f971d0 0x55adb6fb7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208ba00640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f91cc0 0x55adb6fad090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b670 tx=0x7f207c017eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f2083e00640 1 -- 172.21.15.97:0/1178517964 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00acf0 con 0x55adb6f91cc0 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f2083e00640 1 -- 172.21.15.97:0/1178517964 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f207c020440 con 0x55adb6f91cc0 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f2083e00640 1 -- 172.21.15.97:0/1178517964 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c020890 con 0x55adb6f91cc0 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55adb6f0a2d0 con 0x55adb6f91cc0 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.507+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55adb6fad5d0 con 0x55adb6f91cc0 2024-09-10T21:33:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f2083e00640 1 -- 172.21.15.97:0/1178517964 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f207c01e020 con 0x55adb6f91cc0 2024-09-10T21:33:36.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f2083e00640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f206c070fc0 0x7f206c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f2083e00640 1 -- 172.21.15.97:0/1178517964 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f207c0944d0 con 0x55adb6f91cc0 2024-09-10T21:33:36.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f208b000640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f206c070fc0 0x7f206c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f208e770080 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55adb6fb1540 0x55adb6fb1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55adb6fb1e60 con 0x55adb6fb1540 2024-09-10T21:33:36.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f208c400640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55adb6fb1540 0x55adb6fb1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f208b000640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f206c070fc0 0x7f206c073480 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2078011570 tx=0x7f20780017e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.511+0000 7f208c400640 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55adb6fb1540 0x55adb6fb1920 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208c400640 1 -- 172.21.15.97:0/1178517964 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v122'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2084002050 con 0x55adb6fb1540 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55adb6fb1540 msgr2=0x55adb6fb1920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55adb6fb1540 0x55adb6fb1920 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f206c070fc0 msgr2=0x7f206c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f206c070fc0 0x7f206c073480 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2078011570 tx=0x7f20780017e0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f91cc0 msgr2=0x55adb6fad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 --2- 172.21.15.97:0/1178517964 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55adb6f91cc0 0x55adb6fad090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b670 tx=0x7f207c017eb0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208c400640 1 -- 172.21.15.97:0/1178517964 reap_dead start 2024-09-10T21:33:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 shutdown_connections 2024-09-10T21:33:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 >> 172.21.15.97:0/1178517964 conn(0x55adb6f210d0 msgr2=0x55adb6f202f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 shutdown_connections 2024-09-10T21:33:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.515+0000 7f208e770080 1 -- 172.21.15.97:0/1178517964 wait complete. 2024-09-10T21:33:36.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:36.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 -- 172.21.15.97:0/246511520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd414072cc0 msgr2=0x7fd4140ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 --2- 172.21.15.97:0/246511520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd414072cc0 0x7fd4140ce630 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd400009900 tx=0x7fd40001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 -- 172.21.15.97:0/246511520 shutdown_connections 2024-09-10T21:33:36.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 --2- 172.21.15.97:0/246511520 >> v2:172.21.15.194:3300/0 conn(0x7fd4140ad9b0 0x7fd4140ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 --2- 172.21.15.97:0/246511520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd414072cc0 0x7fd4140ce630 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 -- 172.21.15.97:0/246511520 >> 172.21.15.97:0/246511520 conn(0x7fd4140d0070 msgr2=0x7fd4140d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:36.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 -- 172.21.15.97:0/246511520 shutdown_connections 2024-09-10T21:33:36.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.707+0000 7fd419400640 1 -- 172.21.15.97:0/246511520 wait complete. 2024-09-10T21:33:36.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 Processor -- start 2024-09-10T21:33:36.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 -- start start 2024-09-10T21:33:36.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd414072cc0 0x7fd4141fbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4140acec0 0x7fd414153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4140ad9b0 0x7fd4140cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd414169360 con 0x7fd414072cc0 2024-09-10T21:33:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141691e0 con 0x7fd4140ad9b0 2024-09-10T21:33:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd4141694e0 con 0x7fd4140acec0 2024-09-10T21:33:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd414072cc0 0x7fd4141fbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd414072cc0 0x7fd4141fbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43990/0 (socket says 172.21.15.97:43990) 2024-09-10T21:33:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 -- 172.21.15.97:0/1538373036 learned_addr learned my addr 172.21.15.97:0/1538373036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4140acec0 0x7fd414153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd418a00640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4140ad9b0 0x7fd4140cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 -- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4140acec0 msgr2=0x7fd414153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4140acec0 0x7fd414153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 -- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4140ad9b0 msgr2=0x7fd4140cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4140ad9b0 0x7fd4140cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 -- 172.21.15.97:0/1538373036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4141fc0f0 con 0x7fd414072cc0 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4140acec0 0x7fd414153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd418a00640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4140ad9b0 0x7fd4140cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd413e00640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd414072cc0 0x7fd4141fbbb0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c500 tx=0x7fd40400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd40be00640 1 -- 172.21.15.97:0/1538373036 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404017070 con 0x7fd414072cc0 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd40be00640 1 -- 172.21.15.97:0/1538373036 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd404003600 con 0x7fd414072cc0 2024-09-10T21:33:36.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4141f80c0 con 0x7fd414072cc0 2024-09-10T21:33:36.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd40be00640 1 -- 172.21.15.97:0/1538373036 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404003a50 con 0x7fd414072cc0 2024-09-10T21:33:36.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.711+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4141f8680 con 0x7fd414072cc0 2024-09-10T21:33:36.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.715+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4140ad2a0 con 0x7fd414072cc0 2024-09-10T21:33:36.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.715+0000 7fd40be00640 1 -- 172.21.15.97:0/1538373036 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd404003bf0 con 0x7fd414072cc0 2024-09-10T21:33:36.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.715+0000 7fd40be00640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd3f40725e0 0x7fd3f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:36.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.715+0000 7fd40be00640 1 -- 172.21.15.97:0/1538373036 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7fd4040934d0 con 0x7fd414072cc0 2024-09-10T21:33:36.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.715+0000 7fd413400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd3f40725e0 0x7fd3f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:36.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.715+0000 7fd413400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd3f40725e0 0x7fd3f4074aa0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd4000af000 tx=0x7fd40001b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:36.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.719+0000 7fd40be00640 1 -- 172.21.15.97:0/1538373036 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4141fc0f0 con 0x7fd414072cc0 2024-09-10T21:33:36.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:36.919+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd414074a90 con 0x7fd414072cc0 2024-09-10T21:33:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd40be00640 1 -- 172.21.15.97:0/1538373036 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v241) ==== 162+0+0 (secure 0 0 0) 0x7fd40405e160 con 0x7fd414072cc0 2024-09-10T21:33:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd3f40725e0 msgr2=0x7fd3f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd3f40725e0 0x7fd3f4074aa0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd4000af000 tx=0x7fd40001b120 comp rx=0 tx=0).stop 2024-09-10T21:33:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd414072cc0 msgr2=0x7fd4141fbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd414072cc0 0x7fd4141fbbb0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c500 tx=0x7fd40400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 shutdown_connections 2024-09-10T21:33:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd3f40725e0 0x7fd3f4074aa0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4140ad9b0 0x7fd4140cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4140acec0 0x7fd414153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 --2- 172.21.15.97:0/1538373036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd414072cc0 0x7fd4141fbbb0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 >> 172.21.15.97:0/1538373036 conn(0x7fd4140d0070 msgr2=0x7fd414165b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.155+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 shutdown_connections 2024-09-10T21:33:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.159+0000 7fd419400640 1 -- 172.21.15.97:0/1538373036 wait complete. 2024-09-10T21:33:37.585 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:37.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 -- 172.21.15.97:0/3591369154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2806fd50 msgr2=0x7f2b280701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:37.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 --2- 172.21.15.97:0/3591369154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2806fd50 0x7f2b280701a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f2b2815e650 tx=0x7f2b2001cd80 comp rx=0 tx=0).stop 2024-09-10T21:33:37.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 -- 172.21.15.97:0/3591369154 shutdown_connections 2024-09-10T21:33:37.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 --2- 172.21.15.97:0/3591369154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2806fd50 0x7f2b280701a0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 --2- 172.21.15.97:0/3591369154 >> v2:172.21.15.194:3300/0 conn(0x7f2b28144690 0x7f2b280b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 -- 172.21.15.97:0/3591369154 >> 172.21.15.97:0/3591369154 conn(0x7f2b280b1780 msgr2=0x7f2b280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:37.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 -- 172.21.15.97:0/3591369154 shutdown_connections 2024-09-10T21:33:37.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 -- 172.21.15.97:0/3591369154 wait complete. 2024-09-10T21:33:37.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 Processor -- start 2024-09-10T21:33:37.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.731+0000 7f2b2d200640 1 -- start start 2024-09-10T21:33:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b2806fd50 0x7f2b2815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28143f90 0x7f2b2815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b28144690 0x7f2b2815de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b28167c40 con 0x7f2b28143f90 2024-09-10T21:33:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b28167ac0 con 0x7f2b2806fd50 2024-09-10T21:33:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b28167dc0 con 0x7f2b28144690 2024-09-10T21:33:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28143f90 0x7f2b2815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28143f90 0x7f2b2815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51848/0 (socket says 172.21.15.97:51848) 2024-09-10T21:33:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 -- 172.21.15.97:0/897251658 learned_addr learned my addr 172.21.15.97:0/897251658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25e00640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b2806fd50 0x7f2b2815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b26800640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b28144690 0x7f2b2815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 -- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b28144690 msgr2=0x7f2b2815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b28144690 0x7f2b2815de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 -- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b2806fd50 msgr2=0x7f2b2815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b2806fd50 0x7f2b2815ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 -- 172.21.15.97:0/897251658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2815c2a0 con 0x7f2b28143f90 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b26800640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b28144690 0x7f2b2815de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b25400640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28143f90 0x7f2b2815b850 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2b140058d0 tx=0x7f2b1400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b1e000640 1 -- 172.21.15.97:0/897251658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14017070 con 0x7f2b28143f90 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b1e000640 1 -- 172.21.15.97:0/897251658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b14009d20 con 0x7f2b28143f90 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b1e000640 1 -- 172.21.15.97:0/897251658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140106c0 con 0x7f2b28143f90 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2815c580 con 0x7f2b28143f90 2024-09-10T21:33:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2813ac90 con 0x7f2b28143f90 2024-09-10T21:33:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.735+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b280701a0 con 0x7f2b28143f90 2024-09-10T21:33:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.739+0000 7f2b1e000640 1 -- 172.21.15.97:0/897251658 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b14010860 con 0x7f2b28143f90 2024-09-10T21:33:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.739+0000 7f2b1e000640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b08072610 0x7f2b08074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.739+0000 7f2b1e000640 1 -- 172.21.15.97:0/897251658 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f2b140940e0 con 0x7f2b28143f90 2024-09-10T21:33:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.739+0000 7f2b25e00640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b08072610 0x7f2b08074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:37.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.739+0000 7f2b25e00640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b08072610 0x7f2b08074ad0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007fd0 tx=0x7f2b18007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.743+0000 7f2b1e000640 1 -- 172.21.15.97:0/897251658 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b1405e660 con 0x7f2b28143f90 2024-09-10T21:33:37.877 INFO:tasks.ceph.mds.a.smithi097.stderr: -41> 2024-09-10T21:33:33.123+0000 7f0ebd400640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T21:33:37.916 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:33:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b28140c20 con 0x7f2b28143f90 2024-09-10T21:33:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b1e000640 1 -- 172.21.15.97:0/897251658 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 242 v242) ==== 95+0+4404 (secure 0 0 0) 0x7f2b14063090 con 0x7f2b28143f90 2024-09-10T21:33:37.949 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:37.949 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":242,"btime":"2024-09-10T21:33:37:158828+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":242,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:37.158818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":242,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/2264649568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2264649568},{"type":"v1","addr":"172.21.15.97:6835","nonce":2264649568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 242 2024-09-10T21:33:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b08072610 msgr2=0x7f2b08074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b08072610 0x7f2b08074ad0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007fd0 tx=0x7f2b18007e40 comp rx=0 tx=0).stop 2024-09-10T21:33:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28143f90 msgr2=0x7f2b2815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28143f90 0x7f2b2815b850 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2b140058d0 tx=0x7f2b1400dde0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 shutdown_connections 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b08072610 0x7f2b08074ad0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b28144690 0x7f2b2815de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28143f90 0x7f2b2815b850 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 --2- 172.21.15.97:0/897251658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b2806fd50 0x7f2b2815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 >> 172.21.15.97:0/897251658 conn(0x7f2b280b1780 msgr2=0x7f2b28143040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 shutdown_connections 2024-09-10T21:33:37.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:37.951+0000 7f2b2d200640 1 -- 172.21.15.97:0/897251658 wait complete. 2024-09-10T21:33:37.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 242, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:33:37.158818+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9375}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9375': {'gid': 9375, 'name': 'c', 'rank': 0, 'incarnation': 242, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/2264649568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2264649568}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2264649568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:33:37.971 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9375:up:replay 2024-09-10T21:33:37.971 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:33:39.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:39.686 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:43.375 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 -- 172.21.15.97:0/3160348860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4144690 msgr2=0x7f54b40b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3160348860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4144690 0x7f54b40b2a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f549c009870 tx=0x7f549c01c860 comp rx=0 tx=0).stop 2024-09-10T21:33:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 -- 172.21.15.97:0/3160348860 shutdown_connections 2024-09-10T21:33:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3160348860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4144690 0x7f54b40b2a80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 -- 172.21.15.97:0/3160348860 >> 172.21.15.97:0/3160348860 conn(0x7f54b40b1780 msgr2=0x7f54b40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 -- 172.21.15.97:0/3160348860 shutdown_connections 2024-09-10T21:33:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 -- 172.21.15.97:0/3160348860 wait complete. 2024-09-10T21:33:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 Processor -- start 2024-09-10T21:33:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.531+0000 7f54b9e00640 1 -- start start 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b406fd50 0x7f54b4159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4143f90 0x7f54b40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54b4144690 0x7f54b41f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4169410 con 0x7f54b406fd50 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4169290 con 0x7f54b4144690 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f54b4169590 con 0x7f54b4143f90 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b406fd50 0x7f54b4159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b406fd50 0x7f54b4159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51894/0 (socket says 172.21.15.97:51894) 2024-09-10T21:33:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 -- 172.21.15.97:0/3026641527 learned_addr learned my addr 172.21.15.97:0/3026641527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54aa000640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4143f90 0x7f54b40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 -- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4143f90 msgr2=0x7f54b40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4143f90 0x7f54b40c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 -- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54b4144690 msgr2=0x7f54b41f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b3400640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54b4144690 0x7f54b41f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54b4144690 0x7f54b41f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 -- 172.21.15.97:0/3026641527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b415a960 con 0x7f54b406fd50 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54aa000640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4143f90 0x7f54b40c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b3400640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54b4144690 0x7f54b41f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b2a00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b406fd50 0x7f54b4159f90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f54a400cc50 tx=0x7f54a400cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54abe00640 1 -- 172.21.15.97:0/3026641527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4016020 con 0x7f54b406fd50 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54abe00640 1 -- 172.21.15.97:0/3026641527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54a400ab60 con 0x7f54b406fd50 2024-09-10T21:33:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b40c9d10 con 0x7f54b406fd50 2024-09-10T21:33:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54abe00640 1 -- 172.21.15.97:0/3026641527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4012420 con 0x7f54b406fd50 2024-09-10T21:33:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b40c8f30 con 0x7f54b406fd50 2024-09-10T21:33:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.535+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b40b2a80 con 0x7f54b406fd50 2024-09-10T21:33:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.539+0000 7f54abe00640 1 -- 172.21.15.97:0/3026641527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f54a40125c0 con 0x7f54b406fd50 2024-09-10T21:33:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.539+0000 7f54abe00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5490072610 0x7f5490074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.539+0000 7f54abe00640 1 -- 172.21.15.97:0/3026641527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f54a4094540 con 0x7f54b406fd50 2024-09-10T21:33:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.539+0000 7f54aa000640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5490072610 0x7f5490074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.539+0000 7f54aa000640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5490072610 0x7f5490074ad0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f549c0af000 tx=0x7f549c01b0a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.543+0000 7f54abe00640 1 -- 172.21.15.97:0/3026641527 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f54a4020020 con 0x7f54b406fd50 2024-09-10T21:33:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.743+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54b4142900 con 0x7f54b406fd50 2024-09-10T21:33:43.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54abe00640 1 -- 172.21.15.97:0/3026641527 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 245 v245) ==== 95+0+5237 (secure 0 0 0) 0x7f54b415a960 con 0x7f54b406fd50 2024-09-10T21:33:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:43.745 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":245,"btime":"2024-09-10T21:33:40:172483+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2561173061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2561173061},{"type":"v1","addr":"172.21.15.97:6834","nonce":2561173061}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":243}],"filesystems":[{"mdsmap":{"epoch":245,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:40.172469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":242,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/2264649568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2264649568},{"type":"v1","addr":"172.21.15.97:6835","nonce":2264649568}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9375,"qdb_cluster":[9375]},"id":6}]} 2024-09-10T21:33:43.745 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 245 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5490072610 msgr2=0x7f5490074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5490072610 0x7f5490074ad0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f549c0af000 tx=0x7f549c01b0a0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b406fd50 msgr2=0x7f54b4159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b406fd50 0x7f54b4159f90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f54a400cc50 tx=0x7f54a400cd40 comp rx=0 tx=0).stop 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 shutdown_connections 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5490072610 0x7f5490074ad0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54b4144690 0x7f54b41f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54b4143f90 0x7f54b40c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 --2- 172.21.15.97:0/3026641527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54b406fd50 0x7f54b4159f90 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 >> 172.21.15.97:0/3026641527 conn(0x7f54b40b1780 msgr2=0x7f54b40ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 shutdown_connections 2024-09-10T21:33:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.747+0000 7f54b9e00640 1 -- 172.21.15.97:0/3026641527 wait complete. 2024-09-10T21:33:43.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 245, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:33:40.172469+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9375}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9375': {'gid': 9375, 'name': 'c', 'rank': 0, 'incarnation': 242, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6835/2264649568', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2264649568}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2264649568}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9375, 'qdb_cluster': [9375]} 2024-09-10T21:33:43.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:33:43.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:33:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 -- 172.21.15.97:0/2102932834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc144690 msgr2=0x7fdfcc0b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2102932834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc144690 0x7fdfcc0b4120 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009840 tx=0x7fdfb801c820 comp rx=0 tx=0).stop 2024-09-10T21:33:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 -- 172.21.15.97:0/2102932834 shutdown_connections 2024-09-10T21:33:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2102932834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc144690 0x7fdfcc0b4120 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2102932834 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfcc143f90 0x7fdfcc0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 -- 172.21.15.97:0/2102932834 >> 172.21.15.97:0/2102932834 conn(0x7fdfcc0b1780 msgr2=0x7fdfcc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 -- 172.21.15.97:0/2102932834 shutdown_connections 2024-09-10T21:33:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 -- 172.21.15.97:0/2102932834 wait complete. 2024-09-10T21:33:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 Processor -- start 2024-09-10T21:33:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.911+0000 7fdfd3400640 1 -- start start 2024-09-10T21:33:43.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfcc0b4660 0x7fdfcc136970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc143f90 0x7fdfcc164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfcc144690 0x7fdfcc13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1679e0 con 0x7fdfcc143f90 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc167860 con 0x7fdfcc0b4660 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc167b60 con 0x7fdfcc144690 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc143f90 0x7fdfcc164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfcc0b4660 0x7fdfcc136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc143f90 0x7fdfcc164eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51916/0 (socket says 172.21.15.97:51916) 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 -- 172.21.15.97:0/2218417403 learned_addr learned my addr 172.21.15.97:0/2218417403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd1400640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfcc144690 0x7fdfcc13d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 -- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfcc144690 msgr2=0x7fdfcc13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfcc144690 0x7fdfcc13d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 -- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfcc0b4660 msgr2=0x7fdfcc136970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfcc0b4660 0x7fdfcc136970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 -- 172.21.15.97:0/2218417403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc13c520 con 0x7fdfcc143f90 2024-09-10T21:33:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfcbe00640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc143f90 0x7fdfcc164eb0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fdfb801aed0 tx=0x7fdfb80a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfc8c00640 1 -- 172.21.15.97:0/2218417403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80be070 con 0x7fdfcc143f90 2024-09-10T21:33:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfc8c00640 1 -- 172.21.15.97:0/2218417403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfb8004470 con 0x7fdfcc143f90 2024-09-10T21:33:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc13c800 con 0x7fdfcc143f90 2024-09-10T21:33:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfc8c00640 1 -- 172.21.15.97:0/2218417403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80052d0 con 0x7fdfcc143f90 2024-09-10T21:33:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.915+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc13cdc0 con 0x7fdfcc143f90 2024-09-10T21:33:43.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfc8c00640 1 -- 172.21.15.97:0/2218417403 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdfb8005470 con 0x7fdfcc143f90 2024-09-10T21:33:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfc8c00640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfac0725e0 0x7fdfac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 Processor -- start 2024-09-10T21:33:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfc8c00640 1 -- 172.21.15.97:0/2218417403 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7fdfb812e5e0 con 0x7fdfcc143f90 2024-09-10T21:33:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 -- start start 2024-09-10T21:33:43.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfac0725e0 0x7fdfac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfac0725e0 0x7fdfac074aa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00bc00 tx=0x7fdfbc00b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341e99d0 0x55ba341e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341ea2f0 0x55ba3419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ba340be3f0 con 0x55ba341e99d0 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ba33ffbb20 con 0x55ba3419de10 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ba33feaf50 con 0x55ba341ea2f0 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341e99d0 0x55ba341e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341e99d0 0x55ba341e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51920/0 (socket says 172.21.15.97:51920) 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd0a00640 1 -- 172.21.15.97:0/1835104428 learned_addr learned my addr 172.21.15.97:0/1835104428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfcbe00640 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341ea2f0 0x55ba3419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.919+0000 7fdfd1400640 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd0a00640 1 -- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341ea2f0 msgr2=0x55ba3419d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341ea2f0 0x55ba3419d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd0a00640 1 -- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 msgr2=0x55ba3419e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3419e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd0a00640 1 -- 172.21.15.97:0/1835104428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba33fb2590 con 0x55ba341e99d0 2024-09-10T21:33:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341e99d0 0x55ba341e9db0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc005d10 tx=0x7fdfbc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4929502e49fa08f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfc1600640 1 -- 172.21.15.97:0/1835104428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc01e620 con 0x55ba341e99d0 2024-09-10T21:33:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfc1600640 1 -- 172.21.15.97:0/1835104428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfbc01e7c0 con 0x55ba341e99d0 2024-09-10T21:33:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfc1600640 1 -- 172.21.15.97:0/1835104428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc01ebb0 con 0x55ba341e99d0 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- 172.21.15.97:0/1835104428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341e99d0 msgr2=0x55ba341e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341e99d0 0x55ba341e9db0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc005d10 tx=0x7fdfbc013000 comp rx=0 tx=0).stop 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- 172.21.15.97:0/1835104428 shutdown_connections 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3419e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341ea2f0 0x55ba3419d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 --2- 172.21.15.97:0/1835104428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341e99d0 0x55ba341e9db0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- 172.21.15.97:0/1835104428 >> 172.21.15.97:0/1835104428 conn(0x55ba34087130 msgr2=0x55ba341cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- 172.21.15.97:0/1835104428 shutdown_connections 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- 172.21.15.97:0/1835104428 wait complete. 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- start start 2024-09-10T21:33:43.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 Processor -- start 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba3419de10 0x55ba341ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341e99d0 0x55ba341cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba341cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ba33f855d0 con 0x55ba341ea2f0 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ba33feed60 con 0x55ba341e99d0 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ba33f91500 con 0x55ba3419de10 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd6841000 10 client.0 _ll_register_callbacks cb 0x55ba33f95810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba341cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba341cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51934/0 (socket says 172.21.15.97:51934) 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 -- 172.21.15.97:0/2151773558 learned_addr learned my addr 172.21.15.97:0/2151773558 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 -- 172.21.15.97:0/2151773558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba3419de10 msgr2=0x55ba341ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba3419de10 0x55ba341ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 -- 172.21.15.97:0/2151773558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341e99d0 msgr2=0x55ba341cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341e99d0 0x55ba341cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.923+0000 7fdfd1400640 1 -- 172.21.15.97:0/2151773558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba33f89170 con 0x55ba341ea2f0 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd1400640 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba341cb3b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4011520 tx=0x7fdfc40119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b61bd430e1bb4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 1 -- 172.21.15.97:0/2151773558 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003b90 con 0x55ba341ea2f0 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 1 -- 172.21.15.97:0/2151773558 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7fdfc4003d30 con 0x55ba341ea2f0 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba33f830f0 con 0x55ba341ea2f0 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 1 -- 172.21.15.97:0/2151773558 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=245) ==== 24+0+0 (secure 0 0 0) 0x7fdfc4023050 con 0x55ba341ea2f0 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 10 client.9444 fetch_fsmap learned FSMap version 245 2024-09-10T21:33:43.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=245,monmap=2+}) -- 0x55ba3406fef0 con 0x55ba341ea2f0 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 1 -- 172.21.15.97:0/2151773558 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ba33f830f0 con 0x55ba341ea2f0 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 1 -- 172.21.15.97:0/2151773558 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 245) ==== 3502+0+0 (secure 0 0 0) 0x7fdfc40051c0 con 0x55ba341ea2f0 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 10 client.9444 fetch_fsmap finished waiting for FSMap version 245 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 10 client.9444 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 20 client.9444 populate_metadata read hostname 'smithi097' 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341cc0f0 0x55ba341fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 4 client.9444 mds_command: new command op to 9375 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 --> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba33ffda70 con 0x55ba341cc0f0 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341cc0f0 0x55ba341fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341cc0f0 0x55ba341fbc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7d30ae0a1a746e server_cookie=96b314f2b37ecd5e in_seq=0 out_seq=0 2024-09-10T21:33:43.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 10 client.9444 ms_handle_connect on v2:172.21.15.97:6833/2264649568 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 1 -- 172.21.15.97:0/2151773558 <== mds.0 v2:172.21.15.97:6833/2264649568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba33ffda70 con 0x55ba341cc0f0 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 10 client.9444 handle_command_reply: tid=0 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 client.9444 shutdown 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 msgr2=0x55ba341cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba341cb3b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4011520 tx=0x7fdfc40119e0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 shutdown_connections 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341cc0f0 0x55ba341fbc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba341cb3b0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341e99d0 0x55ba341cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2151773558 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba3419de10 0x55ba341ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfb6000640 0 client.9444 ms_handle_reset on v2:172.21.15.97:6833/2264649568 2024-09-10T21:33:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 >> 172.21.15.97:0/2151773558 conn(0x55ba34087130 msgr2=0x55ba3419cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 shutdown_connections 2024-09-10T21:33:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 1 -- 172.21.15.97:0/2151773558 wait complete. 2024-09-10T21:33:43.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.927+0000 7fdfd6841000 20 client.9444 trim_cache size 0 max 16384 2024-09-10T21:33:43.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 Processor -- start 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 -- start start 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba3419de10 0x55ba3425bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341ea2f0 0x55ba3425ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc2059a0 con 0x55ba3419de10 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc2049b0 con 0x55ba341ea2f0 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc1fc0d0 con 0x55ba341e99d0 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba3419de10 0x55ba3425bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba3419de10 0x55ba3425bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51942/0 (socket says 172.21.15.97:51942) 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 -- 172.21.15.97:0/2827220319 learned_addr learned my addr 172.21.15.97:0/2827220319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd1400640 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341ea2f0 0x55ba3425ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfcbe00640 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 -- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 msgr2=0x55ba3425c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 -- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341ea2f0 msgr2=0x55ba3425ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341ea2f0 0x55ba3425ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 -- 172.21.15.97:0/2827220319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba34046d90 con 0x55ba3419de10 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd1400640 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341ea2f0 0x55ba3425ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba3419de10 0x55ba3425bfe0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc022000 tx=0x7fdfbc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acb8ddf0871d540 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.947+0000 7fdfc0c00640 1 -- 172.21.15.97:0/2827220319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc035040 con 0x55ba3419de10 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfc0c00640 1 -- 172.21.15.97:0/2827220319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfbc01e9f0 con 0x55ba3419de10 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfc0c00640 1 -- 172.21.15.97:0/2827220319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc01ee20 con 0x55ba3419de10 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- 172.21.15.97:0/2827220319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba3419de10 msgr2=0x55ba3425bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba3419de10 0x55ba3425bfe0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc022000 tx=0x7fdfbc0046d0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- 172.21.15.97:0/2827220319 shutdown_connections 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba341ea2f0 0x55ba3425ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 --2- 172.21.15.97:0/2827220319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba3419de10 0x55ba3425bfe0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- 172.21.15.97:0/2827220319 >> 172.21.15.97:0/2827220319 conn(0x55ba34087130 msgr2=0x55ba341e85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- 172.21.15.97:0/2827220319 shutdown_connections 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- 172.21.15.97:0/2827220319 wait complete. 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- start start 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 Processor -- start 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3425f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba34260260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ba340aa730 con 0x55ba341ea2f0 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ba33ff3390 con 0x55ba3419de10 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ba3406fa60 con 0x55ba341e99d0 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 10 client.0 _ll_register_callbacks cb 0x7fdfc40021a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba34260260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba34260260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51954/0 (socket says 172.21.15.97:51954) 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 -- 172.21.15.97:0/3790528476 learned_addr learned my addr 172.21.15.97:0/3790528476 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfcbe00640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3425f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 -- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 msgr2=0x55ba3425fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 -- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 msgr2=0x55ba3425f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3425f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 -- 172.21.15.97:0/3790528476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba33fb1a90 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfcbe00640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3425f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd1400640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba34260260 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4001ed0 tx=0x7fdfc4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ad2ed051054110 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfb6000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfb6000640 1 -- 172.21.15.97:0/3790528476 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003d90 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfb6000640 1 -- 172.21.15.97:0/3790528476 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7fdfc4015470 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfbc01e9f0 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfb6000640 1 -- 172.21.15.97:0/3790528476 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=245) ==== 24+0+0 (secure 0 0 0) 0x7fdfc40156b0 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 10 client.9450 fetch_fsmap learned FSMap version 245 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.951+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=245,monmap=2+}) -- 0x55ba33f90050 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfb6000640 1 -- 172.21.15.97:0/3790528476 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfbc01e9f0 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfb6000640 1 -- 172.21.15.97:0/3790528476 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 245) ==== 3502+0+0 (secure 0 0 0) 0x7fdfc4015850 con 0x55ba341ea2f0 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 10 client.9450 fetch_fsmap finished waiting for FSMap version 245 2024-09-10T21:33:43.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 10 client.9450 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:33:43.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 20 client.9450 populate_metadata read hostname 'smithi097' 2024-09-10T21:33:43.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341fed40 0x55ba341fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:43.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 4 client.9450 mds_command: new command op to 9375 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:33:43.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 --> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] -- command(tid 0: {"prefix": "status"}) -- 0x55ba33ff9520 con 0x55ba341fed40 2024-09-10T21:33:43.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341fed40 0x55ba341fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:43.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfb6000640 10 client.9450 ms_handle_connect on v2:172.21.15.97:6833/2264649568 2024-09-10T21:33:43.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd0a00640 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341fed40 0x55ba341fde60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c32fc3f5e08770 server_cookie=58650668bf7cf69f in_seq=0 out_seq=0 2024-09-10T21:33:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfb6000640 1 -- 172.21.15.97:0/3790528476 <== mds.0 v2:172.21.15.97:6833/2264649568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ba33ff9520 con 0x55ba341fed40 2024-09-10T21:33:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfb6000640 10 client.9450 handle_command_reply: tid=0 2024-09-10T21:33:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 client.9450 shutdown 2024-09-10T21:33:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 msgr2=0x55ba34260260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba34260260 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4001ed0 tx=0x7fdfc4007e40 comp rx=0 tx=0).stop 2024-09-10T21:33:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 shutdown_connections 2024-09-10T21:33:43.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:6833/2264649568,v1:172.21.15.97:6835/2264649568] conn(0x55ba341fed40 0x55ba341fde60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba341ea2f0 0x55ba34260260 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ba341e99d0 0x55ba3425fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfb6000640 0 client.9450 ms_handle_reset on v2:172.21.15.97:6833/2264649568 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 --2- 172.21.15.97:0/3790528476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ba3419de10 0x55ba3425f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 >> 172.21.15.97:0/3790528476 conn(0x55ba34087130 msgr2=0x55ba341f6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 shutdown_connections 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 1 -- 172.21.15.97:0/3790528476 wait complete. 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd6841000 20 client.9450 trim_cache size 0 max 16384 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:33:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9375, 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7938067249999996, 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 245, 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 122, 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 122, 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.076534265999999 2024-09-10T21:33:43.961 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:33:43.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfac0725e0 msgr2=0x7fdfac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfac0725e0 0x7fdfac074aa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00bc00 tx=0x7fdfbc00b390 comp rx=0 tx=0).stop 2024-09-10T21:33:43.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc143f90 msgr2=0x7fdfcc164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:43.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc143f90 0x7fdfcc164eb0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fdfb801aed0 tx=0x7fdfb80a66f0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 shutdown_connections 2024-09-10T21:33:43.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfac0725e0 0x7fdfac074aa0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfcc144690 0x7fdfcc13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc143f90 0x7fdfcc164eb0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 --2- 172.21.15.97:0/2218417403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfcc0b4660 0x7fdfcc136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:43.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 >> 172.21.15.97:0/2218417403 conn(0x7fdfcc0b1780 msgr2=0x7fdfcc0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:43.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 shutdown_connections 2024-09-10T21:33:43.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:43.955+0000 7fdfd3400640 1 -- 172.21.15.97:0/2218417403 wait complete. 2024-09-10T21:33:43.973 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:33:43.973 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2024-09-10T21:33:43.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:33:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.115+0000 7f2c95400640 1 -- 172.21.15.97:0/203840675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c900cd580 msgr2=0x7f2c900cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.115+0000 7f2c95400640 1 --2- 172.21.15.97:0/203840675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c900cd580 0x7f2c900cda10 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009bc0 tx=0x7f2c8801cb10 comp rx=0 tx=0).stop 2024-09-10T21:33:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.115+0000 7f2c95400640 1 -- 172.21.15.97:0/203840675 shutdown_connections 2024-09-10T21:33:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.115+0000 7f2c95400640 1 --2- 172.21.15.97:0/203840675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c900cd580 0x7f2c900cda10 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.115+0000 7f2c95400640 1 -- 172.21.15.97:0/203840675 >> 172.21.15.97:0/203840675 conn(0x7f2c900b28c0 msgr2=0x7f2c900b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.115+0000 7f2c95400640 1 -- 172.21.15.97:0/203840675 shutdown_connections 2024-09-10T21:33:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.115+0000 7f2c95400640 1 -- 172.21.15.97:0/203840675 wait complete. 2024-09-10T21:33:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 Processor -- start 2024-09-10T21:33:44.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 -- start start 2024-09-10T21:33:44.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c900cd580 0x7f2c9015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c9015e750 0x7f2c90141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9015f150 0x7f2c90152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901692c0 con 0x7f2c9015f150 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90169140 con 0x7f2c900cd580 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2c90169440 con 0x7f2c9015e750 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9015f150 0x7f2c90152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9015f150 0x7f2c90152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51974/0 (socket says 172.21.15.97:51974) 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 -- 172.21.15.97:0/807875032 learned_addr learned my addr 172.21.15.97:0/807875032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c85e00640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c9015e750 0x7f2c90141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 -- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c9015e750 msgr2=0x7f2c90141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c9015e750 0x7f2c90141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8e800640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c900cd580 0x7f2c9015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 -- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c900cd580 msgr2=0x7f2c9015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c900cd580 0x7f2c9015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 -- 172.21.15.97:0/807875032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c90153d00 con 0x7f2c9015f150 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c85e00640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c9015e750 0x7f2c90141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8e800640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c900cd580 0x7f2c9015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c8f200640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9015f150 0x7f2c90152710 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2c88019720 tx=0x7f2c880a5960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c87400640 1 -- 172.21.15.97:0/807875032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880bb070 con 0x7f2c9015f150 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c87400640 1 -- 172.21.15.97:0/807875032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c88004e00 con 0x7f2c9015f150 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c90153fe0 con 0x7f2c9015f150 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c87400640 1 -- 172.21.15.97:0/807875032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880b2040 con 0x7f2c9015f150 2024-09-10T21:33:44.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9015cdf0 con 0x7f2c9015f150 2024-09-10T21:33:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.119+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c900cda10 con 0x7f2c9015f150 2024-09-10T21:33:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.123+0000 7f2c87400640 1 -- 172.21.15.97:0/807875032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2c880b5030 con 0x7f2c9015f150 2024-09-10T21:33:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.123+0000 7f2c87400640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2c6c0725e0 0x7f2c6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.123+0000 7f2c87400640 1 -- 172.21.15.97:0/807875032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f2c8812dd00 con 0x7f2c9015f150 2024-09-10T21:33:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.123+0000 7f2c8e800640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2c6c0725e0 0x7f2c6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.123+0000 7f2c8e800640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2c6c0725e0 0x7f2c6c074aa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2c80001630 tx=0x7f2c80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.127+0000 7f2c87400640 1 -- 172.21.15.97:0/807875032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2c88016790 con 0x7f2c9015f150 2024-09-10T21:33:44.135 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:33:44.135+0000 7efd31c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:33:44.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.327+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2c900ce7b0 con 0x7f2c9015f150 2024-09-10T21:33:44.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.327+0000 7f2c87400640 1 -- 172.21.15.97:0/807875032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v245) ==== 53+0+73 (secure 0 0 0) 0x7f2c880f8930 con 0x7f2c9015f150 2024-09-10T21:33:44.328 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2c6c0725e0 msgr2=0x7f2c6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2c6c0725e0 0x7f2c6c074aa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2c80001630 tx=0x7f2c80009040 comp rx=0 tx=0).stop 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9015f150 msgr2=0x7f2c90152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9015f150 0x7f2c90152710 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2c88019720 tx=0x7f2c880a5960 comp rx=0 tx=0).stop 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 shutdown_connections 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2c6c0725e0 0x7f2c6c074aa0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9015f150 0x7f2c90152710 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2c9015e750 0x7f2c90141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:44.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 --2- 172.21.15.97:0/807875032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c900cd580 0x7f2c9015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:44.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 >> 172.21.15.97:0/807875032 conn(0x7f2c900b28c0 msgr2=0x7f2c9015b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:44.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 shutdown_connections 2024-09-10T21:33:44.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44.331+0000 7f2c95400640 1 -- 172.21.15.97:0/807875032 wait complete. 2024-09-10T21:33:44.349 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:33:44.350 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:33:44.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:44.350 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:33:44.360 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:33:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:33:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:33:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:33:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41869sec preferred_lft 41869sec 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.362 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:33:44.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:44.363 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:33:44.421 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:33:44.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:44.424 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:33:44.483 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:33:44.483 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:33:44.483 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:33:44.484 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:33:44.484 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:33:44.484 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:33:44.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:33:44.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:33:44.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:33:44.485 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:33:44.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:33:44.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:33:44.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:44.624 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:33:44.680 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:33:44.680 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:33:44.681 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:33:44.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:44.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:33:44.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:33:44.743 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:44.795 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:33:44.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:33:44.796 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:44.847 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:33:44.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:44.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:44.935 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:44.935 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:33:44.936 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:33:44.950 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:44.950 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:33:44.950 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:33:44.951 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:33:44.951 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:33:44.960 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:33:44.963 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:33:44.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:44.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:33:44.983 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:33:45.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:33:45.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:33:45.079 INFO:teuthology.orchestra.run.smithi097.stdout:client7799 2024-09-10T21:33:45.079 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:45.079 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:45.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:45.132 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:33:45.178 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:33:45.186 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:33:45.186 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:33:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:45.187 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:33:45.187 INFO:teuthology.orchestra.run.smithi097.stdout:sixmegs 2024-09-10T21:33:45.187 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2024-09-10T21:33:45.187 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2024-09-10T21:33:45.188 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:45.188 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:45.240 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:45.240 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:45.296 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:33:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 -- 172.21.15.97:0/4033860212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe04c004d50 msgr2=0x7fe04c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 --2- 172.21.15.97:0/4033860212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe06000bc10 tx=0x7fe06001ce80 comp rx=0 tx=0).stop 2024-09-10T21:33:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe067400640 1 -- 172.21.15.97:0/4033860212 reap_dead start 2024-09-10T21:33:45.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 -- 172.21.15.97:0/4033860212 shutdown_connections 2024-09-10T21:33:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 -- 172.21.15.97:0/4033860212 >> 172.21.15.97:0/4033860212 conn(0x7fe0680b1780 msgr2=0x7fe0680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 -- 172.21.15.97:0/4033860212 shutdown_connections 2024-09-10T21:33:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 -- 172.21.15.97:0/4033860212 wait complete. 2024-09-10T21:33:45.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 Processor -- start 2024-09-10T21:33:45.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.487+0000 7fe06dc00640 1 -- start start 2024-09-10T21:33:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c004d50 0x7fe06815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0680702c0 0x7fe068154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe068073680 0x7fe068073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe06813fef0 con 0x7fe068073680 2024-09-10T21:33:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe06815f960 con 0x7fe04c004d50 2024-09-10T21:33:45.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe068140070 con 0x7fe0680702c0 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe068073680 0x7fe068073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe068073680 0x7fe068073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51992/0 (socket says 172.21.15.97:51992) 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 -- 172.21.15.97:0/1908571334 learned_addr learned my addr 172.21.15.97:0/1908571334 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe066a00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c004d50 0x7fe06815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe066000640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0680702c0 0x7fe068154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 -- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0680702c0 msgr2=0x7fe068154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0680702c0 0x7fe068154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 -- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c004d50 msgr2=0x7fe06815e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c004d50 0x7fe06815e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 -- 172.21.15.97:0/1908571334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06815edb0 con 0x7fe068073680 2024-09-10T21:33:45.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe066a00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c004d50 0x7fe06815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe066000640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0680702c0 0x7fe068154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe067400640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe068073680 0x7fe068073b30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fe06000bbb0 tx=0x7fe060005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe05ea00640 1 -- 172.21.15.97:0/1908571334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600be070 con 0x7fe068073680 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe05ea00640 1 -- 172.21.15.97:0/1908571334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe060004bd0 con 0x7fe068073680 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe05ea00640 1 -- 172.21.15.97:0/1908571334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600ab050 con 0x7fe068073680 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06813b310 con 0x7fe068073680 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0681f8510 con 0x7fe068073680 2024-09-10T21:33:45.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.491+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0680cf100 con 0x7fe068073680 2024-09-10T21:33:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.495+0000 7fe05ea00640 1 -- 172.21.15.97:0/1908571334 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe0600af400 con 0x7fe068073680 2024-09-10T21:33:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.495+0000 7fe05ea00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0480725e0 0x7fe048074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.495+0000 7fe05ea00640 1 -- 172.21.15.97:0/1908571334 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7fe06012e4f0 con 0x7fe068073680 2024-09-10T21:33:45.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.495+0000 7fe066a00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0480725e0 0x7fe048074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.495+0000 7fe066a00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0480725e0 0x7fe048074aa0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe058001340 tx=0x7fe058002060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:45.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.499+0000 7fe05ea00640 1 -- 172.21.15.97:0/1908571334 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe06815edb0 con 0x7fe068073680 2024-09-10T21:33:45.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.675+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe068141360 con 0x7fe068073680 2024-09-10T21:33:45.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.675+0000 7fe05ea00640 1 -- 172.21.15.97:0/1908571334 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 52 entries v122) ==== 81+0+3156 (secure 0 0 0) 0x7fe060016b90 con 0x7fe068073680 2024-09-10T21:33:45.674 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:33:45.674 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:33:45.674 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:33:45.674 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:33:45.674 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:33:45.674 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:33:45.675 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:33:45.676 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:33:45.677 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:33:45.678 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:33:45.679 INFO:teuthology.orchestra.run.smithi097.stderr:listed 52 entries 2024-09-10T21:33:45.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0480725e0 msgr2=0x7fe048074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0480725e0 0x7fe048074aa0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe058001340 tx=0x7fe058002060 comp rx=0 tx=0).stop 2024-09-10T21:33:45.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe068073680 msgr2=0x7fe068073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe068073680 0x7fe068073b30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fe06000bbb0 tx=0x7fe060005540 comp rx=0 tx=0).stop 2024-09-10T21:33:45.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 shutdown_connections 2024-09-10T21:33:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe0480725e0 0x7fe048074aa0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe068073680 0x7fe068073b30 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe0680702c0 0x7fe068154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 --2- 172.21.15.97:0/1908571334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c004d50 0x7fe06815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 >> 172.21.15.97:0/1908571334 conn(0x7fe0680b1780 msgr2=0x7fe06815a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 shutdown_connections 2024-09-10T21:33:45.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.679+0000 7fe06dc00640 1 -- 172.21.15.97:0/1908571334 wait complete. 2024-09-10T21:33:45.695 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:33:45.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:45.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:33:45.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:45.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:45.719 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:45.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 -- 172.21.15.97:0/109252658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0b3cf0 msgr2=0x7f604c0ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 --2- 172.21.15.97:0/109252658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0b3cf0 0x7f604c0ceee0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6038009900 tx=0x7f603801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 -- 172.21.15.97:0/109252658 shutdown_connections 2024-09-10T21:33:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 --2- 172.21.15.97:0/109252658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0b3cf0 0x7f604c0ceee0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 --2- 172.21.15.97:0/109252658 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f604c1511f0 0x7f604c0b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 -- 172.21.15.97:0/109252658 >> 172.21.15.97:0/109252658 conn(0x7f604c0b1780 msgr2=0x7f604c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 -- 172.21.15.97:0/109252658 shutdown_connections 2024-09-10T21:33:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 -- 172.21.15.97:0/109252658 wait complete. 2024-09-10T21:33:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 Processor -- start 2024-09-10T21:33:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.959+0000 7f6054200640 1 -- start start 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f604c0b3cf0 0x7f604c06f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0cf420 0x7f604c14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f604c1511f0 0x7f604c15f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f604c06e530 con 0x7f604c0cf420 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f604c06e3b0 con 0x7f604c0b3cf0 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f604c06e6b0 con 0x7f604c1511f0 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0cf420 0x7f604c14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6051800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f604c0b3cf0 0x7f604c06f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0cf420 0x7f604c14dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51998/0 (socket says 172.21.15.97:51998) 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 -- 172.21.15.97:0/3349036225 learned_addr learned my addr 172.21.15.97:0/3349036225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6052200640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f604c1511f0 0x7f604c15f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 -- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f604c1511f0 msgr2=0x7f604c15f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f604c1511f0 0x7f604c15f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 -- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f604c0b3cf0 msgr2=0x7f604c06f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f604c0b3cf0 0x7f604c06f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 -- 172.21.15.97:0/3349036225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1f5510 con 0x7f604c0cf420 2024-09-10T21:33:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6050e00640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0cf420 0x7f604c14dd70 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6038009850 tx=0x7f6038005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6049600640 1 -- 172.21.15.97:0/3349036225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60380be070 con 0x7f604c0cf420 2024-09-10T21:33:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6049600640 1 -- 172.21.15.97:0/3349036225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60380053e0 con 0x7f604c0cf420 2024-09-10T21:33:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6049600640 1 -- 172.21.15.97:0/3349036225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60380b5040 con 0x7f604c0cf420 2024-09-10T21:33:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c0c5480 con 0x7f604c0cf420 2024-09-10T21:33:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c0c5720 con 0x7f604c0cf420 2024-09-10T21:33:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604c0ceee0 con 0x7f604c0cf420 2024-09-10T21:33:45.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.963+0000 7f6049600640 1 -- 172.21.15.97:0/3349036225 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f60380b8030 con 0x7f604c0cf420 2024-09-10T21:33:45.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.967+0000 7f6049600640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c072600 0x7f602c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.967+0000 7f6049600640 1 -- 172.21.15.97:0/3349036225 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8511+0+0 (secure 0 0 0) 0x7f6038130990 con 0x7f604c0cf420 2024-09-10T21:33:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.967+0000 7f6051800640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c072600 0x7f602c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.967+0000 7f6051800640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c072600 0x7f602c074ac0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f604c159ad0 tx=0x7f60400091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:45.971+0000 7f6049600640 1 -- 172.21.15.97:0/3349036225 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6038018c30 con 0x7f604c0cf420 2024-09-10T21:33:46.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.167+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f604c15ab40 con 0x7f604c0cf420 2024-09-10T21:33:46.165 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:46.167+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.167+0000 7f6049600640 1 -- 172.21.15.97:0/3349036225 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v246) ==== 161+0+0 (secure 0 0 0) 0x7f60380faf10 con 0x7f604c0cf420 2024-09-10T21:33:46.167 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c072600 msgr2=0x7f602c074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c072600 0x7f602c074ac0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f604c159ad0 tx=0x7f60400091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0cf420 msgr2=0x7f604c14dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0cf420 0x7f604c14dd70 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6038009850 tx=0x7f6038005e70 comp rx=0 tx=0).stop 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 shutdown_connections 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f602c072600 0x7f602c074ac0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f604c1511f0 0x7f604c15f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604c0cf420 0x7f604c14dd70 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 --2- 172.21.15.97:0/3349036225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f604c0b3cf0 0x7f604c06f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 >> 172.21.15.97:0/3349036225 conn(0x7f604c0b1780 msgr2=0x7f604c15cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:46.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 shutdown_connections 2024-09-10T21:33:46.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.171+0000 7f6054200640 1 -- 172.21.15.97:0/3349036225 wait complete. 2024-09-10T21:33:46.175 INFO:tasks.ceph.mds.c.smithi097.stderr: -553> 2024-09-10T21:33:44.135+0000 7efd31c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:33:46.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:46.216 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:33:46.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 -- 172.21.15.97:0/2550416642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 msgr2=0x7f24d8141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:46.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 --2- 172.21.15.97:0/2550416642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d8141d20 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f24d8141580 tx=0x7f24c801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 -- 172.21.15.97:0/2550416642 shutdown_connections 2024-09-10T21:33:46.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 --2- 172.21.15.97:0/2550416642 >> v2:172.21.15.194:3300/0 conn(0x7f24d815fde0 0x7f24d813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 --2- 172.21.15.97:0/2550416642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d8141d20 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 -- 172.21.15.97:0/2550416642 >> 172.21.15.97:0/2550416642 conn(0x7f24d80b1780 msgr2=0x7f24d80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:46.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 -- 172.21.15.97:0/2550416642 shutdown_connections 2024-09-10T21:33:46.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 -- 172.21.15.97:0/2550416642 wait complete. 2024-09-10T21:33:46.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 Processor -- start 2024-09-10T21:33:46.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 -- start start 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.343+0000 7f24dfc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d80b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce9d0 0x7f24d815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d815fde0 0x7f24d815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8168a70 con 0x7f24d80ce5f0 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8140cd0 con 0x7f24d815fde0 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f24d8168bf0 con 0x7f24d80ce9d0 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d80b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:46.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d80b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52020/0 (socket says 172.21.15.97:52020) 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 -- 172.21.15.97:0/321306655 learned_addr learned my addr 172.21.15.97:0/321306655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24ddc00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d815fde0 0x7f24d815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24cfe00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce9d0 0x7f24d815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 -- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce9d0 msgr2=0x7f24d815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce9d0 0x7f24d815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 -- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d815fde0 msgr2=0x7f24d815c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d815fde0 0x7f24d815c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 -- 172.21.15.97:0/321306655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d8142090 con 0x7f24d80ce5f0 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24cfe00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce9d0 0x7f24d815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:46.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dd200640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d80b42f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f24c80b0040 tx=0x7f24c80a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:46.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24d5600640 1 -- 172.21.15.97:0/321306655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80bb070 con 0x7f24d80ce5f0 2024-09-10T21:33:46.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24d5600640 1 -- 172.21.15.97:0/321306655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c8005070 con 0x7f24d80ce5f0 2024-09-10T21:33:46.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24d5600640 1 -- 172.21.15.97:0/321306655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80b2040 con 0x7f24d80ce5f0 2024-09-10T21:33:46.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d81f78c0 con 0x7f24d80ce5f0 2024-09-10T21:33:46.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81f8100 con 0x7f24d80ce5f0 2024-09-10T21:33:46.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d8141d20 con 0x7f24d80ce5f0 2024-09-10T21:33:46.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.347+0000 7f24d5600640 1 -- 172.21.15.97:0/321306655 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f24c80054c0 con 0x7f24d80ce5f0 2024-09-10T21:33:46.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.351+0000 7f24d5600640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b40725e0 0x7f24b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:46.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.351+0000 7f24d5600640 1 -- 172.21.15.97:0/321306655 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8597+0+0 (secure 0 0 0) 0x7f24c812d490 con 0x7f24d80ce5f0 2024-09-10T21:33:46.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.351+0000 7f24cfe00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b40725e0 0x7f24b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:46.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.351+0000 7f24cfe00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b40725e0 0x7f24b4074aa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f24c00016d0 tx=0x7f24c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:46.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.351+0000 7f24d5600640 1 -- 172.21.15.97:0/321306655 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24d81f78c0 con 0x7f24d80ce5f0 2024-09-10T21:33:46.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:46.579+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f24d815a990 con 0x7f24d80ce5f0 2024-09-10T21:33:47.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.175+0000 7f24d5600640 1 -- 172.21.15.97:0/321306655 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v247) ==== 107+0+0 (secure 0 0 0) 0x7f24c80164b0 con 0x7f24d80ce5f0 2024-09-10T21:33:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:33:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.175+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b40725e0 msgr2=0x7f24b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.175+0000 7f24dfc00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b40725e0 0x7f24b4074aa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f24c00016d0 tx=0x7f24c0009040 comp rx=0 tx=0).stop 2024-09-10T21:33:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 msgr2=0x7f24d80b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d80b42f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f24c80b0040 tx=0x7f24c80a5760 comp rx=0 tx=0).stop 2024-09-10T21:33:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 shutdown_connections 2024-09-10T21:33:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f24b40725e0 0x7f24b4074aa0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d815fde0 0x7f24d815c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f24d80ce9d0 0x7f24d815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 --2- 172.21.15.97:0/321306655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d80ce5f0 0x7f24d80b42f0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 >> 172.21.15.97:0/321306655 conn(0x7f24d80b1780 msgr2=0x7f24d80b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 shutdown_connections 2024-09-10T21:33:47.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.179+0000 7f24dfc00640 1 -- 172.21.15.97:0/321306655 wait complete. 2024-09-10T21:33:47.197 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 Processor -- start 2024-09-10T21:33:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- start start 2024-09-10T21:33:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b059b4aa0 0x563b05a8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6dca0 0x563b05a86b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b05a6e080 0x563b05a870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b05902a00 con 0x563b05a6dca0 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b05960f40 con 0x563b059b4aa0 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563b059e3da0 con 0x563b05a6e080 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6dca0 0x563b05a86b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6dca0 0x563b05a86b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52032/0 (socket says 172.21.15.97:52032) 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 -- 172.21.15.97:0/309378921 learned_addr learned my addr 172.21.15.97:0/309378921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe445200640 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b059b4aa0 0x563b05a8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe445c00640 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b05a6e080 0x563b05a870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 -- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b05a6e080 msgr2=0x563b05a870b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b05a6e080 0x563b05a870b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 -- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b059b4aa0 msgr2=0x563b05a8c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b059b4aa0 0x563b05a8c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 -- 172.21.15.97:0/309378921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b058dbc10 con 0x563b05a6dca0 2024-09-10T21:33:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43fe00640 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6dca0 0x563b05a86b70 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fe430004f60 tx=0x7fe430017fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6558c37db80fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43ea00640 1 -- 172.21.15.97:0/309378921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300b0040 con 0x563b05a6dca0 2024-09-10T21:33:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43ea00640 1 -- 172.21.15.97:0/309378921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4300acb20 con 0x563b05a6dca0 2024-09-10T21:33:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe43ea00640 1 -- 172.21.15.97:0/309378921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300b5410 con 0x563b05a6dca0 2024-09-10T21:33:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- 172.21.15.97:0/309378921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6dca0 msgr2=0x563b05a86b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6dca0 0x563b05a86b70 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fe430004f60 tx=0x7fe430017fb0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- 172.21.15.97:0/309378921 shutdown_connections 2024-09-10T21:33:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b05a6e080 0x563b05a870b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6dca0 0x563b05a86b70 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 --2- 172.21.15.97:0/309378921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b059b4aa0 0x563b05a8c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- 172.21.15.97:0/309378921 >> 172.21.15.97:0/309378921 conn(0x563b059b3140 msgr2=0x563b059b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- 172.21.15.97:0/309378921 shutdown_connections 2024-09-10T21:33:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.287+0000 7fe447f59080 1 -- 172.21.15.97:0/309378921 wait complete. 2024-09-10T21:33:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 Processor -- start 2024-09-10T21:33:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 -- start start 2024-09-10T21:33:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b059b4aa0 0x563b05a8f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b05a6dca0 0x563b05a87380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6e080 0x563b05a8e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b05a8b630 con 0x563b05a6e080 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b05a8b390 con 0x563b05a6dca0 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563b05a89920 con 0x563b059b4aa0 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6e080 0x563b05a8e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6e080 0x563b05a8e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52034/0 (socket says 172.21.15.97:52034) 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 learned_addr learned my addr 172.21.15.97:0/3134655131 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe43fe00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b05a6dca0 0x563b05a87380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445200640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b059b4aa0 0x563b05a8f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b059b4aa0 msgr2=0x563b05a8f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b059b4aa0 0x563b05a8f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b05a6dca0 msgr2=0x563b05a87380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b05a6dca0 0x563b05a87380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b05a8fc70 con 0x563b05a6e080 2024-09-10T21:33:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe445c00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6e080 0x563b05a8e730 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fe44000d3d0 tx=0x7fe44000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe43d600640 1 -- 172.21.15.97:0/3134655131 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440006020 con 0x563b05a6e080 2024-09-10T21:33:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe43d600640 1 -- 172.21.15.97:0/3134655131 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4400062b0 con 0x563b05a6e080 2024-09-10T21:33:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563b05a8ff10 con 0x563b05a6e080 2024-09-10T21:33:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe43d600640 1 -- 172.21.15.97:0/3134655131 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440010750 con 0x563b05a6e080 2024-09-10T21:33:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.291+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b05a8e290 con 0x563b05a6e080 2024-09-10T21:33:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe43d600640 1 -- 172.21.15.97:0/3134655131 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe4400108f0 con 0x563b05a6e080 2024-09-10T21:33:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe43d600640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe424070fc0 0x7fe424073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe43d600640 1 -- 172.21.15.97:0/3134655131 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8597+0+0 (secure 0 0 0) 0x7fe4400922a0 con 0x563b05a6e080 2024-09-10T21:33:47.295 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b05a8ec70 0x563b05b24210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05a914a0 con 0x563b05a8ec70 2024-09-10T21:33:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe445200640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe424070fc0 0x7fe424073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe43fe00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b05a8ec70 0x563b05b24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe445200640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe424070fc0 0x7fe424073480 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c650 tx=0x7fe43800e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.295+0000 7fe43fe00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b05a8ec70 0x563b05b24210 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.299+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v123'815 uv789 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.299+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05a91840 con 0x563b05a8ec70 2024-09-10T21:33:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.299+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v123'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.299+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05a91be0 con 0x563b05a8ec70 2024-09-10T21:33:47.301 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.303+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v123'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.303+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b31320 con 0x563b05a8ec70 2024-09-10T21:33:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.303+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.303+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b316c0 con 0x563b05a8ec70 2024-09-10T21:33:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.307+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.307+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30310 con 0x563b05a8ec70 2024-09-10T21:33:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.307+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'820 uv820 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.307+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.307+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v123'821 uv795 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.307+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.311+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v123'822 uv822 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.311+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.311 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.311+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v123'823 uv823 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.311+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.315+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'824 uv824 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.315+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.315+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'825 uv825 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.315+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.315+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'826 uv826 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.315+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.319+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v123'827 uv802 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.319+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.319+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'828 uv828 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.319+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.321 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.323+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v123'829 uv829 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.323+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.323+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'830 uv830 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.323+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.327+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'831 uv831 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.327+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b306b0 con 0x563b05a8ec70 2024-09-10T21:33:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.327+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'832 uv832 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.327+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30f20 con 0x563b05a8ec70 2024-09-10T21:33:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.327+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'833 uv833 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.327+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.331+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v123'834 uv808 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.331+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.331+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v123'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.331+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.331 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.331+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v123'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.331+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.335+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.335+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.335+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.335+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.339+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.339+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.339+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v123'840 uv814 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.339+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.339+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v123'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.339+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.343+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v123'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.341 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.343+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.343+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.343+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.343+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.343+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05a8ec70 2024-09-10T21:33:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.347+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'845 uv845 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05a8ec70 2024-09-10T21:33:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.347+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b05b31f70 0x563b05b34360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.347+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b30de0 con 0x563b05b31f70 2024-09-10T21:33:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.347+0000 7fe445200640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b05b31f70 0x563b05b34360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.347+0000 7fe445200640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b05b31f70 0x563b05b34360 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.351+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'596 uv575 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.351+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b348a0 con 0x563b05b31f70 2024-09-10T21:33:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.351+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'597 uv597 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.351+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b34c40 con 0x563b05b31f70 2024-09-10T21:33:47.351 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.351+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'598 uv598 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.351+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b35490 con 0x563b05b31f70 2024-09-10T21:33:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.355+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v123'599 uv599 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.355+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b35830 con 0x563b05b31f70 2024-09-10T21:33:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.355+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'600 uv600 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.355+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b35bd0 con 0x563b05b31f70 2024-09-10T21:33:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.359+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v123'601 uv594 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.359+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b35f70 con 0x563b05b31f70 2024-09-10T21:33:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.359+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v123'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.359+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.359 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.359+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v123'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.359+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.363+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'604 uv604 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.363+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.363+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.363+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.363+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v123'606 uv595 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.363+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.367+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v123'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.367+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.366 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.367+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v123'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.367+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3a4e0 con 0x563b05b31f70 2024-09-10T21:33:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.367+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v123'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.367+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3a4e0 con 0x563b05b31f70 2024-09-10T21:33:47.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.371+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'610 uv610 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.371+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.371+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v123'611 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.371+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.371+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v123'612 uv612 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.371+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.372 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:47.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.375+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v123'613 uv613 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.375+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b37320 con 0x563b05b31f70 2024-09-10T21:33:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.375+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'614 uv614 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.375+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b37320 con 0x563b05b31f70 2024-09-10T21:33:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.375+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'615 uv615 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.375+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b36310 con 0x563b05b31f70 2024-09-10T21:33:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.379+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'616 uv616 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b31f70 2024-09-10T21:33:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.379+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b05b36820 0x563b05b38c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.379+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b39290 con 0x563b05b36820 2024-09-10T21:33:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.379+0000 7fe43fe00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b05b36820 0x563b05b38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.379+0000 7fe43fe00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b05b36820 0x563b05b38c10 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.379+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.379+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b39350 con 0x563b05b36820 2024-09-10T21:33:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.383+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.383+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b39770 con 0x563b05b36820 2024-09-10T21:33:47.381 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.383+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.383+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3a780 con 0x563b05b36820 2024-09-10T21:33:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.383+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.383+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3a780 con 0x563b05b36820 2024-09-10T21:33:47.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b39770 con 0x563b05b36820 2024-09-10T21:33:47.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b05b39c80 0x563b05b3c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3c7c0 con 0x563b05b39c80 2024-09-10T21:33:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe445c00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b05b39c80 0x563b05b3c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe445c00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b05b39c80 0x563b05b3c0f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'120 uv119 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.387+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b3ccd0 con 0x563b05b39c80 2024-09-10T21:33:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.391+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'121 uv121 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.391+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3d190 con 0x563b05b39c80 2024-09-10T21:33:47.389 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.391+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'122 uv122 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.391+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3da80 con 0x563b05b39c80 2024-09-10T21:33:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.391+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'123 uv123 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.391+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3de20 con 0x563b05b36820 2024-09-10T21:33:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b3de20 con 0x563b05b36820 2024-09-10T21:33:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3de20 con 0x563b05b36820 2024-09-10T21:33:47.394 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3ee30 con 0x563b05b36820 2024-09-10T21:33:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.395+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3ee30 con 0x563b05b36820 2024-09-10T21:33:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.399+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.399+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3de20 con 0x563b05b36820 2024-09-10T21:33:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.399+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.399+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3de20 con 0x563b05b39c80 2024-09-10T21:33:47.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.399+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v123'294 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.399+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b3e1c0 con 0x563b05b39c80 2024-09-10T21:33:47.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.403+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v123'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.403+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e560 con 0x563b05b39c80 2024-09-10T21:33:47.401 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:47.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.403+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v123'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.403+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3f910 con 0x563b05b39c80 2024-09-10T21:33:47.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.403+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.403+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3f910 con 0x563b05b39c80 2024-09-10T21:33:47.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.407+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.407+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b39c80 2024-09-10T21:33:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.407+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.407+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b39c80 2024-09-10T21:33:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.407+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v123'300 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.407+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b39c80 2024-09-10T21:33:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.411+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v123'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.411+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b39c80 2024-09-10T21:33:47.410 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.411+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v123'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.411+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b39c80 2024-09-10T21:33:47.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.411+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.411+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b39c80 2024-09-10T21:33:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.415+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.415+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b36820 2024-09-10T21:33:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.415+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v123'132 uv131 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.415+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b36820 2024-09-10T21:33:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.415+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v123'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.415+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b36820 2024-09-10T21:33:47.416 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v123'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b36820 2024-09-10T21:33:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b3e900 con 0x563b05b36820 2024-09-10T21:33:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b36820 2024-09-10T21:33:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b05b3ef80 0x563b05b41370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41940 con 0x563b05b3ef80 2024-09-10T21:33:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.419+0000 7fe445200640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b05b3ef80 0x563b05b41370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.423+0000 7fe445200640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b05b3ef80 0x563b05b41370 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.423+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.423+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b41a00 con 0x563b05b3ef80 2024-09-10T21:33:47.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.423+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.423+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.424 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:47.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.423+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.427+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b42e30 con 0x563b05b3ef80 2024-09-10T21:33:47.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.427+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.427+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b42e30 con 0x563b05b3ef80 2024-09-10T21:33:47.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.427+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.427+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.427+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.427+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.431+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v123'263 uv257 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.431+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.431+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v123'264 uv264 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.431+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.431+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v123'265 uv265 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.433 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:47.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.431+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.435+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'266 uv266 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.435+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.435+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'267 uv267 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.435+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.435+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v123'268 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.435+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.439+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v123'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.439+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.437 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:47.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.439+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v123'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.439+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b41e20 con 0x563b05b3ef80 2024-09-10T21:33:47.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.439+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.439+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b44c30 con 0x563b05b3ef80 2024-09-10T21:33:47.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v123'116 uv114 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'117 uv117 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.443 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:47.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v123'118 uv118 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.443+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.447+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'119 uv119 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.447+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.447+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v123'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.447+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.447+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.447+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.448 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:47.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v123'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b422d0 con 0x563b05b39c80 2024-09-10T21:33:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe440058a30 con 0x563b05b39c80 2024-09-10T21:33:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b05b42950 0x563b05b45290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b44d40 con 0x563b05b42950 2024-09-10T21:33:47.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.451+0000 7fe43fe00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b05b42950 0x563b05b45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.455+0000 7fe43fe00640 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b05b42950 0x563b05b45290 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.455+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v123'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b42950 2024-09-10T21:33:47.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.455+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b44d40 con 0x563b05b42950 2024-09-10T21:33:47.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.455+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v123'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b42950 2024-09-10T21:33:47.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.455+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b44d40 con 0x563b05b42950 2024-09-10T21:33:47.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v123'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b42950 2024-09-10T21:33:47.456 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:47.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b44d40 con 0x563b05b42950 2024-09-10T21:33:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b42950 2024-09-10T21:33:47.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b42950 2024-09-10T21:33:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe43fe00640 1 -- 172.21.15.97:0/3134655131 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe4300b3040 con 0x563b05b42950 2024-09-10T21:33:47.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v123'327 uv318 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.459+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.463+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v123'328 uv328 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.463+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.463+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v123'329 uv329 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.462 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:47.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.463+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.463+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v123'330 uv330 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.463+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'331 uv331 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v123'332 uv325 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v123'333 uv333 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.467 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v123'334 uv334 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.467+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b467e0 con 0x563b05b3ef80 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.471+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'335 uv335 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.471+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b467e0 con 0x563b05b3ef80 2024-09-10T21:33:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.471+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'336 uv336 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.471+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b457d0 con 0x563b05b3ef80 2024-09-10T21:33:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.471+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'337 uv337 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.471+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563b05b45c80 con 0x563b05b3ef80 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe445200640 1 -- 172.21.15.97:0/3134655131 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'338 uv338 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe438008a80 con 0x563b05b3ef80 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b05b36820 msgr2=0x563b05b38c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b05b36820 0x563b05b38c10 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b05b3ef80 msgr2=0x563b05b41370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b05b3ef80 0x563b05b41370 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b05b42950 msgr2=0x563b05b45290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b05b42950 0x563b05b45290 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b05b39c80 msgr2=0x563b05b3c0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b05b39c80 0x563b05b3c0f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 reap_dead start 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b05b31f70 msgr2=0x563b05b34360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b05b31f70 0x563b05b34360 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b05a8ec70 msgr2=0x563b05b24210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b05a8ec70 0x563b05b24210 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe424070fc0 msgr2=0x7fe424073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe424070fc0 0x7fe424073480 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c650 tx=0x7fe43800e040 comp rx=0 tx=0).stop 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6e080 msgr2=0x563b05a8e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 --2- 172.21.15.97:0/3134655131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b05a6e080 0x563b05a8e730 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fe44000d3d0 tx=0x7fe44000d890 comp rx=0 tx=0).stop 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe445c00640 1 -- 172.21.15.97:0/3134655131 reap_dead start 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 shutdown_connections 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 >> 172.21.15.97:0/3134655131 conn(0x563b059b3140 msgr2=0x563b059e4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 shutdown_connections 2024-09-10T21:33:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.475+0000 7fe447f59080 1 -- 172.21.15.97:0/3134655131 wait complete. 2024-09-10T21:33:47.477 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2024-09-10T21:33:47.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 Processor -- start 2024-09-10T21:33:47.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 -- start start 2024-09-10T21:33:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ff44940 0x56468ffd4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ffceb00 0x56468ffd2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffd3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56468fe63a00 con 0x56468ffceb00 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56468fec1f40 con 0x56468ffcf2a0 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f6455080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56468ffd0e70 con 0x56468ff44940 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ffceb00 0x56468ffd2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ffceb00 0x56468ffd2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36906/0 (socket says 172.21.15.97:36906) 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 -- 172.21.15.97:0/2265077763 learned_addr learned my addr 172.21.15.97:0/2265077763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f3600640 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ff44940 0x56468ffd4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f4000640 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffd3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 -- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ff44940 msgr2=0x56468ffd4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ff44940 0x56468ffd4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 -- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 msgr2=0x56468ffd3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffd3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 -- 172.21.15.97:0/2265077763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468fe3cc10 con 0x56468ffceb00 2024-09-10T21:33:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f4000640 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffd3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f2c00640 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ffceb00 0x56468ffd2f30 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f71e000b800 tx=0x7f71e0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bda3328755b6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f1800640 1 -- 172.21.15.97:0/2265077763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e000abd0 con 0x56468ffceb00 2024-09-10T21:33:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f1800640 1 -- 172.21.15.97:0/2265077763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e000ad70 con 0x56468ffceb00 2024-09-10T21:33:47.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.567+0000 7f71f1800640 1 -- 172.21.15.97:0/2265077763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0008690 con 0x56468ffceb00 2024-09-10T21:33:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- 172.21.15.97:0/2265077763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ffceb00 msgr2=0x56468ffd2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ffceb00 0x56468ffd2f30 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f71e000b800 tx=0x7f71e0004f90 comp rx=0 tx=0).stop 2024-09-10T21:33:47.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- 172.21.15.97:0/2265077763 shutdown_connections 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffd3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ffceb00 0x56468ffd2f30 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 --2- 172.21.15.97:0/2265077763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ff44940 0x56468ffd4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- 172.21.15.97:0/2265077763 >> 172.21.15.97:0/2265077763 conn(0x56468ff43520 msgr2=0x56468ffeb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- 172.21.15.97:0/2265077763 shutdown_connections 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- 172.21.15.97:0/2265077763 wait complete. 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 Processor -- start 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- start start 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ff44940 0x56468ffcb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ffceb00 0x56468ffea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffe76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56468ffecfc0 con 0x56468ff44940 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56468ffe8120 con 0x56468ffcf2a0 2024-09-10T21:33:47.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56468ffe83c0 con 0x56468ffceb00 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ff44940 0x56468ffcb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ff44940 0x56468ffcb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36914/0 (socket says 172.21.15.97:36914) 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 -- 172.21.15.97:0/853000839 learned_addr learned my addr 172.21.15.97:0/853000839 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f4000640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffe76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f2c00640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ffceb00 0x56468ffea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 -- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ffceb00 msgr2=0x56468ffea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56468ffceb00 0x56468ffea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 -- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 msgr2=0x56468ffe76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56468ffcf2a0 0x56468ffe76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 -- 172.21.15.97:0/853000839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468ffd3fe0 con 0x56468ff44940 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f3600640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ff44940 0x56468ffcb150 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000c10 tx=0x7f71e40ab620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71ebe00640 1 -- 172.21.15.97:0/853000839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e40bc070 con 0x56468ff44940 2024-09-10T21:33:47.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71ebe00640 1 -- 172.21.15.97:0/853000839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e40b1040 con 0x56468ff44940 2024-09-10T21:33:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71ebe00640 1 -- 172.21.15.97:0/853000839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e40b8470 con 0x56468ff44940 2024-09-10T21:33:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56468ffd4280 con 0x56468ff44940 2024-09-10T21:33:47.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.571+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468ff460a0 con 0x56468ff44940 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71ebe00640 1 -- 172.21.15.97:0/853000839 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71e40b6040 con 0x56468ff44940 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71ebe00640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71d4070fc0 0x7f71d4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71ebe00640 1 -- 172.21.15.97:0/853000839 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8597+0+0 (secure 0 0 0) 0x7f71e412dd00 con 0x56468ff44940 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71f6455080 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56468ffe7c20 0x56468ffd4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x56468ffd4c60 con 0x56468ffe7c20 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71f2c00640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71d4070fc0 0x7f71d4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71f4000640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56468ffe7c20 0x56468ffd4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71f4000640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56468ffe7c20 0x56468ffd4720 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.575+0000 7f71f2c00640 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71d4070fc0 0x7f71d4073480 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f71e000b770 tx=0x7f71e0018040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f4000640 1 -- 172.21.15.97:0/853000839 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v123'137 uv137 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f71ec002930 con 0x56468ffe7c20 2024-09-10T21:33:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56468ffe7c20 msgr2=0x56468ffd4720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x56468ffe7c20 0x56468ffd4720 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71d4070fc0 msgr2=0x7f71d4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f71d4070fc0 0x7f71d4073480 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f71e000b770 tx=0x7f71e0018040 comp rx=0 tx=0).stop 2024-09-10T21:33:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ff44940 msgr2=0x56468ffcb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 --2- 172.21.15.97:0/853000839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468ff44940 0x56468ffcb150 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000c10 tx=0x7f71e40ab620 comp rx=0 tx=0).stop 2024-09-10T21:33:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f4000640 1 -- 172.21.15.97:0/853000839 reap_dead start 2024-09-10T21:33:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 shutdown_connections 2024-09-10T21:33:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 >> 172.21.15.97:0/853000839 conn(0x56468ff43520 msgr2=0x56468ff46980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 shutdown_connections 2024-09-10T21:33:47.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.579+0000 7f71f6455080 1 -- 172.21.15.97:0/853000839 wait complete. 2024-09-10T21:33:47.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 -- 172.21.15.97:0/2004242979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0cfcc0 msgr2=0x7fdcac0b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 --2- 172.21.15.97:0/2004242979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0cfcc0 0x7fdcac0b5d20 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009840 tx=0x7fdca401f010 comp rx=0 tx=0).stop 2024-09-10T21:33:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 -- 172.21.15.97:0/2004242979 shutdown_connections 2024-09-10T21:33:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 --2- 172.21.15.97:0/2004242979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac0cfcc0 0x7fdcac0b5d20 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 -- 172.21.15.97:0/2004242979 >> 172.21.15.97:0/2004242979 conn(0x7fdcac0b3c70 msgr2=0x7fdcac0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 -- 172.21.15.97:0/2004242979 shutdown_connections 2024-09-10T21:33:47.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 -- 172.21.15.97:0/2004242979 wait complete. 2024-09-10T21:33:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 Processor -- start 2024-09-10T21:33:47.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.727+0000 7fdcb3400640 1 -- start start 2024-09-10T21:33:47.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac074b20 0x7fdcac14a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0cfcc0 0x7fdcac158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15ac30 0x7fdcac158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac15bab0 con 0x7fdcac15ac30 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac15b930 con 0x7fdcac074b20 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdcac15bc30 con 0x7fdcac0cfcc0 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15ac30 0x7fdcac158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15ac30 0x7fdcac158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36932/0 (socket says 172.21.15.97:36932) 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 -- 172.21.15.97:0/699980261 learned_addr learned my addr 172.21.15.97:0/699980261 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb0a00640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac074b20 0x7fdcac14a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcabe00640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0cfcc0 0x7fdcac158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 -- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0cfcc0 msgr2=0x7fdcac158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0cfcc0 0x7fdcac158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 -- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac074b20 msgr2=0x7fdcac14a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac074b20 0x7fdcac14a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 -- 172.21.15.97:0/699980261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac09a780 con 0x7fdcac15ac30 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb1400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15ac30 0x7fdcac158690 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fdca401bc20 tx=0x7fdca4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdca8c00640 1 -- 172.21.15.97:0/699980261 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca40be070 con 0x7fdcac15ac30 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdca8c00640 1 -- 172.21.15.97:0/699980261 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdca4005230 con 0x7fdcac15ac30 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdca8c00640 1 -- 172.21.15.97:0/699980261 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca40b5040 con 0x7fdcac15ac30 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac09c120 con 0x7fdcac15ac30 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac09c6e0 con 0x7fdcac15ac30 2024-09-10T21:33:47.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcac0b5d20 con 0x7fdcac15ac30 2024-09-10T21:33:47.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.731+0000 7fdca8c00640 1 -- 172.21.15.97:0/699980261 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdca40b8070 con 0x7fdcac15ac30 2024-09-10T21:33:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.735+0000 7fdca8c00640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc8c0725e0 0x7fdc8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.735+0000 7fdca8c00640 1 -- 172.21.15.97:0/699980261 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8597+0+0 (secure 0 0 0) 0x7fdca4130930 con 0x7fdcac15ac30 2024-09-10T21:33:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.735+0000 7fdcb0a00640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc8c0725e0 0x7fdc8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.735+0000 7fdcb0a00640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc8c0725e0 0x7fdc8c074aa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0017d0 tx=0x7fdc9c001740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.739+0000 7fdca8c00640 1 -- 172.21.15.97:0/699980261 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdca4018b70 con 0x7fdcac15ac30 2024-09-10T21:33:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:47.939+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fdcac069820 con 0x7fdcac15ac30 2024-09-10T21:33:48.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdca8c00640 1 -- 172.21.15.97:0/699980261 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v248) ==== 162+0+0 (secure 0 0 0) 0x7fdca40faeb0 con 0x7fdcac15ac30 2024-09-10T21:33:48.289 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc8c0725e0 msgr2=0x7fdc8c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc8c0725e0 0x7fdc8c074aa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0017d0 tx=0x7fdc9c001740 comp rx=0 tx=0).stop 2024-09-10T21:33:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15ac30 msgr2=0x7fdcac158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15ac30 0x7fdcac158690 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fdca401bc20 tx=0x7fdca4004620 comp rx=0 tx=0).stop 2024-09-10T21:33:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 shutdown_connections 2024-09-10T21:33:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc8c0725e0 0x7fdc8c074aa0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcac15ac30 0x7fdcac158690 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdcac0cfcc0 0x7fdcac158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 --2- 172.21.15.97:0/699980261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcac074b20 0x7fdcac14a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 >> 172.21.15.97:0/699980261 conn(0x7fdcac0b3c70 msgr2=0x7fdcac06f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:48.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 shutdown_connections 2024-09-10T21:33:48.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.291+0000 7fdcb3400640 1 -- 172.21.15.97:0/699980261 wait complete. 2024-09-10T21:33:48.300 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:33:48.299+0000 7f886ba00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-10T21:33:48.300 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:48.303+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:48.300 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:48.303+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:48.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 -- 172.21.15.97:0/1751068710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 msgr2=0x7fc7880cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 --2- 172.21.15.97:0/1751068710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 0x7fc7880cff50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc774009840 tx=0x7fc77401c820 comp rx=0 tx=0).stop 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 -- 172.21.15.97:0/1751068710 shutdown_connections 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 --2- 172.21.15.97:0/1751068710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc7880d0490 0x7fc7880ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 --2- 172.21.15.97:0/1751068710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 0x7fc7880cff50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 -- 172.21.15.97:0/1751068710 >> 172.21.15.97:0/1751068710 conn(0x7fc7880d1a60 msgr2=0x7fc7880d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 -- 172.21.15.97:0/1751068710 shutdown_connections 2024-09-10T21:33:48.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 -- 172.21.15.97:0/1751068710 wait complete. 2024-09-10T21:33:48.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.459+0000 7fc78ea00640 1 Processor -- start 2024-09-10T21:33:48.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 -- start start 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 0x7fc7881f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880d0490 0x7fc788154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc78815efc0 0x7fc788155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc788168930 con 0x7fc7880d0490 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881687b0 con 0x7fc7880cdcb0 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc788168ab0 con 0x7fc78815efc0 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880d0490 0x7fc788154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880d0490 0x7fc788154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36936/0 (socket says 172.21.15.97:36936) 2024-09-10T21:33:48.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 -- 172.21.15.97:0/2730652750 learned_addr learned my addr 172.21.15.97:0/2730652750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78d600640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 0x7fc7881f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78e000640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc78815efc0 0x7fc788155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 -- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc78815efc0 msgr2=0x7fc788155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc78815efc0 0x7fc788155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 -- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 msgr2=0x7fc7881f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 0x7fc7881f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 -- 172.21.15.97:0/2730652750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7881f60f0 con 0x7fc7880d0490 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78d600640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 0x7fc7881f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78cc00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880d0490 0x7fc788154d00 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc774009790 tx=0x7fc774005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc785600640 1 -- 172.21.15.97:0/2730652750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740b2040 con 0x7fc7880d0490 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc785600640 1 -- 172.21.15.97:0/2730652750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc774004330 con 0x7fc7880d0490 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc785600640 1 -- 172.21.15.97:0/2730652750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774005240 con 0x7fc7880d0490 2024-09-10T21:33:48.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc788159790 con 0x7fc7880d0490 2024-09-10T21:33:48.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc788159d50 con 0x7fc7880d0490 2024-09-10T21:33:48.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.463+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7880ae930 con 0x7fc7880d0490 2024-09-10T21:33:48.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.467+0000 7fc785600640 1 -- 172.21.15.97:0/2730652750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc7740bf060 con 0x7fc7880d0490 2024-09-10T21:33:48.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.467+0000 7fc785600640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7680725e0 0x7fc768074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.467+0000 7fc785600640 1 -- 172.21.15.97:0/2730652750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7fc77412cfa0 con 0x7fc7880d0490 2024-09-10T21:33:48.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.467+0000 7fc78d600640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7680725e0 0x7fc768074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:48.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.467+0000 7fc78d600640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7680725e0 0x7fc768074aa0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc77c007a60 tx=0x7fc77c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:48.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.471+0000 7fc785600640 1 -- 172.21.15.97:0/2730652750 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc774016410 con 0x7fc7880d0490 2024-09-10T21:33:48.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7880b68a0 con 0x7fc7880d0490 2024-09-10T21:33:48.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc785600640 1 -- 172.21.15.97:0/2730652750 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 250 v250) ==== 95+0+4395 (secure 0 0 0) 0x7fc7740f7a90 con 0x7fc7880d0490 2024-09-10T21:33:48.669 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:48.669 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":250,"btime":"2024-09-10T21:33:48:304307+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2561173061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2561173061},{"type":"v1","addr":"172.21.15.97:6834","nonce":2561173061}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":243}],"filesystems":[{"mdsmap":{"epoch":250,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:48.304249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:48.670 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 250 2024-09-10T21:33:48.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7680725e0 msgr2=0x7fc768074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7680725e0 0x7fc768074aa0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc77c007a60 tx=0x7fc77c009040 comp rx=0 tx=0).stop 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880d0490 msgr2=0x7fc788154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880d0490 0x7fc788154d00 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc774009790 tx=0x7fc774005d20 comp rx=0 tx=0).stop 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 shutdown_connections 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc7680725e0 0x7fc768074aa0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc78815efc0 0x7fc788155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7880d0490 0x7fc788154d00 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 --2- 172.21.15.97:0/2730652750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7880cdcb0 0x7fc7881f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 >> 172.21.15.97:0/2730652750 conn(0x7fc7880d1a60 msgr2=0x7fc7880b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 shutdown_connections 2024-09-10T21:33:48.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.671+0000 7fc78ea00640 1 -- 172.21.15.97:0/2730652750 wait complete. 2024-09-10T21:33:48.691 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:48.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:48.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 -- 172.21.15.97:0/2153987725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00b0d50 msgr2=0x7f3ed00aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/2153987725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00b0d50 0x7f3ed00aedc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0074d10 tx=0x7f3eb801c800 comp rx=0 tx=0).stop 2024-09-10T21:33:48.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 -- 172.21.15.97:0/2153987725 shutdown_connections 2024-09-10T21:33:48.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/2153987725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00b0d50 0x7f3ed00aedc0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/2153987725 >> v2:172.21.15.194:3300/0 conn(0x7f3ed00ce640 0x7f3ed00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 -- 172.21.15.97:0/2153987725 >> 172.21.15.97:0/2153987725 conn(0x7f3ed00d1c30 msgr2=0x7f3ed00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:48.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 -- 172.21.15.97:0/2153987725 shutdown_connections 2024-09-10T21:33:48.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 -- 172.21.15.97:0/2153987725 wait complete. 2024-09-10T21:33:48.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.835+0000 7f3ed5a00640 1 Processor -- start 2024-09-10T21:33:48.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 -- start start 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00af300 0x7f3ed00c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ed00b0d50 0x7f3ed01561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed00ce640 0x7f3ed0156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0169270 con 0x7f3ed00af300 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed01690f0 con 0x7f3ed00ce640 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed01693f0 con 0x7f3ed00b0d50 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00af300 0x7f3ed00c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00af300 0x7f3ed00c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36948/0 (socket says 172.21.15.97:36948) 2024-09-10T21:33:48.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 -- 172.21.15.97:0/1473078575 learned_addr learned my addr 172.21.15.97:0/1473078575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecf400640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed00ce640 0x7f3ed0156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 -- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ed00b0d50 msgr2=0x7f3ed01561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ed00b0d50 0x7f3ed01561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 -- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed00ce640 msgr2=0x7f3ed0156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed00ce640 0x7f3ed0156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 -- 172.21.15.97:0/1473078575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed00c8400 con 0x7f3ed00af300 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecf400640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed00ce640 0x7f3ed0156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ecea00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00af300 0x7f3ed00c6510 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000e7d0 tx=0x7f3ec000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ec7400640 1 -- 172.21.15.97:0/1473078575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000ccc0 con 0x7f3ed00af300 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ec7400640 1 -- 172.21.15.97:0/1473078575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ec0010430 con 0x7f3ed00af300 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ec7400640 1 -- 172.21.15.97:0/1473078575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0010860 con 0x7f3ed00af300 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed00c86a0 con 0x7f3ed00af300 2024-09-10T21:33:48.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed0158330 con 0x7f3ed00af300 2024-09-10T21:33:48.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.839+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed00aedc0 con 0x7f3ed00af300 2024-09-10T21:33:48.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.843+0000 7f3ec7400640 1 -- 172.21.15.97:0/1473078575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ec0010a00 con 0x7f3ed00af300 2024-09-10T21:33:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.843+0000 7f3ec7400640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3eac0725e0 0x7f3eac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.843+0000 7f3ec7400640 1 -- 172.21.15.97:0/1473078575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7f3ec0094820 con 0x7f3ed00af300 2024-09-10T21:33:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.843+0000 7f3ec6000640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3eac0725e0 0x7f3eac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.843+0000 7f3ec6000640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3eac0725e0 0x7f3eac074aa0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80af000 tx=0x7f3eb80195a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:48.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:48.847+0000 7f3ec7400640 1 -- 172.21.15.97:0/1473078575 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ec005f340 con 0x7f3ed00af300 2024-09-10T21:33:49.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.051+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed00b06c0 con 0x7f3ed00af300 2024-09-10T21:33:49.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.051+0000 7f3ec7400640 1 -- 172.21.15.97:0/1473078575 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 250 v250) ==== 95+0+4395 (secure 0 0 0) 0x7f3ec005ec40 con 0x7f3ed00af300 2024-09-10T21:33:49.050 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:49.050 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":250,"btime":"2024-09-10T21:33:48:304307+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2561173061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2561173061},{"type":"v1","addr":"172.21.15.97:6834","nonce":2561173061}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":243}],"filesystems":[{"mdsmap":{"epoch":250,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:48.304249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 250 2024-09-10T21:33:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3eac0725e0 msgr2=0x7f3eac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3eac0725e0 0x7f3eac074aa0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80af000 tx=0x7f3eb80195a0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00af300 msgr2=0x7f3ed00c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00af300 0x7f3ed00c6510 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000e7d0 tx=0x7f3ec000ec90 comp rx=0 tx=0).stop 2024-09-10T21:33:49.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 shutdown_connections 2024-09-10T21:33:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3eac0725e0 0x7f3eac074aa0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed00ce640 0x7f3ed0156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ed00b0d50 0x7f3ed01561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 --2- 172.21.15.97:0/1473078575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed00af300 0x7f3ed00c6510 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 >> 172.21.15.97:0/1473078575 conn(0x7f3ed00d1c30 msgr2=0x7f3ed00d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 shutdown_connections 2024-09-10T21:33:49.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.055+0000 7f3ed5a00640 1 -- 172.21.15.97:0/1473078575 wait complete. 2024-09-10T21:33:49.072 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:49.072 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:49.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- 172.21.15.97:0/1641290058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a60003a10 msgr2=0x7f2a60003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 --2- 172.21.15.97:0/1641290058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a60003a10 0x7f2a60003e30 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009b60 tx=0x7f2a6c01d220 comp rx=0 tx=0).stop 2024-09-10T21:33:49.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a7a800640 1 -- 172.21.15.97:0/1641290058 reap_dead start 2024-09-10T21:33:49.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- 172.21.15.97:0/1641290058 shutdown_connections 2024-09-10T21:33:49.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- 172.21.15.97:0/1641290058 >> 172.21.15.97:0/1641290058 conn(0x7f2a7c0d1cd0 msgr2=0x7f2a7c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:49.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- 172.21.15.97:0/1641290058 shutdown_connections 2024-09-10T21:33:49.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- 172.21.15.97:0/1641290058 wait complete. 2024-09-10T21:33:49.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 Processor -- start 2024-09-10T21:33:49.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- start start 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a60003a10 0x7f2a7c14d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c06edf0 0x7f2a7c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a7c163a80 0x7f2a7c163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b12e0 con 0x7f2a7c06edf0 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b1160 con 0x7f2a60003a10 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.219+0000 7f2a80e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c1666f0 con 0x7f2a7c163a80 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c06edf0 0x7f2a7c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c06edf0 0x7f2a7c06f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36980/0 (socket says 172.21.15.97:36980) 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 -- 172.21.15.97:0/1672752826 learned_addr learned my addr 172.21.15.97:0/1672752826 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a7a800640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a7c163a80 0x7f2a7c163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a79e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a60003a10 0x7f2a7c14d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 -- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a7c163a80 msgr2=0x7f2a7c163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a7c163a80 0x7f2a7c163f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 -- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a60003a10 msgr2=0x7f2a7c14d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a60003a10 0x7f2a7c14d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 -- 172.21.15.97:0/1672752826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c06ddf0 con 0x7f2a7c06edf0 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a79e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a60003a10 0x7f2a7c14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a7a800640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a7c163a80 0x7f2a7c163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a71400640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c06edf0 0x7f2a7c06f250 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2a64001360 tx=0x7f2a6400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a72a00640 1 -- 172.21.15.97:0/1672752826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64015070 con 0x7f2a7c06edf0 2024-09-10T21:33:49.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a72a00640 1 -- 172.21.15.97:0/1672752826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a6400cd10 con 0x7f2a7c06edf0 2024-09-10T21:33:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a72a00640 1 -- 172.21.15.97:0/1672752826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640116a0 con 0x7f2a7c06edf0 2024-09-10T21:33:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c06e0d0 con 0x7f2a7c06edf0 2024-09-10T21:33:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c06e690 con 0x7f2a7c06edf0 2024-09-10T21:33:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c0d27f0 con 0x7f2a7c06edf0 2024-09-10T21:33:49.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.223+0000 7f2a72a00640 1 -- 172.21.15.97:0/1672752826 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a64011840 con 0x7f2a7c06edf0 2024-09-10T21:33:49.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.227+0000 7f2a72a00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a580725e0 0x7f2a58074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.227+0000 7f2a72a00640 1 -- 172.21.15.97:0/1672752826 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7f2a640948a0 con 0x7f2a7c06edf0 2024-09-10T21:33:49.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.227+0000 7f2a79e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a580725e0 0x7f2a58074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.227+0000 7f2a79e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a580725e0 0x7f2a58074aa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0ae000 tx=0x7f2a6c01a020 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:49.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.231+0000 7f2a72a00640 1 -- 172.21.15.97:0/1672752826 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a6405f330 con 0x7f2a7c06edf0 2024-09-10T21:33:49.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.427+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a7c0cfd60 con 0x7f2a7c06edf0 2024-09-10T21:33:49.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a72a00640 1 -- 172.21.15.97:0/1672752826 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 250 v250) ==== 95+0+4395 (secure 0 0 0) 0x7f2a6405ec30 con 0x7f2a7c06edf0 2024-09-10T21:33:49.429 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:49.429 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":250,"btime":"2024-09-10T21:33:48:304307+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2561173061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2561173061},{"type":"v1","addr":"172.21.15.97:6834","nonce":2561173061}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":243}],"filesystems":[{"mdsmap":{"epoch":250,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:48.304249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 250 2024-09-10T21:33:49.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a580725e0 msgr2=0x7f2a58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a580725e0 0x7f2a58074aa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0ae000 tx=0x7f2a6c01a020 comp rx=0 tx=0).stop 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c06edf0 msgr2=0x7f2a7c06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c06edf0 0x7f2a7c06f250 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2a64001360 tx=0x7f2a6400ecf0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 shutdown_connections 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a580725e0 0x7f2a58074aa0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a7c163a80 0x7f2a7c163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c06edf0 0x7f2a7c06f250 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 --2- 172.21.15.97:0/1672752826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a60003a10 0x7f2a7c14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 >> 172.21.15.97:0/1672752826 conn(0x7f2a7c0d1cd0 msgr2=0x7f2a7c072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:49.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 shutdown_connections 2024-09-10T21:33:49.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.431+0000 7f2a80e00640 1 -- 172.21.15.97:0/1672752826 wait complete. 2024-09-10T21:33:49.450 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2024-09-10T21:33:49.450 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2024-09-10T21:33:49.450 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:49.450 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:49.459 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:49.459 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:49.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:49.459 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:49.507 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:49.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:49.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- 172.21.15.97:0/1995387031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 msgr2=0x7f357806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 --2- 172.21.15.97:0/1995387031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 0x7f357806f090 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3560009870 tx=0x7f356001ef30 comp rx=0 tx=0).stop 2024-09-10T21:33:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- 172.21.15.97:0/1995387031 shutdown_connections 2024-09-10T21:33:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 --2- 172.21.15.97:0/1995387031 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35780ceee0 0x7f35780cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 --2- 172.21.15.97:0/1995387031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 0x7f357806f090 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- 172.21.15.97:0/1995387031 >> 172.21.15.97:0/1995387031 conn(0x7f35780b1780 msgr2=0x7f35780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:49.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- 172.21.15.97:0/1995387031 shutdown_connections 2024-09-10T21:33:49.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- 172.21.15.97:0/1995387031 wait complete. 2024-09-10T21:33:49.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 Processor -- start 2024-09-10T21:33:49.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- start start 2024-09-10T21:33:49.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 0x7f35781f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35780ceee0 0x7f35781f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781511f0 0x7f35781593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f357815aac0 con 0x7f35781511f0 2024-09-10T21:33:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f357815a940 con 0x7f357806ec40 2024-09-10T21:33:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357f400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f357815ac40 con 0x7f35780ceee0 2024-09-10T21:33:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.695+0000 7f357d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781511f0 0x7f35781593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781511f0 0x7f35781593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36992/0 (socket says 172.21.15.97:36992) 2024-09-10T21:33:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 -- 172.21.15.97:0/1946677894 learned_addr learned my addr 172.21.15.97:0/1946677894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357ca00640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 0x7f35781f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f356fe00640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35780ceee0 0x7f35781f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 -- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35780ceee0 msgr2=0x7f35781f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35780ceee0 0x7f35781f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 -- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 msgr2=0x7f35781f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 0x7f35781f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 -- 172.21.15.97:0/1946677894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35781f8970 con 0x7f35781511f0 2024-09-10T21:33:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357d400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781511f0 0x7f35781593d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f357000eb20 tx=0x7f357000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f3575600640 1 -- 172.21.15.97:0/1946677894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570004b30 con 0x7f35781511f0 2024-09-10T21:33:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f3575600640 1 -- 172.21.15.97:0/1946677894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3570004cf0 con 0x7f35781511f0 2024-09-10T21:33:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35781f8c50 con 0x7f35781511f0 2024-09-10T21:33:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f3575600640 1 -- 172.21.15.97:0/1946677894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570005280 con 0x7f35781511f0 2024-09-10T21:33:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35781f4ee0 con 0x7f35781511f0 2024-09-10T21:33:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357815b060 con 0x7f35781511f0 2024-09-10T21:33:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.699+0000 7f3575600640 1 -- 172.21.15.97:0/1946677894 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3570005420 con 0x7f35781511f0 2024-09-10T21:33:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.703+0000 7f3575600640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35540725e0 0x7f3554074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.703+0000 7f3575600640 1 -- 172.21.15.97:0/1946677894 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7f3570094ae0 con 0x7f35781511f0 2024-09-10T21:33:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.703+0000 7f357ca00640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35540725e0 0x7f3554074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.703+0000 7f357ca00640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35540725e0 0x7f3554074aa0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3568007990 tx=0x7f3568001da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:49.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.703+0000 7f3575600640 1 -- 172.21.15.97:0/1946677894 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f357005ef20 con 0x7f35781511f0 2024-09-10T21:33:49.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:49.903+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f357806aad0 con 0x7f35781511f0 2024-09-10T21:33:50.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.295+0000 7f3575600640 1 -- 172.21.15.97:0/1946677894 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v251) ==== 161+0+0 (secure 0 0 0) 0x7f3570063950 con 0x7f35781511f0 2024-09-10T21:33:50.296 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35540725e0 msgr2=0x7f3554074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35540725e0 0x7f3554074aa0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3568007990 tx=0x7f3568001da0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781511f0 msgr2=0x7f35781593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781511f0 0x7f35781593d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f357000eb20 tx=0x7f357000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 shutdown_connections 2024-09-10T21:33:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35540725e0 0x7f3554074aa0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781511f0 0x7f35781593d0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35780ceee0 0x7f35781f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 --2- 172.21.15.97:0/1946677894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f357806ec40 0x7f35781f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 >> 172.21.15.97:0/1946677894 conn(0x7f35780b1780 msgr2=0x7f3578150310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 shutdown_connections 2024-09-10T21:33:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.299+0000 7f357f400640 1 -- 172.21.15.97:0/1946677894 wait complete. 2024-09-10T21:33:50.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:50.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.463+0000 7f927e000640 1 -- 172.21.15.97:0/2932303526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780b0d50 msgr2=0x7f92780ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:50.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.463+0000 7f927e000640 1 --2- 172.21.15.97:0/2932303526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780b0d50 0x7f92780ccde0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f9270009bc0 tx=0x7f927001cba0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.463+0000 7f927e000640 1 -- 172.21.15.97:0/2932303526 shutdown_connections 2024-09-10T21:33:50.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.463+0000 7f927e000640 1 --2- 172.21.15.97:0/2932303526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780b0d50 0x7f92780ccde0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.463+0000 7f927e000640 1 -- 172.21.15.97:0/2932303526 >> 172.21.15.97:0/2932303526 conn(0x7f92780d1c30 msgr2=0x7f92780d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:50.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.463+0000 7f927e000640 1 -- 172.21.15.97:0/2932303526 shutdown_connections 2024-09-10T21:33:50.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.463+0000 7f927e000640 1 -- 172.21.15.97:0/2932303526 wait complete. 2024-09-10T21:33:50.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 Processor -- start 2024-09-10T21:33:50.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 -- start start 2024-09-10T21:33:50.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780afdd0 0x7f92780cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:50.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f92780b0d50 0x7f92780b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:50.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92780ce640 0x7f927814aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:50.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9278169ef0 con 0x7f92780afdd0 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9278169d70 con 0x7f92780ce640 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f927816a070 con 0x7f92780b0d50 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780afdd0 0x7f92780cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780afdd0 0x7f92780cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37002/0 (socket says 172.21.15.97:37002) 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 -- 172.21.15.97:0/1008454310 learned_addr learned my addr 172.21.15.97:0/1008454310 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277e00640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92780ce640 0x7f927814aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9276a00640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f92780b0d50 0x7f92780b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 -- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f92780b0d50 msgr2=0x7f92780b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f92780b0d50 0x7f92780b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 -- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92780ce640 msgr2=0x7f927814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:50.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92780ce640 0x7f927814aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:50.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 -- 172.21.15.97:0/1008454310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92780c8af0 con 0x7f92780afdd0 2024-09-10T21:33:50.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f9277400640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780afdd0 0x7f92780cb750 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f926800c430 tx=0x7f926800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:50.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f926f400640 1 -- 172.21.15.97:0/1008454310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268011790 con 0x7f92780afdd0 2024-09-10T21:33:50.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f926f400640 1 -- 172.21.15.97:0/1008454310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9268017070 con 0x7f92780afdd0 2024-09-10T21:33:50.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92780b5d00 con 0x7f92780afdd0 2024-09-10T21:33:50.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f926f400640 1 -- 172.21.15.97:0/1008454310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268011ce0 con 0x7f92780afdd0 2024-09-10T21:33:50.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.467+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92780b62c0 con 0x7f92780afdd0 2024-09-10T21:33:50.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.471+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92780cea20 con 0x7f92780afdd0 2024-09-10T21:33:50.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.471+0000 7f926f400640 1 -- 172.21.15.97:0/1008454310 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f926801c460 con 0x7f92780afdd0 2024-09-10T21:33:50.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.471+0000 7f926f400640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9258072610 0x7f9258074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.471+0000 7f926f400640 1 -- 172.21.15.97:0/1008454310 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7f9268093a50 con 0x7f92780afdd0 2024-09-10T21:33:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.471+0000 7f9276a00640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9258072610 0x7f9258074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.475+0000 7f9276a00640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9258072610 0x7f9258074ad0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9278167340 tx=0x7f9264009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.475+0000 7f926f400640 1 -- 172.21.15.97:0/1008454310 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f926805e690 con 0x7f92780afdd0 2024-09-10T21:33:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:50.699+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f92780d24e0 con 0x7f92780afdd0 2024-09-10T21:33:51.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.307+0000 7f926f400640 1 -- 172.21.15.97:0/1008454310 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v252) ==== 101+0+0 (secure 0 0 0) 0x7f926805df90 con 0x7f92780afdd0 2024-09-10T21:33:51.307 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:51.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9258072610 msgr2=0x7f9258074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9258072610 0x7f9258074ad0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9278167340 tx=0x7f9264009040 comp rx=0 tx=0).stop 2024-09-10T21:33:51.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780afdd0 msgr2=0x7f92780cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780afdd0 0x7f92780cb750 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f926800c430 tx=0x7f926800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 shutdown_connections 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9258072610 0x7f9258074ad0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92780ce640 0x7f927814aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f92780b0d50 0x7f92780b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 --2- 172.21.15.97:0/1008454310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92780afdd0 0x7f92780cb750 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 >> 172.21.15.97:0/1008454310 conn(0x7f92780d1c30 msgr2=0x7f927806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 shutdown_connections 2024-09-10T21:33:51.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.311+0000 7f927e000640 1 -- 172.21.15.97:0/1008454310 wait complete. 2024-09-10T21:33:51.331 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:51.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 Processor -- start 2024-09-10T21:33:51.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- start start 2024-09-10T21:33:51.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b0cc0 0x555cbe542540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b2830 0x555cbe5b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b61d0 0x555cbe5cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555cbe5b37b0 con 0x555cbe5b61d0 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555cbe446a00 con 0x555cbe5b0cc0 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555cbe5b3930 con 0x555cbe5b2830 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b61d0 0x555cbe5cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f4600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b0cc0 0x555cbe542540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f4600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b0cc0 0x555cbe542540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:37162/0 (socket says 172.21.15.97:37162) 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b61d0 0x555cbe5cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37016/0 (socket says 172.21.15.97:37016) 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f4600640 1 -- 172.21.15.97:0/3620896425 learned_addr learned my addr 172.21.15.97:0/3620896425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f3c00640 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b2830 0x555cbe5b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 -- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b2830 msgr2=0x555cbe5b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b2830 0x555cbe5b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 -- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b0cc0 msgr2=0x555cbe542540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b0cc0 0x555cbe542540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 -- 172.21.15.97:0/3620896425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cbe445a20 con 0x555cbe5b61d0 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f4600640 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b0cc0 0x555cbe542540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f5000640 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b61d0 0x555cbe5cdf00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00c6e0 tx=0x7f18ec004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc30867ddadcbc5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f2800640 1 -- 172.21.15.97:0/3620896425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec004020 con 0x555cbe5b61d0 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f2800640 1 -- 172.21.15.97:0/3620896425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18ec0042b0 con 0x555cbe5b61d0 2024-09-10T21:33:51.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f2800640 1 -- 172.21.15.97:0/3620896425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00aa80 con 0x555cbe5b61d0 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- 172.21.15.97:0/3620896425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b61d0 msgr2=0x555cbe5cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b61d0 0x555cbe5cdf00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00c6e0 tx=0x7f18ec004f90 comp rx=0 tx=0).stop 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- 172.21.15.97:0/3620896425 shutdown_connections 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b61d0 0x555cbe5cdf00 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b2830 0x555cbe5b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 --2- 172.21.15.97:0/3620896425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b0cc0 0x555cbe542540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- 172.21.15.97:0/3620896425 >> 172.21.15.97:0/3620896425 conn(0x555cbe5400d0 msgr2=0x555cbe5cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- 172.21.15.97:0/3620896425 shutdown_connections 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.379+0000 7f18f7455080 1 -- 172.21.15.97:0/3620896425 wait complete. 2024-09-10T21:33:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 Processor -- start 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 -- start start 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b0cc0 0x555cbe668360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b2830 0x555cbe5ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b61d0 0x555cbe5cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555cbe526ff0 con 0x555cbe5b0cc0 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555cbe526d50 con 0x555cbe5b2830 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555cbe527290 con 0x555cbe5b61d0 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b0cc0 0x555cbe668360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b0cc0 0x555cbe668360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37026/0 (socket says 172.21.15.97:37026) 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 learned_addr learned my addr 172.21.15.97:0/1626254604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b2830 0x555cbe5ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f5000640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b61d0 0x555cbe5cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b61d0 msgr2=0x555cbe5cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b61d0 0x555cbe5cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b2830 msgr2=0x555cbe5ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b2830 0x555cbe5ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cbe41fc10 con 0x555cbe5b0cc0 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f5000640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555cbe5b61d0 0x555cbe5cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555cbe5b2830 0x555cbe5ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f4600640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b0cc0 0x555cbe668360 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f18e4008750 tx=0x7f18e4008c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f1400640 1 -- 172.21.15.97:0/1626254604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e401a070 con 0x555cbe5b0cc0 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f1400640 1 -- 172.21.15.97:0/1626254604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18e400add0 con 0x555cbe5b0cc0 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555cbe53dfe0 con 0x555cbe5b0cc0 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f1400640 1 -- 172.21.15.97:0/1626254604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4015680 con 0x555cbe5b0cc0 2024-09-10T21:33:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.383+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cbe53e520 con 0x555cbe5b0cc0 2024-09-10T21:33:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f1400640 1 -- 172.21.15.97:0/1626254604 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18e4015820 con 0x555cbe5b0cc0 2024-09-10T21:33:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f1400640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18d4070fc0 0x7f18d4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f1400640 1 -- 172.21.15.97:0/1626254604 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7f18e4092ff0 con 0x555cbe5b0cc0 2024-09-10T21:33:51.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18d4070fc0 0x7f18d4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.387 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555cbe6688a0 0x555cbe5288a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe5aeca0 con 0x555cbe6688a0 2024-09-10T21:33:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f5000640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555cbe6688a0 0x555cbe5288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18d4070fc0 0x7f18d4073480 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f18e00ab000 tx=0x7f18e0019820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.387+0000 7f18f5000640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555cbe6688a0 0x555cbe5288a0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.391+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v124'846 uv820 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.391+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe53ef00 con 0x555cbe6688a0 2024-09-10T21:33:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.391+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v124'847 uv847 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.391+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe53f2d0 con 0x555cbe6688a0 2024-09-10T21:33:51.392 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:51.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.395+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v124'848 uv848 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.395+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe53f670 con 0x555cbe6688a0 2024-09-10T21:33:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.395+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'849 uv849 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.395+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675930 con 0x555cbe6688a0 2024-09-10T21:33:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.395+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'850 uv850 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.395+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe674920 con 0x555cbe6688a0 2024-09-10T21:33:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.399+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.399+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe674d10 con 0x555cbe6688a0 2024-09-10T21:33:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.399+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v124'852 uv826 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.399+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe675100 con 0x555cbe6688a0 2024-09-10T21:33:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.403+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v124'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.403+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675100 con 0x555cbe6688a0 2024-09-10T21:33:51.402 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:51.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.403+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v124'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.403+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.407+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.407+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.407+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.407+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.407+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.407+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.411+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v124'858 uv833 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.411+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.411+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'859 uv859 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.411+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.413 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:51.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.415+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v124'860 uv860 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.415+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.415+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'861 uv861 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.415+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.419+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'862 uv862 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.419+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6750b0 con 0x555cbe6688a0 2024-09-10T21:33:51.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.419+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'863 uv863 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.419+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.419+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'864 uv864 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.419+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.423+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v124'865 uv839 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.423+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.423+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v124'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.423+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.424 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:51.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.427+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v124'867 uv867 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.427+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe676ce0 con 0x555cbe6688a0 2024-09-10T21:33:51.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.427+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'868 uv868 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.427+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe676ce0 con 0x555cbe6688a0 2024-09-10T21:33:51.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.427+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'869 uv869 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.427+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.431+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'870 uv870 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.431+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.431+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v124'871 uv845 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.431+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.435+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v124'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.435+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.434 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:51.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.435+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v124'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.435+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe676ce0 con 0x555cbe6688a0 2024-09-10T21:33:51.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.435+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.435+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe676ce0 con 0x555cbe6688a0 2024-09-10T21:33:51.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.439+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.439+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe675cd0 con 0x555cbe6688a0 2024-09-10T21:33:51.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.439+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe6688a0 2024-09-10T21:33:51.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.439+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x555cbe6761e0 0x555cbe6785d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.439+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe678b10 con 0x555cbe6761e0 2024-09-10T21:33:51.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.443+0000 7f18f4600640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x555cbe6761e0 0x555cbe6785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.443+0000 7f18f4600640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x555cbe6761e0 0x555cbe6785d0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.443+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'617 uv600 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.443+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe678eb0 con 0x555cbe6761e0 2024-09-10T21:33:51.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.447+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'618 uv618 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.447+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe679250 con 0x555cbe6761e0 2024-09-10T21:33:51.446 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:51.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.447+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'619 uv619 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.447+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe679aa0 con 0x555cbe6761e0 2024-09-10T21:33:51.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.447+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v124'620 uv620 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.447+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe679e40 con 0x555cbe6761e0 2024-09-10T21:33:51.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.451+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'621 uv621 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.451+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a1e0 con 0x555cbe6761e0 2024-09-10T21:33:51.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.451+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v124'622 uv605 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.451+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe67a580 con 0x555cbe6761e0 2024-09-10T21:33:51.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.455+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v124'623 uv623 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.455+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.454 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.455+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v124'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.455+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.459+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.459+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.459+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.459+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.459+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v124'627 uv610 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.459+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.463+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v124'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.463+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.462 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:51.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.463+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v124'629 uv629 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.463+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.467+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v124'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.467+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.467+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.467+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.467+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v124'632 uv616 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.467+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.471+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v124'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.471+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.470 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:51.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.471+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v124'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.471+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.471+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'635 uv635 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.471+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.475+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.475+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67a920 con 0x555cbe6761e0 2024-09-10T21:33:51.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.475+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe6761e0 2024-09-10T21:33:51.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.475+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x555cbe67b220 0x555cbe67b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.475+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67dd80 con 0x555cbe67b220 2024-09-10T21:33:51.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.475+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x555cbe67b220 0x555cbe67b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.475+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x555cbe67b220 0x555cbe67b600 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.479+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.479+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe67e120 con 0x555cbe67b220 2024-09-10T21:33:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.479+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.479+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67e4c0 con 0x555cbe67b220 2024-09-10T21:33:51.479 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.479+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.479+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67fd20 con 0x555cbe67b220 2024-09-10T21:33:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6800c0 con 0x555cbe67b220 2024-09-10T21:33:51.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe67ed10 con 0x555cbe67b220 2024-09-10T21:33:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x555cbe67f630 0x555cbe682470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6829b0 con 0x555cbe67f630 2024-09-10T21:33:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f5000640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x555cbe67f630 0x555cbe682470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.483+0000 7f18f5000640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x555cbe67f630 0x555cbe682470 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.487+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'124 uv123 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.487+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe6829b0 con 0x555cbe67f630 2024-09-10T21:33:51.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.487+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'125 uv125 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.487+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6829b0 con 0x555cbe67f630 2024-09-10T21:33:51.487 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.487+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'126 uv126 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.487+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.491+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'127 uv127 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.491+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.491+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.491+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.491+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.491+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.492 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe683dd0 con 0x555cbe67b220 2024-09-10T21:33:51.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe683dd0 con 0x555cbe67b220 2024-09-10T21:33:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.495+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.499+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v124'305 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.499+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.499+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v124'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.499+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.500 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v124'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe683dd0 con 0x555cbe67f630 2024-09-10T21:33:51.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe683dd0 con 0x555cbe67f630 2024-09-10T21:33:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.503+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.507+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v124'311 uv304 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.507+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.507+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v124'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.507+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.507 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:51.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.507+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v124'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.507+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.511+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.511+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67f630 2024-09-10T21:33:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.511+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.511+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.511+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v124'138 uv137 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.511+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v124'139 uv139 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v124'140 uv140 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.513 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:51.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'141 uv141 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe682dc0 con 0x555cbe67b220 2024-09-10T21:33:51.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'142 uv142 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe67b220 2024-09-10T21:33:51.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.515+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x555cbe683440 0x555cbe685830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.519+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe685e40 con 0x555cbe683440 2024-09-10T21:33:51.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.519+0000 7f18f4600640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x555cbe683440 0x555cbe685830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.519+0000 7f18f4600640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x555cbe683440 0x555cbe685830 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.519+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.519+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe685f20 con 0x555cbe683440 2024-09-10T21:33:51.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.519+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.519+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.523+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.521 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.523+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687350 con 0x555cbe683440 2024-09-10T21:33:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.523+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.523+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687350 con 0x555cbe683440 2024-09-10T21:33:51.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.523+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.523+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v124'273 uv267 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v124'274 uv274 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.527 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v124'275 uv275 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.527+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.531+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'276 uv276 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.531+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.531+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'277 uv277 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.531+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.531+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v124'278 uv272 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.531+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v124'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.533 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:51.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v124'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe686340 con 0x555cbe683440 2024-09-10T21:33:51.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe689150 con 0x555cbe683440 2024-09-10T21:33:51.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.535+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.539+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v124'120 uv119 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.539+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.539+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'121 uv121 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.539+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.539+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v124'122 uv122 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.539 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:51.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.539+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.543+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'123 uv123 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.543+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.543+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v124'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.543+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v124'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.545 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v124'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687800 con 0x555cbe67f630 2024-09-10T21:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687800 con 0x555cbe67f630 2024-09-10T21:33:51.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.547+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe6867f0 con 0x555cbe67f630 2024-09-10T21:33:51.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18ec002030 con 0x555cbe67f630 2024-09-10T21:33:51.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x555cbe686e70 0x555cbe687250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe686e70 2024-09-10T21:33:51.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x555cbe686e70 0x555cbe687250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f3c00640 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x555cbe686e70 0x555cbe687250 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v124'140 uv139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe686e70 2024-09-10T21:33:51.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe686e70 2024-09-10T21:33:51.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v124'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe686e70 2024-09-10T21:33:51.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.551+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe686e70 2024-09-10T21:33:51.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.555+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v124'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe686e70 2024-09-10T21:33:51.554 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:51.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.555+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe686e70 2024-09-10T21:33:51.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.555+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe686e70 2024-09-10T21:33:51.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.555+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe686e70 2024-09-10T21:33:51.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.559+0000 7f18f3c00640 1 -- 172.21.15.97:0/1626254604 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f18e00c2040 con 0x555cbe686e70 2024-09-10T21:33:51.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.559+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe683440 2024-09-10T21:33:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.559+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v124'339 uv331 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.559+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe683440 2024-09-10T21:33:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.559+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v124'340 uv340 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.559+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe683440 2024-09-10T21:33:51.560 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.563+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v124'341 uv341 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.563+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687ca0 con 0x555cbe683440 2024-09-10T21:33:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.563+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v124'342 uv342 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.563+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687ca0 con 0x555cbe683440 2024-09-10T21:33:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.563+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'343 uv343 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.563+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687c50 con 0x555cbe683440 2024-09-10T21:33:51.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v124'344 uv338 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x555cbe687c50 con 0x555cbe683440 2024-09-10T21:33:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v124'345 uv345 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe683440 2024-09-10T21:33:51.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v124'346 uv346 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.566 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe683440 2024-09-10T21:33:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'347 uv347 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.567+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe683440 2024-09-10T21:33:51.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'348 uv348 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe687790 con 0x555cbe683440 2024-09-10T21:33:51.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'349 uv349 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x555cbe688ca0 con 0x555cbe683440 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f4600640 1 -- 172.21.15.97:0/1626254604 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'350 uv350 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f18e401c030 con 0x555cbe683440 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x555cbe67b220 msgr2=0x555cbe67b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x555cbe67b220 0x555cbe67b600 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x555cbe683440 msgr2=0x555cbe685830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x555cbe683440 0x555cbe685830 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x555cbe686e70 msgr2=0x555cbe687250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x555cbe686e70 0x555cbe687250 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x555cbe67f630 msgr2=0x555cbe682470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x555cbe67f630 0x555cbe682470 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x555cbe6761e0 msgr2=0x555cbe6785d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x555cbe6761e0 0x555cbe6785d0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 reap_dead start 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555cbe6688a0 msgr2=0x555cbe5288a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555cbe6688a0 0x555cbe5288a0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 reap_dead start 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f5000640 1 -- 172.21.15.97:0/1626254604 reap_dead start 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18d4070fc0 msgr2=0x7f18d4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18d4070fc0 0x7f18d4073480 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f18e00ab000 tx=0x7f18e0019820 comp rx=0 tx=0).stop 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b0cc0 msgr2=0x555cbe668360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b0cc0 0x555cbe668360 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f18e4008750 tx=0x7f18e4008c10 comp rx=0 tx=0).stop 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 shutdown_connections 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18d4070fc0 0x7f18d4073480 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 --2- 172.21.15.97:0/1626254604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cbe5b0cc0 0x555cbe668360 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 >> 172.21.15.97:0/1626254604 conn(0x555cbe5400d0 msgr2=0x555cbe5b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 shutdown_connections 2024-09-10T21:33:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.571+0000 7f18f7455080 1 -- 172.21.15.97:0/1626254604 wait complete. 2024-09-10T21:33:51.576 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2024-09-10T21:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 Processor -- start 2024-09-10T21:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- start start 2024-09-10T21:33:51.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec63cc0 0x55611ebdb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec65830 0x55611ebda4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec691d0 0x55611ed17790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55611eaf9a00 con 0x55611ec691d0 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55611eb57f40 con 0x55611ec63cc0 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55611ec8c440 con 0x55611ec65830 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec691d0 0x55611ed17790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec691d0 0x55611ed17790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37040/0 (socket says 172.21.15.97:37040) 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4b600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec63cc0 0x55611ebdb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 -- 172.21.15.97:0/56399798 learned_addr learned my addr 172.21.15.97:0/56399798 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4ac00640 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec65830 0x55611ebda4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 -- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec65830 msgr2=0x55611ebda4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec65830 0x55611ebda4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 -- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec63cc0 msgr2=0x55611ebdb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec63cc0 0x55611ebdb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 -- 172.21.15.97:0/56399798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611ead2c10 con 0x55611ec691d0 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4b600640 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec63cc0 0x55611ebdb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4c000640 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec691d0 0x55611ed17790 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fdb440016d0 tx=0x7fdb44006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1351023b898f6d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb49800640 1 -- 172.21.15.97:0/56399798 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44016070 con 0x55611ec691d0 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb49800640 1 -- 172.21.15.97:0/56399798 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb4400f800 con 0x55611ec691d0 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb49800640 1 -- 172.21.15.97:0/56399798 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400fc30 con 0x55611ec691d0 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- 172.21.15.97:0/56399798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec691d0 msgr2=0x55611ed17790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec691d0 0x55611ed17790 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fdb440016d0 tx=0x7fdb44006f90 comp rx=0 tx=0).stop 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- 172.21.15.97:0/56399798 shutdown_connections 2024-09-10T21:33:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec691d0 0x55611ed17790 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec65830 0x55611ebda4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/56399798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec63cc0 0x55611ebdb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- 172.21.15.97:0/56399798 >> 172.21.15.97:0/56399798 conn(0x55611ebf30d0 msgr2=0x55611ebd9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- 172.21.15.97:0/56399798 shutdown_connections 2024-09-10T21:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.667+0000 7fdb4e38c080 1 -- 172.21.15.97:0/56399798 wait complete. 2024-09-10T21:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 Processor -- start 2024-09-10T21:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 -- start start 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec63cc0 0x55611ebaba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec65830 0x55611ec61c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec691d0 0x55611ec82110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55611ec67030 con 0x55611ec63cc0 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55611ec66d90 con 0x55611ec65830 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55611ec672d0 con 0x55611ec691d0 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec63cc0 0x55611ebaba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec63cc0 0x55611ebaba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37056/0 (socket says 172.21.15.97:37056) 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 -- 172.21.15.97:0/3734639188 learned_addr learned my addr 172.21.15.97:0/3734639188 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4ac00640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec65830 0x55611ec61c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4c000640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec691d0 0x55611ec82110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 -- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec691d0 msgr2=0x55611ec82110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55611ec691d0 0x55611ec82110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 -- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec65830 msgr2=0x55611ec61c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55611ec65830 0x55611ec61c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 -- 172.21.15.97:0/3734639188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55611ebabfb0 con 0x55611ec63cc0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4b600640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec63cc0 0x55611ebaba70 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00a6f0 tx=0x7fdb3c00abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb43e00640 1 -- 172.21.15.97:0/3734639188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c009bc0 con 0x55611ec63cc0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb43e00640 1 -- 172.21.15.97:0/3734639188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb3c009d60 con 0x55611ec63cc0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55611ebaab50 con 0x55611ec63cc0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb43e00640 1 -- 172.21.15.97:0/3734639188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c008690 con 0x55611ec63cc0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.671+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55611ebab090 con 0x55611ec63cc0 2024-09-10T21:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb43e00640 1 -- 172.21.15.97:0/3734639188 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb3c008830 con 0x55611ec63cc0 2024-09-10T21:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb43e00640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb2c070ff0 0x7fdb2c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb43e00640 1 -- 172.21.15.97:0/3734639188 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7fdb3c093bd0 con 0x55611ec63cc0 2024-09-10T21:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55611ec621b0 0x55611ec7daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55611ed11440 con 0x55611ec621b0 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb4ac00640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb2c070ff0 0x7fdb2c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb4c000640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55611ec621b0 0x55611ec7daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb4c000640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55611ec621b0 0x55611ec7daa0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.675+0000 7fdb4ac00640 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb2c070ff0 0x7fdb2c0734b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdb380ab000 tx=0x7fdb38019a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4c000640 1 -- 172.21.15.97:0/3734639188 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v124'283 uv283 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdb4400f070 con 0x55611ec621b0 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55611ec621b0 msgr2=0x55611ec7daa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55611ec621b0 0x55611ec7daa0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb2c070ff0 msgr2=0x7fdb2c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdb2c070ff0 0x7fdb2c0734b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdb380ab000 tx=0x7fdb38019a00 comp rx=0 tx=0).stop 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec63cc0 msgr2=0x55611ebaba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 --2- 172.21.15.97:0/3734639188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55611ec63cc0 0x55611ebaba70 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00a6f0 tx=0x7fdb3c00abb0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4c000640 1 -- 172.21.15.97:0/3734639188 reap_dead start 2024-09-10T21:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 shutdown_connections 2024-09-10T21:33:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 >> 172.21.15.97:0/3734639188 conn(0x55611ebf30d0 msgr2=0x55611ebd9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 shutdown_connections 2024-09-10T21:33:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.679+0000 7fdb4e38c080 1 -- 172.21.15.97:0/3734639188 wait complete. 2024-09-10T21:33:51.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:51.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.871+0000 7f787b400640 1 -- 172.21.15.97:0/2394831541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f787415f350 msgr2=0x7f78740b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.871+0000 7f787b400640 1 --2- 172.21.15.97:0/2394831541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f787415f350 0x7f78740b3070 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f78740b1210 tx=0x7f786401c830 comp rx=0 tx=0).stop 2024-09-10T21:33:51.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.871+0000 7f787b400640 1 -- 172.21.15.97:0/2394831541 shutdown_connections 2024-09-10T21:33:51.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.871+0000 7f787b400640 1 --2- 172.21.15.97:0/2394831541 >> v2:172.21.15.194:3300/0 conn(0x7f78740d05e0 0x7f78740cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 --2- 172.21.15.97:0/2394831541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f787415f350 0x7f78740b3070 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- 172.21.15.97:0/2394831541 >> 172.21.15.97:0/2394831541 conn(0x7f78740d1c60 msgr2=0x7f78740d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:51.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- 172.21.15.97:0/2394831541 shutdown_connections 2024-09-10T21:33:51.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- 172.21.15.97:0/2394831541 wait complete. 2024-09-10T21:33:51.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 Processor -- start 2024-09-10T21:33:51.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- start start 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78740cff80 0x7f78741f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740d05e0 0x7f78741663b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f787415f350 0x7f787415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7874169020 con 0x7f78740d05e0 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7874168ea0 con 0x7f787415f350 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f78741691a0 con 0x7f78740cff80 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740d05e0 0x7f78741663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740d05e0 0x7f78741663b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37082/0 (socket says 172.21.15.97:37082) 2024-09-10T21:33:51.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 -- 172.21.15.97:0/3974768192 learned_addr learned my addr 172.21.15.97:0/3974768192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f7879400640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f787415f350 0x7f787415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f7878a00640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78740cff80 0x7f78741f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 -- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78740cff80 msgr2=0x7f78741f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78740cff80 0x7f78741f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 -- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f787415f350 msgr2=0x7f787415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f787415f350 0x7f787415a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 -- 172.21.15.97:0/3974768192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78741f8880 con 0x7f78740d05e0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f786be00640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740d05e0 0x7f78741663b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f7871600640 1 -- 172.21.15.97:0/3974768192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c00cd80 con 0x7f78740d05e0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f7871600640 1 -- 172.21.15.97:0/3974768192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f785c010430 con 0x7f78740d05e0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f7871600640 1 -- 172.21.15.97:0/3974768192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c010860 con 0x7f78740d05e0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.875+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78741f8b60 con 0x7f78740d05e0 2024-09-10T21:33:51.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.879+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741f9120 con 0x7f78740d05e0 2024-09-10T21:33:51.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.879+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78740cfa40 con 0x7f78740d05e0 2024-09-10T21:33:51.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.879+0000 7f7871600640 1 -- 172.21.15.97:0/3974768192 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f785c010a00 con 0x7f78740d05e0 2024-09-10T21:33:51.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.879+0000 7f7871600640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78500725e0 0x7f7850074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:51.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.879+0000 7f7871600640 1 -- 172.21.15.97:0/3974768192 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8683+0+0 (secure 0 0 0) 0x7f785c0949c0 con 0x7f78740d05e0 2024-09-10T21:33:51.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.879+0000 7f7878a00640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78500725e0 0x7f7850074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:51.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.883+0000 7f7878a00640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78500725e0 0x7f7850074aa0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f78640af000 tx=0x7f78640a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:51.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:51.883+0000 7f7871600640 1 -- 172.21.15.97:0/3974768192 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f785c05eec0 con 0x7f78740d05e0 2024-09-10T21:33:52.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.083+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f78740b13a0 con 0x7f78740d05e0 2024-09-10T21:33:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.311+0000 7f7871600640 1 -- 172.21.15.97:0/3974768192 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v253) ==== 162+0+0 (secure 0 0 0) 0x7f785c0638f0 con 0x7f78740d05e0 2024-09-10T21:33:52.310 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:52.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78500725e0 msgr2=0x7f7850074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78500725e0 0x7f7850074aa0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f78640af000 tx=0x7f78640a7040 comp rx=0 tx=0).stop 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740d05e0 msgr2=0x7f78741663b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740d05e0 0x7f78741663b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 shutdown_connections 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78500725e0 0x7f7850074aa0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f787415f350 0x7f787415a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78740d05e0 0x7f78741663b0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 --2- 172.21.15.97:0/3974768192 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78740cff80 0x7f78741f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 >> 172.21.15.97:0/3974768192 conn(0x7f78740d1c60 msgr2=0x7f787406f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 shutdown_connections 2024-09-10T21:33:52.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.315+0000 7f787b400640 1 -- 172.21.15.97:0/3974768192 wait complete. 2024-09-10T21:33:52.318 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:33:52.319+0000 7f9bd5200640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-10T21:33:52.319 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:52.319+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:52.319 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:52.319+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:52.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- 172.21.15.97:0/3594247024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55015eed0 msgr2=0x7ff5500d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 --2- 172.21.15.97:0/3594247024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55015eed0 0x7ff5500d27f0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff5500d15e0 tx=0x7ff54001c800 comp rx=0 tx=0).stop 2024-09-10T21:33:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- 172.21.15.97:0/3594247024 shutdown_connections 2024-09-10T21:33:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 --2- 172.21.15.97:0/3594247024 >> v2:172.21.15.194:3300/0 conn(0x7ff5500cdd40 0x7ff5500cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 --2- 172.21.15.97:0/3594247024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff55015eed0 0x7ff5500d27f0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- 172.21.15.97:0/3594247024 >> 172.21.15.97:0/3594247024 conn(0x7ff5500d1cd0 msgr2=0x7ff5500d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- 172.21.15.97:0/3594247024 shutdown_connections 2024-09-10T21:33:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- 172.21.15.97:0/3594247024 wait complete. 2024-09-10T21:33:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 Processor -- start 2024-09-10T21:33:52.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- start start 2024-09-10T21:33:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500cca90 0x7ff5501f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5500cdd40 0x7ff55014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff55015eed0 0x7ff5501f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff550168c50 con 0x7ff5500cca90 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff550168ad0 con 0x7ff55015eed0 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff557200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff550168dd0 con 0x7ff5500cdd40 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500cca90 0x7ff5501f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500cca90 0x7ff5501f95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37096/0 (socket says 172.21.15.97:37096) 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 -- 172.21.15.97:0/706140029 learned_addr learned my addr 172.21.15.97:0/706140029 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff547400640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5500cdd40 0x7ff55014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff555200640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff55015eed0 0x7ff5501f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 -- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5500cdd40 msgr2=0x7ff55014b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5500cdd40 0x7ff55014b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 -- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff55015eed0 msgr2=0x7ff5501f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff55015eed0 0x7ff5501f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.483+0000 7ff54fe00640 1 -- 172.21.15.97:0/706140029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5500b64b0 con 0x7ff5500cca90 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff54fe00640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500cca90 0x7ff5501f95a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff540000c10 tx=0x7ff540004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff54cc00640 1 -- 172.21.15.97:0/706140029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400bf070 con 0x7ff5500cca90 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff54cc00640 1 -- 172.21.15.97:0/706140029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff540004320 con 0x7ff5500cca90 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5500b6790 con 0x7ff5500cca90 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff54cc00640 1 -- 172.21.15.97:0/706140029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400044f0 con 0x7ff5500cca90 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5501f9080 con 0x7ff5500cca90 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5500cc550 con 0x7ff5500cca90 2024-09-10T21:33:52.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff54cc00640 1 -- 172.21.15.97:0/706140029 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff540005290 con 0x7ff5500cca90 2024-09-10T21:33:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff54cc00640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff52c0725e0 0x7ff52c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff54cc00640 1 -- 172.21.15.97:0/706140029 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7ff54012ddd0 con 0x7ff5500cca90 2024-09-10T21:33:52.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.487+0000 7ff547400640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff52c0725e0 0x7ff52c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.491+0000 7ff547400640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff52c0725e0 0x7ff52c074aa0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff550153a90 tx=0x7ff53800b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.491+0000 7ff54cc00640 1 -- 172.21.15.97:0/706140029 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff5400163f0 con 0x7ff5500cca90 2024-09-10T21:33:52.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.691+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5500cfd60 con 0x7ff5500cca90 2024-09-10T21:33:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.691+0000 7ff54cc00640 1 -- 172.21.15.97:0/706140029 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 255 v255) ==== 95+0+3569 (secure 0 0 0) 0x7ff5400f81b0 con 0x7ff5500cca90 2024-09-10T21:33:52.691 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:52.691 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":255,"btime":"2024-09-10T21:33:52:323424+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":255,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:52.323367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 255 2024-09-10T21:33:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff52c0725e0 msgr2=0x7ff52c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff52c0725e0 0x7ff52c074aa0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff550153a90 tx=0x7ff53800b390 comp rx=0 tx=0).stop 2024-09-10T21:33:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500cca90 msgr2=0x7ff5501f95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500cca90 0x7ff5501f95a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff540000c10 tx=0x7ff540004040 comp rx=0 tx=0).stop 2024-09-10T21:33:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 shutdown_connections 2024-09-10T21:33:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff52c0725e0 0x7ff52c074aa0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff55015eed0 0x7ff5501f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5500cdd40 0x7ff55014b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 --2- 172.21.15.97:0/706140029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5500cca90 0x7ff5501f95a0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 >> 172.21.15.97:0/706140029 conn(0x7ff5500d1cd0 msgr2=0x7ff5500b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 shutdown_connections 2024-09-10T21:33:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.695+0000 7ff557200640 1 -- 172.21.15.97:0/706140029 wait complete. 2024-09-10T21:33:52.713 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:52.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 -- 172.21.15.97:0/1285697499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 msgr2=0x7fea440cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 --2- 172.21.15.97:0/1285697499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea440cf7f0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fea4406d940 tx=0x7fea3401c770 comp rx=0 tx=0).stop 2024-09-10T21:33:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 -- 172.21.15.97:0/1285697499 shutdown_connections 2024-09-10T21:33:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 --2- 172.21.15.97:0/1285697499 >> v2:172.21.15.194:3300/0 conn(0x7fea4415f2c0 0x7fea440b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 --2- 172.21.15.97:0/1285697499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea440cf7f0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 -- 172.21.15.97:0/1285697499 >> 172.21.15.97:0/1285697499 conn(0x7fea440b3580 msgr2=0x7fea440b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 -- 172.21.15.97:0/1285697499 shutdown_connections 2024-09-10T21:33:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 -- 172.21.15.97:0/1285697499 wait complete. 2024-09-10T21:33:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 Processor -- start 2024-09-10T21:33:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.855+0000 7fea43200640 1 -- start start 2024-09-10T21:33:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea440b40a0 0x7fea44151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea4414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea4415f2c0 0x7fea4414e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea44168b60 con 0x7fea4415eee0 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea441689e0 con 0x7fea440b40a0 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fea44168ce0 con 0x7fea4415f2c0 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea4414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea440b40a0 0x7fea44151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea4414dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37106/0 (socket says 172.21.15.97:37106) 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41400640 1 -- 172.21.15.97:0/856349696 learned_addr learned my addr 172.21.15.97:0/856349696 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea42800640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea4415f2c0 0x7fea4414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41e00640 1 -- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea4415f2c0 msgr2=0x7fea4414e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41e00640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea4415f2c0 0x7fea4414e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41e00640 1 -- 172.21.15.97:0/856349696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 msgr2=0x7fea4414dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41e00640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea4414dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41e00640 1 -- 172.21.15.97:0/856349696 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4406f510 con 0x7fea440b40a0 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41400640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea4414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea41e00640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea440b40a0 0x7fea44151090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea340ac000 tx=0x7fea340a5560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea3a000640 1 -- 172.21.15.97:0/856349696 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340bb070 con 0x7fea440b40a0 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea3a000640 1 -- 172.21.15.97:0/856349696 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea34004490 con 0x7fea440b40a0 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4406f7f0 con 0x7fea440b40a0 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea3a000640 1 -- 172.21.15.97:0/856349696 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340b2040 con 0x7fea440b40a0 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441515d0 con 0x7fea440b40a0 2024-09-10T21:33:52.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.859+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4415ca00 con 0x7fea440b40a0 2024-09-10T21:33:52.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.863+0000 7fea3a000640 1 -- 172.21.15.97:0/856349696 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fea340b5030 con 0x7fea440b40a0 2024-09-10T21:33:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.863+0000 7fea3a000640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea240725e0 0x7fea24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.863+0000 7fea3a000640 1 -- 172.21.15.97:0/856349696 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7fea3412d2a0 con 0x7fea440b40a0 2024-09-10T21:33:52.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.863+0000 7fea41400640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea240725e0 0x7fea24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.863+0000 7fea41400640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea240725e0 0x7fea24074aa0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fea4415c200 tx=0x7fea300018d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:52.867+0000 7fea3a000640 1 -- 172.21.15.97:0/856349696 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea340163f0 con 0x7fea440b40a0 2024-09-10T21:33:53.054 INFO:tasks.ceph.mds.c.smithi097.stderr: -44> 2024-09-10T21:33:48.299+0000 7f886ba00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-10T21:33:53.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.063+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea440b1240 con 0x7fea440b40a0 2024-09-10T21:33:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.063+0000 7fea3a000640 1 -- 172.21.15.97:0/856349696 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 255 v255) ==== 95+0+3569 (secure 0 0 0) 0x7fea340f77f0 con 0x7fea440b40a0 2024-09-10T21:33:53.063 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:53.063 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":255,"btime":"2024-09-10T21:33:52:323424+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":255,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:52.323367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:53.063 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 255 2024-09-10T21:33:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea240725e0 msgr2=0x7fea24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea240725e0 0x7fea24074aa0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fea4415c200 tx=0x7fea300018d0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea440b40a0 msgr2=0x7fea44151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea440b40a0 0x7fea44151090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea340ac000 tx=0x7fea340a5560 comp rx=0 tx=0).stop 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 shutdown_connections 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fea240725e0 0x7fea24074aa0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fea4415f2c0 0x7fea4414e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea4415eee0 0x7fea4414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 --2- 172.21.15.97:0/856349696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea440b40a0 0x7fea44151090 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 >> 172.21.15.97:0/856349696 conn(0x7fea440b3580 msgr2=0x7fea44151f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 shutdown_connections 2024-09-10T21:33:53.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.067+0000 7fea43200640 1 -- 172.21.15.97:0/856349696 wait complete. 2024-09-10T21:33:53.085 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:53.086 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:53.092 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 -- 172.21.15.97:0/2435954686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300cff80 msgr2=0x7ff3300b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 --2- 172.21.15.97:0/2435954686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300cff80 0x7ff3300b15c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff3300736b0 tx=0x7ff32801c570 comp rx=0 tx=0).stop 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 -- 172.21.15.97:0/2435954686 shutdown_connections 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 --2- 172.21.15.97:0/2435954686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300cff80 0x7ff3300b15c0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 --2- 172.21.15.97:0/2435954686 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff3300d05e0 0x7ff3300cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 --2- 172.21.15.97:0/2435954686 >> v2:172.21.15.194:3300/0 conn(0x7ff33015f350 0x7ff3300b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 -- 172.21.15.97:0/2435954686 >> 172.21.15.97:0/2435954686 conn(0x7ff3300d1c60 msgr2=0x7ff3300d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 -- 172.21.15.97:0/2435954686 shutdown_connections 2024-09-10T21:33:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.231+0000 7ff336000640 1 -- 172.21.15.97:0/2435954686 wait complete. 2024-09-10T21:33:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 Processor -- start 2024-09-10T21:33:53.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 -- start start 2024-09-10T21:33:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3300cff80 0x7ff33014d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300d05e0 0x7ff33014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff33015f350 0x7ff330164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff3300b03e0 con 0x7ff3300d05e0 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff3300b0260 con 0x7ff3300cff80 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff3300b0560 con 0x7ff33015f350 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300d05e0 0x7ff33014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300d05e0 0x7ff33014d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37128/0 (socket says 172.21.15.97:37128) 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 -- 172.21.15.97:0/168134537 learned_addr learned my addr 172.21.15.97:0/168134537 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32f400640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3300cff80 0x7ff33014d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32fe00640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff33015f350 0x7ff330164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 -- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff33015f350 msgr2=0x7ff330164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff33015f350 0x7ff330164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 -- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3300cff80 msgr2=0x7ff33014d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3300cff80 0x7ff33014d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 -- 172.21.15.97:0/168134537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3301661e0 con 0x7ff3300d05e0 2024-09-10T21:33:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff32ea00640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300d05e0 0x7ff33014d350 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00e9f0 tx=0x7ff31c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff327400640 1 -- 172.21.15.97:0/168134537 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c00cd80 con 0x7ff3300d05e0 2024-09-10T21:33:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff327400640 1 -- 172.21.15.97:0/168134537 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff31c010430 con 0x7ff3300d05e0 2024-09-10T21:33:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff327400640 1 -- 172.21.15.97:0/168134537 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c010860 con 0x7ff3300d05e0 2024-09-10T21:33:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3301664c0 con 0x7ff3300d05e0 2024-09-10T21:33:53.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff330165420 con 0x7ff3300d05e0 2024-09-10T21:33:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.235+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3300cfa40 con 0x7ff3300d05e0 2024-09-10T21:33:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.239+0000 7ff327400640 1 -- 172.21.15.97:0/168134537 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff31c010a00 con 0x7ff3300d05e0 2024-09-10T21:33:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.239+0000 7ff327400640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff3100725e0 0x7ff310074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.239+0000 7ff327400640 1 -- 172.21.15.97:0/168134537 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7ff31c0949e0 con 0x7ff3300d05e0 2024-09-10T21:33:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.239+0000 7ff32f400640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff3100725e0 0x7ff310074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.239+0000 7ff32f400640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff3100725e0 0x7ff310074aa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff320009e40 tx=0x7ff320009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:53.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.243+0000 7ff327400640 1 -- 172.21.15.97:0/168134537 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff31c05ee80 con 0x7ff3300d05e0 2024-09-10T21:33:53.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.443+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3300b13a0 con 0x7ff3300d05e0 2024-09-10T21:33:53.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.443+0000 7ff327400640 1 -- 172.21.15.97:0/168134537 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 256 v256) ==== 95+0+4395 (secure 0 0 0) 0x7ff31c0638b0 con 0x7ff3300d05e0 2024-09-10T21:33:53.442 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:53.443 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":256,"btime":"2024-09-10T21:33:53:325105+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3334191869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3334191869},{"type":"v1","addr":"172.21.15.97:6835","nonce":3334191869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":256}],"filesystems":[{"mdsmap":{"epoch":255,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:52.323367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:53.443 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 256 2024-09-10T21:33:53.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff3100725e0 msgr2=0x7ff310074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff3100725e0 0x7ff310074aa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff320009e40 tx=0x7ff320009500 comp rx=0 tx=0).stop 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300d05e0 msgr2=0x7ff33014d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300d05e0 0x7ff33014d350 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00e9f0 tx=0x7ff31c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 shutdown_connections 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff3100725e0 0x7ff310074aa0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff33015f350 0x7ff330164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3300d05e0 0x7ff33014d350 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 --2- 172.21.15.97:0/168134537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3300cff80 0x7ff33014d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 >> 172.21.15.97:0/168134537 conn(0x7ff3300d1c60 msgr2=0x7ff3300cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 shutdown_connections 2024-09-10T21:33:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.447+0000 7ff336000640 1 -- 172.21.15.97:0/168134537 wait complete. 2024-09-10T21:33:53.463 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2024-09-10T21:33:53.463 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2024-09-10T21:33:53.463 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:53.463 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:53.512 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:53.512 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:53.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:53.512 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:53.563 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:53.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:53.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 -- 172.21.15.97:0/1605858778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 msgr2=0x7f5c3c0d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 --2- 172.21.15.97:0/1605858778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 0x7f5c3c0d2610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009900 tx=0x7f5c2801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 -- 172.21.15.97:0/1605858778 shutdown_connections 2024-09-10T21:33:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 --2- 172.21.15.97:0/1605858778 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 0x7f5c3c0d2610 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 --2- 172.21.15.97:0/1605858778 >> v2:172.21.15.194:3300/0 conn(0x7f5c3c15f0a0 0x7f5c3c0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 -- 172.21.15.97:0/1605858778 >> 172.21.15.97:0/1605858778 conn(0x7f5c3c0d1af0 msgr2=0x7f5c3c0d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 -- 172.21.15.97:0/1605858778 shutdown_connections 2024-09-10T21:33:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.755+0000 7f5c3be00640 1 -- 172.21.15.97:0/1605858778 wait complete. 2024-09-10T21:33:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 Processor -- start 2024-09-10T21:33:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 -- start start 2024-09-10T21:33:53.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3c15f0a0 0x7f5c3c1ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 0x7f5c3c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c3c15f960 0x7f5c3c0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c168ca0 con 0x7f5c3c15f0a0 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c0af650 con 0x7f5c3c15f960 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3c168e20 con 0x7f5c3c15f480 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3c15f0a0 0x7f5c3c1ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3c15f0a0 0x7f5c3c1ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37134/0 (socket says 172.21.15.97:37134) 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3aa00640 1 -- 172.21.15.97:0/131372491 learned_addr learned my addr 172.21.15.97:0/131372491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3a000640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 0x7f5c3c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3b400640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c3c15f960 0x7f5c3c0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3a000640 1 -- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c3c15f960 msgr2=0x7f5c3c0b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3a000640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c3c15f960 0x7f5c3c0b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3a000640 1 -- 172.21.15.97:0/131372491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3c15f0a0 msgr2=0x7f5c3c1ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3a000640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3c15f0a0 0x7f5c3c1ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3a000640 1 -- 172.21.15.97:0/131372491 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c165330 con 0x7f5c3c15f480 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3a000640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 0x7f5c3c0cce70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ac000 tx=0x7f5c280a55a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c32a00640 1 -- 172.21.15.97:0/131372491 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280b2040 con 0x7f5c3c15f480 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c32a00640 1 -- 172.21.15.97:0/131372491 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c280042e0 con 0x7f5c3c15f480 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c32a00640 1 -- 172.21.15.97:0/131372491 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28005290 con 0x7f5c3c15f480 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c1ff9a0 con 0x7f5c3c15f480 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c0cc670 con 0x7f5c3c15f480 2024-09-10T21:33:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.759+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c3c0b7a70 con 0x7f5c3c15f480 2024-09-10T21:33:53.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.763+0000 7f5c32a00640 1 -- 172.21.15.97:0/131372491 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5c28005430 con 0x7f5c3c15f480 2024-09-10T21:33:53.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.763+0000 7f5c32a00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c1c0725e0 0x7f5c1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.763+0000 7f5c32a00640 1 -- 172.21.15.97:0/131372491 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7f5c2812d7c0 con 0x7f5c3c15f480 2024-09-10T21:33:53.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.763+0000 7f5c3aa00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c1c0725e0 0x7f5c1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.763+0000 7f5c3aa00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c1c0725e0 0x7f5c1c074aa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009600 tx=0x7f5c2c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:53.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.767+0000 7f5c32a00640 1 -- 172.21.15.97:0/131372491 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c280164d0 con 0x7f5c3c15f480 2024-09-10T21:33:53.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:53.967+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5c3c0ccb00 con 0x7f5c3c15f480 2024-09-10T21:33:54.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c32a00640 1 -- 172.21.15.97:0/131372491 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v257) ==== 161+0+0 (secure 0 0 0) 0x7f5c280f7cb0 con 0x7f5c3c15f480 2024-09-10T21:33:54.333 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c1c0725e0 msgr2=0x7f5c1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c1c0725e0 0x7f5c1c074aa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009600 tx=0x7f5c2c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 msgr2=0x7f5c3c0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 0x7f5c3c0cce70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ac000 tx=0x7f5c280a55a0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 shutdown_connections 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5c1c0725e0 0x7f5c1c074aa0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c3c15f960 0x7f5c3c0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5c3c15f480 0x7f5c3c0cce70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 --2- 172.21.15.97:0/131372491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c3c15f0a0 0x7f5c3c1ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 >> 172.21.15.97:0/131372491 conn(0x7f5c3c0d1af0 msgr2=0x7f5c3c15ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 shutdown_connections 2024-09-10T21:33:54.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.335+0000 7f5c3be00640 1 -- 172.21.15.97:0/131372491 wait complete. 2024-09-10T21:33:54.356 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 -- 172.21.15.97:0/2469437669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040afdd0 msgr2=0x7f46040cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 --2- 172.21.15.97:0/2469437669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040afdd0 0x7f46040cc6d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009870 tx=0x7f45f001c820 comp rx=0 tx=0).stop 2024-09-10T21:33:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 -- 172.21.15.97:0/2469437669 shutdown_connections 2024-09-10T21:33:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 --2- 172.21.15.97:0/2469437669 >> v2:172.21.15.194:3300/0 conn(0x7f46040b0d50 0x7f46040ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 --2- 172.21.15.97:0/2469437669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040afdd0 0x7f46040cc6d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 -- 172.21.15.97:0/2469437669 >> 172.21.15.97:0/2469437669 conn(0x7f46040d1c30 msgr2=0x7f46040d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 -- 172.21.15.97:0/2469437669 shutdown_connections 2024-09-10T21:33:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.499+0000 7f4609400640 1 -- 172.21.15.97:0/2469437669 wait complete. 2024-09-10T21:33:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 Processor -- start 2024-09-10T21:33:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 -- start start 2024-09-10T21:33:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46040afdd0 0x7f46041582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040b0d50 0x7f46040b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46040ce640 0x7f46041588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4604168ee0 con 0x7f46040ce640 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4604168d60 con 0x7f46040afdd0 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4604169060 con 0x7f46040b0d50 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46040ce640 0x7f46041588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46040ce640 0x7f46041588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37146/0 (socket says 172.21.15.97:37146) 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 -- 172.21.15.97:0/4228918419 learned_addr learned my addr 172.21.15.97:0/4228918419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4602800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46040afdd0 0x7f46041582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4601e00640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040b0d50 0x7f46040b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 -- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040b0d50 msgr2=0x7f46040b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040b0d50 0x7f46040b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 -- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46040afdd0 msgr2=0x7f46041582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46040afdd0 0x7f46041582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 -- 172.21.15.97:0/4228918419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040c70e0 con 0x7f46040ce640 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4603200640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46040ce640 0x7f46041588e0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00e9a0 tx=0x7f45fc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f45faa00640 1 -- 172.21.15.97:0/4228918419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00cb10 con 0x7f46040ce640 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f45faa00640 1 -- 172.21.15.97:0/4228918419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45fc00ccb0 con 0x7f46040ce640 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f45faa00640 1 -- 172.21.15.97:0/4228918419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc005280 con 0x7f46040ce640 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.503+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46041fb8f0 con 0x7f46040ce640 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.507+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041fbbd0 con 0x7f46040ce640 2024-09-10T21:33:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.507+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46040cea20 con 0x7f46040ce640 2024-09-10T21:33:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.507+0000 7f45faa00640 1 -- 172.21.15.97:0/4228918419 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f45fc0047b0 con 0x7f46040ce640 2024-09-10T21:33:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.507+0000 7f45faa00640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45e40725e0 0x7f45e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.507+0000 7f45faa00640 1 -- 172.21.15.97:0/4228918419 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7f45fc094970 con 0x7f46040ce640 2024-09-10T21:33:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.507+0000 7f4602800640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45e40725e0 0x7f45e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.507+0000 7f4602800640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45e40725e0 0x7f45e4074aa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f45f4004a40 tx=0x7f45f4001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.511+0000 7f45faa00640 1 -- 172.21.15.97:0/4228918419 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f45fc05ed50 con 0x7f46040ce640 2024-09-10T21:33:54.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:54.735+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f460415c6c0 con 0x7f46040ce640 2024-09-10T21:33:55.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.331+0000 7f45faa00640 1 -- 172.21.15.97:0/4228918419 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v258) ==== 101+0+0 (secure 0 0 0) 0x7f45fc063780 con 0x7f46040ce640 2024-09-10T21:33:55.332 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45e40725e0 msgr2=0x7f45e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45e40725e0 0x7f45e4074aa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f45f4004a40 tx=0x7f45f4001a80 comp rx=0 tx=0).stop 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46040ce640 msgr2=0x7f46041588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46040ce640 0x7f46041588e0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00e9a0 tx=0x7f45fc00ee60 comp rx=0 tx=0).stop 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 shutdown_connections 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f45e40725e0 0x7f45e4074aa0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46040ce640 0x7f46041588e0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f46040b0d50 0x7f46040b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 --2- 172.21.15.97:0/4228918419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46040afdd0 0x7f46041582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 >> 172.21.15.97:0/4228918419 conn(0x7f46040d1c30 msgr2=0x7f46040d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:55.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 shutdown_connections 2024-09-10T21:33:55.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.335+0000 7f4609400640 1 -- 172.21.15.97:0/4228918419 wait complete. 2024-09-10T21:33:55.357 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 Processor -- start 2024-09-10T21:33:55.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 -- start start 2024-09-10T21:33:55.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e96080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5e965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f241d0 0x55b5d5f39d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b5d5db4a00 con 0x55b5d5f241d0 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b5d5e12f40 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b09a4d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b5d5f22710 con 0x55b5d5f20a00 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b06a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f241d0 0x55b5d5f39d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b06a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f241d0 0x55b5d5f39d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37158/0 (socket says 172.21.15.97:37158) 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.403+0000 7f4b06a00640 1 -- 172.21.15.97:0/2125317669 learned_addr learned my addr 172.21.15.97:0/2125317669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b05600640 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5e965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e96080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06a00640 1 -- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 msgr2=0x55b5d5e965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06a00640 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5e965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06a00640 1 -- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f1ecc0 msgr2=0x55b5d5e96080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06a00640 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e96080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06a00640 1 -- 172.21.15.97:0/2125317669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d5d8dc10 con 0x55b5d5f241d0 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e96080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b05600640 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5e965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06a00640 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f241d0 0x55b5d5f39d40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000a550 tx=0x7f4b00018300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ccfa0cb935ff69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4affe00640 1 -- 172.21.15.97:0/2125317669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000a8af0 con 0x55b5d5f241d0 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4affe00640 1 -- 172.21.15.97:0/2125317669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b000a8d80 con 0x55b5d5f241d0 2024-09-10T21:33:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4affe00640 1 -- 172.21.15.97:0/2125317669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000bc680 con 0x55b5d5f241d0 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- 172.21.15.97:0/2125317669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f241d0 msgr2=0x55b5d5f39d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f241d0 0x55b5d5f39d40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000a550 tx=0x7f4b00018300 comp rx=0 tx=0).stop 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- 172.21.15.97:0/2125317669 shutdown_connections 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f241d0 0x55b5d5f39d40 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5e965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/2125317669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e96080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- 172.21.15.97:0/2125317669 >> 172.21.15.97:0/2125317669 conn(0x55b5d5eae0d0 msgr2=0x55b5d5e96e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- 172.21.15.97:0/2125317669 shutdown_connections 2024-09-10T21:33:55.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- 172.21.15.97:0/2125317669 wait complete. 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 Processor -- start 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- start start 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e674f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5eac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f241d0 0x55b5d5f1cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b5d5f1e9c0 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b5d5e699b0 con 0x55b5d5f241d0 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b09a4d080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b5d5eb0450 con 0x55b5d5f20a00 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e674f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37172/0 (socket says 172.21.15.97:37172) 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 learned_addr learned my addr 172.21.15.97:0/3163388316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:55.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06a00640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f241d0 0x55b5d5f1cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5eac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 msgr2=0x55b5d5eac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5eac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f241d0 msgr2=0x55b5d5f1cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b5d5f241d0 0x55b5d5f1cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d5eaa670 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.407+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b5d5f20a00 0x55b5d5eac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4b06000640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e674f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009480 tx=0x7f4af8009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4afea00640 1 -- 172.21.15.97:0/3163388316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80087d0 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4afea00640 1 -- 172.21.15.97:0/3163388316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af8008970 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b5d5eaa910 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4afea00640 1 -- 172.21.15.97:0/3163388316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8008dc0 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d5eaae50 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4afea00640 1 -- 172.21.15.97:0/3163388316 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4af801e440 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4afea00640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ae8070ff0 0x7f4ae80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4afea00640 1 -- 172.21.15.97:0/3163388316 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7f4af8092c10 con 0x55b5d5f1ecc0 2024-09-10T21:33:55.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ae8070ff0 0x7f4ae80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.413 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:55.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b5d5f3a850 0x55b5d5f1bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.411+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5e67a30 con 0x55b5d5f3a850 2024-09-10T21:33:55.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.415+0000 7f4b06a00640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b5d5f3a850 0x55b5d5f1bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.415+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ae8070ff0 0x7f4ae80734b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4af4000f70 tx=0x7f4af4002630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.415+0000 7f4b06a00640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b5d5f3a850 0x55b5d5f1bf20 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.415+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v125'877 uv851 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.415+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5eaceb0 con 0x55b5d5f3a850 2024-09-10T21:33:55.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.419+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v125'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.419+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5f1c460 con 0x55b5d5f3a850 2024-09-10T21:33:55.419 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.419+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v125'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.419+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe3830 con 0x55b5d5f3a850 2024-09-10T21:33:55.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.423+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.423+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe3bd0 con 0x55b5d5f3a850 2024-09-10T21:33:55.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.423+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.423+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea7be0 con 0x55b5d5f3a850 2024-09-10T21:33:55.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.423+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.423+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea7f80 con 0x55b5d5f3a850 2024-09-10T21:33:55.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.427+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v125'883 uv857 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.427+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.427+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v125'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.427+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.429 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:55.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.431+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v125'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.431+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.431+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.431+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.435+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.435+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.435+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'888 uv888 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.435+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.435+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v125'889 uv864 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.435+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.439+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'890 uv890 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.439+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.438 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:55.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.439+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v125'891 uv891 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.439+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.439+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'892 uv892 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.439+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.443+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'893 uv893 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.443+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea8320 con 0x55b5d5f3a850 2024-09-10T21:33:55.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.443+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'894 uv894 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.443+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.447+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'895 uv895 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.447+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.447+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v125'896 uv870 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.447+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.451+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v125'897 uv897 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.451+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.450 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:55.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.451+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v125'898 uv898 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.451+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.451+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'899 uv899 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.451+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.455+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'900 uv900 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.455+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.455+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'901 uv901 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.455+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.459+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v125'902 uv876 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.459+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.459+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v125'903 uv903 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.459+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.459+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v125'904 uv904 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.459 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:55.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.459+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.463+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'905 uv905 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.463+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.463+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'906 uv906 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.463+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ea85a0 con 0x55b5d5f3a850 2024-09-10T21:33:55.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.467+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'907 uv907 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5f3a850 2024-09-10T21:33:55.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.467+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b5d5fe3f70 0x55b5d5fe6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.467+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe68a0 con 0x55b5d5fe3f70 2024-09-10T21:33:55.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.467+0000 7f4b06000640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b5d5fe3f70 0x55b5d5fe6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.467+0000 7f4b06000640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b5d5fe3f70 0x55b5d5fe6360 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.471+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'638 uv621 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.471+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5fe6c40 con 0x55b5d5fe3f70 2024-09-10T21:33:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.471+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'639 uv639 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.471+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe6fe0 con 0x55b5d5fe3f70 2024-09-10T21:33:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.471+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'640 uv640 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.471 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:55.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.471+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7830 con 0x55b5d5fe3f70 2024-09-10T21:33:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.475+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v125'641 uv641 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.475+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7bd0 con 0x55b5d5fe3f70 2024-09-10T21:33:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.475+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'642 uv642 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.475+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.475+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v125'643 uv626 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.475+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.479+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v125'644 uv644 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.479+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.479+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v125'645 uv645 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.479 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:55.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.479+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.483+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'646 uv646 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.483+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.483+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'647 uv647 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.483+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.487+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v125'648 uv631 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.487+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.487+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v125'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.487+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.486 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:55.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.487+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v125'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.487+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fec140 con 0x55b5d5fe3f70 2024-09-10T21:33:55.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.491+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v125'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.491+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fec140 con 0x55b5d5fe3f70 2024-09-10T21:33:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.491+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.491+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.491+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v125'653 uv637 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.491+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.495+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v125'654 uv654 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.495+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.495+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v125'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.494 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:55.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.495+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe8f80 con 0x55b5d5fe3f70 2024-09-10T21:33:55.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.495+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.495+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe8f80 con 0x55b5d5fe3f70 2024-09-10T21:33:55.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fe7f70 con 0x55b5d5fe3f70 2024-09-10T21:33:55.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5fe3f70 2024-09-10T21:33:55.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b5d5fe84d0 0x55b5d5fea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5feaf20 con 0x55b5d5fe84d0 2024-09-10T21:33:55.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b5d5fe84d0 0x55b5d5fea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b5d5fe84d0 0x55b5d5fea8c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.499+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5feb770 con 0x55b5d5fe84d0 2024-09-10T21:33:55.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.503+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.503+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5febb10 con 0x55b5d5fe84d0 2024-09-10T21:33:55.502 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:55.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.503+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.503+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fecf00 con 0x55b5d5fe84d0 2024-09-10T21:33:55.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.503+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.503+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fed360 con 0x55b5d5fe84d0 2024-09-10T21:33:55.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.507+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.507+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5febeb0 con 0x55b5d5fe84d0 2024-09-10T21:33:55.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.507+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.507+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b5d5fec9a0 0x55b5d5fef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.507+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5fefd50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.507+0000 7f4b06a00640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b5d5fec9a0 0x55b5d5fef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.507+0000 7f4b06a00640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b5d5fec9a0 0x55b5d5fef7b0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'128 uv127 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff0140 con 0x55b5d5fec9a0 2024-09-10T21:33:55.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0140 con 0x55b5d5fec9a0 2024-09-10T21:33:55.510 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:55.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0180 con 0x55b5d5fec9a0 2024-09-10T21:33:55.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.511+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fe84d0 2024-09-10T21:33:55.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.515+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.515+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fe84d0 2024-09-10T21:33:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.515+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.515+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fe84d0 2024-09-10T21:33:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.515+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.515 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:55.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.515+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff15f0 con 0x55b5d5fe84d0 2024-09-10T21:33:55.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff15f0 con 0x55b5d5fe84d0 2024-09-10T21:33:55.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fe84d0 2024-09-10T21:33:55.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fec9a0 2024-09-10T21:33:55.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v125'316 uv310 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.519+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fec9a0 2024-09-10T21:33:55.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.523+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v125'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.523+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0270 con 0x55b5d5fec9a0 2024-09-10T21:33:55.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.523+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v125'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.522 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:55.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.523+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff12c0 con 0x55b5d5fec9a0 2024-09-10T21:33:55.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.523+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.523+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff1230 con 0x55b5d5fec9a0 2024-09-10T21:33:55.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.527+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.527+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0220 con 0x55b5d5fec9a0 2024-09-10T21:33:55.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.527+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.527+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0220 con 0x55b5d5fec9a0 2024-09-10T21:33:55.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.527+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v125'322 uv315 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.527+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fec9a0 2024-09-10T21:33:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.531+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v125'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.531+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff05e0 con 0x55b5d5fec9a0 2024-09-10T21:33:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.531+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v125'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.529 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:55.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.531+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0220 con 0x55b5d5fec9a0 2024-09-10T21:33:55.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.531+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.531+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0220 con 0x55b5d5fec9a0 2024-09-10T21:33:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0200 con 0x55b5d5fe84d0 2024-09-10T21:33:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v125'143 uv142 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff0200 con 0x55b5d5fe84d0 2024-09-10T21:33:55.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v125'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0200 con 0x55b5d5fe84d0 2024-09-10T21:33:55.535 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v125'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.535+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0220 con 0x55b5d5fe84d0 2024-09-10T21:33:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.539+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.539+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff0220 con 0x55b5d5fe84d0 2024-09-10T21:33:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.539+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'147 uv147 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5fe84d0 2024-09-10T21:33:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.539+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b5d5ff0be0 0x55b5d5ff1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.539+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff36b0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.539+0000 7f4b06000640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b5d5ff0be0 0x55b5d5ff1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.539+0000 7f4b06000640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b5d5ff0be0 0x55b5d5ff1000 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff36b0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff36b0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.542 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:55.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4ab0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.543+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4ab0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.547+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.547+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.547+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.547+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.547+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v125'284 uv283 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.547+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.551+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v125'285 uv285 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.551+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.550 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.551+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v125'286 uv286 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.551+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.551+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'287 uv287 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.551+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.555+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'288 uv288 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.555+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.555+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v125'289 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.555+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.555+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v125'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.555+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.556 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:55.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.559+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v125'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.559+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3aa0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.559+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.559+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff68b0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.559+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.559+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.563+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v125'124 uv123 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.563+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.563+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'125 uv125 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.563+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.563+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v125'126 uv126 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.563 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:55.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.563+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.563+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'127 uv127 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.567+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.567+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v125'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.567+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.567+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v125'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.567+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.567+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v125'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.567+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4f60 con 0x55b5d5fec9a0 2024-09-10T21:33:55.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4f60 con 0x55b5d5fec9a0 2024-09-10T21:33:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff3f50 con 0x55b5d5fec9a0 2024-09-10T21:33:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4b00002050 con 0x55b5d5fec9a0 2024-09-10T21:33:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b5d5ff4460 0x55b5d5ff4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff4460 2024-09-10T21:33:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b5d5ff4460 0x55b5d5ff4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.571+0000 7f4b05600640 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b5d5ff4460 0x55b5d5ff4860 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.575+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v125'145 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5ff4460 2024-09-10T21:33:55.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.575+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff4460 2024-09-10T21:33:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.575+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v125'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5ff4460 2024-09-10T21:33:55.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.575+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff4460 2024-09-10T21:33:55.575 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.575+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v125'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5ff4460 2024-09-10T21:33:55.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.575+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff4460 2024-09-10T21:33:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.579+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5ff4460 2024-09-10T21:33:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.579+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff4460 2024-09-10T21:33:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.579+0000 7f4b05600640 1 -- 172.21.15.97:0/3163388316 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4af400f810 con 0x55b5d5ff4460 2024-09-10T21:33:55.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.579+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.579+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v125'351 uv343 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.579+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.583+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v125'352 uv352 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.583+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.581 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:55.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.583+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v125'353 uv353 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.583+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.583+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v125'354 uv354 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.583+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'355 uv355 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v125'356 uv350 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v125'357 uv357 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v125'358 uv358 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.587 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:55.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.587+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.591+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'359 uv359 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.591+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.591+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'360 uv360 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.591+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff4da0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.591+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'361 uv361 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.591+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55b5d5ff63c0 con 0x55b5d5ff0be0 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b06000640 1 -- 172.21.15.97:0/3163388316 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'362 uv362 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4af801b020 con 0x55b5d5ff0be0 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b5d5fe84d0 msgr2=0x55b5d5fea8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55b5d5fe84d0 0x55b5d5fea8c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b5d5ff0be0 msgr2=0x55b5d5ff1000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55b5d5ff0be0 0x55b5d5ff1000 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b5d5ff4460 msgr2=0x55b5d5ff4860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55b5d5ff4460 0x55b5d5ff4860 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b5d5fec9a0 msgr2=0x55b5d5fef7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55b5d5fec9a0 0x55b5d5fef7b0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b5d5fe3f70 msgr2=0x55b5d5fe6360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 reap_dead start 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55b5d5fe3f70 0x55b5d5fe6360 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b06a00640 1 -- 172.21.15.97:0/3163388316 reap_dead start 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b5d5f3a850 msgr2=0x55b5d5f1bf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b5d5f3a850 0x55b5d5f1bf20 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ae8070ff0 msgr2=0x7f4ae80734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ae8070ff0 0x7f4ae80734b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4af4000f70 tx=0x7f4af4002630 comp rx=0 tx=0).stop 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f1ecc0 msgr2=0x55b5d5e674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e674f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009480 tx=0x7f4af8009940 comp rx=0 tx=0).stop 2024-09-10T21:33:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 shutdown_connections 2024-09-10T21:33:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b5d5f3a850 0x55b5d5f1bf20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ae8070ff0 0x7f4ae80734b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 --2- 172.21.15.97:0/3163388316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b5d5f1ecc0 0x55b5d5e674f0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 >> 172.21.15.97:0/3163388316 conn(0x55b5d5eae0d0 msgr2=0x55b5d5e96dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 shutdown_connections 2024-09-10T21:33:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.595+0000 7f4b09a4d080 1 -- 172.21.15.97:0/3163388316 wait complete. 2024-09-10T21:33:55.597 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2024-09-10T21:33:55.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 Processor -- start 2024-09-10T21:33:55.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 -- start start 2024-09-10T21:33:55.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa95b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563fa95b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa989000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5563fa909f40 con 0x5563faa1a0f0 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5563fa916d00 con 0x5563faa19d10 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda8125e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5563fa8aba00 con 0x5563fa95daa0 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa989000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa989000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37188/0 (socket says 172.21.15.97:37188) 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7ee00640 1 -- 172.21.15.97:0/1833798269 learned_addr learned my addr 172.21.15.97:0/1833798269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7da00640 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563fa95b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7e400640 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa95b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7da00640 1 -- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 msgr2=0x5563fa95b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:55.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7da00640 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa95b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7da00640 1 -- 172.21.15.97:0/1833798269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 msgr2=0x5563fa989000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:55.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7da00640 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa989000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7da00640 1 -- 172.21.15.97:0/1833798269 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563fa98b670 con 0x5563faa19d10 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7ee00640 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa989000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda7da00640 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563fa95b6f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5563fa961a60 tx=0x7fda6c017f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8387108a22b11c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda77e00640 1 -- 172.21.15.97:0/1833798269 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0b0040 con 0x5563faa19d10 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda77e00640 1 -- 172.21.15.97:0/1833798269 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda6c0ba070 con 0x5563faa19d10 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.687+0000 7fda77e00640 1 -- 172.21.15.97:0/1833798269 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0b5410 con 0x5563faa19d10 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 msgr2=0x5563fa95b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563fa95b6f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5563fa961a60 tx=0x7fda6c017f30 comp rx=0 tx=0).stop 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- 172.21.15.97:0/1833798269 shutdown_connections 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa989000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563fa95b6f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 --2- 172.21.15.97:0/1833798269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa95b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- 172.21.15.97:0/1833798269 >> 172.21.15.97:0/1833798269 conn(0x5563fa95c140 msgr2=0x5563faa18f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:55.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- 172.21.15.97:0/1833798269 shutdown_connections 2024-09-10T21:33:55.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- 172.21.15.97:0/1833798269 wait complete. 2024-09-10T21:33:55.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 Processor -- start 2024-09-10T21:33:55.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- start start 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa989960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563faa310e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa98c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5563fa98bbb0 con 0x5563faa1a0f0 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5563fa98b910 con 0x5563faa19d10 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5563fa98be50 con 0x5563fa95daa0 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa989960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa98c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563faa310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563faa310e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:37420/0 (socket says 172.21.15.97:37420) 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 -- 172.21.15.97:0/2086612843 learned_addr learned my addr 172.21.15.97:0/2086612843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa989960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:44760/0 (socket says 172.21.15.97:44760) 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 -- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 msgr2=0x5563fa989960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5563fa95daa0 0x5563fa989960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 -- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 msgr2=0x5563fa98c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa98c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 -- 172.21.15.97:0/2086612843 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563faa2ff50 con 0x5563faa19d10 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7ee00640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5563faa1a0f0 0x5563fa98c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:55.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda7da00640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563faa310e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fda6c004fb0 tx=0x7fda6c0abb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda76a00640 1 -- 172.21.15.97:0/2086612843 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0c6420 con 0x5563faa19d10 2024-09-10T21:33:55.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda76a00640 1 -- 172.21.15.97:0/2086612843 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda6c0c2070 con 0x5563faa19d10 2024-09-10T21:33:55.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda76a00640 1 -- 172.21.15.97:0/2086612843 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0c6910 con 0x5563faa19d10 2024-09-10T21:33:55.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5563faa301f0 con 0x5563faa19d10 2024-09-10T21:33:55.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.691+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563faa30730 con 0x5563faa19d10 2024-09-10T21:33:55.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda76a00640 1 -- 172.21.15.97:0/2086612843 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fda6c0c6ab0 con 0x5563faa19d10 2024-09-10T21:33:55.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda76a00640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda60070ff0 0x7fda600734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda76a00640 1 -- 172.21.15.97:0/2086612843 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7fda6c12eb60 con 0x5563faa19d10 2024-09-10T21:33:55.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda7e400640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda60070ff0 0x7fda600734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda8125e080 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5563faa31620 0x5563faa2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5563faa2f6f0 con 0x5563faa31620 2024-09-10T21:33:55.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda7ee00640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5563faa31620 0x5563faa2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda7e400640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda60070ff0 0x7fda600734b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fda70011610 tx=0x7fda70002b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.695+0000 7fda7ee00640 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5563faa31620 0x5563faa2f1b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda7ee00640 1 -- 172.21.15.97:0/2086612843 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v125'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fda78002e50 con 0x5563faa31620 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5563faa31620 msgr2=0x5563faa2f1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5563faa31620 0x5563faa2f1b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda60070ff0 msgr2=0x7fda600734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda60070ff0 0x7fda600734b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fda70011610 tx=0x7fda70002b80 comp rx=0 tx=0).stop 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 msgr2=0x5563faa310e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 --2- 172.21.15.97:0/2086612843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563faa19d10 0x5563faa310e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fda6c004fb0 tx=0x7fda6c0abb40 comp rx=0 tx=0).stop 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda7ee00640 1 -- 172.21.15.97:0/2086612843 reap_dead start 2024-09-10T21:33:55.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 shutdown_connections 2024-09-10T21:33:55.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 >> 172.21.15.97:0/2086612843 conn(0x5563fa95c140 msgr2=0x5563fa95ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:55.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 shutdown_connections 2024-09-10T21:33:55.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.699+0000 7fda8125e080 1 -- 172.21.15.97:0/2086612843 wait complete. 2024-09-10T21:33:55.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:55.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 -- 172.21.15.97:0/3005502030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 msgr2=0x7f20300a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 --2- 172.21.15.97:0/3005502030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f20300a94a0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f2020009db0 tx=0x7f202001cd40 comp rx=0 tx=0).stop 2024-09-10T21:33:55.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 -- 172.21.15.97:0/3005502030 shutdown_connections 2024-09-10T21:33:55.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 --2- 172.21.15.97:0/3005502030 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20300b4010 0x7f2030074180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 --2- 172.21.15.97:0/3005502030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f20300a94a0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 -- 172.21.15.97:0/3005502030 >> 172.21.15.97:0/3005502030 conn(0x7f20300cf890 msgr2=0x7f20300cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:55.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 -- 172.21.15.97:0/3005502030 shutdown_connections 2024-09-10T21:33:55.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 -- 172.21.15.97:0/3005502030 wait complete. 2024-09-10T21:33:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.878+0000 7f2035a00640 1 Processor -- start 2024-09-10T21:33:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 -- start start 2024-09-10T21:33:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20300ae890 0x7f20301751c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f203020c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20300b4010 0x7f20300aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f203016f6d0 con 0x7f20300b3c30 2024-09-10T21:33:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f203016f550 con 0x7f20300ae890 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f203016f850 con 0x7f20300b4010 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2027e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f203020c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2027e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f203020c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37218/0 (socket says 172.21.15.97:37218) 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2027e00640 1 -- 172.21.15.97:0/2630164482 learned_addr learned my addr 172.21.15.97:0/2630164482 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202fe00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20300ae890 0x7f20301751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035000640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20300b4010 0x7f20300aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202fe00640 1 -- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20300b4010 msgr2=0x7f20300aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202fe00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20300b4010 0x7f20300aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202fe00640 1 -- 172.21.15.97:0/2630164482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 msgr2=0x7f203020c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202fe00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f203020c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202fe00640 1 -- 172.21.15.97:0/2630164482 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20300aa2a0 con 0x7f20300ae890 2024-09-10T21:33:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2027e00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f203020c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202fe00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20300ae890 0x7f20301751c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f20200ac000 tx=0x7f2020004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202cc00640 1 -- 172.21.15.97:0/2630164482 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20200b2040 con 0x7f20300ae890 2024-09-10T21:33:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202cc00640 1 -- 172.21.15.97:0/2630164482 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2020003600 con 0x7f20300ae890 2024-09-10T21:33:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f202cc00640 1 -- 172.21.15.97:0/2630164482 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020003a50 con 0x7f20300ae890 2024-09-10T21:33:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20300aa580 con 0x7f20300ae890 2024-09-10T21:33:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203020bf00 con 0x7f20300ae890 2024-09-10T21:33:55.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.882+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20301780b0 con 0x7f20300ae890 2024-09-10T21:33:55.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.886+0000 7f202cc00640 1 -- 172.21.15.97:0/2630164482 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f20200b5070 con 0x7f20300ae890 2024-09-10T21:33:55.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.886+0000 7f202cc00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f200c0725e0 0x7f200c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:55.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.886+0000 7f202cc00640 1 -- 172.21.15.97:0/2630164482 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8769+0+0 (secure 0 0 0) 0x7f202012d640 con 0x7f20300ae890 2024-09-10T21:33:55.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.886+0000 7f2027e00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f200c0725e0 0x7f200c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:55.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.886+0000 7f2027e00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f200c0725e0 0x7f200c074aa0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f20300ac560 tx=0x7f2018009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:55.890+0000 7f202cc00640 1 -- 172.21.15.97:0/2630164482 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2020016930 con 0x7f20300ae890 2024-09-10T21:33:56.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.086+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f203006e320 con 0x7f20300ae890 2024-09-10T21:33:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f202cc00640 1 -- 172.21.15.97:0/2630164482 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v259) ==== 162+0+0 (secure 0 0 0) 0x7f20200f7ab0 con 0x7f20300ae890 2024-09-10T21:33:56.349 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:33:56.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f200c0725e0 msgr2=0x7f200c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f200c0725e0 0x7f200c074aa0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f20300ac560 tx=0x7f2018009040 comp rx=0 tx=0).stop 2024-09-10T21:33:56.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20300ae890 msgr2=0x7f20301751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20300ae890 0x7f20301751c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f20200ac000 tx=0x7f2020004040 comp rx=0 tx=0).stop 2024-09-10T21:33:56.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 shutdown_connections 2024-09-10T21:33:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f200c0725e0 0x7f200c074aa0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f20300b4010 0x7f20300aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20300b3c30 0x7f203020c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 --2- 172.21.15.97:0/2630164482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20300ae890 0x7f20301751c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 >> 172.21.15.97:0/2630164482 conn(0x7f20300cf890 msgr2=0x7f20300739e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 shutdown_connections 2024-09-10T21:33:56.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.350+0000 7f2035a00640 1 -- 172.21.15.97:0/2630164482 wait complete. 2024-09-10T21:33:56.357 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:33:56.358+0000 7f0250000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T21:33:56.357 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:56.358+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:33:56.358 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:33:56.358+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:33:56.375 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- 172.21.15.97:0/3434694696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c40062e0 msgr2=0x7f32c400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 --2- 172.21.15.97:0/3434694696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32c40062e0 0x7f32c400cae0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f32d8001f50 tx=0x7f32d801cc40 comp rx=0 tx=0).stop 2024-09-10T21:33:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32dfe00640 1 -- 172.21.15.97:0/3434694696 reap_dead start 2024-09-10T21:33:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- 172.21.15.97:0/3434694696 shutdown_connections 2024-09-10T21:33:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- 172.21.15.97:0/3434694696 >> 172.21.15.97:0/3434694696 conn(0x7f32e00b1780 msgr2=0x7f32e00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- 172.21.15.97:0/3434694696 shutdown_connections 2024-09-10T21:33:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- 172.21.15.97:0/3434694696 wait complete. 2024-09-10T21:33:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 Processor -- start 2024-09-10T21:33:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- start start 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32c40062e0 0x7f32e0153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e0139530 0x7f32e015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32e015c040 0x7f32e01e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32e006f060 con 0x7f32e0139530 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32e015b210 con 0x7f32e015c040 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32e6800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f32e006f1e0 con 0x7f32c40062e0 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e0139530 0x7f32e015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e0139530 0x7f32e015bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37234/0 (socket says 172.21.15.97:37234) 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32d7e00640 1 -- 172.21.15.97:0/2412514158 learned_addr learned my addr 172.21.15.97:0/2412514158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32df400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32c40062e0 0x7f32e0153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.518+0000 7f32dfe00640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32e015c040 0x7f32e01e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32df400640 1 -- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32e015c040 msgr2=0x7f32e01e5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32df400640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32e015c040 0x7f32e01e5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32df400640 1 -- 172.21.15.97:0/2412514158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e0139530 msgr2=0x7f32e015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32df400640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e0139530 0x7f32e015bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32df400640 1 -- 172.21.15.97:0/2412514158 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e01f2af0 con 0x7f32c40062e0 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32d7e00640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e0139530 0x7f32e015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32dfe00640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32e015c040 0x7f32e01e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32df400640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32c40062e0 0x7f32e0153080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f32d0006690 tx=0x7f32d00066c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:56.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32d7400640 1 -- 172.21.15.97:0/2412514158 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0015070 con 0x7f32c40062e0 2024-09-10T21:33:56.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32d7400640 1 -- 172.21.15.97:0/2412514158 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32d0011900 con 0x7f32c40062e0 2024-09-10T21:33:56.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e01f2dd0 con 0x7f32c40062e0 2024-09-10T21:33:56.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32d7400640 1 -- 172.21.15.97:0/2412514158 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0011d50 con 0x7f32c40062e0 2024-09-10T21:33:56.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e01f3390 con 0x7f32c40062e0 2024-09-10T21:33:56.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e00cf450 con 0x7f32c40062e0 2024-09-10T21:33:56.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32d7400640 1 -- 172.21.15.97:0/2412514158 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f32d00103c0 con 0x7f32c40062e0 2024-09-10T21:33:56.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32d7400640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32bc0725e0 0x7f32bc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.522+0000 7f32d7400640 1 -- 172.21.15.97:0/2412514158 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7f32d0094d90 con 0x7f32c40062e0 2024-09-10T21:33:56.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.526+0000 7f32d7e00640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32bc0725e0 0x7f32bc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.526+0000 7f32d7e00640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32bc0725e0 0x7f32bc074aa0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009fd0 tx=0x7f32c80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.526+0000 7f32d7400640 1 -- 172.21.15.97:0/2412514158 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f32d005f210 con 0x7f32c40062e0 2024-09-10T21:33:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.726+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32e015c420 con 0x7f32c40062e0 2024-09-10T21:33:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32d7400640 1 -- 172.21.15.97:0/2412514158 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) ==== 95+0+3569 (secure 0 0 0) 0x7f32d0063c40 con 0x7f32c40062e0 2024-09-10T21:33:56.728 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:56.728 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":261,"btime":"2024-09-10T21:33:56:361731+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:56.361652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 261 2024-09-10T21:33:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32bc0725e0 msgr2=0x7f32bc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32bc0725e0 0x7f32bc074aa0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009fd0 tx=0x7f32c80091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32c40062e0 msgr2=0x7f32e0153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32c40062e0 0x7f32e0153080 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f32d0006690 tx=0x7f32d00066c0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 shutdown_connections 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32bc0725e0 0x7f32bc074aa0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32e015c040 0x7f32e01e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32e0139530 0x7f32e015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 --2- 172.21.15.97:0/2412514158 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f32c40062e0 0x7f32e0153080 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 >> 172.21.15.97:0/2412514158 conn(0x7f32e00b1780 msgr2=0x7f32e00b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 shutdown_connections 2024-09-10T21:33:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.730+0000 7f32e6800640 1 -- 172.21.15.97:0/2412514158 wait complete. 2024-09-10T21:33:56.751 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:56.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 -- 172.21.15.97:0/1119538975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 msgr2=0x7f87e40ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 --2- 172.21.15.97:0/1119538975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e40ceee0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f87e4074f10 tx=0x7f87dc01f580 comp rx=0 tx=0).stop 2024-09-10T21:33:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 -- 172.21.15.97:0/1119538975 shutdown_connections 2024-09-10T21:33:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 --2- 172.21.15.97:0/1119538975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e40ceee0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 --2- 172.21.15.97:0/1119538975 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e40b3cf0 0x7f87e406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 --2- 172.21.15.97:0/1119538975 >> v2:172.21.15.194:3300/0 conn(0x7f87e41511f0 0x7f87e40b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 -- 172.21.15.97:0/1119538975 >> 172.21.15.97:0/1119538975 conn(0x7f87e40b1780 msgr2=0x7f87e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:56.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 -- 172.21.15.97:0/1119538975 shutdown_connections 2024-09-10T21:33:56.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 -- 172.21.15.97:0/1119538975 wait complete. 2024-09-10T21:33:56.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 Processor -- start 2024-09-10T21:33:56.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.894+0000 7f87ea200640 1 -- start start 2024-09-10T21:33:56.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e41f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40b3cf0 0x7f87e40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e41511f0 0x7f87e40c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87e415a8b0 con 0x7f87e406f180 2024-09-10T21:33:56.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87e415a730 con 0x7f87e40b3cf0 2024-09-10T21:33:56.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f87e415aa30 con 0x7f87e41511f0 2024-09-10T21:33:56.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e41f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e41f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37244/0 (socket says 172.21.15.97:37244) 2024-09-10T21:33:56.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 -- 172.21.15.97:0/1247027473 learned_addr learned my addr 172.21.15.97:0/1247027473 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:56.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87daa00640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40b3cf0 0x7f87e40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3e00640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e41511f0 0x7f87e40c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 -- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e41511f0 msgr2=0x7f87e40c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e41511f0 0x7f87e40c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 -- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40b3cf0 msgr2=0x7f87e40c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40b3cf0 0x7f87e40c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 -- 172.21.15.97:0/1247027473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e41f45c0 con 0x7f87e406f180 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87e3400640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e41f8000 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c9f0 tx=0x7f87d400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87dbe00640 1 -- 172.21.15.97:0/1247027473 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4007bb0 con 0x7f87e406f180 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87dbe00640 1 -- 172.21.15.97:0/1247027473 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87d4007e40 con 0x7f87e406f180 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e41f48a0 con 0x7f87e406f180 2024-09-10T21:33:56.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87dbe00640 1 -- 172.21.15.97:0/1247027473 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d400f690 con 0x7f87e406f180 2024-09-10T21:33:56.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41f4270 con 0x7f87e406f180 2024-09-10T21:33:56.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e406ec40 con 0x7f87e406f180 2024-09-10T21:33:56.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.898+0000 7f87dbe00640 1 -- 172.21.15.97:0/1247027473 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f87d4020030 con 0x7f87e406f180 2024-09-10T21:33:56.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.902+0000 7f87dbe00640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c00725e0 0x7f87c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:56.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.902+0000 7f87dbe00640 1 -- 172.21.15.97:0/1247027473 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7f87d4094460 con 0x7f87e406f180 2024-09-10T21:33:56.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.902+0000 7f87daa00640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c00725e0 0x7f87c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:56.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.902+0000 7f87daa00640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c00725e0 0x7f87c0074aa0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f87e415cf90 tx=0x7f87cc005db0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:56.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:56.906+0000 7f87dbe00640 1 -- 172.21.15.97:0/1247027473 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f87d4014030 con 0x7f87e406f180 2024-09-10T21:33:57.071 INFO:tasks.ceph.mds.a.smithi097.stderr: -52> 2024-09-10T21:33:52.319+0000 7f9bd5200640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-10T21:33:57.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.106+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87e41f4c30 con 0x7f87e406f180 2024-09-10T21:33:57.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.106+0000 7f87dbe00640 1 -- 172.21.15.97:0/1247027473 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) ==== 95+0+3569 (secure 0 0 0) 0x7f87d405ee40 con 0x7f87e406f180 2024-09-10T21:33:57.106 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:57.106 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":261,"btime":"2024-09-10T21:33:56:361731+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:56.361652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:57.107 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 261 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c00725e0 msgr2=0x7f87c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c00725e0 0x7f87c0074aa0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f87e415cf90 tx=0x7f87cc005db0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 msgr2=0x7f87e41f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e41f8000 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c9f0 tx=0x7f87d400ceb0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 shutdown_connections 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87c00725e0 0x7f87c0074aa0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87e41511f0 0x7f87e40c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87e40b3cf0 0x7f87e40c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 --2- 172.21.15.97:0/1247027473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87e406f180 0x7f87e41f8000 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 >> 172.21.15.97:0/1247027473 conn(0x7f87e40b1780 msgr2=0x7f87e40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 shutdown_connections 2024-09-10T21:33:57.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.110+0000 7f87ea200640 1 -- 172.21.15.97:0/1247027473 wait complete. 2024-09-10T21:33:57.110 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:33:57.129 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:33:57.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:33:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 -- 172.21.15.97:0/1784733527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc888073a70 msgr2=0x7fc8880b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 --2- 172.21.15.97:0/1784733527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc888073a70 0x7fc8880b5850 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc874009840 tx=0x7fc87401c820 comp rx=0 tx=0).stop 2024-09-10T21:33:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 -- 172.21.15.97:0/1784733527 shutdown_connections 2024-09-10T21:33:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 --2- 172.21.15.97:0/1784733527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc888073a70 0x7fc8880b5850 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 --2- 172.21.15.97:0/1784733527 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc8880d0060 0x7fc8881624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 -- 172.21.15.97:0/1784733527 >> 172.21.15.97:0/1784733527 conn(0x7fc8880b28c0 msgr2=0x7fc8880b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:57.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 -- 172.21.15.97:0/1784733527 shutdown_connections 2024-09-10T21:33:57.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 -- 172.21.15.97:0/1784733527 wait complete. 2024-09-10T21:33:57.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 Processor -- start 2024-09-10T21:33:57.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.274+0000 7fc88fa00640 1 -- start start 2024-09-10T21:33:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88806cb40 0x7fc8881532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc888073a70 0x7fc8881fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8880d0060 0x7fc888156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc88816b650 con 0x7fc88806cb40 2024-09-10T21:33:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880d0630 con 0x7fc8880d0060 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc88816b7d0 con 0x7fc888073a70 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88806cb40 0x7fc8881532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88806cb40 0x7fc8881532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37260/0 (socket says 172.21.15.97:37260) 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 -- 172.21.15.97:0/2935681454 learned_addr learned my addr 172.21.15.97:0/2935681454 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88da00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8880d0060 0x7fc888156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc887e00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc888073a70 0x7fc8881fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 -- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc888073a70 msgr2=0x7fc8881fbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc888073a70 0x7fc8881fbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 -- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8880d0060 msgr2=0x7fc888156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8880d0060 0x7fc888156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 -- 172.21.15.97:0/2935681454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8881537f0 con 0x7fc88806cb40 2024-09-10T21:33:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88d000640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88806cb40 0x7fc8881532b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00e9f0 tx=0x7fc87c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc884c00640 1 -- 172.21.15.97:0/2935681454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00cd80 con 0x7fc88806cb40 2024-09-10T21:33:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc884c00640 1 -- 172.21.15.97:0/2935681454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc87c010430 con 0x7fc88806cb40 2024-09-10T21:33:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc884c00640 1 -- 172.21.15.97:0/2935681454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c010860 con 0x7fc88806cb40 2024-09-10T21:33:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8881fc510 con 0x7fc88806cb40 2024-09-10T21:33:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc888154180 con 0x7fc88806cb40 2024-09-10T21:33:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.278+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8880b5850 con 0x7fc88806cb40 2024-09-10T21:33:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.282+0000 7fc884c00640 1 -- 172.21.15.97:0/2935681454 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc87c010a00 con 0x7fc88806cb40 2024-09-10T21:33:57.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.282+0000 7fc884c00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8680725e0 0x7fc868074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.282+0000 7fc884c00640 1 -- 172.21.15.97:0/2935681454 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7fc87c094a00 con 0x7fc88806cb40 2024-09-10T21:33:57.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.282+0000 7fc887e00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8680725e0 0x7fc868074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.282+0000 7fc887e00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8680725e0 0x7fc868074aa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc8740af000 tx=0x7fc87401b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.286+0000 7fc884c00640 1 -- 172.21.15.97:0/2935681454 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc8881fc510 con 0x7fc88806cb40 2024-09-10T21:33:57.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.486+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88815b7a0 con 0x7fc88806cb40 2024-09-10T21:33:57.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.486+0000 7fc884c00640 1 -- 172.21.15.97:0/2935681454 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 262 v262) ==== 95+0+4395 (secure 0 0 0) 0x7fc87c05ef40 con 0x7fc88806cb40 2024-09-10T21:33:57.487 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:33:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":262,"btime":"2024-09-10T21:33:57:363370+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9654,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2750213748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2750213748},{"type":"v1","addr":"172.21.15.97:6834","nonce":2750213748}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:33:56.361652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:33:57.488 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 262 2024-09-10T21:33:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8680725e0 msgr2=0x7fc868074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8680725e0 0x7fc868074aa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc8740af000 tx=0x7fc87401b060 comp rx=0 tx=0).stop 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88806cb40 msgr2=0x7fc8881532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88806cb40 0x7fc8881532b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00e9f0 tx=0x7fc87c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 shutdown_connections 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc8680725e0 0x7fc868074aa0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8880d0060 0x7fc888156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc888073a70 0x7fc8881fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 --2- 172.21.15.97:0/2935681454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88806cb40 0x7fc8881532b0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 >> 172.21.15.97:0/2935681454 conn(0x7fc8880b28c0 msgr2=0x7fc8880cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 shutdown_connections 2024-09-10T21:33:57.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.490+0000 7fc88fa00640 1 -- 172.21.15.97:0/2935681454 wait complete. 2024-09-10T21:33:57.508 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2024-09-10T21:33:57.508 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2024-09-10T21:33:57.508 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:33:57.508 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:33:57.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:33:57.517 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:33:57.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:33:57.517 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:33:57.570 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:33:57.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:57.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:33:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 -- 172.21.15.97:0/2860047766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c0051d0 msgr2=0x7fa25c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 --2- 172.21.15.97:0/2860047766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c0051d0 0x7fa25c00b9d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa270009b60 tx=0x7fa27001d2b0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 -- 172.21.15.97:0/2860047766 shutdown_connections 2024-09-10T21:33:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 --2- 172.21.15.97:0/2860047766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c0051d0 0x7fa25c00b9d0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 -- 172.21.15.97:0/2860047766 >> 172.21.15.97:0/2860047766 conn(0x7fa2780b1780 msgr2=0x7fa2780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 -- 172.21.15.97:0/2860047766 shutdown_connections 2024-09-10T21:33:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 -- 172.21.15.97:0/2860047766 wait complete. 2024-09-10T21:33:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 Processor -- start 2024-09-10T21:33:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.758+0000 7fa27e400640 1 -- start start 2024-09-10T21:33:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa25c0051d0 0x7fa27806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27806e3b0 0x7fa2781435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2780cbf50 0x7fa27813d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780cee10 con 0x7fa27806e3b0 2024-09-10T21:33:57.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780cec90 con 0x7fa2780cbf50 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa27815f6d0 con 0x7fa25c0051d0 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27806e3b0 0x7fa2781435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27806e3b0 0x7fa2781435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53916/0 (socket says 172.21.15.97:53916) 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 -- 172.21.15.97:0/403690566 learned_addr learned my addr 172.21.15.97:0/403690566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa277400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa25c0051d0 0x7fa27806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa277e00640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2780cbf50 0x7fa27813d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 -- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa25c0051d0 msgr2=0x7fa27806ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa25c0051d0 0x7fa27806ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 -- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2780cbf50 msgr2=0x7fa27813d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2780cbf50 0x7fa27813d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 -- 172.21.15.97:0/403690566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27806e840 con 0x7fa27806e3b0 2024-09-10T21:33:57.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa277e00640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2780cbf50 0x7fa27813d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26ea00640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27806e3b0 0x7fa2781435f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fa260004fd0 tx=0x7fa26000ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26fe00640 1 -- 172.21.15.97:0/403690566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260015070 con 0x7fa27806e3b0 2024-09-10T21:33:57.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26fe00640 1 -- 172.21.15.97:0/403690566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa260010980 con 0x7fa27806e3b0 2024-09-10T21:33:57.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27806eb20 con 0x7fa27806e3b0 2024-09-10T21:33:57.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa26fe00640 1 -- 172.21.15.97:0/403690566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260010dd0 con 0x7fa27806e3b0 2024-09-10T21:33:57.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27806f0e0 con 0x7fa27806e3b0 2024-09-10T21:33:57.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.762+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27815c100 con 0x7fa27806e3b0 2024-09-10T21:33:57.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.766+0000 7fa26fe00640 1 -- 172.21.15.97:0/403690566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa260011460 con 0x7fa27806e3b0 2024-09-10T21:33:57.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.766+0000 7fa26fe00640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa254072610 0x7fa254074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.766+0000 7fa26fe00640 1 -- 172.21.15.97:0/403690566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7fa260094490 con 0x7fa27806e3b0 2024-09-10T21:33:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.766+0000 7fa277400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa254072610 0x7fa254074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:57.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.766+0000 7fa277400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa254072610 0x7fa254074ad0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f90 tx=0x7fa268004a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:57.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.770+0000 7fa26fe00640 1 -- 172.21.15.97:0/403690566 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa26005e820 con 0x7fa27806e3b0 2024-09-10T21:33:57.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:57.970+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa278141360 con 0x7fa27806e3b0 2024-09-10T21:33:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.366+0000 7fa26fe00640 1 -- 172.21.15.97:0/403690566 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v263) ==== 161+0+0 (secure 0 0 0) 0x7fa260063250 con 0x7fa27806e3b0 2024-09-10T21:33:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:33:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa254072610 msgr2=0x7fa254074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa254072610 0x7fa254074ad0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f90 tx=0x7fa268004a10 comp rx=0 tx=0).stop 2024-09-10T21:33:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27806e3b0 msgr2=0x7fa2781435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27806e3b0 0x7fa2781435f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fa260004fd0 tx=0x7fa26000ea20 comp rx=0 tx=0).stop 2024-09-10T21:33:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 shutdown_connections 2024-09-10T21:33:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa254072610 0x7fa254074ad0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2780cbf50 0x7fa27813d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27806e3b0 0x7fa2781435f0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 --2- 172.21.15.97:0/403690566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa25c0051d0 0x7fa27806ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 >> 172.21.15.97:0/403690566 conn(0x7fa2780b1780 msgr2=0x7fa2780731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 shutdown_connections 2024-09-10T21:33:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.370+0000 7fa27e400640 1 -- 172.21.15.97:0/403690566 wait complete. 2024-09-10T21:33:58.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 -- 172.21.15.97:0/2691676662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c141d20 msgr2=0x7fc03c15f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 --2- 172.21.15.97:0/2691676662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c141d20 0x7fc03c15f6d0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fc03c141580 tx=0x7fc02801c8a0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 -- 172.21.15.97:0/2691676662 shutdown_connections 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 --2- 172.21.15.97:0/2691676662 >> v2:172.21.15.194:3300/0 conn(0x7fc03c15fc10 0x7fc03c13f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 --2- 172.21.15.97:0/2691676662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c141d20 0x7fc03c15f6d0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 -- 172.21.15.97:0/2691676662 >> 172.21.15.97:0/2691676662 conn(0x7fc03c0b1780 msgr2=0x7fc03c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 -- 172.21.15.97:0/2691676662 shutdown_connections 2024-09-10T21:33:58.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.530+0000 7fc042200640 1 -- 172.21.15.97:0/2691676662 wait complete. 2024-09-10T21:33:58.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 Processor -- start 2024-09-10T21:33:58.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 -- start start 2024-09-10T21:33:58.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c0ce5f0 0x7fc03c13acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:58.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc03c141d20 0x7fc03c164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:58.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc03c15fc10 0x7fc03c164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c168a70 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c168bf0 con 0x7fc03c15fc10 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c168d70 con 0x7fc03c141d20 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c0ce5f0 0x7fc03c13acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c0ce5f0 0x7fc03c13acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53940/0 (socket says 172.21.15.97:53940) 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 -- 172.21.15.97:0/1888300262 learned_addr learned my addr 172.21.15.97:0/1888300262 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03aa00640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc03c141d20 0x7fc03c164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03be00640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc03c15fc10 0x7fc03c164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 -- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc03c141d20 msgr2=0x7fc03c164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc03c141d20 0x7fc03c164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 -- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc03c15fc10 msgr2=0x7fc03c164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:58.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc03c15fc10 0x7fc03c164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 -- 172.21.15.97:0/1888300262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c0b4810 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03aa00640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc03c141d20 0x7fc03c164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc03b400640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c0ce5f0 0x7fc03c13acf0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c500 tx=0x7fc02c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc033400640 1 -- 172.21.15.97:0/1888300262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c017070 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc033400640 1 -- 172.21.15.97:0/1888300262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc02c003600 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc033400640 1 -- 172.21.15.97:0/1888300262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c003a90 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c137020 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.534+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c13b230 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.538+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c15f6d0 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.538+0000 7fc033400640 1 -- 172.21.15.97:0/1888300262 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc02c003c30 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.538+0000 7fc033400640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc01c0725e0 0x7fc01c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:58.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.538+0000 7fc033400640 1 -- 172.21.15.97:0/1888300262 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7fc02c093650 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.538+0000 7fc03aa00640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc01c0725e0 0x7fc01c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:58.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.538+0000 7fc03aa00640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc01c0725e0 0x7fc01c074aa0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc0280af000 tx=0x7fc02801b0e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:58.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.542+0000 7fc033400640 1 -- 172.21.15.97:0/1888300262 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc02c05e180 con 0x7fc03c0ce5f0 2024-09-10T21:33:58.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:58.770+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc03c141360 con 0x7fc03c0ce5f0 2024-09-10T21:33:59.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.374+0000 7fc033400640 1 -- 172.21.15.97:0/1888300262 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v264) ==== 101+0+0 (secure 0 0 0) 0x7fc02c05da80 con 0x7fc03c0ce5f0 2024-09-10T21:33:59.374 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:33:59.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc01c0725e0 msgr2=0x7fc01c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc01c0725e0 0x7fc01c074aa0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc0280af000 tx=0x7fc02801b0e0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c0ce5f0 msgr2=0x7fc03c13acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c0ce5f0 0x7fc03c13acf0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c500 tx=0x7fc02c00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 shutdown_connections 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc01c0725e0 0x7fc01c074aa0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc03c15fc10 0x7fc03c164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc03c141d20 0x7fc03c164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 --2- 172.21.15.97:0/1888300262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc03c0ce5f0 0x7fc03c13acf0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 >> 172.21.15.97:0/1888300262 conn(0x7fc03c0b1780 msgr2=0x7fc03c1385e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 shutdown_connections 2024-09-10T21:33:59.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.378+0000 7fc042200640 1 -- 172.21.15.97:0/1888300262 wait complete. 2024-09-10T21:33:59.397 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:33:59.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 Processor -- start 2024-09-10T21:33:59.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 -- start start 2024-09-10T21:33:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ff33aa0 0x558520003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffecca0 0x558520004450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ffed080 0x55851ff5e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55851fe81a00 con 0x55851ffecca0 2024-09-10T21:33:59.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55851fedff40 con 0x55851ff33aa0 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5e3e8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5585200132d0 con 0x55851ffed080 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5ac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffecca0 0x558520004450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5ac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffecca0 0x558520004450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53956/0 (socket says 172.21.15.97:53956) 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5ac00640 1 -- 172.21.15.97:0/2198316442 learned_addr learned my addr 172.21.15.97:0/2198316442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5c000640 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ffed080 0x55851ff5e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.442+0000 7fef5b600640 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ff33aa0 0x558520003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5ac00640 1 -- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ffed080 msgr2=0x55851ff5e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5ac00640 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ffed080 0x55851ff5e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5ac00640 1 -- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ff33aa0 msgr2=0x558520003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5ac00640 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ff33aa0 0x558520003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5ac00640 1 -- 172.21.15.97:0/2198316442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851fe5ac10 con 0x55851ffecca0 2024-09-10T21:33:59.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5ac00640 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffecca0 0x558520004450 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fef4800c5c0 tx=0x7fef4800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8684ab2ac6301af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef59800640 1 -- 172.21.15.97:0/2198316442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4800aaf0 con 0x55851ffecca0 2024-09-10T21:33:59.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef59800640 1 -- 172.21.15.97:0/2198316442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef4800ad80 con 0x55851ffecca0 2024-09-10T21:33:59.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef59800640 1 -- 172.21.15.97:0/2198316442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48008690 con 0x55851ffecca0 2024-09-10T21:33:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- 172.21.15.97:0/2198316442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffecca0 msgr2=0x558520004450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffecca0 0x558520004450 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fef4800c5c0 tx=0x7fef4800ca80 comp rx=0 tx=0).stop 2024-09-10T21:33:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- 172.21.15.97:0/2198316442 shutdown_connections 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ffed080 0x55851ff5e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffecca0 0x558520004450 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/2198316442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ff33aa0 0x558520003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- 172.21.15.97:0/2198316442 >> 172.21.15.97:0/2198316442 conn(0x55851ff32140 msgr2=0x558520006090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- 172.21.15.97:0/2198316442 shutdown_connections 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- 172.21.15.97:0/2198316442 wait complete. 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 Processor -- start 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- start start 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ff33aa0 0x558520009460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ffecca0 0x55851ff7c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffed080 0x55851ff7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55852000ad00 con 0x55851ffed080 2024-09-10T21:33:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55852000f690 con 0x55851ffecca0 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5e3e8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55852000afa0 con 0x55851ff33aa0 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffed080 0x55851ff7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffed080 0x55851ff7cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53958/0 (socket says 172.21.15.97:53958) 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 learned_addr learned my addr 172.21.15.97:0/1906956578 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5ac00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ffecca0 0x55851ff7c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5b600640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ff33aa0 0x558520009460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ff33aa0 msgr2=0x558520009460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55851ff33aa0 0x558520009460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ffecca0 msgr2=0x55851ff7c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55851ffecca0 0x55851ff7c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.446+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851ff73e90 con 0x55851ffed080 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef5c000640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffed080 0x55851ff7cb90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fef5400a600 tx=0x7fef54004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef53e00640 1 -- 172.21.15.97:0/1906956578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54004020 con 0x55851ffed080 2024-09-10T21:33:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef53e00640 1 -- 172.21.15.97:0/1906956578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef540041c0 con 0x55851ffed080 2024-09-10T21:33:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55851ff74130 con 0x55851ffed080 2024-09-10T21:33:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef53e00640 1 -- 172.21.15.97:0/1906956578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef540a8c70 con 0x55851ffed080 2024-09-10T21:33:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851ff74670 con 0x55851ffed080 2024-09-10T21:33:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef53e00640 1 -- 172.21.15.97:0/1906956578 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fef540c5030 con 0x55851ffed080 2024-09-10T21:33:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef53e00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef3c070ff0 0x7fef3c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef53e00640 1 -- 172.21.15.97:0/1906956578 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7fef5412d870 con 0x55851ffed080 2024-09-10T21:33:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.450+0000 7fef5b600640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef3c070ff0 0x7fef3c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.452 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:33:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.454+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55851ffec380 0x55851ff730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.454+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55851ff73600 con 0x55851ffec380 2024-09-10T21:33:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.454+0000 7fef5ac00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55851ffec380 0x55851ff730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.454+0000 7fef5b600640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef3c070ff0 0x7fef3c0734b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fef4c003f20 tx=0x7fef4c0038c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.454+0000 7fef5ac00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55851ffec380 0x55851ff730c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.454+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v126'908 uv882 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.454+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x55851ff739d0 con 0x55851ffec380 2024-09-10T21:33:59.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.458+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v126'909 uv909 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.458+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200afca0 con 0x55851ffec380 2024-09-10T21:33:59.458 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:33:59.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.458+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v126'910 uv910 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.458+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b1050 con 0x55851ffec380 2024-09-10T21:33:59.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.462+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'911 uv911 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.462+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b13f0 con 0x55851ffec380 2024-09-10T21:33:59.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.462+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'912 uv912 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.462+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0040 con 0x55851ffec380 2024-09-10T21:33:59.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.462+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'913 uv913 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.466+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.466+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v126'914 uv888 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.466+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.466+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v126'915 uv915 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.466+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.469 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:33:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.470+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v126'916 uv916 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.470+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.470+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'917 uv917 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.470+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.470+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'918 uv918 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.470+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.474+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'919 uv919 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.474+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.474+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v126'920 uv895 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.474+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.478+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'921 uv921 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.478+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.478 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:33:59.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.478+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v126'922 uv922 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.478+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.482+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'923 uv923 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.482+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.482+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'924 uv924 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.482+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b03e0 con 0x55851ffec380 2024-09-10T21:33:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.482+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'925 uv925 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.482+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0c70 con 0x55851ffec380 2024-09-10T21:33:59.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.486+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'926 uv926 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.486+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.486+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v126'927 uv901 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.486+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.486+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v126'928 uv928 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.486+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.489 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:33:59.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.490+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v126'929 uv929 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.490+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.490+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'930 uv930 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.490+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.494+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'931 uv931 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.494+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.494+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'932 uv932 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.494+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.494+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v126'933 uv907 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.494+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.498+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v126'934 uv934 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.498+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.498 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:33:59.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.498+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v126'935 uv935 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.498+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.498+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'936 uv936 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.498+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.502+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'937 uv937 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.502+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x55851ffec380 2024-09-10T21:33:59.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.502+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'938 uv938 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x55851ffec380 2024-09-10T21:33:59.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.502+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5585200b1ca0 0x5585200b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.502+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b0b10 con 0x5585200b1ca0 2024-09-10T21:33:59.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.502+0000 7fef5b600640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5585200b1ca0 0x5585200b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.502+0000 7fef5b600640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5585200b1ca0 0x5585200b4090 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.506+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'659 uv642 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.506+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b45d0 con 0x5585200b1ca0 2024-09-10T21:33:59.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.506+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'660 uv660 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.506+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b4970 con 0x5585200b1ca0 2024-09-10T21:33:59.507 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:33:59.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.506+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'661 uv661 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.506+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b51c0 con 0x5585200b1ca0 2024-09-10T21:33:59.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.510+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v126'662 uv662 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.510+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b5560 con 0x5585200b1ca0 2024-09-10T21:33:59.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.510+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'663 uv663 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.510+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b5900 con 0x5585200b1ca0 2024-09-10T21:33:59.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.510+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v126'664 uv647 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.510+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b5ca0 con 0x5585200b1ca0 2024-09-10T21:33:59.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.514+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v126'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.514+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.514 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:33:59.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.514+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v126'666 uv666 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.514+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.514+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.514+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.518+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.518+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.518+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v126'669 uv652 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.518+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.518+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v126'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.522+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.521 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:33:59.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.522+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v126'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.522+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200ba210 con 0x5585200b1ca0 2024-09-10T21:33:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.522+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v126'672 uv672 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.522+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200ba210 con 0x5585200b1ca0 2024-09-10T21:33:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.526+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'673 uv673 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.526+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.526+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v126'674 uv658 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.526+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.526+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v126'675 uv675 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.526+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.528 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:33:59.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.530+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v126'676 uv676 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.530+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b7050 con 0x5585200b1ca0 2024-09-10T21:33:59.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.530+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'677 uv677 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.530+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b7050 con 0x5585200b1ca0 2024-09-10T21:33:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.530+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'678 uv678 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.530+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b6040 con 0x5585200b1ca0 2024-09-10T21:33:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'679 uv679 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200b1ca0 2024-09-10T21:33:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5585200b6550 0x5585200b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b8e80 con 0x5585200b6550 2024-09-10T21:33:59.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5ac00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5585200b6550 0x5585200b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5ac00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5585200b6550 0x5585200b8940 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200b8f00 con 0x5585200b6550 2024-09-10T21:33:59.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.534+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b92f0 con 0x5585200b6550 2024-09-10T21:33:59.536 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:33:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.538+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.538+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200ba300 con 0x5585200b6550 2024-09-10T21:33:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.538+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.538+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200ba300 con 0x5585200b6550 2024-09-10T21:33:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.538+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.538+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200b92f0 con 0x5585200b6550 2024-09-10T21:33:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.538+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5585200b9970 0x5585200bbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bc330 con 0x5585200b9970 2024-09-10T21:33:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5c000640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5585200b9970 0x5585200bbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5c000640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5585200b9970 0x5585200bbd60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'132 uv131 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200bc840 con 0x5585200b9970 2024-09-10T21:33:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.542+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bcd40 con 0x5585200b9970 2024-09-10T21:33:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.546+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'134 uv134 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.544 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:33:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.546+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bd600 con 0x5585200b9970 2024-09-10T21:33:59.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.546+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'135 uv135 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.546+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bd9a0 con 0x5585200b6550 2024-09-10T21:33:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.546+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.546+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200bd9a0 con 0x5585200b6550 2024-09-10T21:33:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bd9a0 con 0x5585200b6550 2024-09-10T21:33:59.549 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:33:59.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be9b0 con 0x5585200b6550 2024-09-10T21:33:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be9b0 con 0x5585200b6550 2024-09-10T21:33:59.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.550+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bd9a0 con 0x5585200b6550 2024-09-10T21:33:59.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.554+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.554+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bd9a0 con 0x5585200b9970 2024-09-10T21:33:59.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.554+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v126'327 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.554+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200bdd40 con 0x5585200b9970 2024-09-10T21:33:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.554+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v126'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.554+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b9970 2024-09-10T21:33:59.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.558+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v126'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.556 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:33:59.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.558+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bf0f0 con 0x5585200b9970 2024-09-10T21:33:59.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.558+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.558+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200bf0f0 con 0x5585200b9970 2024-09-10T21:33:59.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.558+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.558+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b9970 2024-09-10T21:33:59.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b9970 2024-09-10T21:33:59.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v126'333 uv326 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b9970 2024-09-10T21:33:59.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v126'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b9970 2024-09-10T21:33:59.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v126'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.564 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:33:59.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.562+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b9970 2024-09-10T21:33:59.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.566+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.566+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b9970 2024-09-10T21:33:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.566+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.566+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b6550 2024-09-10T21:33:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.566+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v126'148 uv147 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.566+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b6550 2024-09-10T21:33:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.570+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v126'149 uv149 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.570+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b6550 2024-09-10T21:33:59.569 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:33:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.570+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v126'150 uv150 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.570+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b6550 2024-09-10T21:33:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.570+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'151 uv151 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.570+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200be0e0 con 0x5585200b6550 2024-09-10T21:33:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'152 uv152 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200b6550 2024-09-10T21:33:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5585200be9e0 0x5585200c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1310 con 0x5585200be9e0 2024-09-10T21:33:59.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5b600640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5585200be9e0 0x5585200c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5b600640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5585200be9e0 0x5585200c0dd0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c1310 con 0x5585200be9e0 2024-09-10T21:33:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.574+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.576 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:33:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.578+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.578+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2740 con 0x5585200be9e0 2024-09-10T21:33:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.578+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.578+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2740 con 0x5585200be9e0 2024-09-10T21:33:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.578+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.578+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.582+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.582+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.582+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v126'294 uv288 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.582+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.582+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v126'295 uv295 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.582+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.584 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:33:59.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v126'296 uv296 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'297 uv297 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'298 uv298 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v126'299 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.586+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.590+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v126'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.590+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.589 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:33:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.590+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v126'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.590+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1730 con 0x5585200be9e0 2024-09-10T21:33:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.590+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.590+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c4540 con 0x5585200be9e0 2024-09-10T21:33:59.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v126'128 uv127 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'129 uv129 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v126'130 uv130 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.595 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:33:59.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.594+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.598+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'131 uv131 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.598+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.598+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v126'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.598+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.598+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v126'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.598+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.600 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:33:59.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v126'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2bf0 con 0x5585200b9970 2024-09-10T21:33:59.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2bf0 con 0x5585200b9970 2024-09-10T21:33:59.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c1be0 con 0x5585200b9970 2024-09-10T21:33:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef54002050 con 0x5585200b9970 2024-09-10T21:33:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5585200c20f0 0x5585200c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2a70 con 0x5585200c20f0 2024-09-10T21:33:59.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.602+0000 7fef5ac00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5585200c20f0 0x5585200c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.606+0000 7fef5ac00640 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5585200c20f0 0x5585200c2530 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.606+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v126'151 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200c20f0 2024-09-10T21:33:59.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.606+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c2a70 con 0x5585200c20f0 2024-09-10T21:33:59.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.606+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v126'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200c20f0 2024-09-10T21:33:59.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.606+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2a70 con 0x5585200c20f0 2024-09-10T21:33:59.607 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:33:59.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.606+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v126'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200c20f0 2024-09-10T21:33:59.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.606+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2a70 con 0x5585200c20f0 2024-09-10T21:33:59.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.610+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200c20f0 2024-09-10T21:33:59.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.610+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2a70 con 0x5585200c20f0 2024-09-10T21:33:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.610+0000 7fef5ac00640 1 -- 172.21.15.97:0/1906956578 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4800a8f0 con 0x5585200c20f0 2024-09-10T21:33:59.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.610+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c2a70 con 0x5585200be9e0 2024-09-10T21:33:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.610+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v126'363 uv355 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.610+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c2a70 con 0x5585200be9e0 2024-09-10T21:33:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.614+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v126'364 uv364 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.614+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c30e0 con 0x5585200be9e0 2024-09-10T21:33:59.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.614+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v126'365 uv365 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.614 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:33:59.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.614+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c30e0 con 0x5585200be9e0 2024-09-10T21:33:59.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.614+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v126'366 uv366 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.614+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'367 uv367 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v126'368 uv362 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v126'369 uv369 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v126'370 uv370 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.619 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:33:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.618+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.622+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'371 uv371 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.622+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.622+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'372 uv372 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.622+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.622+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'373 uv373 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.622+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5585200c3040 con 0x5585200be9e0 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5b600640 1 -- 172.21.15.97:0/1906956578 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'374 uv374 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c01b040 con 0x5585200be9e0 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5585200b6550 msgr2=0x5585200b8940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5585200b6550 0x5585200b8940 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5585200be9e0 msgr2=0x5585200c0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5585200be9e0 0x5585200c0dd0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5585200c20f0 msgr2=0x5585200c2530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5585200c20f0 0x5585200c2530 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5585200b9970 msgr2=0x5585200bbd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5585200b9970 0x5585200bbd60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5585200b1ca0 msgr2=0x5585200b4090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5585200b1ca0 0x5585200b4090 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 reap_dead start 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5c000640 1 -- 172.21.15.97:0/1906956578 reap_dead start 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55851ffec380 msgr2=0x55851ff730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55851ffec380 0x55851ff730c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef3c070ff0 msgr2=0x7fef3c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef3c070ff0 0x7fef3c0734b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fef4c003f20 tx=0x7fef4c0038c0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffed080 msgr2=0x55851ff7cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffed080 0x55851ff7cb90 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fef5400a600 tx=0x7fef54004b00 comp rx=0 tx=0).stop 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 shutdown_connections 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55851ffec380 0x55851ff730c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef3c070ff0 0x7fef3c0734b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 --2- 172.21.15.97:0/1906956578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55851ffed080 0x55851ff7cb90 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 >> 172.21.15.97:0/1906956578 conn(0x55851ff32140 msgr2=0x55851ff5e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:59.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 shutdown_connections 2024-09-10T21:33:59.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.626+0000 7fef5e3e8080 1 -- 172.21.15.97:0/1906956578 wait complete. 2024-09-10T21:33:59.630 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2024-09-10T21:33:59.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 Processor -- start 2024-09-10T21:33:59.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- start start 2024-09-10T21:33:59.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb3d0940 0x5640bb477330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb45ab00 0x5640bb45f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb3d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5640bb34df40 con 0x5640bb45b2a0 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5640bb35ad00 con 0x5640bb45ab00 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5640bb2efa00 con 0x5640bb3d0940 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb3d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb3d0940 0x5640bb477330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc177800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb45ab00 0x5640bb45f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb3d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53968/0 (socket says 172.21.15.97:53968) 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178c00640 1 -- 172.21.15.97:0/241133710 learned_addr learned my addr 172.21.15.97:0/241133710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178200640 1 -- 172.21.15.97:0/241133710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb45ab00 msgr2=0x5640bb45f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178200640 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb45ab00 0x5640bb45f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178200640 1 -- 172.21.15.97:0/241133710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 msgr2=0x5640bb3d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178200640 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb3d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178200640 1 -- 172.21.15.97:0/241133710 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640bb460950 con 0x5640bb3d0940 2024-09-10T21:33:59.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178c00640 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb3d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:59.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc178200640 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb3d0940 0x5640bb477330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x5640bb3a8a00 tx=0x7fc168017ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c66814f21cd13ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc176400640 1 -- 172.21.15.97:0/241133710 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680abb80 con 0x5640bb3d0940 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc176400640 1 -- 172.21.15.97:0/241133710 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1680bb070 con 0x5640bb3d0940 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc176400640 1 -- 172.21.15.97:0/241133710 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680b6680 con 0x5640bb3d0940 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- 172.21.15.97:0/241133710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb3d0940 msgr2=0x5640bb477330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb3d0940 0x5640bb477330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x5640bb3a8a00 tx=0x7fc168017ea0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- 172.21.15.97:0/241133710 shutdown_connections 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb3d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb45ab00 0x5640bb45f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 --2- 172.21.15.97:0/241133710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb3d0940 0x5640bb477330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- 172.21.15.97:0/241133710 >> 172.21.15.97:0/241133710 conn(0x5640bb3cf520 msgr2=0x5640bb454dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- 172.21.15.97:0/241133710 shutdown_connections 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.718+0000 7fc17b09b080 1 -- 172.21.15.97:0/241133710 wait complete. 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 Processor -- start 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 -- start start 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb3d0940 0x5640bb4711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb45ab00 0x5640bb474900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb474e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5640bb476e90 con 0x5640bb45b2a0 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5640bb47a5e0 con 0x5640bb3d0940 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5640bb47a840 con 0x5640bb45ab00 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb474e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb3d0940 0x5640bb4711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb474e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53972/0 (socket says 172.21.15.97:53972) 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 -- 172.21.15.97:0/3991328345 learned_addr learned my addr 172.21.15.97:0/3991328345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc177800640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb45ab00 0x5640bb474900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 -- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb45ab00 msgr2=0x5640bb474900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5640bb45ab00 0x5640bb474900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 -- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb3d0940 msgr2=0x5640bb4711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb3d0940 0x5640bb4711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 -- 172.21.15.97:0/3991328345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640bb473160 con 0x5640bb45b2a0 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178200640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5640bb3d0940 0x5640bb4711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:59.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc178c00640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb474e40 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc17000d430 tx=0x7fc17000d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc175000640 1 -- 172.21.15.97:0/3991328345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170018070 con 0x5640bb45b2a0 2024-09-10T21:33:59.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc175000640 1 -- 172.21.15.97:0/3991328345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc170006030 con 0x5640bb45b2a0 2024-09-10T21:33:59.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5640bb473400 con 0x5640bb45b2a0 2024-09-10T21:33:59.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc175000640 1 -- 172.21.15.97:0/3991328345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700125f0 con 0x5640bb45b2a0 2024-09-10T21:33:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.722+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640bb473940 con 0x5640bb45b2a0 2024-09-10T21:33:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc175000640 1 -- 172.21.15.97:0/3991328345 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc170012790 con 0x5640bb45b2a0 2024-09-10T21:33:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc175000640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc158070ff0 0x7fc1580734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc175000640 1 -- 172.21.15.97:0/3991328345 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7fc170092680 con 0x5640bb45b2a0 2024-09-10T21:33:59.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc178200640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc158070ff0 0x7fc1580734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc17b09b080 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5640bb472cc0 0x5640bb3d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x5640bb4716e0 con 0x5640bb472cc0 2024-09-10T21:33:59.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc177800640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5640bb472cc0 0x5640bb3d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc178200640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc158070ff0 0x7fc1580734b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc1680b7000 tx=0x7fc168004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.726+0000 7fc177800640 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5640bb472cc0 0x5640bb3d3d50 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc177800640 1 -- 172.21.15.97:0/3991328345 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v126'375 uv375 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc16400f590 con 0x5640bb472cc0 2024-09-10T21:33:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5640bb472cc0 msgr2=0x5640bb3d3d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5640bb472cc0 0x5640bb3d3d50 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc158070ff0 msgr2=0x7fc1580734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc158070ff0 0x7fc1580734b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc1680b7000 tx=0x7fc168004890 comp rx=0 tx=0).stop 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 msgr2=0x5640bb474e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 --2- 172.21.15.97:0/3991328345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5640bb45b2a0 0x5640bb474e40 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc17000d430 tx=0x7fc17000d8f0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc178c00640 1 -- 172.21.15.97:0/3991328345 reap_dead start 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 shutdown_connections 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 >> 172.21.15.97:0/3991328345 conn(0x5640bb3cf520 msgr2=0x5640bb3eb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 shutdown_connections 2024-09-10T21:33:59.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.730+0000 7fc17b09b080 1 -- 172.21.15.97:0/3991328345 wait complete. 2024-09-10T21:33:59.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 -- 172.21.15.97:0/2484259557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 msgr2=0x7fdaa80cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 --2- 172.21.15.97:0/2484259557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa80cff10 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8069d60 tx=0x7fda9001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 -- 172.21.15.97:0/2484259557 shutdown_connections 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 --2- 172.21.15.97:0/2484259557 >> v2:172.21.15.194:3300/0 conn(0x7fdaa80b1560 0x7fdaa80b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 --2- 172.21.15.97:0/2484259557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa80cff10 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 -- 172.21.15.97:0/2484259557 >> 172.21.15.97:0/2484259557 conn(0x7fdaa80b37c0 msgr2=0x7fdaa80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 -- 172.21.15.97:0/2484259557 shutdown_connections 2024-09-10T21:33:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.922+0000 7fdaad200640 1 -- 172.21.15.97:0/2484259557 wait complete. 2024-09-10T21:33:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 Processor -- start 2024-09-10T21:33:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 -- start start 2024-09-10T21:33:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdaa80b1560 0x7fdaa81f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa80ce830 0x7fdaa81f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80ce1a0 con 0x7fdaa80cec10 2024-09-10T21:33:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80cde60 con 0x7fdaa80ce830 2024-09-10T21:33:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa80ce320 con 0x7fdaa80b1560 2024-09-10T21:33:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa806eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53990/0 (socket says 172.21.15.97:53990) 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 -- 172.21.15.97:0/3685639313 learned_addr learned my addr 172.21.15.97:0/3685639313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fda9d400640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa80ce830 0x7fdaa81f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa5e00640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdaa80b1560 0x7fdaa81f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 -- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdaa80b1560 msgr2=0x7fdaa81f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdaa80b1560 0x7fdaa81f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 -- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa80ce830 msgr2=0x7fdaa81f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa80ce830 0x7fdaa81f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 -- 172.21.15.97:0/3685639313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa806de20 con 0x7fdaa80cec10 2024-09-10T21:33:59.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fda9d400640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa80ce830 0x7fdaa81f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaa6800640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa806eec0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000eb50 tx=0x7fdaa000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fda9ea00640 1 -- 172.21.15.97:0/3685639313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0004d30 con 0x7fdaa80cec10 2024-09-10T21:33:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fda9ea00640 1 -- 172.21.15.97:0/3685639313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaa000ee50 con 0x7fdaa80cec10 2024-09-10T21:33:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fda9ea00640 1 -- 172.21.15.97:0/3685639313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0005280 con 0x7fdaa80cec10 2024-09-10T21:33:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa806e100 con 0x7fdaa80cec10 2024-09-10T21:33:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.926+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa806e6c0 con 0x7fdaa80cec10 2024-09-10T21:33:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.930+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa80cf800 con 0x7fdaa80cec10 2024-09-10T21:33:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.930+0000 7fda9ea00640 1 -- 172.21.15.97:0/3685639313 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdaa0005420 con 0x7fdaa80cec10 2024-09-10T21:33:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.930+0000 7fda9ea00640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda840725e0 0x7fda84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.930+0000 7fda9ea00640 1 -- 172.21.15.97:0/3685639313 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8855+0+0 (secure 0 0 0) 0x7fdaa0094c10 con 0x7fdaa80cec10 2024-09-10T21:33:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.930+0000 7fdaa5e00640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda840725e0 0x7fda84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.930+0000 7fdaa5e00640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda840725e0 0x7fda84074aa0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda98007fd0 tx=0x7fda98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:59.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:33:59.934+0000 7fda9ea00640 1 -- 172.21.15.97:0/3685639313 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdaa005ef20 con 0x7fdaa80cec10 2024-09-10T21:34:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.134+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fdaa8071d10 con 0x7fdaa80cec10 2024-09-10T21:34:00.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.378+0000 7fda9ea00640 1 -- 172.21.15.97:0/3685639313 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v265) ==== 162+0+0 (secure 0 0 0) 0x7fdaa0063950 con 0x7fdaa80cec10 2024-09-10T21:34:00.378 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:34:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda840725e0 msgr2=0x7fda84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda840725e0 0x7fda84074aa0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda98007fd0 tx=0x7fda98009040 comp rx=0 tx=0).stop 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 msgr2=0x7fdaa806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa806eec0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000eb50 tx=0x7fdaa000c6a0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 shutdown_connections 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fda840725e0 0x7fda84074aa0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa80cec10 0x7fdaa806eec0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaa80ce830 0x7fdaa81f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 --2- 172.21.15.97:0/3685639313 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdaa80b1560 0x7fdaa81f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 >> 172.21.15.97:0/3685639313 conn(0x7fdaa80b37c0 msgr2=0x7fdaa80cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 shutdown_connections 2024-09-10T21:34:00.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.382+0000 7fdaad200640 1 -- 172.21.15.97:0/3685639313 wait complete. 2024-09-10T21:34:00.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:00.423 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:00.422+0000 7f0f3c200640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T21:34:00.423 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:00.422+0000 7f0f3c200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:34:00.424 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:00.426+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:34:00.424 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:00.426+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:34:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- 172.21.15.97:0/1063310542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040ce5e0 msgr2=0x7eff040b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 --2- 172.21.15.97:0/1063310542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040ce5e0 0x7eff040b2010 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7eff040d1c70 tx=0x7efef401c800 comp rx=0 tx=0).stop 2024-09-10T21:34:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- 172.21.15.97:0/1063310542 shutdown_connections 2024-09-10T21:34:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 --2- 172.21.15.97:0/1063310542 >> v2:172.21.15.194:3300/0 conn(0x7eff040cde30 0x7eff040cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 --2- 172.21.15.97:0/1063310542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040ce5e0 0x7eff040b2010 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- 172.21.15.97:0/1063310542 >> 172.21.15.97:0/1063310542 conn(0x7eff040d1ca0 msgr2=0x7eff040d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- 172.21.15.97:0/1063310542 shutdown_connections 2024-09-10T21:34:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- 172.21.15.97:0/1063310542 wait complete. 2024-09-10T21:34:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 Processor -- start 2024-09-10T21:34:00.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- start start 2024-09-10T21:34:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040cde30 0x7eff0409f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff040ce5e0 0x7eff041fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff040d0710 0x7eff040cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff040cd300 con 0x7eff040cde30 2024-09-10T21:34:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff040cd180 con 0x7eff040ce5e0 2024-09-10T21:34:00.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff0a600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7eff040cd480 con 0x7eff040d0710 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040cde30 0x7eff0409f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff03400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040cde30 0x7eff0409f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54004/0 (socket says 172.21.15.97:54004) 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.546+0000 7eff03400640 1 -- 172.21.15.97:0/890066363 learned_addr learned my addr 172.21.15.97:0/890066363 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff03e00640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff040d0710 0x7eff040cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7efefaa00640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff040ce5e0 0x7eff041fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff03400640 1 -- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff040d0710 msgr2=0x7eff040cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff03400640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff040d0710 0x7eff040cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff03400640 1 -- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff040ce5e0 msgr2=0x7eff041fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff03400640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff040ce5e0 0x7eff041fb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff03400640 1 -- 172.21.15.97:0/890066363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff041fb8a0 con 0x7eff040cde30 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7efefaa00640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff040ce5e0 0x7eff041fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff03400640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040cde30 0x7eff0409f560 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7efef401cce0 tx=0x7efef40a66c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:00.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7efefbe00640 1 -- 172.21.15.97:0/890066363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef40be020 con 0x7eff040cde30 2024-09-10T21:34:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7efefbe00640 1 -- 172.21.15.97:0/890066363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efef40043b0 con 0x7eff040cde30 2024-09-10T21:34:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7efefbe00640 1 -- 172.21.15.97:0/890066363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4003840 con 0x7eff040cde30 2024-09-10T21:34:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0414de70 con 0x7eff040cde30 2024-09-10T21:34:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0414e430 con 0x7eff040cde30 2024-09-10T21:34:00.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff040b2010 con 0x7eff040cde30 2024-09-10T21:34:00.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7efefbe00640 1 -- 172.21.15.97:0/890066363 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efef40039e0 con 0x7eff040cde30 2024-09-10T21:34:00.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.550+0000 7efefbe00640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efee0072610 0x7efee0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.554+0000 7efefbe00640 1 -- 172.21.15.97:0/890066363 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7efef412f320 con 0x7eff040cde30 2024-09-10T21:34:00.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.554+0000 7efefaa00640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efee0072610 0x7efee0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.554+0000 7efefaa00640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efee0072610 0x7efee0074ad0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7eff040b22d0 tx=0x7efeec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:00.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.554+0000 7efefbe00640 1 -- 172.21.15.97:0/890066363 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efef40c8030 con 0x7eff040cde30 2024-09-10T21:34:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff040d0060 con 0x7eff040cde30 2024-09-10T21:34:00.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7efefbe00640 1 -- 172.21.15.97:0/890066363 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) ==== 95+0+3569 (secure 0 0 0) 0x7eff041fb8a0 con 0x7eff040cde30 2024-09-10T21:34:00.757 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:00.757 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":267,"btime":"2024-09-10T21:34:00:428165+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:00.428106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:00.758 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 267 2024-09-10T21:34:00.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efee0072610 msgr2=0x7efee0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efee0072610 0x7efee0074ad0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7eff040b22d0 tx=0x7efeec009040 comp rx=0 tx=0).stop 2024-09-10T21:34:00.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040cde30 msgr2=0x7eff0409f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040cde30 0x7eff0409f560 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7efef401cce0 tx=0x7efef40a66c0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 shutdown_connections 2024-09-10T21:34:00.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efee0072610 0x7efee0074ad0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff040d0710 0x7eff040cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff040ce5e0 0x7eff041fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 --2- 172.21.15.97:0/890066363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff040cde30 0x7eff0409f560 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 >> 172.21.15.97:0/890066363 conn(0x7eff040d1ca0 msgr2=0x7eff040aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 shutdown_connections 2024-09-10T21:34:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.758+0000 7eff0a600640 1 -- 172.21.15.97:0/890066363 wait complete. 2024-09-10T21:34:00.778 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:34:00.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- 172.21.15.97:0/1746715818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352015efc0 msgr2=0x7f35200cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 --2- 172.21.15.97:0/1746715818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352015efc0 0x7f35200cf840 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f3514009900 tx=0x7f351401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- 172.21.15.97:0/1746715818 shutdown_connections 2024-09-10T21:34:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 --2- 172.21.15.97:0/1746715818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352015efc0 0x7f35200cf840 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- 172.21.15.97:0/1746715818 >> 172.21.15.97:0/1746715818 conn(0x7f35200d1a60 msgr2=0x7f35200d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- 172.21.15.97:0/1746715818 shutdown_connections 2024-09-10T21:34:00.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- 172.21.15.97:0/1746715818 wait complete. 2024-09-10T21:34:00.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 Processor -- start 2024-09-10T21:34:00.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- start start 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35200cdcb0 0x7f35201fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35200d0490 0x7f3520157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352015efc0 0x7f3520156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3520169b50 con 0x7f35200d0490 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f35201699d0 con 0x7f352015efc0 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3525e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3520169cd0 con 0x7f35200cdcb0 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f351fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35200d0490 0x7f3520157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f3524a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35200cdcb0 0x7f35201fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f351fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35200d0490 0x7f3520157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54016/0 (socket says 172.21.15.97:54016) 2024-09-10T21:34:00.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.962+0000 7f351fe00640 1 -- 172.21.15.97:0/947590132 learned_addr learned my addr 172.21.15.97:0/947590132 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:00.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f3525400640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352015efc0 0x7f3520156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351fe00640 1 -- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35200cdcb0 msgr2=0x7f35201fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351fe00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35200cdcb0 0x7f35201fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351fe00640 1 -- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352015efc0 msgr2=0x7f3520156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:00.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351fe00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352015efc0 0x7f3520156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:00.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351fe00640 1 -- 172.21.15.97:0/947590132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3520158680 con 0x7f35200d0490 2024-09-10T21:34:00.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351fe00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35200d0490 0x7f3520157fa0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c420 tx=0x7f350c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:00.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351cc00640 1 -- 172.21.15.97:0/947590132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c017070 con 0x7f35200d0490 2024-09-10T21:34:00.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351cc00640 1 -- 172.21.15.97:0/947590132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f350c003600 con 0x7f35200d0490 2024-09-10T21:34:00.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351cc00640 1 -- 172.21.15.97:0/947590132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c003a90 con 0x7f35200d0490 2024-09-10T21:34:00.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35201fadc0 con 0x7f35200d0490 2024-09-10T21:34:00.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35201fb380 con 0x7f35200d0490 2024-09-10T21:34:00.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35200ae930 con 0x7f35200d0490 2024-09-10T21:34:00.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351cc00640 1 -- 172.21.15.97:0/947590132 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f350c003c30 con 0x7f35200d0490 2024-09-10T21:34:00.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.966+0000 7f351cc00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35000725e0 0x7f3500074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:00.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.970+0000 7f351cc00640 1 -- 172.21.15.97:0/947590132 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f350c093490 con 0x7f35200d0490 2024-09-10T21:34:00.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.970+0000 7f3524a00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35000725e0 0x7f3500074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:00.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.970+0000 7f3524a00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35000725e0 0x7f3500074aa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f35140af000 tx=0x7f351401b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:00.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:00.970+0000 7f351cc00640 1 -- 172.21.15.97:0/947590132 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f350c05d7d0 con 0x7f35200d0490 2024-09-10T21:34:01.111 INFO:tasks.ceph.mds.c.smithi097.stderr: -45> 2024-09-10T21:33:56.358+0000 7f0250000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T21:34:01.147 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:34:01.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.170+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3520157a90 con 0x7f35200d0490 2024-09-10T21:34:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f351cc00640 1 -- 172.21.15.97:0/947590132 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) ==== 95+0+3569 (secure 0 0 0) 0x7f350c062200 con 0x7f35200d0490 2024-09-10T21:34:01.172 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:01.172 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":267,"btime":"2024-09-10T21:34:00:428165+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:00.428106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:01.172 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 267 2024-09-10T21:34:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35000725e0 msgr2=0x7f3500074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35000725e0 0x7f3500074aa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f35140af000 tx=0x7f351401b120 comp rx=0 tx=0).stop 2024-09-10T21:34:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35200d0490 msgr2=0x7f3520157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35200d0490 0x7f3520157fa0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c420 tx=0x7f350c00c8e0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 shutdown_connections 2024-09-10T21:34:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35000725e0 0x7f3500074aa0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f352015efc0 0x7f3520156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35200d0490 0x7f3520157fa0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 --2- 172.21.15.97:0/947590132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35200cdcb0 0x7f35201fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 >> 172.21.15.97:0/947590132 conn(0x7f35200d1a60 msgr2=0x7f352015ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 shutdown_connections 2024-09-10T21:34:01.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.174+0000 7f3525e00640 1 -- 172.21.15.97:0/947590132 wait complete. 2024-09-10T21:34:01.194 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:34:01.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 -- 172.21.15.97:0/1587122175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0afdd0 msgr2=0x7f1b7c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 --2- 172.21.15.97:0/1587122175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0afdd0 0x7f1b7c0cc6d0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0c55b0 tx=0x7f1b6401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 -- 172.21.15.97:0/1587122175 shutdown_connections 2024-09-10T21:34:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 --2- 172.21.15.97:0/1587122175 >> v2:172.21.15.194:3300/0 conn(0x7f1b7c0b0d50 0x7f1b7c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 --2- 172.21.15.97:0/1587122175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0afdd0 0x7f1b7c0cc6d0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 -- 172.21.15.97:0/1587122175 >> 172.21.15.97:0/1587122175 conn(0x7f1b7c0d1c30 msgr2=0x7f1b7c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 -- 172.21.15.97:0/1587122175 shutdown_connections 2024-09-10T21:34:01.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.334+0000 7f1b82400640 1 -- 172.21.15.97:0/1587122175 wait complete. 2024-09-10T21:34:01.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 Processor -- start 2024-09-10T21:34:01.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 -- start start 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0afdd0 0x7f1b7c163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0b0d50 0x7f1b7c1fa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0ce640 0x7f1b7c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c168ee0 con 0x7f1b7c0ce640 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c168d60 con 0x7f1b7c0afdd0 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c169060 con 0x7f1b7c0b0d50 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0ce640 0x7f1b7c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0ce640 0x7f1b7c15e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54054/0 (socket says 172.21.15.97:54054) 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 -- 172.21.15.97:0/3491229237 learned_addr learned my addr 172.21.15.97:0/3491229237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b72a00640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0b0d50 0x7f1b7c1fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7b400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0afdd0 0x7f1b7c163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 -- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0b0d50 msgr2=0x7f1b7c1fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0b0d50 0x7f1b7c1fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 -- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0afdd0 msgr2=0x7f1b7c163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0afdd0 0x7f1b7c163c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 -- 172.21.15.97:0/3491229237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c1642c0 con 0x7f1b7c0ce640 2024-09-10T21:34:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b7be00640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0ce640 0x7f1b7c15e6f0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e990 tx=0x7f1b7400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b73e00640 1 -- 172.21.15.97:0/3491229237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74004d30 con 0x7f1b7c0ce640 2024-09-10T21:34:01.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b73e00640 1 -- 172.21.15.97:0/3491229237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b7400cb00 con 0x7f1b7c0ce640 2024-09-10T21:34:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c1645a0 con 0x7f1b7c0ce640 2024-09-10T21:34:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b73e00640 1 -- 172.21.15.97:0/3491229237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74005020 con 0x7f1b7c0ce640 2024-09-10T21:34:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c0c6950 con 0x7f1b7c0ce640 2024-09-10T21:34:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.338+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c0cea20 con 0x7f1b7c0ce640 2024-09-10T21:34:01.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.342+0000 7f1b73e00640 1 -- 172.21.15.97:0/3491229237 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1b7400f040 con 0x7f1b7c0ce640 2024-09-10T21:34:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.342+0000 7f1b73e00640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b580725e0 0x7f1b58074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.342+0000 7f1b73e00640 1 -- 172.21.15.97:0/3491229237 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f1b740944b0 con 0x7f1b7c0ce640 2024-09-10T21:34:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.342+0000 7f1b7b400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b580725e0 0x7f1b58074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.342+0000 7f1b7b400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b580725e0 0x7f1b58074aa0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000f70 tx=0x7f1b6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:01.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.346+0000 7f1b73e00640 1 -- 172.21.15.97:0/3491229237 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1b74016030 con 0x7f1b7c0ce640 2024-09-10T21:34:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.546+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b7c072290 con 0x7f1b7c0ce640 2024-09-10T21:34:01.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.546+0000 7f1b73e00640 1 -- 172.21.15.97:0/3491229237 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 268 v268) ==== 95+0+4395 (secure 0 0 0) 0x7f1b74094140 con 0x7f1b7c0ce640 2024-09-10T21:34:01.546 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:01.546 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":268,"btime":"2024-09-10T21:34:01:430025+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2872418710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2872418710},{"type":"v1","addr":"172.21.15.97:6835","nonce":2872418710}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":268}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:00.428106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 268 2024-09-10T21:34:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b580725e0 msgr2=0x7f1b58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b580725e0 0x7f1b58074aa0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000f70 tx=0x7f1b6c009040 comp rx=0 tx=0).stop 2024-09-10T21:34:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0ce640 msgr2=0x7f1b7c15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0ce640 0x7f1b7c15e6f0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e990 tx=0x7f1b7400ee50 comp rx=0 tx=0).stop 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 shutdown_connections 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b580725e0 0x7f1b58074aa0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7c0ce640 0x7f1b7c15e6f0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1b7c0b0d50 0x7f1b7c1fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 --2- 172.21.15.97:0/3491229237 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b7c0afdd0 0x7f1b7c163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 >> 172.21.15.97:0/3491229237 conn(0x7f1b7c0d1c30 msgr2=0x7f1b7c0cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 shutdown_connections 2024-09-10T21:34:01.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.550+0000 7f1b82400640 1 -- 172.21.15.97:0/3491229237 wait complete. 2024-09-10T21:34:01.569 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2024-09-10T21:34:01.569 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2024-09-10T21:34:01.569 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:01.569 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:01.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:34:01.578 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:34:01.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:01.578 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:34:01.627 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:34:01.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:34:01.628 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- 172.21.15.97:0/2503022025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40ceee0 msgr2=0x7f1bb40cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 --2- 172.21.15.97:0/2503022025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40ceee0 0x7f1bb40cf390 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009900 tx=0x7f1bac01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- 172.21.15.97:0/2503022025 shutdown_connections 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 --2- 172.21.15.97:0/2503022025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40ceee0 0x7f1bb40cf390 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 --2- 172.21.15.97:0/2503022025 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb41511f0 0x7f1bb40b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- 172.21.15.97:0/2503022025 >> 172.21.15.97:0/2503022025 conn(0x7f1bb40b1780 msgr2=0x7f1bb40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- 172.21.15.97:0/2503022025 shutdown_connections 2024-09-10T21:34:01.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- 172.21.15.97:0/2503022025 wait complete. 2024-09-10T21:34:01.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 Processor -- start 2024-09-10T21:34:01.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- start start 2024-09-10T21:34:01.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40b3cf0 0x7f1bb41f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb40ceee0 0x7f1bb41593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb41511f0 0x7f1bb414c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb415aac0 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb415a940 con 0x7f1bb41511f0 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bbb400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb415ac40 con 0x7f1bb40ceee0 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40b3cf0 0x7f1bb41f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40b3cf0 0x7f1bb41f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54066/0 (socket says 172.21.15.97:54066) 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb8a00640 1 -- 172.21.15.97:0/178058621 learned_addr learned my addr 172.21.15.97:0/178058621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb9400640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb41511f0 0x7f1bb414c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb3e00640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb40ceee0 0x7f1bb41593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb8a00640 1 -- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb40ceee0 msgr2=0x7f1bb41593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb8a00640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb40ceee0 0x7f1bb41593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb8a00640 1 -- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb41511f0 msgr2=0x7f1bb414c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:01.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.814+0000 7f1bb8a00640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb41511f0 0x7f1bb414c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb8a00640 1 -- 172.21.15.97:0/178058621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb4072d50 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb8a00640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40b3cf0 0x7f1bb41f5570 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400ccb0 tx=0x7f1ba400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb0c00640 1 -- 172.21.15.97:0/178058621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400f450 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb0c00640 1 -- 172.21.15.97:0/178058621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ba400f6e0 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb41f5ab0 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb0c00640 1 -- 172.21.15.97:0/178058621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400fb30 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb40ccea0 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb415b0a0 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb0c00640 1 -- 172.21.15.97:0/178058621 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ba4020030 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb0c00640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b940725e0 0x7f1b94074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb0c00640 1 -- 172.21.15.97:0/178058621 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f1ba40946a0 con 0x7f1bb40b3cf0 2024-09-10T21:34:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.818+0000 7f1bb3e00640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b940725e0 0x7f1b94074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:01.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.822+0000 7f1bb3e00640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b940725e0 0x7f1b94074aa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4152320 tx=0x7f1ba0007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:01.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:01.822+0000 7f1bb0c00640 1 -- 172.21.15.97:0/178058621 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ba4014030 con 0x7f1bb40b3cf0 2024-09-10T21:34:02.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.022+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1bb415b900 con 0x7f1bb40b3cf0 2024-09-10T21:34:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.434+0000 7f1bb0c00640 1 -- 172.21.15.97:0/178058621 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v269) ==== 161+0+0 (secure 0 0 0) 0x7f1ba405e9e0 con 0x7f1bb40b3cf0 2024-09-10T21:34:02.434 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:34:02.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b940725e0 msgr2=0x7f1b94074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:02.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b940725e0 0x7f1b94074aa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4152320 tx=0x7f1ba0007e40 comp rx=0 tx=0).stop 2024-09-10T21:34:02.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40b3cf0 msgr2=0x7f1bb41f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:02.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40b3cf0 0x7f1bb41f5570 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400ccb0 tx=0x7f1ba400a620 comp rx=0 tx=0).stop 2024-09-10T21:34:02.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 shutdown_connections 2024-09-10T21:34:02.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1b940725e0 0x7f1b94074aa0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:02.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb41511f0 0x7f1bb414c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:02.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1bb40ceee0 0x7f1bb41593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:02.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 --2- 172.21.15.97:0/178058621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40b3cf0 0x7f1bb41f5570 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:02.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 >> 172.21.15.97:0/178058621 conn(0x7f1bb40b1780 msgr2=0x7f1bb41517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:02.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 shutdown_connections 2024-09-10T21:34:02.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.438+0000 7f1bbb400640 1 -- 172.21.15.97:0/178058621 wait complete. 2024-09-10T21:34:02.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:34:02.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- 172.21.15.97:0/1678597629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c004010 msgr2=0x7f2b1c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:02.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 --2- 172.21.15.97:0/1678597629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c004010 0x7f2b1c004810 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2b200016f0 tx=0x7f2b2001c7b0 comp rx=0 tx=0).stop 2024-09-10T21:34:02.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b37400640 1 -- 172.21.15.97:0/1678597629 reap_dead start 2024-09-10T21:34:02.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- 172.21.15.97:0/1678597629 shutdown_connections 2024-09-10T21:34:02.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- 172.21.15.97:0/1678597629 >> 172.21.15.97:0/1678597629 conn(0x7f2b380d1cd0 msgr2=0x7f2b380d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:02.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- 172.21.15.97:0/1678597629 shutdown_connections 2024-09-10T21:34:02.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- 172.21.15.97:0/1678597629 wait complete. 2024-09-10T21:34:02.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 Processor -- start 2024-09-10T21:34:02.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- start start 2024-09-10T21:34:02.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b1c004010 0x7f2b3814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:02.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3815de40 0x7f2b3815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:02.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b3815acf0 0x7f2b3815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:02.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b381648f0 con 0x7f2b3815de40 2024-09-10T21:34:02.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3815f440 con 0x7f2b3815acf0 2024-09-10T21:34:02.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b3de00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b38164a70 con 0x7f2b1c004010 2024-09-10T21:34:02.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3815de40 0x7f2b3815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3815de40 0x7f2b3815e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54092/0 (socket says 172.21.15.97:54092) 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b2fe00640 1 -- 172.21.15.97:0/4213972855 learned_addr learned my addr 172.21.15.97:0/4213972855 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.602+0000 7f2b36a00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b1c004010 0x7f2b3814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b37400640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b3815acf0 0x7f2b3815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2fe00640 1 -- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b1c004010 msgr2=0x7f2b3814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2fe00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b1c004010 0x7f2b3814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2fe00640 1 -- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b3815acf0 msgr2=0x7f2b3815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2fe00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b3815acf0 0x7f2b3815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2fe00640 1 -- 172.21.15.97:0/4213972855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b380b12d0 con 0x7f2b3815de40 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b37400640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b3815acf0 0x7f2b3815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2fe00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3815de40 0x7f2b3815e2c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2b2001b080 tx=0x7f2b20001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:02.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2f400640 1 -- 172.21.15.97:0/4213972855 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b200a5610 con 0x7f2b3815de40 2024-09-10T21:34:02.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2f400640 1 -- 172.21.15.97:0/4213972855 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b200a58a0 con 0x7f2b3815de40 2024-09-10T21:34:02.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3815da50 con 0x7f2b3815de40 2024-09-10T21:34:02.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2f400640 1 -- 172.21.15.97:0/4213972855 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b200a5c90 con 0x7f2b3815de40 2024-09-10T21:34:02.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b380b50b0 con 0x7f2b3815de40 2024-09-10T21:34:02.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b380cc550 con 0x7f2b3815de40 2024-09-10T21:34:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2f400640 1 -- 172.21.15.97:0/4213972855 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b200c0030 con 0x7f2b3815de40 2024-09-10T21:34:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2f400640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b14072610 0x7f2b14074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:02.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.606+0000 7f2b2f400640 1 -- 172.21.15.97:0/4213972855 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f2b2012d570 con 0x7f2b3815de40 2024-09-10T21:34:02.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.610+0000 7f2b36a00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b14072610 0x7f2b14074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:02.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.610+0000 7f2b36a00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b14072610 0x7f2b14074ad0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2b380712d0 tx=0x7f2b280067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.610+0000 7f2b2f400640 1 -- 172.21.15.97:0/4213972855 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b3815da50 con 0x7f2b3815de40 2024-09-10T21:34:02.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:02.834+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f2b380cfd60 con 0x7f2b3815de40 2024-09-10T21:34:03.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b2f400640 1 -- 172.21.15.97:0/4213972855 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v270) ==== 101+0+0 (secure 0 0 0) 0x7f2b20016510 con 0x7f2b3815de40 2024-09-10T21:34:03.441 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:34:03.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b14072610 msgr2=0x7f2b14074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b14072610 0x7f2b14074ad0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2b380712d0 tx=0x7f2b280067b0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3815de40 msgr2=0x7f2b3815e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3815de40 0x7f2b3815e2c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2b2001b080 tx=0x7f2b20001310 comp rx=0 tx=0).stop 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 shutdown_connections 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b14072610 0x7f2b14074ad0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b3815acf0 0x7f2b3815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3815de40 0x7f2b3815e2c0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 --2- 172.21.15.97:0/4213972855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b1c004010 0x7f2b3814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 >> 172.21.15.97:0/4213972855 conn(0x7f2b380d1cd0 msgr2=0x7f2b38071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 shutdown_connections 2024-09-10T21:34:03.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.442+0000 7f2b3de00640 1 -- 172.21.15.97:0/4213972855 wait complete. 2024-09-10T21:34:03.464 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:34:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 Processor -- start 2024-09-10T21:34:03.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 -- start start 2024-09-10T21:34:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c1b4940 0x556f9c1b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c23eb00 0x556f9c1c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23f2a0 0x556f9c1c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556f9c131f40 con 0x556f9c23eb00 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556f9c13ed00 con 0x556f9c1b4940 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709ffbe080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556f9c0d3a00 con 0x556f9c23f2a0 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c23eb00 0x556f9c1c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c23eb00 0x556f9c1c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54110/0 (socket says 172.21.15.97:54110) 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 -- 172.21.15.97:0/3524834688 learned_addr learned my addr 172.21.15.97:0/3524834688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709d200640 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c1b4940 0x556f9c1b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f709dc00640 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23f2a0 0x556f9c1c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 -- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23f2a0 msgr2=0x556f9c1c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:03.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23f2a0 0x556f9c1c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 -- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c1b4940 msgr2=0x556f9c1b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c1b4940 0x556f9c1b7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 -- 172.21.15.97:0/3524834688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f9c1b5fe0 con 0x556f9c23eb00 2024-09-10T21:34:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.510+0000 7f7097e00640 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c23eb00 0x556f9c1c9820 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f708800b250 tx=0x7f708800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fccb95ad409845a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f7096a00640 1 -- 172.21.15.97:0/3524834688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708800a040 con 0x556f9c23eb00 2024-09-10T21:34:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f7096a00640 1 -- 172.21.15.97:0/3524834688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f708800f620 con 0x556f9c23eb00 2024-09-10T21:34:03.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f7096a00640 1 -- 172.21.15.97:0/3524834688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708800fa70 con 0x556f9c23eb00 2024-09-10T21:34:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- 172.21.15.97:0/3524834688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c23eb00 msgr2=0x556f9c1c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c23eb00 0x556f9c1c9820 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f708800b250 tx=0x7f708800b710 comp rx=0 tx=0).stop 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- 172.21.15.97:0/3524834688 shutdown_connections 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23f2a0 0x556f9c1c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c23eb00 0x556f9c1c9820 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 --2- 172.21.15.97:0/3524834688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c1b4940 0x556f9c1b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- 172.21.15.97:0/3524834688 >> 172.21.15.97:0/3524834688 conn(0x556f9c1b3520 msgr2=0x556f9c184fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- 172.21.15.97:0/3524834688 shutdown_connections 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- 172.21.15.97:0/3524834688 wait complete. 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 Processor -- start 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- start start 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c1b4940 0x556f9c23d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23eb00 0x556f9c259890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c23f2a0 0x556f9c2f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556f9c1b6280 con 0x556f9c1b4940 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556f9c25e090 con 0x556f9c23f2a0 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x556f9c25e330 con 0x556f9c23eb00 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c1b4940 0x556f9c23d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c1b4940 0x556f9c23d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54118/0 (socket says 172.21.15.97:54118) 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 learned_addr learned my addr 172.21.15.97:0/1667283946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f7097e00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23eb00 0x556f9c259890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709dc00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c23f2a0 0x556f9c2f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23eb00 msgr2=0x556f9c259890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x556f9c23eb00 0x556f9c259890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c23f2a0 msgr2=0x556f9c2f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c23f2a0 0x556f9c2f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f9c25a2f0 con 0x556f9c1b4940 2024-09-10T21:34:03.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709dc00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f9c23f2a0 0x556f9c2f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709d200640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c1b4940 0x556f9c23d600 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f7090017dd0 tx=0x7f70900acaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f7095600640 1 -- 172.21.15.97:0/1667283946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70900bb070 con 0x556f9c1b4940 2024-09-10T21:34:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f7095600640 1 -- 172.21.15.97:0/1667283946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70900b1040 con 0x556f9c1b4940 2024-09-10T21:34:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f7095600640 1 -- 172.21.15.97:0/1667283946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70900b7400 con 0x556f9c1b4940 2024-09-10T21:34:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556f9c25a590 con 0x556f9c1b4940 2024-09-10T21:34:03.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.514+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f9c25aad0 con 0x556f9c1b4940 2024-09-10T21:34:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f7095600640 1 -- 172.21.15.97:0/1667283946 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f70900b6040 con 0x556f9c1b4940 2024-09-10T21:34:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f7095600640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f707c070ff0 0x7f707c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f7095600640 1 -- 172.21.15.97:0/1667283946 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f709012da90 con 0x556f9c1b4940 2024-09-10T21:34:03.519 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:34:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f7097e00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f707c070ff0 0x7f707c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x556f9c259dd0 0x556f9c24f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c24fb90 con 0x556f9c259dd0 2024-09-10T21:34:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f709dc00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x556f9c259dd0 0x556f9c24f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f7097e00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f707c070ff0 0x7f707c0734b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7088000c10 tx=0x7f7088001090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.518+0000 7f709dc00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x556f9c259dd0 0x556f9c24f650 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.522+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v127'939 uv913 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.522+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c2f0980 con 0x556f9c259dd0 2024-09-10T21:34:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.526+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v127'940 uv940 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.526+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c301850 con 0x556f9c259dd0 2024-09-10T21:34:03.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.526+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v127'941 uv941 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.526 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:34:03.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.526+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302c00 con 0x556f9c259dd0 2024-09-10T21:34:03.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.526+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'942 uv942 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.526+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302fa0 con 0x556f9c259dd0 2024-09-10T21:34:03.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.530+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'943 uv943 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.530+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c301bf0 con 0x556f9c259dd0 2024-09-10T21:34:03.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.530+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'944 uv944 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.530+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302010 con 0x556f9c259dd0 2024-09-10T21:34:03.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.534+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v127'945 uv919 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.534+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c302400 con 0x556f9c259dd0 2024-09-10T21:34:03.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.534+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v127'946 uv946 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.534+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302400 con 0x556f9c259dd0 2024-09-10T21:34:03.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.538+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v127'947 uv947 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.536 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:34:03.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.538+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.538+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'948 uv948 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.538+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.538+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'949 uv949 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.538+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.542+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'950 uv950 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.542+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.542+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v127'951 uv926 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.542+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.542+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'952 uv952 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.542+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.546+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v127'953 uv953 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.545 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:34:03.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.546+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.546+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'954 uv954 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.546+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.550+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'955 uv955 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.550+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3023b0 con 0x556f9c259dd0 2024-09-10T21:34:03.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.550+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'956 uv956 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.550+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3027c0 con 0x556f9c259dd0 2024-09-10T21:34:03.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.550+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'957 uv957 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.550+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.554+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v127'958 uv932 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.554+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.554+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v127'959 uv959 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.554+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.556 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:34:03.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.558+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v127'960 uv960 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.558+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.558+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'961 uv961 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.558+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.558+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'962 uv962 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.558+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.562+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'963 uv963 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.562+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.562+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v127'964 uv938 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.562+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.566+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v127'965 uv965 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.566+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.566 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:34:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.566+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v127'966 uv966 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.566+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.566+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'967 uv967 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.566+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.570+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'968 uv968 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.570+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c302630 con 0x556f9c259dd0 2024-09-10T21:34:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.570+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'969 uv969 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c259dd0 2024-09-10T21:34:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.570+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556f9c3027a0 0x556f9c305860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.570+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c305da0 con 0x556f9c3027a0 2024-09-10T21:34:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.570+0000 7f709d200640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556f9c3027a0 0x556f9c305860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.574+0000 7f709d200640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556f9c3027a0 0x556f9c305860 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.574+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'680 uv663 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.574+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c306140 con 0x556f9c3027a0 2024-09-10T21:34:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.574+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'681 uv681 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.574+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c306560 con 0x556f9c3027a0 2024-09-10T21:34:03.576 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:34:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.578+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'682 uv682 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.578+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c306e00 con 0x556f9c3027a0 2024-09-10T21:34:03.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.578+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v127'683 uv683 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.578+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3071a0 con 0x556f9c3027a0 2024-09-10T21:34:03.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.578+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'684 uv684 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.578+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307540 con 0x556f9c3027a0 2024-09-10T21:34:03.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v127'685 uv668 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c307930 con 0x556f9c3027a0 2024-09-10T21:34:03.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v127'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307d20 con 0x556f9c3027a0 2024-09-10T21:34:03.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v127'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.582 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:34:03.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307d20 con 0x556f9c3027a0 2024-09-10T21:34:03.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.582+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.586+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.586+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.586+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v127'690 uv673 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.586+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.586+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v127'691 uv691 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.586+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.588 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:34:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.590+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v127'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.590+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.590+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v127'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.590+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.590+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.590+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.594+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v127'695 uv679 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.594+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.594+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v127'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.594+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.598+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v127'697 uv697 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.597 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:34:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.598+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.598+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.598+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c308ce0 con 0x556f9c3027a0 2024-09-10T21:34:03.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.598+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.598+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c307cd0 con 0x556f9c3027a0 2024-09-10T21:34:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.602+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c3027a0 2024-09-10T21:34:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.602+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x556f9c3085f0 0x556f9c3089f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.602+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30b130 con 0x556f9c3085f0 2024-09-10T21:34:03.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.602+0000 7f7097e00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x556f9c3085f0 0x556f9c3089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.602+0000 7f7097e00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x556f9c3085f0 0x556f9c3089f0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.602+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.602+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c30b4d0 con 0x556f9c3085f0 2024-09-10T21:34:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30b870 con 0x556f9c3085f0 2024-09-10T21:34:03.605 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:34:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c306a50 con 0x556f9c3085f0 2024-09-10T21:34:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30d200 con 0x556f9c3085f0 2024-09-10T21:34:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.606+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30c110 con 0x556f9c3085f0 2024-09-10T21:34:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.610+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.610+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556f9c30cbd0 0x556f9c30f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.610+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30fb80 con 0x556f9c30cbd0 2024-09-10T21:34:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.610+0000 7f709dc00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556f9c30cbd0 0x556f9c30f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.610+0000 7f709dc00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556f9c30cbd0 0x556f9c30f620 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.610+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'136 uv135 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.610+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c30ff70 con 0x556f9c30cbd0 2024-09-10T21:34:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.614+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'137 uv137 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.614+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30ff70 con 0x556f9c30cbd0 2024-09-10T21:34:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.614+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'138 uv138 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.613 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:34:03.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.614+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30ff70 con 0x556f9c30cbd0 2024-09-10T21:34:03.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.614+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'139 uv139 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.614+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c30feb0 con 0x556f9c3085f0 2024-09-10T21:34:03.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c310360 con 0x556f9c3085f0 2024-09-10T21:34:03.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310360 con 0x556f9c3085f0 2024-09-10T21:34:03.618 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:34:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c311370 con 0x556f9c3085f0 2024-09-10T21:34:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.618+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c311370 con 0x556f9c3085f0 2024-09-10T21:34:03.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.622+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.622+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310360 con 0x556f9c3085f0 2024-09-10T21:34:03.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.622+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.622+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310360 con 0x556f9c30cbd0 2024-09-10T21:34:03.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v127'338 uv332 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c310360 con 0x556f9c30cbd0 2024-09-10T21:34:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v127'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310310 con 0x556f9c30cbd0 2024-09-10T21:34:03.626 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:34:03.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v127'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310310 con 0x556f9c30cbd0 2024-09-10T21:34:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.626+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3102f0 con 0x556f9c30cbd0 2024-09-10T21:34:03.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.630+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.630+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3106b0 con 0x556f9c30cbd0 2024-09-10T21:34:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.630+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.630+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3106b0 con 0x556f9c30cbd0 2024-09-10T21:34:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.630+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v127'344 uv337 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.630+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c3106b0 con 0x556f9c30cbd0 2024-09-10T21:34:03.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.634+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v127'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.634+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3106b0 con 0x556f9c30cbd0 2024-09-10T21:34:03.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.634+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v127'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.634 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:34:03.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.634+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3106b0 con 0x556f9c30cbd0 2024-09-10T21:34:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.634+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.634+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3102f0 con 0x556f9c30cbd0 2024-09-10T21:34:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.638+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.638+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3102f0 con 0x556f9c3085f0 2024-09-10T21:34:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.638+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v127'153 uv152 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.638+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c310270 con 0x556f9c3085f0 2024-09-10T21:34:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.638+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v127'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.638+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310270 con 0x556f9c3085f0 2024-09-10T21:34:03.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v127'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.640 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:34:03.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310270 con 0x556f9c3085f0 2024-09-10T21:34:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c310270 con 0x556f9c3085f0 2024-09-10T21:34:03.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'157 uv157 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3085f0 2024-09-10T21:34:03.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556f9c310c60 0x556f9c313050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313590 con 0x556f9c310c60 2024-09-10T21:34:03.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f709d200640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556f9c310c60 0x556f9c313050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.642+0000 7f709d200640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556f9c310c60 0x556f9c313050 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c313610 con 0x556f9c310c60 2024-09-10T21:34:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.646 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:34:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c314a40 con 0x556f9c310c60 2024-09-10T21:34:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.646+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c314a40 con 0x556f9c310c60 2024-09-10T21:34:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.650+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.650+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.650+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.650+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.650+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v127'304 uv298 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.650+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.654+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v127'305 uv305 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.654+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.653 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:34:03.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.654+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v127'306 uv306 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.654+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.654+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'307 uv307 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.654+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.658+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'308 uv308 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.658+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.658+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v127'309 uv303 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.658+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.658+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v127'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.658+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.660 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:34:03.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.662+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v127'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.662+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313a30 con 0x556f9c310c60 2024-09-10T21:34:03.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.662+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.662+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316840 con 0x556f9c310c60 2024-09-10T21:34:03.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.662+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.662+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.666+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v127'132 uv131 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.666+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.666+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'133 uv133 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.666+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.666+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v127'134 uv134 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.667 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:34:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.666+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.670+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'135 uv135 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.670+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.670+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v127'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.670+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.670+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v127'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.670+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.672 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:34:03.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v127'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c313ee0 con 0x556f9c30cbd0 2024-09-10T21:34:03.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7098001680 con 0x556f9c30cbd0 2024-09-10T21:34:03.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x556f9c3143f0 0x556f9c316800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.674+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316e40 con 0x556f9c3143f0 2024-09-10T21:34:03.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.678+0000 7f7097e00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x556f9c3143f0 0x556f9c316800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.678+0000 7f7097e00640 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x556f9c3143f0 0x556f9c316800 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.678+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v127'156 uv155 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3143f0 2024-09-10T21:34:03.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.678+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c316e40 con 0x556f9c3143f0 2024-09-10T21:34:03.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.678+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v127'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3143f0 2024-09-10T21:34:03.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.678+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316e40 con 0x556f9c3143f0 2024-09-10T21:34:03.679 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:34:03.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.678+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v127'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3143f0 2024-09-10T21:34:03.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.682+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c317230 con 0x556f9c3143f0 2024-09-10T21:34:03.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.682+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3143f0 2024-09-10T21:34:03.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.682+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c317230 con 0x556f9c3143f0 2024-09-10T21:34:03.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.682+0000 7f7097e00640 1 -- 172.21.15.97:0/1667283946 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f708800d040 con 0x556f9c3143f0 2024-09-10T21:34:03.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.682+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.682+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v127'376 uv375 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.682+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.686+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v127'377 uv377 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.686+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.686+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v127'378 uv378 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.686 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:34:03.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.686+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316fc0 con 0x556f9c310c60 2024-09-10T21:34:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.686+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v127'379 uv379 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.686+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316fc0 con 0x556f9c310c60 2024-09-10T21:34:03.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'380 uv380 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316fc0 con 0x556f9c310c60 2024-09-10T21:34:03.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v127'381 uv374 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v127'382 uv382 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.691 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:34:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v127'383 uv383 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.690+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.694+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'384 uv384 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.694+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.694+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'385 uv385 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.694+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c316f60 con 0x556f9c310c60 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.694+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'386 uv386 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.694+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x556f9c3173d0 con 0x556f9c310c60 2024-09-10T21:34:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709d200640 1 -- 172.21.15.97:0/1667283946 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'387 uv387 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f70900c6070 con 0x556f9c310c60 2024-09-10T21:34:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x556f9c3085f0 msgr2=0x556f9c3089f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x556f9c3085f0 0x556f9c3089f0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556f9c310c60 msgr2=0x556f9c313050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x556f9c310c60 0x556f9c313050 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x556f9c3143f0 msgr2=0x556f9c316800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x556f9c3143f0 0x556f9c316800 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556f9c30cbd0 msgr2=0x556f9c30f620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x556f9c30cbd0 0x556f9c30f620 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 reap_dead start 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556f9c3027a0 msgr2=0x556f9c305860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x556f9c3027a0 0x556f9c305860 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x556f9c259dd0 msgr2=0x556f9c24f650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x556f9c259dd0 0x556f9c24f650 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f707c070ff0 msgr2=0x7f707c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f707c070ff0 0x7f707c0734b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7088000c10 tx=0x7f7088001090 comp rx=0 tx=0).stop 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c1b4940 msgr2=0x556f9c23d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 --2- 172.21.15.97:0/1667283946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f9c1b4940 0x556f9c23d600 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f7090017dd0 tx=0x7f70900acaf0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709dc00640 1 -- 172.21.15.97:0/1667283946 reap_dead start 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 shutdown_connections 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 >> 172.21.15.97:0/1667283946 conn(0x556f9c1b3520 msgr2=0x556f9c184f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 shutdown_connections 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.698+0000 7f709ffbe080 1 -- 172.21.15.97:0/1667283946 wait complete. 2024-09-10T21:34:03.702 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:34:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.790+0000 7f6daa447080 1 Processor -- start 2024-09-10T21:34:03.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.790+0000 7f6daa447080 1 -- start start 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6d8c003640 0x555716c36810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716c2c120 0x555716d6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555716ce0970 0x555716ce0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555716bb6f40 con 0x555716ce0970 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555716bc3d00 con 0x555716c2c120 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555716b58a00 con 0x7f6d8c003640 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555716ce0970 0x555716ce0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555716ce0970 0x555716ce0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54142/0 (socket says 172.21.15.97:54142) 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 -- 172.21.15.97:0/1151710846 learned_addr learned my addr 172.21.15.97:0/1151710846 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:03.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da6a00640 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6d8c003640 0x555716c36810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da6000640 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716c2c120 0x555716d6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 -- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6d8c003640 msgr2=0x555716c36810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6d8c003640 0x555716c36810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 -- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716c2c120 msgr2=0x555716d6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716c2c120 0x555716d6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 -- 172.21.15.97:0/1151710846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555716c38420 con 0x555716ce0970 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da6000640 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716c2c120 0x555716d6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555716ce0970 0x555716ce0d80 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009940 tx=0x7f6d8c004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6379311b452ab115 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da4c00640 1 -- 172.21.15.97:0/1151710846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00092d0 con 0x555716ce0970 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da4c00640 1 -- 172.21.15.97:0/1151710846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da00a76f0 con 0x555716ce0970 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da4c00640 1 -- 172.21.15.97:0/1151710846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00a7aa0 con 0x555716ce0970 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- 172.21.15.97:0/1151710846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555716ce0970 msgr2=0x555716ce0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555716ce0970 0x555716ce0d80 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009940 tx=0x7f6d8c004130 comp rx=0 tx=0).stop 2024-09-10T21:34:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- 172.21.15.97:0/1151710846 shutdown_connections 2024-09-10T21:34:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555716ce0970 0x555716ce0d80 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716c2c120 0x555716d6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- 172.21.15.97:0/1151710846 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6d8c003640 0x555716c36810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- 172.21.15.97:0/1151710846 >> 172.21.15.97:0/1151710846 conn(0x555716c09140 msgr2=0x555716cc5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- 172.21.15.97:0/1151710846 shutdown_connections 2024-09-10T21:34:03.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- 172.21.15.97:0/1151710846 wait complete. 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 Processor -- start 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- start start 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555716c2c120 0x555716d6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716ce0970 0x555716d6b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8c003640 0x555716d6bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555716c38960 con 0x7f6d8c003640 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555716c386c0 con 0x555716ce0970 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6daa447080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555716c38c00 con 0x555716c2c120 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.794+0000 7f6da7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8c003640 0x555716d6bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8c003640 0x555716d6bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54148/0 (socket says 172.21.15.97:54148) 2024-09-10T21:34:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 -- 172.21.15.97:0/3130737900 learned_addr learned my addr 172.21.15.97:0/3130737900 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da6000640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716ce0970 0x555716d6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da6a00640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555716c2c120 0x555716d6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 -- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555716c2c120 msgr2=0x555716d6eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555716c2c120 0x555716d6eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 -- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716ce0970 msgr2=0x555716d6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716ce0970 0x555716d6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 -- 172.21.15.97:0/3130737900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555716cc3ae0 con 0x7f6d8c003640 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da6000640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555716ce0970 0x555716d6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6da7400640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8c003640 0x555716d6bc70 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f6da001e3c0 tx=0x7f6da0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6d9f400640 1 -- 172.21.15.97:0/3130737900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00a8840 con 0x7f6d8c003640 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6d9f400640 1 -- 172.21.15.97:0/3130737900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da00a89e0 con 0x7f6d8c003640 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6d9f400640 1 -- 172.21.15.97:0/3130737900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da00a8db0 con 0x7f6d8c003640 2024-09-10T21:34:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555716cc5eb0 con 0x7f6d8c003640 2024-09-10T21:34:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555716d6f0a0 con 0x7f6d8c003640 2024-09-10T21:34:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6d9f400640 1 -- 172.21.15.97:0/3130737900 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6da00cd060 con 0x7f6d8c003640 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.798+0000 7f6d9f400640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6d88070ff0 0x7f6d880734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6d9f400640 1 -- 172.21.15.97:0/3130737900 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f6da01307c0 con 0x7f6d8c003640 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6da6a00640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6d88070ff0 0x7f6d880734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555716c33ca0 0x555716d6d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x555716c52f80 con 0x555716c33ca0 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6da6000640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555716c33ca0 0x555716d6d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6da6a00640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6d88070ff0 0x7f6d880734b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6d98000c10 tx=0x7f6d98003040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6da6000640 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555716c33ca0 0x555716d6d150 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6da6000640 1 -- 172.21.15.97:0/3130737900 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v127'970 uv970 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6d9400dce0 con 0x555716c33ca0 2024-09-10T21:34:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555716c33ca0 msgr2=0x555716d6d150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555716c33ca0 0x555716d6d150 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6d88070ff0 msgr2=0x7f6d880734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6d88070ff0 0x7f6d880734b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6d98000c10 tx=0x7f6d98003040 comp rx=0 tx=0).stop 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8c003640 msgr2=0x555716d6bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 --2- 172.21.15.97:0/3130737900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8c003640 0x555716d6bc70 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f6da001e3c0 tx=0x7f6da0009620 comp rx=0 tx=0).stop 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6da7400640 1 -- 172.21.15.97:0/3130737900 reap_dead start 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 shutdown_connections 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 >> 172.21.15.97:0/3130737900 conn(0x555716c09140 msgr2=0x555716c37890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 shutdown_connections 2024-09-10T21:34:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:03.802+0000 7f6daa447080 1 -- 172.21.15.97:0/3130737900 wait complete. 2024-09-10T21:34:03.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:34:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- 172.21.15.97:0/3037457478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e5815fde0 msgr2=0x7f4e58166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/3037457478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e5815fde0 0x7f4e58166850 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009bc0 tx=0x7f4e5001cba0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- 172.21.15.97:0/3037457478 shutdown_connections 2024-09-10T21:34:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/3037457478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e5815fde0 0x7f4e58166850 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- 172.21.15.97:0/3037457478 >> 172.21.15.97:0/3037457478 conn(0x7f4e580b1780 msgr2=0x7f4e580b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- 172.21.15.97:0/3037457478 shutdown_connections 2024-09-10T21:34:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- 172.21.15.97:0/3037457478 wait complete. 2024-09-10T21:34:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 Processor -- start 2024-09-10T21:34:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- start start 2024-09-10T21:34:04.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e580ce5f0 0x7f4e58137a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e58141d20 0x7f4e5815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4e5815fde0 0x7f4e5815e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5816a130 con 0x7f4e58141d20 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58169fb0 con 0x7f4e580ce5f0 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5816a2b0 con 0x7f4e5815fde0 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e58141d20 0x7f4e5815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e58141d20 0x7f4e5815e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54168/0 (socket says 172.21.15.97:54168) 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 -- 172.21.15.97:0/2257192338 learned_addr learned my addr 172.21.15.97:0/2257192338 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5de00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4e5815fde0 0x7f4e5815e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5d400640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e580ce5f0 0x7f4e58137a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 -- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4e5815fde0 msgr2=0x7f4e5815e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4e5815fde0 0x7f4e5815e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 -- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e580ce5f0 msgr2=0x7f4e58137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e580ce5f0 0x7f4e58137a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 -- 172.21.15.97:0/2257192338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e58157370 con 0x7f4e58141d20 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5d400640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e580ce5f0 0x7f4e58137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.002+0000 7f4e5ca00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e58141d20 0x7f4e5815e260 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c450 tx=0x7f4e4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e55600640 1 -- 172.21.15.97:0/2257192338 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44005480 con 0x7f4e58141d20 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e55600640 1 -- 172.21.15.97:0/2257192338 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e44003600 con 0x7f4e58141d20 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e55600640 1 -- 172.21.15.97:0/2257192338 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44003a30 con 0x7f4e58141d20 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e58157650 con 0x7f4e58141d20 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e58157c10 con 0x7f4e58141d20 2024-09-10T21:34:04.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e580ce9d0 con 0x7f4e58141d20 2024-09-10T21:34:04.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e55600640 1 -- 172.21.15.97:0/2257192338 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4e44003bd0 con 0x7f4e58141d20 2024-09-10T21:34:04.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e55600640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e380725e0 0x7f4e38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e55600640 1 -- 172.21.15.97:0/2257192338 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f4e44093a60 con 0x7f4e58141d20 2024-09-10T21:34:04.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.006+0000 7f4e5d400640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e380725e0 0x7f4e38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.010+0000 7f4e5d400640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e380725e0 0x7f4e38074aa0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c001750 tx=0x7f4e4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.010+0000 7f4e55600640 1 -- 172.21.15.97:0/2257192338 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4e4405e3f0 con 0x7f4e58141d20 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.214+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4e58141360 con 0x7f4e58141d20 2024-09-10T21:34:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.450+0000 7f4e55600640 1 -- 172.21.15.97:0/2257192338 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v271) ==== 162+0+0 (secure 0 0 0) 0x7f4e4405dcf0 con 0x7f4e58141d20 2024-09-10T21:34:04.450 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:34:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.450+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e380725e0 msgr2=0x7f4e38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.450+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e380725e0 0x7f4e38074aa0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c001750 tx=0x7f4e4c009040 comp rx=0 tx=0).stop 2024-09-10T21:34:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.450+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e58141d20 msgr2=0x7f4e5815e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.450+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e58141d20 0x7f4e5815e260 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c450 tx=0x7f4e4400c910 comp rx=0 tx=0).stop 2024-09-10T21:34:04.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 shutdown_connections 2024-09-10T21:34:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4e380725e0 0x7f4e38074aa0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4e5815fde0 0x7f4e5815e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e58141d20 0x7f4e5815e260 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 --2- 172.21.15.97:0/2257192338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e580ce5f0 0x7f4e58137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 >> 172.21.15.97:0/2257192338 conn(0x7f4e580b1780 msgr2=0x7f4e5813e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 shutdown_connections 2024-09-10T21:34:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:04.454+0000 7f4e5fe00640 1 -- 172.21.15.97:0/2257192338 wait complete. 2024-09-10T21:34:04.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.022+0000 7f11d2e00640 1 -- 172.21.15.97:0/2747793589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce5f0 msgr2=0x7f11cc141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.022+0000 7f11d2e00640 1 --2- 172.21.15.97:0/2747793589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce5f0 0x7f11cc141d20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009840 tx=0x7f11bc01c740 comp rx=0 tx=0).stop 2024-09-10T21:34:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.022+0000 7f11d2e00640 1 -- 172.21.15.97:0/2747793589 shutdown_connections 2024-09-10T21:34:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.022+0000 7f11d2e00640 1 --2- 172.21.15.97:0/2747793589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce5f0 0x7f11cc141d20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.022+0000 7f11d2e00640 1 -- 172.21.15.97:0/2747793589 >> 172.21.15.97:0/2747793589 conn(0x7f11cc0b1780 msgr2=0x7f11cc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.022+0000 7f11d2e00640 1 -- 172.21.15.97:0/2747793589 shutdown_connections 2024-09-10T21:34:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.022+0000 7f11d2e00640 1 -- 172.21.15.97:0/2747793589 wait complete. 2024-09-10T21:34:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 Processor -- start 2024-09-10T21:34:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 -- start start 2024-09-10T21:34:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11cc0ce5f0 0x7f11cc0afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce9d0 0x7f11cc15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11cc15fde0 0x7f11cc15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc169d60 con 0x7f11cc15fde0 2024-09-10T21:34:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc169be0 con 0x7f11cc0ce9d0 2024-09-10T21:34:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc169ee0 con 0x7f11cc0ce5f0 2024-09-10T21:34:05.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11cc15fde0 0x7f11cc15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11cc15fde0 0x7f11cc15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54176/0 (socket says 172.21.15.97:54176) 2024-09-10T21:34:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 -- 172.21.15.97:0/3978330407 learned_addr learned my addr 172.21.15.97:0/3978330407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11cbe00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11cc0ce5f0 0x7f11cc0afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11cb400640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce9d0 0x7f11cc15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 -- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11cc0ce5f0 msgr2=0x7f11cc0afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11cc0ce5f0 0x7f11cc0afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 -- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce9d0 msgr2=0x7f11cc15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce9d0 0x7f11cc15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 -- 172.21.15.97:0/3978330407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc1f8e20 con 0x7f11cc15fde0 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d0e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11cc15fde0 0x7f11cc15a670 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c730 tx=0x7f11c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11c3e00640 1 -- 172.21.15.97:0/3978330407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4003b80 con 0x7f11cc15fde0 2024-09-10T21:34:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11c3e00640 1 -- 172.21.15.97:0/3978330407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11c4003d20 con 0x7f11cc15fde0 2024-09-10T21:34:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11c3e00640 1 -- 172.21.15.97:0/3978330407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400aab0 con 0x7f11cc15fde0 2024-09-10T21:34:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc1f9100 con 0x7f11cc15fde0 2024-09-10T21:34:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc1f96c0 con 0x7f11cc15fde0 2024-09-10T21:34:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.026+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc1f98c0 con 0x7f11cc15fde0 2024-09-10T21:34:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.030+0000 7f11c3e00640 1 -- 172.21.15.97:0/3978330407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f11c400ac50 con 0x7f11cc15fde0 2024-09-10T21:34:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.030+0000 7f11c3e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f11ac0725e0 0x7f11ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.030+0000 7f11c3e00640 1 -- 172.21.15.97:0/3978330407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f11c4093b10 con 0x7f11cc15fde0 2024-09-10T21:34:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.030+0000 7f11cbe00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f11ac0725e0 0x7f11ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:05.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.030+0000 7f11cbe00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f11ac0725e0 0x7f11ac074aa0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0af000 tx=0x7f11bc01af80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:05.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.034+0000 7f11c3e00640 1 -- 172.21.15.97:0/3978330407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f11cc1f98c0 con 0x7f11cc15fde0 2024-09-10T21:34:05.176 INFO:tasks.ceph.mds.a.smithi097.stderr: -51> 2024-09-10T21:34:00.422+0000 7f0f3c200640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T21:34:05.176 INFO:tasks.ceph.mds.a.smithi097.stderr: -41> 2024-09-10T21:34:00.422+0000 7f0f3c200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T21:34:05.214 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:34:05.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.238+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11cc141d20 con 0x7f11cc15fde0 2024-09-10T21:34:05.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.238+0000 7f11c3e00640 1 -- 172.21.15.97:0/3978330407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) ==== 95+0+4404 (secure 0 0 0) 0x7f11c405df40 con 0x7f11cc15fde0 2024-09-10T21:34:05.239 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:05.239 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":272,"btime":"2024-09-10T21:34:04:455211+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":272,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:04.455200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":272,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/2872418710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2872418710},{"type":"v1","addr":"172.21.15.97:6835","nonce":2872418710}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:05.239 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 272 2024-09-10T21:34:05.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f11ac0725e0 msgr2=0x7f11ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:05.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f11ac0725e0 0x7f11ac074aa0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0af000 tx=0x7f11bc01af80 comp rx=0 tx=0).stop 2024-09-10T21:34:05.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11cc15fde0 msgr2=0x7f11cc15a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11cc15fde0 0x7f11cc15a670 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c730 tx=0x7f11c400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 shutdown_connections 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f11ac0725e0 0x7f11ac074aa0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11cc15fde0 0x7f11cc15a670 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11cc0ce9d0 0x7f11cc15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 --2- 172.21.15.97:0/3978330407 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f11cc0ce5f0 0x7f11cc0afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 >> 172.21.15.97:0/3978330407 conn(0x7f11cc0b1780 msgr2=0x7f11cc15f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 shutdown_connections 2024-09-10T21:34:05.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:05.242+0000 7f11d2e00640 1 -- 172.21.15.97:0/3978330407 wait complete. 2024-09-10T21:34:05.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 272, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:34:04.455200+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9720}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9720': {'gid': 9720, 'name': 'c', 'rank': 0, 'incarnation': 272, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/2872418710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2872418710}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2872418710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:34:05.260 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9720:up:replay 2024-09-10T21:34:05.260 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:34:09.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:09.745 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:10.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 -- 172.21.15.97:0/2570907253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84280ce5f0 msgr2=0x7f8428141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 --2- 172.21.15.97:0/2570907253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84280ce5f0 0x7f8428141d20 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f84280725c0 tx=0x7f841801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 -- 172.21.15.97:0/2570907253 shutdown_connections 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 --2- 172.21.15.97:0/2570907253 >> v2:172.21.15.194:3300/0 conn(0x7f842815fc10 0x7f842815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 --2- 172.21.15.97:0/2570907253 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84280ce9d0 0x7f842815f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 --2- 172.21.15.97:0/2570907253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84280ce5f0 0x7f8428141d20 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 -- 172.21.15.97:0/2570907253 >> 172.21.15.97:0/2570907253 conn(0x7f84280b1780 msgr2=0x7f84280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 -- 172.21.15.97:0/2570907253 shutdown_connections 2024-09-10T21:34:10.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 -- 172.21.15.97:0/2570907253 wait complete. 2024-09-10T21:34:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 Processor -- start 2024-09-10T21:34:10.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.818+0000 7f842d800640 1 -- start start 2024-09-10T21:34:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84280ce5f0 0x7f84281661c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84280ce9d0 0x7f842813ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842815fc10 0x7f842815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84281401d0 con 0x7f842815fc10 2024-09-10T21:34:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8428140050 con 0x7f84280ce9d0 2024-09-10T21:34:10.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8428140350 con 0x7f84280ce5f0 2024-09-10T21:34:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842815fc10 0x7f842815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842815fc10 0x7f842815e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54152/0 (socket says 172.21.15.97:54152) 2024-09-10T21:34:10.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 -- 172.21.15.97:0/2607456426 learned_addr learned my addr 172.21.15.97:0/2607456426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 -- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84280ce5f0 msgr2=0x7f84281661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84280ce5f0 0x7f84281661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 -- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84280ce9d0 msgr2=0x7f842813ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84280ce9d0 0x7f842813ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 -- 172.21.15.97:0/2607456426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842813cf80 con 0x7f842815fc10 2024-09-10T21:34:10.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f8427400640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842815fc10 0x7f842815e260 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f842000b970 tx=0x7f842000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f841ea00640 1 -- 172.21.15.97:0/2607456426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420019070 con 0x7f842815fc10 2024-09-10T21:34:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f841ea00640 1 -- 172.21.15.97:0/2607456426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f842000cc50 con 0x7f842815fc10 2024-09-10T21:34:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8428164c20 con 0x7f842815fc10 2024-09-10T21:34:10.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f841ea00640 1 -- 172.21.15.97:0/2607456426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84200146a0 con 0x7f842815fc10 2024-09-10T21:34:10.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281651e0 con 0x7f842815fc10 2024-09-10T21:34:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842815f6d0 con 0x7f842815fc10 2024-09-10T21:34:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.822+0000 7f841ea00640 1 -- 172.21.15.97:0/2607456426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f842001d3b0 con 0x7f842815fc10 2024-09-10T21:34:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.826+0000 7f841ea00640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8408072610 0x7f8408074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:10.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.826+0000 7f841ea00640 1 -- 172.21.15.97:0/2607456426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f8420093fd0 con 0x7f842815fc10 2024-09-10T21:34:10.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.826+0000 7f8426a00640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8408072610 0x7f8408074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:10.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.826+0000 7f8426a00640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8408072610 0x7f8408074ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84180af000 tx=0x7f8418019660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:10.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:10.830+0000 7f841ea00640 1 -- 172.21.15.97:0/2607456426 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f842005e960 con 0x7f842815fc10 2024-09-10T21:34:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.026+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8428141360 con 0x7f842815fc10 2024-09-10T21:34:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.026+0000 7f841ea00640 1 -- 172.21.15.97:0/2607456426 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+5237 (secure 0 0 0) 0x7f842005e260 con 0x7f842815fc10 2024-09-10T21:34:11.027 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:11.027 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":275,"btime":"2024-09-10T21:34:07:470093+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1087733461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1087733461},{"type":"v1","addr":"172.21.15.97:6834","nonce":1087733461}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":275,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:07.470086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":272,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/2872418710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2872418710},{"type":"v1","addr":"172.21.15.97:6835","nonce":2872418710}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9720,"qdb_cluster":[9720]},"id":6}]} 2024-09-10T21:34:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 275 2024-09-10T21:34:11.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8408072610 msgr2=0x7f8408074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8408072610 0x7f8408074ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84180af000 tx=0x7f8418019660 comp rx=0 tx=0).stop 2024-09-10T21:34:11.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842815fc10 msgr2=0x7f842815e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842815fc10 0x7f842815e260 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f842000b970 tx=0x7f842000be30 comp rx=0 tx=0).stop 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 shutdown_connections 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8408072610 0x7f8408074ad0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f842815fc10 0x7f842815e260 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84280ce9d0 0x7f842813ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 --2- 172.21.15.97:0/2607456426 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84280ce5f0 0x7f84281661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 >> 172.21.15.97:0/2607456426 conn(0x7f84280b1780 msgr2=0x7f84280afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 shutdown_connections 2024-09-10T21:34:11.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.030+0000 7f842d800640 1 -- 172.21.15.97:0/2607456426 wait complete. 2024-09-10T21:34:11.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 275, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:34:07.470086+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9720}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9720': {'gid': 9720, 'name': 'c', 'rank': 0, 'incarnation': 272, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6835/2872418710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2872418710}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2872418710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9720, 'qdb_cluster': [9720]} 2024-09-10T21:34:11.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:34:11.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:34:11.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 -- 172.21.15.97:0/1389602769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 msgr2=0x7ff9500b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 --2- 172.21.15.97:0/1389602769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9500b37b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7ff94000adc0 tx=0x7ff940020470 comp rx=0 tx=0).stop 2024-09-10T21:34:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 -- 172.21.15.97:0/1389602769 shutdown_connections 2024-09-10T21:34:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 --2- 172.21.15.97:0/1389602769 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9500ceee0 0x7ff9500cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 --2- 172.21.15.97:0/1389602769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9500b37b0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 -- 172.21.15.97:0/1389602769 >> 172.21.15.97:0/1389602769 conn(0x7ff9500b1780 msgr2=0x7ff9500b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 -- 172.21.15.97:0/1389602769 shutdown_connections 2024-09-10T21:34:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.190+0000 7ff956400640 1 -- 172.21.15.97:0/1389602769 wait complete. 2024-09-10T21:34:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 Processor -- start 2024-09-10T21:34:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 -- start start 2024-09-10T21:34:11.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9500b3cf0 0x7ff95014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9500ceee0 0x7ff950154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9501e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff95006e740 con 0x7ff9501511f0 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff95006e5c0 con 0x7ff9500ceee0 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff95006e8c0 con 0x7ff9500b3cf0 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9501e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9501e6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54162/0 (socket says 172.21.15.97:54162) 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 -- 172.21.15.97:0/890949789 learned_addr learned my addr 172.21.15.97:0/890949789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94f400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9500b3cf0 0x7ff95014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff946a00640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9500ceee0 0x7ff950154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 -- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9500b3cf0 msgr2=0x7ff95014f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9500b3cf0 0x7ff95014f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 -- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9500ceee0 msgr2=0x7ff950154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9500ceee0 0x7ff950154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 -- 172.21.15.97:0/890949789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9501f26a0 con 0x7ff9501511f0 2024-09-10T21:34:11.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff94fe00640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9501e6d90 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7ff94800d780 tx=0x7ff94800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff947e00640 1 -- 172.21.15.97:0/890949789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948017070 con 0x7ff9501511f0 2024-09-10T21:34:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff947e00640 1 -- 172.21.15.97:0/890949789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff94800de90 con 0x7ff9501511f0 2024-09-10T21:34:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff947e00640 1 -- 172.21.15.97:0/890949789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948005020 con 0x7ff9501511f0 2024-09-10T21:34:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9501f2920 con 0x7ff9501511f0 2024-09-10T21:34:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.194+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501f2ee0 con 0x7ff9501511f0 2024-09-10T21:34:11.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff947e00640 1 -- 172.21.15.97:0/890949789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff9480051c0 con 0x7ff9501511f0 2024-09-10T21:34:11.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff947e00640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff92c0725e0 0x7ff92c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 Processor -- start 2024-09-10T21:34:11.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff947e00640 1 -- 172.21.15.97:0/890949789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7ff948093ff0 con 0x7ff9501511f0 2024-09-10T21:34:11.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 -- start start 2024-09-10T21:34:11.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff94f400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff92c0725e0 0x7ff92c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289db1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff94f400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff92c0725e0 0x7ff92c074aa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff9400b2000 tx=0x7ff94001ecb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d2898ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d2898f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563d288ab3f0 con 0x563d289dadc0 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563d287e8b20 con 0x563d2898f220 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.198+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563d287d7f50 con 0x563d289db6e0 2024-09-10T21:34:11.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289db1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289db1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54180/0 (socket says 172.21.15.97:54180) 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 -- 172.21.15.97:0/4243803707 learned_addr learned my addr 172.21.15.97:0/4243803707 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94fe00640 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d2898f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff946a00640 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d2898ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 -- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 msgr2=0x563d2898ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d2898ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 -- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 msgr2=0x563d2898f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d2898f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 -- 172.21.15.97:0/4243803707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2879f590 con 0x563d289dadc0 2024-09-10T21:34:11.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289db1a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7ff94000ace0 tx=0x7ff9400ba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebda19fcc85e906 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff93fe00640 1 -- 172.21.15.97:0/4243803707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400c3050 con 0x563d289dadc0 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff93fe00640 1 -- 172.21.15.97:0/4243803707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9400cd070 con 0x563d289dadc0 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff93fe00640 1 -- 172.21.15.97:0/4243803707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400b5040 con 0x563d289dadc0 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- 172.21.15.97:0/4243803707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 msgr2=0x563d289db1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289db1a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7ff94000ace0 tx=0x7ff9400ba000 comp rx=0 tx=0).stop 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- 172.21.15.97:0/4243803707 shutdown_connections 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d2898f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d2898ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 --2- 172.21.15.97:0/4243803707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289db1a0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- 172.21.15.97:0/4243803707 >> 172.21.15.97:0/4243803707 conn(0x563d28874130 msgr2=0x563d289b9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- 172.21.15.97:0/4243803707 shutdown_connections 2024-09-10T21:34:11.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- 172.21.15.97:0/4243803707 wait complete. 2024-09-10T21:34:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- start start 2024-09-10T21:34:11.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 Processor -- start 2024-09-10T21:34:11.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 0x563d289c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 0x563d289c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563d287725d0 con 0x563d289dadc0 2024-09-10T21:34:11.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563d287dbd60 con 0x563d289db6e0 2024-09-10T21:34:11.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563d2877e500 con 0x563d2898f220 2024-09-10T21:34:11.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff959714000 10 client.0 _ll_register_callbacks cb 0x563d28782810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94f400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 0x563d289c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54194/0 (socket says 172.21.15.97:54194) 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff946a00640 1 -- 172.21.15.97:0/2203600742 learned_addr learned my addr 172.21.15.97:0/2203600742 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.202+0000 7ff94fe00640 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 0x563d289c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff946a00640 1 -- 172.21.15.97:0/2203600742 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 msgr2=0x563d289c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff946a00640 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 0x563d289c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff946a00640 1 -- 172.21.15.97:0/2203600742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 msgr2=0x563d289c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff946a00640 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 0x563d289c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff946a00640 1 -- 172.21.15.97:0/2203600742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d28776170 con 0x563d289dadc0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff946a00640 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289c58a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7ff9380122f0 tx=0x7ff9380127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cf7aa091bc5779 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 1 -- 172.21.15.97:0/2203600742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938005320 con 0x563d289dadc0 2024-09-10T21:34:11.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 1 -- 172.21.15.97:0/2203600742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7ff938007050 con 0x563d289dadc0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d287700f0 con 0x563d289dadc0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 1 -- 172.21.15.97:0/2203600742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7ff938024050 con 0x563d289dadc0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 10 client.9801 fetch_fsmap learned FSMap version 275 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x563d2885cef0 con 0x563d289dadc0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 1 -- 172.21.15.97:0/2203600742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563d287700f0 con 0x563d289dadc0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 1 -- 172.21.15.97:0/2203600742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7ff938007250 con 0x563d289dadc0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 10 client.9801 fetch_fsmap finished waiting for FSMap version 275 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 10 client.9801 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 20 client.9801 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289c67f0 0x563d289ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 4 client.9801 mds_command: new command op to 9720 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 --> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d287eaa70 con 0x563d289c67f0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff94f400640 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289c67f0 0x563d289ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff94f400640 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289c67f0 0x563d289ec070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a94eff0e5daeaf server_cookie=58baee7b4e1383d0 in_seq=0 out_seq=0 2024-09-10T21:34:11.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 10 client.9801 ms_handle_connect on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 1 -- 172.21.15.97:0/2203600742 <== mds.0 v2:172.21.15.97:6833/2872418710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d287eaa70 con 0x563d289c67f0 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 10 client.9801 handle_command_reply: tid=0 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 client.9801 shutdown 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 msgr2=0x563d289c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289c58a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7ff9380122f0 tx=0x7ff9380127b0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 shutdown_connections 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289c67f0 0x563d289ec070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 0x563d289c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289c58a0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 --2- 172.21.15.97:0/2203600742 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 0x563d289c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff93cc00640 0 client.9801 ms_handle_reset on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 >> 172.21.15.97:0/2203600742 conn(0x563d28874130 msgr2=0x563d289898c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 shutdown_connections 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 1 -- 172.21.15.97:0/2203600742 wait complete. 2024-09-10T21:34:11.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.206+0000 7ff959714000 20 client.9801 trim_cache size 0 max 16384 2024-09-10T21:34:11.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 Processor -- start 2024-09-10T21:34:11.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 -- start start 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 0x563d28997a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d28997fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 0x563d289ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9500ce4a0 con 0x563d289dadc0 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563d287e0390 con 0x563d289db6e0 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563d2885ca60 con 0x563d2898f220 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d28997fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d28997fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54196/0 (socket says 172.21.15.97:54196) 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 -- 172.21.15.97:0/722554697 learned_addr learned my addr 172.21.15.97:0/722554697 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 -- 172.21.15.97:0/722554697 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 msgr2=0x563d28997a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 --2- 172.21.15.97:0/722554697 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 0x563d28997a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 -- 172.21.15.97:0/722554697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 msgr2=0x563d289ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 --2- 172.21.15.97:0/722554697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 0x563d289ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 -- 172.21.15.97:0/722554697 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d28833d90 con 0x563d289dadc0 2024-09-10T21:34:11.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff946a00640 1 --2- 172.21.15.97:0/722554697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d28997fa0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ff938006800 tx=0x7ff938001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d51cdf8560e203f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff93f400640 1 -- 172.21.15.97:0/722554697 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938003600 con 0x563d289dadc0 2024-09-10T21:34:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff93f400640 1 -- 172.21.15.97:0/722554697 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9380037a0 con 0x563d289dadc0 2024-09-10T21:34:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.226+0000 7ff93f400640 1 -- 172.21.15.97:0/722554697 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938003bb0 con 0x563d289dadc0 2024-09-10T21:34:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- 172.21.15.97:0/722554697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 msgr2=0x563d28997fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- 172.21.15.97:0/722554697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d28997fa0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ff938006800 tx=0x7ff938001020 comp rx=0 tx=0).stop 2024-09-10T21:34:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- 172.21.15.97:0/722554697 shutdown_connections 2024-09-10T21:34:11.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- 172.21.15.97:0/722554697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d289db6e0 0x563d289ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- 172.21.15.97:0/722554697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d28997fa0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- 172.21.15.97:0/722554697 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d2898f220 0x563d28997a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- 172.21.15.97:0/722554697 >> 172.21.15.97:0/722554697 conn(0x563d28874130 msgr2=0x563d289b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- 172.21.15.97:0/722554697 shutdown_connections 2024-09-10T21:34:11.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- 172.21.15.97:0/722554697 wait complete. 2024-09-10T21:34:11.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- start start 2024-09-10T21:34:11.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 Processor -- start 2024-09-10T21:34:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d289f0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d289ee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563d287db070 con 0x563d289dadc0 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563d28897730 con 0x563d2898f220 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563d287a3ec0 con 0x563d289db6e0 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 10 client.0 _ll_register_callbacks cb 0x7ff938001b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289f0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54202/0 (socket says 172.21.15.97:54202) 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 -- 172.21.15.97:0/732583291 learned_addr learned my addr 172.21.15.97:0/732583291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff94f400640 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d289f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff94fe00640 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d289ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 -- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 msgr2=0x563d289ee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d289ee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 -- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 msgr2=0x563d289f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d289f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 -- 172.21.15.97:0/732583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2879ea90 con 0x563d289dadc0 2024-09-10T21:34:11.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff946a00640 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289f0aa0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff9380014a0 tx=0x7ff938005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57abd6a746852372 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff93cc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff93cc00640 1 -- 172.21.15.97:0/732583291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938003d60 con 0x563d289dadc0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9380037a0 con 0x563d289dadc0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff93cc00640 1 -- 172.21.15.97:0/732583291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7ff938031450 con 0x563d289dadc0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff93cc00640 1 -- 172.21.15.97:0/732583291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7ff938031650 con 0x563d289dadc0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 10 client.9807 fetch_fsmap learned FSMap version 275 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x563d2877d050 con 0x563d289dadc0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff93cc00640 1 -- 172.21.15.97:0/732583291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9380037a0 con 0x563d289dadc0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff93cc00640 1 -- 172.21.15.97:0/732583291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7ff9380317f0 con 0x563d289dadc0 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 10 client.9807 fetch_fsmap finished waiting for FSMap version 275 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 10 client.9807 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 20 client.9807 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289eed50 0x563d289ef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.230+0000 7ff959714000 4 client.9807 mds_command: new command op to 9720 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 --> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] -- command(tid 0: {"prefix": "status"}) -- 0x563d287e6520 con 0x563d289eed50 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff94f400640 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289eed50 0x563d289ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff94f400640 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289eed50 0x563d289ef130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b26b3ab87230c5a server_cookie=9d72804d9ac36d98 in_seq=0 out_seq=0 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff93cc00640 10 client.9807 ms_handle_connect on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff93cc00640 1 -- 172.21.15.97:0/732583291 <== mds.0 v2:172.21.15.97:6833/2872418710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x563d287e6520 con 0x563d289eed50 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff93cc00640 10 client.9807 handle_command_reply: tid=0 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 client.9807 shutdown 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 msgr2=0x563d289f0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289f0aa0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff9380014a0 tx=0x7ff938005550 comp rx=0 tx=0).stop 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 shutdown_connections 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x563d289eed50 0x563d289ef130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563d289db6e0 0x563d289ee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d289dadc0 0x563d289f0aa0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff93cc00640 0 client.9807 ms_handle_reset on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 --2- 172.21.15.97:0/732583291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d2898f220 0x563d289f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 >> 172.21.15.97:0/732583291 conn(0x563d28874130 msgr2=0x563d28998280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 shutdown_connections 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 1 -- 172.21.15.97:0/732583291 wait complete. 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff959714000 20 client.9807 trim_cache size 0 max 16384 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9720, 2024-09-10T21:34:11.239 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.776973259, 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 275, 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 127, 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 127, 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.082070082 2024-09-10T21:34:11.240 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:34:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff92c0725e0 msgr2=0x7ff92c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff92c0725e0 0x7ff92c074aa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff9400b2000 tx=0x7ff94001ecb0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 msgr2=0x7ff9501e6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9501e6d90 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7ff94800d780 tx=0x7ff94800dc40 comp rx=0 tx=0).stop 2024-09-10T21:34:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 shutdown_connections 2024-09-10T21:34:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff92c0725e0 0x7ff92c074aa0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9501511f0 0x7ff9501e6d90 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9500ceee0 0x7ff950154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 --2- 172.21.15.97:0/890949789 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff9500b3cf0 0x7ff95014f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 >> 172.21.15.97:0/890949789 conn(0x7ff9500b1780 msgr2=0x7ff95015a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 shutdown_connections 2024-09-10T21:34:11.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.234+0000 7ff956400640 1 -- 172.21.15.97:0/890949789 wait complete. 2024-09-10T21:34:11.253 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:34:11.253 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2024-09-10T21:34:11.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:34:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 -- 172.21.15.97:0/2293184334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 msgr2=0x7fef84005130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 --2- 172.21.15.97:0/2293184334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 0x7fef84005130 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fef9000e400 tx=0x7fef9001d770 comp rx=0 tx=0).stop 2024-09-10T21:34:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fef9e800640 1 -- 172.21.15.97:0/2293184334 reap_dead start 2024-09-10T21:34:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 -- 172.21.15.97:0/2293184334 shutdown_connections 2024-09-10T21:34:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 -- 172.21.15.97:0/2293184334 >> 172.21.15.97:0/2293184334 conn(0x7fefa00b1780 msgr2=0x7fefa00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 -- 172.21.15.97:0/2293184334 shutdown_connections 2024-09-10T21:34:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 -- 172.21.15.97:0/2293184334 wait complete. 2024-09-10T21:34:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 Processor -- start 2024-09-10T21:34:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.398+0000 7fefa4c00640 1 -- start start 2024-09-10T21:34:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 0x7fefa01e4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefa014f520 0x7fefa014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefa01e3860 0x7fefa01e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fefa015ae40 con 0x7fef84004d30 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefa00b3ad0 con 0x7fefa01e3860 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fefa015afc0 con 0x7fefa014f520 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 0x7fefa01e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 0x7fefa01e4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54206/0 (socket says 172.21.15.97:54206) 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 -- 172.21.15.97:0/2123044142 learned_addr learned my addr 172.21.15.97:0/2123044142 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef95400640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefa014f520 0x7fefa014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9e800640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefa01e3860 0x7fefa01e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 -- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefa014f520 msgr2=0x7fefa014a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefa014f520 0x7fefa014a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 -- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefa01e3860 msgr2=0x7fefa01e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefa01e3860 0x7fefa01e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 -- 172.21.15.97:0/2123044142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa01e4bd0 con 0x7fef84004d30 2024-09-10T21:34:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef9de00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 0x7fefa01e4690 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fef90009fd0 tx=0x7fef900a6c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef96a00640 1 -- 172.21.15.97:0/2123044142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900bb070 con 0x7fef84004d30 2024-09-10T21:34:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef96a00640 1 -- 172.21.15.97:0/2123044142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef900a6d10 con 0x7fef84004d30 2024-09-10T21:34:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa01e3380 con 0x7fef84004d30 2024-09-10T21:34:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef96a00640 1 -- 172.21.15.97:0/2123044142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900b2040 con 0x7fef84004d30 2024-09-10T21:34:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa00c49f0 con 0x7fef84004d30 2024-09-10T21:34:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa00cf4b0 con 0x7fef84004d30 2024-09-10T21:34:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.402+0000 7fef96a00640 1 -- 172.21.15.97:0/2123044142 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fef900ae6a0 con 0x7fef84004d30 2024-09-10T21:34:11.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.406+0000 7fef96a00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef7c072610 0x7fef7c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.406+0000 7fef96a00640 1 -- 172.21.15.97:0/2123044142 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7fef9012f3f0 con 0x7fef84004d30 2024-09-10T21:34:11.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.406+0000 7fef95400640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef7c072610 0x7fef7c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.406+0000 7fef95400640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef7c072610 0x7fef7c074ad0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fef88001250 tx=0x7fef88001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.410+0000 7fef96a00640 1 -- 172.21.15.97:0/2123044142 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fefa01e3380 con 0x7fef84004d30 2024-09-10T21:34:11.435 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:34:11.434+0000 7f169a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:11.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.610+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fefa0070fa0 con 0x7fef84004d30 2024-09-10T21:34:11.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.610+0000 7fef96a00640 1 -- 172.21.15.97:0/2123044142 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v275) ==== 53+0+73 (secure 0 0 0) 0x7fef90017480 con 0x7fef84004d30 2024-09-10T21:34:11.611 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef7c072610 msgr2=0x7fef7c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef7c072610 0x7fef7c074ad0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fef88001250 tx=0x7fef88001120 comp rx=0 tx=0).stop 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 msgr2=0x7fefa01e4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 0x7fefa01e4690 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fef90009fd0 tx=0x7fef900a6c20 comp rx=0 tx=0).stop 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 shutdown_connections 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fef7c072610 0x7fef7c074ad0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefa01e3860 0x7fefa01e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fefa014f520 0x7fefa014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 --2- 172.21.15.97:0/2123044142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84004d30 0x7fefa01e4690 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 >> 172.21.15.97:0/2123044142 conn(0x7fefa00b1780 msgr2=0x7fefa015c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 shutdown_connections 2024-09-10T21:34:11.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11.614+0000 7fefa4c00640 1 -- 172.21.15.97:0/2123044142 wait complete. 2024-09-10T21:34:11.632 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:34:11.632 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:34:11.633 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:34:11.633 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:34:11.633 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:11.633 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:34:11.633 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:34:11.633 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:34:11.633 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:11.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:11.633 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:34:11.642 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:34:11.642 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:34:11.642 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:34:11.642 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.642 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:34:11.643 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41842sec preferred_lft 41842sec 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.644 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:34:11.645 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:34:11.645 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:34:11.645 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:34:11.645 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:34:11.645 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:34:11.645 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:11.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:11.646 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:34:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:34:11.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:11.712 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:34:11.766 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:11.767 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:34:11.768 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:34:11.768 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:34:11.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:34:11.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:34:11.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:34:11.768 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:34:11.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:34:11.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:34:11.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:11.901 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:34:11.910 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:34:11.911 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:34:11.911 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:11.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:11.911 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:34:11.975 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:34:11.975 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:12.027 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:12.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:34:12.028 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:12.078 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:34:12.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:12.079 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:12.168 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:12.168 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:12.169 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:12.184 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:12.184 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:12.184 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:12.184 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:34:12.184 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:34:12.193 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:12.193 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:12.193 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:12.193 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:34:12.193 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:34:12.193 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:34:12.194 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:34:12.194 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:34:12.198 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:34:12.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:12.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:12.218 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:34:12.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:34:12.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:34:12.315 INFO:teuthology.orchestra.run.smithi097.stdout:client9816 2024-09-10T21:34:12.316 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:12.316 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> import json 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-09-10T21:34:12.372 DEBUG:teuthology.orchestra.run.smithi097:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(e.errno) 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> print(json.dumps( 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> indent=2)) 2024-09-10T21:34:12.373 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:34:12.419 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:34:12.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T21:34:12.495 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T21:34:12.495 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-10T21:34:12.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:12.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- 172.21.15.97:0/887820931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f41511f0 msgr2=0x7fa4f40b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:12.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/887820931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f41511f0 0x7fa4f40b37b0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400aa10 tx=0x7fa4e4020420 comp rx=0 tx=0).stop 2024-09-10T21:34:12.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- 172.21.15.97:0/887820931 shutdown_connections 2024-09-10T21:34:12.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/887820931 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4f40b3cf0 0x7fa4f406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/887820931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f41511f0 0x7fa4f40b37b0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- 172.21.15.97:0/887820931 >> 172.21.15.97:0/887820931 conn(0x7fa4f40b1780 msgr2=0x7fa4f40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- 172.21.15.97:0/887820931 shutdown_connections 2024-09-10T21:34:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- 172.21.15.97:0/887820931 wait complete. 2024-09-10T21:34:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 Processor -- start 2024-09-10T21:34:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- start start 2024-09-10T21:34:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f406f180 0x7fa4f4139430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4f40b3cf0 0x7fa4f4147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4f41511f0 0x7fa4f4159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f406e740 con 0x7fa4f406f180 2024-09-10T21:34:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f406e5c0 con 0x7fa4f41511f0 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f8e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f406e8c0 con 0x7fa4f40b3cf0 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f406f180 0x7fa4f4139430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f406f180 0x7fa4f4139430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54228/0 (socket says 172.21.15.97:54228) 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 -- 172.21.15.97:0/184935783 learned_addr learned my addr 172.21.15.97:0/184935783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f2800640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4f41511f0 0x7fa4f4159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1400640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4f40b3cf0 0x7fa4f4147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 -- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4f40b3cf0 msgr2=0x7fa4f4147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4f40b3cf0 0x7fa4f4147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 -- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4f41511f0 msgr2=0x7fa4f4159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4f41511f0 0x7fa4f4159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 -- 172.21.15.97:0/184935783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f41f4b70 con 0x7fa4f406f180 2024-09-10T21:34:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f2800640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4f41511f0 0x7fa4f4159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1400640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4f40b3cf0 0x7fa4f4147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.686+0000 7fa4f1e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f406f180 0x7fa4f4139430 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400a960 tx=0x7fa4e40b0400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4ea000640 1 -- 172.21.15.97:0/184935783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40b5040 con 0x7fa4f406f180 2024-09-10T21:34:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4ea000640 1 -- 172.21.15.97:0/184935783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e40b0cd0 con 0x7fa4f406f180 2024-09-10T21:34:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4ea000640 1 -- 172.21.15.97:0/184935783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4005280 con 0x7fa4f406f180 2024-09-10T21:34:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f41f4e50 con 0x7fa4f406f180 2024-09-10T21:34:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f41f5410 con 0x7fa4f406f180 2024-09-10T21:34:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f406ec40 con 0x7fa4f406f180 2024-09-10T21:34:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4ea000640 1 -- 172.21.15.97:0/184935783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa4e40b8070 con 0x7fa4f406f180 2024-09-10T21:34:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.690+0000 7fa4ea000640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4d40725e0 0x7fa4d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.694+0000 7fa4ea000640 1 -- 172.21.15.97:0/184935783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7fa4e4131570 con 0x7fa4f406f180 2024-09-10T21:34:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.694+0000 7fa4f1400640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4d40725e0 0x7fa4d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.694+0000 7fa4f1400640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4d40725e0 0x7fa4d4074aa0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005fd0 tx=0x7fa4e0005da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.698+0000 7fa4ea000640 1 -- 172.21.15.97:0/184935783 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa4e4019f80 con 0x7fa4f406f180 2024-09-10T21:34:12.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4f414b2a0 con 0x7fa4f406f180 2024-09-10T21:34:12.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4ea000640 1 -- 172.21.15.97:0/184935783 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+5237 (secure 0 0 0) 0x7fa4e40fbf90 con 0x7fa4f406f180 2024-09-10T21:34:12.898 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:12.898 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":275,"btime":"2024-09-10T21:34:07:470093+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":9783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1087733461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1087733461},{"type":"v1","addr":"172.21.15.97:6834","nonce":1087733461}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":275,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:07.470086+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":272,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6835/2872418710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2872418710},{"type":"v1","addr":"172.21.15.97:6835","nonce":2872418710}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9720,"qdb_cluster":[9720]},"id":6}]} 2024-09-10T21:34:12.898 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 275 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4d40725e0 msgr2=0x7fa4d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4d40725e0 0x7fa4d4074aa0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005fd0 tx=0x7fa4e0005da0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f406f180 msgr2=0x7fa4f4139430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f406f180 0x7fa4f4139430 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400a960 tx=0x7fa4e40b0400 comp rx=0 tx=0).stop 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 shutdown_connections 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa4d40725e0 0x7fa4d4074aa0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4f41511f0 0x7fa4f4159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa4f40b3cf0 0x7fa4f4147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 --2- 172.21.15.97:0/184935783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4f406f180 0x7fa4f4139430 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.898+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 >> 172.21.15.97:0/184935783 conn(0x7fa4f40b1780 msgr2=0x7fa4f415cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:12.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.902+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 shutdown_connections 2024-09-10T21:34:12.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:12.902+0000 7fa4f8e00640 1 -- 172.21.15.97:0/184935783 wait complete. 2024-09-10T21:34:12.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-10T21:34:13.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 -- 172.21.15.97:0/1247828662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3580ccc60 msgr2=0x7fa3580b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 --2- 172.21.15.97:0/1247828662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3580ccc60 0x7fa3580b0490 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fa350009840 tx=0x7fa35001c7b0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 -- 172.21.15.97:0/1247828662 shutdown_connections 2024-09-10T21:34:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 --2- 172.21.15.97:0/1247828662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3580ccc60 0x7fa3580b0490 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 --2- 172.21.15.97:0/1247828662 >> v2:172.21.15.194:3300/0 conn(0x7fa35815eed0 0x7fa3580d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 -- 172.21.15.97:0/1247828662 >> 172.21.15.97:0/1247828662 conn(0x7fa3580d1cd0 msgr2=0x7fa3580d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 -- 172.21.15.97:0/1247828662 shutdown_connections 2024-09-10T21:34:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.062+0000 7fa360000640 1 -- 172.21.15.97:0/1247828662 wait complete. 2024-09-10T21:34:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 Processor -- start 2024-09-10T21:34:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 -- start start 2024-09-10T21:34:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ccc60 0x7fa358147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa3580cdd40 0x7fa35815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35815eed0 0x7fa358155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa358168880 con 0x7fa35815eed0 2024-09-10T21:34:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa358168700 con 0x7fa3580ccc60 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa358168a00 con 0x7fa3580cdd40 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35815eed0 0x7fa358155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35815eed0 0x7fa358155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54246/0 (socket says 172.21.15.97:54246) 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 -- 172.21.15.97:0/846665434 learned_addr learned my addr 172.21.15.97:0/846665434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35d600640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ccc60 0x7fa358147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35cc00640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa3580cdd40 0x7fa35815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 -- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa3580cdd40 msgr2=0x7fa35815c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa3580cdd40 0x7fa35815c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 -- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ccc60 msgr2=0x7fa358147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ccc60 0x7fa358147ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 -- 172.21.15.97:0/846665434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa358155140 con 0x7fa35815eed0 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35cc00640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa3580cdd40 0x7fa35815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:13.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa35e000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35815eed0 0x7fa358155ec0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fa3500193c0 tx=0x7fa3500a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa355600640 1 -- 172.21.15.97:0/846665434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500bc070 con 0x7fa35815eed0 2024-09-10T21:34:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa355600640 1 -- 172.21.15.97:0/846665434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3500044b0 con 0x7fa35815eed0 2024-09-10T21:34:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa355600640 1 -- 172.21.15.97:0/846665434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500b2040 con 0x7fa35815eed0 2024-09-10T21:34:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35815cf00 con 0x7fa35815eed0 2024-09-10T21:34:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.066+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581556c0 con 0x7fa35815eed0 2024-09-10T21:34:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa355600640 1 -- 172.21.15.97:0/846665434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa3500b5070 con 0x7fa35815eed0 2024-09-10T21:34:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa355600640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3380725e0 0x7fa338074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa355600640 1 -- 172.21.15.97:0/846665434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7fa35012d5d0 con 0x7fa35815eed0 2024-09-10T21:34:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 Processor -- start 2024-09-10T21:34:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 -- start start 2024-09-10T21:34:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa35d600640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3380725e0 0x7fa338074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa35d600640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3380725e0 0x7fa338074aa0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009ec0 tx=0x7fa34c009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d0fb60 0x564f37d0ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37cc5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37cc62c0 0x564f37cc66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564f37badce0 con 0x564f37d10480 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564f37b21c70 con 0x564f37cc62c0 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564f37b110a0 con 0x564f37d0fb60 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.070+0000 7fa35cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37cc5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37cc5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54254/0 (socket says 172.21.15.97:54254) 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 -- 172.21.15.97:0/712297746 learned_addr learned my addr 172.21.15.97:0/712297746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 -- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d0fb60 msgr2=0x564f37d0ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d0fb60 0x564f37d0ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 -- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37cc62c0 msgr2=0x564f37cc66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35e000640 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37cc62c0 0x564f37cc66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37cc62c0 0x564f37cc66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 -- 172.21.15.97:0/712297746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f37aec270 con 0x564f37d10480 2024-09-10T21:34:13.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35e000640 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37cc62c0 0x564f37cc66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35cc00640 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37cc5d80 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fa344001680 tx=0x7fa344001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d158a3fedf915fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa34a000640 1 -- 172.21.15.97:0/712297746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34400e950 con 0x564f37d10480 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa34a000640 1 -- 172.21.15.97:0/712297746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa344019070 con 0x564f37d10480 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa34a000640 1 -- 172.21.15.97:0/712297746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344014480 con 0x564f37d10480 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- 172.21.15.97:0/712297746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 msgr2=0x564f37cc5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37cc5d80 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fa344001680 tx=0x7fa344001b40 comp rx=0 tx=0).stop 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- 172.21.15.97:0/712297746 shutdown_connections 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37cc62c0 0x564f37cc66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37cc5d80 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 --2- 172.21.15.97:0/712297746 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d0fb60 0x564f37d0ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- 172.21.15.97:0/712297746 >> 172.21.15.97:0/712297746 conn(0x564f37cc2270 msgr2=0x564f37cc4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- 172.21.15.97:0/712297746 shutdown_connections 2024-09-10T21:34:13.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- 172.21.15.97:0/712297746 wait complete. 2024-09-10T21:34:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- start start 2024-09-10T21:34:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 Processor -- start 2024-09-10T21:34:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 0x564f37d16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d17010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564f37aab920 con 0x564f37d10480 2024-09-10T21:34:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564f37b14eb0 con 0x564f37d0fb60 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564f377d6820 con 0x564f37cc62c0 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa363454000 10 client.0 _ll_register_callbacks cb 0x564f37aef7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d17550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54258/0 (socket says 172.21.15.97:54258) 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.074+0000 7fa35e000640 1 -- 172.21.15.97:0/760027323 learned_addr learned my addr 172.21.15.97:0/760027323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35e000640 1 -- 172.21.15.97:0/760027323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 msgr2=0x564f37d16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35e000640 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 0x564f37d16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35e000640 1 -- 172.21.15.97:0/760027323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 msgr2=0x564f37d17010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35e000640 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d17010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35e000640 1 -- 172.21.15.97:0/760027323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f37ac4530 con 0x564f37d10480 2024-09-10T21:34:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35e000640 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d17550 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa350098390 tx=0x7fa3500f74b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e3eaf2bd1ebd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 1 -- 172.21.15.97:0/760027323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500fc3d0 con 0x564f37d10480 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 1 -- 172.21.15.97:0/760027323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7fa3500fc070 con 0x564f37d10480 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f37aa93e0 con 0x564f37d10480 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 1 -- 172.21.15.97:0/760027323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7fa350100df0 con 0x564f37d10480 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 10 client.9840 fetch_fsmap learned FSMap version 275 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x564f37aaf460 con 0x564f37d10480 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 1 -- 172.21.15.97:0/760027323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x564f37aa93e0 con 0x564f37d10480 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 1 -- 172.21.15.97:0/760027323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7fa350100a50 con 0x564f37d10480 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 10 client.9840 fetch_fsmap finished waiting for FSMap version 275 2024-09-10T21:34:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 10 client.9840 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 20 client.9840 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d1eea0 0x564f37d21290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 4 client.9840 mds_command: new command op to 9720 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 --> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f37b23bc0 con 0x564f37d1eea0 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35d600640 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d1eea0 0x564f37d21290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa35d600640 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d1eea0 0x564f37d21290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925bc38d439a5113 server_cookie=e532cfd3410cc93a in_seq=0 out_seq=0 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 10 client.9840 ms_handle_connect on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 1 -- 172.21.15.97:0/760027323 <== mds.0 v2:172.21.15.97:6833/2872418710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f37b23bc0 con 0x564f37d1eea0 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 10 client.9840 handle_command_reply: tid=0 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 client.9840 shutdown 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 msgr2=0x564f37d17550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d17550 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa350098390 tx=0x7fa3500f74b0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 shutdown_connections 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d1eea0 0x564f37d21290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d17550 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d17010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 --2- 172.21.15.97:0/760027323 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 0x564f37d16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa33ea00640 0 client.9840 ms_handle_reset on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 >> 172.21.15.97:0/760027323 conn(0x564f37cc2270 msgr2=0x564f37cc4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 shutdown_connections 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 1 -- 172.21.15.97:0/760027323 wait complete. 2024-09-10T21:34:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.078+0000 7fa363454000 20 client.9840 trim_cache size 0 max 16384 2024-09-10T21:34:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 Processor -- start 2024-09-10T21:34:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- start start 2024-09-10T21:34:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 0x564f37cc5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37cc5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d21290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564f37bd0fa0 con 0x564f37d10480 2024-09-10T21:34:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564f37ccf4f0 con 0x564f37d0fb60 2024-09-10T21:34:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564f37b20a40 con 0x564f37cc62c0 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d21290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d21290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54268/0 (socket says 172.21.15.97:54268) 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 -- 172.21.15.97:0/3112937320 learned_addr learned my addr 172.21.15.97:0/3112937320 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35d600640 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 0x564f37cc5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35cc00640 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37cc5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 -- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 msgr2=0x564f37cc5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 0x564f37cc5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 -- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 msgr2=0x564f37cc5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37cc5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 -- 172.21.15.97:0/3112937320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f37b96cf0 con 0x564f37d10480 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35cc00640 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37cc5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa35e000640 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d21290 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa3500f4c10 tx=0x7fa3500fc3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43b6bf0a16c9ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa349600640 1 -- 172.21.15.97:0/3112937320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350128070 con 0x564f37d10480 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa349600640 1 -- 172.21.15.97:0/3112937320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa350100960 con 0x564f37d10480 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa349600640 1 -- 172.21.15.97:0/3112937320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500f6c00 con 0x564f37d10480 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- 172.21.15.97:0/3112937320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 msgr2=0x564f37d21290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d21290 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa3500f4c10 tx=0x7fa3500fc3d0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- 172.21.15.97:0/3112937320 shutdown_connections 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37d10480 0x564f37d21290 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37cc5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 --2- 172.21.15.97:0/3112937320 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37cc62c0 0x564f37cc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- 172.21.15.97:0/3112937320 >> 172.21.15.97:0/3112937320 conn(0x564f37cc2270 msgr2=0x564f37ccb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- 172.21.15.97:0/3112937320 shutdown_connections 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.094+0000 7fa363454000 1 -- 172.21.15.97:0/3112937320 wait complete. 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 -- start start 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 Processor -- start 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37cc62c0 0x564f37d85620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d85b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d10480 0x564f37d860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564f37abc2e0 con 0x564f37cc62c0 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564f37add2e0 con 0x564f37d0fb60 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564f37b194e0 con 0x564f37d10480 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 10 client.0 _ll_register_callbacks cb 0x7fa350009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37cc62c0 0x564f37d85620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37cc62c0 0x564f37d85620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54282/0 (socket says 172.21.15.97:54282) 2024-09-10T21:34:13.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 -- 172.21.15.97:0/826195562 learned_addr learned my addr 172.21.15.97:0/826195562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35cc00640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d85b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35e000640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d10480 0x564f37d860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 -- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d10480 msgr2=0x564f37d860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d10480 0x564f37d860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 -- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 msgr2=0x564f37d85b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d85b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 -- 172.21.15.97:0/826195562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f37ac7e30 con 0x564f37cc62c0 2024-09-10T21:34:13.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35cc00640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d85b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35d600640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37cc62c0 0x564f37d85620 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x564f37b13f10 tx=0x7fa34c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdac2c7f8e8d7ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa33ea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa33ea00640 1 -- 172.21.15.97:0/826195562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c032060 con 0x564f37cc62c0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa33ea00640 1 -- 172.21.15.97:0/826195562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7fa34c01dcb0 con 0x564f37cc62c0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa350100960 con 0x564f37cc62c0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa33ea00640 1 -- 172.21.15.97:0/826195562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=275) ==== 24+0+0 (secure 0 0 0) 0x7fa34c02d3f0 con 0x564f37cc62c0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 10 client.9846 fetch_fsmap learned FSMap version 275 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=275,monmap=2+}) -- 0x564f37ab62b0 con 0x564f37cc62c0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa33ea00640 1 -- 172.21.15.97:0/826195562 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa350100960 con 0x564f37cc62c0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa33ea00640 1 -- 172.21.15.97:0/826195562 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 275) ==== 3502+0+0 (secure 0 0 0) 0x7fa34c02c070 con 0x564f37cc62c0 2024-09-10T21:34:13.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 10 client.9846 fetch_fsmap finished waiting for FSMap version 275 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 10 client.9846 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 20 client.9846 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d8c870 0x564f37d24dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 4 client.9846 mds_command: new command op to 9720 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 --> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x564f37b1f670 con 0x564f37d8c870 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.098+0000 7fa35cc00640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d8c870 0x564f37d24dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa33ea00640 10 client.9846 ms_handle_connect on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa35cc00640 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d8c870 0x564f37d24dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3aea697afe8b36 server_cookie=dcaa248759e35881 in_seq=0 out_seq=0 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa33ea00640 1 -- 172.21.15.97:0/826195562 <== mds.0 v2:172.21.15.97:6833/2872418710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+215 (crc 0 0 0) 0x564f37b1f670 con 0x564f37d8c870 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa33ea00640 10 client.9846 handle_command_reply: tid=0 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 client.9846 shutdown 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37cc62c0 msgr2=0x564f37d85620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37cc62c0 0x564f37d85620 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x564f37b13f10 tx=0x7fa34c004ac0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 shutdown_connections 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:6833/2872418710,v1:172.21.15.97:6835/2872418710] conn(0x564f37d8c870 0x564f37d24dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564f37d10480 0x564f37d860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564f37d0fb60 0x564f37d85b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa33ea00640 0 client.9846 ms_handle_reset on v2:172.21.15.97:6833/2872418710 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 --2- 172.21.15.97:0/826195562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564f37cc62c0 0x564f37d85620 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 >> 172.21.15.97:0/826195562 conn(0x564f37cc2270 msgr2=0x564f37cc4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 shutdown_connections 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 1 -- 172.21.15.97:0/826195562 wait complete. 2024-09-10T21:34:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa363454000 20 client.9846 trim_cache size 0 max 16384 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "dentry", 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 3830452684, 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511627776, 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: "frag": "*", 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: "dname": "sixmegs", 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_id": "head", 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/subdir/sixmegs" 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:34:13.106 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:34:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3380725e0 msgr2=0x7fa338074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3380725e0 0x7fa338074aa0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009ec0 tx=0x7fa34c009500 comp rx=0 tx=0).stop 2024-09-10T21:34:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35815eed0 msgr2=0x7fa358155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35815eed0 0x7fa358155ec0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fa3500193c0 tx=0x7fa3500a57d0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 shutdown_connections 2024-09-10T21:34:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa3380725e0 0x7fa338074aa0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35815eed0 0x7fa358155ec0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa3580cdd40 0x7fa35815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 --2- 172.21.15.97:0/846665434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ccc60 0x7fa358147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 >> 172.21.15.97:0/846665434 conn(0x7fa3580d1cd0 msgr2=0x7fa3581577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 shutdown_connections 2024-09-10T21:34:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.102+0000 7fa360000640 1 -- 172.21.15.97:0/846665434 wait complete. 2024-09-10T21:34:13.120 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2024-09-10T21:34:13.120 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:13.120 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:13.129 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:13.130 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:13.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:34:13.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.366+0000 7fdc7f000640 1 -- 172.21.15.97:0/3286130168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 msgr2=0x7fdc5c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.366+0000 7fdc7f000640 1 --2- 172.21.15.97:0/3286130168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 0x7fdc5c0051d0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009b90 tx=0x7fdc7001d270 comp rx=0 tx=0).stop 2024-09-10T21:34:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.366+0000 7fdc7d000640 1 -- 172.21.15.97:0/3286130168 reap_dead start 2024-09-10T21:34:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.366+0000 7fdc7f000640 1 -- 172.21.15.97:0/3286130168 shutdown_connections 2024-09-10T21:34:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.366+0000 7fdc7f000640 1 -- 172.21.15.97:0/3286130168 >> 172.21.15.97:0/3286130168 conn(0x7fdc780b1780 msgr2=0x7fdc780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.366+0000 7fdc7f000640 1 -- 172.21.15.97:0/3286130168 shutdown_connections 2024-09-10T21:34:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.366+0000 7fdc7f000640 1 -- 172.21.15.97:0/3286130168 wait complete. 2024-09-10T21:34:13.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 Processor -- start 2024-09-10T21:34:13.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 -- start start 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 0x7fdc7815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc7813f9a0 0x7fdc7815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc780af5d0 0x7fdc780afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78070360 con 0x7fdc5c004d50 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780701e0 con 0x7fdc780af5d0 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdc780704e0 con 0x7fdc7813f9a0 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 0x7fdc7815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 0x7fdc7815cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54306/0 (socket says 172.21.15.97:54306) 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 -- 172.21.15.97:0/1740332457 learned_addr learned my addr 172.21.15.97:0/1740332457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7d000640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc780af5d0 0x7fdc780afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc6f400640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc7813f9a0 0x7fdc7815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 -- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc7813f9a0 msgr2=0x7fdc7815a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc7813f9a0 0x7fdc7815a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 -- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc780af5d0 msgr2=0x7fdc780afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc780af5d0 0x7fdc780afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 -- 172.21.15.97:0/1740332457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc781f4780 con 0x7fdc5c004d50 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc77e00640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 0x7fdc7815cdf0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800e9f0 tx=0x7fdc6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc74c00640 1 -- 172.21.15.97:0/1740332457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68015070 con 0x7fdc5c004d50 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc74c00640 1 -- 172.21.15.97:0/1740332457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc6800ce00 con 0x7fdc5c004d50 2024-09-10T21:34:13.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc780ae4d0 con 0x7fdc5c004d50 2024-09-10T21:34:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc74c00640 1 -- 172.21.15.97:0/1740332457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68003be0 con 0x7fdc5c004d50 2024-09-10T21:34:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7815b390 con 0x7fdc5c004d50 2024-09-10T21:34:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.370+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc780aff90 con 0x7fdc5c004d50 2024-09-10T21:34:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.374+0000 7fdc74c00640 1 -- 172.21.15.97:0/1740332457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdc68011460 con 0x7fdc5c004d50 2024-09-10T21:34:13.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.374+0000 7fdc74c00640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc54072610 0x7fdc54074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.374+0000 7fdc74c00640 1 -- 172.21.15.97:0/1740332457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7fdc68094570 con 0x7fdc5c004d50 2024-09-10T21:34:13.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.374+0000 7fdc6f400640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc54072610 0x7fdc54074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.374+0000 7fdc6f400640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc54072610 0x7fdc54074ad0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdc780b0190 tx=0x7fdc600091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.378+0000 7fdc74c00640 1 -- 172.21.15.97:0/1740332457 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdc6805ea00 con 0x7fdc5c004d50 2024-09-10T21:34:13.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.554+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc780ae9f0 con 0x7fdc5c004d50 2024-09-10T21:34:13.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc74c00640 1 -- 172.21.15.97:0/1740332457 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 62 entries v127) ==== 81+0+3766 (secure 0 0 0) 0x7fdc780ae9f0 con 0x7fdc5c004d50 2024-09-10T21:34:13.556 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:34:13.556 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:34:13.556 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:34:13.557 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:34:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:34:13.559 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:34:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:34:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:34:13.562 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:34:13.562 INFO:teuthology.orchestra.run.smithi097.stderr:listed 62 entries 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc54072610 msgr2=0x7fdc54074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc54072610 0x7fdc54074ad0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdc780b0190 tx=0x7fdc600091e0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 msgr2=0x7fdc7815cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 0x7fdc7815cdf0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800e9f0 tx=0x7fdc6800edf0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 shutdown_connections 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc54072610 0x7fdc54074ad0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc780af5d0 0x7fdc780afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdc7813f9a0 0x7fdc7815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 --2- 172.21.15.97:0/1740332457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc5c004d50 0x7fdc7815cdf0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 >> 172.21.15.97:0/1740332457 conn(0x7fdc780b1780 msgr2=0x7fdc7813a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 shutdown_connections 2024-09-10T21:34:13.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.558+0000 7fdc7f000640 1 -- 172.21.15.97:0/1740332457 wait complete. 2024-09-10T21:34:13.578 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:34:13.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:13.579 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:34:13.601 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:34:13.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:13.602 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:34:13.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:34:13.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.838+0000 7f108ac00640 1 -- 172.21.15.97:0/1413426728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068003c70 msgr2=0x7f1068004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.838+0000 7f108ac00640 1 --2- 172.21.15.97:0/1413426728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068003c70 0x7f1068004070 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f1074001860 tx=0x7f107401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.838+0000 7f1088c00640 1 -- 172.21.15.97:0/1413426728 reap_dead start 2024-09-10T21:34:13.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.838+0000 7f108ac00640 1 -- 172.21.15.97:0/1413426728 shutdown_connections 2024-09-10T21:34:13.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.838+0000 7f108ac00640 1 -- 172.21.15.97:0/1413426728 >> 172.21.15.97:0/1413426728 conn(0x7f10840d1c30 msgr2=0x7f10840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:13.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.838+0000 7f108ac00640 1 -- 172.21.15.97:0/1413426728 shutdown_connections 2024-09-10T21:34:13.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.838+0000 7f108ac00640 1 -- 172.21.15.97:0/1413426728 wait complete. 2024-09-10T21:34:13.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 Processor -- start 2024-09-10T21:34:13.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 -- start start 2024-09-10T21:34:13.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1068003c70 0x7f10841580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840c7030 0x7f10840c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10841589c0 0x7f1084158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10841678e0 con 0x7f10840c7030 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f10840cda40 con 0x7f10841589c0 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1084167a60 con 0x7f1068003c70 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840c7030 0x7f10840c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840c7030 0x7f10840c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54326/0 (socket says 172.21.15.97:54326) 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 -- 172.21.15.97:0/2024423442 learned_addr learned my addr 172.21.15.97:0/2024423442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083e00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1068003c70 0x7f10841580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1088c00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10841589c0 0x7f1084158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 -- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1068003c70 msgr2=0x7f10841580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1068003c70 0x7f10841580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 -- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10841589c0 msgr2=0x7f1084158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10841589c0 0x7f1084158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 -- 172.21.15.97:0/2024423442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10840c7830 con 0x7f10840c7030 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083e00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1068003c70 0x7f10841580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1088c00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10841589c0 0x7f1084158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f1083400640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840c7030 0x7f10840c6170 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f1070000fc0 tx=0x7f107000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f107be00640 1 -- 172.21.15.97:0/2024423442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070015070 con 0x7f10840c7030 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f107be00640 1 -- 172.21.15.97:0/2024423442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10700109a0 con 0x7f10840c7030 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f107be00640 1 -- 172.21.15.97:0/2024423442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070010df0 con 0x7f10840c7030 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10840c7b10 con 0x7f10840c7030 2024-09-10T21:34:13.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10840c80d0 con 0x7f10840c7030 2024-09-10T21:34:13.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.842+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10840cc6d0 con 0x7f10840c7030 2024-09-10T21:34:13.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.846+0000 7f107be00640 1 -- 172.21.15.97:0/2024423442 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1070011460 con 0x7f10840c7030 2024-09-10T21:34:13.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.846+0000 7f107be00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1064072610 0x7f1064074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:13.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.846+0000 7f107be00640 1 -- 172.21.15.97:0/2024423442 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8941+0+0 (secure 0 0 0) 0x7f1070094490 con 0x7f10840c7030 2024-09-10T21:34:13.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.846+0000 7f1083e00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1064072610 0x7f1064074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:13.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.846+0000 7f1083e00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1064072610 0x7f1064074ad0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1074001830 tx=0x7f1074019720 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:13.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:13.850+0000 7f107be00640 1 -- 172.21.15.97:0/2024423442 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f107005eeb0 con 0x7f10840c7030 2024-09-10T21:34:14.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.050+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f10840c88b0 con 0x7f10840c7030 2024-09-10T21:34:14.049 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:14.050+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:34:14.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.050+0000 7f107be00640 1 -- 172.21.15.97:0/2024423442 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v276) ==== 161+0+0 (secure 0 0 0) 0x7f107005e7b0 con 0x7f10840c7030 2024-09-10T21:34:14.052 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:34:14.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1064072610 msgr2=0x7f1064074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:14.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1064072610 0x7f1064074ad0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1074001830 tx=0x7f1074019720 comp rx=0 tx=0).stop 2024-09-10T21:34:14.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840c7030 msgr2=0x7f10840c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:14.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840c7030 0x7f10840c6170 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f1070000fc0 tx=0x7f107000e960 comp rx=0 tx=0).stop 2024-09-10T21:34:14.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 shutdown_connections 2024-09-10T21:34:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1064072610 0x7f1064074ad0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10841589c0 0x7f1084158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840c7030 0x7f10840c6170 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 --2- 172.21.15.97:0/2024423442 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1068003c70 0x7f10841580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 >> 172.21.15.97:0/2024423442 conn(0x7f10840d1c30 msgr2=0x7f108406dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 shutdown_connections 2024-09-10T21:34:14.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.054+0000 7f108ac00640 1 -- 172.21.15.97:0/2024423442 wait complete. 2024-09-10T21:34:14.060 INFO:tasks.ceph.mds.c.smithi097.stderr: -574> 2024-09-10T21:34:11.434+0000 7f169a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:14.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:34:14.101 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:34:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 -- 172.21.15.97:0/2555476523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0b1560 msgr2=0x7f34fc0b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 --2- 172.21.15.97:0/2555476523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0b1560 0x7f34fc0b1970 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0735a0 tx=0x7f34f401f0f0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 -- 172.21.15.97:0/2555476523 shutdown_connections 2024-09-10T21:34:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 --2- 172.21.15.97:0/2555476523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0b1560 0x7f34fc0b1970 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 --2- 172.21.15.97:0/2555476523 >> v2:172.21.15.194:3300/0 conn(0x7f34fc0ce830 0x7f34fc0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 -- 172.21.15.97:0/2555476523 >> 172.21.15.97:0/2555476523 conn(0x7f34fc0b37c0 msgr2=0x7f34fc0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 -- 172.21.15.97:0/2555476523 shutdown_connections 2024-09-10T21:34:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.222+0000 7f3501600640 1 -- 172.21.15.97:0/2555476523 wait complete. 2024-09-10T21:34:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 Processor -- start 2024-09-10T21:34:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 -- start start 2024-09-10T21:34:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34fc0b1560 0x7f34fc0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0ce830 0x7f34fc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34fc0cec10 0x7f34fc0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:14.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc0ce1a0 con 0x7f34fc0ce830 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc0cde60 con 0x7f34fc0b1560 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f34fc0ce320 con 0x7f34fc0cec10 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0ce830 0x7f34fc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0ce830 0x7f34fc15eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54350/0 (socket says 172.21.15.97:54350) 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 -- 172.21.15.97:0/2698189244 learned_addr learned my addr 172.21.15.97:0/2698189244 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34fb400640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34fc0cec10 0x7f34fc0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34faa00640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34fc0b1560 0x7f34fc0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 -- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34fc0cec10 msgr2=0x7f34fc0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34fc0cec10 0x7f34fc0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 -- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34fc0b1560 msgr2=0x7f34fc0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34fc0b1560 0x7f34fc0c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 -- 172.21.15.97:0/2698189244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc0c68e0 con 0x7f34fc0ce830 2024-09-10T21:34:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f2000640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0ce830 0x7f34fc15eef0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c3c0 tx=0x7f34e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f3400640 1 -- 172.21.15.97:0/2698189244 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e40052d0 con 0x7f34fc0ce830 2024-09-10T21:34:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f3400640 1 -- 172.21.15.97:0/2698189244 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e4007020 con 0x7f34fc0ce830 2024-09-10T21:34:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f34f3400640 1 -- 172.21.15.97:0/2698189244 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4007470 con 0x7f34fc0ce830 2024-09-10T21:34:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc0c6bc0 con 0x7f34fc0ce830 2024-09-10T21:34:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.226+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc0c7180 con 0x7f34fc0ce830 2024-09-10T21:34:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.230+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34fc0cf800 con 0x7f34fc0ce830 2024-09-10T21:34:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.230+0000 7f34f3400640 1 -- 172.21.15.97:0/2698189244 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f34e40067e0 con 0x7f34fc0ce830 2024-09-10T21:34:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.230+0000 7f34f3400640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34d80725e0 0x7f34d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:14.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.230+0000 7f34f3400640 1 -- 172.21.15.97:0/2698189244 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9027+0+0 (secure 0 0 0) 0x7f34e40932a0 con 0x7f34fc0ce830 2024-09-10T21:34:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.230+0000 7f34faa00640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34d80725e0 0x7f34d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:14.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.230+0000 7f34faa00640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34d80725e0 0x7f34d8074aa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009f60 tx=0x7f34ec009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:14.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.234+0000 7f34f3400640 1 -- 172.21.15.97:0/2698189244 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34e4001090 con 0x7f34fc0ce830 2024-09-10T21:34:14.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:14.462+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f34fc0c7350 con 0x7f34fc0ce830 2024-09-10T21:34:15.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.058+0000 7f34f3400640 1 -- 172.21.15.97:0/2698189244 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v277) ==== 107+0+0 (secure 0 0 0) 0x7f34e405d670 con 0x7f34fc0ce830 2024-09-10T21:34:15.058 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.058+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34d80725e0 msgr2=0x7f34d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.058+0000 7f3501600640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34d80725e0 0x7f34d8074aa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009f60 tx=0x7f34ec009500 comp rx=0 tx=0).stop 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.058+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0ce830 msgr2=0x7f34fc15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.058+0000 7f3501600640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0ce830 0x7f34fc15eef0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c3c0 tx=0x7f34e400c880 comp rx=0 tx=0).stop 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 shutdown_connections 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f34d80725e0 0x7f34d8074aa0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f34fc0cec10 0x7f34fc0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34fc0ce830 0x7f34fc15eef0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 --2- 172.21.15.97:0/2698189244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34fc0b1560 0x7f34fc0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 >> 172.21.15.97:0/2698189244 conn(0x7f34fc0b37c0 msgr2=0x7f34fc0b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 shutdown_connections 2024-09-10T21:34:15.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.062+0000 7f3501600640 1 -- 172.21.15.97:0/2698189244 wait complete. 2024-09-10T21:34:15.082 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:34:15.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 Processor -- start 2024-09-10T21:34:15.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 -- start start 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 0x55ddb7ad61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7aba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7a2e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ddb79a9f40 con 0x55ddb7ab72a0 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ddb79b6d00 con 0x55ddb7ab6b00 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3d5b4080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ddb794ba00 con 0x55ddb7a2c940 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7a2e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7a2e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54360/0 (socket says 172.21.15.97:54360) 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 -- 172.21.15.97:0/3875181453 learned_addr learned my addr 172.21.15.97:0/3875181453 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a33e00640 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7aba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3a800640 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 0x55ddb7ad61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 -- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 msgr2=0x55ddb7ad61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 0x55ddb7ad61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 -- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 msgr2=0x55ddb7aba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7aba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a3b200640 1 -- 172.21.15.97:0/3875181453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb7ad5570 con 0x55ddb7ab72a0 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.170+0000 7f1a33e00640 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7aba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7a2e130 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f1a340ae040 tx=0x7f1a34018160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25b0cc565de29da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a39400640 1 -- 172.21.15.97:0/3875181453 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340bb030 con 0x55ddb7ab72a0 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a39400640 1 -- 172.21.15.97:0/3875181453 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a34004110 con 0x55ddb7ab72a0 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a39400640 1 -- 172.21.15.97:0/3875181453 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340b0030 con 0x55ddb7ab72a0 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/3875181453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 msgr2=0x55ddb7a2e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7a2e130 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f1a340ae040 tx=0x7f1a34018160 comp rx=0 tx=0).stop 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/3875181453 shutdown_connections 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7a2e130 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7aba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/3875181453 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 0x55ddb7ad61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/3875181453 >> 172.21.15.97:0/3875181453 conn(0x55ddb7a2b520 msgr2=0x55ddb79fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/3875181453 shutdown_connections 2024-09-10T21:34:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/3875181453 wait complete. 2024-09-10T21:34:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 Processor -- start 2024-09-10T21:34:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- start start 2024-09-10T21:34:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 0x55ddb7a2fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7a40c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7ab59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ddb7ad5310 con 0x55ddb7ab72a0 2024-09-10T21:34:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ddb7ad50b0 con 0x55ddb7ab6b00 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3d5b4080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ddb7ad5810 con 0x55ddb7a2c940 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7ab59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7ab59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54362/0 (socket says 172.21.15.97:54362) 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 learned_addr learned my addr 172.21.15.97:0/2199457438 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a33e00640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7a40c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3a800640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 0x55ddb7a2fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 msgr2=0x55ddb7a2fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ddb7a2c940 0x55ddb7a2fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 msgr2=0x55ddb7a40c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ddb7ab6b00 0x55ddb7a40c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb79fd0e0 con 0x55ddb7ab72a0 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a3b200640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7ab59f0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f1a340ba040 tx=0x7f1a340acf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a33400640 1 -- 172.21.15.97:0/2199457438 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340bb040 con 0x55ddb7ab72a0 2024-09-10T21:34:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a33400640 1 -- 172.21.15.97:0/2199457438 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a340c5070 con 0x55ddb7ab72a0 2024-09-10T21:34:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.174+0000 7f1a33400640 1 -- 172.21.15.97:0/2199457438 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340b7780 con 0x55ddb7ab72a0 2024-09-10T21:34:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ddb7a2f2e0 con 0x55ddb7ab72a0 2024-09-10T21:34:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb79fcda0 con 0x55ddb7ab72a0 2024-09-10T21:34:15.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a33400640 1 -- 172.21.15.97:0/2199457438 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1a340b6070 con 0x55ddb7ab72a0 2024-09-10T21:34:15.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a33400640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a18070fc0 0x7f1a18073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a33400640 1 -- 172.21.15.97:0/2199457438 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9027+0+0 (secure 0 0 0) 0x7f1a3412e6c0 con 0x55ddb7ab72a0 2024-09-10T21:34:15.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a3a800640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a18070fc0 0x7f1a18073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.179 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:34:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55ddb7ab5f30 0x55ddb7a43240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7ad1270 con 0x55ddb7ab5f30 2024-09-10T21:34:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a33e00640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55ddb7ab5f30 0x55ddb7a43240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.178+0000 7f1a3a800640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a18070fc0 0x7f1a18073480 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c002ef0 tx=0x7f1a2c002750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.182+0000 7f1a33e00640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55ddb7ab5f30 0x55ddb7a43240 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.182+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v128'971 uv944 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.182+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7a43780 con 0x55ddb7ab5f30 2024-09-10T21:34:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.186+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v128'972 uv972 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.186+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7a43b20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.185 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:34:15.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.186+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v128'973 uv973 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.186+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7a4e0 con 0x55ddb7ab5f30 2024-09-10T21:34:15.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.186+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'974 uv974 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.186+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7a880 con 0x55ddb7ab5f30 2024-09-10T21:34:15.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.190+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'975 uv975 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.190+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b794d0 con 0x55ddb7ab5f30 2024-09-10T21:34:15.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.190+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'976 uv976 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.190+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b798f0 con 0x55ddb7ab5f30 2024-09-10T21:34:15.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.194+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v128'977 uv950 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.194+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b79ce0 con 0x55ddb7ab5f30 2024-09-10T21:34:15.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.194+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v128'978 uv978 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.194+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79ce0 con 0x55ddb7ab5f30 2024-09-10T21:34:15.195 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:34:15.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.194+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v128'979 uv979 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.198+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.198+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'980 uv980 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.198+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.198+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'981 uv981 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.198+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.202+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'982 uv982 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.202+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.202+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v128'983 uv970 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.202+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.206+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'984 uv984 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.206+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.205 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:34:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.206+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v128'985 uv985 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.206+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.206+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'986 uv986 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.206+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.210+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'987 uv987 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.210+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b79c90 con 0x55ddb7ab5f30 2024-09-10T21:34:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.210+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'988 uv988 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.210+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.214+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'989 uv989 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.214+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.214+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v128'990 uv963 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.214+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.214+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v128'991 uv991 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.214+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.216 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:34:15.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.218+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v128'992 uv992 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.218+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7bc30 con 0x55ddb7ab5f30 2024-09-10T21:34:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.218+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'993 uv993 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.218+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7bc30 con 0x55ddb7ab5f30 2024-09-10T21:34:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.218+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'994 uv994 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.222+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.222+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'995 uv995 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.222+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.222+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v128'996 uv969 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.222+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.226+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v128'997 uv997 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.226+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.226 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:34:15.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.226+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v128'998 uv998 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.226+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7bc30 con 0x55ddb7ab5f30 2024-09-10T21:34:15.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.226+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'999 uv999 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.226+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7bc30 con 0x55ddb7ab5f30 2024-09-10T21:34:15.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.230+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'1000 uv1000 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.230+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ac20 con 0x55ddb7ab5f30 2024-09-10T21:34:15.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.230+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'1001 uv1001 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7ab5f30 2024-09-10T21:34:15.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.230+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ddb7b7b130 0x55ddb7b7d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.230+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7da60 con 0x55ddb7b7b130 2024-09-10T21:34:15.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.230+0000 7f1a3a800640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ddb7b7b130 0x55ddb7b7d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.230+0000 7f1a3a800640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ddb7b7b130 0x55ddb7b7d520 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.234+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'705 uv684 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.234+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b7de00 con 0x55ddb7b7b130 2024-09-10T21:34:15.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.234+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'706 uv706 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.234+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7e1a0 con 0x55ddb7b7b130 2024-09-10T21:34:15.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.238+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'707 uv707 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.236 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:34:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.238+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7e9f0 con 0x55ddb7b7b130 2024-09-10T21:34:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.238+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v128'708 uv708 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.238+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7ed90 con 0x55ddb7b7b130 2024-09-10T21:34:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.238+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'709 uv709 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.238+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f130 con 0x55ddb7b7b130 2024-09-10T21:34:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.242+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v128'710 uv703 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.242+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b7f4d0 con 0x55ddb7b7b130 2024-09-10T21:34:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.242+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v128'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.242+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.242+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v128'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.243 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:34:15.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.242+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.246+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.246+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.246+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.246+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.246+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v128'715 uv704 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.246+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.250+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v128'716 uv716 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.250+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.250+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v128'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.249 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:34:15.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.250+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b83a40 con 0x55ddb7b7b130 2024-09-10T21:34:15.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.250+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v128'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.250+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b83a40 con 0x55ddb7b7b130 2024-09-10T21:34:15.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.254+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.254+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.254+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v128'720 uv700 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.254+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.254+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v128'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.254+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.255 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:34:15.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.254+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v128'722 uv722 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b80880 con 0x55ddb7b7b130 2024-09-10T21:34:15.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b80880 con 0x55ddb7b7b130 2024-09-10T21:34:15.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b7f870 con 0x55ddb7b7b130 2024-09-10T21:34:15.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b7b130 2024-09-10T21:34:15.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ddb7b80140 0x55ddb7b82530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.258+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b82a70 con 0x55ddb7b80140 2024-09-10T21:34:15.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.262+0000 7f1a33e00640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ddb7b80140 0x55ddb7b82530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.262+0000 7f1a33e00640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ddb7b80140 0x55ddb7b82530 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.262+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.262+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b82a70 con 0x55ddb7b80140 2024-09-10T21:34:15.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.262+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.262+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b82e60 con 0x55ddb7b80140 2024-09-10T21:34:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.264 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:34:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b83e70 con 0x55ddb7b80140 2024-09-10T21:34:15.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b83e70 con 0x55ddb7b80140 2024-09-10T21:34:15.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b82e60 con 0x55ddb7b80140 2024-09-10T21:34:15.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55ddb7b834e0 0x55ddb7b858d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b85e50 con 0x55ddb7b834e0 2024-09-10T21:34:15.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.266+0000 7f1a3b200640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55ddb7b834e0 0x55ddb7b858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.270+0000 7f1a3b200640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55ddb7b834e0 0x55ddb7b858d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.270+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'142 uv141 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.270+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b86360 con 0x55ddb7b834e0 2024-09-10T21:34:15.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.270+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'143 uv143 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.270+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b86860 con 0x55ddb7b834e0 2024-09-10T21:34:15.273 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:34:15.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.274+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'144 uv144 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.274+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87120 con 0x55ddb7b834e0 2024-09-10T21:34:15.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.274+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'145 uv145 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.274+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b874c0 con 0x55ddb7b80140 2024-09-10T21:34:15.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.274+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.274+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b874c0 con 0x55ddb7b80140 2024-09-10T21:34:15.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b874c0 con 0x55ddb7b80140 2024-09-10T21:34:15.277 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:34:15.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b884d0 con 0x55ddb7b80140 2024-09-10T21:34:15.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b884d0 con 0x55ddb7b80140 2024-09-10T21:34:15.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.278+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b874c0 con 0x55ddb7b80140 2024-09-10T21:34:15.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.282+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.282+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b874c0 con 0x55ddb7b834e0 2024-09-10T21:34:15.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.282+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v128'349 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.282+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b87860 con 0x55ddb7b834e0 2024-09-10T21:34:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.282+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v128'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.282+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b834e0 2024-09-10T21:34:15.286 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:34:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.286+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v128'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.286+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b88c10 con 0x55ddb7b834e0 2024-09-10T21:34:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.286+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.286+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b88c10 con 0x55ddb7b834e0 2024-09-10T21:34:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.286+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.286+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b834e0 2024-09-10T21:34:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b834e0 2024-09-10T21:34:15.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v128'355 uv348 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b834e0 2024-09-10T21:34:15.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v128'356 uv356 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b834e0 2024-09-10T21:34:15.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v128'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.292 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:34:15.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.290+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b834e0 2024-09-10T21:34:15.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.294+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.294+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b834e0 2024-09-10T21:34:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.294+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.294+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b80140 2024-09-10T21:34:15.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.294+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v128'158 uv157 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.294+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b80140 2024-09-10T21:34:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.298+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v128'159 uv159 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.298+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b80140 2024-09-10T21:34:15.297 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:34:15.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.298+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v128'160 uv160 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.298+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b80140 2024-09-10T21:34:15.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.298+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'161 uv161 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.298+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b87c00 con 0x55ddb7b80140 2024-09-10T21:34:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.302+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'162 uv162 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b80140 2024-09-10T21:34:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.302+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ddb7b88280 0x55ddb7b8a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.302+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8ac80 con 0x55ddb7b88280 2024-09-10T21:34:15.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.302+0000 7f1a3a800640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ddb7b88280 0x55ddb7b8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.302+0000 7f1a3a800640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ddb7b88280 0x55ddb7b8a670 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.302+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.302+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8ad40 con 0x55ddb7b88280 2024-09-10T21:34:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.305 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:34:15.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8c140 con 0x55ddb7b88280 2024-09-10T21:34:15.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8c140 con 0x55ddb7b88280 2024-09-10T21:34:15.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.306+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.310+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.310+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.310+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v128'314 uv308 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.310+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.310+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v128'315 uv315 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.310+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.314+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v128'316 uv316 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.312 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:34:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.314+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.314+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'317 uv317 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.314+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.314+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'318 uv318 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.314+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.318+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v128'319 uv313 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.318+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.318+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v128'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.318+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.318 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:34:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.318+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v128'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.318+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8b130 con 0x55ddb7b88280 2024-09-10T21:34:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.322+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.322+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8df20 con 0x55ddb7b88280 2024-09-10T21:34:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.322+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.322+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dee0 con 0x55ddb7b834e0 2024-09-10T21:34:15.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.322+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v128'137 uv135 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.322+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8dee0 con 0x55ddb7b834e0 2024-09-10T21:34:15.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'138 uv138 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dee0 con 0x55ddb7b834e0 2024-09-10T21:34:15.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v128'139 uv139 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.325 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:34:15.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dee0 con 0x55ddb7b834e0 2024-09-10T21:34:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'140 uv140 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dea0 con 0x55ddb7b834e0 2024-09-10T21:34:15.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v128'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.326+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8dea0 con 0x55ddb7b834e0 2024-09-10T21:34:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.330+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v128'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.330+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dea0 con 0x55ddb7b834e0 2024-09-10T21:34:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.330+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v128'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.330 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:34:15.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.330+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dea0 con 0x55ddb7b834e0 2024-09-10T21:34:15.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.330+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.330+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dea0 con 0x55ddb7b834e0 2024-09-10T21:34:15.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8dea0 con 0x55ddb7b834e0 2024-09-10T21:34:15.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a34002050 con 0x55ddb7b834e0 2024-09-10T21:34:15.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55ddb7b8dea0 0x55ddb7b8e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b8dea0 2024-09-10T21:34:15.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a33e00640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55ddb7b8dea0 0x55ddb7b8e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a33e00640 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55ddb7b8dea0 0x55ddb7b8e5e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v128'161 uv160 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b8dea0 2024-09-10T21:34:15.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.334+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b8dea0 2024-09-10T21:34:15.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.338+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v128'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b8dea0 2024-09-10T21:34:15.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.338+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b8dea0 2024-09-10T21:34:15.337 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:34:15.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.338+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v128'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b8dea0 2024-09-10T21:34:15.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.338+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b8dea0 2024-09-10T21:34:15.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.338+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b8dea0 2024-09-10T21:34:15.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.338+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b8dea0 2024-09-10T21:34:15.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a33e00640 1 -- 172.21.15.97:0/2199457438 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a24010690 con 0x55ddb7b8dea0 2024-09-10T21:34:15.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v128'389 uv380 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v128'390 uv390 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v128'391 uv391 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.343 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:34:15.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.342+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.346+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v128'392 uv392 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.346+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.346+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'393 uv393 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.346+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.346+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v128'394 uv387 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.346+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.350+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v128'395 uv395 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.350+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.349 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:34:15.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.350+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v128'396 uv396 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.350+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.350+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'397 uv397 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.350+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'398 uv398 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8da40 con 0x55ddb7b88280 2024-09-10T21:34:15.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'399 uv399 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55ddb7b8eb40 con 0x55ddb7b88280 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3a800640 1 -- 172.21.15.97:0/2199457438 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'400 uv400 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a2c00d840 con 0x55ddb7b88280 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ddb7b80140 msgr2=0x55ddb7b82530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55ddb7b80140 0x55ddb7b82530 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ddb7b88280 msgr2=0x55ddb7b8a670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55ddb7b88280 0x55ddb7b8a670 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55ddb7b8dea0 msgr2=0x55ddb7b8e5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55ddb7b8dea0 0x55ddb7b8e5e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55ddb7b834e0 msgr2=0x55ddb7b858d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55ddb7b834e0 0x55ddb7b858d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ddb7b7b130 msgr2=0x55ddb7b7d520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 reap_dead start 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55ddb7b7b130 0x55ddb7b7d520 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3b200640 1 -- 172.21.15.97:0/2199457438 reap_dead start 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55ddb7ab5f30 msgr2=0x55ddb7a43240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55ddb7ab5f30 0x55ddb7a43240 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a18070fc0 msgr2=0x7f1a18073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a18070fc0 0x7f1a18073480 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c002ef0 tx=0x7f1a2c002750 comp rx=0 tx=0).stop 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 msgr2=0x55ddb7ab59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7ab59f0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f1a340ba040 tx=0x7f1a340acf70 comp rx=0 tx=0).stop 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 shutdown_connections 2024-09-10T21:34:15.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55ddb7ab5f30 0x55ddb7a43240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1a18070fc0 0x7f1a18073480 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 --2- 172.21.15.97:0/2199457438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ddb7ab72a0 0x55ddb7ab59f0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 >> 172.21.15.97:0/2199457438 conn(0x55ddb7a2b520 msgr2=0x55ddb7a49ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:15.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 shutdown_connections 2024-09-10T21:34:15.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.354+0000 7f1a3d5b4080 1 -- 172.21.15.97:0/2199457438 wait complete. 2024-09-10T21:34:15.359 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:34:15.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.450+0000 7fe55fda0080 1 Processor -- start 2024-09-10T21:34:15.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.450+0000 7fe55fda0080 1 -- start start 2024-09-10T21:34:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f15581c80 0x560f1559e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f1559b700 0x560f154c6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f155800a0 0x560f155a3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f15473f40 con 0x560f155800a0 2024-09-10T21:34:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560f15480d00 con 0x560f1559b700 2024-09-10T21:34:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560f15415a00 con 0x560f15581c80 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f155800a0 0x560f155a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f155800a0 0x560f155a3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54392/0 (socket says 172.21.15.97:54392) 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 -- 172.21.15.97:0/2234965474 learned_addr learned my addr 172.21.15.97:0/2234965474 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe557e00640 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f1559b700 0x560f154c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55d000640 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f15581c80 0x560f1559e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 -- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f15581c80 msgr2=0x560f1559e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f15581c80 0x560f1559e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 -- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f1559b700 msgr2=0x560f154c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f1559b700 0x560f154c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 -- 172.21.15.97:0/2234965474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f155a2580 con 0x560f155800a0 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55da00640 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f155800a0 0x560f155a3580 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x560f154b72f0 tx=0x7fe558001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62988fe6921b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe556a00640 1 -- 172.21.15.97:0/2234965474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558019070 con 0x560f155800a0 2024-09-10T21:34:15.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe556a00640 1 -- 172.21.15.97:0/2234965474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe558007590 con 0x560f155800a0 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe556a00640 1 -- 172.21.15.97:0/2234965474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548001c90 con 0x560f155800a0 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- 172.21.15.97:0/2234965474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f155800a0 msgr2=0x560f155a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f155800a0 0x560f155a3580 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x560f154b72f0 tx=0x7fe558001b50 comp rx=0 tx=0).stop 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- 172.21.15.97:0/2234965474 shutdown_connections 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f155800a0 0x560f155a3580 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f1559b700 0x560f154c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 --2- 172.21.15.97:0/2234965474 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f15581c80 0x560f1559e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- 172.21.15.97:0/2234965474 >> 172.21.15.97:0/2234965474 conn(0x560f154c6140 msgr2=0x560f15580be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- 172.21.15.97:0/2234965474 shutdown_connections 2024-09-10T21:34:15.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- 172.21.15.97:0/2234965474 wait complete. 2024-09-10T21:34:15.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 Processor -- start 2024-09-10T21:34:15.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.454+0000 7fe55fda0080 1 -- start start 2024-09-10T21:34:15.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f155800a0 0x560f1557ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f15581c80 0x560f15638d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f1559b700 0x560f156392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f155a2a90 con 0x560f1559b700 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560f1559e770 con 0x560f15581c80 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x560f155a27f0 con 0x560f155800a0 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f1559b700 0x560f156392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55d000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f155800a0 0x560f1557ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f1559b700 0x560f156392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54406/0 (socket says 172.21.15.97:54406) 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 -- 172.21.15.97:0/3361530275 learned_addr learned my addr 172.21.15.97:0/3361530275 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe557e00640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f15581c80 0x560f15638d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 -- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f155800a0 msgr2=0x560f1557ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x560f155800a0 0x560f1557ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 -- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f15581c80 msgr2=0x560f15638d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f15581c80 0x560f15638d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 -- 172.21.15.97:0/3361530275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f15639bc0 con 0x560f1559b700 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe557e00640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560f15581c80 0x560f15638d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55da00640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f1559b700 0x560f156392c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x560f15584a50 tx=0x7fe558002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe555600640 1 -- 172.21.15.97:0/3361530275 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558019040 con 0x560f1559b700 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe555600640 1 -- 172.21.15.97:0/3361530275 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe558023070 con 0x560f1559b700 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe555600640 1 -- 172.21.15.97:0/3361530275 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558014900 con 0x560f1559b700 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f155a52f0 con 0x560f1559b700 2024-09-10T21:34:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f155a5830 con 0x560f1559b700 2024-09-10T21:34:15.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.458+0000 7fe555600640 1 -- 172.21.15.97:0/3361530275 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe558014aa0 con 0x560f1559b700 2024-09-10T21:34:15.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe555600640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe53c070fc0 0x7fe53c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe555600640 1 -- 172.21.15.97:0/3361530275 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9027+0+0 (secure 0 0 0) 0x7fe5580943f0 con 0x560f1559b700 2024-09-10T21:34:15.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55d000640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe53c070fc0 0x7fe53c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560f154e9df0 0x560f154ea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x560f154ea710 con 0x560f154e9df0 2024-09-10T21:34:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe557e00640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560f154e9df0 0x560f154ea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55d000640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe53c070fc0 0x7fe53c073480 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe5500ac000 tx=0x7fe5500b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe557e00640 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560f154e9df0 0x560f154ea1d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe557e00640 1 -- 172.21.15.97:0/3361530275 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v128'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe54800b040 con 0x560f154e9df0 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560f154e9df0 msgr2=0x560f154ea1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x560f154e9df0 0x560f154ea1d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe53c070fc0 msgr2=0x7fe53c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe53c070fc0 0x7fe53c073480 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe5500ac000 tx=0x7fe5500b3040 comp rx=0 tx=0).stop 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f1559b700 msgr2=0x560f156392c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 --2- 172.21.15.97:0/3361530275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f1559b700 0x560f156392c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x560f15584a50 tx=0x7fe558002100 comp rx=0 tx=0).stop 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55da00640 1 -- 172.21.15.97:0/3361530275 reap_dead start 2024-09-10T21:34:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 shutdown_connections 2024-09-10T21:34:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 >> 172.21.15.97:0/3361530275 conn(0x560f154c6140 msgr2=0x560f154c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 shutdown_connections 2024-09-10T21:34:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.462+0000 7fe55fda0080 1 -- 172.21.15.97:0/3361530275 wait complete. 2024-09-10T21:34:15.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:34:15.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- 172.21.15.97:0/2642529223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0b2570 msgr2=0x7f19ec166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2642529223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0b2570 0x7f19ec166fe0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009900 tx=0x7f19e401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- 172.21.15.97:0/2642529223 shutdown_connections 2024-09-10T21:34:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2642529223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0b2570 0x7f19ec166fe0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- 172.21.15.97:0/2642529223 >> 172.21.15.97:0/2642529223 conn(0x7f19ec0b37c0 msgr2=0x7f19ec0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- 172.21.15.97:0/2642529223 shutdown_connections 2024-09-10T21:34:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- 172.21.15.97:0/2642529223 wait complete. 2024-09-10T21:34:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 Processor -- start 2024-09-10T21:34:15.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- start start 2024-09-10T21:34:15.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19ec0b2570 0x7f19ec0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19ec0ce830 0x7f19ec0b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0cec10 0x7f19ec0c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec0cddd0 con 0x7f19ec0cec10 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec0cdc50 con 0x7f19ec0b2570 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19f1c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec0ce1a0 con 0x7f19ec0ce830 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0cec10 0x7f19ec0c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0cec10 0x7f19ec0c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54410/0 (socket says 172.21.15.97:54410) 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 -- 172.21.15.97:0/2272459573 learned_addr learned my addr 172.21.15.97:0/2272459573 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eaa00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19ec0b2570 0x7f19ec0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19ea000640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19ec0ce830 0x7f19ec0b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 -- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19ec0ce830 msgr2=0x7f19ec0b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19ec0ce830 0x7f19ec0b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 -- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19ec0b2570 msgr2=0x7f19ec0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19ec0b2570 0x7f19ec0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:15.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.614+0000 7f19eb400640 1 -- 172.21.15.97:0/2272459573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec1f2540 con 0x7f19ec0cec10 2024-09-10T21:34:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19eb400640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0cec10 0x7f19ec0c74c0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f19e40b3040 tx=0x7f19e40a8dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19e2a00640 1 -- 172.21.15.97:0/2272459573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e40bf070 con 0x7f19ec0cec10 2024-09-10T21:34:15.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19e2a00640 1 -- 172.21.15.97:0/2272459573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19e4005340 con 0x7f19ec0cec10 2024-09-10T21:34:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19e2a00640 1 -- 172.21.15.97:0/2272459573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e40b5040 con 0x7f19ec0cec10 2024-09-10T21:34:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec0b7190 con 0x7f19ec0cec10 2024-09-10T21:34:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec0b7750 con 0x7f19ec0cec10 2024-09-10T21:34:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec0cf800 con 0x7f19ec0cec10 2024-09-10T21:34:15.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19e2a00640 1 -- 172.21.15.97:0/2272459573 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f19e40b8070 con 0x7f19ec0cec10 2024-09-10T21:34:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.618+0000 7f19e2a00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19cc0725e0 0x7f19cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:15.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.622+0000 7f19e2a00640 1 -- 172.21.15.97:0/2272459573 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9027+0+0 (secure 0 0 0) 0x7f19e4130d80 con 0x7f19ec0cec10 2024-09-10T21:34:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.622+0000 7f19eaa00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19cc0725e0 0x7f19cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:15.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.622+0000 7f19eaa00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19cc0725e0 0x7f19cc074aa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0012f0 tx=0x7f19dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:15.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.622+0000 7f19e2a00640 1 -- 172.21.15.97:0/2272459573 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f19e4018c30 con 0x7f19ec0cec10 2024-09-10T21:34:15.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:15.826+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f19ec1586b0 con 0x7f19ec0cec10 2024-09-10T21:34:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.498+0000 7f19e2a00640 1 -- 172.21.15.97:0/2272459573 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v278) ==== 162+0+0 (secure 0 0 0) 0x7f19e40fb060 con 0x7f19ec0cec10 2024-09-10T21:34:16.499 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:34:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19cc0725e0 msgr2=0x7f19cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:16.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19cc0725e0 0x7f19cc074aa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0012f0 tx=0x7f19dc009040 comp rx=0 tx=0).stop 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0cec10 msgr2=0x7f19ec0c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0cec10 0x7f19ec0c74c0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f19e40b3040 tx=0x7f19e40a8dd0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 shutdown_connections 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19cc0725e0 0x7f19cc074aa0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19ec0cec10 0x7f19ec0c74c0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19ec0ce830 0x7f19ec0b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 --2- 172.21.15.97:0/2272459573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19ec0b2570 0x7f19ec0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 >> 172.21.15.97:0/2272459573 conn(0x7f19ec0b37c0 msgr2=0x7f19ec0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 shutdown_connections 2024-09-10T21:34:16.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.502+0000 7f19f1c00640 1 -- 172.21.15.97:0/2272459573 wait complete. 2024-09-10T21:34:16.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- 172.21.15.97:0/3784617837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 msgr2=0x7f6aec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3784617837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6aec00b9d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009ee0 tx=0x7f6afc01d630 comp rx=0 tx=0).stop 2024-09-10T21:34:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- 172.21.15.97:0/3784617837 shutdown_connections 2024-09-10T21:34:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3784617837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6aec00b9d0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- 172.21.15.97:0/3784617837 >> 172.21.15.97:0/3784617837 conn(0x7f6b040b1780 msgr2=0x7f6b040b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- 172.21.15.97:0/3784617837 shutdown_connections 2024-09-10T21:34:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- 172.21.15.97:0/3784617837 wait complete. 2024-09-10T21:34:16.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 Processor -- start 2024-09-10T21:34:16.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- start start 2024-09-10T21:34:16.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6b04157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b04144aa0 0x7f6b04158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b0415c310 0x7f6b0415c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b040cee10 con 0x7f6aec0051d0 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6b040cec90 con 0x7f6b04144aa0 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b0c200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0415f6d0 con 0x7f6b0415c310 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b09800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6b04157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b09800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6b04157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54432/0 (socket says 172.21.15.97:54432) 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.670+0000 7f6b09800640 1 -- 172.21.15.97:0/3624524603 learned_addr learned my addr 172.21.15.97:0/3624524603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b08e00640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b04144aa0 0x7f6b04158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b0a200640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b0415c310 0x7f6b0415c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b09800640 1 -- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b0415c310 msgr2=0x7f6b0415c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b09800640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b0415c310 0x7f6b0415c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b09800640 1 -- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b04144aa0 msgr2=0x7f6b04158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b09800640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b04144aa0 0x7f6b04158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b09800640 1 -- 172.21.15.97:0/3624524603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0415bae0 con 0x7f6aec0051d0 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b09800640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6b04157450 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f6af8006660 tx=0x7f6af80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b01600640 1 -- 172.21.15.97:0/3624524603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8015070 con 0x7f6aec0051d0 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b01600640 1 -- 172.21.15.97:0/3624524603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6af8011840 con 0x7f6aec0051d0 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b01600640 1 -- 172.21.15.97:0/3624524603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8011c90 con 0x7f6aec0051d0 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0406ad40 con 0x7f6aec0051d0 2024-09-10T21:34:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b041595d0 con 0x7f6aec0051d0 2024-09-10T21:34:16.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0415c100 con 0x7f6aec0051d0 2024-09-10T21:34:16.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.674+0000 7f6b01600640 1 -- 172.21.15.97:0/3624524603 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6af800f3c0 con 0x7f6aec0051d0 2024-09-10T21:34:16.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.678+0000 7f6b01600640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ae4072610 0x7f6ae4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:16.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.678+0000 7f6b01600640 1 -- 172.21.15.97:0/3624524603 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9027+0+0 (secure 0 0 0) 0x7f6af80949b0 con 0x7f6aec0051d0 2024-09-10T21:34:16.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.678+0000 7f6b08e00640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ae4072610 0x7f6ae4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:16.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.678+0000 7f6b08e00640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ae4072610 0x7f6ae4074ad0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c10 tx=0x7f6af0006b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:16.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.682+0000 7f6b01600640 1 -- 172.21.15.97:0/3624524603 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6af805ed80 con 0x7f6aec0051d0 2024-09-10T21:34:16.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.878+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b04140c70 con 0x7f6aec0051d0 2024-09-10T21:34:16.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.878+0000 7f6b01600640 1 -- 172.21.15.97:0/3624524603 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5230 (secure 0 0 0) 0x7f6af80637b0 con 0x7f6aec0051d0 2024-09-10T21:34:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:16.879 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":279,"btime":"2024-09-10T21:34:16:505091+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:16.505074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9783":{"gid":9783,"name":"a","rank":0,"incarnation":279,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/1087733461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1087733461},{"type":"v1","addr":"172.21.15.97:6834","nonce":1087733461}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:16.879 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 279 2024-09-10T21:34:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ae4072610 msgr2=0x7f6ae4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ae4072610 0x7f6ae4074ad0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c10 tx=0x7f6af0006b80 comp rx=0 tx=0).stop 2024-09-10T21:34:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 msgr2=0x7f6b04157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6b04157450 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f6af8006660 tx=0x7f6af80039d0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 shutdown_connections 2024-09-10T21:34:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6ae4072610 0x7f6ae4074ad0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6b0415c310 0x7f6b0415c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b04144aa0 0x7f6b04158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 --2- 172.21.15.97:0/3624524603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0051d0 0x7f6b04157450 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 >> 172.21.15.97:0/3624524603 conn(0x7f6b040b1780 msgr2=0x7f6b04073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 shutdown_connections 2024-09-10T21:34:16.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:16.882+0000 7f6b0c200640 1 -- 172.21.15.97:0/3624524603 wait complete. 2024-09-10T21:34:16.901 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:34:19.529 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:19.530+0000 7f6afae00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-10T21:34:19.530 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:19.530+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:34:19.530 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:19.530+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:34:20.466 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:20.466+0000 7f6affe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:21.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 -- 172.21.15.97:0/3457474953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 msgr2=0x7f50900b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 --2- 172.21.15.97:0/3457474953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 0x7f50900b2eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f507c009870 tx=0x7f507c01c7a0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 -- 172.21.15.97:0/3457474953 shutdown_connections 2024-09-10T21:34:22.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 --2- 172.21.15.97:0/3457474953 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 0x7f50900b2eb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 -- 172.21.15.97:0/3457474953 >> 172.21.15.97:0/3457474953 conn(0x7f50900d1a60 msgr2=0x7f50900d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 -- 172.21.15.97:0/3457474953 shutdown_connections 2024-09-10T21:34:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 -- 172.21.15.97:0/3457474953 wait complete. 2024-09-10T21:34:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.054+0000 7f5094a00640 1 Processor -- start 2024-09-10T21:34:22.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 -- start start 2024-09-10T21:34:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50900ae710 0x7f509015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 0x7f509014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5090161290 0x7f5090159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f509016c4f0 con 0x7f50900ae710 2024-09-10T21:34:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f509016c370 con 0x7f5090161290 2024-09-10T21:34:22.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f509016c670 con 0x7f50900b29b0 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50900ae710 0x7f509015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50900ae710 0x7f509015a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34380/0 (socket says 172.21.15.97:34380) 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 -- 172.21.15.97:0/1188814982 learned_addr learned my addr 172.21.15.97:0/1188814982 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508de00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 0x7f509014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508f200640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5090161290 0x7f5090159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 -- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 msgr2=0x7f509014ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 0x7f509014ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 -- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5090161290 msgr2=0x7f5090159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5090161290 0x7f5090159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 -- 172.21.15.97:0/1188814982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509015aeb0 con 0x7f50900ae710 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508de00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 0x7f509014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f508e800640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50900ae710 0x7f509015a970 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f508000c430 tx=0x7f508000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:22.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5086a00640 1 -- 172.21.15.97:0/1188814982 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080017070 con 0x7f50900ae710 2024-09-10T21:34:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5086a00640 1 -- 172.21.15.97:0/1188814982 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50800053c0 con 0x7f50900ae710 2024-09-10T21:34:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509015a230 con 0x7f50900ae710 2024-09-10T21:34:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5086a00640 1 -- 172.21.15.97:0/1188814982 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080003860 con 0x7f50900ae710 2024-09-10T21:34:22.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50900b6090 con 0x7f50900ae710 2024-09-10T21:34:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.058+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50900b2eb0 con 0x7f50900ae710 2024-09-10T21:34:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.062+0000 7f5086a00640 1 -- 172.21.15.97:0/1188814982 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5080003a00 con 0x7f50900ae710 2024-09-10T21:34:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.062+0000 7f5086a00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f50700725e0 0x7f5070074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.062+0000 7f5086a00640 1 -- 172.21.15.97:0/1188814982 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f5080093720 con 0x7f50900ae710 2024-09-10T21:34:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.062+0000 7f508de00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f50700725e0 0x7f5070074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.062+0000 7f508de00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f50700725e0 0x7f5070074aa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f507c0af000 tx=0x7f507c01afe0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:22.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.066+0000 7f5086a00640 1 -- 172.21.15.97:0/1188814982 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f508005e000 con 0x7f50900ae710 2024-09-10T21:34:22.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.262+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50900afd30 con 0x7f50900ae710 2024-09-10T21:34:22.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.262+0000 7f5086a00640 1 -- 172.21.15.97:0/1188814982 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 283 v283) ==== 95+0+4395 (secure 0 0 0) 0x7f508005d900 con 0x7f50900ae710 2024-09-10T21:34:22.262 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:22.262 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":283,"btime":"2024-09-10T21:34:19:534097+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":283,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:19.534094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:22.263 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 283 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f50700725e0 msgr2=0x7f5070074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f50700725e0 0x7f5070074aa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f507c0af000 tx=0x7f507c01afe0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50900ae710 msgr2=0x7f509015a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50900ae710 0x7f509015a970 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f508000c430 tx=0x7f508000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 shutdown_connections 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f50700725e0 0x7f5070074aa0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5090161290 0x7f5090159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f50900b29b0 0x7f509014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 --2- 172.21.15.97:0/1188814982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50900ae710 0x7f509015a970 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 >> 172.21.15.97:0/1188814982 conn(0x7f50900d1a60 msgr2=0x7f50900af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:22.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 shutdown_connections 2024-09-10T21:34:22.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.266+0000 7f5094a00640 1 -- 172.21.15.97:0/1188814982 wait complete. 2024-09-10T21:34:22.284 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:34:22.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:22.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.426+0000 7ff2fea00640 1 -- 172.21.15.97:0/1473259689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f8070720 msgr2=0x7ff2f8072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.426+0000 7ff2fea00640 1 --2- 172.21.15.97:0/1473259689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f8070720 0x7ff2f8072630 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000a950 tx=0x7ff2e001db40 comp rx=0 tx=0).stop 2024-09-10T21:34:22.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.426+0000 7ff2fea00640 1 -- 172.21.15.97:0/1473259689 shutdown_connections 2024-09-10T21:34:22.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.426+0000 7ff2fea00640 1 --2- 172.21.15.97:0/1473259689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f8070720 0x7ff2f8072630 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.426+0000 7ff2fea00640 1 -- 172.21.15.97:0/1473259689 >> 172.21.15.97:0/1473259689 conn(0x7ff2f80b1780 msgr2=0x7ff2f80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:22.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.426+0000 7ff2fea00640 1 -- 172.21.15.97:0/1473259689 shutdown_connections 2024-09-10T21:34:22.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.426+0000 7ff2fea00640 1 -- 172.21.15.97:0/1473259689 wait complete. 2024-09-10T21:34:22.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 Processor -- start 2024-09-10T21:34:22.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 -- start start 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f8070720 0x7ff2f8135030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f8072b70 0x7ff2f813b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f815b110 0x7ff2f814b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8167b60 con 0x7ff2f815b110 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f80b3a40 con 0x7ff2f8072b70 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f8167ce0 con 0x7ff2f8070720 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f815b110 0x7ff2f814b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f815b110 0x7ff2f814b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34400/0 (socket says 172.21.15.97:34400) 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 -- 172.21.15.97:0/2493078754 learned_addr learned my addr 172.21.15.97:0/2493078754 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2ef400640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f8072b70 0x7ff2f813b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2f7e00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f8070720 0x7ff2f8135030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 -- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f8070720 msgr2=0x7ff2f8135030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f8070720 0x7ff2f8135030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 -- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f8072b70 msgr2=0x7ff2f813b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f8072b70 0x7ff2f813b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 -- 172.21.15.97:0/2493078754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f81fac70 con 0x7ff2f815b110 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fca00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f815b110 0x7ff2f814b500 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c3c0 tx=0x7ff2f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2f4c00640 1 -- 172.21.15.97:0/2493078754 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0011790 con 0x7ff2f815b110 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2f4c00640 1 -- 172.21.15.97:0/2493078754 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2f0017070 con 0x7ff2f815b110 2024-09-10T21:34:22.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f81faf50 con 0x7ff2f815b110 2024-09-10T21:34:22.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2f4c00640 1 -- 172.21.15.97:0/2493078754 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0011cc0 con 0x7ff2f815b110 2024-09-10T21:34:22.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.430+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f81fb510 con 0x7ff2f815b110 2024-09-10T21:34:22.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.434+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f8072630 con 0x7ff2f815b110 2024-09-10T21:34:22.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.434+0000 7ff2f4c00640 1 -- 172.21.15.97:0/2493078754 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff2f001c460 con 0x7ff2f815b110 2024-09-10T21:34:22.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.434+0000 7ff2f4c00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d40725e0 0x7ff2d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.434+0000 7ff2f4c00640 1 -- 172.21.15.97:0/2493078754 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7ff2f0093c70 con 0x7ff2f815b110 2024-09-10T21:34:22.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.434+0000 7ff2f7e00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d40725e0 0x7ff2d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.434+0000 7ff2f7e00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d40725e0 0x7ff2d4074aa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80012f0 tx=0x7ff2e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:22.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.438+0000 7ff2f4c00640 1 -- 172.21.15.97:0/2493078754 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff2f005df60 con 0x7ff2f815b110 2024-09-10T21:34:22.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.638+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2f8070340 con 0x7ff2f815b110 2024-09-10T21:34:22.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.638+0000 7ff2f4c00640 1 -- 172.21.15.97:0/2493078754 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 283 v283) ==== 95+0+4395 (secure 0 0 0) 0x7ff2f0062990 con 0x7ff2f815b110 2024-09-10T21:34:22.639 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:22.639 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":283,"btime":"2024-09-10T21:34:19:534097+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":283,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:19.534094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:22.640 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 283 2024-09-10T21:34:22.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d40725e0 msgr2=0x7ff2d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d40725e0 0x7ff2d4074aa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80012f0 tx=0x7ff2e8009040 comp rx=0 tx=0).stop 2024-09-10T21:34:22.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f815b110 msgr2=0x7ff2f814b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f815b110 0x7ff2f814b500 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c3c0 tx=0x7ff2f000c880 comp rx=0 tx=0).stop 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 shutdown_connections 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff2d40725e0 0x7ff2d4074aa0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2f815b110 0x7ff2f814b500 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2f8072b70 0x7ff2f813b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 --2- 172.21.15.97:0/2493078754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2f8070720 0x7ff2f8135030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 >> 172.21.15.97:0/2493078754 conn(0x7ff2f80b1780 msgr2=0x7ff2f8152000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 shutdown_connections 2024-09-10T21:34:22.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.642+0000 7ff2fea00640 1 -- 172.21.15.97:0/2493078754 wait complete. 2024-09-10T21:34:22.661 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:34:22.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:22.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- 172.21.15.97:0/2347058296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe340b3cf0 msgr2=0x7efe3406ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 --2- 172.21.15.97:0/2347058296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe340b3cf0 0x7efe3406ec40 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009900 tx=0x7efe1c01f090 comp rx=0 tx=0).stop 2024-09-10T21:34:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- 172.21.15.97:0/2347058296 shutdown_connections 2024-09-10T21:34:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 --2- 172.21.15.97:0/2347058296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe340b3cf0 0x7efe3406ec40 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 --2- 172.21.15.97:0/2347058296 >> v2:172.21.15.194:3300/0 conn(0x7efe341511f0 0x7efe340b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- 172.21.15.97:0/2347058296 >> 172.21.15.97:0/2347058296 conn(0x7efe340b1780 msgr2=0x7efe340b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- 172.21.15.97:0/2347058296 shutdown_connections 2024-09-10T21:34:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- 172.21.15.97:0/2347058296 wait complete. 2024-09-10T21:34:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 Processor -- start 2024-09-10T21:34:22.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- start start 2024-09-10T21:34:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe3406f180 0x7efe3414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe340b3cf0 0x7efe3415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe341511f0 0x7efe3415c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe3406e740 con 0x7efe341511f0 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efe3406e5c0 con 0x7efe3406f180 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe39200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efe3406e8c0 con 0x7efe340b3cf0 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe32800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe341511f0 0x7efe3415c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe32800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe341511f0 0x7efe3415c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34414/0 (socket says 172.21.15.97:34414) 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe32800640 1 -- 172.21.15.97:0/962920981 learned_addr learned my addr 172.21.15.97:0/962920981 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe29400640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe340b3cf0 0x7efe3415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.806+0000 7efe31e00640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe3406f180 0x7efe3414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe32800640 1 -- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe340b3cf0 msgr2=0x7efe3415c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe32800640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe340b3cf0 0x7efe3415c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe32800640 1 -- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe3406f180 msgr2=0x7efe3414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe32800640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe3406f180 0x7efe3414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe32800640 1 -- 172.21.15.97:0/962920981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe34150070 con 0x7efe341511f0 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe32800640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe341511f0 0x7efe3415c750 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c4d0 tx=0x7efe2c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe2aa00640 1 -- 172.21.15.97:0/962920981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0052d0 con 0x7efe341511f0 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe2aa00640 1 -- 172.21.15.97:0/962920981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe2c005470 con 0x7efe341511f0 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe2aa00640 1 -- 172.21.15.97:0/962920981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c012870 con 0x7efe341511f0 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe341f80c0 con 0x7efe341511f0 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe341f8680 con 0x7efe341511f0 2024-09-10T21:34:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe340b37b0 con 0x7efe341511f0 2024-09-10T21:34:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe2aa00640 1 -- 172.21.15.97:0/962920981 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe2c003600 con 0x7efe341511f0 2024-09-10T21:34:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe2aa00640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe100725e0 0x7efe10074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.810+0000 7efe2aa00640 1 -- 172.21.15.97:0/962920981 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7efe2c093960 con 0x7efe341511f0 2024-09-10T21:34:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.814+0000 7efe31e00640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe100725e0 0x7efe10074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.814+0000 7efe31e00640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe100725e0 0x7efe10074aa0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efe2400aed0 tx=0x7efe2400a640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:22.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:22.814+0000 7efe2aa00640 1 -- 172.21.15.97:0/962920981 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe2c05dc80 con 0x7efe341511f0 2024-09-10T21:34:23.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.014+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe34076120 con 0x7efe341511f0 2024-09-10T21:34:23.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.014+0000 7efe2aa00640 1 -- 172.21.15.97:0/962920981 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 283 v283) ==== 95+0+4395 (secure 0 0 0) 0x7efe2c0626b0 con 0x7efe341511f0 2024-09-10T21:34:23.014 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:23.014 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":283,"btime":"2024-09-10T21:34:19:534097+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":283,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:19.534094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 283 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.014+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe100725e0 msgr2=0x7efe10074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.014+0000 7efe39200640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe100725e0 0x7efe10074aa0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efe2400aed0 tx=0x7efe2400a640 comp rx=0 tx=0).stop 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.014+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe341511f0 msgr2=0x7efe3415c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.014+0000 7efe39200640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe341511f0 0x7efe3415c750 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c4d0 tx=0x7efe2c00c990 comp rx=0 tx=0).stop 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 shutdown_connections 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efe100725e0 0x7efe10074aa0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe341511f0 0x7efe3415c750 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efe340b3cf0 0x7efe3415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 --2- 172.21.15.97:0/962920981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe3406f180 0x7efe3414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 >> 172.21.15.97:0/962920981 conn(0x7efe340b1780 msgr2=0x7efe341507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 shutdown_connections 2024-09-10T21:34:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.018+0000 7efe39200640 1 -- 172.21.15.97:0/962920981 wait complete. 2024-09-10T21:34:23.035 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2024-09-10T21:34:23.035 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2024-09-10T21:34:23.035 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:23.035 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:23.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:34:23.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:34:23.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:23.044 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:34:23.099 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:34:23.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:34:23.100 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:34:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 -- 172.21.15.97:0/669075244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d05e0 msgr2=0x7f91300b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 --2- 172.21.15.97:0/669075244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d05e0 0x7f91300b3070 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f9128009bc0 tx=0x7f912801cb10 comp rx=0 tx=0).stop 2024-09-10T21:34:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 -- 172.21.15.97:0/669075244 shutdown_connections 2024-09-10T21:34:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 --2- 172.21.15.97:0/669075244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d05e0 0x7f91300b3070 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 -- 172.21.15.97:0/669075244 >> 172.21.15.97:0/669075244 conn(0x7f91300d1c60 msgr2=0x7f91300d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 -- 172.21.15.97:0/669075244 shutdown_connections 2024-09-10T21:34:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 -- 172.21.15.97:0/669075244 wait complete. 2024-09-10T21:34:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 Processor -- start 2024-09-10T21:34:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 -- start start 2024-09-10T21:34:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d0150 0x7f91300cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f91300d05e0 0x7f9130070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.290+0000 7f9136000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f913015f350 0x7f91300b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9136000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9130169dc0 con 0x7f91300d0150 2024-09-10T21:34:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9136000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9130169c40 con 0x7f913015f350 2024-09-10T21:34:23.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9136000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9130169f40 con 0x7f91300d05e0 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d0150 0x7f91300cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d0150 0x7f91300cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34432/0 (socket says 172.21.15.97:34432) 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 -- 172.21.15.97:0/1582407604 learned_addr learned my addr 172.21.15.97:0/1582407604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9126a00640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f91300d05e0 0x7f9130070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912fe00640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f913015f350 0x7f91300b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 -- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f91300d05e0 msgr2=0x7f9130070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f91300d05e0 0x7f9130070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 -- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f913015f350 msgr2=0x7f91300b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f913015f350 0x7f91300b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 -- 172.21.15.97:0/1582407604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913015eae0 con 0x7f91300d0150 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912fe00640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f913015f350 0x7f91300b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f912f400640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d0150 0x7f91300cb430 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f912000c3c0 tx=0x7f912000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9127e00640 1 -- 172.21.15.97:0/1582407604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120017070 con 0x7f91300d0150 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9127e00640 1 -- 172.21.15.97:0/1582407604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91200053c0 con 0x7f91300d0150 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9127e00640 1 -- 172.21.15.97:0/1582407604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120003860 con 0x7f91300d0150 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91301f8bf0 con 0x7f91300d0150 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91301f8110 con 0x7f91300d0150 2024-09-10T21:34:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91300b3070 con 0x7f91300d0150 2024-09-10T21:34:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.294+0000 7f9127e00640 1 -- 172.21.15.97:0/1582407604 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9120003a00 con 0x7f91300d0150 2024-09-10T21:34:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.298+0000 7f9127e00640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f910c072610 0x7f910c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.298+0000 7f9127e00640 1 -- 172.21.15.97:0/1582407604 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f91200935d0 con 0x7f91300d0150 2024-09-10T21:34:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.298+0000 7f9126a00640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f910c072610 0x7f910c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.298+0000 7f9126a00640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f910c072610 0x7f910c074ad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f91301f5880 tx=0x7f9118009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:23.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.302+0000 7f9127e00640 1 -- 172.21.15.97:0/1582407604 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f912005dfa0 con 0x7f91300d0150 2024-09-10T21:34:23.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.498+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91301f8810 con 0x7f91300d0150 2024-09-10T21:34:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.546+0000 7f9127e00640 1 -- 172.21.15.97:0/1582407604 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v284) ==== 161+0+0 (secure 0 0 0) 0x7f912005d8a0 con 0x7f91300d0150 2024-09-10T21:34:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:34:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f910c072610 msgr2=0x7f910c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f910c072610 0x7f910c074ad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f91301f5880 tx=0x7f9118009040 comp rx=0 tx=0).stop 2024-09-10T21:34:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d0150 msgr2=0x7f91300cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d0150 0x7f91300cb430 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f912000c3c0 tx=0x7f912000c880 comp rx=0 tx=0).stop 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 shutdown_connections 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f910c072610 0x7f910c074ad0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f913015f350 0x7f91300b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f91300d05e0 0x7f9130070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 --2- 172.21.15.97:0/1582407604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91300d0150 0x7f91300cb430 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 >> 172.21.15.97:0/1582407604 conn(0x7f91300d1c60 msgr2=0x7f9130071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 shutdown_connections 2024-09-10T21:34:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.550+0000 7f9136000640 1 -- 172.21.15.97:0/1582407604 wait complete. 2024-09-10T21:34:23.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:34:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- 172.21.15.97:0/3976718207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 msgr2=0x7fa70c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 --2- 172.21.15.97:0/3976718207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 0x7fa70c004810 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fa71000bbc0 tx=0x7fa71001ce60 comp rx=0 tx=0).stop 2024-09-10T21:34:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72d200640 1 -- 172.21.15.97:0/3976718207 reap_dead start 2024-09-10T21:34:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- 172.21.15.97:0/3976718207 shutdown_connections 2024-09-10T21:34:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- 172.21.15.97:0/3976718207 >> 172.21.15.97:0/3976718207 conn(0x7fa7280b1780 msgr2=0x7fa7280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- 172.21.15.97:0/3976718207 shutdown_connections 2024-09-10T21:34:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- 172.21.15.97:0/3976718207 wait complete. 2024-09-10T21:34:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 Processor -- start 2024-09-10T21:34:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- start start 2024-09-10T21:34:23.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 0x7fa72813d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa72815ea00 0x7fa72815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa7281588e0 0x7fa728158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa728141d20 con 0x7fa70c004030 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa728140bd0 con 0x7fa72815ea00 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.714+0000 7fa72f200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa728141ea0 con 0x7fa7281588e0 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 0x7fa72813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 0x7fa72813d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34458/0 (socket says 172.21.15.97:34458) 2024-09-10T21:34:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 -- 172.21.15.97:0/1599896817 learned_addr learned my addr 172.21.15.97:0/1599896817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:23.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa72d200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa7281588e0 0x7fa728158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 -- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa7281588e0 msgr2=0x7fa728158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa7281588e0 0x7fa728158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 -- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa72815ea00 msgr2=0x7fa72815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:23.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa71f400640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa72815ea00 0x7fa72815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa72815ea00 0x7fa72815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 -- 172.21.15.97:0/1599896817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa728142770 con 0x7fa70c004030 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa72d200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa7281588e0 0x7fa728158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa71f400640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa72815ea00 0x7fa72815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa727e00640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 0x7fa72813d500 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fa718011ab0 tx=0x7fa718011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa724c00640 1 -- 172.21.15.97:0/1599896817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71801e070 con 0x7fa70c004030 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa724c00640 1 -- 172.21.15.97:0/1599896817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa718004460 con 0x7fa70c004030 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa728142a50 con 0x7fa70c004030 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa724c00640 1 -- 172.21.15.97:0/1599896817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71800fc60 con 0x7fa70c004030 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa728143010 con 0x7fa70c004030 2024-09-10T21:34:23.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7281433f0 con 0x7fa70c004030 2024-09-10T21:34:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.718+0000 7fa724c00640 1 -- 172.21.15.97:0/1599896817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa718003600 con 0x7fa70c004030 2024-09-10T21:34:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.722+0000 7fa724c00640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa7040725e0 0x7fa704074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:23.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.722+0000 7fa724c00640 1 -- 172.21.15.97:0/1599896817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7fa718094b60 con 0x7fa70c004030 2024-09-10T21:34:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.722+0000 7fa71f400640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa7040725e0 0x7fa704074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:23.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.722+0000 7fa71f400640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa7040725e0 0x7fa704074aa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa7100af000 tx=0x7fa710019c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:23.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.722+0000 7fa724c00640 1 -- 172.21.15.97:0/1599896817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa7281433f0 con 0x7fa70c004030 2024-09-10T21:34:23.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:23.950+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa7281592a0 con 0x7fa70c004030 2024-09-10T21:34:24.283 INFO:tasks.ceph.mds.a.smithi097.stderr: -53> 2024-09-10T21:34:19.530+0000 7f6afae00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-10T21:34:24.283 INFO:tasks.ceph.mds.a.smithi097.stderr: -33> 2024-09-10T21:34:20.466+0000 7f6affe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:24.322 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:34:24.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.558+0000 7fa724c00640 1 -- 172.21.15.97:0/1599896817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v285) ==== 101+0+0 (secure 0 0 0) 0x7fa71805eee0 con 0x7fa70c004030 2024-09-10T21:34:24.558 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:34:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa7040725e0 msgr2=0x7fa704074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa7040725e0 0x7fa704074aa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa7100af000 tx=0x7fa710019c00 comp rx=0 tx=0).stop 2024-09-10T21:34:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 msgr2=0x7fa72813d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 0x7fa72813d500 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fa718011ab0 tx=0x7fa718011f70 comp rx=0 tx=0).stop 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 shutdown_connections 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa7040725e0 0x7fa704074aa0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa7281588e0 0x7fa728158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa72815ea00 0x7fa72815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 --2- 172.21.15.97:0/1599896817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa70c004030 0x7fa72813d500 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 >> 172.21.15.97:0/1599896817 conn(0x7fa7280b1780 msgr2=0x7fa728073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 shutdown_connections 2024-09-10T21:34:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.562+0000 7fa72f200640 1 -- 172.21.15.97:0/1599896817 wait complete. 2024-09-10T21:34:24.582 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:34:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 Processor -- start 2024-09-10T21:34:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- start start 2024-09-10T21:34:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db4cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db540a00 0x5639db4d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db5441d0 0x5639db4b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5639db3d4a00 con 0x5639db53ecc0 2024-09-10T21:34:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5639db432f40 con 0x5639db5441d0 2024-09-10T21:34:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5639db542710 con 0x5639db540a00 2024-09-10T21:34:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db4cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db4cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34488/0 (socket says 172.21.15.97:34488) 2024-09-10T21:34:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 -- 172.21.15.97:0/2257182437 learned_addr learned my addr 172.21.15.97:0/2257182437 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:24.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9de600640 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db540a00 0x5639db4d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9dfa00640 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db5441d0 0x5639db4b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 -- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db540a00 msgr2=0x5639db4d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:24.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db540a00 0x5639db4d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 -- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db5441d0 msgr2=0x5639db4b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:24.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db5441d0 0x5639db4b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 -- 172.21.15.97:0/2257182437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639db3adc10 con 0x5639db53ecc0 2024-09-10T21:34:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9df000640 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db4cd980 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000b220 tx=0x7fe9d000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5353985a5d21b5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9dd200640 1 -- 172.21.15.97:0/2257182437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d000a040 con 0x5639db53ecc0 2024-09-10T21:34:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9dd200640 1 -- 172.21.15.97:0/2257182437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9d000f620 con 0x5639db53ecc0 2024-09-10T21:34:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9dd200640 1 -- 172.21.15.97:0/2257182437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d000fa70 con 0x5639db53ecc0 2024-09-10T21:34:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/2257182437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 msgr2=0x5639db4cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db4cd980 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000b220 tx=0x7fe9d000b6e0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/2257182437 shutdown_connections 2024-09-10T21:34:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db5441d0 0x5639db4b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db540a00 0x5639db4d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/2257182437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db4cd980 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/2257182437 >> 172.21.15.97:0/2257182437 conn(0x5639db4ce0d0 msgr2=0x5639db4b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.678+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/2257182437 shutdown_connections 2024-09-10T21:34:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/2257182437 wait complete. 2024-09-10T21:34:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 Processor -- start 2024-09-10T21:34:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 -- start start 2024-09-10T21:34:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db53cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db540a00 0x5639db55cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db5441d0 0x5639db55d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5639db55a690 con 0x5639db53ecc0 2024-09-10T21:34:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5639db55abd0 con 0x5639db540a00 2024-09-10T21:34:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5639db55a930 con 0x5639db5441d0 2024-09-10T21:34:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db53cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db53cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34504/0 (socket says 172.21.15.97:34504) 2024-09-10T21:34:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 learned_addr learned my addr 172.21.15.97:0/4226431044 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9de600640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db540a00 0x5639db55cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9dfa00640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db5441d0 0x5639db55d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db5441d0 msgr2=0x5639db55d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db5441d0 0x5639db55d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db540a00 msgr2=0x5639db55cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5639db540a00 0x5639db55cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639db557f30 con 0x5639db53ecc0 2024-09-10T21:34:24.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9dfa00640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5639db5441d0 0x5639db55d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9df000640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db53cab0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0001af0 tx=0x7fe9d000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9d7400640 1 -- 172.21.15.97:0/4226431044 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d001f070 con 0x5639db53ecc0 2024-09-10T21:34:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639db5581d0 con 0x5639db53ecc0 2024-09-10T21:34:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9d7400640 1 -- 172.21.15.97:0/4226431044 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9d001b640 con 0x5639db53ecc0 2024-09-10T21:34:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9d7400640 1 -- 172.21.15.97:0/4226431044 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d001ba90 con 0x5639db53ecc0 2024-09-10T21:34:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.682+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639db558470 con 0x5639db53ecc0 2024-09-10T21:34:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9d7400640 1 -- 172.21.15.97:0/4226431044 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe9d001bc30 con 0x5639db53ecc0 2024-09-10T21:34:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9d7400640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9c0070fc0 0x7fe9c0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9d7400640 1 -- 172.21.15.97:0/4226431044 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7fe9d00944a0 con 0x5639db53ecc0 2024-09-10T21:34:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9de600640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9c0070fc0 0x7fe9c0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.699 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:34:24.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5639db486a50 0x5639db486e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db487370 con 0x5639db486a50 2024-09-10T21:34:24.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9dfa00640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5639db486a50 0x5639db486e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9de600640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9c0070fc0 0x7fe9c0073480 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0ab000 tx=0x7fe9cc019b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.686+0000 7fe9dfa00640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5639db486a50 0x5639db486e30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.690+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v129'1002 uv976 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.690+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db487740 con 0x5639db486a50 2024-09-10T21:34:24.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.694+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v129'1003 uv1003 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.694+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db487ae0 con 0x5639db486a50 2024-09-10T21:34:24.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.694+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v129'1004 uv1004 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.703 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:34:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.694+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db603580 con 0x5639db486a50 2024-09-10T21:34:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.694+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v129'1005 uv1005 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.694+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db603920 con 0x5639db486a50 2024-09-10T21:34:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.698+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v129'1006 uv1006 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.698+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602570 con 0x5639db486a50 2024-09-10T21:34:24.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.698+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v129'1007 uv1007 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.698+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602960 con 0x5639db486a50 2024-09-10T21:34:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.702+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v129'1008 uv982 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.702+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db602d50 con 0x5639db486a50 2024-09-10T21:34:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.702+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v129'1009 uv1009 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.702+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d50 con 0x5639db486a50 2024-09-10T21:34:24.708 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:34:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.706+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v129'1010 uv1010 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.706+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.706+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v129'1011 uv1011 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.706+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.706+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v129'1012 uv1012 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.706+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.710+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v129'1013 uv1013 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.710+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.710+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v129'1014 uv989 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.710+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.714+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v129'1015 uv1015 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.714+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.715 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:34:24.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.714+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v129'1016 uv1016 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.714+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.718+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v129'1017 uv1017 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.718+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.718+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v129'1018 uv1018 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.718+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db602d00 con 0x5639db486a50 2024-09-10T21:34:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.722+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v129'1019 uv1019 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.722+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.722+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v129'1020 uv1020 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.722+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.722+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v129'1021 uv995 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.722+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.726+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v129'1022 uv1022 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.726+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.726 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:34:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.726+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v129'1023 uv1023 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.726+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.730+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v129'1024 uv1024 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.730+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.730+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v129'1025 uv1025 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.730+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.730+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v129'1026 uv1026 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.730+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.734+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v129'1027 uv1001 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.734+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.734+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v129'1028 uv1028 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.734+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.736 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:34:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.738+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v129'1029 uv1029 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.738+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.738+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v129'1030 uv1030 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.738+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.742+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v129'1031 uv1031 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.742+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db486a50 2024-09-10T21:34:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.742+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v129'1032 uv1032 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db486a50 2024-09-10T21:34:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.742+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5639db6041d0 0x5639db6065c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.742+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6031b0 con 0x5639db6041d0 2024-09-10T21:34:24.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.742+0000 7fe9df000640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5639db6041d0 0x5639db6065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.742+0000 7fe9df000640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5639db6041d0 0x5639db6065c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.746+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v129'727 uv709 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.746+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db606b00 con 0x5639db6041d0 2024-09-10T21:34:24.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.746+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v129'728 uv728 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.746+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db606f40 con 0x5639db6041d0 2024-09-10T21:34:24.747 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:34:24.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.746+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v129'729 uv729 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.746+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db607800 con 0x5639db6041d0 2024-09-10T21:34:24.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.750+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v129'730 uv730 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.750+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db607ba0 con 0x5639db6041d0 2024-09-10T21:34:24.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.750+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v129'731 uv731 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.750+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db607f40 con 0x5639db6041d0 2024-09-10T21:34:24.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.754+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v129'732 uv714 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.754+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.754+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v129'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.754+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.754+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v129'734 uv734 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.754 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:34:24.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.754+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.758+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v129'735 uv735 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.758+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.758+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v129'736 uv736 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.758+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.758+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v129'737 uv719 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.758+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.762+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v129'738 uv738 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.762+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.761 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:34:24.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.762+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v129'739 uv739 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.762+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.762+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v129'740 uv740 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.762+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.766+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v129'741 uv741 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.766+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.766+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v129'742 uv725 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.766+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.770+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v129'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.770+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.769 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:34:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.770+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v129'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.770+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.770+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v129'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.770+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6092f0 con 0x5639db6041d0 2024-09-10T21:34:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v129'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6082e0 con 0x5639db6041d0 2024-09-10T21:34:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v129'747 uv747 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db6041d0 2024-09-10T21:34:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5639db6089a0 0x5639db608d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db60b6f0 con 0x5639db6089a0 2024-09-10T21:34:24.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9de600640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5639db6089a0 0x5639db608d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9de600640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5639db6089a0 0x5639db608d80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.774+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db60ba90 con 0x5639db6089a0 2024-09-10T21:34:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.778+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.778+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db60be30 con 0x5639db6089a0 2024-09-10T21:34:24.777 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:34:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.778+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.778+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6073d0 con 0x5639db6089a0 2024-09-10T21:34:24.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.778+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.778+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db60d6e0 con 0x5639db6089a0 2024-09-10T21:34:24.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db60c680 con 0x5639db6089a0 2024-09-10T21:34:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5639db60d160 0x5639db60fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610060 con 0x5639db60d160 2024-09-10T21:34:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9dfa00640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5639db60d160 0x5639db60fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9dfa00640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5639db60d160 0x5639db60fb20 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v129'146 uv145 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.782+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db610450 con 0x5639db60d160 2024-09-10T21:34:24.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.786+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v129'147 uv147 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.786+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610450 con 0x5639db60d160 2024-09-10T21:34:24.785 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:34:24.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.786+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v129'148 uv148 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.786+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610450 con 0x5639db60d160 2024-09-10T21:34:24.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.786+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v129'149 uv149 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.786+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610870 con 0x5639db6089a0 2024-09-10T21:34:24.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.789 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:34:24.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db611c20 con 0x5639db6089a0 2024-09-10T21:34:24.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.790+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db611c20 con 0x5639db6089a0 2024-09-10T21:34:24.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.794+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.794+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.794+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.794+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.794+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v129'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.794+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.798+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v129'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.798+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.797 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:34:24.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.798+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v129'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.798+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db611c20 con 0x5639db60d160 2024-09-10T21:34:24.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.798+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v129'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.798+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db611c20 con 0x5639db60d160 2024-09-10T21:34:24.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v129'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v129'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v129'366 uv359 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v129'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.802+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.804 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:34:24.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.806+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v129'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.806+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.806+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v129'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.806+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db60d160 2024-09-10T21:34:24.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.806+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v129'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.806+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v129'163 uv162 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v129'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.810 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:34:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v129'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v129'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.810+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db610c10 con 0x5639db6089a0 2024-09-10T21:34:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.814+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v129'167 uv167 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db6089a0 2024-09-10T21:34:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.814+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5639db611120 0x5639db613510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.814+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db613b90 con 0x5639db611120 2024-09-10T21:34:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.814+0000 7fe9df000640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5639db611120 0x5639db613510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.814+0000 7fe9df000640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5639db611120 0x5639db613510 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.814+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.814+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db613c50 con 0x5639db611120 2024-09-10T21:34:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.818+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.818+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.817 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:34:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.818+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.818+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615050 con 0x5639db611120 2024-09-10T21:34:24.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.818+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.818+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615050 con 0x5639db611120 2024-09-10T21:34:24.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.822+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.822+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.822+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.822+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.822+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v129'325 uv318 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.822+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v129'326 uv326 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.825 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:34:24.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v129'327 uv327 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v129'328 uv328 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v129'329 uv329 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.826+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.830+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v129'330 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.830+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.830+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v129'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.830+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.831 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:34:24.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.830+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v129'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.830+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db614040 con 0x5639db611120 2024-09-10T21:34:24.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.834+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v129'333 uv333 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.834+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db616e50 con 0x5639db611120 2024-09-10T21:34:24.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.834+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v129'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.834+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.834+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v129'142 uv140 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.834+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.838+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v129'143 uv143 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.838+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.837 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:34:24.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.838+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v129'144 uv144 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.838+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.838+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v129'145 uv145 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.838+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v129'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v129'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.842 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:34:24.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v129'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615500 con 0x5639db60d160 2024-09-10T21:34:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v129'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.842+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615500 con 0x5639db60d160 2024-09-10T21:34:24.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.846+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v129'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.846+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6144f0 con 0x5639db60d160 2024-09-10T21:34:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.846+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v129'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9d8002040 con 0x5639db60d160 2024-09-10T21:34:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.846+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5639db614b70 0x5639db614f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.846+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615490 con 0x5639db614b70 2024-09-10T21:34:24.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.846+0000 7fe9de600640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5639db614b70 0x5639db614f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.846+0000 7fe9de600640 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5639db614b70 0x5639db614f50 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v129'166 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db614b70 2024-09-10T21:34:24.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db615490 con 0x5639db614b70 2024-09-10T21:34:24.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v129'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db614b70 2024-09-10T21:34:24.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615490 con 0x5639db614b70 2024-09-10T21:34:24.850 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:34:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v129'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db614b70 2024-09-10T21:34:24.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615490 con 0x5639db614b70 2024-09-10T21:34:24.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v129'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db614b70 2024-09-10T21:34:24.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.850+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615490 con 0x5639db614b70 2024-09-10T21:34:24.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.854+0000 7fe9de600640 1 -- 172.21.15.97:0/4226431044 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v129'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe9cc0b3040 con 0x5639db614b70 2024-09-10T21:34:24.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.854+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615490 con 0x5639db611120 2024-09-10T21:34:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.854+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v129'402 uv393 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.854+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db615490 con 0x5639db611120 2024-09-10T21:34:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.854+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v129'403 uv403 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.854+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6159a0 con 0x5639db611120 2024-09-10T21:34:24.856 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:34:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.858+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v129'404 uv404 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.858+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db6159a0 con 0x5639db611120 2024-09-10T21:34:24.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.858+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v129'405 uv405 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.858+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615900 con 0x5639db611120 2024-09-10T21:34:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.858+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v129'406 uv406 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.858+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615900 con 0x5639db611120 2024-09-10T21:34:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v129'407 uv400 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x5639db615900 con 0x5639db611120 2024-09-10T21:34:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v129'408 uv408 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615900 con 0x5639db611120 2024-09-10T21:34:24.862 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:34:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v129'409 uv409 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db615900 con 0x5639db611120 2024-09-10T21:34:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v129'410 uv410 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.862+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db616910 con 0x5639db611120 2024-09-10T21:34:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v129'411 uv411 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db616d20 con 0x5639db611120 2024-09-10T21:34:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v129'412 uv412 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5639db616d20 con 0x5639db611120 2024-09-10T21:34:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9df000640 1 -- 172.21.15.97:0/4226431044 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v129'413 uv413 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe9d002b070 con 0x5639db611120 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5639db6089a0 msgr2=0x5639db608d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5639db6089a0 0x5639db608d80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5639db611120 msgr2=0x5639db613510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5639db611120 0x5639db613510 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5639db614b70 msgr2=0x5639db614f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5639db614b70 0x5639db614f50 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5639db60d160 msgr2=0x5639db60fb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5639db60d160 0x5639db60fb20 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5639db6041d0 msgr2=0x5639db6065c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5639db6041d0 0x5639db6065c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 reap_dead start 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9dfa00640 1 -- 172.21.15.97:0/4226431044 reap_dead start 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5639db486a50 msgr2=0x5639db486e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5639db486a50 0x5639db486e30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9c0070fc0 msgr2=0x7fe9c0073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9c0070fc0 0x7fe9c0073480 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0ab000 tx=0x7fe9cc019b20 comp rx=0 tx=0).stop 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 msgr2=0x5639db53cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db53cab0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0001af0 tx=0x7fe9d000b050 comp rx=0 tx=0).stop 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 shutdown_connections 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5639db486a50 0x5639db486e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9c0070fc0 0x7fe9c0073480 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 --2- 172.21.15.97:0/4226431044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639db53ecc0 0x5639db53cab0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 >> 172.21.15.97:0/4226431044 conn(0x5639db4ce0d0 msgr2=0x5639db4ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 shutdown_connections 2024-09-10T21:34:24.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.866+0000 7fe9e1ef0080 1 -- 172.21.15.97:0/4226431044 wait complete. 2024-09-10T21:34:24.872 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:34:24.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 Processor -- start 2024-09-10T21:34:24.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 -- start start 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 0x5626ea7a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea83d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 0x5626ea83daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5626ea61fa00 con 0x5626ea78ab00 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5626ea67df40 con 0x5626ea700940 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.962+0000 7f04fb669080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5626ea78d080 con 0x5626ea78b2a0 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea83d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea83d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34528/0 (socket says 172.21.15.97:34528) 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 -- 172.21.15.97:0/2355341328 learned_addr learned my addr 172.21.15.97:0/2355341328 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:24.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f9200640 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 0x5626ea83daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f8800640 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 0x5626ea7a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 -- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 msgr2=0x5626ea83daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 0x5626ea83daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 -- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 msgr2=0x5626ea7a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 0x5626ea7a31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 -- 172.21.15.97:0/2355341328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ea5f8c10 con 0x5626ea78ab00 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea83d560 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f04e400b230 tx=0x7f04e400b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b7ede34de6ecc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f6a00640 1 -- 172.21.15.97:0/2355341328 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4016070 con 0x5626ea78ab00 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f6a00640 1 -- 172.21.15.97:0/2355341328 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e400d4c0 con 0x5626ea78ab00 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f6a00640 1 -- 172.21.15.97:0/2355341328 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e400d710 con 0x5626ea78ab00 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- 172.21.15.97:0/2355341328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 msgr2=0x5626ea83d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea83d560 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f04e400b230 tx=0x7f04e400b6f0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- 172.21.15.97:0/2355341328 shutdown_connections 2024-09-10T21:34:24.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 0x5626ea83daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea83d560 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 --2- 172.21.15.97:0/2355341328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 0x5626ea7a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- 172.21.15.97:0/2355341328 >> 172.21.15.97:0/2355341328 conn(0x5626ea6ff520 msgr2=0x5626ea6d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- 172.21.15.97:0/2355341328 shutdown_connections 2024-09-10T21:34:24.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- 172.21.15.97:0/2355341328 wait complete. 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 Processor -- start 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- start start 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 0x5626ea7903d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea789990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 0x5626ea7a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5626ea7aa610 con 0x5626ea78ab00 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5626ea7a8cd0 con 0x5626ea700940 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04fb669080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5626ea7a8f30 con 0x5626ea78b2a0 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea789990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea789990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34540/0 (socket says 172.21.15.97:34540) 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 -- 172.21.15.97:0/670846641 learned_addr learned my addr 172.21.15.97:0/670846641 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 0x5626ea7903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f9200640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 0x5626ea7a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 -- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 msgr2=0x5626ea7a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.966+0000 7f04f7e00640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5626ea78b2a0 0x5626ea7a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f7e00640 1 -- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 msgr2=0x5626ea7903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f7e00640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626ea700940 0x5626ea7903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f7e00640 1 -- 172.21.15.97:0/670846641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ea78a110 con 0x5626ea78ab00 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f7e00640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea789990 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f04e400de70 tx=0x7f04e400dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f5600640 1 -- 172.21.15.97:0/670846641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4016040 con 0x5626ea78ab00 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f5600640 1 -- 172.21.15.97:0/670846641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e401f070 con 0x5626ea78ab00 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f5600640 1 -- 172.21.15.97:0/670846641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e40108d0 con 0x5626ea78ab00 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5626ea78a3b0 con 0x5626ea78ab00 2024-09-10T21:34:24.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626ea7a2900 con 0x5626ea78ab00 2024-09-10T21:34:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f5600640 1 -- 172.21.15.97:0/670846641 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04e4010a70 con 0x5626ea78ab00 2024-09-10T21:34:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f5600640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04d8070ff0 0x7f04d80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f5600640 1 -- 172.21.15.97:0/670846641 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f04e40936b0 con 0x5626ea78ab00 2024-09-10T21:34:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04fb669080 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5626ea7a2ad0 0x5626ea787d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04f8800640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04d8070ff0 0x7f04d80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.970+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x5626ea83aa60 con 0x5626ea7a2ad0 2024-09-10T21:34:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04f9200640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5626ea7a2ad0 0x5626ea787d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04f8800640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04d8070ff0 0x7f04d80734b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00c970 tx=0x7f04ec00f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04f9200640 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5626ea7a2ad0 0x5626ea787d20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04f9200640 1 -- 172.21.15.97:0/670846641 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v129'414 uv414 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04f0002050 con 0x5626ea7a2ad0 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5626ea7a2ad0 msgr2=0x5626ea787d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5626ea7a2ad0 0x5626ea787d20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04d8070ff0 msgr2=0x7f04d80734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f04d8070ff0 0x7f04d80734b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00c970 tx=0x7f04ec00f040 comp rx=0 tx=0).stop 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 msgr2=0x5626ea789990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 --2- 172.21.15.97:0/670846641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5626ea78ab00 0x5626ea789990 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f04e400de70 tx=0x7f04e400dea0 comp rx=0 tx=0).stop 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04f9200640 1 -- 172.21.15.97:0/670846641 reap_dead start 2024-09-10T21:34:24.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 shutdown_connections 2024-09-10T21:34:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 >> 172.21.15.97:0/670846641 conn(0x5626ea6ff520 msgr2=0x5626ea791080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 shutdown_connections 2024-09-10T21:34:24.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:24.974+0000 7f04fb669080 1 -- 172.21.15.97:0/670846641 wait complete. 2024-09-10T21:34:24.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:34:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- 172.21.15.97:0/1307937436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 msgr2=0x7f3f2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/1307937436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009b60 tx=0x7f3f3001d270 comp rx=0 tx=0).stop 2024-09-10T21:34:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- 172.21.15.97:0/1307937436 shutdown_connections 2024-09-10T21:34:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/1307937436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f2c004c90 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- 172.21.15.97:0/1307937436 >> 172.21.15.97:0/1307937436 conn(0x7f3f480d1cd0 msgr2=0x7f3f480d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- 172.21.15.97:0/1307937436 shutdown_connections 2024-09-10T21:34:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- 172.21.15.97:0/1307937436 wait complete. 2024-09-10T21:34:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 Processor -- start 2024-09-10T21:34:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- start start 2024-09-10T21:34:25.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f481470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:25.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f48147760 0x7f3f4815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f4815a930 0x7f3f481497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480cdd40 con 0x7f3f2c004810 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480cc8c0 con 0x7f3f4815a930 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ee00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480cdec0 con 0x7f3f48147760 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f481470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f481470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34554/0 (socket says 172.21.15.97:34554) 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f47e00640 1 -- 172.21.15.97:0/871117763 learned_addr learned my addr 172.21.15.97:0/871117763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f3f400640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f48147760 0x7f3f4815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.126+0000 7f3f4ce00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f4815a930 0x7f3f481497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f47e00640 1 -- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f48147760 msgr2=0x7f3f4815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f47e00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f48147760 0x7f3f4815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:25.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f47e00640 1 -- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f4815a930 msgr2=0x7f3f481497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f47e00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f4815a930 0x7f3f481497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f47e00640 1 -- 172.21.15.97:0/871117763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f480b4820 con 0x7f3f2c004810 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f47e00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f481470b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f3f38005f80 tx=0x7f3f38011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f44c00640 1 -- 172.21.15.97:0/871117763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38005320 con 0x7f3f2c004810 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f44c00640 1 -- 172.21.15.97:0/871117763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f38018020 con 0x7f3f2c004810 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f480b7660 con 0x7f3f2c004810 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f44c00640 1 -- 172.21.15.97:0/871117763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38003860 con 0x7f3f2c004810 2024-09-10T21:34:25.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f48159ed0 con 0x7f3f2c004810 2024-09-10T21:34:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f480aec80 con 0x7f3f2c004810 2024-09-10T21:34:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f44c00640 1 -- 172.21.15.97:0/871117763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3f38010040 con 0x7f3f2c004810 2024-09-10T21:34:25.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f44c00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f240725e0 0x7f3f24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:25.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.130+0000 7f3f44c00640 1 -- 172.21.15.97:0/871117763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f3f38097480 con 0x7f3f2c004810 2024-09-10T21:34:25.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.134+0000 7f3f3f400640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f240725e0 0x7f3f24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:25.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.134+0000 7f3f3f400640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f240725e0 0x7f3f24074aa0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ae000 tx=0x7f3f3001a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:25.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.134+0000 7f3f44c00640 1 -- 172.21.15.97:0/871117763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3f38022020 con 0x7f3f2c004810 2024-09-10T21:34:25.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.338+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f3f480cfd60 con 0x7f3f2c004810 2024-09-10T21:34:25.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.566+0000 7f3f44c00640 1 -- 172.21.15.97:0/871117763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v286) ==== 162+0+0 (secure 0 0 0) 0x7f3f38061710 con 0x7f3f2c004810 2024-09-10T21:34:25.566 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:34:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f240725e0 msgr2=0x7f3f24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f240725e0 0x7f3f24074aa0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ae000 tx=0x7f3f3001a010 comp rx=0 tx=0).stop 2024-09-10T21:34:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 msgr2=0x7f3f481470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f481470b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f3f38005f80 tx=0x7f3f38011800 comp rx=0 tx=0).stop 2024-09-10T21:34:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 shutdown_connections 2024-09-10T21:34:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3f240725e0 0x7f3f24074aa0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:25.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f4815a930 0x7f3f481497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:25.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3f48147760 0x7f3f4815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:25.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 --2- 172.21.15.97:0/871117763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f2c004810 0x7f3f481470b0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:25.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 >> 172.21.15.97:0/871117763 conn(0x7f3f480d1cd0 msgr2=0x7f3f48071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:25.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 shutdown_connections 2024-09-10T21:34:25.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:25.570+0000 7f3f4ee00640 1 -- 172.21.15.97:0/871117763 wait complete. 2024-09-10T21:34:25.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:26.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 -- 172.21.15.97:0/1027980257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 msgr2=0x7f23bc0ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:26.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 --2- 172.21.15.97:0/1027980257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc0ccc10 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009bf0 tx=0x7f23b401cba0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 -- 172.21.15.97:0/1027980257 shutdown_connections 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 --2- 172.21.15.97:0/1027980257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc0ccc10 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 --2- 172.21.15.97:0/1027980257 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23bc0afdd0 0x7f23bc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 -- 172.21.15.97:0/1027980257 >> 172.21.15.97:0/1027980257 conn(0x7f23bc0d1c30 msgr2=0x7f23bc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 -- 172.21.15.97:0/1027980257 shutdown_connections 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.138+0000 7f23c4400640 1 -- 172.21.15.97:0/1027980257 wait complete. 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 Processor -- start 2024-09-10T21:34:26.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 -- start start 2024-09-10T21:34:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23bc0afdd0 0x7f23bc0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23bc0ce640 0x7f23bc158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc168f40 con 0x7f23bc0b0d50 2024-09-10T21:34:26.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc168dc0 con 0x7f23bc0afdd0 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f23bc1690c0 con 0x7f23bc0ce640 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34566/0 (socket says 172.21.15.97:34566) 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 -- 172.21.15.97:0/3842723177 learned_addr learned my addr 172.21.15.97:0/3842723177 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1a00640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23bc0afdd0 0x7f23bc0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c2400640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23bc0ce640 0x7f23bc158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 -- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23bc0ce640 msgr2=0x7f23bc158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23bc0ce640 0x7f23bc158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 -- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23bc0afdd0 msgr2=0x7f23bc0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23bc0afdd0 0x7f23bc0b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 -- 172.21.15.97:0/3842723177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc0b58c0 con 0x7f23bc0b0d50 2024-09-10T21:34:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c1000640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc159380 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d980 tx=0x7f23a800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23b9600640 1 -- 172.21.15.97:0/3842723177 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8017070 con 0x7f23bc0b0d50 2024-09-10T21:34:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23b9600640 1 -- 172.21.15.97:0/3842723177 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23a8009d20 con 0x7f23bc0b0d50 2024-09-10T21:34:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc0c5940 con 0x7f23bc0b0d50 2024-09-10T21:34:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23b9600640 1 -- 172.21.15.97:0/3842723177 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004280 con 0x7f23bc0b0d50 2024-09-10T21:34:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc159960 con 0x7f23bc0b0d50 2024-09-10T21:34:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.142+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23bc0c5760 con 0x7f23bc0b0d50 2024-09-10T21:34:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.146+0000 7f23b9600640 1 -- 172.21.15.97:0/3842723177 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f23a8003600 con 0x7f23bc0b0d50 2024-09-10T21:34:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.146+0000 7f23b9600640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f239c0725e0 0x7f239c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.146+0000 7f23b9600640 1 -- 172.21.15.97:0/3842723177 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f23a80940b0 con 0x7f23bc0b0d50 2024-09-10T21:34:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.146+0000 7f23c1a00640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f239c0725e0 0x7f239c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.146+0000 7f23c1a00640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f239c0725e0 0x7f239c074aa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f23b00046c0 tx=0x7f23b0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.150+0000 7f23b9600640 1 -- 172.21.15.97:0/3842723177 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f23bc0c5760 con 0x7f23bc0b0d50 2024-09-10T21:34:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23bc0cea20 con 0x7f23bc0b0d50 2024-09-10T21:34:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23b9600640 1 -- 172.21.15.97:0/3842723177 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 287 v287) ==== 95+0+5230 (secure 0 0 0) 0x7f23a805e3a0 con 0x7f23bc0b0d50 2024-09-10T21:34:26.349 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:26.349 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":287,"btime":"2024-09-10T21:34:25:571826+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":287,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:25.571777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":287,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/1998443497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1998443497},{"type":"v1","addr":"172.21.15.97:6834","nonce":1998443497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:26.349 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 287 2024-09-10T21:34:26.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f239c0725e0 msgr2=0x7f239c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f239c0725e0 0x7f239c074aa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f23b00046c0 tx=0x7f23b0007ec0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 msgr2=0x7f23bc159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc159380 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d980 tx=0x7f23a800de40 comp rx=0 tx=0).stop 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 shutdown_connections 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f239c0725e0 0x7f239c074aa0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f23bc0ce640 0x7f23bc158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0b0d50 0x7f23bc159380 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 --2- 172.21.15.97:0/3842723177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23bc0afdd0 0x7f23bc0b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 >> 172.21.15.97:0/3842723177 conn(0x7f23bc0d1c30 msgr2=0x7f23bc06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 shutdown_connections 2024-09-10T21:34:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:26.350+0000 7f23c4400640 1 -- 172.21.15.97:0/3842723177 wait complete. 2024-09-10T21:34:26.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 287, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:34:25.571777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9942': {'gid': 9942, 'name': 'a', 'rank': 0, 'incarnation': 287, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6834/1998443497', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 1998443497}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 1998443497}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:34:26.369 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9942:up:replay 2024-09-10T21:34:26.370 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:34:29.566 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:29.566+0000 7eff59800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:31.603 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:31.606+0000 7f8d1dc00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T21:34:31.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:31.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 -- 172.21.15.97:0/2445461770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840b1560 msgr2=0x7f97840b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 --2- 172.21.15.97:0/2445461770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840b1560 0x7f97840b1970 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f9784069d60 tx=0x7f977c01f560 comp rx=0 tx=0).stop 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 -- 172.21.15.97:0/2445461770 shutdown_connections 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 --2- 172.21.15.97:0/2445461770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840b1560 0x7f97840b1970 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 --2- 172.21.15.97:0/2445461770 >> v2:172.21.15.194:3300/0 conn(0x7f97840cec10 0x7f97840cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 -- 172.21.15.97:0/2445461770 >> 172.21.15.97:0/2445461770 conn(0x7f97840b37c0 msgr2=0x7f97840b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 -- 172.21.15.97:0/2445461770 shutdown_connections 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.926+0000 7f978ac00640 1 -- 172.21.15.97:0/2445461770 wait complete. 2024-09-10T21:34:31.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 Processor -- start 2024-09-10T21:34:31.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 -- start start 2024-09-10T21:34:31.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97840b1560 0x7f97840c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:31.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840ce830 0x7f978415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:31.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97840cec10 0x7f97841f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:31.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97840ce1a0 con 0x7f97840ce830 2024-09-10T21:34:31.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97840cde60 con 0x7f97840cec10 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f97840ce320 con 0x7f97840b1560 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840ce830 0x7f978415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840ce830 0x7f978415eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52914/0 (socket says 172.21.15.97:52914) 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 -- 172.21.15.97:0/269027308 learned_addr learned my addr 172.21.15.97:0/269027308 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f9783e00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97840b1560 0x7f97840c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f9788c00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97840cec10 0x7f97841f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 -- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97840b1560 msgr2=0x7f97840c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97840b1560 0x7f97840c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 -- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97840cec10 msgr2=0x7f97841f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97840cec10 0x7f97841f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 -- 172.21.15.97:0/269027308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97841f59f0 con 0x7f97840ce830 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f9788c00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97840cec10 0x7f97841f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:31.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f977b400640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840ce830 0x7f978415eb10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f976c00b410 tx=0x7f976c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f9780c00640 1 -- 172.21.15.97:0/269027308 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c015020 con 0x7f97840ce830 2024-09-10T21:34:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f9780c00640 1 -- 172.21.15.97:0/269027308 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f976c011920 con 0x7f97840ce830 2024-09-10T21:34:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97841f5430 con 0x7f97840ce830 2024-09-10T21:34:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f9780c00640 1 -- 172.21.15.97:0/269027308 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c011d70 con 0x7f97840ce830 2024-09-10T21:34:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.930+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97840c8e30 con 0x7f97840ce830 2024-09-10T21:34:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.934+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97840cf800 con 0x7f97840ce830 2024-09-10T21:34:31.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.934+0000 7f9780c00640 1 -- 172.21.15.97:0/269027308 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f976c01a400 con 0x7f97840ce830 2024-09-10T21:34:31.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.934+0000 7f9780c00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9760072610 0x7f9760074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:31.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.934+0000 7f9780c00640 1 -- 172.21.15.97:0/269027308 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f976c094110 con 0x7f97840ce830 2024-09-10T21:34:31.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.934+0000 7f9783e00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9760072610 0x7f9760074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:31.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.938+0000 7f9783e00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9760072610 0x7f9760074ad0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9774007fd0 tx=0x7f9774009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:31.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:31.938+0000 7f9780c00640 1 -- 172.21.15.97:0/269027308 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f976c01f020 con 0x7f97840ce830 2024-09-10T21:34:32.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.138+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9784072220 con 0x7f97840ce830 2024-09-10T21:34:32.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.138+0000 7f9780c00640 1 -- 172.21.15.97:0/269027308 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5237 (secure 0 0 0) 0x7f976c05ea80 con 0x7f97840ce830 2024-09-10T21:34:32.140 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:32.140 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":291,"btime":"2024-09-10T21:34:31:607213+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:31.584395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":287,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6834/1998443497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1998443497},{"type":"v1","addr":"172.21.15.97:6834","nonce":1998443497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9942,"qdb_cluster":[9942]},"id":6}]} 2024-09-10T21:34:32.140 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 291 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9760072610 msgr2=0x7f9760074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9760072610 0x7f9760074ad0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9774007fd0 tx=0x7f9774009040 comp rx=0 tx=0).stop 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840ce830 msgr2=0x7f978415eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840ce830 0x7f978415eb10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f976c00b410 tx=0x7f976c00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 shutdown_connections 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9760072610 0x7f9760074ad0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97840cec10 0x7f97841f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97840ce830 0x7f978415eb10 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 --2- 172.21.15.97:0/269027308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f97840b1560 0x7f97840c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 >> 172.21.15.97:0/269027308 conn(0x7f97840b37c0 msgr2=0x7f978415f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 shutdown_connections 2024-09-10T21:34:32.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.142+0000 7f978ac00640 1 -- 172.21.15.97:0/269027308 wait complete. 2024-09-10T21:34:32.162 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 291, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:34:31.584395+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 129, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9942': {'gid': 9942, 'name': 'a', 'rank': 0, 'incarnation': 287, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.97:6834/1998443497', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 1998443497}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 1998443497}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9942, 'qdb_cluster': [9942]} 2024-09-10T21:34:32.162 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:34:32.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:34:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 -- 172.21.15.97:0/3917316102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300b1560 msgr2=0x7f7c300b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 --2- 172.21.15.97:0/3917316102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300b1560 0x7f7c300b1970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009c80 tx=0x7f7c2801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 -- 172.21.15.97:0/3917316102 shutdown_connections 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 --2- 172.21.15.97:0/3917316102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300b1560 0x7f7c300b1970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 --2- 172.21.15.97:0/3917316102 >> v2:172.21.15.194:3300/0 conn(0x7f7c300ce830 0x7f7c300cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 -- 172.21.15.97:0/3917316102 >> 172.21.15.97:0/3917316102 conn(0x7f7c300b37c0 msgr2=0x7f7c300b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 -- 172.21.15.97:0/3917316102 shutdown_connections 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 -- 172.21.15.97:0/3917316102 wait complete. 2024-09-10T21:34:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 Processor -- start 2024-09-10T21:34:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.306+0000 7f7c37400640 1 -- start start 2024-09-10T21:34:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c300b1560 0x7f7c300cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300ce830 0x7f7c300c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c300cec10 0x7f7c300c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c300ce320 con 0x7f7c300cec10 2024-09-10T21:34:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7c300ce1a0 con 0x7f7c300b1560 2024-09-10T21:34:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7c300d0450 con 0x7f7c300ce830 2024-09-10T21:34:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c300cec10 0x7f7c300c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c300cec10 0x7f7c300c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52920/0 (socket says 172.21.15.97:52920) 2024-09-10T21:34:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c34a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c300b1560 0x7f7c300cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c27e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300ce830 0x7f7c300c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c35400640 1 -- 172.21.15.97:0/2415244105 learned_addr learned my addr 172.21.15.97:0/2415244105 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c34a00640 1 -- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300ce830 msgr2=0x7f7c300c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c34a00640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300ce830 0x7f7c300c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c34a00640 1 -- 172.21.15.97:0/2415244105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c300cec10 msgr2=0x7f7c300c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c34a00640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c300cec10 0x7f7c300c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c34a00640 1 -- 172.21.15.97:0/2415244105 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c300cb8a0 con 0x7f7c300b1560 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c27e00640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300ce830 0x7f7c300c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c35400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c300cec10 0x7f7c300c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:32.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c34a00640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c300b1560 0x7f7c300cb1a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e9f0 tx=0x7f7c2000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c2d600640 1 -- 172.21.15.97:0/2415244105 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000ccc0 con 0x7f7c300b1560 2024-09-10T21:34:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c2d600640 1 -- 172.21.15.97:0/2415244105 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c20010430 con 0x7f7c300b1560 2024-09-10T21:34:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c300cbb80 con 0x7f7c300b1560 2024-09-10T21:34:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c2d600640 1 -- 172.21.15.97:0/2415244105 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20010860 con 0x7f7c300b1560 2024-09-10T21:34:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.310+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c300ca7a0 con 0x7f7c300b1560 2024-09-10T21:34:32.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c2d600640 1 -- 172.21.15.97:0/2415244105 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7c20010a00 con 0x7f7c300b1560 2024-09-10T21:34:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 Processor -- start 2024-09-10T21:34:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c2d600640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c0c0725e0 0x7f7c0c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 -- start start 2024-09-10T21:34:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c2d600640 1 -- 172.21.15.97:0/2415244105 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f7c20094a60 con 0x7f7c300b1560 2024-09-10T21:34:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c0c0725e0 0x7f7c0c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f99db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836f4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836f4e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559836e6e3f0 con 0x559836f4de10 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559836dabb20 con 0x559836f9a2f0 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559836d9af50 con 0x559836f999d0 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836f4e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c34a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f99db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c27e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836f4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c34a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f99db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:40058/0 (socket says 172.21.15.97:40058) 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c34a00640 1 -- 172.21.15.97:0/4091808914 learned_addr learned my addr 172.21.15.97:0/4091808914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c0c0725e0 0x7f7c0c074aa0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7c2801d1d0 tx=0x7f7c2801dde0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 -- 172.21.15.97:0/4091808914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 msgr2=0x559836f99db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 --2- 172.21.15.97:0/4091808914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f99db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 -- 172.21.15.97:0/4091808914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 msgr2=0x559836f4d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 --2- 172.21.15.97:0/4091808914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836f4d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.314+0000 7f7c35400640 1 -- 172.21.15.97:0/4091808914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559836d62590 con 0x559836f4de10 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 --2- 172.21.15.97:0/4091808914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836f4e1f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f7c2801fe40 tx=0x7f7c280be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb475c85a710a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c25600640 1 -- 172.21.15.97:0/4091808914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280c4680 con 0x559836f4de10 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c25600640 1 -- 172.21.15.97:0/4091808914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c280b4040 con 0x559836f4de10 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c25600640 1 -- 172.21.15.97:0/4091808914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280c4b70 con 0x559836f4de10 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- 172.21.15.97:0/4091808914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 msgr2=0x559836f4e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 --2- 172.21.15.97:0/4091808914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836f4e1f0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f7c2801fe40 tx=0x7f7c280be000 comp rx=0 tx=0).stop 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- 172.21.15.97:0/4091808914 shutdown_connections 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 --2- 172.21.15.97:0/4091808914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836f4e1f0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 --2- 172.21.15.97:0/4091808914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836f4d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 --2- 172.21.15.97:0/4091808914 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f99db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- 172.21.15.97:0/4091808914 >> 172.21.15.97:0/4091808914 conn(0x559836e37130 msgr2=0x559836f7ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- 172.21.15.97:0/4091808914 shutdown_connections 2024-09-10T21:34:32.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- 172.21.15.97:0/4091808914 wait complete. 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- start start 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 Processor -- start 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f4de10 0x559836f4f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f4f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x559836f4fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559836d355d0 con 0x559836f9a2f0 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559836d9ed60 con 0x559836f4de10 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559836d41500 con 0x559836f999d0 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c3a757000 10 client.0 _ll_register_callbacks cb 0x559836d45810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x559836f4fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x559836f4fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52938/0 (socket says 172.21.15.97:52938) 2024-09-10T21:34:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 -- 172.21.15.97:0/3342908424 learned_addr learned my addr 172.21.15.97:0/3342908424 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c34a00640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f4de10 0x559836f4f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c27e00640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f4f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 -- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 msgr2=0x559836f4f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f4f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 -- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f4de10 msgr2=0x559836f4f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f4de10 0x559836f4f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 -- 172.21.15.97:0/3342908424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559836d39170 con 0x559836f9a2f0 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c27e00640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f4f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c35400640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x559836f4fcc0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7c2801fe40 tx=0x7f7c2801c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1dda29db45e4de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.318+0000 7f7c1e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 1 -- 172.21.15.97:0/3342908424 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280c4640 con 0x559836f9a2f0 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559836d330f0 con 0x559836f9a2f0 2024-09-10T21:34:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 1 -- 172.21.15.97:0/3342908424 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f7c280c47e0 con 0x559836f9a2f0 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 1 -- 172.21.15.97:0/3342908424 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f7c280de050 con 0x559836f9a2f0 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 10 client.9993 fetch_fsmap learned FSMap version 291 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x559836e1fef0 con 0x559836f9a2f0 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 1 -- 172.21.15.97:0/3342908424 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559836d330f0 con 0x559836f9a2f0 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 1 -- 172.21.15.97:0/3342908424 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 291) ==== 3502+0+0 (secure 0 0 0) 0x7f7c280c2040 con 0x559836f9a2f0 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 10 client.9993 fetch_fsmap finished waiting for FSMap version 291 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 10 client.9993 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 20 client.9993 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fa8c70 0x559836fab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 4 client.9993 mds_command: new command op to 9942 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 --> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559836dada70 con 0x559836fa8c70 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c34a00640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fa8c70 0x559836fab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 10 client.9993 ms_handle_connect on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c34a00640 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fa8c70 0x559836fab060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef146ef690d7e4f server_cookie=6f47083f83c29b3e in_seq=0 out_seq=0 2024-09-10T21:34:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 1 -- 172.21.15.97:0/3342908424 <== mds.0 v2:172.21.15.97:6832/1998443497 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559836dada70 con 0x559836fa8c70 2024-09-10T21:34:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 10 client.9993 handle_command_reply: tid=0 2024-09-10T21:34:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 client.9993 shutdown 2024-09-10T21:34:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 msgr2=0x559836f4fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x559836f4fcc0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7c2801fe40 tx=0x7f7c2801c6c0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 shutdown_connections 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fa8c70 0x559836fab060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x559836f4fcc0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836f4f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3342908424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f4de10 0x559836f4f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c1e000640 0 client.9993 ms_handle_reset on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 >> 172.21.15.97:0/3342908424 conn(0x559836e37130 msgr2=0x559836f4cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 shutdown_connections 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 1 -- 172.21.15.97:0/3342908424 wait complete. 2024-09-10T21:34:32.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.322+0000 7f7c3a757000 20 client.9993 trim_cache size 0 max 16384 2024-09-10T21:34:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 Processor -- start 2024-09-10T21:34:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 -- start start 2024-09-10T21:34:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f4de10 0x55983700b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f999d0 0x55983700bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x55983700c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559836d66ec0 con 0x559836f9a2f0 2024-09-10T21:34:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559836d355d0 con 0x559836f999d0 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559836d9ed60 con 0x559836f4de10 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x55983700c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x55983700c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52942/0 (socket says 172.21.15.97:52942) 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 -- 172.21.15.97:0/3785132052 learned_addr learned my addr 172.21.15.97:0/3785132052 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c34a00640 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f4de10 0x55983700b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c27e00640 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f999d0 0x55983700bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 -- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f4de10 msgr2=0x55983700b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f4de10 0x55983700b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 -- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f999d0 msgr2=0x55983700bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f999d0 0x55983700bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 -- 172.21.15.97:0/3785132052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559836df6d90 con 0x559836f9a2f0 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c27e00640 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f999d0 0x55983700bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c35400640 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x55983700c050 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f7c28004350 tx=0x7f7c280c4b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34add9bdb0ade442 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c24c00640 1 -- 172.21.15.97:0/3785132052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280c4640 con 0x559836f9a2f0 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c24c00640 1 -- 172.21.15.97:0/3785132052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c280d3020 con 0x559836f9a2f0 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c24c00640 1 -- 172.21.15.97:0/3785132052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280cf430 con 0x559836f9a2f0 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 -- 172.21.15.97:0/3785132052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 msgr2=0x55983700c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x55983700c050 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f7c28004350 tx=0x7f7c280c4b90 comp rx=0 tx=0).stop 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 -- 172.21.15.97:0/3785132052 shutdown_connections 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.342+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f9a2f0 0x55983700c050 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f999d0 0x55983700bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3785132052 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f4de10 0x55983700b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- 172.21.15.97:0/3785132052 >> 172.21.15.97:0/3785132052 conn(0x559836e37130 msgr2=0x559836f4d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- 172.21.15.97:0/3785132052 shutdown_connections 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- 172.21.15.97:0/3785132052 wait complete. 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- start start 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 Processor -- start 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836fafc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836fcb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836fcbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559836e1fa60 con 0x559836f4de10 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559836d41500 con 0x559836f9a2f0 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x559836da3390 con 0x559836f999d0 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 10 client.0 _ll_register_callbacks cb 0x7f7c180016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836fafc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836fafc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52950/0 (socket says 172.21.15.97:52950) 2024-09-10T21:34:32.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 -- 172.21.15.97:0/3462864860 learned_addr learned my addr 172.21.15.97:0/3462864860 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c35400640 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836fcbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c27e00640 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836fcb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 -- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 msgr2=0x559836fcb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836fcb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 -- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 msgr2=0x559836fcbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836fcbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 -- 172.21.15.97:0/3462864860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559836d61a90 con 0x559836f4de10 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c34a00640 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836fafc50 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009fd0 tx=0x7f7c2001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a38d461691d0e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c1e000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c1e000640 1 -- 172.21.15.97:0/3462864860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c200681a0 con 0x559836f4de10 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c1e000640 1 -- 172.21.15.97:0/3462864860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f7c2008b020 con 0x559836f4de10 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c280d3020 con 0x559836f4de10 2024-09-10T21:34:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c1e000640 1 -- 172.21.15.97:0/3462864860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f7c2005afc0 con 0x559836f4de10 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 10 client.9999 fetch_fsmap learned FSMap version 291 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x559836d40050 con 0x559836f4de10 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c1e000640 1 -- 172.21.15.97:0/3462864860 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c280d3020 con 0x559836f4de10 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c1e000640 1 -- 172.21.15.97:0/3462864860 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 291) ==== 3502+0+0 (secure 0 0 0) 0x7f7c2005dfb0 con 0x559836f4de10 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 10 client.9999 fetch_fsmap finished waiting for FSMap version 291 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 10 client.9999 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 20 client.9999 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fcc000 0x559837017650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 4 client.9999 mds_command: new command op to 9942 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 --> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] -- command(tid 0: {"prefix": "status"}) -- 0x559836da9520 con 0x559836fcc000 2024-09-10T21:34:32.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.346+0000 7f7c27e00640 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fcc000 0x559837017650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c1e000640 10 client.9999 ms_handle_connect on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c27e00640 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fcc000 0x559837017650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167e27738ea7da9c server_cookie=c8054bef8766df64 in_seq=0 out_seq=0 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c1e000640 1 -- 172.21.15.97:0/3462864860 <== mds.0 v2:172.21.15.97:6832/1998443497 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x559836da9520 con 0x559836fcc000 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c1e000640 10 client.9999 handle_command_reply: tid=0 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 client.9999 shutdown 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 msgr2=0x559836fafc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836fafc50 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009fd0 tx=0x7f7c2001f000 comp rx=0 tx=0).stop 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 shutdown_connections 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x559836fcc000 0x559837017650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559836f9a2f0 0x559836fcbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x559836f999d0 0x559836fcb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 --2- 172.21.15.97:0/3462864860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559836f4de10 0x559836fafc50 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c1e000640 0 client.9999 ms_handle_reset on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 >> 172.21.15.97:0/3462864860 conn(0x559836e37130 msgr2=0x559836fab060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 shutdown_connections 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 1 -- 172.21.15.97:0/3462864860 wait complete. 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c3a757000 20 client.9999 trim_cache size 0 max 16384 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 9942, 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:34:32.353 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:34:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:34:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7760530399999999, 2024-09-10T21:34:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 291, 2024-09-10T21:34:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 129, 2024-09-10T21:34:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 129, 2024-09-10T21:34:32.354 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 8.0228364550000002 2024-09-10T21:34:32.354 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c0c0725e0 msgr2=0x7f7c0c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c0c0725e0 0x7f7c0c074aa0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7c2801d1d0 tx=0x7f7c2801dde0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c300b1560 msgr2=0x7f7c300cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c300b1560 0x7f7c300cb1a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e9f0 tx=0x7f7c2000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 shutdown_connections 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7c0c0725e0 0x7f7c0c074aa0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c300cec10 0x7f7c300c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7c300ce830 0x7f7c300c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 --2- 172.21.15.97:0/2415244105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c300b1560 0x7f7c300cb1a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 >> 172.21.15.97:0/2415244105 conn(0x7f7c300b37c0 msgr2=0x7f7c300b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 shutdown_connections 2024-09-10T21:34:32.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.350+0000 7f7c37400640 1 -- 172.21.15.97:0/2415244105 wait complete. 2024-09-10T21:34:32.368 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:34:32.368 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2024-09-10T21:34:32.368 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:34:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.510+0000 7f2a9c800640 1 -- 172.21.15.97:0/1051328059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 msgr2=0x7f2a7c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.510+0000 7f2a9c800640 1 --2- 172.21.15.97:0/1051328059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 0x7f2a7c004c90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f2a940b36f0 tx=0x7f2a8401d1d0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.510+0000 7f2a9a800640 1 -- 172.21.15.97:0/1051328059 reap_dead start 2024-09-10T21:34:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.510+0000 7f2a9c800640 1 -- 172.21.15.97:0/1051328059 shutdown_connections 2024-09-10T21:34:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.510+0000 7f2a9c800640 1 -- 172.21.15.97:0/1051328059 >> 172.21.15.97:0/1051328059 conn(0x7f2a940d1c30 msgr2=0x7f2a940d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.510+0000 7f2a9c800640 1 -- 172.21.15.97:0/1051328059 shutdown_connections 2024-09-10T21:34:32.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.510+0000 7f2a9c800640 1 -- 172.21.15.97:0/1051328059 wait complete. 2024-09-10T21:34:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 Processor -- start 2024-09-10T21:34:32.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 -- start start 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 0x7f2a941f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a9415ec30 0x7f2a940b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a9415dbe0 0x7f2a94159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a940cc850 con 0x7f2a7c004810 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2a940cc6d0 con 0x7f2a9415ec30 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2a940cc9d0 con 0x7f2a9415dbe0 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 0x7f2a941f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 0x7f2a941f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52964/0 (socket says 172.21.15.97:52964) 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 -- 172.21.15.97:0/423742948 learned_addr learned my addr 172.21.15.97:0/423742948 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:32.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99400640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a9415ec30 0x7f2a940b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9a800640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a9415dbe0 0x7f2a94159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 -- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a9415dbe0 msgr2=0x7f2a94159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a9415dbe0 0x7f2a94159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 -- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a9415ec30 msgr2=0x7f2a940b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a9415ec30 0x7f2a940b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 -- 172.21.15.97:0/423742948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a941f47b0 con 0x7f2a7c004810 2024-09-10T21:34:32.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a99e00640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 0x7f2a941f5220 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f2a880066b0 tx=0x7f2a88003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a8e000640 1 -- 172.21.15.97:0/423742948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88015070 con 0x7f2a7c004810 2024-09-10T21:34:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a8e000640 1 -- 172.21.15.97:0/423742948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a8800f980 con 0x7f2a7c004810 2024-09-10T21:34:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a8e000640 1 -- 172.21.15.97:0/423742948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800fdd0 con 0x7f2a7c004810 2024-09-10T21:34:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a940b5360 con 0x7f2a7c004810 2024-09-10T21:34:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.514+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a941f4d00 con 0x7f2a7c004810 2024-09-10T21:34:32.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.518+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a940cd6a0 con 0x7f2a7c004810 2024-09-10T21:34:32.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.518+0000 7f2a8e000640 1 -- 172.21.15.97:0/423742948 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2a88011430 con 0x7f2a7c004810 2024-09-10T21:34:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.518+0000 7f2a8e000640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a78072610 0x7f2a78074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.518+0000 7f2a8e000640 1 -- 172.21.15.97:0/423742948 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f2a88094c70 con 0x7f2a7c004810 2024-09-10T21:34:32.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.518+0000 7f2a99400640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a78072610 0x7f2a78074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:32.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.518+0000 7f2a99400640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a78072610 0x7f2a78074ad0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2a940b36f0 tx=0x7f2a84019f70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:32.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.522+0000 7f2a8e000640 1 -- 172.21.15.97:0/423742948 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a8805f050 con 0x7f2a7c004810 2024-09-10T21:34:32.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.726+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2a940cebc0 con 0x7f2a7c004810 2024-09-10T21:34:32.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.726+0000 7f2a8e000640 1 -- 172.21.15.97:0/423742948 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v291) ==== 53+0+73 (secure 0 0 0) 0x7f2a88063a80 con 0x7f2a7c004810 2024-09-10T21:34:32.727 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a78072610 msgr2=0x7f2a78074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a78072610 0x7f2a78074ad0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2a940b36f0 tx=0x7f2a84019f70 comp rx=0 tx=0).stop 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 msgr2=0x7f2a941f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 0x7f2a941f5220 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f2a880066b0 tx=0x7f2a88003ab0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 shutdown_connections 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2a78072610 0x7f2a78074ad0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2a9415dbe0 0x7f2a94159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a9415ec30 0x7f2a940b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 --2- 172.21.15.97:0/423742948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7c004810 0x7f2a941f5220 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:32.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 >> 172.21.15.97:0/423742948 conn(0x7f2a940d1c30 msgr2=0x7f2a94068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:32.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 shutdown_connections 2024-09-10T21:34:32.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32.730+0000 7f2a9c800640 1 -- 172.21.15.97:0/423742948 wait complete. 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:34:32.748 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:32.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:32.749 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:34:32.758 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:34:32.758 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:34:32.758 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:32.759 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41821sec preferred_lft 41821sec 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:34:32.760 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.761 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:34:32.762 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:32.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:32.763 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:34:32.820 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:34:32.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:32.823 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:34:32.875 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:32.876 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:32.876 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:32.876 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:34:32.876 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:34:32.876 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:34:32.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:34:32.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:34:32.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:34:32.876 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:34:32.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:34:33.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:34:33.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:33.011 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:34:33.067 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:34:33.067 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:34:33.068 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:33.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:33.068 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:34:33.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:34:33.136 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:33.191 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:33.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:34:33.191 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:33.242 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:34:33.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:33.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:33.331 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:33.331 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:33.331 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:33.346 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:33.346 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:33.346 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:33.346 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:34:33.346 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:34:33.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:34:33.363 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:34:33.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:33.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:33.382 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:34:33.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:34:33.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:34:33.480 INFO:teuthology.orchestra.run.smithi097.stdout:client9121 2024-09-10T21:34:33.480 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:33.480 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:33.489 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:34:33.489 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:34:33.489 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> import json 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> import sys 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> sys.exit(e.errno) 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:34:33.490 DEBUG:teuthology.orchestra.run.smithi097:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T21:34:33.491 DEBUG:teuthology.orchestra.run.smithi097:> print(json.dumps( 2024-09-10T21:34:33.491 DEBUG:teuthology.orchestra.run.smithi097:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T21:34:33.491 DEBUG:teuthology.orchestra.run.smithi097:> indent=2)) 2024-09-10T21:34:33.491 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:34:33.534 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:34:33.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T21:34:33.609 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T21:34:33.609 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-10T21:34:33.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f5601c00640 1 -- 172.21.15.97:0/1264130485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55e0003a10 msgr2=0x7f55e0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f5601c00640 1 --2- 172.21.15.97:0/1264130485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55e0003a10 0x7f55e0003e30 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c290 tx=0x7f55ec01cfa0 comp rx=0 tx=0).stop 2024-09-10T21:34:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f55fb400640 1 -- 172.21.15.97:0/1264130485 reap_dead start 2024-09-10T21:34:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f5601c00640 1 -- 172.21.15.97:0/1264130485 shutdown_connections 2024-09-10T21:34:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f5601c00640 1 -- 172.21.15.97:0/1264130485 >> 172.21.15.97:0/1264130485 conn(0x7f55fc0d1cd0 msgr2=0x7f55fc0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f5601c00640 1 -- 172.21.15.97:0/1264130485 shutdown_connections 2024-09-10T21:34:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f5601c00640 1 -- 172.21.15.97:0/1264130485 wait complete. 2024-09-10T21:34:33.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.798+0000 7f5601c00640 1 Processor -- start 2024-09-10T21:34:33.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 -- start start 2024-09-10T21:34:33.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55e0003a10 0x7f55fc1f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55fc149590 0x7f55fc1fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f55fc1499f0 0x7f55fc159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc166870 con 0x7f55fc149590 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1666f0 con 0x7f55e0003a10 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc1669f0 con 0x7f55fc1499f0 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55fc149590 0x7f55fc1fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55fc149590 0x7f55fc1fb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52980/0 (socket says 172.21.15.97:52980) 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 -- 172.21.15.97:0/2417532652 learned_addr learned my addr 172.21.15.97:0/2417532652 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55faa00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55e0003a10 0x7f55fc1f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55fb400640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f55fc1499f0 0x7f55fc159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 -- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f55fc1499f0 msgr2=0x7f55fc159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f55fc1499f0 0x7f55fc159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 -- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55e0003a10 msgr2=0x7f55fc1f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55e0003a10 0x7f55fc1f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 -- 172.21.15.97:0/2417532652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc1f7cd0 con 0x7f55fc149590 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f2000640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55fc149590 0x7f55fc1fb2c0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009fd0 tx=0x7f55e400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f3400640 1 -- 172.21.15.97:0/2417532652 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4003b80 con 0x7f55fc149590 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f3400640 1 -- 172.21.15.97:0/2417532652 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e4003d20 con 0x7f55fc149590 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f55f3400640 1 -- 172.21.15.97:0/2417532652 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4004290 con 0x7f55fc149590 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc1fbd10 con 0x7f55fc149590 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc1f7f90 con 0x7f55fc149590 2024-09-10T21:34:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.802+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55fc0cc550 con 0x7f55fc149590 2024-09-10T21:34:33.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.806+0000 7f55f3400640 1 -- 172.21.15.97:0/2417532652 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f55e4004430 con 0x7f55fc149590 2024-09-10T21:34:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.806+0000 7f55f3400640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f55d80725e0 0x7f55d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.806+0000 7f55f3400640 1 -- 172.21.15.97:0/2417532652 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f55e4095200 con 0x7f55fc149590 2024-09-10T21:34:33.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.806+0000 7f55faa00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f55d80725e0 0x7f55d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:33.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.806+0000 7f55faa00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f55d80725e0 0x7f55d8074aa0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0ab000 tx=0x7f55ec0b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:33.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:33.810+0000 7f55f3400640 1 -- 172.21.15.97:0/2417532652 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55e405fb70 con 0x7f55fc149590 2024-09-10T21:34:34.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.006+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55fc0cd9c0 con 0x7f55fc149590 2024-09-10T21:34:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.006+0000 7f55f3400640 1 -- 172.21.15.97:0/2417532652 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5237 (secure 0 0 0) 0x7f55e405f470 con 0x7f55fc149590 2024-09-10T21:34:34.007 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:34.007 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":291,"btime":"2024-09-10T21:34:31:607213+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:31.584395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9942":{"gid":9942,"name":"a","rank":0,"incarnation":287,"state":"up:active","state_seq":5,"addr":"172.21.15.97:6834/1998443497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1998443497},{"type":"v1","addr":"172.21.15.97:6834","nonce":1998443497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9942,"qdb_cluster":[9942]},"id":6}]} 2024-09-10T21:34:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 291 2024-09-10T21:34:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f55d80725e0 msgr2=0x7f55d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f55d80725e0 0x7f55d8074aa0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0ab000 tx=0x7f55ec0b1040 comp rx=0 tx=0).stop 2024-09-10T21:34:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55fc149590 msgr2=0x7f55fc1fb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55fc149590 0x7f55fc1fb2c0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009fd0 tx=0x7f55e400f790 comp rx=0 tx=0).stop 2024-09-10T21:34:34.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 shutdown_connections 2024-09-10T21:34:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f55d80725e0 0x7f55d8074aa0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f55fc1499f0 0x7f55fc159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55fc149590 0x7f55fc1fb2c0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 --2- 172.21.15.97:0/2417532652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55e0003a10 0x7f55fc1f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 >> 172.21.15.97:0/2417532652 conn(0x7f55fc0d1cd0 msgr2=0x7f55fc0cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 shutdown_connections 2024-09-10T21:34:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.010+0000 7f5601c00640 1 -- 172.21.15.97:0/2417532652 wait complete. 2024-09-10T21:34:34.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 -- 172.21.15.97:0/3638543518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c15a730 msgr2=0x7f6c3c15abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 --2- 172.21.15.97:0/3638543518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c15a730 0x7f6c3c15abc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009cb0 tx=0x7f6c3401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 -- 172.21.15.97:0/3638543518 shutdown_connections 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 --2- 172.21.15.97:0/3638543518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c15a730 0x7f6c3c15abc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 --2- 172.21.15.97:0/3638543518 >> v2:172.21.15.194:3300/0 conn(0x7f6c3c0ceee0 0x7f6c3c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 -- 172.21.15.97:0/3638543518 >> 172.21.15.97:0/3638543518 conn(0x7f6c3c0b1780 msgr2=0x7f6c3c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 -- 172.21.15.97:0/3638543518 shutdown_connections 2024-09-10T21:34:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 -- 172.21.15.97:0/3638543518 wait complete. 2024-09-10T21:34:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 Processor -- start 2024-09-10T21:34:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.174+0000 7f6c40a00640 1 -- start start 2024-09-10T21:34:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c0ceee0 0x7f6c3c1e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c3c1511f0 0x7f6c3c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c3c15a730 0x7f6c3c14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c06e530 con 0x7f6c3c15a730 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c06e3b0 con 0x7f6c3c1511f0 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3c06e6b0 con 0x7f6c3c0ceee0 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c3c15a730 0x7f6c3c14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c3c15a730 0x7f6c3c14e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52990/0 (socket says 172.21.15.97:52990) 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 -- 172.21.15.97:0/1923304750 learned_addr learned my addr 172.21.15.97:0/1923304750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c39e00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c0ceee0 0x7f6c3c1e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c39400640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c3c1511f0 0x7f6c3c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 -- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c0ceee0 msgr2=0x7f6c3c1e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c0ceee0 0x7f6c3c1e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 -- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c3c1511f0 msgr2=0x7f6c3c1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c3c1511f0 0x7f6c3c1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 -- 172.21.15.97:0/1923304750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1f4fd0 con 0x7f6c3c15a730 2024-09-10T21:34:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c3a800640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c3c15a730 0x7f6c3c14e250 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f6c3401d1d0 tx=0x7f6c340092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c32000640 1 -- 172.21.15.97:0/1923304750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34004a50 con 0x7f6c3c15a730 2024-09-10T21:34:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c32000640 1 -- 172.21.15.97:0/1923304750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c34004ce0 con 0x7f6c3c15a730 2024-09-10T21:34:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1f52b0 con 0x7f6c3c15a730 2024-09-10T21:34:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c32000640 1 -- 172.21.15.97:0/1923304750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340afb00 con 0x7f6c3c15a730 2024-09-10T21:34:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.178+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1f4c70 con 0x7f6c3c15a730 2024-09-10T21:34:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c32000640 1 -- 172.21.15.97:0/1923304750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6c340afca0 con 0x7f6c3c15a730 2024-09-10T21:34:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c32000640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c1c0725e0 0x7f6c1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c32000640 1 -- 172.21.15.97:0/1923304750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f6c34132e10 con 0x7f6c3c15a730 2024-09-10T21:34:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 Processor -- start 2024-09-10T21:34:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 -- start start 2024-09-10T21:34:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c39e00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c1c0725e0 0x7f6c1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c39e00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c1c0725e0 0x7f6c1c074aa0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c007fd0 tx=0x7f6c2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b4083c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b40837bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b40837c2c0 0x55b40837c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b408263ce0 con 0x55b40837c2c0 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b4081d7c70 con 0x55b4083c5b60 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b4081c70a0 con 0x55b4083c6480 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b40837c2c0 0x55b40837c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c39e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b4083c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b40837c2c0 0x55b40837c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53000/0 (socket says 172.21.15.97:53000) 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c39400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b40837bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 -- 172.21.15.97:0/1845284749 learned_addr learned my addr 172.21.15.97:0/1845284749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 -- 172.21.15.97:0/1845284749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 msgr2=0x55b40837bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 --2- 172.21.15.97:0/1845284749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b40837bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 -- 172.21.15.97:0/1845284749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 msgr2=0x55b4083c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 --2- 172.21.15.97:0/1845284749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b4083c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 -- 172.21.15.97:0/1845284749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4081a2270 con 0x55b40837c2c0 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c3a800640 1 --2- 172.21.15.97:0/1845284749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b40837c2c0 0x55b40837c6a0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f6c341304e0 tx=0x7f6c340fc810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46002d4f533dd43c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c26a00640 1 -- 172.21.15.97:0/1845284749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34101ad0 con 0x55b40837c2c0 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c26a00640 1 -- 172.21.15.97:0/1845284749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c340b4020 con 0x55b40837c2c0 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.182+0000 7f6c26a00640 1 -- 172.21.15.97:0/1845284749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340fd560 con 0x55b40837c2c0 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- 172.21.15.97:0/1845284749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b40837c2c0 msgr2=0x55b40837c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 --2- 172.21.15.97:0/1845284749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b40837c2c0 0x55b40837c6a0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f6c341304e0 tx=0x7f6c340fc810 comp rx=0 tx=0).stop 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- 172.21.15.97:0/1845284749 shutdown_connections 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 --2- 172.21.15.97:0/1845284749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b40837c2c0 0x55b40837c6a0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 --2- 172.21.15.97:0/1845284749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b40837bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 --2- 172.21.15.97:0/1845284749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b4083c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- 172.21.15.97:0/1845284749 >> 172.21.15.97:0/1845284749 conn(0x55b408378270 msgr2=0x55b40837a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- 172.21.15.97:0/1845284749 shutdown_connections 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- 172.21.15.97:0/1845284749 wait complete. 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- start start 2024-09-10T21:34:34.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 Processor -- start 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b4083ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c5b60 0x55b4083cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b4083cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b408161920 con 0x55b4083c6480 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b4081caeb0 con 0x55b40837c2c0 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b407e8c820 con 0x55b4083c5b60 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 10 client.0 _ll_register_callbacks cb 0x55b4081a57d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b4083cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b4083cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53014/0 (socket says 172.21.15.97:53014) 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 -- 172.21.15.97:0/221402301 learned_addr learned my addr 172.21.15.97:0/221402301 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c39400640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c5b60 0x55b4083cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c39e00640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b4083ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 -- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c5b60 msgr2=0x55b4083cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c5b60 0x55b4083cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 -- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 msgr2=0x55b4083ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b4083ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 -- 172.21.15.97:0/221402301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40817a530 con 0x55b4083c6480 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c39400640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c5b60 0x55b4083cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c39e00640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b4083ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c3a800640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b4083cd550 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f6c340f9860 tx=0x7f6c34005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f42ad92281c1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c1b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c1b400640 1 -- 172.21.15.97:0/221402301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34101ad0 con 0x55b4083c6480 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c1b400640 1 -- 172.21.15.97:0/221402301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f6c340fd500 con 0x55b4083c6480 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b40815f3e0 con 0x55b4083c6480 2024-09-10T21:34:34.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c1b400640 1 -- 172.21.15.97:0/221402301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f6c341064f0 con 0x55b4083c6480 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 10 client.10020 fetch_fsmap learned FSMap version 291 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x55b408165460 con 0x55b4083c6480 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.186+0000 7f6c1b400640 1 -- 172.21.15.97:0/221402301 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b40815f3e0 con 0x55b4083c6480 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c1b400640 1 -- 172.21.15.97:0/221402301 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 291) ==== 3502+0+0 (secure 0 0 0) 0x7f6c34106060 con 0x55b4083c6480 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 10 client.10020 fetch_fsmap finished waiting for FSMap version 291 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 10 client.10020 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 20 client.10020 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083d4ea0 0x55b4083d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 4 client.10020 mds_command: new command op to 9942 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:34:34.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 --> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4081d9bc0 con 0x55b4083d4ea0 2024-09-10T21:34:34.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c39e00640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083d4ea0 0x55b4083d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c39e00640 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083d4ea0 0x55b4083d7290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc981b765a0968fd server_cookie=d799c31f014eb19a in_seq=0 out_seq=0 2024-09-10T21:34:34.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c1b400640 10 client.10020 ms_handle_connect on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c1b400640 1 -- 172.21.15.97:0/221402301 <== mds.0 v2:172.21.15.97:6832/1998443497 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b4081d9bc0 con 0x55b4083d4ea0 2024-09-10T21:34:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c1b400640 10 client.10020 handle_command_reply: tid=0 2024-09-10T21:34:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 client.10020 shutdown 2024-09-10T21:34:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 msgr2=0x55b4083cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b4083cd550 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f6c340f9860 tx=0x7f6c34005f70 comp rx=0 tx=0).stop 2024-09-10T21:34:34.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 shutdown_connections 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083d4ea0 0x55b4083d7290 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b4083cd550 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c1b400640 0 client.10020 ms_handle_reset on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c5b60 0x55b4083cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 --2- 172.21.15.97:0/221402301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b4083ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 >> 172.21.15.97:0/221402301 conn(0x55b408378270 msgr2=0x55b40837ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 shutdown_connections 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 1 -- 172.21.15.97:0/221402301 wait complete. 2024-09-10T21:34:34.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.190+0000 7f6c43d78000 20 client.10020 trim_cache size 0 max 16384 2024-09-10T21:34:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 Processor -- start 2024-09-10T21:34:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- start start 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b40837b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c5b60 0x55b40837bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b4083d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b408286fa0 con 0x55b4083c5b60 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b4081722e0 con 0x55b40837c2c0 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b4081d6a40 con 0x55b4083c6480 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c5b60 0x55b40837bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c5b60 0x55b40837bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53018/0 (socket says 172.21.15.97:53018) 2024-09-10T21:34:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 -- 172.21.15.97:0/437657423 learned_addr learned my addr 172.21.15.97:0/437657423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c3a800640 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b4083d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39e00640 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b40837b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 -- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 msgr2=0x55b4083d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b4083d7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 -- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 msgr2=0x55b40837b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b40837b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 -- 172.21.15.97:0/437657423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40824ccf0 con 0x55b4083c5b60 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c39400640 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c5b60 0x55b40837bbb0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009d40 tx=0x7f6c28010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ff8dd7f575452d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c26000640 1 -- 172.21.15.97:0/437657423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28020070 con 0x55b4083c5b60 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c26000640 1 -- 172.21.15.97:0/437657423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c2801b6f0 con 0x55b4083c5b60 2024-09-10T21:34:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c26000640 1 -- 172.21.15.97:0/437657423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2801bb20 con 0x55b4083c5b60 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- 172.21.15.97:0/437657423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c5b60 msgr2=0x55b40837bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c5b60 0x55b40837bbb0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009d40 tx=0x7f6c28010800 comp rx=0 tx=0).stop 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- 172.21.15.97:0/437657423 shutdown_connections 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b4083c6480 0x55b4083d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c5b60 0x55b40837bbb0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 --2- 172.21.15.97:0/437657423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b40837c2c0 0x55b40837b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- 172.21.15.97:0/437657423 >> 172.21.15.97:0/437657423 conn(0x55b408378270 msgr2=0x55b4083818d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- 172.21.15.97:0/437657423 shutdown_connections 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- 172.21.15.97:0/437657423 wait complete. 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 -- start start 2024-09-10T21:34:34.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.206+0000 7f6c43d78000 1 Processor -- start 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b40837c2c0 0x55b40843b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b40843b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b40843bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b4081932e0 con 0x55b4083c6480 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b408184650 con 0x55b4083c5b60 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b4081ca1c0 con 0x55b40837c2c0 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 10 client.0 _ll_register_callbacks cb 0x7f6c340c6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b40843bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b40843bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53034/0 (socket says 172.21.15.97:53034) 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 -- 172.21.15.97:0/3400177473 learned_addr learned my addr 172.21.15.97:0/3400177473 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c39e00640 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b40837c2c0 0x55b40843b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c39400640 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b40843b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 -- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b40837c2c0 msgr2=0x55b40843b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b40837c2c0 0x55b40843b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 -- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 msgr2=0x55b40843b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b40843b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 -- 172.21.15.97:0/3400177473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40817de30 con 0x55b4083c6480 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c3a800640 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b40843bc80 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6c340fa310 tx=0x7f6c340fc650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c246106702a8ca8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 1 -- 172.21.15.97:0/3400177473 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340fcbb0 con 0x55b4083c6480 2024-09-10T21:34:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 1 -- 172.21.15.97:0/3400177473 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f6c34112ae0 con 0x55b4083c6480 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2801b6f0 con 0x55b4083c6480 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 1 -- 172.21.15.97:0/3400177473 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f6c34101ad0 con 0x55b4083c6480 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 10 client.10026 fetch_fsmap learned FSMap version 291 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x55b40816c2b0 con 0x55b4083c6480 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 1 -- 172.21.15.97:0/3400177473 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c2801b6f0 con 0x55b4083c6480 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 1 -- 172.21.15.97:0/3400177473 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 291) ==== 3502+0+0 (secure 0 0 0) 0x7f6c34129150 con 0x55b4083c6480 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 10 client.10026 fetch_fsmap finished waiting for FSMap version 291 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 10 client.10026 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 20 client.10026 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083da420 0x55b4083da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 4 client.10026 mds_command: new command op to 9942 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 --> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b4081d5670 con 0x55b4083da420 2024-09-10T21:34:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c39e00640 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083da420 0x55b4083da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c39e00640 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083da420 0x55b4083da800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b74934717461f9 server_cookie=32174cd7e70f751e in_seq=0 out_seq=0 2024-09-10T21:34:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 10 client.10026 ms_handle_connect on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 1 -- 172.21.15.97:0/3400177473 <== mds.0 v2:172.21.15.97:6832/1998443497 1 ==== command_reply(tid 0: 0 ) ==== 8+0+194 (crc 0 0 0) 0x55b4081d5670 con 0x55b4083da420 2024-09-10T21:34:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 10 client.10026 handle_command_reply: tid=0 2024-09-10T21:34:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 client.10026 shutdown 2024-09-10T21:34:34.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 msgr2=0x55b40843bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b40843bc80 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6c340fa310 tx=0x7f6c340fc650 comp rx=0 tx=0).stop 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 shutdown_connections 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:6832/1998443497,v1:172.21.15.97:6834/1998443497] conn(0x55b4083da420 0x55b4083da800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b4083c6480 0x55b40843bc80 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4083c5b60 0x55b40843b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 --2- 172.21.15.97:0/3400177473 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b40837c2c0 0x55b40843b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c1b400640 0 client.10026 ms_handle_reset on v2:172.21.15.97:6832/1998443497 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 >> 172.21.15.97:0/3400177473 conn(0x55b408378270 msgr2=0x55b40837a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 shutdown_connections 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 1 -- 172.21.15.97:0/3400177473 wait complete. 2024-09-10T21:34:34.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.210+0000 7f6c43d78000 20 client.10026 trim_cache size 0 max 16384 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "dentry", 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 1257126858, 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1, 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: "frag": "*", 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: "dname": "subdir", 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_id": "head", 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/subdir" 2024-09-10T21:34:34.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:34:34.219 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c1c0725e0 msgr2=0x7f6c1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c1c0725e0 0x7f6c1c074aa0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c007fd0 tx=0x7f6c2c009040 comp rx=0 tx=0).stop 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c3c15a730 msgr2=0x7f6c3c14e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c3c15a730 0x7f6c3c14e250 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f6c3401d1d0 tx=0x7f6c340092c0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 shutdown_connections 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6c1c0725e0 0x7f6c1c074aa0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c3c15a730 0x7f6c3c14e250 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c3c1511f0 0x7f6c3c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 --2- 172.21.15.97:0/1923304750 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6c3c0ceee0 0x7f6c3c1e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 >> 172.21.15.97:0/1923304750 conn(0x7f6c3c0b1780 msgr2=0x7f6c3c15c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 shutdown_connections 2024-09-10T21:34:34.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.214+0000 7f6c40a00640 1 -- 172.21.15.97:0/1923304750 wait complete. 2024-09-10T21:34:34.231 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2024-09-10T21:34:34.231 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:34.231 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:34.240 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:34.241 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:34.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:34:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- 172.21.15.97:0/3772650887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 msgr2=0x7f0038160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 --2- 172.21.15.97:0/3772650887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038160220 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f002001c670 comp rx=0 tx=0).stop 2024-09-10T21:34:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- 172.21.15.97:0/3772650887 shutdown_connections 2024-09-10T21:34:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 --2- 172.21.15.97:0/3772650887 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f003806cb40 0x7f00380d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 --2- 172.21.15.97:0/3772650887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038160220 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- 172.21.15.97:0/3772650887 >> 172.21.15.97:0/3772650887 conn(0x7f00380b3c70 msgr2=0x7f00380b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- 172.21.15.97:0/3772650887 shutdown_connections 2024-09-10T21:34:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- 172.21.15.97:0/3772650887 wait complete. 2024-09-10T21:34:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 Processor -- start 2024-09-10T21:34:34.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- start start 2024-09-10T21:34:34.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f003806cb40 0x7f00380b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f003815f120 0x7f0038148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00381694e0 con 0x7f00380af520 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0038169360 con 0x7f003815f120 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f003de00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0038169660 con 0x7f003806cb40 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038148370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53050/0 (socket says 172.21.15.97:53050) 2024-09-10T21:34:34.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 -- 172.21.15.97:0/1957061420 learned_addr learned my addr 172.21.15.97:0/1957061420 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:34.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f0036a00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f003806cb40 0x7f00380b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f0037400640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f003815f120 0x7f0038148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 -- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f003806cb40 msgr2=0x7f00380b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f003806cb40 0x7f00380b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 -- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f003815f120 msgr2=0x7f0038148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f003815f120 0x7f0038148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 -- 172.21.15.97:0/1957061420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00381488b0 con 0x7f00380af520 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f0037400640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f003815f120 0x7f0038148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.478+0000 7f002e000640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038148370 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f002001c4a0 tx=0x7f00200a6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f002fe00640 1 -- 172.21.15.97:0/1957061420 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00200ac070 con 0x7f00380af520 2024-09-10T21:34:34.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f002fe00640 1 -- 172.21.15.97:0/1957061420 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0020004310 con 0x7f00380af520 2024-09-10T21:34:34.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f002fe00640 1 -- 172.21.15.97:0/1957061420 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00200a6b70 con 0x7f00380af520 2024-09-10T21:34:34.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0038151c10 con 0x7f00380af520 2024-09-10T21:34:34.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0038152190 con 0x7f00380af520 2024-09-10T21:34:34.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00380d0170 con 0x7f00380af520 2024-09-10T21:34:34.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f002fe00640 1 -- 172.21.15.97:0/1957061420 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f00200a6d10 con 0x7f00380af520 2024-09-10T21:34:34.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f002fe00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f00140725e0 0x7f0014074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f002fe00640 1 -- 172.21.15.97:0/1957061420 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f002012dde0 con 0x7f00380af520 2024-09-10T21:34:34.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.482+0000 7f0036a00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f00140725e0 0x7f0014074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.486+0000 7f0036a00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f00140725e0 0x7f0014074aa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0038156620 tx=0x7f0028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.486+0000 7f002fe00640 1 -- 172.21.15.97:0/1957061420 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f00200bf020 con 0x7f00380af520 2024-09-10T21:34:34.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f00380d1080 con 0x7f00380af520 2024-09-10T21:34:34.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f002fe00640 1 -- 172.21.15.97:0/1957061420 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 66 entries v129) ==== 81+0+4010 (secure 0 0 0) 0x7f00380cea30 con 0x7f00380af520 2024-09-10T21:34:34.665 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1087733461 2024-09-11T21:34:19.533818+0000 2024-09-10T21:34:34.665 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1087733461 2024-09-11T21:34:19.533818+0000 2024-09-10T21:34:34.665 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2872418710 2024-09-11T21:34:14.052935+0000 2024-09-10T21:34:34.665 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:34:34.665 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:34:34.665 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:34:34.666 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2872418710 2024-09-11T21:34:14.052935+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:34:34.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:34:34.668 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:34:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:34:34.670 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:34:34.670 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:34:34.670 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:34:34.670 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:34:34.670 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:34:34.670 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:listed 66 entries 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f00140725e0 msgr2=0x7f0014074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f00140725e0 0x7f0014074aa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0038156620 tx=0x7f0028008040 comp rx=0 tx=0).stop 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 msgr2=0x7f0038148370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038148370 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f002001c4a0 tx=0x7f00200a6490 comp rx=0 tx=0).stop 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 shutdown_connections 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f00140725e0 0x7f0014074aa0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f003815f120 0x7f0038148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380af520 0x7f0038148370 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 --2- 172.21.15.97:0/1957061420 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f003806cb40 0x7f00380b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 >> 172.21.15.97:0/1957061420 conn(0x7f00380b3c70 msgr2=0x7f0038156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 shutdown_connections 2024-09-10T21:34:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.666+0000 7f003de00640 1 -- 172.21.15.97:0/1957061420 wait complete. 2024-09-10T21:34:34.686 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:34:34.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:34.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:34:34.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:34:34.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:34.710 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:34:34.760 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:34:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 -- 172.21.15.97:0/3759673280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 msgr2=0x7f1a1c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 --2- 172.21.15.97:0/3759673280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c0b3030 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009900 tx=0x7f1a0801c810 comp rx=0 tx=0).stop 2024-09-10T21:34:34.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 -- 172.21.15.97:0/3759673280 shutdown_connections 2024-09-10T21:34:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 --2- 172.21.15.97:0/3759673280 >> v2:172.21.15.194:3300/0 conn(0x7f1a1c0af270 0x7f1a1c15f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 --2- 172.21.15.97:0/3759673280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c0b3030 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 -- 172.21.15.97:0/3759673280 >> 172.21.15.97:0/3759673280 conn(0x7f1a1c0b3580 msgr2=0x7f1a1c0b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 -- 172.21.15.97:0/3759673280 shutdown_connections 2024-09-10T21:34:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.946+0000 7f1a1b200640 1 -- 172.21.15.97:0/3759673280 wait complete. 2024-09-10T21:34:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 Processor -- start 2024-09-10T21:34:34.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 -- start start 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1c0af270 0x7f1a1c14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1a1c15f1d0 0x7f1a1c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c0b28a0 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c0b2720 con 0x7f1a1c0af270 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1c1692c0 con 0x7f1a1c15f1d0 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c144e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53082/0 (socket says 172.21.15.97:53082) 2024-09-10T21:34:34.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 -- 172.21.15.97:0/209399744 learned_addr learned my addr 172.21.15.97:0/209399744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19e00640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1c0af270 0x7f1a1c14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1a800640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1a1c15f1d0 0x7f1a1c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 -- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1a1c15f1d0 msgr2=0x7f1a1c0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1a1c15f1d0 0x7f1a1c0ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 -- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1c0af270 msgr2=0x7f1a1c14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1c0af270 0x7f1a1c14ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 -- 172.21.15.97:0/209399744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c14f290 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a19400640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c144e90 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009850 tx=0x7f1a080a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a12000640 1 -- 172.21.15.97:0/209399744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080b2040 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a12000640 1 -- 172.21.15.97:0/209399744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a08005020 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1c14f550 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a12000640 1 -- 172.21.15.97:0/209399744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080044f0 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1c15a5e0 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.950+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a1c15f770 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.954+0000 7f1a12000640 1 -- 172.21.15.97:0/209399744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1a08005450 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.954+0000 7f1a12000640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19fc072610 0x7f19fc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:34.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.954+0000 7f1a12000640 1 -- 172.21.15.97:0/209399744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9113+0+0 (secure 0 0 0) 0x7f1a0812da40 con 0x7f1a1c0b0b30 2024-09-10T21:34:34.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.954+0000 7f1a19e00640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19fc072610 0x7f19fc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:34.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.954+0000 7f1a19e00640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19fc072610 0x7f19fc074ad0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:34.958+0000 7f1a12000640 1 -- 172.21.15.97:0/209399744 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a08016490 con 0x7f1a1c0b0b30 2024-09-10T21:34:35.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.154+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1a1c0b79f0 con 0x7f1a1c0b0b30 2024-09-10T21:34:35.154 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:35.154+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:34:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.158+0000 7f1a12000640 1 -- 172.21.15.97:0/209399744 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v292) ==== 161+0+0 (secure 0 0 0) 0x7f1a080f8320 con 0x7f1a1c0b0b30 2024-09-10T21:34:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19fc072610 msgr2=0x7f19fc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19fc072610 0x7f19fc074ad0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a0c009040 comp rx=0 tx=0).stop 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 msgr2=0x7f1a1c144e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c144e90 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009850 tx=0x7f1a080a56f0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 shutdown_connections 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19fc072610 0x7f19fc074ad0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1a1c15f1d0 0x7f1a1c0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1c0b0b30 0x7f1a1c144e90 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 --2- 172.21.15.97:0/209399744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1c0af270 0x7f1a1c14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 >> 172.21.15.97:0/209399744 conn(0x7f1a1c0b3580 msgr2=0x7f1a1c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:35.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 shutdown_connections 2024-09-10T21:34:35.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.162+0000 7f1a1b200640 1 -- 172.21.15.97:0/209399744 wait complete. 2024-09-10T21:34:35.164 INFO:tasks.ceph.mds.a.smithi097.stderr: -589> 2024-09-10T21:34:29.566+0000 7eff59800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:35.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:34:35.206 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:34:35.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 -- 172.21.15.97:0/603411043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac006ec40 msgr2=0x7f6ac006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:35.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 --2- 172.21.15.97:0/603411043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac006ec40 0x7f6ac006f090 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00aa40 tx=0x7f6aac020400 comp rx=0 tx=0).stop 2024-09-10T21:34:35.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 -- 172.21.15.97:0/603411043 shutdown_connections 2024-09-10T21:34:35.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 --2- 172.21.15.97:0/603411043 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ac00ceee0 0x7f6ac00cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 --2- 172.21.15.97:0/603411043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac006ec40 0x7f6ac006f090 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 -- 172.21.15.97:0/603411043 >> 172.21.15.97:0/603411043 conn(0x7f6ac00b1780 msgr2=0x7f6ac00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:35.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 -- 172.21.15.97:0/603411043 shutdown_connections 2024-09-10T21:34:35.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 -- 172.21.15.97:0/603411043 wait complete. 2024-09-10T21:34:35.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 Processor -- start 2024-09-10T21:34:35.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.326+0000 7f6ac7800640 1 -- start start 2024-09-10T21:34:35.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ac006ec40 0x7f6ac013bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:35.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac00ceee0 0x7f6ac01e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:35.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ac01511f0 0x7f6ac0150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac006e740 con 0x7f6ac00ceee0 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac006e5c0 con 0x7f6ac006ec40 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac006e8c0 con 0x7f6ac01511f0 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac00ceee0 0x7f6ac01e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac00ceee0 0x7f6ac01e7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53134/0 (socket says 172.21.15.97:53134) 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 -- 172.21.15.97:0/1731839754 learned_addr learned my addr 172.21.15.97:0/1731839754 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac5800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ac01511f0 0x7f6ac0150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:35.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac4e00640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ac006ec40 0x7f6ac013bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:35.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 -- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ac01511f0 msgr2=0x7f6ac0150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ac01511f0 0x7f6ac0150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 -- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ac006ec40 msgr2=0x7f6ac013bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ac006ec40 0x7f6ac013bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 -- 172.21.15.97:0/1731839754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac013e8b0 con 0x7f6ac00ceee0 2024-09-10T21:34:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac5800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ac01511f0 0x7f6ac0150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:35.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abfe00640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac00ceee0 0x7f6ac01e7490 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0b3040 tx=0x7f6aac0b0480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:35.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abcc00640 1 -- 172.21.15.97:0/1731839754 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0b5040 con 0x7f6ac00ceee0 2024-09-10T21:34:35.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abcc00640 1 -- 172.21.15.97:0/1731839754 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6aac0b0c30 con 0x7f6ac00ceee0 2024-09-10T21:34:35.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac013eb90 con 0x7f6ac00ceee0 2024-09-10T21:34:35.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6abcc00640 1 -- 172.21.15.97:0/1731839754 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac003860 con 0x7f6ac00ceee0 2024-09-10T21:34:35.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac013f150 con 0x7f6ac00ceee0 2024-09-10T21:34:35.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.330+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac006f090 con 0x7f6ac00ceee0 2024-09-10T21:34:35.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.334+0000 7f6abcc00640 1 -- 172.21.15.97:0/1731839754 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6aac0b8070 con 0x7f6ac00ceee0 2024-09-10T21:34:35.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.334+0000 7f6abcc00640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6aa00725e0 0x7f6aa0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:35.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.334+0000 7f6abcc00640 1 -- 172.21.15.97:0/1731839754 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f6aac131710 con 0x7f6ac00ceee0 2024-09-10T21:34:35.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.334+0000 7f6ac4e00640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6aa00725e0 0x7f6aa0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:35.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.334+0000 7f6ac4e00640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6aa00725e0 0x7f6aa0074aa0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4007fd0 tx=0x7f6ab4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.338+0000 7f6abcc00640 1 -- 172.21.15.97:0/1731839754 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6aac0b0e20 con 0x7f6ac00ceee0 2024-09-10T21:34:35.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:35.566+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f6ac00b1f30 con 0x7f6ac00ceee0 2024-09-10T21:34:36.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.158+0000 7f6abcc00640 1 -- 172.21.15.97:0/1731839754 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v293) ==== 107+0+0 (secure 0 0 0) 0x7f6aac019f60 con 0x7f6ac00ceee0 2024-09-10T21:34:36.159 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:34:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6aa00725e0 msgr2=0x7f6aa0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6aa00725e0 0x7f6aa0074aa0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4007fd0 tx=0x7f6ab4009040 comp rx=0 tx=0).stop 2024-09-10T21:34:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac00ceee0 msgr2=0x7f6ac01e7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac00ceee0 0x7f6ac01e7490 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0b3040 tx=0x7f6aac0b0480 comp rx=0 tx=0).stop 2024-09-10T21:34:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 shutdown_connections 2024-09-10T21:34:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6aa00725e0 0x7f6aa0074aa0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ac01511f0 0x7f6ac0150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ac00ceee0 0x7f6ac01e7490 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 --2- 172.21.15.97:0/1731839754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ac006ec40 0x7f6ac013bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 >> 172.21.15.97:0/1731839754 conn(0x7f6ac00b1780 msgr2=0x7f6ac00b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 shutdown_connections 2024-09-10T21:34:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.162+0000 7f6ac7800640 1 -- 172.21.15.97:0/1731839754 wait complete. 2024-09-10T21:34:36.181 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:34:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 Processor -- start 2024-09-10T21:34:36.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 -- start start 2024-09-10T21:34:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc7c002350 0x5627c1a52430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5627c1ae9600 0x5627c1a2f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae99e0 0x5627c1a2fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5627c197ea00 con 0x5627c1ae99e0 2024-09-10T21:34:36.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5627c19dcf40 con 0x7efc7c002350 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5627c1b053c0 con 0x5627c1ae9600 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae99e0 0x5627c1a2fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae99e0 0x5627c1a2fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53148/0 (socket says 172.21.15.97:53148) 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 -- 172.21.15.97:0/1839096360 learned_addr learned my addr 172.21.15.97:0/1839096360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 -- 172.21.15.97:0/1839096360 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5627c1ae9600 msgr2=0x5627c1a2f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 --2- 172.21.15.97:0/1839096360 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5627c1ae9600 0x5627c1a2f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 -- 172.21.15.97:0/1839096360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc7c002350 msgr2=0x5627c1a52430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 --2- 172.21.15.97:0/1839096360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc7c002350 0x5627c1a52430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 -- 172.21.15.97:0/1839096360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c1957c10 con 0x5627c1ae99e0 2024-09-10T21:34:36.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc96400640 1 --2- 172.21.15.97:0/1839096360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae99e0 0x5627c1a2fa90 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x5627c1a082b0 tx=0x7efc90009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69daaf960112382f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc8f400640 1 -- 172.21.15.97:0/1839096360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90005050 con 0x5627c1ae99e0 2024-09-10T21:34:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc8f400640 1 -- 172.21.15.97:0/1839096360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc90007590 con 0x5627c1ae99e0 2024-09-10T21:34:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc8f400640 1 -- 172.21.15.97:0/1839096360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900079c0 con 0x5627c1ae99e0 2024-09-10T21:34:36.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 -- 172.21.15.97:0/1839096360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae99e0 msgr2=0x5627c1a2fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 --2- 172.21.15.97:0/1839096360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae99e0 0x5627c1a2fa90 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x5627c1a082b0 tx=0x7efc90009940 comp rx=0 tx=0).stop 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 -- 172.21.15.97:0/1839096360 shutdown_connections 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 --2- 172.21.15.97:0/1839096360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae99e0 0x5627c1a2fa90 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 --2- 172.21.15.97:0/1839096360 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5627c1ae9600 0x5627c1a2f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 --2- 172.21.15.97:0/1839096360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc7c002350 0x5627c1a52430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.270+0000 7efc99249080 1 -- 172.21.15.97:0/1839096360 >> 172.21.15.97:0/1839096360 conn(0x5627c1a2f140 msgr2=0x5627c1a5bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- 172.21.15.97:0/1839096360 shutdown_connections 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- 172.21.15.97:0/1839096360 wait complete. 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 Processor -- start 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- start start 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae9600 0x5627c1a53980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc7c002350 0x5627c1ba3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5627c1a54390 0x5627c1a2df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5627c1a5f8d0 con 0x5627c1ae9600 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5627c1ae9ff0 con 0x5627c1a54390 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5627c1a5f390 con 0x7efc7c002350 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae9600 0x5627c1a53980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae9600 0x5627c1a53980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53156/0 (socket says 172.21.15.97:53156) 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 learned_addr learned my addr 172.21.15.97:0/2816280905 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc7c002350 0x5627c1ba3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc96400640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5627c1a54390 0x5627c1a2df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc7c002350 msgr2=0x5627c1ba3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efc7c002350 0x5627c1ba3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5627c1a54390 msgr2=0x5627c1a2df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5627c1a54390 0x5627c1a2df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c1a6fc70 con 0x5627c1ae9600 2024-09-10T21:34:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc95a00640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae9600 0x5627c1a53980 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7efc8801bba0 tx=0x7efc880af620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc8e000640 1 -- 172.21.15.97:0/2816280905 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88004e40 con 0x5627c1ae9600 2024-09-10T21:34:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc8e000640 1 -- 172.21.15.97:0/2816280905 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc880a7420 con 0x5627c1ae9600 2024-09-10T21:34:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627c1a6ff10 con 0x5627c1ae9600 2024-09-10T21:34:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc8e000640 1 -- 172.21.15.97:0/2816280905 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880af9a0 con 0x5627c1ae9600 2024-09-10T21:34:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.274+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c1a70450 con 0x5627c1ae9600 2024-09-10T21:34:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc8e000640 1 -- 172.21.15.97:0/2816280905 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efc880c1060 con 0x5627c1ae9600 2024-09-10T21:34:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc8e000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc78070fc0 0x7efc78073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc8e000640 1 -- 172.21.15.97:0/2816280905 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7efc8812d080 con 0x5627c1ae9600 2024-09-10T21:34:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc95000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc78070fc0 0x7efc78073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.278 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:34:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5627c1a708f0 0x5627c1a70cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1a71210 con 0x5627c1a708f0 2024-09-10T21:34:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc96400640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5627c1a708f0 0x5627c1a70cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc95000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc78070fc0 0x7efc78073480 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7efc84003000 tx=0x7efc84004420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.278+0000 7efc96400640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5627c1a708f0 0x5627c1a70cd0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.282+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v130'1033 uv1007 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc90015410 con 0x5627c1a708f0 2024-09-10T21:34:36.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.282+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1a715b0 con 0x5627c1a708f0 2024-09-10T21:34:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.282+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v130'1034 uv1034 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc90015410 con 0x5627c1a708f0 2024-09-10T21:34:36.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.282+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bad2f0 con 0x5627c1a708f0 2024-09-10T21:34:36.284 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:34:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.286+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v130'1035 uv1035 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.286+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bae6a0 con 0x5627c1a708f0 2024-09-10T21:34:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.286+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v130'1036 uv1036 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.286+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baea40 con 0x5627c1a708f0 2024-09-10T21:34:36.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.290+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v130'1037 uv1037 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.290+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bad690 con 0x5627c1a708f0 2024-09-10T21:34:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.290+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v130'1038 uv1038 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.290+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bada30 con 0x5627c1a708f0 2024-09-10T21:34:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.290+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v130'1039 uv1013 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.290+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bade20 con 0x5627c1a708f0 2024-09-10T21:34:36.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.294+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v130'1040 uv1040 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.294+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bade20 con 0x5627c1a708f0 2024-09-10T21:34:36.294 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:34:36.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.294+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v130'1041 uv1041 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.294+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.298+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v130'1042 uv1042 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.298+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.298+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v130'1043 uv1043 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.298+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.298+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v130'1044 uv1044 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.302+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.302+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v130'1045 uv1020 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.302+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.302+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v130'1046 uv1046 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.302+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.304 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:34:36.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.306+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v130'1047 uv1047 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.306+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.306+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v130'1048 uv1048 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.306+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.306+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v130'1049 uv1049 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.310+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baddd0 con 0x5627c1a708f0 2024-09-10T21:34:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.310+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v130'1050 uv1050 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.310+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.310+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v130'1051 uv1051 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.310+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.314+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v130'1052 uv1026 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.314+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.314+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v130'1053 uv1053 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.314+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.315 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:34:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.318+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v130'1054 uv1054 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.318+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bafdf0 con 0x5627c1a708f0 2024-09-10T21:34:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.318+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v130'1055 uv1055 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.318+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bafdf0 con 0x5627c1a708f0 2024-09-10T21:34:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.318+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v130'1056 uv1056 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.318+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.322+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v130'1057 uv1057 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.322+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.322+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v130'1058 uv1032 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.322+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.326+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v130'1059 uv1059 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.326+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.325 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:34:36.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.326+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v130'1060 uv1060 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.326+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bafdf0 con 0x5627c1a708f0 2024-09-10T21:34:36.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.326+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v130'1061 uv1061 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.326+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bafdf0 con 0x5627c1a708f0 2024-09-10T21:34:36.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.330+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v130'1062 uv1062 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.330+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1baede0 con 0x5627c1a708f0 2024-09-10T21:34:36.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.330+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v130'1063 uv1063 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1a708f0 2024-09-10T21:34:36.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.330+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5627c1baf2f0 0x5627c1bb16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.330+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb1c20 con 0x5627c1baf2f0 2024-09-10T21:34:36.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.330+0000 7efc95a00640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5627c1baf2f0 0x5627c1bb16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.334+0000 7efc95a00640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5627c1baf2f0 0x5627c1bb16e0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.334+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v130'752 uv731 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.334+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bb1fc0 con 0x5627c1baf2f0 2024-09-10T21:34:36.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.334+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v130'753 uv753 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.334+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb2360 con 0x5627c1baf2f0 2024-09-10T21:34:36.336 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:34:36.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.338+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v130'754 uv754 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.338+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb2bb0 con 0x5627c1baf2f0 2024-09-10T21:34:36.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.338+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v130'755 uv755 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.338+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb2f50 con 0x5627c1baf2f0 2024-09-10T21:34:36.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.338+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v130'756 uv756 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.338+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb32f0 con 0x5627c1baf2f0 2024-09-10T21:34:36.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.342+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v130'757 uv750 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.342+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bb3690 con 0x5627c1baf2f0 2024-09-10T21:34:36.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.342+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v130'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.342+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.343 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:34:36.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.342+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v130'759 uv759 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.342+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.346+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v130'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.346+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.346+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v130'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.346+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.346+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v130'762 uv751 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.346+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.350+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v130'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.350+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.350 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:34:36.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.350+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v130'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.350+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb7c00 con 0x5627c1baf2f0 2024-09-10T21:34:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.354+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v130'765 uv765 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.354+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb7c00 con 0x5627c1baf2f0 2024-09-10T21:34:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.354+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v130'766 uv766 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.354+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.354+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v130'767 uv747 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.354+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.358+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v130'768 uv768 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.358+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.357 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:34:36.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.358+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v130'769 uv769 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.358+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb4a40 con 0x5627c1baf2f0 2024-09-10T21:34:36.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.358+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v130'770 uv770 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.358+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb4a40 con 0x5627c1baf2f0 2024-09-10T21:34:36.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.362+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v130'771 uv771 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.362+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb3a30 con 0x5627c1baf2f0 2024-09-10T21:34:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.362+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v130'772 uv772 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1baf2f0 2024-09-10T21:34:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.362+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5627c1bb3f40 0x5627c1bb6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.362+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb6900 con 0x5627c1bb3f40 2024-09-10T21:34:36.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.362+0000 7efc95000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5627c1bb3f40 0x5627c1bb6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.362+0000 7efc95000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5627c1bb3f40 0x5627c1bb6330 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.366+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.366+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bb7180 con 0x5627c1bb3f40 2024-09-10T21:34:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.366+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.366+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb7520 con 0x5627c1bb3f40 2024-09-10T21:34:36.366 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:34:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.366+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.366+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb88d0 con 0x5627c1bb3f40 2024-09-10T21:34:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb8d50 con 0x5627c1bb3f40 2024-09-10T21:34:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bb78c0 con 0x5627c1bb3f40 2024-09-10T21:34:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5627c1bb8380 0x5627c1bbb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbb750 con 0x5627c1bb8380 2024-09-10T21:34:36.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc96400640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5627c1bb8380 0x5627c1bbb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.370+0000 7efc96400640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5627c1bb8380 0x5627c1bbb1a0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.374+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v130'151 uv150 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.374+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbbb40 con 0x5627c1bb8380 2024-09-10T21:34:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.374+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v130'152 uv152 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.374+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbbb40 con 0x5627c1bb8380 2024-09-10T21:34:36.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.374+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v130'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.374 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:34:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.374+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbbb40 con 0x5627c1bb8380 2024-09-10T21:34:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v130'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.379 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:34:36.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.378+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.382+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.382+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.382+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.382+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.382+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.382+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.386+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v130'371 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.386+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.386+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v130'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.386+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.386 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:34:36.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.386+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v130'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.386+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v130'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v130'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v130'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v130'377 uv370 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.390+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.394+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v130'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.394+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.394+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v130'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.394 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:34:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.394+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.394+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v130'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.394+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb8380 2024-09-10T21:34:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v130'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v130'168 uv167 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v130'169 uv169 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbba70 con 0x5627c1bb3f40 2024-09-10T21:34:36.399 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:34:36.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v130'170 uv170 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.398+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbbf60 con 0x5627c1bb3f40 2024-09-10T21:34:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.402+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v130'171 uv171 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.402+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbbf60 con 0x5627c1bb3f40 2024-09-10T21:34:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.402+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v130'172 uv172 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bb3f40 2024-09-10T21:34:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.402+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5627c1bbc470 0x5627c1bbe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.402+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbeda0 con 0x5627c1bbc470 2024-09-10T21:34:36.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.402+0000 7efc95a00640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5627c1bbc470 0x5627c1bbe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.402+0000 7efc95a00640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5627c1bbc470 0x5627c1bbe860 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbee60 con 0x5627c1bbc470 2024-09-10T21:34:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.406 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:34:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc0290 con 0x5627c1bbc470 2024-09-10T21:34:36.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.406+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc0290 con 0x5627c1bbc470 2024-09-10T21:34:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.410+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.410+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.410+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.410+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.410+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v130'335 uv329 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.410+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.414+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v130'336 uv336 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.414+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.414 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:34:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.414+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v130'337 uv337 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.414+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.414+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v130'338 uv338 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.414+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.418+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v130'339 uv339 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.418+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.418+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v130'340 uv334 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.418+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.418+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v130'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.418+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.420 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:34:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.422+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v130'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.422+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf280 con 0x5627c1bbc470 2024-09-10T21:34:36.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.422+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v130'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.422+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc2090 con 0x5627c1bbc470 2024-09-10T21:34:36.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.422+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v130'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.422+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.426+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v130'147 uv145 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.426+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.426+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v130'148 uv148 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.426+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.426+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v130'149 uv149 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.428 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:34:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.426+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v130'150 uv150 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v130'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v130'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v130'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.431 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:34:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.430+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v130'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v130'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bbf730 con 0x5627c1bb8380 2024-09-10T21:34:36.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v130'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc9001e040 con 0x5627c1bb8380 2024-09-10T21:34:36.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5627c1bbfdb0 0x5627c1bc26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbfdb0 2024-09-10T21:34:36.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.434+0000 7efc95000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5627c1bbfdb0 0x5627c1bc26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.438+0000 7efc95000640 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5627c1bbfdb0 0x5627c1bc26b0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.438+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v130'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bbfdb0 2024-09-10T21:34:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.438+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbfdb0 2024-09-10T21:34:36.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.438+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v130'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bbfdb0 2024-09-10T21:34:36.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.438+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbfdb0 2024-09-10T21:34:36.439 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:34:36.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.438+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v130'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bbfdb0 2024-09-10T21:34:36.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.438+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbfdb0 2024-09-10T21:34:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.442+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v130'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bbfdb0 2024-09-10T21:34:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.442+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbfdb0 2024-09-10T21:34:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.442+0000 7efc95000640 1 -- 172.21.15.97:0/2816280905 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v130'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7efc8400a940 con 0x5627c1bbfdb0 2024-09-10T21:34:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.442+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.442+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v130'416 uv406 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.442+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.446+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v130'417 uv417 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.446+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.446+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v130'418 uv418 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.445 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:34:36.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.446+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.446+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v130'419 uv419 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.446+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.450+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v130'420 uv420 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.450+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.450+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v130'421 uv414 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.450+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.450+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v130'422 uv422 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.450+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.454+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v130'423 uv423 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.452 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:34:36.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.454+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.454+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v130'424 uv424 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.454+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.454+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v130'425 uv425 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.454+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc21a0 con 0x5627c1bbc470 2024-09-10T21:34:36.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v130'426 uv426 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x5627c1bc2bf0 con 0x5627c1bbc470 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc95a00640 1 -- 172.21.15.97:0/2816280905 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v130'427 uv427 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7efc880ad070 con 0x5627c1bbc470 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5627c1bb3f40 msgr2=0x5627c1bb6330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5627c1bb3f40 0x5627c1bb6330 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5627c1bbc470 msgr2=0x5627c1bbe860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5627c1bbc470 0x5627c1bbe860 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5627c1bbfdb0 msgr2=0x5627c1bc26b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5627c1bbfdb0 0x5627c1bc26b0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5627c1bb8380 msgr2=0x5627c1bbb1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5627c1bb8380 0x5627c1bbb1a0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5627c1baf2f0 msgr2=0x5627c1bb16e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5627c1baf2f0 0x5627c1bb16e0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5627c1a708f0 msgr2=0x5627c1a70cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5627c1a708f0 0x5627c1a70cd0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 reap_dead start 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 reap_dead start 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 reap_dead start 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc96400640 1 -- 172.21.15.97:0/2816280905 reap_dead start 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc78070fc0 msgr2=0x7efc78073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc78070fc0 0x7efc78073480 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7efc84003000 tx=0x7efc84004420 comp rx=0 tx=0).stop 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae9600 msgr2=0x5627c1a53980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae9600 0x5627c1a53980 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7efc8801bba0 tx=0x7efc880af620 comp rx=0 tx=0).stop 2024-09-10T21:34:36.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 shutdown_connections 2024-09-10T21:34:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efc78070fc0 0x7efc78073480 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 --2- 172.21.15.97:0/2816280905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c1ae9600 0x5627c1a53980 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 >> 172.21.15.97:0/2816280905 conn(0x5627c1a2f140 msgr2=0x5627c1a5e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 shutdown_connections 2024-09-10T21:34:36.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.458+0000 7efc99249080 1 -- 172.21.15.97:0/2816280905 wait complete. 2024-09-10T21:34:36.462 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 Processor -- start 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- start start 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c23752940 0x555c23767820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dcb00 0x555c23767d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c237dd2a0 0x555c2388f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555c236cff40 con 0x555c237dcb00 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555c236dcd00 con 0x555c237dd2a0 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555c23671a00 con 0x555c23752940 2024-09-10T21:34:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dcb00 0x555c23767d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dcb00 0x555c23767d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53176/0 (socket says 172.21.15.97:53176) 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 -- 172.21.15.97:0/4185790269 learned_addr learned my addr 172.21.15.97:0/4185790269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfe200640 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c237dd2a0 0x555c2388f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfd800640 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c23752940 0x555c23767820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 -- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c23752940 msgr2=0x555c23767820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c23752940 0x555c23767820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 -- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c237dd2a0 msgr2=0x555c2388f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c237dd2a0 0x555c2388f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 -- 172.21.15.97:0/4185790269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c237231e0 con 0x555c237dcb00 2024-09-10T21:34:36.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cfce00640 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dcb00 0x555c23767d60 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00a440 tx=0x7f4cec00a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ca13ac5c7c5810 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cf7400640 1 -- 172.21.15.97:0/4185790269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00b930 con 0x555c237dcb00 2024-09-10T21:34:36.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cf7400640 1 -- 172.21.15.97:0/4185790269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cec00f070 con 0x555c237dcb00 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4cf7400640 1 -- 172.21.15.97:0/4185790269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec014470 con 0x555c237dcb00 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- 172.21.15.97:0/4185790269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dcb00 msgr2=0x555c23767d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dcb00 0x555c23767d60 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00a440 tx=0x7f4cec00a900 comp rx=0 tx=0).stop 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- 172.21.15.97:0/4185790269 shutdown_connections 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c237dd2a0 0x555c2388f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dcb00 0x555c23767d60 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 --2- 172.21.15.97:0/4185790269 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c23752940 0x555c23767820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- 172.21.15.97:0/4185790269 >> 172.21.15.97:0/4185790269 conn(0x555c23751520 msgr2=0x555c237fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- 172.21.15.97:0/4185790269 shutdown_connections 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.550+0000 7f4d00566080 1 -- 172.21.15.97:0/4185790269 wait complete. 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 Processor -- start 2024-09-10T21:34:36.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 -- start start 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c23752940 0x555c237f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c237dcb00 0x555c23765e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dd2a0 0x555c237663c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555c23756150 con 0x555c237dd2a0 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555c23756890 con 0x555c23752940 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555c23723480 con 0x555c237dcb00 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dd2a0 0x555c237663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfd800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c23752940 0x555c237f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfce00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c237dcb00 0x555c23765e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dd2a0 0x555c237663c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53178/0 (socket says 172.21.15.97:53178) 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 -- 172.21.15.97:0/3300513284 learned_addr learned my addr 172.21.15.97:0/3300513284 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 -- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c237dcb00 msgr2=0x555c23765e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555c237dcb00 0x555c23765e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 -- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c23752940 msgr2=0x555c237f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555c23752940 0x555c237f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 -- 172.21.15.97:0/3300513284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c238948f0 con 0x555c237dd2a0 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cfe200640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dd2a0 0x555c237663c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8004240 tx=0x7f4cf800d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cf6000640 1 -- 172.21.15.97:0/3300513284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8018070 con 0x555c237dd2a0 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555c23894b90 con 0x555c237dd2a0 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cf6000640 1 -- 172.21.15.97:0/3300513284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cf80030d0 con 0x555c237dd2a0 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4cf6000640 1 -- 172.21.15.97:0/3300513284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8001680 con 0x555c237dd2a0 2024-09-10T21:34:36.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.554+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c238950d0 con 0x555c237dd2a0 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4cf6000640 1 -- 172.21.15.97:0/3300513284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4cf8001a30 con 0x555c237dd2a0 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4cf6000640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ce0070ff0 0x7f4ce00734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4cf6000640 1 -- 172.21.15.97:0/3300513284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f4cf80928b0 con 0x555c237dd2a0 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4d00566080 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555c23895810 0x555c23895bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x555c23896130 con 0x555c23895810 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4cfd800640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ce0070ff0 0x7f4ce00734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4cfce00640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555c23895810 0x555c23895bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4cfd800640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ce0070ff0 0x7f4ce00734b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00ab000 tx=0x7f4cf0019a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.558+0000 7f4cfce00640 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555c23895810 0x555c23895bf0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4cfce00640 1 -- 172.21.15.97:0/3300513284 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v130'1064 uv1064 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4cec01c040 con 0x555c23895810 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555c23895810 msgr2=0x555c23895bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x555c23895810 0x555c23895bf0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ce0070ff0 msgr2=0x7f4ce00734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ce0070ff0 0x7f4ce00734b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00ab000 tx=0x7f4cf0019a00 comp rx=0 tx=0).stop 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dd2a0 msgr2=0x555c237663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 --2- 172.21.15.97:0/3300513284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c237dd2a0 0x555c237663c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8004240 tx=0x7f4cf800d5f0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4cfe200640 1 -- 172.21.15.97:0/3300513284 reap_dead start 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 shutdown_connections 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 >> 172.21.15.97:0/3300513284 conn(0x555c23751520 msgr2=0x555c237fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:36.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 shutdown_connections 2024-09-10T21:34:36.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.562+0000 7f4d00566080 1 -- 172.21.15.97:0/3300513284 wait complete. 2024-09-10T21:34:36.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:34:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 -- 172.21.15.97:0/2138010775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df0003a40 msgr2=0x7f4df0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 --2- 172.21.15.97:0/2138010775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df0003a40 0x7f4df0003e60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009b60 tx=0x7f4dfc01d1b0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e11c00640 1 -- 172.21.15.97:0/2138010775 reap_dead start 2024-09-10T21:34:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 -- 172.21.15.97:0/2138010775 shutdown_connections 2024-09-10T21:34:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 -- 172.21.15.97:0/2138010775 >> 172.21.15.97:0/2138010775 conn(0x7f4e0c0b1780 msgr2=0x7f4e0c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 -- 172.21.15.97:0/2138010775 shutdown_connections 2024-09-10T21:34:36.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 -- 172.21.15.97:0/2138010775 wait complete. 2024-09-10T21:34:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 Processor -- start 2024-09-10T21:34:36.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.758+0000 7f4e13c00640 1 -- start start 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df0003a40 0x7f4e0c1f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e0c0d0300 0x7f4e0c1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0c152810 0x7f4e0c152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c15ae40 con 0x7f4e0c0d0300 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c0b3b30 con 0x7f4e0c152810 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0c15afc0 con 0x7f4df0003a40 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e0c0d0300 0x7f4e0c1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e0c0d0300 0x7f4e0c1434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53184/0 (socket says 172.21.15.97:53184) 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 -- 172.21.15.97:0/2075963346 learned_addr learned my addr 172.21.15.97:0/2075963346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e11200640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df0003a40 0x7f4e0c1f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e11c00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0c152810 0x7f4e0c152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 -- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df0003a40 msgr2=0x7f4e0c1f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df0003a40 0x7f4e0c1f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 -- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0c152810 msgr2=0x7f4e0c152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0c152810 0x7f4e0c152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:36.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 -- 172.21.15.97:0/2075963346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c0d0780 con 0x7f4e0c0d0300 2024-09-10T21:34:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e03e00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e0c0d0300 0x7f4e0c1434f0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f4df4003f10 tx=0x7f4df4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e09600640 1 -- 172.21.15.97:0/2075963346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4011640 con 0x7f4e0c0d0300 2024-09-10T21:34:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e09600640 1 -- 172.21.15.97:0/2075963346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4df4016020 con 0x7f4e0c0d0300 2024-09-10T21:34:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e09600640 1 -- 172.21.15.97:0/2075963346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4011a60 con 0x7f4e0c0d0300 2024-09-10T21:34:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c1f84c0 con 0x7f4e0c0d0300 2024-09-10T21:34:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c1f8a80 con 0x7f4e0c0d0300 2024-09-10T21:34:36.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.762+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0c143a30 con 0x7f4e0c0d0300 2024-09-10T21:34:36.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.766+0000 7f4e09600640 1 -- 172.21.15.97:0/2075963346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4df4011c00 con 0x7f4e0c0d0300 2024-09-10T21:34:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.766+0000 7f4e09600640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4de80725e0 0x7f4de8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.766+0000 7f4e09600640 1 -- 172.21.15.97:0/2075963346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f4df4094ae0 con 0x7f4e0c0d0300 2024-09-10T21:34:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.766+0000 7f4e11200640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4de80725e0 0x7f4de8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:36.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.766+0000 7f4e11200640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4de80725e0 0x7f4de8074aa0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0ae000 tx=0x7f4dfc019f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.770+0000 7f4e09600640 1 -- 172.21.15.97:0/2075963346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4df4020020 con 0x7f4e0c0d0300 2024-09-10T21:34:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:36.970+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4e0c0008e0 con 0x7f4e0c0d0300 2024-09-10T21:34:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.166+0000 7f4e09600640 1 -- 172.21.15.97:0/2075963346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v294) ==== 162+0+0 (secure 0 0 0) 0x7f4df405f370 con 0x7f4e0c0d0300 2024-09-10T21:34:37.166 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:34:37.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4de80725e0 msgr2=0x7f4de8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4de80725e0 0x7f4de8074aa0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0ae000 tx=0x7f4dfc019f50 comp rx=0 tx=0).stop 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e0c0d0300 msgr2=0x7f4e0c1434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e0c0d0300 0x7f4e0c1434f0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f4df4003f10 tx=0x7f4df4001640 comp rx=0 tx=0).stop 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 shutdown_connections 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4de80725e0 0x7f4de8074aa0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0c152810 0x7f4e0c152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e0c0d0300 0x7f4e0c1434f0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 --2- 172.21.15.97:0/2075963346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4df0003a40 0x7f4e0c1f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 >> 172.21.15.97:0/2075963346 conn(0x7f4e0c0b1780 msgr2=0x7f4e0c0ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 shutdown_connections 2024-09-10T21:34:37.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.170+0000 7f4e13c00640 1 -- 172.21.15.97:0/2075963346 wait complete. 2024-09-10T21:34:37.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.738+0000 7fb799a00640 1 -- 172.21.15.97:0/3417599342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 msgr2=0x7fb778004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.738+0000 7fb799a00640 1 --2- 172.21.15.97:0/3417599342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 0x7fb778004810 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fb780006fa0 tx=0x7fb78001cec0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.738+0000 7fb793400640 1 -- 172.21.15.97:0/3417599342 reap_dead start 2024-09-10T21:34:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.738+0000 7fb799a00640 1 -- 172.21.15.97:0/3417599342 shutdown_connections 2024-09-10T21:34:37.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.738+0000 7fb799a00640 1 -- 172.21.15.97:0/3417599342 >> 172.21.15.97:0/3417599342 conn(0x7fb7940b37c0 msgr2=0x7fb7940b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:37.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.738+0000 7fb799a00640 1 -- 172.21.15.97:0/3417599342 shutdown_connections 2024-09-10T21:34:37.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.738+0000 7fb799a00640 1 -- 172.21.15.97:0/3417599342 wait complete. 2024-09-10T21:34:37.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 Processor -- start 2024-09-10T21:34:37.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 -- start start 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 0x7fb7941503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb79406e610 0x7fb79406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb79406ea90 0x7fb79406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940cddd0 con 0x7fb778004060 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940cdc50 con 0x7fb79406ea90 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb7940ce1a0 con 0x7fb79406e610 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 0x7fb7941503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 0x7fb7941503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60190/0 (socket says 172.21.15.97:60190) 2024-09-10T21:34:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 -- 172.21.15.97:0/2047524376 learned_addr learned my addr 172.21.15.97:0/2047524376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 -- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb79406e610 msgr2=0x7fb79406ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb79406e610 0x7fb79406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 -- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb79406ea90 msgr2=0x7fb79406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb79406ea90 0x7fb79406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 -- 172.21.15.97:0/2047524376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940b5ee0 con 0x7fb778004060 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb792a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 0x7fb7941503c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fb784006640 tx=0x7fb7840015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb78aa00640 1 -- 172.21.15.97:0/2047524376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784015020 con 0x7fb778004060 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb78aa00640 1 -- 172.21.15.97:0/2047524376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb784011960 con 0x7fb778004060 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb78aa00640 1 -- 172.21.15.97:0/2047524376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784001d50 con 0x7fb778004060 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7940b61c0 con 0x7fb778004060 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7940b6780 con 0x7fb778004060 2024-09-10T21:34:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.742+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7940cf800 con 0x7fb778004060 2024-09-10T21:34:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.746+0000 7fb78aa00640 1 -- 172.21.15.97:0/2047524376 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb7840103c0 con 0x7fb778004060 2024-09-10T21:34:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.746+0000 7fb78aa00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb774072610 0x7fb774074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.746+0000 7fb78aa00640 1 -- 172.21.15.97:0/2047524376 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7fb784094960 con 0x7fb778004060 2024-09-10T21:34:37.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.746+0000 7fb792000640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb774072610 0x7fb774074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:37.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.746+0000 7fb792000640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb774072610 0x7fb774074ad0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb7800ac000 tx=0x7fb778005840 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:37.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.750+0000 7fb78aa00640 1 -- 172.21.15.97:0/2047524376 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb78401f030 con 0x7fb778004060 2024-09-10T21:34:37.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.946+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7940b0720 con 0x7fb778004060 2024-09-10T21:34:37.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.946+0000 7fb78aa00640 1 -- 172.21.15.97:0/2047524376 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 295 v295) ==== 95+0+5233 (secure 0 0 0) 0x7fb784011d70 con 0x7fb778004060 2024-09-10T21:34:37.946 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:37.947 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":295,"btime":"2024-09-10T21:34:37:174431+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":295,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:37.174387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10047},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10047":{"gid":10047,"name":"a","rank":0,"incarnation":295,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/3083575804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3083575804},{"type":"v1","addr":"172.21.15.97:6834","nonce":3083575804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 295 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb774072610 msgr2=0x7fb774074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb774072610 0x7fb774074ad0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb7800ac000 tx=0x7fb778005840 comp rx=0 tx=0).stop 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 msgr2=0x7fb7941503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 0x7fb7941503c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fb784006640 tx=0x7fb7840015e0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 shutdown_connections 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb774072610 0x7fb774074ad0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb79406ea90 0x7fb79406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb79406e610 0x7fb79406ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 --2- 172.21.15.97:0/2047524376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb778004060 0x7fb7941503c0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 >> 172.21.15.97:0/2047524376 conn(0x7fb7940b37c0 msgr2=0x7fb7940af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 shutdown_connections 2024-09-10T21:34:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:37.950+0000 7fb799a00640 1 -- 172.21.15.97:0/2047524376 wait complete. 2024-09-10T21:34:37.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 295, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:34:37.174387+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 130, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10047}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10047': {'gid': 10047, 'name': 'a', 'rank': 0, 'incarnation': 295, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6834/3083575804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 3083575804}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 3083575804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:34:37.967 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10047:up:replay 2024-09-10T21:34:37.967 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:34:39.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:39.804 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:41.167 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:41.166+0000 7fc7b3200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:43.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:43.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 -- 172.21.15.97:0/3309534629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40b1560 msgr2=0x7fd6b40b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 --2- 172.21.15.97:0/3309534629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40b1560 0x7fd6b40b1990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009bc0 tx=0x7fd6ac01f3b0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 -- 172.21.15.97:0/3309534629 shutdown_connections 2024-09-10T21:34:43.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 --2- 172.21.15.97:0/3309534629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40b1560 0x7fd6b40b1990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 --2- 172.21.15.97:0/3309534629 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6b40ce830 0x7fd6b40cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 -- 172.21.15.97:0/3309534629 >> 172.21.15.97:0/3309534629 conn(0x7fd6b40b37c0 msgr2=0x7fd6b40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 -- 172.21.15.97:0/3309534629 shutdown_connections 2024-09-10T21:34:43.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 -- 172.21.15.97:0/3309534629 wait complete. 2024-09-10T21:34:43.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 Processor -- start 2024-09-10T21:34:43.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.526+0000 7fd6ba800640 1 -- start start 2024-09-10T21:34:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b40b1560 0x7fd6b4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6b40ce830 0x7fd6b40c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40cec10 0x7fd6b415e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b40ce1a0 con 0x7fd6b40b1560 2024-09-10T21:34:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b40cde60 con 0x7fd6b40cec10 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b40ce320 con 0x7fd6b40ce830 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b40b1560 0x7fd6b4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b40b1560 0x7fd6b4155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60194/0 (socket says 172.21.15.97:60194) 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 -- 172.21.15.97:0/2898042540 learned_addr learned my addr 172.21.15.97:0/2898042540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b2a00640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6b40ce830 0x7fd6b40c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3e00640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40cec10 0x7fd6b415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 -- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6b40ce830 msgr2=0x7fd6b40c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6b40ce830 0x7fd6b40c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 -- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40cec10 msgr2=0x7fd6b415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40cec10 0x7fd6b415e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 -- 172.21.15.97:0/2898042540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b41f5dd0 con 0x7fd6b40b1560 2024-09-10T21:34:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3e00640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40cec10 0x7fd6b415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6b3400640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b40b1560 0x7fd6b4155d40 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400ea10 tx=0x7fd6a400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ab400640 1 -- 172.21.15.97:0/2898042540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400cde0 con 0x7fd6b40b1560 2024-09-10T21:34:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ab400640 1 -- 172.21.15.97:0/2898042540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a4004020 con 0x7fd6b40b1560 2024-09-10T21:34:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b41f6070 con 0x7fd6b40b1560 2024-09-10T21:34:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ab400640 1 -- 172.21.15.97:0/2898042540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004470 con 0x7fd6b40b1560 2024-09-10T21:34:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b41f6630 con 0x7fd6b40b1560 2024-09-10T21:34:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b40cf800 con 0x7fd6b40b1560 2024-09-10T21:34:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.530+0000 7fd6ab400640 1 -- 172.21.15.97:0/2898042540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd6a4003600 con 0x7fd6b40b1560 2024-09-10T21:34:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.534+0000 7fd6ab400640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6940725e0 0x7fd694074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.534+0000 7fd6ab400640 1 -- 172.21.15.97:0/2898042540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7fd6a4094980 con 0x7fd6b40b1560 2024-09-10T21:34:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.534+0000 7fd6b2a00640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6940725e0 0x7fd694074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.534+0000 7fd6b2a00640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6940725e0 0x7fd694074aa0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd6b41e7bd0 tx=0x7fd6a0001380 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.538+0000 7fd6ab400640 1 -- 172.21.15.97:0/2898042540 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6a405f2a0 con 0x7fd6b40b1560 2024-09-10T21:34:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6b40b2a50 con 0x7fd6b40b1560 2024-09-10T21:34:43.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ab400640 1 -- 172.21.15.97:0/2898042540 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 298 v298) ==== 95+0+5242 (secure 0 0 0) 0x7fd6a405eba0 con 0x7fd6b40b1560 2024-09-10T21:34:43.737 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:43.737 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":298,"btime":"2024-09-10T21:34:40:186199+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":298,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:40.186192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10047},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10047":{"gid":10047,"name":"a","rank":0,"incarnation":295,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6834/3083575804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3083575804},{"type":"v1","addr":"172.21.15.97:6834","nonce":3083575804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10047,"qdb_cluster":[10047]},"id":6}]} 2024-09-10T21:34:43.737 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 298 2024-09-10T21:34:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6940725e0 msgr2=0x7fd694074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6940725e0 0x7fd694074aa0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd6b41e7bd0 tx=0x7fd6a0001380 comp rx=0 tx=0).stop 2024-09-10T21:34:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b40b1560 msgr2=0x7fd6b4155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b40b1560 0x7fd6b4155d40 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400ea10 tx=0x7fd6a400eed0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 shutdown_connections 2024-09-10T21:34:43.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6940725e0 0x7fd694074aa0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6b40cec10 0x7fd6b415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6b40ce830 0x7fd6b40c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 --2- 172.21.15.97:0/2898042540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b40b1560 0x7fd6b4155d40 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 >> 172.21.15.97:0/2898042540 conn(0x7fd6b40b37c0 msgr2=0x7fd6b40cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 shutdown_connections 2024-09-10T21:34:43.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.738+0000 7fd6ba800640 1 -- 172.21.15.97:0/2898042540 wait complete. 2024-09-10T21:34:43.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 298, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:34:40.186192+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 130, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10047}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10047': {'gid': 10047, 'name': 'a', 'rank': 0, 'incarnation': 295, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6834/3083575804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 3083575804}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 3083575804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10047, 'qdb_cluster': [10047]} 2024-09-10T21:34:43.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:34:43.760 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:34:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- 172.21.15.97:0/2760997117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72f8003c40 msgr2=0x7f72f8004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 --2- 172.21.15.97:0/2760997117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72f8003c40 0x7f72f8004040 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f7304001860 tx=0x7f730401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- 172.21.15.97:0/2760997117 shutdown_connections 2024-09-10T21:34:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 --2- 172.21.15.97:0/2760997117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72f8003c40 0x7f72f8004040 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- 172.21.15.97:0/2760997117 >> 172.21.15.97:0/2760997117 conn(0x7f73100d1c30 msgr2=0x7f73100d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- 172.21.15.97:0/2760997117 shutdown_connections 2024-09-10T21:34:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- 172.21.15.97:0/2760997117 wait complete. 2024-09-10T21:34:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 Processor -- start 2024-09-10T21:34:43.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- start start 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72f8003c40 0x7f73101f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73101f8380 0x7f73100cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73100cb270 0x7f73101faef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73100cce10 con 0x7f73101f8380 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f731015c1d0 con 0x7f73100cb270 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7318800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f73100ccf90 con 0x7f72f8003c40 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7315400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73101f8380 0x7f73100cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7315400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73101f8380 0x7f73100cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60224/0 (socket says 172.21.15.97:60224) 2024-09-10T21:34:43.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7315e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72f8003c40 0x7f73101f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7315400640 1 -- 172.21.15.97:0/1367326295 learned_addr learned my addr 172.21.15.97:0/1367326295 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:43.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.906+0000 7f7316800640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73100cb270 0x7f73101faef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7315400640 1 -- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72f8003c40 msgr2=0x7f73101f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7315400640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72f8003c40 0x7f73101f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7315400640 1 -- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73100cb270 msgr2=0x7f73101faef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7315400640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73100cb270 0x7f73101faef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7315400640 1 -- 172.21.15.97:0/1367326295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73100cb720 con 0x7f73101f8380 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7315400640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73101f8380 0x7f73100cad30 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f7300006660 tx=0x7f73000039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f730a000640 1 -- 172.21.15.97:0/1367326295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300015070 con 0x7f73101f8380 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f730a000640 1 -- 172.21.15.97:0/1367326295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7300011900 con 0x7f73101f8380 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f730a000640 1 -- 172.21.15.97:0/1367326295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300011d50 con 0x7f73101f8380 2024-09-10T21:34:43.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73100c8fd0 con 0x7f73101f8380 2024-09-10T21:34:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73100c9290 con 0x7f73101f8380 2024-09-10T21:34:43.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f730a000640 1 -- 172.21.15.97:0/1367326295 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7300010420 con 0x7f73101f8380 2024-09-10T21:34:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f730a000640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72f40725e0 0x7f72f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f730a000640 1 -- 172.21.15.97:0/1367326295 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f7300094de0 con 0x7f73101f8380 2024-09-10T21:34:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.910+0000 7f731bcad000 1 Processor -- start 2024-09-10T21:34:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- start start 2024-09-10T21:34:43.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315e00640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72f40725e0 0x7f72f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315e00640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72f40725e0 0x7f72f4074aa0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7304001830 tx=0x7f7304019700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc19d0 0x55b6eccc1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc22f0 0x55b6ecc758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecc761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecb963f0 con 0x55b6eccc22f0 2024-09-10T21:34:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecad3b20 con 0x55b6ecc75e10 2024-09-10T21:34:43.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b6ecac2f50 con 0x55b6eccc19d0 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc22f0 0x55b6ecc758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc22f0 0x55b6ecc758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60226/0 (socket says 172.21.15.97:60226) 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 -- 172.21.15.97:0/1366808785 learned_addr learned my addr 172.21.15.97:0/1366808785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 -- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc19d0 msgr2=0x55b6eccc1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315e00640 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc19d0 0x55b6eccc1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7316800640 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecc761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc19d0 0x55b6eccc1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 -- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 msgr2=0x55b6ecc761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecc761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 -- 172.21.15.97:0/1366808785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6eca8a590 con 0x55b6eccc22f0 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315e00640 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc19d0 0x55b6eccc1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7316800640 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecc761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:43.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f7315400640 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc22f0 0x55b6ecc758d0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f730005efb0 tx=0x7f730005eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63395ff38ab104e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f72fea00640 1 -- 172.21.15.97:0/1366808785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300020040 con 0x55b6eccc22f0 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f72fea00640 1 -- 172.21.15.97:0/1366808785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7300063a50 con 0x55b6eccc22f0 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f72fea00640 1 -- 172.21.15.97:0/1366808785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300064e20 con 0x55b6eccc22f0 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- 172.21.15.97:0/1366808785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc22f0 msgr2=0x55b6ecc758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc22f0 0x55b6ecc758d0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f730005efb0 tx=0x7f730005eb30 comp rx=0 tx=0).stop 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- 172.21.15.97:0/1366808785 shutdown_connections 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecc761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc22f0 0x55b6ecc758d0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 --2- 172.21.15.97:0/1366808785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc19d0 0x55b6eccc1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- 172.21.15.97:0/1366808785 >> 172.21.15.97:0/1366808785 conn(0x55b6ecb5f130 msgr2=0x55b6ecca4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- 172.21.15.97:0/1366808785 shutdown_connections 2024-09-10T21:34:43.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- 172.21.15.97:0/1366808785 wait complete. 2024-09-10T21:34:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 -- start start 2024-09-10T21:34:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.914+0000 7f731bcad000 1 Processor -- start 2024-09-10T21:34:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecca26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecca2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecca33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b6eca5d5d0 con 0x55b6ecc75e10 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecac6d60 con 0x55b6eccc19d0 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b6eca69500 con 0x55b6eccc22f0 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 10 client.0 _ll_register_callbacks cb 0x55b6eca6d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecca26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecca26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60232/0 (socket says 172.21.15.97:60232) 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 -- 172.21.15.97:0/1793782289 learned_addr learned my addr 172.21.15.97:0/1793782289 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7316800640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecca33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315400640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecca2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 -- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 msgr2=0x55b6ecca33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecca33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 -- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 msgr2=0x55b6ecca2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecca2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 -- 172.21.15.97:0/1793782289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6eca61170 con 0x55b6ecc75e10 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7316800640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecca33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315e00640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecca26a0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f7304001860 tx=0x7f7304006b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47aa2d44cc949c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 1 -- 172.21.15.97:0/1793782289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73040ca060 con 0x55b6ecc75e10 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 1 -- 172.21.15.97:0/1793782289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f73040b6030 con 0x55b6ecc75e10 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6eca5b0f0 con 0x55b6ecc75e10 2024-09-10T21:34:43.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 1 -- 172.21.15.97:0/1793782289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=298) ==== 24+0+0 (secure 0 0 0) 0x7f73040d1050 con 0x55b6ecc75e10 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 10 client.10107 fetch_fsmap learned FSMap version 298 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=298,monmap=2+}) -- 0x55b6ecb47ef0 con 0x55b6ecc75e10 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 1 -- 172.21.15.97:0/1793782289 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b6eca5b0f0 con 0x55b6ecc75e10 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 1 -- 172.21.15.97:0/1793782289 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 298) ==== 3502+0+0 (secure 0 0 0) 0x7f73040a6c50 con 0x55b6ecc75e10 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 10 client.10107 fetch_fsmap finished waiting for FSMap version 298 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 10 client.10107 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 20 client.10107 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6ecca40f0 0x55b6eccd3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 4 client.10107 mds_command: new command op to 10047 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 --> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ecad5a70 con 0x55b6ecca40f0 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315400640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6ecca40f0 0x55b6eccd3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 10 client.10107 ms_handle_connect on v2:172.21.15.97:6832/3083575804 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f7315400640 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6ecca40f0 0x55b6eccd3c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0851747017903a server_cookie=d2a01337cc1e9eba in_seq=0 out_seq=0 2024-09-10T21:34:43.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 1 -- 172.21.15.97:0/1793782289 <== mds.0 v2:172.21.15.97:6832/3083575804 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6ecad5a70 con 0x55b6ecca40f0 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f72f3400640 10 client.10107 handle_command_reply: tid=0 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.918+0000 7f731bcad000 1 client.10107 shutdown 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 msgr2=0x55b6ecca26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecca26a0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f7304001860 tx=0x7f7304006b20 comp rx=0 tx=0).stop 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 shutdown_connections 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6ecca40f0 0x55b6eccd3c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecca33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecca2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 --2- 172.21.15.97:0/1793782289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecca26a0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f72f3400640 0 client.10107 ms_handle_reset on v2:172.21.15.97:6832/3083575804 2024-09-10T21:34:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 >> 172.21.15.97:0/1793782289 conn(0x55b6ecb5f130 msgr2=0x55b6ecc74fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 shutdown_connections 2024-09-10T21:34:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 1 -- 172.21.15.97:0/1793782289 wait complete. 2024-09-10T21:34:43.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.922+0000 7f731bcad000 20 client.10107 trim_cache size 0 max 16384 2024-09-10T21:34:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 Processor -- start 2024-09-10T21:34:43.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 -- start start 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecd33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc19d0 0x55b6ecd34520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd34a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7310206ca0 con 0x55b6eccc19d0 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7310205c80 con 0x55b6ecc75e10 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7310204fa0 con 0x55b6eccc22f0 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f7315400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc19d0 0x55b6ecd34520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f7315400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc19d0 0x55b6ecd34520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60242/0 (socket says 172.21.15.97:60242) 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.938+0000 7f7315400640 1 -- 172.21.15.97:0/2396442852 learned_addr learned my addr 172.21.15.97:0/2396442852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7316800640 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd34a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecd33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315400640 1 -- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 msgr2=0x55b6ecd34a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315400640 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd34a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315400640 1 -- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 msgr2=0x55b6ecd33fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315400640 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecd33fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315400640 1 -- 172.21.15.97:0/2396442852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ecb1ed90 con 0x55b6eccc19d0 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315400640 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc19d0 0x55b6ecd34520 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f730005c290 tx=0x7f730004a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f45705772a456b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f72fe000640 1 -- 172.21.15.97:0/2396442852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300020040 con 0x55b6eccc19d0 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f72fe000640 1 -- 172.21.15.97:0/2396442852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7300064be0 con 0x55b6eccc19d0 2024-09-10T21:34:43.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f72fe000640 1 -- 172.21.15.97:0/2396442852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730005aec0 con 0x55b6eccc19d0 2024-09-10T21:34:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- 172.21.15.97:0/2396442852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc19d0 msgr2=0x55b6ecd34520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc19d0 0x55b6ecd34520 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f730005c290 tx=0x7f730004a040 comp rx=0 tx=0).stop 2024-09-10T21:34:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- 172.21.15.97:0/2396442852 shutdown_connections 2024-09-10T21:34:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd34a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6eccc19d0 0x55b6ecd34520 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 --2- 172.21.15.97:0/2396442852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6ecc75e10 0x55b6ecd33fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- 172.21.15.97:0/2396442852 >> 172.21.15.97:0/2396442852 conn(0x55b6ecb5f130 msgr2=0x55b6ecc7d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- 172.21.15.97:0/2396442852 shutdown_connections 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- 172.21.15.97:0/2396442852 wait complete. 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- start start 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 Processor -- start 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecd38020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecd38560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd38aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b6eccd8ce0 con 0x55b6ecc75e10 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b6ecacb390 con 0x55b6eccc19d0 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b6ecb47a60 con 0x55b6eccc22f0 2024-09-10T21:34:43.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f731bcad000 10 client.0 _ll_register_callbacks cb 0x7f72f800c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:34:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecd38020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecd38020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60254/0 (socket says 172.21.15.97:60254) 2024-09-10T21:34:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 -- 172.21.15.97:0/2234732401 learned_addr learned my addr 172.21.15.97:0/2234732401 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7316800640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd38aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315400640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecd38560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 -- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 msgr2=0x55b6ecd38aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd38aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 -- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 msgr2=0x55b6ecd38560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecd38560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 -- 172.21.15.97:0/2234732401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6eca89a90 con 0x55b6ecc75e10 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7316800640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd38aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f7315e00640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecd38020 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f730401afc0 tx=0x7f7304003600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a11f3f0b907403f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.942+0000 7f72f3400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 1 -- 172.21.15.97:0/2234732401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73040ca070 con 0x55b6ecc75e10 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 1 -- 172.21.15.97:0/2234732401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f73040c2040 con 0x55b6ecc75e10 2024-09-10T21:34:43.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7300064be0 con 0x55b6ecc75e10 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 1 -- 172.21.15.97:0/2234732401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=298) ==== 24+0+0 (secure 0 0 0) 0x7f73040dc050 con 0x55b6ecc75e10 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 10 client.10113 fetch_fsmap learned FSMap version 298 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=298,monmap=2+}) -- 0x55b6eca68050 con 0x55b6ecc75e10 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 1 -- 172.21.15.97:0/2234732401 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7300064be0 con 0x55b6ecc75e10 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 1 -- 172.21.15.97:0/2234732401 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 298) ==== 3502+0+0 (secure 0 0 0) 0x7f7304003770 con 0x55b6ecc75e10 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 10 client.10113 fetch_fsmap finished waiting for FSMap version 298 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 10 client.10113 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 20 client.10113 populate_metadata read hostname 'smithi097' 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6eccd6bf0 0x55b6eccd5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 4 client.10113 mds_command: new command op to 10047 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 --> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] -- command(tid 0: {"prefix": "status"}) -- 0x55b6ecad1520 con 0x55b6eccd6bf0 2024-09-10T21:34:43.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7315400640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6eccd6bf0 0x55b6eccd5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 10 client.10113 ms_handle_connect on v2:172.21.15.97:6832/3083575804 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7315400640 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6eccd6bf0 0x55b6eccd5d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180d83125dfab52c server_cookie=565e9668de50948b in_seq=0 out_seq=0 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 1 -- 172.21.15.97:0/2234732401 <== mds.0 v2:172.21.15.97:6832/3083575804 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55b6ecad1520 con 0x55b6eccd6bf0 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 10 client.10113 handle_command_reply: tid=0 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 client.10113 shutdown 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 msgr2=0x55b6ecd38020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecd38020 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f730401afc0 tx=0x7f7304003600 comp rx=0 tx=0).stop 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 shutdown_connections 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:6832/3083575804,v1:172.21.15.97:6834/3083575804] conn(0x55b6eccd6bf0 0x55b6eccd5d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b6eccc22f0 0x55b6ecd38aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6eccc19d0 0x55b6ecd38560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 --2- 172.21.15.97:0/2234732401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b6ecc75e10 0x55b6ecd38020 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f72f3400640 0 client.10113 ms_handle_reset on v2:172.21.15.97:6832/3083575804 2024-09-10T21:34:43.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 >> 172.21.15.97:0/2234732401 conn(0x55b6ecb5f130 msgr2=0x55b6ecc7db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 shutdown_connections 2024-09-10T21:34:43.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 1 -- 172.21.15.97:0/2234732401 wait complete. 2024-09-10T21:34:43.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f731bcad000 20 client.10113 trim_cache size 0 max 16384 2024-09-10T21:34:43.952 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:34:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:34:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 10047, 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7707847990000003, 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 298, 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 130, 2024-09-10T21:34:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 130, 2024-09-10T21:34:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 8.7362975840000008 2024-09-10T21:34:43.954 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:34:43.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72f40725e0 msgr2=0x7f72f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72f40725e0 0x7f72f4074aa0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7304001830 tx=0x7f7304019700 comp rx=0 tx=0).stop 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73101f8380 msgr2=0x7f73100cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73101f8380 0x7f73100cad30 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f7300006660 tx=0x7f73000039d0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 shutdown_connections 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f72f40725e0 0x7f72f4074aa0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73100cb270 0x7f73101faef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73101f8380 0x7f73100cad30 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 --2- 172.21.15.97:0/1367326295 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f72f8003c40 0x7f73101f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 >> 172.21.15.97:0/1367326295 conn(0x7f73100d1c30 msgr2=0x7f731015d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 shutdown_connections 2024-09-10T21:34:43.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:43.946+0000 7f7318800640 1 -- 172.21.15.97:0/1367326295 wait complete. 2024-09-10T21:34:43.965 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:34:43.965 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2024-09-10T21:34:43.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:34:44.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 -- 172.21.15.97:0/4210757382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 msgr2=0x7f0be015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 --2- 172.21.15.97:0/4210757382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be015be20 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f0be00753d0 tx=0x7f0bcc01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 -- 172.21.15.97:0/4210757382 shutdown_connections 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 --2- 172.21.15.97:0/4210757382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be015be20 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 --2- 172.21.15.97:0/4210757382 >> v2:172.21.15.194:3300/0 conn(0x7f0be015ba40 0x7f0be00cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 -- 172.21.15.97:0/4210757382 >> 172.21.15.97:0/4210757382 conn(0x7f0be00cd3d0 msgr2=0x7f0be00cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 -- 172.21.15.97:0/4210757382 shutdown_connections 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.106+0000 7f0be5e00640 1 -- 172.21.15.97:0/4210757382 wait complete. 2024-09-10T21:34:44.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 Processor -- start 2024-09-10T21:34:44.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 -- start start 2024-09-10T21:34:44.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be00ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0be015ba40 0x7f0be014f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0be015c360 0x7f0be00c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0be006ced0 con 0x7f0be00cce60 2024-09-10T21:34:44.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0be006cd50 con 0x7f0be015c360 2024-09-10T21:34:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0be006d050 con 0x7f0be015ba40 2024-09-10T21:34:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be00ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be00ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60266/0 (socket says 172.21.15.97:60266) 2024-09-10T21:34:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 -- 172.21.15.97:0/47217479 learned_addr learned my addr 172.21.15.97:0/47217479 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0bdfe00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0be015ba40 0x7f0be014f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5400640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0be015c360 0x7f0be00c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 -- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0be015ba40 msgr2=0x7f0be014f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0be015ba40 0x7f0be014f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 -- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0be015c360 msgr2=0x7f0be00c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0be015c360 0x7f0be00c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 -- 172.21.15.97:0/47217479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be00c6080 con 0x7f0be00cce60 2024-09-10T21:34:44.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0bdfe00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0be015ba40 0x7f0be014f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:44.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be4a00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be00ca5e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400ea20 tx=0x7f0bd400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:44.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0bdcc00640 1 -- 172.21.15.97:0/47217479 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd400cd80 con 0x7f0be00cce60 2024-09-10T21:34:44.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0bdcc00640 1 -- 172.21.15.97:0/47217479 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd4010430 con 0x7f0be00cce60 2024-09-10T21:34:44.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0bdcc00640 1 -- 172.21.15.97:0/47217479 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4010860 con 0x7f0be00cce60 2024-09-10T21:34:44.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.110+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be00c6360 con 0x7f0be00cce60 2024-09-10T21:34:44.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.114+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be0150230 con 0x7f0be00cce60 2024-09-10T21:34:44.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.114+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be00cfa00 con 0x7f0be00cce60 2024-09-10T21:34:44.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.114+0000 7f0bdcc00640 1 -- 172.21.15.97:0/47217479 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0bd4010a00 con 0x7f0be00cce60 2024-09-10T21:34:44.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.114+0000 7f0bdcc00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0bc0072610 0x7f0bc0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.114+0000 7f0bdcc00640 1 -- 172.21.15.97:0/47217479 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f0bd4094b50 con 0x7f0be00cce60 2024-09-10T21:34:44.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.114+0000 7f0bdfe00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0bc0072610 0x7f0bc0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.114+0000 7f0bdfe00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0bc0072610 0x7f0bc0074ad0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0b2000 tx=0x7f0bcc01d8f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:44.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.118+0000 7f0bdcc00640 1 -- 172.21.15.97:0/47217479 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0bd405f4d0 con 0x7f0be00cce60 2024-09-10T21:34:44.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.326+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0be015be20 con 0x7f0be00cce60 2024-09-10T21:34:44.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.326+0000 7f0bdcc00640 1 -- 172.21.15.97:0/47217479 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v298) ==== 53+0+73 (secure 0 0 0) 0x7f0bd405edd0 con 0x7f0be00cce60 2024-09-10T21:34:44.326 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0bc0072610 msgr2=0x7f0bc0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0bc0072610 0x7f0bc0074ad0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0b2000 tx=0x7f0bcc01d8f0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 msgr2=0x7f0be00ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be00ca5e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400ea20 tx=0x7f0bd400eee0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 shutdown_connections 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0bc0072610 0x7f0bc0074ad0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0be015c360 0x7f0be00c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0be015ba40 0x7f0be014f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 --2- 172.21.15.97:0/47217479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be00cce60 0x7f0be00ca5e0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 >> 172.21.15.97:0/47217479 conn(0x7f0be00cd3d0 msgr2=0x7f0be015f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:44.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 shutdown_connections 2024-09-10T21:34:44.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44.330+0000 7f0be5e00640 1 -- 172.21.15.97:0/47217479 wait complete. 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:34:44.348 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:44.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:44.349 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:34:44.357 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:34:44.358 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41809sec preferred_lft 41809sec 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.359 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:34:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:34:44.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:44.362 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:34:44.421 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:34:44.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:44.423 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:34:44.478 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:34:44.478 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:34:44.478 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:34:44.479 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:34:44.479 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:34:44.479 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:34:44.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:34:44.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:34:44.480 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:34:44.480 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:34:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:34:44.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:34:44.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:44.612 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:34:44.621 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:34:44.621 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:34:44.621 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:44.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:44.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:34:44.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:34:44.683 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:44.732 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:34:44.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:34:44.732 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:44.783 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:34:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:44.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:44.875 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:44.875 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:44.875 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:44.890 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:44.890 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:44.890 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:44.890 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:34:44.890 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:34:44.899 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:34:44.899 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:34:44.899 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:34:44.899 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:34:44.900 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:34:44.900 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:34:44.900 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:34:44.900 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:34:44.903 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:34:44.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:44.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:34:44.923 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:34:45.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:34:45.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:34:45.026 INFO:teuthology.orchestra.run.smithi097.stdout:client10122 2024-09-10T21:34:45.027 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:45.027 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:45.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:45.075 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:34:45.121 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:34:45.129 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:45.130+0000 7fc7ae200640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:34:45.129 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:34:45.129 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:34:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:45.129 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:34:45.130 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2024-09-10T21:34:45.130 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2024-09-10T21:34:45.130 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:45.130 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:45.184 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:45.184 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:45.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:34:45.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 -- 172.21.15.97:0/824863044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad4141d20 msgr2=0x7f3ad415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 --2- 172.21.15.97:0/824863044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad4141d20 0x7f3ad415f6d0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4141580 tx=0x7f3ac001c670 comp rx=0 tx=0).stop 2024-09-10T21:34:45.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 -- 172.21.15.97:0/824863044 shutdown_connections 2024-09-10T21:34:45.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 --2- 172.21.15.97:0/824863044 >> v2:172.21.15.194:3300/0 conn(0x7f3ad415fc10 0x7f3ad413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 --2- 172.21.15.97:0/824863044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad4141d20 0x7f3ad415f6d0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 -- 172.21.15.97:0/824863044 >> 172.21.15.97:0/824863044 conn(0x7f3ad40b1780 msgr2=0x7f3ad40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:45.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 -- 172.21.15.97:0/824863044 shutdown_connections 2024-09-10T21:34:45.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.422+0000 7f3adbc00640 1 -- 172.21.15.97:0/824863044 wait complete. 2024-09-10T21:34:45.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 Processor -- start 2024-09-10T21:34:45.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 -- start start 2024-09-10T21:34:45.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad40ce5f0 0x7f3ad41f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ad4141d20 0x7f3ad4139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ad415fc10 0x7f3ad406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad4168a70 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad4168bf0 con 0x7f3ad4141d20 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad4168d70 con 0x7f3ad415fc10 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad40ce5f0 0x7f3ad41f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad40ce5f0 0x7f3ad41f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60292/0 (socket says 172.21.15.97:60292) 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 -- 172.21.15.97:0/1325395187 learned_addr learned my addr 172.21.15.97:0/1325395187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 -- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ad415fc10 msgr2=0x7f3ad406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ad415fc10 0x7f3ad406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 -- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ad4141d20 msgr2=0x7f3ad4139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad3e00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ad4141d20 0x7f3ad4139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ad4141d20 0x7f3ad4139bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 -- 172.21.15.97:0/1325395187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad413a130 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad3e00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ad4141d20 0x7f3ad4139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:45.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad9200640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad40ce5f0 0x7f3ad41f8020 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800c7b0 tx=0x7f3ac800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:45.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad0c00640 1 -- 172.21.15.97:0/1325395187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8017070 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad0c00640 1 -- 172.21.15.97:0/1325395187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ac8005380 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad41647e0 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3ad0c00640 1 -- 172.21.15.97:0/1325395187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800a8a0 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.426+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad4164da0 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.430+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad415f6d0 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.430+0000 7f3ad0c00640 1 -- 172.21.15.97:0/1325395187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ac800aa40 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.430+0000 7f3ad0c00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ab40725e0 0x7f3ab4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.430+0000 7f3ad0c00640 1 -- 172.21.15.97:0/1325395187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f3ac80938b0 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.430+0000 7f3ad3e00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ab40725e0 0x7f3ab4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.430+0000 7f3ad3e00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ab40725e0 0x7f3ab4074aa0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00af000 tx=0x7f3ac001aeb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:45.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.434+0000 7f3ad0c00640 1 -- 172.21.15.97:0/1325395187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ac805e140 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.610+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ad4141360 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.610+0000 7f3ad0c00640 1 -- 172.21.15.97:0/1325395187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 68 entries v130) ==== 81+0+4132 (secure 0 0 0) 0x7f3ac805da40 con 0x7f3ad40ce5f0 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1998443497 2024-09-11T21:34:35.158201+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1087733461 2024-09-11T21:34:19.533818+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1087733461 2024-09-11T21:34:19.533818+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2872418710 2024-09-11T21:34:14.052935+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:34:45.610 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2872418710 2024-09-11T21:34:14.052935+0000 2024-09-10T21:34:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1998443497 2024-09-11T21:34:35.158201+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:34:45.612 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:34:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:34:45.614 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:34:45.615 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:34:45.615 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:34:45.615 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:34:45.615 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:34:45.615 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:listed 68 entries 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ab40725e0 msgr2=0x7f3ab4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ab40725e0 0x7f3ab4074aa0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00af000 tx=0x7f3ac001aeb0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad40ce5f0 msgr2=0x7f3ad41f8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad40ce5f0 0x7f3ad41f8020 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800c7b0 tx=0x7f3ac800cc70 comp rx=0 tx=0).stop 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 shutdown_connections 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3ab40725e0 0x7f3ab4074aa0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ad415fc10 0x7f3ad406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ad4141d20 0x7f3ad4139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 --2- 172.21.15.97:0/1325395187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ad40ce5f0 0x7f3ad41f8020 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 >> 172.21.15.97:0/1325395187 conn(0x7f3ad40b1780 msgr2=0x7f3ad40731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:45.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 shutdown_connections 2024-09-10T21:34:45.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.614+0000 7f3adbc00640 1 -- 172.21.15.97:0/1325395187 wait complete. 2024-09-10T21:34:45.631 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:34:45.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:45.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:34:45.654 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:34:45.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:45.654 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:34:45.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:34:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- 172.21.15.97:0/1476846067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140ce830 msgr2=0x7f90140cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 --2- 172.21.15.97:0/1476846067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140ce830 0x7f90140cf800 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f9004009900 tx=0x7f900401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- 172.21.15.97:0/1476846067 shutdown_connections 2024-09-10T21:34:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 --2- 172.21.15.97:0/1476846067 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f90140b1560 0x7f90140b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 --2- 172.21.15.97:0/1476846067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140ce830 0x7f90140cf800 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- 172.21.15.97:0/1476846067 >> 172.21.15.97:0/1476846067 conn(0x7f90140b37c0 msgr2=0x7f90140b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- 172.21.15.97:0/1476846067 shutdown_connections 2024-09-10T21:34:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- 172.21.15.97:0/1476846067 wait complete. 2024-09-10T21:34:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 Processor -- start 2024-09-10T21:34:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- start start 2024-09-10T21:34:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140b1560 0x7f9014153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f90140ce830 0x7f90140b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90140cec10 0x7f90140b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90140ce320 con 0x7f90140b1560 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f90140ce1a0 con 0x7f90140cec10 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9019000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f90140d0450 con 0x7f90140ce830 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140b1560 0x7f9014153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140b1560 0x7f9014153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60316/0 (socket says 172.21.15.97:60316) 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011e00640 1 -- 172.21.15.97:0/2192981668 learned_addr learned my addr 172.21.15.97:0/2192981668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011400640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f90140ce830 0x7f90140b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9012800640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90140cec10 0x7f90140b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011e00640 1 -- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f90140ce830 msgr2=0x7f90140b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011e00640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f90140ce830 0x7f90140b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011e00640 1 -- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90140cec10 msgr2=0x7f90140b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.894+0000 7f9011e00640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90140cec10 0x7f90140b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f9011e00640 1 -- 172.21.15.97:0/2192981668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90140b7620 con 0x7f90140b1560 2024-09-10T21:34:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f9011e00640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140b1560 0x7f9014153160 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f9004009850 tx=0x7f90040a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f900a000640 1 -- 172.21.15.97:0/2192981668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90040be070 con 0x7f90140b1560 2024-09-10T21:34:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f900a000640 1 -- 172.21.15.97:0/2192981668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9004005400 con 0x7f90140b1560 2024-09-10T21:34:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f900a000640 1 -- 172.21.15.97:0/2192981668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90040b5040 con 0x7f90140b1560 2024-09-10T21:34:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90140b7060 con 0x7f90140b1560 2024-09-10T21:34:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90140c8da0 con 0x7f90140b1560 2024-09-10T21:34:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90140c8bd0 con 0x7f90140b1560 2024-09-10T21:34:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f900a000640 1 -- 172.21.15.97:0/2192981668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f90040b8030 con 0x7f90140b1560 2024-09-10T21:34:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f900a000640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ff40725d0 0x7f8ff4074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f900a000640 1 -- 172.21.15.97:0/2192981668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9199+0+0 (secure 0 0 0) 0x7f9004130c70 con 0x7f90140b1560 2024-09-10T21:34:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.898+0000 7f9011400640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ff40725d0 0x7f8ff4074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.902+0000 7f9011400640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ff40725d0 0x7f8ff4074a90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f90140cb380 tx=0x7f9000009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:45.902+0000 7f900a000640 1 -- 172.21.15.97:0/2192981668 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9004018c30 con 0x7f90140b1560 2024-09-10T21:34:46.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.098+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f90140b1970 con 0x7f90140b1560 2024-09-10T21:34:46.100 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:46.102+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:34:46.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.102+0000 7f900a000640 1 -- 172.21.15.97:0/2192981668 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v299) ==== 161+0+0 (secure 0 0 0) 0x7f90040faf40 con 0x7f90140b1560 2024-09-10T21:34:46.102 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:34:46.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ff40725d0 msgr2=0x7f8ff4074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ff40725d0 0x7f8ff4074a90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f90140cb380 tx=0x7f9000009040 comp rx=0 tx=0).stop 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140b1560 msgr2=0x7f9014153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140b1560 0x7f9014153160 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f9004009850 tx=0x7f90040a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 shutdown_connections 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8ff40725d0 0x7f8ff4074a90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90140cec10 0x7f90140b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f90140ce830 0x7f90140b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 --2- 172.21.15.97:0/2192981668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90140b1560 0x7f9014153160 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 >> 172.21.15.97:0/2192981668 conn(0x7f90140b37c0 msgr2=0x7f90141562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 shutdown_connections 2024-09-10T21:34:46.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.106+0000 7f9019000640 1 -- 172.21.15.97:0/2192981668 wait complete. 2024-09-10T21:34:46.110 INFO:tasks.ceph.mds.a.smithi097.stderr: -672> 2024-09-10T21:34:41.166+0000 7fc7b3200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:46.111 INFO:tasks.ceph.mds.a.smithi097.stderr: -152> 2024-09-10T21:34:45.130+0000 7fc7ae200640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:34:46.128 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:34:46.152 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:34:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 -- 172.21.15.97:0/1656932516 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7240ce5f0 msgr2=0x7fe724141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 --2- 172.21.15.97:0/1656932516 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7240ce5f0 0x7fe724141d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe714009870 tx=0x7fe71401c860 comp rx=0 tx=0).stop 2024-09-10T21:34:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 -- 172.21.15.97:0/1656932516 shutdown_connections 2024-09-10T21:34:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 --2- 172.21.15.97:0/1656932516 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe7240ce5f0 0x7fe724141d20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 -- 172.21.15.97:0/1656932516 >> 172.21.15.97:0/1656932516 conn(0x7fe7240b1780 msgr2=0x7fe7240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 -- 172.21.15.97:0/1656932516 shutdown_connections 2024-09-10T21:34:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 -- 172.21.15.97:0/1656932516 wait complete. 2024-09-10T21:34:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 Processor -- start 2024-09-10T21:34:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.274+0000 7fe72a200640 1 -- start start 2024-09-10T21:34:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7240ce5f0 0x7fe724152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7240ce9d0 0x7fe72413b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe72415fde0 0x7fe72406f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe724169e10 con 0x7fe7240ce5f0 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe724169c90 con 0x7fe7240ce9d0 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe724169f90 con 0x7fe72415fde0 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7240ce5f0 0x7fe724152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7240ce5f0 0x7fe724152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60344/0 (socket says 172.21.15.97:60344) 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 -- 172.21.15.97:0/635544578 learned_addr learned my addr 172.21.15.97:0/635544578 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723e00640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe72415fde0 0x7fe72406f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe71be00640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7240ce9d0 0x7fe72413b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 -- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe72415fde0 msgr2=0x7fe72406f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe72415fde0 0x7fe72406f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 -- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7240ce9d0 msgr2=0x7fe72413b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7240ce9d0 0x7fe72413b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 -- 172.21.15.97:0/635544578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72406e590 con 0x7fe7240ce5f0 2024-09-10T21:34:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe723400640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7240ce5f0 0x7fe724152ca0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe714019470 tx=0x7fe714005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe71b400640 1 -- 172.21.15.97:0/635544578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140b2040 con 0x7fe7240ce5f0 2024-09-10T21:34:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe71b400640 1 -- 172.21.15.97:0/635544578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe714004040 con 0x7fe7240ce5f0 2024-09-10T21:34:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72406e870 con 0x7fe7240ce5f0 2024-09-10T21:34:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe71b400640 1 -- 172.21.15.97:0/635544578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714004430 con 0x7fe7240ce5f0 2024-09-10T21:34:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72406ee30 con 0x7fe7240ce5f0 2024-09-10T21:34:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe724141d20 con 0x7fe7240ce5f0 2024-09-10T21:34:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.278+0000 7fe71b400640 1 -- 172.21.15.97:0/635544578 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe7140b5070 con 0x7fe7240ce5f0 2024-09-10T21:34:46.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.282+0000 7fe71b400640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7000725e0 0x7fe700074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:46.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.282+0000 7fe71b400640 1 -- 172.21.15.97:0/635544578 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9285+0+0 (secure 0 0 0) 0x7fe71412d8e0 con 0x7fe7240ce5f0 2024-09-10T21:34:46.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.282+0000 7fe71be00640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7000725e0 0x7fe700074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:46.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.282+0000 7fe71be00640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7000725e0 0x7fe700074aa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe7241568a0 tx=0x7fe70c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:46.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.286+0000 7fe71b400640 1 -- 172.21.15.97:0/635544578 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe7140c4060 con 0x7fe7240ce5f0 2024-09-10T21:34:46.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:46.514+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe724141360 con 0x7fe7240ce5f0 2024-09-10T21:34:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe71b400640 1 -- 172.21.15.97:0/635544578 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v300) ==== 107+0+0 (secure 0 0 0) 0x7fe714016450 con 0x7fe7240ce5f0 2024-09-10T21:34:47.109 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:34:47.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7000725e0 msgr2=0x7fe700074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7000725e0 0x7fe700074aa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe7241568a0 tx=0x7fe70c009040 comp rx=0 tx=0).stop 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7240ce5f0 msgr2=0x7fe724152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7240ce5f0 0x7fe724152ca0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe714019470 tx=0x7fe714005760 comp rx=0 tx=0).stop 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 shutdown_connections 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe7000725e0 0x7fe700074aa0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe72415fde0 0x7fe72406f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7240ce9d0 0x7fe72413b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 --2- 172.21.15.97:0/635544578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7240ce5f0 0x7fe724152ca0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.110+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 >> 172.21.15.97:0/635544578 conn(0x7fe7240b1780 msgr2=0x7fe724073be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.114+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 shutdown_connections 2024-09-10T21:34:47.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.114+0000 7fe72a200640 1 -- 172.21.15.97:0/635544578 wait complete. 2024-09-10T21:34:47.132 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:34:47.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 Processor -- start 2024-09-10T21:34:47.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- start start 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 0x5653ab41be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36bec0 0x5653ab4176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36c2a0 0x5653ab3f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5653ab289a00 con 0x5653ab36bec0 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5653ab2e7f40 con 0x5653ab33d360 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5653ab3fa560 con 0x5653ab36c2a0 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36bec0 0x5653ab4176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36bec0 0x5653ab4176b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60362/0 (socket says 172.21.15.97:60362) 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 -- 172.21.15.97:0/3086967718 learned_addr learned my addr 172.21.15.97:0/3086967718 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb490600640 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 0x5653ab41be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb491000640 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36c2a0 0x5653ab3f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 -- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36c2a0 msgr2=0x5653ab3f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36c2a0 0x5653ab3f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 -- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 msgr2=0x5653ab41be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 0x5653ab41be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 -- 172.21.15.97:0/3086967718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ab262c10 con 0x5653ab36bec0 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb491000640 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36c2a0 0x5653ab3f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36bec0 0x5653ab4176b0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fb47c004f60 tx=0x7fb47c018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11afecb822fce8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48e800640 1 -- 172.21.15.97:0/3086967718 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0b0040 con 0x5653ab36bec0 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48e800640 1 -- 172.21.15.97:0/3086967718 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb47c0acb20 con 0x5653ab36bec0 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb48e800640 1 -- 172.21.15.97:0/3086967718 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0b5410 con 0x5653ab36bec0 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- 172.21.15.97:0/3086967718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36bec0 msgr2=0x5653ab4176b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36bec0 0x5653ab4176b0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fb47c004f60 tx=0x7fb47c018040 comp rx=0 tx=0).stop 2024-09-10T21:34:47.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- 172.21.15.97:0/3086967718 shutdown_connections 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36c2a0 0x5653ab3f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36bec0 0x5653ab4176b0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3086967718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 0x5653ab41be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- 172.21.15.97:0/3086967718 >> 172.21.15.97:0/3086967718 conn(0x5653ab3830d0 msgr2=0x5653ab41b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- 172.21.15.97:0/3086967718 shutdown_connections 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- 172.21.15.97:0/3086967718 wait complete. 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 Processor -- start 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.222+0000 7fb493e4b080 1 -- start start 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 0x5653ab3f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36bec0 0x5653ab419d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36c2a0 0x5653ab41a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5653ab3f6ca0 con 0x5653ab36c2a0 2024-09-10T21:34:47.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5653ab3f6a00 con 0x5653ab33d360 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5653ab3f6f40 con 0x5653ab36bec0 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36c2a0 0x5653ab41a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36c2a0 0x5653ab41a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60372/0 (socket says 172.21.15.97:60372) 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 learned_addr learned my addr 172.21.15.97:0/3945508037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb490600640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 0x5653ab3f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36bec0 0x5653ab419d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36bec0 msgr2=0x5653ab419d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5653ab36bec0 0x5653ab419d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 msgr2=0x5653ab3f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653ab33d360 0x5653ab3f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ab41a7f0 con 0x5653ab36c2a0 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb491000640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36c2a0 0x5653ab41a2b0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fb4880042a0 tx=0x7fb48800c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb48d400640 1 -- 172.21.15.97:0/3945508037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488017070 con 0x5653ab36c2a0 2024-09-10T21:34:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb48d400640 1 -- 172.21.15.97:0/3945508037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb488001510 con 0x5653ab36c2a0 2024-09-10T21:34:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb48d400640 1 -- 172.21.15.97:0/3945508037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4880018e0 con 0x5653ab36c2a0 2024-09-10T21:34:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653ab41aa80 con 0x5653ab36c2a0 2024-09-10T21:34:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.226+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ab4a3900 con 0x5653ab36c2a0 2024-09-10T21:34:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb48d400640 1 -- 172.21.15.97:0/3945508037 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb488001a80 con 0x5653ab36c2a0 2024-09-10T21:34:47.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb48d400640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb470070fc0 0x7fb470073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb48d400640 1 -- 172.21.15.97:0/3945508037 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9285+0+0 (secure 0 0 0) 0x7fb488092940 con 0x5653ab36c2a0 2024-09-10T21:34:47.230 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:34:47.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb490600640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb470070fc0 0x7fb470073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5653ab3f41a0 0x5653ab3818d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4a5050 con 0x5653ab3f41a0 2024-09-10T21:34:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5653ab3f41a0 0x5653ab3818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb490600640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb470070fc0 0x7fb470073480 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c650 tx=0x7fb4800049f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.230+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5653ab3f41a0 0x5653ab3818d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.234+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v131'1065 uv1038 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.234+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab381e10 con 0x5653ab3f41a0 2024-09-10T21:34:47.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.234+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v131'1066 uv1066 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.234+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab3821e0 con 0x5653ab3f41a0 2024-09-10T21:34:47.236 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:34:47.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.238+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v131'1067 uv1067 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.238+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab3825d0 con 0x5653ab3f41a0 2024-09-10T21:34:47.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.238+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v131'1068 uv1068 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.238+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7e50 con 0x5653ab3f41a0 2024-09-10T21:34:47.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.242+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v131'1069 uv1069 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.242+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b72f0 con 0x5653ab3f41a0 2024-09-10T21:34:47.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.242+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v131'1070 uv1070 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.242+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.242+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v131'1071 uv1044 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.242+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.246+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v131'1072 uv1072 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.246+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.246 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:34:47.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.246+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v131'1073 uv1073 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.246+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.250+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v131'1074 uv1074 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.250+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.250+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v131'1075 uv1075 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.250+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.254+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v131'1076 uv1076 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.254+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.254+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v131'1077 uv1064 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.254+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.258+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v131'1078 uv1078 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.258+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.257 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:34:47.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.258+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v131'1079 uv1079 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.258+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.258+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v131'1080 uv1080 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.258+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.262+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v131'1081 uv1081 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.262+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.262+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v131'1082 uv1082 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.262+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.266+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v131'1083 uv1083 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.266+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.266+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v131'1084 uv1057 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.266+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.266+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v131'1085 uv1085 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.266+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.269 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:34:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.270+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v131'1086 uv1086 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.270+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.270+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v131'1087 uv1087 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.270+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.274+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v131'1088 uv1088 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.274+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.274+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v131'1089 uv1089 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.274+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.278+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v131'1090 uv1063 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.278+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.278+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v131'1091 uv1091 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.278+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.279 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:34:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.278+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v131'1092 uv1092 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.278+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.282+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v131'1093 uv1093 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.282+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.282+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v131'1094 uv1094 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.282+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4b7690 con 0x5653ab3f41a0 2024-09-10T21:34:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.286+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v131'1095 uv1095 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab3f41a0 2024-09-10T21:34:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.286+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5653ab4b8700 0x5653ab4baaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.286+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bb030 con 0x5653ab4b8700 2024-09-10T21:34:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.286+0000 7fb490600640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5653ab4b8700 0x5653ab4baaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.286+0000 7fb490600640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5653ab4b8700 0x5653ab4baaf0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.286+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v131'777 uv756 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.286+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4bb3d0 con 0x5653ab4b8700 2024-09-10T21:34:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.290+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v131'778 uv778 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.290+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bb770 con 0x5653ab4b8700 2024-09-10T21:34:47.290 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:34:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.290+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v131'779 uv779 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.290+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bbbb0 con 0x5653ab4b8700 2024-09-10T21:34:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.294+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v131'780 uv780 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.294+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bbfe0 con 0x5653ab4b8700 2024-09-10T21:34:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.294+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v131'781 uv781 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.294+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bc3d0 con 0x5653ab4b8700 2024-09-10T21:34:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.294+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v131'782 uv775 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.294+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4bc770 con 0x5653ab4b8700 2024-09-10T21:34:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.298+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v131'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.298+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.298+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v131'784 uv784 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.297 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:34:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.298+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.298+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v131'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.298+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.302+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v131'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.302+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.302+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v131'787 uv776 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.302+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.302+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v131'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.302+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.304 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:34:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.306+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v131'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.306+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c0ce0 con 0x5653ab4b8700 2024-09-10T21:34:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.306+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v131'790 uv790 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.306+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c0ce0 con 0x5653ab4b8700 2024-09-10T21:34:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.310+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v131'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.310+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.310+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v131'792 uv772 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.310+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.310+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v131'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.310+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.312 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:34:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.314+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v131'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.314+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bdb20 con 0x5653ab4b8700 2024-09-10T21:34:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.314+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v131'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.314+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bdb20 con 0x5653ab4b8700 2024-09-10T21:34:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.314+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v131'796 uv796 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.314+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bcb10 con 0x5653ab4b8700 2024-09-10T21:34:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.318+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v131'797 uv797 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4b8700 2024-09-10T21:34:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.318+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5653ab4bd190 0x5653ab4bd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.318+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bfef0 con 0x5653ab4bd190 2024-09-10T21:34:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.318+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5653ab4bd190 0x5653ab4bd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.318+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5653ab4bd190 0x5653ab4bd570 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.318+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.318+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4bfef0 con 0x5653ab4bd190 2024-09-10T21:34:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.322+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.322+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4bfef0 con 0x5653ab4bd190 2024-09-10T21:34:47.321 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:34:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.322+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.322+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c12f0 con 0x5653ab4bd190 2024-09-10T21:34:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.322+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.322+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c12f0 con 0x5653ab4bd190 2024-09-10T21:34:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c02e0 con 0x5653ab4bd190 2024-09-10T21:34:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5653ab4c07f0 0x5653ab4c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c32d0 con 0x5653ab4c07f0 2024-09-10T21:34:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb491000640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5653ab4c07f0 0x5653ab4c2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb491000640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5653ab4c07f0 0x5653ab4c2c90 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v131'157 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.326+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c37e0 con 0x5653ab4c07f0 2024-09-10T21:34:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.330+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v131'158 uv158 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.330+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c3ce0 con 0x5653ab4c07f0 2024-09-10T21:34:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.330+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v131'159 uv159 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.329 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:34:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.330+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c45a0 con 0x5653ab4c07f0 2024-09-10T21:34:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.330+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v131'160 uv160 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.330+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4940 con 0x5653ab4bd190 2024-09-10T21:34:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c4940 con 0x5653ab4bd190 2024-09-10T21:34:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4940 con 0x5653ab4bd190 2024-09-10T21:34:47.333 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:34:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5950 con 0x5653ab4bd190 2024-09-10T21:34:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.334+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5950 con 0x5653ab4bd190 2024-09-10T21:34:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.338+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.338+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4940 con 0x5653ab4bd190 2024-09-10T21:34:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.338+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.338+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4940 con 0x5653ab4c07f0 2024-09-10T21:34:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.338+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v131'382 uv376 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.338+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c4ce0 con 0x5653ab4c07f0 2024-09-10T21:34:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.342+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v131'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.342+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4c07f0 2024-09-10T21:34:47.341 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:34:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.342+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v131'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.342+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c6090 con 0x5653ab4c07f0 2024-09-10T21:34:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.342+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v131'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.342+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c6090 con 0x5653ab4c07f0 2024-09-10T21:34:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.346+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v131'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.346+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4c07f0 2024-09-10T21:34:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.346+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v131'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.346+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4c07f0 2024-09-10T21:34:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.346+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v131'388 uv381 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.346+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4c07f0 2024-09-10T21:34:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.350+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v131'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.350+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4c07f0 2024-09-10T21:34:47.349 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:34:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.350+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v131'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.350+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4c07f0 2024-09-10T21:34:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.350+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v131'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.350+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4c07f0 2024-09-10T21:34:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.354+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v131'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.354+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4bd190 2024-09-10T21:34:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.354+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v131'173 uv172 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.354+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4bd190 2024-09-10T21:34:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.354+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v131'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.354+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4bd190 2024-09-10T21:34:47.355 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:34:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v131'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4bd190 2024-09-10T21:34:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v131'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c5080 con 0x5653ab4bd190 2024-09-10T21:34:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v131'177 uv177 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4bd190 2024-09-10T21:34:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5653ab4c5700 0x5653ab4c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c80c0 con 0x5653ab4c5700 2024-09-10T21:34:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb490600640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5653ab4c5700 0x5653ab4c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.358+0000 7fb490600640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5653ab4c5700 0x5653ab4c7af0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.362+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.362+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c5700 2024-09-10T21:34:47.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.362+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.362+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c5700 2024-09-10T21:34:47.363 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:34:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.362+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.362+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c94a0 con 0x5653ab4c5700 2024-09-10T21:34:47.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.366+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.366+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c94a0 con 0x5653ab4c5700 2024-09-10T21:34:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.366+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.366+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c5700 2024-09-10T21:34:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.366+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.366+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c5700 2024-09-10T21:34:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.370+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v131'345 uv339 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.370+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c5700 2024-09-10T21:34:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.370+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v131'346 uv346 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.370+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c5700 2024-09-10T21:34:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.370+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v131'347 uv347 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.370 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:34:47.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.370+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4190 con 0x5653ab4c5700 2024-09-10T21:34:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.374+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v131'348 uv348 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.374+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4190 con 0x5653ab4c5700 2024-09-10T21:34:47.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.374+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v131'349 uv349 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.374+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4190 con 0x5653ab4c5700 2024-09-10T21:34:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.374+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v131'350 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.374+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c4190 con 0x5653ab4c5700 2024-09-10T21:34:47.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.378+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v131'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.378+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4190 con 0x5653ab4c5700 2024-09-10T21:34:47.377 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:34:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.378+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v131'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.378+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c4190 con 0x5653ab4c5700 2024-09-10T21:34:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.378+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v131'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.378+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4cb320 con 0x5653ab4c5700 2024-09-10T21:34:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.382+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v131'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.382+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c07f0 2024-09-10T21:34:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.382+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v131'152 uv150 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.382+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c07f0 2024-09-10T21:34:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.382+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v131'153 uv153 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.382+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c07f0 2024-09-10T21:34:47.383 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:34:47.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.386+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v131'154 uv154 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.386+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8490 con 0x5653ab4c07f0 2024-09-10T21:34:47.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.386+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v131'155 uv155 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.386+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8940 con 0x5653ab4c07f0 2024-09-10T21:34:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.386+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v131'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.386+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c8940 con 0x5653ab4c07f0 2024-09-10T21:34:47.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v131'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8940 con 0x5653ab4c07f0 2024-09-10T21:34:47.388 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:34:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v131'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9950 con 0x5653ab4c07f0 2024-09-10T21:34:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v131'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9950 con 0x5653ab4c07f0 2024-09-10T21:34:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v131'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.390+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c8940 con 0x5653ab4c07f0 2024-09-10T21:34:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v131'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb488019030 con 0x5653ab4c07f0 2024-09-10T21:34:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5653ab4c8e50 0x5653ab4c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c97b0 con 0x5653ab4c8e50 2024-09-10T21:34:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5653ab4c8e50 0x5653ab4c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb48fc00640 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5653ab4c8e50 0x5653ab4c9270 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v131'176 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4c8e50 2024-09-10T21:34:47.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c97b0 con 0x5653ab4c8e50 2024-09-10T21:34:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v131'177 uv177 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4c8e50 2024-09-10T21:34:47.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.394+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c97b0 con 0x5653ab4c8e50 2024-09-10T21:34:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:34:47.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.398+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v131'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4c8e50 2024-09-10T21:34:47.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.398+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c8e50 2024-09-10T21:34:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.398+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v131'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4c8e50 2024-09-10T21:34:47.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.398+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c8e50 2024-09-10T21:34:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.398+0000 7fb48fc00640 1 -- 172.21.15.97:0/3945508037 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v131'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb47c0b3040 con 0x5653ab4c8e50 2024-09-10T21:34:47.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.398+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.402+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v131'429 uv420 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.402+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.402+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v131'430 uv430 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.402+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.402+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v131'431 uv431 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.402 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:34:47.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.402+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v131'432 uv432 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v131'433 uv433 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v131'434 uv427 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v131'435 uv435 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.406+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.408 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:34:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.410+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v131'436 uv436 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.410+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.410+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v131'437 uv437 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.410+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.410+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v131'438 uv438 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.410+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4c9d50 con 0x5653ab4c5700 2024-09-10T21:34:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v131'439 uv439 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x5653ab4ca160 con 0x5653ab4c5700 2024-09-10T21:34:47.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb490600640 1 -- 172.21.15.97:0/3945508037 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v131'440 uv440 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb480008a20 con 0x5653ab4c5700 2024-09-10T21:34:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5653ab4bd190 msgr2=0x5653ab4bd570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5653ab4bd190 0x5653ab4bd570 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5653ab4c5700 msgr2=0x5653ab4c7af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5653ab4c5700 0x5653ab4c7af0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5653ab4c8e50 msgr2=0x5653ab4c9270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x5653ab4c8e50 0x5653ab4c9270 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5653ab4c07f0 msgr2=0x5653ab4c2c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5653ab4c07f0 0x5653ab4c2c90 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5653ab4b8700 msgr2=0x5653ab4baaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x5653ab4b8700 0x5653ab4baaf0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5653ab3f41a0 msgr2=0x5653ab3818d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5653ab3f41a0 0x5653ab3818d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 reap_dead start 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 reap_dead start 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 reap_dead start 2024-09-10T21:34:47.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb491000640 1 -- 172.21.15.97:0/3945508037 reap_dead start 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb470070fc0 msgr2=0x7fb470073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb470070fc0 0x7fb470073480 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c650 tx=0x7fb4800049f0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36c2a0 msgr2=0x5653ab41a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36c2a0 0x5653ab41a2b0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fb4880042a0 tx=0x7fb48800c5e0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 shutdown_connections 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb470070fc0 0x7fb470073480 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 --2- 172.21.15.97:0/3945508037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5653ab36c2a0 0x5653ab41a2b0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 >> 172.21.15.97:0/3945508037 conn(0x5653ab3830d0 msgr2=0x5653ab3f5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 shutdown_connections 2024-09-10T21:34:47.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.414+0000 7fb493e4b080 1 -- 172.21.15.97:0/3945508037 wait complete. 2024-09-10T21:34:47.419 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:34:47.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 Processor -- start 2024-09-10T21:34:47.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 -- start start 2024-09-10T21:34:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1350003b10 0x55b9c1923e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9c18997c0 0x55b9c187f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c189c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b9c1772f40 con 0x7f1350003b10 2024-09-10T21:34:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b9c177fd00 con 0x55b9c18997c0 2024-09-10T21:34:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136f249080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b9c1714a00 con 0x55b9c17c42a0 2024-09-10T21:34:47.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1350003b10 0x55b9c1923e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1350003b10 0x55b9c1923e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38684/0 (socket says 172.21.15.97:38684) 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 -- 172.21.15.97:0/3587896492 learned_addr learned my addr 172.21.15.97:0/3587896492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136c400640 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c189c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136b000640 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9c18997c0 0x55b9c187f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 -- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 msgr2=0x55b9c189c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c189c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 -- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9c18997c0 msgr2=0x55b9c187f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9c18997c0 0x55b9c187f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 -- 172.21.15.97:0/3587896492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c17f25c0 con 0x7f1350003b10 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f136ba00640 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1350003b10 0x55b9c1923e40 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f135c000fe0 tx=0x7f135c004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ae4991a807648f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f1369c00640 1 -- 172.21.15.97:0/3587896492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c010ba0 con 0x7f1350003b10 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f1369c00640 1 -- 172.21.15.97:0/3587896492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f135c010d40 con 0x7f1350003b10 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.506+0000 7f1369c00640 1 -- 172.21.15.97:0/3587896492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c004370 con 0x7f1350003b10 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- 172.21.15.97:0/3587896492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1350003b10 msgr2=0x55b9c1923e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1350003b10 0x55b9c1923e40 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f135c000fe0 tx=0x7f135c004bd0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- 172.21.15.97:0/3587896492 shutdown_connections 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c189c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9c18997c0 0x55b9c187f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 --2- 172.21.15.97:0/3587896492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1350003b10 0x55b9c1923e40 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- 172.21.15.97:0/3587896492 >> 172.21.15.97:0/3587896492 conn(0x55b9c17c5140 msgr2=0x55b9c17f5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- 172.21.15.97:0/3587896492 shutdown_connections 2024-09-10T21:34:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- 172.21.15.97:0/3587896492 wait complete. 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 Processor -- start 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- start start 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c1899090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9c18997c0 0x55b9c17f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1350003b10 0x55b9c17f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b9c189bb40 con 0x55b9c18997c0 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b9c189b400 con 0x7f1350003b10 2024-09-10T21:34:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55b9c18995d0 con 0x55b9c17c42a0 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9c18997c0 0x55b9c17f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9c18997c0 0x55b9c17f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38696/0 (socket says 172.21.15.97:38696) 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136b000640 1 -- 172.21.15.97:0/3330429753 learned_addr learned my addr 172.21.15.97:0/3330429753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136ba00640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c1899090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136c400640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1350003b10 0x55b9c17f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136ba00640 1 -- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1350003b10 msgr2=0x55b9c17f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136ba00640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1350003b10 0x55b9c17f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136ba00640 1 -- 172.21.15.97:0/3330429753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9c18997c0 msgr2=0x55b9c17f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136ba00640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9c18997c0 0x55b9c17f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136ba00640 1 -- 172.21.15.97:0/3330429753 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c1936220 con 0x55b9c17c42a0 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136b000640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b9c18997c0 0x55b9c17f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136ba00640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c1899090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f135c014000 tx=0x7f135c01b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f1363e00640 1 -- 172.21.15.97:0/3330429753 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c001e20 con 0x55b9c17c42a0 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f1363e00640 1 -- 172.21.15.97:0/3330429753 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f135c001440 con 0x55b9c17c42a0 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b9c19364c0 con 0x55b9c17c42a0 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f1363e00640 1 -- 172.21.15.97:0/3330429753 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c01be20 con 0x55b9c17c42a0 2024-09-10T21:34:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.510+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c1936a00 con 0x55b9c17c42a0 2024-09-10T21:34:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f1363e00640 1 -- 172.21.15.97:0/3330429753 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f135c004110 con 0x55b9c17c42a0 2024-09-10T21:34:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f1363e00640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f134c071190 0x7f134c073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f1363e00640 1 -- 172.21.15.97:0/3330429753 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(131..131 src has 1..131) ==== 9285+0+0 (secure 0 0 0) 0x7f135c097cf0 con 0x55b9c17c42a0 2024-09-10T21:34:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f136b000640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f134c071190 0x7f134c073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f136f249080 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b9c17ecd90 0x55b9c17ed170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55b9c17ed6b0 con 0x55b9c17ecd90 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f136c400640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b9c17ecd90 0x55b9c17ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.514+0000 7f136b000640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f134c071190 0x7f134c073650 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f135800dcf0 tx=0x7f1358003040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136c400640 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b9c17ecd90 0x55b9c17ed170 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136c400640 1 -- 172.21.15.97:0/3330429753 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v131'1096 uv1096 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1364002030 con 0x55b9c17ecd90 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b9c17ecd90 msgr2=0x55b9c17ed170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55b9c17ecd90 0x55b9c17ed170 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f134c071190 msgr2=0x7f134c073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f134c071190 0x7f134c073650 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f135800dcf0 tx=0x7f1358003040 comp rx=0 tx=0).stop 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 msgr2=0x55b9c1899090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 --2- 172.21.15.97:0/3330429753 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55b9c17c42a0 0x55b9c1899090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f135c014000 tx=0x7f135c01b9d0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136c400640 1 -- 172.21.15.97:0/3330429753 reap_dead start 2024-09-10T21:34:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 shutdown_connections 2024-09-10T21:34:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 >> 172.21.15.97:0/3330429753 conn(0x55b9c17c5140 msgr2=0x55b9c17f5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 shutdown_connections 2024-09-10T21:34:47.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.518+0000 7f136f249080 1 -- 172.21.15.97:0/3330429753 wait complete. 2024-09-10T21:34:47.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:34:47.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 -- 172.21.15.97:0/2896858620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c06fd50 msgr2=0x7f900c0701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 --2- 172.21.15.97:0/2896858620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c06fd50 0x7f900c0701e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9004009bc0 tx=0x7f900401cba0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 -- 172.21.15.97:0/2896858620 shutdown_connections 2024-09-10T21:34:47.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 --2- 172.21.15.97:0/2896858620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c06fd50 0x7f900c0701e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 --2- 172.21.15.97:0/2896858620 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f900c144690 0x7f900c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 -- 172.21.15.97:0/2896858620 >> 172.21.15.97:0/2896858620 conn(0x7f900c0b1780 msgr2=0x7f900c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 -- 172.21.15.97:0/2896858620 shutdown_connections 2024-09-10T21:34:47.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.710+0000 7f9010c00640 1 -- 172.21.15.97:0/2896858620 wait complete. 2024-09-10T21:34:47.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 Processor -- start 2024-09-10T21:34:47.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 -- start start 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f900c06fd50 0x7f900c141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f900c143f90 0x7f900c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c144690 0x7f900c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f900c167d50 con 0x7f900c06fd50 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f900c167bd0 con 0x7f900c144690 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f900c167ed0 con 0x7f900c143f90 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9009e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f900c06fd50 0x7f900c141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9001400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f900c143f90 0x7f900c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9009e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f900c06fd50 0x7f900c141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38698/0 (socket says 172.21.15.97:38698) 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9009e00640 1 -- 172.21.15.97:0/3498942517 learned_addr learned my addr 172.21.15.97:0/3498942517 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:47.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f900a800640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c144690 0x7f900c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9001400640 1 -- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c144690 msgr2=0x7f900c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9001400640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c144690 0x7f900c159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9001400640 1 -- 172.21.15.97:0/3498942517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f900c06fd50 msgr2=0x7f900c141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9001400640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f900c06fd50 0x7f900c141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9001400640 1 -- 172.21.15.97:0/3498942517 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c137700 con 0x7f900c143f90 2024-09-10T21:34:47.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9001400640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f900c143f90 0x7f900c15cc50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400e6c0 tx=0x7f8ff400eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9002a00640 1 -- 172.21.15.97:0/3498942517 <== mon.2 v2:172.21.15.194:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400cab0 con 0x7f900c143f90 2024-09-10T21:34:47.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9002a00640 1 -- 172.21.15.97:0/3498942517 <== mon.2 v2:172.21.15.194:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ff400cd40 con 0x7f900c143f90 2024-09-10T21:34:47.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c1379e0 con 0x7f900c143f90 2024-09-10T21:34:47.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9002a00640 1 -- 172.21.15.97:0/3498942517 <== mon.2 v2:172.21.15.194:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4010690 con 0x7f900c143f90 2024-09-10T21:34:47.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.714+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c137fa0 con 0x7f900c143f90 2024-09-10T21:34:47.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.718+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c0701e0 con 0x7f900c143f90 2024-09-10T21:34:47.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.718+0000 7f9002a00640 1 -- 172.21.15.97:0/3498942517 <== mon.2 v2:172.21.15.194:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ff403daa0 con 0x7f900c143f90 2024-09-10T21:34:47.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.718+0000 7f9002a00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8fe80725e0 0x7f8fe8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.718+0000 7f9002a00640 1 -- 172.21.15.97:0/3498942517 <== mon.2 v2:172.21.15.194:3301/0 5 ==== osd_map(131..131 src has 1..131) ==== 9285+0+0 (secure 0 0 0) 0x7f8ff40949a0 con 0x7f900c143f90 2024-09-10T21:34:47.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.718+0000 7f9009e00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8fe80725e0 0x7f8fe8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.718+0000 7f9009e00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8fe80725e0 0x7f8fe8074aa0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc001910 tx=0x7f8ffc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.722+0000 7f9002a00640 1 -- 172.21.15.97:0/3498942517 <== mon.2 v2:172.21.15.194:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ff405ec90 con 0x7f900c143f90 2024-09-10T21:34:47.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:47.922+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f900c143450 con 0x7f900c143f90 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9002a00640 1 -- 172.21.15.97:0/3498942517 <== mon.2 v2:172.21.15.194:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v301) ==== 162+0+0 (secure 0 0 0) 0x7f8ff40636c0 con 0x7f900c143f90 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8fe80725e0 msgr2=0x7f8fe8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8fe80725e0 0x7f8fe8074aa0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc001910 tx=0x7f8ffc009040 comp rx=0 tx=0).stop 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f900c143f90 msgr2=0x7f900c15cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f900c143f90 0x7f900c15cc50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400e6c0 tx=0x7f8ff400eb80 comp rx=0 tx=0).stop 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 shutdown_connections 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8fe80725e0 0x7f8fe8074aa0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f900c144690 0x7f900c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f900c143f90 0x7f900c15cc50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 --2- 172.21.15.97:0/3498942517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f900c06fd50 0x7f900c141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 >> 172.21.15.97:0/3498942517 conn(0x7f900c0b1780 msgr2=0x7f900c15e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 shutdown_connections 2024-09-10T21:34:48.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.310+0000 7f9010c00640 1 -- 172.21.15.97:0/3498942517 wait complete. 2024-09-10T21:34:48.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.474+0000 7f87dca00640 1 -- 172.21.15.97:0/2277073806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80ce640 msgr2=0x7f87d80cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.474+0000 7f87dca00640 1 --2- 172.21.15.97:0/2277073806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80ce640 0x7f87d80cea20 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009c80 tx=0x7f87c801cda0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.474+0000 7f87dca00640 1 -- 172.21.15.97:0/2277073806 shutdown_connections 2024-09-10T21:34:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.474+0000 7f87dca00640 1 --2- 172.21.15.97:0/2277073806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80ce640 0x7f87d80cea20 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.474+0000 7f87dca00640 1 -- 172.21.15.97:0/2277073806 >> 172.21.15.97:0/2277073806 conn(0x7f87d80d1c30 msgr2=0x7f87d80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.474+0000 7f87dca00640 1 -- 172.21.15.97:0/2277073806 shutdown_connections 2024-09-10T21:34:48.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.474+0000 7f87dca00640 1 -- 172.21.15.97:0/2277073806 wait complete. 2024-09-10T21:34:48.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 Processor -- start 2024-09-10T21:34:48.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 -- start start 2024-09-10T21:34:48.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b0d50 0x7f87d806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87d80cd6e0 0x7f87d80b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d80ce640 0x7f87d8150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87d816a8a0 con 0x7f87d80b0d50 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87d816a720 con 0x7f87d80ce640 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f87d816aa20 con 0x7f87d80cd6e0 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b0d50 0x7f87d806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b0d50 0x7f87d806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38710/0 (socket says 172.21.15.97:38710) 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 -- 172.21.15.97:0/667585347 learned_addr learned my addr 172.21.15.97:0/667585347 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5400640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87d80cd6e0 0x7f87d80b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d6800640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d80ce640 0x7f87d8150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 -- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87d80cd6e0 msgr2=0x7f87d80b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87d80cd6e0 0x7f87d80b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 -- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d80ce640 msgr2=0x7f87d8150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d80ce640 0x7f87d8150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 -- 172.21.15.97:0/667585347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d806de30 con 0x7f87d80b0d50 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d6800640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d80ce640 0x7f87d8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87d5e00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b0d50 0x7f87d806f6d0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f87c8000f90 tx=0x7f87c80a5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87ce000640 1 -- 172.21.15.97:0/667585347 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80bb070 con 0x7f87d80b0d50 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87ce000640 1 -- 172.21.15.97:0/667585347 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87c8004e00 con 0x7f87d80b0d50 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d806e110 con 0x7f87d80b0d50 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87ce000640 1 -- 172.21.15.97:0/667585347 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80b2040 con 0x7f87d80b0d50 2024-09-10T21:34:48.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d806e6d0 con 0x7f87d80b0d50 2024-09-10T21:34:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.478+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d80cea20 con 0x7f87d80b0d50 2024-09-10T21:34:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.482+0000 7f87ce000640 1 -- 172.21.15.97:0/667585347 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f87c80b5030 con 0x7f87d80b0d50 2024-09-10T21:34:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.482+0000 7f87ce000640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87b80725e0 0x7f87b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.482+0000 7f87ce000640 1 -- 172.21.15.97:0/667585347 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9285+0+0 (secure 0 0 0) 0x7f87c812e110 con 0x7f87d80b0d50 2024-09-10T21:34:48.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.482+0000 7f87d5400640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87b80725e0 0x7f87b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.482+0000 7f87d5400640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87b80725e0 0x7f87b8074aa0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f87d806f0a0 tx=0x7f87c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.486+0000 7f87ce000640 1 -- 172.21.15.97:0/667585347 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f87c80f2de0 con 0x7f87d80b0d50 2024-09-10T21:34:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.686+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87d80cc060 con 0x7f87d80b0d50 2024-09-10T21:34:48.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87ce000640 1 -- 172.21.15.97:0/667585347 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 302 v302) ==== 95+0+5233 (secure 0 0 0) 0x7f87c8016990 con 0x7f87d80b0d50 2024-09-10T21:34:48.688 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:48.688 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":302,"btime":"2024-09-10T21:34:48:311102+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":302,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:48.311092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10146":{"gid":10146,"name":"a","rank":0,"incarnation":302,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/3289603806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3289603806},{"type":"v1","addr":"172.21.15.97:6834","nonce":3289603806}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:48.688 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 302 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87b80725e0 msgr2=0x7f87b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87b80725e0 0x7f87b8074aa0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f87d806f0a0 tx=0x7f87c4009040 comp rx=0 tx=0).stop 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b0d50 msgr2=0x7f87d806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b0d50 0x7f87d806f6d0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f87c8000f90 tx=0x7f87c80a5b80 comp rx=0 tx=0).stop 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 shutdown_connections 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f87b80725e0 0x7f87b8074aa0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d80ce640 0x7f87d8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f87d80cd6e0 0x7f87d80b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 --2- 172.21.15.97:0/667585347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d80b0d50 0x7f87d806f6d0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 >> 172.21.15.97:0/667585347 conn(0x7f87d80d1c30 msgr2=0x7f87d8071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 shutdown_connections 2024-09-10T21:34:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:48.690+0000 7f87dca00640 1 -- 172.21.15.97:0/667585347 wait complete. 2024-09-10T21:34:48.708 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:34:51.338 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:51.338+0000 7f2389000640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:34:51.340 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:51.338+0000 7f2389000640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T21:34:51.340 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:51.338+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:34:51.341 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:34:51.338+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:34:52.116 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:52.118+0000 7f238e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:53.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- 172.21.15.97:0/2781623039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac15fc10 msgr2=0x7fe9ac13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 --2- 172.21.15.97:0/2781623039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac15fc10 0x7fe9ac13f9b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009900 tx=0x7fe9a401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:34:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- 172.21.15.97:0/2781623039 shutdown_connections 2024-09-10T21:34:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 --2- 172.21.15.97:0/2781623039 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac15fc10 0x7fe9ac13f9b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 --2- 172.21.15.97:0/2781623039 >> v2:172.21.15.194:3300/0 conn(0x7fe9ac141d20 0x7fe9ac15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- 172.21.15.97:0/2781623039 >> 172.21.15.97:0/2781623039 conn(0x7fe9ac0b1780 msgr2=0x7fe9ac0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- 172.21.15.97:0/2781623039 shutdown_connections 2024-09-10T21:34:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- 172.21.15.97:0/2781623039 wait complete. 2024-09-10T21:34:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 Processor -- start 2024-09-10T21:34:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- start start 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac0ce5f0 0x7fe9ac15ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ac141d20 0x7fe9ac0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9ac15fc10 0x7fe9ac13a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac168a70 con 0x7fe9ac141d20 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac140d00 con 0x7fe9ac15fc10 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9b1200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac168bf0 con 0x7fe9ac0ce5f0 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ac141d20 0x7fe9ac0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9a9e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac0ce5f0 0x7fe9ac15ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ac141d20 0x7fe9ac0b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38714/0 (socket says 172.21.15.97:38714) 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9a1400640 1 -- 172.21.15.97:0/734238648 learned_addr learned my addr 172.21.15.97:0/734238648 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:53.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.866+0000 7fe9aa800640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9ac15fc10 0x7fe9ac13a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a1400640 1 -- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac0ce5f0 msgr2=0x7fe9ac15ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a1400640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac0ce5f0 0x7fe9ac15ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a1400640 1 -- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9ac15fc10 msgr2=0x7fe9ac13a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a1400640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9ac15fc10 0x7fe9ac13a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a1400640 1 -- 172.21.15.97:0/734238648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac13c750 con 0x7fe9ac141d20 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a1400640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ac141d20 0x7fe9ac0b3e30 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fe99400ec00 tx=0x7fe99400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a2a00640 1 -- 172.21.15.97:0/734238648 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994010630 con 0x7fe9ac141d20 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a2a00640 1 -- 172.21.15.97:0/734238648 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9940108c0 con 0x7fe9ac141d20 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a2a00640 1 -- 172.21.15.97:0/734238648 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994010cf0 con 0x7fe9ac141d20 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac13ca30 con 0x7fe9ac141d20 2024-09-10T21:34:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac13cff0 con 0x7fe9ac141d20 2024-09-10T21:34:53.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac15f6d0 con 0x7fe9ac141d20 2024-09-10T21:34:53.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a2a00640 1 -- 172.21.15.97:0/734238648 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe994018420 con 0x7fe9ac141d20 2024-09-10T21:34:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a2a00640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9880725e0 0x7fe988074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.870+0000 7fe9a2a00640 1 -- 172.21.15.97:0/734238648 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7fe994094e10 con 0x7fe9ac141d20 2024-09-10T21:34:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.874+0000 7fe9a9e00640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9880725e0 0x7fe988074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:53.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.874+0000 7fe9a9e00640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9880725e0 0x7fe988074aa0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe99c007fd0 tx=0x7fe99c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:53.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:53.874+0000 7fe9a2a00640 1 -- 172.21.15.97:0/734238648 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe99405f070 con 0x7fe9ac141d20 2024-09-10T21:34:54.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9ac141360 con 0x7fe9ac141d20 2024-09-10T21:34:54.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9a2a00640 1 -- 172.21.15.97:0/734238648 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 306 v306) ==== 95+0+4395 (secure 0 0 0) 0x7fe994063aa0 con 0x7fe9ac141d20 2024-09-10T21:34:54.072 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:54.072 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":306,"btime":"2024-09-10T21:34:51:343305+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":306,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:51.343300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:54.073 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 306 2024-09-10T21:34:54.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9880725e0 msgr2=0x7fe988074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9880725e0 0x7fe988074aa0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe99c007fd0 tx=0x7fe99c009040 comp rx=0 tx=0).stop 2024-09-10T21:34:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ac141d20 msgr2=0x7fe9ac0b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ac141d20 0x7fe9ac0b3e30 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fe99400ec00 tx=0x7fe99400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 shutdown_connections 2024-09-10T21:34:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe9880725e0 0x7fe988074aa0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9ac15fc10 0x7fe9ac13a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9ac141d20 0x7fe9ac0b3e30 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 --2- 172.21.15.97:0/734238648 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe9ac0ce5f0 0x7fe9ac15ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 >> 172.21.15.97:0/734238648 conn(0x7fe9ac0b1780 msgr2=0x7fe9ac13b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:54.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 shutdown_connections 2024-09-10T21:34:54.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.074+0000 7fe9b1200640 1 -- 172.21.15.97:0/734238648 wait complete. 2024-09-10T21:34:54.094 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:34:54.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- 172.21.15.97:0/793991491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a201511f0 msgr2=0x7f4a200b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 --2- 172.21.15.97:0/793991491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a201511f0 0x7f4a200b37b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009c80 tx=0x7f4a1001f5a0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- 172.21.15.97:0/793991491 shutdown_connections 2024-09-10T21:34:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 --2- 172.21.15.97:0/793991491 >> v2:172.21.15.194:3300/0 conn(0x7f4a200b3cf0 0x7f4a200ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 --2- 172.21.15.97:0/793991491 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a201511f0 0x7f4a200b37b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- 172.21.15.97:0/793991491 >> 172.21.15.97:0/793991491 conn(0x7f4a200b1780 msgr2=0x7f4a200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- 172.21.15.97:0/793991491 shutdown_connections 2024-09-10T21:34:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- 172.21.15.97:0/793991491 wait complete. 2024-09-10T21:34:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 Processor -- start 2024-09-10T21:34:54.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- start start 2024-09-10T21:34:54.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a200b3cf0 0x7f4a20152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a200cf420 0x7f4a20140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a201511f0 0x7f4a201e8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2006e530 con 0x7f4a200cf420 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2006e3b0 con 0x7f4a201511f0 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a27400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2006e6b0 con 0x7f4a200b3cf0 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a200cf420 0x7f4a20140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a200cf420 0x7f4a20140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38722/0 (socket says 172.21.15.97:38722) 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 -- 172.21.15.97:0/1859633189 learned_addr learned my addr 172.21.15.97:0/1859633189 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a24a00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a200b3cf0 0x7f4a20152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a25400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a201511f0 0x7f4a201e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 -- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a200b3cf0 msgr2=0x7f4a20152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a200b3cf0 0x7f4a20152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 -- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a201511f0 msgr2=0x7f4a201e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a201511f0 0x7f4a201e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 -- 172.21.15.97:0/1859633189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a20152f40 con 0x7f4a200cf420 2024-09-10T21:34:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a24a00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a200b3cf0 0x7f4a20152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a25400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a201511f0 0x7f4a201e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.238+0000 7f4a1fe00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a200cf420 0x7f4a20140d40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e9f0 tx=0x7f4a0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a1cc00640 1 -- 172.21.15.97:0/1859633189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00cd20 con 0x7f4a200cf420 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a1cc00640 1 -- 172.21.15.97:0/1859633189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a0c004020 con 0x7f4a200cf420 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a20153220 con 0x7f4a200cf420 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a1cc00640 1 -- 172.21.15.97:0/1859633189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004450 con 0x7f4a200cf420 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a201537e0 con 0x7f4a200cf420 2024-09-10T21:34:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a200ceee0 con 0x7f4a200cf420 2024-09-10T21:34:54.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a1cc00640 1 -- 172.21.15.97:0/1859633189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a0c003600 con 0x7f4a200cf420 2024-09-10T21:34:54.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.242+0000 7f4a1cc00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a000725e0 0x7f4a00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.246+0000 7f4a1cc00640 1 -- 172.21.15.97:0/1859633189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7f4a0c094be0 con 0x7f4a200cf420 2024-09-10T21:34:54.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.246+0000 7f4a24a00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a000725e0 0x7f4a00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.246+0000 7f4a24a00640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a000725e0 0x7f4a00074aa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4a100b3000 tx=0x7f4a1001dde0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.246+0000 7f4a1cc00640 1 -- 172.21.15.97:0/1859633189 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a0c05f490 con 0x7f4a200cf420 2024-09-10T21:34:54.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.450+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a20076120 con 0x7f4a200cf420 2024-09-10T21:34:54.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.450+0000 7f4a1cc00640 1 -- 172.21.15.97:0/1859633189 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 306 v306) ==== 95+0+4395 (secure 0 0 0) 0x7f4a0c05ed90 con 0x7f4a200cf420 2024-09-10T21:34:54.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:54.450 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":306,"btime":"2024-09-10T21:34:51:343305+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":306,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:51.343300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 306 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a000725e0 msgr2=0x7f4a00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a000725e0 0x7f4a00074aa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4a100b3000 tx=0x7f4a1001dde0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a200cf420 msgr2=0x7f4a20140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a200cf420 0x7f4a20140d40 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e9f0 tx=0x7f4a0c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 shutdown_connections 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4a000725e0 0x7f4a00074aa0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a201511f0 0x7f4a201e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a200cf420 0x7f4a20140d40 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 --2- 172.21.15.97:0/1859633189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4a200b3cf0 0x7f4a20152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 >> 172.21.15.97:0/1859633189 conn(0x7f4a200b1780 msgr2=0x7f4a2015c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 shutdown_connections 2024-09-10T21:34:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.454+0000 7f4a27400640 1 -- 172.21.15.97:0/1859633189 wait complete. 2024-09-10T21:34:54.471 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:34:54.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- 172.21.15.97:0/1172675069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00d27f0 msgr2=0x7f4bd00cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 --2- 172.21.15.97:0/1172675069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00d27f0 0x7f4bd00cc550 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009840 tx=0x7f4bb801c820 comp rx=0 tx=0).stop 2024-09-10T21:34:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- 172.21.15.97:0/1172675069 shutdown_connections 2024-09-10T21:34:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 --2- 172.21.15.97:0/1172675069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00d27f0 0x7f4bd00cc550 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- 172.21.15.97:0/1172675069 >> 172.21.15.97:0/1172675069 conn(0x7f4bd00d1cd0 msgr2=0x7f4bd00d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:54.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- 172.21.15.97:0/1172675069 shutdown_connections 2024-09-10T21:34:54.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- 172.21.15.97:0/1172675069 wait complete. 2024-09-10T21:34:54.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 Processor -- start 2024-09-10T21:34:54.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- start start 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bd00cca90 0x7f4bd00b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4bd00d27f0 0x7f4bd014d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd015eed0 0x7f4bd0155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0169fc0 con 0x7f4bd015eed0 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0169e40 con 0x7f4bd00cca90 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bd6800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd016a140 con 0x7f4bd00d27f0 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd015eed0 0x7f4bd0155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd015eed0 0x7f4bd0155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38752/0 (socket says 172.21.15.97:38752) 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 -- 172.21.15.97:0/3834921760 learned_addr learned my addr 172.21.15.97:0/3834921760 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcf400640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bd00cca90 0x7f4bd00b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bc6a00640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4bd00d27f0 0x7f4bd014d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 -- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4bd00d27f0 msgr2=0x7f4bd014d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4bd00d27f0 0x7f4bd014d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 -- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bd00cca90 msgr2=0x7f4bd00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bd00cca90 0x7f4bd00b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.614+0000 7f4bcfe00640 1 -- 172.21.15.97:0/3834921760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd00b48c0 con 0x7f4bd015eed0 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bc6a00640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4bd00d27f0 0x7f4bd014d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bcf400640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bd00cca90 0x7f4bd00b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bcfe00640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd015eed0 0x7f4bd0155b30 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c3c0 tx=0x7f4bc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bc7e00640 1 -- 172.21.15.97:0/3834921760 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8011790 con 0x7f4bd015eed0 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bc7e00640 1 -- 172.21.15.97:0/3834921760 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bc8017070 con 0x7f4bd015eed0 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd00b7af0 con 0x7f4bd015eed0 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bc7e00640 1 -- 172.21.15.97:0/3834921760 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8011cc0 con 0x7f4bd015eed0 2024-09-10T21:34:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd00b45e0 con 0x7f4bd015eed0 2024-09-10T21:34:54.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd00b5570 con 0x7f4bd015eed0 2024-09-10T21:34:54.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bc7e00640 1 -- 172.21.15.97:0/3834921760 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4bc801c460 con 0x7f4bd015eed0 2024-09-10T21:34:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bc7e00640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bac072610 0x7f4bac074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bc7e00640 1 -- 172.21.15.97:0/3834921760 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7f4bc8093f10 con 0x7f4bd015eed0 2024-09-10T21:34:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.618+0000 7f4bcf400640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bac072610 0x7f4bac074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.622+0000 7f4bcf400640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bac072610 0x7f4bac074ad0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00012f0 tx=0x7f4bc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:54.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.622+0000 7f4bc7e00640 1 -- 172.21.15.97:0/3834921760 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4bd00b5570 con 0x7f4bd015eed0 2024-09-10T21:34:54.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.822+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bd00cfd60 con 0x7f4bd015eed0 2024-09-10T21:34:54.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.822+0000 7f4bc7e00640 1 -- 172.21.15.97:0/3834921760 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 306 v306) ==== 95+0+4395 (secure 0 0 0) 0x7f4bd00b7af0 con 0x7f4bd015eed0 2024-09-10T21:34:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":306,"btime":"2024-09-10T21:34:51:343305+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":306,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:51.343300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:54.822 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 306 2024-09-10T21:34:54.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bac072610 msgr2=0x7f4bac074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bac072610 0x7f4bac074ad0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00012f0 tx=0x7f4bc0009040 comp rx=0 tx=0).stop 2024-09-10T21:34:54.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd015eed0 msgr2=0x7f4bd0155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:54.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd015eed0 0x7f4bd0155b30 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c3c0 tx=0x7f4bc800c880 comp rx=0 tx=0).stop 2024-09-10T21:34:54.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 shutdown_connections 2024-09-10T21:34:54.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bac072610 0x7f4bac074ad0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd015eed0 0x7f4bd0155b30 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4bd00d27f0 0x7f4bd014d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3834921760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bd00cca90 0x7f4bd00b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:54.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 >> 172.21.15.97:0/3834921760 conn(0x7f4bd00d1cd0 msgr2=0x7f4bd00cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:54.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 shutdown_connections 2024-09-10T21:34:54.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:54.826+0000 7f4bd6800640 1 -- 172.21.15.97:0/3834921760 wait complete. 2024-09-10T21:34:54.844 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2024-09-10T21:34:54.844 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2024-09-10T21:34:54.844 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:34:54.844 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:34:54.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:34:54.853 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:34:54.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:34:54.853 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:34:54.902 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:34:54.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:34:54.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:34:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- 172.21.15.97:0/1687582775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 msgr2=0x7fcd240b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 --2- 172.21.15.97:0/1687582775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd240b2a80 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fcd24072040 tx=0x7fcd1c01cac0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- 172.21.15.97:0/1687582775 shutdown_connections 2024-09-10T21:34:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 --2- 172.21.15.97:0/1687582775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd240b2a80 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 --2- 172.21.15.97:0/1687582775 >> v2:172.21.15.194:3300/0 conn(0x7fcd240cf0b0 0x7fcd2406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- 172.21.15.97:0/1687582775 >> 172.21.15.97:0/1687582775 conn(0x7fcd240b1780 msgr2=0x7fcd240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:55.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- 172.21.15.97:0/1687582775 shutdown_connections 2024-09-10T21:34:55.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- 172.21.15.97:0/1687582775 wait complete. 2024-09-10T21:34:55.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 Processor -- start 2024-09-10T21:34:55.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- start start 2024-09-10T21:34:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd240cf0b0 0x7fcd24139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd240cf490 0x7fcd24151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd2415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240b3b60 con 0x7fcd2415b110 2024-09-10T21:34:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240b39e0 con 0x7fcd240cf490 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd2b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcd24070970 con 0x7fcd240cf0b0 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd29200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd2415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd29200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd2415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38764/0 (socket says 172.21.15.97:38764) 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd29200640 1 -- 172.21.15.97:0/2459783639 learned_addr learned my addr 172.21.15.97:0/2459783639 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd23e00640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd240cf0b0 0x7fcd24139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.094+0000 7fcd1b400640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd240cf490 0x7fcd24151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd29200640 1 -- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd240cf0b0 msgr2=0x7fcd24139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd29200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd240cf0b0 0x7fcd24139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd29200640 1 -- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd240cf490 msgr2=0x7fcd24151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd29200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd240cf490 0x7fcd24151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd29200640 1 -- 172.21.15.97:0/2459783639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241f8bb0 con 0x7fcd2415b110 2024-09-10T21:34:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd29200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd2415de30 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0196d0 tx=0x7fcd1c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:55.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd20c00640 1 -- 172.21.15.97:0/2459783639 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0bb070 con 0x7fcd2415b110 2024-09-10T21:34:55.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd20c00640 1 -- 172.21.15.97:0/2459783639 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd1c004d40 con 0x7fcd2415b110 2024-09-10T21:34:55.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241f8e90 con 0x7fcd2415b110 2024-09-10T21:34:55.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd20c00640 1 -- 172.21.15.97:0/2459783639 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0b2040 con 0x7fcd2415b110 2024-09-10T21:34:55.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241f9450 con 0x7fcd2415b110 2024-09-10T21:34:55.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2406e3b0 con 0x7fcd2415b110 2024-09-10T21:34:55.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.098+0000 7fcd20c00640 1 -- 172.21.15.97:0/2459783639 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd1c0b5030 con 0x7fcd2415b110 2024-09-10T21:34:55.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.102+0000 7fcd20c00640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd000725e0 0x7fcd00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.102+0000 7fcd20c00640 1 -- 172.21.15.97:0/2459783639 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7fcd1c12dc30 con 0x7fcd2415b110 2024-09-10T21:34:55.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.102+0000 7fcd23e00640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd000725e0 0x7fcd00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.102+0000 7fcd23e00640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd000725e0 0x7fcd00074aa0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400bd00 tx=0x7fcd1400b490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:55.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.106+0000 7fcd20c00640 1 -- 172.21.15.97:0/2459783639 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd1c0166e0 con 0x7fcd2415b110 2024-09-10T21:34:55.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.306+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd24152210 con 0x7fcd2415b110 2024-09-10T21:34:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.358+0000 7fcd20c00640 1 -- 172.21.15.97:0/2459783639 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v307) ==== 161+0+0 (secure 0 0 0) 0x7fcd1c0f7dc0 con 0x7fcd2415b110 2024-09-10T21:34:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd000725e0 msgr2=0x7fcd00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd000725e0 0x7fcd00074aa0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400bd00 tx=0x7fcd1400b490 comp rx=0 tx=0).stop 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 msgr2=0x7fcd2415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd2415de30 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0196d0 tx=0x7fcd1c005b00 comp rx=0 tx=0).stop 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 shutdown_connections 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcd000725e0 0x7fcd00074aa0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd2415b110 0x7fcd2415de30 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd240cf490 0x7fcd24151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 --2- 172.21.15.97:0/2459783639 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcd240cf0b0 0x7fcd24139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 >> 172.21.15.97:0/2459783639 conn(0x7fcd240b1780 msgr2=0x7fcd240af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 shutdown_connections 2024-09-10T21:34:55.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.362+0000 7fcd2b200640 1 -- 172.21.15.97:0/2459783639 wait complete. 2024-09-10T21:34:55.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:34:55.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.522+0000 7f51c8800640 1 -- 172.21.15.97:0/3123866706 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00ce830 msgr2=0x7f51c00cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.522+0000 7f51c8800640 1 --2- 172.21.15.97:0/3123866706 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00ce830 0x7f51c00cf800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009840 tx=0x7f51b401f010 comp rx=0 tx=0).stop 2024-09-10T21:34:55.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- 172.21.15.97:0/3123866706 shutdown_connections 2024-09-10T21:34:55.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 --2- 172.21.15.97:0/3123866706 >> v2:172.21.15.194:3300/0 conn(0x7f51c00cec10 0x7f51c00cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 --2- 172.21.15.97:0/3123866706 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00ce830 0x7f51c00cf800 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- 172.21.15.97:0/3123866706 >> 172.21.15.97:0/3123866706 conn(0x7f51c00b37c0 msgr2=0x7f51c00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:55.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- 172.21.15.97:0/3123866706 shutdown_connections 2024-09-10T21:34:55.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- 172.21.15.97:0/3123866706 wait complete. 2024-09-10T21:34:55.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 Processor -- start 2024-09-10T21:34:55.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- start start 2024-09-10T21:34:55.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c00ce830 0x7f51c015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00cec10 0x7f51c015b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51c00d0280 0x7f51c00c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51c00b4480 con 0x7f51c00ce830 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51c00cde60 con 0x7f51c00d0280 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c8800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f51c00ce1a0 con 0x7f51c00cec10 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c00ce830 0x7f51c015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c00ce830 0x7f51c015aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38768/0 (socket says 172.21.15.97:38768) 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 -- 172.21.15.97:0/1844786913 learned_addr learned my addr 172.21.15.97:0/1844786913 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5400640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00cec10 0x7f51c015b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c6800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51c00d0280 0x7f51c00c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 -- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00cec10 msgr2=0x7f51c015b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00cec10 0x7f51c015b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 -- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51c00d0280 msgr2=0x7f51c00c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:55.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51c00d0280 0x7f51c00c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 -- 172.21.15.97:0/1844786913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c01593c0 con 0x7f51c00ce830 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c6800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51c00d0280 0x7f51c00c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.526+0000 7f51c5e00640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c00ce830 0x7f51c015aff0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f51b401cc40 tx=0x7f51b4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51ba000640 1 -- 172.21.15.97:0/1844786913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b40b5040 con 0x7f51c00ce830 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51ba000640 1 -- 172.21.15.97:0/1844786913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51b4005220 con 0x7f51c00ce830 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c01f5130 con 0x7f51c00ce830 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51ba000640 1 -- 172.21.15.97:0/1844786913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b40046d0 con 0x7f51c00ce830 2024-09-10T21:34:55.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01f5410 con 0x7f51c00ce830 2024-09-10T21:34:55.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c00b2780 con 0x7f51c00ce830 2024-09-10T21:34:55.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51ba000640 1 -- 172.21.15.97:0/1844786913 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f51b40b8070 con 0x7f51c00ce830 2024-09-10T21:34:55.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51ba000640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51a40725e0 0x7f51a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:55.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51ba000640 1 -- 172.21.15.97:0/1844786913 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7f51b4130f60 con 0x7f51c00ce830 2024-09-10T21:34:55.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.530+0000 7f51c5400640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51a40725e0 0x7f51a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:55.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.534+0000 7f51c5400640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51a40725e0 0x7f51a4074aa0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f51c0158e10 tx=0x7f51b00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:55.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.534+0000 7f51ba000640 1 -- 172.21.15.97:0/1844786913 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f51b4018b70 con 0x7f51c00ce830 2024-09-10T21:34:55.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:55.758+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f51c0070e40 con 0x7f51c00ce830 2024-09-10T21:34:56.092 INFO:tasks.ceph.mds.a.smithi097.stderr: -51> 2024-09-10T21:34:51.338+0000 7f2389000640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:34:56.092 INFO:tasks.ceph.mds.a.smithi097.stderr: -48> 2024-09-10T21:34:51.338+0000 7f2389000640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T21:34:56.092 INFO:tasks.ceph.mds.a.smithi097.stderr: -34> 2024-09-10T21:34:52.118+0000 7f238e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:34:56.131 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:34:56.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.370+0000 7f51ba000640 1 -- 172.21.15.97:0/1844786913 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v308) ==== 101+0+0 (secure 0 0 0) 0x7f51b40fb7d0 con 0x7f51c00ce830 2024-09-10T21:34:56.371 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51a40725e0 msgr2=0x7f51a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51a40725e0 0x7f51a4074aa0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f51c0158e10 tx=0x7f51b00091e0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c00ce830 msgr2=0x7f51c015aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c00ce830 0x7f51c015aff0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f51b401cc40 tx=0x7f51b4005dd0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 shutdown_connections 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f51a40725e0 0x7f51a4074aa0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51c00d0280 0x7f51c00c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f51c00cec10 0x7f51c015b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 --2- 172.21.15.97:0/1844786913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c00ce830 0x7f51c015aff0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 >> 172.21.15.97:0/1844786913 conn(0x7f51c00b37c0 msgr2=0x7f51c006de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:56.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 shutdown_connections 2024-09-10T21:34:56.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.374+0000 7f51c8800640 1 -- 172.21.15.97:0/1844786913 wait complete. 2024-09-10T21:34:56.393 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:34:56.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 Processor -- start 2024-09-10T21:34:56.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 -- start start 2024-09-10T21:34:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aa89c80 0x55c49a9fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49ab3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aaa43c0 0x55c49ab3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c49a91da00 con 0x55c49aa8a060 2024-09-10T21:34:56.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c49a97bf40 con 0x55c49aa89c80 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff864c4b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c49aa8bc90 con 0x55c49aaa43c0 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff85be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49ab3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff85be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49ab3b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38792/0 (socket says 172.21.15.97:38792) 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.438+0000 7ff85be00640 1 -- 172.21.15.97:0/1043597431 learned_addr learned my addr 172.21.15.97:0/1043597431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 -- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aaa43c0 msgr2=0x55c49ab3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff861c00640 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aaa43c0 0x55c49ab3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff861200640 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aa89c80 0x55c49a9fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aaa43c0 0x55c49ab3b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 -- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aa89c80 msgr2=0x55c49a9fcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aa89c80 0x55c49a9fcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 -- 172.21.15.97:0/1043597431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49a8f6c10 con 0x55c49aa8a060 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff861c00640 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aaa43c0 0x55c49ab3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:56.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff861200640 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aa89c80 0x55c49a9fcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49ab3b260 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff84c01b630 tx=0x7ff84c017ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55129d999e0878bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85aa00640 1 -- 172.21.15.97:0/1043597431 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0a3e80 con 0x55c49aa8a060 2024-09-10T21:34:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85aa00640 1 -- 172.21.15.97:0/1043597431 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff84c0bc070 con 0x55c49aa8a060 2024-09-10T21:34:56.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85aa00640 1 -- 172.21.15.97:0/1043597431 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0aba70 con 0x55c49aa8a060 2024-09-10T21:34:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- 172.21.15.97:0/1043597431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 msgr2=0x55c49ab3b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49ab3b260 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff84c01b630 tx=0x7ff84c017ef0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- 172.21.15.97:0/1043597431 shutdown_connections 2024-09-10T21:34:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aaa43c0 0x55c49ab3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49ab3b260 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 --2- 172.21.15.97:0/1043597431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aa89c80 0x55c49a9fcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- 172.21.15.97:0/1043597431 >> 172.21.15.97:0/1043597431 conn(0x55c49a9ce140 msgr2=0x55c49a9ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- 172.21.15.97:0/1043597431 shutdown_connections 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- 172.21.15.97:0/1043597431 wait complete. 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 Processor -- start 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- start start 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aa89c80 0x55c49aaa0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49aa88b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aaa43c0 0x55c49a9f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c49aaa7d40 con 0x55c49aa8a060 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c49aaadef0 con 0x55c49aaa43c0 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff864c4b080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c49aaa80f0 con 0x55c49aa89c80 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49aa88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49aa88b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38808/0 (socket says 172.21.15.97:38808) 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 learned_addr learned my addr 172.21.15.97:0/2823162212 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:56.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aa89c80 0x55c49aaa0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff861c00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aaa43c0 0x55c49a9f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aa89c80 msgr2=0x55c49aaa0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aa89c80 0x55c49aaa0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aaa43c0 msgr2=0x55c49a9f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.442+0000 7ff85be00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aaa43c0 0x55c49a9f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49a9f85d0 con 0x55c49aa8a060 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c49aa89c80 0x55c49aaa0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff861c00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c49aaa43c0 0x55c49a9f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff85be00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49aa88b70 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0b1040 tx=0x7ff84c0ab9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff859600640 1 -- 172.21.15.97:0/2823162212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0c1400 con 0x55c49aa8a060 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff859600640 1 -- 172.21.15.97:0/2823162212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff84c0c8070 con 0x55c49aa8a060 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff859600640 1 -- 172.21.15.97:0/2823162212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0c18b0 con 0x55c49aa8a060 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c49a9f8870 con 0x55c49aa8a060 2024-09-10T21:34:56.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49a9f8db0 con 0x55c49aa8a060 2024-09-10T21:34:56.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff859600640 1 -- 172.21.15.97:0/2823162212 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff84c0c1a50 con 0x55c49aa8a060 2024-09-10T21:34:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff859600640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff840070fc0 0x7ff840073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff859600640 1 -- 172.21.15.97:0/2823162212 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7ff84c12f5e0 con 0x55c49aa8a060 2024-09-10T21:34:56.448 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:34:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff840070fc0 0x7ff840073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55c49a9f71c0 0x55c49a9f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.446+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49a9f1b70 con 0x55c49a9f71c0 2024-09-10T21:34:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.450+0000 7ff861c00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55c49a9f71c0 0x55c49a9f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.450+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff840070fc0 0x7ff840073480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff854000f70 tx=0x7ff8540035e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.450+0000 7ff861c00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55c49a9f71c0 0x55c49a9f1630 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.450+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v132'1097 uv1070 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.450+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49a9f1f10 con 0x55c49a9f71c0 2024-09-10T21:34:56.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.454+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v132'1098 uv1098 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.454+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4b780 con 0x55c49a9f71c0 2024-09-10T21:34:56.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.454+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v132'1099 uv1099 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.454 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:34:56.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.454+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4cb30 con 0x55c49a9f71c0 2024-09-10T21:34:56.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.458+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v132'1100 uv1100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.458+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4ced0 con 0x55c49a9f71c0 2024-09-10T21:34:56.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.458+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v132'1101 uv1101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.458+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4bb20 con 0x55c49a9f71c0 2024-09-10T21:34:56.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.462+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v132'1102 uv1102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.462+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4bf40 con 0x55c49a9f71c0 2024-09-10T21:34:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.462+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v132'1103 uv1076 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.462+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab4c330 con 0x55c49a9f71c0 2024-09-10T21:34:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.466+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v132'1104 uv1104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.466+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c330 con 0x55c49a9f71c0 2024-09-10T21:34:56.465 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:34:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.466+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v132'1105 uv1105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.466+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.466+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v132'1106 uv1106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.466+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.470+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v132'1107 uv1107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.470+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.470+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v132'1108 uv1108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.470+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.474+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v132'1109 uv1083 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.474+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.474+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v132'1110 uv1110 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.474+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.475 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:34:56.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.478+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v132'1111 uv1111 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.478+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.478+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v132'1112 uv1112 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.478+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.478+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v132'1113 uv1113 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.478+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.482+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v132'1114 uv1114 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.482+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.482+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v132'1115 uv1115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.482+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.486+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v132'1116 uv1096 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.486+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.486+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v132'1117 uv1117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.486+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.487 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:34:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.490+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v132'1118 uv1118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.490+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.490+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v132'1119 uv1119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.490+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.490+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v132'1120 uv1120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.490+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.494+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v132'1121 uv1121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.494+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.494+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v132'1122 uv1095 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.494+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.498+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v132'1123 uv1123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.498+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.497 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:34:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.498+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v132'1124 uv1124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.498+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.502+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v132'1125 uv1125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.502+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.502+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v132'1126 uv1126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.502+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4c2e0 con 0x55c49a9f71c0 2024-09-10T21:34:56.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.502+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v132'1127 uv1127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49a9f71c0 2024-09-10T21:34:56.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.502+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c49ab4d270 0x55c49ab4f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.502+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab4fba0 con 0x55c49ab4d270 2024-09-10T21:34:56.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.506+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c49ab4d270 0x55c49ab4f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.506+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c49ab4d270 0x55c49ab4f660 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.506+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v132'799 uv781 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.506+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab4ff40 con 0x55c49ab4d270 2024-09-10T21:34:56.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.510+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v132'800 uv800 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.510+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab502e0 con 0x55c49ab4d270 2024-09-10T21:34:56.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.510+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v132'801 uv801 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.509 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:34:56.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.510+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab506e0 con 0x55c49ab4d270 2024-09-10T21:34:56.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.510+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v132'802 uv802 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.510+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab50b10 con 0x55c49ab4d270 2024-09-10T21:34:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.514+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v132'803 uv803 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.514+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab50f20 con 0x55c49ab4d270 2024-09-10T21:34:56.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.514+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v132'804 uv786 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.514+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab51310 con 0x55c49ab4d270 2024-09-10T21:34:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.514+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v132'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.514+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab51700 con 0x55c49ab4d270 2024-09-10T21:34:56.516 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:34:56.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.518+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v132'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.518+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab51700 con 0x55c49ab4d270 2024-09-10T21:34:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.518+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v132'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.518+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.518+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v132'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.518+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.522+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v132'809 uv791 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.522+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.522+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v132'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.522+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.523 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:34:56.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.522+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v132'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.522+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.526+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v132'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.526+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.526+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v132'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.526+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.530+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v132'814 uv797 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.530+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.530+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v132'815 uv815 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.530+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.530 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:34:56.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.530+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v132'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.530+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab52700 con 0x55c49ab4d270 2024-09-10T21:34:56.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v132'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab526c0 con 0x55c49ab4d270 2024-09-10T21:34:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v132'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab516b0 con 0x55c49ab4d270 2024-09-10T21:34:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v132'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab4d270 2024-09-10T21:34:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55c49ab51fb0 0x55c49ab52390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab54b10 con 0x55c49ab51fb0 2024-09-10T21:34:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff85be00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55c49ab51fb0 0x55c49ab52390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.534+0000 7ff85be00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55c49ab51fb0 0x55c49ab52390 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.538+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.538+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab54eb0 con 0x55c49ab51fb0 2024-09-10T21:34:56.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.538+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.538+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab55250 con 0x55c49ab51fb0 2024-09-10T21:34:56.539 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:34:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.538+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.538+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab56b00 con 0x55c49ab51fb0 2024-09-10T21:34:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab56ea0 con 0x55c49ab51fb0 2024-09-10T21:34:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab55af0 con 0x55c49ab51fb0 2024-09-10T21:34:56.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55c49ab56410 0x55c49ab59250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59790 con 0x55c49ab56410 2024-09-10T21:34:56.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.542+0000 7ff861c00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55c49ab56410 0x55c49ab59250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.546+0000 7ff861c00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55c49ab56410 0x55c49ab59250 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.546+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v132'161 uv160 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.546+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab59790 con 0x55c49ab56410 2024-09-10T21:34:56.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.546+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v132'162 uv162 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.546+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59810 con 0x55c49ab56410 2024-09-10T21:34:56.547 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:34:56.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.546+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v132'163 uv163 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.546+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59c00 con 0x55c49ab56410 2024-09-10T21:34:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.550+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v132'164 uv164 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.550+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59c00 con 0x55c49ab51fb0 2024-09-10T21:34:56.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.550+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.550+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.550+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.550+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.552 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:34:56.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.554+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.554+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5afb0 con 0x55c49ab51fb0 2024-09-10T21:34:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.554+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.554+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5afb0 con 0x55c49ab51fb0 2024-09-10T21:34:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.554+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.554+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.558+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.558+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.558+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v132'393 uv387 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.558+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.558+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v132'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.558+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.559 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:34:56.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.562+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v132'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.562+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5afb0 con 0x55c49ab56410 2024-09-10T21:34:56.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.562+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v132'396 uv396 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.562+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5afb0 con 0x55c49ab56410 2024-09-10T21:34:56.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.562+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v132'397 uv397 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.562+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v132'398 uv398 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v132'399 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v132'400 uv400 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.567 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:34:56.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v132'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.566+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.570+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v132'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.570+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab56410 2024-09-10T21:34:56.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.570+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v132'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.570+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.570+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v132'178 uv177 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.570+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.574+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v132'179 uv179 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.574+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.573 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:34:56.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.574+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v132'180 uv180 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.574+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.574+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v132'181 uv181 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.574+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab59fa0 con 0x55c49ab51fb0 2024-09-10T21:34:56.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v132'182 uv182 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab51fb0 2024-09-10T21:34:56.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c49ab5a620 0x55c49ab5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d030 con 0x55c49ab5a620 2024-09-10T21:34:56.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c49ab5a620 0x55c49ab5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff861200640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c49ab5a620 0x55c49ab5ca10 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5d0f0 con 0x55c49ab5a620 2024-09-10T21:34:56.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.578+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.580 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:34:56.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.582+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.582+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e520 con 0x55c49ab5a620 2024-09-10T21:34:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.582+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.582+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e520 con 0x55c49ab5a620 2024-09-10T21:34:56.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.582+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.582+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.586+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.586+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.586+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v132'355 uv349 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.586+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.586+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v132'356 uv356 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.586+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.588 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:34:56.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.590+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v132'357 uv357 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.590+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.590+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v132'358 uv358 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.590+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.590+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v132'359 uv359 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.590+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.594+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v132'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.594+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.594+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v132'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.594+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.594 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:34:56.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.594+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v132'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.594+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d510 con 0x55c49ab5a620 2024-09-10T21:34:56.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.598+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v132'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.598+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab60320 con 0x55c49ab5a620 2024-09-10T21:34:56.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.598+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v132'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.598+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.598+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v132'157 uv155 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.598+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.602+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v132'158 uv158 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.602+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.601 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:34:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.602+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v132'159 uv159 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.602+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.602+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v132'160 uv160 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.602+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.606+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v132'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.606+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.606+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v132'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.606+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.605 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:34:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.606+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v132'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.606+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e9d0 con 0x55c49ab56410 2024-09-10T21:34:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v132'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e9d0 con 0x55c49ab56410 2024-09-10T21:34:56.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v132'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5d9c0 con 0x55c49ab56410 2024-09-10T21:34:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v132'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff85c00e820 con 0x55c49ab56410 2024-09-10T21:34:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55c49ab5ded0 0x55c49ab5e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5ded0 2024-09-10T21:34:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff85be00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55c49ab5ded0 0x55c49ab5e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.610+0000 7ff85be00640 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55c49ab5ded0 0x55c49ab5e2d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.614+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v132'181 uv180 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab5ded0 2024-09-10T21:34:56.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.614+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5ded0 2024-09-10T21:34:56.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.614+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v132'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab5ded0 2024-09-10T21:34:56.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.614+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5ded0 2024-09-10T21:34:56.614 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:34:56.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.614+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v132'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab5ded0 2024-09-10T21:34:56.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.614+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5ded0 2024-09-10T21:34:56.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v132'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab5ded0 2024-09-10T21:34:56.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5ded0 2024-09-10T21:34:56.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff85be00640 1 -- 172.21.15.97:0/2823162212 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v132'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff84c0b3070 con 0x55c49ab5ded0 2024-09-10T21:34:56.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v132'442 uv433 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v132'443 uv443 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.618+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.620 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:34:56.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.622+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v132'444 uv444 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.622+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.622+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v132'445 uv445 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.622+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.622+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v132'446 uv446 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.622+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.626+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v132'447 uv440 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.626+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.626+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v132'448 uv448 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.626+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.626 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:34:56.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.626+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v132'449 uv449 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.626+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5fe60 con 0x55c49ab5a620 2024-09-10T21:34:56.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v132'450 uv450 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v132'451 uv451 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab5e810 con 0x55c49ab5a620 2024-09-10T21:34:56.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v132'452 uv452 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55c49ab60300 con 0x55c49ab5a620 2024-09-10T21:34:56.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff861200640 1 -- 172.21.15.97:0/2823162212 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v132'453 uv453 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff854010450 con 0x55c49ab5a620 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55c49ab51fb0 msgr2=0x55c49ab52390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55c49ab51fb0 0x55c49ab52390 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c49ab5a620 msgr2=0x55c49ab5ca10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55c49ab5a620 0x55c49ab5ca10 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55c49ab5ded0 msgr2=0x55c49ab5e2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55c49ab5ded0 0x55c49ab5e2d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55c49ab56410 msgr2=0x55c49ab59250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55c49ab56410 0x55c49ab59250 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c49ab4d270 msgr2=0x55c49ab4f660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55c49ab4d270 0x55c49ab4f660 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55c49a9f71c0 msgr2=0x55c49a9f1630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55c49a9f71c0 0x55c49a9f1630 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 reap_dead start 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 reap_dead start 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff840070fc0 msgr2=0x7ff840073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff840070fc0 0x7ff840073480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff854000f70 tx=0x7ff8540035e0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff861c00640 1 -- 172.21.15.97:0/2823162212 reap_dead start 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 msgr2=0x55c49aa88b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49aa88b70 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0b1040 tx=0x7ff84c0ab9f0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.630+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 shutdown_connections 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.634+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff840070fc0 0x7ff840073480 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.634+0000 7ff864c4b080 1 --2- 172.21.15.97:0/2823162212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c49aa8a060 0x55c49aa88b70 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.634+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 >> 172.21.15.97:0/2823162212 conn(0x55c49a9ce140 msgr2=0x55c49aaa32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.634+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 shutdown_connections 2024-09-10T21:34:56.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.634+0000 7ff864c4b080 1 -- 172.21.15.97:0/2823162212 wait complete. 2024-09-10T21:34:56.636 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:34:56.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 Processor -- start 2024-09-10T21:34:56.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 -- start start 2024-09-10T21:34:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c0a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 0x55d679ca9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679caa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d679b84f40 con 0x55d679c922a0 2024-09-10T21:34:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d679b91d00 con 0x55d679c07940 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff546c49080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d679b26a00 con 0x55d679c91b00 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679caa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679caa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38816/0 (socket says 172.21.15.97:38816) 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 -- 172.21.15.97:0/1547298565 learned_addr learned my addr 172.21.15.97:0/1547298565 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff542800640 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 0x55d679ca9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543200640 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c0a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 -- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 msgr2=0x55d679ca9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 0x55d679ca9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 -- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 msgr2=0x55d679c0a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c0a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 -- 172.21.15.97:0/1547298565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d679c0b2d0 con 0x55d679c922a0 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543200640 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c0a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff543c00640 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679caa090 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001610 tx=0x7ff53c001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a29321b248ef820 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff541400640 1 -- 172.21.15.97:0/1547298565 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c016070 con 0x55d679c922a0 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff541400640 1 -- 172.21.15.97:0/1547298565 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff53c00f800 con 0x55d679c922a0 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.726+0000 7ff541400640 1 -- 172.21.15.97:0/1547298565 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00fc50 con 0x55d679c922a0 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- 172.21.15.97:0/1547298565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 msgr2=0x55d679caa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679caa090 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001610 tx=0x7ff53c001ad0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- 172.21.15.97:0/1547298565 shutdown_connections 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679caa090 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 0x55d679ca9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 --2- 172.21.15.97:0/1547298565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c0a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- 172.21.15.97:0/1547298565 >> 172.21.15.97:0/1547298565 conn(0x55d679c06520 msgr2=0x55d679cb1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- 172.21.15.97:0/1547298565 shutdown_connections 2024-09-10T21:34:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- 172.21.15.97:0/1547298565 wait complete. 2024-09-10T21:34:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 Processor -- start 2024-09-10T21:34:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- start start 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c1bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 0x55d679c90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679bd5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d679c0b810 con 0x55d679c922a0 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d679c0b570 con 0x55d679c07940 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d679c08d00 con 0x55d679c91b00 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679bd5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff542800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 0x55d679c90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679bd5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38820/0 (socket says 172.21.15.97:38820) 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 -- 172.21.15.97:0/1966812366 learned_addr learned my addr 172.21.15.97:0/1966812366 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543200640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c1bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 -- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 msgr2=0x55d679c90cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d679c91b00 0x55d679c90cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 -- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 msgr2=0x55d679c1bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d679c07940 0x55d679c1bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 -- 172.21.15.97:0/1966812366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d679caaa00 con 0x55d679c922a0 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff543c00640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679bd5d90 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0014b0 tx=0x7ff53c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff53be00640 1 -- 172.21.15.97:0/1966812366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c020030 con 0x55d679c922a0 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff53be00640 1 -- 172.21.15.97:0/1966812366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff53c006130 con 0x55d679c922a0 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d679caaca0 con 0x55d679c922a0 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff53be00640 1 -- 172.21.15.97:0/1966812366 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c016040 con 0x55d679c922a0 2024-09-10T21:34:56.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.730+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d679cab1e0 con 0x55d679c922a0 2024-09-10T21:34:56.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff53be00640 1 -- 172.21.15.97:0/1966812366 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff53c02d060 con 0x55d679c922a0 2024-09-10T21:34:56.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff53be00640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff524070fc0 0x7ff524073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff53be00640 1 -- 172.21.15.97:0/1966812366 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7ff53c093a20 con 0x55d679c922a0 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff543200640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff524070fc0 0x7ff524073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff546c49080 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d679caa560 0x55d679c1d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x55d679c91230 con 0x55d679caa560 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff542800640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d679caa560 0x55d679c1d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff543200640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff524070fc0 0x7ff524073480 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff534002d00 tx=0x7ff534002c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.734+0000 7ff542800640 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d679caa560 0x55d679c1d7d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff542800640 1 -- 172.21.15.97:0/1966812366 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v132'165 uv165 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff5300ac7f0 con 0x55d679caa560 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d679caa560 msgr2=0x55d679c1d7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d679caa560 0x55d679c1d7d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff524070fc0 msgr2=0x7ff524073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff524070fc0 0x7ff524073480 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff534002d00 tx=0x7ff534002c90 comp rx=0 tx=0).stop 2024-09-10T21:34:56.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 msgr2=0x55d679bd5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 --2- 172.21.15.97:0/1966812366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d679c922a0 0x55d679bd5d90 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0014b0 tx=0x7ff53c00ff90 comp rx=0 tx=0).stop 2024-09-10T21:34:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff543c00640 1 -- 172.21.15.97:0/1966812366 reap_dead start 2024-09-10T21:34:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 shutdown_connections 2024-09-10T21:34:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 >> 172.21.15.97:0/1966812366 conn(0x55d679c06520 msgr2=0x55d679cb1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 shutdown_connections 2024-09-10T21:34:56.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.738+0000 7ff546c49080 1 -- 172.21.15.97:0/1966812366 wait complete. 2024-09-10T21:34:56.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:34:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 -- 172.21.15.97:0/869384298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 msgr2=0x7fcc380b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 --2- 172.21.15.97:0/869384298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc380b37b0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800aa40 tx=0x7fcc28020400 comp rx=0 tx=0).stop 2024-09-10T21:34:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 -- 172.21.15.97:0/869384298 shutdown_connections 2024-09-10T21:34:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 --2- 172.21.15.97:0/869384298 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc380ceee0 0x7fcc380cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 --2- 172.21.15.97:0/869384298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc380b37b0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 -- 172.21.15.97:0/869384298 >> 172.21.15.97:0/869384298 conn(0x7fcc380b1780 msgr2=0x7fcc380b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 -- 172.21.15.97:0/869384298 shutdown_connections 2024-09-10T21:34:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.926+0000 7fcc3e400640 1 -- 172.21.15.97:0/869384298 wait complete. 2024-09-10T21:34:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 Processor -- start 2024-09-10T21:34:56.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 -- start start 2024-09-10T21:34:56.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc380b3cf0 0x7fcc3813ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc380ceee0 0x7fcc38154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc3813ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3806e740 con 0x7fcc381511f0 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3806e5c0 con 0x7fcc380b3cf0 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3806e8c0 con 0x7fcc380ceee0 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc3813ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc3813ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38834/0 (socket says 172.21.15.97:38834) 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 -- 172.21.15.97:0/1651423499 learned_addr learned my addr 172.21.15.97:0/1651423499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc36a00640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc380ceee0 0x7fcc38154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc380b3cf0 0x7fcc3813ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 -- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc380ceee0 msgr2=0x7fcc38154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc380ceee0 0x7fcc38154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 -- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc380b3cf0 msgr2=0x7fcc3813ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc380b3cf0 0x7fcc3813ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 -- 172.21.15.97:0/1651423499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc381f5ae0 con 0x7fcc381511f0 2024-09-10T21:34:56.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc380b3cf0 0x7fcc3813ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc37e00640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc3813ea90 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000daa0 tx=0x7fcc3000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc2f400640 1 -- 172.21.15.97:0/1651423499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30017070 con 0x7fcc381511f0 2024-09-10T21:34:56.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc2f400640 1 -- 172.21.15.97:0/1651423499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc30004b30 con 0x7fcc381511f0 2024-09-10T21:34:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc2f400640 1 -- 172.21.15.97:0/1651423499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30005020 con 0x7fcc381511f0 2024-09-10T21:34:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381f5dc0 con 0x7fcc381511f0 2024-09-10T21:34:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.930+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381f6380 con 0x7fcc381511f0 2024-09-10T21:34:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.934+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc380cf370 con 0x7fcc381511f0 2024-09-10T21:34:56.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.934+0000 7fcc2f400640 1 -- 172.21.15.97:0/1651423499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcc300051c0 con 0x7fcc381511f0 2024-09-10T21:34:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.934+0000 7fcc2f400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc180725e0 0x7fcc18074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.938+0000 7fcc2f400640 1 -- 172.21.15.97:0/1651423499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7fcc300944a0 con 0x7fcc381511f0 2024-09-10T21:34:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.938+0000 7fcc37400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc180725e0 0x7fcc18074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:56.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.938+0000 7fcc37400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc180725e0 0x7fcc18074aa0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc280b2000 tx=0x7fcc2801ec40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:56.938+0000 7fcc2f400640 1 -- 172.21.15.97:0/1651423499 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc30020070 con 0x7fcc381511f0 2024-09-10T21:34:57.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.138+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fcc380b4a70 con 0x7fcc381511f0 2024-09-10T21:34:57.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.382+0000 7fcc2f400640 1 -- 172.21.15.97:0/1651423499 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v309) ==== 162+0+0 (secure 0 0 0) 0x7fcc3005e6c0 con 0x7fcc381511f0 2024-09-10T21:34:57.381 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc180725e0 msgr2=0x7fcc18074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc180725e0 0x7fcc18074aa0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc280b2000 tx=0x7fcc2801ec40 comp rx=0 tx=0).stop 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 msgr2=0x7fcc3813ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc3813ea90 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000daa0 tx=0x7fcc3000df60 comp rx=0 tx=0).stop 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 shutdown_connections 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc180725e0 0x7fcc18074aa0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc381511f0 0x7fcc3813ea90 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc380ceee0 0x7fcc38154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 --2- 172.21.15.97:0/1651423499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc380b3cf0 0x7fcc3813ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 >> 172.21.15.97:0/1651423499 conn(0x7fcc380b1780 msgr2=0x7fcc3815c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:57.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 shutdown_connections 2024-09-10T21:34:57.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.386+0000 7fcc3e400640 1 -- 172.21.15.97:0/1651423499 wait complete. 2024-09-10T21:34:57.391 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:34:57.390+0000 7fbca0e00640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-10T21:34:57.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:34:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 -- 172.21.15.97:0/1331880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00b0d50 msgr2=0x7f3de00aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 --2- 172.21.15.97:0/1331880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00b0d50 0x7f3de00aedc0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f3de00733b0 tx=0x7f3dc801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 -- 172.21.15.97:0/1331880947 shutdown_connections 2024-09-10T21:34:57.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 --2- 172.21.15.97:0/1331880947 >> v2:172.21.15.194:3300/0 conn(0x7f3de00af300 0x7f3de00af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 --2- 172.21.15.97:0/1331880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00b0d50 0x7f3de00aedc0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 --2- 172.21.15.97:0/1331880947 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3de00ce640 0x7f3de00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 -- 172.21.15.97:0/1331880947 >> 172.21.15.97:0/1331880947 conn(0x7f3de00d1c30 msgr2=0x7f3de00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:57.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 -- 172.21.15.97:0/1331880947 shutdown_connections 2024-09-10T21:34:57.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.954+0000 7f3de7a00640 1 -- 172.21.15.97:0/1331880947 wait complete. 2024-09-10T21:34:57.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 Processor -- start 2024-09-10T21:34:57.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 -- start start 2024-09-10T21:34:57.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3de00af300 0x7f3de0155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:57.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3de00b0d50 0x7f3de006f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:57.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00ce640 0x7f3de006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3de00cd270 con 0x7f3de00ce640 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3de00cd0f0 con 0x7f3de00b0d50 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3de00cd3f0 con 0x7f3de00af300 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00ce640 0x7f3de006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00ce640 0x7f3de006f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45510/0 (socket says 172.21.15.97:45510) 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 -- 172.21.15.97:0/3804702002 learned_addr learned my addr 172.21.15.97:0/3804702002 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3dd7e00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3de00b0d50 0x7f3de006f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5000640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3de00af300 0x7f3de0155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 -- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3de00af300 msgr2=0x7f3de0155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3de00af300 0x7f3de0155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 -- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3de00b0d50 msgr2=0x7f3de006f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:57.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3de00b0d50 0x7f3de006f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:57.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 -- 172.21.15.97:0/3804702002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de006e330 con 0x7f3de00ce640 2024-09-10T21:34:57.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5000640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3de00af300 0x7f3de0155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:57.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de5a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00ce640 0x7f3de006f8d0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800ca80 tx=0x7f3dd800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:57.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3ddd600640 1 -- 172.21.15.97:0/3804702002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800ae20 con 0x7f3de00ce640 2024-09-10T21:34:57.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3ddd600640 1 -- 172.21.15.97:0/3804702002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dd800f560 con 0x7f3de00ce640 2024-09-10T21:34:57.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de006e8b0 con 0x7f3de00ce640 2024-09-10T21:34:57.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3ddd600640 1 -- 172.21.15.97:0/3804702002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800f9b0 con 0x7f3de00ce640 2024-09-10T21:34:57.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de006ee70 con 0x7f3de00ce640 2024-09-10T21:34:57.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.958+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de00cd950 con 0x7f3de00ce640 2024-09-10T21:34:57.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.962+0000 7f3ddd600640 1 -- 172.21.15.97:0/3804702002 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3dd800fb50 con 0x7f3de00ce640 2024-09-10T21:34:57.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.962+0000 7f3ddd600640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc0725e0 0x7f3dbc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:57.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.962+0000 7f3ddd600640 1 -- 172.21.15.97:0/3804702002 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7f3dd8094b90 con 0x7f3de00ce640 2024-09-10T21:34:57.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.962+0000 7f3de5000640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc0725e0 0x7f3dbc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:57.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.962+0000 7f3de5000640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc0725e0 0x7f3dbc074aa0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005fd0 tx=0x7f3dd000a540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:57.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:57.966+0000 7f3ddd600640 1 -- 172.21.15.97:0/3804702002 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3dd8014030 con 0x7f3de00ce640 2024-09-10T21:34:58.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3de00aee60 con 0x7f3de00ce640 2024-09-10T21:34:58.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3ddd600640 1 -- 172.21.15.97:0/3804702002 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5233 (secure 0 0 0) 0x7f3dd805f370 con 0x7f3de00ce640 2024-09-10T21:34:58.165 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:34:58.165 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":310,"btime":"2024-09-10T21:34:57:387768+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:34:57.387753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":310,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/3930239299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3930239299},{"type":"v1","addr":"172.21.15.97:6834","nonce":3930239299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:34:58.165 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 310 2024-09-10T21:34:58.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc0725e0 msgr2=0x7f3dbc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc0725e0 0x7f3dbc074aa0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005fd0 tx=0x7f3dd000a540 comp rx=0 tx=0).stop 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00ce640 msgr2=0x7f3de006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00ce640 0x7f3de006f8d0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800ca80 tx=0x7f3dd800cf40 comp rx=0 tx=0).stop 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 shutdown_connections 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3dbc0725e0 0x7f3dbc074aa0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de00ce640 0x7f3de006f8d0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3de00b0d50 0x7f3de006f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 --2- 172.21.15.97:0/3804702002 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3de00af300 0x7f3de0155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 >> 172.21.15.97:0/3804702002 conn(0x7f3de00d1c30 msgr2=0x7f3de0071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 shutdown_connections 2024-09-10T21:34:58.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:34:58.166+0000 7f3de7a00640 1 -- 172.21.15.97:0/3804702002 wait complete. 2024-09-10T21:34:58.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 310, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:34:57.387753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 132, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10215}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10215': {'gid': 10215, 'name': 'a', 'rank': 0, 'incarnation': 310, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6834/3930239299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 3930239299}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 3930239299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:34:58.187 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10215:up:replay 2024-09-10T21:34:58.187 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:35:01.379 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:35:01.378+0000 7fbca5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:03.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:03.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 -- 172.21.15.97:0/132073998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 msgr2=0x7f999c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:03.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 --2- 172.21.15.97:0/132073998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f999c004c90 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009b60 tx=0x7f99a001d270 comp rx=0 tx=0).stop 2024-09-10T21:35:03.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 -- 172.21.15.97:0/132073998 shutdown_connections 2024-09-10T21:35:03.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 --2- 172.21.15.97:0/132073998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f999c004c90 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 -- 172.21.15.97:0/132073998 >> 172.21.15.97:0/132073998 conn(0x7f99b40b1780 msgr2=0x7f99b40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:03.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 -- 172.21.15.97:0/132073998 shutdown_connections 2024-09-10T21:35:03.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 -- 172.21.15.97:0/132073998 wait complete. 2024-09-10T21:35:03.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 Processor -- start 2024-09-10T21:35:03.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.746+0000 7f99bc400640 1 -- start start 2024-09-10T21:35:03.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f99b41595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99b413a8c0 0x7f99b41557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99b406f650 0x7f99b406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99b40ce9d0 con 0x7f999c004810 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f99b415f930 con 0x7f99b406f650 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f99b40ceb50 con 0x7f99b413a8c0 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f99b41595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f99b41595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45518/0 (socket says 172.21.15.97:45518) 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 -- 172.21.15.97:0/646748060 learned_addr learned my addr 172.21.15.97:0/646748060 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9000640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99b413a8c0 0x7f99b41557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99ba400640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99b406f650 0x7f99b406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 -- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99b413a8c0 msgr2=0x7f99b41557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99b413a8c0 0x7f99b41557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 -- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99b406f650 msgr2=0x7f99b406fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99b406f650 0x7f99b406fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 -- 172.21.15.97:0/646748060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b4157d80 con 0x7f999c004810 2024-09-10T21:35:03.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b9a00640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f99b41595e0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001810 tx=0x7f99a800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:03.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b1600640 1 -- 172.21.15.97:0/646748060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8012850 con 0x7f999c004810 2024-09-10T21:35:03.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b1600640 1 -- 172.21.15.97:0/646748060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99a8012ae0 con 0x7f999c004810 2024-09-10T21:35:03.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b1600640 1 -- 172.21.15.97:0/646748060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a801a400 con 0x7f999c004810 2024-09-10T21:35:03.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b41f6fe0 con 0x7f999c004810 2024-09-10T21:35:03.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b41f6500 con 0x7f999c004810 2024-09-10T21:35:03.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b415c100 con 0x7f999c004810 2024-09-10T21:35:03.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.750+0000 7f99b1600640 1 -- 172.21.15.97:0/646748060 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f99a801a5a0 con 0x7f999c004810 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.754+0000 7f99b1600640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99940725e0 0x7f9994074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.754+0000 7f99b1600640 1 -- 172.21.15.97:0/646748060 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7f99a80974e0 con 0x7f999c004810 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.754+0000 7f99b9000640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99940725e0 0x7f9994074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:03.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.754+0000 7f99b9000640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99940725e0 0x7f9994074aa0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f99a00ae000 tx=0x7f99a001a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.758+0000 7f99b1600640 1 -- 172.21.15.97:0/646748060 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99b4157d80 con 0x7f999c004810 2024-09-10T21:35:03.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b413ea20 con 0x7f999c004810 2024-09-10T21:35:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99b1600640 1 -- 172.21.15.97:0/646748060 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 313 v313) ==== 95+0+5242 (secure 0 0 0) 0x7f99a8016030 con 0x7f999c004810 2024-09-10T21:35:03.956 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:03.956 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":313,"btime":"2024-09-10T21:35:00:405603+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":313,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:00.405599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":310,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6834/3930239299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3930239299},{"type":"v1","addr":"172.21.15.97:6834","nonce":3930239299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10215,"qdb_cluster":[10215]},"id":6}]} 2024-09-10T21:35:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 313 2024-09-10T21:35:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99940725e0 msgr2=0x7f9994074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99940725e0 0x7f9994074aa0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f99a00ae000 tx=0x7f99a001a010 comp rx=0 tx=0).stop 2024-09-10T21:35:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 msgr2=0x7f99b41595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f99b41595e0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001810 tx=0x7f99a800e770 comp rx=0 tx=0).stop 2024-09-10T21:35:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 shutdown_connections 2024-09-10T21:35:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f99940725e0 0x7f9994074aa0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99b406f650 0x7f99b406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99b413a8c0 0x7f99b41557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 --2- 172.21.15.97:0/646748060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f999c004810 0x7f99b41595e0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 >> 172.21.15.97:0/646748060 conn(0x7f99b40b1780 msgr2=0x7f99b415b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 shutdown_connections 2024-09-10T21:35:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:03.958+0000 7f99bc400640 1 -- 172.21.15.97:0/646748060 wait complete. 2024-09-10T21:35:03.978 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 313, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:30:36.527179+0000', 'modified': '2024-09-10T21:35:00.405599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 132, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10215}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10215': {'gid': 10215, 'name': 'a', 'rank': 0, 'incarnation': 310, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6834/3930239299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 3930239299}, {'type': 'v1', 'addr': '172.21.15.97:6834', 'nonce': 3930239299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10215, 'qdb_cluster': [10215]} 2024-09-10T21:35:03.978 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:35:03.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:35:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 -- 172.21.15.97:0/2443223082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 msgr2=0x7fce98004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 --2- 172.21.15.97:0/2443223082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 0x7fce98004810 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009b60 tx=0x7fcea001d230 comp rx=0 tx=0).stop 2024-09-10T21:35:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb6800640 1 -- 172.21.15.97:0/2443223082 reap_dead start 2024-09-10T21:35:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 -- 172.21.15.97:0/2443223082 shutdown_connections 2024-09-10T21:35:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 -- 172.21.15.97:0/2443223082 >> 172.21.15.97:0/2443223082 conn(0x7fceb00b3c70 msgr2=0x7fceb00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 -- 172.21.15.97:0/2443223082 shutdown_connections 2024-09-10T21:35:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 -- 172.21.15.97:0/2443223082 wait complete. 2024-09-10T21:35:04.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 Processor -- start 2024-09-10T21:35:04.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.122+0000 7fceb8800640 1 -- start start 2024-09-10T21:35:04.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 0x7fceb01f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fceb00cbd30 0x7fceb0146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fceb0149300 0x7fceb0149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00ce410 con 0x7fce98003fe0 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00af7a0 con 0x7fceb00cbd30 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00ce590 con 0x7fceb0149300 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 0x7fceb01f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 0x7fceb01f86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45540/0 (socket says 172.21.15.97:45540) 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 -- 172.21.15.97:0/1626275508 learned_addr learned my addr 172.21.15.97:0/1626275508 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb6800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fceb0149300 0x7fceb0149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5400640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fceb00cbd30 0x7fceb0146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 -- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fceb0149300 msgr2=0x7fceb0149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fceb0149300 0x7fceb0149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 -- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fceb00cbd30 msgr2=0x7fceb0146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fceb00cbd30 0x7fceb0146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 -- 172.21.15.97:0/1626275508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb01f8be0 con 0x7fce98003fe0 2024-09-10T21:35:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5400640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fceb00cbd30 0x7fceb0146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb5e00640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 0x7fceb01f86a0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fcea40017d0 tx=0x7fcea400e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceaa000640 1 -- 172.21.15.97:0/1626275508 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4016070 con 0x7fce98003fe0 2024-09-10T21:35:04.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceaa000640 1 -- 172.21.15.97:0/1626275508 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea4003600 con 0x7fce98003fe0 2024-09-10T21:35:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceaa000640 1 -- 172.21.15.97:0/1626275508 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4003a50 con 0x7fce98003fe0 2024-09-10T21:35:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb01f8e60 con 0x7fce98003fe0 2024-09-10T21:35:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.126+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb01f9420 con 0x7fce98003fe0 2024-09-10T21:35:04.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceaa000640 1 -- 172.21.15.97:0/1626275508 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcea4003bf0 con 0x7fce98003fe0 2024-09-10T21:35:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 Processor -- start 2024-09-10T21:35:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceaa000640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce94072610 0x7fce94074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 -- start start 2024-09-10T21:35:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceaa000640 1 -- 172.21.15.97:0/1626275508 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7fcea4097730 con 0x7fce98003fe0 2024-09-10T21:35:04.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb6800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce94072610 0x7fce94074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc55762db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557168d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dc556373f0 con 0x55dc557632f0 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dc55574b20 con 0x55dc557629d0 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55dc55563f50 con 0x55dc55716e10 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb6800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce94072610 0x7fce94074ad0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fceb0071900 tx=0x7fceac009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557168d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45544/0 (socket says 172.21.15.97:45544) 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 -- 172.21.15.97:0/3661808860 learned_addr learned my addr 172.21.15.97:0/3661808860 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb6800640 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5e00640 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc55762db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 -- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 msgr2=0x55dc557171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 -- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 msgr2=0x55dc55762db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc55762db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 -- 172.21.15.97:0/3661808860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5552b590 con 0x55dc557632f0 2024-09-10T21:35:04.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fceb5400640 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557168d0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fcea00af040 tx=0x7fce98005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b720097a579e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fce9ea00640 1 -- 172.21.15.97:0/3661808860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00bb070 con 0x55dc557632f0 2024-09-10T21:35:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fce9ea00640 1 -- 172.21.15.97:0/3661808860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea00043f0 con 0x55dc557632f0 2024-09-10T21:35:04.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.130+0000 7fce9ea00640 1 -- 172.21.15.97:0/3661808860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00b1040 con 0x55dc557632f0 2024-09-10T21:35:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3661808860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 msgr2=0x55dc557168d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557168d0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fcea00af040 tx=0x7fce98005840 comp rx=0 tx=0).stop 2024-09-10T21:35:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3661808860 shutdown_connections 2024-09-10T21:35:04.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557168d0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3661808860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc55762db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3661808860 >> 172.21.15.97:0/3661808860 conn(0x55dc55600130 msgr2=0x55dc55745e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3661808860 shutdown_connections 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3661808860 wait complete. 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- start start 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 Processor -- start 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557629d0 0x55dc55743e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557632f0 0x55dc557443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dc554fe5d0 con 0x55dc557629d0 2024-09-10T21:35:04.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dc55567d60 con 0x55dc557632f0 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55dc5550a500 con 0x55dc55716e10 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 10 client.0 _ll_register_callbacks cb 0x55dc5550e810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557629d0 0x55dc55743e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557629d0 0x55dc55743e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45558/0 (socket says 172.21.15.97:45558) 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 -- 172.21.15.97:0/3123527918 learned_addr learned my addr 172.21.15.97:0/3123527918 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb6800640 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557632f0 0x55dc557443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5e00640 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 -- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 msgr2=0x55dc557436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 -- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557632f0 msgr2=0x55dc557443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557632f0 0x55dc557443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 -- 172.21.15.97:0/3123527918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc55502170 con 0x55dc557629d0 2024-09-10T21:35:04.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fceb5400640 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557629d0 0x55dc55743e70 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fcea001dad0 tx=0x7fcea00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449a8e98eba68e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fce93400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fce93400640 1 -- 172.21.15.97:0/3123527918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00c4060 con 0x55dc557629d0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc554fc0f0 con 0x55dc557629d0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.134+0000 7fce93400640 1 -- 172.21.15.97:0/3123527918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7fcea0003600 con 0x55dc557629d0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fce93400640 1 -- 172.21.15.97:0/3123527918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=313) ==== 24+0+0 (secure 0 0 0) 0x7fcea00cb050 con 0x55dc557629d0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 10 client.10269 fetch_fsmap learned FSMap version 313 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=313,monmap=2+}) -- 0x55dc555e8ef0 con 0x55dc557629d0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fce93400640 1 -- 172.21.15.97:0/3123527918 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55dc554fc0f0 con 0x55dc557629d0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fce93400640 1 -- 172.21.15.97:0/3123527918 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 313) ==== 3502+0+0 (secure 0 0 0) 0x7fcea0003800 con 0x55dc557629d0 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 10 client.10269 fetch_fsmap finished waiting for FSMap version 313 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 10 client.10269 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 20 client.10269 populate_metadata read hostname 'smithi097' 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc557450f0 0x55dc55774c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 4 client.10269 mds_command: new command op to 10215 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:35:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 --> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc55576a70 con 0x55dc557450f0 2024-09-10T21:35:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fceb5e00640 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc557450f0 0x55dc55774c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fce93400640 10 client.10269 ms_handle_connect on v2:172.21.15.97:6832/3930239299 2024-09-10T21:35:04.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fceb5e00640 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc557450f0 0x55dc55774c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6005cb9a38de5341 server_cookie=88e16dca6c5ab3fc in_seq=0 out_seq=0 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fce93400640 1 -- 172.21.15.97:0/3123527918 <== mds.0 v2:172.21.15.97:6832/3930239299 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc55576a70 con 0x55dc557450f0 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fce93400640 10 client.10269 handle_command_reply: tid=0 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 client.10269 shutdown 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557629d0 msgr2=0x55dc55743e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557629d0 0x55dc55743e70 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fcea001dad0 tx=0x7fcea00056e0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 shutdown_connections 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc557450f0 0x55dc55774c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557632f0 0x55dc557443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557629d0 0x55dc55743e70 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3123527918 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fce93400640 0 client.10269 ms_handle_reset on v2:172.21.15.97:6832/3930239299 2024-09-10T21:35:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 >> 172.21.15.97:0/3123527918 conn(0x55dc55600130 msgr2=0x55dc55715fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 shutdown_connections 2024-09-10T21:35:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3123527918 wait complete. 2024-09-10T21:35:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.138+0000 7fcebbb4b000 20 client.10269 trim_cache size 0 max 16384 2024-09-10T21:35:04.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 Processor -- start 2024-09-10T21:35:04.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- start start 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0203d70 con 0x55dc557632f0 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0202d50 con 0x55dc557629d0 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fceb0202030 con 0x55dc55716e10 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557d5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45568/0 (socket says 172.21.15.97:45568) 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 -- 172.21.15.97:0/1285817345 learned_addr learned my addr 172.21.15.97:0/1285817345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb5e00640 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb5400640 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 -- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 msgr2=0x55dc557d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557d4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 -- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 msgr2=0x55dc557d5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 -- 172.21.15.97:0/1285817345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc555bfd90 con 0x55dc557632f0 2024-09-10T21:35:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fceb6800640 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557d5850 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fceb0072e40 tx=0x7fceac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc2630b3f0ef07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fce9e000640 1 -- 172.21.15.97:0/1285817345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac0047c0 con 0x55dc557632f0 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fce9e000640 1 -- 172.21.15.97:0/1285817345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fceac004b30 con 0x55dc557632f0 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fce9e000640 1 -- 172.21.15.97:0/1285817345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac005020 con 0x55dc557632f0 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- 172.21.15.97:0/1285817345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 msgr2=0x55dc557d5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557d5850 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fceb0072e40 tx=0x7fceac015000 comp rx=0 tx=0).stop 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- 172.21.15.97:0/1285817345 shutdown_connections 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc557632f0 0x55dc557d5850 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/1285817345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc55716e10 0x55dc557d4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- 172.21.15.97:0/1285817345 >> 172.21.15.97:0/1285817345 conn(0x55dc55600130 msgr2=0x55dc5571eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- 172.21.15.97:0/1285817345 shutdown_connections 2024-09-10T21:35:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.158+0000 7fcebbb4b000 1 -- 172.21.15.97:0/1285817345 wait complete. 2024-09-10T21:35:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 -- start start 2024-09-10T21:35:04.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 Processor -- start 2024-09-10T21:35:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc55716e10 0x55dc557d89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc557632f0 0x55dc557d9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dc55567d60 con 0x55dc55716e10 2024-09-10T21:35:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0202680 con 0x55dc557629d0 2024-09-10T21:35:04.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55dc554fe5d0 con 0x55dc557632f0 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 10 client.0 _ll_register_callbacks cb 0x7fcea4004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc55716e10 0x55dc557d89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc55716e10 0x55dc557d89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45576/0 (socket says 172.21.15.97:45576) 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 -- 172.21.15.97:0/3739792425 learned_addr learned my addr 172.21.15.97:0/3739792425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb6800640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc557632f0 0x55dc557d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5400640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 -- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc557632f0 msgr2=0x55dc557d9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc557632f0 0x55dc557d9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 -- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 msgr2=0x55dc557d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 -- 172.21.15.97:0/3739792425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5552aa90 con 0x55dc55716e10 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb6800640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc557632f0 0x55dc557d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:04.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5400640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5e00640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc55716e10 0x55dc557d89f0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fcea4094d60 tx=0x7fcea40013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bfc559c370050 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fce93400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fce93400640 1 -- 172.21.15.97:0/3739792425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4092060 con 0x55dc55716e10 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fce93400640 1 -- 172.21.15.97:0/3739792425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7fcea406aa80 con 0x55dc55716e10 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceac004b30 con 0x55dc55716e10 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fce93400640 1 -- 172.21.15.97:0/3739792425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=313) ==== 24+0+0 (secure 0 0 0) 0x7fcea408b050 con 0x55dc55716e10 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 10 client.10275 fetch_fsmap learned FSMap version 313 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=313,monmap=2+}) -- 0x55dc55509050 con 0x55dc55716e10 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fce93400640 1 -- 172.21.15.97:0/3739792425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fceac004b30 con 0x55dc55716e10 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fce93400640 1 -- 172.21.15.97:0/3739792425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 313) ==== 3502+0+0 (secure 0 0 0) 0x7fcea4086bd0 con 0x55dc55716e10 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 10 client.10275 fetch_fsmap finished waiting for FSMap version 313 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 10 client.10275 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 20 client.10275 populate_metadata read hostname 'smithi097' 2024-09-10T21:35:04.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc55778290 0x55dc557e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 4 client.10275 mds_command: new command op to 10215 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 --> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] -- command(tid 0: {"prefix": "status"}) -- 0x55dc55572520 con 0x55dc55778290 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.162+0000 7fceb5400640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc55778290 0x55dc557e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb5400640 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc55778290 0x55dc557e1c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3950e75ff7c9bfd7 server_cookie=e28bf9141ceef64c in_seq=0 out_seq=0 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fce93400640 10 client.10275 ms_handle_connect on v2:172.21.15.97:6832/3930239299 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fce93400640 1 -- 172.21.15.97:0/3739792425 <== mds.0 v2:172.21.15.97:6832/3930239299 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55dc55572520 con 0x55dc55778290 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fce93400640 10 client.10275 handle_command_reply: tid=0 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 client.10275 shutdown 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc55716e10 msgr2=0x55dc557d89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc55716e10 0x55dc557d89f0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fcea4094d60 tx=0x7fcea40013c0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 shutdown_connections 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:6832/3930239299,v1:172.21.15.97:6834/3930239299] conn(0x55dc55778290 0x55dc557e1c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dc557632f0 0x55dc557d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc557629d0 0x55dc557d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fce93400640 0 client.10275 ms_handle_reset on v2:172.21.15.97:6832/3930239299 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 --2- 172.21.15.97:0/3739792425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc55716e10 0x55dc557d89f0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 >> 172.21.15.97:0/3739792425 conn(0x55dc55600130 msgr2=0x55dc557615b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 shutdown_connections 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 1 -- 172.21.15.97:0/3739792425 wait complete. 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fcebbb4b000 20 client.10275 trim_cache size 0 max 16384 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:35:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 10215, 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.7770866830000003, 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 313, 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 132, 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 132, 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 8.0302805999999993 2024-09-10T21:35:04.169 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:35:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce94072610 msgr2=0x7fce94074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce94072610 0x7fce94074ad0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fceb0071900 tx=0x7fceac009500 comp rx=0 tx=0).stop 2024-09-10T21:35:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 msgr2=0x7fceb01f86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 0x7fceb01f86a0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fcea40017d0 tx=0x7fcea400e770 comp rx=0 tx=0).stop 2024-09-10T21:35:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 shutdown_connections 2024-09-10T21:35:04.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fce94072610 0x7fce94074ad0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fceb0149300 0x7fceb0149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fceb00cbd30 0x7fceb0146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 --2- 172.21.15.97:0/1626275508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce98003fe0 0x7fceb01f86a0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 >> 172.21.15.97:0/1626275508 conn(0x7fceb00b3c70 msgr2=0x7fceb00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 shutdown_connections 2024-09-10T21:35:04.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.166+0000 7fceb8800640 1 -- 172.21.15.97:0/1626275508 wait complete. 2024-09-10T21:35:04.184 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:35:04.184 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2024-09-10T21:35:04.184 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:35:04.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 -- 172.21.15.97:0/3335605058 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680b0d50 msgr2=0x7f2b680ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 --2- 172.21.15.97:0/3335605058 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680b0d50 0x7f2b680ccc10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009bc0 tx=0x7f2b6001cba0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 -- 172.21.15.97:0/3335605058 shutdown_connections 2024-09-10T21:35:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 --2- 172.21.15.97:0/3335605058 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680b0d50 0x7f2b680ccc10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 --2- 172.21.15.97:0/3335605058 >> v2:172.21.15.194:3300/0 conn(0x7f2b680afdd0 0x7f2b680cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 -- 172.21.15.97:0/3335605058 >> 172.21.15.97:0/3335605058 conn(0x7f2b680d1c30 msgr2=0x7f2b680d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 -- 172.21.15.97:0/3335605058 shutdown_connections 2024-09-10T21:35:04.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.326+0000 7f2b6f800640 1 -- 172.21.15.97:0/3335605058 wait complete. 2024-09-10T21:35:04.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 Processor -- start 2024-09-10T21:35:04.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 -- start start 2024-09-10T21:35:04.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b680afdd0 0x7f2b68156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b680b0d50 0x7f2b680b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680ce640 0x7f2b680b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68168e90 con 0x7f2b680b0d50 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68168d10 con 0x7f2b680afdd0 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b68169010 con 0x7f2b680ce640 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b680b0d50 0x7f2b680b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b680b0d50 0x7f2b680b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45582/0 (socket says 172.21.15.97:45582) 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 -- 172.21.15.97:0/2692544963 learned_addr learned my addr 172.21.15.97:0/2692544963 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6ce00640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b680afdd0 0x7f2b68156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6d800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680ce640 0x7f2b680b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 -- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680ce640 msgr2=0x7f2b680b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680ce640 0x7f2b680b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 -- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b680afdd0 msgr2=0x7f2b68156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b680afdd0 0x7f2b68156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 -- 172.21.15.97:0/2692544963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b681572c0 con 0x7f2b680b0d50 2024-09-10T21:35:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6ce00640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b680afdd0 0x7f2b68156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6d800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680ce640 0x7f2b680b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b5fe00640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b680b0d50 0x7f2b680b4110 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000e9f0 tx=0x7f2b5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b65600640 1 -- 172.21.15.97:0/2692544963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5000cd80 con 0x7f2b680b0d50 2024-09-10T21:35:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b65600640 1 -- 172.21.15.97:0/2692544963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b50010430 con 0x7f2b680b0d50 2024-09-10T21:35:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b65600640 1 -- 172.21.15.97:0/2692544963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50010860 con 0x7f2b680b0d50 2024-09-10T21:35:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b680b1400 con 0x7f2b680b0d50 2024-09-10T21:35:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.330+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b680b19c0 con 0x7f2b680b0d50 2024-09-10T21:35:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.334+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b680cea20 con 0x7f2b680b0d50 2024-09-10T21:35:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.334+0000 7f2b65600640 1 -- 172.21.15.97:0/2692544963 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b50010a00 con 0x7f2b680b0d50 2024-09-10T21:35:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.334+0000 7f2b65600640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b440725e0 0x7f2b44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.334+0000 7f2b65600640 1 -- 172.21.15.97:0/2692544963 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7f2b50094c70 con 0x7f2b680b0d50 2024-09-10T21:35:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.334+0000 7f2b6ce00640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b440725e0 0x7f2b44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.334+0000 7f2b6ce00640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b440725e0 0x7f2b44074aa0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2b580016b0 tx=0x7f2b58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.338+0000 7f2b65600640 1 -- 172.21.15.97:0/2692544963 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b5005f600 con 0x7f2b680b0d50 2024-09-10T21:35:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b680d2260 con 0x7f2b680b0d50 2024-09-10T21:35:04.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b65600640 1 -- 172.21.15.97:0/2692544963 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v313) ==== 53+0+73 (secure 0 0 0) 0x7f2b5005ef00 con 0x7f2b680b0d50 2024-09-10T21:35:04.540 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:35:04.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b440725e0 msgr2=0x7f2b44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b440725e0 0x7f2b44074aa0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2b580016b0 tx=0x7f2b58009040 comp rx=0 tx=0).stop 2024-09-10T21:35:04.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b680b0d50 msgr2=0x7f2b680b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b680b0d50 0x7f2b680b4110 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000e9f0 tx=0x7f2b5000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 shutdown_connections 2024-09-10T21:35:04.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b440725e0 0x7f2b44074aa0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b680ce640 0x7f2b680b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b680b0d50 0x7f2b680b4110 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 --2- 172.21.15.97:0/2692544963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b680afdd0 0x7f2b68156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 >> 172.21.15.97:0/2692544963 conn(0x7f2b680d1c30 msgr2=0x7f2b68071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 shutdown_connections 2024-09-10T21:35:04.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04.542+0000 7f2b6f800640 1 -- 172.21.15.97:0/2692544963 wait complete. 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:35:04.562 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:35:04.563 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:35:04.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:04.563 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:35:04.572 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:35:04.572 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:35:04.572 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:35:04.572 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.572 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:04.573 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41789sec preferred_lft 41789sec 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:35:04.574 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:35:04.575 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:04.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:04.576 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:35:04.632 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:35:04.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:04.635 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:35:04.689 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:35:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:35:04.690 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:35:04.691 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:35:04.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:35:04.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:35:04.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:35:04.691 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:35:04.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:35:04.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:35:04.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:04.823 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:35:04.833 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:35:04.833 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:35:04.833 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:35:04.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:04.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:35:04.894 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:35:04.895 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:04.943 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:35:04.944 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:35:04.944 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:04.994 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:35:04.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:04.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:05.082 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:05.082 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:35:05.082 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:35:05.097 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:05.097 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:35:05.097 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:35:05.097 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:35:05.098 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:35:05.107 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:05.108 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:35:05.108 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:35:05.108 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:35:05.108 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:35:05.108 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:35:05.108 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:35:05.108 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:35:05.112 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:35:05.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:05.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:05.131 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:35:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:35:05.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:35:05.230 INFO:teuthology.orchestra.run.smithi097.stdout:client9358 2024-09-10T21:35:05.230 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:05.230 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:05.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:05.239 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T21:35:05.285 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T21:35:05.293 INFO:teuthology.orchestra.run.smithi097.stdout:.: 2024-09-10T21:35:05.293 INFO:teuthology.orchestra.run.smithi097.stdout:subdir 2024-09-10T21:35:05.293 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:05.294 INFO:teuthology.orchestra.run.smithi097.stdout:./subdir: 2024-09-10T21:35:05.294 INFO:teuthology.orchestra.run.smithi097.stdout:sixmegs 2024-09-10T21:35:05.294 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2024-09-10T21:35:05.294 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2024-09-10T21:35:05.295 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:05.295 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:05.347 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:05.347 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:05.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:35:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 -- 172.21.15.97:0/223340621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c1511f0 msgr2=0x7eff2c0b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 --2- 172.21.15.97:0/223340621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c1511f0 0x7eff2c0b37b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7eff2000aa10 tx=0x7eff200203e0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 -- 172.21.15.97:0/223340621 shutdown_connections 2024-09-10T21:35:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 --2- 172.21.15.97:0/223340621 >> v2:172.21.15.194:3300/0 conn(0x7eff2c0ceee0 0x7eff2c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 --2- 172.21.15.97:0/223340621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c1511f0 0x7eff2c0b37b0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 -- 172.21.15.97:0/223340621 >> 172.21.15.97:0/223340621 conn(0x7eff2c0b1780 msgr2=0x7eff2c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 -- 172.21.15.97:0/223340621 shutdown_connections 2024-09-10T21:35:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 -- 172.21.15.97:0/223340621 wait complete. 2024-09-10T21:35:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 Processor -- start 2024-09-10T21:35:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.590+0000 7eff34600640 1 -- start start 2024-09-10T21:35:05.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c0b3cf0 0x7eff2c13f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:05.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff2c0ceee0 0x7eff2c1e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:05.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff2c1511f0 0x7eff2c1e4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:05.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c06e740 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c06e5c0 con 0x7eff2c0ceee0 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c06e8c0 con 0x7eff2c1511f0 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c0b3cf0 0x7eff2c13f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c0b3cf0 0x7eff2c13f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45590/0 (socket says 172.21.15.97:45590) 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 -- 172.21.15.97:0/2912657566 learned_addr learned my addr 172.21.15.97:0/2912657566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff32600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff2c1511f0 0x7eff2c1e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31200640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff2c0ceee0 0x7eff2c1e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 -- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff2c1511f0 msgr2=0x7eff2c1e4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:05.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff2c1511f0 0x7eff2c1e4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 -- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff2c0ceee0 msgr2=0x7eff2c1e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff2c0ceee0 0x7eff2c1e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 -- 172.21.15.97:0/2912657566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c15c020 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff32600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff2c1511f0 0x7eff2c1e4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31200640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff2c0ceee0 0x7eff2c1e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff31c00640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c0b3cf0 0x7eff2c13f0d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7eff2000a960 tx=0x7eff200b0400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff29600640 1 -- 172.21.15.97:0/2912657566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200b5040 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff29600640 1 -- 172.21.15.97:0/2912657566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff200b0cd0 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff29600640 1 -- 172.21.15.97:0/2912657566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20005280 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c1f8fa0 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1f5220 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c0cf350 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.594+0000 7eff29600640 1 -- 172.21.15.97:0/2912657566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff200b8070 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.598+0000 7eff29600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff0c0725e0 0x7eff0c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.598+0000 7eff29600640 1 -- 172.21.15.97:0/2912657566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7eff20131790 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.598+0000 7eff31200640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff0c0725e0 0x7eff0c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:05.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.598+0000 7eff31200640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff0c0725e0 0x7eff0c074aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1e7ce0 tx=0x7eff18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:05.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.602+0000 7eff29600640 1 -- 172.21.15.97:0/2912657566 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff20019f40 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.778+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff2c1e52b0 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.778+0000 7eff29600640 1 -- 172.21.15.97:0/2912657566 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 72 entries v132) ==== 81+0+4376 (secure 0 0 0) 0x7eff200fc000 con 0x7eff2c0b3cf0 2024-09-10T21:35:05.778 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3083575804 2024-09-11T21:34:46.104147+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1998443497 2024-09-11T21:34:35.158201+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1087733461 2024-09-11T21:34:19.533818+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1087733461 2024-09-11T21:34:19.533818+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2872418710 2024-09-11T21:34:14.052935+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2750213748 2024-09-11T21:34:00.427920+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:35:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/923451308 2024-09-11T21:32:09.002263+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3289603806 2024-09-11T21:34:51.343028+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:35:05.780 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2872418710 2024-09-11T21:34:14.052935+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2243090003 2024-09-11T21:32:28.378125+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3083575804 2024-09-11T21:34:46.104147+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1998443497 2024-09-11T21:34:35.158201+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/170708142 2024-09-11T21:32:30.659200+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2364393791 2024-09-11T21:32:02.967748+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2383941694 2024-09-11T21:31:49.749927+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2415223919 2024-09-11T21:32:37.700495+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/873879253 2024-09-11T21:30:50.627295+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2192988988 2024-09-11T21:31:11.752953+0000 2024-09-10T21:35:05.781 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/507601338 2024-09-11T21:33:33.124771+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2702314481 2024-09-11T21:31:51.627563+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2521509514 2024-09-11T21:31:17.802510+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/769007942 2024-09-11T21:31:21.620967+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/895133479 2024-09-11T21:31:25.859540+0000 2024-09-10T21:35:05.782 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2013408839 2024-09-11T21:31:38.828816+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3648960428 2024-09-11T21:32:43.749260+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/1863388792 2024-09-11T21:32:47.786444+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/733923445 2024-09-11T21:33:07.903840+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2841033666 2024-09-11T21:31:55.916991+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3111622295 2024-09-11T21:33:05.844372+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2402514654 2024-09-11T21:33:11.941224+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:35:05.783 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/3289603806 2024-09-11T21:34:51.343028+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3203242917 2024-09-11T21:33:15.973352+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/1929373906 2024-09-11T21:33:20.014467+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/3971758099 2024-09-11T21:31:06.554833+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/2194082021 2024-09-11T21:32:51.642298+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6832/3576291769 2024-09-11T21:33:25.065757+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6834/2561173061 2024-09-11T21:33:52.323112+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/899393328 2024-09-11T21:33:29.083390+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/2264649568 2024-09-11T21:33:46.168737+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6835/4017597023 2024-09-11T21:33:48.304066+0000 2024-09-10T21:35:05.784 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6833/3334191869 2024-09-11T21:33:56.361451+0000 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:listed 72 entries 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff0c0725e0 msgr2=0x7eff0c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff0c0725e0 0x7eff0c074aa0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1e7ce0 tx=0x7eff18008040 comp rx=0 tx=0).stop 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c0b3cf0 msgr2=0x7eff2c13f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c0b3cf0 0x7eff2c13f0d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7eff2000a960 tx=0x7eff200b0400 comp rx=0 tx=0).stop 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 shutdown_connections 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff0c0725e0 0x7eff0c074aa0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff2c1511f0 0x7eff2c1e4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff2c0ceee0 0x7eff2c1e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 --2- 172.21.15.97:0/2912657566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff2c0b3cf0 0x7eff2c13f0d0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 >> 172.21.15.97:0/2912657566 conn(0x7eff2c0b1780 msgr2=0x7eff2c0b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 shutdown_connections 2024-09-10T21:35:05.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:05.782+0000 7eff34600640 1 -- 172.21.15.97:0/2912657566 wait complete. 2024-09-10T21:35:05.799 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:35:05.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:05.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:35:05.822 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:35:05.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:05.823 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:35:05.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:35:06.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 -- 172.21.15.97:0/1913874548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c144690 msgr2=0x7f2b8c0b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 --2- 172.21.15.97:0/1913874548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c144690 0x7f2b8c0b4120 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009900 tx=0x7f2b7801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 -- 172.21.15.97:0/1913874548 shutdown_connections 2024-09-10T21:35:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 --2- 172.21.15.97:0/1913874548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c144690 0x7f2b8c0b4120 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 --2- 172.21.15.97:0/1913874548 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b8c143f90 0x7f2b8c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 -- 172.21.15.97:0/1913874548 >> 172.21.15.97:0/1913874548 conn(0x7f2b8c0b1780 msgr2=0x7f2b8c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 -- 172.21.15.97:0/1913874548 shutdown_connections 2024-09-10T21:35:06.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 -- 172.21.15.97:0/1913874548 wait complete. 2024-09-10T21:35:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 Processor -- start 2024-09-10T21:35:06.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 -- start start 2024-09-10T21:35:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c0b4660 0x7f2b8c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.058+0000 7f2b91400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b8c143f90 0x7f2b8c13cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b91400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b8c144690 0x7f2b8c13d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b91400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c06efd0 con 0x7f2b8c0b4660 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b91400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c06ee50 con 0x7f2b8c144690 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b91400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c06f150 con 0x7f2b8c143f90 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c0b4660 0x7f2b8c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c0b4660 0x7f2b8c1598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45602/0 (socket says 172.21.15.97:45602) 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 -- 172.21.15.97:0/4136827395 learned_addr learned my addr 172.21.15.97:0/4136827395 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8b200640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b8c144690 0x7f2b8c13d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b89e00640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b8c143f90 0x7f2b8c13cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 -- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b8c143f90 msgr2=0x7f2b8c13cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b8c143f90 0x7f2b8c13cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 -- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b8c144690 msgr2=0x7f2b8c13d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b8c144690 0x7f2b8c13d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 -- 172.21.15.97:0/4136827395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c159df0 con 0x7f2b8c0b4660 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8b200640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b8c144690 0x7f2b8c13d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b89e00640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b8c143f90 0x7f2b8c13cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b8a800640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c0b4660 0x7f2b8c1598b0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00bfd0 tx=0x7f2b7c00ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b82a00640 1 -- 172.21.15.97:0/4136827395 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c019070 con 0x7f2b8c0b4660 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b82a00640 1 -- 172.21.15.97:0/4136827395 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b7c007a70 con 0x7f2b8c0b4660 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c135990 con 0x7f2b8c0b4660 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b82a00640 1 -- 172.21.15.97:0/4136827395 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c014410 con 0x7f2b8c0b4660 2024-09-10T21:35:06.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c135f50 con 0x7f2b8c0b4660 2024-09-10T21:35:06.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c0b2a80 con 0x7f2b8c0b4660 2024-09-10T21:35:06.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.062+0000 7f2b82a00640 1 -- 172.21.15.97:0/4136827395 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b7c0145b0 con 0x7f2b8c0b4660 2024-09-10T21:35:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.066+0000 7f2b82a00640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b6c0725e0 0x7f2b6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.066+0000 7f2b82a00640 1 -- 172.21.15.97:0/4136827395 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9371+0+0 (secure 0 0 0) 0x7f2b7c093bf0 con 0x7f2b8c0b4660 2024-09-10T21:35:06.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.066+0000 7f2b89e00640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b6c0725e0 0x7f2b6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.066+0000 7f2b89e00640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b6c0725e0 0x7f2b6c074aa0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2b780af000 tx=0x7f2b7801b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:06.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.070+0000 7f2b82a00640 1 -- 172.21.15.97:0/4136827395 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b7c05de70 con 0x7f2b8c0b4660 2024-09-10T21:35:06.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.266+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2b8c0ce050 con 0x7f2b8c0b4660 2024-09-10T21:35:06.266 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:06.266+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:35:06.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.270+0000 7f2b82a00640 1 -- 172.21.15.97:0/4136827395 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v314) ==== 161+0+0 (secure 0 0 0) 0x7f2b7c0628a0 con 0x7f2b8c0b4660 2024-09-10T21:35:06.269 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.270+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b6c0725e0 msgr2=0x7f2b6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b6c0725e0 0x7f2b6c074aa0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2b780af000 tx=0x7f2b7801b120 comp rx=0 tx=0).stop 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c0b4660 msgr2=0x7f2b8c1598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c0b4660 0x7f2b8c1598b0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00bfd0 tx=0x7f2b7c00ae10 comp rx=0 tx=0).stop 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 shutdown_connections 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b6c0725e0 0x7f2b6c074aa0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b8c144690 0x7f2b8c13d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b8c143f90 0x7f2b8c13cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 --2- 172.21.15.97:0/4136827395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8c0b4660 0x7f2b8c1598b0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 >> 172.21.15.97:0/4136827395 conn(0x7f2b8c0b1780 msgr2=0x7f2b8c073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:06.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 shutdown_connections 2024-09-10T21:35:06.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.274+0000 7f2b91400640 1 -- 172.21.15.97:0/4136827395 wait complete. 2024-09-10T21:35:06.274 INFO:tasks.ceph.mds.a.smithi097.stderr: -1885> 2024-09-10T21:34:57.390+0000 7fbca0e00640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-10T21:35:06.276 INFO:tasks.ceph.mds.a.smithi097.stderr: -685> 2024-09-10T21:35:01.378+0000 7fbca5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:06.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:35:06.318 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:35:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- 172.21.15.97:0/3836120132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840b0d50 msgr2=0x7f09840ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 --2- 172.21.15.97:0/3836120132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840b0d50 0x7f09840ccc10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f097c009bc0 tx=0x7f097c01cba0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- 172.21.15.97:0/3836120132 shutdown_connections 2024-09-10T21:35:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 --2- 172.21.15.97:0/3836120132 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840b0d50 0x7f09840ccc10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 --2- 172.21.15.97:0/3836120132 >> v2:172.21.15.194:3300/0 conn(0x7f09840afdd0 0x7f09840cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- 172.21.15.97:0/3836120132 >> 172.21.15.97:0/3836120132 conn(0x7f09840d1c30 msgr2=0x7f09840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- 172.21.15.97:0/3836120132 shutdown_connections 2024-09-10T21:35:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- 172.21.15.97:0/3836120132 wait complete. 2024-09-10T21:35:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 Processor -- start 2024-09-10T21:35:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- start start 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09840afdd0 0x7f0984165db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09840b0d50 0x7f0984157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840ce640 0x7f098406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0984168e30 con 0x7f09840afdd0 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0984168cb0 con 0x7f09840b0d50 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0989a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0984168fb0 con 0x7f09840ce640 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0982a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09840afdd0 0x7f0984165db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0982a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09840afdd0 0x7f0984165db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45624/0 (socket says 172.21.15.97:45624) 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.442+0000 7f0982a00640 1 -- 172.21.15.97:0/1734098962 learned_addr learned my addr 172.21.15.97:0/1734098962 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0983400640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840ce640 0x7f098406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982000640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09840b0d50 0x7f0984157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982a00640 1 -- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840ce640 msgr2=0x7f098406dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840ce640 0x7f098406dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982a00640 1 -- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09840b0d50 msgr2=0x7f0984157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09840b0d50 0x7f0984157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982a00640 1 -- 172.21.15.97:0/1734098962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09840c6be0 con 0x7f09840afdd0 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982000640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09840b0d50 0x7f0984157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0982a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09840afdd0 0x7f0984165db0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f097400c430 tx=0x7f097400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f097aa00640 1 -- 172.21.15.97:0/1734098962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974011790 con 0x7f09840afdd0 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f097aa00640 1 -- 172.21.15.97:0/1734098962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0974017070 con 0x7f09840afdd0 2024-09-10T21:35:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f097aa00640 1 -- 172.21.15.97:0/1734098962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974011ce0 con 0x7f09840afdd0 2024-09-10T21:35:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09840c6ec0 con 0x7f09840afdd0 2024-09-10T21:35:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09840c7480 con 0x7f09840afdd0 2024-09-10T21:35:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09840cea20 con 0x7f09840afdd0 2024-09-10T21:35:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f097aa00640 1 -- 172.21.15.97:0/1734098962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f097401c460 con 0x7f09840afdd0 2024-09-10T21:35:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.446+0000 7f097aa00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09640725e0 0x7f0964074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.450+0000 7f097aa00640 1 -- 172.21.15.97:0/1734098962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9457+0+0 (secure 0 0 0) 0x7f0974093e80 con 0x7f09840afdd0 2024-09-10T21:35:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.450+0000 7f0982000640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09640725e0 0x7f0964074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:06.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.450+0000 7f0982000640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09640725e0 0x7f0964074aa0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f098415ae90 tx=0x7f09700091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.450+0000 7f097aa00640 1 -- 172.21.15.97:0/1734098962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f097405e6c0 con 0x7f09840afdd0 2024-09-10T21:35:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:06.678+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f09840d04f0 con 0x7f09840afdd0 2024-09-10T21:35:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.270+0000 7f097aa00640 1 -- 172.21.15.97:0/1734098962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v315) ==== 107+0+0 (secure 0 0 0) 0x7f097405dfc0 con 0x7f09840afdd0 2024-09-10T21:35:07.271 INFO:teuthology.orchestra.run.smithi097.stderr:nothing to do: rank is not damaged 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09640725e0 msgr2=0x7f0964074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09640725e0 0x7f0964074aa0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f098415ae90 tx=0x7f09700091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09840afdd0 msgr2=0x7f0984165db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09840afdd0 0x7f0984165db0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f097400c430 tx=0x7f097400c8f0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 shutdown_connections 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f09640725e0 0x7f0964074aa0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f09840ce640 0x7f098406dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09840b0d50 0x7f0984157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 --2- 172.21.15.97:0/1734098962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09840afdd0 0x7f0984165db0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 >> 172.21.15.97:0/1734098962 conn(0x7f09840d1c30 msgr2=0x7f09840d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 shutdown_connections 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.274+0000 7f0989a00640 1 -- 172.21.15.97:0/1734098962 wait complete. 2024-09-10T21:35:07.297 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:35:07.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 Processor -- start 2024-09-10T21:35:07.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 -- start start 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbea5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2bb00 0x558bfbf4b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2c2a0 0x558bfbe6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558bfbdc0a00 con 0x558bfbea1940 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558bfbe1ef40 con 0x558bfbf2bb00 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe962b59080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558bfbf2e080 con 0x558bfbf2c2a0 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbea5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbea5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39432/0 (socket says 172.21.15.97:39432) 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 -- 172.21.15.97:0/165179010 learned_addr learned my addr 172.21.15.97:0/165179010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95f400640 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2bb00 0x558bfbf4b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe960800640 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2c2a0 0x558bfbe6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 -- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2c2a0 msgr2=0x558bfbe6fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2c2a0 0x558bfbe6fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 -- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2bb00 msgr2=0x558bfbf4b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2bb00 0x558bfbf4b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 -- 172.21.15.97:0/165179010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bfbd99c10 con 0x558bfbea1940 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe960800640 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2c2a0 0x558bfbe6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95fe00640 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbea5420 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe95000a820 tx=0x7fe95000ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533698eefe0bde0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95e000640 1 -- 172.21.15.97:0/165179010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95000cac0 con 0x558bfbea1940 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95e000640 1 -- 172.21.15.97:0/165179010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe95000cd50 con 0x558bfbea1940 2024-09-10T21:35:07.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.386+0000 7fe95e000640 1 -- 172.21.15.97:0/165179010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95000f6c0 con 0x558bfbea1940 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- 172.21.15.97:0/165179010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 msgr2=0x558bfbea5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbea5420 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe95000a820 tx=0x7fe95000ace0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- 172.21.15.97:0/165179010 shutdown_connections 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2c2a0 0x558bfbe6fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2bb00 0x558bfbf4b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 --2- 172.21.15.97:0/165179010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbea5420 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- 172.21.15.97:0/165179010 >> 172.21.15.97:0/165179010 conn(0x558bfbea0520 msgr2=0x558bfbf481e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- 172.21.15.97:0/165179010 shutdown_connections 2024-09-10T21:35:07.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- 172.21.15.97:0/165179010 wait complete. 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 Processor -- start 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- start start 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbf440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2bb00 0x558bfbf2ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2c2a0 0x558bfbf2aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558bfbf44c50 con 0x558bfbea1940 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558bfbf449b0 con 0x558bfbf2c2a0 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x558bfbea30b0 con 0x558bfbf2bb00 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbf440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbf440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39448/0 (socket says 172.21.15.97:39448) 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 learned_addr learned my addr 172.21.15.97:0/1201937952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:07.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95f400640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2bb00 0x558bfbf2ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe960800640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2c2a0 0x558bfbf2aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2bb00 msgr2=0x558bfbf2ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x558bfbf2bb00 0x558bfbf2ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2c2a0 msgr2=0x558bfbf2aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bfbf2c2a0 0x558bfbf2aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bfbf30730 con 0x558bfbea1940 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95fe00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbf440f0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fe95000fa60 tx=0x7fe95000fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95cc00640 1 -- 172.21.15.97:0/1201937952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950025070 con 0x558bfbea1940 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95cc00640 1 -- 172.21.15.97:0/1201937952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe95000fe40 con 0x558bfbea1940 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe95cc00640 1 -- 172.21.15.97:0/1201937952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950020670 con 0x558bfbea1940 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558bfbf309d0 con 0x558bfbea1940 2024-09-10T21:35:07.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.390+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bfbf30f10 con 0x558bfbea1940 2024-09-10T21:35:07.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe95cc00640 1 -- 172.21.15.97:0/1201937952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe950017400 con 0x558bfbea1940 2024-09-10T21:35:07.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe95cc00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe940070ff0 0x7fe9400734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe95cc00640 1 -- 172.21.15.97:0/1201937952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9457+0+0 (secure 0 0 0) 0x7fe950094980 con 0x558bfbea1940 2024-09-10T21:35:07.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe95f400640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe940070ff0 0x7fe9400734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.395 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:35:07.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x558bfbf31650 0x558bfbf31a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbf31f70 con 0x558bfbf31650 2024-09-10T21:35:07.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe960800640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x558bfbf31650 0x558bfbf31a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.394+0000 7fe95f400640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe940070ff0 0x7fe9400734b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0ab000 tx=0x7fe94c0ad040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.398+0000 7fe960800640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x558bfbf31650 0x558bfbf31a30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.398+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v133'1128 uv1102 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.398+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbf32340 con 0x558bfbf31650 2024-09-10T21:35:07.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.402+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v133'1129 uv1129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.402+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbf326e0 con 0x558bfbf31650 2024-09-10T21:35:07.402 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:35:07.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.402+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v133'1130 uv1130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.402+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef920 con 0x558bfbf31650 2024-09-10T21:35:07.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.406+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v133'1131 uv1131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.406+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfefcc0 con 0x558bfbf31650 2024-09-10T21:35:07.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.406+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v133'1132 uv1132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.406+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfee910 con 0x558bfbf31650 2024-09-10T21:35:07.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.410+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v133'1133 uv1133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.410+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfeed30 con 0x558bfbf31650 2024-09-10T21:35:07.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.410+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v133'1134 uv1108 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.410+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.414+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v133'1135 uv1135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.414+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.414+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v133'1136 uv1136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.413 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:35:07.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.414+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.414+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v133'1137 uv1137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.414+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.418+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v133'1138 uv1138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.418+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.418+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v133'1139 uv1139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.418+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.422+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v133'1140 uv1115 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.422+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.422+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v133'1141 uv1141 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.422+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.423 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:35:07.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.426+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v133'1142 uv1142 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.426+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.426+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v133'1143 uv1143 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.426+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.426+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v133'1144 uv1144 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.426+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef120 con 0x558bfbf31650 2024-09-10T21:35:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.430+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v133'1145 uv1145 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.430+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef510 con 0x558bfbf31650 2024-09-10T21:35:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.430+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v133'1146 uv1146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.430+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.434+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v133'1147 uv1121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.434+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.434+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v133'1148 uv1148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.434+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.438+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v133'1149 uv1149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.436 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:35:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.438+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.438+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v133'1150 uv1150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.438+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.442+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v133'1151 uv1151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.442+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.442+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v133'1152 uv1152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.442+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.442+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v133'1153 uv1127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.442+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.446+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v133'1154 uv1154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.446+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.446+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v133'1155 uv1155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.446 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:35:07.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.446+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.450+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v133'1156 uv1156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.450+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.450+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v133'1157 uv1157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.450+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbf31650 2024-09-10T21:35:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.450+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v133'1158 uv1158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbf31650 2024-09-10T21:35:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.450+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558bfbff0570 0x558bfbff2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.450+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbfef3a0 con 0x558bfbff0570 2024-09-10T21:35:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.454+0000 7fe95fe00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558bfbff0570 0x558bfbff2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.454+0000 7fe95fe00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558bfbff0570 0x558bfbff2960 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.454+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v133'824 uv803 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.454+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbff2ea0 con 0x558bfbff0570 2024-09-10T21:35:07.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.458+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v133'825 uv825 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.458+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff3260 con 0x558bfbff0570 2024-09-10T21:35:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.458+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v133'826 uv826 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.457 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:35:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.458+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff3b20 con 0x558bfbff0570 2024-09-10T21:35:07.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.458+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v133'827 uv827 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.458+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff3ec0 con 0x558bfbff0570 2024-09-10T21:35:07.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.462+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v133'828 uv828 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.462+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff42b0 con 0x558bfbff0570 2024-09-10T21:35:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.462+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v133'829 uv822 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.462+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbff4650 con 0x558bfbff0570 2024-09-10T21:35:07.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.462+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v133'830 uv830 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.462+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff4a40 con 0x558bfbff0570 2024-09-10T21:35:07.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.466+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v133'831 uv831 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.464 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:35:07.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.466+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff4a40 con 0x558bfbff0570 2024-09-10T21:35:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.466+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v133'832 uv832 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.466+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.466+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v133'833 uv833 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.466+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.470+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v133'834 uv823 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.470+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.470+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v133'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.470+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.470+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v133'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.471 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:35:07.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.470+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.474+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v133'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.474+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.478+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v133'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.478+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.478+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v133'839 uv819 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.478+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.478+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v133'840 uv840 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.482+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff49f0 con 0x558bfbff0570 2024-09-10T21:35:07.480 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:35:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.482+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v133'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.482+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff4a10 con 0x558bfbff0570 2024-09-10T21:35:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.482+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v133'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.482+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff4a10 con 0x558bfbff0570 2024-09-10T21:35:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.482+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v133'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.482+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff4a10 con 0x558bfbff0570 2024-09-10T21:35:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.486+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v133'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbff0570 2024-09-10T21:35:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.486+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x558bfbff50e0 0x558bfbff54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.486+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff7ee0 con 0x558bfbff50e0 2024-09-10T21:35:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.486+0000 7fe95f400640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x558bfbff50e0 0x558bfbff54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.486+0000 7fe95f400640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x558bfbff50e0 0x558bfbff54e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.486+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.486+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbff8280 con 0x558bfbff50e0 2024-09-10T21:35:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.490+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.490+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff8620 con 0x558bfbff50e0 2024-09-10T21:35:07.489 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:35:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.490+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.490+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff9e80 con 0x558bfbff50e0 2024-09-10T21:35:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.490+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.490+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffa220 con 0x558bfbff50e0 2024-09-10T21:35:07.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.494+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.494+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff8e70 con 0x558bfbff50e0 2024-09-10T21:35:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.494+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.494+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x558bfbff9770 0x558bfbffc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.494+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffcb10 con 0x558bfbff9770 2024-09-10T21:35:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.494+0000 7fe960800640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x558bfbff9770 0x558bfbffc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.494+0000 7fe960800640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x558bfbff9770 0x558bfbffc5d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v133'168 uv167 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbffcb10 con 0x558bfbff9770 2024-09-10T21:35:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v133'169 uv169 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffcb90 con 0x558bfbff9770 2024-09-10T21:35:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v133'170 uv170 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.498 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:35:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbff8ac0 con 0x558bfbff9770 2024-09-10T21:35:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v133'171 uv171 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.498+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd100 con 0x558bfbff50e0 2024-09-10T21:35:07.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.502+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.502+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbffd100 con 0x558bfbff50e0 2024-09-10T21:35:07.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.502+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.502+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd100 con 0x558bfbff50e0 2024-09-10T21:35:07.502 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:35:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.502+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.502+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffe150 con 0x558bfbff50e0 2024-09-10T21:35:07.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.506+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.506+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffe110 con 0x558bfbff50e0 2024-09-10T21:35:07.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.506+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.506+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd100 con 0x558bfbff50e0 2024-09-10T21:35:07.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.506+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.506+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd100 con 0x558bfbff9770 2024-09-10T21:35:07.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.510+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v133'404 uv398 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.510+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbffd100 con 0x558bfbff9770 2024-09-10T21:35:07.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.510+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v133'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.510+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff9770 2024-09-10T21:35:07.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.510+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v133'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.510 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:35:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.510+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffe0c0 con 0x558bfbff9770 2024-09-10T21:35:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v133'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffe0c0 con 0x558bfbff9770 2024-09-10T21:35:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v133'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff9770 2024-09-10T21:35:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v133'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff9770 2024-09-10T21:35:07.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v133'410 uv403 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.514+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff9770 2024-09-10T21:35:07.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.518+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v133'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.518+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff9770 2024-09-10T21:35:07.517 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:35:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.518+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v133'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.518+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff9770 2024-09-10T21:35:07.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.518+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v133'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.518+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff9770 2024-09-10T21:35:07.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v133'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff50e0 2024-09-10T21:35:07.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v133'183 uv182 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff50e0 2024-09-10T21:35:07.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v133'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff50e0 2024-09-10T21:35:07.523 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:35:07.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v133'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.522+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff50e0 2024-09-10T21:35:07.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.526+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v133'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.526+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfbffd0b0 con 0x558bfbff50e0 2024-09-10T21:35:07.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.526+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v133'187 uv187 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfbff50e0 2024-09-10T21:35:07.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.526+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558bfbffda20 0x558bfbffde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.526+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc0005c0 con 0x558bfbffda20 2024-09-10T21:35:07.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.526+0000 7fe95fe00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558bfbffda20 0x558bfbffde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.526+0000 7fe95fe00640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558bfbffda20 0x558bfbffde40 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.530+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.530+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc0005c0 con 0x558bfbffda20 2024-09-10T21:35:07.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.530+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.530+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc0005c0 con 0x558bfbffda20 2024-09-10T21:35:07.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.530+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.530 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:35:07.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.530+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc0019f0 con 0x558bfbffda20 2024-09-10T21:35:07.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.534+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.534+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc0019f0 con 0x558bfbffda20 2024-09-10T21:35:07.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.534+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.534+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000730 con 0x558bfbffda20 2024-09-10T21:35:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.534+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.534+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000730 con 0x558bfbffda20 2024-09-10T21:35:07.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.538+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v133'365 uv359 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.538+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc0006b0 con 0x558bfbffda20 2024-09-10T21:35:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.538+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v133'366 uv366 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.538+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc0006b0 con 0x558bfbffda20 2024-09-10T21:35:07.538 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:35:07.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.538+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v133'367 uv367 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.538+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000af0 con 0x558bfbffda20 2024-09-10T21:35:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v133'368 uv368 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000af0 con 0x558bfbffda20 2024-09-10T21:35:07.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v133'369 uv369 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc0007a0 con 0x558bfbffda20 2024-09-10T21:35:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v133'370 uv364 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc000750 con 0x558bfbffda20 2024-09-10T21:35:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v133'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.542+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000750 con 0x558bfbffda20 2024-09-10T21:35:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.546+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v133'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.546 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:35:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.546+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000750 con 0x558bfbffda20 2024-09-10T21:35:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.546+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v133'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.546+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc0035a0 con 0x558bfbffda20 2024-09-10T21:35:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.546+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v133'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.546+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.550+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v133'162 uv160 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.550+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.550+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v133'163 uv163 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.550+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.550 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:35:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.550+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v133'164 uv164 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.550+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.554+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v133'165 uv165 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.554+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.554+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v133'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.554+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.554+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v133'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.554+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.555 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:35:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v133'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v133'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc001c10 con 0x558bfbff9770 2024-09-10T21:35:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v133'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc000c00 con 0x558bfbff9770 2024-09-10T21:35:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v133'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe95800e490 con 0x558bfbff9770 2024-09-10T21:35:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x558bfc001280 0x558bfc001680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.558+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc001bc0 con 0x558bfc001280 2024-09-10T21:35:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.562+0000 7fe95f400640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x558bfc001280 0x558bfc001680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.562+0000 7fe95f400640 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x558bfc001280 0x558bfc001680 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.562+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v133'186 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfc001280 2024-09-10T21:35:07.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.562+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc001bc0 con 0x558bfc001280 2024-09-10T21:35:07.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.562+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v133'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfc001280 2024-09-10T21:35:07.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.562+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc001bc0 con 0x558bfc001280 2024-09-10T21:35:07.563 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:35:07.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.566+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v133'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfc001280 2024-09-10T21:35:07.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.566+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc001bc0 con 0x558bfc001280 2024-09-10T21:35:07.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.566+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v133'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfc001280 2024-09-10T21:35:07.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.566+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfc001280 2024-09-10T21:35:07.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.566+0000 7fe95f400640 1 -- 172.21.15.97:0/1201937952 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v133'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe94c0b3050 con 0x558bfc001280 2024-09-10T21:35:07.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.566+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v133'455 uv446 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v133'456 uv456 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v133'457 uv457 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.570 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:35:07.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v133'458 uv458 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.570+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.574+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v133'459 uv459 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.574+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.574+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v133'460 uv453 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.574+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.574+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v133'461 uv461 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.574+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v133'462 uv462 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.576 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:35:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc002050 con 0x558bfbffda20 2024-09-10T21:35:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v133'463 uv463 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc003060 con 0x558bfbffda20 2024-09-10T21:35:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v133'464 uv464 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc003450 con 0x558bfbffda20 2024-09-10T21:35:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v133'465 uv465 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.578+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x558bfc003450 con 0x558bfbffda20 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe95fe00640 1 -- 172.21.15.97:0/1201937952 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v133'466 uv466 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe950027030 con 0x558bfbffda20 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x558bfbff50e0 msgr2=0x558bfbff54e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x558bfbff50e0 0x558bfbff54e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558bfbffda20 msgr2=0x558bfbffde40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x558bfbffda20 0x558bfbffde40 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x558bfc001280 msgr2=0x558bfc001680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x558bfc001280 0x558bfc001680 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x558bfbff9770 msgr2=0x558bfbffc5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x558bfbff9770 0x558bfbffc5d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 reap_dead start 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558bfbff0570 msgr2=0x558bfbff2960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x558bfbff0570 0x558bfbff2960 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x558bfbf31650 msgr2=0x558bfbf31a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x558bfbf31650 0x558bfbf31a30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe960800640 1 -- 172.21.15.97:0/1201937952 reap_dead start 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe940070ff0 msgr2=0x7fe9400734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe940070ff0 0x7fe9400734b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0ab000 tx=0x7fe94c0ad040 comp rx=0 tx=0).stop 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 msgr2=0x558bfbf440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbf440f0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fe95000fa60 tx=0x7fe95000fa90 comp rx=0 tx=0).stop 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 shutdown_connections 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x558bfbf31650 0x558bfbf31a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe940070ff0 0x7fe9400734b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 --2- 172.21.15.97:0/1201937952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bfbea1940 0x558bfbf440f0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 >> 172.21.15.97:0/1201937952 conn(0x558bfbea0520 msgr2=0x558bfbf482c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 shutdown_connections 2024-09-10T21:35:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.582+0000 7fe962b59080 1 -- 172.21.15.97:0/1201937952 wait complete. 2024-09-10T21:35:07.586 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:35:07.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 Processor -- start 2024-09-10T21:35:07.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- start start 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c039cc0 0x55dd0c055410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03ba00 0x55dd0c055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c03f1d0 0x55dd0c056430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dd0becfa00 con 0x55dd0c039cc0 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dd0bf2df40 con 0x55dd0c03ba00 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55dd0c03d710 con 0x55dd0c03f1d0 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c039cc0 0x55dd0c055410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c039cc0 0x55dd0c055410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39462/0 (socket says 172.21.15.97:39462) 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 -- 172.21.15.97:0/1689070221 learned_addr learned my addr 172.21.15.97:0/1689070221 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:07.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62a00640 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c03f1d0 0x55dd0c056430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a61600640 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03ba00 0x55dd0c055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 -- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c03f1d0 msgr2=0x55dd0c056430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c03f1d0 0x55dd0c056430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 -- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03ba00 msgr2=0x55dd0c055ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03ba00 0x55dd0c055ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 -- 172.21.15.97:0/1689070221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0bea8c10 con 0x55dd0c039cc0 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a62000640 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c039cc0 0x55dd0c055410 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400b2b0 tx=0x7f8a5400b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff065767f69bca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a5be00640 1 -- 172.21.15.97:0/1689070221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54016070 con 0x55dd0c039cc0 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a5be00640 1 -- 172.21.15.97:0/1689070221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a5400d4c0 con 0x55dd0c039cc0 2024-09-10T21:35:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a5be00640 1 -- 172.21.15.97:0/1689070221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400d730 con 0x55dd0c039cc0 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- 172.21.15.97:0/1689070221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c039cc0 msgr2=0x55dd0c055410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c039cc0 0x55dd0c055410 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400b2b0 tx=0x7f8a5400b770 comp rx=0 tx=0).stop 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- 172.21.15.97:0/1689070221 shutdown_connections 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c03f1d0 0x55dd0c056430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03ba00 0x55dd0c055ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 --2- 172.21.15.97:0/1689070221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c039cc0 0x55dd0c055410 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- 172.21.15.97:0/1689070221 >> 172.21.15.97:0/1689070221 conn(0x55dd0bfc90d0 msgr2=0x55dd0bf84360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- 172.21.15.97:0/1689070221 shutdown_connections 2024-09-10T21:35:07.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.634+0000 7f8a64e94080 1 -- 172.21.15.97:0/1689070221 wait complete. 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 Processor -- start 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 -- start start 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c039cc0 0x55dd0c056400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c03ba00 0x55dd0c0537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03f1d0 0x55dd0c058330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dd0c038a70 con 0x55dd0c03ba00 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dd0bfcb2d0 con 0x55dd0c03f1d0 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55dd0bfc2a50 con 0x55dd0c039cc0 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c03ba00 0x55dd0c0537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a62000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c039cc0 0x55dd0c056400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c03ba00 0x55dd0c0537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39470/0 (socket says 172.21.15.97:39470) 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a62000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c039cc0 0x55dd0c056400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3301/0 says I am v2:172.21.15.97:47960/0 (socket says 172.21.15.97:47960) 2024-09-10T21:35:07.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 -- 172.21.15.97:0/2228079428 learned_addr learned my addr 172.21.15.97:0/2228079428 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a62a00640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03f1d0 0x55dd0c058330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 -- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c039cc0 msgr2=0x55dd0c056400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55dd0c039cc0 0x55dd0c056400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 -- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03f1d0 msgr2=0x55dd0c058330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dd0c03f1d0 0x55dd0c058330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 -- 172.21.15.97:0/2228079428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0bfc2c80 con 0x55dd0c03ba00 2024-09-10T21:35:07.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a61600640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c03ba00 0x55dd0c0537c0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000ad30 tx=0x7f8a50008790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a5aa00640 1 -- 172.21.15.97:0/2228079428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50020070 con 0x55dd0c03ba00 2024-09-10T21:35:07.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a5aa00640 1 -- 172.21.15.97:0/2228079428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a50006750 con 0x55dd0c03ba00 2024-09-10T21:35:07.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a5aa00640 1 -- 172.21.15.97:0/2228079428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50006b40 con 0x55dd0c03ba00 2024-09-10T21:35:07.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dd0bfc2f20 con 0x55dd0c03ba00 2024-09-10T21:35:07.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.638+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0bfc3460 con 0x55dd0c03ba00 2024-09-10T21:35:07.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a5aa00640 1 -- 172.21.15.97:0/2228079428 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8a50006ce0 con 0x55dd0c03ba00 2024-09-10T21:35:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a5aa00640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a44070ff0 0x7f8a440734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a5aa00640 1 -- 172.21.15.97:0/2228079428 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9457+0+0 (secure 0 0 0) 0x7f8a50097940 con 0x55dd0c03ba00 2024-09-10T21:35:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a62000640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a44070ff0 0x7f8a440734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a64e94080 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dd0c053d00 0x55dd0bfc6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x55dd0bfc6aa0 con 0x55dd0c053d00 2024-09-10T21:35:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a62a00640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dd0c053d00 0x55dd0bfc6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a62000640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a44070ff0 0x7f8a440734b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400b2b0 tx=0x7f8a54001090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a62a00640 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dd0c053d00 0x55dd0bfc6560 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.642+0000 7f8a62a00640 1 -- 172.21.15.97:0/2228079428 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v133'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8a5c002050 con 0x55dd0c053d00 2024-09-10T21:35:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dd0c053d00 msgr2=0x55dd0bfc6560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55dd0c053d00 0x55dd0bfc6560 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a44070ff0 msgr2=0x7f8a440734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8a44070ff0 0x7f8a440734b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400b2b0 tx=0x7f8a54001090 comp rx=0 tx=0).stop 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c03ba00 msgr2=0x55dd0c0537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 --2- 172.21.15.97:0/2228079428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0c03ba00 0x55dd0c0537c0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000ad30 tx=0x7f8a50008790 comp rx=0 tx=0).stop 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a62a00640 1 -- 172.21.15.97:0/2228079428 reap_dead start 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 shutdown_connections 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 >> 172.21.15.97:0/2228079428 conn(0x55dd0bfc90d0 msgr2=0x55dd0c0380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 shutdown_connections 2024-09-10T21:35:07.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.646+0000 7f8a64e94080 1 -- 172.21.15.97:0/2228079428 wait complete. 2024-09-10T21:35:07.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 -- 172.21.15.97:0/761204706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c0ceee0 msgr2=0x7f443c0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 --2- 172.21.15.97:0/761204706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c0ceee0 0x7f443c0cf350 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f4434009c80 tx=0x7f443401f560 comp rx=0 tx=0).stop 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 -- 172.21.15.97:0/761204706 shutdown_connections 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 --2- 172.21.15.97:0/761204706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c0ceee0 0x7f443c0cf350 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 --2- 172.21.15.97:0/761204706 >> v2:172.21.15.194:3300/0 conn(0x7f443c06ec40 0x7f443c06f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 -- 172.21.15.97:0/761204706 >> 172.21.15.97:0/761204706 conn(0x7f443c0b1780 msgr2=0x7f443c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 -- 172.21.15.97:0/761204706 shutdown_connections 2024-09-10T21:35:07.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 -- 172.21.15.97:0/761204706 wait complete. 2024-09-10T21:35:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 Processor -- start 2024-09-10T21:35:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 -- start start 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c06ec40 0x7f443c1e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.794+0000 7f4442200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f443c0ceee0 0x7f443c1f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4442200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f443c1511f0 0x7f443c13dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4442200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f443c06e740 con 0x7f443c06ec40 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4442200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f443c06e5c0 con 0x7f443c1511f0 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4442200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f443c06e8c0 con 0x7f443c0ceee0 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c06ec40 0x7f443c1e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c06ec40 0x7f443c1e29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39488/0 (socket says 172.21.15.97:39488) 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 -- 172.21.15.97:0/2695581604 learned_addr learned my addr 172.21.15.97:0/2695581604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443be00640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f443c1511f0 0x7f443c13dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4432a00640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f443c0ceee0 0x7f443c1f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 -- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f443c0ceee0 msgr2=0x7f443c1f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f443c0ceee0 0x7f443c1f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 -- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f443c1511f0 msgr2=0x7f443c13dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f443c1511f0 0x7f443c13dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 -- 172.21.15.97:0/2695581604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c141670 con 0x7f443c06ec40 2024-09-10T21:35:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4432a00640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f443c0ceee0 0x7f443c1f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f443b400640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c06ec40 0x7f443c1e29c0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c3f0 tx=0x7f442c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4433e00640 1 -- 172.21.15.97:0/2695581604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c017070 con 0x7f443c06ec40 2024-09-10T21:35:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4433e00640 1 -- 172.21.15.97:0/2695581604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442c0053c0 con 0x7f443c06ec40 2024-09-10T21:35:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c1408b0 con 0x7f443c06ec40 2024-09-10T21:35:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4433e00640 1 -- 172.21.15.97:0/2695581604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c003860 con 0x7f443c06ec40 2024-09-10T21:35:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c140e70 con 0x7f443c06ec40 2024-09-10T21:35:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c06f090 con 0x7f443c06ec40 2024-09-10T21:35:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.798+0000 7f4433e00640 1 -- 172.21.15.97:0/2695581604 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f442c003a00 con 0x7f443c06ec40 2024-09-10T21:35:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.802+0000 7f4433e00640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44180725e0 0x7f4418074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.802+0000 7f4433e00640 1 -- 172.21.15.97:0/2695581604 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9457+0+0 (secure 0 0 0) 0x7f442c093720 con 0x7f443c06ec40 2024-09-10T21:35:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.802+0000 7f4432a00640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44180725e0 0x7f4418074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.802+0000 7f4432a00640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44180725e0 0x7f4418074aa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f443c14edc0 tx=0x7f442400a2f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:07.806+0000 7f4433e00640 1 -- 172.21.15.97:0/2695581604 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f442c05d850 con 0x7f443c06ec40 2024-09-10T21:35:08.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.002+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f443c0b1560 con 0x7f443c06ec40 2024-09-10T21:35:08.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.430+0000 7f4433e00640 1 -- 172.21.15.97:0/2695581604 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v316) ==== 162+0+0 (secure 0 0 0) 0x7f442c062280 con 0x7f443c06ec40 2024-09-10T21:35:08.431 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44180725e0 msgr2=0x7f4418074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44180725e0 0x7f4418074aa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f443c14edc0 tx=0x7f442400a2f0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c06ec40 msgr2=0x7f443c1e29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c06ec40 0x7f443c1e29c0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c3f0 tx=0x7f442c00c8b0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 shutdown_connections 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44180725e0 0x7f4418074aa0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f443c1511f0 0x7f443c13dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f443c0ceee0 0x7f443c1f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 --2- 172.21.15.97:0/2695581604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f443c06ec40 0x7f443c1e29c0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 >> 172.21.15.97:0/2695581604 conn(0x7f443c0b1780 msgr2=0x7f443c15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:08.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 shutdown_connections 2024-09-10T21:35:08.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.434+0000 7f4442200640 1 -- 172.21.15.97:0/2695581604 wait complete. 2024-09-10T21:35:08.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 -- 172.21.15.97:0/2574750645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c815fde0 msgr2=0x7fd6c813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 --2- 172.21.15.97:0/2574750645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c815fde0 0x7fd6c813f9b0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009c80 tx=0x7fd6c001cda0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 -- 172.21.15.97:0/2574750645 shutdown_connections 2024-09-10T21:35:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 --2- 172.21.15.97:0/2574750645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c815fde0 0x7fd6c813f9b0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 --2- 172.21.15.97:0/2574750645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce5f0 0x7fd6c8141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 -- 172.21.15.97:0/2574750645 >> 172.21.15.97:0/2574750645 conn(0x7fd6c80b1780 msgr2=0x7fd6c80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 -- 172.21.15.97:0/2574750645 shutdown_connections 2024-09-10T21:35:08.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.602+0000 7fd6cde00640 1 -- 172.21.15.97:0/2574750645 wait complete. 2024-09-10T21:35:08.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 Processor -- start 2024-09-10T21:35:08.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 -- start start 2024-09-10T21:35:08.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce5f0 0x7fd6c8072630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:08.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80ce9d0 0x7fd6c806e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:08.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c815fde0 0x7fd6c806f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:08.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8168bf0 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8168a70 con 0x7fd6c815fde0 2024-09-10T21:35:08.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c8168d70 con 0x7fd6c80ce5f0 2024-09-10T21:35:08.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80ce9d0 0x7fd6c806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80ce9d0 0x7fd6c806e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39508/0 (socket says 172.21.15.97:39508) 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 -- 172.21.15.97:0/3564302985 learned_addr learned my addr 172.21.15.97:0/3564302985 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6a00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce5f0 0x7fd6c8072630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c7400640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c815fde0 0x7fd6c806f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 -- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce5f0 msgr2=0x7fd6c8072630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce5f0 0x7fd6c8072630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 -- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c815fde0 msgr2=0x7fd6c806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c815fde0 0x7fd6c806f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 -- 172.21.15.97:0/3564302985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c8072b70 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6c6000640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80ce9d0 0x7fd6c806e610 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c410 tx=0x7fd6b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6bf400640 1 -- 172.21.15.97:0/3564302985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4011790 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6bf400640 1 -- 172.21.15.97:0/3564302985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6b4017070 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c813bff0 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6bf400640 1 -- 172.21.15.97:0/3564302985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4011cc0 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c813c5b0 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.606+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c8141d20 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.610+0000 7fd6bf400640 1 -- 172.21.15.97:0/3564302985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd6b4039b30 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.610+0000 7fd6bf400640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a80725e0 0x7fd6a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:08.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.610+0000 7fd6bf400640 1 -- 172.21.15.97:0/3564302985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9457+0+0 (secure 0 0 0) 0x7fd6b4093dc0 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.610+0000 7fd6c6a00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a80725e0 0x7fd6a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:08.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.610+0000 7fd6c6a00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a80725e0 0x7fd6a8074aa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8004630 tx=0x7fd6b800b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:08.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.614+0000 7fd6bf400640 1 -- 172.21.15.97:0/3564302985 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6b4021020 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6c806ab60 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6bf400640 1 -- 172.21.15.97:0/3564302985 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 317 v317) ==== 95+0+5233 (secure 0 0 0) 0x7fd6b405df70 con 0x7fd6c80ce9d0 2024-09-10T21:35:08.813 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:08.813 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":317,"btime":"2024-09-10T21:35:08:437790+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":317,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:08.437773+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10296":{"gid":10296,"name":"a","rank":0,"incarnation":317,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6834/3735064631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3735064631},{"type":"v1","addr":"172.21.15.97:6834","nonce":3735064631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 317 2024-09-10T21:35:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a80725e0 msgr2=0x7fd6a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a80725e0 0x7fd6a8074aa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8004630 tx=0x7fd6b800b390 comp rx=0 tx=0).stop 2024-09-10T21:35:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80ce9d0 msgr2=0x7fd6c806e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80ce9d0 0x7fd6c806e610 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c410 tx=0x7fd6b400c8d0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 shutdown_connections 2024-09-10T21:35:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd6a80725e0 0x7fd6a8074aa0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6c815fde0 0x7fd6c806f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c80ce9d0 0x7fd6c806e610 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 --2- 172.21.15.97:0/3564302985 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd6c80ce5f0 0x7fd6c8072630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.814+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 >> 172.21.15.97:0/3564302985 conn(0x7fd6c80b1780 msgr2=0x7fd6c813e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.818+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 shutdown_connections 2024-09-10T21:35:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:08.818+0000 7fd6cde00640 1 -- 172.21.15.97:0/3564302985 wait complete. 2024-09-10T21:35:08.834 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:35:09.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:09.868 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:11.465 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:35:11.466+0000 7f4143200640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:35:11.466 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:11.466+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:35:11.466 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:11.466+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:35:12.280 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:35:12.282+0000 7f4148200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:13.835 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 -- 172.21.15.97:0/3014384826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 msgr2=0x7faa500cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 --2- 172.21.15.97:0/3014384826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa500cf840 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7faa40009900 tx=0x7faa4001c770 comp rx=0 tx=0).stop 2024-09-10T21:35:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 -- 172.21.15.97:0/3014384826 shutdown_connections 2024-09-10T21:35:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 --2- 172.21.15.97:0/3014384826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa500cf840 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 -- 172.21.15.97:0/3014384826 >> 172.21.15.97:0/3014384826 conn(0x7faa500d1a60 msgr2=0x7faa500d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 -- 172.21.15.97:0/3014384826 shutdown_connections 2024-09-10T21:35:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 -- 172.21.15.97:0/3014384826 wait complete. 2024-09-10T21:35:13.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.986+0000 7faa56200640 1 Processor -- start 2024-09-10T21:35:13.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 -- start start 2024-09-10T21:35:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa500cdcb0 0x7faa500c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faa500d0490 0x7faa50070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa501569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa500af3d0 con 0x7faa5015efc0 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faa500af250 con 0x7faa500cdcb0 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7faa500af550 con 0x7faa500d0490 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa501569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa501569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39520/0 (socket says 172.21.15.97:39520) 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 -- 172.21.15.97:0/1688869367 learned_addr learned my addr 172.21.15.97:0/1688869367 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa4fe00640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faa500d0490 0x7faa50070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa54e00640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa500cdcb0 0x7faa500c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:13.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 -- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faa500d0490 msgr2=0x7faa50070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:13.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faa500d0490 0x7faa50070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:13.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 -- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa500cdcb0 msgr2=0x7faa500c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:13.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa500cdcb0 0x7faa500c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 -- 172.21.15.97:0/1688869367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa500cb3a0 con 0x7faa5015efc0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa55800640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa501569b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7faa480046e0 tx=0x7faa4800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa4cc00640 1 -- 172.21.15.97:0/1688869367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4801a070 con 0x7faa5015efc0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa4cc00640 1 -- 172.21.15.97:0/1688869367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa480069e0 con 0x7faa5015efc0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa500cb680 con 0x7faa5015efc0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa4cc00640 1 -- 172.21.15.97:0/1688869367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48006e10 con 0x7faa5015efc0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa500c96b0 con 0x7faa5015efc0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.990+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa500afad0 con 0x7faa5015efc0 2024-09-10T21:35:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.994+0000 7faa4cc00640 1 -- 172.21.15.97:0/1688869367 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faa48008070 con 0x7faa5015efc0 2024-09-10T21:35:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.994+0000 7faa4cc00640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faa300725e0 0x7faa30074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.994+0000 7faa4cc00640 1 -- 172.21.15.97:0/1688869367 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7faa48097080 con 0x7faa5015efc0 2024-09-10T21:35:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.994+0000 7faa54e00640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faa300725e0 0x7faa30074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:13.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.994+0000 7faa54e00640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faa300725e0 0x7faa30074aa0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7faa400af000 tx=0x7faa4001afb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:13.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:13.998+0000 7faa4cc00640 1 -- 172.21.15.97:0/1688869367 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa48061160 con 0x7faa5015efc0 2024-09-10T21:35:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa5006f280 con 0x7faa5015efc0 2024-09-10T21:35:14.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa4cc00640 1 -- 172.21.15.97:0/1688869367 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 321 v321) ==== 95+0+4395 (secure 0 0 0) 0x7faa48065b90 con 0x7faa5015efc0 2024-09-10T21:35:14.197 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:14.197 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":321,"btime":"2024-09-10T21:35:11:471029+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":321,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:11.471023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:14.197 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 321 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faa300725e0 msgr2=0x7faa30074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faa300725e0 0x7faa30074aa0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7faa400af000 tx=0x7faa4001afb0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 msgr2=0x7faa501569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa501569b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7faa480046e0 tx=0x7faa4800d430 comp rx=0 tx=0).stop 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 shutdown_connections 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faa300725e0 0x7faa30074aa0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5015efc0 0x7faa501569b0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faa500d0490 0x7faa50070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 --2- 172.21.15.97:0/1688869367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa500cdcb0 0x7faa500c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 >> 172.21.15.97:0/1688869367 conn(0x7faa500d1a60 msgr2=0x7faa500716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 shutdown_connections 2024-09-10T21:35:14.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.198+0000 7faa56200640 1 -- 172.21.15.97:0/1688869367 wait complete. 2024-09-10T21:35:14.219 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:35:14.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 -- 172.21.15.97:0/203052875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 msgr2=0x7f8584166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 --2- 172.21.15.97:0/203052875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584166fe0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f8584069d60 tx=0x7f857c01f580 comp rx=0 tx=0).stop 2024-09-10T21:35:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 -- 172.21.15.97:0/203052875 shutdown_connections 2024-09-10T21:35:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 --2- 172.21.15.97:0/203052875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584166fe0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 -- 172.21.15.97:0/203052875 >> 172.21.15.97:0/203052875 conn(0x7f85840b37c0 msgr2=0x7f85840b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 -- 172.21.15.97:0/203052875 shutdown_connections 2024-09-10T21:35:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 -- 172.21.15.97:0/203052875 wait complete. 2024-09-10T21:35:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 Processor -- start 2024-09-10T21:35:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 -- start start 2024-09-10T21:35:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.362+0000 7f8589000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85840ce830 0x7f85840c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85840cec10 0x7f85840c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85840cddd0 con 0x7f85840b2570 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f85840cdc50 con 0x7f85840cec10 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f85840ce1a0 con 0x7f85840ce830 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39548/0 (socket says 172.21.15.97:39548) 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 -- 172.21.15.97:0/459848447 learned_addr learned my addr 172.21.15.97:0/459848447 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8582800640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85840cec10 0x7f85840c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581400640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85840ce830 0x7f85840c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 -- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85840ce830 msgr2=0x7f85840c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85840ce830 0x7f85840c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 -- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85840cec10 msgr2=0x7f85840c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85840cec10 0x7f85840c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 -- 172.21.15.97:0/459848447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85841521b0 con 0x7f85840b2570 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581400640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85840ce830 0x7f85840c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8581e00640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584151c70 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f857400c450 tx=0x7f857400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f857a000640 1 -- 172.21.15.97:0/459848447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574012830 con 0x7f85840b2570 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f857a000640 1 -- 172.21.15.97:0/459848447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8574017070 con 0x7f85840b2570 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f857a000640 1 -- 172.21.15.97:0/459848447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574012d80 con 0x7f85840b2570 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85841f8c10 con 0x7f85840b2570 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85840c7580 con 0x7f85840b2570 2024-09-10T21:35:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85840cf800 con 0x7f85840b2570 2024-09-10T21:35:14.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.366+0000 7f857a000640 1 -- 172.21.15.97:0/459848447 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f857401c460 con 0x7f85840b2570 2024-09-10T21:35:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.370+0000 7f857a000640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85640725e0 0x7f8564074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.370+0000 7f857a000640 1 -- 172.21.15.97:0/459848447 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7f8574093e80 con 0x7f85840b2570 2024-09-10T21:35:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.370+0000 7f8581400640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85640725e0 0x7f8564074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.370+0000 7f8581400640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85640725e0 0x7f8564074aa0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f85840b79b0 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.374+0000 7f857a000640 1 -- 172.21.15.97:0/459848447 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f857405e5b0 con 0x7f85840b2570 2024-09-10T21:35:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.574+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8584158c20 con 0x7f85840b2570 2024-09-10T21:35:14.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.574+0000 7f857a000640 1 -- 172.21.15.97:0/459848447 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 321 v321) ==== 95+0+4395 (secure 0 0 0) 0x7f857405deb0 con 0x7f85840b2570 2024-09-10T21:35:14.576 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:14.576 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":321,"btime":"2024-09-10T21:35:11:471029+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":321,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:11.471023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:14.576 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 321 2024-09-10T21:35:14.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85640725e0 msgr2=0x7f8564074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85640725e0 0x7f8564074aa0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f85840b79b0 tx=0x7f8570009040 comp rx=0 tx=0).stop 2024-09-10T21:35:14.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 msgr2=0x7f8584151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584151c70 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f857400c450 tx=0x7f857400c910 comp rx=0 tx=0).stop 2024-09-10T21:35:14.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 shutdown_connections 2024-09-10T21:35:14.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f85640725e0 0x7f8564074aa0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85840cec10 0x7f85840c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f85840ce830 0x7f85840c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 --2- 172.21.15.97:0/459848447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85840b2570 0x7f8584151c70 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 >> 172.21.15.97:0/459848447 conn(0x7f85840b37c0 msgr2=0x7f85840c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:14.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 shutdown_connections 2024-09-10T21:35:14.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.578+0000 7f8589000640 1 -- 172.21.15.97:0/459848447 wait complete. 2024-09-10T21:35:14.597 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:35:14.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 -- 172.21.15.97:0/2987349461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0ce640 msgr2=0x7f053c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 --2- 172.21.15.97:0/2987349461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0ce640 0x7f053c0cea20 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f053c0b36f0 tx=0x7f052c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 -- 172.21.15.97:0/2987349461 shutdown_connections 2024-09-10T21:35:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 --2- 172.21.15.97:0/2987349461 >> v2:172.21.15.194:3300/0 conn(0x7f053c0b0d50 0x7f053c0aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 --2- 172.21.15.97:0/2987349461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0ce640 0x7f053c0cea20 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 -- 172.21.15.97:0/2987349461 >> 172.21.15.97:0/2987349461 conn(0x7f053c0d1c30 msgr2=0x7f053c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 -- 172.21.15.97:0/2987349461 shutdown_connections 2024-09-10T21:35:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 -- 172.21.15.97:0/2987349461 wait complete. 2024-09-10T21:35:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.742+0000 7f0540a00640 1 Processor -- start 2024-09-10T21:35:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 -- start start 2024-09-10T21:35:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f053c0af300 0x7f053c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0b0d50 0x7f053c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f053c0ce640 0x7f053c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f053c169270 con 0x7f053c0b0d50 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1690f0 con 0x7f053c0ce640 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f053c1693f0 con 0x7f053c0af300 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0b0d50 0x7f053c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0b0d50 0x7f053c157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39572/0 (socket says 172.21.15.97:39572) 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 -- 172.21.15.97:0/1500718683 learned_addr learned my addr 172.21.15.97:0/1500718683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539e00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f053c0af300 0x7f053c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f053a800640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f053c0ce640 0x7f053c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 -- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f053c0af300 msgr2=0x7f053c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f053c0af300 0x7f053c0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 -- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f053c0ce640 msgr2=0x7f053c152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f053c0ce640 0x7f053c152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 -- 172.21.15.97:0/1500718683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c158920 con 0x7f053c0b0d50 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f053a800640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f053c0ce640 0x7f053c152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0539400640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0b0d50 0x7f053c157fb0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0532000640 1 -- 172.21.15.97:0/1500718683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528017070 con 0x7f053c0b0d50 2024-09-10T21:35:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0532000640 1 -- 172.21.15.97:0/1500718683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0528009d20 con 0x7f053c0b0d50 2024-09-10T21:35:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c158c00 con 0x7f053c0b0d50 2024-09-10T21:35:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0532000640 1 -- 172.21.15.97:0/1500718683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280106c0 con 0x7f053c0b0d50 2024-09-10T21:35:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c152940 con 0x7f053c0b0d50 2024-09-10T21:35:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.746+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c0aedc0 con 0x7f053c0b0d50 2024-09-10T21:35:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.750+0000 7f0532000640 1 -- 172.21.15.97:0/1500718683 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0528010860 con 0x7f053c0b0d50 2024-09-10T21:35:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.750+0000 7f0532000640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f051c0725e0 0x7f051c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.750+0000 7f0532000640 1 -- 172.21.15.97:0/1500718683 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7f05280944a0 con 0x7f053c0b0d50 2024-09-10T21:35:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.750+0000 7f0539e00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f051c0725e0 0x7f051c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.750+0000 7f0539e00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f051c0725e0 0x7f051c074aa0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f052c0af000 tx=0x7f052c0a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.754+0000 7f0532000640 1 -- 172.21.15.97:0/1500718683 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f052805e610 con 0x7f053c0b0d50 2024-09-10T21:35:14.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.950+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f053c158dd0 con 0x7f053c0b0d50 2024-09-10T21:35:14.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0532000640 1 -- 172.21.15.97:0/1500718683 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 321 v321) ==== 95+0+4395 (secure 0 0 0) 0x7f0528063040 con 0x7f053c0b0d50 2024-09-10T21:35:14.952 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:14.952 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":321,"btime":"2024-09-10T21:35:11:471029+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":321,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:11.471023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:14.952 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 321 2024-09-10T21:35:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f051c0725e0 msgr2=0x7f051c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f051c0725e0 0x7f051c074aa0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f052c0af000 tx=0x7f052c0a7040 comp rx=0 tx=0).stop 2024-09-10T21:35:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0b0d50 msgr2=0x7f053c157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0b0d50 0x7f053c157fb0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800de10 comp rx=0 tx=0).stop 2024-09-10T21:35:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 shutdown_connections 2024-09-10T21:35:14.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f051c0725e0 0x7f051c074aa0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f053c0ce640 0x7f053c152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f053c0b0d50 0x7f053c157fb0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 --2- 172.21.15.97:0/1500718683 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f053c0af300 0x7f053c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 >> 172.21.15.97:0/1500718683 conn(0x7f053c0d1c30 msgr2=0x7f053c15d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 shutdown_connections 2024-09-10T21:35:14.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:14.954+0000 7f0540a00640 1 -- 172.21.15.97:0/1500718683 wait complete. 2024-09-10T21:35:14.973 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2024-09-10T21:35:14.973 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2024-09-10T21:35:14.973 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:14.973 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:14.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:35:14.983 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:35:14.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:14.983 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:35:15.038 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:35:15.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:35:15.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 -- 172.21.15.97:0/1684001733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 msgr2=0x7f53980cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 --2- 172.21.15.97:0/1684001733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980cea20 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f53980c8c90 tx=0x7f538801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 -- 172.21.15.97:0/1684001733 shutdown_connections 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 --2- 172.21.15.97:0/1684001733 >> v2:172.21.15.194:3300/0 conn(0x7f53980cd6e0 0x7f53980af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 --2- 172.21.15.97:0/1684001733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980cea20 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 -- 172.21.15.97:0/1684001733 >> 172.21.15.97:0/1684001733 conn(0x7f53980d1c30 msgr2=0x7f53980d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 -- 172.21.15.97:0/1684001733 shutdown_connections 2024-09-10T21:35:15.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 -- 172.21.15.97:0/1684001733 wait complete. 2024-09-10T21:35:15.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 Processor -- start 2024-09-10T21:35:15.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.230+0000 7f539ca00640 1 -- start start 2024-09-10T21:35:15.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53980b0d50 0x7f53981faed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53980cd6e0 0x7f53980b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5398168d20 con 0x7f53980ce640 2024-09-10T21:35:15.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5398168ba0 con 0x7f53980cd6e0 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5398168ea0 con 0x7f53980b0d50 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39592/0 (socket says 172.21.15.97:39592) 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 -- 172.21.15.97:0/1565087748 learned_addr learned my addr 172.21.15.97:0/1565087748 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5395e00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53980b0d50 0x7f53981faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f538fe00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53980cd6e0 0x7f53980b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 -- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53980b0d50 msgr2=0x7f53981faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53980b0d50 0x7f53981faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 -- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53980cd6e0 msgr2=0x7f53980b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53980cd6e0 0x7f53980b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 -- 172.21.15.97:0/1565087748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53981fb410 con 0x7f53980ce640 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f538fe00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53980cd6e0 0x7f53980b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f5396800640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980b5b90 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f539000e990 tx=0x7f539000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f538e000640 1 -- 172.21.15.97:0/1565087748 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390004d30 con 0x7f53980ce640 2024-09-10T21:35:15.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f538e000640 1 -- 172.21.15.97:0/1565087748 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f539000cb00 con 0x7f53980ce640 2024-09-10T21:35:15.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53981fb6d0 con 0x7f53980ce640 2024-09-10T21:35:15.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f538e000640 1 -- 172.21.15.97:0/1565087748 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390005020 con 0x7f53980ce640 2024-09-10T21:35:15.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53981fbc90 con 0x7f53980ce640 2024-09-10T21:35:15.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53980c5600 con 0x7f53980ce640 2024-09-10T21:35:15.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.234+0000 7f538e000640 1 -- 172.21.15.97:0/1565087748 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f539000f040 con 0x7f53980ce640 2024-09-10T21:35:15.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.238+0000 7f538e000640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53740725e0 0x7f5374074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.238+0000 7f538e000640 1 -- 172.21.15.97:0/1565087748 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7f53900946c0 con 0x7f53980ce640 2024-09-10T21:35:15.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.238+0000 7f5395e00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53740725e0 0x7f5374074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.238+0000 7f5395e00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53740725e0 0x7f5374074aa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f53880af000 tx=0x7f5388019660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.242+0000 7f538e000640 1 -- 172.21.15.97:0/1565087748 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5390016030 con 0x7f53980ce640 2024-09-10T21:35:15.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.438+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f53980d0640 con 0x7f53980ce640 2024-09-10T21:35:15.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.482+0000 7f538e000640 1 -- 172.21.15.97:0/1565087748 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v322) ==== 161+0+0 (secure 0 0 0) 0x7f539005e7a0 con 0x7f53980ce640 2024-09-10T21:35:15.481 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:35:15.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53740725e0 msgr2=0x7f5374074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53740725e0 0x7f5374074aa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f53880af000 tx=0x7f5388019660 comp rx=0 tx=0).stop 2024-09-10T21:35:15.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 msgr2=0x7f53980b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980b5b90 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f539000e990 tx=0x7f539000ee50 comp rx=0 tx=0).stop 2024-09-10T21:35:15.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 shutdown_connections 2024-09-10T21:35:15.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f53740725e0 0x7f5374074aa0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53980ce640 0x7f53980b5b90 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53980cd6e0 0x7f53980b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 --2- 172.21.15.97:0/1565087748 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f53980b0d50 0x7f53981faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 >> 172.21.15.97:0/1565087748 conn(0x7f53980d1c30 msgr2=0x7f539806f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:15.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 shutdown_connections 2024-09-10T21:35:15.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.486+0000 7f539ca00640 1 -- 172.21.15.97:0/1565087748 wait complete. 2024-09-10T21:35:15.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:35:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- 172.21.15.97:0/204417785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f438406fd50 msgr2=0x7f43840701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 --2- 172.21.15.97:0/204417785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f438406fd50 0x7f43840701e0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f437c009c80 tx=0x7f437c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- 172.21.15.97:0/204417785 shutdown_connections 2024-09-10T21:35:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 --2- 172.21.15.97:0/204417785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f438406fd50 0x7f43840701e0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 --2- 172.21.15.97:0/204417785 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4384144690 0x7f43840b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- 172.21.15.97:0/204417785 >> 172.21.15.97:0/204417785 conn(0x7f43840b1780 msgr2=0x7f43840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- 172.21.15.97:0/204417785 shutdown_connections 2024-09-10T21:35:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- 172.21.15.97:0/204417785 wait complete. 2024-09-10T21:35:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 Processor -- start 2024-09-10T21:35:15.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- start start 2024-09-10T21:35:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f438406fd50 0x7f43841fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4384143f90 0x7f438415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4384144690 0x7f4384163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4384167d50 con 0x7f4384143f90 2024-09-10T21:35:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4384167bd0 con 0x7f438406fd50 2024-09-10T21:35:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438c000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4384167ed0 con 0x7f4384144690 2024-09-10T21:35:15.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4384143f90 0x7f438415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4384143f90 0x7f438415c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39610/0 (socket says 172.21.15.97:39610) 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 -- 172.21.15.97:0/849891019 learned_addr learned my addr 172.21.15.97:0/849891019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4389600640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f438406fd50 0x7f43841fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f438a000640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4384144690 0x7f4384163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 -- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4384144690 msgr2=0x7f4384163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4384144690 0x7f4384163bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 -- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f438406fd50 msgr2=0x7f43841fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f438406fd50 0x7f43841fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.650+0000 7f4388c00640 1 -- 172.21.15.97:0/849891019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43841f8920 con 0x7f4384143f90 2024-09-10T21:35:15.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4388c00640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4384143f90 0x7f438415c030 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f437000e7d0 tx=0x7f437000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4381600640 1 -- 172.21.15.97:0/849891019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437000ccc0 con 0x7f4384143f90 2024-09-10T21:35:15.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4381600640 1 -- 172.21.15.97:0/849891019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4370010430 con 0x7f4384143f90 2024-09-10T21:35:15.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43841f8c00 con 0x7f4384143f90 2024-09-10T21:35:15.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4381600640 1 -- 172.21.15.97:0/849891019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370010860 con 0x7f4384143f90 2024-09-10T21:35:15.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43841fc2e0 con 0x7f4384143f90 2024-09-10T21:35:15.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43840701e0 con 0x7f4384143f90 2024-09-10T21:35:15.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4381600640 1 -- 172.21.15.97:0/849891019 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4370010a00 con 0x7f4384143f90 2024-09-10T21:35:15.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4381600640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43640725e0 0x7f4364074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4381600640 1 -- 172.21.15.97:0/849891019 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7f4370094b60 con 0x7f4384143f90 2024-09-10T21:35:15.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.654+0000 7f4389600640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43640725e0 0x7f4364074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.658+0000 7f4389600640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43640725e0 0x7f4364074aa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4378007fd0 tx=0x7f4378009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.658+0000 7f4381600640 1 -- 172.21.15.97:0/849891019 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f437005ec40 con 0x7f4384143f90 2024-09-10T21:35:15.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:15.886+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f4384141bf0 con 0x7f4384143f90 2024-09-10T21:35:16.219 INFO:tasks.ceph.mds.a.smithi097.stderr: -55> 2024-09-10T21:35:11.466+0000 7f4143200640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:35:16.219 INFO:tasks.ceph.mds.a.smithi097.stderr: -33> 2024-09-10T21:35:12.282+0000 7f4148200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:16.258 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:35:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f4381600640 1 -- 172.21.15.97:0/849891019 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v323) ==== 101+0+0 (secure 0 0 0) 0x7f4370063670 con 0x7f4384143f90 2024-09-10T21:35:16.493 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43640725e0 msgr2=0x7f4364074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43640725e0 0x7f4364074aa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4378007fd0 tx=0x7f4378009040 comp rx=0 tx=0).stop 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4384143f90 msgr2=0x7f438415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4384143f90 0x7f438415c030 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f437000e7d0 tx=0x7f437000ec90 comp rx=0 tx=0).stop 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 shutdown_connections 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f43640725e0 0x7f4364074aa0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4384144690 0x7f4384163bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4384143f90 0x7f438415c030 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 --2- 172.21.15.97:0/849891019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f438406fd50 0x7f43841fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 >> 172.21.15.97:0/849891019 conn(0x7f43840b1780 msgr2=0x7f43840b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 shutdown_connections 2024-09-10T21:35:16.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.494+0000 7f438c000640 1 -- 172.21.15.97:0/849891019 wait complete. 2024-09-10T21:35:16.516 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:35:16.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.562+0000 7f1d806d8080 1 Processor -- start 2024-09-10T21:35:16.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.562+0000 7f1d806d8080 1 -- start start 2024-09-10T21:35:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.562+0000 7f1d806d8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d64002350 0x563b776195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.562+0000 7f1d806d8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b77646e40 0x563b777825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.562+0000 7f1d806d8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b7761a0f0 0x563b77782b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b77566a00 con 0x563b77646e40 2024-09-10T21:35:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b775c4f40 con 0x563b7761a0f0 2024-09-10T21:35:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563b776d1e50 con 0x7f1d64002350 2024-09-10T21:35:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b77646e40 0x563b777825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b77646e40 0x563b777825d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39636/0 (socket says 172.21.15.97:39636) 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 -- 172.21.15.97:0/1986709913 learned_addr learned my addr 172.21.15.97:0/1986709913 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 -- 172.21.15.97:0/1986709913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d64002350 msgr2=0x563b776195c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 --2- 172.21.15.97:0/1986709913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d64002350 0x563b776195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 -- 172.21.15.97:0/1986709913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b7761a0f0 msgr2=0x563b77782b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 --2- 172.21.15.97:0/1986709913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b7761a0f0 0x563b77782b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 -- 172.21.15.97:0/1986709913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b7753fc10 con 0x563b77646e40 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 --2- 172.21.15.97:0/1986709913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b77646e40 0x563b777825d0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b770 tx=0x7f1d68004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a092cdfb89df51c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77400640 1 -- 172.21.15.97:0/1986709913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800adb0 con 0x563b77646e40 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77400640 1 -- 172.21.15.97:0/1986709913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d68008440 con 0x563b77646e40 2024-09-10T21:35:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77400640 1 -- 172.21.15.97:0/1986709913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68008830 con 0x563b77646e40 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- 172.21.15.97:0/1986709913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b77646e40 msgr2=0x563b777825d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 --2- 172.21.15.97:0/1986709913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b77646e40 0x563b777825d0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b770 tx=0x7f1d68004890 comp rx=0 tx=0).stop 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- 172.21.15.97:0/1986709913 shutdown_connections 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 --2- 172.21.15.97:0/1986709913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b7761a0f0 0x563b77782b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 --2- 172.21.15.97:0/1986709913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b77646e40 0x563b777825d0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 --2- 172.21.15.97:0/1986709913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d64002350 0x563b776195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- 172.21.15.97:0/1986709913 >> 172.21.15.97:0/1986709913 conn(0x563b776600d0 msgr2=0x563b776ec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- 172.21.15.97:0/1986709913 shutdown_connections 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- 172.21.15.97:0/1986709913 wait complete. 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 Processor -- start 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- start start 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b7761a0f0 0x563b776eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d64002350 0x563b77618250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b77618790 0x563b776175a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b77648000 con 0x7f1d64002350 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b77647d60 con 0x563b77618790 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d806d8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x563b776482a0 con 0x563b7761a0f0 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d64002350 0x563b77618250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d64002350 0x563b77618250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39648/0 (socket says 172.21.15.97:39648) 2024-09-10T21:35:16.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 learned_addr learned my addr 172.21.15.97:0/878794951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d7e200640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b77618790 0x563b776175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.566+0000 7f1d7d800640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b7761a0f0 0x563b776eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b7761a0f0 msgr2=0x563b776eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d77e00640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x563b7761a0f0 0x563b776eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b77618790 msgr2=0x563b776175a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d77e00640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b77618790 0x563b776175a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b776ef2f0 con 0x7f1d64002350 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d77e00640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d64002350 0x563b77618250 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800a810 tx=0x7f1d680186d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d76000640 1 -- 172.21.15.97:0/878794951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68026020 con 0x7f1d64002350 2024-09-10T21:35:16.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d76000640 1 -- 172.21.15.97:0/878794951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d68018b00 con 0x7f1d64002350 2024-09-10T21:35:16.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d76000640 1 -- 172.21.15.97:0/878794951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68022420 con 0x7f1d64002350 2024-09-10T21:35:16.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563b776ef590 con 0x7f1d64002350 2024-09-10T21:35:16.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b776cdd50 con 0x7f1d64002350 2024-09-10T21:35:16.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d76000640 1 -- 172.21.15.97:0/878794951 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d68008440 con 0x7f1d64002350 2024-09-10T21:35:16.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d76000640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d5c070fc0 0x7f1d5c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.570+0000 7f1d76000640 1 -- 172.21.15.97:0/878794951 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7f1d68094ba0 con 0x7f1d64002350 2024-09-10T21:35:16.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d7d800640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d5c070fc0 0x7f1d5c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.572 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:35:16.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b776ce700 0x563b776ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b776cf020 con 0x563b776ce700 2024-09-10T21:35:16.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d7e200640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b776ce700 0x563b776ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d7d800640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d5c070fc0 0x7f1d5c073480 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000b660 tx=0x7f1d700b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d7e200640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b776ce700 0x563b776ceae0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v134'1159 uv1133 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.574+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b77795520 con 0x563b776ce700 2024-09-10T21:35:16.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.578+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v134'1160 uv1160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.578+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777958c0 con 0x563b776ce700 2024-09-10T21:35:16.578 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:35:16.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.578+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v134'1161 uv1161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.578+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796c70 con 0x563b776ce700 2024-09-10T21:35:16.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.582+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v134'1162 uv1162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.582+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77797010 con 0x563b776ce700 2024-09-10T21:35:16.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.582+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v134'1163 uv1163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.582+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77795c60 con 0x563b776ce700 2024-09-10T21:35:16.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.586+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v134'1164 uv1164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.586+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796080 con 0x563b776ce700 2024-09-10T21:35:16.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.586+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v134'1165 uv1139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.586+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b77796080 con 0x563b776ce700 2024-09-10T21:35:16.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.590+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v134'1166 uv1166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.590+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796030 con 0x563b776ce700 2024-09-10T21:35:16.590 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:35:16.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.590+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v134'1167 uv1167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.590+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.594+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v134'1168 uv1168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.594+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.594+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v134'1169 uv1169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.594+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.598+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v134'1170 uv1170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.598+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.598+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v134'1171 uv1146 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.598+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.602+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v134'1172 uv1172 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.602+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.601 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:35:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.602+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v134'1173 uv1173 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.602+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.602+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v134'1174 uv1174 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.602+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.606+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v134'1175 uv1175 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.606+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796000 con 0x563b776ce700 2024-09-10T21:35:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.606+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v134'1176 uv1176 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.606+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777963d0 con 0x563b776ce700 2024-09-10T21:35:16.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.610+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v134'1177 uv1177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.610+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.610+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v134'1178 uv1152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.610+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.614+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v134'1179 uv1179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.614+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.613 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:35:16.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.614+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v134'1180 uv1180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.614+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.614+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v134'1181 uv1181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.614+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.618+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v134'1182 uv1182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.618+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.618+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v134'1183 uv1183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.618+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.622+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v134'1184 uv1158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.622+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.622+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v134'1185 uv1185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.622+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.623 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:35:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.626+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v134'1186 uv1186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.626+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.626+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v134'1187 uv1187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.626+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.626+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v134'1188 uv1188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.626+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b776ce700 2024-09-10T21:35:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.630+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v134'1189 uv1189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b776ce700 2024-09-10T21:35:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.630+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b777973b0 0x563b777997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.630+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b77796280 con 0x563b777973b0 2024-09-10T21:35:16.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.630+0000 7f1d7d800640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b777973b0 0x563b777997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.630+0000 7f1d7d800640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b777973b0 0x563b777997a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.634+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v134'846 uv828 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.634+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b77799ce0 con 0x563b777973b0 2024-09-10T21:35:16.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.634+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v134'847 uv847 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.634+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779a080 con 0x563b777973b0 2024-09-10T21:35:16.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.634+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v134'848 uv848 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.635 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:35:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.634+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779a8d0 con 0x563b777973b0 2024-09-10T21:35:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.638+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v134'849 uv849 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.638+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779ac70 con 0x563b777973b0 2024-09-10T21:35:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.638+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v134'850 uv850 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.638+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b010 con 0x563b777973b0 2024-09-10T21:35:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.642+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v134'851 uv833 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.642+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b7779b3b0 con 0x563b777973b0 2024-09-10T21:35:16.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.642+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v134'852 uv852 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.642+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.642+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v134'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.643 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:35:16.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.642+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.646+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v134'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.646+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.646+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v134'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.646+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.650+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v134'856 uv838 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.650+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.650+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v134'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.650+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.650+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v134'858 uv858 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.650 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:35:16.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.650+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f920 con 0x563b777973b0 2024-09-10T21:35:16.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.654+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v134'859 uv859 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.654+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f920 con 0x563b777973b0 2024-09-10T21:35:16.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.654+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v134'860 uv860 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.654+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.654+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v134'861 uv844 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.654+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v134'862 uv862 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.656 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:35:16.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v134'863 uv863 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779c760 con 0x563b777973b0 2024-09-10T21:35:16.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v134'864 uv864 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779c760 con 0x563b777973b0 2024-09-10T21:35:16.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v134'865 uv865 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.658+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779b750 con 0x563b777973b0 2024-09-10T21:35:16.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.662+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v134'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777973b0 2024-09-10T21:35:16.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.662+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b7779bc60 0x563b7779e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.662+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779e590 con 0x563b7779bc60 2024-09-10T21:35:16.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.662+0000 7f1d77e00640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b7779bc60 0x563b7779e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.662+0000 7f1d77e00640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b7779bc60 0x563b7779e050 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.662+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.662+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b7779e930 con 0x563b7779bc60 2024-09-10T21:35:16.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.666+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.666+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779ede0 con 0x563b7779bc60 2024-09-10T21:35:16.665 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:35:16.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.666+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.666+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a0690 con 0x563b7779bc60 2024-09-10T21:35:16.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.666+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.666+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a0a30 con 0x563b7779bc60 2024-09-10T21:35:16.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f680 con 0x563b7779bc60 2024-09-10T21:35:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b7779ff80 0x563b777a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3320 con 0x563b7779ff80 2024-09-10T21:35:16.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d7e200640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b7779ff80 0x563b777a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d7e200640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b7779ff80 0x563b777a2de0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v134'172 uv171 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.670+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a3320 con 0x563b7779ff80 2024-09-10T21:35:16.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.674+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v134'173 uv173 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.674+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a33a0 con 0x563b7779ff80 2024-09-10T21:35:16.673 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:35:16.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.674+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v134'174 uv174 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.674+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3790 con 0x563b7779ff80 2024-09-10T21:35:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.674+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v134'175 uv175 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.674+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3790 con 0x563b7779bc60 2024-09-10T21:35:16.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.678+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.678+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.678+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.678+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.678 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:35:16.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.678+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.678+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a4b40 con 0x563b7779bc60 2024-09-10T21:35:16.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.682+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.682+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a4b40 con 0x563b7779bc60 2024-09-10T21:35:16.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.682+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.682+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.682+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.682+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.686+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v134'415 uv409 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.686+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.686+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v134'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.686+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.686 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:35:16.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.686+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v134'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.686+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a4b40 con 0x563b7779ff80 2024-09-10T21:35:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.690+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v134'418 uv418 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.690+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a4b40 con 0x563b7779ff80 2024-09-10T21:35:16.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.690+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v134'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.690+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.690+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v134'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.690+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.694+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v134'421 uv414 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.694+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.694+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v134'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.694+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.696 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:35:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.694+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v134'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.694+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.698+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v134'424 uv424 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.698+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779ff80 2024-09-10T21:35:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.698+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v134'425 uv425 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.698+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.698+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v134'188 uv187 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.698+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v134'189 uv189 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.700 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:35:16.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v134'190 uv190 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v134'191 uv191 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a3b30 con 0x563b7779bc60 2024-09-10T21:35:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v134'192 uv192 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b7779bc60 2024-09-10T21:35:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b777a4040 0x563b777a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.702+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6a90 con 0x563b777a4040 2024-09-10T21:35:16.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.706+0000 7f1d7d800640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b777a4040 0x563b777a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.706+0000 7f1d7d800640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b777a4040 0x563b777a6430 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.706+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.706+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a6e60 con 0x563b777a4040 2024-09-10T21:35:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.706+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.706+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6e60 con 0x563b777a4040 2024-09-10T21:35:16.707 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:35:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.710+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.710+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a7e70 con 0x563b777a4040 2024-09-10T21:35:16.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.710+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.710+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a7e70 con 0x563b777a4040 2024-09-10T21:35:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.710+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.710+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6e60 con 0x563b777a4040 2024-09-10T21:35:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6e60 con 0x563b777a4040 2024-09-10T21:35:16.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v134'376 uv369 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a6e60 con 0x563b777a4040 2024-09-10T21:35:16.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v134'377 uv377 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6e60 con 0x563b777a4040 2024-09-10T21:35:16.715 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:35:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v134'378 uv378 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.714+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a7220 con 0x563b777a4040 2024-09-10T21:35:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.718+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v134'379 uv379 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.718+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6f00 con 0x563b777a4040 2024-09-10T21:35:16.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.718+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v134'380 uv380 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.718+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6f00 con 0x563b777a4040 2024-09-10T21:35:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.718+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v134'381 uv375 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.718+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777a6f00 con 0x563b777a4040 2024-09-10T21:35:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.722+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v134'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.722+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6f00 con 0x563b777a4040 2024-09-10T21:35:16.721 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:35:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.722+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v134'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.722+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777a6f00 con 0x563b777a4040 2024-09-10T21:35:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.722+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v134'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.722+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f2a0 con 0x563b777a4040 2024-09-10T21:35:16.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.726+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v134'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.726+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f260 con 0x563b7779ff80 2024-09-10T21:35:16.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.726+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v134'167 uv165 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.726+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b7779f260 con 0x563b7779ff80 2024-09-10T21:35:16.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.726+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v134'168 uv168 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.726+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f260 con 0x563b7779ff80 2024-09-10T21:35:16.727 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:35:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.726+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v134'169 uv169 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.730+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f260 con 0x563b7779ff80 2024-09-10T21:35:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.730+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v134'170 uv170 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.730+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f220 con 0x563b7779ff80 2024-09-10T21:35:16.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.730+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v134'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.730+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b7779f220 con 0x563b7779ff80 2024-09-10T21:35:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.730+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v134'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.730+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f220 con 0x563b7779ff80 2024-09-10T21:35:16.731 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:35:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.734+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v134'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.734+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f220 con 0x563b7779ff80 2024-09-10T21:35:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.734+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v134'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.734+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f220 con 0x563b7779ff80 2024-09-10T21:35:16.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.734+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v134'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.734+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b7779f220 con 0x563b7779ff80 2024-09-10T21:35:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.734+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v134'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d780099c0 con 0x563b7779ff80 2024-09-10T21:35:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b777a7cb0 0x563b777aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a7cb0 2024-09-10T21:35:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d77e00640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b777a7cb0 0x563b777aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d77e00640 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b777a7cb0 0x563b777aa3e0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v134'191 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b777a7cb0 2024-09-10T21:35:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a7cb0 2024-09-10T21:35:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v134'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b777a7cb0 2024-09-10T21:35:16.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.738+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a7cb0 2024-09-10T21:35:16.739 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:35:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.742+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v134'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b777a7cb0 2024-09-10T21:35:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.742+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a7cb0 2024-09-10T21:35:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.742+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v134'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b777a7cb0 2024-09-10T21:35:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.742+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a7cb0 2024-09-10T21:35:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.742+0000 7f1d77e00640 1 -- 172.21.15.97:0/878794951 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v134'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1d68028030 con 0x563b777a7cb0 2024-09-10T21:35:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.742+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v134'468 uv459 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v134'469 uv469 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v134'470 uv470 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.746 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:35:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v134'471 uv471 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.746+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.750+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v134'472 uv472 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.750+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.750+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v134'473 uv466 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.750+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.750+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v134'474 uv474 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.750+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v134'475 uv475 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.751 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:35:16.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777ab930 con 0x563b777a4040 2024-09-10T21:35:16.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v134'476 uv476 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777ab930 con 0x563b777a4040 2024-09-10T21:35:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v134'477 uv477 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aa920 con 0x563b777a4040 2024-09-10T21:35:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v134'478 uv478 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.754+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x563b777aadd0 con 0x563b777a4040 2024-09-10T21:35:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d7d800640 1 -- 172.21.15.97:0/878794951 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v134'479 uv479 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1d700ad070 con 0x563b777a4040 2024-09-10T21:35:16.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b7779bc60 msgr2=0x563b7779e050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x563b7779bc60 0x563b7779e050 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b777a4040 msgr2=0x563b777a6430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x563b777a4040 0x563b777a6430 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b777a7cb0 msgr2=0x563b777aa3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x563b777a7cb0 0x563b777aa3e0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b7779ff80 msgr2=0x563b777a2de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x563b7779ff80 0x563b777a2de0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 reap_dead start 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b777973b0 msgr2=0x563b777997a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x563b777973b0 0x563b777997a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b776ce700 msgr2=0x563b776ceae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d7e200640 1 -- 172.21.15.97:0/878794951 reap_dead start 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b776ce700 0x563b776ceae0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d5c070fc0 msgr2=0x7f1d5c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d5c070fc0 0x7f1d5c073480 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000b660 tx=0x7f1d700b1040 comp rx=0 tx=0).stop 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d64002350 msgr2=0x563b77618250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d64002350 0x563b77618250 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800a810 tx=0x7f1d680186d0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 shutdown_connections 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x563b776ce700 0x563b776ceae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d5c070fc0 0x7f1d5c073480 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 --2- 172.21.15.97:0/878794951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d64002350 0x563b77618250 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 >> 172.21.15.97:0/878794951 conn(0x563b776600d0 msgr2=0x563b776eade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 shutdown_connections 2024-09-10T21:35:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.758+0000 7f1d806d8080 1 -- 172.21.15.97:0/878794951 wait complete. 2024-09-10T21:35:16.761 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:35:16.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 Processor -- start 2024-09-10T21:35:16.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 -- start start 2024-09-10T21:35:16.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 0x55d9e6c12210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c12b00 0x55d9e6b8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c132a0 0x55d9e6c0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d9e6c14e70 con 0x55d9e6c12b00 2024-09-10T21:35:16.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d9e6aa7a00 con 0x55d9e6c132a0 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd452fe8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d9e6c14ff0 con 0x55d9e6b88940 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c12b00 0x55d9e6b8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c12b00 0x55d9e6b8c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39664/0 (socket says 172.21.15.97:39664) 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd450200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 0x55d9e6c12210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 -- 172.21.15.97:0/3488146875 learned_addr learned my addr 172.21.15.97:0/3488146875 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd450c00640 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c132a0 0x55d9e6c0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 -- 172.21.15.97:0/3488146875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 msgr2=0x55d9e6c12210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 0x55d9e6c12210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 -- 172.21.15.97:0/3488146875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c132a0 msgr2=0x55d9e6c0b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c132a0 0x55d9e6c0b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 -- 172.21.15.97:0/3488146875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e6aa6a20 con 0x55d9e6c12b00 2024-09-10T21:35:16.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd447e00640 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c12b00 0x55d9e6b8c220 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c610 tx=0x7fd43800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99455214c70ce1d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd44ee00640 1 -- 172.21.15.97:0/3488146875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43800aa90 con 0x55d9e6c12b00 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd44ee00640 1 -- 172.21.15.97:0/3488146875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd43800ac30 con 0x55d9e6c12b00 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.850+0000 7fd44ee00640 1 -- 172.21.15.97:0/3488146875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438015440 con 0x55d9e6c12b00 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- 172.21.15.97:0/3488146875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c12b00 msgr2=0x55d9e6b8c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c12b00 0x55d9e6b8c220 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c610 tx=0x7fd43800cad0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- 172.21.15.97:0/3488146875 shutdown_connections 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c132a0 0x55d9e6c0b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c12b00 0x55d9e6b8c220 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3488146875 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 0x55d9e6c12210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- 172.21.15.97:0/3488146875 >> 172.21.15.97:0/3488146875 conn(0x55d9e6b87520 msgr2=0x55d9e6b56d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- 172.21.15.97:0/3488146875 shutdown_connections 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- 172.21.15.97:0/3488146875 wait complete. 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 Processor -- start 2024-09-10T21:35:16.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- start start 2024-09-10T21:35:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 0x55d9e6c2ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c12b00 0x55d9e6c2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c132a0 0x55d9e6c2e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d9e6c302d0 con 0x55d9e6c132a0 2024-09-10T21:35:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d9e6c12750 con 0x55d9e6c12b00 2024-09-10T21:35:16.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55d9e6c2d1d0 con 0x55d9e6b88940 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c132a0 0x55d9e6c2e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c132a0 0x55d9e6c2e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39668/0 (socket says 172.21.15.97:39668) 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 -- 172.21.15.97:0/3896268402 learned_addr learned my addr 172.21.15.97:0/3896268402 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450200640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 0x55d9e6c2ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd447e00640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c12b00 0x55d9e6c2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 -- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 msgr2=0x55d9e6c2ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55d9e6b88940 0x55d9e6c2ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 -- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c12b00 msgr2=0x55d9e6c2b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9e6c12b00 0x55d9e6c2b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 -- 172.21.15.97:0/3896268402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e6a80c10 con 0x55d9e6c132a0 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd450c00640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c132a0 0x55d9e6c2e0f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a550 tx=0x7fd4480aa5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd44da00640 1 -- 172.21.15.97:0/3896268402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480bb070 con 0x55d9e6c132a0 2024-09-10T21:35:16.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd44da00640 1 -- 172.21.15.97:0/3896268402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4480b0040 con 0x55d9e6c132a0 2024-09-10T21:35:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd44da00640 1 -- 172.21.15.97:0/3896268402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480af070 con 0x55d9e6c132a0 2024-09-10T21:35:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d9e6c2d470 con 0x55d9e6c132a0 2024-09-10T21:35:16.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.854+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e6c2d9b0 con 0x55d9e6c132a0 2024-09-10T21:35:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd44da00640 1 -- 172.21.15.97:0/3896268402 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4480c7060 con 0x55d9e6c132a0 2024-09-10T21:35:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd44da00640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd42c070fc0 0x7fd42c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd44da00640 1 -- 172.21.15.97:0/3896268402 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7fd44812d790 con 0x55d9e6c132a0 2024-09-10T21:35:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd450200640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd42c070fc0 0x7fd42c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d9e6cc9330 0x55d9e6cc9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55d9e6c0bd90 con 0x55d9e6cc9330 2024-09-10T21:35:16.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd447e00640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d9e6cc9330 0x55d9e6cc9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd450200640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd42c070fc0 0x7fd42c073480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c710 tx=0x7fd44000e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.858+0000 7fd447e00640 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d9e6cc9330 0x55d9e6cc9710 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd447e00640 1 -- 172.21.15.97:0/3896268402 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v134'171 uv171 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd438004c50 con 0x55d9e6cc9330 2024-09-10T21:35:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d9e6cc9330 msgr2=0x55d9e6cc9710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55d9e6cc9330 0x55d9e6cc9710 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd42c070fc0 msgr2=0x7fd42c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd42c070fc0 0x7fd42c073480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c710 tx=0x7fd44000e040 comp rx=0 tx=0).stop 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c132a0 msgr2=0x55d9e6c2e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 --2- 172.21.15.97:0/3896268402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e6c132a0 0x55d9e6c2e0f0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fd44800a550 tx=0x7fd4480aa5a0 comp rx=0 tx=0).stop 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd450c00640 1 -- 172.21.15.97:0/3896268402 reap_dead start 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 shutdown_connections 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 >> 172.21.15.97:0/3896268402 conn(0x55d9e6b87520 msgr2=0x55d9e6c161a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 shutdown_connections 2024-09-10T21:35:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:16.862+0000 7fd452fe8080 1 -- 172.21.15.97:0/3896268402 wait complete. 2024-09-10T21:35:16.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:35:17.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.010+0000 7fcbf2e00640 1 -- 172.21.15.97:0/2165447148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec0b5d20 msgr2=0x7fcbec0d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.010+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/2165447148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec0b5d20 0x7fcbec0d0350 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009bc0 tx=0x7fcbe401cb60 comp rx=0 tx=0).stop 2024-09-10T21:35:17.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.010+0000 7fcbf2e00640 1 -- 172.21.15.97:0/2165447148 shutdown_connections 2024-09-10T21:35:17.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.010+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/2165447148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec0b5d20 0x7fcbec0d0350 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.010+0000 7fcbf2e00640 1 -- 172.21.15.97:0/2165447148 >> 172.21.15.97:0/2165447148 conn(0x7fcbec0b3c70 msgr2=0x7fcbec0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:17.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.010+0000 7fcbf2e00640 1 -- 172.21.15.97:0/2165447148 shutdown_connections 2024-09-10T21:35:17.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.010+0000 7fcbf2e00640 1 -- 172.21.15.97:0/2165447148 wait complete. 2024-09-10T21:35:17.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 Processor -- start 2024-09-10T21:35:17.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 -- start start 2024-09-10T21:35:17.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec074990 0x7fcbec147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcbec0b5d20 0x7fcbec15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbec0ccda0 0x7fcbec15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec16c590 con 0x7fcbec0ccda0 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec06d2f0 con 0x7fcbec074990 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcbec16c710 con 0x7fcbec0b5d20 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbec0ccda0 0x7fcbec15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbec0ccda0 0x7fcbec15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39672/0 (socket says 172.21.15.97:39672) 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 -- 172.21.15.97:0/3113089281 learned_addr learned my addr 172.21.15.97:0/3113089281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbeb400640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcbec0b5d20 0x7fcbec15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbebe00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec074990 0x7fcbec147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 -- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcbec0b5d20 msgr2=0x7fcbec15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcbec0b5d20 0x7fcbec15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 -- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec074990 msgr2=0x7fcbec147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec074990 0x7fcbec147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 -- 172.21.15.97:0/3113089281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbec15ad00 con 0x7fcbec0ccda0 2024-09-10T21:35:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbebe00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec074990 0x7fcbec147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:17.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf0e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbec0ccda0 0x7fcbec15d590 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009b10 tx=0x7fcbe4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbe3e00640 1 -- 172.21.15.97:0/3113089281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40bb070 con 0x7fcbec0ccda0 2024-09-10T21:35:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbe3e00640 1 -- 172.21.15.97:0/3113089281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbe4004d40 con 0x7fcbec0ccda0 2024-09-10T21:35:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbe3e00640 1 -- 172.21.15.97:0/3113089281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40b2040 con 0x7fcbec0ccda0 2024-09-10T21:35:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbec070400 con 0x7fcbec0ccda0 2024-09-10T21:35:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.014+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbec15afa0 con 0x7fcbec0ccda0 2024-09-10T21:35:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.018+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbec074280 con 0x7fcbec0ccda0 2024-09-10T21:35:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.018+0000 7fcbe3e00640 1 -- 172.21.15.97:0/3113089281 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcbe40b5030 con 0x7fcbec0ccda0 2024-09-10T21:35:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.018+0000 7fcbe3e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcbcc0725e0 0x7fcbcc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.018+0000 7fcbe3e00640 1 -- 172.21.15.97:0/3113089281 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9543+0+0 (secure 0 0 0) 0x7fcbe412de30 con 0x7fcbec0ccda0 2024-09-10T21:35:17.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.018+0000 7fcbebe00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcbcc0725e0 0x7fcbcc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:17.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.018+0000 7fcbebe00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcbcc0725e0 0x7fcbcc074aa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc007a60 tx=0x7fcbdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.022+0000 7fcbe3e00640 1 -- 172.21.15.97:0/3113089281 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcbe4016750 con 0x7fcbec0ccda0 2024-09-10T21:35:17.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.222+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fcbec158590 con 0x7fcbec0ccda0 2024-09-10T21:35:17.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbe3e00640 1 -- 172.21.15.97:0/3113089281 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v324) ==== 162+0+0 (secure 0 0 0) 0x7fcbe40f7f20 con 0x7fcbec0ccda0 2024-09-10T21:35:17.500 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:35:17.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcbcc0725e0 msgr2=0x7fcbcc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcbcc0725e0 0x7fcbcc074aa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc007a60 tx=0x7fcbdc009040 comp rx=0 tx=0).stop 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbec0ccda0 msgr2=0x7fcbec15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbec0ccda0 0x7fcbec15d590 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009b10 tx=0x7fcbe4009220 comp rx=0 tx=0).stop 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 shutdown_connections 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcbcc0725e0 0x7fcbcc074aa0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbec0ccda0 0x7fcbec15d590 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcbec0b5d20 0x7fcbec15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 --2- 172.21.15.97:0/3113089281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbec074990 0x7fcbec147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 >> 172.21.15.97:0/3113089281 conn(0x7fcbec0b3c70 msgr2=0x7fcbec0cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 shutdown_connections 2024-09-10T21:35:17.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.502+0000 7fcbf2e00640 1 -- 172.21.15.97:0/3113089281 wait complete. 2024-09-10T21:35:17.509 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:35:17.510+0000 7f0ff3a00640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:35:17.509 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:17.510+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:35:17.509 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:17.510+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:35:17.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:17.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.670+0000 7f3abf600640 1 -- 172.21.15.97:0/1886296511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a9c004810 msgr2=0x7f3a9c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.670+0000 7f3abf600640 1 --2- 172.21.15.97:0/1886296511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a9c004810 0x7f3a9c004c90 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009b60 tx=0x7f3aa401d270 comp rx=0 tx=0).stop 2024-09-10T21:35:17.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.670+0000 7f3abf600640 1 -- 172.21.15.97:0/1886296511 shutdown_connections 2024-09-10T21:35:17.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.670+0000 7f3abf600640 1 --2- 172.21.15.97:0/1886296511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a9c004810 0x7f3a9c004c90 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.670+0000 7f3abf600640 1 -- 172.21.15.97:0/1886296511 >> 172.21.15.97:0/1886296511 conn(0x7f3ab80d1c30 msgr2=0x7f3ab80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:17.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.670+0000 7f3abf600640 1 -- 172.21.15.97:0/1886296511 shutdown_connections 2024-09-10T21:35:17.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.670+0000 7f3abf600640 1 -- 172.21.15.97:0/1886296511 wait complete. 2024-09-10T21:35:17.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 Processor -- start 2024-09-10T21:35:17.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 -- start start 2024-09-10T21:35:17.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a9c004810 0x7f3ab80c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ab80c5ea0 0x7f3ab80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ab80c6320 0x7f3ab80b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab80ceba0 con 0x7f3ab80c6320 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab80cea20 con 0x7f3a9c004810 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab80ced20 con 0x7f3ab80c5ea0 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ab80c6320 0x7f3ab80b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ab80c6320 0x7f3ab80b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54898/0 (socket says 172.21.15.97:54898) 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 -- 172.21.15.97:0/2674140826 learned_addr learned my addr 172.21.15.97:0/2674140826 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 -- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ab80c5ea0 msgr2=0x7f3ab80b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ab80c5ea0 0x7f3ab80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 -- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a9c004810 msgr2=0x7f3ab80c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a9c004810 0x7f3ab80c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 -- 172.21.15.97:0/2674140826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab80c68f0 con 0x7f3ab80c6320 2024-09-10T21:35:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abd600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ab80c6320 0x7f3ab80b37b0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80b7ce0 tx=0x7f3ab0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3ab4c00640 1 -- 172.21.15.97:0/2674140826 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0011660 con 0x7f3ab80c6320 2024-09-10T21:35:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3ab4c00640 1 -- 172.21.15.97:0/2674140826 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ab0017070 con 0x7f3ab80c6320 2024-09-10T21:35:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3ab4c00640 1 -- 172.21.15.97:0/2674140826 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0011b90 con 0x7f3ab80c6320 2024-09-10T21:35:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81f50b0 con 0x7f3ab80c6320 2024-09-10T21:35:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.674+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81f5670 con 0x7f3ab80c6320 2024-09-10T21:35:17.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.678+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab81f5840 con 0x7f3ab80c6320 2024-09-10T21:35:17.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.678+0000 7f3ab4c00640 1 -- 172.21.15.97:0/2674140826 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3ab001c460 con 0x7f3ab80c6320 2024-09-10T21:35:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.678+0000 7f3ab4c00640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a980725e0 0x7f3a98074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.678+0000 7f3ab4c00640 1 -- 172.21.15.97:0/2674140826 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7f3ab00945f0 con 0x7f3ab80c6320 2024-09-10T21:35:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.678+0000 7f3abcc00640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a980725e0 0x7f3a98074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:17.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.678+0000 7f3abcc00640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a980725e0 0x7f3a98074aa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8001bf0 tx=0x7f3aa8004800 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:17.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.682+0000 7f3ab4c00640 1 -- 172.21.15.97:0/2674140826 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3ab005ecd0 con 0x7f3ab80c6320 2024-09-10T21:35:17.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ab80af6f0 con 0x7f3ab80c6320 2024-09-10T21:35:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3ab4c00640 1 -- 172.21.15.97:0/2674140826 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 326 v326) ==== 95+0+4395 (secure 0 0 0) 0x7f3ab00673f0 con 0x7f3ab80c6320 2024-09-10T21:35:17.881 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:17.881 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":326,"btime":"2024-09-10T21:35:17:514214+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":326,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:17.514150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:17.881 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 326 2024-09-10T21:35:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a980725e0 msgr2=0x7f3a98074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a980725e0 0x7f3a98074aa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8001bf0 tx=0x7f3aa8004800 comp rx=0 tx=0).stop 2024-09-10T21:35:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ab80c6320 msgr2=0x7f3ab80b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ab80c6320 0x7f3ab80b37b0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80b7ce0 tx=0x7f3ab0001640 comp rx=0 tx=0).stop 2024-09-10T21:35:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 shutdown_connections 2024-09-10T21:35:17.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3a980725e0 0x7f3a98074aa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ab80c6320 0x7f3ab80b37b0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3ab80c5ea0 0x7f3ab80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 --2- 172.21.15.97:0/2674140826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a9c004810 0x7f3ab80c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 >> 172.21.15.97:0/2674140826 conn(0x7f3ab80d1c30 msgr2=0x7f3ab8072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 shutdown_connections 2024-09-10T21:35:17.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:17.882+0000 7f3abf600640 1 -- 172.21.15.97:0/2674140826 wait complete. 2024-09-10T21:35:17.903 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:35:17.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- 172.21.15.97:0/2683414645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5340ccc60 msgr2=0x7ff5340b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 --2- 172.21.15.97:0/2683414645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5340ccc60 0x7ff5340b0490 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009900 tx=0x7ff52c01c870 comp rx=0 tx=0).stop 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- 172.21.15.97:0/2683414645 shutdown_connections 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 --2- 172.21.15.97:0/2683414645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5340ccc60 0x7ff5340b0490 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 --2- 172.21.15.97:0/2683414645 >> v2:172.21.15.194:3300/0 conn(0x7ff53415eed0 0x7ff5340d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- 172.21.15.97:0/2683414645 >> 172.21.15.97:0/2683414645 conn(0x7ff5340d1cd0 msgr2=0x7ff5340d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- 172.21.15.97:0/2683414645 shutdown_connections 2024-09-10T21:35:18.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- 172.21.15.97:0/2683414645 wait complete. 2024-09-10T21:35:18.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 Processor -- start 2024-09-10T21:35:18.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- start start 2024-09-10T21:35:18.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5340ccc60 0x7ff5340b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5340cdd40 0x7ff5340cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff53415eed0 0x7ff5340cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff534168880 con 0x7ff53415eed0 2024-09-10T21:35:18.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff534168700 con 0x7ff5340ccc60 2024-09-10T21:35:18.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff53be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff534168a00 con 0x7ff5340cdd40 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff539e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff53415eed0 0x7ff5340cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff539e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff53415eed0 0x7ff5340cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54914/0 (socket says 172.21.15.97:54914) 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.046+0000 7ff539e00640 1 -- 172.21.15.97:0/1835668903 learned_addr learned my addr 172.21.15.97:0/1835668903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff539400640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5340ccc60 0x7ff5340b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff538a00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5340cdd40 0x7ff5340cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff539e00640 1 -- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5340cdd40 msgr2=0x7ff5340cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff539e00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5340cdd40 0x7ff5340cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff539e00640 1 -- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5340ccc60 msgr2=0x7ff5340b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff539e00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5340ccc60 0x7ff5340b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff539e00640 1 -- 172.21.15.97:0/1835668903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff534155ca0 con 0x7ff53415eed0 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff538a00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5340cdd40 0x7ff5340cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff539e00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff53415eed0 0x7ff5340cb590 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0b0040 tx=0x7ff52c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff531600640 1 -- 172.21.15.97:0/1835668903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0bc070 con 0x7ff53415eed0 2024-09-10T21:35:18.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff531600640 1 -- 172.21.15.97:0/1835668903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff52c005030 con 0x7ff53415eed0 2024-09-10T21:35:18.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5341faad0 con 0x7ff53415eed0 2024-09-10T21:35:18.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff531600640 1 -- 172.21.15.97:0/1835668903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0b2040 con 0x7ff53415eed0 2024-09-10T21:35:18.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5341fb090 con 0x7ff53415eed0 2024-09-10T21:35:18.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5340b0490 con 0x7ff53415eed0 2024-09-10T21:35:18.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff531600640 1 -- 172.21.15.97:0/1835668903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff52c005480 con 0x7ff53415eed0 2024-09-10T21:35:18.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.050+0000 7ff531600640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5140725e0 0x7ff514074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.054+0000 7ff531600640 1 -- 172.21.15.97:0/1835668903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7ff52c12da40 con 0x7ff53415eed0 2024-09-10T21:35:18.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.054+0000 7ff539400640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5140725e0 0x7ff514074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.054+0000 7ff539400640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5140725e0 0x7ff514074aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff5340b4990 tx=0x7ff5280095b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:18.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.058+0000 7ff531600640 1 -- 172.21.15.97:0/1835668903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff52c016490 con 0x7ff53415eed0 2024-09-10T21:35:18.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.254+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5340cfd60 con 0x7ff53415eed0 2024-09-10T21:35:18.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.254+0000 7ff531600640 1 -- 172.21.15.97:0/1835668903 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 326 v326) ==== 95+0+4395 (secure 0 0 0) 0x7ff52c0f7ad0 con 0x7ff53415eed0 2024-09-10T21:35:18.255 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:18.255 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":326,"btime":"2024-09-10T21:35:17:514214+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":326,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:17.514150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 326 2024-09-10T21:35:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5140725e0 msgr2=0x7ff514074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5140725e0 0x7ff514074aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff5340b4990 tx=0x7ff5280095b0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff53415eed0 msgr2=0x7ff5340cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff53415eed0 0x7ff5340cb590 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0b0040 tx=0x7ff52c004040 comp rx=0 tx=0).stop 2024-09-10T21:35:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 shutdown_connections 2024-09-10T21:35:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5140725e0 0x7ff514074aa0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff53415eed0 0x7ff5340cb590 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5340cdd40 0x7ff5340cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 --2- 172.21.15.97:0/1835668903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5340ccc60 0x7ff5340b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 >> 172.21.15.97:0/1835668903 conn(0x7ff5340d1cd0 msgr2=0x7ff53414c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 shutdown_connections 2024-09-10T21:35:18.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.258+0000 7ff53be00640 1 -- 172.21.15.97:0/1835668903 wait complete. 2024-09-10T21:35:18.277 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:35:18.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 -- 172.21.15.97:0/1907093582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 msgr2=0x7f1f380cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 --2- 172.21.15.97:0/1907093582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f380cc6d0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f1f380c55b0 tx=0x7f1f2401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 -- 172.21.15.97:0/1907093582 shutdown_connections 2024-09-10T21:35:18.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 --2- 172.21.15.97:0/1907093582 >> v2:172.21.15.194:3300/0 conn(0x7f1f380b0d50 0x7f1f380ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 --2- 172.21.15.97:0/1907093582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f380cc6d0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 -- 172.21.15.97:0/1907093582 >> 172.21.15.97:0/1907093582 conn(0x7f1f380d1c30 msgr2=0x7f1f380d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 -- 172.21.15.97:0/1907093582 shutdown_connections 2024-09-10T21:35:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.418+0000 7f1f3e800640 1 -- 172.21.15.97:0/1907093582 wait complete. 2024-09-10T21:35:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 Processor -- start 2024-09-10T21:35:18.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 -- start start 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f38163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f380b0d50 0x7f1f380b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f380ce640 0x7f1f38153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38168ee0 con 0x7f1f380afdd0 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38168d60 con 0x7f1f380b0d50 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1f38169060 con 0x7f1f380ce640 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f38163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f38163c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54942/0 (socket says 172.21.15.97:54942) 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 -- 172.21.15.97:0/2039160470 learned_addr learned my addr 172.21.15.97:0/2039160470 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f36a00640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f380b0d50 0x7f1f380b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37e00640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f380ce640 0x7f1f38153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 -- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f380ce640 msgr2=0x7f1f38153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f380ce640 0x7f1f38153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 -- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f380b0d50 msgr2=0x7f1f380b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f380b0d50 0x7f1f380b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 -- 172.21.15.97:0/2039160470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f380b4c40 con 0x7f1f380afdd0 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f37400640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f38163c30 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800ba80 tx=0x7f1f2800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f2f400640 1 -- 172.21.15.97:0/2039160470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800cc00 con 0x7f1f380afdd0 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f2f400640 1 -- 172.21.15.97:0/2039160470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f28003980 con 0x7f1f380afdd0 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f2f400640 1 -- 172.21.15.97:0/2039160470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28003e10 con 0x7f1f380afdd0 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f380b4f20 con 0x7f1f380afdd0 2024-09-10T21:35:18.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f38154eb0 con 0x7f1f380afdd0 2024-09-10T21:35:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.422+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f380cea20 con 0x7f1f380afdd0 2024-09-10T21:35:18.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.426+0000 7f1f2f400640 1 -- 172.21.15.97:0/2039160470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1f28003fb0 con 0x7f1f380afdd0 2024-09-10T21:35:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.426+0000 7f1f2f400640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f18072610 0x7f1f18074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.426+0000 7f1f2f400640 1 -- 172.21.15.97:0/2039160470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7f1f280940e0 con 0x7f1f380afdd0 2024-09-10T21:35:18.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.426+0000 7f1f36a00640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f18072610 0x7f1f18074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.426+0000 7f1f36a00640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f18072610 0x7f1f18074ad0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1f240af000 tx=0x7f1f2401b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:18.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.430+0000 7f1f2f400640 1 -- 172.21.15.97:0/2039160470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1f2805e170 con 0x7f1f380afdd0 2024-09-10T21:35:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f38159e90 con 0x7f1f380afdd0 2024-09-10T21:35:18.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f2f400640 1 -- 172.21.15.97:0/2039160470 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 326 v326) ==== 95+0+4395 (secure 0 0 0) 0x7f1f28062ba0 con 0x7f1f380afdd0 2024-09-10T21:35:18.629 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:18.629 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":326,"btime":"2024-09-10T21:35:17:514214+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":277}],"filesystems":[{"mdsmap":{"epoch":326,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:17.514150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:18.629 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 326 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f18072610 msgr2=0x7f1f18074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f18072610 0x7f1f18074ad0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1f240af000 tx=0x7f1f2401b100 comp rx=0 tx=0).stop 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 msgr2=0x7f1f38163c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f38163c30 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800ba80 tx=0x7f1f2800bf40 comp rx=0 tx=0).stop 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 shutdown_connections 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1f18072610 0x7f1f18074ad0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1f380ce640 0x7f1f38153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f380b0d50 0x7f1f380b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 --2- 172.21.15.97:0/2039160470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f380afdd0 0x7f1f38163c30 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 >> 172.21.15.97:0/2039160470 conn(0x7f1f380d1c30 msgr2=0x7f1f3815d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 shutdown_connections 2024-09-10T21:35:18.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.630+0000 7f1f3e800640 1 -- 172.21.15.97:0/2039160470 wait complete. 2024-09-10T21:35:18.650 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2024-09-10T21:35:18.650 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2024-09-10T21:35:18.651 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:18.651 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:18.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:35:18.660 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:35:18.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:18.660 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:35:18.710 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:35:18.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:35:18.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:35:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 -- 172.21.15.97:0/1078213791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 msgr2=0x7f303c0cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 --2- 172.21.15.97:0/1078213791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c0cefe0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f3034009900 tx=0x7f303401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 -- 172.21.15.97:0/1078213791 shutdown_connections 2024-09-10T21:35:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 --2- 172.21.15.97:0/1078213791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c0cefe0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 --2- 172.21.15.97:0/1078213791 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f303c0b3ab0 0x7f303c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 -- 172.21.15.97:0/1078213791 >> 172.21.15.97:0/1078213791 conn(0x7f303c06cb40 msgr2=0x7f303c06cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 -- 172.21.15.97:0/1078213791 shutdown_connections 2024-09-10T21:35:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.898+0000 7f303be00640 1 -- 172.21.15.97:0/1078213791 wait complete. 2024-09-10T21:35:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 Processor -- start 2024-09-10T21:35:18.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 -- start start 2024-09-10T21:35:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f303c0725f0 0x7f303c144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f303c0b3ab0 0x7f303c150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f303c167e60 con 0x7f303c072a60 2024-09-10T21:35:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f303c167fe0 con 0x7f303c0b3ab0 2024-09-10T21:35:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f303c168160 con 0x7f303c0725f0 2024-09-10T21:35:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c14e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54954/0 (socket says 172.21.15.97:54954) 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 -- 172.21.15.97:0/3076211427 learned_addr learned my addr 172.21.15.97:0/3076211427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303aa00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f303c0725f0 0x7f303c144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303b400640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f303c0b3ab0 0x7f303c150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 -- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f303c0725f0 msgr2=0x7f303c144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f303c0725f0 0x7f303c144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 -- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f303c0b3ab0 msgr2=0x7f303c150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f303c0b3ab0 0x7f303c150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 -- 172.21.15.97:0/3076211427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c1f7b70 con 0x7f303c072a60 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303a000640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c14e3f0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f302800c450 tx=0x7f302800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f3033400640 1 -- 172.21.15.97:0/3076211427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028012830 con 0x7f303c072a60 2024-09-10T21:35:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f3033400640 1 -- 172.21.15.97:0/3076211427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3028017070 con 0x7f303c072a60 2024-09-10T21:35:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f3033400640 1 -- 172.21.15.97:0/3076211427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028012d80 con 0x7f303c072a60 2024-09-10T21:35:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c1f7e50 con 0x7f303c072a60 2024-09-10T21:35:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.902+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c1f8410 con 0x7f303c072a60 2024-09-10T21:35:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.906+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c0b4320 con 0x7f303c072a60 2024-09-10T21:35:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.906+0000 7f3033400640 1 -- 172.21.15.97:0/3076211427 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f302801c460 con 0x7f303c072a60 2024-09-10T21:35:18.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.906+0000 7f3033400640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f301c0725e0 0x7f301c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:18.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.906+0000 7f3033400640 1 -- 172.21.15.97:0/3076211427 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7f3028093eb0 con 0x7f303c072a60 2024-09-10T21:35:18.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.906+0000 7f303aa00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f301c0725e0 0x7f301c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:18.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.910+0000 7f303aa00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f301c0725e0 0x7f301c074aa0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f302c005fd0 tx=0x7f302c00a590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:18.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:18.910+0000 7f3033400640 1 -- 172.21.15.97:0/3076211427 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f302805df40 con 0x7f303c072a60 2024-09-10T21:35:19.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.114+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f303c06d9a0 con 0x7f303c072a60 2024-09-10T21:35:19.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.514+0000 7f3033400640 1 -- 172.21.15.97:0/3076211427 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v327) ==== 161+0+0 (secure 0 0 0) 0x7f3028062970 con 0x7f303c072a60 2024-09-10T21:35:19.515 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:35:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f301c0725e0 msgr2=0x7f301c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f301c0725e0 0x7f301c074aa0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f302c005fd0 tx=0x7f302c00a590 comp rx=0 tx=0).stop 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 msgr2=0x7f303c14e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c14e3f0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f302800c450 tx=0x7f302800c910 comp rx=0 tx=0).stop 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 shutdown_connections 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f301c0725e0 0x7f301c074aa0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f303c0b3ab0 0x7f303c150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f303c072a60 0x7f303c14e3f0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 --2- 172.21.15.97:0/3076211427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f303c0725f0 0x7f303c144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 >> 172.21.15.97:0/3076211427 conn(0x7f303c06cb40 msgr2=0x7f303c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 shutdown_connections 2024-09-10T21:35:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.518+0000 7f303be00640 1 -- 172.21.15.97:0/3076211427 wait complete. 2024-09-10T21:35:19.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T21:35:19.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.682+0000 7fdec3400640 1 -- 172.21.15.97:0/906696650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc15fde0 msgr2=0x7fdebc13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:19.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.682+0000 7fdec3400640 1 --2- 172.21.15.97:0/906696650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc15fde0 0x7fdebc13f9b0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009900 tx=0x7fdeb401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- 172.21.15.97:0/906696650 shutdown_connections 2024-09-10T21:35:19.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 --2- 172.21.15.97:0/906696650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc15fde0 0x7fdebc13f9b0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 --2- 172.21.15.97:0/906696650 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdebc0ce5f0 0x7fdebc141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- 172.21.15.97:0/906696650 >> 172.21.15.97:0/906696650 conn(0x7fdebc0b1780 msgr2=0x7fdebc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:19.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- 172.21.15.97:0/906696650 shutdown_connections 2024-09-10T21:35:19.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- 172.21.15.97:0/906696650 wait complete. 2024-09-10T21:35:19.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 Processor -- start 2024-09-10T21:35:19.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- start start 2024-09-10T21:35:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc0ce5f0 0x7fdebc13a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdebc0ce9d0 0x7fdebc1fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:19.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdebc15fde0 0x7fdebc1fb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc168a70 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc168bf0 con 0x7fdebc0ce9d0 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec3400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc168d70 con 0x7fdebc15fde0 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc0ce5f0 0x7fdebc13a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc0ce5f0 0x7fdebc13a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54968/0 (socket says 172.21.15.97:54968) 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 -- 172.21.15.97:0/1670597896 learned_addr learned my addr 172.21.15.97:0/1670597896 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec1400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdebc15fde0 0x7fdebc1fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdebbe00640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdebc0ce9d0 0x7fdebc1fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 -- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdebc15fde0 msgr2=0x7fdebc1fb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdebc15fde0 0x7fdebc1fb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 -- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdebc0ce9d0 msgr2=0x7fdebc1fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdebc0ce9d0 0x7fdebc1fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 -- 172.21.15.97:0/1670597896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc165bf0 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec1400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdebc15fde0 0x7fdebc1fb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.686+0000 7fdec0a00640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc0ce5f0 0x7fdebc13a880 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00ec00 tx=0x7fdeac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdeb8c00640 1 -- 172.21.15.97:0/1670597896 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac010630 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdeb8c00640 1 -- 172.21.15.97:0/1670597896 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdeac0108c0 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc074600 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdeb8c00640 1 -- 172.21.15.97:0/1670597896 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac010cf0 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc165630 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc141d20 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdeb8c00640 1 -- 172.21.15.97:0/1670597896 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdeac018420 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdeb8c00640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde9c0725e0 0x7fde9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:19.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdeb8c00640 1 -- 172.21.15.97:0/1670597896 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7fdeac094ff0 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.690+0000 7fdebbe00640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde9c0725e0 0x7fde9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:19.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.694+0000 7fdebbe00640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde9c0725e0 0x7fde9c074aa0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdebc1555e0 tx=0x7fdea8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:19.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.694+0000 7fdeb8c00640 1 -- 172.21.15.97:0/1670597896 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdeac05f780 con 0x7fdebc0ce5f0 2024-09-10T21:35:19.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:19.922+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fdebc1599d0 con 0x7fdebc0ce5f0 2024-09-10T21:35:20.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.526+0000 7fdeb8c00640 1 -- 172.21.15.97:0/1670597896 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v328) ==== 101+0+0 (secure 0 0 0) 0x7fdeac05f080 con 0x7fdebc0ce5f0 2024-09-10T21:35:20.526 INFO:teuthology.orchestra.run.smithi097.stderr:repaired: restoring rank 6:0 2024-09-10T21:35:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde9c0725e0 msgr2=0x7fde9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde9c0725e0 0x7fde9c074aa0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdebc1555e0 tx=0x7fdea8009040 comp rx=0 tx=0).stop 2024-09-10T21:35:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc0ce5f0 msgr2=0x7fdebc13a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc0ce5f0 0x7fdebc13a880 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00ec00 tx=0x7fdeac00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 shutdown_connections 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fde9c0725e0 0x7fde9c074aa0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdebc15fde0 0x7fdebc1fb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdebc0ce9d0 0x7fdebc1fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 --2- 172.21.15.97:0/1670597896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdebc0ce5f0 0x7fdebc13a880 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 >> 172.21.15.97:0/1670597896 conn(0x7fdebc0b1780 msgr2=0x7fdebc13d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 shutdown_connections 2024-09-10T21:35:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.530+0000 7fdec3400640 1 -- 172.21.15.97:0/1670597896 wait complete. 2024-09-10T21:35:20.548 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T21:35:20.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 Processor -- start 2024-09-10T21:35:20.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 -- start start 2024-09-10T21:35:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 0x55a31ebc6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba9290 0x55a31eaef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba8620 0x55a31eba8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a31ea9cf40 con 0x55a31eba8620 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a31eaa9d00 con 0x55a31eba9290 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc542a49080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55a31ea3ea00 con 0x7fc5240036b0 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba8620 0x55a31eba8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba8620 0x55a31eba8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54996/0 (socket says 172.21.15.97:54996) 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 -- 172.21.15.97:0/3072104957 learned_addr learned my addr 172.21.15.97:0/3072104957 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 -- 172.21.15.97:0/3072104957 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 msgr2=0x55a31ebc6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3072104957 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 0x55a31ebc6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 -- 172.21.15.97:0/3072104957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba9290 msgr2=0x55a31eaef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3072104957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba9290 0x55a31eaef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.594+0000 7fc53fc00640 1 -- 172.21.15.97:0/3072104957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31eba8f70 con 0x55a31eba8620 2024-09-10T21:35:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3072104957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba8620 0x55a31eba8a30 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc538009850 tx=0x7fc524004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9ea9abb6b2de63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53d400640 1 -- 172.21.15.97:0/3072104957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380019c0 con 0x55a31eba8620 2024-09-10T21:35:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53d400640 1 -- 172.21.15.97:0/3072104957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5380074d0 con 0x55a31eba8620 2024-09-10T21:35:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53d400640 1 -- 172.21.15.97:0/3072104957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380078c0 con 0x55a31eba8620 2024-09-10T21:35:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- 172.21.15.97:0/3072104957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba8620 msgr2=0x55a31eba8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 --2- 172.21.15.97:0/3072104957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba8620 0x55a31eba8a30 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc538009850 tx=0x7fc524004130 comp rx=0 tx=0).stop 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- 172.21.15.97:0/3072104957 shutdown_connections 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 --2- 172.21.15.97:0/3072104957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba8620 0x55a31eba8a30 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 --2- 172.21.15.97:0/3072104957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba9290 0x55a31eaef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 --2- 172.21.15.97:0/3072104957 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 0x55a31ebc6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- 172.21.15.97:0/3072104957 >> 172.21.15.97:0/3072104957 conn(0x55a31eb1e520 msgr2=0x55a31ebac360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- 172.21.15.97:0/3072104957 shutdown_connections 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- 172.21.15.97:0/3072104957 wait complete. 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 Processor -- start 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- start start 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba8620 0x55a31ec60960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba9290 0x55a31eba5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 0x55a31eba4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a31ebae3a0 con 0x55a31eba9290 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a31ebae100 con 0x55a31eba8620 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55a31ebae640 con 0x7fc5240036b0 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba9290 0x55a31eba5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba9290 0x55a31eba5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55010/0 (socket says 172.21.15.97:55010) 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 learned_addr learned my addr 172.21.15.97:0/3353739055 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 0x55a31eba4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 msgr2=0x55a31eba4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba8620 0x55a31ec60960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 0x55a31eba4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba8620 msgr2=0x55a31ec60960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba8620 0x55a31ec60960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31eba0a00 con 0x55a31eba9290 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc5240036b0 0x55a31eba4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a31eba8620 0x55a31ec60960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc53e800640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba9290 0x55a31eba5730 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00d590 tx=0x7fc52c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc537e00640 1 -- 172.21.15.97:0/3353739055 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c013070 con 0x55a31eba9290 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc537e00640 1 -- 172.21.15.97:0/3353739055 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc52c00f8e0 con 0x55a31eba9290 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a31eba0ca0 con 0x55a31eba9290 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.598+0000 7fc537e00640 1 -- 172.21.15.97:0/3353739055 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00fcf0 con 0x55a31eba9290 2024-09-10T21:35:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31eba11e0 con 0x55a31eba9290 2024-09-10T21:35:20.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc537e00640 1 -- 172.21.15.97:0/3353739055 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc52c00e3a0 con 0x55a31eba9290 2024-09-10T21:35:20.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc537e00640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc520070fc0 0x7fc520073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc537e00640 1 -- 172.21.15.97:0/3353739055 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7fc52c093550 con 0x55a31eba9290 2024-09-10T21:35:20.603 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pool 2024-09-10T21:35:20.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc520070fc0 0x7fc520073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a31eba5c70 0x55a31eba1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31eba2370 con 0x55a31eba5c70 2024-09-10T21:35:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.602+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a31eba5c70 0x55a31eba1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.606+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc520070fc0 0x7fc520073480 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc530004fd0 tx=0x7fc530002fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.606+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a31eba5c70 0x55a31eba1e30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.606+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(1 601.00000000 [create] v135'1190 uv1164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.606+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31eba2710 con 0x55a31eba5c70 2024-09-10T21:35:20.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.610+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== osd_op_reply(2 601.00000000 [delete] v135'1191 uv1191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.610+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6d750 con 0x55a31eba5c70 2024-09-10T21:35:20.609 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T21:35:20.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.610+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 3 ==== osd_op_reply(3 601.00000000 [create] v135'1192 uv1192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.610+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6eb00 con 0x55a31eba5c70 2024-09-10T21:35:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.614+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v135'1193 uv1193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.614+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6eea0 con 0x55a31eba5c70 2024-09-10T21:35:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.614+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v135'1194 uv1194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.614+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6daf0 con 0x55a31eba5c70 2024-09-10T21:35:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.618+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v135'1195 uv1195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.618+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.618+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 7 ==== osd_op_reply(7 602.00000000 [create] v135'1196 uv1170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.618+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.618+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 8 ==== osd_op_reply(8 602.00000000 [delete] v135'1197 uv1197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.618+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.620 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T21:35:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.622+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 9 ==== osd_op_reply(9 602.00000000 [create] v135'1198 uv1198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.622+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.622+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v135'1199 uv1199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.622+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.626+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v135'1200 uv1200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.626+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.626+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v135'1201 uv1201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.626+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.630+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 13 ==== osd_op_reply(13 10000000000.00000000 [create] v135'1202 uv1177 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.630+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.630+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v135'1203 uv1203 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.630+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.631 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T21:35:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.630+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 15 ==== osd_op_reply(15 10000000000.00000000 [create] v135'1204 uv1204 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.630+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.634+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v135'1205 uv1205 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.634+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.634+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v135'1206 uv1206 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.634+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.638+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v135'1207 uv1207 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.638+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=532b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.638+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v135'1208 uv1208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.638+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.642+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 20 ==== osd_op_reply(20 600.00000000 [create] v135'1209 uv1183 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.642+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.642+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 21 ==== osd_op_reply(21 600.00000000 [delete] v135'1210 uv1210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.642+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.643 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T21:35:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.642+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 22 ==== osd_op_reply(22 600.00000000 [create] v135'1211 uv1211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.642+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.646+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v135'1212 uv1212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.646+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.646+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v135'1213 uv1213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.646+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.650+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v135'1214 uv1214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.650+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.650+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 26 ==== osd_op_reply(26 603.00000000 [create] v135'1215 uv1189 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.650+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.654+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 27 ==== osd_op_reply(27 603.00000000 [delete] v135'1216 uv1216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.654+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.654 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T21:35:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.654+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 28 ==== osd_op_reply(28 603.00000000 [create] v135'1217 uv1217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.654+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.658+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v135'1218 uv1218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.658+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.658+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v135'1219 uv1219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.658+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31eba5c70 2024-09-10T21:35:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.662+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.7 v2:172.21.15.194:6823/3380827268 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v135'1220 uv1220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31eba5c70 2024-09-10T21:35:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.662+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55a31ec6f240 0x55a31ec71630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.662+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec6df10 con 0x55a31ec6f240 2024-09-10T21:35:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.662+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55a31ec6f240 0x55a31ec71630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.662+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55a31ec6f240 0x55a31ec71630 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.662+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== osd_op_reply(32 1.00000000.inode [create] v135'867 uv850 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.662+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec71b70 con 0x55a31ec6f240 2024-09-10T21:35:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.666+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== osd_op_reply(33 1.00000000.inode [delete] v135'868 uv868 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.666+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec71f10 con 0x55a31ec6f240 2024-09-10T21:35:20.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.666+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 3 ==== osd_op_reply(34 1.00000000.inode [create] v135'869 uv869 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.665 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T21:35:20.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.666+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec72760 con 0x55a31ec6f240 2024-09-10T21:35:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.666+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v135'870 uv870 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.666+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec72b00 con 0x55a31ec6f240 2024-09-10T21:35:20.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.670+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v135'871 uv871 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.670+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec72ea0 con 0x55a31ec6f240 2024-09-10T21:35:20.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.670+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 6 ==== osd_op_reply(37 200.00000000 [create] v135'872 uv855 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.670+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec73240 con 0x55a31ec6f240 2024-09-10T21:35:20.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.670+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 7 ==== osd_op_reply(38 200.00000000 [delete] v135'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.670+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.672 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T21:35:20.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.674+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 8 ==== osd_op_reply(39 200.00000000 [create] v135'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.674+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.674+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v135'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.674+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.674+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v135'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.674+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.678+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 11 ==== osd_op_reply(42 200.00000001 [create] v135'877 uv860 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.678+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.678+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 12 ==== osd_op_reply(43 200.00000001 [delete] v135'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.678+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.678 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T21:35:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.678+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 13 ==== osd_op_reply(44 200.00000001 [create] v135'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.678+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~16801 in=16801b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec777b0 con 0x55a31ec6f240 2024-09-10T21:35:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.682+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 14 ==== osd_op_reply(45 200.00000001 [write 0~16801] v135'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.682+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec777b0 con 0x55a31ec6f240 2024-09-10T21:35:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.682+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v135'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.682+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.686+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 16 ==== osd_op_reply(47 606.00000000 [create] v135'882 uv866 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.686+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.686+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 17 ==== osd_op_reply(48 606.00000000 [delete] v135'883 uv883 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.686+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.685 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T21:35:20.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.686+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 18 ==== osd_op_reply(49 606.00000000 [create] v135'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.686+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec745f0 con 0x55a31ec6f240 2024-09-10T21:35:20.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v135'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec745f0 con 0x55a31ec6f240 2024-09-10T21:35:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v135'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec735e0 con 0x55a31ec6f240 2024-09-10T21:35:20.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.6 v2:172.21.15.194:6810/1793675643 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v135'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec6f240 2024-09-10T21:35:20.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55a31ec73c60 0x55a31ec76050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec76620 con 0x55a31ec73c60 2024-09-10T21:35:20.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc53e800640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55a31ec73c60 0x55a31ec76050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.690+0000 7fc53e800640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55a31ec73c60 0x55a31ec76050 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.694+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(53 607.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.694+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec76af0 con 0x55a31ec73c60 2024-09-10T21:35:20.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.694+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== osd_op_reply(54 607.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.694+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec76fd0 con 0x55a31ec73c60 2024-09-10T21:35:20.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.694+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 3 ==== osd_op_reply(55 607.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.694 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T21:35:20.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.694+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec78880 con 0x55a31ec73c60 2024-09-10T21:35:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec78c20 con 0x55a31ec73c60 2024-09-10T21:35:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec77870 con 0x55a31ec73c60 2024-09-10T21:35:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55a31ec78190 0x55a31ec7afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7b510 con 0x55a31ec78190 2024-09-10T21:35:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55a31ec78190 0x55a31ec7afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.698+0000 7fc53fc00640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55a31ec78190 0x55a31ec7afd0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v135'176 uv175 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7b510 con 0x55a31ec78190 2024-09-10T21:35:20.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v135'177 uv177 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7b590 con 0x55a31ec78190 2024-09-10T21:35:20.702 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T21:35:20.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v135'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec774c0 con 0x55a31ec78190 2024-09-10T21:35:20.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v135'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.702+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bb00 con 0x55a31ec73c60 2024-09-10T21:35:20.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.706+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 7 ==== osd_op_reply(63 608.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.706+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7bb00 con 0x55a31ec73c60 2024-09-10T21:35:20.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.706+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 8 ==== osd_op_reply(64 608.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.706+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bb00 con 0x55a31ec73c60 2024-09-10T21:35:20.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.706+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 9 ==== osd_op_reply(65 608.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.706 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T21:35:20.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.706+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7cb10 con 0x55a31ec73c60 2024-09-10T21:35:20.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.710+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.710+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7cb10 con 0x55a31ec73c60 2024-09-10T21:35:20.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.710+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.710+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bb00 con 0x55a31ec73c60 2024-09-10T21:35:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.710+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.710+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bb00 con 0x55a31ec78190 2024-09-10T21:35:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.714+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(69 604.00000000 [create] v135'426 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.714+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7bb00 con 0x55a31ec78190 2024-09-10T21:35:20.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.714+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(70 604.00000000 [delete] v135'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.714+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.714+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 7 ==== osd_op_reply(71 604.00000000 [create] v135'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.714 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T21:35:20.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.714+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.718+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v135'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.718+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.718+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v135'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.718+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.718+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v135'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.718+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.722+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 11 ==== osd_op_reply(75 500.00000000 [create] v135'432 uv425 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.722+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.722+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 12 ==== osd_op_reply(76 500.00000000 [delete] v135'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.722+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.722+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 13 ==== osd_op_reply(77 500.00000000 [create] v135'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.722 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T21:35:20.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.722+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.726+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v135'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.726+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec78190 2024-09-10T21:35:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.726+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v135'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.726+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec73c60 2024-09-10T21:35:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.726+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 13 ==== osd_op_reply(80 mds_snaptable [create] v135'193 uv192 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.726+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec73c60 2024-09-10T21:35:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 14 ==== osd_op_reply(81 mds_snaptable [delete] v135'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec73c60 2024-09-10T21:35:20.728 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T21:35:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 15 ==== osd_op_reply(82 mds_snaptable [create] v135'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec73c60 2024-09-10T21:35:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v135'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7bab0 con 0x55a31ec73c60 2024-09-10T21:35:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.0 v2:172.21.15.97:6824/2907498194 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v135'197 uv197 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec73c60 2024-09-10T21:35:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a31ec7c470 0x55a31ec7e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7eda0 con 0x55a31ec7c470 2024-09-10T21:35:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.730+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a31ec7c470 0x55a31ec7e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.734+0000 7fc53f200640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a31ec7c470 0x55a31ec7e860 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.734+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(85 605.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.734+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7eda0 con 0x55a31ec7c470 2024-09-10T21:35:20.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.734+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== osd_op_reply(86 605.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.734+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.735 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T21:35:20.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 3 ==== osd_op_reply(87 605.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec801d0 con 0x55a31ec7c470 2024-09-10T21:35:20.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec801d0 con 0x55a31ec7c470 2024-09-10T21:35:20.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.738+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.742+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 7 ==== osd_op_reply(91 mds0_inotable [create] v135'386 uv380 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.742+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.742+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 8 ==== osd_op_reply(92 mds0_inotable [delete] v135'387 uv387 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.742+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.742 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T21:35:20.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.742+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 9 ==== osd_op_reply(93 mds0_inotable [create] v135'388 uv388 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.742+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v135'389 uv389 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v135'390 uv390 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 12 ==== osd_op_reply(96 100.00000000 [create] v135'391 uv385 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 13 ==== osd_op_reply(97 100.00000000 [delete] v135'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.746+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.748 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T21:35:20.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.750+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 14 ==== osd_op_reply(98 100.00000000 [create] v135'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.750+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f1c0 con 0x55a31ec7c470 2024-09-10T21:35:20.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.750+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v135'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.750+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec81fd0 con 0x55a31ec7c470 2024-09-10T21:35:20.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.750+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v135'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.750+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.754+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 16 ==== osd_op_reply(101 mds0_sessionmap [create] v135'172 uv171 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.754+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.754+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v135'173 uv173 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.754+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.754 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T21:35:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.754+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 18 ==== osd_op_reply(103 mds0_sessionmap [create] v135'174 uv174 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.754+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v135'175 uv175 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 20 ==== osd_op_reply(105 609.00000000 [create] v135'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 21 ==== osd_op_reply(106 609.00000000 [delete] v135'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.758 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T21:35:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 22 ==== osd_op_reply(107 609.00000000 [create] v135'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.758+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec80680 con 0x55a31ec78190 2024-09-10T21:35:20.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v135'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec80680 con 0x55a31ec78190 2024-09-10T21:35:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v135'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec7f670 con 0x55a31ec78190 2024-09-10T21:35:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 <== osd.3 v2:172.21.15.97:6800/1724813549 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v135'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5380b5020 con 0x55a31ec78190 2024-09-10T21:35:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55a31ec7fb80 0x55a31ec7ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7fb80 2024-09-10T21:35:20.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.762+0000 7fc53e800640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55a31ec7fb80 0x55a31ec7ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.766+0000 7fc53e800640 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55a31ec7fb80 0x55a31ec7ff60 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.766+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(111 400.00000000 [create] v135'196 uv195 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec7fb80 2024-09-10T21:35:20.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.766+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7fb80 2024-09-10T21:35:20.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.766+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== osd_op_reply(112 400.00000000 [delete] v135'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec7fb80 2024-09-10T21:35:20.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.766+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7fb80 2024-09-10T21:35:20.766 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T21:35:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.766+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.2 v2:172.21.15.97:6808/3581819336 3 ==== osd_op_reply(113 400.00000000 [create] v135'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec7fb80 2024-09-10T21:35:20.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.766+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7fb80 2024-09-10T21:35:20.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.770+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.2 v2:172.21.15.97:6808/3581819336 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v135'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec7fb80 2024-09-10T21:35:20.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.770+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7fb80 2024-09-10T21:35:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.770+0000 7fc53e800640 1 -- 172.21.15.97:0/3353739055 <== osd.2 v2:172.21.15.97:6808/3581819336 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v135'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc52c020070 con 0x55a31ec7fb80 2024-09-10T21:35:20.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.770+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.770+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 17 ==== osd_op_reply(116 100.00000000.inode [create] v135'480 uv472 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.770+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.774+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 18 ==== osd_op_reply(117 100.00000000.inode [delete] v135'481 uv481 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.774+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.773 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T21:35:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.774+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 19 ==== osd_op_reply(118 100.00000000.inode [create] v135'482 uv482 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.774+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.774+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v135'483 uv483 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.774+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v135'484 uv484 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 22 ==== osd_op_reply(121 1.00000000 [create] v135'485 uv479 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 23 ==== osd_op_reply(122 1.00000000 [delete] v135'486 uv486 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.779 INFO:teuthology.orchestra.run.smithi097.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T21:35:20.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 24 ==== osd_op_reply(123 1.00000000 [create] v135'487 uv487 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.778+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.782+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v135'488 uv488 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.782+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.782+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v135'489 uv489 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.782+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec804a0 con 0x55a31ec7c470 2024-09-10T21:35:20.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.782+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v135'490 uv490 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.782+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55a31ec81ae0 con 0x55a31ec7c470 2024-09-10T21:35:20.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc53f200640 1 -- 172.21.15.97:0/3353739055 <== osd.1 v2:172.21.15.97:6809/3744897448 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v135'491 uv491 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc53000aae0 con 0x55a31ec7c470 2024-09-10T21:35:20.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55a31ec73c60 msgr2=0x55a31ec76050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x55a31ec73c60 0x55a31ec76050 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a31ec7c470 msgr2=0x55a31ec7e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55a31ec7c470 0x55a31ec7e860 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55a31ec7fb80 msgr2=0x55a31ec7ff60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x55a31ec7fb80 0x55a31ec7ff60 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55a31ec78190 msgr2=0x55a31ec7afd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x55a31ec78190 0x55a31ec7afd0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55a31ec6f240 msgr2=0x55a31ec71630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55a31ec6f240 0x55a31ec71630 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 reap_dead start 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc53fc00640 1 -- 172.21.15.97:0/3353739055 reap_dead start 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a31eba5c70 msgr2=0x55a31eba1e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a31eba5c70 0x55a31eba1e30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc520070fc0 msgr2=0x7fc520073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc520070fc0 0x7fc520073480 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc530004fd0 tx=0x7fc530002fc0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba9290 msgr2=0x55a31eba5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba9290 0x55a31eba5730 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00d590 tx=0x7fc52c00da50 comp rx=0 tx=0).stop 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 shutdown_connections 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x55a31ec6f240 0x55a31ec71630 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x55a31eba5c70 0x55a31eba1e30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc520070fc0 0x7fc520073480 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 --2- 172.21.15.97:0/3353739055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a31eba9290 0x55a31eba5730 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 >> 172.21.15.97:0/3353739055 conn(0x55a31eb1e520 msgr2=0x55a31ebac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 shutdown_connections 2024-09-10T21:35:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.786+0000 7fc542a49080 1 -- 172.21.15.97:0/3353739055 wait complete. 2024-09-10T21:35:20.791 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:35:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 Processor -- start 2024-09-10T21:35:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- start start 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f22fc70 0x55651f3079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f2e8ca0 0x55651f300b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f3010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55651f17da00 con 0x55651f22fc70 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55651f1dbf40 con 0x55651f2e9080 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55651f30f2d0 con 0x55651f2e8ca0 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f22fc70 0x55651f3079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f22fc70 0x55651f3079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55034/0 (socket says 172.21.15.97:55034) 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 -- 172.21.15.97:0/1165326829 learned_addr learned my addr 172.21.15.97:0/1165326829 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ade00640 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f3010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8aca00640 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f2e8ca0 0x55651f300b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 -- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f2e8ca0 msgr2=0x55651f300b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f2e8ca0 0x55651f300b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 -- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 msgr2=0x55651f3010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f3010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 -- 172.21.15.97:0/1165326829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651f156c10 con 0x55651f22fc70 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8ad400640 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f22fc70 0x55651f3079f0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000b7d0 tx=0x7fd8a000bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fc36322ca6fb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8a7400640 1 -- 172.21.15.97:0/1165326829 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000abb0 con 0x55651f22fc70 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8a7400640 1 -- 172.21.15.97:0/1165326829 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8a000ad50 con 0x55651f22fc70 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8a7400640 1 -- 172.21.15.97:0/1165326829 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00086c0 con 0x55651f22fc70 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- 172.21.15.97:0/1165326829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f22fc70 msgr2=0x55651f3079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f22fc70 0x55651f3079f0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000b7d0 tx=0x7fd8a000bc90 comp rx=0 tx=0).stop 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- 172.21.15.97:0/1165326829 shutdown_connections 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f3010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f2e8ca0 0x55651f300b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 --2- 172.21.15.97:0/1165326829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f22fc70 0x55651f3079f0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- 172.21.15.97:0/1165326829 >> 172.21.15.97:0/1165326829 conn(0x55651f22e140 msgr2=0x55651f302050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:20.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- 172.21.15.97:0/1165326829 shutdown_connections 2024-09-10T21:35:20.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.882+0000 7fd8b022f080 1 -- 172.21.15.97:0/1165326829 wait complete. 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 Processor -- start 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 -- start start 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f22fc70 0x55651f39fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f2e8ca0 0x55651f26f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f26f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55651f2e84a0 con 0x55651f2e8ca0 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55651f302b80 con 0x55651f2e9080 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55651f3028e0 con 0x55651f22fc70 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f2e8ca0 0x55651f26f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f2e8ca0 0x55651f26f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55042/0 (socket says 172.21.15.97:55042) 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 -- 172.21.15.97:0/3340259335 learned_addr learned my addr 172.21.15.97:0/3340259335 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8ad400640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f22fc70 0x55651f39fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8ade00640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f26f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 -- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f22fc70 msgr2=0x55651f39fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55651f22fc70 0x55651f39fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 -- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 msgr2=0x55651f26f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f26f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 -- 172.21.15.97:0/3340259335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651f26fd70 con 0x55651f2e8ca0 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8ade00640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651f2e9080 0x55651f26f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8aca00640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f2e8ca0 0x55651f26f110 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00dee0 tx=0x7fd89c00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8a6000640 1 -- 172.21.15.97:0/3340259335 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c014020 con 0x55651f2e8ca0 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8a6000640 1 -- 172.21.15.97:0/3340259335 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd89c001400 con 0x55651f2e8ca0 2024-09-10T21:35:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8a6000640 1 -- 172.21.15.97:0/3340259335 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00a040 con 0x55651f2e8ca0 2024-09-10T21:35:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55651f270010 con 0x55651f2e8ca0 2024-09-10T21:35:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.886+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651f270550 con 0x55651f2e8ca0 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8a6000640 1 -- 172.21.15.97:0/3340259335 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd89c001830 con 0x55651f2e8ca0 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8a6000640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd890070fc0 0x7fd890073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8a6000640 1 -- 172.21.15.97:0/3340259335 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7fd89c092ff0 con 0x55651f2e8ca0 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8ad400640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd890070fc0 0x7fd890073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8b022f080 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55651f39e270 0x55651f30ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55651f30bfc0 con 0x55651f39e270 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8ade00640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55651f39e270 0x55651f30ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8ad400640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd890070fc0 0x7fd890073480 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c10 tx=0x7fd8a0019040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.890+0000 7fd8ade00640 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55651f39e270 0x55651f30ba80 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8ade00640 1 -- 172.21.15.97:0/3340259335 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v135'492 uv492 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd8a8002050 con 0x55651f39e270 2024-09-10T21:35:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55651f39e270 msgr2=0x55651f30ba80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x55651f39e270 0x55651f30ba80 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd890070fc0 msgr2=0x7fd890073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd890070fc0 0x7fd890073480 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c10 tx=0x7fd8a0019040 comp rx=0 tx=0).stop 2024-09-10T21:35:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f2e8ca0 msgr2=0x55651f26f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 --2- 172.21.15.97:0/3340259335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55651f2e8ca0 0x55651f26f110 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00dee0 tx=0x7fd89c00b5c0 comp rx=0 tx=0).stop 2024-09-10T21:35:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8ade00640 1 -- 172.21.15.97:0/3340259335 reap_dead start 2024-09-10T21:35:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 shutdown_connections 2024-09-10T21:35:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 >> 172.21.15.97:0/3340259335 conn(0x55651f22e140 msgr2=0x55651f2e7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 shutdown_connections 2024-09-10T21:35:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:20.894+0000 7fd8b022f080 1 -- 172.21.15.97:0/3340259335 wait complete. 2024-09-10T21:35:20.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:35:21.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 -- 172.21.15.97:0/629743382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c15bdd0 msgr2=0x7f1c3c074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 --2- 172.21.15.97:0/629743382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c15bdd0 0x7f1c3c074380 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0b07d0 tx=0x7f1c2c01c7d0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 -- 172.21.15.97:0/629743382 shutdown_connections 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 --2- 172.21.15.97:0/629743382 >> v2:172.21.15.194:3300/0 conn(0x7f1c3c0cfad0 0x7f1c3c0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 --2- 172.21.15.97:0/629743382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c15bdd0 0x7f1c3c074380 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 -- 172.21.15.97:0/629743382 >> 172.21.15.97:0/629743382 conn(0x7f1c3c0b28c0 msgr2=0x7f1c3c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 -- 172.21.15.97:0/629743382 shutdown_connections 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 -- 172.21.15.97:0/629743382 wait complete. 2024-09-10T21:35:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 Processor -- start 2024-09-10T21:35:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.042+0000 7f1c42c00640 1 -- start start 2024-09-10T21:35:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c3c06cb40 0x7f1c3c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c0cfad0 0x7f1c3c1fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c3c15bdd0 0x7f1c3c15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c169340 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c1691c0 con 0x7f1c3c06cb40 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3c1694c0 con 0x7f1c3c15bdd0 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c0cfad0 0x7f1c3c1fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c0cfad0 0x7f1c3c1fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55060/0 (socket says 172.21.15.97:55060) 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 -- 172.21.15.97:0/1524998909 learned_addr learned my addr 172.21.15.97:0/1524998909 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3be00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c3c06cb40 0x7f1c3c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c40c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c3c15bdd0 0x7f1c3c15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 -- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c3c15bdd0 msgr2=0x7f1c3c15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c3c15bdd0 0x7f1c3c15ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 -- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c3c06cb40 msgr2=0x7f1c3c151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c3c06cb40 0x7f1c3c151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 -- 172.21.15.97:0/1524998909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c152260 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c40c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c3c15bdd0 0x7f1c3c15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3be00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c3c06cb40 0x7f1c3c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c3b400640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c0cfad0 0x7f1c3c1fc180 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800d980 tx=0x7f1c2800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c33e00640 1 -- 172.21.15.97:0/1524998909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28017030 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c33e00640 1 -- 172.21.15.97:0/1524998909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c28009b20 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c33e00640 1 -- 172.21.15.97:0/1524998909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28004020 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3c152540 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3c1fbc60 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c3c1fb770 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.046+0000 7f1c33e00640 1 -- 172.21.15.97:0/1524998909 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1c28024060 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.050+0000 7f1c33e00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c1c072610 0x7f1c1c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.050+0000 7f1c33e00640 1 -- 172.21.15.97:0/1524998909 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7f1c28094480 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.050+0000 7f1c3be00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c1c072610 0x7f1c1c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.050+0000 7f1c3be00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c1c072610 0x7f1c1c074ad0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0af000 tx=0x7f1c2c01b010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:21.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.054+0000 7f1c33e00640 1 -- 172.21.15.97:0/1524998909 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1c2805ebf0 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.250+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1c3c0cff40 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c33e00640 1 -- 172.21.15.97:0/1524998909 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v329) ==== 162+0+0 (secure 0 0 0) 0x7f1c2805e4f0 con 0x7f1c3c0cfad0 2024-09-10T21:35:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:35:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c1c072610 msgr2=0x7f1c1c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c1c072610 0x7f1c1c074ad0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0af000 tx=0x7f1c2c01b010 comp rx=0 tx=0).stop 2024-09-10T21:35:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c0cfad0 msgr2=0x7f1c3c1fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c0cfad0 0x7f1c3c1fc180 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800d980 tx=0x7f1c2800de40 comp rx=0 tx=0).stop 2024-09-10T21:35:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 shutdown_connections 2024-09-10T21:35:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1c1c072610 0x7f1c1c074ad0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c3c15bdd0 0x7f1c3c15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c3c0cfad0 0x7f1c3c1fc180 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 --2- 172.21.15.97:0/1524998909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c3c06cb40 0x7f1c3c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 >> 172.21.15.97:0/1524998909 conn(0x7f1c3c0b28c0 msgr2=0x7f1c3c0d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 shutdown_connections 2024-09-10T21:35:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.530+0000 7f1c42c00640 1 -- 172.21.15.97:0/1524998909 wait complete. 2024-09-10T21:35:21.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 -- 172.21.15.97:0/2161810509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0b1560 msgr2=0x7f7a8c0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 --2- 172.21.15.97:0/2161810509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0b1560 0x7f7a8c0b1990 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009c80 tx=0x7f7a8401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 -- 172.21.15.97:0/2161810509 shutdown_connections 2024-09-10T21:35:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 --2- 172.21.15.97:0/2161810509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0b1560 0x7f7a8c0b1990 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 --2- 172.21.15.97:0/2161810509 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a8c0ce830 0x7f7a8c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 -- 172.21.15.97:0/2161810509 >> 172.21.15.97:0/2161810509 conn(0x7f7a8c0b37c0 msgr2=0x7f7a8c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:21.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 -- 172.21.15.97:0/2161810509 shutdown_connections 2024-09-10T21:35:21.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.694+0000 7f7a94600640 1 -- 172.21.15.97:0/2161810509 wait complete. 2024-09-10T21:35:21.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 Processor -- start 2024-09-10T21:35:21.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 -- start start 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a8c0b1560 0x7f7a8c0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0ce830 0x7f7a8c0b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a8c0cec10 0x7f7a8c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0ce1a0 con 0x7f7a8c0ce830 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0cde60 con 0x7f7a8c0cec10 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c0ce320 con 0x7f7a8c0b1560 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0ce830 0x7f7a8c0b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0ce830 0x7f7a8c0b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55078/0 (socket says 172.21.15.97:55078) 2024-09-10T21:35:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a92600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a8c0cec10 0x7f7a8c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 -- 172.21.15.97:0/4006979518 learned_addr learned my addr 172.21.15.97:0/4006979518 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91c00640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a8c0b1560 0x7f7a8c0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 -- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a8c0b1560 msgr2=0x7f7a8c0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a8c0b1560 0x7f7a8c0c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 -- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a8c0cec10 msgr2=0x7f7a8c0b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a8c0cec10 0x7f7a8c0b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 -- 172.21.15.97:0/4006979518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c0b6680 con 0x7f7a8c0ce830 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a91200640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0ce830 0x7f7a8c0b5b10 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c4b0 tx=0x7f7a7800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a89600640 1 -- 172.21.15.97:0/4006979518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a780052d0 con 0x7f7a8c0ce830 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a89600640 1 -- 172.21.15.97:0/4006979518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a78005470 con 0x7f7a8c0ce830 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.698+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c0b6960 con 0x7f7a8c0ce830 2024-09-10T21:35:21.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a89600640 1 -- 172.21.15.97:0/4006979518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78012870 con 0x7f7a8c0ce830 2024-09-10T21:35:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1f4e70 con 0x7f7a8c0ce830 2024-09-10T21:35:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c0b7af0 con 0x7f7a8c0ce830 2024-09-10T21:35:21.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a89600640 1 -- 172.21.15.97:0/4006979518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7a78003600 con 0x7f7a8c0ce830 2024-09-10T21:35:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a89600640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a6c0725e0 0x7f7a6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a89600640 1 -- 172.21.15.97:0/4006979518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9629+0+0 (secure 0 0 0) 0x7f7a78093cc0 con 0x7f7a8c0ce830 2024-09-10T21:35:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a91c00640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a6c0725e0 0x7f7a6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:21.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.702+0000 7f7a91c00640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a6c0725e0 0x7f7a6c074aa0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7a80004630 tx=0x7f7a8000b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:21.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.706+0000 7f7a89600640 1 -- 172.21.15.97:0/4006979518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7a7809a080 con 0x7f7a8c0ce830 2024-09-10T21:35:21.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.902+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a8c0ca5c0 con 0x7f7a8c0ce830 2024-09-10T21:35:21.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a89600640 1 -- 172.21.15.97:0/4006979518 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 330 v330) ==== 95+0+4404 (secure 0 0 0) 0x7f7a7805dd60 con 0x7f7a8c0ce830 2024-09-10T21:35:21.904 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:21.904 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":330,"btime":"2024-09-10T21:35:21:534970+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":330,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:21.534956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8051":{"gid":8051,"name":"c","rank":0,"incarnation":330,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/3450288328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3450288328},{"type":"v1","addr":"172.21.15.97:6835","nonce":3450288328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:21.904 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 330 2024-09-10T21:35:21.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a6c0725e0 msgr2=0x7f7a6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a6c0725e0 0x7f7a6c074aa0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7a80004630 tx=0x7f7a8000b390 comp rx=0 tx=0).stop 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0ce830 msgr2=0x7f7a8c0b5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0ce830 0x7f7a8c0b5b10 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c4b0 tx=0x7f7a7800c970 comp rx=0 tx=0).stop 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 shutdown_connections 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7a6c0725e0 0x7f7a6c074aa0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a8c0cec10 0x7f7a8c0b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c0ce830 0x7f7a8c0b5b10 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 --2- 172.21.15.97:0/4006979518 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7a8c0b1560 0x7f7a8c0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 >> 172.21.15.97:0/4006979518 conn(0x7f7a8c0b37c0 msgr2=0x7f7a8c06de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 shutdown_connections 2024-09-10T21:35:21.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:21.906+0000 7f7a94600640 1 -- 172.21.15.97:0/4006979518 wait complete. 2024-09-10T21:35:21.923 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T21:35:22.263 INFO:tasks.ceph.mds.a.smithi097.stderr: -42> 2024-09-10T21:35:17.510+0000 7f0ff3a00640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:35:22.302 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:35:24.549 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:35:24.550+0000 7f8e53a00640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:35:24.549 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:35:24.550+0000 7f8e53a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T21:35:24.556 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:24.558+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:35:24.556 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:24.558+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T21:35:25.068 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-09-10T21:35:25.070+0000 7f8e58a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:26.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- 172.21.15.97:0/192665693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 msgr2=0x7f28ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 --2- 172.21.15.97:0/192665693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 0x7f28ec0051d0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f2904070a60 tx=0x7f28fc01d1a0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290a400640 1 -- 172.21.15.97:0/192665693 reap_dead start 2024-09-10T21:35:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- 172.21.15.97:0/192665693 shutdown_connections 2024-09-10T21:35:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- 172.21.15.97:0/192665693 >> 172.21.15.97:0/192665693 conn(0x7f29040b3c70 msgr2=0x7f29040b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- 172.21.15.97:0/192665693 shutdown_connections 2024-09-10T21:35:27.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- 172.21.15.97:0/192665693 wait complete. 2024-09-10T21:35:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 Processor -- start 2024-09-10T21:35:27.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- start start 2024-09-10T21:35:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 0x7f290409da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29041f5870 0x7f29041f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29041f7400 0x7f29041598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f29040b61d0 con 0x7f28ec004d50 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f29040b6050 con 0x7f29041f7400 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290c400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f290415b720 con 0x7f29041f5870 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 0x7f290409da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 0x7f290409da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55094/0 (socket says 172.21.15.97:55094) 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 -- 172.21.15.97:0/600412883 learned_addr learned my addr 172.21.15.97:0/600412883 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290a400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29041f7400 0x7f29041598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909000640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29041f5870 0x7f29041f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 -- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29041f5870 msgr2=0x7f29041f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29041f5870 0x7f29041f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 -- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29041f7400 msgr2=0x7f29041598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29041f7400 0x7f29041598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 -- 172.21.15.97:0/600412883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29041f5670 con 0x7f28ec004d50 2024-09-10T21:35:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f290a400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29041f7400 0x7f29041598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2909a00640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 0x7f290409da80 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f28f8006600 tx=0x7f28f8001740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.086+0000 7f2901600640 1 -- 172.21.15.97:0/600412883 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8011700 con 0x7f28ec004d50 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f2901600640 1 -- 172.21.15.97:0/600412883 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28f8016020 con 0x7f28ec004d50 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29041f5cb0 con 0x7f28ec004d50 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f2901600640 1 -- 172.21.15.97:0/600412883 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8011b40 con 0x7f28ec004d50 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2904157100 con 0x7f28ec004d50 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290415bcb0 con 0x7f28ec004d50 2024-09-10T21:35:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f2901600640 1 -- 172.21.15.97:0/600412883 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f28f8011ce0 con 0x7f28ec004d50 2024-09-10T21:35:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f2901600640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f28e40725e0 0x7f28e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f2901600640 1 -- 172.21.15.97:0/600412883 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f28f8094fe0 con 0x7f28ec004d50 2024-09-10T21:35:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.090+0000 7f2909000640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f28e40725e0 0x7f28e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.094+0000 7f2909000640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f28e40725e0 0x7f28e4074aa0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f29040b5cb0 tx=0x7f28f00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.094+0000 7f2901600640 1 -- 172.21.15.97:0/600412883 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f28f8020020 con 0x7f28ec004d50 2024-09-10T21:35:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2904069820 con 0x7f28ec004d50 2024-09-10T21:35:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f2901600640 1 -- 172.21.15.97:0/600412883 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) ==== 95+0+4396 (secure 0 0 0) 0x7f29041f5670 con 0x7f28ec004d50 2024-09-10T21:35:27.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:27.292 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":334,"btime":"2024-09-10T21:35:24:560912+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:24.560907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:27.292 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 334 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f28e40725e0 msgr2=0x7f28e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f28e40725e0 0x7f28e4074aa0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f29040b5cb0 tx=0x7f28f00091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 msgr2=0x7f290409da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 0x7f290409da80 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f28f8006600 tx=0x7f28f8001740 comp rx=0 tx=0).stop 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 shutdown_connections 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f28e40725e0 0x7f28e4074aa0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29041f7400 0x7f29041598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29041f5870 0x7f29041f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 --2- 172.21.15.97:0/600412883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28ec004d50 0x7f290409da80 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 >> 172.21.15.97:0/600412883 conn(0x7f29040b3c70 msgr2=0x7f290406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 shutdown_connections 2024-09-10T21:35:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.294+0000 7f290c400640 1 -- 172.21.15.97:0/600412883 wait complete. 2024-09-10T21:35:27.314 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T21:35:27.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 -- 172.21.15.97:0/3308709339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9990004da0 msgr2=0x7f9990005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 --2- 172.21.15.97:0/3308709339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9990004da0 0x7f9990005200 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009eb0 tx=0x7f99a001d5b0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99ae000640 1 -- 172.21.15.97:0/3308709339 reap_dead start 2024-09-10T21:35:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 -- 172.21.15.97:0/3308709339 shutdown_connections 2024-09-10T21:35:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 -- 172.21.15.97:0/3308709339 >> 172.21.15.97:0/3308709339 conn(0x7f99a80d1c60 msgr2=0x7f99a80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 -- 172.21.15.97:0/3308709339 shutdown_connections 2024-09-10T21:35:27.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 -- 172.21.15.97:0/3308709339 wait complete. 2024-09-10T21:35:27.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 Processor -- start 2024-09-10T21:35:27.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 -- start start 2024-09-10T21:35:27.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.458+0000 7f99b0000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9990004da0 0x7f99a815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99a80c7c50 0x7f99a815a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99a815b610 0x7f99a815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99a80b3450 con 0x7f99a815b610 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f99a80b32d0 con 0x7f9990004da0 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f99a80b0350 con 0x7f99a80c7c50 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99a815b610 0x7f99a815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99a815b610 0x7f99a815bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42620/0 (socket says 172.21.15.97:42620) 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 -- 172.21.15.97:0/2314013602 learned_addr learned my addr 172.21.15.97:0/2314013602 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:27.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ad600640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9990004da0 0x7f99a815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99acc00640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99a80c7c50 0x7f99a815a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 -- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99a80c7c50 msgr2=0x7f99a815a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99a80c7c50 0x7f99a815a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 -- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9990004da0 msgr2=0x7f99a815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9990004da0 0x7f99a815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 -- 172.21.15.97:0/2314013602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a80c8120 con 0x7f99a815b610 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99ae000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99a815b610 0x7f99a815bac0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f99a0002240 tx=0x7f99a00abe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99a5600640 1 -- 172.21.15.97:0/2314013602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0009200 con 0x7f99a815b610 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99a5600640 1 -- 172.21.15.97:0/2314013602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99a00abea0 con 0x7f99a815b610 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99a5600640 1 -- 172.21.15.97:0/2314013602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a00a94f0 con 0x7f99a815b610 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a80c8400 con 0x7f99a815b610 2024-09-10T21:35:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a806de60 con 0x7f99a815b610 2024-09-10T21:35:27.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a806dc50 con 0x7f99a815b610 2024-09-10T21:35:27.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.462+0000 7f99a5600640 1 -- 172.21.15.97:0/2314013602 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f99a0004a50 con 0x7f99a815b610 2024-09-10T21:35:27.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.466+0000 7f99a5600640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9988072610 0x7f9988074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.466+0000 7f99a5600640 1 -- 172.21.15.97:0/2314013602 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f99a012edb0 con 0x7f99a815b610 2024-09-10T21:35:27.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.466+0000 7f99ad600640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9988072610 0x7f9988074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.466+0000 7f99ad600640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9988072610 0x7f9988074ad0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f999c006fd0 tx=0x7f999c006d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:27.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.470+0000 7f99a5600640 1 -- 172.21.15.97:0/2314013602 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99a0133070 con 0x7f99a815b610 2024-09-10T21:35:27.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.666+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99a80afc40 con 0x7f99a815b610 2024-09-10T21:35:27.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.666+0000 7f99a5600640 1 -- 172.21.15.97:0/2314013602 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) ==== 95+0+4396 (secure 0 0 0) 0x7f99a00ad030 con 0x7f99a815b610 2024-09-10T21:35:27.666 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:27.666 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":334,"btime":"2024-09-10T21:35:24:560912+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:24.560907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:27.666 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 334 2024-09-10T21:35:27.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9988072610 msgr2=0x7f9988074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9988072610 0x7f9988074ad0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f999c006fd0 tx=0x7f999c006d30 comp rx=0 tx=0).stop 2024-09-10T21:35:27.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99a815b610 msgr2=0x7f99a815bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99a815b610 0x7f99a815bac0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f99a0002240 tx=0x7f99a00abe30 comp rx=0 tx=0).stop 2024-09-10T21:35:27.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 shutdown_connections 2024-09-10T21:35:27.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9988072610 0x7f9988074ad0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99a815b610 0x7f99a815bac0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f99a80c7c50 0x7f99a815a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 --2- 172.21.15.97:0/2314013602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9990004da0 0x7f99a815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 >> 172.21.15.97:0/2314013602 conn(0x7f99a80d1c60 msgr2=0x7f99a80cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:27.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 shutdown_connections 2024-09-10T21:35:27.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.670+0000 7f99b0000640 1 -- 172.21.15.97:0/2314013602 wait complete. 2024-09-10T21:35:27.688 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:35:27.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.830+0000 7f4fea000640 1 -- 172.21.15.97:0/1512737106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc8007980 msgr2=0x7f4fc8009e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.830+0000 7f4fea000640 1 --2- 172.21.15.97:0/1512737106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc8007980 0x7f4fc8009e10 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009b60 tx=0x7f4fd001cfb0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.830+0000 7f4fe3e00640 1 -- 172.21.15.97:0/1512737106 reap_dead start 2024-09-10T21:35:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.830+0000 7f4fea000640 1 -- 172.21.15.97:0/1512737106 shutdown_connections 2024-09-10T21:35:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.830+0000 7f4fea000640 1 -- 172.21.15.97:0/1512737106 >> 172.21.15.97:0/1512737106 conn(0x7f4fe40b1780 msgr2=0x7f4fe40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.830+0000 7f4fea000640 1 -- 172.21.15.97:0/1512737106 shutdown_connections 2024-09-10T21:35:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.830+0000 7f4fea000640 1 -- 172.21.15.97:0/1512737106 wait complete. 2024-09-10T21:35:27.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 Processor -- start 2024-09-10T21:35:27.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 -- start start 2024-09-10T21:35:27.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fc8007980 0x7f4fe4148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe41515b0 0x7f4fe4151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe4155f90 0x7f4fe4156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe40729c0 con 0x7f4fe4155f90 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4072840 con 0x7f4fc8007980 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe4072b40 con 0x7f4fe41515b0 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe4155f90 0x7f4fe4156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe4155f90 0x7f4fe4156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42638/0 (socket says 172.21.15.97:42638) 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 -- 172.21.15.97:0/2284608193 learned_addr learned my addr 172.21.15.97:0/2284608193 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3400640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fc8007980 0x7f4fe4148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe2a00640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe41515b0 0x7f4fe4151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 -- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe41515b0 msgr2=0x7f4fe4151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe41515b0 0x7f4fe4151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 -- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fc8007980 msgr2=0x7f4fe4148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:27.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fc8007980 0x7f4fe4148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:27.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 -- 172.21.15.97:0/2284608193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe4156c50 con 0x7f4fe4155f90 2024-09-10T21:35:27.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fe3e00640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe4155f90 0x7f4fe4156410 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc006fd0 tx=0x7f4fdc006640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:27.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fdb400640 1 -- 172.21.15.97:0/2284608193 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc003b80 con 0x7f4fe4155f90 2024-09-10T21:35:27.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fdb400640 1 -- 172.21.15.97:0/2284608193 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fdc003d20 con 0x7f4fe4155f90 2024-09-10T21:35:27.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fdb400640 1 -- 172.21.15.97:0/2284608193 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc011b00 con 0x7f4fe4155f90 2024-09-10T21:35:27.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe4148a60 con 0x7f4fe4155f90 2024-09-10T21:35:27.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.834+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe4154c30 con 0x7f4fe4155f90 2024-09-10T21:35:27.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.838+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe406e3b0 con 0x7f4fe4155f90 2024-09-10T21:35:27.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.838+0000 7f4fdb400640 1 -- 172.21.15.97:0/2284608193 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4fdc010040 con 0x7f4fe4155f90 2024-09-10T21:35:27.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.838+0000 7f4fdb400640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fc4072610 0x7f4fc4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:27.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.838+0000 7f4fdb400640 1 -- 172.21.15.97:0/2284608193 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f4fdc0978f0 con 0x7f4fe4155f90 2024-09-10T21:35:27.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.838+0000 7f4fe3400640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fc4072610 0x7f4fc4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:27.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.838+0000 7f4fe3400640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fc4072610 0x7f4fc4074ad0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c10 tx=0x7f4fd4001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:27.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:27.842+0000 7f4fdb400640 1 -- 172.21.15.97:0/2284608193 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4fdc061f70 con 0x7f4fe4155f90 2024-09-10T21:35:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.038+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe406aad0 con 0x7f4fe4155f90 2024-09-10T21:35:28.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fdb400640 1 -- 172.21.15.97:0/2284608193 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) ==== 95+0+4396 (secure 0 0 0) 0x7f4fdc061870 con 0x7f4fe4155f90 2024-09-10T21:35:28.040 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:28.040 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":334,"btime":"2024-09-10T21:35:24:560912+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:24.560907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:28.040 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 334 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fc4072610 msgr2=0x7f4fc4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fc4072610 0x7f4fc4074ad0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c10 tx=0x7f4fd4001120 comp rx=0 tx=0).stop 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe4155f90 msgr2=0x7f4fe4156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe4155f90 0x7f4fe4156410 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc006fd0 tx=0x7f4fdc006640 comp rx=0 tx=0).stop 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 shutdown_connections 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4fc4072610 0x7f4fc4074ad0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fe4155f90 0x7f4fe4156410 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4fe41515b0 0x7f4fe4151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 --2- 172.21.15.97:0/2284608193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fc8007980 0x7f4fe4148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 >> 172.21.15.97:0/2284608193 conn(0x7f4fe40b1780 msgr2=0x7f4fe40b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 shutdown_connections 2024-09-10T21:35:28.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.042+0000 7f4fea000640 1 -- 172.21.15.97:0/2284608193 wait complete. 2024-09-10T21:35:28.061 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2024-09-10T21:35:28.061 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2024-09-10T21:35:28.062 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:35:28.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:35:28.154 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:35:28.250 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:35:28.250 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:35:28.250 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:28.251 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:28.259 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:35:28.259 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:35:28.259 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:35:28.259 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:28.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:35:28.269 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:35:28.269 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:28.269 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:28.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:35:28.310 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:35:28.311 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:35:28.311 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:28.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:35:28.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:28.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- 172.21.15.97:0/2027437106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 msgr2=0x7f1e60166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 --2- 172.21.15.97:0/2027437106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e60166850 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009c80 tx=0x7f1e5801cda0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- 172.21.15.97:0/2027437106 shutdown_connections 2024-09-10T21:35:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 --2- 172.21.15.97:0/2027437106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e60166850 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- 172.21.15.97:0/2027437106 >> 172.21.15.97:0/2027437106 conn(0x7f1e600b1780 msgr2=0x7f1e600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- 172.21.15.97:0/2027437106 shutdown_connections 2024-09-10T21:35:28.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- 172.21.15.97:0/2027437106 wait complete. 2024-09-10T21:35:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 Processor -- start 2024-09-10T21:35:28.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- start start 2024-09-10T21:35:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e600ce5f0 0x7f1e6013c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e60141d20 0x7f1e600b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e600b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6016a130 con 0x7f1e6015fde0 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e60169fb0 con 0x7f1e60141d20 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.498+0000 7f1e64a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6016a2b0 con 0x7f1e600ce5f0 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e600b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e600b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42656/0 (socket says 172.21.15.97:42656) 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 -- 172.21.15.97:0/4051070169 learned_addr learned my addr 172.21.15.97:0/4051070169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5d400640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e60141d20 0x7f1e600b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5de00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e600ce5f0 0x7f1e6013c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 -- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e600ce5f0 msgr2=0x7f1e6013c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e600ce5f0 0x7f1e6013c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 -- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e60141d20 msgr2=0x7f1e600b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e60141d20 0x7f1e600b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 -- 172.21.15.97:0/4051070169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e60140180 con 0x7f1e6015fde0 2024-09-10T21:35:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5d400640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e60141d20 0x7f1e600b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:28.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e5e800640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e600b47f0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009bd0 tx=0x7f1e580093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e56000640 1 -- 172.21.15.97:0/4051070169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e580bc070 con 0x7f1e6015fde0 2024-09-10T21:35:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e56000640 1 -- 172.21.15.97:0/4051070169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e580042f0 con 0x7f1e6015fde0 2024-09-10T21:35:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e56000640 1 -- 172.21.15.97:0/4051070169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e580b2040 con 0x7f1e6015fde0 2024-09-10T21:35:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e600b2d20 con 0x7f1e6015fde0 2024-09-10T21:35:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e600b32e0 con 0x7f1e6015fde0 2024-09-10T21:35:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e600ce9d0 con 0x7f1e6015fde0 2024-09-10T21:35:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.502+0000 7f1e56000640 1 -- 172.21.15.97:0/4051070169 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1e580b5070 con 0x7f1e6015fde0 2024-09-10T21:35:28.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.506+0000 7f1e56000640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e400725e0 0x7f1e40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.506+0000 7f1e56000640 1 -- 172.21.15.97:0/4051070169 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f1e5812dcb0 con 0x7f1e6015fde0 2024-09-10T21:35:28.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.506+0000 7f1e5de00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e400725e0 0x7f1e40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.506+0000 7f1e5de00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e400725e0 0x7f1e40074aa0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004e30 tx=0x7f1e50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.510+0000 7f1e56000640 1 -- 172.21.15.97:0/4051070169 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1e58016990 con 0x7f1e6015fde0 2024-09-10T21:35:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.710+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e60076120 con 0x7f1e6015fde0 2024-09-10T21:35:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.710+0000 7f1e56000640 1 -- 172.21.15.97:0/4051070169 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) ==== 95+0+4396 (secure 0 0 0) 0x7f1e580f8420 con 0x7f1e6015fde0 2024-09-10T21:35:28.709 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:28.710 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":334,"btime":"2024-09-10T21:35:24:560912+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:24.560907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 334 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e400725e0 msgr2=0x7f1e40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e400725e0 0x7f1e40074aa0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004e30 tx=0x7f1e50009040 comp rx=0 tx=0).stop 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 msgr2=0x7f1e600b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e600b47f0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009bd0 tx=0x7f1e580093d0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 shutdown_connections 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1e400725e0 0x7f1e40074aa0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e6015fde0 0x7f1e600b47f0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e60141d20 0x7f1e600b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 --2- 172.21.15.97:0/4051070169 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1e600ce5f0 0x7f1e6013c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 >> 172.21.15.97:0/4051070169 conn(0x7f1e600b1780 msgr2=0x7f1e6013f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 shutdown_connections 2024-09-10T21:35:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.714+0000 7f1e64a00640 1 -- 172.21.15.97:0/4051070169 wait complete. 2024-09-10T21:35:28.732 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:28.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.874+0000 7f1d69200640 1 -- 172.21.15.97:0/846532492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d48004810 msgr2=0x7f1d48004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.874+0000 7f1d69200640 1 --2- 172.21.15.97:0/846532492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d48004810 0x7f1d48004c90 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009b60 tx=0x7f1d4c01d210 comp rx=0 tx=0).stop 2024-09-10T21:35:28.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.874+0000 7f1d63e00640 1 -- 172.21.15.97:0/846532492 reap_dead start 2024-09-10T21:35:28.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.874+0000 7f1d69200640 1 -- 172.21.15.97:0/846532492 shutdown_connections 2024-09-10T21:35:28.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.874+0000 7f1d69200640 1 -- 172.21.15.97:0/846532492 >> 172.21.15.97:0/846532492 conn(0x7f1d640d1a60 msgr2=0x7f1d640d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:28.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.874+0000 7f1d69200640 1 -- 172.21.15.97:0/846532492 shutdown_connections 2024-09-10T21:35:28.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.874+0000 7f1d69200640 1 -- 172.21.15.97:0/846532492 wait complete. 2024-09-10T21:35:28.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 Processor -- start 2024-09-10T21:35:28.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 -- start start 2024-09-10T21:35:28.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d48004810 0x7f1d641f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d640cbb10 0x7f1d641f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d640ae8a0 0x7f1d640aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6415f510 con 0x7f1d640ae8a0 2024-09-10T21:35:28.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640afb50 con 0x7f1d640cbb10 2024-09-10T21:35:28.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640cfd80 con 0x7f1d48004810 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d640ae8a0 0x7f1d640aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d640ae8a0 0x7f1d640aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42670/0 (socket says 172.21.15.97:42670) 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 -- 172.21.15.97:0/4178736500 learned_addr learned my addr 172.21.15.97:0/4178736500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d5aa00640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d640cbb10 0x7f1d641f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63400640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d48004810 0x7f1d641f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 -- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d48004810 msgr2=0x7f1d641f87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d48004810 0x7f1d641f87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 -- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d640cbb10 msgr2=0x7f1d641f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d640cbb10 0x7f1d641f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 -- 172.21.15.97:0/4178736500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d640b77a0 con 0x7f1d640ae8a0 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d5aa00640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d640cbb10 0x7f1d641f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d63e00640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d640ae8a0 0x7f1d640aed20 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0115a0 tx=0x7f1d5c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:28.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d5be00640 1 -- 172.21.15.97:0/4178736500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c017070 con 0x7f1d640ae8a0 2024-09-10T21:35:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d5be00640 1 -- 172.21.15.97:0/4178736500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d5c013a10 con 0x7f1d640ae8a0 2024-09-10T21:35:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d640b52b0 con 0x7f1d640ae8a0 2024-09-10T21:35:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d5be00640 1 -- 172.21.15.97:0/4178736500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c013e60 con 0x7f1d640ae8a0 2024-09-10T21:35:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d640b5870 con 0x7f1d640ae8a0 2024-09-10T21:35:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.878+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640cf840 con 0x7f1d640ae8a0 2024-09-10T21:35:28.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.882+0000 7f1d5be00640 1 -- 172.21.15.97:0/4178736500 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d5c010040 con 0x7f1d640ae8a0 2024-09-10T21:35:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.882+0000 7f1d5be00640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d40072610 0x7f1d40074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.882+0000 7f1d5be00640 1 -- 172.21.15.97:0/4178736500 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f1d5c097de0 con 0x7f1d640ae8a0 2024-09-10T21:35:28.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.882+0000 7f1d63400640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d40072610 0x7f1d40074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:28.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.882+0000 7f1d63400640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d40072610 0x7f1d40074ad0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009bb0 tx=0x7f1d540091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:28.886+0000 7f1d5be00640 1 -- 172.21.15.97:0/4178736500 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d5c0624f0 con 0x7f1d640ae8a0 2024-09-10T21:35:29.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.082+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d640712a0 con 0x7f1d640ae8a0 2024-09-10T21:35:29.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d5be00640 1 -- 172.21.15.97:0/4178736500 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) ==== 95+0+4396 (secure 0 0 0) 0x7f1d5c061df0 con 0x7f1d640ae8a0 2024-09-10T21:35:29.083 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:29.084 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":334,"btime":"2024-09-10T21:35:24:560912+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:24.560907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:29.084 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 334 2024-09-10T21:35:29.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d40072610 msgr2=0x7f1d40074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d40072610 0x7f1d40074ad0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009bb0 tx=0x7f1d540091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d640ae8a0 msgr2=0x7f1d640aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d640ae8a0 0x7f1d640aed20 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0115a0 tx=0x7f1d5c011a60 comp rx=0 tx=0).stop 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 shutdown_connections 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1d40072610 0x7f1d40074ad0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d640ae8a0 0x7f1d640aed20 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d640cbb10 0x7f1d641f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 --2- 172.21.15.97:0/4178736500 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1d48004810 0x7f1d641f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 >> 172.21.15.97:0/4178736500 conn(0x7f1d640d1a60 msgr2=0x7f1d6406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 shutdown_connections 2024-09-10T21:35:29.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.086+0000 7f1d69200640 1 -- 172.21.15.97:0/4178736500 wait complete. 2024-09-10T21:35:29.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 -- 172.21.15.97:0/1736318334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0afdd0 msgr2=0x7f014c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 --2- 172.21.15.97:0/1736318334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0afdd0 0x7f014c0cc6d0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f014c0c55b0 tx=0x7f013401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 -- 172.21.15.97:0/1736318334 shutdown_connections 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 --2- 172.21.15.97:0/1736318334 >> v2:172.21.15.194:3300/0 conn(0x7f014c0b0d50 0x7f014c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 --2- 172.21.15.97:0/1736318334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0afdd0 0x7f014c0cc6d0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 -- 172.21.15.97:0/1736318334 >> 172.21.15.97:0/1736318334 conn(0x7f014c0d1c30 msgr2=0x7f014c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 -- 172.21.15.97:0/1736318334 shutdown_connections 2024-09-10T21:35:29.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.246+0000 7f0152e00640 1 -- 172.21.15.97:0/1736318334 wait complete. 2024-09-10T21:35:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 Processor -- start 2024-09-10T21:35:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 -- start start 2024-09-10T21:35:29.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f014c0afdd0 0x7f014c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f014c0b0d50 0x7f014c163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0ce640 0x7f014c164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f014c168ee0 con 0x7f014c0ce640 2024-09-10T21:35:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f014c168d60 con 0x7f014c0afdd0 2024-09-10T21:35:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f014c169060 con 0x7f014c0b0d50 2024-09-10T21:35:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0ce640 0x7f014c164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0ce640 0x7f014c164170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42682/0 (socket says 172.21.15.97:42682) 2024-09-10T21:35:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 -- 172.21.15.97:0/1945568782 learned_addr learned my addr 172.21.15.97:0/1945568782 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f014be00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f014c0afdd0 0x7f014c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0143400640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f014c0b0d50 0x7f014c163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 -- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f014c0b0d50 msgr2=0x7f014c163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f014c0b0d50 0x7f014c163c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 -- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f014c0afdd0 msgr2=0x7f014c0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f014c0afdd0 0x7f014c0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 -- 172.21.15.97:0/1945568782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c0c68e0 con 0x7f014c0ce640 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0150e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0ce640 0x7f014c164170 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f014400e960 tx=0x7f014400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0148c00640 1 -- 172.21.15.97:0/1945568782 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144004d30 con 0x7f014c0ce640 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0148c00640 1 -- 172.21.15.97:0/1945568782 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f014400cb20 con 0x7f014c0ce640 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0148c00640 1 -- 172.21.15.97:0/1945568782 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144005020 con 0x7f014c0ce640 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c0c6bc0 con 0x7f014c0ce640 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c1647f0 con 0x7f014c0ce640 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.250+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c0cea20 con 0x7f014c0ce640 2024-09-10T21:35:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.254+0000 7f0148c00640 1 -- 172.21.15.97:0/1945568782 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f014400f040 con 0x7f014c0ce640 2024-09-10T21:35:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.254+0000 7f0148c00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01280725e0 0x7f0128074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.254+0000 7f0148c00640 1 -- 172.21.15.97:0/1945568782 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f01440946b0 con 0x7f014c0ce640 2024-09-10T21:35:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.254+0000 7f014be00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01280725e0 0x7f0128074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.254+0000 7f014be00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01280725e0 0x7f0128074aa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f013c007fd0 tx=0x7f013c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.258+0000 7f0148c00640 1 -- 172.21.15.97:0/1945568782 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0144016030 con 0x7f014c0ce640 2024-09-10T21:35:29.305 INFO:tasks.ceph.mds.c.smithi097.stderr: -56> 2024-09-10T21:35:24.550+0000 7f8e53a00640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T21:35:29.306 INFO:tasks.ceph.mds.c.smithi097.stderr: -53> 2024-09-10T21:35:24.550+0000 7f8e53a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T21:35:29.306 INFO:tasks.ceph.mds.c.smithi097.stderr: -36> 2024-09-10T21:35:25.070+0000 7f8e58a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:29.344 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:35:29.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.430+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f014c072290 con 0x7f014c0ce640 2024-09-10T21:35:29.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0148c00640 1 -- 172.21.15.97:0/1945568782 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+22428 (secure 0 0 0) 0x7f014405e7d0 con 0x7f014c0ce640 2024-09-10T21:35:29.432 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:29.432 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":136,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:24.560618+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:30:34.216652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T21:30:35.493513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.7","mappings":[{"from":1,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.1a","mappings":[{"from":7,"to":3}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/3450288328":"2024-09-11T21:35:24.560596+0000","172.21.15.97:6833/3450288328":"2024-09-11T21:35:24.560596+0000","172.21.15.97:6834/884798951":"2024-09-11T21:35:17.513883+0000","172.21.15.97:6834/3735064631":"2024-09-11T21:35:11.470738+0000","172.21.15.97:6834/3930239299":"2024-09-11T21:35:06.270774+0000","172.21.15.97:6832/3930239299":"2024-09-11T21:35:06.270774+0000","172.21.15.97:6834/3083575804":"2024-09-11T21:34:46.104147+0000","172.21.15.97:6834/1998443497":"2024-09-11T21:34:35.158201+0000","172.21.15.97:6834/1087733461":"2024-09-11T21:34:19.533818+0000","172.21.15.97:6832/1087733461":"2024-09-11T21:34:19.533818+0000","172.21.15.97:6833/2872418710":"2024-09-11T21:34:14.052935+0000","172.21.15.97:6834/2750213748":"2024-09-11T21:34:00.427920+0000","172.21.15.97:6832/2750213748":"2024-09-11T21:34:00.427920+0000","172.21.15.97:6835/3334191869":"2024-09-11T21:33:56.361451+0000","172.21.15.97:6833/2243090003":"2024-09-11T21:32:28.378125+0000","172.21.15.97:6834/507601338":"2024-09-11T21:33:33.124771+0000","172.21.15.97:6832/2415223919":"2024-09-11T21:32:37.700495+0000","172.21.15.97:6834/923451308":"2024-09-11T21:32:09.002263+0000","172.21.15.97:6832/884798951":"2024-09-11T21:35:17.513883+0000","172.21.15.97:6834/3576291769":"2024-09-11T21:33:25.065757+0000","172.21.15.97:6835/170708142":"2024-09-11T21:32:30.659200+0000","172.21.15.97:6832/923451308":"2024-09-11T21:32:09.002263+0000","172.21.15.97:6835/733923445":"2024-09-11T21:33:07.903840+0000","172.21.15.97:6834/2402514654":"2024-09-11T21:33:11.941224+0000","172.21.15.97:6832/3289603806":"2024-09-11T21:34:51.343028+0000","172.21.15.97:6834/2364393791":"2024-09-11T21:32:02.967748+0000","172.21.15.97:6832/3648960428":"2024-09-11T21:32:43.749260+0000","172.21.15.97:6832/2383941694":"2024-09-11T21:31:49.749927+0000","172.21.15.97:6834/2013408839":"2024-09-11T21:31:38.828816+0000","172.21.15.97:6833/873879253":"2024-09-11T21:30:50.627295+0000","172.21.15.97:6834/1929373906":"2024-09-11T21:33:20.014467+0000","172.21.15.97:6833/3971758099":"2024-09-11T21:31:06.554833+0000","172.21.15.97:6835/2872418710":"2024-09-11T21:34:14.052935+0000","172.21.15.97:6835/2243090003":"2024-09-11T21:32:28.378125+0000","172.21.15.97:6832/3083575804":"2024-09-11T21:34:46.104147+0000","172.21.15.97:6832/769007942":"2024-09-11T21:31:21.620967+0000","172.21.15.97:6832/1998443497":"2024-09-11T21:34:35.158201+0000","172.21.15.97:6833/2192988988":"2024-09-11T21:31:11.752953+0000","172.21.15.97:6833/170708142":"2024-09-11T21:32:30.659200+0000","172.21.15.97:6834/2702314481":"2024-09-11T21:31:51.627563+0000","172.21.15.97:6832/2364393791":"2024-09-11T21:32:02.967748+0000","172.21.15.97:6834/2383941694":"2024-09-11T21:31:49.749927+0000","172.21.15.97:6834/2415223919":"2024-09-11T21:32:37.700495+0000","172.21.15.97:6835/873879253":"2024-09-11T21:30:50.627295+0000","172.21.15.97:6835/2192988988":"2024-09-11T21:31:11.752953+0000","172.21.15.97:6832/507601338":"2024-09-11T21:33:33.124771+0000","172.21.15.97:6833/895133479":"2024-09-11T21:31:25.859540+0000","172.21.15.97:6833/2521509514":"2024-09-11T21:31:17.802510+0000","172.21.15.97:6832/2702314481":"2024-09-11T21:31:51.627563+0000","172.21.15.97:6833/3111622295":"2024-09-11T21:33:05.844372+0000","172.21.15.97:6835/2521509514":"2024-09-11T21:31:17.802510+0000","172.21.15.97:6834/769007942":"2024-09-11T21:31:21.620967+0000","172.21.15.97:6835/2841033666":"2024-09-11T21:31:55.916991+0000","172.21.15.97:6835/899393328":"2024-09-11T21:33:29.083390+0000","172.21.15.97:6835/895133479":"2024-09-11T21:31:25.859540+0000","172.21.15.97:6832/2013408839":"2024-09-11T21:31:38.828816+0000","172.21.15.97:6835/2264649568":"2024-09-11T21:33:46.168737+0000","172.21.15.97:6834/3648960428":"2024-09-11T21:32:43.749260+0000","172.21.15.97:6833/1863388792":"2024-09-11T21:32:47.786444+0000","172.21.15.97:6835/1863388792":"2024-09-11T21:32:47.786444+0000","172.21.15.97:6832/2561173061":"2024-09-11T21:33:52.323112+0000","172.21.15.97:6833/733923445":"2024-09-11T21:33:07.903840+0000","172.21.15.97:6832/3735064631":"2024-09-11T21:35:11.470738+0000","172.21.15.97:6834/2194082021":"2024-09-11T21:32:51.642298+0000","172.21.15.97:6833/2841033666":"2024-09-11T21:31:55.916991+0000","172.21.15.97:6835/3111622295":"2024-09-11T21:33:05.844372+0000","172.21.15.97:6832/2402514654":"2024-09-11T21:33:11.941224+0000","172.21.15.97:6833/3203242917":"2024-09-11T21:33:15.973352+0000","172.21.15.97:6833/4017597023":"2024-09-11T21:33:48.304066+0000","172.21.15.97:6834/3289603806":"2024-09-11T21:34:51.343028+0000","172.21.15.97:6835/3203242917":"2024-09-11T21:33:15.973352+0000","172.21.15.97:6832/1929373906":"2024-09-11T21:33:20.014467+0000","172.21.15.97:6835/3971758099":"2024-09-11T21:31:06.554833+0000","172.21.15.97:6832/2194082021":"2024-09-11T21:32:51.642298+0000","172.21.15.97:6832/3576291769":"2024-09-11T21:33:25.065757+0000","172.21.15.97:6834/2561173061":"2024-09-11T21:33:52.323112+0000","172.21.15.97:6833/899393328":"2024-09-11T21:33:29.083390+0000","172.21.15.97:6833/2264649568":"2024-09-11T21:33:46.168737+0000","172.21.15.97:6835/4017597023":"2024-09-11T21:33:48.304066+0000","172.21.15.97:6833/3334191869":"2024-09-11T21:33:56.361451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:35:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01280725e0 msgr2=0x7f0128074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01280725e0 0x7f0128074aa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f013c007fd0 tx=0x7f013c009040 comp rx=0 tx=0).stop 2024-09-10T21:35:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0ce640 msgr2=0x7f014c164170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0ce640 0x7f014c164170 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f014400e960 tx=0x7f014400ee20 comp rx=0 tx=0).stop 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 shutdown_connections 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f01280725e0 0x7f0128074aa0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f014c0ce640 0x7f014c164170 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f014c0b0d50 0x7f014c163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 --2- 172.21.15.97:0/1945568782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f014c0afdd0 0x7f014c0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 >> 172.21.15.97:0/1945568782 conn(0x7f014c0d1c30 msgr2=0x7f014c0cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 shutdown_connections 2024-09-10T21:35:29.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.434+0000 7f0152e00640 1 -- 172.21.15.97:0/1945568782 wait complete. 2024-09-10T21:35:29.453 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:35:29.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- 172.21.15.97:0/2836205827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 msgr2=0x7fa1880cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 --2- 172.21.15.97:0/2836205827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880cf800 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fa178009840 tx=0x7fa17801f010 comp rx=0 tx=0).stop 2024-09-10T21:35:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- 172.21.15.97:0/2836205827 shutdown_connections 2024-09-10T21:35:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 --2- 172.21.15.97:0/2836205827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880cf800 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- 172.21.15.97:0/2836205827 >> 172.21.15.97:0/2836205827 conn(0x7fa1880b37c0 msgr2=0x7fa1880b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- 172.21.15.97:0/2836205827 shutdown_connections 2024-09-10T21:35:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- 172.21.15.97:0/2836205827 wait complete. 2024-09-10T21:35:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 Processor -- start 2024-09-10T21:35:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- start start 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1880b1560 0x7fa1880c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1880cec10 0x7fa18815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880ce320 con 0x7fa1880ce830 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880ce1a0 con 0x7fa1880cec10 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18f000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa1880d0450 con 0x7fa1880b1560 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa17f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa17f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42736/0 (socket says 172.21.15.97:42736) 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa17f400640 1 -- 172.21.15.97:0/307715422 learned_addr learned my addr 172.21.15.97:0/307715422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa18d000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1880cec10 0x7fa18815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.598+0000 7fa187e00640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1880b1560 0x7fa1880c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa17f400640 1 -- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1880b1560 msgr2=0x7fa1880c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa17f400640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1880b1560 0x7fa1880c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa17f400640 1 -- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1880cec10 msgr2=0x7fa18815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa17f400640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1880cec10 0x7fa18815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa17f400640 1 -- 172.21.15.97:0/307715422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1880c6170 con 0x7fa1880ce830 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa18d000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1880cec10 0x7fa18815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa17f400640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880c7dd0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fa17000c420 tx=0x7fa17000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa184c00640 1 -- 172.21.15.97:0/307715422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170017070 con 0x7fa1880ce830 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa184c00640 1 -- 172.21.15.97:0/307715422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1700053c0 con 0x7fa1880ce830 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1880c6410 con 0x7fa1880ce830 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa184c00640 1 -- 172.21.15.97:0/307715422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170003860 con 0x7fa1880ce830 2024-09-10T21:35:29.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1880c7820 con 0x7fa1880ce830 2024-09-10T21:35:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1880c6a90 con 0x7fa1880ce830 2024-09-10T21:35:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa184c00640 1 -- 172.21.15.97:0/307715422 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa170003a00 con 0x7fa1880ce830 2024-09-10T21:35:29.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.602+0000 7fa184c00640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa164072610 0x7fa164074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.606+0000 7fa184c00640 1 -- 172.21.15.97:0/307715422 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7fa1700938b0 con 0x7fa1880ce830 2024-09-10T21:35:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.606+0000 7fa187e00640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa164072610 0x7fa164074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.606+0000 7fa187e00640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa164072610 0x7fa164074ad0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b2000 tx=0x7fa17801d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:29.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.606+0000 7fa184c00640 1 -- 172.21.15.97:0/307715422 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa17005d850 con 0x7fa1880ce830 2024-09-10T21:35:29.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.806+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1880b1970 con 0x7fa1880ce830 2024-09-10T21:35:29.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.806+0000 7fa184c00640 1 -- 172.21.15.97:0/307715422 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+5220 (secure 0 0 0) 0x7fa170062280 con 0x7fa1880ce830 2024-09-10T21:35:29.807 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:29.807 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":335,"btime":"2024-09-10T21:35:29:567868+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331},{"gid":10518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/640676491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":640676491},{"type":"v1","addr":"172.21.15.97:6835","nonce":640676491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:24.560907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:29.807 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 335 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa164072610 msgr2=0x7fa164074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa164072610 0x7fa164074ad0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b2000 tx=0x7fa17801d850 comp rx=0 tx=0).stop 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 msgr2=0x7fa1880c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880c7dd0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fa17000c420 tx=0x7fa17000c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 shutdown_connections 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa164072610 0x7fa164074ad0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1880cec10 0x7fa18815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1880ce830 0x7fa1880c7dd0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 --2- 172.21.15.97:0/307715422 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa1880b1560 0x7fa1880c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 >> 172.21.15.97:0/307715422 conn(0x7fa1880b37c0 msgr2=0x7fa1880b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:29.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 shutdown_connections 2024-09-10T21:35:29.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.810+0000 7fa18f000640 1 -- 172.21.15.97:0/307715422 wait complete. 2024-09-10T21:35:29.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:29.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 -- 172.21.15.97:0/3228348157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 msgr2=0x7fc2bc168b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 --2- 172.21.15.97:0/3228348157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc168b00 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009900 tx=0x7fc2b401d280 comp rx=0 tx=0).stop 2024-09-10T21:35:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 -- 172.21.15.97:0/3228348157 shutdown_connections 2024-09-10T21:35:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 --2- 172.21.15.97:0/3228348157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc168b00 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 -- 172.21.15.97:0/3228348157 >> 172.21.15.97:0/3228348157 conn(0x7fc2bc0d1c30 msgr2=0x7fc2bc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 -- 172.21.15.97:0/3228348157 shutdown_connections 2024-09-10T21:35:29.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 -- 172.21.15.97:0/3228348157 wait complete. 2024-09-10T21:35:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.970+0000 7fc2c2200640 1 Processor -- start 2024-09-10T21:35:29.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 -- start start 2024-09-10T21:35:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2bc161600 0x7fc2bc144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2bc145010 0x7fc2bc1f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc16c1e0 con 0x7fc2bc0b1590 2024-09-10T21:35:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc16c060 con 0x7fc2bc161600 2024-09-10T21:35:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc16c360 con 0x7fc2bc145010 2024-09-10T21:35:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc145660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42766/0 (socket says 172.21.15.97:42766) 2024-09-10T21:35:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 -- 172.21.15.97:0/34506136 learned_addr learned my addr 172.21.15.97:0/34506136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2baa00640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2bc161600 0x7fc2bc144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bbe00640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2bc145010 0x7fc2bc1f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 -- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2bc145010 msgr2=0x7fc2bc1f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2bc145010 0x7fc2bc1f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 -- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2bc161600 msgr2=0x7fc2bc144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:29.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2bc161600 0x7fc2bc144ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 -- 172.21.15.97:0/34506136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc0b34c0 con 0x7fc2bc0b1590 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2bb400640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc145660 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c750 tx=0x7fc2ac00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2b3400640 1 -- 172.21.15.97:0/34506136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac017070 con 0x7fc2bc0b1590 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2b3400640 1 -- 172.21.15.97:0/34506136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2ac005070 con 0x7fc2bc0b1590 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2b3400640 1 -- 172.21.15.97:0/34506136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00ce60 con 0x7fc2bc0b1590 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc0c88a0 con 0x7fc2bc0b1590 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc15b820 con 0x7fc2bc0b1590 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.974+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2bc0cfe30 con 0x7fc2bc0b1590 2024-09-10T21:35:29.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.978+0000 7fc2b3400640 1 -- 172.21.15.97:0/34506136 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc2ac0054c0 con 0x7fc2bc0b1590 2024-09-10T21:35:29.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.978+0000 7fc2b3400640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc29c072610 0x7fc29c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.978+0000 7fc2b3400640 1 -- 172.21.15.97:0/34506136 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7fc2ac093a20 con 0x7fc2bc0b1590 2024-09-10T21:35:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.978+0000 7fc2baa00640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc29c072610 0x7fc29c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:29.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.978+0000 7fc2baa00640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc29c072610 0x7fc29c074ad0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0b6d80 tx=0x7fc2a8007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:29.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:29.982+0000 7fc2b3400640 1 -- 172.21.15.97:0/34506136 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc2ac0057e0 con 0x7fc2bc0b1590 2024-09-10T21:35:30.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.178+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2bc145420 con 0x7fc2bc0b1590 2024-09-10T21:35:30.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2b3400640 1 -- 172.21.15.97:0/34506136 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 335 v335) ==== 95+0+5220 (secure 0 0 0) 0x7fc2ac05db40 con 0x7fc2bc0b1590 2024-09-10T21:35:30.180 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:30.180 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":335,"btime":"2024-09-10T21:35:29:567868+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331},{"gid":10518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/640676491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":640676491},{"type":"v1","addr":"172.21.15.97:6835","nonce":640676491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335}],"filesystems":[{"mdsmap":{"epoch":334,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:30:36.527179+0000","modified":"2024-09-10T21:35:24.560907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:35:30.180 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 335 2024-09-10T21:35:30.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc29c072610 msgr2=0x7fc29c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc29c072610 0x7fc29c074ad0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0b6d80 tx=0x7fc2a8007e40 comp rx=0 tx=0).stop 2024-09-10T21:35:30.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 msgr2=0x7fc2bc145660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc145660 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c750 tx=0x7fc2ac00cc10 comp rx=0 tx=0).stop 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 shutdown_connections 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc29c072610 0x7fc29c074ad0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc2bc145010 0x7fc2bc1f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2bc161600 0x7fc2bc144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 --2- 172.21.15.97:0/34506136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2bc0b1590 0x7fc2bc145660 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 >> 172.21.15.97:0/34506136 conn(0x7fc2bc0d1c30 msgr2=0x7fc2bc0721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 shutdown_connections 2024-09-10T21:35:30.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.182+0000 7fc2c2200640 1 -- 172.21.15.97:0/34506136 wait complete. 2024-09-10T21:35:30.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- 172.21.15.97:0/2282644362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2006fd50 msgr2=0x7f5f200701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 --2- 172.21.15.97:0/2282644362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2006fd50 0x7f5f200701a0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f5f2015e650 tx=0x7f5f1801cb80 comp rx=0 tx=0).stop 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- 172.21.15.97:0/2282644362 shutdown_connections 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 --2- 172.21.15.97:0/2282644362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f2006fd50 0x7f5f200701a0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 --2- 172.21.15.97:0/2282644362 >> v2:172.21.15.194:3300/0 conn(0x7f5f20144690 0x7f5f200b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- 172.21.15.97:0/2282644362 >> 172.21.15.97:0/2282644362 conn(0x7f5f200b1780 msgr2=0x7f5f200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- 172.21.15.97:0/2282644362 shutdown_connections 2024-09-10T21:35:30.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- 172.21.15.97:0/2282644362 wait complete. 2024-09-10T21:35:30.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 Processor -- start 2024-09-10T21:35:30.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- start start 2024-09-10T21:35:30.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f2006fd50 0x7f5f201433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20143f90 0x7f5f2013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20144690 0x7f5f20135700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20167c40 con 0x7f5f20144690 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20167ac0 con 0x7f5f20143f90 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f25c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5f20167dc0 con 0x7f5f2006fd50 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20144690 0x7f5f20135700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20144690 0x7f5f20135700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42790/0 (socket says 172.21.15.97:42790) 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 -- 172.21.15.97:0/202884413 learned_addr learned my addr 172.21.15.97:0/202884413 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f17e00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20143f90 0x7f5f2013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1ea00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f2006fd50 0x7f5f201433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 -- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f2006fd50 msgr2=0x7f5f201433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f2006fd50 0x7f5f201433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 -- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20143f90 msgr2=0x7f5f2013e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20143f90 0x7f5f2013e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.346+0000 7f5f1f400640 1 -- 172.21.15.97:0/202884413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f20137a40 con 0x7f5f20144690 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f1f400640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20144690 0x7f5f20135700 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009b10 tx=0x7f5f180a5a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f16a00640 1 -- 172.21.15.97:0/202884413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180bb070 con 0x7f5f20144690 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f16a00640 1 -- 172.21.15.97:0/202884413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f18004c00 con 0x7f5f20144690 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f20137d20 con 0x7f5f20144690 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f16a00640 1 -- 172.21.15.97:0/202884413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180b2040 con 0x7f5f20144690 2024-09-10T21:35:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f201385c0 con 0x7f5f20144690 2024-09-10T21:35:30.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f200701a0 con 0x7f5f20144690 2024-09-10T21:35:30.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f16a00640 1 -- 172.21.15.97:0/202884413 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f180b5030 con 0x7f5f20144690 2024-09-10T21:35:30.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f16a00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc0725e0 0x7f5efc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f16a00640 1 -- 172.21.15.97:0/202884413 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f5f1812ddd0 con 0x7f5f20144690 2024-09-10T21:35:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.350+0000 7f5f1ea00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc0725e0 0x7f5efc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.354+0000 7f5f1ea00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc0725e0 0x7f5efc074aa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5f10001690 tx=0x7f5f10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:30.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.354+0000 7f5f16a00640 1 -- 172.21.15.97:0/202884413 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f18016770 con 0x7f5f20144690 2024-09-10T21:35:30.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.522+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f200b09c0 con 0x7f5f20144690 2024-09-10T21:35:30.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f16a00640 1 -- 172.21.15.97:0/202884413 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+22428 (secure 0 0 0) 0x7f5f180f7d80 con 0x7f5f20144690 2024-09-10T21:35:30.524 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:30.524 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":136,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:24.560618+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:30:34.216652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T21:30:35.493513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.7","mappings":[{"from":1,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.1a","mappings":[{"from":7,"to":3}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/3450288328":"2024-09-11T21:35:24.560596+0000","172.21.15.97:6833/3450288328":"2024-09-11T21:35:24.560596+0000","172.21.15.97:6834/884798951":"2024-09-11T21:35:17.513883+0000","172.21.15.97:6834/3735064631":"2024-09-11T21:35:11.470738+0000","172.21.15.97:6834/3930239299":"2024-09-11T21:35:06.270774+0000","172.21.15.97:6832/3930239299":"2024-09-11T21:35:06.270774+0000","172.21.15.97:6834/3083575804":"2024-09-11T21:34:46.104147+0000","172.21.15.97:6834/1998443497":"2024-09-11T21:34:35.158201+0000","172.21.15.97:6834/1087733461":"2024-09-11T21:34:19.533818+0000","172.21.15.97:6832/1087733461":"2024-09-11T21:34:19.533818+0000","172.21.15.97:6833/2872418710":"2024-09-11T21:34:14.052935+0000","172.21.15.97:6834/2750213748":"2024-09-11T21:34:00.427920+0000","172.21.15.97:6832/2750213748":"2024-09-11T21:34:00.427920+0000","172.21.15.97:6835/3334191869":"2024-09-11T21:33:56.361451+0000","172.21.15.97:6833/2243090003":"2024-09-11T21:32:28.378125+0000","172.21.15.97:6834/507601338":"2024-09-11T21:33:33.124771+0000","172.21.15.97:6832/2415223919":"2024-09-11T21:32:37.700495+0000","172.21.15.97:6834/923451308":"2024-09-11T21:32:09.002263+0000","172.21.15.97:6832/884798951":"2024-09-11T21:35:17.513883+0000","172.21.15.97:6834/3576291769":"2024-09-11T21:33:25.065757+0000","172.21.15.97:6835/170708142":"2024-09-11T21:32:30.659200+0000","172.21.15.97:6832/923451308":"2024-09-11T21:32:09.002263+0000","172.21.15.97:6835/733923445":"2024-09-11T21:33:07.903840+0000","172.21.15.97:6834/2402514654":"2024-09-11T21:33:11.941224+0000","172.21.15.97:6832/3289603806":"2024-09-11T21:34:51.343028+0000","172.21.15.97:6834/2364393791":"2024-09-11T21:32:02.967748+0000","172.21.15.97:6832/3648960428":"2024-09-11T21:32:43.749260+0000","172.21.15.97:6832/2383941694":"2024-09-11T21:31:49.749927+0000","172.21.15.97:6834/2013408839":"2024-09-11T21:31:38.828816+0000","172.21.15.97:6833/873879253":"2024-09-11T21:30:50.627295+0000","172.21.15.97:6834/1929373906":"2024-09-11T21:33:20.014467+0000","172.21.15.97:6833/3971758099":"2024-09-11T21:31:06.554833+0000","172.21.15.97:6835/2872418710":"2024-09-11T21:34:14.052935+0000","172.21.15.97:6835/2243090003":"2024-09-11T21:32:28.378125+0000","172.21.15.97:6832/3083575804":"2024-09-11T21:34:46.104147+0000","172.21.15.97:6832/769007942":"2024-09-11T21:31:21.620967+0000","172.21.15.97:6832/1998443497":"2024-09-11T21:34:35.158201+0000","172.21.15.97:6833/2192988988":"2024-09-11T21:31:11.752953+0000","172.21.15.97:6833/170708142":"2024-09-11T21:32:30.659200+0000","172.21.15.97:6834/2702314481":"2024-09-11T21:31:51.627563+0000","172.21.15.97:6832/2364393791":"2024-09-11T21:32:02.967748+0000","172.21.15.97:6834/2383941694":"2024-09-11T21:31:49.749927+0000","172.21.15.97:6834/2415223919":"2024-09-11T21:32:37.700495+0000","172.21.15.97:6835/873879253":"2024-09-11T21:30:50.627295+0000","172.21.15.97:6835/2192988988":"2024-09-11T21:31:11.752953+0000","172.21.15.97:6832/507601338":"2024-09-11T21:33:33.124771+0000","172.21.15.97:6833/895133479":"2024-09-11T21:31:25.859540+0000","172.21.15.97:6833/2521509514":"2024-09-11T21:31:17.802510+0000","172.21.15.97:6832/2702314481":"2024-09-11T21:31:51.627563+0000","172.21.15.97:6833/3111622295":"2024-09-11T21:33:05.844372+0000","172.21.15.97:6835/2521509514":"2024-09-11T21:31:17.802510+0000","172.21.15.97:6834/769007942":"2024-09-11T21:31:21.620967+0000","172.21.15.97:6835/2841033666":"2024-09-11T21:31:55.916991+0000","172.21.15.97:6835/899393328":"2024-09-11T21:33:29.083390+0000","172.21.15.97:6835/895133479":"2024-09-11T21:31:25.859540+0000","172.21.15.97:6832/2013408839":"2024-09-11T21:31:38.828816+0000","172.21.15.97:6835/2264649568":"2024-09-11T21:33:46.168737+0000","172.21.15.97:6834/3648960428":"2024-09-11T21:32:43.749260+0000","172.21.15.97:6833/1863388792":"2024-09-11T21:32:47.786444+0000","172.21.15.97:6835/1863388792":"2024-09-11T21:32:47.786444+0000","172.21.15.97:6832/2561173061":"2024-09-11T21:33:52.323112+0000","172.21.15.97:6833/733923445":"2024-09-11T21:33:07.903840+0000","172.21.15.97:6832/3735064631":"2024-09-11T21:35:11.470738+0000","172.21.15.97:6834/2194082021":"2024-09-11T21:32:51.642298+0000","172.21.15.97:6833/2841033666":"2024-09-11T21:31:55.916991+0000","172.21.15.97:6835/3111622295":"2024-09-11T21:33:05.844372+0000","172.21.15.97:6832/2402514654":"2024-09-11T21:33:11.941224+0000","172.21.15.97:6833/3203242917":"2024-09-11T21:33:15.973352+0000","172.21.15.97:6833/4017597023":"2024-09-11T21:33:48.304066+0000","172.21.15.97:6834/3289603806":"2024-09-11T21:34:51.343028+0000","172.21.15.97:6835/3203242917":"2024-09-11T21:33:15.973352+0000","172.21.15.97:6832/1929373906":"2024-09-11T21:33:20.014467+0000","172.21.15.97:6835/3971758099":"2024-09-11T21:31:06.554833+0000","172.21.15.97:6832/2194082021":"2024-09-11T21:32:51.642298+0000","172.21.15.97:6832/3576291769":"2024-09-11T21:33:25.065757+0000","172.21.15.97:6834/2561173061":"2024-09-11T21:33:52.323112+0000","172.21.15.97:6833/899393328":"2024-09-11T21:33:29.083390+0000","172.21.15.97:6833/2264649568":"2024-09-11T21:33:46.168737+0000","172.21.15.97:6835/4017597023":"2024-09-11T21:33:48.304066+0000","172.21.15.97:6833/3334191869":"2024-09-11T21:33:56.361451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:35:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc0725e0 msgr2=0x7f5efc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc0725e0 0x7f5efc074aa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5f10001690 tx=0x7f5f10009040 comp rx=0 tx=0).stop 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20144690 msgr2=0x7f5f20135700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20144690 0x7f5f20135700 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009b10 tx=0x7f5f180a5a50 comp rx=0 tx=0).stop 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 shutdown_connections 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5efc0725e0 0x7f5efc074aa0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20144690 0x7f5f20135700 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20143f90 0x7f5f2013e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 --2- 172.21.15.97:0/202884413 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f2006fd50 0x7f5f201433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 >> 172.21.15.97:0/202884413 conn(0x7f5f200b1780 msgr2=0x7f5f200b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 shutdown_connections 2024-09-10T21:35:30.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.526+0000 7f5f25c00640 1 -- 172.21.15.97:0/202884413 wait complete. 2024-09-10T21:35:30.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:35:30.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 -- 172.21.15.97:0/1688952846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e80b3cf0 msgr2=0x7fd7e806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 --2- 172.21.15.97:0/1688952846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e80b3cf0 0x7fd7e806ec40 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009900 tx=0x7fd7d401f090 comp rx=0 tx=0).stop 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 -- 172.21.15.97:0/1688952846 shutdown_connections 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 --2- 172.21.15.97:0/1688952846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e80b3cf0 0x7fd7e806ec40 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 --2- 172.21.15.97:0/1688952846 >> v2:172.21.15.194:3300/0 conn(0x7fd7e81511f0 0x7fd7e80b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 -- 172.21.15.97:0/1688952846 >> 172.21.15.97:0/1688952846 conn(0x7fd7e80b1780 msgr2=0x7fd7e80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 -- 172.21.15.97:0/1688952846 shutdown_connections 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 -- 172.21.15.97:0/1688952846 wait complete. 2024-09-10T21:35:30.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 Processor -- start 2024-09-10T21:35:30.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.690+0000 7fd7efa00640 1 -- start start 2024-09-10T21:35:30.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e806f180 0x7fd7e80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7e80b3cf0 0x7fd7e814fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7e81511f0 0x7fd7e81f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e806e740 con 0x7fd7e806f180 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e806e5c0 con 0x7fd7e80b3cf0 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd7e806e8c0 con 0x7fd7e81511f0 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e806f180 0x7fd7e80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e806f180 0x7fd7e80cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42804/0 (socket says 172.21.15.97:42804) 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 -- 172.21.15.97:0/2054855794 learned_addr learned my addr 172.21.15.97:0/2054855794 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7eda00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7e81511f0 0x7fd7e81f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7e7e00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7e80b3cf0 0x7fd7e814fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 -- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7e81511f0 msgr2=0x7fd7e81f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7e81511f0 0x7fd7e81f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 -- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7e80b3cf0 msgr2=0x7fd7e814fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:30.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7e80b3cf0 0x7fd7e814fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:30.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 -- 172.21.15.97:0/2054855794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e80af170 con 0x7fd7e806f180 2024-09-10T21:35:30.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7ed000640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e806f180 0x7fd7e80cd590 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00db90 tx=0x7fd7dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:30.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7e4c00640 1 -- 172.21.15.97:0/2054855794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc01b070 con 0x7fd7e806f180 2024-09-10T21:35:30.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7e4c00640 1 -- 172.21.15.97:0/2054855794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7dc00ae50 con 0x7fd7e806f180 2024-09-10T21:35:30.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e80ade50 con 0x7fd7e806f180 2024-09-10T21:35:30.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7e4c00640 1 -- 172.21.15.97:0/2054855794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00e9c0 con 0x7fd7e806f180 2024-09-10T21:35:30.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e80ae410 con 0x7fd7e806f180 2024-09-10T21:35:30.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.694+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e80b37b0 con 0x7fd7e806f180 2024-09-10T21:35:30.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.698+0000 7fd7e4c00640 1 -- 172.21.15.97:0/2054855794 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd7dc00b040 con 0x7fd7e806f180 2024-09-10T21:35:30.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.698+0000 7fd7e4c00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7c80725e0 0x7fd7c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:30.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.698+0000 7fd7e4c00640 1 -- 172.21.15.97:0/2054855794 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7fd7dc094710 con 0x7fd7e806f180 2024-09-10T21:35:30.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.698+0000 7fd7e7e00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7c80725e0 0x7fd7c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:30.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.698+0000 7fd7e7e00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7c80725e0 0x7fd7c8074aa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40b2000 tx=0x7fd7d401d8d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:30.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.702+0000 7fd7e4c00640 1 -- 172.21.15.97:0/2054855794 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd7e80af170 con 0x7fd7e806f180 2024-09-10T21:35:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:30.902+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd7e80cc960 con 0x7fd7e806f180 2024-09-10T21:35:31.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.586+0000 7fd7e4c00640 1 -- 172.21.15.97:0/2054855794 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v336) ==== 161+0+0 (secure 0 0 0) 0x7fd7dc05e740 con 0x7fd7e806f180 2024-09-10T21:35:31.585 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7c80725e0 msgr2=0x7fd7c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7c80725e0 0x7fd7c8074aa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40b2000 tx=0x7fd7d401d8d0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e806f180 msgr2=0x7fd7e80cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e806f180 0x7fd7e80cd590 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00db90 tx=0x7fd7dc00a620 comp rx=0 tx=0).stop 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 shutdown_connections 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7c80725e0 0x7fd7c8074aa0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7e81511f0 0x7fd7e81f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7e80b3cf0 0x7fd7e814fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 --2- 172.21.15.97:0/2054855794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7e806f180 0x7fd7e80cd590 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 >> 172.21.15.97:0/2054855794 conn(0x7fd7e80b1780 msgr2=0x7fd7e80af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 shutdown_connections 2024-09-10T21:35:31.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.590+0000 7fd7efa00640 1 -- 172.21.15.97:0/2054855794 wait complete. 2024-09-10T21:35:31.611 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 -- 172.21.15.97:0/196481140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68144690 msgr2=0x7f4d680b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 --2- 172.21.15.97:0/196481140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68144690 0x7f4d680b4120 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f4d68074ee0 tx=0x7f4d5001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 -- 172.21.15.97:0/196481140 shutdown_connections 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 --2- 172.21.15.97:0/196481140 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4d680b4660 0x7f4d6806fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 --2- 172.21.15.97:0/196481140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68144690 0x7f4d680b4120 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 --2- 172.21.15.97:0/196481140 >> v2:172.21.15.194:3300/0 conn(0x7f4d68143f90 0x7f4d680b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 -- 172.21.15.97:0/196481140 >> 172.21.15.97:0/196481140 conn(0x7f4d680b1780 msgr2=0x7f4d680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 -- 172.21.15.97:0/196481140 shutdown_connections 2024-09-10T21:35:31.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.758+0000 7f4d6d600640 1 -- 172.21.15.97:0/196481140 wait complete. 2024-09-10T21:35:31.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 Processor -- start 2024-09-10T21:35:31.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 -- start start 2024-09-10T21:35:31.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d680b4660 0x7f4d68159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:31.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68143f90 0x7f4d6806f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:31.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4d68144690 0x7f4d6806f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6815ed60 con 0x7f4d68143f90 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6815ebe0 con 0x7f4d680b4660 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6815eee0 con 0x7f4d68144690 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68143f90 0x7f4d6806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68143f90 0x7f4d6806f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42822/0 (socket says 172.21.15.97:42822) 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 -- 172.21.15.97:0/2828065939 learned_addr learned my addr 172.21.15.97:0/2828065939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d66a00640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d680b4660 0x7f4d68159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d67400640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4d68144690 0x7f4d6806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 -- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4d68144690 msgr2=0x7f4d6806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4d68144690 0x7f4d6806f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 -- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d680b4660 msgr2=0x7f4d68159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d680b4660 0x7f4d68159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 -- 172.21.15.97:0/2828065939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d681f8080 con 0x7f4d68143f90 2024-09-10T21:35:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d67400640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4d68144690 0x7f4d6806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5e000640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68143f90 0x7f4d6806f000 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4d500b0040 tx=0x7f4d500a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5f400640 1 -- 172.21.15.97:0/2828065939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d500b2040 con 0x7f4d68143f90 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5f400640 1 -- 172.21.15.97:0/2828065939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d50004340 con 0x7f4d68143f90 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d681f8360 con 0x7f4d68143f90 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d5f400640 1 -- 172.21.15.97:0/2828065939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50003860 con 0x7f4d68143f90 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681f8920 con 0x7f4d68143f90 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.762+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6815f4b0 con 0x7f4d68143f90 2024-09-10T21:35:31.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.766+0000 7f4d5f400640 1 -- 172.21.15.97:0/2828065939 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4d50003a00 con 0x7f4d68143f90 2024-09-10T21:35:31.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.766+0000 7f4d5f400640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4d440725e0 0x7f4d44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:31.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.766+0000 7f4d5f400640 1 -- 172.21.15.97:0/2828065939 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7f4d5012e350 con 0x7f4d68143f90 2024-09-10T21:35:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.766+0000 7f4d66a00640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4d440725e0 0x7f4d44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:31.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.766+0000 7f4d66a00640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4d440725e0 0x7f4d44074aa0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:31.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.770+0000 7f4d5f400640 1 -- 172.21.15.97:0/2828065939 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4d500164b0 con 0x7f4d68143f90 2024-09-10T21:35:31.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:31.970+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4d6815ad70 con 0x7f4d68143f90 2024-09-10T21:35:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d5f400640 1 -- 172.21.15.97:0/2828065939 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v337) ==== 104+0+0 (secure 0 0 0) 0x7f4d500f8ac0 con 0x7f4d68143f90 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4d440725e0 msgr2=0x7f4d44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4d440725e0 0x7f4d44074aa0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009110 comp rx=0 tx=0).stop 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68143f90 msgr2=0x7f4d6806f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68143f90 0x7f4d6806f000 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4d500b0040 tx=0x7f4d500a5650 comp rx=0 tx=0).stop 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 shutdown_connections 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4d440725e0 0x7f4d44074aa0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4d68144690 0x7f4d6806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d68143f90 0x7f4d6806f000 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 --2- 172.21.15.97:0/2828065939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d680b4660 0x7f4d68159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 >> 172.21.15.97:0/2828065939 conn(0x7f4d680b1780 msgr2=0x7f4d68163bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:32.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 shutdown_connections 2024-09-10T21:35:32.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.594+0000 7f4d6d600640 1 -- 172.21.15.97:0/2828065939 wait complete. 2024-09-10T21:35:32.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:35:32.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- 172.21.15.97:0/4049323650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 msgr2=0x7fd370004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:32.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 --2- 172.21.15.97:0/4049323650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 0x7fd370004070 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009b60 tx=0x7fd37c01d270 comp rx=0 tx=0).stop 2024-09-10T21:35:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd38be00640 1 -- 172.21.15.97:0/4049323650 reap_dead start 2024-09-10T21:35:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- 172.21.15.97:0/4049323650 shutdown_connections 2024-09-10T21:35:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- 172.21.15.97:0/4049323650 >> 172.21.15.97:0/4049323650 conn(0x7fd38c0b1780 msgr2=0x7fd38c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- 172.21.15.97:0/4049323650 shutdown_connections 2024-09-10T21:35:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- 172.21.15.97:0/4049323650 wait complete. 2024-09-10T21:35:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 Processor -- start 2024-09-10T21:35:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- start start 2024-09-10T21:35:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 0x7fd38c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd38c13b040 0x7fd38c139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd38c15e260 0x7fd38c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c167c40 con 0x7fd370003c70 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c167ac0 con 0x7fd38c13b040 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.766+0000 7fd392600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c167dc0 con 0x7fd38c15e260 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 0x7fd38c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 0x7fd38c1590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42836/0 (socket says 172.21.15.97:42836) 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 -- 172.21.15.97:0/2545574251 learned_addr learned my addr 172.21.15.97:0/2545574251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38aa00640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd38c13b040 0x7fd38c139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38be00640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd38c15e260 0x7fd38c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 -- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd38c15e260 msgr2=0x7fd38c15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd38c15e260 0x7fd38c15e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 -- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd38c13b040 msgr2=0x7fd38c139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd38c13b040 0x7fd38c139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 -- 172.21.15.97:0/2545574251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c157460 con 0x7fd370003c70 2024-09-10T21:35:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38be00640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd38c15e260 0x7fd38c15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:32.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd38b400640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 0x7fd38c1590d0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fd37c01d0a0 tx=0x7fd37c005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd383400640 1 -- 172.21.15.97:0/2545574251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0b1040 con 0x7fd370003c70 2024-09-10T21:35:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd383400640 1 -- 172.21.15.97:0/2545574251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd37c0bb070 con 0x7fd370003c70 2024-09-10T21:35:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd383400640 1 -- 172.21.15.97:0/2545574251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c004290 con 0x7fd370003c70 2024-09-10T21:35:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c157740 con 0x7fd370003c70 2024-09-10T21:35:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c13b4c0 con 0x7fd370003c70 2024-09-10T21:35:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c157c60 con 0x7fd370003c70 2024-09-10T21:35:32.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.770+0000 7fd383400640 1 -- 172.21.15.97:0/2545574251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd37c0046d0 con 0x7fd370003c70 2024-09-10T21:35:32.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.774+0000 7fd383400640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd36c0725e0 0x7fd36c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:32.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.774+0000 7fd383400640 1 -- 172.21.15.97:0/2545574251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9715+0+0 (secure 0 0 0) 0x7fd37c12f210 con 0x7fd370003c70 2024-09-10T21:35:32.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.774+0000 7fd38aa00640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd36c0725e0 0x7fd36c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:32.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.774+0000 7fd38aa00640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd36c0725e0 0x7fd36c074aa0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd3780017f0 tx=0x7fd378006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:32.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.778+0000 7fd383400640 1 -- 172.21.15.97:0/2545574251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd37c016f80 con 0x7fd370003c70 2024-09-10T21:35:32.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:32.942+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd38c15f6d0 con 0x7fd370003c70 2024-09-10T21:35:33.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.594+0000 7fd383400640 1 -- 172.21.15.97:0/2545574251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) ==== 181+0+0 (secure 0 0 0) 0x7fd37c0f9240 con 0x7fd370003c70 2024-09-10T21:35:33.593 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd36c0725e0 msgr2=0x7fd36c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd36c0725e0 0x7fd36c074aa0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd3780017f0 tx=0x7fd378006040 comp rx=0 tx=0).stop 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 msgr2=0x7fd38c1590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 0x7fd38c1590d0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fd37c01d0a0 tx=0x7fd37c005590 comp rx=0 tx=0).stop 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 shutdown_connections 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd36c0725e0 0x7fd36c074aa0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd38c15e260 0x7fd38c15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd38c13b040 0x7fd38c139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 --2- 172.21.15.97:0/2545574251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd370003c70 0x7fd38c1590d0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 >> 172.21.15.97:0/2545574251 conn(0x7fd38c0b1780 msgr2=0x7fd38c15c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:33.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 shutdown_connections 2024-09-10T21:35:33.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.598+0000 7fd392600640 1 -- 172.21.15.97:0/2545574251 wait complete. 2024-09-10T21:35:33.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 -- 172.21.15.97:0/2799004394 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00b1560 msgr2=0x7fbad00b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 --2- 172.21.15.97:0/2799004394 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00b1560 0x7fbad00b1970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009bc0 tx=0x7fbac801f3b0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 -- 172.21.15.97:0/2799004394 shutdown_connections 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 --2- 172.21.15.97:0/2799004394 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00b1560 0x7fbad00b1970 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 --2- 172.21.15.97:0/2799004394 >> v2:172.21.15.194:3300/0 conn(0x7fbad00cec10 0x7fbad00cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 -- 172.21.15.97:0/2799004394 >> 172.21.15.97:0/2799004394 conn(0x7fbad00b37c0 msgr2=0x7fbad00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 -- 172.21.15.97:0/2799004394 shutdown_connections 2024-09-10T21:35:33.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.766+0000 7fbad4e00640 1 -- 172.21.15.97:0/2799004394 wait complete. 2024-09-10T21:35:33.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 Processor -- start 2024-09-10T21:35:33.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 -- start start 2024-09-10T21:35:33.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbad00b1560 0x7fbad01f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad00ce830 0x7fbad0156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00cec10 0x7fbad00b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbad00ce1a0 con 0x7fbad00ce830 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbad00cde60 con 0x7fbad00b1560 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbad00ce320 con 0x7fbad00cec10 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad00ce830 0x7fbad0156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad00ce830 0x7fbad0156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42852/0 (socket says 172.21.15.97:42852) 2024-09-10T21:35:33.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 -- 172.21.15.97:0/3049538015 learned_addr learned my addr 172.21.15.97:0/3049538015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:33.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbace800640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00cec10 0x7fbad00b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:33.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacde00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbad00b1560 0x7fbad01f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:33.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 -- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00cec10 msgr2=0x7fbad00b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:33.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00cec10 0x7fbad00b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 -- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbad00b1560 msgr2=0x7fbad01f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbad00b1560 0x7fbad01f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 -- 172.21.15.97:0/3049538015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad01f5350 con 0x7fbad00ce830 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbace800640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00cec10 0x7fbad00b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacde00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbad00b1560 0x7fbad01f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbacd400640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad00ce830 0x7fbad0156190 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e970 tx=0x7fbabc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbac6000640 1 -- 172.21.15.97:0/3049538015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00cd80 con 0x7fbad00ce830 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad00b7570 con 0x7fbad00ce830 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbac6000640 1 -- 172.21.15.97:0/3049538015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbabc004020 con 0x7fbad00ce830 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbac6000640 1 -- 172.21.15.97:0/3049538015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc004470 con 0x7fbad00ce830 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.770+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad00c5a40 con 0x7fbad00ce830 2024-09-10T21:35:33.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.774+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad00cf800 con 0x7fbad00ce830 2024-09-10T21:35:33.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.774+0000 7fbac6000640 1 -- 172.21.15.97:0/3049538015 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbabc003600 con 0x7fbad00ce830 2024-09-10T21:35:33.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.774+0000 7fbac6000640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab00725e0 0x7fbab0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:33.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.774+0000 7fbac6000640 1 -- 172.21.15.97:0/3049538015 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9248+0+0 (secure 0 0 0) 0x7fbabc094a90 con 0x7fbad00ce830 2024-09-10T21:35:33.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.774+0000 7fbacde00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab00725e0 0x7fbab0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:33.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.778+0000 7fbacde00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab00725e0 0x7fbab0074aa0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbac0001250 tx=0x7fbac0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:33.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.778+0000 7fbac6000640 1 -- 172.21.15.97:0/3049538015 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbad01f5350 con 0x7fbad00ce830 2024-09-10T21:35:33.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:33.946+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbad0071f30 con 0x7fbad00ce830 2024-09-10T21:35:34.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.610+0000 7fbac6000640 1 -- 172.21.15.97:0/3049538015 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) ==== 169+0+0 (secure 0 0 0) 0x7fbabc05f290 con 0x7fbad00ce830 2024-09-10T21:35:34.608 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-09-10T21:35:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab00725e0 msgr2=0x7fbab0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab00725e0 0x7fbab0074aa0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbac0001250 tx=0x7fbac0009040 comp rx=0 tx=0).stop 2024-09-10T21:35:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad00ce830 msgr2=0x7fbad0156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad00ce830 0x7fbad0156190 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e970 tx=0x7fbabc00ee30 comp rx=0 tx=0).stop 2024-09-10T21:35:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 shutdown_connections 2024-09-10T21:35:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbab00725e0 0x7fbab0074aa0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbad00cec10 0x7fbad00b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad00ce830 0x7fbad0156190 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 --2- 172.21.15.97:0/3049538015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbad00b1560 0x7fbad01f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 >> 172.21.15.97:0/3049538015 conn(0x7fbad00b37c0 msgr2=0x7fbad00b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 shutdown_connections 2024-09-10T21:35:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.614+0000 7fbad4e00640 1 -- 172.21.15.97:0/3049538015 wait complete. 2024-09-10T21:35:34.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:35:34.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 -- 172.21.15.97:0/455329185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe59015fc10 msgr2=0x7fe59013f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:34.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 --2- 172.21.15.97:0/455329185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe59015fc10 0x7fe59013f9b0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fe590074800 tx=0x7fe58801cb60 comp rx=0 tx=0).stop 2024-09-10T21:35:34.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 -- 172.21.15.97:0/455329185 shutdown_connections 2024-09-10T21:35:34.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 --2- 172.21.15.97:0/455329185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe59015fc10 0x7fe59013f9b0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 --2- 172.21.15.97:0/455329185 >> v2:172.21.15.194:3300/0 conn(0x7fe590141d20 0x7fe59015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 -- 172.21.15.97:0/455329185 >> 172.21.15.97:0/455329185 conn(0x7fe5900b1780 msgr2=0x7fe5900b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:34.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 -- 172.21.15.97:0/455329185 shutdown_connections 2024-09-10T21:35:34.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 -- 172.21.15.97:0/455329185 wait complete. 2024-09-10T21:35:34.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 Processor -- start 2024-09-10T21:35:34.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 -- start start 2024-09-10T21:35:34.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5900ce5f0 0x7fe5901fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:34.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.782+0000 7fe596800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe590141d20 0x7fe59013c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:34.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe596800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59015fc10 0x7fe59013c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:34.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe596800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe590168a70 con 0x7fe5900ce5f0 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe596800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe590140cd0 con 0x7fe59015fc10 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe596800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe590168bf0 con 0x7fe590141d20 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5900ce5f0 0x7fe5901fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5900ce5f0 0x7fe5901fbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42870/0 (socket says 172.21.15.97:42870) 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 -- 172.21.15.97:0/2105980669 learned_addr learned my addr 172.21.15.97:0/2105980669 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58ea00640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe590141d20 0x7fe59013c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58fe00640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59015fc10 0x7fe59013c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 -- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe590141d20 msgr2=0x7fe59013c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe590141d20 0x7fe59013c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 -- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59015fc10 msgr2=0x7fe59013c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59015fc10 0x7fe59013c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 -- 172.21.15.97:0/2105980669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59006f2f0 con 0x7fe5900ce5f0 2024-09-10T21:35:34.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe58f400640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5900ce5f0 0x7fe5901fbb30 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c500 tx=0x7fe58000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:34.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe587400640 1 -- 172.21.15.97:0/2105980669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580017070 con 0x7fe5900ce5f0 2024-09-10T21:35:34.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe587400640 1 -- 172.21.15.97:0/2105980669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe580003600 con 0x7fe5900ce5f0 2024-09-10T21:35:34.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe587400640 1 -- 172.21.15.97:0/2105980669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580003a50 con 0x7fe5900ce5f0 2024-09-10T21:35:34.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901fc070 con 0x7fe5900ce5f0 2024-09-10T21:35:34.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901fb330 con 0x7fe5900ce5f0 2024-09-10T21:35:34.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5900b4b70 con 0x7fe5900ce5f0 2024-09-10T21:35:34.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.786+0000 7fe587400640 1 -- 172.21.15.97:0/2105980669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe580003bf0 con 0x7fe5900ce5f0 2024-09-10T21:35:34.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.790+0000 7fe587400640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5700725e0 0x7fe570074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:34.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.790+0000 7fe587400640 1 -- 172.21.15.97:0/2105980669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7fe5800935d0 con 0x7fe5900ce5f0 2024-09-10T21:35:34.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.790+0000 7fe58ea00640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5700725e0 0x7fe570074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:34.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.790+0000 7fe58ea00640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5700725e0 0x7fe570074aa0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005fd0 tx=0x7fe57c005db0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:34.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.794+0000 7fe587400640 1 -- 172.21.15.97:0/2105980669 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5901fc070 con 0x7fe5900ce5f0 2024-09-10T21:35:34.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.958+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fe590166260 con 0x7fe5900ce5f0 2024-09-10T21:35:34.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.962+0000 7fe587400640 1 -- 172.21.15.97:0/2105980669 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7fe59006f2f0 con 0x7fe5900ce5f0 2024-09-10T21:35:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.962+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5700725e0 msgr2=0x7fe570074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.962+0000 7fe596800640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5700725e0 0x7fe570074aa0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005fd0 tx=0x7fe57c005db0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.962+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5900ce5f0 msgr2=0x7fe5901fbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:34.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.962+0000 7fe596800640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5900ce5f0 0x7fe5901fbb30 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c500 tx=0x7fe58000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 shutdown_connections 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5700725e0 0x7fe570074aa0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59015fc10 0x7fe59013c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe590141d20 0x7fe59013c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 --2- 172.21.15.97:0/2105980669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5900ce5f0 0x7fe5901fbb30 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 >> 172.21.15.97:0/2105980669 conn(0x7fe5900b1780 msgr2=0x7fe590139e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 shutdown_connections 2024-09-10T21:35:34.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:34.966+0000 7fe596800640 1 -- 172.21.15.97:0/2105980669 wait complete. 2024-09-10T21:35:34.984 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-10T21:35:35.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.126+0000 7f0af8800640 1 -- 172.21.15.97:0/1979655316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ad8004810 msgr2=0x7f0ad8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.126+0000 7f0af8800640 1 --2- 172.21.15.97:0/1979655316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ad8004810 0x7f0ad8004c90 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009b60 tx=0x7f0ae001d270 comp rx=0 tx=0).stop 2024-09-10T21:35:35.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.126+0000 7f0af6800640 1 -- 172.21.15.97:0/1979655316 reap_dead start 2024-09-10T21:35:35.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.126+0000 7f0af8800640 1 -- 172.21.15.97:0/1979655316 shutdown_connections 2024-09-10T21:35:35.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.126+0000 7f0af8800640 1 -- 172.21.15.97:0/1979655316 >> 172.21.15.97:0/1979655316 conn(0x7f0af00d1c30 msgr2=0x7f0af00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:35.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.126+0000 7f0af8800640 1 -- 172.21.15.97:0/1979655316 shutdown_connections 2024-09-10T21:35:35.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.126+0000 7f0af8800640 1 -- 172.21.15.97:0/1979655316 wait complete. 2024-09-10T21:35:35.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 Processor -- start 2024-09-10T21:35:35.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 -- start start 2024-09-10T21:35:35.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ad8004810 0x7f0af0155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0af01562a0 0x7f0af0164200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af00b68e0 0x7f0af00b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0af0168c80 con 0x7f0af00b68e0 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0af0168b00 con 0x7f0af01562a0 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0af0168e00 con 0x7f0ad8004810 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af00b68e0 0x7f0af00b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af00b68e0 0x7f0af00b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42900/0 (socket says 172.21.15.97:42900) 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 -- 172.21.15.97:0/2797775645 learned_addr learned my addr 172.21.15.97:0/2797775645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af5400640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0af01562a0 0x7f0af0164200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af5e00640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ad8004810 0x7f0af0155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 -- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ad8004810 msgr2=0x7f0af0155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ad8004810 0x7f0af0155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 -- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0af01562a0 msgr2=0x7f0af0164200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0af01562a0 0x7f0af0164200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 -- 172.21.15.97:0/2797775645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af00b5db0 con 0x7f0af00b68e0 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af6800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af00b68e0 0x7f0af00b6d60 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f0af009fc30 tx=0x7f0aec010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0aea000640 1 -- 172.21.15.97:0/2797775645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec00ed50 con 0x7f0af00b68e0 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0aea000640 1 -- 172.21.15.97:0/2797775645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0aec015070 con 0x7f0af00b68e0 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af00b5510 con 0x7f0af00b68e0 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0aea000640 1 -- 172.21.15.97:0/2797775645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec003bc0 con 0x7f0af00b68e0 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.130+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af00b5ad0 con 0x7f0af00b68e0 2024-09-10T21:35:35.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.134+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af006dc50 con 0x7f0af00b68e0 2024-09-10T21:35:35.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.134+0000 7f0aea000640 1 -- 172.21.15.97:0/2797775645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0aec024060 con 0x7f0af00b68e0 2024-09-10T21:35:35.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.134+0000 7f0aea000640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ad4072610 0x7f0ad4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.134+0000 7f0aea000640 1 -- 172.21.15.97:0/2797775645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f0aec0940e0 con 0x7f0af00b68e0 2024-09-10T21:35:35.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.134+0000 7f0af5e00640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ad4072610 0x7f0ad4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.134+0000 7f0af5e00640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ad4072610 0x7f0ad4074ad0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0af015a220 tx=0x7f0ae4001ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:35.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.138+0000 7f0aea000640 1 -- 172.21.15.97:0/2797775645 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0aec05e510 con 0x7f0af00b68e0 2024-09-10T21:35:35.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.362+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7f0af00d20e0 con 0x7f0af00b68e0 2024-09-10T21:35:35.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.610+0000 7f0aea000640 1 -- 172.21.15.97:0/2797775645 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v588) ==== 135+0+0 (secure 0 0 0) 0x7f0aec062f40 con 0x7f0af00b68e0 2024-09-10T21:35:35.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ad4072610 msgr2=0x7f0ad4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ad4072610 0x7f0ad4074ad0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0af015a220 tx=0x7f0ae4001ce0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af00b68e0 msgr2=0x7f0af00b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af00b68e0 0x7f0af00b6d60 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f0af009fc30 tx=0x7f0aec010e90 comp rx=0 tx=0).stop 2024-09-10T21:35:35.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 shutdown_connections 2024-09-10T21:35:35.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0ad4072610 0x7f0ad4074ad0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af00b68e0 0x7f0af00b6d60 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0af01562a0 0x7f0af0164200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 --2- 172.21.15.97:0/2797775645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0ad8004810 0x7f0af0155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 >> 172.21.15.97:0/2797775645 conn(0x7f0af00d1c30 msgr2=0x7f0af01675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:35.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 shutdown_connections 2024-09-10T21:35:35.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.614+0000 7f0af8800640 1 -- 172.21.15.97:0/2797775645 wait complete. 2024-09-10T21:35:35.632 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2024-09-10T21:35:35.632 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2024-09-10T21:35:35.633 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-10T21:35:35.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:35:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 -- 172.21.15.97:0/2215130698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800cec10 msgr2=0x7fd7800cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 --2- 172.21.15.97:0/2215130698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800cec10 0x7fd7800cff10 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fd7800b4860 tx=0x7fd76c01f070 comp rx=0 tx=0).stop 2024-09-10T21:35:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 -- 172.21.15.97:0/2215130698 shutdown_connections 2024-09-10T21:35:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 --2- 172.21.15.97:0/2215130698 >> v2:172.21.15.194:3300/0 conn(0x7fd7800b1560 0x7fd7800b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 --2- 172.21.15.97:0/2215130698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800cec10 0x7fd7800cff10 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 -- 172.21.15.97:0/2215130698 >> 172.21.15.97:0/2215130698 conn(0x7fd7800b37c0 msgr2=0x7fd7800b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 -- 172.21.15.97:0/2215130698 shutdown_connections 2024-09-10T21:35:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.774+0000 7fd785c00640 1 -- 172.21.15.97:0/2215130698 wait complete. 2024-09-10T21:35:35.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 Processor -- start 2024-09-10T21:35:35.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 -- start start 2024-09-10T21:35:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800b1560 0x7fd7801f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7800ce830 0x7fd78015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7800cec10 0x7fd78015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7800ce1a0 con 0x7fd7800b1560 2024-09-10T21:35:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd7800cde60 con 0x7fd7800cec10 2024-09-10T21:35:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd7800ce320 con 0x7fd7800ce830 2024-09-10T21:35:35.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800b1560 0x7fd7801f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800b1560 0x7fd7801f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42916/0 (socket says 172.21.15.97:42916) 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 -- 172.21.15.97:0/403564436 learned_addr learned my addr 172.21.15.97:0/403564436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77f400640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7800cec10 0x7fd78015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77e000640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7800ce830 0x7fd78015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 -- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7800ce830 msgr2=0x7fd78015bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7800ce830 0x7fd78015bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 -- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7800cec10 msgr2=0x7fd78015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7800cec10 0x7fd78015dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 -- 172.21.15.97:0/403564436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78006f1d0 con 0x7fd7800b1560 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd77ea00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800b1560 0x7fd7801f4fd0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd77000c450 tx=0x7fd77000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd776a00640 1 -- 172.21.15.97:0/403564436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770012830 con 0x7fd7800b1560 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd776a00640 1 -- 172.21.15.97:0/403564436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd770017070 con 0x7fd7800b1560 2024-09-10T21:35:35.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78015c550 con 0x7fd7800b1560 2024-09-10T21:35:35.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd776a00640 1 -- 172.21.15.97:0/403564436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770012d80 con 0x7fd7800b1560 2024-09-10T21:35:35.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.778+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78006e410 con 0x7fd7800b1560 2024-09-10T21:35:35.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.782+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7800cf800 con 0x7fd7800b1560 2024-09-10T21:35:35.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.782+0000 7fd776a00640 1 -- 172.21.15.97:0/403564436 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd77001c460 con 0x7fd7800b1560 2024-09-10T21:35:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.782+0000 7fd776a00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7600725e0 0x7fd760074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.782+0000 7fd776a00640 1 -- 172.21.15.97:0/403564436 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7fd770093b10 con 0x7fd7800b1560 2024-09-10T21:35:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.782+0000 7fd77e000640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7600725e0 0x7fd760074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.782+0000 7fd77e000640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7600725e0 0x7fd760074aa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0b2000 tx=0x7fd76c01d8b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.786+0000 7fd776a00640 1 -- 172.21.15.97:0/403564436 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd77005df40 con 0x7fd7800b1560 2024-09-10T21:35:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.950+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7800b27b0 con 0x7fd7800b1560 2024-09-10T21:35:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd776a00640 1 -- 172.21.15.97:0/403564436 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+18383 (secure 0 0 0) 0x7fd770062970 con 0x7fd7800b1560 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":138,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:34.604710+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/3450288328":"2024-09-11T21:35:24.560596+0000","172.21.15.97:6833/3450288328":"2024-09-11T21:35:24.560596+0000","172.21.15.97:6834/884798951":"2024-09-11T21:35:17.513883+0000","172.21.15.97:6834/3735064631":"2024-09-11T21:35:11.470738+0000","172.21.15.97:6834/3930239299":"2024-09-11T21:35:06.270774+0000","172.21.15.97:6832/3930239299":"2024-09-11T21:35:06.270774+0000","172.21.15.97:6834/3083575804":"2024-09-11T21:34:46.104147+0000","172.21.15.97:6834/1998443497":"2024-09-11T21:34:35.158201+0000","172.21.15.97:6834/1087733461":"2024-09-11T21:34:19.533818+0000","172.21.15.97:6832/1087733461":"2024-09-11T21:34:19.533818+0000","172.21.15.97:6833/2872418710":"2024-09-11T21:34:14.052935+0000","172.21.15.97:6834/2750213748":"2024-09-11T21:34:00.427920+0000","172.21.15.97:6832/2750213748":"2024-09-11T21:34:00.427920+0000","172.21.15.97:6835/3334191869":"2024-09-11T21:33:56.361451+0000","172.21.15.97:6833/2243090003":"2024-09-11T21:32:28.378125+0000","172.21.15.97:6834/507601338":"2024-09-11T21:33:33.124771+0000","172.21.15.97:6832/2415223919":"2024-09-11T21:32:37.700495+0000","172.21.15.97:6834/923451308":"2024-09-11T21:32:09.002263+0000","172.21.15.97:6832/884798951":"2024-09-11T21:35:17.513883+0000","172.21.15.97:6834/3576291769":"2024-09-11T21:33:25.065757+0000","172.21.15.97:6835/170708142":"2024-09-11T21:32:30.659200+0000","172.21.15.97:6832/923451308":"2024-09-11T21:32:09.002263+0000","172.21.15.97:6835/733923445":"2024-09-11T21:33:07.903840+0000","172.21.15.97:6834/2402514654":"2024-09-11T21:33:11.941224+0000","172.21.15.97:6832/3289603806":"2024-09-11T21:34:51.343028+0000","172.21.15.97:6834/2364393791":"2024-09-11T21:32:02.967748+0000","172.21.15.97:6832/3648960428":"2024-09-11T21:32:43.749260+0000","172.21.15.97:6832/2383941694":"2024-09-11T21:31:49.749927+0000","172.21.15.97:6834/2013408839":"2024-09-11T21:31:38.828816+0000","172.21.15.97:6833/873879253":"2024-09-11T21:30:50.627295+0000","172.21.15.97:6834/1929373906":"2024-09-11T21:33:20.014467+0000","172.21.15.97:6833/3971758099":"2024-09-11T21:31:06.554833+0000","172.21.15.97:6835/2872418710":"2024-09-11T21:34:14.052935+0000","172.21.15.97:6835/2243090003":"2024-09-11T21:32:28.378125+0000","172.21.15.97:6832/3083575804":"2024-09-11T21:34:46.104147+0000","172.21.15.97:6832/769007942":"2024-09-11T21:31:21.620967+0000","172.21.15.97:6832/1998443497":"2024-09-11T21:34:35.158201+0000","172.21.15.97:6833/2192988988":"2024-09-11T21:31:11.752953+0000","172.21.15.97:6833/170708142":"2024-09-11T21:32:30.659200+0000","172.21.15.97:6834/2702314481":"2024-09-11T21:31:51.627563+0000","172.21.15.97:6832/2364393791":"2024-09-11T21:32:02.967748+0000","172.21.15.97:6834/2383941694":"2024-09-11T21:31:49.749927+0000","172.21.15.97:6834/2415223919":"2024-09-11T21:32:37.700495+0000","172.21.15.97:6835/873879253":"2024-09-11T21:30:50.627295+0000","172.21.15.97:6835/2192988988":"2024-09-11T21:31:11.752953+0000","172.21.15.97:6832/507601338":"2024-09-11T21:33:33.124771+0000","172.21.15.97:6833/895133479":"2024-09-11T21:31:25.859540+0000","172.21.15.97:6833/2521509514":"2024-09-11T21:31:17.802510+0000","172.21.15.97:6832/2702314481":"2024-09-11T21:31:51.627563+0000","172.21.15.97:6833/3111622295":"2024-09-11T21:33:05.844372+0000","172.21.15.97:6835/2521509514":"2024-09-11T21:31:17.802510+0000","172.21.15.97:6834/769007942":"2024-09-11T21:31:21.620967+0000","172.21.15.97:6835/2841033666":"2024-09-11T21:31:55.916991+0000","172.21.15.97:6835/899393328":"2024-09-11T21:33:29.083390+0000","172.21.15.97:6835/895133479":"2024-09-11T21:31:25.859540+0000","172.21.15.97:6832/2013408839":"2024-09-11T21:31:38.828816+0000","172.21.15.97:6835/2264649568":"2024-09-11T21:33:46.168737+0000","172.21.15.97:6834/3648960428":"2024-09-11T21:32:43.749260+0000","172.21.15.97:6833/1863388792":"2024-09-11T21:32:47.786444+0000","172.21.15.97:6835/1863388792":"2024-09-11T21:32:47.786444+0000","172.21.15.97:6832/2561173061":"2024-09-11T21:33:52.323112+0000","172.21.15.97:6833/733923445":"2024-09-11T21:33:07.903840+0000","172.21.15.97:6832/3735064631":"2024-09-11T21:35:11.470738+0000","172.21.15.97:6834/2194082021":"2024-09-11T21:32:51.642298+0000","172.21.15.97:6833/2841033666":"2024-09-11T21:31:55.916991+0000","172.21.15.97:6835/3111622295":"2024-09-11T21:33:05.844372+0000","172.21.15.97:6832/2402514654":"2024-09-11T21:33:11.941224+0000","172.21.15.97:6833/3203242917":"2024-09-11T21:33:15.973352+0000","172.21.15.97:6833/4017597023":"2024-09-11T21:33:48.304066+0000","172.21.15.97:6834/3289603806":"2024-09-11T21:34:51.343028+0000","172.21.15.97:6835/3203242917":"2024-09-11T21:33:15.973352+0000","172.21.15.97:6832/1929373906":"2024-09-11T21:33:20.014467+0000","172.21.15.97:6835/3971758099":"2024-09-11T21:31:06.554833+0000","172.21.15.97:6832/2194082021":"2024-09-11T21:32:51.642298+0000","172.21.15.97:6832/3576291769":"2024-09-11T21:33:25.065757+0000","172.21.15.97:6834/2561173061":"2024-09-11T21:33:52.323112+0000","172.21.15.97:6833/899393328":"2024-09-11T21:33:29.083390+0000","172.21.15.97:6833/2264649568":"2024-09-11T21:33:46.168737+0000","172.21.15.97:6835/4017597023":"2024-09-11T21:33:48.304066+0000","172.21.15.97:6833/3334191869":"2024-09-11T21:33:56.361451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:35:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7600725e0 msgr2=0x7fd760074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7600725e0 0x7fd760074aa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0b2000 tx=0x7fd76c01d8b0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800b1560 msgr2=0x7fd7801f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800b1560 0x7fd7801f4fd0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd77000c450 tx=0x7fd77000c910 comp rx=0 tx=0).stop 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 shutdown_connections 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd7600725e0 0x7fd760074aa0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7800cec10 0x7fd78015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd7800ce830 0x7fd78015bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 --2- 172.21.15.97:0/403564436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7800b1560 0x7fd7801f4fd0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 >> 172.21.15.97:0/403564436 conn(0x7fd7800b37c0 msgr2=0x7fd7800cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 shutdown_connections 2024-09-10T21:35:35.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:35.954+0000 7fd785c00640 1 -- 172.21.15.97:0/403564436 wait complete. 2024-09-10T21:35:35.974 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:24:02.092708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:35:35.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:35:36.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 -- 172.21.15.97:0/2598708796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27040ce9d0 msgr2=0x7f270415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 --2- 172.21.15.97:0/2598708796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27040ce9d0 0x7f270415f6d0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f2704074ae0 tx=0x7f26ec01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 -- 172.21.15.97:0/2598708796 shutdown_connections 2024-09-10T21:35:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 --2- 172.21.15.97:0/2598708796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27040ce9d0 0x7f270415f6d0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 --2- 172.21.15.97:0/2598708796 >> v2:172.21.15.194:3300/0 conn(0x7f27040ce5f0 0x7f2704141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 -- 172.21.15.97:0/2598708796 >> 172.21.15.97:0/2598708796 conn(0x7f27040b1780 msgr2=0x7f27040b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 -- 172.21.15.97:0/2598708796 shutdown_connections 2024-09-10T21:35:36.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.118+0000 7f2709000640 1 -- 172.21.15.97:0/2598708796 wait complete. 2024-09-10T21:35:36.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 Processor -- start 2024-09-10T21:35:36.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 -- start start 2024-09-10T21:35:36.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27040ce5f0 0x7f270413b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27040ce9d0 0x7f2704143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f270415fc10 0x7f27040736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27041690c0 con 0x7f270415fc10 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2704168f40 con 0x7f27040ce9d0 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2704169240 con 0x7f27040ce5f0 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f270415fc10 0x7f27040736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f270415fc10 0x7f27040736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42930/0 (socket says 172.21.15.97:42930) 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 -- 172.21.15.97:0/3776493967 learned_addr learned my addr 172.21.15.97:0/3776493967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f26f9400640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27040ce9d0 0x7f2704143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2701e00640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27040ce5f0 0x7f270413b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 -- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27040ce5f0 msgr2=0x7f270413b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27040ce5f0 0x7f270413b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 -- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27040ce9d0 msgr2=0x7f2704143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27040ce9d0 0x7f2704143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 -- 172.21.15.97:0/3776493967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270406ad10 con 0x7f270415fc10 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2702800640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f270415fc10 0x7f27040736f0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c6b0 tx=0x7f26fc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f26faa00640 1 -- 172.21.15.97:0/3776493967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc003b80 con 0x7f270415fc10 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f26faa00640 1 -- 172.21.15.97:0/3776493967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26fc003d40 con 0x7f270415fc10 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f26faa00640 1 -- 172.21.15.97:0/3776493967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc005330 con 0x7f270415fc10 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27041440d0 con 0x7f270415fc10 2024-09-10T21:35:36.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.122+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2704144690 con 0x7f270415fc10 2024-09-10T21:35:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.126+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2704141d20 con 0x7f270415fc10 2024-09-10T21:35:36.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.126+0000 7f26faa00640 1 -- 172.21.15.97:0/3776493967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f26fc0054d0 con 0x7f270415fc10 2024-09-10T21:35:36.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.126+0000 7f26faa00640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26e0072610 0x7f26e0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.126+0000 7f26faa00640 1 -- 172.21.15.97:0/3776493967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f26fc093990 con 0x7f270415fc10 2024-09-10T21:35:36.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.126+0000 7f2701e00640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26e0072610 0x7f26e0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.126+0000 7f2701e00640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26e0072610 0x7f26e0074ad0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f27040ccc20 tx=0x7f26f400b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:36.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.130+0000 7f26faa00640 1 -- 172.21.15.97:0/3776493967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f26fc05de50 con 0x7f270415fc10 2024-09-10T21:35:36.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f270413ba20 con 0x7f270415fc10 2024-09-10T21:35:36.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f26faa00640 1 -- 172.21.15.97:0/3776493967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) ==== 93+0+10 (secure 0 0 0) 0x7f26fc062880 con 0x7f270415fc10 2024-09-10T21:35:36.296 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-09-10T21:35:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26e0072610 msgr2=0x7f26e0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26e0072610 0x7f26e0074ad0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f27040ccc20 tx=0x7f26f400b390 comp rx=0 tx=0).stop 2024-09-10T21:35:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f270415fc10 msgr2=0x7f27040736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f270415fc10 0x7f27040736f0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c6b0 tx=0x7f26fc00cb70 comp rx=0 tx=0).stop 2024-09-10T21:35:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 shutdown_connections 2024-09-10T21:35:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26e0072610 0x7f26e0074ad0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f270415fc10 0x7f27040736f0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27040ce9d0 0x7f2704143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 --2- 172.21.15.97:0/3776493967 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27040ce5f0 0x7f270413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 >> 172.21.15.97:0/3776493967 conn(0x7f27040b1780 msgr2=0x7f270415f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 shutdown_connections 2024-09-10T21:35:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.298+0000 7f2709000640 1 -- 172.21.15.97:0/3776493967 wait complete. 2024-09-10T21:35:36.317 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- 172.21.15.97:0/2492993203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f822015dee0 msgr2=0x7f822006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 --2- 172.21.15.97:0/2492993203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f822015dee0 0x7f822006fe00 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f8208009930 tx=0x7f820801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- 172.21.15.97:0/2492993203 shutdown_connections 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 --2- 172.21.15.97:0/2492993203 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8220070340 0x7f82200b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 --2- 172.21.15.97:0/2492993203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f822015dee0 0x7f822006fe00 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- 172.21.15.97:0/2492993203 >> 172.21.15.97:0/2492993203 conn(0x7f82200b1780 msgr2=0x7f82200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- 172.21.15.97:0/2492993203 shutdown_connections 2024-09-10T21:35:36.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- 172.21.15.97:0/2492993203 wait complete. 2024-09-10T21:35:36.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 Processor -- start 2024-09-10T21:35:36.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- start start 2024-09-10T21:35:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8220070340 0x7f8220147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f822015db00 0x7f822015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f822015dee0 0x7f822015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f822016ae30 con 0x7f8220070340 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f822016acb0 con 0x7f822015db00 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8225e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f822016afb0 con 0x7f822015dee0 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8220070340 0x7f8220147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8220070340 0x7f8220147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42940/0 (socket says 172.21.15.97:42940) 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 -- 172.21.15.97:0/104717843 learned_addr learned my addr 172.21.15.97:0/104717843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f8216000640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f822015db00 0x7f822015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821f400640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f822015dee0 0x7f822015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 -- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f822015dee0 msgr2=0x7f822015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f822015dee0 0x7f822015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 -- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f822015db00 msgr2=0x7f822015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f822015db00 0x7f822015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:36.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821ea00640 1 -- 172.21.15.97:0/104717843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8220147b50 con 0x7f8220070340 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.466+0000 7f821f400640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f822015dee0 0x7f822015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f821ea00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8220070340 0x7f8220147610 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f821000c7c0 tx=0x7f821000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8217e00640 1 -- 172.21.15.97:0/104717843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210016020 con 0x7f8220070340 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8217e00640 1 -- 172.21.15.97:0/104717843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f821000ab40 con 0x7f8220070340 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8217e00640 1 -- 172.21.15.97:0/104717843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210012420 con 0x7f8220070340 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822015edb0 con 0x7f8220070340 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201558f0 con 0x7f8220070340 2024-09-10T21:35:36.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82200b3150 con 0x7f8220070340 2024-09-10T21:35:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8217e00640 1 -- 172.21.15.97:0/104717843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f821001b400 con 0x7f8220070340 2024-09-10T21:35:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8217e00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81fc0725e0 0x7f81fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8217e00640 1 -- 172.21.15.97:0/104717843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f8210094140 con 0x7f8220070340 2024-09-10T21:35:36.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.470+0000 7f8216000640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81fc0725e0 0x7f81fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:36.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.474+0000 7f8216000640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81fc0725e0 0x7f81fc074aa0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f82080af000 tx=0x7f820801b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:36.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.474+0000 7f8217e00640 1 -- 172.21.15.97:0/104717843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8210020030 con 0x7f8220070340 2024-09-10T21:35:36.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:36.698+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f8220161cb0 con 0x7f8220070340 2024-09-10T21:35:37.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.618+0000 7f8217e00640 1 -- 172.21.15.97:0/104717843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v590) ==== 138+0+0 (secure 0 0 0) 0x7f821005ecb0 con 0x7f8220070340 2024-09-10T21:35:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81fc0725e0 msgr2=0x7f81fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81fc0725e0 0x7f81fc074aa0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f82080af000 tx=0x7f820801b120 comp rx=0 tx=0).stop 2024-09-10T21:35:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8220070340 msgr2=0x7f8220147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8220070340 0x7f8220147610 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f821000c7c0 tx=0x7f821000cc80 comp rx=0 tx=0).stop 2024-09-10T21:35:37.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 shutdown_connections 2024-09-10T21:35:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f81fc0725e0 0x7f81fc074aa0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f822015dee0 0x7f822015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f822015db00 0x7f822015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 --2- 172.21.15.97:0/104717843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8220070340 0x7f8220147610 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 >> 172.21.15.97:0/104717843 conn(0x7f82200b1780 msgr2=0x7f82200af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 shutdown_connections 2024-09-10T21:35:37.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.622+0000 7f8225e00640 1 -- 172.21.15.97:0/104717843 wait complete. 2024-09-10T21:35:37.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:35:37.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- 172.21.15.97:0/1897213669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 msgr2=0x7f597400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:37.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 --2- 172.21.15.97:0/1897213669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f597400b9d0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f598800c630 tx=0x7f598801d2c0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- 172.21.15.97:0/1897213669 shutdown_connections 2024-09-10T21:35:37.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 --2- 172.21.15.97:0/1897213669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f597400b9d0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- 172.21.15.97:0/1897213669 >> 172.21.15.97:0/1897213669 conn(0x7f59900d1c30 msgr2=0x7f59900d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:37.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- 172.21.15.97:0/1897213669 shutdown_connections 2024-09-10T21:35:37.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- 172.21.15.97:0/1897213669 wait complete. 2024-09-10T21:35:37.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 Processor -- start 2024-09-10T21:35:37.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- start start 2024-09-10T21:35:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f59900b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59901f51a0 0x7f59900c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59900c4680 0x7f59900c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f59900cea20 con 0x7f59740052f0 2024-09-10T21:35:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f599015c1d0 con 0x7f59900c4680 2024-09-10T21:35:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5997c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f59900ceba0 con 0x7f59901f51a0 2024-09-10T21:35:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f59900b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f59900b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52306/0 (socket says 172.21.15.97:52306) 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 -- 172.21.15.97:0/2274381921 learned_addr learned my addr 172.21.15.97:0/2274381921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995c00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59900c4680 0x7f59900c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5987e00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59901f51a0 0x7f59900c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 -- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59901f51a0 msgr2=0x7f59900c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59901f51a0 0x7f59900c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 -- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59900c4680 msgr2=0x7f59900c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59900c4680 0x7f59900c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 -- 172.21.15.97:0/2274381921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59900b7920 con 0x7f59740052f0 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f5995200640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f59900b72c0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f5980003ee0 tx=0x7f5980001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:37.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.794+0000 7f598d600640 1 -- 172.21.15.97:0/2274381921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980015070 con 0x7f59740052f0 2024-09-10T21:35:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f598d600640 1 -- 172.21.15.97:0/2274381921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5980011840 con 0x7f59740052f0 2024-09-10T21:35:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f598d600640 1 -- 172.21.15.97:0/2274381921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980011c70 con 0x7f59740052f0 2024-09-10T21:35:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59900c70a0 con 0x7f59740052f0 2024-09-10T21:35:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59900c7660 con 0x7f59740052f0 2024-09-10T21:35:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59900aedc0 con 0x7f59740052f0 2024-09-10T21:35:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f598d600640 1 -- 172.21.15.97:0/2274381921 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f59800103c0 con 0x7f59740052f0 2024-09-10T21:35:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f598d600640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f596c0725e0 0x7f596c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f598d600640 1 -- 172.21.15.97:0/2274381921 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f5980094e10 con 0x7f59740052f0 2024-09-10T21:35:37.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.798+0000 7f5987e00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f596c0725e0 0x7f596c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:37.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.802+0000 7f5987e00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f596c0725e0 0x7f596c074aa0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5978005fd0 tx=0x7f597800b260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.802+0000 7f598d600640 1 -- 172.21.15.97:0/2274381921 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5980098030 con 0x7f59740052f0 2024-09-10T21:35:37.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.969+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f599015e610 con 0x7f59740052f0 2024-09-10T21:35:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.969+0000 7f598d600640 1 -- 172.21.15.97:0/2274381921 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+1998 (secure 0 0 0) 0x7f598005f890 con 0x7f59740052f0 2024-09-10T21:35:37.969 INFO:teuthology.orchestra.run.smithi097.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:35:37.969 INFO:teuthology.orchestra.run.smithi097.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:35:37.969 INFO:teuthology.orchestra.run.smithi097.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:35:37.969 INFO:teuthology.orchestra.run.smithi097.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:35:37.969 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3032.483965 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3656.533946 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3351.408335 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3363.485497 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 8164.284498 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4391.956543 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4391.204852 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4799.171341 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds 20/20 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced debug_ms 2/2 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_frag true 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:35:37.970 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:mds dev mds_verify_scatter true 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced client_mount_timeout 600 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_client 20/20 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced debug_ms 1/1 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:35:37.971 INFO:teuthology.orchestra.run.smithi097.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:35:37.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.969+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f596c0725e0 msgr2=0x7f596c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:37.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.969+0000 7f5997c00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f596c0725e0 0x7f596c074aa0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5978005fd0 tx=0x7f597800b260 comp rx=0 tx=0).stop 2024-09-10T21:35:37.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.969+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 msgr2=0x7f59900b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:37.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f59900b72c0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f5980003ee0 tx=0x7f5980001640 comp rx=0 tx=0).stop 2024-09-10T21:35:37.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 shutdown_connections 2024-09-10T21:35:37.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f596c0725e0 0x7f596c074aa0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59900c4680 0x7f59900c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f59901f51a0 0x7f59900c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 --2- 172.21.15.97:0/2274381921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59740052f0 0x7f59900b72c0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:37.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 >> 172.21.15.97:0/2274381921 conn(0x7f59900d1c30 msgr2=0x7f599015ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:37.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 shutdown_connections 2024-09-10T21:35:37.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:37.973+0000 7f5997c00640 1 -- 172.21.15.97:0/2274381921 wait complete. 2024-09-10T21:35:37.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:35:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 -- 172.21.15.97:0/3500109514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb415fc10 msgr2=0x7fccb413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 --2- 172.21.15.97:0/3500109514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb415fc10 0x7fccb413f9b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fccac009bc0 tx=0x7fccac01cba0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 -- 172.21.15.97:0/3500109514 shutdown_connections 2024-09-10T21:35:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 --2- 172.21.15.97:0/3500109514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb415fc10 0x7fccb413f9b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 --2- 172.21.15.97:0/3500109514 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fccb4141d20 0x7fccb415f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 -- 172.21.15.97:0/3500109514 >> 172.21.15.97:0/3500109514 conn(0x7fccb40b1780 msgr2=0x7fccb40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:38.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 -- 172.21.15.97:0/3500109514 shutdown_connections 2024-09-10T21:35:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 -- 172.21.15.97:0/3500109514 wait complete. 2024-09-10T21:35:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 Processor -- start 2024-09-10T21:35:38.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.137+0000 7fccba800640 1 -- start start 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb40ce5f0 0x7fccb413b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccb4141d20 0x7fccb4074130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fccb415fc10 0x7fccb40b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4168a70 con 0x7fccb4141d20 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4140c80 con 0x7fccb40ce5f0 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fccb4168bf0 con 0x7fccb415fc10 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccb4141d20 0x7fccb4074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccb4141d20 0x7fccb4074130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52314/0 (socket says 172.21.15.97:52314) 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 -- 172.21.15.97:0/2145397559 learned_addr learned my addr 172.21.15.97:0/2145397559 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb3e00640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fccb415fc10 0x7fccb40b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb3400640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb40ce5f0 0x7fccb413b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 -- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fccb415fc10 msgr2=0x7fccb40b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fccb415fc10 0x7fccb40b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 -- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb40ce5f0 msgr2=0x7fccb413b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb40ce5f0 0x7fccb413b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 -- 172.21.15.97:0/2145397559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb40b4780 con 0x7fccb4141d20 2024-09-10T21:35:38.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb3400640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb40ce5f0 0x7fccb413b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccb2a00640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccb4141d20 0x7fccb4074130 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fcca000ea20 tx=0x7fcca000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccab400640 1 -- 172.21.15.97:0/2145397559 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca000cd80 con 0x7fccb4141d20 2024-09-10T21:35:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccab400640 1 -- 172.21.15.97:0/2145397559 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcca0010430 con 0x7fccb4141d20 2024-09-10T21:35:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb40b4a60 con 0x7fccb4141d20 2024-09-10T21:35:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccab400640 1 -- 172.21.15.97:0/2145397559 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0010860 con 0x7fccb4141d20 2024-09-10T21:35:38.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb41f7c80 con 0x7fccb4141d20 2024-09-10T21:35:38.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb415f6d0 con 0x7fccb4141d20 2024-09-10T21:35:38.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.141+0000 7fccab400640 1 -- 172.21.15.97:0/2145397559 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcca0010a00 con 0x7fccb4141d20 2024-09-10T21:35:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.145+0000 7fccab400640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc940725e0 0x7fcc94074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.145+0000 7fccab400640 1 -- 172.21.15.97:0/2145397559 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7fcca0094a40 con 0x7fccb4141d20 2024-09-10T21:35:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.145+0000 7fccb3400640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc940725e0 0x7fcc94074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.145+0000 7fccb3400640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc940725e0 0x7fcc94074aa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcca4007a60 tx=0x7fcca4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:38.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.149+0000 7fccab400640 1 -- 172.21.15.97:0/2145397559 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcca005edf0 con 0x7fccb4141d20 2024-09-10T21:35:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.317+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fccb40cca70 con 0x7fccb4141d20 2024-09-10T21:35:38.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccab400640 1 -- 172.21.15.97:0/2145397559 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fcca0063820 con 0x7fccb4141d20 2024-09-10T21:35:38.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc940725e0 msgr2=0x7fcc94074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc940725e0 0x7fcc94074aa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcca4007a60 tx=0x7fcca4009040 comp rx=0 tx=0).stop 2024-09-10T21:35:38.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccb4141d20 msgr2=0x7fccb4074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccb4141d20 0x7fccb4074130 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fcca000ea20 tx=0x7fcca000eee0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 shutdown_connections 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc940725e0 0x7fcc94074aa0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fccb415fc10 0x7fccb40b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccb4141d20 0x7fccb4074130 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 --2- 172.21.15.97:0/2145397559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fccb40ce5f0 0x7fccb413b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 >> 172.21.15.97:0/2145397559 conn(0x7fccb40b1780 msgr2=0x7fccb415b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 shutdown_connections 2024-09-10T21:35:38.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.321+0000 7fccba800640 1 -- 172.21.15.97:0/2145397559 wait complete. 2024-09-10T21:35:38.342 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:35:38.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:35:38.442 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:35:38.522 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:35:38.522 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:38.523 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:38.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:35:38.532 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:35:38.532 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:38.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:35:38.541 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:38.541 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:38.583 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:35:38.583 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:35:38.583 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:38.592 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:35:38.592 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:35:38.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:38.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.769+0000 7fbc74a00640 1 -- 172.21.15.97:0/2897156219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540039f0 msgr2=0x7fbc54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.769+0000 7fbc74a00640 1 --2- 172.21.15.97:0/2897156219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540039f0 0x7fbc54003dd0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fbc600017f0 tx=0x7fbc6001c900 comp rx=0 tx=0).stop 2024-09-10T21:35:38.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.769+0000 7fbc6f200640 1 -- 172.21.15.97:0/2897156219 reap_dead start 2024-09-10T21:35:38.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.769+0000 7fbc74a00640 1 -- 172.21.15.97:0/2897156219 shutdown_connections 2024-09-10T21:35:38.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.769+0000 7fbc74a00640 1 -- 172.21.15.97:0/2897156219 >> 172.21.15.97:0/2897156219 conn(0x7fbc700d1a70 msgr2=0x7fbc700d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:38.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.769+0000 7fbc74a00640 1 -- 172.21.15.97:0/2897156219 shutdown_connections 2024-09-10T21:35:38.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.769+0000 7fbc74a00640 1 -- 172.21.15.97:0/2897156219 wait complete. 2024-09-10T21:35:38.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 Processor -- start 2024-09-10T21:35:38.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 -- start start 2024-09-10T21:35:38.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540039f0 0x7fbc7015bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc701f8180 0x7fbc700c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc70160e30 0x7fbc701612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70167c90 con 0x7fbc701f8180 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70167b10 con 0x7fbc70160e30 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbc70167e10 con 0x7fbc540039f0 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc701f8180 0x7fbc700c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc701f8180 0x7fbc700c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52342/0 (socket says 172.21.15.97:52342) 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 -- 172.21.15.97:0/2137134287 learned_addr learned my addr 172.21.15.97:0/2137134287 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6e800640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540039f0 0x7fbc7015bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6f200640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc70160e30 0x7fbc701612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 -- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540039f0 msgr2=0x7fbc7015bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540039f0 0x7fbc7015bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 -- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc70160e30 msgr2=0x7fbc701612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc70160e30 0x7fbc701612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 -- 172.21.15.97:0/2137134287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc701f85e0 con 0x7fbc701f8180 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc6de00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc701f8180 0x7fbc700c7ea0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c011860 tx=0x7fbc5c011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:38.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc66a00640 1 -- 172.21.15.97:0/2137134287 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c004460 con 0x7fbc701f8180 2024-09-10T21:35:38.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc66a00640 1 -- 172.21.15.97:0/2137134287 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc5c016070 con 0x7fbc701f8180 2024-09-10T21:35:38.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc66a00640 1 -- 172.21.15.97:0/2137134287 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c005350 con 0x7fbc701f8180 2024-09-10T21:35:38.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7015a0e0 con 0x7fbc701f8180 2024-09-10T21:35:38.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7015a6a0 con 0x7fbc701f8180 2024-09-10T21:35:38.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.773+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7015f9c0 con 0x7fbc701f8180 2024-09-10T21:35:38.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.777+0000 7fbc66a00640 1 -- 172.21.15.97:0/2137134287 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbc5c0054f0 con 0x7fbc701f8180 2024-09-10T21:35:38.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.777+0000 7fbc66a00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc50072610 0x7fbc50074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:38.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.777+0000 7fbc66a00640 1 -- 172.21.15.97:0/2137134287 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7fbc5c094480 con 0x7fbc701f8180 2024-09-10T21:35:38.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.777+0000 7fbc6e800640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc50072610 0x7fbc50074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:38.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.777+0000 7fbc6e800640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc50072610 0x7fbc50074ad0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbc600017c0 tx=0x7fbc600b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:38.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.781+0000 7fbc66a00640 1 -- 172.21.15.97:0/2137134287 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbc5c05e940 con 0x7fbc701f8180 2024-09-10T21:35:38.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc701ed520 con 0x7fbc701f8180 2024-09-10T21:35:38.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc66a00640 1 -- 172.21.15.97:0/2137134287 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 337 v337) ==== 95+0+3919 (secure 0 0 0) 0x7fbc5c063370 con 0x7fbc701f8180 2024-09-10T21:35:38.980 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:38.980 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":337,"btime":"2024-09-10T21:35:32:589987+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3626341358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3626341358},{"type":"v1","addr":"172.21.15.194:6837","nonce":3626341358}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":6538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/3455669369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3455669369},{"type":"v1","addr":"172.21.15.194:6836","nonce":3455669369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":10470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/2268667754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2268667754},{"type":"v1","addr":"172.21.15.97:6834","nonce":2268667754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":331},{"gid":10518,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/640676491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":640676491},{"type":"v1","addr":"172.21.15.97:6835","nonce":640676491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335}],"filesystems":[]} 2024-09-10T21:35:38.981 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 337 2024-09-10T21:35:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc50072610 msgr2=0x7fbc50074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc50072610 0x7fbc50074ad0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbc600017c0 tx=0x7fbc600b1040 comp rx=0 tx=0).stop 2024-09-10T21:35:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc701f8180 msgr2=0x7fbc700c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc701f8180 0x7fbc700c7ea0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c011860 tx=0x7fbc5c011d20 comp rx=0 tx=0).stop 2024-09-10T21:35:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 shutdown_connections 2024-09-10T21:35:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc50072610 0x7fbc50074ad0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc70160e30 0x7fbc701612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc701f8180 0x7fbc700c7ea0 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 --2- 172.21.15.97:0/2137134287 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540039f0 0x7fbc7015bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 >> 172.21.15.97:0/2137134287 conn(0x7fbc700d1a70 msgr2=0x7fbc7015fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 shutdown_connections 2024-09-10T21:35:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:38.981+0000 7fbc74a00640 1 -- 172.21.15.97:0/2137134287 wait complete. 2024-09-10T21:35:39.002 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:35:39.003 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:35:39.003 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:35:39.003 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:35:39.003 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:35:39.003 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:35:39.003 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:35:39.004 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:35:39.004 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:35:39.004 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:35:39.004 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:35:39.004 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:35:39.005 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:35:39.005 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:35:39.005 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:35:39.005 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:35:39.016 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:35:39.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:35:39.017 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:35:39.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:35:39.018 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:35:39.019 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:35:39.020 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:35:39.020 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:35:39.062 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:35:39.062 DEBUG:teuthology.parallel:result is None 2024-09-10T21:35:39.063 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:35:39.064 DEBUG:teuthology.parallel:result is None 2024-09-10T21:35:39.067 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:35:39.067 DEBUG:teuthology.parallel:result is None 2024-09-10T21:35:39.068 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:35:39.069 DEBUG:teuthology.parallel:result is None 2024-09-10T21:35:39.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:35:39.148 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-09-10T21:35:39.148 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:35:39.159 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:35:39.159 INFO:tasks.ceph.mds.b.smithi194.stdout:starting mds.b at 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- 172.21.15.97:0/1695612823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce5f0 msgr2=0x7f3810141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 --2- 172.21.15.97:0/1695612823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce5f0 0x7f3810141d20 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f3804009900 tx=0x7f380401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- 172.21.15.97:0/1695612823 shutdown_connections 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 --2- 172.21.15.97:0/1695612823 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f381015fde0 0x7f381013f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 --2- 172.21.15.97:0/1695612823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce5f0 0x7f3810141d20 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- 172.21.15.97:0/1695612823 >> 172.21.15.97:0/1695612823 conn(0x7f38100b1780 msgr2=0x7f38100b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- 172.21.15.97:0/1695612823 shutdown_connections 2024-09-10T21:35:39.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- 172.21.15.97:0/1695612823 wait complete. 2024-09-10T21:35:39.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 Processor -- start 2024-09-10T21:35:39.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- start start 2024-09-10T21:35:39.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38100ce5f0 0x7f3810165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce9d0 0x7f381013baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381015fde0 0x7f381015e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3810168a70 con 0x7f38100ce9d0 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3810140d00 con 0x7f381015fde0 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3817e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3810168bf0 con 0x7f38100ce5f0 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381015fde0 0x7f381015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3814a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce9d0 0x7f381013baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38100ce5f0 0x7f3810165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381015fde0 0x7f381015e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:51198/0 (socket says 172.21.15.97:51198) 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 -- 172.21.15.97:0/1786830427 learned_addr learned my addr 172.21.15.97:0/1786830427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 -- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38100ce5f0 msgr2=0x7f3810165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38100ce5f0 0x7f3810165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 -- 172.21.15.97:0/1786830427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce9d0 msgr2=0x7f381013baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce9d0 0x7f381013baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 -- 172.21.15.97:0/1786830427 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38100b46b0 con 0x7f381015fde0 2024-09-10T21:35:39.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815400640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38100ce5f0 0x7f3810165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:35:39.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.221+0000 7f3815e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381015fde0 0x7f381015e260 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3808009fd0 tx=0x7f380800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:39.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f380d600640 1 -- 172.21.15.97:0/1786830427 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380800ce20 con 0x7f381015fde0 2024-09-10T21:35:39.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f380d600640 1 -- 172.21.15.97:0/1786830427 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808004b30 con 0x7f381015fde0 2024-09-10T21:35:39.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f380d600640 1 -- 172.21.15.97:0/1786830427 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808005020 con 0x7f381015fde0 2024-09-10T21:35:39.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38100b4990 con 0x7f381015fde0 2024-09-10T21:35:39.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38100b3630 con 0x7f381015fde0 2024-09-10T21:35:39.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3810141d20 con 0x7f381015fde0 2024-09-10T21:35:39.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f380d600640 1 -- 172.21.15.97:0/1786830427 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f38080051c0 con 0x7f381015fde0 2024-09-10T21:35:39.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f380d600640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37f00725e0 0x7f37f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f380d600640 1 -- 172.21.15.97:0/1786830427 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8689+0+0 (secure 0 0 0) 0x7f3808094e00 con 0x7f381015fde0 2024-09-10T21:35:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.225+0000 7f3815400640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37f00725e0 0x7f37f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.229+0000 7f3815400640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37f00725e0 0x7f37f0074aa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f38040af000 tx=0x7f380401b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:39.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.229+0000 7f380d600640 1 -- 172.21.15.97:0/1786830427 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f380805f340 con 0x7f381015fde0 2024-09-10T21:35:39.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.405+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f38100b3800 con 0x7f381015fde0 2024-09-10T21:35:39.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f380d600640 1 -- 172.21.15.97:0/1786830427 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v140) ==== 97+0+0 (secure 0 0 0) 0x7f38080680b0 con 0x7f381015fde0 2024-09-10T21:35:39.640 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-09-10T21:35:39.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37f00725e0 msgr2=0x7f37f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37f00725e0 0x7f37f0074aa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f38040af000 tx=0x7f380401b120 comp rx=0 tx=0).stop 2024-09-10T21:35:39.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381015fde0 msgr2=0x7f381015e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381015fde0 0x7f381015e260 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3808009fd0 tx=0x7f380800ef10 comp rx=0 tx=0).stop 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 shutdown_connections 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37f00725e0 0x7f37f0074aa0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381015fde0 0x7f381015e260 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38100ce9d0 0x7f381013baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 --2- 172.21.15.97:0/1786830427 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f38100ce5f0 0x7f3810165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 >> 172.21.15.97:0/1786830427 conn(0x7f38100b1780 msgr2=0x7f381015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 shutdown_connections 2024-09-10T21:35:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.641+0000 7f3817e00640 1 -- 172.21.15.97:0/1786830427 wait complete. 2024-09-10T21:35:39.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:35:39.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 -- 172.21.15.97:0/1936750448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f74004d50 msgr2=0x7f8f740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 --2- 172.21.15.97:0/1936750448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f74004d50 0x7f8f740051d0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009ee0 tx=0x7f8f8801d5f0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f8fe00640 1 -- 172.21.15.97:0/1936750448 reap_dead start 2024-09-10T21:35:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 -- 172.21.15.97:0/1936750448 shutdown_connections 2024-09-10T21:35:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 -- 172.21.15.97:0/1936750448 >> 172.21.15.97:0/1936750448 conn(0x7f8f900d1c30 msgr2=0x7f8f900d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 -- 172.21.15.97:0/1936750448 shutdown_connections 2024-09-10T21:35:39.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 -- 172.21.15.97:0/1936750448 wait complete. 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 Processor -- start 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.809+0000 7f8f96800640 1 -- start start 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f74004d50 0x7f8f900c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f900b7280 0x7f8f900c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f900c80c0 0x7f8f900c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8f900ccf60 con 0x7f8f900b7280 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f900ccde0 con 0x7f8f900c80c0 2024-09-10T21:35:39.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8f900cd0e0 con 0x7f8f74004d50 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f900b7280 0x7f8f900c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f900b7280 0x7f8f900c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52412/0 (socket says 172.21.15.97:52412) 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 -- 172.21.15.97:0/1538375089 learned_addr learned my addr 172.21.15.97:0/1538375089 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8f400640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f74004d50 0x7f8f900c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8fe00640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f900c80c0 0x7f8f900c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 -- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f74004d50 msgr2=0x7f8f900c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f74004d50 0x7f8f900c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 -- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f900c80c0 msgr2=0x7f8f900c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f900c80c0 0x7f8f900c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 -- 172.21.15.97:0/1538375089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f900c6c90 con 0x7f8f900b7280 2024-09-10T21:35:39.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f8ea00640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f900b7280 0x7f8f900c9ba0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c003eb0 tx=0x7f8f7c001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:39.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f87400640 1 -- 172.21.15.97:0/1538375089 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c015070 con 0x7f8f900b7280 2024-09-10T21:35:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f87400640 1 -- 172.21.15.97:0/1538375089 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f7c011900 con 0x7f8f900b7280 2024-09-10T21:35:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f87400640 1 -- 172.21.15.97:0/1538375089 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c011d50 con 0x7f8f900b7280 2024-09-10T21:35:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f900b4c00 con 0x7f8f900b7280 2024-09-10T21:35:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f900b54a0 con 0x7f8f900b7280 2024-09-10T21:35:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f900d0690 con 0x7f8f900b7280 2024-09-10T21:35:39.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.813+0000 7f8f87400640 1 -- 172.21.15.97:0/1538375089 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8f7c0103c0 con 0x7f8f900b7280 2024-09-10T21:35:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.817+0000 7f8f87400640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8f700725e0 0x7f8f70074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.817+0000 7f8f87400640 1 -- 172.21.15.97:0/1538375089 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5249+0+0 (secure 0 0 0) 0x7f8f7c0940e0 con 0x7f8f900b7280 2024-09-10T21:35:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.817+0000 7f8f8f400640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8f700725e0 0x7f8f70074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.817+0000 7f8f8f400640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8f700725e0 0x7f8f70074aa0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8f80001ce0 tx=0x7f8f800011d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:39.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:39.821+0000 7f8f87400640 1 -- 172.21.15.97:0/1538375089 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8f7c05f290 con 0x7f8f900b7280 2024-09-10T21:35:39.949 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:39.953 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.045+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8f900cd670 con 0x7f8f900b7280 2024-09-10T21:35:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.045+0000 7f8f87400640 1 -- 172.21.15.97:0/1538375089 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f8f7c063cc0 con 0x7f8f900b7280 2024-09-10T21:35:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:40.046 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:35:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-09-10T21:35:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmPoo4NRAAVmXQDgh9pF8kFFMh0WJ+BA==", 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmYygqBhAAuFiWTIb345efacgw+4RC4g==", 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmR9sEARAABIzg0rYZ9H/kK4guI+riXw==", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:35:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm9KCGDRAAoiCQJIDo4uG0CLoaNjZVpA==", 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDCuOBmoKzIFBAAbmTri6pSmbmpYNRFvD4T7Q==", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDFuOBmkoklIxAAKt7RBgGZZZmsXPol9n9G3Q==", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJuOBmnTJkCRAAH0ZmaAYErgkn5uZK+dU3Fg==", 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-09-10T21:35:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDMuOBmxRVYHxAACEfEswq+Yd2qXCzkFPQfGQ==", 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSuOBm6OavJBAAJhy7Uzq5/I7uNug4BhJeqQ==", 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVuOBmLXdHOxAA/FhYjS9sRCDU+9vtInCYqw==", 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-09-10T21:35:40.053 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDZuOBmjDvPFBAAYMdVz2/DPHlOC4BQJVaBuQ==", 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDcuOBmu/tkIxAAzwVb0RlbSYqGWTtHTQfz1Q==", 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-09-10T21:35:40.054 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q==", 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.055 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmjmEUGxAAGl7y8Ft2CaBELPu4sL6VsQ==", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmmzNcFxAAuubFP4tiZ4Azz+FLuIpSag==", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.056 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC/uOBmT6f7IBAA3B/5JAbN3SlKYOcN7DpyhA==", 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBmjCI0FRAATFUoCxw0bU6HSEUnlyrVJQ==", 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-09-10T21:35:40.058 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm02gILRAAy5FFywUl/Md90YqB5K2DVA==", 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC+uOBm6U8AKRAAYMafLRJIqdGS12q+uL0Miw==", 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-09-10T21:35:40.059 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-09-10T21:35:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.060 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-10T21:35:40.060 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8f700725e0 msgr2=0x7f8f70074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8f700725e0 0x7f8f70074aa0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8f80001ce0 tx=0x7f8f800011d0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f900b7280 msgr2=0x7f8f900c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f900b7280 0x7f8f900c9ba0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c003eb0 tx=0x7f8f7c001810 comp rx=0 tx=0).stop 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 shutdown_connections 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8f700725e0 0x7f8f70074aa0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f900c80c0 0x7f8f900c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f900b7280 0x7f8f900c9ba0 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 --2- 172.21.15.97:0/1538375089 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8f74004d50 0x7f8f900c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 >> 172.21.15.97:0/1538375089 conn(0x7f8f900d1c30 msgr2=0x7f8f900cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 shutdown_connections 2024-09-10T21:35:40.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.049+0000 7f8f96800640 1 -- 172.21.15.97:0/1538375089 wait complete. 2024-09-10T21:35:40.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:35:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 -- 172.21.15.97:0/839494908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff415efc0 msgr2=0x7f6ff40cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/839494908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff415efc0 0x7f6ff40cf840 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40b3410 tx=0x7f6fe401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 -- 172.21.15.97:0/839494908 shutdown_connections 2024-09-10T21:35:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/839494908 >> v2:172.21.15.194:3300/0 conn(0x7f6ff40cdcb0 0x7f6ff40cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/839494908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff415efc0 0x7f6ff40cf840 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 -- 172.21.15.97:0/839494908 >> 172.21.15.97:0/839494908 conn(0x7f6ff40d1a60 msgr2=0x7f6ff40d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:40.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 -- 172.21.15.97:0/839494908 shutdown_connections 2024-09-10T21:35:40.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 -- 172.21.15.97:0/839494908 wait complete. 2024-09-10T21:35:40.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 Processor -- start 2024-09-10T21:35:40.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.209+0000 7f6ff8a00640 1 -- start start 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ff40cdcb0 0x7f6ff4154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40d02c0 0x7f6ff415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ff415efc0 0x7f6ff415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4168d50 con 0x7f6ff40d02c0 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4168bd0 con 0x7f6ff415efc0 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff4168ed0 con 0x7f6ff40cdcb0 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40d02c0 0x7f6ff415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40d02c0 0x7f6ff415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52418/0 (socket says 172.21.15.97:52418) 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 -- 172.21.15.97:0/1069450801 learned_addr learned my addr 172.21.15.97:0/1069450801 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:40.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff3e00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ff415efc0 0x7f6ff415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff3400640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ff40cdcb0 0x7f6ff4154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 -- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ff40cdcb0 msgr2=0x7f6ff4154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ff40cdcb0 0x7f6ff4154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 -- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ff415efc0 msgr2=0x7f6ff415a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ff415efc0 0x7f6ff415a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 -- 172.21.15.97:0/1069450801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff4159cb0 con 0x7f6ff40d02c0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff2a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40d02c0 0x7f6ff415a160 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000d950 tx=0x7f6fe000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6feb400640 1 -- 172.21.15.97:0/1069450801 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0017070 con 0x7f6ff40d02c0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6feb400640 1 -- 172.21.15.97:0/1069450801 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fe0009d20 con 0x7f6ff40d02c0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6feb400640 1 -- 172.21.15.97:0/1069450801 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00106c0 con 0x7f6ff40d02c0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff40b5d50 con 0x7f6ff40d02c0 2024-09-10T21:35:40.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff40b6310 con 0x7f6ff40d02c0 2024-09-10T21:35:40.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.213+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff40ce1e0 con 0x7f6ff40d02c0 2024-09-10T21:35:40.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.217+0000 7f6feb400640 1 -- 172.21.15.97:0/1069450801 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6fe0010860 con 0x7f6ff40d02c0 2024-09-10T21:35:40.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.217+0000 7f6feb400640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6fd4072640 0x7f6fd4074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.217+0000 7f6feb400640 1 -- 172.21.15.97:0/1069450801 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5249+0+0 (secure 0 0 0) 0x7f6fe00933e0 con 0x7f6ff40d02c0 2024-09-10T21:35:40.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.217+0000 7f6ff3400640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6fd4072640 0x7f6fd4074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.217+0000 7f6ff3400640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6fd4072640 0x7f6fd4074b00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40af000 tx=0x7f6fe40a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:40.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.221+0000 7f6feb400640 1 -- 172.21.15.97:0/1069450801 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6fe005e620 con 0x7f6ff40d02c0 2024-09-10T21:35:40.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6ff40cc6b0 con 0x7f6ff40d02c0 2024-09-10T21:35:40.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6feb400640 1 -- 172.21.15.97:0/1069450801 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v142) ==== 84+0+65 (secure 0 0 0) 0x7f6fe0063050 con 0x7f6ff40d02c0 2024-09-10T21:35:40.388 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:40.389 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:35:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:35:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-09-10T21:35:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-09-10T21:35:40.389 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:35:40.389 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:35:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6fd4072640 msgr2=0x7f6fd4074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6fd4072640 0x7f6fd4074b00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40af000 tx=0x7f6fe40a7040 comp rx=0 tx=0).stop 2024-09-10T21:35:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40d02c0 msgr2=0x7f6ff415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40d02c0 0x7f6ff415a160 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000d950 tx=0x7f6fe000de10 comp rx=0 tx=0).stop 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 shutdown_connections 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6fd4072640 0x7f6fd4074b00 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ff415efc0 0x7f6ff415a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ff40d02c0 0x7f6ff415a160 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 --2- 172.21.15.97:0/1069450801 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ff40cdcb0 0x7f6ff4154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 >> 172.21.15.97:0/1069450801 conn(0x7f6ff40d1a60 msgr2=0x7f6ff415b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 shutdown_connections 2024-09-10T21:35:40.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.389+0000 7f6ff8a00640 1 -- 172.21.15.97:0/1069450801 wait complete. 2024-09-10T21:35:40.409 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:35:40.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:35:40.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 -- 172.21.15.97:0/2024892443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 msgr2=0x7efda4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 --2- 172.21.15.97:0/2024892443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 0x7efda4004810 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7efdc00718b0 tx=0x7efda801ce80 comp rx=0 tx=0).stop 2024-09-10T21:35:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdbe800640 1 -- 172.21.15.97:0/2024892443 reap_dead start 2024-09-10T21:35:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 -- 172.21.15.97:0/2024892443 shutdown_connections 2024-09-10T21:35:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 -- 172.21.15.97:0/2024892443 >> 172.21.15.97:0/2024892443 conn(0x7efdc00b28c0 msgr2=0x7efdc00b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 -- 172.21.15.97:0/2024892443 shutdown_connections 2024-09-10T21:35:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 -- 172.21.15.97:0/2024892443 wait complete. 2024-09-10T21:35:40.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 Processor -- start 2024-09-10T21:35:40.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.557+0000 7efdc4e00640 1 -- start start 2024-09-10T21:35:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 0x7efdc01497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdc015ad50 0x7efdc01669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc0165610 0x7efdc0166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efdc006d4a0 con 0x7efda4003f80 2024-09-10T21:35:40.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efdc006d320 con 0x7efdc0165610 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efdc016a440 con 0x7efdc015ad50 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 0x7efdc01497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 0x7efdc01497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52440/0 (socket says 172.21.15.97:52440) 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 -- 172.21.15.97:0/977420768 learned_addr learned my addr 172.21.15.97:0/977420768 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbe800640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc0165610 0x7efdc0166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdb5400640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdc015ad50 0x7efdc01669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 -- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdc015ad50 msgr2=0x7efdc01669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdc015ad50 0x7efdc01669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 -- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc0165610 msgr2=0x7efdc0166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc0165610 0x7efdc0166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 -- 172.21.15.97:0/977420768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc00cf910 con 0x7efda4003f80 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdb5400640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdc015ad50 0x7efdc01669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdbde00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 0x7efdc01497f0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009fd0 tx=0x7efdb000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:40.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdb6a00640 1 -- 172.21.15.97:0/977420768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0017070 con 0x7efda4003f80 2024-09-10T21:35:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdb6a00640 1 -- 172.21.15.97:0/977420768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdb0011730 con 0x7efda4003f80 2024-09-10T21:35:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc015e580 con 0x7efda4003f80 2024-09-10T21:35:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdb6a00640 1 -- 172.21.15.97:0/977420768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0011b20 con 0x7efda4003f80 2024-09-10T21:35:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc0165df0 con 0x7efda4003f80 2024-09-10T21:35:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc00cead0 con 0x7efda4003f80 2024-09-10T21:35:40.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.561+0000 7efdb6a00640 1 -- 172.21.15.97:0/977420768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efdb0011cc0 con 0x7efda4003f80 2024-09-10T21:35:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.565+0000 7efdb6a00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd9c0725e0 0x7efd9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.565+0000 7efdb6a00640 1 -- 172.21.15.97:0/977420768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5249+0+0 (secure 0 0 0) 0x7efdb0093f00 con 0x7efda4003f80 2024-09-10T21:35:40.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.565+0000 7efdb5400640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd9c0725e0 0x7efd9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:40.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.565+0000 7efdb5400640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd9c0725e0 0x7efd9c074aa0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efda40057f0 tx=0x7efda8019cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.569+0000 7efdb6a00640 1 -- 172.21.15.97:0/977420768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efdb005f700 con 0x7efda4003f80 2024-09-10T21:35:40.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:40.737+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efdc00b2e70 con 0x7efda4003f80 2024-09-10T21:35:41.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.657+0000 7efdb6a00640 1 -- 172.21.15.97:0/977420768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v143) ==== 138+0+0 (secure 0 0 0) 0x7efdb005f000 con 0x7efda4003f80 2024-09-10T21:35:41.657 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd9c0725e0 msgr2=0x7efd9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd9c0725e0 0x7efd9c074aa0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efda40057f0 tx=0x7efda8019cc0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 msgr2=0x7efdc01497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 0x7efdc01497f0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009fd0 tx=0x7efdb000e720 comp rx=0 tx=0).stop 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 shutdown_connections 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efd9c0725e0 0x7efd9c074aa0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdc0165610 0x7efdc0166ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdc015ad50 0x7efdc01669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 --2- 172.21.15.97:0/977420768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4003f80 0x7efdc01497f0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 >> 172.21.15.97:0/977420768 conn(0x7efdc00b28c0 msgr2=0x7efdc0161cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 shutdown_connections 2024-09-10T21:35:41.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.661+0000 7efdc4e00640 1 -- 172.21.15.97:0/977420768 wait complete. 2024-09-10T21:35:41.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 -- 172.21.15.97:0/3119560865 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac15efc0 msgr2=0x7f26ac0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 --2- 172.21.15.97:0/3119560865 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac15efc0 0x7f26ac0cf840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f269c009840 tx=0x7f269c01c740 comp rx=0 tx=0).stop 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 -- 172.21.15.97:0/3119560865 shutdown_connections 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 --2- 172.21.15.97:0/3119560865 >> v2:172.21.15.194:3300/0 conn(0x7f26ac0cdcb0 0x7f26ac0cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 --2- 172.21.15.97:0/3119560865 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac15efc0 0x7f26ac0cf840 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 -- 172.21.15.97:0/3119560865 >> 172.21.15.97:0/3119560865 conn(0x7f26ac0d1a60 msgr2=0x7f26ac0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 -- 172.21.15.97:0/3119560865 shutdown_connections 2024-09-10T21:35:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 -- 172.21.15.97:0/3119560865 wait complete. 2024-09-10T21:35:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 Processor -- start 2024-09-10T21:35:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.837+0000 7f26abe00640 1 -- start start 2024-09-10T21:35:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0cdcb0 0x7f26ac166140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d02c0 0x7f26ac1f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15efc0 0x7f26ac15d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac168f20 con 0x7f26ac0d02c0 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac168da0 con 0x7f26ac15efc0 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f26ac1690a0 con 0x7f26ac0cdcb0 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d02c0 0x7f26ac1f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d02c0 0x7f26ac1f8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52446/0 (socket says 172.21.15.97:52446) 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26aaa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0cdcb0 0x7f26ac166140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 -- 172.21.15.97:0/512229806 learned_addr learned my addr 172.21.15.97:0/512229806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26ab400640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15efc0 0x7f26ac15d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 -- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0cdcb0 msgr2=0x7f26ac166140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0cdcb0 0x7f26ac166140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 -- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15efc0 msgr2=0x7f26ac15d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15efc0 0x7f26ac15d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 -- 172.21.15.97:0/512229806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac1f8d80 con 0x7f26ac0d02c0 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a2000640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d02c0 0x7f26ac1f8840 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f269400e9c0 tx=0x7f269400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a3400640 1 -- 172.21.15.97:0/512229806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269400cd80 con 0x7f26ac0d02c0 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a3400640 1 -- 172.21.15.97:0/512229806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2694004020 con 0x7f26ac0d02c0 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac155fb0 con 0x7f26ac0d02c0 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26a3400640 1 -- 172.21.15.97:0/512229806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694004450 con 0x7f26ac0d02c0 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac156570 con 0x7f26ac0d02c0 2024-09-10T21:35:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.841+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ac0ce1e0 con 0x7f26ac0d02c0 2024-09-10T21:35:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.845+0000 7f26a3400640 1 -- 172.21.15.97:0/512229806 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2694003600 con 0x7f26ac0d02c0 2024-09-10T21:35:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.845+0000 7f26a3400640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26880725e0 0x7f2688074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.845+0000 7f26a3400640 1 -- 172.21.15.97:0/512229806 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5619+0+0 (secure 0 0 0) 0x7f2694093c00 con 0x7f26ac0d02c0 2024-09-10T21:35:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.845+0000 7f26aaa00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26880725e0 0x7f2688074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.845+0000 7f26aaa00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26880725e0 0x7f2688074aa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f269c0af000 tx=0x7f269c01af80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:41.849+0000 7f26a3400640 1 -- 172.21.15.97:0/512229806 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f26ac1f8d80 con 0x7f26ac0d02c0 2024-09-10T21:35:42.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.017+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f26ac158b50 con 0x7f26ac0d02c0 2024-09-10T21:35:42.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.665+0000 7f26a3400640 1 -- 172.21.15.97:0/512229806 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v144) ==== 170+0+0 (secure 0 0 0) 0x7f269405ec40 con 0x7f26ac0d02c0 2024-09-10T21:35:42.667 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-09-10T21:35:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26880725e0 msgr2=0x7f2688074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26880725e0 0x7f2688074aa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f269c0af000 tx=0x7f269c01af80 comp rx=0 tx=0).stop 2024-09-10T21:35:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d02c0 msgr2=0x7f26ac1f8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d02c0 0x7f26ac1f8840 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f269400e9c0 tx=0x7f269400ee80 comp rx=0 tx=0).stop 2024-09-10T21:35:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 shutdown_connections 2024-09-10T21:35:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f26880725e0 0x7f2688074aa0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ac15efc0 0x7f26ac15d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26ac0d02c0 0x7f26ac1f8840 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 --2- 172.21.15.97:0/512229806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f26ac0cdcb0 0x7f26ac166140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 >> 172.21.15.97:0/512229806 conn(0x7f26ac0d1a60 msgr2=0x7f26ac0b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 shutdown_connections 2024-09-10T21:35:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.669+0000 7f26abe00640 1 -- 172.21.15.97:0/512229806 wait complete. 2024-09-10T21:35:42.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:35:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 -- 172.21.15.97:0/4179888710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a00b1590 msgr2=0x7f74a00d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4179888710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a00b1590 0x7f74a00d0370 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7498009c80 tx=0x7f749801d770 comp rx=0 tx=0).stop 2024-09-10T21:35:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 -- 172.21.15.97:0/4179888710 shutdown_connections 2024-09-10T21:35:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4179888710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a00b1590 0x7f74a00d0370 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4179888710 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74a006dc50 0x7f74a00cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 -- 172.21.15.97:0/4179888710 >> 172.21.15.97:0/4179888710 conn(0x7f74a00d1c30 msgr2=0x7f74a00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 -- 172.21.15.97:0/4179888710 shutdown_connections 2024-09-10T21:35:42.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.853+0000 7f74a4c00640 1 -- 172.21.15.97:0/4179888710 wait complete. 2024-09-10T21:35:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 Processor -- start 2024-09-10T21:35:42.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 -- start start 2024-09-10T21:35:42.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74a00b1590 0x7f74a01507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74a0161600 0x7f74a01597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a0159cf0 0x7f74a0153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74a016aea0 con 0x7f74a00b1590 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f74a016ad20 con 0x7f74a0159cf0 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f74a016b020 con 0x7f74a0161600 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74a00b1590 0x7f74a01507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74a00b1590 0x7f74a01507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52458/0 (socket says 172.21.15.97:52458) 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 -- 172.21.15.97:0/4268435308 learned_addr learned my addr 172.21.15.97:0/4268435308 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749d400640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74a0161600 0x7f74a01597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749e800640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a0159cf0 0x7f74a0153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 -- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74a0161600 msgr2=0x7f74a01597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74a0161600 0x7f74a01597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 -- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a0159cf0 msgr2=0x7f74a0153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a0159cf0 0x7f74a0153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 -- 172.21.15.97:0/4268435308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a00b5a70 con 0x7f74a00b1590 2024-09-10T21:35:42.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749e800640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a0159cf0 0x7f74a0153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:35:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749d400640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74a0161600 0x7f74a01597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:35:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f749de00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74a00b1590 0x7f74a01507d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f749000c3f0 tx=0x7f749000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f7496000640 1 -- 172.21.15.97:0/4268435308 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490017070 con 0x7f74a00b1590 2024-09-10T21:35:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f7496000640 1 -- 172.21.15.97:0/4268435308 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74900053c0 con 0x7f74a00b1590 2024-09-10T21:35:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f7496000640 1 -- 172.21.15.97:0/4268435308 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490003860 con 0x7f74a00b1590 2024-09-10T21:35:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a00b4cb0 con 0x7f74a00b1590 2024-09-10T21:35:42.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.857+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a00b5270 con 0x7f74a00b1590 2024-09-10T21:35:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.861+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a00d0a60 con 0x7f74a00b1590 2024-09-10T21:35:42.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.861+0000 7f7496000640 1 -- 172.21.15.97:0/4268435308 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7490003a00 con 0x7f74a00b1590 2024-09-10T21:35:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.861+0000 7f7496000640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480072610 0x7f7480074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.861+0000 7f7496000640 1 -- 172.21.15.97:0/4268435308 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6001+0+0 (secure 0 0 0) 0x7f7490092950 con 0x7f74a00b1590 2024-09-10T21:35:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.861+0000 7f749d400640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480072610 0x7f7480074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.861+0000 7f749d400640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480072610 0x7f7480074ad0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f74a0160cd0 tx=0x7f748c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:42.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:42.865+0000 7f7496000640 1 -- 172.21.15.97:0/4268435308 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f749005d820 con 0x7f74a00b1590 2024-09-10T21:35:43.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.065+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f74a00d2510 con 0x7f74a00b1590 2024-09-10T21:35:43.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.669+0000 7f7496000640 1 -- 172.21.15.97:0/4268435308 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v340) ==== 174+0+0 (secure 0 0 0) 0x7f7490062250 con 0x7f74a00b1590 2024-09-10T21:35:43.669 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 15 and data pool 16 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480072610 msgr2=0x7f7480074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480072610 0x7f7480074ad0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f74a0160cd0 tx=0x7f748c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74a00b1590 msgr2=0x7f74a01507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74a00b1590 0x7f74a01507d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f749000c3f0 tx=0x7f749000c8b0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 shutdown_connections 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7480072610 0x7f7480074ad0 secure :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f74a0160cd0 tx=0x7f748c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74a0159cf0 0x7f74a0153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f74a0161600 0x7f74a01597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 --2- 172.21.15.97:0/4268435308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74a00b1590 0x7f74a01507d0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 >> 172.21.15.97:0/4268435308 conn(0x7f74a00d1c30 msgr2=0x7f74a00cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 shutdown_connections 2024-09-10T21:35:43.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.673+0000 7f74a4c00640 1 -- 172.21.15.97:0/4268435308 wait complete. 2024-09-10T21:35:43.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 -- 172.21.15.97:0/472251632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc15fde0 msgr2=0x7f5cfc13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 --2- 172.21.15.97:0/472251632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc13f9b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009bc0 tx=0x7f5cf401cba0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 -- 172.21.15.97:0/472251632 shutdown_connections 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 --2- 172.21.15.97:0/472251632 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc13f9b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 --2- 172.21.15.97:0/472251632 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cfc0ce5f0 0x7f5cfc141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 -- 172.21.15.97:0/472251632 >> 172.21.15.97:0/472251632 conn(0x7f5cfc0b1780 msgr2=0x7f5cfc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 -- 172.21.15.97:0/472251632 shutdown_connections 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 -- 172.21.15.97:0/472251632 wait complete. 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 Processor -- start 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.861+0000 7f5d01000640 1 -- start start 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cfc0ce5f0 0x7f5cfc1661f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc0ce9d0 0x7f5cfc154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc168bf0 con 0x7f5cfc15fde0 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc168a70 con 0x7f5cfc0ce9d0 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5cfc168d70 con 0x7f5cfc0ce5f0 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52464/0 (socket says 172.21.15.97:52464) 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 -- 172.21.15.97:0/2240425936 learned_addr learned my addr 172.21.15.97:0/2240425936 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cf9e00640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cfc0ce5f0 0x7f5cfc1661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cf3e00640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc0ce9d0 0x7f5cfc154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 -- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cfc0ce5f0 msgr2=0x7f5cfc1661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cfc0ce5f0 0x7f5cfc1661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 -- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc0ce9d0 msgr2=0x7f5cfc154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc0ce9d0 0x7f5cfc154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 -- 172.21.15.97:0/2240425936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc0b4ad0 con 0x7f5cfc15fde0 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cfa800640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc15a370 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f5cf401a7d0 tx=0x7f5cf4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cf2000640 1 -- 172.21.15.97:0/2240425936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40bb070 con 0x7f5cfc15fde0 2024-09-10T21:35:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cf2000640 1 -- 172.21.15.97:0/2240425936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cf4004b40 con 0x7f5cfc15fde0 2024-09-10T21:35:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5cf2000640 1 -- 172.21.15.97:0/2240425936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40b2040 con 0x7f5cfc15fde0 2024-09-10T21:35:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cfc164c20 con 0x7f5cfc15fde0 2024-09-10T21:35:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cfc1651e0 con 0x7f5cfc15fde0 2024-09-10T21:35:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.865+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cfc141d20 con 0x7f5cfc15fde0 2024-09-10T21:35:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.869+0000 7f5cf2000640 1 -- 172.21.15.97:0/2240425936 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5cf40b5030 con 0x7f5cfc15fde0 2024-09-10T21:35:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.869+0000 7f5cf2000640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cd80725e0 0x7f5cd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.869+0000 7f5cf2000640 1 -- 172.21.15.97:0/2240425936 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f5cf412cc40 con 0x7f5cfc15fde0 2024-09-10T21:35:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.869+0000 7f5cf9e00640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cd80725e0 0x7f5cd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.869+0000 7f5cf9e00640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cd80725e0 0x7f5cd8074aa0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009b80 tx=0x7f5cec009260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:43.873+0000 7f5cf2000640 1 -- 172.21.15.97:0/2240425936 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5cf4016790 con 0x7f5cfc15fde0 2024-09-10T21:35:44.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.037+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cfc06ef40 con 0x7f5cfc15fde0 2024-09-10T21:35:44.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5cf2000640 1 -- 172.21.15.97:0/2240425936 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16997 (secure 0 0 0) 0x7f5cf40f80f0 con 0x7f5cfc15fde0 2024-09-10T21:35:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:44.040 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":145,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:43.069763+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:35:40.751434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T21:35:42.030160+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cd80725e0 msgr2=0x7f5cd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cd80725e0 0x7f5cd8074aa0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009b80 tx=0x7f5cec009260 comp rx=0 tx=0).stop 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cfc15fde0 msgr2=0x7f5cfc15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc15a370 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f5cf401a7d0 tx=0x7f5cf4003f60 comp rx=0 tx=0).stop 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 shutdown_connections 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5cd80725e0 0x7f5cd8074aa0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cfc15fde0 0x7f5cfc15a370 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cfc0ce9d0 0x7f5cfc154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 --2- 172.21.15.97:0/2240425936 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5cfc0ce5f0 0x7f5cfc1661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 >> 172.21.15.97:0/2240425936 conn(0x7f5cfc0b1780 msgr2=0x7f5cfc072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:44.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 shutdown_connections 2024-09-10T21:35:44.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.041+0000 7f5d01000640 1 -- 172.21.15.97:0/2240425936 wait complete. 2024-09-10T21:35:44.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 -- 172.21.15.97:0/2408018505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 msgr2=0x7f57080cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 --2- 172.21.15.97:0/2408018505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 0x7f57080cc6d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009930 tx=0x7f56f001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 -- 172.21.15.97:0/2408018505 shutdown_connections 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 --2- 172.21.15.97:0/2408018505 >> v2:172.21.15.194:3300/0 conn(0x7f57080b0d50 0x7f57080ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 --2- 172.21.15.97:0/2408018505 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 0x7f57080cc6d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 -- 172.21.15.97:0/2408018505 >> 172.21.15.97:0/2408018505 conn(0x7f57080d1c30 msgr2=0x7f57080d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 -- 172.21.15.97:0/2408018505 shutdown_connections 2024-09-10T21:35:44.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.201+0000 7f570e400640 1 -- 172.21.15.97:0/2408018505 wait complete. 2024-09-10T21:35:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 Processor -- start 2024-09-10T21:35:44.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 -- start start 2024-09-10T21:35:44.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 0x7f57080b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57080b0d50 0x7f57081491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57080ce640 0x7f570815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5708168ee0 con 0x7f57080b0d50 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5708168d60 con 0x7f57080ce640 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5708169060 con 0x7f57080afdd0 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57080b0d50 0x7f57081491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57080b0d50 0x7f57081491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52480/0 (socket says 172.21.15.97:52480) 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 -- 172.21.15.97:0/2218114344 learned_addr learned my addr 172.21.15.97:0/2218114344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f5707400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 0x7f57080b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f5707e00640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57080ce640 0x7f570815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 -- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 msgr2=0x7f57080b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 0x7f57080b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 -- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57080ce640 msgr2=0x7f570815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57080ce640 0x7f570815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 -- 172.21.15.97:0/2218114344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57080c8a20 con 0x7f57080b0d50 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f5707e00640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57080ce640 0x7f570815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56fea00640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57080b0d50 0x7f57081491c0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f56f00b0040 tx=0x7f56f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56ffe00640 1 -- 172.21.15.97:0/2218114344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00b2040 con 0x7f57080b0d50 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57080b2ec0 con 0x7f57080b0d50 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56ffe00640 1 -- 172.21.15.97:0/2218114344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56f0005030 con 0x7f57080b0d50 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f56ffe00640 1 -- 172.21.15.97:0/2218114344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0005480 con 0x7f57080b0d50 2024-09-10T21:35:44.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57081f9390 con 0x7f57080b0d50 2024-09-10T21:35:44.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.205+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57080cea20 con 0x7f57080b0d50 2024-09-10T21:35:44.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.209+0000 7f56ffe00640 1 -- 172.21.15.97:0/2218114344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f56f00046d0 con 0x7f57080b0d50 2024-09-10T21:35:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.209+0000 7f56ffe00640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56e40725e0 0x7f56e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.209+0000 7f56ffe00640 1 -- 172.21.15.97:0/2218114344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7f56f012cb30 con 0x7f57080b0d50 2024-09-10T21:35:44.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.209+0000 7f5707400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56e40725e0 0x7f56e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.209+0000 7f5707400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56e40725e0 0x7f56e4074aa0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f56f8007fd0 tx=0x7f56f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:44.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.213+0000 7f56ffe00640 1 -- 172.21.15.97:0/2218114344 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f57080c8a20 con 0x7f57080b0d50 2024-09-10T21:35:44.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.377+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f570806ac40 con 0x7f57080b0d50 2024-09-10T21:35:44.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.377+0000 7f56ffe00640 1 -- 172.21.15.97:0/2218114344 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16997 (secure 0 0 0) 0x7f56f00164d0 con 0x7f57080b0d50 2024-09-10T21:35:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:44.379 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":145,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:43.069763+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:35:40.751434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T21:35:42.030160+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:35:44.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56e40725e0 msgr2=0x7f56e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56e40725e0 0x7f56e4074aa0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f56f8007fd0 tx=0x7f56f8009040 comp rx=0 tx=0).stop 2024-09-10T21:35:44.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57080b0d50 msgr2=0x7f57081491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57080b0d50 0x7f57081491c0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f56f00b0040 tx=0x7f56f0004040 comp rx=0 tx=0).stop 2024-09-10T21:35:44.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 shutdown_connections 2024-09-10T21:35:44.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f56e40725e0 0x7f56e4074aa0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57080ce640 0x7f570815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57080b0d50 0x7f57081491c0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 --2- 172.21.15.97:0/2218114344 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f57080afdd0 0x7f57080b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 >> 172.21.15.97:0/2218114344 conn(0x7f57080d1c30 msgr2=0x7f570815e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:44.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 shutdown_connections 2024-09-10T21:35:44.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.381+0000 7f570e400640 1 -- 172.21.15.97:0/2218114344 wait complete. 2024-09-10T21:35:44.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:35:44.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 -- 172.21.15.97:0/1513525034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc815efc0 msgr2=0x7effc80cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 --2- 172.21.15.97:0/1513525034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc815efc0 0x7effc80cf840 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7effb8009840 tx=0x7effb801c820 comp rx=0 tx=0).stop 2024-09-10T21:35:44.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 -- 172.21.15.97:0/1513525034 shutdown_connections 2024-09-10T21:35:44.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 --2- 172.21.15.97:0/1513525034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc815efc0 0x7effc80cf840 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 -- 172.21.15.97:0/1513525034 >> 172.21.15.97:0/1513525034 conn(0x7effc80d1a60 msgr2=0x7effc80d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:44.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 -- 172.21.15.97:0/1513525034 shutdown_connections 2024-09-10T21:35:44.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 -- 172.21.15.97:0/1513525034 wait complete. 2024-09-10T21:35:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 Processor -- start 2024-09-10T21:35:44.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 -- start start 2024-09-10T21:35:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.545+0000 7effcd800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc80cdcb0 0x7effc80b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7effc80d0490 0x7effc80b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effc815efc0 0x7effc8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7effc8169b50 con 0x7effc815efc0 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7effc81699d0 con 0x7effc80cdcb0 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7effc8169cd0 con 0x7effc80d0490 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effc815efc0 0x7effc8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effc815efc0 0x7effc8150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52482/0 (socket says 172.21.15.97:52482) 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 -- 172.21.15.97:0/1501096776 learned_addr learned my addr 172.21.15.97:0/1501096776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effc7e00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc80cdcb0 0x7effc80b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effbf400640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7effc80d0490 0x7effc80b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 -- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7effc80d0490 msgr2=0x7effc80b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7effc80d0490 0x7effc80b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 -- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc80cdcb0 msgr2=0x7effc80b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc80cdcb0 0x7effc80b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 -- 172.21.15.97:0/1501096776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc80b4dd0 con 0x7effc815efc0 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effc7e00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc80cdcb0 0x7effc80b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcce00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effc815efc0 0x7effc8150740 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7effc000c730 tx=0x7effc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effc4c00640 1 -- 172.21.15.97:0/1501096776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0003db0 con 0x7effc815efc0 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effc4c00640 1 -- 172.21.15.97:0/1501096776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effc000ce40 con 0x7effc815efc0 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc80b5050 con 0x7effc815efc0 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effc4c00640 1 -- 172.21.15.97:0/1501096776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc00052d0 con 0x7effc815efc0 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc80b5610 con 0x7effc815efc0 2024-09-10T21:35:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc80ae930 con 0x7effc815efc0 2024-09-10T21:35:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.549+0000 7effc4c00640 1 -- 172.21.15.97:0/1501096776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7effc0005470 con 0x7effc815efc0 2024-09-10T21:35:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.553+0000 7effc4c00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7effa40725e0 0x7effa4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.553+0000 7effc4c00640 1 -- 172.21.15.97:0/1501096776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6101+0+0 (secure 0 0 0) 0x7effc00930e0 con 0x7effc815efc0 2024-09-10T21:35:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.553+0000 7effc7e00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7effa40725e0 0x7effa4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.553+0000 7effc7e00640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7effa40725e0 0x7effa4074aa0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7effb80af000 tx=0x7effb801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.557+0000 7effc4c00640 1 -- 172.21.15.97:0/1501096776 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7effc80b4dd0 con 0x7effc815efc0 2024-09-10T21:35:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:44.761+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7effc81578c0 con 0x7effc815efc0 2024-09-10T21:35:45.650 INFO:tasks.ceph.mds.d.smithi194.stderr:2024-09-10T21:35:45.649+0000 7f5747000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:45.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.697+0000 7effc4c00640 1 -- 172.21.15.97:0/1501096776 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v343) ==== 118+0+0 (secure 0 0 0) 0x7effc005dfd0 con 0x7effc815efc0 2024-09-10T21:35:45.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7effa40725e0 msgr2=0x7effa4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:45.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7effa40725e0 0x7effa4074aa0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7effb80af000 tx=0x7effb801b060 comp rx=0 tx=0).stop 2024-09-10T21:35:45.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effc815efc0 msgr2=0x7effc8150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:45.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effc815efc0 0x7effc8150740 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7effc000c730 tx=0x7effc000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:35:45.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 shutdown_connections 2024-09-10T21:35:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7effa40725e0 0x7effa4074aa0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effc815efc0 0x7effc8150740 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7effc80d0490 0x7effc80b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 --2- 172.21.15.97:0/1501096776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7effc80cdcb0 0x7effc80b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 >> 172.21.15.97:0/1501096776 conn(0x7effc80d1a60 msgr2=0x7effc80b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 shutdown_connections 2024-09-10T21:35:45.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.701+0000 7effcd800640 1 -- 172.21.15.97:0/1501096776 wait complete. 2024-09-10T21:35:45.722 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:35:45.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:35:45.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- 172.21.15.97:0/2927172226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 msgr2=0x7f3af400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:45.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 --2- 172.21.15.97:0/2927172226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 0x7f3af400cae0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800c640 tx=0x7f3b0801d240 comp rx=0 tx=0).stop 2024-09-10T21:35:45.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b0f400640 1 -- 172.21.15.97:0/2927172226 reap_dead start 2024-09-10T21:35:45.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- 172.21.15.97:0/2927172226 shutdown_connections 2024-09-10T21:35:45.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- 172.21.15.97:0/2927172226 >> 172.21.15.97:0/2927172226 conn(0x7f3b100b1780 msgr2=0x7f3b100b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:45.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- 172.21.15.97:0/2927172226 shutdown_connections 2024-09-10T21:35:45.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- 172.21.15.97:0/2927172226 wait complete. 2024-09-10T21:35:45.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 Processor -- start 2024-09-10T21:35:45.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- start start 2024-09-10T21:35:45.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 0x7f3b101f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:45.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b10143530 0x7f3b101439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:45.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b10146af0 0x7f3b10146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:45.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1006f060 con 0x7f3af4005f80 2024-09-10T21:35:45.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1015b270 con 0x7f3b10146af0 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b15e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1006f1e0 con 0x7f3b10143530 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b0ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 0x7f3b101f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b0ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 0x7f3b101f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52492/0 (socket says 172.21.15.97:52492) 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b0ea00640 1 -- 172.21.15.97:0/1460861520 learned_addr learned my addr 172.21.15.97:0/1460861520 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b07e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b10143530 0x7f3b101439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.873+0000 7f3b0f400640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b10146af0 0x7f3b10146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b0ea00640 1 -- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b10143530 msgr2=0x7f3b101439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b0ea00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b10143530 0x7f3b101439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b0ea00640 1 -- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b10146af0 msgr2=0x7f3b10146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b0ea00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b10146af0 0x7f3b10146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b0ea00640 1 -- 172.21.15.97:0/1460861520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b10144030 con 0x7f3af4005f80 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b0f400640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b10146af0 0x7f3b10146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:45.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b0ea00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 0x7f3b101f63f0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f3b00004f10 tx=0x7f3b00001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:45.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b07400640 1 -- 172.21.15.97:0/1460861520 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00015020 con 0x7f3af4005f80 2024-09-10T21:35:45.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b07400640 1 -- 172.21.15.97:0/1460861520 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b00011960 con 0x7f3af4005f80 2024-09-10T21:35:45.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b07400640 1 -- 172.21.15.97:0/1460861520 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00011d90 con 0x7f3af4005f80 2024-09-10T21:35:45.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b10144310 con 0x7f3af4005f80 2024-09-10T21:35:45.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b101f5ed0 con 0x7f3af4005f80 2024-09-10T21:35:45.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b101f59e0 con 0x7f3af4005f80 2024-09-10T21:35:45.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b07400640 1 -- 172.21.15.97:0/1460861520 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3b0001a400 con 0x7f3af4005f80 2024-09-10T21:35:45.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b07400640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3aec072610 0x7f3aec074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:45.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.877+0000 7f3b07400640 1 -- 172.21.15.97:0/1460861520 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f3b000941b0 con 0x7f3af4005f80 2024-09-10T21:35:45.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.881+0000 7f3b07e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3aec072610 0x7f3aec074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:45.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.881+0000 7f3b07e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3aec072610 0x7f3aec074ad0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b03f0 tx=0x7f3af800d410 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:45.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:45.885+0000 7f3b07400640 1 -- 172.21.15.97:0/1460861520 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3b0001f020 con 0x7f3af4005f80 2024-09-10T21:35:46.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.085+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3b100cfc10 con 0x7f3af4005f80 2024-09-10T21:35:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b07400640 1 -- 172.21.15.97:0/1460861520 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v344) ==== 122+0+0 (secure 0 0 0) 0x7f3b0005f660 con 0x7f3af4005f80 2024-09-10T21:35:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3aec072610 msgr2=0x7f3aec074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3aec072610 0x7f3aec074ad0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b03f0 tx=0x7f3af800d410 comp rx=0 tx=0).stop 2024-09-10T21:35:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 msgr2=0x7f3b101f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 0x7f3b101f63f0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f3b00004f10 tx=0x7f3b00001640 comp rx=0 tx=0).stop 2024-09-10T21:35:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 shutdown_connections 2024-09-10T21:35:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f3aec072610 0x7f3aec074ad0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b10146af0 0x7f3b10146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3b10143530 0x7f3b101439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 --2- 172.21.15.97:0/1460861520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3af4005f80 0x7f3b101f63f0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 >> 172.21.15.97:0/1460861520 conn(0x7f3b100b1780 msgr2=0x7f3b100b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 shutdown_connections 2024-09-10T21:35:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.705+0000 7f3b15e00640 1 -- 172.21.15.97:0/1460861520 wait complete. 2024-09-10T21:35:46.728 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:35:46.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:46.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 -- 172.21.15.97:0/442144799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 msgr2=0x7f5fc80d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 --2- 172.21.15.97:0/442144799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 0x7f5fc80d0370 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009900 tx=0x7f5fc001d2b0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 -- 172.21.15.97:0/442144799 shutdown_connections 2024-09-10T21:35:46.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 --2- 172.21.15.97:0/442144799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 0x7f5fc80d0370 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 --2- 172.21.15.97:0/442144799 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5fc806dc50 0x7f5fc80cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 -- 172.21.15.97:0/442144799 >> 172.21.15.97:0/442144799 conn(0x7f5fc80d1c30 msgr2=0x7f5fc80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:46.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 -- 172.21.15.97:0/442144799 shutdown_connections 2024-09-10T21:35:46.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.869+0000 7f5fcd000640 1 -- 172.21.15.97:0/442144799 wait complete. 2024-09-10T21:35:46.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 Processor -- start 2024-09-10T21:35:46.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 -- start start 2024-09-10T21:35:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 0x7f5fc8152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5fc8161600 0x7f5fc815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc81fae60 0x7f5fc81517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc816aea0 con 0x7f5fc81fae60 2024-09-10T21:35:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc816ad20 con 0x7f5fc80b1590 2024-09-10T21:35:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc816b020 con 0x7f5fc8161600 2024-09-10T21:35:46.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc81fae60 0x7f5fc81517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc81fae60 0x7f5fc81517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52504/0 (socket says 172.21.15.97:52504) 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 -- 172.21.15.97:0/3708229016 learned_addr learned my addr 172.21.15.97:0/3708229016 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc5e00640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 0x7f5fc8152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fbd400640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5fc8161600 0x7f5fc815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 -- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5fc8161600 msgr2=0x7f5fc815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5fc8161600 0x7f5fc815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 -- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 msgr2=0x7f5fc8152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 0x7f5fc8152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 -- 172.21.15.97:0/3708229016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc81fb340 con 0x7f5fc81fae60 2024-09-10T21:35:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fc6800640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc81fae60 0x7f5fc81517e0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00b0040 tx=0x7f5fc0005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fbea00640 1 -- 172.21.15.97:0/3708229016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00b2040 con 0x7f5fc81fae60 2024-09-10T21:35:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fbea00640 1 -- 172.21.15.97:0/3708229016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fc0004240 con 0x7f5fc81fae60 2024-09-10T21:35:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fbea00640 1 -- 172.21.15.97:0/3708229016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0005030 con 0x7f5fc81fae60 2024-09-10T21:35:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc81fb620 con 0x7f5fc81fae60 2024-09-10T21:35:46.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.873+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc81fbbe0 con 0x7f5fc81fae60 2024-09-10T21:35:46.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.877+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc80d0a60 con 0x7f5fc81fae60 2024-09-10T21:35:46.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.877+0000 7f5fbea00640 1 -- 172.21.15.97:0/3708229016 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5fc00051d0 con 0x7f5fc81fae60 2024-09-10T21:35:46.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.877+0000 7f5fbea00640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fa4072610 0x7f5fa4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:46.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.877+0000 7f5fbea00640 1 -- 172.21.15.97:0/3708229016 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f5fc012cc70 con 0x7f5fc81fae60 2024-09-10T21:35:46.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.877+0000 7f5fc5e00640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fa4072610 0x7f5fa4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:46.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.877+0000 7f5fc5e00640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fa4072610 0x7f5fa4074ad0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8157940 tx=0x7f5fb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:46.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:46.881+0000 7f5fbea00640 1 -- 172.21.15.97:0/3708229016 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5fc00164d0 con 0x7f5fc81fae60 2024-09-10T21:35:47.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fc80d2500 con 0x7f5fc81fae60 2024-09-10T21:35:47.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fbea00640 1 -- 172.21.15.97:0/3708229016 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 344 v344) ==== 95+0+5241 (secure 0 0 0) 0x7f5fc00f7ad0 con 0x7f5fc81fae60 2024-09-10T21:35:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:47.081 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":344,"btime":"2024-09-10T21:35:46:701370+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":10629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339}],"filesystems":[{"mdsmap":{"epoch":344,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:35:46.088263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10647},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10647":{"gid":10647,"name":"d","rank":0,"incarnation":340,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6837/3945561983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3945561983},{"type":"v1","addr":"172.21.15.194:6837","nonce":3945561983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10647,"qdb_cluster":[10647]},"id":7}]} 2024-09-10T21:35:47.081 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 344 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fa4072610 msgr2=0x7f5fa4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fa4072610 0x7f5fa4074ad0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8157940 tx=0x7f5fb8009040 comp rx=0 tx=0).stop 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc81fae60 msgr2=0x7f5fc81517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc81fae60 0x7f5fc81517e0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00b0040 tx=0x7f5fc0005df0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 shutdown_connections 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5fa4072610 0x7f5fa4074ad0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc81fae60 0x7f5fc81517e0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5fc8161600 0x7f5fc815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 --2- 172.21.15.97:0/3708229016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fc80b1590 0x7f5fc8152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 >> 172.21.15.97:0/3708229016 conn(0x7f5fc80d1c30 msgr2=0x7f5fc8070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 shutdown_connections 2024-09-10T21:35:47.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.081+0000 7f5fcd000640 1 -- 172.21.15.97:0/3708229016 wait complete. 2024-09-10T21:35:47.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:35:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- 172.21.15.97:0/353421849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 msgr2=0x7fcc740b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/353421849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc740b37b0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009840 tx=0x7fcc6401c700 comp rx=0 tx=0).stop 2024-09-10T21:35:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- 172.21.15.97:0/353421849 shutdown_connections 2024-09-10T21:35:47.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/353421849 >> v2:172.21.15.194:3300/0 conn(0x7fcc740b3cf0 0x7fcc7406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/353421849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc740b37b0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- 172.21.15.97:0/353421849 >> 172.21.15.97:0/353421849 conn(0x7fcc740b1780 msgr2=0x7fcc740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- 172.21.15.97:0/353421849 shutdown_connections 2024-09-10T21:35:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- 172.21.15.97:0/353421849 wait complete. 2024-09-10T21:35:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 Processor -- start 2024-09-10T21:35:47.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- start start 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc7406f180 0x7fcc741e8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc740b3cf0 0x7fcc741f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc741414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7415aac0 con 0x7fcc741511f0 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7415a940 con 0x7fcc740b3cf0 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc7aa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7415ac40 con 0x7fcc7406f180 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc78a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc741414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc78a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc741414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52522/0 (socket says 172.21.15.97:52522) 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc78a00640 1 -- 172.21.15.97:0/3378418572 learned_addr learned my addr 172.21.15.97:0/3378418572 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc6b400640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc740b3cf0 0x7fcc741f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.245+0000 7fcc73e00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc7406f180 0x7fcc741e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc78a00640 1 -- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc7406f180 msgr2=0x7fcc741e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc78a00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc7406f180 0x7fcc741e8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc78a00640 1 -- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc740b3cf0 msgr2=0x7fcc741f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc78a00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc740b3cf0 0x7fcc741f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc78a00640 1 -- 172.21.15.97:0/3378418572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc741f8590 con 0x7fcc741511f0 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc78a00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc741414d0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00fd40 tx=0x7fcc6c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc70c00640 1 -- 172.21.15.97:0/3378418572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c01b070 con 0x7fcc741511f0 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc70c00640 1 -- 172.21.15.97:0/3378418572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc6c006b90 con 0x7fcc741511f0 2024-09-10T21:35:47.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc70c00640 1 -- 172.21.15.97:0/3378418572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0080d0 con 0x7fcc741511f0 2024-09-10T21:35:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc741f8870 con 0x7fcc741511f0 2024-09-10T21:35:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc741e4c60 con 0x7fcc741511f0 2024-09-10T21:35:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc741f8d90 con 0x7fcc741511f0 2024-09-10T21:35:47.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.249+0000 7fcc70c00640 1 -- 172.21.15.97:0/3378418572 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcc6c008270 con 0x7fcc741511f0 2024-09-10T21:35:47.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.253+0000 7fcc70c00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc500725e0 0x7fcc50074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.253+0000 7fcc70c00640 1 -- 172.21.15.97:0/3378418572 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fcc6c0961e0 con 0x7fcc741511f0 2024-09-10T21:35:47.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.253+0000 7fcc73e00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc500725e0 0x7fcc50074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.253+0000 7fcc73e00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc500725e0 0x7fcc50074aa0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcc640af000 tx=0x7fcc6401af40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.257+0000 7fcc70c00640 1 -- 172.21.15.97:0/3378418572 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc741f8d90 con 0x7fcc741511f0 2024-09-10T21:35:47.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.421+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc7415b0f0 con 0x7fcc741511f0 2024-09-10T21:35:47.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc70c00640 1 -- 172.21.15.97:0/3378418572 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16984 (secure 0 0 0) 0x7fcc6c061170 con 0x7fcc741511f0 2024-09-10T21:35:47.424 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:47.424 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":146,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:44.684955+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:35:40.751434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T21:35:42.030160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:35:47.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc500725e0 msgr2=0x7fcc50074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc500725e0 0x7fcc50074aa0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcc640af000 tx=0x7fcc6401af40 comp rx=0 tx=0).stop 2024-09-10T21:35:47.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 msgr2=0x7fcc741414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc741414d0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00fd40 tx=0x7fcc6c00d430 comp rx=0 tx=0).stop 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 shutdown_connections 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fcc500725e0 0x7fcc50074aa0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc741511f0 0x7fcc741414d0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc740b3cf0 0x7fcc741f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 --2- 172.21.15.97:0/3378418572 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fcc7406f180 0x7fcc741e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 >> 172.21.15.97:0/3378418572 conn(0x7fcc740b1780 msgr2=0x7fcc741507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 shutdown_connections 2024-09-10T21:35:47.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.425+0000 7fcc7aa00640 1 -- 172.21.15.97:0/3378418572 wait complete. 2024-09-10T21:35:47.447 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:35:47.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:35:47.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- 172.21.15.97:0/933759647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0144690 msgr2=0x7f5ec00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/933759647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0144690 0x7f5ec00b2a80 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009840 tx=0x7f5ea801c820 comp rx=0 tx=0).stop 2024-09-10T21:35:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- 172.21.15.97:0/933759647 shutdown_connections 2024-09-10T21:35:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/933759647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0144690 0x7f5ec00b2a80 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- 172.21.15.97:0/933759647 >> 172.21.15.97:0/933759647 conn(0x7f5ec00b1780 msgr2=0x7f5ec00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- 172.21.15.97:0/933759647 shutdown_connections 2024-09-10T21:35:47.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- 172.21.15.97:0/933759647 wait complete. 2024-09-10T21:35:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 Processor -- start 2024-09-10T21:35:47.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- start start 2024-09-10T21:35:47.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ec006fd50 0x7f5ec0136420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0143f90 0x7f5ec00b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ec0144690 0x7f5ec013e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec0169410 con 0x7f5ec0143f90 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec0169290 con 0x7f5ec006fd50 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5ec7a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec0169590 con 0x7f5ec0144690 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.589+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0143f90 0x7f5ec00b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0143f90 0x7f5ec00b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44998/0 (socket says 172.21.15.97:44998) 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 -- 172.21.15.97:0/1411625882 learned_addr learned my addr 172.21.15.97:0/1411625882 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ec5000640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ec006fd50 0x7f5ec0136420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ec5a00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ec0144690 0x7f5ec013e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 -- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ec0144690 msgr2=0x7f5ec013e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ec0144690 0x7f5ec013e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 -- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ec006fd50 msgr2=0x7f5ec0136420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ec006fd50 0x7f5ec0136420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 -- 172.21.15.97:0/1411625882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec00af5d0 con 0x7f5ec0143f90 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5eb7e00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0143f90 0x7f5ec00b0ec0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009790 tx=0x7f5ea80a6780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ebd600640 1 -- 172.21.15.97:0/1411625882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004140 con 0x7f5ec0143f90 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ebd600640 1 -- 172.21.15.97:0/1411625882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ea80043d0 con 0x7f5ec0143f90 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ebd600640 1 -- 172.21.15.97:0/1411625882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003600 con 0x7f5ec0143f90 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec01fb690 con 0x7f5ec0143f90 2024-09-10T21:35:47.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec01fbc50 con 0x7f5ec0143f90 2024-09-10T21:35:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec00af7a0 con 0x7f5ec0143f90 2024-09-10T21:35:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.593+0000 7f5ebd600640 1 -- 172.21.15.97:0/1411625882 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5ea80037a0 con 0x7f5ec0143f90 2024-09-10T21:35:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.597+0000 7f5ebd600640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e9c0725e0 0x7f5e9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.597+0000 7f5ebd600640 1 -- 172.21.15.97:0/1411625882 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f5ea812daa0 con 0x7f5ec0143f90 2024-09-10T21:35:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.597+0000 7f5ec5000640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e9c0725e0 0x7f5e9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.597+0000 7f5ec5000640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e9c0725e0 0x7f5e9c074aa0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00016f0 tx=0x7f5eb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.601+0000 7f5ebd600640 1 -- 172.21.15.97:0/1411625882 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5ea8016410 con 0x7f5ec0143f90 2024-09-10T21:35:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.765+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ec013f9d0 con 0x7f5e9c0725e0 2024-09-10T21:35:47.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.769+0000 7f5ebd600640 1 -- 172.21.15.97:0/1411625882 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238736 (secure 0 0 0) 0x7f5ec013f9d0 con 0x7f5e9c0725e0 2024-09-10T21:35:47.772 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:47.773 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:35:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e9c0725e0 msgr2=0x7f5e9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e9c0725e0 0x7f5e9c074aa0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00016f0 tx=0x7f5eb0009040 comp rx=0 tx=0).stop 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0143f90 msgr2=0x7f5ec00b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0143f90 0x7f5ec00b0ec0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009790 tx=0x7f5ea80a6780 comp rx=0 tx=0).stop 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 shutdown_connections 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5e9c0725e0 0x7f5e9c074aa0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5ec0144690 0x7f5ec013e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ec0143f90 0x7f5ec00b0ec0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 --2- 172.21.15.97:0/1411625882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ec006fd50 0x7f5ec0136420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 >> 172.21.15.97:0/1411625882 conn(0x7f5ec00b1780 msgr2=0x7f5ec00ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 shutdown_connections 2024-09-10T21:35:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.773+0000 7f5ec7a00640 1 -- 172.21.15.97:0/1411625882 wait complete. 2024-09-10T21:35:47.792 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-10T21:35:46.249315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":38,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":281692,"kb_used_data":15520,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754693028,"statfs":{"total":773094113280,"available":772805660672,"internally_reserved":0,"allocated":15892480,"data_stored":8601540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.584364"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683461+0000","last_change":"2024-09-10T21:35:42.683461+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683461+0000","last_undegraded":"2024-09-10T21:35:42.683461+0000","last_fullsized":"2024-09-10T21:35:42.683461+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683460+0000","last_change":"2024-09-10T21:35:42.683460+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683460+0000","last_undegraded":"2024-09-10T21:35:42.683460+0000","last_fullsized":"2024-09-10T21:35:42.683460+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682808+0000","last_change":"2024-09-10T21:35:42.682808+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682808+0000","last_undegraded":"2024-09-10T21:35:42.682808+0000","last_fullsized":"2024-09-10T21:35:42.682808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683255+0000","last_change":"2024-09-10T21:35:42.683255+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683255+0000","last_undegraded":"2024-09-10T21:35:42.683255+0000","last_fullsized":"2024-09-10T21:35:42.683255+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682503+0000","last_change":"2024-09-10T21:35:42.682503+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682503+0000","last_undegraded":"2024-09-10T21:35:42.682503+0000","last_fullsized":"2024-09-10T21:35:42.682503+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681124+0000","last_change":"2024-09-10T21:35:42.681124+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681124+0000","last_undegraded":"2024-09-10T21:35:42.681124+0000","last_fullsized":"2024-09-10T21:35:42.681124+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681555+0000","last_change":"2024-09-10T21:35:42.681555+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681555+0000","last_undegraded":"2024-09-10T21:35:42.681555+0000","last_fullsized":"2024-09-10T21:35:42.681555+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681498+0000","last_change":"2024-09-10T21:35:42.681498+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681498+0000","last_undegraded":"2024-09-10T21:35:42.681498+0000","last_fullsized":"2024-09-10T21:35:42.681498+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680925+0000","last_change":"2024-09-10T21:35:42.680925+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680925+0000","last_undegraded":"2024-09-10T21:35:42.680925+0000","last_fullsized":"2024-09-10T21:35:42.680925+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680542+0000","last_change":"2024-09-10T21:35:42.680542+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680542+0000","last_undegraded":"2024-09-10T21:35:42.680542+0000","last_fullsized":"2024-09-10T21:35:42.680542+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701846+0000","last_change":"2024-09-10T21:35:42.693092+0000","last_active":"2024-09-10T21:35:42.701846+0000","last_peered":"2024-09-10T21:35:42.701846+0000","last_clean":"2024-09-10T21:35:42.701846+0000","last_became_active":"2024-09-10T21:35:42.692798+0000","last_became_peered":"2024-09-10T21:35:42.692798+0000","last_unstale":"2024-09-10T21:35:42.701846+0000","last_undegraded":"2024-09-10T21:35:42.701846+0000","last_fullsized":"2024-09-10T21:35:42.701846+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:18.074505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.692613+0000","last_change":"2024-09-10T21:35:42.689212+0000","last_active":"2024-09-10T21:35:42.692613+0000","last_peered":"2024-09-10T21:35:42.692613+0000","last_clean":"2024-09-10T21:35:42.692613+0000","last_became_active":"2024-09-10T21:35:42.688903+0000","last_became_peered":"2024-09-10T21:35:42.688903+0000","last_unstale":"2024-09-10T21:35:42.692613+0000","last_undegraded":"2024-09-10T21:35:42.692613+0000","last_fullsized":"2024-09-10T21:35:42.692613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:01.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682184+0000","last_change":"2024-09-10T21:35:42.682184+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682184+0000","last_undegraded":"2024-09-10T21:35:42.682184+0000","last_fullsized":"2024-09-10T21:35:42.682184+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681938+0000","last_change":"2024-09-10T21:35:42.681938+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681938+0000","last_undegraded":"2024-09-10T21:35:42.681938+0000","last_fullsized":"2024-09-10T21:35:42.681938+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701896+0000","last_change":"2024-09-10T21:35:42.692774+0000","last_active":"2024-09-10T21:35:42.701896+0000","last_peered":"2024-09-10T21:35:42.701896+0000","last_clean":"2024-09-10T21:35:42.701896+0000","last_became_active":"2024-09-10T21:35:42.692448+0000","last_became_peered":"2024-09-10T21:35:42.692448+0000","last_unstale":"2024-09-10T21:35:42.701896+0000","last_undegraded":"2024-09-10T21:35:42.701896+0000","last_fullsized":"2024-09-10T21:35:42.701896+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:36.661466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683005+0000","last_change":"2024-09-10T21:35:42.683005+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683005+0000","last_undegraded":"2024-09-10T21:35:42.683005+0000","last_fullsized":"2024-09-10T21:35:42.683005+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682047+0000","last_change":"2024-09-10T21:35:42.682047+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682047+0000","last_undegraded":"2024-09-10T21:35:42.682047+0000","last_fullsized":"2024-09-10T21:35:42.682047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701861+0000","last_change":"2024-09-10T21:35:42.695716+0000","last_active":"2024-09-10T21:35:42.701861+0000","last_peered":"2024-09-10T21:35:42.701861+0000","last_clean":"2024-09-10T21:35:42.701861+0000","last_became_active":"2024-09-10T21:35:42.695410+0000","last_became_peered":"2024-09-10T21:35:42.695410+0000","last_unstale":"2024-09-10T21:35:42.701861+0000","last_undegraded":"2024-09-10T21:35:42.701861+0000","last_fullsized":"2024-09-10T21:35:42.701861+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:13.034229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":344,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T21:35:40.438141+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:35:40.438141+0000","last_peered":"2024-09-10T21:35:40.438141+0000","last_clean":"2024-09-10T21:35:40.438141+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:35:40.438141+0000","last_undegraded":"2024-09-10T21:35:40.438141+0000","last_fullsized":"2024-09-10T21:35:40.438141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":1},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37152,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28820,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343020,"statfs":{"total":96636764160,"available":96607252480,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33560,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338280,"statfs":{"total":96636764160,"available":96602398720,"internally_reserved":0,"allocated":2428928,"data_stored":1516359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41308,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330532,"statfs":{"total":96636764160,"available":96594464768,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32980,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37724,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334116,"statfs":{"total":96636764160,"available":96598134784,"internally_reserved":0,"allocated":2428928,"data_stored":1516359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37144,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":938867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:35:47.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- 172.21.15.97:0/3866071277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798144690 msgr2=0x7f37980b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 --2- 172.21.15.97:0/3866071277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798144690 0x7f37980b4120 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f379815f790 tx=0x7f378001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- 172.21.15.97:0/3866071277 shutdown_connections 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 --2- 172.21.15.97:0/3866071277 >> v2:172.21.15.194:3300/0 conn(0x7f37980b4660 0x7f379806fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 --2- 172.21.15.97:0/3866071277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798144690 0x7f37980b4120 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 --2- 172.21.15.97:0/3866071277 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f3798143f90 0x7f37980b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- 172.21.15.97:0/3866071277 >> 172.21.15.97:0/3866071277 conn(0x7f37980b1780 msgr2=0x7f37980b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- 172.21.15.97:0/3866071277 shutdown_connections 2024-09-10T21:35:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- 172.21.15.97:0/3866071277 wait complete. 2024-09-10T21:35:47.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 Processor -- start 2024-09-10T21:35:47.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- start start 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f37980b4660 0x7f37980cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798143f90 0x7f379813aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3798144690 0x7f3798159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f379815ede0 con 0x7f3798143f90 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f379815ec60 con 0x7f3798144690 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379f600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f379815ef60 con 0x7f37980b4660 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798143f90 0x7f379813aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798143f90 0x7f379813aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45008/0 (socket says 172.21.15.97:45008) 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 -- 172.21.15.97:0/3768314726 learned_addr learned my addr 172.21.15.97:0/3768314726 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379cc00640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f37980b4660 0x7f37980cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f379d600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3798144690 0x7f3798159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 -- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f37980b4660 msgr2=0x7f37980cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f37980b4660 0x7f37980cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 -- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3798144690 msgr2=0x7f3798159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3798144690 0x7f3798159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 -- 172.21.15.97:0/3768314726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3798139c40 con 0x7f3798143f90 2024-09-10T21:35:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.977+0000 7f378fe00640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798143f90 0x7f379813aa00 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f37800b0040 tx=0x7f37800a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f3795600640 1 -- 172.21.15.97:0/3768314726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800b2040 con 0x7f3798143f90 2024-09-10T21:35:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f3795600640 1 -- 172.21.15.97:0/3768314726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37800043b0 con 0x7f3798143f90 2024-09-10T21:35:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3798139f20 con 0x7f3798143f90 2024-09-10T21:35:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f3795600640 1 -- 172.21.15.97:0/3768314726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780003860 con 0x7f3798143f90 2024-09-10T21:35:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379813a4e0 con 0x7f3798143f90 2024-09-10T21:35:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37980b2a80 con 0x7f3798143f90 2024-09-10T21:35:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f3795600640 1 -- 172.21.15.97:0/3768314726 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3780003a00 con 0x7f3798143f90 2024-09-10T21:35:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f3795600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37740725e0 0x7f3774074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f3795600640 1 -- 172.21.15.97:0/3768314726 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f378012d150 con 0x7f3798143f90 2024-09-10T21:35:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.981+0000 7f379cc00640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37740725e0 0x7f3774074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.985+0000 7f379cc00640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37740725e0 0x7f3774074aa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3788009990 tx=0x7f3788009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:47.985+0000 7f3795600640 1 -- 172.21.15.97:0/3768314726 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f37800bd060 con 0x7f3798143f90 2024-09-10T21:35:48.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.153+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37980cc550 con 0x7f37740725e0 2024-09-10T21:35:48.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.157+0000 7f3795600640 1 -- 172.21.15.97:0/3768314726 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238736 (secure 0 0 0) 0x7f37980cc550 con 0x7f37740725e0 2024-09-10T21:35:48.158 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:48.160 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37740725e0 msgr2=0x7f3774074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37740725e0 0x7f3774074aa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3788009990 tx=0x7f3788009110 comp rx=0 tx=0).stop 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798143f90 msgr2=0x7f379813aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798143f90 0x7f379813aa00 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f37800b0040 tx=0x7f37800a55c0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 shutdown_connections 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f37740725e0 0x7f3774074aa0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3798144690 0x7f3798159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3798143f90 0x7f379813aa00 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 --2- 172.21.15.97:0/3768314726 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f37980b4660 0x7f37980cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 >> 172.21.15.97:0/3768314726 conn(0x7f37980b1780 msgr2=0x7f37980730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:48.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 shutdown_connections 2024-09-10T21:35:48.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.161+0000 7f379f600640 1 -- 172.21.15.97:0/3768314726 wait complete. 2024-09-10T21:35:48.178 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-10T21:35:46.249315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":38,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":281692,"kb_used_data":15520,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754693028,"statfs":{"total":773094113280,"available":772805660672,"internally_reserved":0,"allocated":15892480,"data_stored":8601540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.584364"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683461+0000","last_change":"2024-09-10T21:35:42.683461+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683461+0000","last_undegraded":"2024-09-10T21:35:42.683461+0000","last_fullsized":"2024-09-10T21:35:42.683461+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683460+0000","last_change":"2024-09-10T21:35:42.683460+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683460+0000","last_undegraded":"2024-09-10T21:35:42.683460+0000","last_fullsized":"2024-09-10T21:35:42.683460+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682808+0000","last_change":"2024-09-10T21:35:42.682808+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682808+0000","last_undegraded":"2024-09-10T21:35:42.682808+0000","last_fullsized":"2024-09-10T21:35:42.682808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683255+0000","last_change":"2024-09-10T21:35:42.683255+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683255+0000","last_undegraded":"2024-09-10T21:35:42.683255+0000","last_fullsized":"2024-09-10T21:35:42.683255+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682503+0000","last_change":"2024-09-10T21:35:42.682503+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682503+0000","last_undegraded":"2024-09-10T21:35:42.682503+0000","last_fullsized":"2024-09-10T21:35:42.682503+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681124+0000","last_change":"2024-09-10T21:35:42.681124+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681124+0000","last_undegraded":"2024-09-10T21:35:42.681124+0000","last_fullsized":"2024-09-10T21:35:42.681124+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681555+0000","last_change":"2024-09-10T21:35:42.681555+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681555+0000","last_undegraded":"2024-09-10T21:35:42.681555+0000","last_fullsized":"2024-09-10T21:35:42.681555+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681498+0000","last_change":"2024-09-10T21:35:42.681498+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681498+0000","last_undegraded":"2024-09-10T21:35:42.681498+0000","last_fullsized":"2024-09-10T21:35:42.681498+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680925+0000","last_change":"2024-09-10T21:35:42.680925+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680925+0000","last_undegraded":"2024-09-10T21:35:42.680925+0000","last_fullsized":"2024-09-10T21:35:42.680925+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680542+0000","last_change":"2024-09-10T21:35:42.680542+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680542+0000","last_undegraded":"2024-09-10T21:35:42.680542+0000","last_fullsized":"2024-09-10T21:35:42.680542+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701846+0000","last_change":"2024-09-10T21:35:42.693092+0000","last_active":"2024-09-10T21:35:42.701846+0000","last_peered":"2024-09-10T21:35:42.701846+0000","last_clean":"2024-09-10T21:35:42.701846+0000","last_became_active":"2024-09-10T21:35:42.692798+0000","last_became_peered":"2024-09-10T21:35:42.692798+0000","last_unstale":"2024-09-10T21:35:42.701846+0000","last_undegraded":"2024-09-10T21:35:42.701846+0000","last_fullsized":"2024-09-10T21:35:42.701846+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:18.074505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.692613+0000","last_change":"2024-09-10T21:35:42.689212+0000","last_active":"2024-09-10T21:35:42.692613+0000","last_peered":"2024-09-10T21:35:42.692613+0000","last_clean":"2024-09-10T21:35:42.692613+0000","last_became_active":"2024-09-10T21:35:42.688903+0000","last_became_peered":"2024-09-10T21:35:42.688903+0000","last_unstale":"2024-09-10T21:35:42.692613+0000","last_undegraded":"2024-09-10T21:35:42.692613+0000","last_fullsized":"2024-09-10T21:35:42.692613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:01.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682184+0000","last_change":"2024-09-10T21:35:42.682184+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682184+0000","last_undegraded":"2024-09-10T21:35:42.682184+0000","last_fullsized":"2024-09-10T21:35:42.682184+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681938+0000","last_change":"2024-09-10T21:35:42.681938+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681938+0000","last_undegraded":"2024-09-10T21:35:42.681938+0000","last_fullsized":"2024-09-10T21:35:42.681938+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701896+0000","last_change":"2024-09-10T21:35:42.692774+0000","last_active":"2024-09-10T21:35:42.701896+0000","last_peered":"2024-09-10T21:35:42.701896+0000","last_clean":"2024-09-10T21:35:42.701896+0000","last_became_active":"2024-09-10T21:35:42.692448+0000","last_became_peered":"2024-09-10T21:35:42.692448+0000","last_unstale":"2024-09-10T21:35:42.701896+0000","last_undegraded":"2024-09-10T21:35:42.701896+0000","last_fullsized":"2024-09-10T21:35:42.701896+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:36.661466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683005+0000","last_change":"2024-09-10T21:35:42.683005+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683005+0000","last_undegraded":"2024-09-10T21:35:42.683005+0000","last_fullsized":"2024-09-10T21:35:42.683005+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682047+0000","last_change":"2024-09-10T21:35:42.682047+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682047+0000","last_undegraded":"2024-09-10T21:35:42.682047+0000","last_fullsized":"2024-09-10T21:35:42.682047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:41.657585+0000","last_change":"2024-09-10T21:35:41.657585+0000","last_active":"2024-09-10T21:35:41.657585+0000","last_peered":"2024-09-10T21:35:41.657585+0000","last_clean":"2024-09-10T21:35:41.657585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:41.657585+0000","last_undegraded":"2024-09-10T21:35:41.657585+0000","last_fullsized":"2024-09-10T21:35:41.657585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:35:42.661776+0000","last_change":"2024-09-10T21:35:42.661776+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.661776+0000","last_undegraded":"2024-09-10T21:35:42.661776+0000","last_fullsized":"2024-09-10T21:35:42.661776+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701861+0000","last_change":"2024-09-10T21:35:42.695716+0000","last_active":"2024-09-10T21:35:42.701861+0000","last_peered":"2024-09-10T21:35:42.701861+0000","last_clean":"2024-09-10T21:35:42.701861+0000","last_became_active":"2024-09-10T21:35:42.695410+0000","last_became_peered":"2024-09-10T21:35:42.695410+0000","last_unstale":"2024-09-10T21:35:42.701861+0000","last_undegraded":"2024-09-10T21:35:42.701861+0000","last_fullsized":"2024-09-10T21:35:42.701861+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:13.034229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":344,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-10T21:35:40.438141+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:35:40.438141+0000","last_peered":"2024-09-10T21:35:40.438141+0000","last_clean":"2024-09-10T21:35:40.438141+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:35:40.438141+0000","last_undegraded":"2024-09-10T21:35:40.438141+0000","last_fullsized":"2024-09-10T21:35:40.438141+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":1},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37152,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28820,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343020,"statfs":{"total":96636764160,"available":96607252480,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33560,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338280,"statfs":{"total":96636764160,"available":96602398720,"internally_reserved":0,"allocated":2428928,"data_stored":1516359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41308,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330532,"statfs":{"total":96636764160,"available":96594464768,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32980,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37724,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334116,"statfs":{"total":96636764160,"available":96598134784,"internally_reserved":0,"allocated":2428928,"data_stored":1516359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37144,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":1835008,"data_stored":925991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":938867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:35:48.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:35:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 -- 172.21.15.97:0/2864982806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 msgr2=0x7f0414004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 --2- 172.21.15.97:0/2864982806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 0x7f0414004c90 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f041800f490 tx=0x7f041801e410 comp rx=0 tx=0).stop 2024-09-10T21:35:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f042fe00640 1 -- 172.21.15.97:0/2864982806 reap_dead start 2024-09-10T21:35:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 -- 172.21.15.97:0/2864982806 shutdown_connections 2024-09-10T21:35:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 -- 172.21.15.97:0/2864982806 >> 172.21.15.97:0/2864982806 conn(0x7f04300b1780 msgr2=0x7f04300b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 -- 172.21.15.97:0/2864982806 shutdown_connections 2024-09-10T21:35:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 -- 172.21.15.97:0/2864982806 wait complete. 2024-09-10T21:35:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 Processor -- start 2024-09-10T21:35:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.361+0000 7f0436400640 1 -- start start 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 0x7f043013eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043014c1f0 0x7f0430153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0430154230 0x7f04301546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f043015aac0 con 0x7f0414004810 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f043015a940 con 0x7f043014c1f0 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f043015ac40 con 0x7f0430154230 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 0x7f043013eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 0x7f043013eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45026/0 (socket says 172.21.15.97:45026) 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 -- 172.21.15.97:0/2504004166 learned_addr learned my addr 172.21.15.97:0/2504004166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0426a00640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043014c1f0 0x7f0430153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042fe00640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0430154230 0x7f04301546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 -- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0430154230 msgr2=0x7f04301546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0430154230 0x7f04301546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 -- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043014c1f0 msgr2=0x7f0430153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043014c1f0 0x7f0430153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 -- 172.21.15.97:0/2504004166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043014e710 con 0x7f0414004810 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f042f400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 0x7f043013eb60 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f0420011b10 tx=0x7f0420009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0427e00640 1 -- 172.21.15.97:0/2504004166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042000fd00 con 0x7f0414004810 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0427e00640 1 -- 172.21.15.97:0/2504004166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0420004020 con 0x7f0414004810 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0427e00640 1 -- 172.21.15.97:0/2504004166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042000fea0 con 0x7f0414004810 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043014e9f0 con 0x7f0414004810 2024-09-10T21:35:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043014efb0 con 0x7f0414004810 2024-09-10T21:35:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043015b1d0 con 0x7f0414004810 2024-09-10T21:35:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.365+0000 7f0427e00640 1 -- 172.21.15.97:0/2504004166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0420003600 con 0x7f0414004810 2024-09-10T21:35:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.369+0000 7f0427e00640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f040c0725e0 0x7f040c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.369+0000 7f0427e00640 1 -- 172.21.15.97:0/2504004166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f0420093fb0 con 0x7f0414004810 2024-09-10T21:35:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.369+0000 7f0426a00640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f040c0725e0 0x7f040c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.369+0000 7f0426a00640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f040c0725e0 0x7f040c074aa0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f04180b1000 tx=0x7f041801b1b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:48.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.373+0000 7f0427e00640 1 -- 172.21.15.97:0/2504004166 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0420004470 con 0x7f0414004810 2024-09-10T21:35:48.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.537+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0430138e00 con 0x7f040c0725e0 2024-09-10T21:35:48.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.541+0000 7f0427e00640 1 -- 172.21.15.97:0/2504004166 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251319 (secure 0 0 0) 0x7f04180b5040 con 0x7f040c0725e0 2024-09-10T21:35:48.543 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:48.544 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f040c0725e0 msgr2=0x7f040c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f040c0725e0 0x7f040c074aa0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f04180b1000 tx=0x7f041801b1b0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 msgr2=0x7f043013eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 0x7f043013eb60 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f0420011b10 tx=0x7f0420009f20 comp rx=0 tx=0).stop 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 shutdown_connections 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f040c0725e0 0x7f040c074aa0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0430154230 0x7f04301546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043014c1f0 0x7f0430153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 --2- 172.21.15.97:0/2504004166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0414004810 0x7f043013eb60 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 >> 172.21.15.97:0/2504004166 conn(0x7f04300b1780 msgr2=0x7f043014b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:48.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 shutdown_connections 2024-09-10T21:35:48.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.545+0000 7f0436400640 1 -- 172.21.15.97:0/2504004166 wait complete. 2024-09-10T21:35:48.563 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T21:35:48.249753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":140,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282072,"kb_used_data":15904,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692648,"statfs":{"total":773094113280,"available":772805271552,"internally_reserved":0,"allocated":16285696,"data_stored":8786779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001455"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683461+0000","last_change":"2024-09-10T21:35:42.683461+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683461+0000","last_undegraded":"2024-09-10T21:35:42.683461+0000","last_fullsized":"2024-09-10T21:35:42.683461+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562668+0000","last_change":"2024-09-10T21:35:43.704550+0000","last_active":"2024-09-10T21:35:45.562668+0000","last_peered":"2024-09-10T21:35:45.562668+0000","last_clean":"2024-09-10T21:35:45.562668+0000","last_became_active":"2024-09-10T21:35:43.704086+0000","last_became_peered":"2024-09-10T21:35:43.704086+0000","last_unstale":"2024-09-10T21:35:45.562668+0000","last_undegraded":"2024-09-10T21:35:45.562668+0000","last_fullsized":"2024-09-10T21:35:45.562668+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:14.393495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563678+0000","last_change":"2024-09-10T21:35:43.702123+0000","last_active":"2024-09-10T21:35:45.563678+0000","last_peered":"2024-09-10T21:35:45.563678+0000","last_clean":"2024-09-10T21:35:45.563678+0000","last_became_active":"2024-09-10T21:35:43.700634+0000","last_became_peered":"2024-09-10T21:35:43.700634+0000","last_unstale":"2024-09-10T21:35:45.563678+0000","last_undegraded":"2024-09-10T21:35:45.563678+0000","last_fullsized":"2024-09-10T21:35:45.563678+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:13.113908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683460+0000","last_change":"2024-09-10T21:35:42.683460+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683460+0000","last_undegraded":"2024-09-10T21:35:42.683460+0000","last_fullsized":"2024-09-10T21:35:42.683460+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682808+0000","last_change":"2024-09-10T21:35:42.682808+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682808+0000","last_undegraded":"2024-09-10T21:35:42.682808+0000","last_fullsized":"2024-09-10T21:35:42.682808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692726+0000","last_change":"2024-09-10T21:35:43.710080+0000","last_active":"2024-09-10T21:35:44.692726+0000","last_peered":"2024-09-10T21:35:44.692726+0000","last_clean":"2024-09-10T21:35:44.692726+0000","last_became_active":"2024-09-10T21:35:43.709693+0000","last_became_peered":"2024-09-10T21:35:43.709693+0000","last_unstale":"2024-09-10T21:35:44.692726+0000","last_undegraded":"2024-09-10T21:35:44.692726+0000","last_fullsized":"2024-09-10T21:35:44.692726+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:38.047565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692510+0000","last_change":"2024-09-10T21:35:43.711852+0000","last_active":"2024-09-10T21:35:44.692510+0000","last_peered":"2024-09-10T21:35:44.692510+0000","last_clean":"2024-09-10T21:35:44.692510+0000","last_became_active":"2024-09-10T21:35:43.711654+0000","last_became_peered":"2024-09-10T21:35:43.711654+0000","last_unstale":"2024-09-10T21:35:44.692510+0000","last_undegraded":"2024-09-10T21:35:44.692510+0000","last_fullsized":"2024-09-10T21:35:44.692510+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:12.060947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075008+0000","last_change":"2024-09-10T21:35:43.713260+0000","last_active":"2024-09-10T21:35:45.075008+0000","last_peered":"2024-09-10T21:35:45.075008+0000","last_clean":"2024-09-10T21:35:45.075008+0000","last_became_active":"2024-09-10T21:35:43.712978+0000","last_became_peered":"2024-09-10T21:35:43.712978+0000","last_unstale":"2024-09-10T21:35:45.075008+0000","last_undegraded":"2024-09-10T21:35:45.075008+0000","last_fullsized":"2024-09-10T21:35:45.075008+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:36.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562758+0000","last_change":"2024-09-10T21:35:43.711304+0000","last_active":"2024-09-10T21:35:45.562758+0000","last_peered":"2024-09-10T21:35:45.562758+0000","last_clean":"2024-09-10T21:35:45.562758+0000","last_became_active":"2024-09-10T21:35:43.711066+0000","last_became_peered":"2024-09-10T21:35:43.711066+0000","last_unstale":"2024-09-10T21:35:45.562758+0000","last_undegraded":"2024-09-10T21:35:45.562758+0000","last_fullsized":"2024-09-10T21:35:45.562758+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:32.539648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075220+0000","last_change":"2024-09-10T21:35:43.708804+0000","last_active":"2024-09-10T21:35:45.075220+0000","last_peered":"2024-09-10T21:35:45.075220+0000","last_clean":"2024-09-10T21:35:45.075220+0000","last_became_active":"2024-09-10T21:35:43.708489+0000","last_became_peered":"2024-09-10T21:35:43.708489+0000","last_unstale":"2024-09-10T21:35:45.075220+0000","last_undegraded":"2024-09-10T21:35:45.075220+0000","last_fullsized":"2024-09-10T21:35:45.075220+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:47.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880679+0000","last_change":"2024-09-10T21:35:43.693534+0000","last_active":"2024-09-10T21:35:44.880679+0000","last_peered":"2024-09-10T21:35:44.880679+0000","last_clean":"2024-09-10T21:35:44.880679+0000","last_became_active":"2024-09-10T21:35:43.693285+0000","last_became_peered":"2024-09-10T21:35:43.693285+0000","last_unstale":"2024-09-10T21:35:44.880679+0000","last_undegraded":"2024-09-10T21:35:44.880679+0000","last_fullsized":"2024-09-10T21:35:44.880679+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:18.734510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693052+0000","last_change":"2024-09-10T21:35:43.689168+0000","last_active":"2024-09-10T21:35:44.693052+0000","last_peered":"2024-09-10T21:35:44.693052+0000","last_clean":"2024-09-10T21:35:44.693052+0000","last_became_active":"2024-09-10T21:35:43.688873+0000","last_became_peered":"2024-09-10T21:35:43.688873+0000","last_unstale":"2024-09-10T21:35:44.693052+0000","last_undegraded":"2024-09-10T21:35:44.693052+0000","last_fullsized":"2024-09-10T21:35:44.693052+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:16.639213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683255+0000","last_change":"2024-09-10T21:35:42.683255+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683255+0000","last_undegraded":"2024-09-10T21:35:42.683255+0000","last_fullsized":"2024-09-10T21:35:42.683255+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692849+0000","last_change":"2024-09-10T21:35:43.689434+0000","last_active":"2024-09-10T21:35:44.692849+0000","last_peered":"2024-09-10T21:35:44.692849+0000","last_clean":"2024-09-10T21:35:44.692849+0000","last_became_active":"2024-09-10T21:35:43.689163+0000","last_became_peered":"2024-09-10T21:35:43.689163+0000","last_unstale":"2024-09-10T21:35:44.692849+0000","last_undegraded":"2024-09-10T21:35:44.692849+0000","last_fullsized":"2024-09-10T21:35:44.692849+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:21.413539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562494+0000","last_change":"2024-09-10T21:35:43.698275+0000","last_active":"2024-09-10T21:35:45.562494+0000","last_peered":"2024-09-10T21:35:45.562494+0000","last_clean":"2024-09-10T21:35:45.562494+0000","last_became_active":"2024-09-10T21:35:43.698023+0000","last_became_peered":"2024-09-10T21:35:43.698023+0000","last_unstale":"2024-09-10T21:35:45.562494+0000","last_undegraded":"2024-09-10T21:35:45.562494+0000","last_fullsized":"2024-09-10T21:35:45.562494+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:37.116752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062129+0000","last_change":"2024-09-10T21:35:43.702042+0000","last_active":"2024-09-10T21:35:45.062129+0000","last_peered":"2024-09-10T21:35:45.062129+0000","last_clean":"2024-09-10T21:35:45.062129+0000","last_became_active":"2024-09-10T21:35:43.701794+0000","last_became_peered":"2024-09-10T21:35:43.701794+0000","last_unstale":"2024-09-10T21:35:45.062129+0000","last_undegraded":"2024-09-10T21:35:45.062129+0000","last_fullsized":"2024-09-10T21:35:45.062129+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:42.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563696+0000","last_change":"2024-09-10T21:35:43.706781+0000","last_active":"2024-09-10T21:35:45.563696+0000","last_peered":"2024-09-10T21:35:45.563696+0000","last_clean":"2024-09-10T21:35:45.563696+0000","last_became_active":"2024-09-10T21:35:43.706511+0000","last_became_peered":"2024-09-10T21:35:43.706511+0000","last_unstale":"2024-09-10T21:35:45.563696+0000","last_undegraded":"2024-09-10T21:35:45.563696+0000","last_fullsized":"2024-09-10T21:35:45.563696+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:56.867984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562875+0000","last_change":"2024-09-10T21:35:43.710248+0000","last_active":"2024-09-10T21:35:45.562875+0000","last_peered":"2024-09-10T21:35:45.562875+0000","last_clean":"2024-09-10T21:35:45.562875+0000","last_became_active":"2024-09-10T21:35:43.710000+0000","last_became_peered":"2024-09-10T21:35:43.710000+0000","last_unstale":"2024-09-10T21:35:45.562875+0000","last_undegraded":"2024-09-10T21:35:45.562875+0000","last_fullsized":"2024-09-10T21:35:45.562875+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:24.749734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682503+0000","last_change":"2024-09-10T21:35:42.682503+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682503+0000","last_undegraded":"2024-09-10T21:35:42.682503+0000","last_fullsized":"2024-09-10T21:35:42.682503+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062438+0000","last_change":"2024-09-10T21:35:43.695973+0000","last_active":"2024-09-10T21:35:45.062438+0000","last_peered":"2024-09-10T21:35:45.062438+0000","last_clean":"2024-09-10T21:35:45.062438+0000","last_became_active":"2024-09-10T21:35:43.695709+0000","last_became_peered":"2024-09-10T21:35:43.695709+0000","last_unstale":"2024-09-10T21:35:45.062438+0000","last_undegraded":"2024-09-10T21:35:45.062438+0000","last_fullsized":"2024-09-10T21:35:45.062438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:21.413539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563488+0000","last_change":"2024-09-10T21:35:43.695495+0000","last_active":"2024-09-10T21:35:45.563488+0000","last_peered":"2024-09-10T21:35:45.563488+0000","last_clean":"2024-09-10T21:35:45.563488+0000","last_became_active":"2024-09-10T21:35:43.695095+0000","last_became_peered":"2024-09-10T21:35:43.695095+0000","last_unstale":"2024-09-10T21:35:45.563488+0000","last_undegraded":"2024-09-10T21:35:45.563488+0000","last_fullsized":"2024-09-10T21:35:45.563488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:54.813769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694816+0000","last_change":"2024-09-10T21:35:43.696103+0000","last_active":"2024-09-10T21:35:44.694816+0000","last_peered":"2024-09-10T21:35:44.694816+0000","last_clean":"2024-09-10T21:35:44.694816+0000","last_became_active":"2024-09-10T21:35:43.695633+0000","last_became_peered":"2024-09-10T21:35:43.695633+0000","last_unstale":"2024-09-10T21:35:44.694816+0000","last_undegraded":"2024-09-10T21:35:44.694816+0000","last_fullsized":"2024-09-10T21:35:44.694816+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:56.055893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074781+0000","last_change":"2024-09-10T21:35:43.699096+0000","last_active":"2024-09-10T21:35:45.074781+0000","last_peered":"2024-09-10T21:35:45.074781+0000","last_clean":"2024-09-10T21:35:45.074781+0000","last_became_active":"2024-09-10T21:35:43.698826+0000","last_became_peered":"2024-09-10T21:35:43.698826+0000","last_unstale":"2024-09-10T21:35:45.074781+0000","last_undegraded":"2024-09-10T21:35:45.074781+0000","last_fullsized":"2024-09-10T21:35:45.074781+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:31.736444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074767+0000","last_change":"2024-09-10T21:35:43.710788+0000","last_active":"2024-09-10T21:35:45.074767+0000","last_peered":"2024-09-10T21:35:45.074767+0000","last_clean":"2024-09-10T21:35:45.074767+0000","last_became_active":"2024-09-10T21:35:43.710492+0000","last_became_peered":"2024-09-10T21:35:43.710492+0000","last_unstale":"2024-09-10T21:35:45.074767+0000","last_undegraded":"2024-09-10T21:35:45.074767+0000","last_fullsized":"2024-09-10T21:35:45.074767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:09.641780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062559+0000","last_change":"2024-09-10T21:35:43.695952+0000","last_active":"2024-09-10T21:35:45.062559+0000","last_peered":"2024-09-10T21:35:45.062559+0000","last_clean":"2024-09-10T21:35:45.062559+0000","last_became_active":"2024-09-10T21:35:43.694886+0000","last_became_peered":"2024-09-10T21:35:43.694886+0000","last_unstale":"2024-09-10T21:35:45.062559+0000","last_undegraded":"2024-09-10T21:35:45.062559+0000","last_fullsized":"2024-09-10T21:35:45.062559+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:41.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062558+0000","last_change":"2024-09-10T21:35:43.691902+0000","last_active":"2024-09-10T21:35:45.062558+0000","last_peered":"2024-09-10T21:35:45.062558+0000","last_clean":"2024-09-10T21:35:45.062558+0000","last_became_active":"2024-09-10T21:35:43.691638+0000","last_became_peered":"2024-09-10T21:35:43.691638+0000","last_unstale":"2024-09-10T21:35:45.062558+0000","last_undegraded":"2024-09-10T21:35:45.062558+0000","last_fullsized":"2024-09-10T21:35:45.062558+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:16.639213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062518+0000","last_change":"2024-09-10T21:35:43.696465+0000","last_active":"2024-09-10T21:35:45.062518+0000","last_peered":"2024-09-10T21:35:45.062518+0000","last_clean":"2024-09-10T21:35:45.062518+0000","last_became_active":"2024-09-10T21:35:43.695584+0000","last_became_peered":"2024-09-10T21:35:43.695584+0000","last_unstale":"2024-09-10T21:35:45.062518+0000","last_undegraded":"2024-09-10T21:35:45.062518+0000","last_fullsized":"2024-09-10T21:35:45.062518+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:29.000102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692740+0000","last_change":"2024-09-10T21:35:43.686570+0000","last_active":"2024-09-10T21:35:44.692740+0000","last_peered":"2024-09-10T21:35:44.692740+0000","last_clean":"2024-09-10T21:35:44.692740+0000","last_became_active":"2024-09-10T21:35:43.686287+0000","last_became_peered":"2024-09-10T21:35:43.686287+0000","last_unstale":"2024-09-10T21:35:44.692740+0000","last_undegraded":"2024-09-10T21:35:44.692740+0000","last_fullsized":"2024-09-10T21:35:44.692740+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:49.069960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074985+0000","last_change":"2024-09-10T21:35:43.697251+0000","last_active":"2024-09-10T21:35:45.074985+0000","last_peered":"2024-09-10T21:35:45.074985+0000","last_clean":"2024-09-10T21:35:45.074985+0000","last_became_active":"2024-09-10T21:35:43.696685+0000","last_became_peered":"2024-09-10T21:35:43.696685+0000","last_unstale":"2024-09-10T21:35:45.074985+0000","last_undegraded":"2024-09-10T21:35:45.074985+0000","last_fullsized":"2024-09-10T21:35:45.074985+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:19.847487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880695+0000","last_change":"2024-09-10T21:35:43.695764+0000","last_active":"2024-09-10T21:35:44.880695+0000","last_peered":"2024-09-10T21:35:44.880695+0000","last_clean":"2024-09-10T21:35:44.880695+0000","last_became_active":"2024-09-10T21:35:43.695460+0000","last_became_peered":"2024-09-10T21:35:43.695460+0000","last_unstale":"2024-09-10T21:35:44.880695+0000","last_undegraded":"2024-09-10T21:35:44.880695+0000","last_fullsized":"2024-09-10T21:35:44.880695+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:05.693752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692686+0000","last_change":"2024-09-10T21:35:43.702866+0000","last_active":"2024-09-10T21:35:44.692686+0000","last_peered":"2024-09-10T21:35:44.692686+0000","last_clean":"2024-09-10T21:35:44.692686+0000","last_became_active":"2024-09-10T21:35:43.702624+0000","last_became_peered":"2024-09-10T21:35:43.702624+0000","last_unstale":"2024-09-10T21:35:44.692686+0000","last_undegraded":"2024-09-10T21:35:44.692686+0000","last_fullsized":"2024-09-10T21:35:44.692686+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:42.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694552+0000","last_change":"2024-09-10T21:35:43.691043+0000","last_active":"2024-09-10T21:35:44.694552+0000","last_peered":"2024-09-10T21:35:44.694552+0000","last_clean":"2024-09-10T21:35:44.694552+0000","last_became_active":"2024-09-10T21:35:43.690768+0000","last_became_peered":"2024-09-10T21:35:43.690768+0000","last_unstale":"2024-09-10T21:35:44.694552+0000","last_undegraded":"2024-09-10T21:35:44.694552+0000","last_fullsized":"2024-09-10T21:35:44.694552+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:59.974397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563300+0000","last_change":"2024-09-10T21:35:43.690260+0000","last_active":"2024-09-10T21:35:45.563300+0000","last_peered":"2024-09-10T21:35:45.563300+0000","last_clean":"2024-09-10T21:35:45.563300+0000","last_became_active":"2024-09-10T21:35:43.690001+0000","last_became_peered":"2024-09-10T21:35:43.690001+0000","last_unstale":"2024-09-10T21:35:45.563300+0000","last_undegraded":"2024-09-10T21:35:45.563300+0000","last_fullsized":"2024-09-10T21:35:45.563300+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:11.506883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696123+0000","last_change":"2024-09-10T21:35:42.684988+0000","last_active":"2024-09-10T21:35:44.696123+0000","last_peered":"2024-09-10T21:35:44.696123+0000","last_clean":"2024-09-10T21:35:44.696123+0000","last_became_active":"2024-09-10T21:35:42.684730+0000","last_became_peered":"2024-09-10T21:35:42.684730+0000","last_unstale":"2024-09-10T21:35:44.696123+0000","last_undegraded":"2024-09-10T21:35:44.696123+0000","last_fullsized":"2024-09-10T21:35:44.696123+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:22.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074844+0000","last_change":"2024-09-10T21:35:43.697903+0000","last_active":"2024-09-10T21:35:45.074844+0000","last_peered":"2024-09-10T21:35:45.074844+0000","last_clean":"2024-09-10T21:35:45.074844+0000","last_became_active":"2024-09-10T21:35:43.697647+0000","last_became_peered":"2024-09-10T21:35:43.697647+0000","last_unstale":"2024-09-10T21:35:45.074844+0000","last_undegraded":"2024-09-10T21:35:45.074844+0000","last_fullsized":"2024-09-10T21:35:45.074844+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:02.827939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563603+0000","last_change":"2024-09-10T21:35:42.683390+0000","last_active":"2024-09-10T21:35:45.563603+0000","last_peered":"2024-09-10T21:35:45.563603+0000","last_clean":"2024-09-10T21:35:45.563603+0000","last_became_active":"2024-09-10T21:35:42.683095+0000","last_became_peered":"2024-09-10T21:35:42.683095+0000","last_unstale":"2024-09-10T21:35:45.563603+0000","last_undegraded":"2024-09-10T21:35:45.563603+0000","last_fullsized":"2024-09-10T21:35:45.563603+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:08.275738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562597+0000","last_change":"2024-09-10T21:35:43.704762+0000","last_active":"2024-09-10T21:35:45.562597+0000","last_peered":"2024-09-10T21:35:45.562597+0000","last_clean":"2024-09-10T21:35:45.562597+0000","last_became_active":"2024-09-10T21:35:43.704137+0000","last_became_peered":"2024-09-10T21:35:43.704137+0000","last_unstale":"2024-09-10T21:35:45.562597+0000","last_undegraded":"2024-09-10T21:35:45.562597+0000","last_fullsized":"2024-09-10T21:35:45.562597+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:11.824013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562772+0000","last_change":"2024-09-10T21:35:42.693628+0000","last_active":"2024-09-10T21:35:45.562772+0000","last_peered":"2024-09-10T21:35:45.562772+0000","last_clean":"2024-09-10T21:35:45.562772+0000","last_became_active":"2024-09-10T21:35:42.692980+0000","last_became_peered":"2024-09-10T21:35:42.692980+0000","last_unstale":"2024-09-10T21:35:45.562772+0000","last_undegraded":"2024-09-10T21:35:45.562772+0000","last_fullsized":"2024-09-10T21:35:45.562772+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:25.656976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880707+0000","last_change":"2024-09-10T21:35:43.698119+0000","last_active":"2024-09-10T21:35:44.880707+0000","last_peered":"2024-09-10T21:35:44.880707+0000","last_clean":"2024-09-10T21:35:44.880707+0000","last_became_active":"2024-09-10T21:35:43.697892+0000","last_became_peered":"2024-09-10T21:35:43.697892+0000","last_unstale":"2024-09-10T21:35:44.880707+0000","last_undegraded":"2024-09-10T21:35:44.880707+0000","last_fullsized":"2024-09-10T21:35:44.880707+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:13.854892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074953+0000","last_change":"2024-09-10T21:35:42.697366+0000","last_active":"2024-09-10T21:35:45.074953+0000","last_peered":"2024-09-10T21:35:45.074953+0000","last_clean":"2024-09-10T21:35:45.074953+0000","last_became_active":"2024-09-10T21:35:42.697095+0000","last_became_peered":"2024-09-10T21:35:42.697095+0000","last_unstale":"2024-09-10T21:35:45.074953+0000","last_undegraded":"2024-09-10T21:35:45.074953+0000","last_fullsized":"2024-09-10T21:35:45.074953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:23.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681124+0000","last_change":"2024-09-10T21:35:42.681124+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681124+0000","last_undegraded":"2024-09-10T21:35:42.681124+0000","last_fullsized":"2024-09-10T21:35:42.681124+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880614+0000","last_change":"2024-09-10T21:35:42.695787+0000","last_active":"2024-09-10T21:35:44.880614+0000","last_peered":"2024-09-10T21:35:44.880614+0000","last_clean":"2024-09-10T21:35:44.880614+0000","last_became_active":"2024-09-10T21:35:42.695491+0000","last_became_peered":"2024-09-10T21:35:42.695491+0000","last_unstale":"2024-09-10T21:35:44.880614+0000","last_undegraded":"2024-09-10T21:35:44.880614+0000","last_fullsized":"2024-09-10T21:35:44.880614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:09.005666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681555+0000","last_change":"2024-09-10T21:35:42.681555+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681555+0000","last_undegraded":"2024-09-10T21:35:42.681555+0000","last_fullsized":"2024-09-10T21:35:42.681555+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062193+0000","last_change":"2024-09-10T21:35:42.694006+0000","last_active":"2024-09-10T21:35:45.062193+0000","last_peered":"2024-09-10T21:35:45.062193+0000","last_clean":"2024-09-10T21:35:45.062193+0000","last_became_active":"2024-09-10T21:35:42.693763+0000","last_became_peered":"2024-09-10T21:35:42.693763+0000","last_unstale":"2024-09-10T21:35:45.062193+0000","last_undegraded":"2024-09-10T21:35:45.062193+0000","last_fullsized":"2024-09-10T21:35:45.062193+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:01.408836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880823+0000","last_change":"2024-09-10T21:35:43.691603+0000","last_active":"2024-09-10T21:35:44.880823+0000","last_peered":"2024-09-10T21:35:44.880823+0000","last_clean":"2024-09-10T21:35:44.880823+0000","last_became_active":"2024-09-10T21:35:43.691298+0000","last_became_peered":"2024-09-10T21:35:43.691298+0000","last_unstale":"2024-09-10T21:35:44.880823+0000","last_undegraded":"2024-09-10T21:35:44.880823+0000","last_fullsized":"2024-09-10T21:35:44.880823+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:17.652436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062225+0000","last_change":"2024-09-10T21:35:42.689996+0000","last_active":"2024-09-10T21:35:45.062225+0000","last_peered":"2024-09-10T21:35:45.062225+0000","last_clean":"2024-09-10T21:35:45.062225+0000","last_became_active":"2024-09-10T21:35:42.689693+0000","last_became_peered":"2024-09-10T21:35:42.689693+0000","last_unstale":"2024-09-10T21:35:45.062225+0000","last_undegraded":"2024-09-10T21:35:45.062225+0000","last_fullsized":"2024-09-10T21:35:45.062225+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.810200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681498+0000","last_change":"2024-09-10T21:35:42.681498+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681498+0000","last_undegraded":"2024-09-10T21:35:42.681498+0000","last_fullsized":"2024-09-10T21:35:42.681498+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694720+0000","last_change":"2024-09-10T21:35:42.690501+0000","last_active":"2024-09-10T21:35:44.694720+0000","last_peered":"2024-09-10T21:35:44.694720+0000","last_clean":"2024-09-10T21:35:44.694720+0000","last_became_active":"2024-09-10T21:35:42.690137+0000","last_became_peered":"2024-09-10T21:35:42.690137+0000","last_unstale":"2024-09-10T21:35:44.694720+0000","last_undegraded":"2024-09-10T21:35:44.694720+0000","last_fullsized":"2024-09-10T21:35:44.694720+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:05.467377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680925+0000","last_change":"2024-09-10T21:35:42.680925+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680925+0000","last_undegraded":"2024-09-10T21:35:42.680925+0000","last_fullsized":"2024-09-10T21:35:42.680925+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693373+0000","last_change":"2024-09-10T21:35:42.698961+0000","last_active":"2024-09-10T21:35:44.693373+0000","last_peered":"2024-09-10T21:35:44.693373+0000","last_clean":"2024-09-10T21:35:44.693373+0000","last_became_active":"2024-09-10T21:35:42.697755+0000","last_became_peered":"2024-09-10T21:35:42.697755+0000","last_unstale":"2024-09-10T21:35:44.693373+0000","last_undegraded":"2024-09-10T21:35:44.693373+0000","last_fullsized":"2024-09-10T21:35:44.693373+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:22.195866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680542+0000","last_change":"2024-09-10T21:35:42.680542+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680542+0000","last_undegraded":"2024-09-10T21:35:42.680542+0000","last_fullsized":"2024-09-10T21:35:42.680542+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075143+0000","last_change":"2024-09-10T21:35:42.695674+0000","last_active":"2024-09-10T21:35:45.075143+0000","last_peered":"2024-09-10T21:35:45.075143+0000","last_clean":"2024-09-10T21:35:45.075143+0000","last_became_active":"2024-09-10T21:35:42.695387+0000","last_became_peered":"2024-09-10T21:35:42.695387+0000","last_unstale":"2024-09-10T21:35:45.075143+0000","last_undegraded":"2024-09-10T21:35:45.075143+0000","last_fullsized":"2024-09-10T21:35:45.075143+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:28.688577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075067+0000","last_change":"2024-09-10T21:35:43.704682+0000","last_active":"2024-09-10T21:35:45.075067+0000","last_peered":"2024-09-10T21:35:45.075067+0000","last_clean":"2024-09-10T21:35:45.075067+0000","last_became_active":"2024-09-10T21:35:43.704396+0000","last_became_peered":"2024-09-10T21:35:43.704396+0000","last_unstale":"2024-09-10T21:35:45.075067+0000","last_undegraded":"2024-09-10T21:35:45.075067+0000","last_fullsized":"2024-09-10T21:35:45.075067+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:50.258626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693160+0000","last_change":"2024-09-10T21:35:42.694971+0000","last_active":"2024-09-10T21:35:44.693160+0000","last_peered":"2024-09-10T21:35:44.693160+0000","last_clean":"2024-09-10T21:35:44.693160+0000","last_became_active":"2024-09-10T21:35:42.694748+0000","last_became_peered":"2024-09-10T21:35:42.694748+0000","last_unstale":"2024-09-10T21:35:44.693160+0000","last_undegraded":"2024-09-10T21:35:44.693160+0000","last_fullsized":"2024-09-10T21:35:44.693160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:00.243184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695083+0000","last_change":"2024-09-10T21:35:43.685624+0000","last_active":"2024-09-10T21:35:44.695083+0000","last_peered":"2024-09-10T21:35:44.695083+0000","last_clean":"2024-09-10T21:35:44.695083+0000","last_became_active":"2024-09-10T21:35:43.685331+0000","last_became_peered":"2024-09-10T21:35:43.685331+0000","last_unstale":"2024-09-10T21:35:44.695083+0000","last_undegraded":"2024-09-10T21:35:44.695083+0000","last_fullsized":"2024-09-10T21:35:44.695083+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:12.359273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880557+0000","last_change":"2024-09-10T21:35:42.697365+0000","last_active":"2024-09-10T21:35:44.880557+0000","last_peered":"2024-09-10T21:35:44.880557+0000","last_clean":"2024-09-10T21:35:44.880557+0000","last_became_active":"2024-09-10T21:35:42.693107+0000","last_became_peered":"2024-09-10T21:35:42.693107+0000","last_unstale":"2024-09-10T21:35:44.880557+0000","last_undegraded":"2024-09-10T21:35:44.880557+0000","last_fullsized":"2024-09-10T21:35:44.880557+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:15.635021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562895+0000","last_change":"2024-09-10T21:35:43.710713+0000","last_active":"2024-09-10T21:35:45.562895+0000","last_peered":"2024-09-10T21:35:45.562895+0000","last_clean":"2024-09-10T21:35:45.562895+0000","last_became_active":"2024-09-10T21:35:43.710463+0000","last_became_peered":"2024-09-10T21:35:43.710463+0000","last_unstale":"2024-09-10T21:35:45.562895+0000","last_undegraded":"2024-09-10T21:35:45.562895+0000","last_fullsized":"2024-09-10T21:35:45.562895+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:34.876598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701846+0000","last_change":"2024-09-10T21:35:42.693092+0000","last_active":"2024-09-10T21:35:42.701846+0000","last_peered":"2024-09-10T21:35:42.701846+0000","last_clean":"2024-09-10T21:35:42.701846+0000","last_became_active":"2024-09-10T21:35:42.692798+0000","last_became_peered":"2024-09-10T21:35:42.692798+0000","last_unstale":"2024-09-10T21:35:42.701846+0000","last_undegraded":"2024-09-10T21:35:42.701846+0000","last_fullsized":"2024-09-10T21:35:42.701846+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:18.074505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562873+0000","last_change":"2024-09-10T21:35:43.706761+0000","last_active":"2024-09-10T21:35:45.562873+0000","last_peered":"2024-09-10T21:35:45.562873+0000","last_clean":"2024-09-10T21:35:45.562873+0000","last_became_active":"2024-09-10T21:35:43.706277+0000","last_became_peered":"2024-09-10T21:35:43.706277+0000","last_unstale":"2024-09-10T21:35:45.562873+0000","last_undegraded":"2024-09-10T21:35:45.562873+0000","last_fullsized":"2024-09-10T21:35:45.562873+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:10.403267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075228+0000","last_change":"2024-09-10T21:35:42.698748+0000","last_active":"2024-09-10T21:35:45.075228+0000","last_peered":"2024-09-10T21:35:45.075228+0000","last_clean":"2024-09-10T21:35:45.075228+0000","last_became_active":"2024-09-10T21:35:42.698507+0000","last_became_peered":"2024-09-10T21:35:42.698507+0000","last_unstale":"2024-09-10T21:35:45.075228+0000","last_undegraded":"2024-09-10T21:35:45.075228+0000","last_fullsized":"2024-09-10T21:35:45.075228+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:02.345574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695906+0000","last_change":"2024-09-10T21:35:43.702996+0000","last_active":"2024-09-10T21:35:44.695906+0000","last_peered":"2024-09-10T21:35:44.695906+0000","last_clean":"2024-09-10T21:35:44.695906+0000","last_became_active":"2024-09-10T21:35:43.702647+0000","last_became_peered":"2024-09-10T21:35:43.702647+0000","last_unstale":"2024-09-10T21:35:44.695906+0000","last_undegraded":"2024-09-10T21:35:44.695906+0000","last_fullsized":"2024-09-10T21:35:44.695906+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:32.575627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.692613+0000","last_change":"2024-09-10T21:35:42.689212+0000","last_active":"2024-09-10T21:35:42.692613+0000","last_peered":"2024-09-10T21:35:42.692613+0000","last_clean":"2024-09-10T21:35:42.692613+0000","last_became_active":"2024-09-10T21:35:42.688903+0000","last_became_peered":"2024-09-10T21:35:42.688903+0000","last_unstale":"2024-09-10T21:35:42.692613+0000","last_undegraded":"2024-09-10T21:35:42.692613+0000","last_fullsized":"2024-09-10T21:35:42.692613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:01.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563808+0000","last_change":"2024-09-10T21:35:43.701294+0000","last_active":"2024-09-10T21:35:45.563808+0000","last_peered":"2024-09-10T21:35:45.563808+0000","last_clean":"2024-09-10T21:35:45.563808+0000","last_became_active":"2024-09-10T21:35:43.701016+0000","last_became_peered":"2024-09-10T21:35:43.701016+0000","last_unstale":"2024-09-10T21:35:45.563808+0000","last_undegraded":"2024-09-10T21:35:45.563808+0000","last_fullsized":"2024-09-10T21:35:45.563808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:58.553078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880497+0000","last_change":"2024-09-10T21:35:42.694284+0000","last_active":"2024-09-10T21:35:44.880497+0000","last_peered":"2024-09-10T21:35:44.880497+0000","last_clean":"2024-09-10T21:35:44.880497+0000","last_became_active":"2024-09-10T21:35:42.694026+0000","last_became_peered":"2024-09-10T21:35:42.694026+0000","last_unstale":"2024-09-10T21:35:44.880497+0000","last_undegraded":"2024-09-10T21:35:44.880497+0000","last_fullsized":"2024-09-10T21:35:44.880497+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:04.880365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075284+0000","last_change":"2024-09-10T21:35:43.699828+0000","last_active":"2024-09-10T21:35:45.075284+0000","last_peered":"2024-09-10T21:35:45.075284+0000","last_clean":"2024-09-10T21:35:45.075284+0000","last_became_active":"2024-09-10T21:35:43.699553+0000","last_became_peered":"2024-09-10T21:35:43.699553+0000","last_unstale":"2024-09-10T21:35:45.075284+0000","last_undegraded":"2024-09-10T21:35:45.075284+0000","last_fullsized":"2024-09-10T21:35:45.075284+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:24.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693437+0000","last_change":"2024-09-10T21:35:42.692258+0000","last_active":"2024-09-10T21:35:44.693437+0000","last_peered":"2024-09-10T21:35:44.693437+0000","last_clean":"2024-09-10T21:35:44.693437+0000","last_became_active":"2024-09-10T21:35:42.691757+0000","last_became_peered":"2024-09-10T21:35:42.691757+0000","last_unstale":"2024-09-10T21:35:44.693437+0000","last_undegraded":"2024-09-10T21:35:44.693437+0000","last_fullsized":"2024-09-10T21:35:44.693437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.810200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062140+0000","last_change":"2024-09-10T21:35:43.708695+0000","last_active":"2024-09-10T21:35:45.062140+0000","last_peered":"2024-09-10T21:35:45.062140+0000","last_clean":"2024-09-10T21:35:45.062140+0000","last_became_active":"2024-09-10T21:35:43.705922+0000","last_became_peered":"2024-09-10T21:35:43.705922+0000","last_unstale":"2024-09-10T21:35:45.062140+0000","last_undegraded":"2024-09-10T21:35:45.062140+0000","last_fullsized":"2024-09-10T21:35:45.062140+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:38.047565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062415+0000","last_change":"2024-09-10T21:35:42.687000+0000","last_active":"2024-09-10T21:35:45.062415+0000","last_peered":"2024-09-10T21:35:45.062415+0000","last_clean":"2024-09-10T21:35:45.062415+0000","last_became_active":"2024-09-10T21:35:42.686730+0000","last_became_peered":"2024-09-10T21:35:42.686730+0000","last_unstale":"2024-09-10T21:35:45.062415+0000","last_undegraded":"2024-09-10T21:35:45.062415+0000","last_fullsized":"2024-09-10T21:35:45.062415+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:28.386650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694749+0000","last_change":"2024-09-10T21:35:43.688538+0000","last_active":"2024-09-10T21:35:44.694749+0000","last_peered":"2024-09-10T21:35:44.694749+0000","last_clean":"2024-09-10T21:35:44.694749+0000","last_became_active":"2024-09-10T21:35:43.688259+0000","last_became_peered":"2024-09-10T21:35:43.688259+0000","last_unstale":"2024-09-10T21:35:44.694749+0000","last_undegraded":"2024-09-10T21:35:44.694749+0000","last_fullsized":"2024-09-10T21:35:44.694749+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:36.626231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692993+0000","last_change":"2024-09-10T21:35:42.692966+0000","last_active":"2024-09-10T21:35:44.692993+0000","last_peered":"2024-09-10T21:35:44.692993+0000","last_clean":"2024-09-10T21:35:44.692993+0000","last_became_active":"2024-09-10T21:35:42.691729+0000","last_became_peered":"2024-09-10T21:35:42.691729+0000","last_unstale":"2024-09-10T21:35:44.692993+0000","last_undegraded":"2024-09-10T21:35:44.692993+0000","last_fullsized":"2024-09-10T21:35:44.692993+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:10.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682184+0000","last_change":"2024-09-10T21:35:42.682184+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682184+0000","last_undegraded":"2024-09-10T21:35:42.682184+0000","last_fullsized":"2024-09-10T21:35:42.682184+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563454+0000","last_change":"2024-09-10T21:35:42.690902+0000","last_active":"2024-09-10T21:35:45.563454+0000","last_peered":"2024-09-10T21:35:45.563454+0000","last_clean":"2024-09-10T21:35:45.563454+0000","last_became_active":"2024-09-10T21:35:42.690591+0000","last_became_peered":"2024-09-10T21:35:42.690591+0000","last_unstale":"2024-09-10T21:35:45.563454+0000","last_undegraded":"2024-09-10T21:35:45.563454+0000","last_fullsized":"2024-09-10T21:35:45.563454+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:20.723287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695436+0000","last_change":"2024-09-10T21:35:43.710518+0000","last_active":"2024-09-10T21:35:44.695436+0000","last_peered":"2024-09-10T21:35:44.695436+0000","last_clean":"2024-09-10T21:35:44.695436+0000","last_became_active":"2024-09-10T21:35:43.710108+0000","last_became_peered":"2024-09-10T21:35:43.710108+0000","last_unstale":"2024-09-10T21:35:44.695436+0000","last_undegraded":"2024-09-10T21:35:44.695436+0000","last_fullsized":"2024-09-10T21:35:44.695436+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:52.921264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563242+0000","last_change":"2024-09-10T21:35:42.686431+0000","last_active":"2024-09-10T21:35:45.563242+0000","last_peered":"2024-09-10T21:35:45.563242+0000","last_clean":"2024-09-10T21:35:45.563242+0000","last_became_active":"2024-09-10T21:35:42.684143+0000","last_became_peered":"2024-09-10T21:35:42.684143+0000","last_unstale":"2024-09-10T21:35:45.563242+0000","last_undegraded":"2024-09-10T21:35:45.563242+0000","last_fullsized":"2024-09-10T21:35:45.563242+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:23.749610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695890+0000","last_change":"2024-09-10T21:35:43.691613+0000","last_active":"2024-09-10T21:35:44.695890+0000","last_peered":"2024-09-10T21:35:44.695890+0000","last_clean":"2024-09-10T21:35:44.695890+0000","last_became_active":"2024-09-10T21:35:43.691305+0000","last_became_peered":"2024-09-10T21:35:43.691305+0000","last_unstale":"2024-09-10T21:35:44.695890+0000","last_undegraded":"2024-09-10T21:35:44.695890+0000","last_fullsized":"2024-09-10T21:35:44.695890+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:45.661154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692885+0000","last_change":"2024-09-10T21:35:42.693678+0000","last_active":"2024-09-10T21:35:44.692885+0000","last_peered":"2024-09-10T21:35:44.692885+0000","last_clean":"2024-09-10T21:35:44.692885+0000","last_became_active":"2024-09-10T21:35:42.693394+0000","last_became_peered":"2024-09-10T21:35:42.693394+0000","last_unstale":"2024-09-10T21:35:44.692885+0000","last_undegraded":"2024-09-10T21:35:44.692885+0000","last_fullsized":"2024-09-10T21:35:44.692885+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:58.991101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681938+0000","last_change":"2024-09-10T21:35:42.681938+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681938+0000","last_undegraded":"2024-09-10T21:35:42.681938+0000","last_fullsized":"2024-09-10T21:35:42.681938+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075147+0000","last_change":"2024-09-10T21:35:42.693798+0000","last_active":"2024-09-10T21:35:45.075147+0000","last_peered":"2024-09-10T21:35:45.075147+0000","last_clean":"2024-09-10T21:35:45.075147+0000","last_became_active":"2024-09-10T21:35:42.693499+0000","last_became_peered":"2024-09-10T21:35:42.693499+0000","last_unstale":"2024-09-10T21:35:45.075147+0000","last_undegraded":"2024-09-10T21:35:45.075147+0000","last_fullsized":"2024-09-10T21:35:45.075147+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:10.502692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694617+0000","last_change":"2024-09-10T21:35:43.696424+0000","last_active":"2024-09-10T21:35:44.694617+0000","last_peered":"2024-09-10T21:35:44.694617+0000","last_clean":"2024-09-10T21:35:44.694617+0000","last_became_active":"2024-09-10T21:35:43.694483+0000","last_became_peered":"2024-09-10T21:35:43.694483+0000","last_unstale":"2024-09-10T21:35:44.694617+0000","last_undegraded":"2024-09-10T21:35:44.694617+0000","last_fullsized":"2024-09-10T21:35:44.694617+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:50.417439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563854+0000","last_change":"2024-09-10T21:35:42.695829+0000","last_active":"2024-09-10T21:35:45.563854+0000","last_peered":"2024-09-10T21:35:45.563854+0000","last_clean":"2024-09-10T21:35:45.563854+0000","last_became_active":"2024-09-10T21:35:42.695550+0000","last_became_peered":"2024-09-10T21:35:42.695550+0000","last_unstale":"2024-09-10T21:35:45.563854+0000","last_undegraded":"2024-09-10T21:35:45.563854+0000","last_fullsized":"2024-09-10T21:35:45.563854+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:29.998665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880558+0000","last_change":"2024-09-10T21:35:43.687717+0000","last_active":"2024-09-10T21:35:44.880558+0000","last_peered":"2024-09-10T21:35:44.880558+0000","last_clean":"2024-09-10T21:35:44.880558+0000","last_became_active":"2024-09-10T21:35:43.687440+0000","last_became_peered":"2024-09-10T21:35:43.687440+0000","last_unstale":"2024-09-10T21:35:44.880558+0000","last_undegraded":"2024-09-10T21:35:44.880558+0000","last_fullsized":"2024-09-10T21:35:44.880558+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:18.372062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"146'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.684833+0000","last_change":"2024-09-10T21:35:42.694598+0000","last_active":"2024-09-10T21:35:45.684833+0000","last_peered":"2024-09-10T21:35:45.684833+0000","last_clean":"2024-09-10T21:35:45.684833+0000","last_became_active":"2024-09-10T21:35:42.694267+0000","last_became_peered":"2024-09-10T21:35:42.694267+0000","last_unstale":"2024-09-10T21:35:45.684833+0000","last_undegraded":"2024-09-10T21:35:45.684833+0000","last_fullsized":"2024-09-10T21:35:45.684833+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:09.399075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075103+0000","last_change":"2024-09-10T21:35:43.708665+0000","last_active":"2024-09-10T21:35:45.075103+0000","last_peered":"2024-09-10T21:35:45.075103+0000","last_clean":"2024-09-10T21:35:45.075103+0000","last_became_active":"2024-09-10T21:35:43.707009+0000","last_became_peered":"2024-09-10T21:35:43.707009+0000","last_unstale":"2024-09-10T21:35:45.075103+0000","last_undegraded":"2024-09-10T21:35:45.075103+0000","last_fullsized":"2024-09-10T21:35:45.075103+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:41.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880692+0000","last_change":"2024-09-10T21:35:42.698430+0000","last_active":"2024-09-10T21:35:44.880692+0000","last_peered":"2024-09-10T21:35:44.880692+0000","last_clean":"2024-09-10T21:35:44.880692+0000","last_became_active":"2024-09-10T21:35:42.698195+0000","last_became_peered":"2024-09-10T21:35:42.698195+0000","last_unstale":"2024-09-10T21:35:44.880692+0000","last_undegraded":"2024-09-10T21:35:44.880692+0000","last_fullsized":"2024-09-10T21:35:44.880692+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:27.995910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062378+0000","last_change":"2024-09-10T21:35:43.697700+0000","last_active":"2024-09-10T21:35:45.062378+0000","last_peered":"2024-09-10T21:35:45.062378+0000","last_clean":"2024-09-10T21:35:45.062378+0000","last_became_active":"2024-09-10T21:35:43.694375+0000","last_became_peered":"2024-09-10T21:35:43.694375+0000","last_unstale":"2024-09-10T21:35:45.062378+0000","last_undegraded":"2024-09-10T21:35:45.062378+0000","last_fullsized":"2024-09-10T21:35:45.062378+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:49.069960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701896+0000","last_change":"2024-09-10T21:35:42.692774+0000","last_active":"2024-09-10T21:35:42.701896+0000","last_peered":"2024-09-10T21:35:42.701896+0000","last_clean":"2024-09-10T21:35:42.701896+0000","last_became_active":"2024-09-10T21:35:42.692448+0000","last_became_peered":"2024-09-10T21:35:42.692448+0000","last_unstale":"2024-09-10T21:35:42.701896+0000","last_undegraded":"2024-09-10T21:35:42.701896+0000","last_fullsized":"2024-09-10T21:35:42.701896+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:36.661466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683005+0000","last_change":"2024-09-10T21:35:42.683005+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683005+0000","last_undegraded":"2024-09-10T21:35:42.683005+0000","last_fullsized":"2024-09-10T21:35:42.683005+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880713+0000","last_change":"2024-09-10T21:35:42.691107+0000","last_active":"2024-09-10T21:35:44.880713+0000","last_peered":"2024-09-10T21:35:44.880713+0000","last_clean":"2024-09-10T21:35:44.880713+0000","last_became_active":"2024-09-10T21:35:42.690747+0000","last_became_peered":"2024-09-10T21:35:42.690747+0000","last_unstale":"2024-09-10T21:35:44.880713+0000","last_undegraded":"2024-09-10T21:35:44.880713+0000","last_fullsized":"2024-09-10T21:35:44.880713+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:26.876968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682047+0000","last_change":"2024-09-10T21:35:42.682047+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682047+0000","last_undegraded":"2024-09-10T21:35:42.682047+0000","last_fullsized":"2024-09-10T21:35:42.682047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563377+0000","last_change":"2024-09-10T21:35:42.697886+0000","last_active":"2024-09-10T21:35:45.563377+0000","last_peered":"2024-09-10T21:35:45.563377+0000","last_clean":"2024-09-10T21:35:45.563377+0000","last_became_active":"2024-09-10T21:35:42.697625+0000","last_became_peered":"2024-09-10T21:35:42.697625+0000","last_unstale":"2024-09-10T21:35:45.563377+0000","last_undegraded":"2024-09-10T21:35:45.563377+0000","last_fullsized":"2024-09-10T21:35:45.563377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:23.745543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693346+0000","last_change":"2024-09-10T21:35:43.687136+0000","last_active":"2024-09-10T21:35:44.693346+0000","last_peered":"2024-09-10T21:35:44.693346+0000","last_clean":"2024-09-10T21:35:44.693346+0000","last_became_active":"2024-09-10T21:35:43.686689+0000","last_became_peered":"2024-09-10T21:35:43.686689+0000","last_unstale":"2024-09-10T21:35:44.693346+0000","last_undegraded":"2024-09-10T21:35:44.693346+0000","last_fullsized":"2024-09-10T21:35:44.693346+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:10.009857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062254+0000","last_change":"2024-09-10T21:35:42.695868+0000","last_active":"2024-09-10T21:35:45.062254+0000","last_peered":"2024-09-10T21:35:45.062254+0000","last_clean":"2024-09-10T21:35:45.062254+0000","last_became_active":"2024-09-10T21:35:42.695165+0000","last_became_peered":"2024-09-10T21:35:42.695165+0000","last_unstale":"2024-09-10T21:35:45.062254+0000","last_undegraded":"2024-09-10T21:35:45.062254+0000","last_fullsized":"2024-09-10T21:35:45.062254+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:10.502692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562882+0000","last_change":"2024-09-10T21:35:43.706228+0000","last_active":"2024-09-10T21:35:45.562882+0000","last_peered":"2024-09-10T21:35:45.562882+0000","last_clean":"2024-09-10T21:35:45.562882+0000","last_became_active":"2024-09-10T21:35:43.705968+0000","last_became_peered":"2024-09-10T21:35:43.705968+0000","last_unstale":"2024-09-10T21:35:45.562882+0000","last_undegraded":"2024-09-10T21:35:45.562882+0000","last_fullsized":"2024-09-10T21:35:45.562882+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:34.416314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563243+0000","last_change":"2024-09-10T21:35:42.697093+0000","last_active":"2024-09-10T21:35:45.563243+0000","last_peered":"2024-09-10T21:35:45.563243+0000","last_clean":"2024-09-10T21:35:45.563243+0000","last_became_active":"2024-09-10T21:35:42.696400+0000","last_became_peered":"2024-09-10T21:35:42.696400+0000","last_unstale":"2024-09-10T21:35:45.563243+0000","last_undegraded":"2024-09-10T21:35:45.563243+0000","last_fullsized":"2024-09-10T21:35:45.563243+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:41.463058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692962+0000","last_change":"2024-09-10T21:35:43.705454+0000","last_active":"2024-09-10T21:35:44.692962+0000","last_peered":"2024-09-10T21:35:44.692962+0000","last_clean":"2024-09-10T21:35:44.692962+0000","last_became_active":"2024-09-10T21:35:43.704324+0000","last_became_peered":"2024-09-10T21:35:43.704324+0000","last_unstale":"2024-09-10T21:35:44.692962+0000","last_undegraded":"2024-09-10T21:35:44.692962+0000","last_fullsized":"2024-09-10T21:35:44.692962+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:29.000102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701861+0000","last_change":"2024-09-10T21:35:42.695716+0000","last_active":"2024-09-10T21:35:42.701861+0000","last_peered":"2024-09-10T21:35:42.701861+0000","last_clean":"2024-09-10T21:35:42.701861+0000","last_became_active":"2024-09-10T21:35:42.695410+0000","last_became_peered":"2024-09-10T21:35:42.695410+0000","last_unstale":"2024-09-10T21:35:42.701861+0000","last_undegraded":"2024-09-10T21:35:42.701861+0000","last_fullsized":"2024-09-10T21:35:42.701861+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:13.034229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":352,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.061985+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:35:45.061985+0000","last_peered":"2024-09-10T21:35:45.061985+0000","last_clean":"2024-09-10T21:35:45.061985+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:35:45.061985+0000","last_undegraded":"2024-09-10T21:35:45.061985+0000","last_fullsized":"2024-09-10T21:35:45.061985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509585,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1892352,"data_stored":953378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509583,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28884,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342956,"statfs":{"total":96636764160,"available":96607186944,"internally_reserved":0,"allocated":1896448,"data_stored":954338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509585,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33604,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338236,"statfs":{"total":96636764160,"available":96602353664,"internally_reserved":0,"allocated":2473984,"data_stored":1536526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":954338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509584,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1892352,"data_stored":952838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509585,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2486272,"data_stored":1543186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1888256,"data_stored":953308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":938867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3f is not active+clean 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.683461+0000', 'last_change': '2024-09-10T21:35:42.683461+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.683461+0000', 'last_undegraded': '2024-09-10T21:35:42.683461+0000', 'last_fullsized': '2024-09-10T21:35:42.683461+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3c is not active+clean 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.683460+0000', 'last_change': '2024-09-10T21:35:42.683460+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.683460+0000', 'last_undegraded': '2024-09-10T21:35:42.683460+0000', 'last_fullsized': '2024-09-10T21:35:42.683460+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3b is not active+clean 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.682808+0000', 'last_change': '2024-09-10T21:35:42.682808+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.682808+0000', 'last_undegraded': '2024-09-10T21:35:42.682808+0000', 'last_fullsized': '2024-09-10T21:35:42.682808+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:PG 16.33 is not active+clean 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.683255+0000', 'last_change': '2024-09-10T21:35:42.683255+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.683255+0000', 'last_undegraded': '2024-09-10T21:35:42.683255+0000', 'last_fullsized': '2024-09-10T21:35:42.683255+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.567 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2d is not active+clean 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.682503+0000', 'last_change': '2024-09-10T21:35:42.682503+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.682503+0000', 'last_undegraded': '2024-09-10T21:35:42.682503+0000', 'last_fullsized': '2024-09-10T21:35:42.682503+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:PG 16.4 is not active+clean 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.681124+0000', 'last_change': '2024-09-10T21:35:42.681124+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.681124+0000', 'last_undegraded': '2024-09-10T21:35:42.681124+0000', 'last_fullsized': '2024-09-10T21:35:42.681124+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:PG 16.5 is not active+clean 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.681555+0000', 'last_change': '2024-09-10T21:35:42.681555+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.681555+0000', 'last_undegraded': '2024-09-10T21:35:42.681555+0000', 'last_fullsized': '2024-09-10T21:35:42.681555+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:PG 16.7 is not active+clean 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.681498+0000', 'last_change': '2024-09-10T21:35:42.681498+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.681498+0000', 'last_undegraded': '2024-09-10T21:35:42.681498+0000', 'last_fullsized': '2024-09-10T21:35:42.681498+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:PG 16.8 is not active+clean 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.680925+0000', 'last_change': '2024-09-10T21:35:42.680925+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.680925+0000', 'last_undegraded': '2024-09-10T21:35:42.680925+0000', 'last_fullsized': '2024-09-10T21:35:42.680925+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:PG 16.9 is not active+clean 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.680542+0000', 'last_change': '2024-09-10T21:35:42.680542+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.680542+0000', 'last_undegraded': '2024-09-10T21:35:42.680542+0000', 'last_fullsized': '2024-09-10T21:35:42.680542+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.568 INFO:tasks.ceph.ceph_manager.ceph:PG 16.14 is not active+clean 2024-09-10T21:35:48.569 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.682184+0000', 'last_change': '2024-09-10T21:35:42.682184+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.682184+0000', 'last_undegraded': '2024-09-10T21:35:42.682184+0000', 'last_fullsized': '2024-09-10T21:35:42.682184+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.569 INFO:tasks.ceph.ceph_manager.ceph:PG 16.17 is not active+clean 2024-09-10T21:35:48.569 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.681938+0000', 'last_change': '2024-09-10T21:35:42.681938+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.681938+0000', 'last_undegraded': '2024-09-10T21:35:42.681938+0000', 'last_fullsized': '2024-09-10T21:35:42.681938+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.569 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1c is not active+clean 2024-09-10T21:35:48.569 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.683005+0000', 'last_change': '2024-09-10T21:35:42.683005+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.683005+0000', 'last_undegraded': '2024-09-10T21:35:42.683005+0000', 'last_fullsized': '2024-09-10T21:35:42.683005+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.569 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1d is not active+clean 2024-09-10T21:35:48.569 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 144, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:35:42.682047+0000', 'last_change': '2024-09-10T21:35:42.682047+0000', 'last_active': '2024-09-10T21:35:42.661776+0000', 'last_peered': '2024-09-10T21:35:42.661776+0000', 'last_clean': '2024-09-10T21:35:42.661776+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:35:42.682047+0000', 'last_undegraded': '2024-09-10T21:35:42.682047+0000', 'last_fullsized': '2024-09-10T21:35:42.682047+0000', 'mapping_epoch': 144, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 144, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'last_clean_scrub_stamp': '2024-09-10T21:35:42.661776+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:35:48.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:35:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 -- 172.21.15.97:0/3032543555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b406d250 msgr2=0x7f61b40745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 --2- 172.21.15.97:0/3032543555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b406d250 0x7f61b40745f0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f619c009bc0 tx=0x7f619c01cb10 comp rx=0 tx=0).stop 2024-09-10T21:35:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 -- 172.21.15.97:0/3032543555 shutdown_connections 2024-09-10T21:35:48.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 --2- 172.21.15.97:0/3032543555 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61b40cd650 0x7f61b40cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 --2- 172.21.15.97:0/3032543555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b406d250 0x7f61b40745f0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 -- 172.21.15.97:0/3032543555 >> 172.21.15.97:0/3032543555 conn(0x7f61b40b28c0 msgr2=0x7f61b40b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:48.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 -- 172.21.15.97:0/3032543555 shutdown_connections 2024-09-10T21:35:48.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 -- 172.21.15.97:0/3032543555 wait complete. 2024-09-10T21:35:48.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 Processor -- start 2024-09-10T21:35:48.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.753+0000 7f61b8a00640 1 -- start start 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61b406d250 0x7f61b41487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b40cd650 0x7f61b40b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61b4159980 0x7f61b4070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4168a90 con 0x7f61b40cd650 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4168910 con 0x7f61b4159980 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f61b4168c10 con 0x7f61b406d250 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b40cd650 0x7f61b40b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b40cd650 0x7f61b40b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45040/0 (socket says 172.21.15.97:45040) 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 -- 172.21.15.97:0/4131383116 learned_addr learned my addr 172.21.15.97:0/4131383116 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b1e00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61b406d250 0x7f61b41487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b2800640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61b4159980 0x7f61b4070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 -- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61b406d250 msgr2=0x7f61b41487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61b406d250 0x7f61b41487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 -- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61b4159980 msgr2=0x7f61b4070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61b4159980 0x7f61b4070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 -- 172.21.15.97:0/4131383116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b40b5080 con 0x7f61b40cd650 2024-09-10T21:35:48.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b1e00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61b406d250 0x7f61b41487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61a9400640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b40cd650 0x7f61b40b56a0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f619c01b1c0 tx=0x7f619c0a5970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61aaa00640 1 -- 172.21.15.97:0/4131383116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c0ba020 con 0x7f61b40cd650 2024-09-10T21:35:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61aaa00640 1 -- 172.21.15.97:0/4131383116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f619c004d40 con 0x7f61b40cd650 2024-09-10T21:35:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b40aefe0 con 0x7f61b40cd650 2024-09-10T21:35:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61aaa00640 1 -- 172.21.15.97:0/4131383116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c0b2040 con 0x7f61b40cd650 2024-09-10T21:35:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b40af5a0 con 0x7f61b40cd650 2024-09-10T21:35:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b4158ab0 con 0x7f61b40cd650 2024-09-10T21:35:48.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.757+0000 7f61aaa00640 1 -- 172.21.15.97:0/4131383116 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f619c0b5070 con 0x7f61b40cd650 2024-09-10T21:35:48.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.761+0000 7f61aaa00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61900725e0 0x7f6190074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:48.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.761+0000 7f61aaa00640 1 -- 172.21.15.97:0/4131383116 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f619c12d700 con 0x7f61b40cd650 2024-09-10T21:35:48.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.761+0000 7f61b1e00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61900725e0 0x7f6190074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:48.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.761+0000 7f61b1e00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61900725e0 0x7f6190074aa0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f61a4000f70 tx=0x7f61a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:48.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.765+0000 7f61aaa00640 1 -- 172.21.15.97:0/4131383116 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f619c0c4020 con 0x7f61b40cd650 2024-09-10T21:35:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.989+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f61b40745f0 con 0x7f61b40cd650 2024-09-10T21:35:48.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.989+0000 7f61aaa00640 1 -- 172.21.15.97:0/4131383116 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1145 (secure 0 0 0) 0x7f619c016790 con 0x7f61b40cd650 2024-09-10T21:35:48.990 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:48.990 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":722,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":146,"num_osds":8,"num_up_osds":8,"osd_up_since":1726003441,"num_in_osds":8,"osd_in_since":1726003430,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"creating+peering","count":14}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592744,"bytes_used":288841728,"bytes_avail":772805271552,"bytes_total":773094113280,"inactive_pgs_ratio":0.14432989060878754,"write_bytes_sec":1194,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":344,"btime":"2024-09-10T21:35:46:701370+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":10647}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":48,"modified":"2024-09-10T21:35:18.241986+0000","services":{}},"progress_events":{}} 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61900725e0 msgr2=0x7f6190074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61900725e0 0x7f6190074aa0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f61a4000f70 tx=0x7f61a4009040 comp rx=0 tx=0).stop 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b40cd650 msgr2=0x7f61b40b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b40cd650 0x7f61b40b56a0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f619c01b1c0 tx=0x7f619c0a5970 comp rx=0 tx=0).stop 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 shutdown_connections 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f61900725e0 0x7f6190074aa0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f61b4159980 0x7f61b4070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61b40cd650 0x7f61b40b56a0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 --2- 172.21.15.97:0/4131383116 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f61b406d250 0x7f61b41487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:48.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 >> 172.21.15.97:0/4131383116 conn(0x7f61b40b28c0 msgr2=0x7f61b4157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:48.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 shutdown_connections 2024-09-10T21:35:48.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:48.993+0000 7f61b8a00640 1 -- 172.21.15.97:0/4131383116 wait complete. 2024-09-10T21:35:49.012 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:35:49.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:35:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.153+0000 7f9dab400640 1 -- 172.21.15.97:0/422096645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 msgr2=0x7f9d880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.153+0000 7f9dab400640 1 --2- 172.21.15.97:0/422096645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 0x7f9d880051d0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009f10 tx=0x7f9d9c01d5d0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.153+0000 7f9da9400640 1 -- 172.21.15.97:0/422096645 reap_dead start 2024-09-10T21:35:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.153+0000 7f9dab400640 1 -- 172.21.15.97:0/422096645 shutdown_connections 2024-09-10T21:35:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.153+0000 7f9dab400640 1 -- 172.21.15.97:0/422096645 >> 172.21.15.97:0/422096645 conn(0x7f9da40b1780 msgr2=0x7f9da40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.153+0000 7f9dab400640 1 -- 172.21.15.97:0/422096645 shutdown_connections 2024-09-10T21:35:49.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.153+0000 7f9dab400640 1 -- 172.21.15.97:0/422096645 wait complete. 2024-09-10T21:35:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 Processor -- start 2024-09-10T21:35:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 -- start start 2024-09-10T21:35:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 0x7f9da414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9da41f88a0 0x7f9da41f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9da4150b90 0x7f9da4150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4072f00 con 0x7f9d88004d50 2024-09-10T21:35:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4072d80 con 0x7f9da4150b90 2024-09-10T21:35:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9da4070990 con 0x7f9da41f88a0 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 0x7f9da414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 0x7f9da414c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45052/0 (socket says 172.21.15.97:45052) 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 -- 172.21.15.97:0/1427305573 learned_addr learned my addr 172.21.15.97:0/1427305573 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9da41f88a0 0x7f9da41f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da9400640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9da4150b90 0x7f9da4150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 -- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9da41f88a0 msgr2=0x7f9da41f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9da41f88a0 0x7f9da41f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 -- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9da4150b90 msgr2=0x7f9da4150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9da4150b90 0x7f9da4150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 -- 172.21.15.97:0/1427305573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da414faf0 con 0x7f9d88004d50 2024-09-10T21:35:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da8a00640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 0x7f9da414c3f0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f9d94006690 tx=0x7f9d940066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da1600640 1 -- 172.21.15.97:0/1427305573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94011700 con 0x7f9d88004d50 2024-09-10T21:35:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da1600640 1 -- 172.21.15.97:0/1427305573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d94016020 con 0x7f9d88004d50 2024-09-10T21:35:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9da1600640 1 -- 172.21.15.97:0/1427305573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94011b40 con 0x7f9d88004d50 2024-09-10T21:35:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da414fdd0 con 0x7f9d88004d50 2024-09-10T21:35:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.157+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da4150390 con 0x7f9d88004d50 2024-09-10T21:35:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.161+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da4072630 con 0x7f9d88004d50 2024-09-10T21:35:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.161+0000 7f9da1600640 1 -- 172.21.15.97:0/1427305573 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d94011ce0 con 0x7f9d88004d50 2024-09-10T21:35:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.161+0000 7f9da1600640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9d80072610 0x7f9d80074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.161+0000 7f9da1600640 1 -- 172.21.15.97:0/1427305573 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f9d94093e20 con 0x7f9d88004d50 2024-09-10T21:35:49.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.161+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9d80072610 0x7f9d80074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.161+0000 7f9d9be00640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9d80072610 0x7f9d80074ad0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9da40729f0 tx=0x7f9d8c00e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.165+0000 7f9da1600640 1 -- 172.21.15.97:0/1427305573 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d94020020 con 0x7f9d88004d50 2024-09-10T21:35:49.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.329+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9da406e690 con 0x7f9d80072610 2024-09-10T21:35:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.333+0000 7f9da1600640 1 -- 172.21.15.97:0/1427305573 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251319 (secure 0 0 0) 0x7f9da406e690 con 0x7f9d80072610 2024-09-10T21:35:49.335 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:35:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9d80072610 msgr2=0x7f9d80074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9d80072610 0x7f9d80074ad0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9da40729f0 tx=0x7f9d8c00e040 comp rx=0 tx=0).stop 2024-09-10T21:35:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 msgr2=0x7f9da414c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:49.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 0x7f9da414c3f0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f9d94006690 tx=0x7f9d940066c0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 shutdown_connections 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9d80072610 0x7f9d80074ad0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9da4150b90 0x7f9da4150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9da41f88a0 0x7f9da41f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 --2- 172.21.15.97:0/1427305573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d88004d50 0x7f9da414c3f0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 >> 172.21.15.97:0/1427305573 conn(0x7f9da40b1780 msgr2=0x7f9da415d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 shutdown_connections 2024-09-10T21:35:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:49.337+0000 7f9dab400640 1 -- 172.21.15.97:0/1427305573 wait complete. 2024-09-10T21:35:49.355 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T21:35:48.249753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":140,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282072,"kb_used_data":15904,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692648,"statfs":{"total":773094113280,"available":772805271552,"internally_reserved":0,"allocated":16285696,"data_stored":8786779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001455"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683461+0000","last_change":"2024-09-10T21:35:42.683461+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683461+0000","last_undegraded":"2024-09-10T21:35:42.683461+0000","last_fullsized":"2024-09-10T21:35:42.683461+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562668+0000","last_change":"2024-09-10T21:35:43.704550+0000","last_active":"2024-09-10T21:35:45.562668+0000","last_peered":"2024-09-10T21:35:45.562668+0000","last_clean":"2024-09-10T21:35:45.562668+0000","last_became_active":"2024-09-10T21:35:43.704086+0000","last_became_peered":"2024-09-10T21:35:43.704086+0000","last_unstale":"2024-09-10T21:35:45.562668+0000","last_undegraded":"2024-09-10T21:35:45.562668+0000","last_fullsized":"2024-09-10T21:35:45.562668+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:14.393495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563678+0000","last_change":"2024-09-10T21:35:43.702123+0000","last_active":"2024-09-10T21:35:45.563678+0000","last_peered":"2024-09-10T21:35:45.563678+0000","last_clean":"2024-09-10T21:35:45.563678+0000","last_became_active":"2024-09-10T21:35:43.700634+0000","last_became_peered":"2024-09-10T21:35:43.700634+0000","last_unstale":"2024-09-10T21:35:45.563678+0000","last_undegraded":"2024-09-10T21:35:45.563678+0000","last_fullsized":"2024-09-10T21:35:45.563678+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:13.113908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683460+0000","last_change":"2024-09-10T21:35:42.683460+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683460+0000","last_undegraded":"2024-09-10T21:35:42.683460+0000","last_fullsized":"2024-09-10T21:35:42.683460+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682808+0000","last_change":"2024-09-10T21:35:42.682808+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682808+0000","last_undegraded":"2024-09-10T21:35:42.682808+0000","last_fullsized":"2024-09-10T21:35:42.682808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692726+0000","last_change":"2024-09-10T21:35:43.710080+0000","last_active":"2024-09-10T21:35:44.692726+0000","last_peered":"2024-09-10T21:35:44.692726+0000","last_clean":"2024-09-10T21:35:44.692726+0000","last_became_active":"2024-09-10T21:35:43.709693+0000","last_became_peered":"2024-09-10T21:35:43.709693+0000","last_unstale":"2024-09-10T21:35:44.692726+0000","last_undegraded":"2024-09-10T21:35:44.692726+0000","last_fullsized":"2024-09-10T21:35:44.692726+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:38.047565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692510+0000","last_change":"2024-09-10T21:35:43.711852+0000","last_active":"2024-09-10T21:35:44.692510+0000","last_peered":"2024-09-10T21:35:44.692510+0000","last_clean":"2024-09-10T21:35:44.692510+0000","last_became_active":"2024-09-10T21:35:43.711654+0000","last_became_peered":"2024-09-10T21:35:43.711654+0000","last_unstale":"2024-09-10T21:35:44.692510+0000","last_undegraded":"2024-09-10T21:35:44.692510+0000","last_fullsized":"2024-09-10T21:35:44.692510+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:12.060947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075008+0000","last_change":"2024-09-10T21:35:43.713260+0000","last_active":"2024-09-10T21:35:45.075008+0000","last_peered":"2024-09-10T21:35:45.075008+0000","last_clean":"2024-09-10T21:35:45.075008+0000","last_became_active":"2024-09-10T21:35:43.712978+0000","last_became_peered":"2024-09-10T21:35:43.712978+0000","last_unstale":"2024-09-10T21:35:45.075008+0000","last_undegraded":"2024-09-10T21:35:45.075008+0000","last_fullsized":"2024-09-10T21:35:45.075008+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:36.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562758+0000","last_change":"2024-09-10T21:35:43.711304+0000","last_active":"2024-09-10T21:35:45.562758+0000","last_peered":"2024-09-10T21:35:45.562758+0000","last_clean":"2024-09-10T21:35:45.562758+0000","last_became_active":"2024-09-10T21:35:43.711066+0000","last_became_peered":"2024-09-10T21:35:43.711066+0000","last_unstale":"2024-09-10T21:35:45.562758+0000","last_undegraded":"2024-09-10T21:35:45.562758+0000","last_fullsized":"2024-09-10T21:35:45.562758+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:32.539648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075220+0000","last_change":"2024-09-10T21:35:43.708804+0000","last_active":"2024-09-10T21:35:45.075220+0000","last_peered":"2024-09-10T21:35:45.075220+0000","last_clean":"2024-09-10T21:35:45.075220+0000","last_became_active":"2024-09-10T21:35:43.708489+0000","last_became_peered":"2024-09-10T21:35:43.708489+0000","last_unstale":"2024-09-10T21:35:45.075220+0000","last_undegraded":"2024-09-10T21:35:45.075220+0000","last_fullsized":"2024-09-10T21:35:45.075220+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:47.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880679+0000","last_change":"2024-09-10T21:35:43.693534+0000","last_active":"2024-09-10T21:35:44.880679+0000","last_peered":"2024-09-10T21:35:44.880679+0000","last_clean":"2024-09-10T21:35:44.880679+0000","last_became_active":"2024-09-10T21:35:43.693285+0000","last_became_peered":"2024-09-10T21:35:43.693285+0000","last_unstale":"2024-09-10T21:35:44.880679+0000","last_undegraded":"2024-09-10T21:35:44.880679+0000","last_fullsized":"2024-09-10T21:35:44.880679+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:18.734510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693052+0000","last_change":"2024-09-10T21:35:43.689168+0000","last_active":"2024-09-10T21:35:44.693052+0000","last_peered":"2024-09-10T21:35:44.693052+0000","last_clean":"2024-09-10T21:35:44.693052+0000","last_became_active":"2024-09-10T21:35:43.688873+0000","last_became_peered":"2024-09-10T21:35:43.688873+0000","last_unstale":"2024-09-10T21:35:44.693052+0000","last_undegraded":"2024-09-10T21:35:44.693052+0000","last_fullsized":"2024-09-10T21:35:44.693052+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:16.639213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683255+0000","last_change":"2024-09-10T21:35:42.683255+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683255+0000","last_undegraded":"2024-09-10T21:35:42.683255+0000","last_fullsized":"2024-09-10T21:35:42.683255+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692849+0000","last_change":"2024-09-10T21:35:43.689434+0000","last_active":"2024-09-10T21:35:44.692849+0000","last_peered":"2024-09-10T21:35:44.692849+0000","last_clean":"2024-09-10T21:35:44.692849+0000","last_became_active":"2024-09-10T21:35:43.689163+0000","last_became_peered":"2024-09-10T21:35:43.689163+0000","last_unstale":"2024-09-10T21:35:44.692849+0000","last_undegraded":"2024-09-10T21:35:44.692849+0000","last_fullsized":"2024-09-10T21:35:44.692849+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:21.413539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562494+0000","last_change":"2024-09-10T21:35:43.698275+0000","last_active":"2024-09-10T21:35:45.562494+0000","last_peered":"2024-09-10T21:35:45.562494+0000","last_clean":"2024-09-10T21:35:45.562494+0000","last_became_active":"2024-09-10T21:35:43.698023+0000","last_became_peered":"2024-09-10T21:35:43.698023+0000","last_unstale":"2024-09-10T21:35:45.562494+0000","last_undegraded":"2024-09-10T21:35:45.562494+0000","last_fullsized":"2024-09-10T21:35:45.562494+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:37.116752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062129+0000","last_change":"2024-09-10T21:35:43.702042+0000","last_active":"2024-09-10T21:35:45.062129+0000","last_peered":"2024-09-10T21:35:45.062129+0000","last_clean":"2024-09-10T21:35:45.062129+0000","last_became_active":"2024-09-10T21:35:43.701794+0000","last_became_peered":"2024-09-10T21:35:43.701794+0000","last_unstale":"2024-09-10T21:35:45.062129+0000","last_undegraded":"2024-09-10T21:35:45.062129+0000","last_fullsized":"2024-09-10T21:35:45.062129+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:42.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563696+0000","last_change":"2024-09-10T21:35:43.706781+0000","last_active":"2024-09-10T21:35:45.563696+0000","last_peered":"2024-09-10T21:35:45.563696+0000","last_clean":"2024-09-10T21:35:45.563696+0000","last_became_active":"2024-09-10T21:35:43.706511+0000","last_became_peered":"2024-09-10T21:35:43.706511+0000","last_unstale":"2024-09-10T21:35:45.563696+0000","last_undegraded":"2024-09-10T21:35:45.563696+0000","last_fullsized":"2024-09-10T21:35:45.563696+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:56.867984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562875+0000","last_change":"2024-09-10T21:35:43.710248+0000","last_active":"2024-09-10T21:35:45.562875+0000","last_peered":"2024-09-10T21:35:45.562875+0000","last_clean":"2024-09-10T21:35:45.562875+0000","last_became_active":"2024-09-10T21:35:43.710000+0000","last_became_peered":"2024-09-10T21:35:43.710000+0000","last_unstale":"2024-09-10T21:35:45.562875+0000","last_undegraded":"2024-09-10T21:35:45.562875+0000","last_fullsized":"2024-09-10T21:35:45.562875+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:24.749734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682503+0000","last_change":"2024-09-10T21:35:42.682503+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682503+0000","last_undegraded":"2024-09-10T21:35:42.682503+0000","last_fullsized":"2024-09-10T21:35:42.682503+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062438+0000","last_change":"2024-09-10T21:35:43.695973+0000","last_active":"2024-09-10T21:35:45.062438+0000","last_peered":"2024-09-10T21:35:45.062438+0000","last_clean":"2024-09-10T21:35:45.062438+0000","last_became_active":"2024-09-10T21:35:43.695709+0000","last_became_peered":"2024-09-10T21:35:43.695709+0000","last_unstale":"2024-09-10T21:35:45.062438+0000","last_undegraded":"2024-09-10T21:35:45.062438+0000","last_fullsized":"2024-09-10T21:35:45.062438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:21.413539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563488+0000","last_change":"2024-09-10T21:35:43.695495+0000","last_active":"2024-09-10T21:35:45.563488+0000","last_peered":"2024-09-10T21:35:45.563488+0000","last_clean":"2024-09-10T21:35:45.563488+0000","last_became_active":"2024-09-10T21:35:43.695095+0000","last_became_peered":"2024-09-10T21:35:43.695095+0000","last_unstale":"2024-09-10T21:35:45.563488+0000","last_undegraded":"2024-09-10T21:35:45.563488+0000","last_fullsized":"2024-09-10T21:35:45.563488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:54.813769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694816+0000","last_change":"2024-09-10T21:35:43.696103+0000","last_active":"2024-09-10T21:35:44.694816+0000","last_peered":"2024-09-10T21:35:44.694816+0000","last_clean":"2024-09-10T21:35:44.694816+0000","last_became_active":"2024-09-10T21:35:43.695633+0000","last_became_peered":"2024-09-10T21:35:43.695633+0000","last_unstale":"2024-09-10T21:35:44.694816+0000","last_undegraded":"2024-09-10T21:35:44.694816+0000","last_fullsized":"2024-09-10T21:35:44.694816+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:56.055893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074781+0000","last_change":"2024-09-10T21:35:43.699096+0000","last_active":"2024-09-10T21:35:45.074781+0000","last_peered":"2024-09-10T21:35:45.074781+0000","last_clean":"2024-09-10T21:35:45.074781+0000","last_became_active":"2024-09-10T21:35:43.698826+0000","last_became_peered":"2024-09-10T21:35:43.698826+0000","last_unstale":"2024-09-10T21:35:45.074781+0000","last_undegraded":"2024-09-10T21:35:45.074781+0000","last_fullsized":"2024-09-10T21:35:45.074781+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:31.736444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074767+0000","last_change":"2024-09-10T21:35:43.710788+0000","last_active":"2024-09-10T21:35:45.074767+0000","last_peered":"2024-09-10T21:35:45.074767+0000","last_clean":"2024-09-10T21:35:45.074767+0000","last_became_active":"2024-09-10T21:35:43.710492+0000","last_became_peered":"2024-09-10T21:35:43.710492+0000","last_unstale":"2024-09-10T21:35:45.074767+0000","last_undegraded":"2024-09-10T21:35:45.074767+0000","last_fullsized":"2024-09-10T21:35:45.074767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:09.641780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062559+0000","last_change":"2024-09-10T21:35:43.695952+0000","last_active":"2024-09-10T21:35:45.062559+0000","last_peered":"2024-09-10T21:35:45.062559+0000","last_clean":"2024-09-10T21:35:45.062559+0000","last_became_active":"2024-09-10T21:35:43.694886+0000","last_became_peered":"2024-09-10T21:35:43.694886+0000","last_unstale":"2024-09-10T21:35:45.062559+0000","last_undegraded":"2024-09-10T21:35:45.062559+0000","last_fullsized":"2024-09-10T21:35:45.062559+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:41.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062558+0000","last_change":"2024-09-10T21:35:43.691902+0000","last_active":"2024-09-10T21:35:45.062558+0000","last_peered":"2024-09-10T21:35:45.062558+0000","last_clean":"2024-09-10T21:35:45.062558+0000","last_became_active":"2024-09-10T21:35:43.691638+0000","last_became_peered":"2024-09-10T21:35:43.691638+0000","last_unstale":"2024-09-10T21:35:45.062558+0000","last_undegraded":"2024-09-10T21:35:45.062558+0000","last_fullsized":"2024-09-10T21:35:45.062558+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:16.639213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062518+0000","last_change":"2024-09-10T21:35:43.696465+0000","last_active":"2024-09-10T21:35:45.062518+0000","last_peered":"2024-09-10T21:35:45.062518+0000","last_clean":"2024-09-10T21:35:45.062518+0000","last_became_active":"2024-09-10T21:35:43.695584+0000","last_became_peered":"2024-09-10T21:35:43.695584+0000","last_unstale":"2024-09-10T21:35:45.062518+0000","last_undegraded":"2024-09-10T21:35:45.062518+0000","last_fullsized":"2024-09-10T21:35:45.062518+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:29.000102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692740+0000","last_change":"2024-09-10T21:35:43.686570+0000","last_active":"2024-09-10T21:35:44.692740+0000","last_peered":"2024-09-10T21:35:44.692740+0000","last_clean":"2024-09-10T21:35:44.692740+0000","last_became_active":"2024-09-10T21:35:43.686287+0000","last_became_peered":"2024-09-10T21:35:43.686287+0000","last_unstale":"2024-09-10T21:35:44.692740+0000","last_undegraded":"2024-09-10T21:35:44.692740+0000","last_fullsized":"2024-09-10T21:35:44.692740+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:49.069960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074985+0000","last_change":"2024-09-10T21:35:43.697251+0000","last_active":"2024-09-10T21:35:45.074985+0000","last_peered":"2024-09-10T21:35:45.074985+0000","last_clean":"2024-09-10T21:35:45.074985+0000","last_became_active":"2024-09-10T21:35:43.696685+0000","last_became_peered":"2024-09-10T21:35:43.696685+0000","last_unstale":"2024-09-10T21:35:45.074985+0000","last_undegraded":"2024-09-10T21:35:45.074985+0000","last_fullsized":"2024-09-10T21:35:45.074985+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:19.847487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880695+0000","last_change":"2024-09-10T21:35:43.695764+0000","last_active":"2024-09-10T21:35:44.880695+0000","last_peered":"2024-09-10T21:35:44.880695+0000","last_clean":"2024-09-10T21:35:44.880695+0000","last_became_active":"2024-09-10T21:35:43.695460+0000","last_became_peered":"2024-09-10T21:35:43.695460+0000","last_unstale":"2024-09-10T21:35:44.880695+0000","last_undegraded":"2024-09-10T21:35:44.880695+0000","last_fullsized":"2024-09-10T21:35:44.880695+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:05.693752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692686+0000","last_change":"2024-09-10T21:35:43.702866+0000","last_active":"2024-09-10T21:35:44.692686+0000","last_peered":"2024-09-10T21:35:44.692686+0000","last_clean":"2024-09-10T21:35:44.692686+0000","last_became_active":"2024-09-10T21:35:43.702624+0000","last_became_peered":"2024-09-10T21:35:43.702624+0000","last_unstale":"2024-09-10T21:35:44.692686+0000","last_undegraded":"2024-09-10T21:35:44.692686+0000","last_fullsized":"2024-09-10T21:35:44.692686+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:42.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694552+0000","last_change":"2024-09-10T21:35:43.691043+0000","last_active":"2024-09-10T21:35:44.694552+0000","last_peered":"2024-09-10T21:35:44.694552+0000","last_clean":"2024-09-10T21:35:44.694552+0000","last_became_active":"2024-09-10T21:35:43.690768+0000","last_became_peered":"2024-09-10T21:35:43.690768+0000","last_unstale":"2024-09-10T21:35:44.694552+0000","last_undegraded":"2024-09-10T21:35:44.694552+0000","last_fullsized":"2024-09-10T21:35:44.694552+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:59.974397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563300+0000","last_change":"2024-09-10T21:35:43.690260+0000","last_active":"2024-09-10T21:35:45.563300+0000","last_peered":"2024-09-10T21:35:45.563300+0000","last_clean":"2024-09-10T21:35:45.563300+0000","last_became_active":"2024-09-10T21:35:43.690001+0000","last_became_peered":"2024-09-10T21:35:43.690001+0000","last_unstale":"2024-09-10T21:35:45.563300+0000","last_undegraded":"2024-09-10T21:35:45.563300+0000","last_fullsized":"2024-09-10T21:35:45.563300+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:11.506883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696123+0000","last_change":"2024-09-10T21:35:42.684988+0000","last_active":"2024-09-10T21:35:44.696123+0000","last_peered":"2024-09-10T21:35:44.696123+0000","last_clean":"2024-09-10T21:35:44.696123+0000","last_became_active":"2024-09-10T21:35:42.684730+0000","last_became_peered":"2024-09-10T21:35:42.684730+0000","last_unstale":"2024-09-10T21:35:44.696123+0000","last_undegraded":"2024-09-10T21:35:44.696123+0000","last_fullsized":"2024-09-10T21:35:44.696123+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:22.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074844+0000","last_change":"2024-09-10T21:35:43.697903+0000","last_active":"2024-09-10T21:35:45.074844+0000","last_peered":"2024-09-10T21:35:45.074844+0000","last_clean":"2024-09-10T21:35:45.074844+0000","last_became_active":"2024-09-10T21:35:43.697647+0000","last_became_peered":"2024-09-10T21:35:43.697647+0000","last_unstale":"2024-09-10T21:35:45.074844+0000","last_undegraded":"2024-09-10T21:35:45.074844+0000","last_fullsized":"2024-09-10T21:35:45.074844+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:02.827939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563603+0000","last_change":"2024-09-10T21:35:42.683390+0000","last_active":"2024-09-10T21:35:45.563603+0000","last_peered":"2024-09-10T21:35:45.563603+0000","last_clean":"2024-09-10T21:35:45.563603+0000","last_became_active":"2024-09-10T21:35:42.683095+0000","last_became_peered":"2024-09-10T21:35:42.683095+0000","last_unstale":"2024-09-10T21:35:45.563603+0000","last_undegraded":"2024-09-10T21:35:45.563603+0000","last_fullsized":"2024-09-10T21:35:45.563603+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:08.275738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562597+0000","last_change":"2024-09-10T21:35:43.704762+0000","last_active":"2024-09-10T21:35:45.562597+0000","last_peered":"2024-09-10T21:35:45.562597+0000","last_clean":"2024-09-10T21:35:45.562597+0000","last_became_active":"2024-09-10T21:35:43.704137+0000","last_became_peered":"2024-09-10T21:35:43.704137+0000","last_unstale":"2024-09-10T21:35:45.562597+0000","last_undegraded":"2024-09-10T21:35:45.562597+0000","last_fullsized":"2024-09-10T21:35:45.562597+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:11.824013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562772+0000","last_change":"2024-09-10T21:35:42.693628+0000","last_active":"2024-09-10T21:35:45.562772+0000","last_peered":"2024-09-10T21:35:45.562772+0000","last_clean":"2024-09-10T21:35:45.562772+0000","last_became_active":"2024-09-10T21:35:42.692980+0000","last_became_peered":"2024-09-10T21:35:42.692980+0000","last_unstale":"2024-09-10T21:35:45.562772+0000","last_undegraded":"2024-09-10T21:35:45.562772+0000","last_fullsized":"2024-09-10T21:35:45.562772+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:25.656976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880707+0000","last_change":"2024-09-10T21:35:43.698119+0000","last_active":"2024-09-10T21:35:44.880707+0000","last_peered":"2024-09-10T21:35:44.880707+0000","last_clean":"2024-09-10T21:35:44.880707+0000","last_became_active":"2024-09-10T21:35:43.697892+0000","last_became_peered":"2024-09-10T21:35:43.697892+0000","last_unstale":"2024-09-10T21:35:44.880707+0000","last_undegraded":"2024-09-10T21:35:44.880707+0000","last_fullsized":"2024-09-10T21:35:44.880707+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:13.854892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074953+0000","last_change":"2024-09-10T21:35:42.697366+0000","last_active":"2024-09-10T21:35:45.074953+0000","last_peered":"2024-09-10T21:35:45.074953+0000","last_clean":"2024-09-10T21:35:45.074953+0000","last_became_active":"2024-09-10T21:35:42.697095+0000","last_became_peered":"2024-09-10T21:35:42.697095+0000","last_unstale":"2024-09-10T21:35:45.074953+0000","last_undegraded":"2024-09-10T21:35:45.074953+0000","last_fullsized":"2024-09-10T21:35:45.074953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:23.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681124+0000","last_change":"2024-09-10T21:35:42.681124+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681124+0000","last_undegraded":"2024-09-10T21:35:42.681124+0000","last_fullsized":"2024-09-10T21:35:42.681124+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880614+0000","last_change":"2024-09-10T21:35:42.695787+0000","last_active":"2024-09-10T21:35:44.880614+0000","last_peered":"2024-09-10T21:35:44.880614+0000","last_clean":"2024-09-10T21:35:44.880614+0000","last_became_active":"2024-09-10T21:35:42.695491+0000","last_became_peered":"2024-09-10T21:35:42.695491+0000","last_unstale":"2024-09-10T21:35:44.880614+0000","last_undegraded":"2024-09-10T21:35:44.880614+0000","last_fullsized":"2024-09-10T21:35:44.880614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:09.005666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681555+0000","last_change":"2024-09-10T21:35:42.681555+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681555+0000","last_undegraded":"2024-09-10T21:35:42.681555+0000","last_fullsized":"2024-09-10T21:35:42.681555+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062193+0000","last_change":"2024-09-10T21:35:42.694006+0000","last_active":"2024-09-10T21:35:45.062193+0000","last_peered":"2024-09-10T21:35:45.062193+0000","last_clean":"2024-09-10T21:35:45.062193+0000","last_became_active":"2024-09-10T21:35:42.693763+0000","last_became_peered":"2024-09-10T21:35:42.693763+0000","last_unstale":"2024-09-10T21:35:45.062193+0000","last_undegraded":"2024-09-10T21:35:45.062193+0000","last_fullsized":"2024-09-10T21:35:45.062193+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:01.408836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880823+0000","last_change":"2024-09-10T21:35:43.691603+0000","last_active":"2024-09-10T21:35:44.880823+0000","last_peered":"2024-09-10T21:35:44.880823+0000","last_clean":"2024-09-10T21:35:44.880823+0000","last_became_active":"2024-09-10T21:35:43.691298+0000","last_became_peered":"2024-09-10T21:35:43.691298+0000","last_unstale":"2024-09-10T21:35:44.880823+0000","last_undegraded":"2024-09-10T21:35:44.880823+0000","last_fullsized":"2024-09-10T21:35:44.880823+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:17.652436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062225+0000","last_change":"2024-09-10T21:35:42.689996+0000","last_active":"2024-09-10T21:35:45.062225+0000","last_peered":"2024-09-10T21:35:45.062225+0000","last_clean":"2024-09-10T21:35:45.062225+0000","last_became_active":"2024-09-10T21:35:42.689693+0000","last_became_peered":"2024-09-10T21:35:42.689693+0000","last_unstale":"2024-09-10T21:35:45.062225+0000","last_undegraded":"2024-09-10T21:35:45.062225+0000","last_fullsized":"2024-09-10T21:35:45.062225+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.810200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681498+0000","last_change":"2024-09-10T21:35:42.681498+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681498+0000","last_undegraded":"2024-09-10T21:35:42.681498+0000","last_fullsized":"2024-09-10T21:35:42.681498+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694720+0000","last_change":"2024-09-10T21:35:42.690501+0000","last_active":"2024-09-10T21:35:44.694720+0000","last_peered":"2024-09-10T21:35:44.694720+0000","last_clean":"2024-09-10T21:35:44.694720+0000","last_became_active":"2024-09-10T21:35:42.690137+0000","last_became_peered":"2024-09-10T21:35:42.690137+0000","last_unstale":"2024-09-10T21:35:44.694720+0000","last_undegraded":"2024-09-10T21:35:44.694720+0000","last_fullsized":"2024-09-10T21:35:44.694720+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:05.467377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680925+0000","last_change":"2024-09-10T21:35:42.680925+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680925+0000","last_undegraded":"2024-09-10T21:35:42.680925+0000","last_fullsized":"2024-09-10T21:35:42.680925+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693373+0000","last_change":"2024-09-10T21:35:42.698961+0000","last_active":"2024-09-10T21:35:44.693373+0000","last_peered":"2024-09-10T21:35:44.693373+0000","last_clean":"2024-09-10T21:35:44.693373+0000","last_became_active":"2024-09-10T21:35:42.697755+0000","last_became_peered":"2024-09-10T21:35:42.697755+0000","last_unstale":"2024-09-10T21:35:44.693373+0000","last_undegraded":"2024-09-10T21:35:44.693373+0000","last_fullsized":"2024-09-10T21:35:44.693373+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:22.195866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.680542+0000","last_change":"2024-09-10T21:35:42.680542+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.680542+0000","last_undegraded":"2024-09-10T21:35:42.680542+0000","last_fullsized":"2024-09-10T21:35:42.680542+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075143+0000","last_change":"2024-09-10T21:35:42.695674+0000","last_active":"2024-09-10T21:35:45.075143+0000","last_peered":"2024-09-10T21:35:45.075143+0000","last_clean":"2024-09-10T21:35:45.075143+0000","last_became_active":"2024-09-10T21:35:42.695387+0000","last_became_peered":"2024-09-10T21:35:42.695387+0000","last_unstale":"2024-09-10T21:35:45.075143+0000","last_undegraded":"2024-09-10T21:35:45.075143+0000","last_fullsized":"2024-09-10T21:35:45.075143+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:28.688577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075067+0000","last_change":"2024-09-10T21:35:43.704682+0000","last_active":"2024-09-10T21:35:45.075067+0000","last_peered":"2024-09-10T21:35:45.075067+0000","last_clean":"2024-09-10T21:35:45.075067+0000","last_became_active":"2024-09-10T21:35:43.704396+0000","last_became_peered":"2024-09-10T21:35:43.704396+0000","last_unstale":"2024-09-10T21:35:45.075067+0000","last_undegraded":"2024-09-10T21:35:45.075067+0000","last_fullsized":"2024-09-10T21:35:45.075067+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:50.258626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693160+0000","last_change":"2024-09-10T21:35:42.694971+0000","last_active":"2024-09-10T21:35:44.693160+0000","last_peered":"2024-09-10T21:35:44.693160+0000","last_clean":"2024-09-10T21:35:44.693160+0000","last_became_active":"2024-09-10T21:35:42.694748+0000","last_became_peered":"2024-09-10T21:35:42.694748+0000","last_unstale":"2024-09-10T21:35:44.693160+0000","last_undegraded":"2024-09-10T21:35:44.693160+0000","last_fullsized":"2024-09-10T21:35:44.693160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:00.243184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695083+0000","last_change":"2024-09-10T21:35:43.685624+0000","last_active":"2024-09-10T21:35:44.695083+0000","last_peered":"2024-09-10T21:35:44.695083+0000","last_clean":"2024-09-10T21:35:44.695083+0000","last_became_active":"2024-09-10T21:35:43.685331+0000","last_became_peered":"2024-09-10T21:35:43.685331+0000","last_unstale":"2024-09-10T21:35:44.695083+0000","last_undegraded":"2024-09-10T21:35:44.695083+0000","last_fullsized":"2024-09-10T21:35:44.695083+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:12.359273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880557+0000","last_change":"2024-09-10T21:35:42.697365+0000","last_active":"2024-09-10T21:35:44.880557+0000","last_peered":"2024-09-10T21:35:44.880557+0000","last_clean":"2024-09-10T21:35:44.880557+0000","last_became_active":"2024-09-10T21:35:42.693107+0000","last_became_peered":"2024-09-10T21:35:42.693107+0000","last_unstale":"2024-09-10T21:35:44.880557+0000","last_undegraded":"2024-09-10T21:35:44.880557+0000","last_fullsized":"2024-09-10T21:35:44.880557+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:15.635021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562895+0000","last_change":"2024-09-10T21:35:43.710713+0000","last_active":"2024-09-10T21:35:45.562895+0000","last_peered":"2024-09-10T21:35:45.562895+0000","last_clean":"2024-09-10T21:35:45.562895+0000","last_became_active":"2024-09-10T21:35:43.710463+0000","last_became_peered":"2024-09-10T21:35:43.710463+0000","last_unstale":"2024-09-10T21:35:45.562895+0000","last_undegraded":"2024-09-10T21:35:45.562895+0000","last_fullsized":"2024-09-10T21:35:45.562895+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:34.876598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701846+0000","last_change":"2024-09-10T21:35:42.693092+0000","last_active":"2024-09-10T21:35:42.701846+0000","last_peered":"2024-09-10T21:35:42.701846+0000","last_clean":"2024-09-10T21:35:42.701846+0000","last_became_active":"2024-09-10T21:35:42.692798+0000","last_became_peered":"2024-09-10T21:35:42.692798+0000","last_unstale":"2024-09-10T21:35:42.701846+0000","last_undegraded":"2024-09-10T21:35:42.701846+0000","last_fullsized":"2024-09-10T21:35:42.701846+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:18.074505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562873+0000","last_change":"2024-09-10T21:35:43.706761+0000","last_active":"2024-09-10T21:35:45.562873+0000","last_peered":"2024-09-10T21:35:45.562873+0000","last_clean":"2024-09-10T21:35:45.562873+0000","last_became_active":"2024-09-10T21:35:43.706277+0000","last_became_peered":"2024-09-10T21:35:43.706277+0000","last_unstale":"2024-09-10T21:35:45.562873+0000","last_undegraded":"2024-09-10T21:35:45.562873+0000","last_fullsized":"2024-09-10T21:35:45.562873+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:10.403267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075228+0000","last_change":"2024-09-10T21:35:42.698748+0000","last_active":"2024-09-10T21:35:45.075228+0000","last_peered":"2024-09-10T21:35:45.075228+0000","last_clean":"2024-09-10T21:35:45.075228+0000","last_became_active":"2024-09-10T21:35:42.698507+0000","last_became_peered":"2024-09-10T21:35:42.698507+0000","last_unstale":"2024-09-10T21:35:45.075228+0000","last_undegraded":"2024-09-10T21:35:45.075228+0000","last_fullsized":"2024-09-10T21:35:45.075228+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:02.345574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695906+0000","last_change":"2024-09-10T21:35:43.702996+0000","last_active":"2024-09-10T21:35:44.695906+0000","last_peered":"2024-09-10T21:35:44.695906+0000","last_clean":"2024-09-10T21:35:44.695906+0000","last_became_active":"2024-09-10T21:35:43.702647+0000","last_became_peered":"2024-09-10T21:35:43.702647+0000","last_unstale":"2024-09-10T21:35:44.695906+0000","last_undegraded":"2024-09-10T21:35:44.695906+0000","last_fullsized":"2024-09-10T21:35:44.695906+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:32.575627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.692613+0000","last_change":"2024-09-10T21:35:42.689212+0000","last_active":"2024-09-10T21:35:42.692613+0000","last_peered":"2024-09-10T21:35:42.692613+0000","last_clean":"2024-09-10T21:35:42.692613+0000","last_became_active":"2024-09-10T21:35:42.688903+0000","last_became_peered":"2024-09-10T21:35:42.688903+0000","last_unstale":"2024-09-10T21:35:42.692613+0000","last_undegraded":"2024-09-10T21:35:42.692613+0000","last_fullsized":"2024-09-10T21:35:42.692613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:01.158195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563808+0000","last_change":"2024-09-10T21:35:43.701294+0000","last_active":"2024-09-10T21:35:45.563808+0000","last_peered":"2024-09-10T21:35:45.563808+0000","last_clean":"2024-09-10T21:35:45.563808+0000","last_became_active":"2024-09-10T21:35:43.701016+0000","last_became_peered":"2024-09-10T21:35:43.701016+0000","last_unstale":"2024-09-10T21:35:45.563808+0000","last_undegraded":"2024-09-10T21:35:45.563808+0000","last_fullsized":"2024-09-10T21:35:45.563808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:58.553078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880497+0000","last_change":"2024-09-10T21:35:42.694284+0000","last_active":"2024-09-10T21:35:44.880497+0000","last_peered":"2024-09-10T21:35:44.880497+0000","last_clean":"2024-09-10T21:35:44.880497+0000","last_became_active":"2024-09-10T21:35:42.694026+0000","last_became_peered":"2024-09-10T21:35:42.694026+0000","last_unstale":"2024-09-10T21:35:44.880497+0000","last_undegraded":"2024-09-10T21:35:44.880497+0000","last_fullsized":"2024-09-10T21:35:44.880497+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:04.880365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075284+0000","last_change":"2024-09-10T21:35:43.699828+0000","last_active":"2024-09-10T21:35:45.075284+0000","last_peered":"2024-09-10T21:35:45.075284+0000","last_clean":"2024-09-10T21:35:45.075284+0000","last_became_active":"2024-09-10T21:35:43.699553+0000","last_became_peered":"2024-09-10T21:35:43.699553+0000","last_unstale":"2024-09-10T21:35:45.075284+0000","last_undegraded":"2024-09-10T21:35:45.075284+0000","last_fullsized":"2024-09-10T21:35:45.075284+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:24.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693437+0000","last_change":"2024-09-10T21:35:42.692258+0000","last_active":"2024-09-10T21:35:44.693437+0000","last_peered":"2024-09-10T21:35:44.693437+0000","last_clean":"2024-09-10T21:35:44.693437+0000","last_became_active":"2024-09-10T21:35:42.691757+0000","last_became_peered":"2024-09-10T21:35:42.691757+0000","last_unstale":"2024-09-10T21:35:44.693437+0000","last_undegraded":"2024-09-10T21:35:44.693437+0000","last_fullsized":"2024-09-10T21:35:44.693437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.810200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062140+0000","last_change":"2024-09-10T21:35:43.708695+0000","last_active":"2024-09-10T21:35:45.062140+0000","last_peered":"2024-09-10T21:35:45.062140+0000","last_clean":"2024-09-10T21:35:45.062140+0000","last_became_active":"2024-09-10T21:35:43.705922+0000","last_became_peered":"2024-09-10T21:35:43.705922+0000","last_unstale":"2024-09-10T21:35:45.062140+0000","last_undegraded":"2024-09-10T21:35:45.062140+0000","last_fullsized":"2024-09-10T21:35:45.062140+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:38.047565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062415+0000","last_change":"2024-09-10T21:35:42.687000+0000","last_active":"2024-09-10T21:35:45.062415+0000","last_peered":"2024-09-10T21:35:45.062415+0000","last_clean":"2024-09-10T21:35:45.062415+0000","last_became_active":"2024-09-10T21:35:42.686730+0000","last_became_peered":"2024-09-10T21:35:42.686730+0000","last_unstale":"2024-09-10T21:35:45.062415+0000","last_undegraded":"2024-09-10T21:35:45.062415+0000","last_fullsized":"2024-09-10T21:35:45.062415+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:28.386650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694749+0000","last_change":"2024-09-10T21:35:43.688538+0000","last_active":"2024-09-10T21:35:44.694749+0000","last_peered":"2024-09-10T21:35:44.694749+0000","last_clean":"2024-09-10T21:35:44.694749+0000","last_became_active":"2024-09-10T21:35:43.688259+0000","last_became_peered":"2024-09-10T21:35:43.688259+0000","last_unstale":"2024-09-10T21:35:44.694749+0000","last_undegraded":"2024-09-10T21:35:44.694749+0000","last_fullsized":"2024-09-10T21:35:44.694749+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:36.626231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692993+0000","last_change":"2024-09-10T21:35:42.692966+0000","last_active":"2024-09-10T21:35:44.692993+0000","last_peered":"2024-09-10T21:35:44.692993+0000","last_clean":"2024-09-10T21:35:44.692993+0000","last_became_active":"2024-09-10T21:35:42.691729+0000","last_became_peered":"2024-09-10T21:35:42.691729+0000","last_unstale":"2024-09-10T21:35:44.692993+0000","last_undegraded":"2024-09-10T21:35:44.692993+0000","last_fullsized":"2024-09-10T21:35:44.692993+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:10.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682184+0000","last_change":"2024-09-10T21:35:42.682184+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682184+0000","last_undegraded":"2024-09-10T21:35:42.682184+0000","last_fullsized":"2024-09-10T21:35:42.682184+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563454+0000","last_change":"2024-09-10T21:35:42.690902+0000","last_active":"2024-09-10T21:35:45.563454+0000","last_peered":"2024-09-10T21:35:45.563454+0000","last_clean":"2024-09-10T21:35:45.563454+0000","last_became_active":"2024-09-10T21:35:42.690591+0000","last_became_peered":"2024-09-10T21:35:42.690591+0000","last_unstale":"2024-09-10T21:35:45.563454+0000","last_undegraded":"2024-09-10T21:35:45.563454+0000","last_fullsized":"2024-09-10T21:35:45.563454+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:20.723287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695436+0000","last_change":"2024-09-10T21:35:43.710518+0000","last_active":"2024-09-10T21:35:44.695436+0000","last_peered":"2024-09-10T21:35:44.695436+0000","last_clean":"2024-09-10T21:35:44.695436+0000","last_became_active":"2024-09-10T21:35:43.710108+0000","last_became_peered":"2024-09-10T21:35:43.710108+0000","last_unstale":"2024-09-10T21:35:44.695436+0000","last_undegraded":"2024-09-10T21:35:44.695436+0000","last_fullsized":"2024-09-10T21:35:44.695436+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:52.921264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563242+0000","last_change":"2024-09-10T21:35:42.686431+0000","last_active":"2024-09-10T21:35:45.563242+0000","last_peered":"2024-09-10T21:35:45.563242+0000","last_clean":"2024-09-10T21:35:45.563242+0000","last_became_active":"2024-09-10T21:35:42.684143+0000","last_became_peered":"2024-09-10T21:35:42.684143+0000","last_unstale":"2024-09-10T21:35:45.563242+0000","last_undegraded":"2024-09-10T21:35:45.563242+0000","last_fullsized":"2024-09-10T21:35:45.563242+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:23.749610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695890+0000","last_change":"2024-09-10T21:35:43.691613+0000","last_active":"2024-09-10T21:35:44.695890+0000","last_peered":"2024-09-10T21:35:44.695890+0000","last_clean":"2024-09-10T21:35:44.695890+0000","last_became_active":"2024-09-10T21:35:43.691305+0000","last_became_peered":"2024-09-10T21:35:43.691305+0000","last_unstale":"2024-09-10T21:35:44.695890+0000","last_undegraded":"2024-09-10T21:35:44.695890+0000","last_fullsized":"2024-09-10T21:35:44.695890+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:45.661154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692885+0000","last_change":"2024-09-10T21:35:42.693678+0000","last_active":"2024-09-10T21:35:44.692885+0000","last_peered":"2024-09-10T21:35:44.692885+0000","last_clean":"2024-09-10T21:35:44.692885+0000","last_became_active":"2024-09-10T21:35:42.693394+0000","last_became_peered":"2024-09-10T21:35:42.693394+0000","last_unstale":"2024-09-10T21:35:44.692885+0000","last_undegraded":"2024-09-10T21:35:44.692885+0000","last_fullsized":"2024-09-10T21:35:44.692885+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:58.991101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.681938+0000","last_change":"2024-09-10T21:35:42.681938+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.681938+0000","last_undegraded":"2024-09-10T21:35:42.681938+0000","last_fullsized":"2024-09-10T21:35:42.681938+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075147+0000","last_change":"2024-09-10T21:35:42.693798+0000","last_active":"2024-09-10T21:35:45.075147+0000","last_peered":"2024-09-10T21:35:45.075147+0000","last_clean":"2024-09-10T21:35:45.075147+0000","last_became_active":"2024-09-10T21:35:42.693499+0000","last_became_peered":"2024-09-10T21:35:42.693499+0000","last_unstale":"2024-09-10T21:35:45.075147+0000","last_undegraded":"2024-09-10T21:35:45.075147+0000","last_fullsized":"2024-09-10T21:35:45.075147+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:10.502692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694617+0000","last_change":"2024-09-10T21:35:43.696424+0000","last_active":"2024-09-10T21:35:44.694617+0000","last_peered":"2024-09-10T21:35:44.694617+0000","last_clean":"2024-09-10T21:35:44.694617+0000","last_became_active":"2024-09-10T21:35:43.694483+0000","last_became_peered":"2024-09-10T21:35:43.694483+0000","last_unstale":"2024-09-10T21:35:44.694617+0000","last_undegraded":"2024-09-10T21:35:44.694617+0000","last_fullsized":"2024-09-10T21:35:44.694617+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:50.417439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563854+0000","last_change":"2024-09-10T21:35:42.695829+0000","last_active":"2024-09-10T21:35:45.563854+0000","last_peered":"2024-09-10T21:35:45.563854+0000","last_clean":"2024-09-10T21:35:45.563854+0000","last_became_active":"2024-09-10T21:35:42.695550+0000","last_became_peered":"2024-09-10T21:35:42.695550+0000","last_unstale":"2024-09-10T21:35:45.563854+0000","last_undegraded":"2024-09-10T21:35:45.563854+0000","last_fullsized":"2024-09-10T21:35:45.563854+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:29.998665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880558+0000","last_change":"2024-09-10T21:35:43.687717+0000","last_active":"2024-09-10T21:35:44.880558+0000","last_peered":"2024-09-10T21:35:44.880558+0000","last_clean":"2024-09-10T21:35:44.880558+0000","last_became_active":"2024-09-10T21:35:43.687440+0000","last_became_peered":"2024-09-10T21:35:43.687440+0000","last_unstale":"2024-09-10T21:35:44.880558+0000","last_undegraded":"2024-09-10T21:35:44.880558+0000","last_fullsized":"2024-09-10T21:35:44.880558+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:18.372062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"146'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.684833+0000","last_change":"2024-09-10T21:35:42.694598+0000","last_active":"2024-09-10T21:35:45.684833+0000","last_peered":"2024-09-10T21:35:45.684833+0000","last_clean":"2024-09-10T21:35:45.684833+0000","last_became_active":"2024-09-10T21:35:42.694267+0000","last_became_peered":"2024-09-10T21:35:42.694267+0000","last_unstale":"2024-09-10T21:35:45.684833+0000","last_undegraded":"2024-09-10T21:35:45.684833+0000","last_fullsized":"2024-09-10T21:35:45.684833+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:09.399075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075103+0000","last_change":"2024-09-10T21:35:43.708665+0000","last_active":"2024-09-10T21:35:45.075103+0000","last_peered":"2024-09-10T21:35:45.075103+0000","last_clean":"2024-09-10T21:35:45.075103+0000","last_became_active":"2024-09-10T21:35:43.707009+0000","last_became_peered":"2024-09-10T21:35:43.707009+0000","last_unstale":"2024-09-10T21:35:45.075103+0000","last_undegraded":"2024-09-10T21:35:45.075103+0000","last_fullsized":"2024-09-10T21:35:45.075103+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:41.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880692+0000","last_change":"2024-09-10T21:35:42.698430+0000","last_active":"2024-09-10T21:35:44.880692+0000","last_peered":"2024-09-10T21:35:44.880692+0000","last_clean":"2024-09-10T21:35:44.880692+0000","last_became_active":"2024-09-10T21:35:42.698195+0000","last_became_peered":"2024-09-10T21:35:42.698195+0000","last_unstale":"2024-09-10T21:35:44.880692+0000","last_undegraded":"2024-09-10T21:35:44.880692+0000","last_fullsized":"2024-09-10T21:35:44.880692+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:27.995910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062378+0000","last_change":"2024-09-10T21:35:43.697700+0000","last_active":"2024-09-10T21:35:45.062378+0000","last_peered":"2024-09-10T21:35:45.062378+0000","last_clean":"2024-09-10T21:35:45.062378+0000","last_became_active":"2024-09-10T21:35:43.694375+0000","last_became_peered":"2024-09-10T21:35:43.694375+0000","last_unstale":"2024-09-10T21:35:45.062378+0000","last_undegraded":"2024-09-10T21:35:45.062378+0000","last_fullsized":"2024-09-10T21:35:45.062378+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:49.069960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701896+0000","last_change":"2024-09-10T21:35:42.692774+0000","last_active":"2024-09-10T21:35:42.701896+0000","last_peered":"2024-09-10T21:35:42.701896+0000","last_clean":"2024-09-10T21:35:42.701896+0000","last_became_active":"2024-09-10T21:35:42.692448+0000","last_became_peered":"2024-09-10T21:35:42.692448+0000","last_unstale":"2024-09-10T21:35:42.701896+0000","last_undegraded":"2024-09-10T21:35:42.701896+0000","last_fullsized":"2024-09-10T21:35:42.701896+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:36.661466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.683005+0000","last_change":"2024-09-10T21:35:42.683005+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.683005+0000","last_undegraded":"2024-09-10T21:35:42.683005+0000","last_fullsized":"2024-09-10T21:35:42.683005+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880713+0000","last_change":"2024-09-10T21:35:42.691107+0000","last_active":"2024-09-10T21:35:44.880713+0000","last_peered":"2024-09-10T21:35:44.880713+0000","last_clean":"2024-09-10T21:35:44.880713+0000","last_became_active":"2024-09-10T21:35:42.690747+0000","last_became_peered":"2024-09-10T21:35:42.690747+0000","last_unstale":"2024-09-10T21:35:44.880713+0000","last_undegraded":"2024-09-10T21:35:44.880713+0000","last_fullsized":"2024-09-10T21:35:44.880713+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:26.876968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":144,"state":"creating+peering","last_fresh":"2024-09-10T21:35:42.682047+0000","last_change":"2024-09-10T21:35:42.682047+0000","last_active":"2024-09-10T21:35:42.661776+0000","last_peered":"2024-09-10T21:35:42.661776+0000","last_clean":"2024-09-10T21:35:42.661776+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:35:42.682047+0000","last_undegraded":"2024-09-10T21:35:42.682047+0000","last_fullsized":"2024-09-10T21:35:42.682047+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563377+0000","last_change":"2024-09-10T21:35:42.697886+0000","last_active":"2024-09-10T21:35:45.563377+0000","last_peered":"2024-09-10T21:35:45.563377+0000","last_clean":"2024-09-10T21:35:45.563377+0000","last_became_active":"2024-09-10T21:35:42.697625+0000","last_became_peered":"2024-09-10T21:35:42.697625+0000","last_unstale":"2024-09-10T21:35:45.563377+0000","last_undegraded":"2024-09-10T21:35:45.563377+0000","last_fullsized":"2024-09-10T21:35:45.563377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:23.745543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693346+0000","last_change":"2024-09-10T21:35:43.687136+0000","last_active":"2024-09-10T21:35:44.693346+0000","last_peered":"2024-09-10T21:35:44.693346+0000","last_clean":"2024-09-10T21:35:44.693346+0000","last_became_active":"2024-09-10T21:35:43.686689+0000","last_became_peered":"2024-09-10T21:35:43.686689+0000","last_unstale":"2024-09-10T21:35:44.693346+0000","last_undegraded":"2024-09-10T21:35:44.693346+0000","last_fullsized":"2024-09-10T21:35:44.693346+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:10.009857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062254+0000","last_change":"2024-09-10T21:35:42.695868+0000","last_active":"2024-09-10T21:35:45.062254+0000","last_peered":"2024-09-10T21:35:45.062254+0000","last_clean":"2024-09-10T21:35:45.062254+0000","last_became_active":"2024-09-10T21:35:42.695165+0000","last_became_peered":"2024-09-10T21:35:42.695165+0000","last_unstale":"2024-09-10T21:35:45.062254+0000","last_undegraded":"2024-09-10T21:35:45.062254+0000","last_fullsized":"2024-09-10T21:35:45.062254+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:10.502692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562882+0000","last_change":"2024-09-10T21:35:43.706228+0000","last_active":"2024-09-10T21:35:45.562882+0000","last_peered":"2024-09-10T21:35:45.562882+0000","last_clean":"2024-09-10T21:35:45.562882+0000","last_became_active":"2024-09-10T21:35:43.705968+0000","last_became_peered":"2024-09-10T21:35:43.705968+0000","last_unstale":"2024-09-10T21:35:45.562882+0000","last_undegraded":"2024-09-10T21:35:45.562882+0000","last_fullsized":"2024-09-10T21:35:45.562882+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:34.416314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563243+0000","last_change":"2024-09-10T21:35:42.697093+0000","last_active":"2024-09-10T21:35:45.563243+0000","last_peered":"2024-09-10T21:35:45.563243+0000","last_clean":"2024-09-10T21:35:45.563243+0000","last_became_active":"2024-09-10T21:35:42.696400+0000","last_became_peered":"2024-09-10T21:35:42.696400+0000","last_unstale":"2024-09-10T21:35:45.563243+0000","last_undegraded":"2024-09-10T21:35:45.563243+0000","last_fullsized":"2024-09-10T21:35:45.563243+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:41.463058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692962+0000","last_change":"2024-09-10T21:35:43.705454+0000","last_active":"2024-09-10T21:35:44.692962+0000","last_peered":"2024-09-10T21:35:44.692962+0000","last_clean":"2024-09-10T21:35:44.692962+0000","last_became_active":"2024-09-10T21:35:43.704324+0000","last_became_peered":"2024-09-10T21:35:43.704324+0000","last_unstale":"2024-09-10T21:35:44.692962+0000","last_undegraded":"2024-09-10T21:35:44.692962+0000","last_fullsized":"2024-09-10T21:35:44.692962+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:29.000102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-10T21:35:42.701861+0000","last_change":"2024-09-10T21:35:42.695716+0000","last_active":"2024-09-10T21:35:42.701861+0000","last_peered":"2024-09-10T21:35:42.701861+0000","last_clean":"2024-09-10T21:35:42.701861+0000","last_became_active":"2024-09-10T21:35:42.695410+0000","last_became_peered":"2024-09-10T21:35:42.695410+0000","last_unstale":"2024-09-10T21:35:42.701861+0000","last_undegraded":"2024-09-10T21:35:42.701861+0000","last_fullsized":"2024-09-10T21:35:42.701861+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:13.034229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":352,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.061985+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:35:45.061985+0000","last_peered":"2024-09-10T21:35:45.061985+0000","last_clean":"2024-09-10T21:35:45.061985+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:35:45.061985+0000","last_undegraded":"2024-09-10T21:35:45.061985+0000","last_fullsized":"2024-09-10T21:35:45.061985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509585,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1892352,"data_stored":953378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509583,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28884,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342956,"statfs":{"total":96636764160,"available":96607186944,"internally_reserved":0,"allocated":1896448,"data_stored":954338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509585,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33604,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338236,"statfs":{"total":96636764160,"available":96602353664,"internally_reserved":0,"allocated":2473984,"data_stored":1536526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":954338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509584,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1892352,"data_stored":952838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509585,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2486272,"data_stored":1543186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1888256,"data_stored":953308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":938867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:35:52.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:35:52.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 -- 172.21.15.97:0/695687790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0159330 msgr2=0x7f35f015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 --2- 172.21.15.97:0/695687790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0159330 0x7f35f015f490 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009900 tx=0x7f35e001c8b0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 -- 172.21.15.97:0/695687790 shutdown_connections 2024-09-10T21:35:52.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 --2- 172.21.15.97:0/695687790 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35f006db50 0x7f35f006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 --2- 172.21.15.97:0/695687790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f0159330 0x7f35f015f490 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 -- 172.21.15.97:0/695687790 >> 172.21.15.97:0/695687790 conn(0x7f35f00ccac0 msgr2=0x7f35f00cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:52.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 -- 172.21.15.97:0/695687790 shutdown_connections 2024-09-10T21:35:52.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 -- 172.21.15.97:0/695687790 wait complete. 2024-09-10T21:35:52.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 Processor -- start 2024-09-10T21:35:52.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.513+0000 7f35f6400640 1 -- start start 2024-09-10T21:35:52.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35f006db50 0x7f35f0165700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00af420 0x7f35f014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35f0159330 0x7f35f01543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0168cb0 con 0x7f35f00af420 2024-09-10T21:35:52.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0168b30 con 0x7f35f006db50 2024-09-10T21:35:52.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f35f0168e30 con 0x7f35f0159330 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00af420 0x7f35f014c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00af420 0x7f35f014c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45074/0 (socket says 172.21.15.97:45074) 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 -- 172.21.15.97:0/282664913 learned_addr learned my addr 172.21.15.97:0/282664913 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f5000640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35f006db50 0x7f35f0165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f5a00640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35f0159330 0x7f35f01543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 -- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35f0159330 msgr2=0x7f35f01543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35f0159330 0x7f35f01543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 -- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35f006db50 msgr2=0x7f35f0165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35f006db50 0x7f35f0165700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 -- 172.21.15.97:0/282664913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f014ca60 con 0x7f35f00af420 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35e7e00640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00af420 0x7f35f014c520 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c500 tx=0x7f35d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35ed600640 1 -- 172.21.15.97:0/282664913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8017070 con 0x7f35f00af420 2024-09-10T21:35:52.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35ed600640 1 -- 172.21.15.97:0/282664913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35d80053c0 con 0x7f35f00af420 2024-09-10T21:35:52.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35ed600640 1 -- 172.21.15.97:0/282664913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8012890 con 0x7f35f00af420 2024-09-10T21:35:52.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f01f86f0 con 0x7f35f00af420 2024-09-10T21:35:52.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f0164f00 con 0x7f35f00af420 2024-09-10T21:35:52.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f006df70 con 0x7f35f00af420 2024-09-10T21:35:52.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.517+0000 7f35ed600640 1 -- 172.21.15.97:0/282664913 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f35d8003600 con 0x7f35f00af420 2024-09-10T21:35:52.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.521+0000 7f35ed600640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35cc0725e0 0x7f35cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.521+0000 7f35ed600640 1 -- 172.21.15.97:0/282664913 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f35d8092990 con 0x7f35f00af420 2024-09-10T21:35:52.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.521+0000 7f35f5000640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35cc0725e0 0x7f35cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.521+0000 7f35f5000640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35cc0725e0 0x7f35cc074aa0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f35e00af000 tx=0x7f35e001b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:52.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.525+0000 7f35ed600640 1 -- 172.21.15.97:0/282664913 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f35d805d800 con 0x7f35f00af420 2024-09-10T21:35:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.685+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35f00b5fc0 con 0x7f35cc0725e0 2024-09-10T21:35:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.693+0000 7f35ed600640 1 -- 172.21.15.97:0/282664913 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252393 (secure 0 0 0) 0x7f35e00faa00 con 0x7f35cc0725e0 2024-09-10T21:35:52.694 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:52.696 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35cc0725e0 msgr2=0x7f35cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35cc0725e0 0x7f35cc074aa0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f35e00af000 tx=0x7f35e001b0f0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00af420 msgr2=0x7f35f014c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00af420 0x7f35f014c520 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c500 tx=0x7f35d800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 shutdown_connections 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f35cc0725e0 0x7f35cc074aa0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f35f0159330 0x7f35f01543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35f00af420 0x7f35f014c520 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 --2- 172.21.15.97:0/282664913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35f006db50 0x7f35f0165700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 >> 172.21.15.97:0/282664913 conn(0x7f35f00ccac0 msgr2=0x7f35f0068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:52.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 shutdown_connections 2024-09-10T21:35:52.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.697+0000 7f35f6400640 1 -- 172.21.15.97:0/282664913 wait complete. 2024-09-10T21:35:52.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-10T21:35:52.250892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282124,"kb_used_data":15944,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692596,"statfs":{"total":773094113280,"available":772805218304,"internally_reserved":0,"allocated":16326656,"data_stored":8807364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.575023"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.697134+0000","last_change":"2024-09-10T21:35:43.709410+0000","last_active":"2024-09-10T21:35:44.697134+0000","last_peered":"2024-09-10T21:35:44.697134+0000","last_clean":"2024-09-10T21:35:44.697134+0000","last_became_active":"2024-09-10T21:35:43.708591+0000","last_became_peered":"2024-09-10T21:35:43.708591+0000","last_unstale":"2024-09-10T21:35:44.697134+0000","last_undegraded":"2024-09-10T21:35:44.697134+0000","last_fullsized":"2024-09-10T21:35:44.697134+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:44.603593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562668+0000","last_change":"2024-09-10T21:35:43.704550+0000","last_active":"2024-09-10T21:35:45.562668+0000","last_peered":"2024-09-10T21:35:45.562668+0000","last_clean":"2024-09-10T21:35:45.562668+0000","last_became_active":"2024-09-10T21:35:43.704086+0000","last_became_peered":"2024-09-10T21:35:43.704086+0000","last_unstale":"2024-09-10T21:35:45.562668+0000","last_undegraded":"2024-09-10T21:35:45.562668+0000","last_fullsized":"2024-09-10T21:35:45.562668+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:14.393495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563678+0000","last_change":"2024-09-10T21:35:43.702123+0000","last_active":"2024-09-10T21:35:45.563678+0000","last_peered":"2024-09-10T21:35:45.563678+0000","last_clean":"2024-09-10T21:35:45.563678+0000","last_became_active":"2024-09-10T21:35:43.700634+0000","last_became_peered":"2024-09-10T21:35:43.700634+0000","last_unstale":"2024-09-10T21:35:45.563678+0000","last_undegraded":"2024-09-10T21:35:45.563678+0000","last_fullsized":"2024-09-10T21:35:45.563678+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:13.113908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696155+0000","last_change":"2024-09-10T21:35:43.709341+0000","last_active":"2024-09-10T21:35:44.696155+0000","last_peered":"2024-09-10T21:35:44.696155+0000","last_clean":"2024-09-10T21:35:44.696155+0000","last_became_active":"2024-09-10T21:35:43.707803+0000","last_became_peered":"2024-09-10T21:35:43.707803+0000","last_unstale":"2024-09-10T21:35:44.696155+0000","last_undegraded":"2024-09-10T21:35:44.696155+0000","last_fullsized":"2024-09-10T21:35:44.696155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:20.676077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696319+0000","last_change":"2024-09-10T21:35:43.703401+0000","last_active":"2024-09-10T21:35:44.696319+0000","last_peered":"2024-09-10T21:35:44.696319+0000","last_clean":"2024-09-10T21:35:44.696319+0000","last_became_active":"2024-09-10T21:35:43.703135+0000","last_became_peered":"2024-09-10T21:35:43.703135+0000","last_unstale":"2024-09-10T21:35:44.696319+0000","last_undegraded":"2024-09-10T21:35:44.696319+0000","last_fullsized":"2024-09-10T21:35:44.696319+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:28.444330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692726+0000","last_change":"2024-09-10T21:35:43.710080+0000","last_active":"2024-09-10T21:35:44.692726+0000","last_peered":"2024-09-10T21:35:44.692726+0000","last_clean":"2024-09-10T21:35:44.692726+0000","last_became_active":"2024-09-10T21:35:43.709693+0000","last_became_peered":"2024-09-10T21:35:43.709693+0000","last_unstale":"2024-09-10T21:35:44.692726+0000","last_undegraded":"2024-09-10T21:35:44.692726+0000","last_fullsized":"2024-09-10T21:35:44.692726+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:38.047565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692510+0000","last_change":"2024-09-10T21:35:43.711852+0000","last_active":"2024-09-10T21:35:44.692510+0000","last_peered":"2024-09-10T21:35:44.692510+0000","last_clean":"2024-09-10T21:35:44.692510+0000","last_became_active":"2024-09-10T21:35:43.711654+0000","last_became_peered":"2024-09-10T21:35:43.711654+0000","last_unstale":"2024-09-10T21:35:44.692510+0000","last_undegraded":"2024-09-10T21:35:44.692510+0000","last_fullsized":"2024-09-10T21:35:44.692510+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:12.060947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075008+0000","last_change":"2024-09-10T21:35:43.713260+0000","last_active":"2024-09-10T21:35:45.075008+0000","last_peered":"2024-09-10T21:35:45.075008+0000","last_clean":"2024-09-10T21:35:45.075008+0000","last_became_active":"2024-09-10T21:35:43.712978+0000","last_became_peered":"2024-09-10T21:35:43.712978+0000","last_unstale":"2024-09-10T21:35:45.075008+0000","last_undegraded":"2024-09-10T21:35:45.075008+0000","last_fullsized":"2024-09-10T21:35:45.075008+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:36.100932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562758+0000","last_change":"2024-09-10T21:35:43.711304+0000","last_active":"2024-09-10T21:35:45.562758+0000","last_peered":"2024-09-10T21:35:45.562758+0000","last_clean":"2024-09-10T21:35:45.562758+0000","last_became_active":"2024-09-10T21:35:43.711066+0000","last_became_peered":"2024-09-10T21:35:43.711066+0000","last_unstale":"2024-09-10T21:35:45.562758+0000","last_undegraded":"2024-09-10T21:35:45.562758+0000","last_fullsized":"2024-09-10T21:35:45.562758+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:32.539648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075220+0000","last_change":"2024-09-10T21:35:43.708804+0000","last_active":"2024-09-10T21:35:45.075220+0000","last_peered":"2024-09-10T21:35:45.075220+0000","last_clean":"2024-09-10T21:35:45.075220+0000","last_became_active":"2024-09-10T21:35:43.708489+0000","last_became_peered":"2024-09-10T21:35:43.708489+0000","last_unstale":"2024-09-10T21:35:45.075220+0000","last_undegraded":"2024-09-10T21:35:45.075220+0000","last_fullsized":"2024-09-10T21:35:45.075220+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:47.091344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880679+0000","last_change":"2024-09-10T21:35:43.693534+0000","last_active":"2024-09-10T21:35:44.880679+0000","last_peered":"2024-09-10T21:35:44.880679+0000","last_clean":"2024-09-10T21:35:44.880679+0000","last_became_active":"2024-09-10T21:35:43.693285+0000","last_became_peered":"2024-09-10T21:35:43.693285+0000","last_unstale":"2024-09-10T21:35:44.880679+0000","last_undegraded":"2024-09-10T21:35:44.880679+0000","last_fullsized":"2024-09-10T21:35:44.880679+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:18.734510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693052+0000","last_change":"2024-09-10T21:35:43.689168+0000","last_active":"2024-09-10T21:35:44.693052+0000","last_peered":"2024-09-10T21:35:44.693052+0000","last_clean":"2024-09-10T21:35:44.693052+0000","last_became_active":"2024-09-10T21:35:43.688873+0000","last_became_peered":"2024-09-10T21:35:43.688873+0000","last_unstale":"2024-09-10T21:35:44.693052+0000","last_undegraded":"2024-09-10T21:35:44.693052+0000","last_fullsized":"2024-09-10T21:35:44.693052+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:16.639213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696087+0000","last_change":"2024-09-10T21:35:43.711130+0000","last_active":"2024-09-10T21:35:44.696087+0000","last_peered":"2024-09-10T21:35:44.696087+0000","last_clean":"2024-09-10T21:35:44.696087+0000","last_became_active":"2024-09-10T21:35:43.710876+0000","last_became_peered":"2024-09-10T21:35:43.710876+0000","last_unstale":"2024-09-10T21:35:44.696087+0000","last_undegraded":"2024-09-10T21:35:44.696087+0000","last_fullsized":"2024-09-10T21:35:44.696087+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:35.279264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692849+0000","last_change":"2024-09-10T21:35:43.689434+0000","last_active":"2024-09-10T21:35:44.692849+0000","last_peered":"2024-09-10T21:35:44.692849+0000","last_clean":"2024-09-10T21:35:44.692849+0000","last_became_active":"2024-09-10T21:35:43.689163+0000","last_became_peered":"2024-09-10T21:35:43.689163+0000","last_unstale":"2024-09-10T21:35:44.692849+0000","last_undegraded":"2024-09-10T21:35:44.692849+0000","last_fullsized":"2024-09-10T21:35:44.692849+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:21.413539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562494+0000","last_change":"2024-09-10T21:35:43.698275+0000","last_active":"2024-09-10T21:35:45.562494+0000","last_peered":"2024-09-10T21:35:45.562494+0000","last_clean":"2024-09-10T21:35:45.562494+0000","last_became_active":"2024-09-10T21:35:43.698023+0000","last_became_peered":"2024-09-10T21:35:43.698023+0000","last_unstale":"2024-09-10T21:35:45.562494+0000","last_undegraded":"2024-09-10T21:35:45.562494+0000","last_fullsized":"2024-09-10T21:35:45.562494+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:37.116752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062129+0000","last_change":"2024-09-10T21:35:43.702042+0000","last_active":"2024-09-10T21:35:45.062129+0000","last_peered":"2024-09-10T21:35:45.062129+0000","last_clean":"2024-09-10T21:35:45.062129+0000","last_became_active":"2024-09-10T21:35:43.701794+0000","last_became_peered":"2024-09-10T21:35:43.701794+0000","last_unstale":"2024-09-10T21:35:45.062129+0000","last_undegraded":"2024-09-10T21:35:45.062129+0000","last_fullsized":"2024-09-10T21:35:45.062129+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:42.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563696+0000","last_change":"2024-09-10T21:35:43.706781+0000","last_active":"2024-09-10T21:35:45.563696+0000","last_peered":"2024-09-10T21:35:45.563696+0000","last_clean":"2024-09-10T21:35:45.563696+0000","last_became_active":"2024-09-10T21:35:43.706511+0000","last_became_peered":"2024-09-10T21:35:43.706511+0000","last_unstale":"2024-09-10T21:35:45.563696+0000","last_undegraded":"2024-09-10T21:35:45.563696+0000","last_fullsized":"2024-09-10T21:35:45.563696+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:56.867984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562875+0000","last_change":"2024-09-10T21:35:43.710248+0000","last_active":"2024-09-10T21:35:45.562875+0000","last_peered":"2024-09-10T21:35:45.562875+0000","last_clean":"2024-09-10T21:35:45.562875+0000","last_became_active":"2024-09-10T21:35:43.710000+0000","last_became_peered":"2024-09-10T21:35:43.710000+0000","last_unstale":"2024-09-10T21:35:45.562875+0000","last_undegraded":"2024-09-10T21:35:45.562875+0000","last_fullsized":"2024-09-10T21:35:45.562875+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:24.749734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696793+0000","last_change":"2024-09-10T21:35:43.712690+0000","last_active":"2024-09-10T21:35:44.696793+0000","last_peered":"2024-09-10T21:35:44.696793+0000","last_clean":"2024-09-10T21:35:44.696793+0000","last_became_active":"2024-09-10T21:35:43.712391+0000","last_became_peered":"2024-09-10T21:35:43.712391+0000","last_unstale":"2024-09-10T21:35:44.696793+0000","last_undegraded":"2024-09-10T21:35:44.696793+0000","last_fullsized":"2024-09-10T21:35:44.696793+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:00.178908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062438+0000","last_change":"2024-09-10T21:35:43.695973+0000","last_active":"2024-09-10T21:35:45.062438+0000","last_peered":"2024-09-10T21:35:45.062438+0000","last_clean":"2024-09-10T21:35:45.062438+0000","last_became_active":"2024-09-10T21:35:43.695709+0000","last_became_peered":"2024-09-10T21:35:43.695709+0000","last_unstale":"2024-09-10T21:35:45.062438+0000","last_undegraded":"2024-09-10T21:35:45.062438+0000","last_fullsized":"2024-09-10T21:35:45.062438+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:21.413539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563488+0000","last_change":"2024-09-10T21:35:43.695495+0000","last_active":"2024-09-10T21:35:45.563488+0000","last_peered":"2024-09-10T21:35:45.563488+0000","last_clean":"2024-09-10T21:35:45.563488+0000","last_became_active":"2024-09-10T21:35:43.695095+0000","last_became_peered":"2024-09-10T21:35:43.695095+0000","last_unstale":"2024-09-10T21:35:45.563488+0000","last_undegraded":"2024-09-10T21:35:45.563488+0000","last_fullsized":"2024-09-10T21:35:45.563488+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:54.813769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694816+0000","last_change":"2024-09-10T21:35:43.696103+0000","last_active":"2024-09-10T21:35:44.694816+0000","last_peered":"2024-09-10T21:35:44.694816+0000","last_clean":"2024-09-10T21:35:44.694816+0000","last_became_active":"2024-09-10T21:35:43.695633+0000","last_became_peered":"2024-09-10T21:35:43.695633+0000","last_unstale":"2024-09-10T21:35:44.694816+0000","last_undegraded":"2024-09-10T21:35:44.694816+0000","last_fullsized":"2024-09-10T21:35:44.694816+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:56.055893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074781+0000","last_change":"2024-09-10T21:35:43.699096+0000","last_active":"2024-09-10T21:35:45.074781+0000","last_peered":"2024-09-10T21:35:45.074781+0000","last_clean":"2024-09-10T21:35:45.074781+0000","last_became_active":"2024-09-10T21:35:43.698826+0000","last_became_peered":"2024-09-10T21:35:43.698826+0000","last_unstale":"2024-09-10T21:35:45.074781+0000","last_undegraded":"2024-09-10T21:35:45.074781+0000","last_fullsized":"2024-09-10T21:35:45.074781+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:31.736444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074767+0000","last_change":"2024-09-10T21:35:43.710788+0000","last_active":"2024-09-10T21:35:45.074767+0000","last_peered":"2024-09-10T21:35:45.074767+0000","last_clean":"2024-09-10T21:35:45.074767+0000","last_became_active":"2024-09-10T21:35:43.710492+0000","last_became_peered":"2024-09-10T21:35:43.710492+0000","last_unstale":"2024-09-10T21:35:45.074767+0000","last_undegraded":"2024-09-10T21:35:45.074767+0000","last_fullsized":"2024-09-10T21:35:45.074767+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:09.641780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062559+0000","last_change":"2024-09-10T21:35:43.695952+0000","last_active":"2024-09-10T21:35:45.062559+0000","last_peered":"2024-09-10T21:35:45.062559+0000","last_clean":"2024-09-10T21:35:45.062559+0000","last_became_active":"2024-09-10T21:35:43.694886+0000","last_became_peered":"2024-09-10T21:35:43.694886+0000","last_unstale":"2024-09-10T21:35:45.062559+0000","last_undegraded":"2024-09-10T21:35:45.062559+0000","last_fullsized":"2024-09-10T21:35:45.062559+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:41.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062558+0000","last_change":"2024-09-10T21:35:43.691902+0000","last_active":"2024-09-10T21:35:45.062558+0000","last_peered":"2024-09-10T21:35:45.062558+0000","last_clean":"2024-09-10T21:35:45.062558+0000","last_became_active":"2024-09-10T21:35:43.691638+0000","last_became_peered":"2024-09-10T21:35:43.691638+0000","last_unstale":"2024-09-10T21:35:45.062558+0000","last_undegraded":"2024-09-10T21:35:45.062558+0000","last_fullsized":"2024-09-10T21:35:45.062558+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:16.639213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062518+0000","last_change":"2024-09-10T21:35:43.696465+0000","last_active":"2024-09-10T21:35:45.062518+0000","last_peered":"2024-09-10T21:35:45.062518+0000","last_clean":"2024-09-10T21:35:45.062518+0000","last_became_active":"2024-09-10T21:35:43.695584+0000","last_became_peered":"2024-09-10T21:35:43.695584+0000","last_unstale":"2024-09-10T21:35:45.062518+0000","last_undegraded":"2024-09-10T21:35:45.062518+0000","last_fullsized":"2024-09-10T21:35:45.062518+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:29.000102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692740+0000","last_change":"2024-09-10T21:35:43.686570+0000","last_active":"2024-09-10T21:35:44.692740+0000","last_peered":"2024-09-10T21:35:44.692740+0000","last_clean":"2024-09-10T21:35:44.692740+0000","last_became_active":"2024-09-10T21:35:43.686287+0000","last_became_peered":"2024-09-10T21:35:43.686287+0000","last_unstale":"2024-09-10T21:35:44.692740+0000","last_undegraded":"2024-09-10T21:35:44.692740+0000","last_fullsized":"2024-09-10T21:35:44.692740+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:49.069960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074985+0000","last_change":"2024-09-10T21:35:43.697251+0000","last_active":"2024-09-10T21:35:45.074985+0000","last_peered":"2024-09-10T21:35:45.074985+0000","last_clean":"2024-09-10T21:35:45.074985+0000","last_became_active":"2024-09-10T21:35:43.696685+0000","last_became_peered":"2024-09-10T21:35:43.696685+0000","last_unstale":"2024-09-10T21:35:45.074985+0000","last_undegraded":"2024-09-10T21:35:45.074985+0000","last_fullsized":"2024-09-10T21:35:45.074985+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:19.847487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880695+0000","last_change":"2024-09-10T21:35:43.695764+0000","last_active":"2024-09-10T21:35:44.880695+0000","last_peered":"2024-09-10T21:35:44.880695+0000","last_clean":"2024-09-10T21:35:44.880695+0000","last_became_active":"2024-09-10T21:35:43.695460+0000","last_became_peered":"2024-09-10T21:35:43.695460+0000","last_unstale":"2024-09-10T21:35:44.880695+0000","last_undegraded":"2024-09-10T21:35:44.880695+0000","last_fullsized":"2024-09-10T21:35:44.880695+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:05.693752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692686+0000","last_change":"2024-09-10T21:35:43.702866+0000","last_active":"2024-09-10T21:35:44.692686+0000","last_peered":"2024-09-10T21:35:44.692686+0000","last_clean":"2024-09-10T21:35:44.692686+0000","last_became_active":"2024-09-10T21:35:43.702624+0000","last_became_peered":"2024-09-10T21:35:43.702624+0000","last_unstale":"2024-09-10T21:35:44.692686+0000","last_undegraded":"2024-09-10T21:35:44.692686+0000","last_fullsized":"2024-09-10T21:35:44.692686+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:42.139950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694552+0000","last_change":"2024-09-10T21:35:43.691043+0000","last_active":"2024-09-10T21:35:44.694552+0000","last_peered":"2024-09-10T21:35:44.694552+0000","last_clean":"2024-09-10T21:35:44.694552+0000","last_became_active":"2024-09-10T21:35:43.690768+0000","last_became_peered":"2024-09-10T21:35:43.690768+0000","last_unstale":"2024-09-10T21:35:44.694552+0000","last_undegraded":"2024-09-10T21:35:44.694552+0000","last_fullsized":"2024-09-10T21:35:44.694552+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:59.974397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563300+0000","last_change":"2024-09-10T21:35:43.690260+0000","last_active":"2024-09-10T21:35:45.563300+0000","last_peered":"2024-09-10T21:35:45.563300+0000","last_clean":"2024-09-10T21:35:45.563300+0000","last_became_active":"2024-09-10T21:35:43.690001+0000","last_became_peered":"2024-09-10T21:35:43.690001+0000","last_unstale":"2024-09-10T21:35:45.563300+0000","last_undegraded":"2024-09-10T21:35:45.563300+0000","last_fullsized":"2024-09-10T21:35:45.563300+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:11.506883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696123+0000","last_change":"2024-09-10T21:35:42.684988+0000","last_active":"2024-09-10T21:35:44.696123+0000","last_peered":"2024-09-10T21:35:44.696123+0000","last_clean":"2024-09-10T21:35:44.696123+0000","last_became_active":"2024-09-10T21:35:42.684730+0000","last_became_peered":"2024-09-10T21:35:42.684730+0000","last_unstale":"2024-09-10T21:35:44.696123+0000","last_undegraded":"2024-09-10T21:35:44.696123+0000","last_fullsized":"2024-09-10T21:35:44.696123+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:22.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074844+0000","last_change":"2024-09-10T21:35:43.697903+0000","last_active":"2024-09-10T21:35:45.074844+0000","last_peered":"2024-09-10T21:35:45.074844+0000","last_clean":"2024-09-10T21:35:45.074844+0000","last_became_active":"2024-09-10T21:35:43.697647+0000","last_became_peered":"2024-09-10T21:35:43.697647+0000","last_unstale":"2024-09-10T21:35:45.074844+0000","last_undegraded":"2024-09-10T21:35:45.074844+0000","last_fullsized":"2024-09-10T21:35:45.074844+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:02.827939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563603+0000","last_change":"2024-09-10T21:35:42.683390+0000","last_active":"2024-09-10T21:35:45.563603+0000","last_peered":"2024-09-10T21:35:45.563603+0000","last_clean":"2024-09-10T21:35:45.563603+0000","last_became_active":"2024-09-10T21:35:42.683095+0000","last_became_peered":"2024-09-10T21:35:42.683095+0000","last_unstale":"2024-09-10T21:35:45.563603+0000","last_undegraded":"2024-09-10T21:35:45.563603+0000","last_fullsized":"2024-09-10T21:35:45.563603+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:08.275738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562597+0000","last_change":"2024-09-10T21:35:43.704762+0000","last_active":"2024-09-10T21:35:45.562597+0000","last_peered":"2024-09-10T21:35:45.562597+0000","last_clean":"2024-09-10T21:35:45.562597+0000","last_became_active":"2024-09-10T21:35:43.704137+0000","last_became_peered":"2024-09-10T21:35:43.704137+0000","last_unstale":"2024-09-10T21:35:45.562597+0000","last_undegraded":"2024-09-10T21:35:45.562597+0000","last_fullsized":"2024-09-10T21:35:45.562597+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:11.824013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562772+0000","last_change":"2024-09-10T21:35:42.693628+0000","last_active":"2024-09-10T21:35:45.562772+0000","last_peered":"2024-09-10T21:35:45.562772+0000","last_clean":"2024-09-10T21:35:45.562772+0000","last_became_active":"2024-09-10T21:35:42.692980+0000","last_became_peered":"2024-09-10T21:35:42.692980+0000","last_unstale":"2024-09-10T21:35:45.562772+0000","last_undegraded":"2024-09-10T21:35:45.562772+0000","last_fullsized":"2024-09-10T21:35:45.562772+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:25.656976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880707+0000","last_change":"2024-09-10T21:35:43.698119+0000","last_active":"2024-09-10T21:35:44.880707+0000","last_peered":"2024-09-10T21:35:44.880707+0000","last_clean":"2024-09-10T21:35:44.880707+0000","last_became_active":"2024-09-10T21:35:43.697892+0000","last_became_peered":"2024-09-10T21:35:43.697892+0000","last_unstale":"2024-09-10T21:35:44.880707+0000","last_undegraded":"2024-09-10T21:35:44.880707+0000","last_fullsized":"2024-09-10T21:35:44.880707+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:13.854892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.074953+0000","last_change":"2024-09-10T21:35:42.697366+0000","last_active":"2024-09-10T21:35:45.074953+0000","last_peered":"2024-09-10T21:35:45.074953+0000","last_clean":"2024-09-10T21:35:45.074953+0000","last_became_active":"2024-09-10T21:35:42.697095+0000","last_became_peered":"2024-09-10T21:35:42.697095+0000","last_unstale":"2024-09-10T21:35:45.074953+0000","last_undegraded":"2024-09-10T21:35:45.074953+0000","last_fullsized":"2024-09-10T21:35:45.074953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:23.457506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696555+0000","last_change":"2024-09-10T21:35:43.708047+0000","last_active":"2024-09-10T21:35:44.696555+0000","last_peered":"2024-09-10T21:35:44.696555+0000","last_clean":"2024-09-10T21:35:44.696555+0000","last_became_active":"2024-09-10T21:35:43.707396+0000","last_became_peered":"2024-09-10T21:35:43.707396+0000","last_unstale":"2024-09-10T21:35:44.696555+0000","last_undegraded":"2024-09-10T21:35:44.696555+0000","last_fullsized":"2024-09-10T21:35:44.696555+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:22.794213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880614+0000","last_change":"2024-09-10T21:35:42.695787+0000","last_active":"2024-09-10T21:35:44.880614+0000","last_peered":"2024-09-10T21:35:44.880614+0000","last_clean":"2024-09-10T21:35:44.880614+0000","last_became_active":"2024-09-10T21:35:42.695491+0000","last_became_peered":"2024-09-10T21:35:42.695491+0000","last_unstale":"2024-09-10T21:35:44.880614+0000","last_undegraded":"2024-09-10T21:35:44.880614+0000","last_fullsized":"2024-09-10T21:35:44.880614+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:09.005666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.697020+0000","last_change":"2024-09-10T21:35:43.704575+0000","last_active":"2024-09-10T21:35:44.697020+0000","last_peered":"2024-09-10T21:35:44.697020+0000","last_clean":"2024-09-10T21:35:44.697020+0000","last_became_active":"2024-09-10T21:35:43.704285+0000","last_became_peered":"2024-09-10T21:35:43.704285+0000","last_unstale":"2024-09-10T21:35:44.697020+0000","last_undegraded":"2024-09-10T21:35:44.697020+0000","last_fullsized":"2024-09-10T21:35:44.697020+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:17.563856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062193+0000","last_change":"2024-09-10T21:35:42.694006+0000","last_active":"2024-09-10T21:35:45.062193+0000","last_peered":"2024-09-10T21:35:45.062193+0000","last_clean":"2024-09-10T21:35:45.062193+0000","last_became_active":"2024-09-10T21:35:42.693763+0000","last_became_peered":"2024-09-10T21:35:42.693763+0000","last_unstale":"2024-09-10T21:35:45.062193+0000","last_undegraded":"2024-09-10T21:35:45.062193+0000","last_fullsized":"2024-09-10T21:35:45.062193+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:01.408836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880823+0000","last_change":"2024-09-10T21:35:43.691603+0000","last_active":"2024-09-10T21:35:44.880823+0000","last_peered":"2024-09-10T21:35:44.880823+0000","last_clean":"2024-09-10T21:35:44.880823+0000","last_became_active":"2024-09-10T21:35:43.691298+0000","last_became_peered":"2024-09-10T21:35:43.691298+0000","last_unstale":"2024-09-10T21:35:44.880823+0000","last_undegraded":"2024-09-10T21:35:44.880823+0000","last_fullsized":"2024-09-10T21:35:44.880823+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:17.652436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062225+0000","last_change":"2024-09-10T21:35:42.689996+0000","last_active":"2024-09-10T21:35:45.062225+0000","last_peered":"2024-09-10T21:35:45.062225+0000","last_clean":"2024-09-10T21:35:45.062225+0000","last_became_active":"2024-09-10T21:35:42.689693+0000","last_became_peered":"2024-09-10T21:35:42.689693+0000","last_unstale":"2024-09-10T21:35:45.062225+0000","last_undegraded":"2024-09-10T21:35:45.062225+0000","last_fullsized":"2024-09-10T21:35:45.062225+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.810200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.697310+0000","last_change":"2024-09-10T21:35:43.706646+0000","last_active":"2024-09-10T21:35:44.697310+0000","last_peered":"2024-09-10T21:35:44.697310+0000","last_clean":"2024-09-10T21:35:44.697310+0000","last_became_active":"2024-09-10T21:35:43.706305+0000","last_became_peered":"2024-09-10T21:35:43.706305+0000","last_unstale":"2024-09-10T21:35:44.697310+0000","last_undegraded":"2024-09-10T21:35:44.697310+0000","last_fullsized":"2024-09-10T21:35:44.697310+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:24.474370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694720+0000","last_change":"2024-09-10T21:35:42.690501+0000","last_active":"2024-09-10T21:35:44.694720+0000","last_peered":"2024-09-10T21:35:44.694720+0000","last_clean":"2024-09-10T21:35:44.694720+0000","last_became_active":"2024-09-10T21:35:42.690137+0000","last_became_peered":"2024-09-10T21:35:42.690137+0000","last_unstale":"2024-09-10T21:35:44.694720+0000","last_undegraded":"2024-09-10T21:35:44.694720+0000","last_fullsized":"2024-09-10T21:35:44.694720+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:05.467377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694729+0000","last_change":"2024-09-10T21:35:43.698840+0000","last_active":"2024-09-10T21:35:44.694729+0000","last_peered":"2024-09-10T21:35:44.694729+0000","last_clean":"2024-09-10T21:35:44.694729+0000","last_became_active":"2024-09-10T21:35:43.698440+0000","last_became_peered":"2024-09-10T21:35:43.698440+0000","last_unstale":"2024-09-10T21:35:44.694729+0000","last_undegraded":"2024-09-10T21:35:44.694729+0000","last_fullsized":"2024-09-10T21:35:44.694729+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:23.396988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693373+0000","last_change":"2024-09-10T21:35:42.698961+0000","last_active":"2024-09-10T21:35:44.693373+0000","last_peered":"2024-09-10T21:35:44.693373+0000","last_clean":"2024-09-10T21:35:44.693373+0000","last_became_active":"2024-09-10T21:35:42.697755+0000","last_became_peered":"2024-09-10T21:35:42.697755+0000","last_unstale":"2024-09-10T21:35:44.693373+0000","last_undegraded":"2024-09-10T21:35:44.693373+0000","last_fullsized":"2024-09-10T21:35:44.693373+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:22.195866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695346+0000","last_change":"2024-09-10T21:35:43.714759+0000","last_active":"2024-09-10T21:35:44.695346+0000","last_peered":"2024-09-10T21:35:44.695346+0000","last_clean":"2024-09-10T21:35:44.695346+0000","last_became_active":"2024-09-10T21:35:43.714450+0000","last_became_peered":"2024-09-10T21:35:43.714450+0000","last_unstale":"2024-09-10T21:35:44.695346+0000","last_undegraded":"2024-09-10T21:35:44.695346+0000","last_fullsized":"2024-09-10T21:35:44.695346+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:06.356554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075143+0000","last_change":"2024-09-10T21:35:42.695674+0000","last_active":"2024-09-10T21:35:45.075143+0000","last_peered":"2024-09-10T21:35:45.075143+0000","last_clean":"2024-09-10T21:35:45.075143+0000","last_became_active":"2024-09-10T21:35:42.695387+0000","last_became_peered":"2024-09-10T21:35:42.695387+0000","last_unstale":"2024-09-10T21:35:45.075143+0000","last_undegraded":"2024-09-10T21:35:45.075143+0000","last_fullsized":"2024-09-10T21:35:45.075143+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:28.688577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075067+0000","last_change":"2024-09-10T21:35:43.704682+0000","last_active":"2024-09-10T21:35:45.075067+0000","last_peered":"2024-09-10T21:35:45.075067+0000","last_clean":"2024-09-10T21:35:45.075067+0000","last_became_active":"2024-09-10T21:35:43.704396+0000","last_became_peered":"2024-09-10T21:35:43.704396+0000","last_unstale":"2024-09-10T21:35:45.075067+0000","last_undegraded":"2024-09-10T21:35:45.075067+0000","last_fullsized":"2024-09-10T21:35:45.075067+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:50.258626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693160+0000","last_change":"2024-09-10T21:35:42.694971+0000","last_active":"2024-09-10T21:35:44.693160+0000","last_peered":"2024-09-10T21:35:44.693160+0000","last_clean":"2024-09-10T21:35:44.693160+0000","last_became_active":"2024-09-10T21:35:42.694748+0000","last_became_peered":"2024-09-10T21:35:42.694748+0000","last_unstale":"2024-09-10T21:35:44.693160+0000","last_undegraded":"2024-09-10T21:35:44.693160+0000","last_fullsized":"2024-09-10T21:35:44.693160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:00.243184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695083+0000","last_change":"2024-09-10T21:35:43.685624+0000","last_active":"2024-09-10T21:35:44.695083+0000","last_peered":"2024-09-10T21:35:44.695083+0000","last_clean":"2024-09-10T21:35:44.695083+0000","last_became_active":"2024-09-10T21:35:43.685331+0000","last_became_peered":"2024-09-10T21:35:43.685331+0000","last_unstale":"2024-09-10T21:35:44.695083+0000","last_undegraded":"2024-09-10T21:35:44.695083+0000","last_fullsized":"2024-09-10T21:35:44.695083+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:12.359273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880557+0000","last_change":"2024-09-10T21:35:42.697365+0000","last_active":"2024-09-10T21:35:44.880557+0000","last_peered":"2024-09-10T21:35:44.880557+0000","last_clean":"2024-09-10T21:35:44.880557+0000","last_became_active":"2024-09-10T21:35:42.693107+0000","last_became_peered":"2024-09-10T21:35:42.693107+0000","last_unstale":"2024-09-10T21:35:44.880557+0000","last_undegraded":"2024-09-10T21:35:44.880557+0000","last_fullsized":"2024-09-10T21:35:44.880557+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:15.635021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562895+0000","last_change":"2024-09-10T21:35:43.710713+0000","last_active":"2024-09-10T21:35:45.562895+0000","last_peered":"2024-09-10T21:35:45.562895+0000","last_clean":"2024-09-10T21:35:45.562895+0000","last_became_active":"2024-09-10T21:35:43.710463+0000","last_became_peered":"2024-09-10T21:35:43.710463+0000","last_unstale":"2024-09-10T21:35:45.562895+0000","last_undegraded":"2024-09-10T21:35:45.562895+0000","last_fullsized":"2024-09-10T21:35:45.562895+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:34.876598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696606+0000","last_change":"2024-09-10T21:35:42.693092+0000","last_active":"2024-09-10T21:35:44.696606+0000","last_peered":"2024-09-10T21:35:44.696606+0000","last_clean":"2024-09-10T21:35:44.696606+0000","last_became_active":"2024-09-10T21:35:42.692798+0000","last_became_peered":"2024-09-10T21:35:42.692798+0000","last_unstale":"2024-09-10T21:35:44.696606+0000","last_undegraded":"2024-09-10T21:35:44.696606+0000","last_fullsized":"2024-09-10T21:35:44.696606+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:15.290226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562873+0000","last_change":"2024-09-10T21:35:43.706761+0000","last_active":"2024-09-10T21:35:45.562873+0000","last_peered":"2024-09-10T21:35:45.562873+0000","last_clean":"2024-09-10T21:35:45.562873+0000","last_became_active":"2024-09-10T21:35:43.706277+0000","last_became_peered":"2024-09-10T21:35:43.706277+0000","last_unstale":"2024-09-10T21:35:45.562873+0000","last_undegraded":"2024-09-10T21:35:45.562873+0000","last_fullsized":"2024-09-10T21:35:45.562873+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:10.403267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075228+0000","last_change":"2024-09-10T21:35:42.698748+0000","last_active":"2024-09-10T21:35:45.075228+0000","last_peered":"2024-09-10T21:35:45.075228+0000","last_clean":"2024-09-10T21:35:45.075228+0000","last_became_active":"2024-09-10T21:35:42.698507+0000","last_became_peered":"2024-09-10T21:35:42.698507+0000","last_unstale":"2024-09-10T21:35:45.075228+0000","last_undegraded":"2024-09-10T21:35:45.075228+0000","last_fullsized":"2024-09-10T21:35:45.075228+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:02.345574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695906+0000","last_change":"2024-09-10T21:35:43.702996+0000","last_active":"2024-09-10T21:35:44.695906+0000","last_peered":"2024-09-10T21:35:44.695906+0000","last_clean":"2024-09-10T21:35:44.695906+0000","last_became_active":"2024-09-10T21:35:43.702647+0000","last_became_peered":"2024-09-10T21:35:43.702647+0000","last_unstale":"2024-09-10T21:35:44.695906+0000","last_undegraded":"2024-09-10T21:35:44.695906+0000","last_fullsized":"2024-09-10T21:35:44.695906+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:32.575627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695465+0000","last_change":"2024-09-10T21:35:42.689212+0000","last_active":"2024-09-10T21:35:44.695465+0000","last_peered":"2024-09-10T21:35:44.695465+0000","last_clean":"2024-09-10T21:35:44.695465+0000","last_became_active":"2024-09-10T21:35:42.688903+0000","last_became_peered":"2024-09-10T21:35:42.688903+0000","last_unstale":"2024-09-10T21:35:44.695465+0000","last_undegraded":"2024-09-10T21:35:44.695465+0000","last_fullsized":"2024-09-10T21:35:44.695465+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:56.782389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563808+0000","last_change":"2024-09-10T21:35:43.701294+0000","last_active":"2024-09-10T21:35:45.563808+0000","last_peered":"2024-09-10T21:35:45.563808+0000","last_clean":"2024-09-10T21:35:45.563808+0000","last_became_active":"2024-09-10T21:35:43.701016+0000","last_became_peered":"2024-09-10T21:35:43.701016+0000","last_unstale":"2024-09-10T21:35:45.563808+0000","last_undegraded":"2024-09-10T21:35:45.563808+0000","last_fullsized":"2024-09-10T21:35:45.563808+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:58.553078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"145'8","reported_seq":23,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880497+0000","last_change":"2024-09-10T21:35:42.694284+0000","last_active":"2024-09-10T21:35:44.880497+0000","last_peered":"2024-09-10T21:35:44.880497+0000","last_clean":"2024-09-10T21:35:44.880497+0000","last_became_active":"2024-09-10T21:35:42.694026+0000","last_became_peered":"2024-09-10T21:35:42.694026+0000","last_unstale":"2024-09-10T21:35:44.880497+0000","last_undegraded":"2024-09-10T21:35:44.880497+0000","last_fullsized":"2024-09-10T21:35:44.880497+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:04.880365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075284+0000","last_change":"2024-09-10T21:35:43.699828+0000","last_active":"2024-09-10T21:35:45.075284+0000","last_peered":"2024-09-10T21:35:45.075284+0000","last_clean":"2024-09-10T21:35:45.075284+0000","last_became_active":"2024-09-10T21:35:43.699553+0000","last_became_peered":"2024-09-10T21:35:43.699553+0000","last_unstale":"2024-09-10T21:35:45.075284+0000","last_undegraded":"2024-09-10T21:35:45.075284+0000","last_fullsized":"2024-09-10T21:35:45.075284+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:24.216065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693437+0000","last_change":"2024-09-10T21:35:42.692258+0000","last_active":"2024-09-10T21:35:44.693437+0000","last_peered":"2024-09-10T21:35:44.693437+0000","last_clean":"2024-09-10T21:35:44.693437+0000","last_became_active":"2024-09-10T21:35:42.691757+0000","last_became_peered":"2024-09-10T21:35:42.691757+0000","last_unstale":"2024-09-10T21:35:44.693437+0000","last_undegraded":"2024-09-10T21:35:44.693437+0000","last_fullsized":"2024-09-10T21:35:44.693437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:50.810200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062140+0000","last_change":"2024-09-10T21:35:43.708695+0000","last_active":"2024-09-10T21:35:45.062140+0000","last_peered":"2024-09-10T21:35:45.062140+0000","last_clean":"2024-09-10T21:35:45.062140+0000","last_became_active":"2024-09-10T21:35:43.705922+0000","last_became_peered":"2024-09-10T21:35:43.705922+0000","last_unstale":"2024-09-10T21:35:45.062140+0000","last_undegraded":"2024-09-10T21:35:45.062140+0000","last_fullsized":"2024-09-10T21:35:45.062140+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:38.047565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062415+0000","last_change":"2024-09-10T21:35:42.687000+0000","last_active":"2024-09-10T21:35:45.062415+0000","last_peered":"2024-09-10T21:35:45.062415+0000","last_clean":"2024-09-10T21:35:45.062415+0000","last_became_active":"2024-09-10T21:35:42.686730+0000","last_became_peered":"2024-09-10T21:35:42.686730+0000","last_unstale":"2024-09-10T21:35:45.062415+0000","last_undegraded":"2024-09-10T21:35:45.062415+0000","last_fullsized":"2024-09-10T21:35:45.062415+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:28.386650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694749+0000","last_change":"2024-09-10T21:35:43.688538+0000","last_active":"2024-09-10T21:35:44.694749+0000","last_peered":"2024-09-10T21:35:44.694749+0000","last_clean":"2024-09-10T21:35:44.694749+0000","last_became_active":"2024-09-10T21:35:43.688259+0000","last_became_peered":"2024-09-10T21:35:43.688259+0000","last_unstale":"2024-09-10T21:35:44.694749+0000","last_undegraded":"2024-09-10T21:35:44.694749+0000","last_fullsized":"2024-09-10T21:35:44.694749+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:36.626231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692993+0000","last_change":"2024-09-10T21:35:42.692966+0000","last_active":"2024-09-10T21:35:44.692993+0000","last_peered":"2024-09-10T21:35:44.692993+0000","last_clean":"2024-09-10T21:35:44.692993+0000","last_became_active":"2024-09-10T21:35:42.691729+0000","last_became_peered":"2024-09-10T21:35:42.691729+0000","last_unstale":"2024-09-10T21:35:44.692993+0000","last_undegraded":"2024-09-10T21:35:44.692993+0000","last_fullsized":"2024-09-10T21:35:44.692993+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:10.063389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696390+0000","last_change":"2024-09-10T21:35:43.708070+0000","last_active":"2024-09-10T21:35:44.696390+0000","last_peered":"2024-09-10T21:35:44.696390+0000","last_clean":"2024-09-10T21:35:44.696390+0000","last_became_active":"2024-09-10T21:35:43.707512+0000","last_became_peered":"2024-09-10T21:35:43.707512+0000","last_unstale":"2024-09-10T21:35:44.696390+0000","last_undegraded":"2024-09-10T21:35:44.696390+0000","last_fullsized":"2024-09-10T21:35:44.696390+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:36.836585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563454+0000","last_change":"2024-09-10T21:35:42.690902+0000","last_active":"2024-09-10T21:35:45.563454+0000","last_peered":"2024-09-10T21:35:45.563454+0000","last_clean":"2024-09-10T21:35:45.563454+0000","last_became_active":"2024-09-10T21:35:42.690591+0000","last_became_peered":"2024-09-10T21:35:42.690591+0000","last_unstale":"2024-09-10T21:35:45.563454+0000","last_undegraded":"2024-09-10T21:35:45.563454+0000","last_fullsized":"2024-09-10T21:35:45.563454+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:20.723287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695436+0000","last_change":"2024-09-10T21:35:43.710518+0000","last_active":"2024-09-10T21:35:44.695436+0000","last_peered":"2024-09-10T21:35:44.695436+0000","last_clean":"2024-09-10T21:35:44.695436+0000","last_became_active":"2024-09-10T21:35:43.710108+0000","last_became_peered":"2024-09-10T21:35:43.710108+0000","last_unstale":"2024-09-10T21:35:44.695436+0000","last_undegraded":"2024-09-10T21:35:44.695436+0000","last_fullsized":"2024-09-10T21:35:44.695436+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:52.921264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563242+0000","last_change":"2024-09-10T21:35:42.686431+0000","last_active":"2024-09-10T21:35:45.563242+0000","last_peered":"2024-09-10T21:35:45.563242+0000","last_clean":"2024-09-10T21:35:45.563242+0000","last_became_active":"2024-09-10T21:35:42.684143+0000","last_became_peered":"2024-09-10T21:35:42.684143+0000","last_unstale":"2024-09-10T21:35:45.563242+0000","last_undegraded":"2024-09-10T21:35:45.563242+0000","last_fullsized":"2024-09-10T21:35:45.563242+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:23.749610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.695890+0000","last_change":"2024-09-10T21:35:43.691613+0000","last_active":"2024-09-10T21:35:44.695890+0000","last_peered":"2024-09-10T21:35:44.695890+0000","last_clean":"2024-09-10T21:35:44.695890+0000","last_became_active":"2024-09-10T21:35:43.691305+0000","last_became_peered":"2024-09-10T21:35:43.691305+0000","last_unstale":"2024-09-10T21:35:44.695890+0000","last_undegraded":"2024-09-10T21:35:44.695890+0000","last_fullsized":"2024-09-10T21:35:44.695890+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:45.661154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692885+0000","last_change":"2024-09-10T21:35:42.693678+0000","last_active":"2024-09-10T21:35:44.692885+0000","last_peered":"2024-09-10T21:35:44.692885+0000","last_clean":"2024-09-10T21:35:44.692885+0000","last_became_active":"2024-09-10T21:35:42.693394+0000","last_became_peered":"2024-09-10T21:35:42.693394+0000","last_unstale":"2024-09-10T21:35:44.692885+0000","last_undegraded":"2024-09-10T21:35:44.692885+0000","last_fullsized":"2024-09-10T21:35:44.692885+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:58.991101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.697204+0000","last_change":"2024-09-10T21:35:43.708882+0000","last_active":"2024-09-10T21:35:44.697204+0000","last_peered":"2024-09-10T21:35:44.697204+0000","last_clean":"2024-09-10T21:35:44.697204+0000","last_became_active":"2024-09-10T21:35:43.707200+0000","last_became_peered":"2024-09-10T21:35:43.707200+0000","last_unstale":"2024-09-10T21:35:44.697204+0000","last_undegraded":"2024-09-10T21:35:44.697204+0000","last_fullsized":"2024-09-10T21:35:44.697204+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:27.687087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075147+0000","last_change":"2024-09-10T21:35:42.693798+0000","last_active":"2024-09-10T21:35:45.075147+0000","last_peered":"2024-09-10T21:35:45.075147+0000","last_clean":"2024-09-10T21:35:45.075147+0000","last_became_active":"2024-09-10T21:35:42.693499+0000","last_became_peered":"2024-09-10T21:35:42.693499+0000","last_unstale":"2024-09-10T21:35:45.075147+0000","last_undegraded":"2024-09-10T21:35:45.075147+0000","last_fullsized":"2024-09-10T21:35:45.075147+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:10.502692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.694617+0000","last_change":"2024-09-10T21:35:43.696424+0000","last_active":"2024-09-10T21:35:44.694617+0000","last_peered":"2024-09-10T21:35:44.694617+0000","last_clean":"2024-09-10T21:35:44.694617+0000","last_became_active":"2024-09-10T21:35:43.694483+0000","last_became_peered":"2024-09-10T21:35:43.694483+0000","last_unstale":"2024-09-10T21:35:44.694617+0000","last_undegraded":"2024-09-10T21:35:44.694617+0000","last_fullsized":"2024-09-10T21:35:44.694617+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:50.417439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563854+0000","last_change":"2024-09-10T21:35:42.695829+0000","last_active":"2024-09-10T21:35:45.563854+0000","last_peered":"2024-09-10T21:35:45.563854+0000","last_clean":"2024-09-10T21:35:45.563854+0000","last_became_active":"2024-09-10T21:35:42.695550+0000","last_became_peered":"2024-09-10T21:35:42.695550+0000","last_unstale":"2024-09-10T21:35:45.563854+0000","last_undegraded":"2024-09-10T21:35:45.563854+0000","last_fullsized":"2024-09-10T21:35:45.563854+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:29.998665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880558+0000","last_change":"2024-09-10T21:35:43.687717+0000","last_active":"2024-09-10T21:35:44.880558+0000","last_peered":"2024-09-10T21:35:44.880558+0000","last_clean":"2024-09-10T21:35:44.880558+0000","last_became_active":"2024-09-10T21:35:43.687440+0000","last_became_peered":"2024-09-10T21:35:43.687440+0000","last_unstale":"2024-09-10T21:35:44.880558+0000","last_undegraded":"2024-09-10T21:35:44.880558+0000","last_fullsized":"2024-09-10T21:35:44.880558+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:18.372062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"146'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.684833+0000","last_change":"2024-09-10T21:35:42.694598+0000","last_active":"2024-09-10T21:35:45.684833+0000","last_peered":"2024-09-10T21:35:45.684833+0000","last_clean":"2024-09-10T21:35:45.684833+0000","last_became_active":"2024-09-10T21:35:42.694267+0000","last_became_peered":"2024-09-10T21:35:42.694267+0000","last_unstale":"2024-09-10T21:35:45.684833+0000","last_undegraded":"2024-09-10T21:35:45.684833+0000","last_fullsized":"2024-09-10T21:35:45.684833+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:09.399075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.075103+0000","last_change":"2024-09-10T21:35:43.708665+0000","last_active":"2024-09-10T21:35:45.075103+0000","last_peered":"2024-09-10T21:35:45.075103+0000","last_clean":"2024-09-10T21:35:45.075103+0000","last_became_active":"2024-09-10T21:35:43.707009+0000","last_became_peered":"2024-09-10T21:35:43.707009+0000","last_unstale":"2024-09-10T21:35:45.075103+0000","last_undegraded":"2024-09-10T21:35:45.075103+0000","last_fullsized":"2024-09-10T21:35:45.075103+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:41.152095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880692+0000","last_change":"2024-09-10T21:35:42.698430+0000","last_active":"2024-09-10T21:35:44.880692+0000","last_peered":"2024-09-10T21:35:44.880692+0000","last_clean":"2024-09-10T21:35:44.880692+0000","last_became_active":"2024-09-10T21:35:42.698195+0000","last_became_peered":"2024-09-10T21:35:42.698195+0000","last_unstale":"2024-09-10T21:35:44.880692+0000","last_undegraded":"2024-09-10T21:35:44.880692+0000","last_fullsized":"2024-09-10T21:35:44.880692+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:27.995910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062378+0000","last_change":"2024-09-10T21:35:43.697700+0000","last_active":"2024-09-10T21:35:45.062378+0000","last_peered":"2024-09-10T21:35:45.062378+0000","last_clean":"2024-09-10T21:35:45.062378+0000","last_became_active":"2024-09-10T21:35:43.694375+0000","last_became_peered":"2024-09-10T21:35:43.694375+0000","last_unstale":"2024-09-10T21:35:45.062378+0000","last_undegraded":"2024-09-10T21:35:45.062378+0000","last_fullsized":"2024-09-10T21:35:45.062378+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:49.069960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696532+0000","last_change":"2024-09-10T21:35:42.692774+0000","last_active":"2024-09-10T21:35:44.696532+0000","last_peered":"2024-09-10T21:35:44.696532+0000","last_clean":"2024-09-10T21:35:44.696532+0000","last_became_active":"2024-09-10T21:35:42.692448+0000","last_became_peered":"2024-09-10T21:35:42.692448+0000","last_unstale":"2024-09-10T21:35:44.696532+0000","last_undegraded":"2024-09-10T21:35:44.696532+0000","last_fullsized":"2024-09-10T21:35:44.696532+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:18.354140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696297+0000","last_change":"2024-09-10T21:35:43.709006+0000","last_active":"2024-09-10T21:35:44.696297+0000","last_peered":"2024-09-10T21:35:44.696297+0000","last_clean":"2024-09-10T21:35:44.696297+0000","last_became_active":"2024-09-10T21:35:43.707719+0000","last_became_peered":"2024-09-10T21:35:43.707719+0000","last_unstale":"2024-09-10T21:35:44.696297+0000","last_undegraded":"2024-09-10T21:35:44.696297+0000","last_fullsized":"2024-09-10T21:35:44.696297+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:34.379910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.880713+0000","last_change":"2024-09-10T21:35:42.691107+0000","last_active":"2024-09-10T21:35:44.880713+0000","last_peered":"2024-09-10T21:35:44.880713+0000","last_clean":"2024-09-10T21:35:44.880713+0000","last_became_active":"2024-09-10T21:35:42.690747+0000","last_became_peered":"2024-09-10T21:35:42.690747+0000","last_unstale":"2024-09-10T21:35:44.880713+0000","last_undegraded":"2024-09-10T21:35:44.880713+0000","last_fullsized":"2024-09-10T21:35:44.880713+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:26.876968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696810+0000","last_change":"2024-09-10T21:35:43.707971+0000","last_active":"2024-09-10T21:35:44.696810+0000","last_peered":"2024-09-10T21:35:44.696810+0000","last_clean":"2024-09-10T21:35:44.696810+0000","last_became_active":"2024-09-10T21:35:43.707565+0000","last_became_peered":"2024-09-10T21:35:43.707565+0000","last_unstale":"2024-09-10T21:35:44.696810+0000","last_undegraded":"2024-09-10T21:35:44.696810+0000","last_fullsized":"2024-09-10T21:35:44.696810+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:46.080500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563377+0000","last_change":"2024-09-10T21:35:42.697886+0000","last_active":"2024-09-10T21:35:45.563377+0000","last_peered":"2024-09-10T21:35:45.563377+0000","last_clean":"2024-09-10T21:35:45.563377+0000","last_became_active":"2024-09-10T21:35:42.697625+0000","last_became_peered":"2024-09-10T21:35:42.697625+0000","last_unstale":"2024-09-10T21:35:45.563377+0000","last_undegraded":"2024-09-10T21:35:45.563377+0000","last_fullsized":"2024-09-10T21:35:45.563377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:23.745543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.693346+0000","last_change":"2024-09-10T21:35:43.687136+0000","last_active":"2024-09-10T21:35:44.693346+0000","last_peered":"2024-09-10T21:35:44.693346+0000","last_clean":"2024-09-10T21:35:44.693346+0000","last_became_active":"2024-09-10T21:35:43.686689+0000","last_became_peered":"2024-09-10T21:35:43.686689+0000","last_unstale":"2024-09-10T21:35:44.693346+0000","last_undegraded":"2024-09-10T21:35:44.693346+0000","last_fullsized":"2024-09-10T21:35:44.693346+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:10.009857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.062254+0000","last_change":"2024-09-10T21:35:42.695868+0000","last_active":"2024-09-10T21:35:45.062254+0000","last_peered":"2024-09-10T21:35:45.062254+0000","last_clean":"2024-09-10T21:35:45.062254+0000","last_became_active":"2024-09-10T21:35:42.695165+0000","last_became_peered":"2024-09-10T21:35:42.695165+0000","last_unstale":"2024-09-10T21:35:45.062254+0000","last_undegraded":"2024-09-10T21:35:45.062254+0000","last_fullsized":"2024-09-10T21:35:45.062254+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:10.502692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.562882+0000","last_change":"2024-09-10T21:35:43.706228+0000","last_active":"2024-09-10T21:35:45.562882+0000","last_peered":"2024-09-10T21:35:45.562882+0000","last_clean":"2024-09-10T21:35:45.562882+0000","last_became_active":"2024-09-10T21:35:43.705968+0000","last_became_peered":"2024-09-10T21:35:43.705968+0000","last_unstale":"2024-09-10T21:35:45.562882+0000","last_undegraded":"2024-09-10T21:35:45.562882+0000","last_fullsized":"2024-09-10T21:35:45.562882+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:34.416314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.563243+0000","last_change":"2024-09-10T21:35:42.697093+0000","last_active":"2024-09-10T21:35:45.563243+0000","last_peered":"2024-09-10T21:35:45.563243+0000","last_clean":"2024-09-10T21:35:45.563243+0000","last_became_active":"2024-09-10T21:35:42.696400+0000","last_became_peered":"2024-09-10T21:35:42.696400+0000","last_unstale":"2024-09-10T21:35:45.563243+0000","last_undegraded":"2024-09-10T21:35:45.563243+0000","last_fullsized":"2024-09-10T21:35:45.563243+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:41.463058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.692962+0000","last_change":"2024-09-10T21:35:43.705454+0000","last_active":"2024-09-10T21:35:44.692962+0000","last_peered":"2024-09-10T21:35:44.692962+0000","last_clean":"2024-09-10T21:35:44.692962+0000","last_became_active":"2024-09-10T21:35:43.704324+0000","last_became_peered":"2024-09-10T21:35:43.704324+0000","last_unstale":"2024-09-10T21:35:44.692962+0000","last_undegraded":"2024-09-10T21:35:44.692962+0000","last_fullsized":"2024-09-10T21:35:44.692962+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:42.661776+0000","last_clean_scrub_stamp":"2024-09-10T21:35:42.661776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:29.000102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:44.696102+0000","last_change":"2024-09-10T21:35:42.695716+0000","last_active":"2024-09-10T21:35:44.696102+0000","last_peered":"2024-09-10T21:35:44.696102+0000","last_clean":"2024-09-10T21:35:44.696102+0000","last_became_active":"2024-09-10T21:35:42.695410+0000","last_became_peered":"2024-09-10T21:35:42.695410+0000","last_unstale":"2024-09-10T21:35:44.696102+0000","last_undegraded":"2024-09-10T21:35:44.696102+0000","last_fullsized":"2024-09-10T21:35:44.696102+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:35:41.657585+0000","last_clean_scrub_stamp":"2024-09-10T21:35:41.657585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:30.466770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":352,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-10T21:35:45.061985+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:35:45.061985+0000","last_peered":"2024-09-10T21:35:45.061985+0000","last_clean":"2024-09-10T21:35:45.061985+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:35:45.061985+0000","last_undegraded":"2024-09-10T21:35:45.061985+0000","last_fullsized":"2024-09-10T21:35:45.061985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509586,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1892352,"data_stored":953378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509584,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28884,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342956,"statfs":{"total":96636764160,"available":96607186944,"internally_reserved":0,"allocated":1896448,"data_stored":954338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":2486272,"data_stored":1543218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509586,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41364,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330476,"statfs":{"total":96636764160,"available":96594407424,"internally_reserved":0,"allocated":1896448,"data_stored":954338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509585,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1892352,"data_stored":952838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509586,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2486272,"data_stored":1543186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509586,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1888256,"data_stored":953308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1888256,"data_stored":952760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:35:52.719 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:35:52.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:35:52.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 -- 172.21.15.97:0/2684379922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 msgr2=0x7fe6040b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 --2- 172.21.15.97:0/2684379922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe6040b4120 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fe60415f790 tx=0x7fe5f001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 -- 172.21.15.97:0/2684379922 shutdown_connections 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 --2- 172.21.15.97:0/2684379922 >> v2:172.21.15.194:3300/0 conn(0x7fe6040b4660 0x7fe60406fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 --2- 172.21.15.97:0/2684379922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe6040b4120 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 --2- 172.21.15.97:0/2684379922 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe604143f90 0x7fe6040b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 -- 172.21.15.97:0/2684379922 >> 172.21.15.97:0/2684379922 conn(0x7fe6040b1780 msgr2=0x7fe6040b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 -- 172.21.15.97:0/2684379922 shutdown_connections 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.905+0000 7fe609c00640 1 -- 172.21.15.97:0/2684379922 wait complete. 2024-09-10T21:35:52.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 Processor -- start 2024-09-10T21:35:52.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 -- start start 2024-09-10T21:35:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6040b4660 0x7fe6041f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe604143f90 0x7fe60413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe60413c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe60415edc0 con 0x7fe604144690 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe60415ec40 con 0x7fe6040b4660 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe60415ef40 con 0x7fe604143f90 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe60413c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe60413c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45084/0 (socket says 172.21.15.97:45084) 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 -- 172.21.15.97:0/1266740898 learned_addr learned my addr 172.21.15.97:0/1266740898 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe602a00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6040b4660 0x7fe6041f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe602000640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe604143f90 0x7fe60413bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 -- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe604143f90 msgr2=0x7fe60413bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe604143f90 0x7fe60413bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 -- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6040b4660 msgr2=0x7fe6041f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6040b4660 0x7fe6041f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 -- 172.21.15.97:0/1266740898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60415e0e0 con 0x7fe604144690 2024-09-10T21:35:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe602a00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6040b4660 0x7fe6041f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:52.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe603400640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe60413c160 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00eab0 tx=0x7fe5fc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe5faa00640 1 -- 172.21.15.97:0/1266740898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc004b30 con 0x7fe604144690 2024-09-10T21:35:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe5faa00640 1 -- 172.21.15.97:0/1266740898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5fc004dc0 con 0x7fe604144690 2024-09-10T21:35:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe604137240 con 0x7fe604144690 2024-09-10T21:35:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe5faa00640 1 -- 172.21.15.97:0/1266740898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc005280 con 0x7fe604144690 2024-09-10T21:35:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe604137770 con 0x7fe604144690 2024-09-10T21:35:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.909+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60415f4d0 con 0x7fe604144690 2024-09-10T21:35:52.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.913+0000 7fe5faa00640 1 -- 172.21.15.97:0/1266740898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe5fc020030 con 0x7fe604144690 2024-09-10T21:35:52.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.913+0000 7fe5faa00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5e40725e0 0x7fe5e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:52.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.913+0000 7fe5faa00640 1 -- 172.21.15.97:0/1266740898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fe5fc093f70 con 0x7fe604144690 2024-09-10T21:35:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.913+0000 7fe602a00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5e40725e0 0x7fe5e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.913+0000 7fe602a00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5e40725e0 0x7fe5e4074aa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009c30 tx=0x7fe5f40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:52.917+0000 7fe5faa00640 1 -- 172.21.15.97:0/1266740898 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5fc05f550 con 0x7fe604144690 2024-09-10T21:35:53.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.145+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe6041435b0 con 0x7fe604144690 2024-09-10T21:35:53.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.145+0000 7fe5faa00640 1 -- 172.21.15.97:0/1266740898 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7fe5fc05ee50 con 0x7fe604144690 2024-09-10T21:35:53.147 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-10T21:35:53.147 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC/uOBm/UBKERAAggR2Wulenvf2WVcXZ/849Q== 2024-09-10T21:35:53.147 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-09-10T21:35:53.147 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-09-10T21:35:53.147 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:35:53.147 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-09-10T21:35:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5e40725e0 msgr2=0x7fe5e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5e40725e0 0x7fe5e4074aa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009c30 tx=0x7fe5f40091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 msgr2=0x7fe60413c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe60413c160 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00eab0 tx=0x7fe5fc00ef70 comp rx=0 tx=0).stop 2024-09-10T21:35:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 shutdown_connections 2024-09-10T21:35:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe5e40725e0 0x7fe5e4074aa0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe604144690 0x7fe60413c160 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe604143f90 0x7fe60413bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 --2- 172.21.15.97:0/1266740898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6040b4660 0x7fe6041f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 >> 172.21.15.97:0/1266740898 conn(0x7fe6040b1780 msgr2=0x7fe6040cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 shutdown_connections 2024-09-10T21:35:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.149+0000 7fe609c00640 1 -- 172.21.15.97:0/1266740898 wait complete. 2024-09-10T21:35:53.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 -- 172.21.15.97:0/3822903737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040d2690 msgr2=0x7fee040b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 --2- 172.21.15.97:0/3822903737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040d2690 0x7fee040b4710 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fedec009840 tx=0x7fedec01c820 comp rx=0 tx=0).stop 2024-09-10T21:35:53.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 -- 172.21.15.97:0/3822903737 shutdown_connections 2024-09-10T21:35:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 --2- 172.21.15.97:0/3822903737 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fee040b3280 0x7fee040b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 --2- 172.21.15.97:0/3822903737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040d2690 0x7fee040b4710 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 -- 172.21.15.97:0/3822903737 >> 172.21.15.97:0/3822903737 conn(0x7fee040d3e70 msgr2=0x7fee040d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 -- 172.21.15.97:0/3822903737 shutdown_connections 2024-09-10T21:35:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.309+0000 7fee0b400640 1 -- 172.21.15.97:0/3822903737 wait complete. 2024-09-10T21:35:53.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 Processor -- start 2024-09-10T21:35:53.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 -- start start 2024-09-10T21:35:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040b3280 0x7fee041538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee040d2690 0x7fee040b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fee0415b9c0 0x7fee040d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fee0416b970 con 0x7fee040b3280 2024-09-10T21:35:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fee0416b7f0 con 0x7fee040d2690 2024-09-10T21:35:53.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fee0416baf0 con 0x7fee0415b9c0 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040b3280 0x7fee041538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040b3280 0x7fee041538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45098/0 (socket says 172.21.15.97:45098) 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 -- 172.21.15.97:0/1823681135 learned_addr learned my addr 172.21.15.97:0/1823681135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee09400640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fee0415b9c0 0x7fee040d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fedfbe00640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee040d2690 0x7fee040b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 -- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fee0415b9c0 msgr2=0x7fee040d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fee0415b9c0 0x7fee040d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 -- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee040d2690 msgr2=0x7fee040b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee040d2690 0x7fee040b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 -- 172.21.15.97:0/1823681135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee040b7890 con 0x7fee040b3280 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fedfbe00640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee040d2690 0x7fee040b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee08a00640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040b3280 0x7fee041538e0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c430 tx=0x7fedf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee01600640 1 -- 172.21.15.97:0/1823681135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4011790 con 0x7fee040b3280 2024-09-10T21:35:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee01600640 1 -- 172.21.15.97:0/1823681135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedf4017070 con 0x7fee040b3280 2024-09-10T21:35:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee01600640 1 -- 172.21.15.97:0/1823681135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4011ce0 con 0x7fee040b3280 2024-09-10T21:35:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee040b7bd0 con 0x7fee040b3280 2024-09-10T21:35:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.313+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee040b8410 con 0x7fee040b3280 2024-09-10T21:35:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.317+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee040d1f80 con 0x7fee040b3280 2024-09-10T21:35:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.317+0000 7fee01600640 1 -- 172.21.15.97:0/1823681135 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fedf401c460 con 0x7fee040b3280 2024-09-10T21:35:53.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.317+0000 7fee01600640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fede00725e0 0x7fede0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:53.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.317+0000 7fee01600640 1 -- 172.21.15.97:0/1823681135 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7fedf40931a0 con 0x7fee040b3280 2024-09-10T21:35:53.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.317+0000 7fedfbe00640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fede00725e0 0x7fede0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:53.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.317+0000 7fedfbe00640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fede00725e0 0x7fede0074aa0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fedec0af000 tx=0x7fedec01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:53.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.321+0000 7fee01600640 1 -- 172.21.15.97:0/1823681135 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fee040b7bd0 con 0x7fee040b3280 2024-09-10T21:35:53.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.517+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee0408db10 con 0x7fee040b3280 2024-09-10T21:35:53.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.517+0000 7fee01600640 1 -- 172.21.15.97:0/1823681135 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 344 v344) ==== 95+0+5241 (secure 0 0 0) 0x7fedf405e7a0 con 0x7fee040b3280 2024-09-10T21:35:53.519 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:53.519 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":344,"btime":"2024-09-10T21:35:46:701370+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":10629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339}],"filesystems":[{"mdsmap":{"epoch":344,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:35:46.088263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10647},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10647":{"gid":10647,"name":"d","rank":0,"incarnation":340,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6837/3945561983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3945561983},{"type":"v1","addr":"172.21.15.194:6837","nonce":3945561983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10647,"qdb_cluster":[10647]},"id":7}]} 2024-09-10T21:35:53.519 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 344 2024-09-10T21:35:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fede00725e0 msgr2=0x7fede0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fede00725e0 0x7fede0074aa0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fedec0af000 tx=0x7fedec01b060 comp rx=0 tx=0).stop 2024-09-10T21:35:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040b3280 msgr2=0x7fee041538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040b3280 0x7fee041538e0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fedf400c430 tx=0x7fedf400c8f0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 shutdown_connections 2024-09-10T21:35:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fede00725e0 0x7fede0074aa0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fee0415b9c0 0x7fee040d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee040d2690 0x7fee040b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 --2- 172.21.15.97:0/1823681135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee040b3280 0x7fee041538e0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 >> 172.21.15.97:0/1823681135 conn(0x7fee040d3e70 msgr2=0x7fee041550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 shutdown_connections 2024-09-10T21:35:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:53.521+0000 7fee0b400640 1 -- 172.21.15.97:0/1823681135 wait complete. 2024-09-10T21:35:53.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 344, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:35:43.069729+0000', 'modified': '2024-09-10T21:35:46.088263+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10647}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10647': {'gid': 10647, 'name': 'd', 'rank': 0, 'incarnation': 340, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.194:6837/3945561983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6835', 'nonce': 3945561983}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3945561983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10647, 'qdb_cluster': [10647]} 2024-09-10T21:35:53.944 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:35:53.944 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:35:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- 172.21.15.97:0/535881763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0b0d50 msgr2=0x7f7adc0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/535881763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0b0d50 0x7f7adc0ccde0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009bc0 tx=0x7f7ad401cba0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- 172.21.15.97:0/535881763 shutdown_connections 2024-09-10T21:35:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/535881763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0b0d50 0x7f7adc0ccde0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- 172.21.15.97:0/535881763 >> 172.21.15.97:0/535881763 conn(0x7f7adc0d1c30 msgr2=0x7f7adc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- 172.21.15.97:0/535881763 shutdown_connections 2024-09-10T21:35:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- 172.21.15.97:0/535881763 wait complete. 2024-09-10T21:35:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 Processor -- start 2024-09-10T21:35:54.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- start start 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0afdd0 0x7f7adc1f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7adc0b0d50 0x7f7adc167100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7adc0ce640 0x7f7adc14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc169ef0 con 0x7f7adc0ce640 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc169d70 con 0x7f7adc0afdd0 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae3e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc16a070 con 0x7f7adc0b0d50 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7adc0ce640 0x7f7adc14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7adc0ce640 0x7f7adc14ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45110/0 (socket says 172.21.15.97:45110) 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1131935877 learned_addr learned my addr 172.21.15.97:0/1131935877 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7adc0b0d50 0x7f7adc167100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1400640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0afdd0 0x7f7adc1f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7adc0b0d50 msgr2=0x7f7adc167100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7adc0b0d50 0x7f7adc167100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0afdd0 msgr2=0x7f7adc1f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0afdd0 0x7f7adc1f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.089+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1131935877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc1f83c0 con 0x7f7adc0ce640 2024-09-10T21:35:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7adc0ce640 0x7f7adc14ee60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b0040 tx=0x7f7ad4005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ad9600640 1 -- 172.21.15.97:0/1131935877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40bb070 con 0x7f7adc0ce640 2024-09-10T21:35:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ad9600640 1 -- 172.21.15.97:0/1131935877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ad4004c50 con 0x7f7adc0ce640 2024-09-10T21:35:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc1f86a0 con 0x7f7adc0ce640 2024-09-10T21:35:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ad9600640 1 -- 172.21.15.97:0/1131935877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40b2040 con 0x7f7adc0ce640 2024-09-10T21:35:54.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc1f8c60 con 0x7f7adc0ce640 2024-09-10T21:35:54.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ad9600640 1 -- 172.21.15.97:0/1131935877 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7ad40b5070 con 0x7f7adc0ce640 2024-09-10T21:35:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ad9600640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7abc0725e0 0x7f7abc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.093+0000 7f7ad9600640 1 -- 172.21.15.97:0/1131935877 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f7ad412cf70 con 0x7f7adc0ce640 2024-09-10T21:35:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 Processor -- start 2024-09-10T21:35:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- start start 2024-09-10T21:35:54.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1400640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7abc0725e0 0x7f7abc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1400640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7abc0725e0 0x7f7abc074aa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0007fd0 tx=0x7f7ad0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2ef4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2ea88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2ea91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55abc2dc93f0 con 0x55abc2ea8e10 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abc2d06b20 con 0x55abc2ef49d0 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55abc2cf5f50 con 0x55abc2ef52f0 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2ea91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2ea91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45122/0 (socket says 172.21.15.97:45122) 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2ef4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 -- 172.21.15.97:0/54344279 learned_addr learned my addr 172.21.15.97:0/54344279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2ea88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 -- 172.21.15.97:0/54344279 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 msgr2=0x55abc2ea88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2ea88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 -- 172.21.15.97:0/54344279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 msgr2=0x55abc2ef4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2ef4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 -- 172.21.15.97:0/54344279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc2cbd590 con 0x55abc2ea8e10 2024-09-10T21:35:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2ea91f0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40f7df0 tx=0x7f7ad4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81d2321e169fbff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ace000640 1 -- 172.21.15.97:0/54344279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40f7970 con 0x55abc2ea8e10 2024-09-10T21:35:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ace000640 1 -- 172.21.15.97:0/54344279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ad40fc890 con 0x55abc2ea8e10 2024-09-10T21:35:54.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ace000640 1 -- 172.21.15.97:0/54344279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4101340 con 0x55abc2ea8e10 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- 172.21.15.97:0/54344279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 msgr2=0x55abc2ea91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2ea91f0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40f7df0 tx=0x7f7ad4005e20 comp rx=0 tx=0).stop 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- 172.21.15.97:0/54344279 shutdown_connections 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2ea91f0 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2ea88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 --2- 172.21.15.97:0/54344279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2ef4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- 172.21.15.97:0/54344279 >> 172.21.15.97:0/54344279 conn(0x55abc2d92130 msgr2=0x55abc2ed7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- 172.21.15.97:0/54344279 shutdown_connections 2024-09-10T21:35:54.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- 172.21.15.97:0/54344279 wait complete. 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.097+0000 7f7ae7129000 1 -- start start 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 Processor -- start 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ea8e10 0x55abc2eaa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef49d0 0x55abc2eaa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef52f0 0x55abc2eaacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55abc2c905d0 con 0x55abc2ef49d0 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abc2cf9d60 con 0x55abc2ef52f0 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55abc2c9c500 con 0x55abc2ea8e10 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 10 client.0 _ll_register_callbacks cb 0x55abc2ca0810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef49d0 0x55abc2eaa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef49d0 0x55abc2eaa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45134/0 (socket says 172.21.15.97:45134) 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 -- 172.21.15.97:0/4100877393 learned_addr learned my addr 172.21.15.97:0/4100877393 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae1400640 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ea8e10 0x55abc2eaa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef52f0 0x55abc2eaacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 -- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ea8e10 msgr2=0x55abc2eaa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ea8e10 0x55abc2eaa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 -- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef52f0 msgr2=0x55abc2eaacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef52f0 0x55abc2eaacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 -- 172.21.15.97:0/4100877393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc2c94170 con 0x55abc2ef49d0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef49d0 0x55abc2eaa780 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8012520 tx=0x7f7ac80129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2610c55a95d70601 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ac2a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ac2a00640 1 -- 172.21.15.97:0/4100877393 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac801a020 con 0x55abc2ef49d0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55abc2c8e0f0 con 0x55abc2ef49d0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ac2a00640 1 -- 172.21.15.97:0/4100877393 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f7ac8014670 con 0x55abc2ef49d0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ac2a00640 1 -- 172.21.15.97:0/4100877393 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=344) ==== 24+0+0 (secure 0 0 0) 0x7f7ac8024050 con 0x55abc2ef49d0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 10 client.10779 fetch_fsmap learned FSMap version 344 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=344,monmap=2+}) -- 0x55abc2d7aef0 con 0x55abc2ef49d0 2024-09-10T21:35:54.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ac2a00640 1 -- 172.21.15.97:0/4100877393 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55abc2c8e0f0 con 0x55abc2ef49d0 2024-09-10T21:35:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ac2a00640 1 -- 172.21.15.97:0/4100877393 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 344) ==== 3502+0+0 (secure 0 0 0) 0x7f7ac8027020 con 0x55abc2ef49d0 2024-09-10T21:35:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 10 client.10779 fetch_fsmap finished waiting for FSMap version 344 2024-09-10T21:35:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 10 client.10779 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:35:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 20 client.10779 populate_metadata read hostname 'smithi097' 2024-09-10T21:35:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f03c70 0x55abc2f06060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 4 client.10779 mds_command: new command op to 10647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:35:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 --> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc2d08a70 con 0x55abc2f03c70 2024-09-10T21:35:54.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.101+0000 7f7ae1400640 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f03c70 0x55abc2f06060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae1400640 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f03c70 0x55abc2f06060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cb74d5eb9c5e83 server_cookie=b4052ba22ed548ac in_seq=0 out_seq=0 2024-09-10T21:35:54.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ac2a00640 10 client.10779 ms_handle_connect on v2:172.21.15.194:6835/3945561983 2024-09-10T21:35:54.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ac2a00640 1 -- 172.21.15.97:0/4100877393 <== mds.0 v2:172.21.15.194:6835/3945561983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55abc2d08a70 con 0x55abc2f03c70 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ac2a00640 10 client.10779 handle_command_reply: tid=0 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 client.10779 shutdown 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef49d0 msgr2=0x55abc2eaa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef49d0 0x55abc2eaa780 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8012520 tx=0x7f7ac80129e0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 shutdown_connections 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f03c70 0x55abc2f06060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef52f0 0x55abc2eaacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef49d0 0x55abc2eaa780 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ac2a00640 0 client.10779 ms_handle_reset on v2:172.21.15.194:6835/3945561983 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 --2- 172.21.15.97:0/4100877393 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ea8e10 0x55abc2eaa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 >> 172.21.15.97:0/4100877393 conn(0x55abc2d92130 msgr2=0x55abc2ea7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 shutdown_connections 2024-09-10T21:35:54.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 1 -- 172.21.15.97:0/4100877393 wait complete. 2024-09-10T21:35:54.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.105+0000 7f7ae7129000 20 client.10779 trim_cache size 0 max 16384 2024-09-10T21:35:54.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.121+0000 7f7ae7129000 1 Processor -- start 2024-09-10T21:35:54.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.121+0000 7f7ae7129000 1 -- start start 2024-09-10T21:35:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ea8e10 0x55abc2f6eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef49d0 0x55abc2f6f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef52f0 0x55abc2f6fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1fc0d0 con 0x55abc2ef52f0 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1fb430 con 0x55abc2ea8e10 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc1fba80 con 0x55abc2ef49d0 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef52f0 0x55abc2f6fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef52f0 0x55abc2f6fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45150/0 (socket says 172.21.15.97:45150) 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1081386081 learned_addr learned my addr 172.21.15.97:0/1081386081 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1400640 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ea8e10 0x55abc2f6eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef49d0 0x55abc2f6f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef49d0 msgr2=0x55abc2f6f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef49d0 0x55abc2f6f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ea8e10 msgr2=0x55abc2f6eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ea8e10 0x55abc2f6eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 -- 172.21.15.97:0/1081386081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc2d51d90 con 0x55abc2ef52f0 2024-09-10T21:35:54.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef52f0 0x55abc2f6fa70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40197b0 tx=0x7f7ad412b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad67553d807fdb56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7acd600640 1 -- 172.21.15.97:0/1081386081 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40f7e60 con 0x55abc2ef52f0 2024-09-10T21:35:54.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7acd600640 1 -- 172.21.15.97:0/1081386081 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ad411f070 con 0x55abc2ef52f0 2024-09-10T21:35:54.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7acd600640 1 -- 172.21.15.97:0/1081386081 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4101340 con 0x55abc2ef52f0 2024-09-10T21:35:54.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- 172.21.15.97:0/1081386081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef52f0 msgr2=0x55abc2f6fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef52f0 0x55abc2f6fa70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40197b0 tx=0x7f7ad412b140 comp rx=0 tx=0).stop 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- 172.21.15.97:0/1081386081 shutdown_connections 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ef52f0 0x55abc2f6fa70 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef49d0 0x55abc2f6f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1081386081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ea8e10 0x55abc2f6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- 172.21.15.97:0/1081386081 >> 172.21.15.97:0/1081386081 conn(0x55abc2d92130 msgr2=0x55abc2f06060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- 172.21.15.97:0/1081386081 shutdown_connections 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- 172.21.15.97:0/1081386081 wait complete. 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 -- start start 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.125+0000 7f7ae7129000 1 Processor -- start 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2f0b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2f2ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2f2f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55abc2cf9070 con 0x55abc2ea8e10 2024-09-10T21:35:54.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abc2cfe390 con 0x55abc2ef49d0 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55abc2cc1ec0 con 0x55abc2ef52f0 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 10 client.0 _ll_register_callbacks cb 0x7f7ac8007b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2f0b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2f0b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45162/0 (socket says 172.21.15.97:45162) 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 -- 172.21.15.97:0/1337173389 learned_addr learned my addr 172.21.15.97:0/1337173389 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1e00640 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2f2f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2f2ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 -- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 msgr2=0x55abc2f2f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2f2f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 -- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 msgr2=0x55abc2f2ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2f2ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 -- 172.21.15.97:0/1337173389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc2cbca90 con 0x55abc2ea8e10 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae1400640 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2f0b980 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x55abc2d78f40 tx=0x7f7ad0001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6dfb5a9f6cb5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:35:54.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 1 -- 172.21.15.97:0/1337173389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0014450 con 0x55abc2ea8e10 2024-09-10T21:35:54.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 1 -- 172.21.15.97:0/1337173389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 6101+0+0 (secure 0 0 0) 0x7f7ad00145f0 con 0x55abc2ea8e10 2024-09-10T21:35:54.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad411f070 con 0x55abc2ea8e10 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 1 -- 172.21.15.97:0/1337173389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=344) ==== 24+0+0 (secure 0 0 0) 0x7f7ad003a050 con 0x55abc2ea8e10 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 10 client.10785 fetch_fsmap learned FSMap version 344 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=344,monmap=2+}) -- 0x55abc2c9b050 con 0x55abc2ea8e10 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 1 -- 172.21.15.97:0/1337173389 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ad411f070 con 0x55abc2ea8e10 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 1 -- 172.21.15.97:0/1337173389 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 344) ==== 3502+0+0 (secure 0 0 0) 0x7f7ad003d020 con 0x55abc2ea8e10 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 10 client.10785 fetch_fsmap finished waiting for FSMap version 344 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 10 client.10785 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 20 client.10785 populate_metadata read hostname 'smithi097' 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f2f810 0x55abc2f73240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 4 client.10785 mds_command: new command op to 10647 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 --> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] -- command(tid 0: {"prefix": "status"}) -- 0x55abc2d04520 con 0x55abc2f2f810 2024-09-10T21:35:54.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f2f810 0x55abc2f73240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae0a00640 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f2f810 0x55abc2f73240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd44e9d4f9976c7f server_cookie=4d65baed6e10e724 in_seq=0 out_seq=0 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 10 client.10785 ms_handle_connect on v2:172.21.15.194:6835/3945561983 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 1 -- 172.21.15.97:0/1337173389 <== mds.0 v2:172.21.15.194:6835/3945561983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55abc2d04520 con 0x55abc2f2f810 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 10 client.10785 handle_command_reply: tid=0 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 client.10785 shutdown 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 msgr2=0x55abc2f0b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2f0b980 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x55abc2d78f40 tx=0x7f7ad0001090 comp rx=0 tx=0).stop 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 shutdown_connections 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:6835/3945561983,v1:172.21.15.194:6837/3945561983] conn(0x55abc2f2f810 0x55abc2f73240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55abc2ef52f0 0x55abc2f2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abc2ef49d0 0x55abc2f2ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ac2a00640 0 client.10785 ms_handle_reset on v2:172.21.15.194:6835/3945561983 2024-09-10T21:35:54.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 --2- 172.21.15.97:0/1337173389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55abc2ea8e10 0x55abc2f0b980 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 >> 172.21.15.97:0/1337173389 conn(0x55abc2d92130 msgr2=0x55abc2f06880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 shutdown_connections 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 1 -- 172.21.15.97:0/1337173389 wait complete. 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.129+0000 7f7ae7129000 20 client.10785 trim_cache size 0 max 16384 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 10647, 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:35:54.135 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:35:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 10.460089014999999, 2024-09-10T21:35:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 344, 2024-09-10T21:35:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 146, 2024-09-10T21:35:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 146, 2024-09-10T21:35:54.136 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 14.968000787999999 2024-09-10T21:35:54.136 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7abc0725e0 msgr2=0x7f7abc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7abc0725e0 0x7f7abc074aa0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0007fd0 tx=0x7f7ad0009040 comp rx=0 tx=0).stop 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7adc0ce640 msgr2=0x7f7adc14ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7adc0ce640 0x7f7adc14ee60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b0040 tx=0x7f7ad4005ac0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 shutdown_connections 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7abc0725e0 0x7f7abc074aa0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7adc0ce640 0x7f7adc14ee60 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7adc0b0d50 0x7f7adc167100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 --2- 172.21.15.97:0/1131935877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7adc0afdd0 0x7f7adc1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 >> 172.21.15.97:0/1131935877 conn(0x7f7adc0d1c30 msgr2=0x7f7adc071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 shutdown_connections 2024-09-10T21:35:54.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.133+0000 7f7ae3e00640 1 -- 172.21.15.97:0/1131935877 wait complete. 2024-09-10T21:35:54.151 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:35:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 -- 172.21.15.97:0/4003485740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66bc004810 msgr2=0x7f66bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 --2- 172.21.15.97:0/4003485740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66bc004810 0x7f66bc004c90 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009ee0 tx=0x7f66c001d4f0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 -- 172.21.15.97:0/4003485740 shutdown_connections 2024-09-10T21:35:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 --2- 172.21.15.97:0/4003485740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66bc004810 0x7f66bc004c90 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 -- 172.21.15.97:0/4003485740 >> 172.21.15.97:0/4003485740 conn(0x7f66d80cd680 msgr2=0x7f66d80cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 -- 172.21.15.97:0/4003485740 shutdown_connections 2024-09-10T21:35:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 -- 172.21.15.97:0/4003485740 wait complete. 2024-09-10T21:35:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 Processor -- start 2024-09-10T21:35:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.297+0000 7f66de600640 1 -- start start 2024-09-10T21:35:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f66bc004810 0x7f66d809e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66d8142c80 0x7f66d809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d809af80 0x7f66d81f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8073b50 con 0x7f66d809af80 2024-09-10T21:35:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66d80749f0 con 0x7f66d8142c80 2024-09-10T21:35:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f66d8073cd0 con 0x7f66bc004810 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d809af80 0x7f66d81f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d809af80 0x7f66d81f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45190/0 (socket says 172.21.15.97:45190) 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 -- 172.21.15.97:0/829435213 learned_addr learned my addr 172.21.15.97:0/829435213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7400640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f66bc004810 0x7f66d809e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66cea00640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66d8142c80 0x7f66d809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 -- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f66bc004810 msgr2=0x7f66d809e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f66bc004810 0x7f66d809e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 -- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66d8142c80 msgr2=0x7f66d809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66d8142c80 0x7f66d809aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 -- 172.21.15.97:0/829435213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d809ee70 con 0x7f66d809af80 2024-09-10T21:35:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66d7e00640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d809af80 0x7f66d81f3cc0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f66d80ce300 tx=0x7f66d00129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66cfe00640 1 -- 172.21.15.97:0/829435213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0013920 con 0x7f66d809af80 2024-09-10T21:35:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66cfe00640 1 -- 172.21.15.97:0/829435213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66d0013ac0 con 0x7f66d809af80 2024-09-10T21:35:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66cfe00640 1 -- 172.21.15.97:0/829435213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0010480 con 0x7f66d809af80 2024-09-10T21:35:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d809c2e0 con 0x7f66d809af80 2024-09-10T21:35:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d809c8a0 con 0x7f66d809af80 2024-09-10T21:35:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.301+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d80b3d30 con 0x7f66d809af80 2024-09-10T21:35:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.305+0000 7f66cfe00640 1 -- 172.21.15.97:0/829435213 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f66d0010620 con 0x7f66d809af80 2024-09-10T21:35:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.305+0000 7f66cfe00640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66b40725e0 0x7f66b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.305+0000 7f66cfe00640 1 -- 172.21.15.97:0/829435213 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6217+0+0 (secure 0 0 0) 0x7f66d0097680 con 0x7f66d809af80 2024-09-10T21:35:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.305+0000 7f66d7400640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66b40725e0 0x7f66b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.305+0000 7f66d7400640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66b40725e0 0x7f66b4074aa0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f66d81e5ee0 tx=0x7f66c80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.305+0000 7f66cfe00640 1 -- 172.21.15.97:0/829435213 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66d0062580 con 0x7f66d809af80 2024-09-10T21:35:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.505+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f66d806ee30 con 0x7f66d809af80 2024-09-10T21:35:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.505+0000 7f66cfe00640 1 -- 172.21.15.97:0/829435213 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v344) ==== 53+0+73 (secure 0 0 0) 0x7f66d0066fb0 con 0x7f66d809af80 2024-09-10T21:35:54.506 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:35:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66b40725e0 msgr2=0x7f66b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66b40725e0 0x7f66b4074aa0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f66d81e5ee0 tx=0x7f66c80091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d809af80 msgr2=0x7f66d81f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d809af80 0x7f66d81f3cc0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f66d80ce300 tx=0x7f66d00129e0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 shutdown_connections 2024-09-10T21:35:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f66b40725e0 0x7f66b4074aa0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d809af80 0x7f66d81f3cc0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66d8142c80 0x7f66d809aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 --2- 172.21.15.97:0/829435213 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f66bc004810 0x7f66d809e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 >> 172.21.15.97:0/829435213 conn(0x7f66d80cd680 msgr2=0x7f66d80b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 shutdown_connections 2024-09-10T21:35:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54.509+0000 7f66de600640 1 -- 172.21.15.97:0/829435213 wait complete. 2024-09-10T21:35:54.526 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:35:54.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:35:54.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:35:54.526 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:35:54.527 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:54.527 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:35:54.527 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:35:54.527 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:35:54.527 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:35:54.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:54.527 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:35:54.536 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:35:54.536 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:35:54.536 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:35:54.536 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:35:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41739sec preferred_lft 41739sec 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:35:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:35:54.539 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:35:54.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:54.540 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:35:54.604 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:35:54.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:54.607 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:35:54.666 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:35:54.666 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:35:54.666 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:35:54.666 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:35:54.667 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:35:54.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:35:54.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:35:54.667 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:35:54.667 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:35:54.667 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:35:54.667 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:35:54.667 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:35:54.668 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:35:54.668 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:35:54.668 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:35:54.668 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:35:54.668 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:35:54.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:35:54.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:35:54.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:54.801 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:35:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:35:54.810 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:35:54.810 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:35:54.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:54.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:35:54.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:35:54.870 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:54.919 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:35:54.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:35:54.919 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:54.974 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:35:54.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:54.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:55.066 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:55.066 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:35:55.066 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:35:55.081 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:55.081 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:35:55.081 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:35:55.081 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:35:55.081 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:35:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:35:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:35:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:35:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:35:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:35:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:35:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:35:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:35:55.094 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:35:55.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:55.154 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:35:55.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:35:55.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:35:55.267 INFO:teuthology.orchestra.run.smithi097.stdout:client10797 2024-09-10T21:35:55.267 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:55.267 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:55.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.277 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2024-09-10T21:35:55.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.335 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2024-09-10T21:35:55.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.390 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2024-09-10T21:35:55.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.441 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2024-09-10T21:35:55.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.501 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-09-10T21:35:55.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.554 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2024-09-10T21:35:55.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:55.613 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2024-09-10T21:35:55.665 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:35:55.665 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:35:55.665 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:35:55.665 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:35:55.665 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2024-09-10T21:35:55.665 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:35:55.769 INFO:teuthology.orchestra.run.smithi097.stdout:1099511627777 2024-09-10T21:35:55.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:35:55.778 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:35:55.779 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:35:55.779 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:35:55.779 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:35:55.779 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2024-09-10T21:35:55.779 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:35:55.886 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628279 2024-09-10T21:35:55.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:35:55.895 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-09-10T21:35:55.895 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-09-10T21:35:55.895 DEBUG:teuthology.orchestra.run.smithi097:> import stat 2024-09-10T21:35:55.895 DEBUG:teuthology.orchestra.run.smithi097:> 2024-09-10T21:35:55.895 DEBUG:teuthology.orchestra.run.smithi097:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2024-09-10T21:35:55.895 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-09-10T21:35:56.002 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628278 2024-09-10T21:35:56.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:35:56.011 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:56.011 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:56.064 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:35:56.064 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:35:56.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:35:56.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.301+0000 7ff98f000640 1 -- 172.21.15.97:0/2660574620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff96c003a60 msgr2=0x7ff96c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.301+0000 7ff98f000640 1 --2- 172.21.15.97:0/2660574620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff96c003a60 0x7ff96c003e80 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7ff978009b60 tx=0x7ff97801d2b0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.301+0000 7ff98d000640 1 -- 172.21.15.97:0/2660574620 reap_dead start 2024-09-10T21:35:56.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.301+0000 7ff98f000640 1 -- 172.21.15.97:0/2660574620 shutdown_connections 2024-09-10T21:35:56.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.301+0000 7ff98f000640 1 -- 172.21.15.97:0/2660574620 >> 172.21.15.97:0/2660574620 conn(0x7ff9880d1c30 msgr2=0x7ff9880d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:56.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.301+0000 7ff98f000640 1 -- 172.21.15.97:0/2660574620 shutdown_connections 2024-09-10T21:35:56.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.301+0000 7ff98f000640 1 -- 172.21.15.97:0/2660574620 wait complete. 2024-09-10T21:35:56.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 Processor -- start 2024-09-10T21:35:56.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 -- start start 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff96c003a60 0x7ff9880b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff988162480 0x7ff9881fa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff988154070 0x7ff9881562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881670e0 con 0x7ff988162480 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff9880b00d0 con 0x7ff988154070 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff988167260 con 0x7ff96c003a60 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff988162480 0x7ff9881fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff988162480 0x7ff9881fa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45216/0 (socket says 172.21.15.97:45216) 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 -- 172.21.15.97:0/886856795 learned_addr learned my addr 172.21.15.97:0/886856795 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:56.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff987e00640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff96c003a60 0x7ff9880b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98d000640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff988154070 0x7ff9881562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 -- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff96c003a60 msgr2=0x7ff9880b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff96c003a60 0x7ff9880b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 -- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff988154070 msgr2=0x7ff9881562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff988154070 0x7ff9881562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 -- 172.21.15.97:0/886856795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9880b7030 con 0x7ff988162480 2024-09-10T21:35:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff97f400640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff988162480 0x7ff9881fa420 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7ff970010c40 tx=0x7ff97000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff984c00640 1 -- 172.21.15.97:0/886856795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970004d30 con 0x7ff988162480 2024-09-10T21:35:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff984c00640 1 -- 172.21.15.97:0/886856795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff970003980 con 0x7ff988162480 2024-09-10T21:35:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff984c00640 1 -- 172.21.15.97:0/886856795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970003db0 con 0x7ff988162480 2024-09-10T21:35:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9880b7310 con 0x7ff988162480 2024-09-10T21:35:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff988154500 con 0x7ff988162480 2024-09-10T21:35:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.305+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9880cea20 con 0x7ff988162480 2024-09-10T21:35:56.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.309+0000 7ff984c00640 1 -- 172.21.15.97:0/886856795 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff970023030 con 0x7ff988162480 2024-09-10T21:35:56.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.309+0000 7ff984c00640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9640725e0 0x7ff964074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.309+0000 7ff984c00640 1 -- 172.21.15.97:0/886856795 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6217+0+0 (secure 0 0 0) 0x7ff970093c50 con 0x7ff988162480 2024-09-10T21:35:56.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.309+0000 7ff987e00640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9640725e0 0x7ff964074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.309+0000 7ff987e00640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9640725e0 0x7ff964074aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff9780ae000 tx=0x7ff97801a050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:56.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.313+0000 7ff984c00640 1 -- 172.21.15.97:0/886856795 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff97005eb70 con 0x7ff988162480 2024-09-10T21:35:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.489+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff9880c50b0 con 0x7ff988162480 2024-09-10T21:35:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.489+0000 7ff984c00640 1 -- 172.21.15.97:0/886856795 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) ==== 80+0+0 (secure 0 0 0) 0x7ff9700635a0 con 0x7ff988162480 2024-09-10T21:35:56.490 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-09-10T21:35:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9640725e0 msgr2=0x7ff964074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9640725e0 0x7ff964074aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff9780ae000 tx=0x7ff97801a050 comp rx=0 tx=0).stop 2024-09-10T21:35:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff988162480 msgr2=0x7ff9881fa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff988162480 0x7ff9881fa420 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7ff970010c40 tx=0x7ff97000e720 comp rx=0 tx=0).stop 2024-09-10T21:35:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 shutdown_connections 2024-09-10T21:35:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff9640725e0 0x7ff964074aa0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff988154070 0x7ff9881562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff988162480 0x7ff9881fa420 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 --2- 172.21.15.97:0/886856795 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff96c003a60 0x7ff9880b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 >> 172.21.15.97:0/886856795 conn(0x7ff9880d1c30 msgr2=0x7ff9880af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 shutdown_connections 2024-09-10T21:35:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.493+0000 7ff98f000640 1 -- 172.21.15.97:0/886856795 wait complete. 2024-09-10T21:35:56.511 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:35:56.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:56.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:35:56.561 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:35:56.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:35:56.561 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:35:56.578 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 -- 172.21.15.97:0/1290577066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 msgr2=0x7fb2c00aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 --2- 172.21.15.97:0/1290577066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c00aedc0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0074d10 tx=0x7fb2ac01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 -- 172.21.15.97:0/1290577066 shutdown_connections 2024-09-10T21:35:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 --2- 172.21.15.97:0/1290577066 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb2c00af300 0x7fb2c00af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 --2- 172.21.15.97:0/1290577066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c00aedc0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 --2- 172.21.15.97:0/1290577066 >> v2:172.21.15.194:3300/0 conn(0x7fb2c00ce640 0x7fb2c00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 -- 172.21.15.97:0/1290577066 >> 172.21.15.97:0/1290577066 conn(0x7fb2c00d1c30 msgr2=0x7fb2c00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 -- 172.21.15.97:0/1290577066 shutdown_connections 2024-09-10T21:35:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.761+0000 7fb2c8400640 1 -- 172.21.15.97:0/1290577066 wait complete. 2024-09-10T21:35:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 Processor -- start 2024-09-10T21:35:56.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 -- start start 2024-09-10T21:35:56.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb2c00af300 0x7fb2c00b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c0140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2c00ce640 0x7fb2c01f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c00cd270 con 0x7fb2c00b0d50 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c00cd0f0 con 0x7fb2c00ce640 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c00cd3f0 con 0x7fb2c00af300 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c0140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c0140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45242/0 (socket says 172.21.15.97:45242) 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 -- 172.21.15.97:0/2987046950 learned_addr learned my addr 172.21.15.97:0/2987046950 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5a00640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb2c00af300 0x7fb2c00b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c6400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2c00ce640 0x7fb2c01f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 -- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb2c00af300 msgr2=0x7fb2c00b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb2c00af300 0x7fb2c00b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 -- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2c00ce640 msgr2=0x7fb2c01f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2c00ce640 0x7fb2c01f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 -- 172.21.15.97:0/2987046950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c00c82f0 con 0x7fb2c00b0d50 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c6400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2c00ce640 0x7fb2c01f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c5000640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c0140ec0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac01af70 tx=0x7fb2ac0a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2bd600640 1 -- 172.21.15.97:0/2987046950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0b2040 con 0x7fb2c00b0d50 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2bd600640 1 -- 172.21.15.97:0/2987046950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2ac0044a0 con 0x7fb2c00b0d50 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2bd600640 1 -- 172.21.15.97:0/2987046950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0052d0 con 0x7fb2c00b0d50 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c01f9e40 con 0x7fb2c00b0d50 2024-09-10T21:35:56.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c014f1d0 con 0x7fb2c00b0d50 2024-09-10T21:35:56.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.765+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c00cd950 con 0x7fb2c00b0d50 2024-09-10T21:35:56.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.769+0000 7fb2bd600640 1 -- 172.21.15.97:0/2987046950 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb2ac005470 con 0x7fb2c00b0d50 2024-09-10T21:35:56.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.769+0000 7fb2bd600640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb2a0072610 0x7fb2a0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:56.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.769+0000 7fb2bd600640 1 -- 172.21.15.97:0/2987046950 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6217+0+0 (secure 0 0 0) 0x7fb2ac12ca00 con 0x7fb2c00b0d50 2024-09-10T21:35:56.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.769+0000 7fb2c5a00640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb2a0072610 0x7fb2a0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:56.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.769+0000 7fb2c5a00640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb2a0072610 0x7fb2a0074ad0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00c8a10 tx=0x7fb2b400b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:56.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.773+0000 7fb2bd600640 1 -- 172.21.15.97:0/2987046950 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb2ac0164b0 con 0x7fb2c00b0d50 2024-09-10T21:35:56.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.973+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2c00d03b0 con 0x7fb2c00b0d50 2024-09-10T21:35:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.973+0000 7fb2bd600640 1 -- 172.21.15.97:0/2987046950 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 344 v344) ==== 95+0+5241 (secure 0 0 0) 0x7fb2ac0f7ed0 con 0x7fb2c00b0d50 2024-09-10T21:35:56.973 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:56.973 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":344,"btime":"2024-09-10T21:35:46:701370+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":10629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339}],"filesystems":[{"mdsmap":{"epoch":344,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:35:46.088263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10647},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10647":{"gid":10647,"name":"d","rank":0,"incarnation":340,"state":"up:active","state_seq":3,"addr":"172.21.15.194:6837/3945561983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3945561983},{"type":"v1","addr":"172.21.15.194:6837","nonce":3945561983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10647,"qdb_cluster":[10647]},"id":7}]} 2024-09-10T21:35:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 344 2024-09-10T21:35:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.973+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb2a0072610 msgr2=0x7fb2a0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.973+0000 7fb2c8400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb2a0072610 0x7fb2a0074ad0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00c8a10 tx=0x7fb2b400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.973+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 msgr2=0x7fb2c0140ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.973+0000 7fb2c8400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c0140ec0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac01af70 tx=0x7fb2ac0a6f60 comp rx=0 tx=0).stop 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 shutdown_connections 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fb2a0072610 0x7fb2a0074ad0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2c00ce640 0x7fb2c01f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00b0d50 0x7fb2c0140ec0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 --2- 172.21.15.97:0/2987046950 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fb2c00af300 0x7fb2c00b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 >> 172.21.15.97:0/2987046950 conn(0x7fb2c00d1c30 msgr2=0x7fb2c015b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 shutdown_connections 2024-09-10T21:35:56.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:56.977+0000 7fb2c8400640 1 -- 172.21.15.97:0/2987046950 wait complete. 2024-09-10T21:35:56.995 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-09-10T21:35:57.168 INFO:teuthology.orchestra.run.smithi194.stdout:{"message":"","return_code":0} 2024-09-10T21:35:57.168 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:35:57.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 -- 172.21.15.97:0/1994595166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640b0d50 msgr2=0x7f27640aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 --2- 172.21.15.97:0/1994595166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640b0d50 0x7f27640aedc0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f2750009840 tx=0x7f275001c820 comp rx=0 tx=0).stop 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 -- 172.21.15.97:0/1994595166 shutdown_connections 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 --2- 172.21.15.97:0/1994595166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640b0d50 0x7f27640aedc0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 --2- 172.21.15.97:0/1994595166 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27640ce640 0x7f27640cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 -- 172.21.15.97:0/1994595166 >> 172.21.15.97:0/1994595166 conn(0x7f27640d1c30 msgr2=0x7f27640d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 -- 172.21.15.97:0/1994595166 shutdown_connections 2024-09-10T21:35:57.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.309+0000 7f276a200640 1 -- 172.21.15.97:0/1994595166 wait complete. 2024-09-10T21:35:57.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 Processor -- start 2024-09-10T21:35:57.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 -- start start 2024-09-10T21:35:57.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640af300 0x7f27640cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27640b0d50 0x7f2764158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27640ce640 0x7f276406edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2764169270 con 0x7f27640af300 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27641690f0 con 0x7f27640b0d50 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f27641693f0 con 0x7f27640ce640 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640af300 0x7f27640cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640af300 0x7f27640cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46440/0 (socket says 172.21.15.97:46440) 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 -- 172.21.15.97:0/496533868 learned_addr learned my addr 172.21.15.97:0/496533868 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763e00640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27640ce640 0x7f276406edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2762a00640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27640b0d50 0x7f2764158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 -- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27640ce640 msgr2=0x7f276406edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27640ce640 0x7f276406edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 -- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27640b0d50 msgr2=0x7f2764158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27640b0d50 0x7f2764158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 -- 172.21.15.97:0/496533868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27640cc050 con 0x7f27640af300 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763e00640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27640ce640 0x7f276406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2762a00640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27640b0d50 0x7f2764158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f2763400640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640af300 0x7f27640cb520 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f275400ebf0 tx=0x7f275400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f275b400640 1 -- 172.21.15.97:0/496533868 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754010630 con 0x7f27640af300 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f275b400640 1 -- 172.21.15.97:0/496533868 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27540108c0 con 0x7f27640af300 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f275b400640 1 -- 172.21.15.97:0/496533868 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754010cf0 con 0x7f27640af300 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27640cbd70 con 0x7f27640af300 2024-09-10T21:35:57.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.313+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27640c81c0 con 0x7f27640af300 2024-09-10T21:35:57.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.317+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27640aedc0 con 0x7f27640af300 2024-09-10T21:35:57.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.317+0000 7f275b400640 1 -- 172.21.15.97:0/496533868 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2754018420 con 0x7f27640af300 2024-09-10T21:35:57.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.317+0000 7f275b400640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27440725e0 0x7f2744074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.317+0000 7f275b400640 1 -- 172.21.15.97:0/496533868 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6217+0+0 (secure 0 0 0) 0x7f27540941f0 con 0x7f27640af300 2024-09-10T21:35:57.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.317+0000 7f2762a00640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27440725e0 0x7f2744074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.317+0000 7f2762a00640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27440725e0 0x7f2744074aa0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f27500af000 tx=0x7f275001b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.321+0000 7f275b400640 1 -- 172.21.15.97:0/496533868 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f275405f6c0 con 0x7f27640af300 2024-09-10T21:35:57.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.517+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27640c7cb0 con 0x7f27640af300 2024-09-10T21:35:57.518 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:35:57.517+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:35:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.521+0000 7f275b400640 1 -- 172.21.15.97:0/496533868 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v345) ==== 161+0+0 (secure 0 0 0) 0x7f275405efc0 con 0x7f27640af300 2024-09-10T21:35:57.521 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27440725e0 msgr2=0x7f2744074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27440725e0 0x7f2744074aa0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f27500af000 tx=0x7f275001b060 comp rx=0 tx=0).stop 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640af300 msgr2=0x7f27640cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640af300 0x7f27640cb520 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f275400ebf0 tx=0x7f275400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 shutdown_connections 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f27440725e0 0x7f2744074aa0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f27640ce640 0x7f276406edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27640b0d50 0x7f2764158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 --2- 172.21.15.97:0/496533868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27640af300 0x7f27640cb520 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 >> 172.21.15.97:0/496533868 conn(0x7f27640d1c30 msgr2=0x7f276415dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 shutdown_connections 2024-09-10T21:35:57.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.525+0000 7f276a200640 1 -- 172.21.15.97:0/496533868 wait complete. 2024-09-10T21:35:57.536 INFO:tasks.ceph.mds.d.smithi194.stderr: -7195> 2024-09-10T21:35:45.649+0000 7f5747000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:35:57.544 DEBUG:teuthology.orchestra.run.smithi097:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-09-10T21:35:57.586 INFO:tasks.ceph.mds.d.smithi194.stdout:starting mds.d at 2024-09-10T21:35:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.589+0000 7f6381879400 1 -- start start 2024-09-10T21:35:57.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.589+0000 7f6381879400 1 Processor -- start 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56215511d800 0x562155ef0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56215511d400 0x562155ef0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56215511d000 0x562155ef0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562155f12600 con 0x56215511d000 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562155f12480 con 0x56215511d800 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x562155f12300 con 0x56215511d400 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56215511d000 0x562155ef0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f637fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56215511d800 0x562155ef0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56215511d000 0x562155ef0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46452/0 (socket says 172.21.15.97:46452) 2024-09-10T21:35:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f637f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56215511d400 0x562155ef0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 -- 172.21.15.97:0/3266023077 learned_addr learned my addr 172.21.15.97:0/3266023077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 -- 172.21.15.97:0/3266023077 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56215511d400 msgr2=0x562155ef0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 --2- 172.21.15.97:0/3266023077 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56215511d400 0x562155ef0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 -- 172.21.15.97:0/3266023077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56215511d800 msgr2=0x562155ef0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 --2- 172.21.15.97:0/3266023077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56215511d800 0x562155ef0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 -- 172.21.15.97:0/3266023077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562155eea960 con 0x56215511d000 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6380400640 1 --2- 172.21.15.97:0/3266023077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56215511d000 0x562155ef0b00 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x562155f89b30 tx=0x562155200c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d53032254bdc5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f637c800640 1 -- 172.21.15.97:0/3266023077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562155155520 con 0x56215511d000 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f637c800640 1 -- 172.21.15.97:0/3266023077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562155172fc0 con 0x56215511d000 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f637c800640 1 -- 172.21.15.97:0/3266023077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5621551551e0 con 0x56215511d000 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 -- 172.21.15.97:0/3266023077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562155eead20 con 0x56215511d000 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 -- 172.21.15.97:0/3266023077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562155eeaf00 con 0x56215511d000 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 4 waiting for MDS map... 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f637c800640 1 -- 172.21.15.97:0/3266023077 <== mon.0 v2:172.21.15.97:3300/0 4 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x562155ef5000 con 0x56215511d000 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f637c800640 1 -- 172.21.15.97:0/3266023077 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 345) ==== 2893+0+0 (secure 0 0 0) 0x562155175180 con 0x56215511d000 2024-09-10T21:35:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 4 Got MDS map 345 2024-09-10T21:35:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 10 main: JournalTool::main 2024-09-10T21:35:57.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 4 main: JournalTool: connecting to RADOS... 2024-09-10T21:35:57.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 Processor -- start 2024-09-10T21:35:57.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.593+0000 7f6381879400 1 -- start start 2024-09-10T21:35:57.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562155f12180 con 0x562156055800 2024-09-10T21:35:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562155f12900 con 0x562156055400 2024-09-10T21:35:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56215517f800 con 0x562156055000 2024-09-10T21:35:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46454/0 (socket says 172.21.15.97:46454) 2024-09-10T21:35:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:48828/0 (socket says 172.21.15.97:48828) 2024-09-10T21:35:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/699510476 learned_addr learned my addr 172.21.15.97:0/699510476 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/699510476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055000 msgr2=0x562155ef1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- 172.21.15.97:0/699510476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/699510476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 msgr2=0x562155ef1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:57.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- 172.21.15.97:0/699510476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/699510476 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562155eead20 con 0x562156055800 2024-09-10T21:35:57.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- 172.21.15.97:0/699510476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055800 0x562155ef1b80 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x562155f89260 tx=0x56215606ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c343dce62baae544 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637a000640 1 -- 172.21.15.97:0/699510476 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562155155380 con 0x562156055800 2024-09-10T21:35:57.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637a000640 1 -- 172.21.15.97:0/699510476 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562155172e00 con 0x562156055800 2024-09-10T21:35:57.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637a000640 1 -- 172.21.15.97:0/699510476 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562155155520 con 0x562156055800 2024-09-10T21:35:57.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- 172.21.15.97:0/699510476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055800 msgr2=0x562155ef1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- 172.21.15.97:0/699510476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055800 0x562155ef1b80 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x562155f89260 tx=0x56215606ac60 comp rx=0 tx=0).stop 2024-09-10T21:35:57.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- 172.21.15.97:0/699510476 shutdown_connections 2024-09-10T21:35:57.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- 172.21.15.97:0/699510476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- 172.21.15.97:0/699510476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- 172.21.15.97:0/699510476 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- 172.21.15.97:0/699510476 >> 172.21.15.97:0/699510476 conn(0x56215511cc00 msgr2=0x56215514b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- 172.21.15.97:0/699510476 shutdown_connections 2024-09-10T21:35:57.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- 172.21.15.97:0/699510476 wait complete. 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 Processor -- start 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- start start 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562155f13380 con 0x562156055000 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562155f12900 con 0x562156055400 2024-09-10T21:35:57.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x562155f13500 con 0x562156055800 2024-09-10T21:35:57.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46470/0 (socket says 172.21.15.97:46470) 2024-09-10T21:35:57.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/1806411142 learned_addr learned my addr 172.21.15.97:0/1806411142 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:57.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637fa00640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055800 msgr2=0x562155ef1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 msgr2=0x562155ef1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.597+0000 7f637f000640 1 -- 172.21.15.97:0/1806411142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562155eebc20 con 0x562156055000 2024-09-10T21:35:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f637f000640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055000 0x562155ef1080 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x56215604b050 tx=0x562155efe480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6378200640 1 -- 172.21.15.97:0/1806411142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5621551551e0 con 0x562156055000 2024-09-10T21:35:57.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6378200640 1 -- 172.21.15.97:0/1806411142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562155172c40 con 0x562156055000 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6378200640 1 -- 172.21.15.97:0/1806411142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562155154820 con 0x562156055000 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562155eebe00 con 0x562156055000 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562155eea1e0 con 0x562156055000 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6381879400 4 main: JournalTool: resolving pool 15 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6378200640 1 -- 172.21.15.97:0/1806411142 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x562155175500 con 0x562156055000 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6378200640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x562155574000 0x562155ef2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6378200640 1 -- 172.21.15.97:0/1806411142 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x562156042400 con 0x562156055000 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6381879400 4 main: JournalTool: creating IoCtx.. 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f6381879400 4 main: Executing for rank 0 2024-09-10T21:35:57.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f637fa00640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x562155574000 0x562155ef2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.601+0000 7f637fa00640 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x562155574000 0x562155ef2100 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x5621552090e0 tx=0x56215555baa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 -- start start 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 Processor -- start 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562155575000 0x562155ef2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562155575400 0x562155ef2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562155575800 0x562155ef3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562155f13800 con 0x562155575000 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562155f13980 con 0x562155575800 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x56215517f980 con 0x562155575400 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562155575000 0x562155ef2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562155575000 0x562155ef2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46480/0 (socket says 172.21.15.97:46480) 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 -- 172.21.15.97:0/931956346 learned_addr learned my addr 172.21.15.97:0/931956346 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f637fa00640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562155575400 0x562155ef2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f637f000640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562155575800 0x562155ef3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562155575400 msgr2=0x562155ef2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562155575400 0x562155ef2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562155575800 msgr2=0x562155ef3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562155575800 0x562155ef3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621555c2780 con 0x562155575000 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f637f000640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562155575800 0x562155ef3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f637fa00640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562155575400 0x562155ef2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6380400640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562155575000 0x562155ef2680 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x56215550dd10 tx=0x562155207260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad22f32aeead609 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6374600640 1 -- 172.21.15.97:0/931956346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562155155040 con 0x562155575000 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6374600640 1 -- 172.21.15.97:0/931956346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562155172e00 con 0x562155575000 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621555c2960 con 0x562155575000 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5621555c2b40 con 0x562155575000 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 4 waiting for MDS map... 2024-09-10T21:35:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6374600640 1 -- 172.21.15.97:0/931956346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56215549c820 con 0x562155575000 2024-09-10T21:35:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6374600640 1 -- 172.21.15.97:0/931956346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x5621555d4200 con 0x562155575000 2024-09-10T21:35:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6374600640 1 -- 172.21.15.97:0/931956346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 345) ==== 2893+0+0 (secure 0 0 0) 0x562155175880 con 0x562155575000 2024-09-10T21:35:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 4 Got MDS map 345 2024-09-10T21:35:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x562155575c00 0x562155ef3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149) -- 0x5621555d7800 con 0x562155575c00 2024-09-10T21:35:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f637fa00640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x562155575c00 0x562155ef3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.605+0000 7f637fa00640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x562155575c00 0x562155ef3700 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=e4cb9ba9538636ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f637fa00640 1 -- 172.21.15.97:0/931956346 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x56215523c000 con 0x562155575c00 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5621555d7c00 0x5621555f2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149) -- 0x5621555f6000 con 0x5621555d7c00 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f637f000640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5621555d7c00 0x5621555f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f637f000640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5621555d7c00 0x5621555f2000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=236802d97586c6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv22 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x56215523c500 con 0x5621555d7c00 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f6372800640 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e149) -- 0x5621555f8000 con 0x5621555d7c00 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f6372800640 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e149) -- 0x5621555f8400 con 0x5621555d7c00 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x56215523c500 con 0x5621555d7c00 2024-09-10T21:35:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 <== osd.3 v2:172.21.15.97:6800/1724813549 3 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56215523c500 con 0x5621555d7c00 2024-09-10T21:35:57.623 INFO:teuthology.orchestra.run.smithi097.stdout:old journal was 4224631~2030 2024-09-10T21:35:57.623 INFO:teuthology.orchestra.run.smithi097.stdout:new journal start will be 8388608 (4161947 bytes past old end) 2024-09-10T21:35:57.623 INFO:teuthology.orchestra.run.smithi097.stdout:writing journal head 2024-09-10T21:35:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.609+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:5 15.14 15:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f6400 con 0x5621555d7c00 2024-09-10T21:35:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 <== osd.3 v2:172.21.15.97:6800/1724813549 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v149'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56215523c500 con 0x5621555d7c00 2024-09-10T21:35:57.624 INFO:teuthology.orchestra.run.smithi097.stdout:writing EResetJournal entry 2024-09-10T21:35:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:6 15.1b 15:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f6800 con 0x5621555d7c00 2024-09-10T21:35:57.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x5621555f6c00 0x5621555f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- osd_op(unknown.0.0:7 15.b 15:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f7000 con 0x5621555f6c00 2024-09-10T21:35:57.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5621555f7400 0x5621555f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- osd_op(unknown.0.0:8 15.1f 15:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f7800 con 0x5621555f7400 2024-09-10T21:35:57.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5621555f7c00 0x5621555f3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- osd_op(unknown.0.0:9 15.1a 15:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f8000 con 0x5621555f7c00 2024-09-10T21:35:57.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- osd_op(unknown.0.0:10 15.1e 15:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f8400 con 0x5621555f6c00 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5621555f8c00 0x5621555f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- osd_op(unknown.0.0:11 15.1d 15:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f9000 con 0x5621555f8c00 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f637f000640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5621555f7c00 0x5621555f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6380400640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5621555f8c00 0x5621555f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6380400640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x5621555f6c00 0x5621555f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f637fa00640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5621555f7400 0x5621555f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6380400640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5621555f8c00 0x5621555f3600 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=6e1daae483080cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f6380400640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x5621555f6c00 0x5621555f2580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=6d1d724f9c0371a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f637f000640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5621555f7c00 0x5621555f3080 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=3d8b268b7ef34940 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f637fa00640 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5621555f7400 0x5621555f2b00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=e3ceb9b5a2cd4ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.613+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 <== osd.3 v2:172.21.15.97:6800/1724813549 5 ==== osd_op_reply(6 200.00000002 [delete] v149'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56215523c500 con 0x5621555d7c00 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f637fa00640 1 -- 172.21.15.97:0/931956346 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== osd_op_reply(8 200.00000004 [delete] v149'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56215523c000 con 0x5621555f7400 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== osd_op_reply(9 200.00000005 [delete] v149'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56215523c500 con 0x5621555f7c00 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6380400640 1 -- 172.21.15.97:0/931956346 <== osd.5 v2:172.21.15.194:6818/3659294892 1 ==== osd_op_reply(7 200.00000003 [delete] v149'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56215523c780 con 0x5621555f6c00 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6372800640 1 -- 172.21.15.97:0/931956346 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- osd_op(unknown.0.0:12 15.1b 15:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x5621555f8800 con 0x5621555d7c00 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6380400640 1 -- 172.21.15.97:0/931956346 <== osd.5 v2:172.21.15.194:6818/3659294892 2 ==== osd_op_reply(10 200.00000006 [delete] v149'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56215523c780 con 0x5621555f6c00 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6380400640 1 -- 172.21.15.97:0/931956346 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== osd_op_reply(11 200.00000007 [delete] v149'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56215523c780 con 0x5621555f8c00 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 <== osd.3 v2:172.21.15.97:6800/1724813549 6 ==== osd_op_reply(12 200.00000002 [write 0~48] v149'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56215523c500 con 0x5621555d7c00 2024-09-10T21:35:57.630 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-09-10T21:35:57.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5621555f8c00 msgr2=0x5621555f3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x5621555f8c00 0x5621555f3600 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5621555f7c00 msgr2=0x5621555f3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.617+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x5621555f7c00 0x5621555f3080 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x562155575c00 msgr2=0x562155ef3700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x562155575c00 0x562155ef3700 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5621555d7c00 msgr2=0x5621555f2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x5621555d7c00 0x5621555f2000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x5621555f6c00 msgr2=0x5621555f2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x5621555f6c00 0x5621555f2580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5621555f7400 msgr2=0x5621555f2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x5621555f7400 0x5621555f2b00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 reap_dead start 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 reap_dead start 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562155575000 msgr2=0x562155ef2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/931956346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562155575000 0x562155ef2680 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x56215550dd10 tx=0x562155207260 comp rx=0 tx=0).stop 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f637f000640 1 -- 172.21.15.97:0/931956346 reap_dead start 2024-09-10T21:35:57.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 shutdown_connections 2024-09-10T21:35:57.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 >> 172.21.15.97:0/931956346 conn(0x562156055c00 msgr2=0x5621555ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 shutdown_connections 2024-09-10T21:35:57.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/931956346 wait complete. 2024-09-10T21:35:57.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x562155574000 msgr2=0x562155ef2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x562155574000 0x562155ef2100 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x5621552090e0 tx=0x56215555baa0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055000 msgr2=0x562155ef1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055000 0x562155ef1080 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x56215604b050 tx=0x562155efe480 comp rx=0 tx=0).stop 2024-09-10T21:35:57.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 shutdown_connections 2024-09-10T21:35:57.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x562155574000 0x562155ef2100 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x562156055800 0x562155ef1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562156055400 0x562155ef1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/1806411142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562156055000 0x562155ef1080 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 >> 172.21.15.97:0/1806411142 conn(0x56215511cc00 msgr2=0x56215514b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 shutdown_connections 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/1806411142 wait complete. 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/3266023077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56215511d000 msgr2=0x562155ef0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/3266023077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56215511d000 0x562155ef0b00 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x562155f89b30 tx=0x562155200c00 comp rx=0 tx=0).stop 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/3266023077 shutdown_connections 2024-09-10T21:35:57.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/3266023077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56215511d000 0x562155ef0b00 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/3266023077 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56215511d400 0x562155ef0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 --2- 172.21.15.97:0/3266023077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56215511d800 0x562155ef0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/3266023077 >> 172.21.15.97:0/3266023077 conn(0x56215511dc00 msgr2=0x56215514b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/3266023077 shutdown_connections 2024-09-10T21:35:57.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.621+0000 7f6381879400 1 -- 172.21.15.97:0/3266023077 wait complete. 2024-09-10T21:35:57.639 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.095766, result: old journal was 4224631~2030 new journal start will be 8388608 (4161947 bytes past old end) writing journal head writing EResetJournal entry done 2024-09-10T21:35:57.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:35:57.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- 172.21.15.97:0/2196096456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415e750 msgr2=0x7f4c1415b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 --2- 172.21.15.97:0/2196096456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415e750 0x7f4c1415b030 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009840 tx=0x7f4c0001c790 comp rx=0 tx=0).stop 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- 172.21.15.97:0/2196096456 shutdown_connections 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 --2- 172.21.15.97:0/2196096456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415e750 0x7f4c1415b030 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- 172.21.15.97:0/2196096456 >> 172.21.15.97:0/2196096456 conn(0x7f4c140b28c0 msgr2=0x7f4c140b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- 172.21.15.97:0/2196096456 shutdown_connections 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- 172.21.15.97:0/2196096456 wait complete. 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 Processor -- start 2024-09-10T21:35:57.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- start start 2024-09-10T21:35:57.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c1415b570 0x7f4c140b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1415e750 0x7f4c1415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415f150 0x7f4c14158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14168f40 con 0x7f4c1415e750 2024-09-10T21:35:57.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14168dc0 con 0x7f4c1415f150 2024-09-10T21:35:57.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c1be00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4c141690c0 con 0x7f4c1415b570 2024-09-10T21:35:57.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c18a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1415e750 0x7f4c1415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c18a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1415e750 0x7f4c1415a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46482/0 (socket says 172.21.15.97:46482) 2024-09-10T21:35:57.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c18a00640 1 -- 172.21.15.97:0/3492029749 learned_addr learned my addr 172.21.15.97:0/3492029749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c19400640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c1415b570 0x7f4c140b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.833+0000 7f4c19e00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415f150 0x7f4c14158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c18a00640 1 -- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c1415b570 msgr2=0x7f4c140b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c18a00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c1415b570 0x7f4c140b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c18a00640 1 -- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415f150 msgr2=0x7f4c14158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c18a00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415f150 0x7f4c14158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c18a00640 1 -- 172.21.15.97:0/3492029749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c140b1c00 con 0x7f4c1415e750 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c18a00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1415e750 0x7f4c1415a7b0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f4c000193a0 tx=0x7f4c00005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c11600640 1 -- 172.21.15.97:0/3492029749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c000b2040 con 0x7f4c1415e750 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c11600640 1 -- 172.21.15.97:0/3492029749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c00004240 con 0x7f4c1415e750 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c141511e0 con 0x7f4c1415e750 2024-09-10T21:35:57.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c11600640 1 -- 172.21.15.97:0/3492029749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00005020 con 0x7f4c1415e750 2024-09-10T21:35:57.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c141593b0 con 0x7f4c1415e750 2024-09-10T21:35:57.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1415b030 con 0x7f4c1415e750 2024-09-10T21:35:57.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c11600640 1 -- 172.21.15.97:0/3492029749 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4c000b5070 con 0x7f4c1415e750 2024-09-10T21:35:57.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.837+0000 7f4c11600640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bf40725e0 0x7f4bf4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.841+0000 7f4c11600640 1 -- 172.21.15.97:0/3492029749 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f4c0012c9e0 con 0x7f4c1415e750 2024-09-10T21:35:57.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.841+0000 7f4c19400640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bf40725e0 0x7f4bf4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.841+0000 7f4c19400640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bf40725e0 0x7f4bf4074aa0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4c08007f30 tx=0x7f4c08007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:57.841+0000 7f4c11600640 1 -- 172.21.15.97:0/3492029749 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4c00016410 con 0x7f4c1415e750 2024-09-10T21:35:58.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.045+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4c140ce7b0 con 0x7f4c1415e750 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.521+0000 7f4c11600640 1 -- 172.21.15.97:0/3492029749 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v346) ==== 162+0+0 (secure 0 0 0) 0x7f4c000f7780 con 0x7f4c1415e750 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bf40725e0 msgr2=0x7f4bf4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bf40725e0 0x7f4bf4074aa0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4c08007f30 tx=0x7f4c08007ec0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1415e750 msgr2=0x7f4c1415a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1415e750 0x7f4c1415a7b0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f4c000193a0 tx=0x7f4c00005ca0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 shutdown_connections 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4bf40725e0 0x7f4bf4074aa0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c1415f150 0x7f4c14158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c1415e750 0x7f4c1415a7b0 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 --2- 172.21.15.97:0/3492029749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4c1415b570 0x7f4c140b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 >> 172.21.15.97:0/3492029749 conn(0x7f4c140b28c0 msgr2=0x7f4c1415d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 shutdown_connections 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.525+0000 7f4c1be00640 1 -- 172.21.15.97:0/3492029749 wait complete. 2024-09-10T21:35:58.553 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:35:58.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 -- 172.21.15.97:0/2209100131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80b5960 msgr2=0x7f6ce815eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 --2- 172.21.15.97:0/2209100131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80b5960 0x7f6ce815eb00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009840 tx=0x7f6cdc01c820 comp rx=0 tx=0).stop 2024-09-10T21:35:58.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 -- 172.21.15.97:0/2209100131 shutdown_connections 2024-09-10T21:35:58.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 --2- 172.21.15.97:0/2209100131 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ce80cef00 0x7f6ce80b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 --2- 172.21.15.97:0/2209100131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80b5960 0x7f6ce815eb00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 -- 172.21.15.97:0/2209100131 >> 172.21.15.97:0/2209100131 conn(0x7f6ce80afa50 msgr2=0x7f6ce8074980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 -- 172.21.15.97:0/2209100131 shutdown_connections 2024-09-10T21:35:58.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 -- 172.21.15.97:0/2209100131 wait complete. 2024-09-10T21:35:58.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 Processor -- start 2024-09-10T21:35:58.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.697+0000 7f6cee400640 1 -- start start 2024-09-10T21:35:58.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80b0e70 0x7f6ce80a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ce80b5960 0x7f6ce815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80cef00 0x7f6ce815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce8167670 con 0x7f6ce80b0e70 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce815f200 con 0x7f6ce80cef00 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce81677f0 con 0x7f6ce80b5960 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80b0e70 0x7f6ce80a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80b0e70 0x7f6ce80a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46494/0 (socket says 172.21.15.97:46494) 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 -- 172.21.15.97:0/4067756345 learned_addr learned my addr 172.21.15.97:0/4067756345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ce7e00640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ce80b5960 0x7f6ce815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ceda00640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80cef00 0x7f6ce815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 -- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ce80b5960 msgr2=0x7f6ce815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ce80b5960 0x7f6ce815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 -- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80cef00 msgr2=0x7f6ce815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80cef00 0x7f6ce815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 -- 172.21.15.97:0/4067756345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce81fb440 con 0x7f6ce80b0e70 2024-09-10T21:35:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ced000640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80b0e70 0x7f6ce80a98a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009790 tx=0x7f6cdc0a6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ce4c00640 1 -- 172.21.15.97:0/4067756345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0bc030 con 0x7f6ce80b0e70 2024-09-10T21:35:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ce4c00640 1 -- 172.21.15.97:0/4067756345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cdc004120 con 0x7f6ce80b0e70 2024-09-10T21:35:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce8156030 con 0x7f6ce80b0e70 2024-09-10T21:35:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6ce4c00640 1 -- 172.21.15.97:0/4067756345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc005020 con 0x7f6ce80b0e70 2024-09-10T21:35:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce81565f0 con 0x7f6ce80b0e70 2024-09-10T21:35:58.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.701+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce815eb00 con 0x7f6ce80b0e70 2024-09-10T21:35:58.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.705+0000 7f6ce4c00640 1 -- 172.21.15.97:0/4067756345 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6cdc0051c0 con 0x7f6ce80b0e70 2024-09-10T21:35:58.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.705+0000 7f6ce4c00640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cc80725e0 0x7f6cc8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.705+0000 7f6ce4c00640 1 -- 172.21.15.97:0/4067756345 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f6cdc12de80 con 0x7f6ce80b0e70 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.705+0000 7f6ce7e00640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cc80725e0 0x7f6cc8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.705+0000 7f6ce7e00640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cc80725e0 0x7f6cc8074aa0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80aee50 tx=0x7f6cd4001460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.709+0000 7f6ce4c00640 1 -- 172.21.15.97:0/4067756345 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6cdc016410 con 0x7f6ce80b0e70 2024-09-10T21:35:58.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.909+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ce815d310 con 0x7f6ce80b0e70 2024-09-10T21:35:58.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.909+0000 7f6ce4c00640 1 -- 172.21.15.97:0/4067756345 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 347 v347) ==== 95+0+5233 (secure 0 0 0) 0x7f6cdc0f8cb0 con 0x7f6ce80b0e70 2024-09-10T21:35:58.909 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:35:58.909 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":347,"btime":"2024-09-10T21:35:58:531142+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":347,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:35:58.531114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:replay","state_seq":1,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:35:58.910 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 347 2024-09-10T21:35:58.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.909+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cc80725e0 msgr2=0x7f6cc8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.909+0000 7f6cee400640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cc80725e0 0x7f6cc8074aa0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80aee50 tx=0x7f6cd4001460 comp rx=0 tx=0).stop 2024-09-10T21:35:58.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.909+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80b0e70 msgr2=0x7f6ce80a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.909+0000 7f6cee400640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80b0e70 0x7f6ce80a98a0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009790 tx=0x7f6cdc0a6670 comp rx=0 tx=0).stop 2024-09-10T21:35:58.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 shutdown_connections 2024-09-10T21:35:58.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f6cc80725e0 0x7f6cc8074aa0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ce80cef00 0x7f6ce815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6ce80b5960 0x7f6ce815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 --2- 172.21.15.97:0/4067756345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ce80b0e70 0x7f6ce80a98a0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 >> 172.21.15.97:0/4067756345 conn(0x7f6ce80afa50 msgr2=0x7f6ce813fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 shutdown_connections 2024-09-10T21:35:58.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:35:58.913+0000 7f6cee400640 1 -- 172.21.15.97:0/4067756345 wait complete. 2024-09-10T21:35:59.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 347, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:35:43.069729+0000', 'modified': '2024-09-10T21:35:58.531114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 149, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10629}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10629': {'gid': 10629, 'name': 'a', 'rank': 0, 'incarnation': 347, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/2844434294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2844434294}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2844434294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:35:59.333 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10629:up:replay 2024-09-10T21:36:00.334 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:00.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 -- 172.21.15.97:0/1265418536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddb40051d0 msgr2=0x7fddb400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:00.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 --2- 172.21.15.97:0/1265418536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddb40051d0 0x7fddb400b9d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fddc800c290 tx=0x7fddc801f700 comp rx=0 tx=0).stop 2024-09-10T21:36:00.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddce800640 1 -- 172.21.15.97:0/1265418536 reap_dead start 2024-09-10T21:36:00.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 -- 172.21.15.97:0/1265418536 shutdown_connections 2024-09-10T21:36:00.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 -- 172.21.15.97:0/1265418536 >> 172.21.15.97:0/1265418536 conn(0x7fddd00b1780 msgr2=0x7fddd00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:00.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 -- 172.21.15.97:0/1265418536 shutdown_connections 2024-09-10T21:36:00.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 -- 172.21.15.97:0/1265418536 wait complete. 2024-09-10T21:36:00.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 Processor -- start 2024-09-10T21:36:00.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.481+0000 7fddd5200640 1 -- start start 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddb40051d0 0x7fddd01401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddd01507d0 0x7fddd01f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddd01f88b0 0x7fddd01495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fddd00b3b40 con 0x7fddd01f88b0 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fddd00b39c0 con 0x7fddb40051d0 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fddd006e3b0 con 0x7fddd01507d0 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddd01f88b0 0x7fddd01495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddd01f88b0 0x7fddd01495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46506/0 (socket says 172.21.15.97:46506) 2024-09-10T21:36:00.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 -- 172.21.15.97:0/2779249102 learned_addr learned my addr 172.21.15.97:0/2779249102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddcde00640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddb40051d0 0x7fddd01401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddc5400640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddd01507d0 0x7fddd01f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 -- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddd01507d0 msgr2=0x7fddd01f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddd01507d0 0x7fddd01f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 -- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddb40051d0 msgr2=0x7fddd01401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddb40051d0 0x7fddd01401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 -- 172.21.15.97:0/2779249102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd014e300 con 0x7fddd01f88b0 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddc5400640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddd01507d0 0x7fddd01f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddce800640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddd01f88b0 0x7fddd01495c0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fddc80263d0 tx=0x7fddc801f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:00.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddc6a00640 1 -- 172.21.15.97:0/2779249102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc80b3070 con 0x7fddd01f88b0 2024-09-10T21:36:00.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddc6a00640 1 -- 172.21.15.97:0/2779249102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddc8006380 con 0x7fddd01f88b0 2024-09-10T21:36:00.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddc6a00640 1 -- 172.21.15.97:0/2779249102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8003600 con 0x7fddd01f88b0 2024-09-10T21:36:00.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd014e5e0 con 0x7fddd01f88b0 2024-09-10T21:36:00.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd0140a40 con 0x7fddd01f88b0 2024-09-10T21:36:00.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd00ceee0 con 0x7fddd01f88b0 2024-09-10T21:36:00.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.485+0000 7fddc6a00640 1 -- 172.21.15.97:0/2779249102 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fddc80b9070 con 0x7fddd01f88b0 2024-09-10T21:36:00.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.489+0000 7fddc6a00640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddac0725e0 0x7fddac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:00.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.489+0000 7fddc6a00640 1 -- 172.21.15.97:0/2779249102 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fddc8130720 con 0x7fddd01f88b0 2024-09-10T21:36:00.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.489+0000 7fddcde00640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddac0725e0 0x7fddac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:00.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.489+0000 7fddcde00640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddac0725e0 0x7fddac074aa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fddd0074f10 tx=0x7fddc000a610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.493+0000 7fddc6a00640 1 -- 172.21.15.97:0/2779249102 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fddd0140a40 con 0x7fddd01f88b0 2024-09-10T21:36:00.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.693+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddd01e2ad0 con 0x7fddd01f88b0 2024-09-10T21:36:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.693+0000 7fddc6a00640 1 -- 172.21.15.97:0/2779249102 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+5233 (secure 0 0 0) 0x7fddc80c1060 con 0x7fddd01f88b0 2024-09-10T21:36:00.693 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:00.693 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":349,"btime":"2024-09-10T21:36:00:537784+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":349,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:35:59.538997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:36:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 349 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddac0725e0 msgr2=0x7fddac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddac0725e0 0x7fddac074aa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fddd0074f10 tx=0x7fddc000a610 comp rx=0 tx=0).stop 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddd01f88b0 msgr2=0x7fddd01495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddd01f88b0 0x7fddd01495c0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fddc80263d0 tx=0x7fddc801f950 comp rx=0 tx=0).stop 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 shutdown_connections 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fddac0725e0 0x7fddac074aa0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddd01f88b0 0x7fddd01495c0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fddd01507d0 0x7fddd01f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 --2- 172.21.15.97:0/2779249102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddb40051d0 0x7fddd01401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 >> 172.21.15.97:0/2779249102 conn(0x7fddd00b1780 msgr2=0x7fddd006e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 shutdown_connections 2024-09-10T21:36:00.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:00.697+0000 7fddd5200640 1 -- 172.21.15.97:0/2779249102 wait complete. 2024-09-10T21:36:01.118 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 349, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:35:43.069729+0000', 'modified': '2024-09-10T21:35:59.538997+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 149, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10629}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10629': {'gid': 10629, 'name': 'a', 'rank': 0, 'incarnation': 347, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.97:6835/2844434294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2844434294}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2844434294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:36:01.118 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10629:up:rejoin 2024-09-10T21:36:02.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 -- 172.21.15.97:0/3613137909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 msgr2=0x7f1ad8004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 --2- 172.21.15.97:0/3613137909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 0x7f1ad8004020 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4001f50 tx=0x7f1ae401cc40 comp rx=0 tx=0).stop 2024-09-10T21:36:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1af3e00640 1 -- 172.21.15.97:0/3613137909 reap_dead start 2024-09-10T21:36:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 -- 172.21.15.97:0/3613137909 shutdown_connections 2024-09-10T21:36:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 -- 172.21.15.97:0/3613137909 >> 172.21.15.97:0/3613137909 conn(0x7f1af40b1780 msgr2=0x7f1af40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 -- 172.21.15.97:0/3613137909 shutdown_connections 2024-09-10T21:36:02.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 -- 172.21.15.97:0/3613137909 wait complete. 2024-09-10T21:36:02.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 Processor -- start 2024-09-10T21:36:02.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.273+0000 7f1afa600640 1 -- start start 2024-09-10T21:36:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 0x7f1af413e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1af406f400 0x7f1af406f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1af4142470 0x7f1af41428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4167b20 con 0x7f1ad8003c20 2024-09-10T21:36:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4160080 con 0x7f1af406f400 2024-09-10T21:36:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1af4167ca0 con 0x7f1af4142470 2024-09-10T21:36:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 0x7f1af413e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 0x7f1af413e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46524/0 (socket says 172.21.15.97:46524) 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 -- 172.21.15.97:0/663530414 learned_addr learned my addr 172.21.15.97:0/663530414 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3e00640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1af4142470 0x7f1af41428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af2a00640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1af406f400 0x7f1af406f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 -- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1af4142470 msgr2=0x7f1af41428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1af4142470 0x7f1af41428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 -- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1af406f400 msgr2=0x7f1af406f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1af406f400 0x7f1af406f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 -- 172.21.15.97:0/663530414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af413eb30 con 0x7f1ad8003c20 2024-09-10T21:36:02.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1af3400640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 0x7f1af413e5f0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f1ae401b340 tx=0x7f1ae40a5f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1aeb400640 1 -- 172.21.15.97:0/663530414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4006c50 con 0x7f1ad8003c20 2024-09-10T21:36:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1aeb400640 1 -- 172.21.15.97:0/663530414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ae40b4030 con 0x7f1ad8003c20 2024-09-10T21:36:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1aeb400640 1 -- 172.21.15.97:0/663530414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4006150 con 0x7f1ad8003c20 2024-09-10T21:36:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af41f5a80 con 0x7f1ad8003c20 2024-09-10T21:36:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af41f5d40 con 0x7f1ad8003c20 2024-09-10T21:36:02.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af40ce9d0 con 0x7f1ad8003c20 2024-09-10T21:36:02.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.277+0000 7f1aeb400640 1 -- 172.21.15.97:0/663530414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ae40062f0 con 0x7f1ad8003c20 2024-09-10T21:36:02.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.281+0000 7f1aeb400640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ad4072610 0x7f1ad4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.281+0000 7f1aeb400640 1 -- 172.21.15.97:0/663530414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f1ae412d850 con 0x7f1ad8003c20 2024-09-10T21:36:02.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.281+0000 7f1af2a00640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ad4072610 0x7f1ad4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:02.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.281+0000 7f1af2a00640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ad4072610 0x7f1ad4074ad0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1af4074380 tx=0x7f1ae00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:02.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.285+0000 7f1aeb400640 1 -- 172.21.15.97:0/663530414 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ae40169f0 con 0x7f1ad8003c20 2024-09-10T21:36:02.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1af4141360 con 0x7f1ad8003c20 2024-09-10T21:36:02.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1aeb400640 1 -- 172.21.15.97:0/663530414 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7f1ae40f8d50 con 0x7f1ad8003c20 2024-09-10T21:36:02.481 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:02.481 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:02.481 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ad4072610 msgr2=0x7f1ad4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ad4072610 0x7f1ad4074ad0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1af4074380 tx=0x7f1ae00091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 msgr2=0x7f1af413e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 0x7f1af413e5f0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f1ae401b340 tx=0x7f1ae40a5f30 comp rx=0 tx=0).stop 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 shutdown_connections 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1ad4072610 0x7f1ad4074ad0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1af4142470 0x7f1af41428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1af406f400 0x7f1af406f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 --2- 172.21.15.97:0/663530414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad8003c20 0x7f1af413e5f0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.481+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 >> 172.21.15.97:0/663530414 conn(0x7f1af40b1780 msgr2=0x7f1af415c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.485+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 shutdown_connections 2024-09-10T21:36:02.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:02.485+0000 7f1afa600640 1 -- 172.21.15.97:0/663530414 wait complete. 2024-09-10T21:36:02.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 350, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:35:43.069729+0000', 'modified': '2024-09-10T21:36:01.544302+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 149, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10629}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10629': {'gid': 10629, 'name': 'a', 'rank': 0, 'incarnation': 347, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.97:6835/2844434294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6833', 'nonce': 2844434294}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 2844434294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10629, 'qdb_cluster': [10629]} 2024-09-10T21:36:02.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:36:02.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- 172.21.15.97:0/4022953063 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b440ce5f0 msgr2=0x7f2b44141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 --2- 172.21.15.97:0/4022953063 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b440ce5f0 0x7f2b44141d20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009840 tx=0x7f2b3401c820 comp rx=0 tx=0).stop 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- 172.21.15.97:0/4022953063 shutdown_connections 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 --2- 172.21.15.97:0/4022953063 >> v2:172.21.15.194:3300/0 conn(0x7f2b4415fde0 0x7f2b4413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 --2- 172.21.15.97:0/4022953063 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b440ce5f0 0x7f2b44141d20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- 172.21.15.97:0/4022953063 >> 172.21.15.97:0/4022953063 conn(0x7f2b440b1780 msgr2=0x7f2b440b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- 172.21.15.97:0/4022953063 shutdown_connections 2024-09-10T21:36:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- 172.21.15.97:0/4022953063 wait complete. 2024-09-10T21:36:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 Processor -- start 2024-09-10T21:36:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- start start 2024-09-10T21:36:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b440ce5f0 0x7f2b4413ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b440ce9d0 0x7f2b4413d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b4415fde0 0x7f2b4413d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44168bf0 con 0x7f2b440ce5f0 2024-09-10T21:36:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44168a70 con 0x7f2b440ce9d0 2024-09-10T21:36:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.049+0000 7f2b4b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b44168d70 con 0x7f2b4415fde0 2024-09-10T21:36:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b440ce5f0 0x7f2b4413ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b440ce5f0 0x7f2b4413ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46536/0 (socket says 172.21.15.97:46536) 2024-09-10T21:36:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 -- 172.21.15.97:0/68616959 learned_addr learned my addr 172.21.15.97:0/68616959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b49200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b4415fde0 0x7f2b4413d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43400640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b440ce9d0 0x7f2b4413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 -- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b4415fde0 msgr2=0x7f2b4413d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b4415fde0 0x7f2b4413d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 -- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b440ce9d0 msgr2=0x7f2b4413d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b440ce9d0 0x7f2b4413d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 -- 172.21.15.97:0/68616959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4413cb70 con 0x7f2b440ce5f0 2024-09-10T21:36:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43400640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b440ce9d0 0x7f2b4413d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b43e00640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b440ce5f0 0x7f2b4413ac00 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009790 tx=0x7f2b34005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b3be00640 1 -- 172.21.15.97:0/68616959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340b2040 con 0x7f2b440ce5f0 2024-09-10T21:36:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b3be00640 1 -- 172.21.15.97:0/68616959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b34004320 con 0x7f2b440ce5f0 2024-09-10T21:36:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b3be00640 1 -- 172.21.15.97:0/68616959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340044f0 con 0x7f2b440ce5f0 2024-09-10T21:36:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4413b140 con 0x7f2b440ce5f0 2024-09-10T21:36:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.053+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b441f8a60 con 0x7f2b440ce5f0 2024-09-10T21:36:03.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b3be00640 1 -- 172.21.15.97:0/68616959 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b340b5030 con 0x7f2b440ce5f0 2024-09-10T21:36:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 Processor -- start 2024-09-10T21:36:03.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 -- start start 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b3be00640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b24072610 0x7f2b24074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b3be00640 1 -- 172.21.15.97:0/68616959 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f2b3412cb00 con 0x7f2b440ce5f0 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b49200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b24072610 0x7f2b24074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7049d0 0x5625aa704db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7052f0 0x5625aa6b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5625aa5d93f0 con 0x5625aa6b8e10 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5625aa516b20 con 0x5625aa7049d0 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5625aa505f50 con 0x5625aa7052f0 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7049d0 0x5625aa704db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7052f0 0x5625aa6b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7049d0 0x5625aa704db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:48952/0 (socket says 172.21.15.97:48952) 2024-09-10T21:36:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 -- 172.21.15.97:0/3221363163 learned_addr learned my addr 172.21.15.97:0/3221363163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:03.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b49200640 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 -- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7052f0 msgr2=0x5625aa6b88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7052f0 0x5625aa6b88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 -- 172.21.15.97:0/3221363163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 msgr2=0x5625aa6b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 -- 172.21.15.97:0/3221363163 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625aa4cd590 con 0x5625aa7049d0 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b49200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b24072610 0x7f2b24074ad0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0099c0 tx=0x7f2b3c011810 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43400640 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7052f0 0x5625aa6b88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b43e00640 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7049d0 0x5625aa704db0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f7810 tx=0x7f2b340f7710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f27efa99143348 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b38c00640 1 -- 172.21.15.97:0/3221363163 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340f7f80 con 0x5625aa7049d0 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b38c00640 1 -- 172.21.15.97:0/3221363163 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b340fc360 con 0x5625aa7049d0 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.057+0000 7f2b38c00640 1 -- 172.21.15.97:0/3221363163 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34100d80 con 0x5625aa7049d0 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7049d0 msgr2=0x5625aa704db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7049d0 0x5625aa704db0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f7810 tx=0x7f2b340f7710 comp rx=0 tx=0).stop 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/3221363163 shutdown_connections 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7052f0 0x5625aa6b88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/3221363163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7049d0 0x5625aa704db0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/3221363163 >> 172.21.15.97:0/3221363163 conn(0x5625aa5a2130 msgr2=0x5625aa6e7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/3221363163 shutdown_connections 2024-09-10T21:36:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/3221363163 wait complete. 2024-09-10T21:36:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- start start 2024-09-10T21:36:03.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 Processor -- start 2024-09-10T21:36:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa6e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5625aa4a05d0 con 0x5625aa6b8e10 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5625aa509d60 con 0x5625aa7052f0 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5625aa4ac500 con 0x5625aa7049d0 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 10 client.0 _ll_register_callbacks cb 0x5625aa4b0810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6e56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46546/0 (socket says 172.21.15.97:46546) 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 -- 172.21.15.97:0/326835006 learned_addr learned my addr 172.21.15.97:0/326835006 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43400640 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa6e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b49200640 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 -- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 msgr2=0x5625aa6e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa6e5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 -- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 msgr2=0x5625aa6e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6e63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 -- 172.21.15.97:0/326835006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625aa4a4170 con 0x5625aa6b8e10 2024-09-10T21:36:03.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b43e00640 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6e56a0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f3890 tx=0x7f2b34005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6193fb9aee8a007 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b2d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b2d600640 1 -- 172.21.15.97:0/326835006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340f7880 con 0x5625aa6b8e10 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b2d600640 1 -- 172.21.15.97:0/326835006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f2b340f7ae0 con 0x5625aa6b8e10 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5625aa49e0f0 con 0x5625aa6b8e10 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b2d600640 1 -- 172.21.15.97:0/326835006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7f2b34100d80 con 0x5625aa6b8e10 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 10 client.10878 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x5625aa58aef0 con 0x5625aa6b8e10 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b2d600640 1 -- 172.21.15.97:0/326835006 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5625aa49e0f0 con 0x5625aa6b8e10 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b2d600640 1 -- 172.21.15.97:0/326835006 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7f2b341008f0 con 0x5625aa6b8e10 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 10 client.10878 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 10 client.10878 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 20 client.10878 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa6e70f0 0x5625aa716c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 4 client.10878 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:36:03.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.061+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625aa518a70 con 0x5625aa6e70f0 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b43400640 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa6e70f0 0x5625aa716c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b2d600640 10 client.10878 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b43400640 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa6e70f0 0x5625aa716c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f5908fedf7dcf0 server_cookie=382165385073e8b5 in_seq=0 out_seq=0 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b2d600640 1 -- 172.21.15.97:0/326835006 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625aa518a70 con 0x5625aa6e70f0 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b2d600640 10 client.10878 handle_command_reply: tid=0 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 client.10878 shutdown 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 msgr2=0x5625aa6e56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6e56a0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f3890 tx=0x7f2b34005f20 comp rx=0 tx=0).stop 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 shutdown_connections 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa6e70f0 0x5625aa716c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa6e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b2d600640 0 client.10878 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:03.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/326835006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa6e56a0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 >> 172.21.15.97:0/326835006 conn(0x5625aa5a2130 msgr2=0x5625aa6b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 shutdown_connections 2024-09-10T21:36:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/326835006 wait complete. 2024-09-10T21:36:03.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.065+0000 7f2b4e6ca000 20 client.10878 trim_cache size 0 max 16384 2024-09-10T21:36:03.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 Processor -- start 2024-09-10T21:36:03.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- start start 2024-09-10T21:36:03.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa71b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa71b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b442014d0 con 0x5625aa6b8e10 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441f5fe0 con 0x5625aa7052f0 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f2b44200e80 con 0x5625aa7049d0 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa71b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa71b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46548/0 (socket says 172.21.15.97:46548) 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 -- 172.21.15.97:0/1554796121 learned_addr learned my addr 172.21.15.97:0/1554796121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43400640 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa71b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b49200640 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 -- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 msgr2=0x5625aa71b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa71b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 -- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 msgr2=0x5625aa6b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 -- 172.21.15.97:0/1554796121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625aa561d90 con 0x5625aa6b8e10 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b43e00640 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa71b0a0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f4040 tx=0x7f2b340a59e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb006db1464ac01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b2fe00640 1 -- 172.21.15.97:0/1554796121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340f6ff0 con 0x5625aa6b8e10 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b2fe00640 1 -- 172.21.15.97:0/1554796121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b340fc360 con 0x5625aa6b8e10 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b2fe00640 1 -- 172.21.15.97:0/1554796121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b341008f0 con 0x5625aa6b8e10 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1554796121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 msgr2=0x5625aa71b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa71b0a0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f4040 tx=0x7f2b340a59e0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1554796121 shutdown_connections 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa6b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa71b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1554796121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa71b0a0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1554796121 >> 172.21.15.97:0/1554796121 conn(0x5625aa5a2130 msgr2=0x5625aa7035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1554796121 shutdown_connections 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1554796121 wait complete. 2024-09-10T21:36:03.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 -- start start 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.085+0000 7f2b4e6ca000 1 Processor -- start 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa77ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa77f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa77f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5625aa4a05d0 con 0x5625aa6b8e10 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5625aa4d1ec0 con 0x5625aa7052f0 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5625aa509d60 con 0x5625aa7049d0 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 10 client.0 _ll_register_callbacks cb 0x7f2b3c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa77ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa77ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46550/0 (socket says 172.21.15.97:46550) 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 -- 172.21.15.97:0/1524987208 learned_addr learned my addr 172.21.15.97:0/1524987208 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43400640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa77f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b49200640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa77f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 -- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 msgr2=0x5625aa77f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa77f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 -- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 msgr2=0x5625aa77f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa77f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 -- 172.21.15.97:0/1524987208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625aa4cca90 con 0x5625aa6b8e10 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b49200640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa77f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43e00640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa77ebc0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f3e00 tx=0x7f2b340fc250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c694b0ecfd87c7b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 1 -- 172.21.15.97:0/1524987208 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340f3720 con 0x5625aa6b8e10 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 1 -- 172.21.15.97:0/1524987208 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f2b340f7cf0 con 0x5625aa6b8e10 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b340fc360 con 0x5625aa6b8e10 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 1 -- 172.21.15.97:0/1524987208 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7f2b340f8bb0 con 0x5625aa6b8e10 2024-09-10T21:36:03.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 10 client.10884 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x5625aa4ab050 con 0x5625aa6b8e10 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 1 -- 172.21.15.97:0/1524987208 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b340fc360 con 0x5625aa6b8e10 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 1 -- 172.21.15.97:0/1524987208 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7f2b34005280 con 0x5625aa6b8e10 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 10 client.10884 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 10 client.10884 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 20 client.10884 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa73a690 0x5625aa73aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 4 client.10884 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "status"}) -- 0x5625aa514520 con 0x5625aa73a690 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43400640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa73a690 0x5625aa73aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b43400640 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa73a690 0x5625aa73aa70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad13241cd31b0d5 server_cookie=9ad5872f05e88af3 in_seq=0 out_seq=0 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 10 client.10884 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 1 -- 172.21.15.97:0/1524987208 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x5625aa514520 con 0x5625aa73a690 2024-09-10T21:36:03.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 10 client.10884 handle_command_reply: tid=0 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 client.10884 shutdown 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 msgr2=0x5625aa77ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa77ebc0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f2b340f3e00 tx=0x7f2b340fc250 comp rx=0 tx=0).stop 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 shutdown_connections 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5625aa73a690 0x5625aa73aa70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625aa7052f0 0x5625aa77f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5625aa7049d0 0x5625aa77f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b2d600640 0 client.10884 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 --2- 172.21.15.97:0/1524987208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625aa6b8e10 0x5625aa77ebc0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 >> 172.21.15.97:0/1524987208 conn(0x5625aa5a2130 msgr2=0x5625aa703ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 shutdown_connections 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 1 -- 172.21.15.97:0/1524987208 wait complete. 2024-09-10T21:36:03.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.089+0000 7f2b4e6ca000 20 client.10884 trim_cache size 0 max 16384 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d1b4b680-9a7d-4173-b750-f3c8b85e2f7b", 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 10629, 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 4.5584573979999998, 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 350, 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 149, 2024-09-10T21:36:03.096 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 149, 2024-09-10T21:36:03.097 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 23.938107914 2024-09-10T21:36:03.097 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:03.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b24072610 msgr2=0x7f2b24074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b24072610 0x7f2b24074ad0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0099c0 tx=0x7f2b3c011810 comp rx=0 tx=0).stop 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b440ce5f0 msgr2=0x7f2b4413ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b440ce5f0 0x7f2b4413ac00 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009790 tx=0x7f2b34005dd0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 shutdown_connections 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f2b24072610 0x7f2b24074ad0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f2b4415fde0 0x7f2b4413d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b440ce9d0 0x7f2b4413d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 --2- 172.21.15.97:0/68616959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b440ce5f0 0x7f2b4413ac00 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 >> 172.21.15.97:0/68616959 conn(0x7f2b440b1780 msgr2=0x7f2b44072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 shutdown_connections 2024-09-10T21:36:03.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.093+0000 7f2b4b200640 1 -- 172.21.15.97:0/68616959 wait complete. 2024-09-10T21:36:03.111 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:36:03.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 -- 172.21.15.97:0/1473046312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190815f6d0 msgr2=0x7f190815fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 --2- 172.21.15.97:0/1473046312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190815f6d0 0x7f190815fb40 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f19080d1430 tx=0x7f18f401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 -- 172.21.15.97:0/1473046312 shutdown_connections 2024-09-10T21:36:03.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 --2- 172.21.15.97:0/1473046312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190815f6d0 0x7f190815fb40 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 --2- 172.21.15.97:0/1473046312 >> v2:172.21.15.194:3300/0 conn(0x7f1908071a40 0x7f190815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 -- 172.21.15.97:0/1473046312 >> 172.21.15.97:0/1473046312 conn(0x7f19080b2ee0 msgr2=0x7f19080d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 -- 172.21.15.97:0/1473046312 shutdown_connections 2024-09-10T21:36:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 -- 172.21.15.97:0/1473046312 wait complete. 2024-09-10T21:36:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 Processor -- start 2024-09-10T21:36:03.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.253+0000 7f1907e00640 1 -- start start 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190806cb40 0x7f19081f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1908071a40 0x7f19081655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f190815f6d0 0x7f190815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1908169070 con 0x7f190806cb40 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1908071220 con 0x7f1908071a40 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f19081691f0 con 0x7f190815f6d0 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190806cb40 0x7f19081f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190806cb40 0x7f19081f9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46568/0 (socket says 172.21.15.97:46568) 2024-09-10T21:36:03.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 -- 172.21.15.97:0/59744868 learned_addr learned my addr 172.21.15.97:0/59744868 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906000640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1908071a40 0x7f19081655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907400640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f190815f6d0 0x7f190815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 -- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f190815f6d0 msgr2=0x7f190815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f190815f6d0 0x7f190815e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 -- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1908071a40 msgr2=0x7f19081655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1908071a40 0x7f19081655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 -- 172.21.15.97:0/59744868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19081fa070 con 0x7f190806cb40 2024-09-10T21:36:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1906a00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190806cb40 0x7f19081f9b30 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d950 tx=0x7f18f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f18fea00640 1 -- 172.21.15.97:0/59744868 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8017070 con 0x7f190806cb40 2024-09-10T21:36:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f18fea00640 1 -- 172.21.15.97:0/59744868 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18f8009d20 con 0x7f190806cb40 2024-09-10T21:36:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f18fea00640 1 -- 172.21.15.97:0/59744868 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f80106c0 con 0x7f190806cb40 2024-09-10T21:36:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1908150ba0 con 0x7f190806cb40 2024-09-10T21:36:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190814fde0 con 0x7f190806cb40 2024-09-10T21:36:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190815cb60 con 0x7f190806cb40 2024-09-10T21:36:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.257+0000 7f18fea00640 1 -- 172.21.15.97:0/59744868 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18f8010860 con 0x7f190806cb40 2024-09-10T21:36:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.261+0000 7f18fea00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18e80725e0 0x7f18e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.261+0000 7f18fea00640 1 -- 172.21.15.97:0/59744868 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f18f8093860 con 0x7f190806cb40 2024-09-10T21:36:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.261+0000 7f1906000640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18e80725e0 0x7f18e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.261+0000 7f1906000640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18e80725e0 0x7f18e8074aa0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f18f40af000 tx=0x7f18f40a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:03.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.265+0000 7f18fea00640 1 -- 172.21.15.97:0/59744868 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f19081fa070 con 0x7f190806cb40 2024-09-10T21:36:03.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f19080b22c0 con 0x7f190806cb40 2024-09-10T21:36:03.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f18fea00640 1 -- 172.21.15.97:0/59744868 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v350) ==== 53+0+73 (secure 0 0 0) 0x7f18f805e690 con 0x7f190806cb40 2024-09-10T21:36:03.464 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:36:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18e80725e0 msgr2=0x7f18e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18e80725e0 0x7f18e8074aa0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f18f40af000 tx=0x7f18f40a7040 comp rx=0 tx=0).stop 2024-09-10T21:36:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190806cb40 msgr2=0x7f19081f9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190806cb40 0x7f19081f9b30 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f18f800d950 tx=0x7f18f800de10 comp rx=0 tx=0).stop 2024-09-10T21:36:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 shutdown_connections 2024-09-10T21:36:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f18e80725e0 0x7f18e8074aa0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f190815f6d0 0x7f190815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1908071a40 0x7f19081655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 --2- 172.21.15.97:0/59744868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f190806cb40 0x7f19081f9b30 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 >> 172.21.15.97:0/59744868 conn(0x7f19080b2ee0 msgr2=0x7f19080cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 shutdown_connections 2024-09-10T21:36:03.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03.465+0000 7f1907e00640 1 -- 172.21.15.97:0/59744868 wait complete. 2024-09-10T21:36:03.485 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:36:03.485 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:36:03.485 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-10T21:36:03.485 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:36:03.486 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:36:03.486 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:36:03.486 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:36:03.486 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:36:03.486 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:36:03.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:03.486 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-10T21:36:03.495 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:36:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:36:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41730sec preferred_lft 41730sec 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:b3:65:e9:1f brd ff:ff:ff:ff:ff:ff 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:36:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T21:36:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:36:03.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:03.499 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:36:03.566 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-10T21:36:03.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:03.568 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-10T21:36:03.622 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T21:36:03.623 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:36:03.624 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:36:03.624 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-10T21:36:03.624 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:36:03.624 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:36:03.624 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:36:03.624 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-10T21:36:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:36:03.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:36:03.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:03.760 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:36:03.769 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:36:03.769 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:36:03.769 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:36:03.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:03.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-10T21:36:03.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:36:03.830 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:36:03.879 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:36:03.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:36:03.879 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:36:03.930 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T21:36:03.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:03.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:36:04.019 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:36:04.019 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:36:04.019 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:36:04.034 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:36:04.034 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:36:04.034 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:36:04.034 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:36:04.034 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:36:04.042 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T21:36:04.043 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T21:36:04.043 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-10T21:36:04.043 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.194:3300,172.21.15.194:3301" 2024-09-10T21:36:04.043 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T21:36:04.043 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790" 2024-09-10T21:36:04.043 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.194:6789,172.21.15.194:6790:/ 2024-09-10T21:36:04.043 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d1b4b680-9a7d-4173-b750-f3c8b85e2f7b" will pass to kernel 2024-09-10T21:36:04.046 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:36:04.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:04.047 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:36:04.066 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:36:04.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:36:04.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:36:04.169 INFO:teuthology.orchestra.run.smithi097.stdout:client9928 2024-09-10T21:36:04.169 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:36:04.169 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:04.178 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-10T21:36:04.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.265+0000 7f76a93a0080 1 Processor -- start 2024-09-10T21:36:04.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- start start 2024-09-10T21:36:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564885126cc0 0x5648850b8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885128a00 0x5648850b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56488512c1d0 0x5648850b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56488501af40 con 0x564885126cc0 2024-09-10T21:36:04.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564885027d00 con 0x564885128a00 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x564884fbca00 con 0x56488512c1d0 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564885126cc0 0x5648850b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564885126cc0 0x5648850b8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46598/0 (socket says 172.21.15.97:46598) 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 -- 172.21.15.97:0/3495975734 learned_addr learned my addr 172.21.15.97:0/3495975734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a5c00640 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885128a00 0x5648850b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a7000640 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56488512c1d0 0x5648850b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 -- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56488512c1d0 msgr2=0x5648850b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56488512c1d0 0x5648850b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 -- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885128a00 msgr2=0x5648850b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885128a00 0x5648850b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 -- 172.21.15.97:0/3495975734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648850b5a10 con 0x564885126cc0 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a5c00640 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885128a00 0x5648850b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:04.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a7000640 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56488512c1d0 0x5648850b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:04.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a6600640 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564885126cc0 0x5648850b8540 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f7698004ea0 tx=0x7f7698017fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471bd29f4ad38e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f769fe00640 1 -- 172.21.15.97:0/3495975734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980b0040 con 0x564885126cc0 2024-09-10T21:36:04.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f769fe00640 1 -- 172.21.15.97:0/3495975734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76980acc00 con 0x564885126cc0 2024-09-10T21:36:04.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f769fe00640 1 -- 172.21.15.97:0/3495975734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980b5410 con 0x564885126cc0 2024-09-10T21:36:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- 172.21.15.97:0/3495975734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564885126cc0 msgr2=0x5648850b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564885126cc0 0x5648850b8540 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f7698004ea0 tx=0x7f7698017fa0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- 172.21.15.97:0/3495975734 shutdown_connections 2024-09-10T21:36:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x56488512c1d0 0x5648850b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885128a00 0x5648850b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3495975734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564885126cc0 0x5648850b8540 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- 172.21.15.97:0/3495975734 >> 172.21.15.97:0/3495975734 conn(0x5648850b60d0 msgr2=0x564885126450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- 172.21.15.97:0/3495975734 shutdown_connections 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- 172.21.15.97:0/3495975734 wait complete. 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 Processor -- start 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.269+0000 7f76a93a0080 1 -- start start 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885126cc0 0x564885124f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564885128a00 0x564885070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56488512c1d0 0x56488509dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648851411e0 con 0x56488512c1d0 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564885140f40 con 0x564885126cc0 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5648850b5cb0 con 0x564885128a00 2024-09-10T21:36:04.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56488512c1d0 0x56488509dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56488512c1d0 0x56488509dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46604/0 (socket says 172.21.15.97:46604) 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 -- 172.21.15.97:0/3405588069 learned_addr learned my addr 172.21.15.97:0/3405588069 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a6600640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885126cc0 0x564885124f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a5c00640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564885128a00 0x564885070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 -- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564885128a00 msgr2=0x564885070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x564885128a00 0x564885070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 -- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885126cc0 msgr2=0x564885124f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564885126cc0 0x564885124f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 -- 172.21.15.97:0/3405588069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56488506e510 con 0x56488512c1d0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a7000640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56488512c1d0 0x56488509dec0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009230 tx=0x7f76a00096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f769ea00640 1 -- 172.21.15.97:0/3405588069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0014020 con 0x56488512c1d0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f769ea00640 1 -- 172.21.15.97:0/3405588069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76a0010910 con 0x56488512c1d0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f769ea00640 1 -- 172.21.15.97:0/3405588069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0010d60 con 0x56488512c1d0 2024-09-10T21:36:04.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56488506e7b0 con 0x56488512c1d0 2024-09-10T21:36:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.273+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488506ecf0 con 0x56488512c1d0 2024-09-10T21:36:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f769ea00640 1 -- 172.21.15.97:0/3405588069 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76a00183b0 con 0x56488512c1d0 2024-09-10T21:36:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f769ea00640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7688070fc0 0x7f7688073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f769ea00640 1 -- 172.21.15.97:0/3405588069 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f76a00923e0 con 0x56488512c1d0 2024-09-10T21:36:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x5648850712f0 0x56488506f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e149) -- 0x5648851254b0 con 0x5648850712f0 2024-09-10T21:36:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a6600640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7688070fc0 0x7f7688073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a5c00640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x5648850712f0 0x56488506f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a6600640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7688070fc0 0x7f7688073480 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7698004f80 tx=0x7f7698004890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a5c00640 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x5648850712f0 0x56488506f940 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a5c00640 1 -- 172.21.15.97:0/3405588069 <== osd.4 v2:172.21.15.194:6800/4073109273 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) ==== 164+0+91 (crc 0 0 0) 0x7f769400d980 con 0x5648850712f0 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x5648850712f0 msgr2=0x56488506f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x5648850712f0 0x56488506f940 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7688070fc0 msgr2=0x7f7688073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7688070fc0 0x7f7688073480 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7698004f80 tx=0x7f7698004890 comp rx=0 tx=0).stop 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56488512c1d0 msgr2=0x56488509dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 --2- 172.21.15.97:0/3405588069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56488512c1d0 0x56488509dec0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009230 tx=0x7f76a00096f0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a7000640 1 -- 172.21.15.97:0/3405588069 reap_dead start 2024-09-10T21:36:04.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 shutdown_connections 2024-09-10T21:36:04.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 >> 172.21.15.97:0/3405588069 conn(0x5648850b60d0 msgr2=0x564885144f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:04.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 shutdown_connections 2024-09-10T21:36:04.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.277+0000 7f76a93a0080 1 -- 172.21.15.97:0/3405588069 wait complete. 2024-09-10T21:36:04.283 INFO:teuthology.orchestra.run.smithi097.stdout:Ufile1dir1 2024-09-10T21:36:04.283 DEBUG:teuthology.orchestra.run.smithi097:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511627777, 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: "ancestors": [ 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: "dirino": 1099511627776, 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: "dname": "file1", 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 8 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:36:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: "dirino": 1, 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: "dname": "dir1", 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 29 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: "pool": 16, 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: "old_pools": [] 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:04.399 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent rhubarb 2024-09-10T21:36:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 Processor -- start 2024-09-10T21:36:04.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 -- start start 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60ad67540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60add7a00 0x55c60ad63ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60addb1d0 0x55c60ad64410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c60acc9f40 con 0x55c60addb1d0 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c60acd6d00 con 0x55c60add7a00 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa6951e8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c60ac6ba00 con 0x55c60add5cc0 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60addb1d0 0x55c60ad64410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60addb1d0 0x55c60ad64410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46618/0 (socket says 172.21.15.97:46618) 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 -- 172.21.15.97:0/1037913095 learned_addr learned my addr 172.21.15.97:0/1037913095 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:04.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692400640 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60ad67540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa691a00640 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60add7a00 0x55c60ad63ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 -- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 msgr2=0x55c60ad67540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60ad67540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 -- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60add7a00 msgr2=0x55c60ad63ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60add7a00 0x55c60ad63ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 -- 172.21.15.97:0/1037913095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60ad64a10 con 0x55c60addb1d0 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa691a00640 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60add7a00 0x55c60ad63ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692400640 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60ad67540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa692e00640 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60addb1d0 0x55c60ad64410 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0026c0 tx=0x7fa68c002b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a279c08349495 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa68be00640 1 -- 172.21.15.97:0/1037913095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c016070 con 0x55c60addb1d0 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa68be00640 1 -- 172.21.15.97:0/1037913095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa68c00f820 con 0x55c60addb1d0 2024-09-10T21:36:04.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.489+0000 7fa68be00640 1 -- 172.21.15.97:0/1037913095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00fc70 con 0x55c60addb1d0 2024-09-10T21:36:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- 172.21.15.97:0/1037913095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60addb1d0 msgr2=0x55c60ad64410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60addb1d0 0x55c60ad64410 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0026c0 tx=0x7fa68c002b80 comp rx=0 tx=0).stop 2024-09-10T21:36:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- 172.21.15.97:0/1037913095 shutdown_connections 2024-09-10T21:36:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60addb1d0 0x55c60ad64410 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60add7a00 0x55c60ad63ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 --2- 172.21.15.97:0/1037913095 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60ad67540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- 172.21.15.97:0/1037913095 >> 172.21.15.97:0/1037913095 conn(0x55c60ad650d0 msgr2=0x55c60add5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- 172.21.15.97:0/1037913095 shutdown_connections 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- 172.21.15.97:0/1037913095 wait complete. 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 Processor -- start 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- start start 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60add3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60add7a00 0x55c60adf3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60addb1d0 0x55c60ad4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c60adf01e0 con 0x55c60add7a00 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c60adeff40 con 0x55c60addb1d0 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c60ad64cb0 con 0x55c60add5cc0 2024-09-10T21:36:04.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60add7a00 0x55c60adf3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60add7a00 0x55c60adf3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46620/0 (socket says 172.21.15.97:46620) 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 -- 172.21.15.97:0/3774140744 learned_addr learned my addr 172.21.15.97:0/3774140744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa692400640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60add3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa692e00640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60addb1d0 0x55c60ad4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 -- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 msgr2=0x55c60add3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c60add5cc0 0x55c60add3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 -- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60addb1d0 msgr2=0x55c60ad4cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c60addb1d0 0x55c60ad4cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 -- 172.21.15.97:0/3774140744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60adf2e10 con 0x55c60add7a00 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa691a00640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60add7a00 0x55c60adf3d30 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b4e0 tx=0x7fa68000b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa68aa00640 1 -- 172.21.15.97:0/3774140744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800032e0 con 0x55c60add7a00 2024-09-10T21:36:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa68aa00640 1 -- 172.21.15.97:0/3774140744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa680016070 con 0x55c60add7a00 2024-09-10T21:36:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa68aa00640 1 -- 172.21.15.97:0/3774140744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800016c0 con 0x55c60add7a00 2024-09-10T21:36:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c60adf30b0 con 0x55c60add7a00 2024-09-10T21:36:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.493+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c60adf35f0 con 0x55c60add7a00 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa68aa00640 1 -- 172.21.15.97:0/3774140744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa680001880 con 0x55c60add7a00 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa68aa00640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674070fc0 0x7fa674073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa68aa00640 1 -- 172.21.15.97:0/3774140744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fa680092280 con 0x55c60add7a00 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa692400640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674070fc0 0x7fa674073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa6951e8080 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x55c60adf2900 0x55c60adf6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [setxattr parent (7) in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149) -- 0x55c60adf50e0 con 0x55c60adf2900 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa692e00640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x55c60adf2900 0x55c60adf6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa692400640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674070fc0 0x7fa674073480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa6840ab000 tx=0x7fa684019a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.497+0000 7fa692e00640 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x55c60adf2900 0x55c60adf6b20 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa692e00640 1 -- 172.21.15.97:0/3774140744 <== osd.4 v2:172.21.15.194:6800/4073109273 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (7)] v149'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa68c019040 con 0x55c60adf2900 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x55c60adf2900 msgr2=0x55c60adf6b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x55c60adf2900 0x55c60adf6b20 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674070fc0 msgr2=0x7fa674073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa674070fc0 0x7fa674073480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa6840ab000 tx=0x7fa684019a40 comp rx=0 tx=0).stop 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60add7a00 msgr2=0x55c60adf3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 --2- 172.21.15.97:0/3774140744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c60add7a00 0x55c60adf3d30 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b4e0 tx=0x7fa68000b9a0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa692e00640 1 -- 172.21.15.97:0/3774140744 reap_dead start 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 shutdown_connections 2024-09-10T21:36:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 >> 172.21.15.97:0/3774140744 conn(0x55c60ad650d0 msgr2=0x55c60ad4de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 shutdown_connections 2024-09-10T21:36:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.501+0000 7fa6951e8080 1 -- 172.21.15.97:0/3774140744 wait complete. 2024-09-10T21:36:04.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:04.506 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink1) 2024-09-10T21:36:04.517 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:36:04.517+0000 7fe000400640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: Decoder at 'void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=114 minimal_decoder=104: Malformed input [buffer:3] 2024-09-10T21:36:04.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:36:04.518 INFO:teuthology.orchestra.run.smithi097.stderr:stat: cannot statx 'testdir/hardlink1': Input/output error 2024-09-10T21:36:04.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- 172.21.15.97:0/1983251128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640cec10 msgr2=0x7faf640cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 --2- 172.21.15.97:0/1983251128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640cec10 0x7faf640cff10 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009840 tx=0x7faf4c01f010 comp rx=0 tx=0).stop 2024-09-10T21:36:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- 172.21.15.97:0/1983251128 shutdown_connections 2024-09-10T21:36:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 --2- 172.21.15.97:0/1983251128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640cec10 0x7faf640cff10 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- 172.21.15.97:0/1983251128 >> 172.21.15.97:0/1983251128 conn(0x7faf640b37c0 msgr2=0x7faf640b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- 172.21.15.97:0/1983251128 shutdown_connections 2024-09-10T21:36:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- 172.21.15.97:0/1983251128 wait complete. 2024-09-10T21:36:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 Processor -- start 2024-09-10T21:36:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- start start 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640b1560 0x7faf6415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf640ce830 0x7faf640b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640cec10 0x7faf64071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf640ce1a0 con 0x7faf640b1560 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf640cde60 con 0x7faf640ce830 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf6a600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7faf640ce320 con 0x7faf640cec10 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640b1560 0x7faf6415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640b1560 0x7faf6415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46640/0 (socket says 172.21.15.97:46640) 2024-09-10T21:36:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 -- 172.21.15.97:0/2265364278 learned_addr learned my addr 172.21.15.97:0/2265364278 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf5aa00640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf640ce830 0x7faf640b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63e00640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640cec10 0x7faf64071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 -- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640cec10 msgr2=0x7faf64071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640cec10 0x7faf64071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 -- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf640ce830 msgr2=0x7faf640b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf640ce830 0x7faf640b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.701+0000 7faf63400640 1 -- 172.21.15.97:0/2265364278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf640b5b40 con 0x7faf640b1560 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf63400640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640b1560 0x7faf6415ef40 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c480 tx=0x7faf5400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf5be00640 1 -- 172.21.15.97:0/2265364278 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54016020 con 0x7faf640b1560 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf5be00640 1 -- 172.21.15.97:0/2265364278 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf54012940 con 0x7faf640b1560 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf640b66e0 con 0x7faf640b1560 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf5be00640 1 -- 172.21.15.97:0/2265364278 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54012d90 con 0x7faf640b1560 2024-09-10T21:36:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf640b6ca0 con 0x7faf640b1560 2024-09-10T21:36:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf640cf800 con 0x7faf640b1560 2024-09-10T21:36:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf5be00640 1 -- 172.21.15.97:0/2265364278 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faf5401a3b0 con 0x7faf640b1560 2024-09-10T21:36:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf5be00640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf400725e0 0x7faf40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.705+0000 7faf5be00640 1 -- 172.21.15.97:0/2265364278 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7faf54093460 con 0x7faf640b1560 2024-09-10T21:36:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.709+0000 7faf5aa00640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf400725e0 0x7faf40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.709+0000 7faf5aa00640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf400725e0 0x7faf40074aa0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0b2000 tx=0x7faf4c01d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.709+0000 7faf5be00640 1 -- 172.21.15.97:0/2265364278 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faf54020030 con 0x7faf640b1560 2024-09-10T21:36:04.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.909+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf6415f710 con 0x7faf640b1560 2024-09-10T21:36:04.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.909+0000 7faf5be00640 1 -- 172.21.15.97:0/2265364278 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7faf5405e980 con 0x7faf640b1560 2024-09-10T21:36:04.910 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:04.910 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:04.911 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:04.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf400725e0 msgr2=0x7faf40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf400725e0 0x7faf40074aa0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0b2000 tx=0x7faf4c01d850 comp rx=0 tx=0).stop 2024-09-10T21:36:04.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640b1560 msgr2=0x7faf6415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640b1560 0x7faf6415ef40 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c480 tx=0x7faf5400c940 comp rx=0 tx=0).stop 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 shutdown_connections 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faf400725e0 0x7faf40074aa0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faf640cec10 0x7faf64071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf640ce830 0x7faf640b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 --2- 172.21.15.97:0/2265364278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf640b1560 0x7faf6415ef40 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 >> 172.21.15.97:0/2265364278 conn(0x7faf640b37c0 msgr2=0x7faf640b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 shutdown_connections 2024-09-10T21:36:04.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:04.913+0000 7faf6a600640 1 -- 172.21.15.97:0/2265364278 wait complete. 2024-09-10T21:36:04.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T21:36:05.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.073+0000 7fd3bbe00640 1 -- 172.21.15.97:0/2497592835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c003c20 msgr2=0x7fd39c004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.073+0000 7fd3bbe00640 1 --2- 172.21.15.97:0/2497592835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39c003c20 0x7fd39c004020 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c290 tx=0x7fd3a801ced0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.073+0000 7fd3b9e00640 1 -- 172.21.15.97:0/2497592835 reap_dead start 2024-09-10T21:36:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.073+0000 7fd3bbe00640 1 -- 172.21.15.97:0/2497592835 shutdown_connections 2024-09-10T21:36:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.073+0000 7fd3bbe00640 1 -- 172.21.15.97:0/2497592835 >> 172.21.15.97:0/2497592835 conn(0x7fd3b40cfb10 msgr2=0x7fd3b40cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 -- 172.21.15.97:0/2497592835 shutdown_connections 2024-09-10T21:36:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 -- 172.21.15.97:0/2497592835 wait complete. 2024-09-10T21:36:05.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 Processor -- start 2024-09-10T21:36:05.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 -- start start 2024-09-10T21:36:05.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd39c003c20 0x7fd3b4134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3b4135310 0x7fd3b4154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3b406e3b0 0x7fd3b4140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41674f0 con 0x7fd3b406e3b0 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4167370 con 0x7fd3b4135310 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3bbe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b4167670 con 0x7fd39c003c20 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3b406e3b0 0x7fd3b4140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3b406e3b0 0x7fd3b4140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46658/0 (socket says 172.21.15.97:46658) 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 -- 172.21.15.97:0/3152724245 learned_addr learned my addr 172.21.15.97:0/3152724245 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9400640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd39c003c20 0x7fd3b4134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3b4135310 0x7fd3b4154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 -- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd39c003c20 msgr2=0x7fd3b4134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd39c003c20 0x7fd3b4134dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 -- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3b4135310 msgr2=0x7fd3b4154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3b4135310 0x7fd3b4154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 -- 172.21.15.97:0/3152724245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b406f1a0 con 0x7fd3b406e3b0 2024-09-10T21:36:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3b406e3b0 0x7fd3b4140040 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac010ab0 tx=0x7fd3ac010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.077+0000 7fd3b1600640 1 -- 172.21.15.97:0/3152724245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00eea0 con 0x7fd3b406e3b0 2024-09-10T21:36:05.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3b1600640 1 -- 172.21.15.97:0/3152724245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3ac015070 con 0x7fd3b406e3b0 2024-09-10T21:36:05.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b406e900 con 0x7fd3b406e3b0 2024-09-10T21:36:05.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3b1600640 1 -- 172.21.15.97:0/3152724245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac003d10 con 0x7fd3b406e3b0 2024-09-10T21:36:05.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b406eec0 con 0x7fd3b406e3b0 2024-09-10T21:36:05.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3b1600640 1 -- 172.21.15.97:0/3152724245 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd3ac004030 con 0x7fd3b406e3b0 2024-09-10T21:36:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3b1600640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd394072610 0x7fd394074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3bf18e000 1 Processor -- start 2024-09-10T21:36:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3b1600640 1 -- 172.21.15.97:0/3152724245 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fd3ac093c10 con 0x7fd3b406e3b0 2024-09-10T21:36:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3bf18e000 1 -- start start 2024-09-10T21:36:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.081+0000 7fd3b9400640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd394072610 0x7fd394074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17db60 0x55f4ec17df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec133d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4ec01bce0 con 0x55f4ec17e480 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f4ebf8fc70 con 0x55f4ec17db60 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f4ebf7f0a0 con 0x55f4ec1342c0 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b9400640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd394072610 0x7fd394074ad0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80ac000 tx=0x7fd3a8019d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec133d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46664/0 (socket says 172.21.15.97:46664) 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 -- 172.21.15.97:0/3189361280 learned_addr learned my addr 172.21.15.97:0/3189361280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b9400640 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17db60 0x55f4ec17df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 -- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 msgr2=0x55f4ec1346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 -- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17db60 msgr2=0x55f4ec17df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17db60 0x55f4ec17df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 -- 172.21.15.97:0/3189361280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ebf5a270 con 0x55f4ec17e480 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec133d80 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00064a0 tx=0x7fd3a00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f49ecdbbf952878 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3a6000640 1 -- 172.21.15.97:0/3189361280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0001760 con 0x55f4ec17e480 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3a6000640 1 -- 172.21.15.97:0/3189361280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3a0016030 con 0x55f4ec17e480 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3a6000640 1 -- 172.21.15.97:0/3189361280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0001bf0 con 0x55f4ec17e480 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- 172.21.15.97:0/3189361280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 msgr2=0x55f4ec133d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec133d80 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00064a0 tx=0x7fd3a00049e0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- 172.21.15.97:0/3189361280 shutdown_connections 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec133d80 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/3189361280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17db60 0x55f4ec17df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- 172.21.15.97:0/3189361280 >> 172.21.15.97:0/3189361280 conn(0x55f4ec130270 msgr2=0x55f4ec132660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- 172.21.15.97:0/3189361280 shutdown_connections 2024-09-10T21:36:05.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- 172.21.15.97:0/3189361280 wait complete. 2024-09-10T21:36:05.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 -- start start 2024-09-10T21:36:05.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.085+0000 7fd3bf18e000 1 Processor -- start 2024-09-10T21:36:05.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17db60 0x55f4ec168d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17e480 0x55f4ec169240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4ebf19920 con 0x55f4ec17db60 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f4ebf82eb0 con 0x55f4ec17e480 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f4ebc44820 con 0x55f4ec1342c0 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 10 client.0 _ll_register_callbacks cb 0x55f4ebf5d7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17db60 0x55f4ec168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17db60 0x55f4ec168d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46668/0 (socket says 172.21.15.97:46668) 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 -- 172.21.15.97:0/2275372624 learned_addr learned my addr 172.21.15.97:0/2275372624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b9400640 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17e480 0x55f4ec169240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 -- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 msgr2=0x55f4ec1683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 -- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17e480 msgr2=0x55f4ec169240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17e480 0x55f4ec169240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 -- 172.21.15.97:0/2275372624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ebf32530 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17db60 0x55f4ec168d00 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0017040 tx=0x7fd3a00015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336decd89df9f822 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 1 -- 172.21.15.97:0/2275372624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0001960 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4ebf173e0 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 1 -- 172.21.15.97:0/2275372624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fd3a0028d20 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 1 -- 172.21.15.97:0/2275372624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fd3a0001b00 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 10 client.10935 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x55f4ebf1d460 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 1 -- 172.21.15.97:0/2275372624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f4ebf173e0 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 1 -- 172.21.15.97:0/2275372624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fd3a002f050 con 0x55f4ec17db60 2024-09-10T21:36:05.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 10 client.10935 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 10 client.10935 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 20 client.10935 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec169f80 0x55f4ec18fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 4 client.10935 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4ebf91bc0 con 0x55f4ec169f80 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b9400640 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec169f80 0x55f4ec18fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3b9400640 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec169f80 0x55f4ec18fe20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fde120bb96f78f0 server_cookie=103d1743f490d7f7 in_seq=0 out_seq=0 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 10 client.10935 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 1 -- 172.21.15.97:0/2275372624 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4ebf91bc0 con 0x55f4ec169f80 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 10 client.10935 handle_command_reply: tid=0 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 client.10935 shutdown 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17db60 msgr2=0x55f4ec168d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17db60 0x55f4ec168d00 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0017040 tx=0x7fd3a00015e0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 shutdown_connections 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec169f80 0x55f4ec18fe20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec17e480 0x55f4ec169240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17db60 0x55f4ec168d00 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2275372624 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec1342c0 0x55f4ec1683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 >> 172.21.15.97:0/2275372624 conn(0x55f4ec130270 msgr2=0x55f4ec132e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.089+0000 7fd39aa00640 0 client.10935 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.093+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 shutdown_connections 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.093+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2275372624 wait complete. 2024-09-10T21:36:05.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.093+0000 7fd3bf18e000 20 client.10935 trim_cache size 0 max 16384 2024-09-10T21:36:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 Processor -- start 2024-09-10T21:36:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 -- start start 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec17d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec17d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec132660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4ebf8ea40 con 0x55f4ec17e480 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f4ec13d4f0 con 0x55f4ec1342c0 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f4ebf4b2e0 con 0x55f4ec17db60 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec132660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec132660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46672/0 (socket says 172.21.15.97:46672) 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 -- 172.21.15.97:0/2871611616 learned_addr learned my addr 172.21.15.97:0/2871611616 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9400640 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec17d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec17d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 -- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 msgr2=0x55f4ec17d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec17d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 -- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 msgr2=0x55f4ec17d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec17d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 -- 172.21.15.97:0/2871611616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ec17e2b0 con 0x55f4ec17e480 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9400640 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec17d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec17d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec132660 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac05bc20 tx=0x7fd3ac05e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7264cc36a5d770 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3a5600640 1 -- 172.21.15.97:0/2871611616 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac067e00 con 0x55f4ec17e480 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3a5600640 1 -- 172.21.15.97:0/2871611616 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3ac05ac20 con 0x55f4ec17e480 2024-09-10T21:36:05.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.105+0000 7fd3a5600640 1 -- 172.21.15.97:0/2871611616 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac067bd0 con 0x55f4ec17e480 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2871611616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 msgr2=0x55f4ec132660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec132660 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac05bc20 tx=0x7fd3ac05e5d0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2871611616 shutdown_connections 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec132660 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec17d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/2871611616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec17d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2871611616 >> 172.21.15.97:0/2871611616 conn(0x55f4ec130270 msgr2=0x55f4ec1398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2871611616 shutdown_connections 2024-09-10T21:36:05.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/2871611616 wait complete. 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- start start 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 Processor -- start 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec1f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f4ebf874e0 con 0x55f4ec17e480 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f4ebf2a2e0 con 0x55f4ec1342c0 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55f4ebf3c650 con 0x55f4ec17db60 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 10 client.0 _ll_register_callbacks cb 0x7fd3ac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec1f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec1f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46688/0 (socket says 172.21.15.97:46688) 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 -- 172.21.15.97:0/128557117 learned_addr learned my addr 172.21.15.97:0/128557117 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b8a00640 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9400640 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 -- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 msgr2=0x55f4ec1f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec1f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 -- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 msgr2=0x55f4ec1f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec1f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 -- 172.21.15.97:0/128557117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ec004cf0 con 0x55f4ec17e480 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3b9e00640 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec1f41e0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac091e20 tx=0x7fd3ac067c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3129b0dd8b8f2e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd39aa00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd39aa00640 1 -- 172.21.15.97:0/128557117 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac067e00 con 0x55f4ec17e480 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd39aa00640 1 -- 172.21.15.97:0/128557117 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fd3ac05a300 con 0x55f4ec17e480 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ac05ac20 con 0x55f4ec17e480 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd39aa00640 1 -- 172.21.15.97:0/128557117 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fd3ac05a500 con 0x55f4ec17e480 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 10 client.10941 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x55f4ebf35e30 con 0x55f4ec17e480 2024-09-10T21:36:05.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd39aa00640 1 -- 172.21.15.97:0/128557117 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3ac05ac20 con 0x55f4ec17e480 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd39aa00640 1 -- 172.21.15.97:0/128557117 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fd3ac05d680 con 0x55f4ec17e480 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 10 client.10941 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 10 client.10941 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 20 client.10941 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec191da0 0x55f4ec192180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 4 client.10941 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.109+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f4ebf8d670 con 0x55f4ec191da0 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3b9400640 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec191da0 0x55f4ec192180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd39aa00640 10 client.10941 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3b9400640 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec191da0 0x55f4ec192180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b929f9d601c58b server_cookie=ada75dc3575ee454 in_seq=0 out_seq=0 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd39aa00640 1 -- 172.21.15.97:0/128557117 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+145 (crc 0 0 0) 0x55f4ebf8d670 con 0x55f4ec191da0 2024-09-10T21:36:05.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd39aa00640 10 client.10941 handle_command_reply: tid=0 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 client.10941 shutdown 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 msgr2=0x55f4ec1f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec1f41e0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac091e20 tx=0x7fd3ac067c20 comp rx=0 tx=0).stop 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 shutdown_connections 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f4ec17e480 0x55f4ec1f41e0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55f4ec17db60 0x55f4ec1f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55f4ec191da0 0x55f4ec192180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 --2- 172.21.15.97:0/128557117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4ec1342c0 0x55f4ec1f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 >> 172.21.15.97:0/128557117 conn(0x55f4ec130270 msgr2=0x55f4ec17c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd39aa00640 0 client.10941 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 shutdown_connections 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 1 -- 172.21.15.97:0/128557117 wait complete. 2024-09-10T21:36:05.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bf18e000 20 client.10941 trim_cache size 0 max 16384 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "backtrace", 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 2585168771, 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511627777, 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/testdir/hardlink1" 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:36:05.118 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:36:05.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd394072610 msgr2=0x7fd394074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd394072610 0x7fd394074ad0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80ac000 tx=0x7fd3a8019d10 comp rx=0 tx=0).stop 2024-09-10T21:36:05.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3b406e3b0 msgr2=0x7fd3b4140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3b406e3b0 0x7fd3b4140040 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac010ab0 tx=0x7fd3ac010f70 comp rx=0 tx=0).stop 2024-09-10T21:36:05.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 shutdown_connections 2024-09-10T21:36:05.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd394072610 0x7fd394074ad0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3b406e3b0 0x7fd3b4140040 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3b4135310 0x7fd3b4154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 --2- 172.21.15.97:0/3152724245 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd39c003c20 0x7fd3b4134dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 >> 172.21.15.97:0/3152724245 conn(0x7fd3b40cfb10 msgr2=0x7fd3b41383b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 shutdown_connections 2024-09-10T21:36:05.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.113+0000 7fd3bbe00640 1 -- 172.21.15.97:0/3152724245 wait complete. 2024-09-10T21:36:05.132 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 -- 172.21.15.97:0/1649148363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc540ceee0 msgr2=0x7fbc540cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1649148363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc540ceee0 0x7fbc540cf390 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009bc0 tx=0x7fbc4c01f3b0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 -- 172.21.15.97:0/1649148363 shutdown_connections 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1649148363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc540ceee0 0x7fbc540cf390 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1649148363 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc541511f0 0x7fbc540b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 -- 172.21.15.97:0/1649148363 >> 172.21.15.97:0/1649148363 conn(0x7fbc540b1780 msgr2=0x7fbc540b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 -- 172.21.15.97:0/1649148363 shutdown_connections 2024-09-10T21:36:05.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.273+0000 7fbc5a800640 1 -- 172.21.15.97:0/1649148363 wait complete. 2024-09-10T21:36:05.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 Processor -- start 2024-09-10T21:36:05.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 -- start start 2024-09-10T21:36:05.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540b3cf0 0x7fbc54145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540ceee0 0x7fbc5406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc541511f0 0x7fbc5406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5406e740 con 0x7fbc540b3cf0 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5406e5c0 con 0x7fbc541511f0 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5406e8c0 con 0x7fbc540ceee0 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540b3cf0 0x7fbc54145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540b3cf0 0x7fbc54145450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46698/0 (socket says 172.21.15.97:46698) 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 -- 172.21.15.97:0/1799477010 learned_addr learned my addr 172.21.15.97:0/1799477010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53e00640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc541511f0 0x7fbc5406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc52a00640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540ceee0 0x7fbc5406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 -- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540ceee0 msgr2=0x7fbc5406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540ceee0 0x7fbc5406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 -- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc541511f0 msgr2=0x7fbc5406fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc541511f0 0x7fbc5406fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 -- 172.21.15.97:0/1799477010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc54145990 con 0x7fbc540b3cf0 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc52a00640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540ceee0 0x7fbc5406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc53400640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540b3cf0 0x7fbc54145450 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400ea70 tx=0x7fbc4400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc4b400640 1 -- 172.21.15.97:0/1799477010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400cd40 con 0x7fbc540b3cf0 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc4b400640 1 -- 172.21.15.97:0/1799477010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc44004020 con 0x7fbc540b3cf0 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc4b400640 1 -- 172.21.15.97:0/1799477010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc440043f0 con 0x7fbc540b3cf0 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc541f49e0 con 0x7fbc540b3cf0 2024-09-10T21:36:05.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc541f4fa0 con 0x7fbc540b3cf0 2024-09-10T21:36:05.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.277+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc540cf390 con 0x7fbc540b3cf0 2024-09-10T21:36:05.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.281+0000 7fbc4b400640 1 -- 172.21.15.97:0/1799477010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbc4400b040 con 0x7fbc540b3cf0 2024-09-10T21:36:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.281+0000 7fbc4b400640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc340725e0 0x7fbc34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.281+0000 7fbc4b400640 1 -- 172.21.15.97:0/1799477010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fbc44093620 con 0x7fbc540b3cf0 2024-09-10T21:36:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.281+0000 7fbc52a00640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc340725e0 0x7fbc34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.281+0000 7fbc52a00640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc340725e0 0x7fbc34074aa0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbc541f2430 tx=0x7fbc40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.285+0000 7fbc4b400640 1 -- 172.21.15.97:0/1799477010 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbc4405eb40 con 0x7fbc540b3cf0 2024-09-10T21:36:05.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc540ce3e0 con 0x7fbc540b3cf0 2024-09-10T21:36:05.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc4b400640 1 -- 172.21.15.97:0/1799477010 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7fbc4405e440 con 0x7fbc540b3cf0 2024-09-10T21:36:05.484 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:05.484 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:05.484 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc340725e0 msgr2=0x7fbc34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc340725e0 0x7fbc34074aa0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbc541f2430 tx=0x7fbc40009040 comp rx=0 tx=0).stop 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540b3cf0 msgr2=0x7fbc54145450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540b3cf0 0x7fbc54145450 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400ea70 tx=0x7fbc4400ef30 comp rx=0 tx=0).stop 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 shutdown_connections 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbc340725e0 0x7fbc34074aa0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc541511f0 0x7fbc5406fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbc540ceee0 0x7fbc5406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 --2- 172.21.15.97:0/1799477010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc540b3cf0 0x7fbc54145450 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 >> 172.21.15.97:0/1799477010 conn(0x7fbc540b1780 msgr2=0x7fbc5415ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 shutdown_connections 2024-09-10T21:36:05.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.485+0000 7fbc5a800640 1 -- 172.21.15.97:0/1799477010 wait complete. 2024-09-10T21:36:05.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2585168771 2024-09-10T21:36:05.649 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:36:05.649+0000 7fe005400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:36:05.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.649+0000 7fd4f3400640 1 -- 172.21.15.97:0/602749351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec15fc10 msgr2=0x7fd4ec13f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.649+0000 7fd4f3400640 1 --2- 172.21.15.97:0/602749351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec15fc10 0x7fd4ec13f9b0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009bc0 tx=0x7fd4e401cba0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- 172.21.15.97:0/602749351 shutdown_connections 2024-09-10T21:36:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 --2- 172.21.15.97:0/602749351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec15fc10 0x7fd4ec13f9b0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 --2- 172.21.15.97:0/602749351 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4ec141d20 0x7fd4ec15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- 172.21.15.97:0/602749351 >> 172.21.15.97:0/602749351 conn(0x7fd4ec0b1780 msgr2=0x7fd4ec0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- 172.21.15.97:0/602749351 shutdown_connections 2024-09-10T21:36:05.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- 172.21.15.97:0/602749351 wait complete. 2024-09-10T21:36:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 Processor -- start 2024-09-10T21:36:05.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- start start 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0ce5f0 0x7fd4ec1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec141d20 0x7fd4ec15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4ec15fc10 0x7fd4ec06f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec168bf0 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec168a70 con 0x7fd4ec141d20 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec168d70 con 0x7fd4ec15fc10 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0ce5f0 0x7fd4ec1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0ce5f0 0x7fd4ec1f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46716/0 (socket says 172.21.15.97:46716) 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 -- 172.21.15.97:0/1642297306 learned_addr learned my addr 172.21.15.97:0/1642297306 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec141d20 0x7fd4ec15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f1400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4ec15fc10 0x7fd4ec06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 -- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4ec15fc10 msgr2=0x7fd4ec06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4ec15fc10 0x7fd4ec06f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 -- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec141d20 msgr2=0x7fd4ec15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec141d20 0x7fd4ec15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 -- 172.21.15.97:0/1642297306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec15f230 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f1400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4ec15fc10 0x7fd4ec06f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0ce5f0 0x7fd4ec1f7cb0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c490 tx=0x7fd4dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4e9600640 1 -- 172.21.15.97:0/1642297306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc016020 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4e9600640 1 -- 172.21.15.97:0/1642297306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4dc012940 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4e9600640 1 -- 172.21.15.97:0/1642297306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc012d90 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.653+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1f8320 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.657+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1f88e0 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.657+0000 7fd4e9600640 1 -- 172.21.15.97:0/1642297306 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4dc01b400 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.657+0000 7fd4e9600640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4c8072610 0x7fd4c8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.657+0000 7fd4e9600640 1 -- 172.21.15.97:0/1642297306 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fd4dc093340 con 0x7fd4ec0ce5f0 2024-09-10T21:36:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.657+0000 7fd4f67b3000 1 Processor -- start 2024-09-10T21:36:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.657+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4c8072610 0x7fd4c8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.657+0000 7fd4f67b3000 1 -- start start 2024-09-10T21:36:05.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4c8072610 0x7fd4c8074ad0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec1598f0 tx=0x7fd4d400b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c55f970 0x55c14c55fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c560290 0x55c14c512da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 0x55c14c5136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c14c3fca80 con 0x55c14c560290 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c14c371c90 con 0x55c14c5132e0 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c14c3610c0 con 0x55c14c55f970 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c560290 0x55c14c512da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c560290 0x55c14c512da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46726/0 (socket says 172.21.15.97:46726) 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 -- 172.21.15.97:0/763065721 learned_addr learned my addr 172.21.15.97:0/763065721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c55f970 0x55c14c55fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f1400640 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 0x55c14c5136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 -- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c55f970 msgr2=0x55c14c55fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c55f970 0x55c14c55fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 -- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 msgr2=0x55c14c5136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 0x55c14c5136c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 -- 172.21.15.97:0/763065721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14c305350 con 0x55c14c560290 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c560290 0x55c14c512da0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4011010 tx=0x7fd4d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c63753879be2761 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e1600640 1 -- 172.21.15.97:0/763065721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d401e620 con 0x55c14c560290 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e1600640 1 -- 172.21.15.97:0/763065721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4d401e7c0 con 0x55c14c560290 2024-09-10T21:36:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4e1600640 1 -- 172.21.15.97:0/763065721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d401ebd0 con 0x55c14c560290 2024-09-10T21:36:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- 172.21.15.97:0/763065721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c560290 msgr2=0x55c14c512da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c560290 0x55c14c512da0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4011010 tx=0x7fd4d4013000 comp rx=0 tx=0).stop 2024-09-10T21:36:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- 172.21.15.97:0/763065721 shutdown_connections 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 0x55c14c5136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c560290 0x55c14c512da0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/763065721 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c55f970 0x55c14c55fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- 172.21.15.97:0/763065721 >> 172.21.15.97:0/763065721 conn(0x55c14c540780 msgr2=0x55c14c542ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- 172.21.15.97:0/763065721 shutdown_connections 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- 172.21.15.97:0/763065721 wait complete. 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 -- start start 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.661+0000 7fd4f67b3000 1 Processor -- start 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c5132e0 0x55c14c566af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c567030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c560290 0x55c14c567570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c14c3e5600 con 0x55c14c55f970 2024-09-10T21:36:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c14c4201e0 con 0x55c14c560290 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c14c30ab10 con 0x55c14c5132e0 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 10 client.0 _ll_register_callbacks cb 0x55c14c500c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c567030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c567030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46734/0 (socket says 172.21.15.97:46734) 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 -- 172.21.15.97:0/368814071 learned_addr learned my addr 172.21.15.97:0/368814071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c5132e0 0x55c14c566af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f1400640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c560290 0x55c14c567570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 -- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c5132e0 msgr2=0x55c14c566af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c5132e0 0x55c14c566af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 -- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c560290 msgr2=0x55c14c567570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c560290 0x55c14c567570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 -- 172.21.15.97:0/368814071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14c2ff260 con 0x55c14c55f970 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f1400640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c560290 0x55c14c567570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c567030 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400ba00 tx=0x7fd4d401e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dadf520f8ba14d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 1 -- 172.21.15.97:0/368814071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d401ea50 con 0x55c14c55f970 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c14c2f91e0 con 0x55c14c55f970 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 1 -- 172.21.15.97:0/368814071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fd4d401ebf0 con 0x55c14c55f970 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 1 -- 172.21.15.97:0/368814071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fd4d4038050 con 0x55c14c55f970 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 10 client.10959 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x55c14c32aac0 con 0x55c14c55f970 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 1 -- 172.21.15.97:0/368814071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c14c2f91e0 con 0x55c14c55f970 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 1 -- 172.21.15.97:0/368814071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fd4d403b020 con 0x55c14c55f970 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 10 client.10959 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 10 client.10959 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 20 client.10959 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c56eec0 0x55c14c5712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 4 client.10959 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c14c373be0 con 0x55c14c56eec0 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c56eec0 0x55c14c5712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c56eec0 0x55c14c5712b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269c22a28edbd560 server_cookie=42b3be37424c6bea in_seq=0 out_seq=0 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 10 client.10959 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 1 -- 172.21.15.97:0/368814071 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c14c373be0 con 0x55c14c56eec0 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 10 client.10959 handle_command_reply: tid=0 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 client.10959 shutdown 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 msgr2=0x55c14c567030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c567030 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400ba00 tx=0x7fd4d401e650 comp rx=0 tx=0).stop 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 shutdown_connections 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c56eec0 0x55c14c5712b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c560290 0x55c14c567570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c567030 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/368814071 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c5132e0 0x55c14c566af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 >> 172.21.15.97:0/368814071 conn(0x55c14c540780 msgr2=0x55c14c542b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.665+0000 7fd4da000640 0 client.10959 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.669+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 shutdown_connections 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.669+0000 7fd4f67b3000 1 -- 172.21.15.97:0/368814071 wait complete. 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.669+0000 7fd4f67b3000 20 client.10959 trim_cache size 0 max 16384 2024-09-10T21:36:05.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 Processor -- start 2024-09-10T21:36:05.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 -- start start 2024-09-10T21:36:05.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 0x55c14c511570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c511ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 0x55c14c5712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec2036b0 con 0x55c14c55f970 2024-09-10T21:36:05.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1fb670 con 0x55c14c5132e0 2024-09-10T21:36:05.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec1fbcc0 con 0x55c14c560290 2024-09-10T21:36:05.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c511ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c511ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46738/0 (socket says 172.21.15.97:46738) 2024-09-10T21:36:05.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 -- 172.21.15.97:0/889884869 learned_addr learned my addr 172.21.15.97:0/889884869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 -- 172.21.15.97:0/889884869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 msgr2=0x55c14c5712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/889884869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 0x55c14c5712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 -- 172.21.15.97:0/889884869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 msgr2=0x55c14c511570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/889884869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 0x55c14c511570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 -- 172.21.15.97:0/889884869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14c5600c0 con 0x55c14c55f970 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.681+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/889884869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c511ab0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400b490 tx=0x7fd4d401ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67b87dd763f6d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4e0c00640 1 -- 172.21.15.97:0/889884869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4035070 con 0x55c14c55f970 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4e0c00640 1 -- 172.21.15.97:0/889884869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4d4030650 con 0x55c14c55f970 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4e0c00640 1 -- 172.21.15.97:0/889884869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4030aa0 con 0x55c14c55f970 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- 172.21.15.97:0/889884869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 msgr2=0x55c14c511ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/889884869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c511ab0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400b490 tx=0x7fd4d401ec80 comp rx=0 tx=0).stop 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- 172.21.15.97:0/889884869 shutdown_connections 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/889884869 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 0x55c14c5712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/889884869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c55f970 0x55c14c511ab0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/889884869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c5132e0 0x55c14c511570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- 172.21.15.97:0/889884869 >> 172.21.15.97:0/889884869 conn(0x55c14c540780 msgr2=0x55c14c51b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- 172.21.15.97:0/889884869 shutdown_connections 2024-09-10T21:36:05.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- 172.21.15.97:0/889884869 wait complete. 2024-09-10T21:36:05.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- start start 2024-09-10T21:36:05.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 Processor -- start 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c5132e0 0x55c14c573b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c55f970 0x55c14c574040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 0x55c14c574580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c14c370a60 con 0x55c14c5132e0 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c14c305b10 con 0x55c14c55f970 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55c14c3641e0 con 0x55c14c560290 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 10 client.0 _ll_register_callbacks cb 0x7fd4e40b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c5132e0 0x55c14c573b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c5132e0 0x55c14c573b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46742/0 (socket says 172.21.15.97:46742) 2024-09-10T21:36:05.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 -- 172.21.15.97:0/3390351465 learned_addr learned my addr 172.21.15.97:0/3390351465 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f1400640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 0x55c14c574580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c55f970 0x55c14c574040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 -- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 msgr2=0x55c14c574580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 0x55c14c574580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 -- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c55f970 msgr2=0x55c14c574040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c55f970 0x55c14c574040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 -- 172.21.15.97:0/3390351465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14c3e5a90 con 0x55c14c5132e0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c55f970 0x55c14c574040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f0a00640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c5132e0 0x55c14c573b00 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc05e1b0 tx=0x7fd4dc093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c3839afedec512 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4da000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4da000640 1 -- 172.21.15.97:0/3390351465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc08e070 con 0x55c14c5132e0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4da000640 1 -- 172.21.15.97:0/3390351465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fd4dc0930b0 con 0x55c14c5132e0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d4030650 con 0x55c14c5132e0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4da000640 1 -- 172.21.15.97:0/3390351465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fd4dc062d00 con 0x55c14c5132e0 2024-09-10T21:36:05.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 10 client.10965 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.685+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x55c14c306330 con 0x55c14c5132e0 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4da000640 1 -- 172.21.15.97:0/3390351465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4d4030650 con 0x55c14c5132e0 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4da000640 1 -- 172.21.15.97:0/3390351465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fd4dc05a490 con 0x55c14c5132e0 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 10 client.10965 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 10 client.10965 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 20 client.10965 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c574ac0 0x55c14c574ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 4 client.10965 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2585168771}] 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2585168771}) -- 0x55c14c36f690 con 0x55c14c574ac0 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c574ac0 0x55c14c574ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4da000640 10 client.10965 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4e3e00640 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c574ac0 0x55c14c574ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb860ac3ac44eabc server_cookie=9b501003dd591803 in_seq=0 out_seq=0 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4da000640 1 -- 172.21.15.97:0/3390351465 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55c14c36f690 con 0x55c14c574ac0 2024-09-10T21:36:05.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4da000640 10 client.10965 handle_command_reply: tid=0 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 client.10965 shutdown 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c5132e0 msgr2=0x55c14c573b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c5132e0 0x55c14c573b00 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc05e1b0 tx=0x7fd4dc093040 comp rx=0 tx=0).stop 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 shutdown_connections 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55c14c560290 0x55c14c574580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55c14c574ac0 0x55c14c574ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c14c55f970 0x55c14c574040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 --2- 172.21.15.97:0/3390351465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c14c5132e0 0x55c14c573b00 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4da000640 0 client.10965 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 >> 172.21.15.97:0/3390351465 conn(0x55c14c540780 msgr2=0x55c14c56c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 shutdown_connections 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 1 -- 172.21.15.97:0/3390351465 wait complete. 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f67b3000 20 client.10965 trim_cache size 0 max 16384 2024-09-10T21:36:05.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4c8072610 msgr2=0x7fd4c8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4c8072610 0x7fd4c8074ad0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec1598f0 tx=0x7fd4d400b390 comp rx=0 tx=0).stop 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0ce5f0 msgr2=0x7fd4ec1f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0ce5f0 0x7fd4ec1f7cb0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00c490 tx=0x7fd4dc00c950 comp rx=0 tx=0).stop 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 shutdown_connections 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd4c8072610 0x7fd4c8074ad0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd4ec15fc10 0x7fd4ec06f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4ec141d20 0x7fd4ec15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 --2- 172.21.15.97:0/1642297306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4ec0ce5f0 0x7fd4ec1f7cb0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 >> 172.21.15.97:0/1642297306 conn(0x7fd4ec0b1780 msgr2=0x7fd4ec15a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 shutdown_connections 2024-09-10T21:36:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.689+0000 7fd4f3400640 1 -- 172.21.15.97:0/1642297306 wait complete. 2024-09-10T21:36:05.709 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100000001f6.00000000 2024-09-10T21:36:05.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 Processor -- start 2024-09-10T21:36:05.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 -- start start 2024-09-10T21:36:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc7767800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc776b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc77f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555bc77dc7b0 con 0x555bc77df1d0 2024-09-10T21:36:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555bc766fa00 con 0x555bc77db830 2024-09-10T21:36:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.753+0000 7fdcbff9c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555bc77dc930 con 0x555bc77d9cc0 2024-09-10T21:36:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc77f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc77f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46768/0 (socket says 172.21.15.97:46768) 2024-09-10T21:36:05.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 -- 172.21.15.97:0/1185970268 learned_addr learned my addr 172.21.15.97:0/1185970268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbd200640 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc7767800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcb7e00640 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc776b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 -- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 msgr2=0x555bc7767800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc7767800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 -- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 msgr2=0x555bc776b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc776b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 -- 172.21.15.97:0/1185970268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bc766ea20 con 0x555bc77df1d0 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc77f4af0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800b8f0 tx=0x7fdcb800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d03bd814d0e6b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcb6a00640 1 -- 172.21.15.97:0/1185970268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800db30 con 0x555bc77df1d0 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcb6a00640 1 -- 172.21.15.97:0/1185970268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcb800ddc0 con 0x555bc77df1d0 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcb6a00640 1 -- 172.21.15.97:0/1185970268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800f690 con 0x555bc77df1d0 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- 172.21.15.97:0/1185970268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 msgr2=0x555bc77f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc77f4af0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800b8f0 tx=0x7fdcb800bdb0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- 172.21.15.97:0/1185970268 shutdown_connections 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc77f4af0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc776b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/1185970268 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc7767800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- 172.21.15.97:0/1185970268 >> 172.21.15.97:0/1185970268 conn(0x555bc77690d0 msgr2=0x555bc77d9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- 172.21.15.97:0/1185970268 shutdown_connections 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- 172.21.15.97:0/1185970268 wait complete. 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 Processor -- start 2024-09-10T21:36:05.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- start start 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc77f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc77d8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc78917f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555bc7768300 con 0x555bc77df1d0 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555bc7768060 con 0x555bc77db830 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbff9c080 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x555bc77685a0 con 0x555bc77d9cc0 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc78917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc78917f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46776/0 (socket says 172.21.15.97:46776) 2024-09-10T21:36:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 -- 172.21.15.97:0/2001965809 learned_addr learned my addr 172.21.15.97:0/2001965809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcb7e00640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc77d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbd200640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc77f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.757+0000 7fdcbdc00640 1 -- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 msgr2=0x555bc77f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbdc00640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc77f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbdc00640 1 -- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 msgr2=0x555bc77d8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbdc00640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc77d8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbdc00640 1 -- 172.21.15.97:0/2001965809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bc7648c10 con 0x555bc77df1d0 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbd200640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x555bc77d9cc0 0x555bc77f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcb7e00640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555bc77db830 0x555bc77d8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbdc00640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc78917f0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d720 tx=0x7fdcb800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcb5600640 1 -- 172.21.15.97:0/2001965809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8010790 con 0x555bc77df1d0 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcb5600640 1 -- 172.21.15.97:0/2001965809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcb8010950 con 0x555bc77df1d0 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555bc77f72f0 con 0x555bc77df1d0 2024-09-10T21:36:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcb5600640 1 -- 172.21.15.97:0/2001965809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8010da0 con 0x555bc77df1d0 2024-09-10T21:36:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bc77f7830 con 0x555bc77df1d0 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcb5600640 1 -- 172.21.15.97:0/2001965809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdcb802b430 con 0x555bc77df1d0 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcb5600640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c070ff0 0x7fdc9c0734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcb5600640 1 -- 172.21.15.97:0/2001965809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fdcb80941b0 con 0x555bc77df1d0 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbd200640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c070ff0 0x7fdc9c0734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x555bc77f6e40 0x555bc77f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.761+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- osd_op(unknown.0.0:1 15.a 15:57fe7535:::100000001f6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e149) -- 0x555bc78926f0 con 0x555bc77f6e40 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcb7e00640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x555bc77f6e40 0x555bc77f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbd200640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c070ff0 0x7fdc9c0734b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00ab000 tx=0x7fdcb0019820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcb7e00640 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x555bc77f6e40 0x555bc77f2310 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcb7e00640 1 -- 172.21.15.97:0/2001965809 <== osd.5 v2:172.21.15.194:6818/3659294892 1 ==== osd_op_reply(1 100000001f6.00000000 [delete] v149'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdca800c9b0 con 0x555bc77f6e40 2024-09-10T21:36:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x555bc77f6e40 msgr2=0x555bc77f2310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x555bc77f6e40 0x555bc77f2310 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c070ff0 msgr2=0x7fdc9c0734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdc9c070ff0 0x7fdc9c0734b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00ab000 tx=0x7fdcb0019820 comp rx=0 tx=0).stop 2024-09-10T21:36:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 msgr2=0x555bc78917f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 --2- 172.21.15.97:0/2001965809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bc77df1d0 0x555bc78917f0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d720 tx=0x7fdcb800ded0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbdc00640 1 -- 172.21.15.97:0/2001965809 reap_dead start 2024-09-10T21:36:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 shutdown_connections 2024-09-10T21:36:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 >> 172.21.15.97:0/2001965809 conn(0x555bc77690d0 msgr2=0x555bc77f5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 shutdown_connections 2024-09-10T21:36:05.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:05.765+0000 7fdcbff9c080 1 -- 172.21.15.97:0/2001965809 wait complete. 2024-09-10T21:36:05.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:05.770 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink2) 2024-09-10T21:36:05.825 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-09-10T21:36:05.825+0000 7fe000400640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-09-10T21:36:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:stat: cannot statx 'testdir/hardlink2': Input/output error 2024-09-10T21:36:05.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:36:05.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.009+0000 7f44bfc00640 1 -- 172.21.15.97:0/3780190582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b80af520 msgr2=0x7f44b8160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.009+0000 7f44bfc00640 1 --2- 172.21.15.97:0/3780190582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b80af520 0x7f44b8160220 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009840 tx=0x7f44a001c790 comp rx=0 tx=0).stop 2024-09-10T21:36:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- 172.21.15.97:0/3780190582 shutdown_connections 2024-09-10T21:36:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 --2- 172.21.15.97:0/3780190582 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b806cb40 0x7f44b80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 --2- 172.21.15.97:0/3780190582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b80af520 0x7f44b8160220 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- 172.21.15.97:0/3780190582 >> 172.21.15.97:0/3780190582 conn(0x7f44b80b3c70 msgr2=0x7f44b80b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- 172.21.15.97:0/3780190582 shutdown_connections 2024-09-10T21:36:06.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- 172.21.15.97:0/3780190582 wait complete. 2024-09-10T21:36:06.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 Processor -- start 2024-09-10T21:36:06.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- start start 2024-09-10T21:36:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b806cb40 0x7f44b8158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b80af520 0x7f44b814fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b815f120 0x7f44b814a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81694e0 con 0x7f44b815f120 2024-09-10T21:36:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8169360 con 0x7f44b806cb40 2024-09-10T21:36:06.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f44b8169660 con 0x7f44b80af520 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b815f120 0x7f44b814a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b815f120 0x7f44b814a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46790/0 (socket says 172.21.15.97:46790) 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 -- 172.21.15.97:0/819271285 learned_addr learned my addr 172.21.15.97:0/819271285 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bd200640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b806cb40 0x7f44b8158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44afe00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b80af520 0x7f44b814fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 -- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b80af520 msgr2=0x7f44b814fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b80af520 0x7f44b814fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 -- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b806cb40 msgr2=0x7f44b8158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b806cb40 0x7f44b8158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 -- 172.21.15.97:0/819271285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b80b1fd0 con 0x7f44b815f120 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bdc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b815f120 0x7f44b814a6c0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f44b000eb20 tx=0x7f44b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44b5600640 1 -- 172.21.15.97:0/819271285 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0004b30 con 0x7f44b815f120 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44b5600640 1 -- 172.21.15.97:0/819271285 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44b0004cf0 con 0x7f44b815f120 2024-09-10T21:36:06.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.013+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b81503b0 con 0x7f44b815f120 2024-09-10T21:36:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.017+0000 7f44b5600640 1 -- 172.21.15.97:0/819271285 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0005280 con 0x7f44b815f120 2024-09-10T21:36:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.017+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b80b1a10 con 0x7f44b815f120 2024-09-10T21:36:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.017+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b80d0170 con 0x7f44b815f120 2024-09-10T21:36:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.017+0000 7f44b5600640 1 -- 172.21.15.97:0/819271285 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f44b0005420 con 0x7f44b815f120 2024-09-10T21:36:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.017+0000 7f44b5600640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44940725e0 0x7f4494074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.017+0000 7f44b5600640 1 -- 172.21.15.97:0/819271285 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f44b00942c0 con 0x7f44b815f120 2024-09-10T21:36:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.017+0000 7f44bd200640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44940725e0 0x7f4494074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.021+0000 7f44bd200640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44940725e0 0x7f4494074aa0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f44a8001400 tx=0x7f44a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.021+0000 7f44b5600640 1 -- 172.21.15.97:0/819271285 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44b80b1fd0 con 0x7f44b815f120 2024-09-10T21:36:06.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.217+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44b80d1080 con 0x7f44b815f120 2024-09-10T21:36:06.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44b5600640 1 -- 172.21.15.97:0/819271285 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7f44b005f6b0 con 0x7f44b815f120 2024-09-10T21:36:06.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:06.220 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:06.220 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44940725e0 msgr2=0x7f4494074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44940725e0 0x7f4494074aa0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f44a8001400 tx=0x7f44a8009040 comp rx=0 tx=0).stop 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b815f120 msgr2=0x7f44b814a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b815f120 0x7f44b814a6c0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f44b000eb20 tx=0x7f44b000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 shutdown_connections 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f44940725e0 0x7f4494074aa0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44b815f120 0x7f44b814a6c0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f44b80af520 0x7f44b814fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 --2- 172.21.15.97:0/819271285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b806cb40 0x7f44b8158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 >> 172.21.15.97:0/819271285 conn(0x7f44b80b3c70 msgr2=0x7f44b8068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 shutdown_connections 2024-09-10T21:36:06.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.221+0000 7f44bfc00640 1 -- 172.21.15.97:0/819271285 wait complete. 2024-09-10T21:36:06.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T21:36:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 -- 172.21.15.97:0/3915206316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 msgr2=0x7fbcf40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 --2- 172.21.15.97:0/3915206316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf40cea20 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40c8c90 tx=0x7fbce401cb80 comp rx=0 tx=0).stop 2024-09-10T21:36:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 -- 172.21.15.97:0/3915206316 shutdown_connections 2024-09-10T21:36:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 --2- 172.21.15.97:0/3915206316 >> v2:172.21.15.194:3300/0 conn(0x7fbcf40cd6e0 0x7fbcf40af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 --2- 172.21.15.97:0/3915206316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf40cea20 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 -- 172.21.15.97:0/3915206316 >> 172.21.15.97:0/3915206316 conn(0x7fbcf40d1c30 msgr2=0x7fbcf40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 -- 172.21.15.97:0/3915206316 shutdown_connections 2024-09-10T21:36:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.385+0000 7fbcf9200640 1 -- 172.21.15.97:0/3915206316 wait complete. 2024-09-10T21:36:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 Processor -- start 2024-09-10T21:36:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 -- start start 2024-09-10T21:36:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcf40b0d50 0x7fbcf4156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcf40cd6e0 0x7fbcf41568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf41fb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4168d80 con 0x7fbcf40ce640 2024-09-10T21:36:06.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4168c00 con 0x7fbcf40cd6e0 2024-09-10T21:36:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf4168f00 con 0x7fbcf40b0d50 2024-09-10T21:36:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf41fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf41fb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46804/0 (socket says 172.21.15.97:46804) 2024-09-10T21:36:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 -- 172.21.15.97:0/2899099567 learned_addr learned my addr 172.21.15.97:0/2899099567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcf40b0d50 0x7fbcf4156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbce9400640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcf40cd6e0 0x7fbcf41568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 -- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcf40b0d50 msgr2=0x7fbcf4156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcf40b0d50 0x7fbcf4156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 -- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcf40cd6e0 msgr2=0x7fbcf41568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcf40cd6e0 0x7fbcf41568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 -- 172.21.15.97:0/2899099567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf41fbda0 con 0x7fbcf40ce640 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf2800640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf41fb130 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c970 tx=0x7fbcec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbceaa00640 1 -- 172.21.15.97:0/2899099567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec007bb0 con 0x7fbcf40ce640 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbceaa00640 1 -- 172.21.15.97:0/2899099567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcec00f450 con 0x7fbcf40ce640 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf4158960 con 0x7fbcf40ce640 2024-09-10T21:36:06.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbceaa00640 1 -- 172.21.15.97:0/2899099567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec007e60 con 0x7fbcf40ce640 2024-09-10T21:36:06.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.389+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf4158f20 con 0x7fbcf40ce640 2024-09-10T21:36:06.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.393+0000 7fbceaa00640 1 -- 172.21.15.97:0/2899099567 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbcec00f8a0 con 0x7fbcf40ce640 2024-09-10T21:36:06.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.393+0000 7fbceaa00640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbcd00725e0 0x7fbcd0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.393+0000 7fbcfc52f000 1 Processor -- start 2024-09-10T21:36:06.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.393+0000 7fbceaa00640 1 -- 172.21.15.97:0/2899099567 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fbcec093b00 con 0x7fbcf40ce640 2024-09-10T21:36:06.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.393+0000 7fbcfc52f000 1 -- start start 2024-09-10T21:36:06.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.393+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbcd00725e0 0x7fbcd0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbcd00725e0 0x7fbcd0074aa0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbce40af000 tx=0x7fbce4019920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688eeb60 0x5560688eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688ef480 0x5560688a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55606878cce0 con 0x5560688eeb60 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556068700c70 con 0x5560688a52c0 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5560686f00a0 con 0x5560688ef480 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688eeb60 0x5560688eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688eeb60 0x5560688eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46814/0 (socket says 172.21.15.97:46814) 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3746398157 learned_addr learned my addr 172.21.15.97:0/3746398157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf2800640 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbce9400640 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688ef480 0x5560688a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688ef480 msgr2=0x5560688a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688ef480 0x5560688a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 msgr2=0x5560688a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3746398157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560686cb270 con 0x5560688eeb60 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbce9400640 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688ef480 0x5560688a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688eeb60 0x5560688eef40 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fbce4098680 tx=0x7fbce40b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48676a47d948a008 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbce2a00640 1 -- 172.21.15.97:0/3746398157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce40bc070 con 0x5560688eeb60 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbce2a00640 1 -- 172.21.15.97:0/3746398157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbce40c8070 con 0x5560688eeb60 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbce2a00640 1 -- 172.21.15.97:0/3746398157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce40b2040 con 0x5560688eeb60 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3746398157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688eeb60 msgr2=0x5560688eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688eeb60 0x5560688eef40 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fbce4098680 tx=0x7fbce40b7000 comp rx=0 tx=0).stop 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3746398157 shutdown_connections 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688ef480 0x5560688a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3746398157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688eeb60 0x5560688eef40 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3746398157 >> 172.21.15.97:0/3746398157 conn(0x5560688a1270 msgr2=0x5560688a3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3746398157 shutdown_connections 2024-09-10T21:36:06.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3746398157 wait complete. 2024-09-10T21:36:06.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 -- start start 2024-09-10T21:36:06.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.397+0000 7fbcfc52f000 1 Processor -- start 2024-09-10T21:36:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x5560688f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688f6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x5560688f6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55606868a920 con 0x5560688a52c0 2024-09-10T21:36:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5560686f3eb0 con 0x5560688ef480 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5560683b5820 con 0x5560688eeb60 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 10 client.0 _ll_register_callbacks cb 0x5560686ce7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x5560688f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x5560688f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46828/0 (socket says 172.21.15.97:46828) 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3105306671 learned_addr learned my addr 172.21.15.97:0/3105306671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbce9400640 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf2800640 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x5560688f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 msgr2=0x5560688f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 msgr2=0x5560688f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x5560688f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3105306671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560686a3530 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x5560688f5ad0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fbce4098680 tx=0x7fbce4003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3e9480ae156267 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 1 -- 172.21.15.97:0/3105306671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce40bc070 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 1 -- 172.21.15.97:0/3105306671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fbce4004300 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560686883e0 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 1 -- 172.21.15.97:0/3105306671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fbce40db050 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 10 client.10998 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x55606868e460 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 1 -- 172.21.15.97:0/3105306671 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5560686883e0 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 1 -- 172.21.15.97:0/3105306671 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fbce40de020 con 0x5560688a52c0 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 10 client.10998 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 10 client.10998 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 20 client.10998 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560688fdea0 0x556068900290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 4 client.10998 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556068702bc0 con 0x5560688fdea0 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbce9400640 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560688fdea0 0x556068900290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbce9400640 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560688fdea0 0x556068900290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41048d361f94bce6 server_cookie=2c5a0a926570b6a4 in_seq=0 out_seq=0 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 10 client.10998 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 1 -- 172.21.15.97:0/3105306671 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556068702bc0 con 0x5560688fdea0 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 10 client.10998 handle_command_reply: tid=0 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 client.10998 shutdown 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 msgr2=0x5560688f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x5560688f5ad0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fbce4098680 tx=0x7fbce4003ec0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 shutdown_connections 2024-09-10T21:36:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560688fdea0 0x556068900290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x5560688f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3105306671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x5560688f5ad0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcd7400640 0 client.10998 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 >> 172.21.15.97:0/3105306671 conn(0x5560688a1270 msgr2=0x5560688a3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 shutdown_connections 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.401+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3105306671 wait complete. 2024-09-10T21:36:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.405+0000 7fbcfc52f000 20 client.10998 trim_cache size 0 max 16384 2024-09-10T21:36:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 Processor -- start 2024-09-10T21:36:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 -- start start 2024-09-10T21:36:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688ef480 0x556068900290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5560688ae4f0 con 0x5560688ef480 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5560686ad650 con 0x5560688a52c0 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5560687affa0 con 0x5560688eeb60 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688ef480 0x556068900290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688ef480 0x556068900290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46830/0 (socket says 172.21.15.97:46830) 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 -- 172.21.15.97:0/786201904 learned_addr learned my addr 172.21.15.97:0/786201904 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbce9400640 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 -- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 msgr2=0x5560688a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 -- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 msgr2=0x5560688a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 -- 172.21.15.97:0/786201904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556068775cf0 con 0x5560688ef480 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbce9400640 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.417+0000 7fbcf2800640 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688ef480 0x556068900290 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fbcec05bba0 tx=0x7fbcec05b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f57297aa96e9f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbce2000640 1 -- 172.21.15.97:0/786201904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec05e930 con 0x5560688ef480 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbce2000640 1 -- 172.21.15.97:0/786201904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcec063360 con 0x5560688ef480 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbce2000640 1 -- 172.21.15.97:0/786201904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec067b30 con 0x5560688ef480 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- 172.21.15.97:0/786201904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688ef480 msgr2=0x556068900290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688ef480 0x556068900290 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fbcec05bba0 tx=0x7fbcec05b0b0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- 172.21.15.97:0/786201904 shutdown_connections 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688ef480 0x556068900290 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x5560688a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/786201904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688a52c0 0x5560688a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- 172.21.15.97:0/786201904 >> 172.21.15.97:0/786201904 conn(0x5560688a1270 msgr2=0x5560688aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- 172.21.15.97:0/786201904 shutdown_connections 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- 172.21.15.97:0/786201904 wait complete. 2024-09-10T21:36:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- start start 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 Processor -- start 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x556068963ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x556068964530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x556068964a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5560686ffa40 con 0x5560688a52c0 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5560686f31c0 con 0x5560688ef480 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5560686bc2e0 con 0x5560688eeb60 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 10 client.0 _ll_register_callbacks cb 0x7fbcec05af80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x556068963ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x556068963ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46840/0 (socket says 172.21.15.97:46840) 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3110923519 learned_addr learned my addr 172.21.15.97:0/3110923519 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbce9400640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x556068964530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf2800640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x556068964a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 msgr2=0x556068964530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x556068964530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 msgr2=0x556068964a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x556068964a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 -- 172.21.15.97:0/3110923519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560686a6e30 con 0x5560688a52c0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf2800640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x556068964a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcf1e00640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x556068963ff0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fbce401d420 tx=0x7fbce4004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bb84527eed2b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcd7400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcd7400640 1 -- 172.21.15.97:0/3110923519 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce40bc040 con 0x5560688a52c0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcd7400640 1 -- 172.21.15.97:0/3110923519 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fbce40d0a20 con 0x5560688a52c0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcec063360 con 0x5560688a52c0 2024-09-10T21:36:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcd7400640 1 -- 172.21.15.97:0/3110923519 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fbce40b2040 con 0x5560688a52c0 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 10 client.11004 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.421+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x5560686952b0 con 0x5560688a52c0 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcd7400640 1 -- 172.21.15.97:0/3110923519 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcec063360 con 0x5560688a52c0 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcd7400640 1 -- 172.21.15.97:0/3110923519 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fbce40de070 con 0x5560688a52c0 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 10 client.11004 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 10 client.11004 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 20 client.11004 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560689033c0 0x5560689037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 4 client.11004 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5560686fe670 con 0x5560689033c0 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbce9400640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560689033c0 0x5560689037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcd7400640 10 client.11004 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbce9400640 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560689033c0 0x5560689037a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe60aadcff778da server_cookie=18a51b859ccfa005 in_seq=0 out_seq=0 2024-09-10T21:36:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcd7400640 1 -- 172.21.15.97:0/3110923519 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+294 (crc 0 0 0) 0x5560686fe670 con 0x5560689033c0 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcd7400640 10 client.11004 handle_command_reply: tid=0 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 client.11004 shutdown 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 msgr2=0x556068963ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x556068963ff0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fbce401d420 tx=0x7fbce4004950 comp rx=0 tx=0).stop 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 shutdown_connections 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5560689033c0 0x5560689037a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5560688ef480 0x556068964a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5560688eeb60 0x556068964530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 --2- 172.21.15.97:0/3110923519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560688a52c0 0x556068963ff0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcd7400640 0 client.11004 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 >> 172.21.15.97:0/3110923519 conn(0x5560688a1270 msgr2=0x5560688a3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 shutdown_connections 2024-09-10T21:36:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 1 -- 172.21.15.97:0/3110923519 wait complete. 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcfc52f000 20 client.11004 trim_cache size 0 max 16384 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "dir_frag", 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 2957520968, 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511628278, 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: "frag": "*", 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/dir2" 2024-09-10T21:36:06.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-10T21:36:06.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-10T21:36:06.437 INFO:teuthology.orchestra.run.smithi097.stdout: "damage_type": "backtrace", 2024-09-10T21:36:06.437 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 3824306573, 2024-09-10T21:36:06.437 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 1099511628279, 2024-09-10T21:36:06.437 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/testdir/hardlink2" 2024-09-10T21:36:06.437 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-10T21:36:06.437 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbcd00725e0 msgr2=0x7fbcd0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbcd00725e0 0x7fbcd0074aa0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbce40af000 tx=0x7fbce4019920 comp rx=0 tx=0).stop 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 msgr2=0x7fbcf41fb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf41fb130 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c970 tx=0x7fbcec00ce30 comp rx=0 tx=0).stop 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 shutdown_connections 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbcd00725e0 0x7fbcd0074aa0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcf40ce640 0x7fbcf41fb130 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcf40cd6e0 0x7fbcf41568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 --2- 172.21.15.97:0/2899099567 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbcf40b0d50 0x7fbcf4156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 >> 172.21.15.97:0/2899099567 conn(0x7fbcf40d1c30 msgr2=0x7fbcf40d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 shutdown_connections 2024-09-10T21:36:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.425+0000 7fbcf9200640 1 -- 172.21.15.97:0/2899099567 wait complete. 2024-09-10T21:36:06.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:06.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- 172.21.15.97:0/3474255591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec144690 msgr2=0x7faeec0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 --2- 172.21.15.97:0/3474255591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec144690 0x7faeec0b2a80 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7faed8009840 tx=0x7faed801c820 comp rx=0 tx=0).stop 2024-09-10T21:36:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- 172.21.15.97:0/3474255591 shutdown_connections 2024-09-10T21:36:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 --2- 172.21.15.97:0/3474255591 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec06fd50 0x7faeec0701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 --2- 172.21.15.97:0/3474255591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec144690 0x7faeec0b2a80 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- 172.21.15.97:0/3474255591 >> 172.21.15.97:0/3474255591 conn(0x7faeec0b1780 msgr2=0x7faeec0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- 172.21.15.97:0/3474255591 shutdown_connections 2024-09-10T21:36:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- 172.21.15.97:0/3474255591 wait complete. 2024-09-10T21:36:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 Processor -- start 2024-09-10T21:36:06.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- start start 2024-09-10T21:36:06.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec06fd50 0x7faeec1f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec143f90 0x7faeec15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec144690 0x7faeec1fb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faeec167c50 con 0x7faeec143f90 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faeec167ad0 con 0x7faeec144690 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faef1800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7faeec167dd0 con 0x7faeec06fd50 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.589+0000 7faeea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec143f90 0x7faeec15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec143f90 0x7faeec15a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46850/0 (socket says 172.21.15.97:46850) 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 -- 172.21.15.97:0/4157986459 learned_addr learned my addr 172.21.15.97:0/4157986459 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 -- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec06fd50 msgr2=0x7faeec1f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeeaa00640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec06fd50 0x7faeec1f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeeb400640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec144690 0x7faeec1fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec06fd50 0x7faeec1f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 -- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec144690 msgr2=0x7faeec1fb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec144690 0x7faeec1fb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 -- 172.21.15.97:0/4157986459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec1fbd50 con 0x7faeec143f90 2024-09-10T21:36:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeeaa00640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec06fd50 0x7faeec1f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeeb400640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec144690 0x7faeec1fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faeea000640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec143f90 0x7faeec15a1c0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7faed8009790 tx=0x7faed80a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faee2a00640 1 -- 172.21.15.97:0/4157986459 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80be070 con 0x7faeec143f90 2024-09-10T21:36:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faee2a00640 1 -- 172.21.15.97:0/4157986459 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faed8004470 con 0x7faeec143f90 2024-09-10T21:36:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faee2a00640 1 -- 172.21.15.97:0/4157986459 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80052d0 con 0x7faeec143f90 2024-09-10T21:36:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec1f7ea0 con 0x7faeec143f90 2024-09-10T21:36:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec1f8460 con 0x7faeec143f90 2024-09-10T21:36:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec0cade0 con 0x7faeec143f90 2024-09-10T21:36:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faee2a00640 1 -- 172.21.15.97:0/4157986459 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faed8005470 con 0x7faeec143f90 2024-09-10T21:36:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.593+0000 7faee2a00640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.597+0000 7faee2a00640 1 -- 172.21.15.97:0/4157986459 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7faed812d960 con 0x7faeec143f90 2024-09-10T21:36:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.597+0000 7faeeaa00640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.597+0000 7faeeaa00640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7faedc0021a0 tx=0x7faedc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.597+0000 7faee2a00640 1 -- 172.21.15.97:0/4157986459 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faeec1fbd50 con 0x7faeec143f90 2024-09-10T21:36:06.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.801+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeec0701e0 con 0x7faeec143f90 2024-09-10T21:36:06.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.801+0000 7faee2a00640 1 -- 172.21.15.97:0/4157986459 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7faed8016410 con 0x7faeec143f90 2024-09-10T21:36:06.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:06.802 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:06.802 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 msgr2=0x7faecc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7faedc0021a0 tx=0x7faedc009040 comp rx=0 tx=0).stop 2024-09-10T21:36:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec143f90 msgr2=0x7faeec15a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec143f90 0x7faeec15a1c0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7faed8009790 tx=0x7faed80a66f0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 shutdown_connections 2024-09-10T21:36:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7faecc0725e0 0x7faecc074aa0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeec144690 0x7faeec1fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeec143f90 0x7faeec15a1c0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 --2- 172.21.15.97:0/4157986459 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7faeec06fd50 0x7faeec1f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 >> 172.21.15.97:0/4157986459 conn(0x7faeec0b1780 msgr2=0x7faeec0b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 shutdown_connections 2024-09-10T21:36:06.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.805+0000 7faef1800640 1 -- 172.21.15.97:0/4157986459 wait complete. 2024-09-10T21:36:06.822 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2957520968 2024-09-10T21:36:06.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- 172.21.15.97:0/1211677664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8214004d50 msgr2=0x7f82140051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 --2- 172.21.15.97:0/1211677664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8214004d50 0x7f82140051b0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f8228001dc0 tx=0x7f822801cb30 comp rx=0 tx=0).stop 2024-09-10T21:36:06.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f822f200640 1 -- 172.21.15.97:0/1211677664 reap_dead start 2024-09-10T21:36:06.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- 172.21.15.97:0/1211677664 shutdown_connections 2024-09-10T21:36:06.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- 172.21.15.97:0/1211677664 >> 172.21.15.97:0/1211677664 conn(0x7f82300b1780 msgr2=0x7f82300b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- 172.21.15.97:0/1211677664 shutdown_connections 2024-09-10T21:36:06.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- 172.21.15.97:0/1211677664 wait complete. 2024-09-10T21:36:06.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 Processor -- start 2024-09-10T21:36:06.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- start start 2024-09-10T21:36:06.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8214004d50 0x7f8230155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f823013dc70 0x7f823015d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f823013e0f0 0x7f8230156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82301400c0 con 0x7f823013e0f0 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8230160080 con 0x7f823013dc70 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f8235400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8230140240 con 0x7f8214004d50 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f822f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f823013e0f0 0x7f8230156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f822f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f823013e0f0 0x7f8230156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46860/0 (socket says 172.21.15.97:46860) 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f822f200640 1 -- 172.21.15.97:0/2559874749 learned_addr learned my addr 172.21.15.97:0/2559874749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f822e800640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8214004d50 0x7f8230155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.969+0000 7f822de00640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f823013dc70 0x7f823015d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f822f200640 1 -- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8214004d50 msgr2=0x7f8230155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f822f200640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8214004d50 0x7f8230155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f822f200640 1 -- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f823013dc70 msgr2=0x7f823015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f822f200640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f823013dc70 0x7f823015d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f822f200640 1 -- 172.21.15.97:0/2559874749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82300b40a0 con 0x7f823013e0f0 2024-09-10T21:36:06.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f822f200640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f823013e0f0 0x7f8230156710 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f82280ab040 tx=0x7f8228006e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f8226a00640 1 -- 172.21.15.97:0/2559874749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228003d90 con 0x7f823013e0f0 2024-09-10T21:36:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f8226a00640 1 -- 172.21.15.97:0/2559874749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82280060d0 con 0x7f823013e0f0 2024-09-10T21:36:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f8226a00640 1 -- 172.21.15.97:0/2559874749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280064e0 con 0x7f823013e0f0 2024-09-10T21:36:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82300b4380 con 0x7f823013e0f0 2024-09-10T21:36:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82300b4940 con 0x7f823013e0f0 2024-09-10T21:36:06.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.973+0000 7f8226a00640 1 -- 172.21.15.97:0/2559874749 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f82280b6020 con 0x7f823013e0f0 2024-09-10T21:36:06.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f8226a00640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8210072610 0x7f8210074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 Processor -- start 2024-09-10T21:36:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f8226a00640 1 -- 172.21.15.97:0/2559874749 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f822812ca90 con 0x7f823013e0f0 2024-09-10T21:36:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- start start 2024-09-10T21:36:06.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8210072610 0x7f8210074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb180fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb1810290 0x55ffb17c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb17c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ffb16aca80 con 0x55ffb180f970 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ffb1621c90 con 0x55ffb17c32e0 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ffb16110c0 con 0x55ffb1810290 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8210072610 0x7f8210074ad0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8220001ec0 tx=0x7f8220006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb180fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb180fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46872/0 (socket says 172.21.15.97:46872) 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 -- 172.21.15.97:0/4038935173 learned_addr learned my addr 172.21.15.97:0/4038935173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822f200640 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb17c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822de00640 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb1810290 0x55ffb17c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 -- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb1810290 msgr2=0x55ffb17c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb1810290 0x55ffb17c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 -- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 msgr2=0x55ffb17c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb17c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 -- 172.21.15.97:0/4038935173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb15b5350 con 0x55ffb180f970 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f822e800640 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb180fd50 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f8220011d00 tx=0x7f8220014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec864d901886e060 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f821b400640 1 -- 172.21.15.97:0/4038935173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220011d30 con 0x55ffb180f970 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f821b400640 1 -- 172.21.15.97:0/4038935173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8220016070 con 0x55ffb180f970 2024-09-10T21:36:06.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f821b400640 1 -- 172.21.15.97:0/4038935173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822001f690 con 0x55ffb180f970 2024-09-10T21:36:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- 172.21.15.97:0/4038935173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 msgr2=0x55ffb180fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb180fd50 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f8220011d00 tx=0x7f8220014070 comp rx=0 tx=0).stop 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- 172.21.15.97:0/4038935173 shutdown_connections 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb17c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb1810290 0x55ffb17c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 --2- 172.21.15.97:0/4038935173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb180fd50 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- 172.21.15.97:0/4038935173 >> 172.21.15.97:0/4038935173 conn(0x55ffb17f0780 msgr2=0x55ffb17f2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- 172.21.15.97:0/4038935173 shutdown_connections 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.977+0000 7f82387e7000 1 -- 172.21.15.97:0/4038935173 wait complete. 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 -- start start 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 Processor -- start 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1816af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1817030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1817570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ffb1695600 con 0x55ffb1810290 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ffb16d01e0 con 0x55ffb17c32e0 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ffb15bab10 con 0x55ffb180f970 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 10 client.0 _ll_register_callbacks cb 0x55ffb17b0c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1817570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1817570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46874/0 (socket says 172.21.15.97:46874) 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 -- 172.21.15.97:0/3023353597 learned_addr learned my addr 172.21.15.97:0/3023353597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822de00640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1817030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822e800640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1816af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 -- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 msgr2=0x55ffb1817030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1817030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 -- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 msgr2=0x55ffb1816af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1816af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 -- 172.21.15.97:0/3023353597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb15af260 con 0x55ffb1810290 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822e800640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1816af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822f200640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1817570 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f8228003fd0 tx=0x7f82280f3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d8787d143cc0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f820fe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f820fe00640 1 -- 172.21.15.97:0/3023353597 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280f7030 con 0x55ffb1810290 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f820fe00640 1 -- 172.21.15.97:0/3023353597 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f82280f8b40 con 0x55ffb1810290 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffb15a91e0 con 0x55ffb1810290 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f820fe00640 1 -- 172.21.15.97:0/3023353597 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7f82280fd480 con 0x55ffb1810290 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 10 client.11025 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:06.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x55ffb15daac0 con 0x55ffb1810290 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f820fe00640 1 -- 172.21.15.97:0/3023353597 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ffb15a91e0 con 0x55ffb1810290 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f820fe00640 1 -- 172.21.15.97:0/3023353597 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7f822811cbd0 con 0x55ffb1810290 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 10 client.11025 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 10 client.11025 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 20 client.11025 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb181eec0 0x55ffb18212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 4 client.11025 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffb1623be0 con 0x55ffb181eec0 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822e800640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb181eec0 0x55ffb18212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f820fe00640 10 client.11025 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.981+0000 7f822e800640 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb181eec0 0x55ffb18212b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28ed74c3579eae8 server_cookie=11db8e8f036ab64e in_seq=0 out_seq=0 2024-09-10T21:36:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f820fe00640 1 -- 172.21.15.97:0/3023353597 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffb1623be0 con 0x55ffb181eec0 2024-09-10T21:36:06.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f820fe00640 10 client.11025 handle_command_reply: tid=0 2024-09-10T21:36:06.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 client.11025 shutdown 2024-09-10T21:36:06.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 msgr2=0x55ffb1817570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:06.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1817570 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f8228003fd0 tx=0x7f82280f3de0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 shutdown_connections 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1817570 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1817030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb181eec0 0x55ffb18212b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 --2- 172.21.15.97:0/3023353597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1816af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f820fe00640 0 client.11025 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 >> 172.21.15.97:0/3023353597 conn(0x55ffb17f0780 msgr2=0x55ffb17f2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 shutdown_connections 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 1 -- 172.21.15.97:0/3023353597 wait complete. 2024-09-10T21:36:06.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.985+0000 7f82387e7000 20 client.11025 trim_cache size 0 max 16384 2024-09-10T21:36:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 Processor -- start 2024-09-10T21:36:06.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 -- start start 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb17c32e0 0x55ffb181cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb181d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb1810290 0x55ffb18212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8230200ea0 con 0x55ffb180f970 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8230200200 con 0x55ffb1810290 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f8230200850 con 0x55ffb17c32e0 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f822de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb181d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f822de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb181d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46890/0 (socket says 172.21.15.97:46890) 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:06.997+0000 7f822de00640 1 -- 172.21.15.97:0/1510612534 learned_addr learned my addr 172.21.15.97:0/1510612534 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb1810290 0x55ffb18212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822e800640 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb17c32e0 0x55ffb181cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822de00640 1 -- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb17c32e0 msgr2=0x55ffb181cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822de00640 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb17c32e0 0x55ffb181cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822de00640 1 -- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb1810290 msgr2=0x55ffb18212b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822de00640 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb1810290 0x55ffb18212b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822de00640 1 -- 172.21.15.97:0/1510612534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb18100c0 con 0x55ffb180f970 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb1810290 0x55ffb18212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:07.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822de00640 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb181d530 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f821c009e60 tx=0x7f821c0091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86e3d94396dd6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f821aa00640 1 -- 172.21.15.97:0/1510612534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c004250 con 0x55ffb180f970 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f821aa00640 1 -- 172.21.15.97:0/1510612534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f821c0043f0 con 0x55ffb180f970 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f821aa00640 1 -- 172.21.15.97:0/1510612534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c003860 con 0x55ffb180f970 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- 172.21.15.97:0/1510612534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 msgr2=0x55ffb181d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb181d530 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f821c009e60 tx=0x7f821c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- 172.21.15.97:0/1510612534 shutdown_connections 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb1810290 0x55ffb18212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb180f970 0x55ffb181d530 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 --2- 172.21.15.97:0/1510612534 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb17c32e0 0x55ffb181cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- 172.21.15.97:0/1510612534 >> 172.21.15.97:0/1510612534 conn(0x55ffb17f0780 msgr2=0x55ffb181c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- 172.21.15.97:0/1510612534 shutdown_connections 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- 172.21.15.97:0/1510612534 wait complete. 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- start start 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 Processor -- start 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1886230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1886770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1886cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ffb1620a60 con 0x55ffb1810290 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ffb15b5b10 con 0x55ffb17c32e0 2024-09-10T21:36:07.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x55ffb16141e0 con 0x55ffb180f970 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f82387e7000 10 client.0 _ll_register_callbacks cb 0x7f82280f3f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1886cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1886cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46896/0 (socket says 172.21.15.97:46896) 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 -- 172.21.15.97:0/2048087250 learned_addr learned my addr 172.21.15.97:0/2048087250 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822de00640 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1886770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822e800640 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1886230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 -- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 msgr2=0x55ffb1886770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1886770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 -- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 msgr2=0x55ffb1886230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1886230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.001+0000 7f822f200640 1 -- 172.21.15.97:0/2048087250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb1695a90 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f822f200640 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1886cb0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f82280f4b30 tx=0x7f82280f7520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729b01b7f2b164fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 1 -- 172.21.15.97:0/2048087250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228120060 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 1 -- 172.21.15.97:0/2048087250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f82280f7d20 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821c0043f0 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 1 -- 172.21.15.97:0/2048087250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7f82280f7690 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 10 client.11031 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x55ffb15b6330 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 1 -- 172.21.15.97:0/2048087250 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f821c0043f0 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 1 -- 172.21.15.97:0/2048087250 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7f82280f8b40 con 0x55ffb1810290 2024-09-10T21:36:07.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 10 client.11031 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 10 client.11031 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 20 client.11031 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb18244c0 0x55ffb18248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 4 client.11031 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2957520968}] 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2957520968}) -- 0x55ffb161f690 con 0x55ffb18244c0 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f822e800640 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb18244c0 0x55ffb18248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 10 client.11031 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f822e800640 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb18244c0 0x55ffb18248a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c648f750a7a90aa6 server_cookie=75620cbe9adeeb37 in_seq=0 out_seq=0 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 1 -- 172.21.15.97:0/2048087250 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55ffb161f690 con 0x55ffb18244c0 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 10 client.11031 handle_command_reply: tid=0 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 client.11031 shutdown 2024-09-10T21:36:07.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 msgr2=0x55ffb1886cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1886cb0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f82280f4b30 tx=0x7f82280f7520 comp rx=0 tx=0).stop 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 shutdown_connections 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffb1810290 0x55ffb1886cb0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x55ffb180f970 0x55ffb1886770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x55ffb18244c0 0x55ffb18248a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 --2- 172.21.15.97:0/2048087250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ffb17c32e0 0x55ffb1886230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 >> 172.21.15.97:0/2048087250 conn(0x55ffb17f0780 msgr2=0x55ffb17cb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f820fe00640 0 client.11031 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 shutdown_connections 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 1 -- 172.21.15.97:0/2048087250 wait complete. 2024-09-10T21:36:07.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f82387e7000 20 client.11031 trim_cache size 0 max 16384 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8210072610 msgr2=0x7f8210074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8210072610 0x7f8210074ad0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8220001ec0 tx=0x7f8220006040 comp rx=0 tx=0).stop 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f823013e0f0 msgr2=0x7f8230156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f823013e0f0 0x7f8230156710 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f82280ab040 tx=0x7f8228006e80 comp rx=0 tx=0).stop 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 shutdown_connections 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8210072610 0x7f8210074ad0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f823013e0f0 0x7f8230156710 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f823013dc70 0x7f823015d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 --2- 172.21.15.97:0/2559874749 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f8214004d50 0x7f8230155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 >> 172.21.15.97:0/2559874749 conn(0x7f82300b1780 msgr2=0x7f82300b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 shutdown_connections 2024-09-10T21:36:07.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.005+0000 7f8235400640 1 -- 172.21.15.97:0/2559874749 wait complete. 2024-09-10T21:36:07.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:07.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.165+0000 7f9b85600640 1 -- 172.21.15.97:0/3918987307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b64003a40 msgr2=0x7f9b64003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.165+0000 7f9b85600640 1 --2- 172.21.15.97:0/3918987307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b64003a40 0x7f9b64003e60 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009b60 tx=0x7f9b7001d270 comp rx=0 tx=0).stop 2024-09-10T21:36:07.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.165+0000 7f9b7f400640 1 -- 172.21.15.97:0/3918987307 reap_dead start 2024-09-10T21:36:07.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.165+0000 7f9b85600640 1 -- 172.21.15.97:0/3918987307 shutdown_connections 2024-09-10T21:36:07.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.165+0000 7f9b85600640 1 -- 172.21.15.97:0/3918987307 >> 172.21.15.97:0/3918987307 conn(0x7f9b800b37c0 msgr2=0x7f9b800b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.165+0000 7f9b85600640 1 -- 172.21.15.97:0/3918987307 shutdown_connections 2024-09-10T21:36:07.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.165+0000 7f9b85600640 1 -- 172.21.15.97:0/3918987307 wait complete. 2024-09-10T21:36:07.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 Processor -- start 2024-09-10T21:36:07.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 -- start start 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b64003a40 0x7f9b800c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b800b6dc0 0x7f9b800c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800b7240 0x7f9b800c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b800ce1a0 con 0x7f9b800b7240 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9b800cde60 con 0x7f9b800b6dc0 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9b800ce320 con 0x7f9b64003a40 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800b7240 0x7f9b800c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800b7240 0x7f9b800c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46906/0 (socket says 172.21.15.97:46906) 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 -- 172.21.15.97:0/3782919809 learned_addr learned my addr 172.21.15.97:0/3782919809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b76000640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b800b6dc0 0x7f9b800c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7ea00640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b64003a40 0x7f9b800c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 -- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b64003a40 msgr2=0x7f9b800c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b64003a40 0x7f9b800c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 -- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b800b6dc0 msgr2=0x7f9b800c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b800b6dc0 0x7f9b800c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 -- 172.21.15.97:0/3782919809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b801e4af0 con 0x7f9b800b7240 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b76000640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b800b6dc0 0x7f9b800c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b7f400640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800b7240 0x7f9b800c6620 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f9b80069d30 tx=0x7f9b780019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b77400640 1 -- 172.21.15.97:0/3782919809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78015020 con 0x7f9b800b7240 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b77400640 1 -- 172.21.15.97:0/3782919809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b780020c0 con 0x7f9b800b7240 2024-09-10T21:36:07.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b77400640 1 -- 172.21.15.97:0/3782919809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011c20 con 0x7f9b800b7240 2024-09-10T21:36:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b801e4dd0 con 0x7f9b800b7240 2024-09-10T21:36:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.169+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b801e50b0 con 0x7f9b800b7240 2024-09-10T21:36:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.173+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b800cf800 con 0x7f9b800b7240 2024-09-10T21:36:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.173+0000 7f9b77400640 1 -- 172.21.15.97:0/3782919809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9b780103c0 con 0x7f9b800b7240 2024-09-10T21:36:07.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.173+0000 7f9b77400640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b5c0725e0 0x7f9b5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.173+0000 7f9b77400640 1 -- 172.21.15.97:0/3782919809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f9b78093eb0 con 0x7f9b800b7240 2024-09-10T21:36:07.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.173+0000 7f9b7ea00640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b5c0725e0 0x7f9b5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.173+0000 7f9b7ea00640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b5c0725e0 0x7f9b5c074aa0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9b700ae000 tx=0x7f9b7001a010 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.177+0000 7f9b77400640 1 -- 172.21.15.97:0/3782919809 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9b7801f030 con 0x7f9b800b7240 2024-09-10T21:36:07.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b800b24a0 con 0x7f9b800b7240 2024-09-10T21:36:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b77400640 1 -- 172.21.15.97:0/3782919809 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7f9b7805f330 con 0x7f9b800b7240 2024-09-10T21:36:07.376 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:07.376 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:07.376 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:07.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b5c0725e0 msgr2=0x7f9b5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b5c0725e0 0x7f9b5c074aa0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f9b700ae000 tx=0x7f9b7001a010 comp rx=0 tx=0).stop 2024-09-10T21:36:07.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800b7240 msgr2=0x7f9b800c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800b7240 0x7f9b800c6620 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f9b80069d30 tx=0x7f9b780019a0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 shutdown_connections 2024-09-10T21:36:07.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9b5c0725e0 0x7f9b5c074aa0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800b7240 0x7f9b800c6620 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b800b6dc0 0x7f9b800c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 --2- 172.21.15.97:0/3782919809 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9b64003a40 0x7f9b800c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 >> 172.21.15.97:0/3782919809 conn(0x7f9b800b37c0 msgr2=0x7f9b800cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 shutdown_connections 2024-09-10T21:36:07.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.377+0000 7f9b85600640 1 -- 172.21.15.97:0/3782919809 wait complete. 2024-09-10T21:36:07.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3824306573 2024-09-10T21:36:07.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 -- 172.21.15.97:0/9483957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 msgr2=0x7fe83c0b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 --2- 172.21.15.97:0/9483957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c0b4120 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe83c074ee0 tx=0x7fe82801c800 comp rx=0 tx=0).stop 2024-09-10T21:36:07.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 -- 172.21.15.97:0/9483957 shutdown_connections 2024-09-10T21:36:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 --2- 172.21.15.97:0/9483957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c0b4120 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 --2- 172.21.15.97:0/9483957 >> v2:172.21.15.194:3300/0 conn(0x7fe83c143f90 0x7fe83c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 -- 172.21.15.97:0/9483957 >> 172.21.15.97:0/9483957 conn(0x7fe83c0b1780 msgr2=0x7fe83c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 -- 172.21.15.97:0/9483957 shutdown_connections 2024-09-10T21:36:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 -- 172.21.15.97:0/9483957 wait complete. 2024-09-10T21:36:07.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 Processor -- start 2024-09-10T21:36:07.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.541+0000 7fe844000640 1 -- start start 2024-09-10T21:36:07.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe83c0b4660 0x7fe83c13da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe83c143f90 0x7fe83c136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c13abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1679e0 con 0x7fe83c144690 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c167860 con 0x7fe83c143f90 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c167b60 con 0x7fe83c0b4660 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c13abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c13abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52780/0 (socket says 172.21.15.97:52780) 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 -- 172.21.15.97:0/1490049026 learned_addr learned my addr 172.21.15.97:0/1490049026 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe841600640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe83c0b4660 0x7fe83c13da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe840c00640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe83c143f90 0x7fe83c136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 -- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe83c0b4660 msgr2=0x7fe83c13da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe83c0b4660 0x7fe83c13da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 -- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe83c143f90 msgr2=0x7fe83c136ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe83c143f90 0x7fe83c136ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 -- 172.21.15.97:0/1490049026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c13b2e0 con 0x7fe83c144690 2024-09-10T21:36:07.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe842000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c13abe0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe83400cbc0 tx=0x7fe83400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe839600640 1 -- 172.21.15.97:0/1490049026 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834016020 con 0x7fe83c144690 2024-09-10T21:36:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe839600640 1 -- 172.21.15.97:0/1490049026 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe83400ab00 con 0x7fe83c144690 2024-09-10T21:36:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c137dd0 con 0x7fe83c144690 2024-09-10T21:36:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe839600640 1 -- 172.21.15.97:0/1490049026 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834012420 con 0x7fe83c144690 2024-09-10T21:36:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.545+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c14eb60 con 0x7fe83c144690 2024-09-10T21:36:07.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe839600640 1 -- 172.21.15.97:0/1490049026 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe8340125c0 con 0x7fe83c144690 2024-09-10T21:36:07.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe839600640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe81c0725e0 0x7fe81c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe839600640 1 -- 172.21.15.97:0/1490049026 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fe834093950 con 0x7fe83c144690 2024-09-10T21:36:07.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 Processor -- start 2024-09-10T21:36:07.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe841600640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe81c0725e0 0x7fe81c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 -- start start 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe841600640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe81c0725e0 0x7fe81c074aa0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe830009be0 tx=0x7fe8300091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e51970 0x5612f2e51d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2e056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612f2ceea80 con 0x5612f2e52290 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612f2c63c90 con 0x5612f2e052e0 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5612f2c530c0 con 0x5612f2e51970 2024-09-10T21:36:07.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e04da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52784/0 (socket says 172.21.15.97:52784) 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 -- 172.21.15.97:0/2752663167 learned_addr learned my addr 172.21.15.97:0/2752663167 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe841600640 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e51970 0x5612f2e51d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe842000640 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2e056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 -- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e51970 msgr2=0x5612f2e51d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e51970 0x5612f2e51d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 -- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 msgr2=0x5612f2e056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2e056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 -- 172.21.15.97:0/2752663167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f2bf7350 con 0x5612f2e52290 2024-09-10T21:36:07.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe842000640 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2e056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:07.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe840c00640 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e04da0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fe828000c10 tx=0x7fe8280195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c725e57bf926c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe82e000640 1 -- 172.21.15.97:0/2752663167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828004200 con 0x5612f2e52290 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe82e000640 1 -- 172.21.15.97:0/2752663167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe828004490 con 0x5612f2e52290 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.549+0000 7fe82e000640 1 -- 172.21.15.97:0/2752663167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828005280 con 0x5612f2e52290 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- 172.21.15.97:0/2752663167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 msgr2=0x5612f2e04da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e04da0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fe828000c10 tx=0x7fe8280195a0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- 172.21.15.97:0/2752663167 shutdown_connections 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2e056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e04da0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 --2- 172.21.15.97:0/2752663167 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e51970 0x5612f2e51d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- 172.21.15.97:0/2752663167 >> 172.21.15.97:0/2752663167 conn(0x5612f2e32780 msgr2=0x5612f2e34ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- 172.21.15.97:0/2752663167 shutdown_connections 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- 172.21.15.97:0/2752663167 wait complete. 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- start start 2024-09-10T21:36:07.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 Processor -- start 2024-09-10T21:36:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e58af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e59030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e59570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612f2cd7600 con 0x5612f2e52290 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612f2d121e0 con 0x5612f2e51970 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5612f2bfcb10 con 0x5612f2e052e0 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 10 client.0 _ll_register_callbacks cb 0x5612f2df2c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e59570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e59570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52796/0 (socket says 172.21.15.97:52796) 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 -- 172.21.15.97:0/4141679035 learned_addr learned my addr 172.21.15.97:0/4141679035 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe840c00640 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e59030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe841600640 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e58af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 -- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 msgr2=0x5612f2e58af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e58af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 -- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 msgr2=0x5612f2e59030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e59030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 -- 172.21.15.97:0/4141679035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f2bf1260 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe842000640 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e59570 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fe83400cb90 tx=0x7fe83405df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5424814380b444eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe822a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe822a00640 1 -- 172.21.15.97:0/4141679035 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834063120 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe822a00640 1 -- 172.21.15.97:0/4141679035 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fe834065330 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612f2beb1e0 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe822a00640 1 -- 172.21.15.97:0/4141679035 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fe83408f050 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 10 client.11052 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x5612f2c1cac0 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.553+0000 7fe822a00640 1 -- 172.21.15.97:0/4141679035 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5612f2beb1e0 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe822a00640 1 -- 172.21.15.97:0/4141679035 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fe83407fb00 con 0x5612f2e52290 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 10 client.11052 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 10 client.11052 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:07.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 20 client.11052 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:07.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e60ec0 0x5612f2e632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 4 client.11052 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:36:07.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f2c65be0 con 0x5612f2e60ec0 2024-09-10T21:36:07.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe841600640 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e60ec0 0x5612f2e632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe841600640 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e60ec0 0x5612f2e632b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b7f1bf31e138d0 server_cookie=375775d9c0975b15 in_seq=0 out_seq=0 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe822a00640 10 client.11052 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe822a00640 1 -- 172.21.15.97:0/4141679035 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612f2c65be0 con 0x5612f2e60ec0 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe822a00640 10 client.11052 handle_command_reply: tid=0 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 client.11052 shutdown 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 msgr2=0x5612f2e59570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e59570 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fe83400cb90 tx=0x7fe83405df90 comp rx=0 tx=0).stop 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 shutdown_connections 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e59570 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e60ec0 0x5612f2e632b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e59030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 --2- 172.21.15.97:0/4141679035 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e58af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe822a00640 0 client.11052 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 >> 172.21.15.97:0/4141679035 conn(0x5612f2e32780 msgr2=0x5612f2e34b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 shutdown_connections 2024-09-10T21:36:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 1 -- 172.21.15.97:0/4141679035 wait complete. 2024-09-10T21:36:07.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.557+0000 7fe8473d5000 20 client.11052 trim_cache size 0 max 16384 2024-09-10T21:36:07.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 Processor -- start 2024-09-10T21:36:07.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- start start 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e03d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c2036c0 con 0x5612f2e52290 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c138fc0 con 0x5612f2e51970 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c203070 con 0x5612f2e052e0 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52804/0 (socket says 172.21.15.97:52804) 2024-09-10T21:36:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 -- 172.21.15.97:0/1341633373 learned_addr learned my addr 172.21.15.97:0/1341633373 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe841600640 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e03d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe840c00640 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 -- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 msgr2=0x5612f2e03d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e03d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 -- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 msgr2=0x5612f2e042d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e042d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 -- 172.21.15.97:0/1341633373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f2cd7a90 con 0x5612f2e52290 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe842000640 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e598c0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fe83400af70 tx=0x7fe83405add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0e4b29189bc921 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe82d600640 1 -- 172.21.15.97:0/1341633373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8340782c0 con 0x5612f2e52290 2024-09-10T21:36:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe82d600640 1 -- 172.21.15.97:0/1341633373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe834087070 con 0x5612f2e52290 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe82d600640 1 -- 172.21.15.97:0/1341633373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834065330 con 0x5612f2e52290 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- 172.21.15.97:0/1341633373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 msgr2=0x5612f2e598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e598c0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fe83400af70 tx=0x7fe83405add0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- 172.21.15.97:0/1341633373 shutdown_connections 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e52290 0x5612f2e598c0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e51970 0x5612f2e042d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1341633373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e052e0 0x5612f2e03d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- 172.21.15.97:0/1341633373 >> 172.21.15.97:0/1341633373 conn(0x5612f2e32780 msgr2=0x5612f2e0a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- 172.21.15.97:0/1341633373 shutdown_connections 2024-09-10T21:36:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.573+0000 7fe8473d5000 1 -- 172.21.15.97:0/1341633373 wait complete. 2024-09-10T21:36:07.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 -- start start 2024-09-10T21:36:07.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 Processor -- start 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2ec7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e51970 0x5612f2ec7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e52290 0x5612f2ec84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612f2c62a60 con 0x5612f2e51970 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5612f2bf7b10 con 0x5612f2e052e0 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x5612f2c561e0 con 0x5612f2e52290 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 10 client.0 _ll_register_callbacks cb 0x7fe834001c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2ec7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2ec7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.97:38592/0 (socket says 172.21.15.97:38592) 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 -- 172.21.15.97:0/1683706504 learned_addr learned my addr 172.21.15.97:0/1683706504 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe840c00640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e51970 0x5612f2ec7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe842000640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e52290 0x5612f2ec84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 -- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e52290 msgr2=0x5612f2ec84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e52290 0x5612f2ec84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 -- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e51970 msgr2=0x5612f2ec7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:36:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e51970 0x5612f2ec7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 -- 172.21.15.97:0/1683706504 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f2bf8330 con 0x5612f2e052e0 2024-09-10T21:36:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe840c00640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e51970 0x5612f2ec7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:36:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe841600640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2ec7a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8300118d0 tx=0x7fe830004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4035454351d896d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:07.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe822a00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe822a00640 1 -- 172.21.15.97:0/1683706504 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830005020 con 0x5612f2e052e0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe822a00640 1 -- 172.21.15.97:0/1683706504 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fe83002d040 con 0x5612f2e052e0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe834087070 con 0x5612f2e052e0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe822a00640 1 -- 172.21.15.97:0/1683706504 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=350) ==== 24+0+0 (secure 0 0 0) 0x7fe83003c050 con 0x5612f2e052e0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 10 client.10036 fetch_fsmap learned FSMap version 350 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=350,monmap=2+}) -- 0x5612f2c1bcb0 con 0x5612f2e052e0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe822a00640 1 -- 172.21.15.97:0/1683706504 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe834087070 con 0x5612f2e052e0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe822a00640 1 -- 172.21.15.97:0/1683706504 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 350) ==== 3502+0+0 (secure 0 0 0) 0x7fe83003f020 con 0x5612f2e052e0 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 10 client.10036 fetch_fsmap finished waiting for FSMap version 350 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 10 client.10036 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 20 client.10036 populate_metadata read hostname 'smithi097' 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e5e120 0x5612f2e5d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 4 client.10036 mds_command: new command op to 10629 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3824306573}] 2024-09-10T21:36:07.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.577+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 --> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3824306573}) -- 0x5612f2c61690 con 0x5612f2e5e120 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe840c00640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e5e120 0x5612f2e5d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe840c00640 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e5e120 0x5612f2e5d240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2788d3d47231721 server_cookie=b5b00cc81b70d15e in_seq=0 out_seq=0 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe822a00640 10 client.10036 ms_handle_connect on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe822a00640 1 -- 172.21.15.97:0/1683706504 <== mds.0 v2:172.21.15.97:6833/2844434294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5612f2c61690 con 0x5612f2e5e120 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe822a00640 10 client.10036 handle_command_reply: tid=0 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 client.10036 shutdown 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 msgr2=0x5612f2ec7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2ec7a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8300118d0 tx=0x7fe830004740 comp rx=0 tx=0).stop 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 shutdown_connections 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x5612f2e52290 0x5612f2ec84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f2e51970 0x5612f2ec7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.97:6833/2844434294,v1:172.21.15.97:6835/2844434294] conn(0x5612f2e5e120 0x5612f2e5d240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 --2- 172.21.15.97:0/1683706504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5612f2e052e0 0x5612f2ec7a60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe822a00640 0 client.10036 ms_handle_reset on v2:172.21.15.97:6833/2844434294 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 >> 172.21.15.97:0/1683706504 conn(0x5612f2e32780 msgr2=0x5612f2e03d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 shutdown_connections 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 1 -- 172.21.15.97:0/1683706504 wait complete. 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe8473d5000 20 client.10036 trim_cache size 0 max 16384 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe81c0725e0 msgr2=0x7fe81c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe81c0725e0 0x7fe81c074aa0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe830009be0 tx=0x7fe8300091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 msgr2=0x7fe83c13abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c13abe0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe83400cbc0 tx=0x7fe83400ccb0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 shutdown_connections 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe81c0725e0 0x7fe81c074aa0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c144690 0x7fe83c13abe0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe83c143f90 0x7fe83c136ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 --2- 172.21.15.97:0/1490049026 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe83c0b4660 0x7fe83c13da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:07.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 >> 172.21.15.97:0/1490049026 conn(0x7fe83c0b1780 msgr2=0x7fe83c1422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:07.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 shutdown_connections 2024-09-10T21:36:07.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:07.581+0000 7fe844000640 1 -- 172.21.15.97:0/1490049026 wait complete. 2024-09-10T21:36:07.599 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:36:07.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:36:07.681 DEBUG:teuthology.orchestra.run.smithi194:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:36:07.782 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:36:07.782 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:36:07.782 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:36:07.782 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:07.791 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:36:07.791 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:07.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:36:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- 172.21.15.97:0/4218904947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0143f90 msgr2=0x7f84e00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 --2- 172.21.15.97:0/4218904947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0143f90 0x7f84e00b2a80 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f84e0073080 tx=0x7f84d001cb80 comp rx=0 tx=0).stop 2024-09-10T21:36:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- 172.21.15.97:0/4218904947 shutdown_connections 2024-09-10T21:36:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 --2- 172.21.15.97:0/4218904947 >> v2:172.21.15.194:3300/0 conn(0x7f84e006fd50 0x7f84e0070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 --2- 172.21.15.97:0/4218904947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0143f90 0x7f84e00b2a80 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- 172.21.15.97:0/4218904947 >> 172.21.15.97:0/4218904947 conn(0x7f84e00b1780 msgr2=0x7f84e00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:08.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- 172.21.15.97:0/4218904947 shutdown_connections 2024-09-10T21:36:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- 172.21.15.97:0/4218904947 wait complete. 2024-09-10T21:36:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 Processor -- start 2024-09-10T21:36:08.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- start start 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e006fd50 0x7f84e01416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e0143f90 0x7f84e01fbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0144690 0x7f84e0141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0167ca0 con 0x7f84e0144690 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0167b20 con 0x7f84e0143f90 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84e5400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f84e0167e20 con 0x7f84e006fd50 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84df200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0144690 0x7f84e0141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84df200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0144690 0x7f84e0141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52826/0 (socket says 172.21.15.97:52826) 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84df200640 1 -- 172.21.15.97:0/975118110 learned_addr learned my addr 172.21.15.97:0/975118110 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84de800640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e006fd50 0x7f84e01416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.029+0000 7f84d5e00640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e0143f90 0x7f84e01fbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84df200640 1 -- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e006fd50 msgr2=0x7f84e01416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84df200640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e006fd50 0x7f84e01416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84df200640 1 -- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e0143f90 msgr2=0x7f84e01fbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84df200640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e0143f90 0x7f84e01fbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84df200640 1 -- 172.21.15.97:0/975118110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e01fc490 con 0x7f84e0144690 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84df200640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0144690 0x7f84e0141ec0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f84d800d750 tx=0x7f84d800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84d7400640 1 -- 172.21.15.97:0/975118110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8017070 con 0x7f84e0144690 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84d7400640 1 -- 172.21.15.97:0/975118110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84d8009cc0 con 0x7f84e0144690 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84d7400640 1 -- 172.21.15.97:0/975118110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800de60 con 0x7f84e0144690 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e01f8330 con 0x7f84e0144690 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01f88f0 con 0x7f84e0144690 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e0070160 con 0x7f84e0144690 2024-09-10T21:36:08.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84d7400640 1 -- 172.21.15.97:0/975118110 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f84d8005280 con 0x7f84e0144690 2024-09-10T21:36:08.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84d7400640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84bc072610 0x7f84bc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.033+0000 7f84d7400640 1 -- 172.21.15.97:0/975118110 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f84d8093410 con 0x7f84e0144690 2024-09-10T21:36:08.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.037+0000 7f84de800640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84bc072610 0x7f84bc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.037+0000 7f84de800640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84bc072610 0x7f84bc074ad0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f84d00af000 tx=0x7f84d001b3c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:08.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.037+0000 7f84d7400640 1 -- 172.21.15.97:0/975118110 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f84e01fc490 con 0x7f84e0144690 2024-09-10T21:36:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84e00b45f0 con 0x7f84e0144690 2024-09-10T21:36:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84d7400640 1 -- 172.21.15.97:0/975118110 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v149) ==== 80+0+124 (secure 0 0 0) 0x7f84d805e2e0 con 0x7f84e0144690 2024-09-10T21:36:08.216 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.194:6837/3945561983 2024-09-11T21:35:57.522868+0000 2024-09-10T21:36:08.216 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.194:6835/3945561983 2024-09-11T21:35:57.522868+0000 2024-09-10T21:36:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:listed 2 entries 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84bc072610 msgr2=0x7f84bc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84bc072610 0x7f84bc074ad0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f84d00af000 tx=0x7f84d001b3c0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0144690 msgr2=0x7f84e0141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0144690 0x7f84e0141ec0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f84d800d750 tx=0x7f84d800dc10 comp rx=0 tx=0).stop 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 shutdown_connections 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f84bc072610 0x7f84bc074ad0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e0144690 0x7f84e0141ec0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e0143f90 0x7f84e01fbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 --2- 172.21.15.97:0/975118110 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f84e006fd50 0x7f84e01416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 >> 172.21.15.97:0/975118110 conn(0x7f84e00b1780 msgr2=0x7f84e00cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 shutdown_connections 2024-09-10T21:36:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.217+0000 7f84e5400640 1 -- 172.21.15.97:0/975118110 wait complete. 2024-09-10T21:36:08.237 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:36:08.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:08.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:36:08.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:36:08.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:08.263 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:36:08.316 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:36:08.316 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:36:08.316 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:08.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:36:08.326 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:36:08.326 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:36:08.326 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:08.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:36:08.367 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:36:08.367 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:36:08.367 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:08.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:36:08.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:08.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- 172.21.15.97:0/2414292855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40ce640 msgr2=0x7fc1e40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2414292855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40ce640 0x7fc1e40cea20 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009840 tx=0x7fc1d401c820 comp rx=0 tx=0).stop 2024-09-10T21:36:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- 172.21.15.97:0/2414292855 shutdown_connections 2024-09-10T21:36:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2414292855 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1e40cd6e0 0x7fc1e40af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2414292855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40ce640 0x7fc1e40cea20 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- 172.21.15.97:0/2414292855 >> 172.21.15.97:0/2414292855 conn(0x7fc1e40d1c30 msgr2=0x7fc1e40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- 172.21.15.97:0/2414292855 shutdown_connections 2024-09-10T21:36:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- 172.21.15.97:0/2414292855 wait complete. 2024-09-10T21:36:08.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 Processor -- start 2024-09-10T21:36:08.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- start start 2024-09-10T21:36:08.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1e40b0d50 0x7fc1e41f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40cd6e0 0x7fc1e40b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1e40ce640 0x7fc1e40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4168cc0 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4168b40 con 0x7fc1e40b0d50 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1eb200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e4168e40 con 0x7fc1e40ce640 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40cd6e0 0x7fc1e40b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.553+0000 7fc1db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40cd6e0 0x7fc1e40b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52854/0 (socket says 172.21.15.97:52854) 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1db400640 1 -- 172.21.15.97:0/2665804189 learned_addr learned my addr 172.21.15.97:0/2665804189 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e9200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1e40ce640 0x7fc1e40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e3e00640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1e40b0d50 0x7fc1e41f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1db400640 1 -- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1e40ce640 msgr2=0x7fc1e40c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1db400640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1e40ce640 0x7fc1e40c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1db400640 1 -- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1e40b0d50 msgr2=0x7fc1e41f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1db400640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1e40b0d50 0x7fc1e41f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1db400640 1 -- 172.21.15.97:0/2665804189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e40b34f0 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e9200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1e40ce640 0x7fc1e40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e3e00640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1e40b0d50 0x7fc1e41f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1db400640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40cd6e0 0x7fc1e40b6100 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00c420 tx=0x7fc1cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e0c00640 1 -- 172.21.15.97:0/2665804189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc017070 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e0c00640 1 -- 172.21.15.97:0/2665804189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1cc003600 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e41f6ef0 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e0c00640 1 -- 172.21.15.97:0/2665804189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc003a90 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e41f74b0 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e40cea20 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.557+0000 7fc1e0c00640 1 -- 172.21.15.97:0/2665804189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc1cc003c30 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.561+0000 7fc1e0c00640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1c0072610 0x7fc1c0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.561+0000 7fc1e0c00640 1 -- 172.21.15.97:0/2665804189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fc1cc092a50 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.561+0000 7fc1e3e00640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1c0072610 0x7fc1c0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.561+0000 7fc1e3e00640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1c0072610 0x7fc1c0074ad0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40af000 tx=0x7fc1d40195c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:08.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.561+0000 7fc1e0c00640 1 -- 172.21.15.97:0/2665804189 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc1cc05ded0 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.761+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1e4075f90 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1e0c00640 1 -- 172.21.15.97:0/2665804189 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7fc1cc05d7d0 con 0x7fc1e40cd6e0 2024-09-10T21:36:08.763 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:08.764 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:08.764 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1c0072610 msgr2=0x7fc1c0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1c0072610 0x7fc1c0074ad0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40af000 tx=0x7fc1d40195c0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40cd6e0 msgr2=0x7fc1e40b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40cd6e0 0x7fc1e40b6100 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00c420 tx=0x7fc1cc00c8e0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 shutdown_connections 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fc1c0072610 0x7fc1c0074ad0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fc1e40ce640 0x7fc1e40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1e40cd6e0 0x7fc1e40b6100 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 --2- 172.21.15.97:0/2665804189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1e40b0d50 0x7fc1e41f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 >> 172.21.15.97:0/2665804189 conn(0x7fc1e40d1c30 msgr2=0x7fc1e40b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 shutdown_connections 2024-09-10T21:36:08.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.765+0000 7fc1eb200640 1 -- 172.21.15.97:0/2665804189 wait complete. 2024-09-10T21:36:08.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- 172.21.15.97:0/3024196657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 msgr2=0x7f1c20141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 --2- 172.21.15.97:0/3024196657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c20141d20 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f1c20074800 tx=0x7f1c1001c800 comp rx=0 tx=0).stop 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- 172.21.15.97:0/3024196657 shutdown_connections 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 --2- 172.21.15.97:0/3024196657 >> v2:172.21.15.194:3300/0 conn(0x7f1c200ce9d0 0x7f1c2015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 --2- 172.21.15.97:0/3024196657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c20141d20 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- 172.21.15.97:0/3024196657 >> 172.21.15.97:0/3024196657 conn(0x7f1c200b1780 msgr2=0x7f1c200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- 172.21.15.97:0/3024196657 shutdown_connections 2024-09-10T21:36:08.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- 172.21.15.97:0/3024196657 wait complete. 2024-09-10T21:36:08.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 Processor -- start 2024-09-10T21:36:08.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- start start 2024-09-10T21:36:08.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c200b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c200ce9d0 0x7f1c20073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c2015fc10 0x7f1c200740d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20169020 con 0x7f1c200ce5f0 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20168ea0 con 0x7f1c2015fc10 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c24c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f1c201691a0 con 0x7f1c200ce9d0 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c200b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c200b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52866/0 (socket says 172.21.15.97:52866) 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 -- 172.21.15.97:0/2323510404 learned_addr learned my addr 172.21.15.97:0/2323510404 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c15400640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c200ce9d0 0x7f1c20073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1e800640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c2015fc10 0x7f1c200740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 -- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c200ce9d0 msgr2=0x7f1c20073b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c200ce9d0 0x7f1c20073b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 -- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c2015fc10 msgr2=0x7f1c200740d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c2015fc10 0x7f1c200740d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:08.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1de00640 1 -- 172.21.15.97:0/2323510404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2013bc60 con 0x7f1c200ce5f0 2024-09-10T21:36:08.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.929+0000 7f1c1e800640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c2015fc10 0x7f1c200740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:08.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c1de00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c200b46a0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c10 tx=0x7f1c10004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c16a00640 1 -- 172.21.15.97:0/2323510404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10004430 con 0x7f1c200ce5f0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c16a00640 1 -- 172.21.15.97:0/2323510404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c100c0070 con 0x7f1c200ce5f0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c16a00640 1 -- 172.21.15.97:0/2323510404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10003930 con 0x7f1c200ce5f0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2013bf40 con 0x7f1c200ce5f0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2013c500 con 0x7f1c200ce5f0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c20141d20 con 0x7f1c200ce5f0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c16a00640 1 -- 172.21.15.97:0/2323510404 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1c10003ad0 con 0x7f1c200ce5f0 2024-09-10T21:36:08.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c16a00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:08.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c16a00640 1 -- 172.21.15.97:0/2323510404 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f1c1012e790 con 0x7f1c200ce5f0 2024-09-10T21:36:08.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.933+0000 7f1c15400640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:08.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.937+0000 7f1c15400640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1c20159ba0 tx=0x7f1c0800b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:08.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:08.937+0000 7f1c16a00640 1 -- 172.21.15.97:0/2323510404 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1c100163f0 con 0x7f1c200ce5f0 2024-09-10T21:36:09.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.133+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c20166630 con 0x7f1c200ce5f0 2024-09-10T21:36:09.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c16a00640 1 -- 172.21.15.97:0/2323510404 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7f1c100f9c20 con 0x7f1c200ce5f0 2024-09-10T21:36:09.136 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:09.136 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:09.136 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 msgr2=0x7f1bfc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1c20159ba0 tx=0x7f1c0800b390 comp rx=0 tx=0).stop 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 msgr2=0x7f1c200b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c200b46a0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c10 tx=0x7f1c10004040 comp rx=0 tx=0).stop 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 shutdown_connections 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f1bfc0725e0 0x7f1bfc074aa0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c2015fc10 0x7f1c200740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f1c200ce9d0 0x7f1c20073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 --2- 172.21.15.97:0/2323510404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c200ce5f0 0x7f1c200b46a0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 >> 172.21.15.97:0/2323510404 conn(0x7f1c200b1780 msgr2=0x7f1c2013f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:09.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 shutdown_connections 2024-09-10T21:36:09.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.137+0000 7f1c24c00640 1 -- 172.21.15.97:0/2323510404 wait complete. 2024-09-10T21:36:09.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:36:09.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 -- 172.21.15.97:0/1904525816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce5f0 msgr2=0x7f745c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 --2- 172.21.15.97:0/1904525816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce5f0 0x7f745c141d20 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f745c141580 tx=0x7f745001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 -- 172.21.15.97:0/1904525816 shutdown_connections 2024-09-10T21:36:09.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 --2- 172.21.15.97:0/1904525816 >> v2:172.21.15.194:3300/0 conn(0x7f745c15fde0 0x7f745c13f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 --2- 172.21.15.97:0/1904525816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce5f0 0x7f745c141d20 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 -- 172.21.15.97:0/1904525816 >> 172.21.15.97:0/1904525816 conn(0x7f745c0b1780 msgr2=0x7f745c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:09.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 -- 172.21.15.97:0/1904525816 shutdown_connections 2024-09-10T21:36:09.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.297+0000 7f7464400640 1 -- 172.21.15.97:0/1904525816 wait complete. 2024-09-10T21:36:09.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 Processor -- start 2024-09-10T21:36:09.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 -- start start 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f745c0ce5f0 0x7f745c136040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce9d0 0x7f745c166110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f745c15fde0 0x7f745c13c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f745c168bf0 con 0x7f745c0ce9d0 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f745c168a70 con 0x7f745c0ce5f0 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f745c168d70 con 0x7f745c15fde0 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce9d0 0x7f745c166110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce9d0 0x7f745c166110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52894/0 (socket says 172.21.15.97:52894) 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 -- 172.21.15.97:0/630944946 learned_addr learned my addr 172.21.15.97:0/630944946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461a00640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f745c0ce5f0 0x7f745c136040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7462400640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f745c15fde0 0x7f745c13c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 -- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f745c15fde0 msgr2=0x7f745c13c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f745c15fde0 0x7f745c13c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 -- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f745c0ce5f0 msgr2=0x7f745c136040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f745c0ce5f0 0x7f745c136040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 -- 172.21.15.97:0/630944946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c166650 con 0x7f745c0ce9d0 2024-09-10T21:36:09.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7461000640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce9d0 0x7f745c166110 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f744800c450 tx=0x7f744800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:09.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7459600640 1 -- 172.21.15.97:0/630944946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448012830 con 0x7f745c0ce9d0 2024-09-10T21:36:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7459600640 1 -- 172.21.15.97:0/630944946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7448017070 con 0x7f745c0ce9d0 2024-09-10T21:36:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7459600640 1 -- 172.21.15.97:0/630944946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448012d80 con 0x7f745c0ce9d0 2024-09-10T21:36:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c159680 con 0x7f745c0ce9d0 2024-09-10T21:36:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.301+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c165000 con 0x7f745c0ce9d0 2024-09-10T21:36:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.305+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c0cca70 con 0x7f745c0ce9d0 2024-09-10T21:36:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.305+0000 7f7459600640 1 -- 172.21.15.97:0/630944946 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f744801c460 con 0x7f745c0ce9d0 2024-09-10T21:36:09.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.305+0000 7f7459600640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f743c0725e0 0x7f743c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.305+0000 7f7459600640 1 -- 172.21.15.97:0/630944946 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f7448093070 con 0x7f745c0ce9d0 2024-09-10T21:36:09.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.305+0000 7f7461a00640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f743c0725e0 0x7f743c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.305+0000 7f7461a00640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f743c0725e0 0x7f743c074aa0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f74500af000 tx=0x7f745001b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:09.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.309+0000 7f7459600640 1 -- 172.21.15.97:0/630944946 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f744805df40 con 0x7f745c0ce9d0 2024-09-10T21:36:09.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.477+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f745c141360 con 0x7f745c0ce9d0 2024-09-10T21:36:09.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.477+0000 7f7459600640 1 -- 172.21.15.97:0/630944946 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+17308 (secure 0 0 0) 0x7f7448062970 con 0x7f745c0ce9d0 2024-09-10T21:36:09.478 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:09.478 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":149,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:57.522888+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:35:40.751434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T21:35:42.030160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"16.11","mappings":[{"from":4,"to":6}]},{"pgid":"16.1a","mappings":[{"from":2,"to":3}]},{"pgid":"16.37","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6837/3945561983":"2024-09-11T21:35:57.522868+0000","172.21.15.194:6835/3945561983":"2024-09-11T21:35:57.522868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f743c0725e0 msgr2=0x7f743c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f743c0725e0 0x7f743c074aa0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f74500af000 tx=0x7f745001b100 comp rx=0 tx=0).stop 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce9d0 msgr2=0x7f745c166110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce9d0 0x7f745c166110 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f744800c450 tx=0x7f744800c910 comp rx=0 tx=0).stop 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 shutdown_connections 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f743c0725e0 0x7f743c074aa0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f745c15fde0 0x7f745c13c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f745c0ce9d0 0x7f745c166110 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 --2- 172.21.15.97:0/630944946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f745c0ce5f0 0x7f745c136040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 >> 172.21.15.97:0/630944946 conn(0x7f745c0b1780 msgr2=0x7f745c072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 shutdown_connections 2024-09-10T21:36:09.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.481+0000 7f7464400640 1 -- 172.21.15.97:0/630944946 wait complete. 2024-09-10T21:36:09.499 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:36:09.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 -- 172.21.15.97:0/347550515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 msgr2=0x7ffba813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 --2- 172.21.15.97:0/347550515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba813f9b0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009bc0 tx=0x7ffba001cba0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 -- 172.21.15.97:0/347550515 shutdown_connections 2024-09-10T21:36:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 --2- 172.21.15.97:0/347550515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba813f9b0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 --2- 172.21.15.97:0/347550515 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffba80ce5f0 0x7ffba8141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 -- 172.21.15.97:0/347550515 >> 172.21.15.97:0/347550515 conn(0x7ffba80b1780 msgr2=0x7ffba80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 -- 172.21.15.97:0/347550515 shutdown_connections 2024-09-10T21:36:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 -- 172.21.15.97:0/347550515 wait complete. 2024-09-10T21:36:09.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 Processor -- start 2024-09-10T21:36:09.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.641+0000 7ffbafc00640 1 -- start start 2024-09-10T21:36:09.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffba80ce5f0 0x7ffba815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffba80ce9d0 0x7ffba80b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba8164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8168bf0 con 0x7ffba815fde0 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8168a70 con 0x7ffba80ce9d0 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ffba8168d70 con 0x7ffba80ce5f0 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba8164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba8164b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52926/0 (socket says 172.21.15.97:52926) 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 -- 172.21.15.97:0/1743225430 learned_addr learned my addr 172.21.15.97:0/1743225430 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbad200640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffba80ce5f0 0x7ffba815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffb9fe00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffba80ce9d0 0x7ffba80b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 -- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffba80ce5f0 msgr2=0x7ffba815ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffba80ce5f0 0x7ffba815ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 -- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffba80ce9d0 msgr2=0x7ffba80b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffba80ce9d0 0x7ffba80b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 -- 172.21.15.97:0/1743225430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba80b3550 con 0x7ffba815fde0 2024-09-10T21:36:09.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbad200640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffba80ce5f0 0x7ffba815ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:09.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffb9fe00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffba80ce9d0 0x7ffba80b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:09.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbadc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba8164b80 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7ffba001a7d0 tx=0x7ffba0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffba5600640 1 -- 172.21.15.97:0/1743225430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba00bb070 con 0x7ffba815fde0 2024-09-10T21:36:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffba5600640 1 -- 172.21.15.97:0/1743225430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffba0004c00 con 0x7ffba815fde0 2024-09-10T21:36:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba80b37e0 con 0x7ffba815fde0 2024-09-10T21:36:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffba5600640 1 -- 172.21.15.97:0/1743225430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba00b2040 con 0x7ffba815fde0 2024-09-10T21:36:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba815d960 con 0x7ffba815fde0 2024-09-10T21:36:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.645+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba8141d20 con 0x7ffba815fde0 2024-09-10T21:36:09.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.649+0000 7ffba5600640 1 -- 172.21.15.97:0/1743225430 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffba00b5030 con 0x7ffba815fde0 2024-09-10T21:36:09.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.649+0000 7ffba5600640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb840725e0 0x7ffb84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:09.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.649+0000 7ffba5600640 1 -- 172.21.15.97:0/1743225430 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7ffba012ccd0 con 0x7ffba815fde0 2024-09-10T21:36:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.649+0000 7ffbad200640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb840725e0 0x7ffb84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:09.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.649+0000 7ffbad200640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb840725e0 0x7ffb84074aa0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ffb980058d0 tx=0x7ffb98005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:09.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.653+0000 7ffba5600640 1 -- 172.21.15.97:0/1743225430 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffba0016790 con 0x7ffba815fde0 2024-09-10T21:36:09.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.853+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffba81650c0 con 0x7ffba815fde0 2024-09-10T21:36:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.853+0000 7ffba5600640 1 -- 172.21.15.97:0/1743225430 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7ffba00f7b20 con 0x7ffba815fde0 2024-09-10T21:36:09.855 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:09.855 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:09.855 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb840725e0 msgr2=0x7ffb84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb840725e0 0x7ffb84074aa0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ffb980058d0 tx=0x7ffb98005860 comp rx=0 tx=0).stop 2024-09-10T21:36:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 msgr2=0x7ffba8164b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba8164b80 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7ffba001a7d0 tx=0x7ffba0003f60 comp rx=0 tx=0).stop 2024-09-10T21:36:09.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 shutdown_connections 2024-09-10T21:36:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ffb840725e0 0x7ffb84074aa0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba815fde0 0x7ffba8164b80 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffba80ce9d0 0x7ffba80b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 --2- 172.21.15.97:0/1743225430 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ffba80ce5f0 0x7ffba815ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 >> 172.21.15.97:0/1743225430 conn(0x7ffba80b1780 msgr2=0x7ffba81550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 shutdown_connections 2024-09-10T21:36:09.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:09.857+0000 7ffbafc00640 1 -- 172.21.15.97:0/1743225430 wait complete. 2024-09-10T21:36:09.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:10.008 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:10.011 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.017+0000 7ff7f7e00640 1 -- 172.21.15.97:0/2935188542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 msgr2=0x7ff7f0072630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.017+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/2935188542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 0x7ff7f0072630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0092e0 tx=0x7ff7dc01c750 comp rx=0 tx=0).stop 2024-09-10T21:36:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- 172.21.15.97:0/2935188542 shutdown_connections 2024-09-10T21:36:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/2935188542 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 0x7ff7f0072630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- 172.21.15.97:0/2935188542 >> 172.21.15.97:0/2935188542 conn(0x7ff7f00b1780 msgr2=0x7ff7f00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- 172.21.15.97:0/2935188542 shutdown_connections 2024-09-10T21:36:10.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- 172.21.15.97:0/2935188542 wait complete. 2024-09-10T21:36:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 Processor -- start 2024-09-10T21:36:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- start start 2024-09-10T21:36:10.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f00cf0b0 0x7ff7f0159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 0x7ff7f014ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7f015b110 0x7ff7f014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0168190 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0168010 con 0x7ff7f015b110 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f7e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f0168310 con 0x7ff7f00cf490 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f00cf0b0 0x7ff7f0159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f00cf0b0 0x7ff7f0159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52936/0 (socket says 172.21.15.97:52936) 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 -- 172.21.15.97:0/912737671 learned_addr learned my addr 172.21.15.97:0/912737671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f4a00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 0x7ff7f014ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7f015b110 0x7ff7f014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 -- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 msgr2=0x7ff7f014ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 0x7ff7f014ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 -- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7f015b110 msgr2=0x7ff7f014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7f015b110 0x7ff7f014cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 -- 172.21.15.97:0/912737671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f014d630 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7f015b110 0x7ff7f014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.021+0000 7ff7f5400640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f00cf0b0 0x7ff7f0159710 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4001230 tx=0x7ff7e400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7ed600640 1 -- 172.21.15.97:0/912737671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4016020 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7ed600640 1 -- 172.21.15.97:0/912737671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7e400a930 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f014d910 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7ed600640 1 -- 172.21.15.97:0/912737671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400ad80 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f014ded0 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f0072630 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7ed600640 1 -- 172.21.15.97:0/912737671 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7e401b400 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7ed600640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7d00725e0 0x7ff7d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7ed600640 1 -- 172.21.15.97:0/912737671 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7ff7e4093860 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.025+0000 7ff7f4a00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7d00725e0 0x7ff7d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.029+0000 7ff7f4a00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7d00725e0 0x7ff7d0074aa0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0af000 tx=0x7ff7dc01af90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:10.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.033+0000 7ff7ed600640 1 -- 172.21.15.97:0/912737671 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7e4020020 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f00b3f70 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7ed600640 1 -- 172.21.15.97:0/912737671 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5242 (secure 0 0 0) 0x7ff7e405ecf0 con 0x7ff7f00cf0b0 2024-09-10T21:36:10.232 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:10.232 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":350,"btime":"2024-09-10T21:36:01:544310+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:35:43.069729+0000","modified":"2024-09-10T21:36:01.544302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10629":{"gid":10629,"name":"a","rank":0,"incarnation":347,"state":"up:active","state_seq":8,"addr":"172.21.15.97:6835/2844434294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2844434294},{"type":"v1","addr":"172.21.15.97:6835","nonce":2844434294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10629,"qdb_cluster":[10629]},"id":7}]} 2024-09-10T21:36:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 350 2024-09-10T21:36:10.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7d00725e0 msgr2=0x7ff7d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7d00725e0 0x7ff7d0074aa0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0af000 tx=0x7ff7dc01af90 comp rx=0 tx=0).stop 2024-09-10T21:36:10.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f00cf0b0 msgr2=0x7ff7f0159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f00cf0b0 0x7ff7f0159710 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4001230 tx=0x7ff7e400cb90 comp rx=0 tx=0).stop 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 shutdown_connections 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff7d00725e0 0x7ff7d0074aa0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff7f015b110 0x7ff7f014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff7f00cf490 0x7ff7f014ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 --2- 172.21.15.97:0/912737671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7f00cf0b0 0x7ff7f0159710 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 >> 172.21.15.97:0/912737671 conn(0x7ff7f00b1780 msgr2=0x7ff7f00b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 shutdown_connections 2024-09-10T21:36:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.233+0000 7ff7f7e00640 1 -- 172.21.15.97:0/912737671 wait complete. 2024-09-10T21:36:10.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- 172.21.15.97:0/263600927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 msgr2=0x7f32280cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 --2- 172.21.15.97:0/263600927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32280cf370 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f322000ad90 tx=0x7f3220020490 comp rx=0 tx=0).stop 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- 172.21.15.97:0/263600927 shutdown_connections 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 --2- 172.21.15.97:0/263600927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32280cf370 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 --2- 172.21.15.97:0/263600927 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f322806ec40 0x7f322806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- 172.21.15.97:0/263600927 >> 172.21.15.97:0/263600927 conn(0x7f32280b1780 msgr2=0x7f32280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- 172.21.15.97:0/263600927 shutdown_connections 2024-09-10T21:36:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- 172.21.15.97:0/263600927 wait complete. 2024-09-10T21:36:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 Processor -- start 2024-09-10T21:36:10.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- start start 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f322806ec40 0x7f322814e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32281e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32281511f0 0x7f32281ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f322806e740 con 0x7f32280ceee0 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f322806e5c0 con 0x7f32281511f0 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f322806e8c0 con 0x7f322806ec40 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f3227400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32281e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f3227400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32281e60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52958/0 (socket says 172.21.15.97:52958) 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f3227400640 1 -- 172.21.15.97:0/1237587392 learned_addr learned my addr 172.21.15.97:0/1237587392 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:10.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f3227e00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f322806ec40 0x7f322814e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.397+0000 7f322ca00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32281511f0 0x7f32281ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f3227400640 1 -- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f322806ec40 msgr2=0x7f322814e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f3227400640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f322806ec40 0x7f322814e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f3227400640 1 -- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32281511f0 msgr2=0x7f32281ea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f3227400640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32281511f0 0x7f32281ea2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f3227400640 1 -- 172.21.15.97:0/1237587392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3228153380 con 0x7f32280ceee0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f3227400640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32281e60b0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f321400e7a0 tx=0x7f321400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f321fe00640 1 -- 172.21.15.97:0/1237587392 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321400ccc0 con 0x7f32280ceee0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f321fe00640 1 -- 172.21.15.97:0/1237587392 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3214010430 con 0x7f32280ceee0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32281e3570 con 0x7f32280ceee0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f321fe00640 1 -- 172.21.15.97:0/1237587392 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214010860 con 0x7f32280ceee0 2024-09-10T21:36:10.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281e3b30 con 0x7f32280ceee0 2024-09-10T21:36:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322806f060 con 0x7f32280ceee0 2024-09-10T21:36:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f321fe00640 1 -- 172.21.15.97:0/1237587392 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3214010a00 con 0x7f32280ceee0 2024-09-10T21:36:10.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f321fe00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32080725e0 0x7f3208074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f321fe00640 1 -- 172.21.15.97:0/1237587392 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7f3214093e40 con 0x7f32280ceee0 2024-09-10T21:36:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.401+0000 7f3227e00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32080725e0 0x7f3208074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.405+0000 7f3227e00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32080725e0 0x7f3208074aa0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3218007fd0 tx=0x7f3218009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:10.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.405+0000 7f321fe00640 1 -- 172.21.15.97:0/1237587392 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f321405ec70 con 0x7f32280ceee0 2024-09-10T21:36:10.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.573+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3228154600 con 0x7f32280ceee0 2024-09-10T21:36:10.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.573+0000 7f321fe00640 1 -- 172.21.15.97:0/1237587392 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+17308 (secure 0 0 0) 0x7f32140636a0 con 0x7f32280ceee0 2024-09-10T21:36:10.573 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:10.573 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":149,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:35:57.522888+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:35:40.751434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T21:35:42.030160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"16.11","mappings":[{"from":4,"to":6}]},{"pgid":"16.1a","mappings":[{"from":2,"to":3}]},{"pgid":"16.37","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6837/3945561983":"2024-09-11T21:35:57.522868+0000","172.21.15.194:6835/3945561983":"2024-09-11T21:35:57.522868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32080725e0 msgr2=0x7f3208074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32080725e0 0x7f3208074aa0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3218007fd0 tx=0x7f3218009040 comp rx=0 tx=0).stop 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 msgr2=0x7f32281e60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32281e60b0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f321400e7a0 tx=0x7f321400ec60 comp rx=0 tx=0).stop 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 shutdown_connections 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32080725e0 0x7f3208074aa0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32281511f0 0x7f32281ea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280ceee0 0x7f32281e60b0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 --2- 172.21.15.97:0/1237587392 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f322806ec40 0x7f322814e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 >> 172.21.15.97:0/1237587392 conn(0x7f32280b1780 msgr2=0x7f3228159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 shutdown_connections 2024-09-10T21:36:10.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.577+0000 7f322ea00640 1 -- 172.21.15.97:0/1237587392 wait complete. 2024-09-10T21:36:10.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:36:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 -- 172.21.15.97:0/772824515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa618004d50 msgr2=0x7fa6180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 --2- 172.21.15.97:0/772824515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa618004d50 0x7fa6180051d0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0016f0 tx=0x7fa62c01c7b0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa633400640 1 -- 172.21.15.97:0/772824515 reap_dead start 2024-09-10T21:36:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 -- 172.21.15.97:0/772824515 shutdown_connections 2024-09-10T21:36:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 -- 172.21.15.97:0/772824515 >> 172.21.15.97:0/772824515 conn(0x7fa6340b1780 msgr2=0x7fa6340b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 -- 172.21.15.97:0/772824515 shutdown_connections 2024-09-10T21:36:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 -- 172.21.15.97:0/772824515 wait complete. 2024-09-10T21:36:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 Processor -- start 2024-09-10T21:36:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.741+0000 7fa639a00640 1 -- start start 2024-09-10T21:36:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa618004d50 0x7fa634131ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6341592a0 0x7fa6341f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634159720 0x7fa6341f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341400c0 con 0x7fa634159720 2024-09-10T21:36:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa634160080 con 0x7fa618004d50 2024-09-10T21:36:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fa634140240 con 0x7fa6341592a0 2024-09-10T21:36:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634159720 0x7fa6341f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa632a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa618004d50 0x7fa634131ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634159720 0x7fa6341f8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52980/0 (socket says 172.21.15.97:52980) 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 -- 172.21.15.97:0/693026179 learned_addr learned my addr 172.21.15.97:0/693026179 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa632000640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6341592a0 0x7fa6341f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 -- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6341592a0 msgr2=0x7fa6341f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6341592a0 0x7fa6341f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 -- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa618004d50 msgr2=0x7fa634131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa618004d50 0x7fa634131ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 -- 172.21.15.97:0/693026179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340cb860 con 0x7fa634159720 2024-09-10T21:36:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa633400640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634159720 0x7fa6341f8500 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fa62c01cd20 tx=0x7fa62c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa62aa00640 1 -- 172.21.15.97:0/693026179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c001310 con 0x7fa634159720 2024-09-10T21:36:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa62aa00640 1 -- 172.21.15.97:0/693026179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa62c0a5910 con 0x7fa634159720 2024-09-10T21:36:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6340cbb40 con 0x7fa634159720 2024-09-10T21:36:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa62aa00640 1 -- 172.21.15.97:0/693026179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c006930 con 0x7fa634159720 2024-09-10T21:36:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63406e830 con 0x7fa634159720 2024-09-10T21:36:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6340cbd80 con 0x7fa634159720 2024-09-10T21:36:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.745+0000 7fa62aa00640 1 -- 172.21.15.97:0/693026179 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa62c0a5cc0 con 0x7fa634159720 2024-09-10T21:36:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.749+0000 7fa62aa00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6140725e0 0x7fa614074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.749+0000 7fa62aa00640 1 -- 172.21.15.97:0/693026179 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6303+0+0 (secure 0 0 0) 0x7fa62c12c950 con 0x7fa634159720 2024-09-10T21:36:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.749+0000 7fa632a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6140725e0 0x7fa614074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.749+0000 7fa632a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6140725e0 0x7fa614074aa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa63413dfa0 tx=0x7fa624006b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:10.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.753+0000 7fa62aa00640 1 -- 172.21.15.97:0/693026179 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa6340cbd80 con 0x7fa634159720 2024-09-10T21:36:10.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.949+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa6340cf100 con 0x7fa634159720 2024-09-10T21:36:10.950 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-10T21:36:10.949+0000 7f8d1aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:36:10.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.953+0000 7fa62aa00640 1 -- 172.21.15.97:0/693026179 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v351) ==== 161+0+0 (secure 0 0 0) 0x7fa62c0b4070 con 0x7fa634159720 2024-09-10T21:36:10.954 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:36:10.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6140725e0 msgr2=0x7fa614074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6140725e0 0x7fa614074aa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa63413dfa0 tx=0x7fa624006b30 comp rx=0 tx=0).stop 2024-09-10T21:36:10.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634159720 msgr2=0x7fa6341f8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:10.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634159720 0x7fa6341f8500 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fa62c01cd20 tx=0x7fa62c01ca00 comp rx=0 tx=0).stop 2024-09-10T21:36:10.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 shutdown_connections 2024-09-10T21:36:10.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fa6140725e0 0x7fa614074aa0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa634159720 0x7fa6341f8500 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fa6341592a0 0x7fa6341f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 --2- 172.21.15.97:0/693026179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa618004d50 0x7fa634131ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:10.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 >> 172.21.15.97:0/693026179 conn(0x7fa6340b1780 msgr2=0x7fa6340b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:10.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 shutdown_connections 2024-09-10T21:36:10.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:10.957+0000 7fa639a00640 1 -- 172.21.15.97:0/693026179 wait complete. 2024-09-10T21:36:10.964 INFO:tasks.ceph.mds.a.smithi097.stderr: -684> 2024-09-10T21:36:04.517+0000 7fe000400640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: Decoder at 'void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=114 minimal_decoder=104: Malformed input [buffer:3] 2024-09-10T21:36:10.964 INFO:tasks.ceph.mds.a.smithi097.stderr: -514> 2024-09-10T21:36:05.649+0000 7fe005400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:36:10.964 INFO:tasks.ceph.mds.a.smithi097.stderr: -431> 2024-09-10T21:36:05.825+0000 7fe000400640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-09-10T21:36:10.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:36:11.004 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-09-10T21:36:11.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.125+0000 7f4efea00640 1 -- 172.21.15.97:0/2933052993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 msgr2=0x7f4edc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:11.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.125+0000 7f4efea00640 1 --2- 172.21.15.97:0/2933052993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 0x7f4edc003e30 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800c310 tx=0x7f4ee801cf60 comp rx=0 tx=0).stop 2024-09-10T21:36:11.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.125+0000 7f4efca00640 1 -- 172.21.15.97:0/2933052993 reap_dead start 2024-09-10T21:36:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.125+0000 7f4efea00640 1 -- 172.21.15.97:0/2933052993 shutdown_connections 2024-09-10T21:36:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.125+0000 7f4efea00640 1 -- 172.21.15.97:0/2933052993 >> 172.21.15.97:0/2933052993 conn(0x7f4ef80d1cd0 msgr2=0x7f4ef80d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.125+0000 7f4efea00640 1 -- 172.21.15.97:0/2933052993 shutdown_connections 2024-09-10T21:36:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.125+0000 7f4efea00640 1 -- 172.21.15.97:0/2933052993 wait complete. 2024-09-10T21:36:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 Processor -- start 2024-09-10T21:36:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 -- start start 2024-09-10T21:36:11.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 0x7f4ef814eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ef814ae90 0x7f4ef806dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ef815a740 0x7f4ef815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8166870 con 0x7f4edc003a10 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81666f0 con 0x7f4ef814ae90 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef81669f0 con 0x7f4ef815a740 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 0x7f4ef814eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 0x7f4ef814eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53008/0 (socket says 172.21.15.97:53008) 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 -- 172.21.15.97:0/3295815845 learned_addr learned my addr 172.21.15.97:0/3295815845 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efca00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ef815a740 0x7f4ef815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7400640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ef814ae90 0x7f4ef806dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:11.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 -- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ef815a740 msgr2=0x7f4ef815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ef815a740 0x7f4ef815abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 -- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ef814ae90 msgr2=0x7f4ef806dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ef814ae90 0x7f4ef806dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 -- 172.21.15.97:0/3295815845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef80b63a0 con 0x7f4edc003a10 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4ef7e00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 0x7f4ef814eb90 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8098910 tx=0x7f4ee8006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4eefe00640 1 -- 172.21.15.97:0/3295815845 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80013e0 con 0x7f4edc003a10 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4eefe00640 1 -- 172.21.15.97:0/3295815845 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ee8001580 con 0x7f4edc003a10 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef806e4e0 con 0x7f4edc003a10 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4eefe00640 1 -- 172.21.15.97:0/3295815845 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8006330 con 0x7f4edc003a10 2024-09-10T21:36:11.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef806eaa0 con 0x7f4edc003a10 2024-09-10T21:36:11.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.129+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef80b0490 con 0x7f4edc003a10 2024-09-10T21:36:11.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.133+0000 7f4eefe00640 1 -- 172.21.15.97:0/3295815845 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4ee80c1030 con 0x7f4edc003a10 2024-09-10T21:36:11.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.133+0000 7f4eefe00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ed80725e0 0x7f4ed8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:11.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.133+0000 7f4eefe00640 1 -- 172.21.15.97:0/3295815845 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6389+0+0 (secure 0 0 0) 0x7f4ee812c920 con 0x7f4edc003a10 2024-09-10T21:36:11.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.133+0000 7f4ef7400640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ed80725e0 0x7f4ed8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:11.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.133+0000 7f4ef7400640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ed80725e0 0x7f4ed8074aa0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80cbb10 tx=0x7f4ee40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:11.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.137+0000 7f4eefe00640 1 -- 172.21.15.97:0/3295815845 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ee8016d10 con 0x7f4edc003a10 2024-09-10T21:36:11.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.337+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4ef80cfd60 con 0x7f4edc003a10 2024-09-10T21:36:11.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4eefe00640 1 -- 172.21.15.97:0/3295815845 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v352) ==== 104+0+0 (secure 0 0 0) 0x7f4ee80f7d50 con 0x7f4edc003a10 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ed80725e0 msgr2=0x7f4ed8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ed80725e0 0x7f4ed8074aa0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80cbb10 tx=0x7f4ee40091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 msgr2=0x7f4ef814eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 0x7f4ef814eb90 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8098910 tx=0x7f4ee8006d80 comp rx=0 tx=0).stop 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 shutdown_connections 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f4ed80725e0 0x7f4ed8074aa0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4ef815a740 0x7f4ef815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ef814ae90 0x7f4ef806dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 --2- 172.21.15.97:0/3295815845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4edc003a10 0x7f4ef814eb90 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 >> 172.21.15.97:0/3295815845 conn(0x7f4ef80d1cd0 msgr2=0x7f4ef815bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:11.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 shutdown_connections 2024-09-10T21:36:11.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:11.957+0000 7f4efea00640 1 -- 172.21.15.97:0/3295815845 wait complete. 2024-09-10T21:36:11.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 -- 172.21.15.97:0/1570003850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19481511f0 msgr2=0x7f19480b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 --2- 172.21.15.97:0/1570003850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19481511f0 0x7f19480b37b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f193800a950 tx=0x7f1938020360 comp rx=0 tx=0).stop 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 -- 172.21.15.97:0/1570003850 shutdown_connections 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 --2- 172.21.15.97:0/1570003850 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19480b3cf0 0x7f194806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 --2- 172.21.15.97:0/1570003850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19481511f0 0x7f19480b37b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 -- 172.21.15.97:0/1570003850 >> 172.21.15.97:0/1570003850 conn(0x7f19480b1780 msgr2=0x7f19480b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 -- 172.21.15.97:0/1570003850 shutdown_connections 2024-09-10T21:36:12.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.169+0000 7f194d800640 1 -- 172.21.15.97:0/1570003850 wait complete. 2024-09-10T21:36:12.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 Processor -- start 2024-09-10T21:36:12.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 -- start start 2024-09-10T21:36:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194806f180 0x7f194814f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19480b3cf0 0x7f19481e4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19481511f0 0x7f19481e3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f194806e740 con 0x7f194806f180 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f194806e5c0 con 0x7f19480b3cf0 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f194806e8c0 con 0x7f19481511f0 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194806f180 0x7f194814f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194806f180 0x7f194814f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53020/0 (socket says 172.21.15.97:53020) 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 -- 172.21.15.97:0/1320775625 learned_addr learned my addr 172.21.15.97:0/1320775625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946000640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19480b3cf0 0x7f19481e4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 -- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19481511f0 msgr2=0x7f19481e3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1947400640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19481511f0 0x7f19481e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19481511f0 0x7f19481e3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 -- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19480b3cf0 msgr2=0x7f19481e4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19480b3cf0 0x7f19481e4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 -- 172.21.15.97:0/1320775625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194814fed0 con 0x7f194806f180 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946000640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19480b3cf0 0x7f19481e4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1947400640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19481511f0 0x7f19481e3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f1946a00640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194806f180 0x7f194814f990 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f193800a8a0 tx=0x7f1938004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f193ea00640 1 -- 172.21.15.97:0/1320775625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19380be020 con 0x7f194806f180 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f193ea00640 1 -- 172.21.15.97:0/1320775625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19380b6040 con 0x7f194806f180 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19481e2b60 con 0x7f194806f180 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f193ea00640 1 -- 172.21.15.97:0/1320775625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19380b5070 con 0x7f194806f180 2024-09-10T21:36:12.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.173+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19481e5060 con 0x7f194806f180 2024-09-10T21:36:12.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.177+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f194806ec40 con 0x7f194806f180 2024-09-10T21:36:12.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.177+0000 7f193ea00640 1 -- 172.21.15.97:0/1320775625 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f193800b040 con 0x7f194806f180 2024-09-10T21:36:12.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.177+0000 7f193ea00640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19280725e0 0x7f1928074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.177+0000 7f193ea00640 1 -- 172.21.15.97:0/1320775625 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6389+0+0 (secure 0 0 0) 0x7f1938130c80 con 0x7f194806f180 2024-09-10T21:36:12.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.177+0000 7f1946000640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19280725e0 0x7f1928074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:12.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.177+0000 7f1946000640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19280725e0 0x7f1928074aa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f194806ad60 tx=0x7f19340091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:12.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.181+0000 7f193ea00640 1 -- 172.21.15.97:0/1320775625 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1938019ec0 con 0x7f194806f180 2024-09-10T21:36:12.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.345+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1948154680 con 0x7f194806f180 2024-09-10T21:36:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.965+0000 7f193ea00640 1 -- 172.21.15.97:0/1320775625 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) ==== 181+0+0 (secure 0 0 0) 0x7f19380fc020 con 0x7f194806f180 2024-09-10T21:36:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:36:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19280725e0 msgr2=0x7f1928074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19280725e0 0x7f1928074aa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f194806ad60 tx=0x7f19340091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194806f180 msgr2=0x7f194814f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194806f180 0x7f194814f990 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f193800a8a0 tx=0x7f1938004620 comp rx=0 tx=0).stop 2024-09-10T21:36:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 shutdown_connections 2024-09-10T21:36:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f19280725e0 0x7f1928074aa0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f19481511f0 0x7f19481e3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19480b3cf0 0x7f19481e4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 --2- 172.21.15.97:0/1320775625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f194806f180 0x7f194814f990 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 >> 172.21.15.97:0/1320775625 conn(0x7f19480b1780 msgr2=0x7f194815b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 shutdown_connections 2024-09-10T21:36:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:12.969+0000 7f194d800640 1 -- 172.21.15.97:0/1320775625 wait complete. 2024-09-10T21:36:12.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:36:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- 172.21.15.97:0/3372177550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 msgr2=0x7fe8900b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 --2- 172.21.15.97:0/3372177550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8900b1970 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fe8900b4860 tx=0x7fe88801eff0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- 172.21.15.97:0/3372177550 shutdown_connections 2024-09-10T21:36:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 --2- 172.21.15.97:0/3372177550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8900b1970 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 --2- 172.21.15.97:0/3372177550 >> v2:172.21.15.194:3300/0 conn(0x7fe8900cec10 0x7fe8900cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- 172.21.15.97:0/3372177550 >> 172.21.15.97:0/3372177550 conn(0x7fe8900b37c0 msgr2=0x7fe8900b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:13.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- 172.21.15.97:0/3372177550 shutdown_connections 2024-09-10T21:36:13.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- 172.21.15.97:0/3372177550 wait complete. 2024-09-10T21:36:13.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 Processor -- start 2024-09-10T21:36:13.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- start start 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8901f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8900ce830 0x7fe89015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8900cec10 0x7fe8900cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900ce1a0 con 0x7fe8900b1560 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe8900cde60 con 0x7fe8900cec10 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe896800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe8900ce320 con 0x7fe8900ce830 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8901f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8901f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53028/0 (socket says 172.21.15.97:53028) 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 -- 172.21.15.97:0/2457121774 learned_addr learned my addr 172.21.15.97:0/2457121774 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88fe00640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8900cec10 0x7fe8900cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88ea00640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8900ce830 0x7fe89015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 -- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8900ce830 msgr2=0x7fe89015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8900ce830 0x7fe89015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 -- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8900cec10 msgr2=0x7fe8900cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8900cec10 0x7fe8900cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 -- 172.21.15.97:0/2457121774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8900c92b0 con 0x7fe8900b1560 2024-09-10T21:36:13.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.149+0000 7fe88f400640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8901f5200 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fe88000cbd0 tx=0x7fe88000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe887400640 1 -- 172.21.15.97:0/2457121774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880016020 con 0x7fe8900b1560 2024-09-10T21:36:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe887400640 1 -- 172.21.15.97:0/2457121774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe88000aa50 con 0x7fe8900b1560 2024-09-10T21:36:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe887400640 1 -- 172.21.15.97:0/2457121774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88000ae60 con 0x7fe8900b1560 2024-09-10T21:36:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8901f4720 con 0x7fe8900b1560 2024-09-10T21:36:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8900c9590 con 0x7fe8900b1560 2024-09-10T21:36:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8900cffc0 con 0x7fe8900b1560 2024-09-10T21:36:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe887400640 1 -- 172.21.15.97:0/2457121774 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe880012420 con 0x7fe8900b1560 2024-09-10T21:36:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe887400640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8700725e0 0x7fe870074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe887400640 1 -- 172.21.15.97:0/2457121774 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5951+0+0 (secure 0 0 0) 0x7fe880093820 con 0x7fe8900b1560 2024-09-10T21:36:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.153+0000 7fe88ea00640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8700725e0 0x7fe870074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.157+0000 7fe88ea00640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8700725e0 0x7fe870074aa0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c005f20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.157+0000 7fe887400640 1 -- 172.21.15.97:0/2457121774 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe880020020 con 0x7fe8900b1560 2024-09-10T21:36:13.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.325+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe890070af0 con 0x7fe8900b1560 2024-09-10T21:36:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.969+0000 7fe887400640 1 -- 172.21.15.97:0/2457121774 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) ==== 169+0+0 (secure 0 0 0) 0x7fe88005ee00 con 0x7fe8900b1560 2024-09-10T21:36:13.969 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8700725e0 msgr2=0x7fe870074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8700725e0 0x7fe870074aa0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe87c005fd0 tx=0x7fe87c005f20 comp rx=0 tx=0).stop 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 msgr2=0x7fe8901f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8901f5200 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fe88000cbd0 tx=0x7fe88000ccc0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 shutdown_connections 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe8700725e0 0x7fe870074aa0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8900cec10 0x7fe8900cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe8900ce830 0x7fe89015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 --2- 172.21.15.97:0/2457121774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8900b1560 0x7fe8901f5200 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 >> 172.21.15.97:0/2457121774 conn(0x7fe8900b37c0 msgr2=0x7fe89006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:13.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 shutdown_connections 2024-09-10T21:36:13.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:13.973+0000 7fe896800640 1 -- 172.21.15.97:0/2457121774 wait complete. 2024-09-10T21:36:13.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 -- 172.21.15.97:0/310814410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 msgr2=0x7f920c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 --2- 172.21.15.97:0/310814410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0cc6d0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 -- 172.21.15.97:0/310814410 shutdown_connections 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 --2- 172.21.15.97:0/310814410 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f920c0b0d50 0x7f920c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 --2- 172.21.15.97:0/310814410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0cc6d0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 -- 172.21.15.97:0/310814410 >> 172.21.15.97:0/310814410 conn(0x7f920c0d1c30 msgr2=0x7f920c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 -- 172.21.15.97:0/310814410 shutdown_connections 2024-09-10T21:36:14.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.145+0000 7f9211400640 1 -- 172.21.15.97:0/310814410 wait complete. 2024-09-10T21:36:14.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 Processor -- start 2024-09-10T21:36:14.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 -- start start 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f920c0b0d50 0x7f920c158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f920c0ce640 0x7f920c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f920c168e90 con 0x7f920c0afdd0 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f920c168d10 con 0x7f920c0ce640 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f920c169010 con 0x7f920c0b0d50 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53056/0 (socket says 172.21.15.97:53056) 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 -- 172.21.15.97:0/1787639806 learned_addr learned my addr 172.21.15.97:0/1787639806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9209e00640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f920c0b0d50 0x7f920c158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920b200640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f920c0ce640 0x7f920c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 -- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f920c0b0d50 msgr2=0x7f920c158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f920c0b0d50 0x7f920c158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 -- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f920c0ce640 msgr2=0x7f920c159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f920c0ce640 0x7f920c159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 -- 172.21.15.97:0/1787639806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c0c5800 con 0x7f920c0afdd0 2024-09-10T21:36:14.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9209e00640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f920c0b0d50 0x7f920c158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:14.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f920a800640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0c8250 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c3c0 tx=0x7f91fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:14.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9202a00640 1 -- 172.21.15.97:0/1787639806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc017070 con 0x7f920c0afdd0 2024-09-10T21:36:14.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9202a00640 1 -- 172.21.15.97:0/1787639806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91fc0053c0 con 0x7f920c0afdd0 2024-09-10T21:36:14.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9202a00640 1 -- 172.21.15.97:0/1787639806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc003860 con 0x7f920c0afdd0 2024-09-10T21:36:14.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c15e960 con 0x7f920c0afdd0 2024-09-10T21:36:14.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.149+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c0c5d80 con 0x7f920c0afdd0 2024-09-10T21:36:14.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.153+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920c0c87f0 con 0x7f920c0afdd0 2024-09-10T21:36:14.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.153+0000 7f9202a00640 1 -- 172.21.15.97:0/1787639806 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f91fc003a00 con 0x7f920c0afdd0 2024-09-10T21:36:14.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.153+0000 7f9202a00640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f91ec0725e0 0x7f91ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.153+0000 7f9202a00640 1 -- 172.21.15.97:0/1787639806 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f91fc092720 con 0x7f920c0afdd0 2024-09-10T21:36:14.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.153+0000 7f9209e00640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f91ec0725e0 0x7f91ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.157+0000 7f9209e00640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f91ec0725e0 0x7f91ec074aa0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f91f80af000 tx=0x7f91f801b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:14.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.157+0000 7f9202a00640 1 -- 172.21.15.97:0/1787639806 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91fc05df20 con 0x7f920c0afdd0 2024-09-10T21:36:14.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.325+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f920c0cea20 con 0x7f920c0afdd0 2024-09-10T21:36:14.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.329+0000 7f9202a00640 1 -- 172.21.15.97:0/1787639806 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v46) ==== 71+0+0 (secure 0 0 0) 0x7f91fc05d820 con 0x7f920c0afdd0 2024-09-10T21:36:14.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f91ec0725e0 msgr2=0x7f91ec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f91ec0725e0 0x7f91ec074aa0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f91f80af000 tx=0x7f91f801b120 comp rx=0 tx=0).stop 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 msgr2=0x7f920c0c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0c8250 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c3c0 tx=0x7f91fc00c880 comp rx=0 tx=0).stop 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 shutdown_connections 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f91ec0725e0 0x7f91ec074aa0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f920c0ce640 0x7f920c159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f920c0b0d50 0x7f920c158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 --2- 172.21.15.97:0/1787639806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f920c0afdd0 0x7f920c0c8250 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 >> 172.21.15.97:0/1787639806 conn(0x7f920c0d1c30 msgr2=0x7f920c06e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 shutdown_connections 2024-09-10T21:36:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.333+0000 7f9211400640 1 -- 172.21.15.97:0/1787639806 wait complete. 2024-09-10T21:36:14.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-10T21:36:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- 172.21.15.97:0/3969359307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015f150 msgr2=0x7f80200b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 --2- 172.21.15.97:0/3969359307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015f150 0x7f80200b4380 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f8010009840 tx=0x7f801001c790 comp rx=0 tx=0).stop 2024-09-10T21:36:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- 172.21.15.97:0/3969359307 shutdown_connections 2024-09-10T21:36:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 --2- 172.21.15.97:0/3969359307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f802015f150 0x7f80200b4380 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- 172.21.15.97:0/3969359307 >> 172.21.15.97:0/3969359307 conn(0x7f80200b28c0 msgr2=0x7f80200b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- 172.21.15.97:0/3969359307 shutdown_connections 2024-09-10T21:36:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- 172.21.15.97:0/3969359307 wait complete. 2024-09-10T21:36:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 Processor -- start 2024-09-10T21:36:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- start start 2024-09-10T21:36:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f802015e750 0x7f80201fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f802015f150 0x7f80201fef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8020156a60 0x7f802015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8020168f40 con 0x7f8020156a60 2024-09-10T21:36:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8020168dc0 con 0x7f802015e750 2024-09-10T21:36:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f8026800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f80201690c0 con 0x7f802015f150 2024-09-10T21:36:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.493+0000 7f801fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8020156a60 0x7f802015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8020156a60 0x7f802015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53062/0 (socket says 172.21.15.97:53062) 2024-09-10T21:36:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 -- 172.21.15.97:0/1094238214 learned_addr learned my addr 172.21.15.97:0/1094238214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801f400640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f802015e750 0x7f80201fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8016a00640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f802015f150 0x7f80201fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 -- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f802015f150 msgr2=0x7f80201fef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f802015f150 0x7f80201fef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 -- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f802015e750 msgr2=0x7f80201fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f802015e750 0x7f80201fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 -- 172.21.15.97:0/1094238214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80201ff450 con 0x7f8020156a60 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f801fe00640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8020156a60 0x7f802015d570 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f801800c440 tx=0x7f801800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8017e00640 1 -- 172.21.15.97:0/1094238214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018017070 con 0x7f8020156a60 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8017e00640 1 -- 172.21.15.97:0/1094238214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8018003600 con 0x7f8020156a60 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8017e00640 1 -- 172.21.15.97:0/1094238214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018003a50 con 0x7f8020156a60 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8020156f20 con 0x7f8020156a60 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201574e0 con 0x7f8020156a60 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80200ce7b0 con 0x7f8020156a60 2024-09-10T21:36:14.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.497+0000 7f8017e00640 1 -- 172.21.15.97:0/1094238214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8018003bf0 con 0x7f8020156a60 2024-09-10T21:36:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.501+0000 7f8017e00640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ffc0725e0 0x7f7ffc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.501+0000 7f8017e00640 1 -- 172.21.15.97:0/1094238214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f80180921a0 con 0x7f8020156a60 2024-09-10T21:36:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.501+0000 7f801f400640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ffc0725e0 0x7f7ffc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:14.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.501+0000 7f801f400640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ffc0725e0 0x7f7ffc074aa0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f80100af000 tx=0x7f801001afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.505+0000 7f8017e00640 1 -- 172.21.15.97:0/1094238214 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f801805d340 con 0x7f8020156a60 2024-09-10T21:36:14.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.729+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f80200b4360 con 0x7f8020156a60 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.977+0000 7f8017e00640 1 -- 172.21.15.97:0/1094238214 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v624) ==== 135+0+0 (secure 0 0 0) 0x7f80180660b0 con 0x7f8020156a60 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ffc0725e0 msgr2=0x7f7ffc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ffc0725e0 0x7f7ffc074aa0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f80100af000 tx=0x7f801001afd0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8020156a60 msgr2=0x7f802015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8020156a60 0x7f802015d570 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f801800c440 tx=0x7f801800c900 comp rx=0 tx=0).stop 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 shutdown_connections 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7ffc0725e0 0x7f7ffc074aa0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8020156a60 0x7f802015d570 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f802015f150 0x7f80201fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 --2- 172.21.15.97:0/1094238214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f802015e750 0x7f80201fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:14.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 >> 172.21.15.97:0/1094238214 conn(0x7f80200b28c0 msgr2=0x7f802006f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:14.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 shutdown_connections 2024-09-10T21:36:14.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:14.981+0000 7f8026800640 1 -- 172.21.15.97:0/1094238214 wait complete. 2024-09-10T21:36:15.000 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ok 2024-09-10T21:36:15.000 INFO:tasks.cephfs_test_runner: 2024-09-10T21:36:15.000 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T21:36:15.000 INFO:tasks.cephfs_test_runner:Ran 6 tests in 684.472s 2024-09-10T21:36:15.000 INFO:tasks.cephfs_test_runner: 2024-09-10T21:36:15.000 INFO:tasks.cephfs_test_runner:OK 2024-09-10T21:36:15.001 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T21:36:15.012 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T21:36:15.038 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T21:36:15.039 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:36:15.039 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:15.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:36:15.048 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-10T21:36:15.048 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:15.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2787 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:36:15.099 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:36:15.099 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:15.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:36:15.109 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-10T21:36:15.109 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:36:15.159 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3448 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:36:15.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:15.159 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:36:15.169 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:36:15.169 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:36:15.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:15.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:36:15.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:15.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:36:15.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:15.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-09-10T21:36:15.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:15.874 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-10T21:36:15.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:15.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-09-10T21:36:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-09-10T21:36:16.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:36:16.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:16.293 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-09-10T21:36:16.303 INFO:teuthology.orchestra.run.smithi194.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:36:16.303 INFO:teuthology.orchestra.run.smithi194.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:36:16.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:16.304 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:36:16.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:16.369 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:36:16.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:16.854 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link delete ceph-brx 2024-09-10T21:36:17.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:17.046 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-09-10T21:36:17.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:36:17.056 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link delete ceph-brx 2024-09-10T21:36:17.437 INFO:teuthology.orchestra.run.smithi194.stderr:Cannot find device "ceph-brx" 2024-09-10T21:36:17.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:36:17.439 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T21:36:17.447 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:36:17.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- 172.21.15.97:0/3638442584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580144690 msgr2=0x7f45800b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 --2- 172.21.15.97:0/3638442584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580144690 0x7f45800b4120 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f4580074ee0 tx=0x7f456801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- 172.21.15.97:0/3638442584 shutdown_connections 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 --2- 172.21.15.97:0/3638442584 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f45800b4660 0x7f458006fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 --2- 172.21.15.97:0/3638442584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580144690 0x7f45800b4120 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 --2- 172.21.15.97:0/3638442584 >> v2:172.21.15.194:3300/0 conn(0x7f4580143f90 0x7f45800b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- 172.21.15.97:0/3638442584 >> 172.21.15.97:0/3638442584 conn(0x7f45800b1780 msgr2=0x7f45800b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:17.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- 172.21.15.97:0/3638442584 shutdown_connections 2024-09-10T21:36:17.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- 172.21.15.97:0/3638442584 wait complete. 2024-09-10T21:36:17.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 Processor -- start 2024-09-10T21:36:17.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- start start 2024-09-10T21:36:17.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45800b4660 0x7f4580154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580143f90 0x7f458013c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4580144690 0x7f45800cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f458015ed60 con 0x7f4580143f90 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f458015ebe0 con 0x7f45800b4660 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.597+0000 7f4587600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f458015eee0 con 0x7f4580144690 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580143f90 0x7f458013c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580143f90 0x7f458013c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56406/0 (socket says 172.21.15.97:56406) 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 -- 172.21.15.97:0/2362306833 learned_addr learned my addr 172.21.15.97:0/2362306833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4584c00640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45800b4660 0x7f4580154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4585600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4580144690 0x7f45800cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 -- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4580144690 msgr2=0x7f45800cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4580144690 0x7f45800cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 -- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45800b4660 msgr2=0x7f4580154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45800b4660 0x7f4580154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 -- 172.21.15.97:0/2362306833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458013c870 con 0x7f4580143f90 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4577e00640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580143f90 0x7f458013c330 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f45680b0040 tx=0x7f45680a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f457d600640 1 -- 172.21.15.97:0/2362306833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680b2040 con 0x7f4580143f90 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f457d600640 1 -- 172.21.15.97:0/2362306833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4568004340 con 0x7f4580143f90 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f457d600640 1 -- 172.21.15.97:0/2362306833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568003860 con 0x7f4580143f90 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45801f6180 con 0x7f4580143f90 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45801f6740 con 0x7f4580143f90 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458015f4b0 con 0x7f4580143f90 2024-09-10T21:36:17.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.601+0000 7f457d600640 1 -- 172.21.15.97:0/2362306833 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4568003a00 con 0x7f4580143f90 2024-09-10T21:36:17.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.605+0000 7f457d600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f455c0725e0 0x7f455c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.605+0000 7f457d600640 1 -- 172.21.15.97:0/2362306833 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f456812d300 con 0x7f4580143f90 2024-09-10T21:36:17.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.605+0000 7f4584c00640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f455c0725e0 0x7f455c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.605+0000 7f4584c00640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f455c0725e0 0x7f455c074aa0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4570009990 tx=0x7f4570009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:17.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.609+0000 7f457d600640 1 -- 172.21.15.97:0/2362306833 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f458013c870 con 0x7f4580143f90 2024-09-10T21:36:17.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.769+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45801f5f30 con 0x7f455c0725e0 2024-09-10T21:36:17.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.769+0000 7f457d600640 1 -- 172.21.15.97:0/2362306833 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13425 (secure 0 0 0) 0x7f45801f5f30 con 0x7f455c0725e0 2024-09-10T21:36:17.770 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:17.770 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:36:17.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f455c0725e0 msgr2=0x7f455c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f455c0725e0 0x7f455c074aa0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4570009990 tx=0x7f4570009110 comp rx=0 tx=0).stop 2024-09-10T21:36:17.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580143f90 msgr2=0x7f458013c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580143f90 0x7f458013c330 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f45680b0040 tx=0x7f45680a5650 comp rx=0 tx=0).stop 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 shutdown_connections 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f455c0725e0 0x7f455c074aa0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f4580144690 0x7f45800cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580143f90 0x7f458013c330 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 --2- 172.21.15.97:0/2362306833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45800b4660 0x7f4580154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 >> 172.21.15.97:0/2362306833 conn(0x7f45800b1780 msgr2=0x7f45800cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 shutdown_connections 2024-09-10T21:36:17.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.773+0000 7f4587600640 1 -- 172.21.15.97:0/2362306833 wait complete. 2024-09-10T21:36:17.791 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-10T21:36:16.257494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282452,"kb_used_data":16320,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692268,"statfs":{"total":773094113280,"available":772804882432,"internally_reserved":0,"allocated":16711680,"data_stored":9040980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34150,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-98,"num_write_kb":-66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001728"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":361,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T21:36:11.265133+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:36:11.265133+0000","last_peered":"2024-09-10T21:36:11.265133+0000","last_clean":"2024-09-10T21:36:11.265133+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:36:11.265133+0000","last_undegraded":"2024-09-10T21:36:11.265133+0000","last_fullsized":"2024-09-10T21:36:11.265133+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1929216,"data_stored":972941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":1005318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33648,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338192,"statfs":{"total":96636764160,"available":96602308608,"internally_reserved":0,"allocated":2523136,"data_stored":1562797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41440,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330400,"statfs":{"total":96636764160,"available":96594329600,"internally_reserved":0,"allocated":1978368,"data_stored":1012441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1929216,"data_stored":972401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37812,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334028,"statfs":{"total":96636764160,"available":96598044672,"internally_reserved":0,"allocated":2527232,"data_stored":1563197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1925120,"data_stored":972871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":979014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:36:17.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:36:17.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.933+0000 7fbfbda00640 1 -- 172.21.15.97:0/449188627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 msgr2=0x7fbf9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.933+0000 7fbfbda00640 1 --2- 172.21.15.97:0/449188627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbf9c00b9d0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb001f6a0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- 172.21.15.97:0/449188627 shutdown_connections 2024-09-10T21:36:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 --2- 172.21.15.97:0/449188627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbf9c00b9d0 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- 172.21.15.97:0/449188627 >> 172.21.15.97:0/449188627 conn(0x7fbfb80d1c70 msgr2=0x7fbfb80d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- 172.21.15.97:0/449188627 shutdown_connections 2024-09-10T21:36:17.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- 172.21.15.97:0/449188627 wait complete. 2024-09-10T21:36:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 Processor -- start 2024-09-10T21:36:17.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- start start 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbfb8153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfb81540b0 0x7fbfb80b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbfb80b4fe0 0x7fbfb80b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb80ccb10 con 0x7fbf9c0052f0 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb80cc990 con 0x7fbfb81540b0 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfbda00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb80ccc90 con 0x7fbfb80b4fe0 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbfb8153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbfb8153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56436/0 (socket says 172.21.15.97:56436) 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 -- 172.21.15.97:0/3071399201 learned_addr learned my addr 172.21.15.97:0/3071399201 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6000640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfb81540b0 0x7fbfb80b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb7400640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbfb80b4fe0 0x7fbfb80b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 -- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbfb80b4fe0 msgr2=0x7fbfb80b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbfb80b4fe0 0x7fbfb80b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 -- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfb81540b0 msgr2=0x7fbfb80b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfb81540b0 0x7fbfb80b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 -- 172.21.15.97:0/3071399201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb815d110 con 0x7fbf9c0052f0 2024-09-10T21:36:17.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb7400640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbfb80b4fe0 0x7fbfb80b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:17.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.937+0000 7fbfb6a00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbfb8153840 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8002230 tx=0x7fbfa8006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:17.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfaea00640 1 -- 172.21.15.97:0/3071399201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8015070 con 0x7fbf9c0052f0 2024-09-10T21:36:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfaea00640 1 -- 172.21.15.97:0/3071399201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfa8011900 con 0x7fbf9c0052f0 2024-09-10T21:36:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb815d3f0 con 0x7fbf9c0052f0 2024-09-10T21:36:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfaea00640 1 -- 172.21.15.97:0/3071399201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8011d30 con 0x7fbf9c0052f0 2024-09-10T21:36:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb815d9b0 con 0x7fbf9c0052f0 2024-09-10T21:36:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb80ce4b0 con 0x7fbf9c0052f0 2024-09-10T21:36:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfaea00640 1 -- 172.21.15.97:0/3071399201 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbfa80103c0 con 0x7fbf9c0052f0 2024-09-10T21:36:17.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfaea00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf980725e0 0x7fbf98074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:17.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfaea00640 1 -- 172.21.15.97:0/3071399201 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fbfa8093f20 con 0x7fbf9c0052f0 2024-09-10T21:36:17.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.941+0000 7fbfb6000640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf980725e0 0x7fbf98074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:17.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.945+0000 7fbfb6000640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf980725e0 0x7fbf98074aa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbfb815f740 tx=0x7fbfa40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:17.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:17.945+0000 7fbfaea00640 1 -- 172.21.15.97:0/3071399201 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbfa805f730 con 0x7fbf9c0052f0 2024-09-10T21:36:18.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbfb806e9f0 con 0x7fbf980725e0 2024-09-10T21:36:18.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfaea00640 1 -- 172.21.15.97:0/3071399201 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13425 (secure 0 0 0) 0x7fbfb806e9f0 con 0x7fbf980725e0 2024-09-10T21:36:18.113 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:18.113 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf980725e0 msgr2=0x7fbf98074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf980725e0 0x7fbf98074aa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbfb815f740 tx=0x7fbfa40091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 msgr2=0x7fbfb8153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbfb8153840 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8002230 tx=0x7fbfa8006430 comp rx=0 tx=0).stop 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 shutdown_connections 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fbf980725e0 0x7fbf98074aa0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fbfb80b4fe0 0x7fbfb80b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfb81540b0 0x7fbfb80b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 --2- 172.21.15.97:0/3071399201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0052f0 0x7fbfb8153840 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 >> 172.21.15.97:0/3071399201 conn(0x7fbfb80d1c70 msgr2=0x7fbfb80b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:18.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 shutdown_connections 2024-09-10T21:36:18.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.113+0000 7fbfbda00640 1 -- 172.21.15.97:0/3071399201 wait complete. 2024-09-10T21:36:18.134 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-10T21:36:16.257494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282452,"kb_used_data":16320,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692268,"statfs":{"total":773094113280,"available":772804882432,"internally_reserved":0,"allocated":16711680,"data_stored":9040980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34150,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-98,"num_write_kb":-66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001728"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":361,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T21:36:11.265133+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:36:11.265133+0000","last_peered":"2024-09-10T21:36:11.265133+0000","last_clean":"2024-09-10T21:36:11.265133+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:36:11.265133+0000","last_undegraded":"2024-09-10T21:36:11.265133+0000","last_fullsized":"2024-09-10T21:36:11.265133+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1929216,"data_stored":972941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":1005318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33648,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338192,"statfs":{"total":96636764160,"available":96602308608,"internally_reserved":0,"allocated":2523136,"data_stored":1562797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41440,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330400,"statfs":{"total":96636764160,"available":96594329600,"internally_reserved":0,"allocated":1978368,"data_stored":1012441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509589,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1929216,"data_stored":972401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37812,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334028,"statfs":{"total":96636764160,"available":96598044672,"internally_reserved":0,"allocated":2527232,"data_stored":1563197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1925120,"data_stored":972871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":979014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:36:18.135 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:36:18.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:36:18.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 -- 172.21.15.97:0/1200736574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2900cf5f0 msgr2=0x7ff29006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 --2- 172.21.15.97:0/1200736574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2900cf5f0 0x7ff29006f090 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7ff288009cb0 tx=0x7ff28801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 -- 172.21.15.97:0/1200736574 shutdown_connections 2024-09-10T21:36:18.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 --2- 172.21.15.97:0/1200736574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2900cf5f0 0x7ff29006f090 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 -- 172.21.15.97:0/1200736574 >> 172.21.15.97:0/1200736574 conn(0x7ff2900b1780 msgr2=0x7ff2900b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:18.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 -- 172.21.15.97:0/1200736574 shutdown_connections 2024-09-10T21:36:18.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 -- 172.21.15.97:0/1200736574 wait complete. 2024-09-10T21:36:18.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 Processor -- start 2024-09-10T21:36:18.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.276+0000 7ff295600640 1 -- start start 2024-09-10T21:36:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff29006ec40 0x7ff290155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2900cf5f0 0x7ff290158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2901511f0 0x7ff2900aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff29006e800 con 0x7ff2901511f0 2024-09-10T21:36:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff29006e680 con 0x7ff29006ec40 2024-09-10T21:36:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff29006e980 con 0x7ff2900cf5f0 2024-09-10T21:36:18.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2901511f0 0x7ff2900aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2901511f0 0x7ff2900aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56464/0 (socket says 172.21.15.97:56464) 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 -- 172.21.15.97:0/2293904373 learned_addr learned my addr 172.21.15.97:0/2293904373 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28ea00640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff29006ec40 0x7ff290155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff286000640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2900cf5f0 0x7ff290158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 -- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2900cf5f0 msgr2=0x7ff290158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2900cf5f0 0x7ff290158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 -- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff29006ec40 msgr2=0x7ff290155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff29006ec40 0x7ff290155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 -- 172.21.15.97:0/2293904373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff290158940 con 0x7ff2901511f0 2024-09-10T21:36:18.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff28f400640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2901511f0 0x7ff2900aeeb0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7ff288009bd0 tx=0x7ff2880af570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:18.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff287400640 1 -- 172.21.15.97:0/2293904373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288004e60 con 0x7ff2901511f0 2024-09-10T21:36:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff287400640 1 -- 172.21.15.97:0/2293904373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2880be030 con 0x7ff2901511f0 2024-09-10T21:36:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff287400640 1 -- 172.21.15.97:0/2293904373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880afc30 con 0x7ff2901511f0 2024-09-10T21:36:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2900af3f0 con 0x7ff2901511f0 2024-09-10T21:36:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff290155fd0 con 0x7ff2901511f0 2024-09-10T21:36:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29006f090 con 0x7ff2901511f0 2024-09-10T21:36:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.280+0000 7ff287400640 1 -- 172.21.15.97:0/2293904373 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff288004000 con 0x7ff2901511f0 2024-09-10T21:36:18.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.284+0000 7ff287400640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff26c0725e0 0x7ff26c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.284+0000 7ff287400640 1 -- 172.21.15.97:0/2293904373 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7ff288131af0 con 0x7ff2901511f0 2024-09-10T21:36:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.284+0000 7ff28ea00640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff26c0725e0 0x7ff26c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.284+0000 7ff28ea00640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff26c0725e0 0x7ff26c074aa0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff280007fd0 tx=0x7ff280009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.288+0000 7ff287400640 1 -- 172.21.15.97:0/2293904373 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff290158940 con 0x7ff2901511f0 2024-09-10T21:36:18.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.448+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2900afa40 con 0x7ff26c0725e0 2024-09-10T21:36:18.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.448+0000 7ff287400640 1 -- 172.21.15.97:0/2293904373 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13492 (secure 0 0 0) 0x7ff2900afa40 con 0x7ff26c0725e0 2024-09-10T21:36:18.451 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:18.451 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:36:18.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff26c0725e0 msgr2=0x7ff26c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff26c0725e0 0x7ff26c074aa0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff280007fd0 tx=0x7ff280009040 comp rx=0 tx=0).stop 2024-09-10T21:36:18.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2901511f0 msgr2=0x7ff2900aeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2901511f0 0x7ff2900aeeb0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7ff288009bd0 tx=0x7ff2880af570 comp rx=0 tx=0).stop 2024-09-10T21:36:18.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 shutdown_connections 2024-09-10T21:36:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff26c0725e0 0x7ff26c074aa0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2901511f0 0x7ff2900aeeb0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff2900cf5f0 0x7ff290158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 --2- 172.21.15.97:0/2293904373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff29006ec40 0x7ff290155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 >> 172.21.15.97:0/2293904373 conn(0x7ff2900b1780 msgr2=0x7ff2900b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 shutdown_connections 2024-09-10T21:36:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.452+0000 7ff295600640 1 -- 172.21.15.97:0/2293904373 wait complete. 2024-09-10T21:36:18.470 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-09-10T21:36:18.257950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":286804,"kb_used_data":16492,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687916,"statfs":{"total":773094113280,"available":772800425984,"internally_reserved":0,"allocated":16887808,"data_stored":9101454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":276614707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-34150,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-98,"num_write_kb":-66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.296724"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":365,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T21:36:14.033587+0000","last_change":"2024-09-10T21:24:03.518311+0000","last_active":"2024-09-10T21:36:14.033587+0000","last_peered":"2024-09-10T21:36:14.033587+0000","last_clean":"2024-09-10T21:36:14.033587+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:36:14.033587+0000","last_undegraded":"2024-09-10T21:36:14.033587+0000","last_fullsized":"2024-09-10T21:36:14.033587+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:02.108145+0000","last_clean_scrub_stamp":"2024-09-10T21:24:02.108145+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:04.168504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28948,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342892,"statfs":{"total":96636764160,"available":96607121408,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509591,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33692,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338148,"statfs":{"total":96636764160,"available":96602263552,"internally_reserved":0,"allocated":2560000,"data_stored":1582040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41428,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330412,"statfs":{"total":96636764160,"available":96594341888,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42012,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329828,"statfs":{"total":96636764160,"available":96593743872,"internally_reserved":0,"allocated":2560000,"data_stored":1582040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":979014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:36:18.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- 172.21.15.97:0/276178034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 msgr2=0x7fdfd415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 --2- 172.21.15.97:0/276178034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415f6d0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4141580 tx=0x7fdfbc01c8a0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- 172.21.15.97:0/276178034 shutdown_connections 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 --2- 172.21.15.97:0/276178034 >> v2:172.21.15.194:3300/0 conn(0x7fdfd415fc10 0x7fdfd413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 --2- 172.21.15.97:0/276178034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415f6d0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- 172.21.15.97:0/276178034 >> 172.21.15.97:0/276178034 conn(0x7fdfd40b1780 msgr2=0x7fdfd40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- 172.21.15.97:0/276178034 shutdown_connections 2024-09-10T21:36:18.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- 172.21.15.97:0/276178034 wait complete. 2024-09-10T21:36:18.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 Processor -- start 2024-09-10T21:36:18.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- start start 2024-09-10T21:36:18.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfd40ce5f0 0x7fdfd41524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfd415fc10 0x7fdfd413ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4168a70 con 0x7fdfd4141d20 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4168bf0 con 0x7fdfd40ce5f0 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.612+0000 7fdfdb000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd4168d70 con 0x7fdfd415fc10 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56482/0 (socket says 172.21.15.97:56482) 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 -- 172.21.15.97:0/2753353332 learned_addr learned my addr 172.21.15.97:0/2753353332 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfd3e00640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfd40ce5f0 0x7fdfd41524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfd9000640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfd415fc10 0x7fdfd413ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 -- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfd415fc10 msgr2=0x7fdfd413ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfd415fc10 0x7fdfd413ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 -- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfd40ce5f0 msgr2=0x7fdfd41524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfd40ce5f0 0x7fdfd41524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 -- 172.21.15.97:0/2753353332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd415dca0 con 0x7fdfd4141d20 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfcb400640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415ea60 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0b0040 tx=0x7fdfbc0a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfd0c00640 1 -- 172.21.15.97:0/2753353332 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0b2040 con 0x7fdfd4141d20 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfd0c00640 1 -- 172.21.15.97:0/2753353332 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfbc004440 con 0x7fdfd4141d20 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfd0c00640 1 -- 172.21.15.97:0/2753353332 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0052d0 con 0x7fdfd4141d20 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd415df80 con 0x7fdfd4141d20 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd413d230 con 0x7fdfd4141d20 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd415f6d0 con 0x7fdfd4141d20 2024-09-10T21:36:18.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.616+0000 7fdfd0c00640 1 -- 172.21.15.97:0/2753353332 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdfbc005470 con 0x7fdfd4141d20 2024-09-10T21:36:18.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.620+0000 7fdfd0c00640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfb00725e0 0x7fdfb0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:18.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.620+0000 7fdfd0c00640 1 -- 172.21.15.97:0/2753353332 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fdfbc12cc10 con 0x7fdfd4141d20 2024-09-10T21:36:18.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.620+0000 7fdfd3e00640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfb00725e0 0x7fdfb0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:18.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.620+0000 7fdfd3e00640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfb00725e0 0x7fdfb0074aa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007fd0 tx=0x7fdfc4007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:18.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.624+0000 7fdfd0c00640 1 -- 172.21.15.97:0/2753353332 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdfbc016490 con 0x7fdfd4141d20 2024-09-10T21:36:18.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.788+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfd4141340 con 0x7fdfd4141d20 2024-09-10T21:36:18.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfd0c00640 1 -- 172.21.15.97:0/2753353332 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13471 (secure 0 0 0) 0x7fdfbc0f7db0 con 0x7fdfd4141d20 2024-09-10T21:36:18.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:18.792 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":152,"fsid":"d1b4b680-9a7d-4173-b750-f3c8b85e2f7b","created":"2024-09-10T21:23:46.767526+0000","modified":"2024-09-10T21:36:13.969469+0000","last_up_change":"2024-09-10T21:24:01.106177+0000","last_in_change":"2024-09-10T21:23:50.422719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:02.092708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c747c78-dec2-4402-8839-4c4b5f98748b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6825","nonce":2907498194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6827","nonce":2907498194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6831","nonce":2907498194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2907498194},{"type":"v1","addr":"172.21.15.97:6829","nonce":2907498194}]},"public_addr":"172.21.15.97:6825/2907498194","cluster_addr":"172.21.15.97:6827/2907498194","heartbeat_back_addr":"172.21.15.97:6831/2907498194","heartbeat_front_addr":"172.21.15.97:6829/2907498194","state":["exists","up"]},{"osd":1,"uuid":"e0d62fef-5f34-4f25-9a73-100f3fbefb45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6812","nonce":3744897448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6816","nonce":3744897448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6823","nonce":3744897448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3744897448},{"type":"v1","addr":"172.21.15.97:6820","nonce":3744897448}]},"public_addr":"172.21.15.97:6812/3744897448","cluster_addr":"172.21.15.97:6816/3744897448","heartbeat_back_addr":"172.21.15.97:6823/3744897448","heartbeat_front_addr":"172.21.15.97:6820/3744897448","state":["exists","up"]},{"osd":2,"uuid":"8a1e08dc-aee0-4208-9e1b-e4541d75f5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6810","nonce":3581819336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6813","nonce":3581819336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6821","nonce":3581819336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":3581819336},{"type":"v1","addr":"172.21.15.97:6817","nonce":3581819336}]},"public_addr":"172.21.15.97:6810/3581819336","cluster_addr":"172.21.15.97:6813/3581819336","heartbeat_back_addr":"172.21.15.97:6821/3581819336","heartbeat_front_addr":"172.21.15.97:6817/3581819336","state":["exists","up"]},{"osd":3,"uuid":"dc9ae9e1-4f76-400b-a131-78d535e430f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6801","nonce":1724813549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6803","nonce":1724813549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6807","nonce":1724813549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1724813549},{"type":"v1","addr":"172.21.15.97:6805","nonce":1724813549}]},"public_addr":"172.21.15.97:6801/1724813549","cluster_addr":"172.21.15.97:6803/1724813549","heartbeat_back_addr":"172.21.15.97:6807/1724813549","heartbeat_front_addr":"172.21.15.97:6805/1724813549","state":["exists","up"]},{"osd":4,"uuid":"4a9b1edd-edd8-4957-8cb6-1446b54e78f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6801","nonce":4073109273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6803","nonce":4073109273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6807","nonce":4073109273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4073109273},{"type":"v1","addr":"172.21.15.194:6805","nonce":4073109273}]},"public_addr":"172.21.15.194:6801/4073109273","cluster_addr":"172.21.15.194:6803/4073109273","heartbeat_back_addr":"172.21.15.194:6807/4073109273","heartbeat_front_addr":"172.21.15.194:6805/4073109273","state":["exists","up"]},{"osd":5,"uuid":"f888558a-4098-46da-baf8-900de83e509c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6819","nonce":3659294892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6821","nonce":3659294892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6828","nonce":3659294892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3659294892},{"type":"v1","addr":"172.21.15.194:6824","nonce":3659294892}]},"public_addr":"172.21.15.194:6819/3659294892","cluster_addr":"172.21.15.194:6821/3659294892","heartbeat_back_addr":"172.21.15.194:6828/3659294892","heartbeat_front_addr":"172.21.15.194:6824/3659294892","state":["exists","up"]},{"osd":6,"uuid":"7fa0195a-7dd5-4116-9704-58395df8e70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6811","nonce":1793675643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6813","nonce":1793675643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6817","nonce":1793675643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1793675643},{"type":"v1","addr":"172.21.15.194:6815","nonce":1793675643}]},"public_addr":"172.21.15.194:6811/1793675643","cluster_addr":"172.21.15.194:6813/1793675643","heartbeat_back_addr":"172.21.15.194:6817/1793675643","heartbeat_front_addr":"172.21.15.194:6815/1793675643","state":["exists","up"]},{"osd":7,"uuid":"360c2eb5-82b4-4c2b-a316-be76b4817f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6823","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6825","nonce":3380827268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6827","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6829","nonce":3380827268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6833","nonce":3380827268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3380827268},{"type":"v1","addr":"172.21.15.194:6831","nonce":3380827268}]},"public_addr":"172.21.15.194:6825/3380827268","cluster_addr":"172.21.15.194:6829/3380827268","heartbeat_back_addr":"172.21.15.194:6833/3380827268","heartbeat_front_addr":"172.21.15.194:6831/3380827268","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.659707+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.422889+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.670229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.332069+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.164724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.524358+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.395617+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:23:57.484947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/2844434294":"2024-09-11T21:36:10.954176+0000","172.21.15.97:6833/2844434294":"2024-09-11T21:36:10.954176+0000","172.21.15.194:6837/3945561983":"2024-09-11T21:35:57.522868+0000","172.21.15.194:6835/3945561983":"2024-09-11T21:35:57.522868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfb00725e0 msgr2=0x7fdfb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfb00725e0 0x7fdfb0074aa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007fd0 tx=0x7fdfc4007e40 comp rx=0 tx=0).stop 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 msgr2=0x7fdfd415ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415ea60 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0b0040 tx=0x7fdfbc0a5720 comp rx=0 tx=0).stop 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 shutdown_connections 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fdfb00725e0 0x7fdfb0074aa0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fdfd415fc10 0x7fdfd413ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4141d20 0x7fdfd415ea60 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 --2- 172.21.15.97:0/2753353332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfd40ce5f0 0x7fdfd41524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:18.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 >> 172.21.15.97:0/2753353332 conn(0x7fdfd40b1780 msgr2=0x7fdfd406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:18.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 shutdown_connections 2024-09-10T21:36:18.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:18.792+0000 7fdfdb000640 1 -- 172.21.15.97:0/2753353332 wait complete. 2024-09-10T21:36:19.814 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T21:36:19.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 -- 172.21.15.97:0/1018348289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c06fd50 msgr2=0x7f833c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 --2- 172.21.15.97:0/1018348289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c06fd50 0x7f833c070160 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f833c074ee0 tx=0x7f833401cba0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 -- 172.21.15.97:0/1018348289 shutdown_connections 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 --2- 172.21.15.97:0/1018348289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c06fd50 0x7f833c070160 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 --2- 172.21.15.97:0/1018348289 >> v2:172.21.15.194:3300/0 conn(0x7f833c143f90 0x7f833c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 -- 172.21.15.97:0/1018348289 >> 172.21.15.97:0/1018348289 conn(0x7f833c0b1780 msgr2=0x7f833c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 -- 172.21.15.97:0/1018348289 shutdown_connections 2024-09-10T21:36:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.968+0000 7f8341600640 1 -- 172.21.15.97:0/1018348289 wait complete. 2024-09-10T21:36:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 Processor -- start 2024-09-10T21:36:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 -- start start 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f833c06fd50 0x7f833c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f833c143f90 0x7f833c13a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c144690 0x7f833c159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f833c167cb0 con 0x7f833c144690 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f833c167b30 con 0x7f833c143f90 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f833c167e30 con 0x7f833c06fd50 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c144690 0x7f833c159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c144690 0x7f833c159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56508/0 (socket says 172.21.15.97:56508) 2024-09-10T21:36:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 -- 172.21.15.97:0/1465713390 learned_addr learned my addr 172.21.15.97:0/1465713390 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8332000640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f833c143f90 0x7f833c13a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833aa00640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f833c06fd50 0x7f833c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 -- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f833c06fd50 msgr2=0x7f833c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f833c06fd50 0x7f833c0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 -- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f833c143f90 msgr2=0x7f833c13a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f833c143f90 0x7f833c13a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 -- 172.21.15.97:0/1465713390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c1fb3b0 con 0x7f833c144690 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f833b400640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c144690 0x7f833c159f60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f833401b250 tx=0x7f83340a5aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83340bb070 con 0x7f833c144690 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8334004d40 con 0x7f833c144690 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83340b2040 con 0x7f833c144690 2024-09-10T21:36:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c1fb690 con 0x7f833c144690 2024-09-10T21:36:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.972+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c1fbc50 con 0x7f833c144690 2024-09-10T21:36:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f833c1fc020 con 0x7f833c144690 2024-09-10T21:36:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f83340b5030 con 0x7f833c144690 2024-09-10T21:36:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8333400640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8318072610 0x7f8318074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f833412d6d0 con 0x7f833c144690 2024-09-10T21:36:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8333400640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f8318078b50 0x7f831807af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83340ade50 con 0x7f8318078b50 2024-09-10T21:36:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f833412db60 con 0x7f833c144690 2024-09-10T21:36:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f833aa00640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8318072610 0x7f8318074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8332000640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f8318078b50 0x7f831807af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f8332000640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f8318078b50 0x7f831807af90 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.976+0000 7f833aa00640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8318072610 0x7f8318074ad0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f832c00abe0 tx=0x7f832c00a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.980+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== osd.0 v2:172.21.15.97:6824/2907498194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83340ade50 con 0x7f8318078b50 2024-09-10T21:36:19.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 --> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f833c070160 con 0x7f8318078b50 2024-09-10T21:36:19.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8333400640 1 -- 172.21.15.97:0/1465713390 <== osd.0 v2:172.21.15.97:6824/2907498194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f833c070160 con 0x7f8318078b50 2024-09-10T21:36:19.993 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:19.993 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:19.993 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f8318078b50 msgr2=0x7f831807af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:6824/2907498194,v1:172.21.15.97:6825/2907498194] conn(0x7f8318078b50 0x7f831807af90 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8318072610 msgr2=0x7f8318074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f8318072610 0x7f8318074ad0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f832c00abe0 tx=0x7f832c00a2d0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c144690 msgr2=0x7f833c159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:19.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 --2- 172.21.15.97:0/1465713390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833c144690 0x7f833c159f60 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f833401b250 tx=0x7f83340a5aa0 comp rx=0 tx=0).stop 2024-09-10T21:36:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f833b400640 1 -- 172.21.15.97:0/1465713390 reap_dead start 2024-09-10T21:36:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 shutdown_connections 2024-09-10T21:36:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 >> 172.21.15.97:0/1465713390 conn(0x7f833c0b1780 msgr2=0x7f833c164ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 shutdown_connections 2024-09-10T21:36:19.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:19.992+0000 7f8341600640 1 -- 172.21.15.97:0/1465713390 wait complete. 2024-09-10T21:36:20.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T21:36:20.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.152+0000 7f21f9600640 1 -- 172.21.15.97:0/3320631251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40b1560 msgr2=0x7f21f40b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.152+0000 7f21f9600640 1 --2- 172.21.15.97:0/3320631251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40b1560 0x7f21f40b19b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009840 tx=0x7f21ec01f010 comp rx=0 tx=0).stop 2024-09-10T21:36:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- 172.21.15.97:0/3320631251 shutdown_connections 2024-09-10T21:36:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 --2- 172.21.15.97:0/3320631251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40b1560 0x7f21f40b19b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 --2- 172.21.15.97:0/3320631251 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21f40cec10 0x7f21f40cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- 172.21.15.97:0/3320631251 >> 172.21.15.97:0/3320631251 conn(0x7f21f40b37c0 msgr2=0x7f21f40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- 172.21.15.97:0/3320631251 shutdown_connections 2024-09-10T21:36:20.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- 172.21.15.97:0/3320631251 wait complete. 2024-09-10T21:36:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 Processor -- start 2024-09-10T21:36:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- start start 2024-09-10T21:36:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f40b1560 0x7f21f41f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40ce830 0x7f21f40cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21f40cec10 0x7f21f40c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40ce1a0 con 0x7f21f40b1560 2024-09-10T21:36:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40cde60 con 0x7f21f40ce830 2024-09-10T21:36:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f9600640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f21f40ce320 con 0x7f21f40cec10 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f40b1560 0x7f21f41f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40ce830 0x7f21f40cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f40b1560 0x7f21f41f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56510/0 (socket says 172.21.15.97:56510) 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2a00640 1 -- 172.21.15.97:0/1699820524 learned_addr learned my addr 172.21.15.97:0/1699820524 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f3400640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21f40cec10 0x7f21f40c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2000640 1 -- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21f40cec10 msgr2=0x7f21f40c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2000640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21f40cec10 0x7f21f40c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2000640 1 -- 172.21.15.97:0/1699820524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f40b1560 msgr2=0x7f21f41f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2000640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f40b1560 0x7f21f41f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2000640 1 -- 172.21.15.97:0/1699820524 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40cb1d0 con 0x7f21f40ce830 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2a00640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f40b1560 0x7f21f41f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.156+0000 7f21f2000640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40ce830 0x7f21f40cabd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00e6c0 tx=0x7f21dc00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21eaa00640 1 -- 172.21.15.97:0/1699820524 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00cab0 con 0x7f21f40ce830 2024-09-10T21:36:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21eaa00640 1 -- 172.21.15.97:0/1699820524 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21dc00cd40 con 0x7f21f40ce830 2024-09-10T21:36:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f40cb4b0 con 0x7f21f40ce830 2024-09-10T21:36:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21eaa00640 1 -- 172.21.15.97:0/1699820524 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc010690 con 0x7f21f40ce830 2024-09-10T21:36:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f40cba70 con 0x7f21f40ce830 2024-09-10T21:36:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f40cf800 con 0x7f21f40ce830 2024-09-10T21:36:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21eaa00640 1 -- 172.21.15.97:0/1699820524 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f21dc029080 con 0x7f21f40ce830 2024-09-10T21:36:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21eaa00640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21d00725e0 0x7f21d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.160+0000 7f21eaa00640 1 -- 172.21.15.97:0/1699820524 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f21dc093fb0 con 0x7f21f40ce830 2024-09-10T21:36:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.164+0000 7f21f2a00640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21d00725e0 0x7f21d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.164+0000 7f21f2a00640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21d00725e0 0x7f21d0074aa0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f21e4002240 tx=0x7f21e4001b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:20.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.164+0000 7f21eaa00640 1 -- 172.21.15.97:0/1699820524 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f21dc05f140 con 0x7f21f40ce830 2024-09-10T21:36:20.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.336+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f21f415f9b0 con 0x7f21d00725e0 2024-09-10T21:36:20.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21eaa00640 1 -- 172.21.15.97:0/1699820524 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f21f415f9b0 con 0x7f21d00725e0 2024-09-10T21:36:20.340 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T21:36:20.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21d00725e0 msgr2=0x7f21d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21d00725e0 0x7f21d0074aa0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f21e4002240 tx=0x7f21e4001b60 comp rx=0 tx=0).stop 2024-09-10T21:36:20.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40ce830 msgr2=0x7f21f40cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40ce830 0x7f21f40cabd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00e6c0 tx=0x7f21dc00eb80 comp rx=0 tx=0).stop 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 shutdown_connections 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f21d00725e0 0x7f21d0074aa0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f21f40cec10 0x7f21f40c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f40ce830 0x7f21f40cabd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 --2- 172.21.15.97:0/1699820524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21f40b1560 0x7f21f41f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 >> 172.21.15.97:0/1699820524 conn(0x7f21f40b37c0 msgr2=0x7f21f40b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 shutdown_connections 2024-09-10T21:36:20.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.340+0000 7f21f9600640 1 -- 172.21.15.97:0/1699820524 wait complete. 2024-09-10T21:36:20.361 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T21:36:20.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.504+0000 7f78ea000640 1 -- 172.21.15.97:0/4129945262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 msgr2=0x7f78c800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.504+0000 7f78ea000640 1 --2- 172.21.15.97:0/4129945262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 0x7f78c800b9d0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009ee0 tx=0x7f78dc01d630 comp rx=0 tx=0).stop 2024-09-10T21:36:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- 172.21.15.97:0/4129945262 shutdown_connections 2024-09-10T21:36:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 --2- 172.21.15.97:0/4129945262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 0x7f78c800b9d0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- 172.21.15.97:0/4129945262 >> 172.21.15.97:0/4129945262 conn(0x7f78e40b1780 msgr2=0x7f78e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- 172.21.15.97:0/4129945262 shutdown_connections 2024-09-10T21:36:20.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- 172.21.15.97:0/4129945262 wait complete. 2024-09-10T21:36:20.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 Processor -- start 2024-09-10T21:36:20.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- start start 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 0x7f78e40ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78e40aedc0 0x7f78e40abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78e4140ef0 0x7f78e41f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78e40b2c00 con 0x7f78c80051d0 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f78e40b2a80 con 0x7f78e4140ef0 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78ea000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f78e40b2d80 con 0x7f78e40aedc0 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 0x7f78e40ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 0x7f78e40ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56528/0 (socket says 172.21.15.97:56528) 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 -- 172.21.15.97:0/2832401293 learned_addr learned my addr 172.21.15.97:0/2832401293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78daa00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78e40aedc0 0x7f78e40abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3e00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78e4140ef0 0x7f78e41f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 -- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78e40aedc0 msgr2=0x7f78e40abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f78e40aedc0 0x7f78e40abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 -- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78e4140ef0 msgr2=0x7f78e41f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78e4140ef0 0x7f78e41f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 -- 172.21.15.97:0/2832401293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e40af240 con 0x7f78c80051d0 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.508+0000 7f78e3400640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 0x7f78e40ad740 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f78d4004e30 tx=0x7f78d4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4015020 con 0x7f78c80051d0 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78d4011960 con 0x7f78c80051d0 2024-09-10T21:36:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e40ac1d0 con 0x7f78c80051d0 2024-09-10T21:36:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4011d90 con 0x7f78c80051d0 2024-09-10T21:36:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e40ae000 con 0x7f78c80051d0 2024-09-10T21:36:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78e40700d0 con 0x7f78c80051d0 2024-09-10T21:36:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f78d401a400 con 0x7f78c80051d0 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78c0072610 0x7f78c0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f78d4094470 con 0x7f78c80051d0 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f78c0078b50 0x7f78c007af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78c007b660 con 0x7f78c0078b50 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f78d4094860 con 0x7f78c80051d0 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78daa00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78c0072610 0x7f78c0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.512+0000 7f78e3e00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f78c0078b50 0x7f78c007af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.516+0000 7f78e3e00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f78c0078b50 0x7f78c007af90 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.516+0000 7f78daa00640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78c0072610 0x7f78c0074ad0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f78e4140940 tx=0x7f78cc0067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.516+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== osd.1 v2:172.21.15.97:6809/3744897448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78c007b660 con 0x7f78c0078b50 2024-09-10T21:36:20.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 --> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f78e40b1200 con 0x7f78c0078b50 2024-09-10T21:36:20.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78dbe00640 1 -- 172.21.15.97:0/2832401293 <== osd.1 v2:172.21.15.97:6809/3744897448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f78e40b1200 con 0x7f78c0078b50 2024-09-10T21:36:20.530 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:20.531 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f78c0078b50 msgr2=0x7f78c007af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:6809/3744897448,v1:172.21.15.97:6812/3744897448] conn(0x7f78c0078b50 0x7f78c007af90 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78c0072610 msgr2=0x7f78c0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f78c0072610 0x7f78c0074ad0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f78e4140940 tx=0x7f78cc0067b0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 msgr2=0x7f78e40ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 --2- 172.21.15.97:0/2832401293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78c80051d0 0x7f78e40ad740 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f78d4004e30 tx=0x7f78d4001640 comp rx=0 tx=0).stop 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78e3e00640 1 -- 172.21.15.97:0/2832401293 reap_dead start 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 shutdown_connections 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 >> 172.21.15.97:0/2832401293 conn(0x7f78e40b1780 msgr2=0x7f78e41416e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 shutdown_connections 2024-09-10T21:36:20.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.528+0000 7f78ea000640 1 -- 172.21.15.97:0/2832401293 wait complete. 2024-09-10T21:36:20.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T21:36:20.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 -- 172.21.15.97:0/3463543304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80ce830 msgr2=0x7f7de80cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 --2- 172.21.15.97:0/3463543304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80ce830 0x7f7de80cf800 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f7de80b4860 tx=0x7f7ddc01f580 comp rx=0 tx=0).stop 2024-09-10T21:36:20.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 -- 172.21.15.97:0/3463543304 shutdown_connections 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 --2- 172.21.15.97:0/3463543304 >> v2:172.21.15.194:3300/0 conn(0x7f7de80cec10 0x7f7de80cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 --2- 172.21.15.97:0/3463543304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80ce830 0x7f7de80cf800 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 -- 172.21.15.97:0/3463543304 >> 172.21.15.97:0/3463543304 conn(0x7f7de80b37c0 msgr2=0x7f7de80b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 -- 172.21.15.97:0/3463543304 shutdown_connections 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 -- 172.21.15.97:0/3463543304 wait complete. 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 Processor -- start 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.688+0000 7f7defe00640 1 -- start start 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7de80ce830 0x7f7de814afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7de80cec10 0x7f7de81f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80d0280 0x7f7de81f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7de80b4420 con 0x7f7de80d0280 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7de80cde60 con 0x7f7de80cec10 2024-09-10T21:36:20.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f7de80ce1a0 con 0x7f7de80ce830 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80d0280 0x7f7de81f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7ded400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7de80ce830 0x7f7de814afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80d0280 0x7f7de81f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56540/0 (socket says 172.21.15.97:56540) 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 -- 172.21.15.97:0/1238873456 learned_addr learned my addr 172.21.15.97:0/1238873456 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7deca00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7de80cec10 0x7f7de81f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 -- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7de80ce830 msgr2=0x7f7de814afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7de80ce830 0x7f7de814afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 -- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7de80cec10 msgr2=0x7f7de81f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7de80cec10 0x7f7de81f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 -- 172.21.15.97:0/1238873456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de80c9d00 con 0x7f7de80d0280 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7dede00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80d0280 0x7f7de81f6480 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f7de000c3c0 tx=0x7f7de000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7de5600640 1 -- 172.21.15.97:0/1238873456 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00052d0 con 0x7f7de80d0280 2024-09-10T21:36:20.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7de5600640 1 -- 172.21.15.97:0/1238873456 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7de0007020 con 0x7f7de80d0280 2024-09-10T21:36:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7de5600640 1 -- 172.21.15.97:0/1238873456 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0007470 con 0x7f7de80d0280 2024-09-10T21:36:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de80c9fa0 con 0x7f7de80d0280 2024-09-10T21:36:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de80ca560 con 0x7f7de80d0280 2024-09-10T21:36:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.692+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de80b26e0 con 0x7f7de80d0280 2024-09-10T21:36:20.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.696+0000 7f7de5600640 1 -- 172.21.15.97:0/1238873456 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7de00067e0 con 0x7f7de80d0280 2024-09-10T21:36:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.696+0000 7f7de5600640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7dc80725e0 0x7f7dc8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.696+0000 7f7de5600640 1 -- 172.21.15.97:0/1238873456 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f7de0092610 con 0x7f7de80d0280 2024-09-10T21:36:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.696+0000 7f7ded400640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7dc80725e0 0x7f7dc8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:20.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.696+0000 7f7ded400640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7dc80725e0 0x7f7dc8074aa0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0b3000 tx=0x7f7ddc01ddc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:20.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.700+0000 7f7de5600640 1 -- 172.21.15.97:0/1238873456 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7de005d720 con 0x7f7de80d0280 2024-09-10T21:36:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.864+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f7de8072410 con 0x7f7dc80725e0 2024-09-10T21:36:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7de5600640 1 -- 172.21.15.97:0/1238873456 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7de8072410 con 0x7f7dc80725e0 2024-09-10T21:36:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T21:36:20.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7dc80725e0 msgr2=0x7f7dc8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7dc80725e0 0x7f7dc8074aa0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0b3000 tx=0x7f7ddc01ddc0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80d0280 msgr2=0x7f7de81f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80d0280 0x7f7de81f6480 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f7de000c3c0 tx=0x7f7de000c7c0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 shutdown_connections 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f7dc80725e0 0x7f7dc8074aa0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7de80d0280 0x7f7de81f6480 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7de80cec10 0x7f7de81f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 --2- 172.21.15.97:0/1238873456 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f7de80ce830 0x7f7de814afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 >> 172.21.15.97:0/1238873456 conn(0x7f7de80b37c0 msgr2=0x7f7de80709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 shutdown_connections 2024-09-10T21:36:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:20.868+0000 7f7defe00640 1 -- 172.21.15.97:0/1238873456 wait complete. 2024-09-10T21:36:20.890 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T21:36:20.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- 172.21.15.97:0/970661110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8143f90 msgr2=0x7f54d80b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 --2- 172.21.15.97:0/970661110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8143f90 0x7f54d80b2a80 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f54d815e650 tx=0x7f54c801c800 comp rx=0 tx=0).stop 2024-09-10T21:36:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- 172.21.15.97:0/970661110 shutdown_connections 2024-09-10T21:36:21.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 --2- 172.21.15.97:0/970661110 >> v2:172.21.15.194:3300/0 conn(0x7f54d8144690 0x7f54d80b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 --2- 172.21.15.97:0/970661110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8143f90 0x7f54d80b2a80 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- 172.21.15.97:0/970661110 >> 172.21.15.97:0/970661110 conn(0x7f54d80b1780 msgr2=0x7f54d80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- 172.21.15.97:0/970661110 shutdown_connections 2024-09-10T21:36:21.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- 172.21.15.97:0/970661110 wait complete. 2024-09-10T21:36:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 Processor -- start 2024-09-10T21:36:21.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- start start 2024-09-10T21:36:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54d80b4660 0x7f54d8137e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54d8143f90 0x7f54d8138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8144690 0x7f54d806f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81679e0 con 0x7f54d8144690 2024-09-10T21:36:21.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8167860 con 0x7f54d80b4660 2024-09-10T21:36:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.036+0000 7f54de400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f54d8167b60 con 0x7f54d8143f90 2024-09-10T21:36:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8144690 0x7f54d806f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54cea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54d8143f90 0x7f54d8138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8144690 0x7f54d806f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56552/0 (socket says 172.21.15.97:56552) 2024-09-10T21:36:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 -- 172.21.15.97:0/2689783281 learned_addr learned my addr 172.21.15.97:0/2689783281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:21.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7400640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54d80b4660 0x7f54d8137e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 -- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54d8143f90 msgr2=0x7f54d8138900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f54d8143f90 0x7f54d8138900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 -- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54d80b4660 msgr2=0x7f54d8137e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54d80b4660 0x7f54d8137e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 -- 172.21.15.97:0/2689783281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d813c2b0 con 0x7f54d8144690 2024-09-10T21:36:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54d7e00640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8144690 0x7f54d806f3c0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f54d000c730 tx=0x7f54d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0003db0 con 0x7f54d8144690 2024-09-10T21:36:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54d000ce40 con 0x7f54d8144690 2024-09-10T21:36:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d81384d0 con 0x7f54d8144690 2024-09-10T21:36:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00052d0 con 0x7f54d8144690 2024-09-10T21:36:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d813ce00 con 0x7f54d8144690 2024-09-10T21:36:21.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54d80b2a80 con 0x7f54d8144690 2024-09-10T21:36:21.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.040+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f54d0005470 con 0x7f54d8144690 2024-09-10T21:36:21.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cfe00640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54b40725e0 0x7f54b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f54d0093960 con 0x7f54d8144690 2024-09-10T21:36:21.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cfe00640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f54b4078b20 0x7f54b407af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54b407b630 con 0x7f54b4078b20 2024-09-10T21:36:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f54d0093d10 con 0x7f54d8144690 2024-09-10T21:36:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cea00640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f54b4078b20 0x7f54b407af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54d7400640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54b40725e0 0x7f54b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cea00640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f54b4078b20 0x7f54b407af60 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54d7400640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54b40725e0 0x7f54b4074aa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f54c80af000 tx=0x7f54c801b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.044+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== osd.2 v2:172.21.15.97:6808/3581819336 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54b407b630 con 0x7f54b4078b20 2024-09-10T21:36:21.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 --> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54d80cc470 con 0x7f54b4078b20 2024-09-10T21:36:21.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54cfe00640 1 -- 172.21.15.97:0/2689783281 <== osd.2 v2:172.21.15.97:6808/3581819336 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f54d80cc470 con 0x7f54b4078b20 2024-09-10T21:36:21.058 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:21.058 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f54b4078b20 msgr2=0x7f54b407af60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:6808/3581819336,v1:172.21.15.97:6810/3581819336] conn(0x7f54b4078b20 0x7f54b407af60 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54b40725e0 msgr2=0x7f54b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f54b40725e0 0x7f54b4074aa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f54c80af000 tx=0x7f54c801b040 comp rx=0 tx=0).stop 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8144690 msgr2=0x7f54d806f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 --2- 172.21.15.97:0/2689783281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8144690 0x7f54d806f3c0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f54d000c730 tx=0x7f54d000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54d7e00640 1 -- 172.21.15.97:0/2689783281 reap_dead start 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 shutdown_connections 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 >> 172.21.15.97:0/2689783281 conn(0x7f54d80b1780 msgr2=0x7f54d80afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 shutdown_connections 2024-09-10T21:36:21.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.056+0000 7f54de400640 1 -- 172.21.15.97:0/2689783281 wait complete. 2024-09-10T21:36:21.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T21:36:21.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.216+0000 7fe2e9c00640 1 -- 172.21.15.97:0/3049457347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e4144690 msgr2=0x7fe2e40b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.216+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/3049457347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e4144690 0x7fe2e40b2a80 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009840 tx=0x7fe2d001c820 comp rx=0 tx=0).stop 2024-09-10T21:36:21.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.216+0000 7fe2e9c00640 1 -- 172.21.15.97:0/3049457347 shutdown_connections 2024-09-10T21:36:21.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.216+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/3049457347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e4144690 0x7fe2e40b2a80 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.216+0000 7fe2e9c00640 1 -- 172.21.15.97:0/3049457347 >> 172.21.15.97:0/3049457347 conn(0x7fe2e40b1780 msgr2=0x7fe2e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.216+0000 7fe2e9c00640 1 -- 172.21.15.97:0/3049457347 shutdown_connections 2024-09-10T21:36:21.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.216+0000 7fe2e9c00640 1 -- 172.21.15.97:0/3049457347 wait complete. 2024-09-10T21:36:21.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 Processor -- start 2024-09-10T21:36:21.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 -- start start 2024-09-10T21:36:21.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e406fd50 0x7fe2e41429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2e4143f90 0x7fe2e415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2e4144690 0x7fe2e413d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e406efd0 con 0x7fe2e406fd50 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e406ee50 con 0x7fe2e4144690 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e406f150 con 0x7fe2e4143f90 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e406fd50 0x7fe2e41429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e406fd50 0x7fe2e41429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56570/0 (socket says 172.21.15.97:56570) 2024-09-10T21:36:21.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 -- 172.21.15.97:0/4103216318 learned_addr learned my addr 172.21.15.97:0/4103216318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e3400640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2e4144690 0x7fe2e413d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2000640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2e4143f90 0x7fe2e415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 -- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2e4143f90 msgr2=0x7fe2e415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2e4143f90 0x7fe2e415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 -- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2e4144690 msgr2=0x7fe2e413d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2e4144690 0x7fe2e413d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 -- 172.21.15.97:0/4103216318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e4142f00 con 0x7fe2e406fd50 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e2a00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e406fd50 0x7fe2e41429c0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c7c0 tx=0x7fe2d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2daa00640 1 -- 172.21.15.97:0/4103216318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4016020 con 0x7fe2e406fd50 2024-09-10T21:36:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2daa00640 1 -- 172.21.15.97:0/4103216318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2d400ab40 con 0x7fe2e406fd50 2024-09-10T21:36:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e4132ae0 con 0x7fe2e406fd50 2024-09-10T21:36:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2daa00640 1 -- 172.21.15.97:0/4103216318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4012420 con 0x7fe2e406fd50 2024-09-10T21:36:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.220+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e4135410 con 0x7fe2e406fd50 2024-09-10T21:36:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.224+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e40b2a80 con 0x7fe2e406fd50 2024-09-10T21:36:21.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.224+0000 7fe2daa00640 1 -- 172.21.15.97:0/4103216318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe2d401b400 con 0x7fe2e406fd50 2024-09-10T21:36:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.224+0000 7fe2daa00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe2c40725e0 0x7fe2c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.224+0000 7fe2daa00640 1 -- 172.21.15.97:0/4103216318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fe2d40934c0 con 0x7fe2e406fd50 2024-09-10T21:36:21.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.224+0000 7fe2e2000640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe2c40725e0 0x7fe2c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.224+0000 7fe2e2000640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe2c40725e0 0x7fe2c4074aa0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00af000 tx=0x7fe2d001b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.228+0000 7fe2daa00640 1 -- 172.21.15.97:0/4103216318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe2d4020030 con 0x7fe2e406fd50 2024-09-10T21:36:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.392+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fe2e4073a10 con 0x7fe2c40725e0 2024-09-10T21:36:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2daa00640 1 -- 172.21.15.97:0/4103216318 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe2e4073a10 con 0x7fe2c40725e0 2024-09-10T21:36:21.396 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T21:36:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe2c40725e0 msgr2=0x7fe2c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe2c40725e0 0x7fe2c4074aa0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00af000 tx=0x7fe2d001b060 comp rx=0 tx=0).stop 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e406fd50 msgr2=0x7fe2e41429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e406fd50 0x7fe2e41429c0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c7c0 tx=0x7fe2d400cc80 comp rx=0 tx=0).stop 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 shutdown_connections 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fe2c40725e0 0x7fe2c4074aa0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2e4144690 0x7fe2e413d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fe2e4143f90 0x7fe2e415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 --2- 172.21.15.97:0/4103216318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2e406fd50 0x7fe2e41429c0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 >> 172.21.15.97:0/4103216318 conn(0x7fe2e40b1780 msgr2=0x7fe2e4143220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 shutdown_connections 2024-09-10T21:36:21.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.396+0000 7fe2e9c00640 1 -- 172.21.15.97:0/4103216318 wait complete. 2024-09-10T21:36:21.418 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T21:36:21.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:21.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- 172.21.15.97:0/605484606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 msgr2=0x7f33bc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 --2- 172.21.15.97:0/605484606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 0x7f33bc003e80 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c640 tx=0x7f33c801fa00 comp rx=0 tx=0).stop 2024-09-10T21:36:21.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dca00640 1 -- 172.21.15.97:0/605484606 reap_dead start 2024-09-10T21:36:21.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- 172.21.15.97:0/605484606 shutdown_connections 2024-09-10T21:36:21.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- 172.21.15.97:0/605484606 >> 172.21.15.97:0/605484606 conn(0x7f33d80b1780 msgr2=0x7f33d80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- 172.21.15.97:0/605484606 shutdown_connections 2024-09-10T21:36:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- 172.21.15.97:0/605484606 wait complete. 2024-09-10T21:36:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 Processor -- start 2024-09-10T21:36:21.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- start start 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 0x7f33d81f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33d81599a0 0x7f33d8147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33d814d700 0x7f33d8145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33d806f3f0 con 0x7f33bc003a60 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f33d806f270 con 0x7f33d814d700 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dea00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f33d806e3b0 con 0x7f33d81599a0 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 0x7f33d81f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 0x7f33d81f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56584/0 (socket says 172.21.15.97:56584) 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 -- 172.21.15.97:0/96974840 learned_addr learned my addr 172.21.15.97:0/96974840 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33cf400640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33d81599a0 0x7f33d8147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33dca00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33d814d700 0x7f33d8145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 -- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33d81599a0 msgr2=0x7f33d8147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f33d81599a0 0x7f33d8147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 -- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33d814d700 msgr2=0x7f33d8145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33d814d700 0x7f33d8145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 -- 172.21.15.97:0/96974840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d8159de0 con 0x7f33bc003a60 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.564+0000 7f33d7e00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 0x7f33d81f2f70 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006040 tx=0x7f33c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8001740 con 0x7f33bc003a60 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33c80018e0 con 0x7f33bc003a60 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8003be0 con 0x7f33bc003a60 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d815a080 con 0x7f33bc003a60 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d814b780 con 0x7f33bc003a60 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33d814be40 con 0x7f33bc003a60 2024-09-10T21:36:21.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f33c80a95d0 con 0x7f33bc003a60 2024-09-10T21:36:21.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33b4072610 0x7f33b4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f33c8130820 con 0x7f33bc003a60 2024-09-10T21:36:21.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f33b4078b50 0x7f33b407af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33c80acec0 con 0x7f33b4078b50 2024-09-10T21:36:21.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f33c8130bd0 con 0x7f33bc003a60 2024-09-10T21:36:21.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33cf400640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33b4072610 0x7f33b4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.568+0000 7f33dca00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f33b4078b50 0x7f33b407af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.572+0000 7f33cf400640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33b4072610 0x7f33b4074ad0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f33d8071010 tx=0x7f33c00091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.572+0000 7f33dca00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f33b4078b50 0x7f33b407af90 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.572+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== osd.3 v2:172.21.15.97:6800/1724813549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33c80acec0 con 0x7f33b4078b50 2024-09-10T21:36:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 --> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f33d806e940 con 0x7f33b4078b50 2024-09-10T21:36:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33d4c00640 1 -- 172.21.15.97:0/96974840 <== osd.3 v2:172.21.15.97:6800/1724813549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f33d806e940 con 0x7f33b4078b50 2024-09-10T21:36:21.586 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:21.586 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f33b4078b50 msgr2=0x7f33b407af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:6800/1724813549,v1:172.21.15.97:6801/1724813549] conn(0x7f33b4078b50 0x7f33b407af90 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33b4072610 msgr2=0x7f33b4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f33b4072610 0x7f33b4074ad0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f33d8071010 tx=0x7f33c00091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 msgr2=0x7f33d81f2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 --2- 172.21.15.97:0/96974840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33bc003a60 0x7f33d81f2f70 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006040 tx=0x7f33c800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dca00640 1 -- 172.21.15.97:0/96974840 reap_dead start 2024-09-10T21:36:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 shutdown_connections 2024-09-10T21:36:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 >> 172.21.15.97:0/96974840 conn(0x7f33d80b1780 msgr2=0x7f33d8147530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 shutdown_connections 2024-09-10T21:36:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.584+0000 7f33dea00640 1 -- 172.21.15.97:0/96974840 wait complete. 2024-09-10T21:36:21.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T21:36:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- 172.21.15.97:0/1035474617 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 msgr2=0x7f0b440cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 --2- 172.21.15.97:0/1035474617 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 0x7f0b440cf800 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009840 tx=0x7f0b3401f010 comp rx=0 tx=0).stop 2024-09-10T21:36:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- 172.21.15.97:0/1035474617 shutdown_connections 2024-09-10T21:36:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 --2- 172.21.15.97:0/1035474617 >> v2:172.21.15.194:3300/0 conn(0x7f0b440cec10 0x7f0b440cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 --2- 172.21.15.97:0/1035474617 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 0x7f0b440cf800 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- 172.21.15.97:0/1035474617 >> 172.21.15.97:0/1035474617 conn(0x7f0b440b37c0 msgr2=0x7f0b440b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- 172.21.15.97:0/1035474617 shutdown_connections 2024-09-10T21:36:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- 172.21.15.97:0/1035474617 wait complete. 2024-09-10T21:36:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 Processor -- start 2024-09-10T21:36:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- start start 2024-09-10T21:36:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 0x7f0b44157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b440cec10 0x7f0b440cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b440d0280 0x7f0b440cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b440b4480 con 0x7f0b440cec10 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0b440cde60 con 0x7f0b440d0280 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b4a000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0b440ce1a0 con 0x7f0b440ce830 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b3aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b440cec10 0x7f0b440cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b3aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b440cec10 0x7f0b440cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56592/0 (socket says 172.21.15.97:56592) 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b3aa00640 1 -- 172.21.15.97:0/149103448 learned_addr learned my addr 172.21.15.97:0/149103448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b43400640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 0x7f0b44157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.748+0000 7f0b43e00640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b440d0280 0x7f0b440cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3aa00640 1 -- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 msgr2=0x7f0b44157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3aa00640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 0x7f0b44157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3aa00640 1 -- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b440d0280 msgr2=0x7f0b440cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3aa00640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b440d0280 0x7f0b440cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3aa00640 1 -- 172.21.15.97:0/149103448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b441f5ae0 con 0x7f0b440cec10 2024-09-10T21:36:21.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3aa00640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b440cec10 0x7f0b440cb510 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00ea10 tx=0x7f0b2c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3be00640 1 -- 172.21.15.97:0/149103448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c00cd80 con 0x7f0b440cec10 2024-09-10T21:36:21.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3be00640 1 -- 172.21.15.97:0/149103448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b2c004020 con 0x7f0b440cec10 2024-09-10T21:36:21.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b441f5dc0 con 0x7f0b440cec10 2024-09-10T21:36:21.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3be00640 1 -- 172.21.15.97:0/149103448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c004470 con 0x7f0b440cec10 2024-09-10T21:36:21.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b441f6380 con 0x7f0b440cec10 2024-09-10T21:36:21.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b440b2780 con 0x7f0b440cec10 2024-09-10T21:36:21.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.752+0000 7f0b3be00640 1 -- 172.21.15.97:0/149103448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0b2c003600 con 0x7f0b440cec10 2024-09-10T21:36:21.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.756+0000 7f0b3be00640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b200725e0 0x7f0b20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:21.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.756+0000 7f0b3be00640 1 -- 172.21.15.97:0/149103448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f0b2c0939c0 con 0x7f0b440cec10 2024-09-10T21:36:21.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.756+0000 7f0b43400640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b200725e0 0x7f0b20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:21.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.756+0000 7f0b43400640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b200725e0 0x7f0b20074aa0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0b340b2000 tx=0x7f0b3401d850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:21.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.760+0000 7f0b3be00640 1 -- 172.21.15.97:0/149103448 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b2c05eb60 con 0x7f0b440cec10 2024-09-10T21:36:21.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.928+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f0b44071400 con 0x7f0b200725e0 2024-09-10T21:36:21.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.928+0000 7f0b3be00640 1 -- 172.21.15.97:0/149103448 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0b44071400 con 0x7f0b200725e0 2024-09-10T21:36:21.930 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T21:36:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b200725e0 msgr2=0x7f0b20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b200725e0 0x7f0b20074aa0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0b340b2000 tx=0x7f0b3401d850 comp rx=0 tx=0).stop 2024-09-10T21:36:21.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b440cec10 msgr2=0x7f0b440cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b440cec10 0x7f0b440cb510 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00ea10 tx=0x7f0b2c00eed0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 shutdown_connections 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0b200725e0 0x7f0b20074aa0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b440d0280 0x7f0b440cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b440cec10 0x7f0b440cb510 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 --2- 172.21.15.97:0/149103448 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0b440ce830 0x7f0b44157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 >> 172.21.15.97:0/149103448 conn(0x7f0b440b37c0 msgr2=0x7f0b440b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 shutdown_connections 2024-09-10T21:36:21.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:21.932+0000 7f0b4a000640 1 -- 172.21.15.97:0/149103448 wait complete. 2024-09-10T21:36:21.951 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T21:36:21.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:22.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 -- 172.21.15.97:0/2557185001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9edc004d50 msgr2=0x7f9edc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 --2- 172.21.15.97:0/2557185001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9edc004d50 0x7f9edc0051d0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009ee0 tx=0x7f9ef001d680 comp rx=0 tx=0).stop 2024-09-10T21:36:22.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9ef6800640 1 -- 172.21.15.97:0/2557185001 reap_dead start 2024-09-10T21:36:22.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 -- 172.21.15.97:0/2557185001 shutdown_connections 2024-09-10T21:36:22.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 -- 172.21.15.97:0/2557185001 >> 172.21.15.97:0/2557185001 conn(0x7f9ef80b1780 msgr2=0x7f9ef80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 -- 172.21.15.97:0/2557185001 shutdown_connections 2024-09-10T21:36:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 -- 172.21.15.97:0/2557185001 wait complete. 2024-09-10T21:36:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 Processor -- start 2024-09-10T21:36:22.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.096+0000 7f9efcc00640 1 -- start start 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9edc004d50 0x7f9ef8149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ef8149860 0x7f9ef80cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef814b470 0x7f9ef814b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cf700 con 0x7f9ef814b470 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cf580 con 0x7f9edc004d50 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef8152500 con 0x7f9ef8149860 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef814b470 0x7f9ef814b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef814b470 0x7f9ef814b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56614/0 (socket says 172.21.15.97:56614) 2024-09-10T21:36:22.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 -- 172.21.15.97:0/444384843 learned_addr learned my addr 172.21.15.97:0/444384843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef5e00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9edc004d50 0x7f9ef8149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9eed400640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ef8149860 0x7f9ef80cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 -- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ef8149860 msgr2=0x7f9ef80cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f9ef8149860 0x7f9ef80cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 -- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9edc004d50 msgr2=0x7f9ef8149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9edc004d50 0x7f9ef8149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 -- 172.21.15.97:0/444384843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef81518f0 con 0x7f9ef814b470 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef5e00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9edc004d50 0x7f9ef8149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9ef6800640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef814b470 0x7f9ef814b900 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00991b0 tx=0x7f9ef0004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00ad050 con 0x7f9ef814b470 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ef00a7b20 con 0x7f9ef814b470 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0003980 con 0x7f9ef814b470 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef8151bd0 con 0x7f9ef814b470 2024-09-10T21:36:22.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef8152190 con 0x7f9ef814b470 2024-09-10T21:36:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ef806e730 con 0x7f9ef814b470 2024-09-10T21:36:22.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.100+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9ef0003b20 con 0x7f9ef814b470 2024-09-10T21:36:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eeea00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9ed40725e0 0x7f9ed4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f9ef012e6d0 con 0x7f9ef814b470 2024-09-10T21:36:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eeea00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f9ed4078b20 0x7f9ed407af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ef00b5e50 con 0x7f9ed4078b20 2024-09-10T21:36:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f9ef012ea80 con 0x7f9ef814b470 2024-09-10T21:36:22.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9ef5e00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9ed40725e0 0x7f9ed4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eed400640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f9ed4078b20 0x7f9ed407af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9ef5e00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9ed40725e0 0x7f9ed4074aa0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8006ca0 tx=0x7f9ee80067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eed400640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f9ed4078b20 0x7f9ed407af60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.104+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== osd.4 v2:172.21.15.194:6800/4073109273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ef00b5e50 con 0x7f9ed4078b20 2024-09-10T21:36:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.116+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 --> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9ef80b49e0 con 0x7f9ed4078b20 2024-09-10T21:36:22.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9eeea00640 1 -- 172.21.15.97:0/444384843 <== osd.4 v2:172.21.15.194:6800/4073109273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9ef80b49e0 con 0x7f9ed4078b20 2024-09-10T21:36:22.120 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:22.120 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f9ed4078b20 msgr2=0x7f9ed407af60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6800/4073109273,v1:172.21.15.194:6801/4073109273] conn(0x7f9ed4078b20 0x7f9ed407af60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9ed40725e0 msgr2=0x7f9ed4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f9ed40725e0 0x7f9ed4074aa0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8006ca0 tx=0x7f9ee80067b0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef814b470 msgr2=0x7f9ef814b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 --2- 172.21.15.97:0/444384843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef814b470 0x7f9ef814b900 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00991b0 tx=0x7f9ef0004f10 comp rx=0 tx=0).stop 2024-09-10T21:36:22.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9ef6800640 1 -- 172.21.15.97:0/444384843 reap_dead start 2024-09-10T21:36:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 shutdown_connections 2024-09-10T21:36:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 >> 172.21.15.97:0/444384843 conn(0x7f9ef80b1780 msgr2=0x7f9ef81504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 shutdown_connections 2024-09-10T21:36:22.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.120+0000 7f9efcc00640 1 -- 172.21.15.97:0/444384843 wait complete. 2024-09-10T21:36:22.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T21:36:22.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 -- 172.21.15.97:0/327562300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580ce830 msgr2=0x7ff5580cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 --2- 172.21.15.97:0/327562300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580ce830 0x7ff5580cf800 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ff5580b4860 tx=0x7ff54801ef10 comp rx=0 tx=0).stop 2024-09-10T21:36:22.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 -- 172.21.15.97:0/327562300 shutdown_connections 2024-09-10T21:36:22.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 --2- 172.21.15.97:0/327562300 >> v2:172.21.15.194:3300/0 conn(0x7ff5580cec10 0x7ff5580cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 --2- 172.21.15.97:0/327562300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580ce830 0x7ff5580cf800 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 -- 172.21.15.97:0/327562300 >> 172.21.15.97:0/327562300 conn(0x7ff5580b37c0 msgr2=0x7ff5580b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 -- 172.21.15.97:0/327562300 shutdown_connections 2024-09-10T21:36:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 -- 172.21.15.97:0/327562300 wait complete. 2024-09-10T21:36:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 Processor -- start 2024-09-10T21:36:22.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.280+0000 7ff55d200640 1 -- start start 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5580ce830 0x7ff5581f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5580cec10 0x7ff5580cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d0280 0x7ff558153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5580b4450 con 0x7ff5580d0280 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff5580cde60 con 0x7ff5580ce830 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7ff5580ce1a0 con 0x7ff5580cec10 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d0280 0x7ff558153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d0280 0x7ff558153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56646/0 (socket says 172.21.15.97:56646) 2024-09-10T21:36:22.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 -- 172.21.15.97:0/2261460248 learned_addr learned my addr 172.21.15.97:0/2261460248 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff555e00640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5580ce830 0x7ff5581f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff54d400640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5580cec10 0x7ff5580cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 -- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5580cec10 msgr2=0x7ff5580cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5580cec10 0x7ff5580cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 -- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5580ce830 msgr2=0x7ff5581f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5580ce830 0x7ff5581f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 -- 172.21.15.97:0/2261460248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff558154360 con 0x7ff5580d0280 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff54d400640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5580cec10 0x7ff5580cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff556800640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d0280 0x7ff558153e20 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c790 tx=0x7ff55000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff54ea00640 1 -- 172.21.15.97:0/2261460248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550017070 con 0x7ff5580d0280 2024-09-10T21:36:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff54ea00640 1 -- 172.21.15.97:0/2261460248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff550005070 con 0x7ff5580d0280 2024-09-10T21:36:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff54ea00640 1 -- 172.21.15.97:0/2261460248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500054c0 con 0x7ff5580d0280 2024-09-10T21:36:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5580cb590 con 0x7ff5580d0280 2024-09-10T21:36:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5580b6060 con 0x7ff5580d0280 2024-09-10T21:36:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.284+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5580b2650 con 0x7ff5580d0280 2024-09-10T21:36:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.288+0000 7ff54ea00640 1 -- 172.21.15.97:0/2261460248 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff550005660 con 0x7ff5580d0280 2024-09-10T21:36:22.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.288+0000 7ff54ea00640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5340725e0 0x7ff534074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.288+0000 7ff54ea00640 1 -- 172.21.15.97:0/2261460248 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7ff550092ab0 con 0x7ff5580d0280 2024-09-10T21:36:22.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.288+0000 7ff555e00640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5340725e0 0x7ff534074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.288+0000 7ff555e00640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5340725e0 0x7ff534074aa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff5480b2000 tx=0x7ff54801d750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.292+0000 7ff54ea00640 1 -- 172.21.15.97:0/2261460248 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff55005dc50 con 0x7ff5580d0280 2024-09-10T21:36:22.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.456+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7ff5581e8810 con 0x7ff5340725e0 2024-09-10T21:36:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.456+0000 7ff54ea00640 1 -- 172.21.15.97:0/2261460248 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff5581e8810 con 0x7ff5340725e0 2024-09-10T21:36:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5340725e0 msgr2=0x7ff534074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5340725e0 0x7ff534074aa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff5480b2000 tx=0x7ff54801d750 comp rx=0 tx=0).stop 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d0280 msgr2=0x7ff558153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d0280 0x7ff558153e20 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c790 tx=0x7ff55000cc50 comp rx=0 tx=0).stop 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 shutdown_connections 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7ff5340725e0 0x7ff534074aa0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5580d0280 0x7ff558153e20 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7ff5580cec10 0x7ff5580cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 --2- 172.21.15.97:0/2261460248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5580ce830 0x7ff5581f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 >> 172.21.15.97:0/2261460248 conn(0x7ff5580b37c0 msgr2=0x7ff5580cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 shutdown_connections 2024-09-10T21:36:22.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.460+0000 7ff55d200640 1 -- 172.21.15.97:0/2261460248 wait complete. 2024-09-10T21:36:22.481 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T21:36:22.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.624+0000 7fd12e200640 1 -- 172.21.15.97:0/1413271379 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1280b2570 msgr2=0x7fd128166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.624+0000 7fd12e200640 1 --2- 172.21.15.97:0/1413271379 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1280b2570 0x7fd128166fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd120009870 tx=0x7fd12001ef70 comp rx=0 tx=0).stop 2024-09-10T21:36:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.624+0000 7fd12e200640 1 -- 172.21.15.97:0/1413271379 shutdown_connections 2024-09-10T21:36:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.624+0000 7fd12e200640 1 --2- 172.21.15.97:0/1413271379 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1280b2570 0x7fd128166fe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.624+0000 7fd12e200640 1 -- 172.21.15.97:0/1413271379 >> 172.21.15.97:0/1413271379 conn(0x7fd1280b37c0 msgr2=0x7fd1280b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.624+0000 7fd12e200640 1 -- 172.21.15.97:0/1413271379 shutdown_connections 2024-09-10T21:36:22.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.624+0000 7fd12e200640 1 -- 172.21.15.97:0/1413271379 wait complete. 2024-09-10T21:36:22.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 Processor -- start 2024-09-10T21:36:22.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 -- start start 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1280b2570 0x7fd1281f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1280ce830 0x7fd12815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1280cec10 0x7fd128158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1280b4470 con 0x7fd1280ce830 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd1280cde60 con 0x7fd1280b2570 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7fd1280ce1a0 con 0x7fd1280cec10 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1280ce830 0x7fd12815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1280ce830 0x7fd12815aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56654/0 (socket says 172.21.15.97:56654) 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 -- 172.21.15.97:0/2410036495 learned_addr learned my addr 172.21.15.97:0/2410036495 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd127400640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1280b2570 0x7fd1281f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd127e00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1280cec10 0x7fd128158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 -- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1280cec10 msgr2=0x7fd128158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7fd1280cec10 0x7fd128158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 -- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1280b2570 msgr2=0x7fd1281f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1280b2570 0x7fd1281f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 -- 172.21.15.97:0/2410036495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1280af0c0 con 0x7fd1280ce830 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11ea00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1280ce830 0x7fd12815aad0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fd11000cfc0 tx=0x7fd110001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110016020 con 0x7fd1280ce830 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd11000ab40 con 0x7fd1280ce830 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1280c5500 con 0x7fd1280ce830 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110012420 con 0x7fd1280ce830 2024-09-10T21:36:22.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.628+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12806fc50 con 0x7fd1280ce830 2024-09-10T21:36:22.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1280cf800 con 0x7fd1280ce830 2024-09-10T21:36:22.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1100125c0 con 0x7fd1280ce830 2024-09-10T21:36:22.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd11fe00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1040725e0 0x7fd104074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7fd110093a40 con 0x7fd1280ce830 2024-09-10T21:36:22.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd11fe00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7fd104078b20 0x7fd10407af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd127400640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1040725e0 0x7fd104074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd110019ec0 con 0x7fd104078b20 2024-09-10T21:36:22.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fd1100982a0 con 0x7fd1280ce830 2024-09-10T21:36:22.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd127e00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7fd104078b20 0x7fd10407af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd127400640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1040725e0 0x7fd104074aa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd118007fd0 tx=0x7fd118009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.632+0000 7fd127e00640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7fd104078b20 0x7fd10407af60 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.636+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== osd.5 v2:172.21.15.194:6818/3659294892 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd110019ec0 con 0x7fd104078b20 2024-09-10T21:36:22.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 --> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd12815c310 con 0x7fd104078b20 2024-09-10T21:36:22.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd11fe00640 1 -- 172.21.15.97:0/2410036495 <== osd.5 v2:172.21.15.194:6818/3659294892 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd12815c310 con 0x7fd104078b20 2024-09-10T21:36:22.649 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:22.649 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:22.650 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7fd104078b20 msgr2=0x7fd10407af60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6818/3659294892,v1:172.21.15.194:6819/3659294892] conn(0x7fd104078b20 0x7fd10407af60 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1040725e0 msgr2=0x7fd104074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7fd1040725e0 0x7fd104074aa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd118007fd0 tx=0x7fd118009040 comp rx=0 tx=0).stop 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1280ce830 msgr2=0x7fd12815aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 --2- 172.21.15.97:0/2410036495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1280ce830 0x7fd12815aad0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fd11000cfc0 tx=0x7fd110001fe0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd127e00640 1 -- 172.21.15.97:0/2410036495 reap_dead start 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 shutdown_connections 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 >> 172.21.15.97:0/2410036495 conn(0x7fd1280b37c0 msgr2=0x7fd1280af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 shutdown_connections 2024-09-10T21:36:22.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.648+0000 7fd12e200640 1 -- 172.21.15.97:0/2410036495 wait complete. 2024-09-10T21:36:22.668 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T21:36:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- 172.21.15.97:0/3352899529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 msgr2=0x7efdd4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 --2- 172.21.15.97:0/3352899529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 0x7efdd4004810 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7efdd8006fd0 tx=0x7efdd801cee0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf1e00640 1 -- 172.21.15.97:0/3352899529 reap_dead start 2024-09-10T21:36:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- 172.21.15.97:0/3352899529 shutdown_connections 2024-09-10T21:36:22.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- 172.21.15.97:0/3352899529 >> 172.21.15.97:0/3352899529 conn(0x7efdec0b1780 msgr2=0x7efdec0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- 172.21.15.97:0/3352899529 shutdown_connections 2024-09-10T21:36:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- 172.21.15.97:0/3352899529 wait complete. 2024-09-10T21:36:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 Processor -- start 2024-09-10T21:36:22.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- start start 2024-09-10T21:36:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 0x7efdec1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdec141000 0x7efdec0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec0ceb10 0x7efdec13ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efdec0b2c00 con 0x7efdd4003f80 2024-09-10T21:36:22.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efdec0b2a80 con 0x7efdec0ceb10 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf3e00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7efdec0b2d80 con 0x7efdec141000 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 0x7efdec1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 0x7efdec1f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56662/0 (socket says 172.21.15.97:56662) 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf1400640 1 -- 172.21.15.97:0/3978809561 learned_addr learned my addr 172.21.15.97:0/3978809561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf0a00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdec141000 0x7efdec0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.812+0000 7efdf1e00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec0ceb10 0x7efdec13ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf1400640 1 -- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdec141000 msgr2=0x7efdec0aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf1400640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdec141000 0x7efdec0aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf1400640 1 -- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec0ceb10 msgr2=0x7efdec13ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf1400640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec0ceb10 0x7efdec13ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf1400640 1 -- 172.21.15.97:0/3978809561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec141480 con 0x7efdd4003f80 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf1400640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 0x7efdec1f4550 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7efde0011670 tx=0x7efde0011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efde9600640 1 -- 172.21.15.97:0/3978809561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000f740 con 0x7efdd4003f80 2024-09-10T21:36:22.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efde9600640 1 -- 172.21.15.97:0/3978809561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efde000f900 con 0x7efdd4003f80 2024-09-10T21:36:22.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efde9600640 1 -- 172.21.15.97:0/3978809561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000fd50 con 0x7efdd4003f80 2024-09-10T21:36:22.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec0cda70 con 0x7efdd4003f80 2024-09-10T21:36:22.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec0ce030 con 0x7efdd4003f80 2024-09-10T21:36:22.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec0b4120 con 0x7efdd4003f80 2024-09-10T21:36:22.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.816+0000 7efde9600640 1 -- 172.21.15.97:0/3978809561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efde0010040 con 0x7efdd4003f80 2024-09-10T21:36:22.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.820+0000 7efde9600640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdcc0725e0 0x7efdcc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:22.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.820+0000 7efde9600640 1 -- 172.21.15.97:0/3978809561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7efde0096710 con 0x7efdd4003f80 2024-09-10T21:36:22.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.820+0000 7efdf0a00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdcc0725e0 0x7efdcc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:22.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.820+0000 7efdf0a00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdcc0725e0 0x7efdcc074aa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efdd80ab000 tx=0x7efdd80a5f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:22.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.824+0000 7efde9600640 1 -- 172.21.15.97:0/3978809561 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efde0017070 con 0x7efdd4003f80 2024-09-10T21:36:22.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7efdec0cd820 con 0x7efdcc0725e0 2024-09-10T21:36:22.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efde9600640 1 -- 172.21.15.97:0/3978809561 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efdec0cd820 con 0x7efdcc0725e0 2024-09-10T21:36:22.988 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdcc0725e0 msgr2=0x7efdcc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdcc0725e0 0x7efdcc074aa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efdd80ab000 tx=0x7efdd80a5f90 comp rx=0 tx=0).stop 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 msgr2=0x7efdec1f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 0x7efdec1f4550 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7efde0011670 tx=0x7efde0011a70 comp rx=0 tx=0).stop 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 shutdown_connections 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7efdcc0725e0 0x7efdcc074aa0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdec0ceb10 0x7efdec13ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7efdec141000 0x7efdec0aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 --2- 172.21.15.97:0/3978809561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdd4003f80 0x7efdec1f4550 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 >> 172.21.15.97:0/3978809561 conn(0x7efdec0b1780 msgr2=0x7efdec0cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:22.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 shutdown_connections 2024-09-10T21:36:22.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:22.988+0000 7efdf3e00640 1 -- 172.21.15.97:0/3978809561 wait complete. 2024-09-10T21:36:23.009 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T21:36:23.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:23.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.152+0000 7f648fa00640 1 -- 172.21.15.97:0/3290485495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488143f90 msgr2=0x7f64880b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.152+0000 7f648fa00640 1 --2- 172.21.15.97:0/3290485495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488143f90 0x7f64880b2a80 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f6478009840 tx=0x7f647801c820 comp rx=0 tx=0).stop 2024-09-10T21:36:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.152+0000 7f648fa00640 1 -- 172.21.15.97:0/3290485495 shutdown_connections 2024-09-10T21:36:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.152+0000 7f648fa00640 1 --2- 172.21.15.97:0/3290485495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488143f90 0x7f64880b2a80 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.152+0000 7f648fa00640 1 -- 172.21.15.97:0/3290485495 >> 172.21.15.97:0/3290485495 conn(0x7f64880b1780 msgr2=0x7f64880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.152+0000 7f648fa00640 1 -- 172.21.15.97:0/3290485495 shutdown_connections 2024-09-10T21:36:23.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.152+0000 7f648fa00640 1 -- 172.21.15.97:0/3290485495 wait complete. 2024-09-10T21:36:23.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 Processor -- start 2024-09-10T21:36:23.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 -- start start 2024-09-10T21:36:23.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f648806fd50 0x7f64881faa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6488143f90 0x7f64880aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488144690 0x7f648813d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6488168e80 con 0x7f6488144690 2024-09-10T21:36:23.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6488168d00 con 0x7f648806fd50 2024-09-10T21:36:23.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f6488169000 con 0x7f6488143f90 2024-09-10T21:36:23.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488144690 0x7f648813d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488144690 0x7f648813d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56684/0 (socket says 172.21.15.97:56684) 2024-09-10T21:36:23.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 -- 172.21.15.97:0/1922537913 learned_addr learned my addr 172.21.15.97:0/1922537913 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:23.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648d000640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f648806fd50 0x7f64881faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 -- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6488143f90 msgr2=0x7f64880aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f6487e00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6488143f90 0x7f64880aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6488143f90 0x7f64880aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 -- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f648806fd50 msgr2=0x7f64881faa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f648806fd50 0x7f64881faa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 -- 172.21.15.97:0/1922537913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64880ae2c0 con 0x7f6488144690 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648d000640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f648806fd50 0x7f64881faa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f6487e00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f6488143f90 0x7f64880aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648da00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488144690 0x7f648813d4f0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f648000c780 tx=0x7f648000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480017070 con 0x7f6488144690 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6480005070 con 0x7f6488144690 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64880ae580 con 0x7f6488144690 2024-09-10T21:36:23.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.156+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480005500 con 0x7f6488144690 2024-09-10T21:36:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64880af750 con 0x7f6488144690 2024-09-10T21:36:23.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64880aeaa0 con 0x7f6488144690 2024-09-10T21:36:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f648000a5e0 con 0x7f6488144690 2024-09-10T21:36:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f6484c00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64680725e0 0x7f6468074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f6480092a80 con 0x7f6488144690 2024-09-10T21:36:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f6484c00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f6468078b20 0x7f646807af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f646807b630 con 0x7f6468078b20 2024-09-10T21:36:23.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f64800972a0 con 0x7f6488144690 2024-09-10T21:36:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f648d000640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64680725e0 0x7f6468074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.160+0000 7f6487e00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f6468078b20 0x7f646807af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.164+0000 7f6487e00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f6468078b20 0x7f646807af60 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.164+0000 7f648d000640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64680725e0 0x7f6468074aa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f64780af000 tx=0x7f647801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.164+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== osd.6 v2:172.21.15.194:6810/1793675643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f646807b630 con 0x7f6468078b20 2024-09-10T21:36:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 --> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f64880b2a80 con 0x7f6468078b20 2024-09-10T21:36:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f6484c00640 1 -- 172.21.15.97:0/1922537913 <== osd.6 v2:172.21.15.194:6810/1793675643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f64880b2a80 con 0x7f6468078b20 2024-09-10T21:36:23.178 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:23.178 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:23.178 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f6468078b20 msgr2=0x7f646807af60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6810/1793675643,v1:172.21.15.194:6811/1793675643] conn(0x7f6468078b20 0x7f646807af60 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64680725e0 msgr2=0x7f6468074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f64680725e0 0x7f6468074aa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f64780af000 tx=0x7f647801b060 comp rx=0 tx=0).stop 2024-09-10T21:36:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488144690 msgr2=0x7f648813d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 --2- 172.21.15.97:0/1922537913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488144690 0x7f648813d4f0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f648000c780 tx=0x7f648000cc40 comp rx=0 tx=0).stop 2024-09-10T21:36:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648da00640 1 -- 172.21.15.97:0/1922537913 reap_dead start 2024-09-10T21:36:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 shutdown_connections 2024-09-10T21:36:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 >> 172.21.15.97:0/1922537913 conn(0x7f64880b1780 msgr2=0x7f64880cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 shutdown_connections 2024-09-10T21:36:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.176+0000 7f648fa00640 1 -- 172.21.15.97:0/1922537913 wait complete. 2024-09-10T21:36:23.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T21:36:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.340+0000 7f5f1b200640 1 -- 172.21.15.97:0/2593390565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f00004810 msgr2=0x7f5f00004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.340+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2593390565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f00004810 0x7f5f00004c90 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f5f04001860 tx=0x7f5f0401f030 comp rx=0 tx=0).stop 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.340+0000 7f5f1b200640 1 -- 172.21.15.97:0/2593390565 shutdown_connections 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.340+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2593390565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f00004810 0x7f5f00004c90 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.340+0000 7f5f1b200640 1 -- 172.21.15.97:0/2593390565 >> 172.21.15.97:0/2593390565 conn(0x7f5f1c0d1a60 msgr2=0x7f5f1c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.340+0000 7f5f1b200640 1 -- 172.21.15.97:0/2593390565 shutdown_connections 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.340+0000 7f5f1b200640 1 -- 172.21.15.97:0/2593390565 wait complete. 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 Processor -- start 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 -- start start 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f00004810 0x7f5f1c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b5fe0 0x7f5f1c0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c1f8110 0x7f5f1c1f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c0cfbd0 con 0x7f5f1c1f8110 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c0cfa50 con 0x7f5f1c0b5fe0 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1c0cfd50 con 0x7f5f00004810 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c1f8110 0x7f5f1c1f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c1f8110 0x7f5f1c1f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56702/0 (socket says 172.21.15.97:56702) 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 -- 172.21.15.97:0/2116119783 learned_addr learned my addr 172.21.15.97:0/2116119783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f11400640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b5fe0 0x7f5f1c0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f19e00640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f00004810 0x7f5f1c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 -- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f00004810 msgr2=0x7f5f1c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f00004810 0x7f5f1c0af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 -- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b5fe0 msgr2=0x7f5f1c0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b5fe0 0x7f5f1c0af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 -- 172.21.15.97:0/2116119783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c1f4c60 con 0x7f5f1c1f8110 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1a800640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c1f8110 0x7f5f1c1f8590 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f5f14000c10 tx=0x7f5f140039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f12a00640 1 -- 172.21.15.97:0/2116119783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14011840 con 0x7f5f1c1f8110 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f12a00640 1 -- 172.21.15.97:0/2116119783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f14018070 con 0x7f5f1c1f8110 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f12a00640 1 -- 172.21.15.97:0/2116119783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14011c80 con 0x7f5f1c1f8110 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c1f4f40 con 0x7f5f1c1f8110 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c1f5500 con 0x7f5f1c1f8110 2024-09-10T21:36:23.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.344+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1c0ce1e0 con 0x7f5f1c1f8110 2024-09-10T21:36:23.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.348+0000 7f5f12a00640 1 -- 172.21.15.97:0/2116119783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f1401d460 con 0x7f5f1c1f8110 2024-09-10T21:36:23.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.348+0000 7f5f12a00640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ef80725e0 0x7f5ef8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.348+0000 7f5f12a00640 1 -- 172.21.15.97:0/2116119783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f5f140937c0 con 0x7f5f1c1f8110 2024-09-10T21:36:23.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.348+0000 7f5f19e00640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ef80725e0 0x7f5ef8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.348+0000 7f5f19e00640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ef80725e0 0x7f5ef8074aa0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0767e0 tx=0x7f5f0c009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.352+0000 7f5f12a00640 1 -- 172.21.15.97:0/2116119783 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f14011e20 con 0x7f5f1c1f8110 2024-09-10T21:36:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.516+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f5f1c06f720 con 0x7f5ef80725e0 2024-09-10T21:36:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.516+0000 7f5f12a00640 1 -- 172.21.15.97:0/2116119783 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5f1c06f720 con 0x7f5ef80725e0 2024-09-10T21:36:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ef80725e0 msgr2=0x7f5ef8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ef80725e0 0x7f5ef8074aa0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0767e0 tx=0x7f5f0c009500 comp rx=0 tx=0).stop 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c1f8110 msgr2=0x7f5f1c1f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c1f8110 0x7f5f1c1f8590 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f5f14000c10 tx=0x7f5f140039d0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 shutdown_connections 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f5ef80725e0 0x7f5ef8074aa0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f1c1f8110 0x7f5f1c1f8590 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f1c0b5fe0 0x7f5f1c0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 --2- 172.21.15.97:0/2116119783 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f5f00004810 0x7f5f1c0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 >> 172.21.15.97:0/2116119783 conn(0x7f5f1c0d1a60 msgr2=0x7f5f1c071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 shutdown_connections 2024-09-10T21:36:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.520+0000 7f5f1b200640 1 -- 172.21.15.97:0/2116119783 wait complete. 2024-09-10T21:36:23.539 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T21:36:23.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:36:23.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 -- 172.21.15.97:0/3229009822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff24004d50 msgr2=0x7eff240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 --2- 172.21.15.97:0/3229009822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff24004d50 0x7eff240051d0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7eff38009b10 tx=0x7eff3801d230 comp rx=0 tx=0).stop 2024-09-10T21:36:23.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff3fe00640 1 -- 172.21.15.97:0/3229009822 reap_dead start 2024-09-10T21:36:23.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 -- 172.21.15.97:0/3229009822 shutdown_connections 2024-09-10T21:36:23.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 -- 172.21.15.97:0/3229009822 >> 172.21.15.97:0/3229009822 conn(0x7eff400b1780 msgr2=0x7eff400b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:23.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 -- 172.21.15.97:0/3229009822 shutdown_connections 2024-09-10T21:36:23.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 -- 172.21.15.97:0/3229009822 wait complete. 2024-09-10T21:36:23.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 Processor -- start 2024-09-10T21:36:23.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.684+0000 7eff46000640 1 -- start start 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff24004d50 0x7eff4006fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff40154f20 0x7eff40152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff40141630 0x7eff400cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff400b2c00 con 0x7eff40154f20 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff400b2a80 con 0x7eff24004d50 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7eff400b2d80 con 0x7eff40141630 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff40154f20 0x7eff40152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff40154f20 0x7eff40152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56712/0 (socket says 172.21.15.97:56712) 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 -- 172.21.15.97:0/2887855234 learned_addr learned my addr 172.21.15.97:0/2887855234 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff3f400640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff24004d50 0x7eff4006fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff3fe00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff40141630 0x7eff400cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 -- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff40141630 msgr2=0x7eff400cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff40141630 0x7eff400cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 -- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff24004d50 msgr2=0x7eff4006fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff24004d50 0x7eff4006fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 -- 172.21.15.97:0/2887855234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff40155380 con 0x7eff40154f20 2024-09-10T21:36:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff3fe00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7eff40141630 0x7eff400cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff36a00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff40154f20 0x7eff40152480 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7eff28000fc0 tx=0x7eff2800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28004440 con 0x7eff40154f20 2024-09-10T21:36:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff2800ee70 con 0x7eff40154f20 2024-09-10T21:36:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2800ca40 con 0x7eff40154f20 2024-09-10T21:36:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff400ca2e0 con 0x7eff40154f20 2024-09-10T21:36:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff400cab80 con 0x7eff40154f20 2024-09-10T21:36:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.688+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff400b2fe0 con 0x7eff40154f20 2024-09-10T21:36:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff2800cbe0 con 0x7eff40154f20 2024-09-10T21:36:23.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff37e00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff1c0725e0 0x7eff1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7eff280934a0 con 0x7eff40154f20 2024-09-10T21:36:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff37e00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7eff1c078b20 0x7eff1c07af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff1c07b630 con 0x7eff1c078b20 2024-09-10T21:36:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7eff2809a050 con 0x7eff40154f20 2024-09-10T21:36:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff3f400640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff1c0725e0 0x7eff1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff3fe00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7eff1c078b20 0x7eff1c07af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff3f400640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff1c0725e0 0x7eff1c074aa0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7eff30000f90 tx=0x7eff30006e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff3fe00640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7eff1c078b20 0x7eff1c07af60 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.692+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== osd.7 v2:172.21.15.194:6823/3380827268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff1c07b630 con 0x7eff1c078b20 2024-09-10T21:36:23.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 --> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7eff400d03a0 con 0x7eff1c078b20 2024-09-10T21:36:23.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff37e00640 1 -- 172.21.15.97:0/2887855234 <== osd.7 v2:172.21.15.194:6823/3380827268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7eff400d03a0 con 0x7eff1c078b20 2024-09-10T21:36:23.708 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-10T21:36:23.709 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:36:23.709 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-10T21:36:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7eff1c078b20 msgr2=0x7eff1c07af60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6823/3380827268,v1:172.21.15.194:6825/3380827268] conn(0x7eff1c078b20 0x7eff1c07af60 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff1c0725e0 msgr2=0x7eff1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7eff1c0725e0 0x7eff1c074aa0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7eff30000f90 tx=0x7eff30006e70 comp rx=0 tx=0).stop 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff40154f20 msgr2=0x7eff40152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 --2- 172.21.15.97:0/2887855234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff40154f20 0x7eff40152480 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7eff28000fc0 tx=0x7eff2800eb60 comp rx=0 tx=0).stop 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff3fe00640 1 -- 172.21.15.97:0/2887855234 reap_dead start 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 shutdown_connections 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 >> 172.21.15.97:0/2887855234 conn(0x7eff400b1780 msgr2=0x7eff400cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 shutdown_connections 2024-09-10T21:36:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.708+0000 7eff46000640 1 -- 172.21.15.97:0/2887855234 wait complete. 2024-09-10T21:36:23.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.868+0000 7f296fe00640 1 -- 172.21.15.97:0/1175011961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29700b4320 msgr2=0x7f29700725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.868+0000 7f296fe00640 1 --2- 172.21.15.97:0/1175011961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29700b4320 0x7f29700725f0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f296000aa10 tx=0x7f296001dc30 comp rx=0 tx=0).stop 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- 172.21.15.97:0/1175011961 shutdown_connections 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 --2- 172.21.15.97:0/1175011961 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f297006f1c0 0x7f29700ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 --2- 172.21.15.97:0/1175011961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29700b4320 0x7f29700725f0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- 172.21.15.97:0/1175011961 >> 172.21.15.97:0/1175011961 conn(0x7f297006cb40 msgr2=0x7f297006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- 172.21.15.97:0/1175011961 shutdown_connections 2024-09-10T21:36:23.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- 172.21.15.97:0/1175011961 wait complete. 2024-09-10T21:36:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 Processor -- start 2024-09-10T21:36:23.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- start start 2024-09-10T21:36:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297006f1c0 0x7f2970146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2970072b30 0x7f29701450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29700b4320 0x7f297014f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f29700b29a0 con 0x7f297006f1c0 2024-09-10T21:36:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f29700cf630 con 0x7f2970072b30 2024-09-10T21:36:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f29700b2b20 con 0x7f29700b4320 2024-09-10T21:36:23.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297006f1c0 0x7f2970146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297006f1c0 0x7f2970146b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56728/0 (socket says 172.21.15.97:56728) 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 -- 172.21.15.97:0/2234119560 learned_addr learned my addr 172.21.15.97:0/2234119560 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296f400640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29700b4320 0x7f297014f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296e000640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2970072b30 0x7f29701450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 -- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29700b4320 msgr2=0x7f297014f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29700b4320 0x7f297014f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 -- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2970072b30 msgr2=0x7f29701450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2970072b30 0x7f29701450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 -- 172.21.15.97:0/2234119560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2970154e70 con 0x7f297006f1c0 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296e000640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2970072b30 0x7f29701450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296ea00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297006f1c0 0x7f2970146b20 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f296000a960 tx=0x7f29600a6a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f2966a00640 1 -- 172.21.15.97:0/2234119560 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29600b3070 con 0x7f297006f1c0 2024-09-10T21:36:23.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f2966a00640 1 -- 172.21.15.97:0/2234119560 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2960003b80 con 0x7f297006f1c0 2024-09-10T21:36:23.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.872+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2970151520 con 0x7f297006f1c0 2024-09-10T21:36:23.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.876+0000 7f2966a00640 1 -- 172.21.15.97:0/2234119560 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29600ae440 con 0x7f297006f1c0 2024-09-10T21:36:23.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.876+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2970151ae0 con 0x7f297006f1c0 2024-09-10T21:36:23.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.876+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29700ce8d0 con 0x7f297006f1c0 2024-09-10T21:36:23.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.876+0000 7f2966a00640 1 -- 172.21.15.97:0/2234119560 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f296000b040 con 0x7f297006f1c0 2024-09-10T21:36:23.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.876+0000 7f2966a00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29500725e0 0x7f2950074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:23.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.876+0000 7f2966a00640 1 -- 172.21.15.97:0/2234119560 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f296012cdd0 con 0x7f297006f1c0 2024-09-10T21:36:23.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.876+0000 7f296e000640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29500725e0 0x7f2950074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:23.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.880+0000 7f296e000640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29500725e0 0x7f2950074aa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f295c003ec0 tx=0x7f295c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:23.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:23.880+0000 7f2966a00640 1 -- 172.21.15.97:0/2234119560 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2960017820 con 0x7f297006f1c0 2024-09-10T21:36:24.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f297006d9a0 con 0x7f29500725e0 2024-09-10T21:36:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f2966a00640 1 -- 172.21.15.97:0/2234119560 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f297006d9a0 con 0x7f29500725e0 2024-09-10T21:36:24.049 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29500725e0 msgr2=0x7f2950074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29500725e0 0x7f2950074aa0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f295c003ec0 tx=0x7f295c009040 comp rx=0 tx=0).stop 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297006f1c0 msgr2=0x7f2970146b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297006f1c0 0x7f2970146b20 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f296000a960 tx=0x7f29600a6a90 comp rx=0 tx=0).stop 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 shutdown_connections 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f29500725e0 0x7f2950074aa0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f29700b4320 0x7f297014f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2970072b30 0x7f29701450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 --2- 172.21.15.97:0/2234119560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297006f1c0 0x7f2970146b20 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 >> 172.21.15.97:0/2234119560 conn(0x7f297006cb40 msgr2=0x7f2970145c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:24.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 shutdown_connections 2024-09-10T21:36:24.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.048+0000 7f296fe00640 1 -- 172.21.15.97:0/2234119560 wait complete. 2024-09-10T21:36:24.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:36:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 -- 172.21.15.97:0/1197598451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 msgr2=0x7f0d680af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 --2- 172.21.15.97:0/1197598451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 0x7f0d680af4d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009c80 tx=0x7f0d6001cda0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 -- 172.21.15.97:0/1197598451 shutdown_connections 2024-09-10T21:36:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 --2- 172.21.15.97:0/1197598451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 0x7f0d680af4d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 --2- 172.21.15.97:0/1197598451 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0d680ce640 0x7f0d680cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 -- 172.21.15.97:0/1197598451 >> 172.21.15.97:0/1197598451 conn(0x7f0d680d1c30 msgr2=0x7f0d680d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 -- 172.21.15.97:0/1197598451 shutdown_connections 2024-09-10T21:36:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.208+0000 7f0d70400640 1 -- 172.21.15.97:0/1197598451 wait complete. 2024-09-10T21:36:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 Processor -- start 2024-09-10T21:36:24.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 -- start start 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0d680b0d50 0x7f0d680c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 0x7f0d680b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d680ce640 0x7f0d680c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68168cc0 con 0x7f0d680ce640 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68168b40 con 0x7f0d680cd6e0 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f0d68168e40 con 0x7f0d680b0d50 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d680ce640 0x7f0d680c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d680ce640 0x7f0d680c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56734/0 (socket says 172.21.15.97:56734) 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 -- 172.21.15.97:0/250115645 learned_addr learned my addr 172.21.15.97:0/250115645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6d000640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 0x7f0d680b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6da00640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0d680b0d50 0x7f0d680c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 -- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0d680b0d50 msgr2=0x7f0d680c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0d680b0d50 0x7f0d680c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 -- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 msgr2=0x7f0d680b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 0x7f0d680b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 -- 172.21.15.97:0/250115645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d680c6bc0 con 0x7f0d680ce640 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d6e400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d680ce640 0x7f0d680c4a40 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009bd0 tx=0x7f0d60003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d65600640 1 -- 172.21.15.97:0/250115645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d600bb070 con 0x7f0d680ce640 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d65600640 1 -- 172.21.15.97:0/250115645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d60004d40 con 0x7f0d680ce640 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d680c6600 con 0x7f0d680ce640 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d65600640 1 -- 172.21.15.97:0/250115645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d600b2040 con 0x7f0d680ce640 2024-09-10T21:36:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.212+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d680b4b00 con 0x7f0d680ce640 2024-09-10T21:36:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.216+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d680cea20 con 0x7f0d680ce640 2024-09-10T21:36:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.216+0000 7f0d65600640 1 -- 172.21.15.97:0/250115645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0d600b5030 con 0x7f0d680ce640 2024-09-10T21:36:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.216+0000 7f0d65600640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0d480725e0 0x7f0d48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.216+0000 7f0d65600640 1 -- 172.21.15.97:0/250115645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f0d6012ccc0 con 0x7f0d680ce640 2024-09-10T21:36:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.216+0000 7f0d6da00640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0d480725e0 0x7f0d48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.216+0000 7f0d6da00640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0d480725e0 0x7f0d48074aa0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009600 tx=0x7f0d5c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:24.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.220+0000 7f0d65600640 1 -- 172.21.15.97:0/250115645 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d60133080 con 0x7f0d680ce640 2024-09-10T21:36:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.388+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 --> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d6815f2c0 con 0x7f0d480725e0 2024-09-10T21:36:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.388+0000 7f0d65600640 1 -- 172.21.15.97:0/250115645 <== mgr.4105 v2:172.21.15.194:6808/751391564 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7f0d6815f2c0 con 0x7f0d480725e0 2024-09-10T21:36:24.390 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0d480725e0 msgr2=0x7f0d48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0d480725e0 0x7f0d48074aa0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009600 tx=0x7f0d5c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d680ce640 msgr2=0x7f0d680c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d680ce640 0x7f0d680c4a40 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009bd0 tx=0x7f0d60003f60 comp rx=0 tx=0).stop 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 shutdown_connections 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f0d480725e0 0x7f0d48074aa0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d680ce640 0x7f0d680c4a40 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d680cd6e0 0x7f0d680b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 --2- 172.21.15.97:0/250115645 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f0d680b0d50 0x7f0d680c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 >> 172.21.15.97:0/250115645 conn(0x7f0d680d1c30 msgr2=0x7f0d680b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 shutdown_connections 2024-09-10T21:36:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.392+0000 7f0d70400640 1 -- 172.21.15.97:0/250115645 wait complete. 2024-09-10T21:36:24.412 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-10T21:36:24.259433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":286828,"kb_used_data":16520,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687892,"statfs":{"total":773094113280,"available":772800401408,"internally_reserved":0,"allocated":16916480,"data_stored":9114112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":276614707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.284267"},"pg_stats":[{"pgid":"1.0","version":"152'74","reported_seq":375,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T21:36:20.995561+0000","last_change":"2024-09-10T21:36:20.995561+0000","last_active":"2024-09-10T21:36:20.995561+0000","last_peered":"2024-09-10T21:36:20.995561+0000","last_clean":"2024-09-10T21:36:20.995561+0000","last_became_active":"2024-09-10T21:24:03.517291+0000","last_became_peered":"2024-09-10T21:24:03.517291+0000","last_unstale":"2024-09-10T21:36:20.995561+0000","last_undegraded":"2024-09-10T21:36:20.995561+0000","last_fullsized":"2024-09-10T21:36:20.995561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'74","last_scrub_stamp":"2024-09-10T21:36:20.995485+0000","last_deep_scrub":"152'74","last_deep_scrub_stamp":"2024-09-10T21:36:20.995485+0000","last_clean_scrub_stamp":"2024-09-10T21:36:20.995485+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:24.835426+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28948,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342892,"statfs":{"total":96636764160,"available":96607121408,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509592,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33684,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338156,"statfs":{"total":96636764160,"available":96602271744,"internally_reserved":0,"allocated":2560000,"data_stored":1582040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41428,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330412,"statfs":{"total":96636764160,"available":96594341888,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42012,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329828,"statfs":{"total":96636764160,"available":96593743872,"internally_reserved":0,"allocated":2560000,"data_stored":1582040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33112,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1966080,"data_stored":991672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:36:24.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- 172.21.15.97:0/880904905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15eed0 msgr2=0x7f330c0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 --2- 172.21.15.97:0/880904905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15eed0 0x7f330c0d27f0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f330c0d0490 tx=0x7f330001c870 comp rx=0 tx=0).stop 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- 172.21.15.97:0/880904905 shutdown_connections 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 --2- 172.21.15.97:0/880904905 >> v2:172.21.15.194:3300/0 conn(0x7f330c0cca90 0x7f330c0aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 --2- 172.21.15.97:0/880904905 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f330c0cdd40 0x7f330c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 --2- 172.21.15.97:0/880904905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15eed0 0x7f330c0d27f0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- 172.21.15.97:0/880904905 >> 172.21.15.97:0/880904905 conn(0x7f330c0d1cd0 msgr2=0x7f330c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- 172.21.15.97:0/880904905 shutdown_connections 2024-09-10T21:36:24.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- 172.21.15.97:0/880904905 wait complete. 2024-09-10T21:36:24.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 Processor -- start 2024-09-10T21:36:24.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- start start 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0cca90 0x7f330c14c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f330c0cdd40 0x7f330c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f330c15eed0 0x7f330c0b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0b0820 con 0x7f330c0cca90 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0b06a0 con 0x7f330c15eed0 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3314800640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0b09a0 con 0x7f330c0cdd40 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0cca90 0x7f330c14c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0cca90 0x7f330c14c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56756/0 (socket says 172.21.15.97:56756) 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 -- 172.21.15.97:0/1691537496 learned_addr learned my addr 172.21.15.97:0/1691537496 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3312800640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f330c15eed0 0x7f330c0b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311400640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f330c0cdd40 0x7f330c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 -- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f330c0cdd40 msgr2=0x7f330c14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f330c0cdd40 0x7f330c14bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 -- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f330c15eed0 msgr2=0x7f330c0b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f330c15eed0 0x7f330c0b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 -- 172.21.15.97:0/1691537496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c0b7050 con 0x7f330c0cca90 2024-09-10T21:36:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3311e00640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0cca90 0x7f330c14c610 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f330001af20 tx=0x7f33000a5850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.556+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33000b2040 con 0x7f330c0cca90 2024-09-10T21:36:24.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33000044b0 con 0x7f330c0cca90 2024-09-10T21:36:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300005290 con 0x7f330c0cca90 2024-09-10T21:36:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c0b75b0 con 0x7f330c0cca90 2024-09-10T21:36:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c0b2950 con 0x7f330c0cca90 2024-09-10T21:36:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c0b3070 con 0x7f330c0cca90 2024-09-10T21:36:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3300005430 con 0x7f330c0cca90 2024-09-10T21:36:24.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3306000640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32f00725e0 0x7f32f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f330012ca80 con 0x7f330c0cca90 2024-09-10T21:36:24.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.560+0000 7f3311400640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32f00725e0 0x7f32f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.564+0000 7f3311400640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32f00725e0 0x7f32f0074aa0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f330c15e8c0 tx=0x7f32fc00b510 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:24.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.564+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f330012ee20 con 0x7f330c0cca90 2024-09-10T21:36:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.728+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f330c0cc550 con 0x7f330c0cca90 2024-09-10T21:36:24.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v47)=0 v47) ==== 123+0+0 (secure 0 0 0) 0x7f330c0b7050 con 0x7f330c0cca90 2024-09-10T21:36:24.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3306000640 1 -- 172.21.15.97:0/1691537496 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f3300132080 con 0x7f330c0cca90 2024-09-10T21:36:24.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32f00725e0 msgr2=0x7f32f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32f00725e0 0x7f32f0074aa0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f330c15e8c0 tx=0x7f32fc00b510 comp rx=0 tx=0).stop 2024-09-10T21:36:24.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0cca90 msgr2=0x7f330c14c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0cca90 0x7f330c14c610 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f330001af20 tx=0x7f33000a5850 comp rx=0 tx=0).stop 2024-09-10T21:36:24.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 shutdown_connections 2024-09-10T21:36:24.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f32f00725e0 0x7f32f0074aa0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f330c15eed0 0x7f330c0b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f330c0cdd40 0x7f330c14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 --2- 172.21.15.97:0/1691537496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0cca90 0x7f330c14c610 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 >> 172.21.15.97:0/1691537496 conn(0x7f330c0d1cd0 msgr2=0x7f330c06dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:24.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 shutdown_connections 2024-09-10T21:36:24.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.736+0000 7f3314800640 1 -- 172.21.15.97:0/1691537496 wait complete. 2024-09-10T21:36:24.758 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:36:24.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 -- 172.21.15.97:0/2771311484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 msgr2=0x7f88300cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 --2- 172.21.15.97:0/2771311484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cea20 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f8820009840 tx=0x7f882001c820 comp rx=0 tx=0).stop 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 -- 172.21.15.97:0/2771311484 shutdown_connections 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 --2- 172.21.15.97:0/2771311484 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f88300b0d50 0x7f88300aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 --2- 172.21.15.97:0/2771311484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cea20 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 -- 172.21.15.97:0/2771311484 >> 172.21.15.97:0/2771311484 conn(0x7f88300d1c30 msgr2=0x7f88300d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 -- 172.21.15.97:0/2771311484 shutdown_connections 2024-09-10T21:36:24.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.896+0000 7f8836c00640 1 -- 172.21.15.97:0/2771311484 wait complete. 2024-09-10T21:36:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 Processor -- start 2024-09-10T21:36:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 -- start start 2024-09-10T21:36:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 --2- >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f88300af300 0x7f883014b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88300b0d50 0x7f8830152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8830169270 con 0x7f88300ce640 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88301690f0 con 0x7f88300b0d50 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 -- --> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] -- mon_getmap magic: 0 -- 0x7f88301693f0 con 0x7f88300af300 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56768/0 (socket says 172.21.15.97:56768) 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 -- 172.21.15.97:0/2560103655 learned_addr learned my addr 172.21.15.97:0/2560103655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f882fe00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f88300af300 0x7f883014b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f882f400640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88300b0d50 0x7f8830152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 -- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f88300af300 msgr2=0x7f883014b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f88300af300 0x7f883014b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 -- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88300b0d50 msgr2=0x7f8830152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88300b0d50 0x7f8830152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 -- 172.21.15.97:0/2560103655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883015f030 con 0x7f88300ce640 2024-09-10T21:36:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8834c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cb1e0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f882800cb10 tx=0x7f882800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8827e00640 1 -- 172.21.15.97:0/2560103655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828016020 con 0x7f88300ce640 2024-09-10T21:36:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8827e00640 1 -- 172.21.15.97:0/2560103655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f882800ce50 con 0x7f88300ce640 2024-09-10T21:36:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8827e00640 1 -- 172.21.15.97:0/2560103655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882800acd0 con 0x7f88300ce640 2024-09-10T21:36:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883015c530 con 0x7f88300ce640 2024-09-10T21:36:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.900+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301533a0 con 0x7f88300ce640 2024-09-10T21:36:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.904+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88300aedc0 con 0x7f88300ce640 2024-09-10T21:36:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.904+0000 7f8827e00640 1 -- 172.21.15.97:0/2560103655 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f882801b400 con 0x7f88300ce640 2024-09-10T21:36:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.904+0000 7f8827e00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88100725e0 0x7f8810074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.904+0000 7f8827e00640 1 -- 172.21.15.97:0/2560103655 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5421+0+0 (secure 0 0 0) 0x7f8828093470 con 0x7f88300ce640 2024-09-10T21:36:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.904+0000 7f882fe00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88100725e0 0x7f8810074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:24.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.904+0000 7f882fe00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88100725e0 0x7f8810074aa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f88200af000 tx=0x7f88200195c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:24.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:24.908+0000 7f8827e00640 1 -- 172.21.15.97:0/2560103655 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8828020020 con 0x7f88300ce640 2024-09-10T21:36:25.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.104+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88300d21b0 con 0x7f88300ce640 2024-09-10T21:36:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.104+0000 7f8827e00640 1 -- 172.21.15.97:0/2560103655 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 352 v352) ==== 95+0+3921 (secure 0 0 0) 0x7f882805e6b0 con 0x7f88300ce640 2024-09-10T21:36:25.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-10T21:36:25.107 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":352,"btime":"2024-09-10T21:36:11:958189+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3895211076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6835","nonce":3895211076},{"type":"v1","addr":"172.21.15.194:6837","nonce":3895211076}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346},{"gid":9685,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6836/299359831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":299359831},{"type":"v1","addr":"172.21.15.194:6836","nonce":299359831}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":339},{"gid":10626,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/3443330181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3443330181},{"type":"v1","addr":"172.21.15.97:6834","nonce":3443330181}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":338},{"gid":11127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3912200626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":3912200626},{"type":"v1","addr":"172.21.15.97:6835","nonce":3912200626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":352}],"filesystems":[]} 2024-09-10T21:36:25.107 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 352 2024-09-10T21:36:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88100725e0 msgr2=0x7f8810074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88100725e0 0x7f8810074aa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f88200af000 tx=0x7f88200195c0 comp rx=0 tx=0).stop 2024-09-10T21:36:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 msgr2=0x7f88300cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cb1e0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f882800cb10 tx=0x7f882800cc00 comp rx=0 tx=0).stop 2024-09-10T21:36:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 shutdown_connections 2024-09-10T21:36:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:6808/751391564,v1:172.21.15.194:6809/751391564] conn(0x7f88100725e0 0x7f8810074aa0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:25.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88300ce640 0x7f88300cb1e0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88300b0d50 0x7f8830152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 --2- 172.21.15.97:0/2560103655 >> [v2:172.21.15.194:3301/0,v1:172.21.15.194:6790/0] conn(0x7f88300af300 0x7f883014b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 >> 172.21.15.97:0/2560103655 conn(0x7f88300d1c30 msgr2=0x7f8830068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 shutdown_connections 2024-09-10T21:36:25.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-10T21:36:25.108+0000 7f8836c00640 1 -- 172.21.15.97:0/2560103655 wait complete. 2024-09-10T21:36:25.128 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T21:36:25.128 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T21:36:25.129 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:36:25.129 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:25.142 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:36:25.142 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:36:25.142 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:25.195 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:36:25.195 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:36:25.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:25.209 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:36:25.209 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:36:25.209 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:25.264 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:36:25.264 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T21:36:25.264 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T21:36:25.264 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:25.363 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T21:36:25.363 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T21:36:25.363 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:31.465 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T21:36:31.465 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T21:36:31.465 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:31.550 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T21:36:31.550 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T21:36:31.550 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:31.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-10T21:36:37.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-10T21:36:37.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-10T21:36:37.652 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T21:36:37.653 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T21:36:37.653 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:37.725 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T21:36:37.726 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T21:36:37.726 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:40.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:40.074 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:43.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-10T21:36:43.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-10T21:36:43.827 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T21:36:43.827 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T21:36:43.827 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:43.898 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T21:36:43.898 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T21:36:43.898 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:48.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-10T21:36:48.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-10T21:36:50.000 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T21:36:50.001 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T21:36:50.001 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T21:36:50.001 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:50.061 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T21:36:50.061 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T21:36:50.061 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:50.122 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T21:36:50.122 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T21:36:50.122 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:36:50.122 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:50.143 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:36:50.144 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:36:50.144 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:50.160 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:36:50.160 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:36:50.160 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:50.223 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:36:50.223 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T21:36:50.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | egrep -v 'MDS abort because newly corrupt dentry' | head -n 1 2024-09-10T21:36:50.278 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:36:50.278 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T21:36:50.452 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:36:50.453 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T21:36:50.633 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:36:50.633 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T21:36:50.855 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:36:50.856 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T21:36:51.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:36:51.026 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T21:36:51.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:36:51.235 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T21:36:51.435 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:36:51.436 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T21:36:51.637 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:36:51.638 DEBUG:teuthology.orchestra.run.smithi194:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T21:36:51.861 INFO:tasks.ceph:Archiving mon data... 2024-09-10T21:36:51.862 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/data/mon.a.tgz 2024-09-10T21:36:51.863 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-10T21:36:51.872 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.ruiamocBNt 2024-09-10T21:36:51.872 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ruiamocBNt 2024-09-10T21:36:52.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.ruiamocBNt 2024-09-10T21:36:52.139 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.ruiamocBNt is 658KB 2024-09-10T21:36:52.206 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.ruiamocBNt 2024-09-10T21:36:52.214 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/data/mon.b.tgz 2024-09-10T21:36:52.214 DEBUG:teuthology.orchestra.run.smithi194:> mktemp 2024-09-10T21:36:52.224 INFO:teuthology.orchestra.run.smithi194.stdout:/tmp/tmp.7aEthSa3VA 2024-09-10T21:36:52.224 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7aEthSa3VA 2024-09-10T21:36:52.418 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0666 /tmp/tmp.7aEthSa3VA 2024-09-10T21:36:52.490 DEBUG:teuthology.orchestra.remote:smithi194:/tmp/tmp.7aEthSa3VA is 683KB 2024-09-10T21:36:52.557 DEBUG:teuthology.orchestra.run.smithi194:> rm -fr /tmp/tmp.7aEthSa3VA 2024-09-10T21:36:52.565 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/data/mon.c.tgz 2024-09-10T21:36:52.565 DEBUG:teuthology.orchestra.run.smithi194:> mktemp 2024-09-10T21:36:52.618 INFO:teuthology.orchestra.run.smithi194.stdout:/tmp/tmp.4mfpYBGQT9 2024-09-10T21:36:52.618 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4mfpYBGQT9 2024-09-10T21:36:52.808 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0666 /tmp/tmp.4mfpYBGQT9 2024-09-10T21:36:52.882 DEBUG:teuthology.orchestra.remote:smithi194:/tmp/tmp.4mfpYBGQT9 is 683KB 2024-09-10T21:36:52.949 DEBUG:teuthology.orchestra.run.smithi194:> rm -fr /tmp/tmp.4mfpYBGQT9 2024-09-10T21:36:52.957 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T21:36:52.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:36:52.960 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:36:53.027 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:36:53.070 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:36:53.070 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T21:36:53.072 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/remote/smithi097/crash 2024-09-10T21:36:53.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:36:53.101 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/remote/smithi194/crash 2024-09-10T21:36:53.102 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:36:53.154 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T21:36:53.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:53.158 DEBUG:teuthology.orchestra.run.smithi194:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:53.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:53.255 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:53.274 INFO:tasks.ceph:Compressing logs... 2024-09-10T21:36:53.274 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:36:53.305 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:36:53.322 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35536.log 2024-09-10T21:36:53.322 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46123.log 2024-09-10T21:36:53.323 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60831.log 2024-09-10T21:36:53.324 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35536.log: /var/log/ceph/ceph-client.admin.46123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43815.log 2024-09-10T21:36:53.324 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46123.log.gz 2024-09-10T21:36:53.325 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35536.log.gz 2024-09-10T21:36:53.325 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52977.log 2024-09-10T21:36:53.325 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60831.log.gz 2024-09-10T21:36:53.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40924.log 2024-09-10T21:36:53.326 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43815.log.gz 2024-09-10T21:36:53.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 2024-09-10T21:36:53.327 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52977.log.gz 2024-09-10T21:36:53.328 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52700.log 2024-09-10T21:36:53.328 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40924.log.gz 2024-09-10T21:36:53.328 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56239.log 2024-09-10T21:36:53.328 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-09-10T21:36:53.329 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39768.log 2024-09-10T21:36:53.329 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52700.log.gz 2024-09-10T21:36:53.330 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60600.log 2024-09-10T21:36:53.330 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56239.log.gz 2024-09-10T21:36:53.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52474.log 2024-09-10T21:36:53.331 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39768.log.gz 2024-09-10T21:36:53.332 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52209.log 2024-09-10T21:36:53.332 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60600.log.gz 2024-09-10T21:36:53.332 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52508.log 2024-09-10T21:36:53.335 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29010.log 2024-09-10T21:36:53.335 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29113.log 2024-09-10T21:36:53.335 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.29010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28962.log 2024-09-10T21:36:53.335 INFO:teuthology.orchestra.run.smithi194.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29010.log.gz 2024-09-10T21:36:53.335 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.29113.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T21:36:53.336 INFO:teuthology.orchestra.run.smithi194.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29113.log.gz 2024-09-10T21:36:53.336 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.28962.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T21:36:53.336 INFO:teuthology.orchestra.run.smithi194.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28962.log.gz 2024-09-10T21:36:53.336 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29034.log 2024-09-10T21:36:53.337 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52474.log.gz 2024-09-10T21:36:53.337 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54398.log 2024-09-10T21:36:53.337 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52209.log.gz 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58110.log 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54398.log.gz 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.52508.log.gz 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48942.log 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58110.log.gz 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51694.log 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49211.log 2024-09-10T21:36:53.338 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48942.log.gz 2024-09-10T21:36:53.339 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-09-10T21:36:53.339 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51694.log.gz 2024-09-10T21:36:53.344 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29089.log 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.29034.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29034.log.gz 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.29089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34650.log 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29089.log.gz 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29058.log 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.1.34650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34650.log.gz 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:36:53.345 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.29058.log: /var/log/ceph/ceph-mgr.x.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29058.log.gz 2024-09-10T21:36:53.346 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28986.log 2024-09-10T21:36:53.346 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29137.log 2024-09-10T21:36:53.346 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.28986.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T21:36:53.346 INFO:teuthology.orchestra.run.smithi194.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28986.log.gz 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55976.log 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49211.log.gz 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57302.log 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39664.log 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55976.log.gz 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-09-10T21:36:53.348 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57302.log.gz 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54277.log 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39664.log.gz 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58497.log 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54277.log.gz 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52643.log 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48527.log 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58497.log.gz 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52643.log.gz 2024-09-10T21:36:53.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58048.log 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48527.log.gz 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60425.log 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49997.log 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58048.log.gz 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51134.log 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60425.log.gz 2024-09-10T21:36:53.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35432.log 2024-09-10T21:36:53.351 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.admin.29137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34668.log 2024-09-10T21:36:53.351 INFO:teuthology.orchestra.run.smithi194.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29137.log.gz 2024-09-10T21:36:53.351 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:36:53.351 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.1.34668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34668.log.gz 2024-09-10T21:36:53.351 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph.audit.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:36:53.352 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T21:36:53.352 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T21:36:53.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43901.log 2024-09-10T21:36:53.354 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51134.log.gz 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.49997.log.gz 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55025.log 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35432.log.gz 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55541.log 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43901.log.gz 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55025.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.55025.log.gz -5 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57835.log 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45200.log 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55541.log.gz 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48178.log 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57835.log.gz 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38354.log 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45200.log.gz 2024-09-10T21:36:53.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28992.log 2024-09-10T21:36:53.356 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48178.log.gz 2024-09-10T21:36:53.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35380.log 2024-09-10T21:36:53.357 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T21:36:53.357 INFO:teuthology.orchestra.run.smithi194.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:36:53.357 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T21:36:53.358 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38354.log.gz 2024-09-10T21:36:53.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42517.log 2024-09-10T21:36:53.358 INFO:teuthology.orchestra.run.smithi097.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28992.log.gz 2024-09-10T21:36:53.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53628.log 2024-09-10T21:36:53.358 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35380.log.gz 2024-09-10T21:36:53.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.42517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61065.log 2024-09-10T21:36:53.359 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42517.log.gz 2024-09-10T21:36:53.359 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53628.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T21:36:53.359 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53628.log.gz 2024-09-10T21:36:53.359 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47385.log 2024-09-10T21:36:53.359 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61065.log.gz 2024-09-10T21:36:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41854.log 2024-09-10T21:36:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49660.log 2024-09-10T21:36:53.360 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47385.log.gz 2024-09-10T21:36:53.360 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46923.log 2024-09-10T21:36:53.361 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41854.log.gz 2024-09-10T21:36:53.361 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51329.log 2024-09-10T21:36:53.361 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49660.log.gz 2024-09-10T21:36:53.362 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49329.log 2024-09-10T21:36:53.362 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46923.log.gz 2024-09-10T21:36:53.362 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54518.log 2024-09-10T21:36:53.363 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51329.log.gz 2024-09-10T21:36:53.363 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60548.log 2024-09-10T21:36:53.364 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54518.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49329.log.gz 2024-09-10T21:36:53.364 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36366.log 2024-09-10T21:36:53.364 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54518.log.gz 2024-09-10T21:36:53.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36573.log 2024-09-10T21:36:53.365 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60548.log.gz 2024-09-10T21:36:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57450.log 2024-09-10T21:36:53.367 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36366.log.gz 94.6% 2024-09-10T21:36:53.367 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T21:36:53.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48153.log 2024-09-10T21:36:53.367 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36573.log.gz 2024-09-10T21:36:53.368 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34752.log 2024-09-10T21:36:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47011.log 2024-09-10T21:36:53.369 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57450.log.gz 2024-09-10T21:36:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55125.log 2024-09-10T21:36:53.369 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48153.log.gz 2024-09-10T21:36:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46201.log 2024-09-10T21:36:53.370 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47011.log.gz 2024-09-10T21:36:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44722.log 2024-09-10T21:36:53.370 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55125.log.gz 2024-09-10T21:36:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48479.log 2024-09-10T21:36:53.371 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46201.log.gz 2024-09-10T21:36:53.372 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34757.log 2024-09-10T21:36:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29040.log 2024-09-10T21:36:53.372 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44722.log.gz 2024-09-10T21:36:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55151.log 2024-09-10T21:36:53.372 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48479.log.gz 2024-09-10T21:36:53.373 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.3.34752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34752.log.gz 2024-09-10T21:36:53.374 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47590.log 2024-09-10T21:36:53.374 INFO:teuthology.orchestra.run.smithi097.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29040.log.gz 2024-09-10T21:36:53.374 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44265.log 2024-09-10T21:36:53.374 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55151.log.gz 2024-09-10T21:36:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44291.log 2024-09-10T21:36:53.375 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47590.log.gz 2024-09-10T21:36:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58972.log 2024-09-10T21:36:53.376 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44265.log.gz 2024-09-10T21:36:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2024-09-10T21:36:53.376 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/ceph-client.3.34757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34757.log.gz 2024-09-10T21:36:53.377 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44291.log.gz 2024-09-10T21:36:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57813.log 2024-09-10T21:36:53.377 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58972.log.gz 2024-09-10T21:36:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-09-10T21:36:53.378 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2024-09-10T21:36:53.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60935.log 2024-09-10T21:36:53.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49598.log 2024-09-10T21:36:53.379 INFO:teuthology.orchestra.run.smithi097.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-09-10T21:36:53.379 INFO:teuthology.orchestra.run.smithi097.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.57813.log.gz 2024-09-10T21:36:53.379 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2024-09-10T21:36:53.380 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60935.log.gz 2024-09-10T21:36:53.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46857.log 2024-09-10T21:36:53.381 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49598.log.gz 2024-09-10T21:36:53.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47407.log 2024-09-10T21:36:53.381 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2024-09-10T21:36:53.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38436.log 2024-09-10T21:36:53.382 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46857.log.gz 2024-09-10T21:36:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36845.log 2024-09-10T21:36:53.383 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47407.log.gz 2024-09-10T21:36:53.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46813.log 2024-09-10T21:36:53.383 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38436.log.gz 2024-09-10T21:36:53.384 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36845.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36845.log.gz 2024-09-10T21:36:53.384 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-09-10T21:36:53.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58744.log 2024-09-10T21:36:53.385 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46813.log.gz 2024-09-10T21:36:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51839.log 2024-09-10T21:36:53.386 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2024-09-10T21:36:53.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51584.log 2024-09-10T21:36:53.386 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58744.log.gz 2024-09-10T21:36:53.387 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56752.log 2024-09-10T21:36:53.387 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51839.log.gz 2024-09-10T21:36:53.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58367.log 2024-09-10T21:36:53.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58601.log 2024-09-10T21:36:53.389 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56752.log.gz 2024-09-10T21:36:53.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58367.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.51584.log.gz 2024-09-10T21:36:53.389 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53547.log 2024-09-10T21:36:53.390 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58367.log.gz 2024-09-10T21:36:53.390 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58601.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T21:36:53.390 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58601.log.gz 2024-09-10T21:36:53.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41279.log 2024-09-10T21:36:53.391 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53547.log.gz 2024-09-10T21:36:53.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38289.log 2024-09-10T21:36:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36376.log 2024-09-10T21:36:53.393 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41279.log.gz 2024-09-10T21:36:53.393 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36354.log 2024-09-10T21:36:53.394 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38289.log.gz 2024-09-10T21:36:53.394 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58296.log 2024-09-10T21:36:53.395 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36376.log.gz 2024-09-10T21:36:53.395 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55490.log 2024-09-10T21:36:53.395 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36354.log.gz 2024-09-10T21:36:53.396 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-09-10T21:36:53.396 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58296.log.gz 2024-09-10T21:36:53.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.37264.log 2024-09-10T21:36:53.397 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55490.log.gz 2024-09-10T21:36:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz -5 2024-09-10T21:36:53.398 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39813.log 2024-09-10T21:36:53.398 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.2.37264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37725.log 2024-09-10T21:36:53.398 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.37264.log.gz 2024-09-10T21:36:53.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53883.log 2024-09-10T21:36:53.399 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39813.log.gz 2024-09-10T21:36:53.400 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52050.log 2024-09-10T21:36:53.400 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37725.log.gz 2024-09-10T21:36:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-09-10T21:36:53.401 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47767.log 2024-09-10T21:36:53.402 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.53883.log.gz 2024-09-10T21:36:53.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44079.log 2024-09-10T21:36:53.403 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.52050.log.gz 2024-09-10T21:36:53.403 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-09-10T21:36:53.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37131.log 2024-09-10T21:36:53.403 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47767.log.gz 2024-09-10T21:36:53.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40483.log 2024-09-10T21:36:53.404 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44079.log.gz 2024-09-10T21:36:53.405 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.37131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52745.log 2024-09-10T21:36:53.405 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37131.log.gz 2024-09-10T21:36:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45909.log 2024-09-10T21:36:53.406 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40483.log.gz 2024-09-10T21:36:53.406 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56644.log 2024-09-10T21:36:53.407 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52745.log.gz 2024-09-10T21:36:53.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53804.log 2024-09-10T21:36:53.408 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45909.log.gz 2024-09-10T21:36:53.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46548.log 2024-09-10T21:36:53.409 INFO:teuthology.orchestra.run.smithi097.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56644.log.gz 2024-09-10T21:36:53.409 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47912.log 2024-09-10T21:36:53.409 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53804.log.gz 2024-09-10T21:36:53.410 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59361.log 2024-09-10T21:36:53.410 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46548.log.gz 2024-09-10T21:36:53.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55383.log 2024-09-10T21:36:53.411 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47912.log.gz 2024-09-10T21:36:53.412 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61091.log 2024-09-10T21:36:53.412 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59361.log.gz 2024-09-10T21:36:53.412 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54037.log 2024-09-10T21:36:53.413 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55383.log.gz 2024-09-10T21:36:53.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57269.log 2024-09-10T21:36:53.414 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61091.log.gz 2024-09-10T21:36:53.414 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58244.log 2024-09-10T21:36:53.414 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54037.log.gz 2024-09-10T21:36:53.415 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48686.log 2024-09-10T21:36:53.415 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57269.log.gz 2024-09-10T21:36:53.416 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39638.log 2024-09-10T21:36:53.416 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58244.log.gz 2024-09-10T21:36:53.417 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39586.log 2024-09-10T21:36:53.417 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48686.log.gz 2024-09-10T21:36:53.417 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58549.log 2024-09-10T21:36:53.418 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39638.log.gz 2024-09-10T21:36:53.418 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47253.log 2024-09-10T21:36:53.418 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39586.log.gz 2024-09-10T21:36:53.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38745.log 2024-09-10T21:36:53.419 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58549.log.gz 2024-09-10T21:36:53.420 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-09-10T21:36:53.420 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47253.log.gz 2024-09-10T21:36:53.421 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52617.log 2024-09-10T21:36:53.421 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38745.log.gz 2024-09-10T21:36:53.421 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2024-09-10T21:36:53.422 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-09-10T21:36:53.422 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57860.log 2024-09-10T21:36:53.423 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52617.log.gz 2024-09-10T21:36:53.423 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51064.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz 2024-09-10T21:36:53.423 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57425.log 2024-09-10T21:36:53.424 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35179.log 2024-09-10T21:36:53.425 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57860.log.gz 2024-09-10T21:36:53.425 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.57425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47982.log 2024-09-10T21:36:53.425 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57425.log.gz 2024-09-10T21:36:53.425 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53038.log 2024-09-10T21:36:53.426 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35179.log.gz 2024-09-10T21:36:53.426 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56968.log 2024-09-10T21:36:53.426 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47982.log.gz 2024-09-10T21:36:53.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47483.log 2024-09-10T21:36:53.427 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53038.log.gz 2024-09-10T21:36:53.428 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39329.log 2024-09-10T21:36:53.428 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56968.log.gz 2024-09-10T21:36:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55051.log 2024-09-10T21:36:53.429 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47483.log.gz 2024-09-10T21:36:53.429 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59506.log 2024-09-10T21:36:53.430 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.39329.log.gz 2024-09-10T21:36:53.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37445.log 2024-09-10T21:36:53.431 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55051.log.gz 2024-09-10T21:36:53.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59506.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.59506.log.gz --verbose 2024-09-10T21:36:53.431 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.49551.log 2024-09-10T21:36:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56853.log 2024-09-10T21:36:53.432 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37445.log.gz 2024-09-10T21:36:53.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41721.log 2024-09-10T21:36:53.434 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35484.log 2024-09-10T21:36:53.434 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2024-09-10T21:36:53.434 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56853.log.gz 2024-09-10T21:36:53.434 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43133.log 2024-09-10T21:36:53.434 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41721.log.gz 2024-09-10T21:36:53.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2024-09-10T21:36:53.435 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35484.log.gz 2024-09-10T21:36:53.436 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47275.log 2024-09-10T21:36:53.436 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43133.log.gz 2024-09-10T21:36:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47429.log 2024-09-10T21:36:53.437 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-09-10T21:36:53.437 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49189.log 2024-09-10T21:36:53.438 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47275.log.gz 2024-09-10T21:36:53.438 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42832.log 2024-09-10T21:36:53.439 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47429.log.gz 2024-09-10T21:36:53.439 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60961.log 2024-09-10T21:36:53.440 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2024-09-10T21:36:53.440 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42832.log.gz 2024-09-10T21:36:53.440 INFO:teuthology.orchestra.run.smithi097.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.49189.log.gz 2024-09-10T21:36:53.441 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55764.log 2024-09-10T21:36:53.441 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60961.log.gz 2024-09-10T21:36:53.442 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42936.log 2024-09-10T21:36:53.442 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2024-09-10T21:36:53.442 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48344.log 2024-09-10T21:36:53.443 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55764.log.gz 2024-09-10T21:36:53.443 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61013.log 2024-09-10T21:36:53.443 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42936.log.gz 2024-09-10T21:36:53.444 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57232.log 2024-09-10T21:36:53.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48968.log 2024-09-10T21:36:53.445 INFO:teuthology.orchestra.run.smithi097.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.48344.log.gz 2024-09-10T21:36:53.445 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61013.log.gz 2024-09-10T21:36:53.446 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53370.log 2024-09-10T21:36:53.446 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57232.log.gz 2024-09-10T21:36:53.446 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2024-09-10T21:36:53.447 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48968.log.gz 2024-09-10T21:36:53.447 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2024-09-10T21:36:53.448 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53370.log.gz 2024-09-10T21:36:53.448 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41906.log 2024-09-10T21:36:53.449 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2024-09-10T21:36:53.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49741.log 2024-09-10T21:36:53.449 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2024-09-10T21:36:53.450 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49089.log 2024-09-10T21:36:53.450 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41906.log.gz 2024-09-10T21:36:53.451 INFO:teuthology.orchestra.run.smithi194.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T21:36:53.452 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28908.log 2024-09-10T21:36:53.452 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49741.log.gz 2024-09-10T21:36:53.452 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36063.log 2024-09-10T21:36:53.452 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49089.log.gz 2024-09-10T21:36:53.453 INFO:teuthology.orchestra.run.smithi194.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T21:36:53.453 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46901.log 2024-09-10T21:36:53.453 INFO:teuthology.orchestra.run.smithi097.stderr: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.28908.log.gz 2024-09-10T21:36:53.454 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57948.log 2024-09-10T21:36:53.454 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36063.log.gz 2024-09-10T21:36:53.454 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40640.log 2024-09-10T21:36:53.454 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46901.log.gz 2024-09-10T21:36:53.455 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45883.log 2024-09-10T21:36:53.455 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57948.log.gz 2024-09-10T21:36:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38235.log 2024-09-10T21:36:53.456 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40640.log.gz 2024-09-10T21:36:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53270.log 2024-09-10T21:36:53.457 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45883.log.gz 2024-09-10T21:36:53.457 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36089.log 2024-09-10T21:36:53.457 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38235.log.gz 2024-09-10T21:36:53.458 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60371.log 2024-09-10T21:36:53.459 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53270.log.gz 2024-09-10T21:36:53.459 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-09-10T21:36:53.459 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36089.log.gz 2024-09-10T21:36:53.460 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35458.log 2024-09-10T21:36:53.460 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60371.log.gz 2024-09-10T21:36:53.461 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51171.log 2024-09-10T21:36:53.461 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-09-10T21:36:53.461 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-09-10T21:36:53.462 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35458.log.gz 2024-09-10T21:36:53.462 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48295.log 2024-09-10T21:36:53.462 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51171.log.gz 2024-09-10T21:36:53.463 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36161.log 2024-09-10T21:36:53.463 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-09-10T21:36:53.464 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-09-10T21:36:53.464 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48295.log.gz 2024-09-10T21:36:53.465 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43633.log 2024-09-10T21:36:53.465 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36161.log.gz 2024-09-10T21:36:53.465 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48008.log 2024-09-10T21:36:53.466 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52440.log 2024-09-10T21:36:53.466 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-09-10T21:36:53.467 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43633.log.gz 2024-09-10T21:36:53.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48008.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.48008.log.gz 2024-09-10T21:36:53.467 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43659.log 2024-09-10T21:36:53.468 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56002.log 2024-09-10T21:36:53.468 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52440.log.gz 2024-09-10T21:36:53.469 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50279.log 2024-09-10T21:36:53.469 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43659.log.gz 2024-09-10T21:36:53.470 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40349.log 2024-09-10T21:36:53.470 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56002.log.gz 2024-09-10T21:36:53.471 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56607.log 2024-09-10T21:36:53.471 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43206.log 2024-09-10T21:36:53.471 INFO:teuthology.orchestra.run.smithi097.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.50279.log.gz 2024-09-10T21:36:53.472 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40349.log.gz 2024-09-10T21:36:53.472 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35233.log 2024-09-10T21:36:53.473 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56607.log.gz 2024-09-10T21:36:53.473 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46681.log 2024-09-10T21:36:53.473 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43206.log.gz 2024-09-10T21:36:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50832.log 2024-09-10T21:36:53.474 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35233.log.gz 2024-09-10T21:36:53.475 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47319.log 2024-09-10T21:36:53.475 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50832.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.46681.log.gz 2024-09-10T21:36:53.476 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52261.log 2024-09-10T21:36:53.476 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50832.log.gz 2024-09-10T21:36:53.476 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45531.log 2024-09-10T21:36:53.476 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47319.log.gz 2024-09-10T21:36:53.477 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52261.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T21:36:53.478 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52261.log.gz 2024-09-10T21:36:53.478 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43068.log 2024-09-10T21:36:53.478 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz 2024-09-10T21:36:53.479 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60319.log 2024-09-10T21:36:53.480 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50130.log 2024-09-10T21:36:53.480 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43068.log.gz 2024-09-10T21:36:53.481 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39088.log 2024-09-10T21:36:53.481 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60319.log.gz 2024-09-10T21:36:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50130.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50130.log.gz 2024-09-10T21:36:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46504.log 2024-09-10T21:36:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.39088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39088.log.gz 2024-09-10T21:36:53.482 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-09-10T21:36:53.483 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49427.log 2024-09-10T21:36:53.484 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.46504.log.gz 2024-09-10T21:36:53.484 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36585.log 2024-09-10T21:36:53.485 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-09-10T21:36:53.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.0.49427.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36011.log 2024-09-10T21:36:53.485 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49427.log.gz 2024-09-10T21:36:53.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51631.log 2024-09-10T21:36:53.486 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36585.log.gz 2024-09-10T21:36:53.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36011.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36011.log.gz 2024-09-10T21:36:53.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41590.log 2024-09-10T21:36:53.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59602.log 2024-09-10T21:36:53.489 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51631.log.gz 2024-09-10T21:36:53.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53521.log 2024-09-10T21:36:53.489 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41590.log.gz 2024-09-10T21:36:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59602.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59602.log.gz 2024-09-10T21:36:53.490 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54606.log 2024-09-10T21:36:53.491 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35354.log 2024-09-10T21:36:53.491 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53521.log.gz 2024-09-10T21:36:53.492 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54606.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54606.log.gz -5 2024-09-10T21:36:53.492 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36565.log 2024-09-10T21:36:53.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40535.log 2024-09-10T21:36:53.493 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35354.log.gz 2024-09-10T21:36:53.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40509.log 2024-09-10T21:36:53.494 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36565.log.gz 2024-09-10T21:36:53.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58863.log 2024-09-10T21:36:53.495 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40535.log.gz 2024-09-10T21:36:53.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41962.log 2024-09-10T21:36:53.496 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40509.log.gz 2024-09-10T21:36:53.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36367.log 2024-09-10T21:36:53.497 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58863.log.gz 2024-09-10T21:36:53.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47341.log 2024-09-10T21:36:53.497 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41962.log.gz 2024-09-10T21:36:53.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43555.log 2024-09-10T21:36:53.498 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36367.log.gz 2024-09-10T21:36:53.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-09-10T21:36:53.499 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47341.log.gz 2024-09-10T21:36:53.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58627.log 2024-09-10T21:36:53.500 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43555.log.gz 2024-09-10T21:36:53.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50211.log 2024-09-10T21:36:53.501 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-10T21:36:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35772.log 2024-09-10T21:36:53.501 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58627.log.gz 2024-09-10T21:36:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57548.log 2024-09-10T21:36:53.502 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50211.log.gz 2024-09-10T21:36:53.503 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44617.log 2024-09-10T21:36:53.503 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35772.log.gz 2024-09-10T21:36:53.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51953.log 2024-09-10T21:36:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44617.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44617.log.gz 2024-09-10T21:36:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59128.log 2024-09-10T21:36:53.505 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.57548.log.gz 2024-09-10T21:36:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-09-10T21:36:53.506 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51953.log.gz 2024-09-10T21:36:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50019.log 2024-09-10T21:36:53.507 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59128.log.gz 2024-09-10T21:36:53.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53993.log 2024-09-10T21:36:53.508 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-09-10T21:36:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50019.log: gzip -5 --verbose 89.6% -- -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz /var/log/ceph/ceph-client.admin.50416.log 2024-09-10T21:36:53.509 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-10T21:36:53.509 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56209.log 2024-09-10T21:36:53.509 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53993.log.gz 2024-09-10T21:36:53.510 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-09-10T21:36:53.510 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-09-10T21:36:53.511 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60485.log 2024-09-10T21:36:53.511 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56209.log.gz 2024-09-10T21:36:53.512 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48391.log 2024-09-10T21:36:53.512 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-09-10T21:36:53.513 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60485.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60485.log.gz 2024-09-10T21:36:53.513 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T21:36:53.514 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39431.log 2024-09-10T21:36:53.514 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48391.log.gz 2024-09-10T21:36:53.515 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55357.log 2024-09-10T21:36:53.515 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.39431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39431.log.gz 2024-09-10T21:36:53.516 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-09-10T21:36:53.517 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55357.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55357.log.gz 2024-09-10T21:36:53.517 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38209.log 2024-09-10T21:36:53.518 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49379.log 2024-09-10T21:36:53.518 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-09-10T21:36:53.519 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58393.log 2024-09-10T21:36:53.519 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38209.log.gz 2024-09-10T21:36:53.520 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58920.log 2024-09-10T21:36:53.520 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49379.log.gz 2024-09-10T21:36:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40056.log 2024-09-10T21:36:53.521 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58393.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58393.log.gz 2024-09-10T21:36:53.522 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51927.log 2024-09-10T21:36:53.522 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58920.log.gz 2024-09-10T21:36:53.523 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51561.log 2024-09-10T21:36:53.523 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40056.log.gz 2024-09-10T21:36:53.524 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45226.log 2024-09-10T21:36:53.524 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51927.log: /var/log/ceph/ceph-client.admin.51561.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51927.log.gz 2024-09-10T21:36:53.525 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53573.log 2024-09-10T21:36:53.525 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51561.log.gz 2024-09-10T21:36:53.526 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35907.log 2024-09-10T21:36:53.526 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45226.log.gz 2024-09-10T21:36:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49070.log 2024-09-10T21:36:53.527 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53573.log.gz 2024-09-10T21:36:53.528 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47620.log 2024-09-10T21:36:53.528 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35907.log.gz 2024-09-10T21:36:53.529 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.49070.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.49070.log.gz 2024-09-10T21:36:53.529 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29034.log 2024-09-10T21:36:53.530 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50471.log 2024-09-10T21:36:53.530 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47620.log: /var/log/ceph/ceph-client.admin.29034.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47620.log.gz 2024-09-10T21:36:53.531 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57896.log 2024-09-10T21:36:53.531 INFO:teuthology.orchestra.run.smithi097.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29034.log.gz 2024-09-10T21:36:53.532 INFO:teuthology.orchestra.run.smithi194.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T21:36:53.532 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 87.5% 2024-09-10T21:36:53.532 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50471.log.gz 2024-09-10T21:36:53.533 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56887.log 2024-09-10T21:36:53.533 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57896.log.gz 2024-09-10T21:36:53.534 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38157.log 2024-09-10T21:36:53.534 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gz 2024-09-10T21:36:53.535 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56887.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56887.log.gz 2024-09-10T21:36:53.535 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56384.log 2024-09-10T21:36:53.536 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40786.log 2024-09-10T21:36:53.536 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38157.log: /var/log/ceph/ceph-client.admin.56384.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38157.log.gz 2024-09-10T21:36:53.537 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46658.log 2024-09-10T21:36:53.537 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56384.log.gz 2024-09-10T21:36:53.538 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47187.log 2024-09-10T21:36:53.538 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40786.log.gz 2024-09-10T21:36:53.539 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60574.log 2024-09-10T21:36:53.539 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46658.log.gz 2024-09-10T21:36:53.540 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45479.log 2024-09-10T21:36:53.540 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47187.log.gz 2024-09-10T21:36:53.541 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52311.log 2024-09-10T21:36:53.541 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60574.log.gz 2024-09-10T21:36:53.542 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.45479.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47528.log 2024-09-10T21:36:53.542 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45479.log.gz 2024-09-10T21:36:53.543 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35855.log 2024-09-10T21:36:53.543 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52311.log.gz 2024-09-10T21:36:53.544 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35799.log 2024-09-10T21:36:53.544 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47528.log.gz 2024-09-10T21:36:53.545 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59050.log 2024-09-10T21:36:53.545 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35855.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35855.log.gz 2024-09-10T21:36:53.546 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37288.log 2024-09-10T21:36:53.546 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35799.log.gz 2024-09-10T21:36:53.547 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59999.log 2024-09-10T21:36:53.547 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59050.log.gz 2024-09-10T21:36:53.548 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53860.log 2024-09-10T21:36:53.548 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37288.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37288.log.gz 2024-09-10T21:36:53.549 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46614.log 2024-09-10T21:36:53.549 INFO:teuthology.orchestra.run.smithi097.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59999.log.gz 2024-09-10T21:36:53.550 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47099.log 2024-09-10T21:36:53.550 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53860.log.gz 2024-09-10T21:36:53.551 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38991.log 2024-09-10T21:36:53.551 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47099.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46614.log.gz 2024-09-10T21:36:53.552 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51817.log 2024-09-10T21:36:53.552 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47099.log.gz 2024-09-10T21:36:53.553 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45505.log 2024-09-10T21:36:53.553 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38991.log.gz 2024-09-10T21:36:53.554 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44643.log 2024-09-10T21:36:53.555 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59155.log 2024-09-10T21:36:53.555 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45505.log.gz 2024-09-10T21:36:53.556 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.51817.log.gz 2024-09-10T21:36:53.556 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.44643.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46835.log 2024-09-10T21:36:53.557 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44643.log.gz 2024-09-10T21:36:53.558 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51371.log 2024-09-10T21:36:53.558 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59155.log.gz 2024-09-10T21:36:53.558 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2024-09-10T21:36:53.559 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51371.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46835.log.gz 2024-09-10T21:36:53.559 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44161.log 2024-09-10T21:36:53.561 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41190.log 2024-09-10T21:36:53.561 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.51371.log.gz 2024-09-10T21:36:53.561 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-09-10T21:36:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59876.log 2024-09-10T21:36:53.562 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44161.log.gz 2024-09-10T21:36:53.562 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59450.log 2024-09-10T21:36:53.563 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41190.log.gz 2024-09-10T21:36:53.564 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44422.log 2024-09-10T21:36:53.565 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59876.log.gz 2024-09-10T21:36:53.565 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37617.log 2024-09-10T21:36:53.565 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59450.log.gz 2024-09-10T21:36:53.566 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55946.log 2024-09-10T21:36:53.566 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44422.log.gz 2024-09-10T21:36:53.567 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38592.log 2024-09-10T21:36:53.567 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37617.log.gz 2024-09-10T21:36:53.568 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-09-10T21:36:53.568 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55946.log.gz 2024-09-10T21:36:53.568 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5/var/log/ceph/ceph-client.admin.38592.log: --verbose -- /var/log/ceph/ceph-client.0.59408.log 2024-09-10T21:36:53.569 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38592.log.gz 2024-09-10T21:36:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29059.log 2024-09-10T21:36:53.570 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-09-10T21:36:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.59408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59408.log.gz 2024-09-10T21:36:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45739.log 2024-09-10T21:36:53.572 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48623.log 2024-09-10T21:36:53.572 INFO:teuthology.orchestra.run.smithi097.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29059.log.gz 2024-09-10T21:36:53.573 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-09-10T21:36:53.573 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45739.log.gz 2024-09-10T21:36:53.574 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60397.log 2024-09-10T21:36:53.574 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48623.log.gz 2024-09-10T21:36:53.575 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44343.log 2024-09-10T21:36:53.575 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47363.log: /var/log/ceph/ceph-client.admin.60397.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-09-10T21:36:53.576 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45142.log 2024-09-10T21:36:53.576 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60397.log.gz 2024-09-10T21:36:53.577 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38052.log 2024-09-10T21:36:53.577 INFO:teuthology.orchestra.run.smithi097.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44343.log.gz 2024-09-10T21:36:53.578 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37393.log 2024-09-10T21:36:53.578 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45142.log: /var/log/ceph/ceph-client.admin.38052.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45142.log.gz 2024-09-10T21:36:53.579 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59181.log 2024-09-10T21:36:53.579 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38052.log.gz 2024-09-10T21:36:53.580 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-09-10T21:36:53.580 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37393.log.gz 2024-09-10T21:36:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2024-09-10T21:36:53.581 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59181.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59181.log.gz 2024-09-10T21:36:53.582 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52591.log 2024-09-10T21:36:53.582 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-09-10T21:36:53.583 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37803.log 2024-09-10T21:36:53.583 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2024-09-10T21:36:53.584 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38462.log 2024-09-10T21:36:53.584 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52591.log.gz 2024-09-10T21:36:53.585 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43291.log 2024-09-10T21:36:53.585 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37803.log.gz 2024-09-10T21:36:53.586 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54687.log 2024-09-10T21:36:53.586 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38462.log.gz 2024-09-10T21:36:53.588 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-09-10T21:36:53.588 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43291.log.gz 2024-09-10T21:36:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54687.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54687.log.gz 2024-09-10T21:36:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60883.log 2024-09-10T21:36:53.589 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45791.log 2024-09-10T21:36:53.590 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44050.log: /var/log/ceph/ceph-client.admin.60883.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-09-10T21:36:53.590 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36897.log 2024-09-10T21:36:53.590 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60883.log.gz 2024-09-10T21:36:53.591 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52027.log 2024-09-10T21:36:53.591 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45791.log.gz 2024-09-10T21:36:53.592 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49787.log 2024-09-10T21:36:53.592 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36897.log.gz 2024-09-10T21:36:53.593 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-09-10T21:36:53.593 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52027.log.gz 2024-09-10T21:36:53.594 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44832.log 2024-09-10T21:36:53.594 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49787.log.gz 2024-09-10T21:36:53.595 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37987.log 2024-09-10T21:36:53.595 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44550.log: /var/log/ceph/ceph-client.admin.44832.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-09-10T21:36:53.596 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47077.log 2024-09-10T21:36:53.596 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44832.log.gz 2024-09-10T21:36:53.597 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37777.log 2024-09-10T21:36:53.597 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37987.log.gz 2024-09-10T21:36:53.598 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.47077.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-09-10T21:36:53.599 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47077.log.gz 2024-09-10T21:36:53.599 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29066.log 2024-09-10T21:36:53.599 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37777.log.gz 2024-09-10T21:36:53.600 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60345.log 2024-09-10T21:36:53.601 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29066.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-09-10T21:36:53.601 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-09-10T21:36:53.602 INFO:teuthology.orchestra.run.smithi097.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29066.log.gz 2024-09-10T21:36:53.602 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40379.log 2024-09-10T21:36:53.603 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60345.log.gz 2024-09-10T21:36:53.603 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57514.log 2024-09-10T21:36:53.604 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-09-10T21:36:53.604 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41538.log 2024-09-10T21:36:53.605 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40379.log.gz 2024-09-10T21:36:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56676.log 2024-09-10T21:36:53.606 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57514.log.gz 2024-09-10T21:36:53.606 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50703.log 2024-09-10T21:36:53.607 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41538.log.gz 2024-09-10T21:36:53.607 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2024-09-10T21:36:53.608 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56676.log.gz 2024-09-10T21:36:53.609 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53164.log 2024-09-10T21:36:53.609 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50703.log.gz 2024-09-10T21:36:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37471.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-09-10T21:36:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57328.log 2024-09-10T21:36:53.610 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2024-09-10T21:36:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53164.log: /var/log/ceph/ceph-client.admin.57328.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53164.log.gz 2024-09-10T21:36:53.611 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54964.log 2024-09-10T21:36:53.612 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57328.log.gz 2024-09-10T21:36:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54710.log 2024-09-10T21:36:53.613 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2024-09-10T21:36:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58575.log 2024-09-10T21:36:53.614 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54964.log.gz 2024-09-10T21:36:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56362.log 2024-09-10T21:36:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35205.log 2024-09-10T21:36:53.616 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58575.log.gz 2024-09-10T21:36:53.616 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.54710.log.gz 2024-09-10T21:36:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.56362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50957.log 2024-09-10T21:36:53.617 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56362.log.gz 2024-09-10T21:36:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49573.log 2024-09-10T21:36:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35205.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35205.log.gz 2024-09-10T21:36:53.619 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49517.log 2024-09-10T21:36:53.619 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50957.log.gz 2024-09-10T21:36:53.620 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36584.log 2024-09-10T21:36:53.620 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49573.log.gz 2024-09-10T21:36:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42183.log 2024-09-10T21:36:53.621 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49517.log.gz 2024-09-10T21:36:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29088.log 2024-09-10T21:36:53.622 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36584.log.gz 2024-09-10T21:36:53.623 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49432.log 2024-09-10T21:36:53.623 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42183.log.gz 2024-09-10T21:36:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49634.log 2024-09-10T21:36:53.624 INFO:teuthology.orchestra.run.smithi097.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.29088.log.gz 2024-09-10T21:36:53.625 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.49432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49432.log.gz 2024-09-10T21:36:53.625 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.37259.log 2024-09-10T21:36:53.626 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46879.log 2024-09-10T21:36:53.626 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49634.log.gz 2024-09-10T21:36:53.627 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.2.37259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.37259.log.gz 2024-09-10T21:36:53.627 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59259.log 2024-09-10T21:36:53.630 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38380.log 2024-09-10T21:36:53.630 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46879.log: /var/log/ceph/ceph-client.admin.59259.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46879.log.gz 2024-09-10T21:36:53.630 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48214.log 2024-09-10T21:36:53.630 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59259.log.gz 2024-09-10T21:36:53.630 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38380.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38380.log.gz 2024-09-10T21:36:53.631 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29076.log 2024-09-10T21:36:53.631 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42105.log 2024-09-10T21:36:53.632 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29076.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48214.log.gz 2024-09-10T21:36:53.632 INFO:teuthology.orchestra.run.smithi097.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29076.log.gz 2024-09-10T21:36:53.632 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61143.log 2024-09-10T21:36:53.633 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59800.log 2024-09-10T21:36:53.634 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42105.log.gz 2024-09-10T21:36:53.635 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51009.log 2024-09-10T21:36:53.635 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61143.log.gz 2024-09-10T21:36:53.635 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59480.log 2024-09-10T21:36:53.636 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59800.log.gz 2024-09-10T21:36:53.637 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44597.log 2024-09-10T21:36:53.637 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51009.log.gz 2024-09-10T21:36:53.638 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59480.log: gzip -5 87.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.59480.log.gz 2024-09-10T21:36:53.638 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.50156.log 2024-09-10T21:36:53.639 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38828.log 2024-09-10T21:36:53.639 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.44597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44597.log.gz 2024-09-10T21:36:53.640 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50156.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.50156.log.gz --verbose 2024-09-10T21:36:53.640 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.46570.log 2024-09-10T21:36:53.641 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54067.log 2024-09-10T21:36:53.641 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38828.log: /var/log/ceph/ceph-client.admin.46570.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38828.log.gz 2024-09-10T21:36:53.642 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46526.log 2024-09-10T21:36:53.642 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46570.log.gz 2024-09-10T21:36:53.643 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54067.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.54067.log.gz 2024-09-10T21:36:53.643 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T21:36:53.644 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59660.log 2024-09-10T21:36:53.644 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46526.log: /var/log/ceph/ceph-osd.3.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46526.log.gz 2024-09-10T21:36:53.645 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37419.log 2024-09-10T21:36:53.646 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48089.log 2024-09-10T21:36:53.646 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59660.log.gz 2024-09-10T21:36:53.647 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57006.log 2024-09-10T21:36:53.647 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37419.log.gz 2024-09-10T21:36:53.648 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42261.log 2024-09-10T21:36:53.648 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48089.log.gz 2024-09-10T21:36:53.649 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43363.log 2024-09-10T21:36:53.649 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57006.log.gz 2024-09-10T21:36:53.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53905.log 2024-09-10T21:36:53.651 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42261.log.gz 2024-09-10T21:36:53.651 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37926.log 2024-09-10T21:36:53.652 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43363.log.gz 2024-09-10T21:36:53.652 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44213.log 2024-09-10T21:36:53.653 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53905.log.gz 2024-09-10T21:36:53.654 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46253.log 2024-09-10T21:36:53.654 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37926.log.gz 2024-09-10T21:36:53.654 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54793.log 2024-09-10T21:36:53.655 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44213.log.gz 2024-09-10T21:36:53.656 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36156.log 2024-09-10T21:36:53.657 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54793.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46253.log.gz 2024-09-10T21:36:53.657 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50921.log 2024-09-10T21:36:53.657 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54793.log.gz 2024-09-10T21:36:53.658 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59736.log 2024-09-10T21:36:53.658 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36156.log.gz 2024-09-10T21:36:53.659 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60779.log 2024-09-10T21:36:53.660 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50921.log.gz 2024-09-10T21:36:53.660 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44317.log 2024-09-10T21:36:53.660 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59736.log.gz 2024-09-10T21:36:53.661 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58211.log 2024-09-10T21:36:53.661 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60779.log.gz 2024-09-10T21:36:53.663 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59565.log 2024-09-10T21:36:53.663 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44317.log.gz 2024-09-10T21:36:53.663 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59788.log 2024-09-10T21:36:53.664 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58211.log.gz 2024-09-10T21:36:53.664 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52770.log 2024-09-10T21:36:53.665 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59565.log.gz 2024-09-10T21:36:53.666 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.59788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55257.log 2024-09-10T21:36:53.666 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59788.log.gz 2024-09-10T21:36:53.667 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35406.log 2024-09-10T21:36:53.667 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52770.log.gz 2024-09-10T21:36:53.668 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55453.log 2024-09-10T21:36:53.669 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55257.log.gz 2024-09-10T21:36:53.669 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53296.log 2024-09-10T21:36:53.669 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35406.log.gz 2024-09-10T21:36:53.670 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51274.log 2024-09-10T21:36:53.670 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55453.log.gz 2024-09-10T21:36:53.671 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43581.log 2024-09-10T21:36:53.672 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53296.log.gz 2024-09-10T21:36:53.672 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58340.log 2024-09-10T21:36:53.673 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51274.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51274.log.gz 2024-09-10T21:36:53.673 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38871.log 2024-09-10T21:36:53.674 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43581.log.gz 2024-09-10T21:36:53.675 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55221.log 2024-09-10T21:36:53.675 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58340.log.gz 2024-09-10T21:36:53.676 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45820.log 2024-09-10T21:36:53.676 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38871.log.gz 2024-09-10T21:36:53.677 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36152.log 2024-09-10T21:36:53.677 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55221.log.gz 2024-09-10T21:36:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45935.log 2024-09-10T21:36:53.678 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45820.log.gz 2024-09-10T21:36:53.679 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35959.log 2024-09-10T21:36:53.679 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36152.log.gz 2024-09-10T21:36:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-09-10T21:36:53.680 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45935.log.gz 2024-09-10T21:36:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44187.log 2024-09-10T21:36:53.682 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35959.log.gz 2024-09-10T21:36:53.682 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37367.log 2024-09-10T21:36:53.683 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-09-10T21:36:53.683 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54138.log 2024-09-10T21:36:53.683 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44187.log.gz 2024-09-10T21:36:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58946.log 2024-09-10T21:36:53.685 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37367.log.gz 2024-09-10T21:36:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54893.log 2024-09-10T21:36:53.686 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54138.log.gz 2024-09-10T21:36:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46747.log 2024-09-10T21:36:53.687 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58946.log.gz 2024-09-10T21:36:53.688 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49874.log 2024-09-10T21:36:53.688 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54893.log.gz 2024-09-10T21:36:53.689 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56921.log 2024-09-10T21:36:53.689 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46747.log.gz 2024-09-10T21:36:53.690 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50044.log 2024-09-10T21:36:53.690 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49874.log.gz 2024-09-10T21:36:53.691 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38715.log 2024-09-10T21:36:53.692 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60128.log 2024-09-10T21:36:53.692 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50044.log.gz 2024-09-10T21:36:53.693 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.56921.log.gz 2024-09-10T21:36:53.693 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.38715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38715.log.gz 2024-09-10T21:36:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46769.log 2024-09-10T21:36:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58809.log 2024-09-10T21:36:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46769.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60128.log.gz 2024-09-10T21:36:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54543.log 2024-09-10T21:36:53.696 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46769.log.gz 2024-09-10T21:36:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38261.log 2024-09-10T21:36:53.697 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58809.log.gz 2024-09-10T21:36:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-09-10T21:36:53.698 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54543.log.gz 2024-09-10T21:36:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60293.log 2024-09-10T21:36:53.699 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38261.log.gz 2024-09-10T21:36:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43339.log 2024-09-10T21:36:53.700 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-09-10T21:36:53.701 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35933.log 2024-09-10T21:36:53.701 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60293.log.gz 2024-09-10T21:36:53.702 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.43339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47231.log 2024-09-10T21:36:53.702 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43339.log.gz 2024-09-10T21:36:53.703 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45965.log 2024-09-10T21:36:53.703 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35933.log.gz 2024-09-10T21:36:53.704 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38131.log 2024-09-10T21:36:53.705 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47231.log.gz 2024-09-10T21:36:53.705 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37751.log 2024-09-10T21:36:53.705 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45965.log.gz 2024-09-10T21:36:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50081.log 2024-09-10T21:36:53.706 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38131.log.gz 2024-09-10T21:36:53.707 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41140.log 2024-09-10T21:36:53.708 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37751.log.gz 2024-09-10T21:36:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52001.log 2024-09-10T21:36:53.709 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50081.log.gz 2024-09-10T21:36:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38668.log 2024-09-10T21:36:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52001.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41140.log.gz 2024-09-10T21:36:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T21:36:53.711 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52001.log.gz 2024-09-10T21:36:53.712 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41564.log 2024-09-10T21:36:53.712 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38668.log.gz 2024-09-10T21:36:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41050.log 2024-09-10T21:36:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-09-10T21:36:53.714 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41564.log.gz 2024-09-10T21:36:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2024-09-10T21:36:53.715 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41050.log.gz 2024-09-10T21:36:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42363.log 2024-09-10T21:36:53.717 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz 2024-09-10T21:36:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-09-10T21:36:53.718 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2024-09-10T21:36:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.42363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37149.log 2024-09-10T21:36:53.719 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42363.log.gz 2024-09-10T21:36:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51454.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz 2024-09-10T21:36:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57143.log 2024-09-10T21:36:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.37149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37149.log.gz 2024-09-10T21:36:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40234.log 2024-09-10T21:36:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40561.log 2024-09-10T21:36:53.724 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57143.log.gz 2024-09-10T21:36:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40234.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40234.log.gz 2024-09-10T21:36:53.725 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40561.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.40561.log.gz 2024-09-10T21:36:53.726 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53064.log 2024-09-10T21:36:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56560.log 2024-09-10T21:36:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51222.log 2024-09-10T21:36:53.729 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53064.log.gz 2024-09-10T21:36:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56560.log: /var/log/ceph/ceph-client.admin.51222.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51222.log.gz 2024-09-10T21:36:53.731 INFO:teuthology.orchestra.run.smithi097.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.56560.log.gz 2024-09-10T21:36:53.731 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-09-10T21:36:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49715.log 2024-09-10T21:36:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41934.log 2024-09-10T21:36:53.733 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-09-10T21:36:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49715.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49715.log.gz 2024-09-10T21:36:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59952.log 2024-09-10T21:36:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41934.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41934.log.gz 2024-09-10T21:36:53.736 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59233.log 2024-09-10T21:36:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54428.log 2024-09-10T21:36:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59233.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59952.log.gz 2024-09-10T21:36:53.739 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59233.log.gz 2024-09-10T21:36:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54428.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54428.log.gz 2024-09-10T21:36:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52410.log 2024-09-10T21:36:53.741 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54819.log 2024-09-10T21:36:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55617.log 2024-09-10T21:36:53.743 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2024-09-10T21:36:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54819.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54819.log.gz 2024-09-10T21:36:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55617.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55617.log.gz 2024-09-10T21:36:53.745 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59285.log 2024-09-10T21:36:53.746 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36765.log 2024-09-10T21:36:53.748 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2024-09-10T21:36:53.748 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59285.log.gz 2024-09-10T21:36:53.749 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36765.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36765.log.gz 2024-09-10T21:36:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37829.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2024-09-10T21:36:53.750 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-09-10T21:36:53.752 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59630.log 2024-09-10T21:36:53.753 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52384.log 2024-09-10T21:36:53.753 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-09-10T21:36:53.754 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-09-10T21:36:53.754 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59630.log.gz 2024-09-10T21:36:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.52384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52384.log.gz 2024-09-10T21:36:53.755 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45427.log 2024-09-10T21:36:53.756 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36375.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-09-10T21:36:53.757 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39445.log 2024-09-10T21:36:53.758 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-09-10T21:36:53.758 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45427.log.gz 2024-09-10T21:36:53.759 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39445.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39445.log.gz 2024-09-10T21:36:53.760 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35052.log 2024-09-10T21:36:53.761 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51667.log 2024-09-10T21:36:53.761 INFO:teuthology.orchestra.run.smithi097.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-09-10T21:36:53.762 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35296.log 2024-09-10T21:36:53.762 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35052.log.gz 2024-09-10T21:36:53.763 INFO:teuthology.orchestra.run.smithi194.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T21:36:53.763 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51667.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51667.log.gz 2024-09-10T21:36:53.764 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35296.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35296.log.gz 2024-09-10T21:36:53.765 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-09-10T21:36:53.766 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47506.log 2024-09-10T21:36:53.767 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46227.log 2024-09-10T21:36:53.768 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-09-10T21:36:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47506.log: /var/log/ceph/ceph-client.admin.46227.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46227.log.gz 2024-09-10T21:36:53.770 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.47506.log.gz 2024-09-10T21:36:53.770 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40088.log 2024-09-10T21:36:53.771 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36563.log 2024-09-10T21:36:53.772 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39891.log 2024-09-10T21:36:53.773 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40088.log.gz 2024-09-10T21:36:53.773 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36563.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36563.log.gz 2024-09-10T21:36:53.774 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39891.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39891.log.gz 2024-09-10T21:36:53.775 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61117.log 2024-09-10T21:36:53.776 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38965.log 2024-09-10T21:36:53.777 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55428.log 2024-09-10T21:36:53.778 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61117.log.gz 2024-09-10T21:36:53.778 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41216.log 2024-09-10T21:36:53.778 INFO:teuthology.orchestra.run.smithi097.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38965.log.gz 2024-09-10T21:36:53.780 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55428.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55428.log.gz 2024-09-10T21:36:53.781 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41216.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41216.log.gz 2024-09-10T21:36:53.781 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38104.log 2024-09-10T21:36:53.782 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2024-09-10T21:36:53.784 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55794.log 2024-09-10T21:36:53.784 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38104.log.gz 2024-09-10T21:36:53.784 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44886.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2024-09-10T21:36:53.786 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55794.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55794.log.gz 2024-09-10T21:36:53.786 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58026.log 2024-09-10T21:36:53.787 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-09-10T21:36:53.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40323.log 2024-09-10T21:36:53.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42861.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.58026.log.gz 2024-09-10T21:36:53.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40323.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-09-10T21:36:53.793 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40323.log.gz 2024-09-10T21:36:53.793 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48266.log 2024-09-10T21:36:53.793 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57480.log 2024-09-10T21:36:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51535.log 2024-09-10T21:36:53.794 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48266.log.gz 2024-09-10T21:36:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50806.log 2024-09-10T21:36:53.795 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57480.log.gz 2024-09-10T21:36:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51535.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51535.log.gz 2024-09-10T21:36:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53209.log 2024-09-10T21:36:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50806.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50806.log.gz 2024-09-10T21:36:53.797 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55862.log 2024-09-10T21:36:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41307.log 2024-09-10T21:36:53.799 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53209.log.gz 2024-09-10T21:36:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55862.log: /var/log/ceph/ceph-client.admin.41307.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41307.log.gz 2024-09-10T21:36:53.801 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.55862.log.gz 2024-09-10T21:36:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2024-09-10T21:36:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55712.log 2024-09-10T21:36:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54863.log 2024-09-10T21:36:53.804 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-09-10T21:36:53.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-10T21:36:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55712.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55712.log.gz 2024-09-10T21:36:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54863.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54863.log.gz 2024-09-10T21:36:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40587.log 2024-09-10T21:36:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55669.log 2024-09-10T21:36:53.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46303.log 2024-09-10T21:36:53.809 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40587.log.gz 2024-09-10T21:36:53.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.55669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55669.log.gz 2024-09-10T21:36:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46303.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46303.log.gz 2024-09-10T21:36:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2024-09-10T21:36:53.812 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46411.log 2024-09-10T21:36:53.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37588.log 2024-09-10T21:36:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46411.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46411.log.gz 2024-09-10T21:36:53.815 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2024-09-10T21:36:53.815 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60909.log 2024-09-10T21:36:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37588.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37588.log.gz 2024-09-10T21:36:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36163.log 2024-09-10T21:36:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60909.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60909.log.gz 2024-09-10T21:36:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2024-09-10T21:36:53.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49303.log 2024-09-10T21:36:53.820 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36163.log.gz 2024-09-10T21:36:53.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53760.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2024-09-10T21:36:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49303.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49303.log.gz 2024-09-10T21:36:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46071.log 2024-09-10T21:36:53.824 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49483.log 2024-09-10T21:36:53.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35985.log 2024-09-10T21:36:53.825 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46071.log.gz 2024-09-10T21:36:53.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49686.log 2024-09-10T21:36:53.826 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49483.log.gz 2024-09-10T21:36:53.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35985.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35985.log.gz 2024-09-10T21:36:53.828 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41997.log 2024-09-10T21:36:53.828 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49686.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49686.log.gz 2024-09-10T21:36:53.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41754.log 2024-09-10T21:36:53.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50445.log 2024-09-10T21:36:53.830 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41997.log.gz 2024-09-10T21:36:53.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41754.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41754.log.gz 2024-09-10T21:36:53.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50445.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50445.log.gz 2024-09-10T21:36:53.833 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-09-10T21:36:53.834 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40023.log 2024-09-10T21:36:53.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52072.log 2024-09-10T21:36:53.836 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-09-10T21:36:53.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40023.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40023.log.gz 2024-09-10T21:36:53.838 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52072.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52072.log.gz 2024-09-10T21:36:53.838 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39865.log 2024-09-10T21:36:53.839 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45642.log 2024-09-10T21:36:53.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43094.log 2024-09-10T21:36:53.841 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39865.log.gz 2024-09-10T21:36:53.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45642.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45642.log.gz 2024-09-10T21:36:53.842 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40666.log 2024-09-10T21:36:53.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43094.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43094.log.gz 2024-09-10T21:36:53.843 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29329.log 2024-09-10T21:36:53.845 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39093.log 2024-09-10T21:36:53.845 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40666.log.gz/var/log/ceph/ceph-client.admin.29329.log: 2024-09-10T21:36:53.846 INFO:teuthology.orchestra.run.smithi097.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29329.log.gz 2024-09-10T21:36:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.39093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39093.log.gz 2024-09-10T21:36:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39690.log 2024-09-10T21:36:53.848 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:36:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60805.log 2024-09-10T21:36:53.850 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39690.log.gz 2024-09-10T21:36:53.850 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29382.log 2024-09-10T21:36:53.852 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60805.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60805.log.gz 2024-09-10T21:36:53.853 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48453.log 2024-09-10T21:36:53.853 INFO:teuthology.orchestra.run.smithi097.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29382.log.gz 2024-09-10T21:36:53.854 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45674.log 2024-09-10T21:36:53.855 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48453.log: 87.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:36:53.856 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48453.log.gz 2024-09-10T21:36:53.856 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43420.log 2024-09-10T21:36:53.856 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45674.log.gz 2024-09-10T21:36:53.857 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2024-09-10T21:36:53.858 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42512.log 2024-09-10T21:36:53.858 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43420.log.gz 2024-09-10T21:36:53.860 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43503.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2024-09-10T21:36:53.860 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.42512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42512.log.gz 2024-09-10T21:36:53.860 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59823.log 2024-09-10T21:36:53.861 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56057.log 2024-09-10T21:36:53.863 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45993.log 2024-09-10T21:36:53.863 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59823.log.gz 2024-09-10T21:36:53.864 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56057.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56057.log.gz 2024-09-10T21:36:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45993.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45993.log.gz 2024-09-10T21:36:53.865 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39496.log 2024-09-10T21:36:53.866 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51606.log 2024-09-10T21:36:53.867 INFO:teuthology.orchestra.run.smithi194.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T21:36:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42592.log 2024-09-10T21:36:53.869 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51606.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39496.log.gz 2024-09-10T21:36:53.869 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51606.log.gz 2024-09-10T21:36:53.870 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42592.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42592.log.gz 2024-09-10T21:36:53.870 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59783.log 2024-09-10T21:36:53.871 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58185.log 2024-09-10T21:36:53.873 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.59783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43737.log 2024-09-10T21:36:53.873 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59783.log.gz 2024-09-10T21:36:53.874 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58185.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58185.log.gz 2024-09-10T21:36:53.875 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43737.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43737.log.gz 2024-09-10T21:36:53.876 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40862.log 2024-09-10T21:36:53.877 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51768.log 2024-09-10T21:36:53.878 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35326.log 2024-09-10T21:36:53.878 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40862.log.gz 2024-09-10T21:36:53.879 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51768.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51768.log.gz 2024-09-10T21:36:53.880 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35326.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35326.log.gz 2024-09-10T21:36:53.881 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41441.log 2024-09-10T21:36:53.882 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49119.log 2024-09-10T21:36:53.883 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44753.log 2024-09-10T21:36:53.883 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41441.log.gz 2024-09-10T21:36:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58159.log 2024-09-10T21:36:53.885 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49119.log.gz 2024-09-10T21:36:53.885 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55327.log 2024-09-10T21:36:53.886 INFO:teuthology.orchestra.run.smithi097.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.44753.log.gz 2024-09-10T21:36:53.887 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58159.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58159.log.gz 2024-09-10T21:36:53.887 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36377.log 2024-09-10T21:36:53.888 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55327.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55327.log.gz 2024-09-10T21:36:53.888 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28881.log 2024-09-10T21:36:53.889 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48131.log 2024-09-10T21:36:53.890 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36377.log.gz 2024-09-10T21:36:53.891 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28881.log: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.28881.log.gz 2024-09-10T21:36:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48131.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.48131.log.gz 2024-09-10T21:36:53.893 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46175.log 2024-09-10T21:36:53.894 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60026.log 2024-09-10T21:36:53.896 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38319.log 2024-09-10T21:36:53.896 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46175.log.gz 2024-09-10T21:36:53.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60026.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60026.log.gz 2024-09-10T21:36:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38319.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38319.log.gz 2024-09-10T21:36:53.898 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-09-10T21:36:53.899 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60857.log 2024-09-10T21:36:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53930.log 2024-09-10T21:36:53.902 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60857.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-09-10T21:36:53.902 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60857.log.gz 2024-09-10T21:36:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53930.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53930.log.gz 2024-09-10T21:36:53.903 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54989.log 2024-09-10T21:36:53.905 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54632.log 2024-09-10T21:36:53.906 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38909.log 2024-09-10T21:36:53.906 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54989.log.gz 2024-09-10T21:36:53.907 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52832.log 2024-09-10T21:36:53.907 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54632.log.gz 2024-09-10T21:36:53.908 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38909.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38909.log.gz 2024-09-10T21:36:53.908 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39742.log 2024-09-10T21:36:53.909 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52832.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52832.log.gz 2024-09-10T21:36:53.910 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44912.log 2024-09-10T21:36:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42131.log 2024-09-10T21:36:53.911 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39742.log.gz 2024-09-10T21:36:53.912 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44912.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44912.log.gz 2024-09-10T21:36:53.913 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42131.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42131.log.gz 2024-09-10T21:36:53.914 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48240.log 2024-09-10T21:36:53.915 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51087.log 2024-09-10T21:36:53.917 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60675.log 2024-09-10T21:36:53.917 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48240.log.gz 2024-09-10T21:36:53.919 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51087.log: /var/log/ceph/ceph-client.admin.60675.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60675.log.gz 2024-09-10T21:36:53.919 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.51087.log.gz 2024-09-10T21:36:53.919 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54661.log 2024-09-10T21:36:53.920 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50755.log 2024-09-10T21:36:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55283.log 2024-09-10T21:36:53.922 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54661.log.gz 2024-09-10T21:36:53.922 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.50755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56028.log 2024-09-10T21:36:53.922 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50755.log.gz 2024-09-10T21:36:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55283.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55283.log.gz 2024-09-10T21:36:53.924 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.56028.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49812.log 2024-09-10T21:36:53.925 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56028.log.gz 2024-09-10T21:36:53.925 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56526.log 2024-09-10T21:36:53.926 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47646.log 2024-09-10T21:36:53.927 INFO:teuthology.orchestra.run.smithi097.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.49812.log.gz 2024-09-10T21:36:53.928 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56526.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56526.log.gz 2024-09-10T21:36:53.929 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47646.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47646.log.gz 2024-09-10T21:36:53.930 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48856.log 2024-09-10T21:36:53.931 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50627.log 2024-09-10T21:36:53.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47055.log 2024-09-10T21:36:53.933 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48856.log.gz 2024-09-10T21:36:53.933 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50627.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50627.log.gz 2024-09-10T21:36:53.934 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47055.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47055.log.gz 2024-09-10T21:36:53.935 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58837.log 2024-09-10T21:36:53.936 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40950.log 2024-09-10T21:36:53.937 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42413.log 2024-09-10T21:36:53.938 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58837.log.gz 2024-09-10T21:36:53.938 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40950.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40950.log.gz 2024-09-10T21:36:53.940 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42413.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42413.log.gz 2024-09-10T21:36:53.940 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37673.log 2024-09-10T21:36:53.941 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55196.log 2024-09-10T21:36:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60649.log 2024-09-10T21:36:53.943 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37673.log.gz 2024-09-10T21:36:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60204.log 2024-09-10T21:36:53.944 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55196.log.gz 2024-09-10T21:36:53.945 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60649.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60649.log.gz 2024-09-10T21:36:53.946 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.60204.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45765.log 2024-09-10T21:36:53.947 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60204.log.gz 2024-09-10T21:36:53.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50983.log 2024-09-10T21:36:53.948 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36164.log 2024-09-10T21:36:53.948 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45765.log.gz 2024-09-10T21:36:53.949 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50983.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50983.log.gz 2024-09-10T21:36:53.950 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36164.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36164.log.gz 2024-09-10T21:36:53.950 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52906.log 2024-09-10T21:36:53.952 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-09-10T21:36:53.953 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59850.log 2024-09-10T21:36:53.953 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52906.log.gz 2024-09-10T21:36:53.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45848.log 2024-09-10T21:36:53.954 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-09-10T21:36:53.955 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59850.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59850.log.gz 2024-09-10T21:36:53.956 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.45848.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58471.log 2024-09-10T21:36:53.957 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45848.log.gz 2024-09-10T21:36:53.957 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58445.log 2024-09-10T21:36:53.958 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50362.log 2024-09-10T21:36:53.959 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58471.log.gz 2024-09-10T21:36:53.959 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58445.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58445.log.gz 2024-09-10T21:36:53.960 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50362.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50362.log.gz 2024-09-10T21:36:53.961 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-09-10T21:36:53.962 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39470.log 2024-09-10T21:36:53.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40980.log 2024-09-10T21:36:53.964 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-09-10T21:36:53.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39470.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39470.log.gz 2024-09-10T21:36:53.966 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40980.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40980.log.gz 2024-09-10T21:36:53.966 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57977.log 2024-09-10T21:36:53.967 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55095.log 2024-09-10T21:36:53.968 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41512.log 2024-09-10T21:36:53.968 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57977.log.gz 2024-09-10T21:36:53.970 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55095.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55095.log.gz 2024-09-10T21:36:53.971 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41512.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41512.log.gz 2024-09-10T21:36:53.971 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40768.log 2024-09-10T21:36:53.972 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47553.log 2024-09-10T21:36:53.973 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.40768.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.57570.log 2024-09-10T21:36:53.974 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.0.40768.log.gz 2024-09-10T21:36:53.975 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-09-10T21:36:53.975 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47553.log.gz 2024-09-10T21:36:53.976 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-09-10T21:36:53.976 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57570.log.gz 2024-09-10T21:36:53.977 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36570.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-09-10T21:36:53.977 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52389.log 2024-09-10T21:36:53.978 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39041.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-09-10T21:36:53.978 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50237.log 2024-09-10T21:36:53.979 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.52389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58653.log 2024-09-10T21:36:53.980 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52389.log.gz 2024-09-10T21:36:53.981 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50237.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50237.log.gz 2024-09-10T21:36:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58653.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58653.log.gz 2024-09-10T21:36:53.982 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58998.log 2024-09-10T21:36:53.983 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56172.log 2024-09-10T21:36:53.985 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53967.log 2024-09-10T21:36:53.985 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58998.log.gz 2024-09-10T21:36:53.986 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56172.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56172.log.gz 2024-09-10T21:36:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53967.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53967.log.gz 2024-09-10T21:36:53.987 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56410.log 2024-09-10T21:36:53.988 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49236.log 2024-09-10T21:36:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54462.log 2024-09-10T21:36:53.990 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49236.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56410.log.gz 2024-09-10T21:36:53.991 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49236.log.gz 2024-09-10T21:36:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54462.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54462.log.gz 2024-09-10T21:36:53.992 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-09-10T21:36:53.993 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49453.log 2024-09-10T21:36:53.995 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40405.log 2024-09-10T21:36:53.996 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49453.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49453.log.gz 2024-09-10T21:36:53.996 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-09-10T21:36:53.997 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40405.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40405.log.gz 2024-09-10T21:36:53.997 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59686.log 2024-09-10T21:36:53.999 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45031.log 2024-09-10T21:36:54.000 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41616.log 2024-09-10T21:36:54.001 INFO:teuthology.orchestra.run.smithi097.stderr: 91.3%/var/log/ceph/ceph-client.admin.45031.log: -- replaced with /var/log/ceph/ceph-client.admin.59686.log.gz 2024-09-10T21:36:54.001 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45031.log.gz 2024-09-10T21:36:54.002 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41616.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41616.log.gz 2024-09-10T21:36:54.002 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-09-10T21:36:54.004 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59529.log 2024-09-10T21:36:54.005 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39969.log 2024-09-10T21:36:54.005 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-09-10T21:36:54.006 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47209.log 2024-09-10T21:36:54.007 INFO:teuthology.orchestra.run.smithi097.stderr: 91.4%/var/log/ceph/ceph-client.admin.39969.log: -- replaced with /var/log/ceph/ceph-client.admin.59529.log.gz 2024-09-10T21:36:54.007 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39969.log.gz 2024-09-10T21:36:54.008 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38406.log 2024-09-10T21:36:54.009 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56083.log 2024-09-10T21:36:54.009 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47209.log.gz 2024-09-10T21:36:54.010 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46149.log 2024-09-10T21:36:54.010 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38406.log.gz 2024-09-10T21:36:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56083.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56083.log.gz 2024-09-10T21:36:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41880.log 2024-09-10T21:36:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46149.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46149.log.gz 2024-09-10T21:36:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-09-10T21:36:54.014 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-09-10T21:36:54.014 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41880.log.gz 2024-09-10T21:36:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36115.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-09-10T21:36:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48553.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-09-10T21:36:54.017 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35714.log 2024-09-10T21:36:54.018 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38078.log 2024-09-10T21:36:54.021 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38514.log 2024-09-10T21:36:54.021 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38078.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38078.log.gz 2024-09-10T21:36:54.022 INFO:teuthology.orchestra.run.smithi097.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.35714.log.gz 2024-09-10T21:36:54.022 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38514.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38514.log.gz 2024-09-10T21:36:54.022 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45609.log 2024-09-10T21:36:54.024 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40297.log 2024-09-10T21:36:54.025 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56367.log 2024-09-10T21:36:54.025 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45609.log.gz 2024-09-10T21:36:54.026 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55406.log 2024-09-10T21:36:54.026 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40297.log.gz 2024-09-10T21:36:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.56367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56367.log.gz 2024-09-10T21:36:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49273.log 2024-09-10T21:36:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59207.log 2024-09-10T21:36:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49273.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.55406.log.gz 2024-09-10T21:36:54.030 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49273.log.gz 2024-09-10T21:36:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59207.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59207.log.gz 2024-09-10T21:36:54.031 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50494.log 2024-09-10T21:36:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42988.log 2024-09-10T21:36:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-09-10T21:36:54.035 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42988.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42988.log.gz 2024-09-10T21:36:54.035 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.50494.log.gz 2024-09-10T21:36:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45304.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-09-10T21:36:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47748.log 2024-09-10T21:36:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50541.log 2024-09-10T21:36:54.038 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.47748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39379.log 2024-09-10T21:36:54.039 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47748.log.gz 2024-09-10T21:36:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50541.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50541.log.gz 2024-09-10T21:36:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39379.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39379.log.gz 2024-09-10T21:36:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36361.log 2024-09-10T21:36:54.042 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44135.log 2024-09-10T21:36:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57430.log 2024-09-10T21:36:54.044 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36361.log.gz 2024-09-10T21:36:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44135.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44135.log.gz 2024-09-10T21:36:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.57430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57430.log.gz 2024-09-10T21:36:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59413.log 2024-09-10T21:36:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55174.log 2024-09-10T21:36:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.59413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60052.log 2024-09-10T21:36:54.049 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59413.log.gz 2024-09-10T21:36:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55174.log: /var/log/ceph/ceph-client.admin.60052.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.55174.log.gz 2024-09-10T21:36:54.051 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60052.log.gz 2024-09-10T21:36:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52806.log 2024-09-10T21:36:54.053 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52723.log 2024-09-10T21:36:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60522.log 2024-09-10T21:36:54.055 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52806.log.gz 2024-09-10T21:36:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52723.log: /var/log/ceph/ceph-client.admin.60522.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60522.log.gz 2024-09-10T21:36:54.057 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.52723.log.gz 2024-09-10T21:36:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58523.log 2024-09-10T21:36:54.058 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37855.log 2024-09-10T21:36:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35153.log 2024-09-10T21:36:54.060 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58523.log.gz 2024-09-10T21:36:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37855.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37855.log.gz 2024-09-10T21:36:54.062 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35153.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35153.log.gz 2024-09-10T21:36:54.062 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36564.log 2024-09-10T21:36:54.063 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55828.log 2024-09-10T21:36:54.065 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29065.log 2024-09-10T21:36:54.065 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36564.log.gz 2024-09-10T21:36:54.065 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55828.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55828.log.gz 2024-09-10T21:36:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29065.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29065.log.gz 2024-09-10T21:36:54.067 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2024-09-10T21:36:54.068 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44448.log 2024-09-10T21:36:54.070 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57062.log 2024-09-10T21:36:54.070 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2024-09-10T21:36:54.071 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57117.log 2024-09-10T21:36:54.071 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44448.log.gz 2024-09-10T21:36:54.072 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57062.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57062.log.gz 2024-09-10T21:36:54.072 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51864.log 2024-09-10T21:36:54.073 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57117.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57117.log.gz 2024-09-10T21:36:54.073 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57088.log 2024-09-10T21:36:54.075 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57664.log 2024-09-10T21:36:54.075 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51864.log.gz 2024-09-10T21:36:54.076 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57088.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57088.log.gz 2024-09-10T21:36:54.077 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57664.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57664.log.gz 2024-09-10T21:36:54.077 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39112.log 2024-09-10T21:36:54.078 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40893.log 2024-09-10T21:36:54.080 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58679.log 2024-09-10T21:36:54.080 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39112.log.gz 2024-09-10T21:36:54.081 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40893.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40893.log.gz 2024-09-10T21:36:54.082 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58679.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58679.log.gz 2024-09-10T21:36:54.083 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51794.log 2024-09-10T21:36:54.083 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60102.log 2024-09-10T21:36:54.085 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35881.log 2024-09-10T21:36:54.085 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51794.log.gz 2024-09-10T21:36:54.086 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60102.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60102.log.gz 2024-09-10T21:36:54.087 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35881.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35881.log.gz 2024-09-10T21:36:54.088 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53187.log 2024-09-10T21:36:54.295 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42962.log 2024-09-10T21:36:54.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58712.log 2024-09-10T21:36:54.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42962.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42962.log.gz 2024-09-10T21:36:54.295 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.53187.log.gz 2024-09-10T21:36:54.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58712.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58712.log.gz 2024-09-10T21:36:54.295 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45356.log 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47143.log 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45356.log.gz 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47297.log 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47143.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47143.log.gz 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53466.log 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47297.log.gz 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr: 89.7%/var/log/ceph/ceph-client.admin.53466.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53466.log.gz 2024-09-10T21:36:54.296 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29084.log 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43016.log 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29084.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.29084.log.gz 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43016.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43016.log.gz 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42209.log 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50750.log 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42209.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42209.log.gz 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.50750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50750.log.gz 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60727.log 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44806.log 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60727.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60727.log.gz 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44806.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44806.log.gz 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-10T21:36:54.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41485.log 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39839.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41485.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41485.log.gz 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45557.log 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38540.log 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45557.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45557.log.gz 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38540.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38540.log.gz 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61039.log 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56125.log 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61039.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.61039.log.gz 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56125.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.56125.log.gz 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56315.log 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52555.log 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56315.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56315.log.gz 2024-09-10T21:36:54.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52555.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52555.log.gz 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52876.log 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52876.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52876.log.gz 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56702.log 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28833.log 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56702.log.gz 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36791.log 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28833.log: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.28833.log.gz 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54379.log 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz 2024-09-10T21:36:54.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36791.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36791.log.gz 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.54379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59102.log 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54379.log.gz 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59102.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59102.log.gz 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48786.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57745.log 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57745.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57745.log.gz 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43685.log 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35586.log 2024-09-10T21:36:54.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43685.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43685.log.gz 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35586.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35586.log.gz 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59076.log 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59076.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59076.log.gz 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45401.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37173.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57378.log 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57378.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57378.log.gz 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42023.log 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37699.log 2024-09-10T21:36:54.301 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42023.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42023.log.gz 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37699.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37699.log.gz 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36923.log 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36923.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36923.log.gz 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52178.log 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55884.log 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52178.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52178.log.gz 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55884.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55884.log.gz 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47121.log 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47121.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47121.log.gz 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35261.log 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35261.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35261.log.gz 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48321.log 2024-09-10T21:36:54.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48321.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48321.log.gz 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40153.log 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44474.log 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40153.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40153.log.gz 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44474.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44474.log.gz 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43344.log 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.43344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43344.log.gz 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29016.log 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29016.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29016.log.gz 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46351.log 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.46351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46351.log.gz 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54580.log 2024-09-10T21:36:54.303 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55567.log 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54580.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54580.log.gz 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40262.log 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55567.log.gz 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45085.log 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45085.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45085.log.gz 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53419.log 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40262.log.gz 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40205.log 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.53419.log.gz 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2024-09-10T21:36:54.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40205.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40205.log.gz 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.60987.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50578.log 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41006.log 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46356.log 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50578.log: /var/log/ceph/ceph-client.admin.41006.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41006.log.gz 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.46356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46356.log.gz 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50578.log.gz 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51901.log 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51901.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51901.log.gz 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45174.log 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45174.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45174.log.gz 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60261.log 2024-09-10T21:36:54.305 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60261.log.gz 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log/var/log/ceph/ceph-client.admin.51109.log: 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52134.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48034.log 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56462.log 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48034.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48034.log.gz 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49918.log/var/log/ceph/ceph-client.admin.56462.log: 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7%/var/log/ceph/ceph-client.admin.49918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46725.log 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56462.log.gz 2024-09-10T21:36:54.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35510.log 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46725.log.gz 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49065.log 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35510.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35510.log.gz 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.49065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49065.log.gz 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46791.log 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46791.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46791.log.gz 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40614.log 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40614.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40614.log.gz 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42383.log 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42383.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42383.log.gz 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50896.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54496.log 2024-09-10T21:36:54.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50516.log 2024-09-10T21:36:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54496.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.54496.log.gz 2024-09-10T21:36:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50516.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50516.log.gz 2024-09-10T21:36:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42358.log 2024-09-10T21:36:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.42358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42358.log.gz 2024-09-10T21:36:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55909.log 2024-09-10T21:36:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55909.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55909.log.gz 2024-09-10T21:36:54.308 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43933.log 2024-09-10T21:36:54.310 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43933.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43933.log.gz 2024-09-10T21:36:54.310 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49848.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48893.log 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48893.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48893.log.gz 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53108.log 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48730.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36169.log 2024-09-10T21:36:54.311 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53108.log.gz 2024-09-10T21:36:54.312 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36169.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36169.log.gz 2024-09-10T21:36:54.312 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41642.log 2024-09-10T21:36:54.314 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41642.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41642.log.gz 2024-09-10T21:36:54.314 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43841.log 2024-09-10T21:36:54.317 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43841.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43841.log.gz 2024-09-10T21:36:54.317 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44369.log 2024-09-10T21:36:54.321 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57632.log 2024-09-10T21:36:54.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44369.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44369.log.gz 2024-09-10T21:36:54.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58003.log 2024-09-10T21:36:54.322 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57632.log.gz 2024-09-10T21:36:54.324 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58003.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58003.log.gz 2024-09-10T21:36:54.324 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53340.log 2024-09-10T21:36:54.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53340.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53340.log.gz 2024-09-10T21:36:54.327 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45583.log 2024-09-10T21:36:54.330 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45583.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45583.log.gz 2024-09-10T21:36:54.330 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35829.log 2024-09-10T21:36:54.331 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44975.log 2024-09-10T21:36:54.333 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35829.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35829.log.gz 2024-09-10T21:36:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44975.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44975.log.gz 2024-09-10T21:36:54.335 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58892.log 2024-09-10T21:36:54.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58892.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58892.log.gz 2024-09-10T21:36:54.338 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53734.log 2024-09-10T21:36:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54919.log 2024-09-10T21:36:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53734.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53734.log.gz 2024-09-10T21:36:54.343 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54919.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54919.log.gz 2024-09-10T21:36:54.344 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47165.log 2024-09-10T21:36:54.347 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54200.log 2024-09-10T21:36:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61169.log 2024-09-10T21:36:54.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54200.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47165.log.gz 2024-09-10T21:36:54.351 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61169.log: -- /var/log/ceph/ceph-client.admin.43477.log 2024-09-10T21:36:54.352 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61169.log.gz 2024-09-10T21:36:54.352 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54200.log.gz 2024-09-10T21:36:54.353 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59926.log 2024-09-10T21:36:54.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43477.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-09-10T21:36:54.356 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37315.log 2024-09-10T21:36:54.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59926.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59926.log.gz 2024-09-10T21:36:54.361 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35085.log 2024-09-10T21:36:54.361 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37315.log.gz 2024-09-10T21:36:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38183.log 2024-09-10T21:36:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38183.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38183.log.gz 2024-09-10T21:36:54.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35085.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35085.log.gz 2024-09-10T21:36:54.368 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46045.log 2024-09-10T21:36:54.371 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46045.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46045.log.gz 2024-09-10T21:36:54.371 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51393.log 2024-09-10T21:36:54.374 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51393.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51393.log.gz 2024-09-10T21:36:54.374 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56492.log 2024-09-10T21:36:54.376 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56492.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56492.log.gz 2024-09-10T21:36:54.377 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52337.log 2024-09-10T21:36:54.379 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39917.log 2024-09-10T21:36:54.379 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52337.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52337.log.gz 2024-09-10T21:36:54.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39917.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39917.log.gz 2024-09-10T21:36:54.383 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57036.log 2024-09-10T21:36:54.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57036.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57036.log.gz 2024-09-10T21:36:54.385 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37647.log 2024-09-10T21:36:54.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37647.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37647.log.gz 2024-09-10T21:36:54.388 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52672.log 2024-09-10T21:36:54.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52672.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52672.log.gz 2024-09-10T21:36:54.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40431.log 2024-09-10T21:36:54.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28874.log 2024-09-10T21:36:54.394 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T21:36:54.394 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40431.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40431.log.gz 2024-09-10T21:36:54.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28874.log.gz 2024-09-10T21:36:54.397 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59024.log 2024-09-10T21:36:54.398 INFO:teuthology.orchestra.run.smithi097.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T21:36:54.398 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38775.log 2024-09-10T21:36:54.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59024.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59024.log.gz 2024-09-10T21:36:54.400 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38775.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38775.log.gz 2024-09-10T21:36:54.400 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43042.log 2024-09-10T21:36:54.402 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57185.log 2024-09-10T21:36:54.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52955.log 2024-09-10T21:36:54.403 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43042.log.gz 2024-09-10T21:36:54.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57185.log: /var/log/ceph/ceph-client.admin.52955.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.57185.log.gz 2024-09-10T21:36:54.408 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.52955.log.gz 2024-09-10T21:36:54.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57595.log 2024-09-10T21:36:54.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37341.log 2024-09-10T21:36:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53396.log 2024-09-10T21:36:54.409 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57595.log.gz 2024-09-10T21:36:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37341.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37341.log.gz 2024-09-10T21:36:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53396.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53396.log.gz 2024-09-10T21:36:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44602.log 2024-09-10T21:36:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-09-10T21:36:54.414 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.44602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39276.log 2024-09-10T21:36:54.414 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44602.log.gz 2024-09-10T21:36:54.415 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54942.log 2024-09-10T21:36:54.415 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-09-10T21:36:54.417 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56799.log 2024-09-10T21:36:54.417 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39276.log.gz 2024-09-10T21:36:54.418 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43789.log 2024-09-10T21:36:54.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.56799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56799.log.gz 2024-09-10T21:36:54.419 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.54942.log.gz 2024-09-10T21:36:54.420 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58270.log 2024-09-10T21:36:54.421 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51480.log 2024-09-10T21:36:54.421 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43789.log.gz 2024-09-10T21:36:54.422 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50182.log 2024-09-10T21:36:54.422 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58270.log.gz 2024-09-10T21:36:54.423 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51480.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51480.log.gz 2024-09-10T21:36:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50182.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50182.log.gz 2024-09-10T21:36:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48598.log 2024-09-10T21:36:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44860.log 2024-09-10T21:36:54.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54732.log 2024-09-10T21:36:54.427 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48598.log.gz 2024-09-10T21:36:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44860.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44860.log.gz 2024-09-10T21:36:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54732.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54732.log.gz 2024-09-10T21:36:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37900.log 2024-09-10T21:36:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51038.log 2024-09-10T21:36:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47797.log 2024-09-10T21:36:54.433 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37900.log.gz 2024-09-10T21:36:54.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37955.log 2024-09-10T21:36:54.433 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51038.log.gz 2024-09-10T21:36:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47797.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47797.log.gz 2024-09-10T21:36:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44395.log 2024-09-10T21:36:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37955.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37955.log.gz 2024-09-10T21:36:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28932.log 2024-09-10T21:36:54.437 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56147.log 2024-09-10T21:36:54.437 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44395.log.gz 2024-09-10T21:36:54.439 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28932.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.28932.log.gz 2024-09-10T21:36:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56147.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56147.log.gz 2024-09-10T21:36:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57207.log 2024-09-10T21:36:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44500.log 2024-09-10T21:36:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56582.log 2024-09-10T21:36:54.443 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57207.log.gz 2024-09-10T21:36:54.444 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44500.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-09-10T21:36:54.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56582.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56582.log.gz 2024-09-10T21:36:54.445 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43998.log 2024-09-10T21:36:54.446 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44777.log 2024-09-10T21:36:54.448 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42049.log 2024-09-10T21:36:54.448 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43998.log.gz 2024-09-10T21:36:54.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44777.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44777.log.gz 2024-09-10T21:36:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42049.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42049.log.gz 2024-09-10T21:36:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47743.log 2024-09-10T21:36:54.452 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42730.log 2024-09-10T21:36:54.453 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.47743.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.47743.log.gz -5 2024-09-10T21:36:54.453 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57922.log 2024-09-10T21:36:54.454 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50301.log 2024-09-10T21:36:54.454 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42730.log.gz 2024-09-10T21:36:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57922.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57922.log.gz 2024-09-10T21:36:54.456 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52530.log 2024-09-10T21:36:54.456 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50301.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50301.log.gz 2024-09-10T21:36:54.457 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43711.log 2024-09-10T21:36:54.458 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41242.log 2024-09-10T21:36:54.458 INFO:teuthology.orchestra.run.smithi097.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52530.log.gz 2024-09-10T21:36:54.459 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43711.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43711.log.gz 2024-09-10T21:36:54.459 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44239.log 2024-09-10T21:36:54.461 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41076.log 2024-09-10T21:36:54.461 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41242.log.gz 2024-09-10T21:36:54.462 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42157.log 2024-09-10T21:36:54.462 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44239.log.gz 2024-09-10T21:36:54.463 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41076.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41076.log.gz 2024-09-10T21:36:54.464 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42157.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42157.log.gz 2024-09-10T21:36:54.465 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45453.log 2024-09-10T21:36:54.466 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40716.log 2024-09-10T21:36:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36583.log 2024-09-10T21:36:54.467 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45453.log.gz 2024-09-10T21:36:54.468 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40716.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40716.log.gz 2024-09-10T21:36:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36583.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36583.log.gz 2024-09-10T21:36:54.469 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28920.log 2024-09-10T21:36:54.471 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34080.log 2024-09-10T21:36:54.472 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38618.log 2024-09-10T21:36:54.473 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28920.log.gz 2024-09-10T21:36:54.473 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34080.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34080.log.gz 2024-09-10T21:36:54.475 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38618.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38618.log.gz 2024-09-10T21:36:54.475 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48831.log 2024-09-10T21:36:54.477 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39226.log 2024-09-10T21:36:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39997.log 2024-09-10T21:36:54.478 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48831.log.gz 2024-09-10T21:36:54.478 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39138.log 2024-09-10T21:36:54.479 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39997.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.39226.log.gz 2024-09-10T21:36:54.480 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39997.log.gz 2024-09-10T21:36:54.481 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43763.log 2024-09-10T21:36:54.482 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54163.log 2024-09-10T21:36:54.482 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39138.log.gz 2024-09-10T21:36:54.483 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39716.log 2024-09-10T21:36:54.483 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43763.log.gz 2024-09-10T21:36:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54163.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54163.log.gz 2024-09-10T21:36:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50388.log 2024-09-10T21:36:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39716.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39716.log.gz 2024-09-10T21:36:54.485 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38720.log 2024-09-10T21:36:54.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41389.log 2024-09-10T21:36:54.487 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50388.log.gz 2024-09-10T21:36:54.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.38720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38720.log.gz 2024-09-10T21:36:54.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41389.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41389.log.gz 2024-09-10T21:36:54.490 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38935.log 2024-09-10T21:36:54.491 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54093.log 2024-09-10T21:36:54.492 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40763.log 2024-09-10T21:36:54.492 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38935.log.gz 2024-09-10T21:36:54.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54374.log 2024-09-10T21:36:54.493 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54093.log.gz 2024-09-10T21:36:54.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.40763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40763.log.gz 2024-09-10T21:36:54.495 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50874.log 2024-09-10T21:36:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.54374.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40836.log 2024-09-10T21:36:54.496 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.0.54374.log.gz 2024-09-10T21:36:54.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-09-10T21:36:54.498 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40836.log: 87.2% 93.0% -- replaced with /var/log/ceph/ceph-client.admin.50874.log.gz 2024-09-10T21:36:54.499 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40836.log.gz 2024-09-10T21:36:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-09-10T21:36:54.499 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-09-10T21:36:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53138.log 2024-09-10T21:36:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54251.log 2024-09-10T21:36:54.501 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz 2024-09-10T21:36:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-10T21:36:54.502 INFO:teuthology.orchestra.run.smithi097.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53138.log.gz 2024-09-10T21:36:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54251.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54251.log.gz 2024-09-10T21:36:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46967.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-09-10T21:36:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-09-10T21:36:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36949.log 2024-09-10T21:36:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54327.log 2024-09-10T21:36:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48760.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-09-10T21:36:54.509 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36949.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.36949.log.gz 2024-09-10T21:36:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54327.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54327.log.gz 2024-09-10T21:36:54.510 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56823.log 2024-09-10T21:36:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58073.log 2024-09-10T21:36:54.513 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46464.log 2024-09-10T21:36:54.513 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56823.log.gz 2024-09-10T21:36:54.514 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58073.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58073.log.gz 2024-09-10T21:36:54.516 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46464.log: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.46464.log.gz 2024-09-10T21:36:54.517 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53673.log 2024-09-10T21:36:54.518 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42463.log 2024-09-10T21:36:54.519 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52932.log 2024-09-10T21:36:54.520 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53673.log.gz 2024-09-10T21:36:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42463.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42463.log.gz 2024-09-10T21:36:54.521 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52932.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52932.log.gz 2024-09-10T21:36:54.522 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39426.log 2024-09-10T21:36:54.523 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36817.log 2024-09-10T21:36:54.524 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.39426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41668.log 2024-09-10T21:36:54.524 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39426.log.gz 2024-09-10T21:36:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36817.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36817.log.gz 2024-09-10T21:36:54.526 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41668.log: /var/log/ceph/ceph-client.admin.40179.log 2024-09-10T21:36:54.526 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41668.log.gz 2024-09-10T21:36:54.527 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28896.log 2024-09-10T21:36:54.528 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49018.log 2024-09-10T21:36:54.529 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40179.log.gz 2024-09-10T21:36:54.530 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.28896.log: 76.6% -- replaced with /var/log/ceph/ceph-client.admin.28896.log.gz 2024-09-10T21:36:54.531 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49018.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2024-09-10T21:36:54.531 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36370.log 2024-09-10T21:36:54.532 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50775.log 2024-09-10T21:36:54.534 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38488.log 2024-09-10T21:36:54.534 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36370.log.gz 2024-09-10T21:36:54.534 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50775.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50775.log.gz 2024-09-10T21:36:54.536 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38488.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38488.log.gz 2024-09-10T21:36:54.536 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47887.log 2024-09-10T21:36:54.537 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56943.log 2024-09-10T21:36:54.539 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38566.log 2024-09-10T21:36:54.539 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47887.log.gz 2024-09-10T21:36:54.540 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58419.log 2024-09-10T21:36:54.540 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56943.log.gz 2024-09-10T21:36:54.541 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38566.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38566.log.gz 2024-09-10T21:36:54.542 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51303.log 2024-09-10T21:36:54.542 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58419.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58419.log.gz 2024-09-10T21:36:54.542 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-09-10T21:36:54.544 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39943.log 2024-09-10T21:36:54.545 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51303.log.gz 2024-09-10T21:36:54.545 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46636.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz 2024-09-10T21:36:54.547 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39943.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39943.log.gz 2024-09-10T21:36:54.547 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46019.log 2024-09-10T21:36:54.548 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57716.log 2024-09-10T21:36:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61219.log 2024-09-10T21:36:54.550 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46019.log.gz 2024-09-10T21:36:54.550 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57716.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57716.log.gz 2024-09-10T21:36:54.551 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61219.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61219.log.gz 2024-09-10T21:36:54.552 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-09-10T21:36:54.553 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39612.log 2024-09-10T21:36:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49765.log 2024-09-10T21:36:54.555 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-09-10T21:36:54.555 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2024-09-10T21:36:54.556 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39612.log.gz 2024-09-10T21:36:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49765.log: /var/log/ceph/ceph-client.admin.46441.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2024-09-10T21:36:54.558 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.49765.log.gz 2024-09-10T21:36:54.558 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-09-10T21:36:54.559 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29012.log 2024-09-10T21:36:54.561 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48427.log 2024-09-10T21:36:54.561 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-09-10T21:36:54.561 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29012.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29012.log.gz 2024-09-10T21:36:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57690.log 2024-09-10T21:36:54.562 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48427.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48427.log.gz 2024-09-10T21:36:54.563 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59311.log 2024-09-10T21:36:54.565 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44949.log 2024-09-10T21:36:54.565 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57690.log.gz 2024-09-10T21:36:54.566 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59311.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59311.log.gz 2024-09-10T21:36:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44949.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44949.log.gz 2024-09-10T21:36:54.567 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54116.log 2024-09-10T21:36:54.568 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52097.log 2024-09-10T21:36:54.569 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42637.log 2024-09-10T21:36:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52097.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52097.log.gz 2024-09-10T21:36:54.571 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.54116.log.gz 2024-09-10T21:36:54.571 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-09-10T21:36:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42637.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42637.log.gz 2024-09-10T21:36:54.573 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39555.log 2024-09-10T21:36:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37015.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-09-10T21:36:54.574 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60451.log 2024-09-10T21:36:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53002.log 2024-09-10T21:36:54.576 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-09-10T21:36:54.576 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60451.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.60451.log.gz 2024-09-10T21:36:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53002.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53002.log.gz 2024-09-10T21:36:54.578 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42566.log 2024-09-10T21:36:54.579 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24008.log 2024-09-10T21:36:54.581 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2024-09-10T21:36:54.581 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42566.log.gz 2024-09-10T21:36:54.582 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.tmp-client.admin.24008.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24008.log.gz 2024-09-10T21:36:54.583 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44107.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2024-09-10T21:36:54.583 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49974.log 2024-09-10T21:36:54.584 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60178.log 2024-09-10T21:36:54.586 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29469.log 2024-09-10T21:36:54.586 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49974.log.gz 2024-09-10T21:36:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60178.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60178.log.gz 2024-09-10T21:36:54.589 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29469.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29469.log.gz 2024-09-10T21:36:54.589 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60701.log 2024-09-10T21:36:54.589 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41694.log 2024-09-10T21:36:54.591 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47831.log 2024-09-10T21:36:54.591 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60701.log.gz 2024-09-10T21:36:54.592 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41694.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41694.log.gz 2024-09-10T21:36:54.593 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47831.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47831.log.gz 2024-09-10T21:36:54.593 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55686.log 2024-09-10T21:36:54.594 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43607.log 2024-09-10T21:36:54.596 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53834.log 2024-09-10T21:36:54.596 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55686.log.gz 2024-09-10T21:36:54.597 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43607.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43607.log.gz 2024-09-10T21:36:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53834.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53834.log.gz 2024-09-10T21:36:54.599 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50653.log 2024-09-10T21:36:54.599 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61193.log 2024-09-10T21:36:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-09-10T21:36:54.602 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2%/var/log/ceph/ceph-client.admin.61193.log: -- replaced with /var/log/ceph/ceph-client.admin.50653.log.gz 2024-09-10T21:36:54.602 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61193.log.gz 2024-09-10T21:36:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41415.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-09-10T21:36:54.604 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57771.log 2024-09-10T21:36:54.605 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48063.log 2024-09-10T21:36:54.606 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39196.log 2024-09-10T21:36:54.607 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57771.log.gz 2024-09-10T21:36:54.607 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36154.log 2024-09-10T21:36:54.607 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48063.log.gz 2024-09-10T21:36:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39196.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39196.log.gz 2024-09-10T21:36:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56265.log 2024-09-10T21:36:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36154.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36154.log.gz 2024-09-10T21:36:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60753.log 2024-09-10T21:36:54.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56804.log 2024-09-10T21:36:54.613 INFO:teuthology.orchestra.run.smithi097.stderr: 91.3%/var/log/ceph/ceph-client.admin.60753.log: -- replaced with /var/log/ceph/ceph-client.admin.56265.log.gz 2024-09-10T21:36:54.613 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60753.log.gz 2024-09-10T21:36:54.614 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.56804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56804.log.gz 2024-09-10T21:36:54.614 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37557.log 2024-09-10T21:36:54.615 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55664.log 2024-09-10T21:36:54.616 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42235.log 2024-09-10T21:36:54.617 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37557.log.gz 2024-09-10T21:36:54.617 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.55664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55664.log.gz 2024-09-10T21:36:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42235.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42235.log.gz 2024-09-10T21:36:54.619 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41808.log 2024-09-10T21:36:54.621 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41808.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41808.log.gz 2024-09-10T21:36:54.751 INFO:teuthology.orchestra.run.smithi097.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T21:36:54.778 INFO:teuthology.orchestra.run.smithi194.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T21:36:54.843 INFO:teuthology.orchestra.run.smithi194.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T21:36:54.968 INFO:teuthology.orchestra.run.smithi194.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T21:36:55.109 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T21:36:55.155 INFO:teuthology.orchestra.run.smithi097.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T21:36:55.384 INFO:teuthology.orchestra.run.smithi194.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T21:36:55.386 INFO:teuthology.orchestra.run.smithi194.stderr: 2024-09-10T21:36:55.387 INFO:teuthology.orchestra.run.smithi194.stderr:real 0m2.066s 2024-09-10T21:36:55.387 INFO:teuthology.orchestra.run.smithi194.stderr:user 0m7.274s 2024-09-10T21:36:55.387 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m0.632s 2024-09-10T21:36:55.641 INFO:teuthology.orchestra.run.smithi097.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T21:36:55.837 INFO:teuthology.orchestra.run.smithi097.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T21:36:55.839 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-10T21:36:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m2.530s 2024-09-10T21:36:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m9.540s 2024-09-10T21:36:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m2.265s 2024-09-10T21:36:55.840 INFO:tasks.ceph:Archiving logs... 2024-09-10T21:36:55.841 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/remote/smithi097/log 2024-09-10T21:36:55.842 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:36:57.501 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/remote/smithi194/log 2024-09-10T21:36:57.502 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:36:58.370 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T21:36:58.379 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T21:36:58.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:36:58.383 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:36:58.483 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T21:36:58.492 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T21:36:58.493 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:36:58.496 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:36:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:36:58.509 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-09-10T21:36:58.509 INFO:teuthology.orchestra.run.smithi097.stdout:*hv01.front.sepi 67.205.162.81 3 u 21 64 377 0.089 -3.633 1.146 2024-09-10T21:36:58.509 INFO:teuthology.orchestra.run.smithi097.stdout:+hv02.front.sepi 96.245.170.99 3 u 15 64 377 0.075 -5.702 1.132 2024-09-10T21:36:58.509 INFO:teuthology.orchestra.run.smithi097.stdout:+hv03.front.sepi 74.6.168.72 3 u 15 64 377 0.069 -2.022 1.307 2024-09-10T21:36:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-10T21:36:58.512 INFO:teuthology.orchestra.run.smithi194.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:36:58.512 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================== 2024-09-10T21:36:58.512 INFO:teuthology.orchestra.run.smithi194.stdout:+hv01.front.sepi 67.205.162.81 3 u 15 64 377 0.087 -3.036 1.235 2024-09-10T21:36:58.512 INFO:teuthology.orchestra.run.smithi194.stdout:+hv02.front.sepi 96.245.170.99 3 u 14 64 377 0.092 -6.742 1.554 2024-09-10T21:36:58.512 INFO:teuthology.orchestra.run.smithi194.stdout:*hv03.front.sepi 74.6.168.72 3 u 19 64 377 0.098 -2.868 1.090 2024-09-10T21:36:58.512 INFO:teuthology.orchestra.run.smithi194.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-10T21:36:58.513 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T21:36:58.522 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T21:36:58.522 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T21:36:58.532 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T21:36:58.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T21:36:58.550 INFO:teuthology.task.internal:Duration was 1499.056124 seconds 2024-09-10T21:36:58.550 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T21:36:58.559 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T21:36:58.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:36:58.562 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:36:58.649 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T21:36:58.649 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-09-10T21:36:58.649 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:36:58.670 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-09-10T21:36:58.670 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:36:58.690 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T21:36:58.690 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:36:58.714 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:36:58.772 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T21:36:58.773 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:36:58.798 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:36:58.889 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T21:36:58.899 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T21:36:58.899 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:36:58.918 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:36:58.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T21:36:58.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:36:58.962 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:36:58.976 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-09-10T21:36:58.999 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-09-10T21:36:59.017 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:36:59.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:36:59.042 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:36:59.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:36:59.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T21:36:59.078 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T21:36:59.079 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/remote/smithi097 2024-09-10T21:36:59.079 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:36:59.110 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899672/remote/smithi194 2024-09-10T21:36:59.110 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:36:59.135 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T21:36:59.135 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:36:59.149 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:36:59.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T21:36:59.193 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T21:36:59.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T21:36:59.203 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T21:36:59.203 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:36:59.206 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:36:59.211 INFO:teuthology.orchestra.run.smithi097.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:36 /home/ubuntu/cephtest 2024-09-10T21:36:59.235 INFO:teuthology.orchestra.run.smithi194.stdout: 658142 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:36 /home/ubuntu/cephtest 2024-09-10T21:36:59.237 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T21:36:59.267 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T21:36:59.790 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage} duration: 1499.0561242103577 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T21:36:59.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:36:59.885 INFO:teuthology.run:pass